Upload
mhamdimourad
View
249
Download
0
Embed Size (px)
Citation preview
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
1/61
Keynote SIGOS GmbH
Klingenhofstrae 50d
90411 Nuernberg, Germany
Fon +49 / 911 / 9 51 68-0
Fax +49 / 911 / 9 51 68-355
www.keynote-sigos.com
Testing is our competence
SITE 2.1 Training
Reporting & Alarming
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
2/61
Keynote SIGOS GmbH 2008, page 2
real time
reporting
alarming
Measurements
Strategy
Test
Executions
Test Execution on
different locations
KPIs
o v e r v i e w - S I T E r e p o r t e n v i r o n m e n t
Base
Measurements
(Input and Output)
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
3/61
Keynote SIGOS GmbH 2008, page 3
How to create a report?
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
4/61
Keynote SIGOS GmbH 2008, page 4
Toolbox for Reports, Report Editor
Reports are created with the Report Editor.
You can open the Report Editor with the corresponding button in the toolbar.
Together with the Report Editor the Toolbox for Reports will be opened in the
navigation area...
Edit Report Definition
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
5/61
Keynote SIGOS GmbH 2008, page 5
Toolbox for Reports, Report Editor
You can open the Toolbox for Reports manually with the regarded button in the
navigation bar:
Toolbox for Reports
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
6/61
Keynote SIGOS GmbH 2008, page 6
Toolbox for Reports, Report Editor
The Toolbox for Reports contains all items you need to define your reports:
Folder to build up a hierarchy in your report definitions
Report Definition defines a new report which consists
of one or more charts
Chart is a diagram which shows one or more
measurements
Alarm defines a threshold on a measurement. If the
threshold is reached a defined action will be
executed.
Measurement Databases contain the measurements of the
testruns. The layout is defined by the Measurement
Strategy.
Measurements are results of the testruns. The calculation of
the measurements is defines in the Measurement
Strategy
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
7/61
Keynote SIGOS GmbH 2008, page 7
Toolbox for Reports, Report Editor
You can build up a structure in the Report Editor by dragging items from the
Toolbox for Reports (navigation area) and dropping them in the working area:
Toolbox For Reports Report Editor
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
8/61
Keynote SIGOS GmbH 2008, page 8
Toolbox For Reports, Report Editor
You can create reports out of these
structures
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
9/61
Keynote SIGOS GmbH 2008, page 9
Toolbox For Reports, Report Editor
You create a report by selecting the report definition node in the Report Editor
and pressing the Show Current Report button afterwards.
Show Current Report
or by using the context menu
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
10/61
Keynote SIGOS GmbH 2008, page 10
f i r s t r e p o r t
open the report editor
drag a new folderfrom the toolbox for reports to the structure tree
drag a new report definition from the toolbox beneath the new folder
drag the KPI Rx_level from the toolbox beneath the new chart
show current report by opening the context menu of the new report or
selecting the icon from the toolbar.
Demonstration: Display the Rx_Level of the testruns within the last 2 hours
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
11/61
Keynote SIGOS GmbH 2008, page 11
Report Definition
When you click on a Report Definition you will find the parameter tree in the
working area:
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
12/61
Keynote SIGOS GmbH 2008, page 12
Report Definition - size
With the size parameter you can define the initial size of the
charts in the report.When set to automatic the value is taken from
the table ConfigurationParameters (report image width and report
image height) in the database sitecu. Default is 500x300)
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
13/61
Keynote SIGOS GmbH 2008, page 13
Report definition Synchronize charts
Synchronize the timelines across all
charts in the report.
By setting to 'no' it is possible to
define individual timelines for each
chart.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
14/61
Keynote SIGOS GmbH 2008, page 14
Report Definition - Report Interval
The timing parameter defines the time range for the report. Only results of
testruns with a Timestamp value within this range will be shown in the report.
The timestamp value is the time when the final verdict of a testrun is set. So in
most cases the Timestamp will match the end time in the TestcaseStatus view.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
15/61
Keynote SIGOS GmbH 2008, page 15
Report Definition Report Interval
Fixed (account KPIs from a certain start time till a certain end time)
start time end time
Rubberband (account KPIs from a certain start time till time of report generation)
start time Report generation
Window (account all KPIs within a certain time windowbefore report generation)
window Report generation
start time end time
start time Report generation
Report generation
Report generation
window Report generation
t
t
t
t
t
t
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
16/61
Keynote SIGOS GmbH 2008, page 16
Report Definition - Data Aggregation
In the Report Timing dialog you also
setup the Data Aggregation Interval.
In case you use aggregated
measurements, the Report Interval is
split into slots of the selected
duration.
For each time slot the aggregated
measurements will be combined to a
single data point in the report which
is placed at the end of the time slot.
Note:
If you select Window the time range displayed is aligned according to the data
aggregation interval. If you select Until now you will force the system to extend
the end point of the report window to the time of the report generation.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
17/61
Keynote SIGOS GmbH 2008, page 17
Report Definition - Data Aggregation
1
2
3
1
2
3
4
Aggregation method: count
min
max
average
t
t
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
18/61
Keynote SIGOS GmbH 2008, page 18
Report Definition - Report Schedule
You can schedule a report to be generated one or more times. Any time a
scheduled report is generated, the report data is stored in the database.
You can also enter an email address which the generated reports will be sent
to.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
19/61
Keynote SIGOS GmbH 2008, page 19
Report Definition - eMail Report Example
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
20/61
Keynote SIGOS GmbH 2008, page 20
Report Definition - Filter
The filter parameters allow you to define
a filter for all measurements used in the
charts of the report.
The filter is valid for all measurements
which are taken from the filtered table.
(A filter on the Main table will be valid
for all measurements originating from the
same database.)
You can define following filter:
db_name (Measurement Target), db_table (Testcase), column_filter
If include_invalid is set to no measurements marked as 'invalid'will not be
included in the report.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
21/61
Keynote SIGOS GmbH 2008, page 21
Report Definition - Column Filter
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
22/61
Keynote SIGOS GmbH 2008, page 22
Chart Definition
When you click on a Chart Definition you will find the chart parameter tree in
the working area:
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
23/61
Keynote SIGOS GmbH 2008, page 23
Chart Definition - Chart Type
marker lines lines
areas
marker
bar stack bar group bar percentage
Chart Types:
matrix single
matrix multi
s1
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
24/61
Slide 23
s1 matrix multi: multiple KPI, just column headers (x-value)
matrix single: just one KPI, column and row headerssigos, 1/17/2008
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
25/61
Keynote SIGOS GmbH 2008, page 24
Chart Definition Aggregation Method
The aggregation method is
necessary for the matrix
report types
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
26/61
Keynote SIGOS GmbH 2008, page 25
Chart Definition Timing
If the parameter synchronize_charts
in the report definition is set to no,
the timeline (report interval) can be
specified for each chart in the report,
separately.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
27/61
Keynote SIGOS GmbH 2008, page 26
Chart Definition - Chart Title
You can enter a title of a chart. The title will be displayed at the top of the chart.
There are some text macros available (that will be substituted by the actual values):
: Begin of timing
: End of timing
: Report-definition-Path : Report-definition-Name
: Time zone setting of the report's owner
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
28/61
Keynote SIGOS GmbH 2008, page 27
Chart Definition - Chart Title
Here you can modify the information
contained in the legend within the chart:
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
29/61
Keynote SIGOS GmbH 2008, page 28
Chart Definition - Chart Filter
Here you can define a filter for all measurements in
the chart(not in the whole report).
Remember: The filter will only be taken into account if
it is set on the same table as the measurements
actually used in the chart. (The main table will be validfor all measurements of the same target.)
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
30/61
Keynote SIGOS GmbH 2008, page 29
Chart Definition - Chart Filter
You can select a background color for
the chart. Default is white.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
31/61
Keynote SIGOS GmbH 2008, page 30
Chart Definition - Bar Labels
If you choose chart type bar stack or
bar percentage you can enable the
display of labels for each bar. You can
choose between labels showing the
absolute value or the percentage
value.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
32/61
Keynote SIGOS GmbH 2008, page 31
KPI Definition
When you click on a KPI Definition you will find the KPI parameter tree in the
working area:
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
33/61
Keynote SIGOS GmbH 2008, page 32
KPI Definition Data Source
The data source of the KPI can
still be changed.
Changing this parameter have the same
effect as dragging a different KPI,
respectively a KPI from a different KPI
database and/or a different KPI table, intothe chart.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
34/61
Keynote SIGOS GmbH 2008, page 33
KPI Definition - Type
You can define a column and/or a test
definition filter on measurements.
The column filter is a string pattern filter
on string value or a numerical filter on
numeric columns on a table which is a
join of the Main table and the testcasespecific table from which the
measurement was taken.
The test definition filter is a filter on the
test definition node in the test editor.
Only results of the given test definition
will be taken of the report.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
35/61
Keynote SIGOS GmbH 2008, page 34
Difference test definition and column filter on TestDefPath
You can define a test definition filter or you can define a column filter that
filters for certain test definition paths.
Both filters will only allow measurements of a specific test definition.What's
the difference?
Works after moving the test definition
to another position in the structure or
after renaming
Works after export and reimport tothe same location
column filter testdef filter
no yes
yes no
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
36/61
Keynote SIGOS GmbH 2008, page 35
KPI Definition - Type
Here you can choose the "type" of the
KPI in regard to its role to the chart.
Following types are available:
y1-value The value will be displayed on
the y-axis.
y2-value The value will be displayed onthe y-axis, but a second scale will be
used.
y3-value, y4-value Additional y-scales
like y2-value
x-value-string All other KPIs of the chart
will be aggregated on the different
values of this KPI. These differentvalues build up the x-axis of the chart.
x-value-numeric Similar to x-value-string
but the points on the x-axis will be
scaled according to the value.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
37/61
Keynote SIGOS GmbH 2008, page 36
y2-value example
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
38/61
Keynote SIGOS GmbH 2008, page 37
x-value-string example
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
39/61
Keynote SIGOS GmbH 2008, page 38
x-value-numeric example
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
40/61
Keynote SIGOS GmbH 2008, page 39
KPI Definition - Factor And Scale
In the y_scale_factorand scalingsettings you
can adjust scaling setting. You can multiply
the measurement values with a given factor oradjust the range of the y-axis.
You can also define a label for the axis and a
unit description.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
41/61
Keynote SIGOS GmbH 2008, page 40
KPI Definition - Aggregation
There are different aggregation methods
available:
raw no aggregation is done
min/max/avg the minimal (maximal, average)
value of all values in each aggregationinterval is calculated
sum all values in the specific aggregation
interval are summarized
count is the number of database entries
matching all filter criteria
Remember: In the chart the aggregatedvalues will be displayed at the end of the
aggregation interval.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
42/61
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
43/61
Keynote SIGOS GmbH 2008, page 42
KPI Definition Alarm Expiry
Here you can define an expiry period.
If there is no KPI data for at least the
duration of the specified period the
severity of a defined alarm will be set to
indeterminate.
The period can be defined in following
formats: sec, min:sec, h:min:sec. If the
expiry is set to 0 a trapped alarm will
not expire.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
44/61
Keynote SIGOS GmbH 2008, page 43
Let's have a closer lookat the report...
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
45/61
Keynote SIGOS GmbH 2008, page 44
Report Properties
If you look at a report you canadjust the timing settings
temporarily.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
46/61
Keynote SIGOS GmbH 2008, page 45
Report Properties
It's also possible to change the
color of the measurements
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
47/61
Keynote SIGOS GmbH 2008, page 46
Interactive Chart Viewer
With the chart viewer you are
able to browse through the
measurements of your report.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
48/61
Keynote SIGOS GmbH 2008, page 47
Interactive Chart Viewer
With the cursor you can runthrough the results by using the
arrow keys. You will see the
Timestamp and the Value for
each data point.
(If the cursor is at the exact x
position the information is
displayed with the color of theaccording KPI. Otherwise the
color of the information is grey
which means, that you see the
information of a data point which
is near to the current position of
the cursor). In case
unaggregated values are used inthe chart you will also see the
TestcaseId.
If you see the TestcaseId you are
able to go to the trace of the
testrun directly.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
49/61
Keynote SIGOS GmbH 2008, page 48
Interactive Chart Viewer - Drill down
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
50/61
Keynote SIGOS GmbH 2008, page 49
Measurement Data
From the report dialog or the
interactive chart viewer you can
open the measurement data view.
Here you can see the list of all
values used for the report.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
51/61
Keynote SIGOS GmbH 2008, page 50
Measurement Data Drill down
In the Measurement Data view you can open the traces of the testcases
which resulted in the measurements had the chart consists of.
Note: Drill down is only possible if you use raw data.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
52/61
Keynote SIGOS GmbH 2008, page 51
Chart Viewer - Toolbar
Change ReportProperties
Open Interactive
Chart Viewer
Show
Measurement
Data
Show
Report Info
Save Chart
as Picture
Store Report
to Database
Enable/Disable
Cyclic Update
Update view of
current Report
Send link to report
graphic via email
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
53/61
Keynote SIGOS GmbH 2008, page 52
Chart Viewer - Toolbar
Change Report
Properties
Show
Measurement
Data
Save Chart
as Picture
Enable/Disable
Cyclic Update
Update view of
current Report
Open Trace
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
54/61
Keynote SIGOS GmbH 2008, page 53
Alarming
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
55/61
Keynote SIGOS GmbH 2008, page 54
Alarm Definition
The first step to define an alarm is to drag an alarm item from the Toolbox For
Reports beneath a KPI Item of a report in the Report Editor via Drag and Drop.
Toolbox For Reports Report Editor
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
56/61
Keynote SIGOS GmbH 2008, page 55
Alarm Parameter
If you click on an Alarm definition you will find a parameter tree like this in the
working area:
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
57/61
Keynote SIGOS GmbH 2008, page 56
Alarm Definition - Parameter
An alarm can be activated
or deactivated. Alarm traps
will only be sent when the
alarm is activated.
The alarm threshold can be
shown in the report chart.
The color of the alarm line in the
report chart can be changed.
An alarm has a certain severity.
You can choose between
warning, minor, major, critical
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
58/61
Keynote SIGOS GmbH 2008, page 57
Alarm Definition - Threshold
The alarm will be trapped if the
measurement will be above, below,
equal or not equal (condition) a
given thresholdfor at least count
times.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
59/61
Keynote SIGOS GmbH 2008, page 58
Alarm Definition - Actions
The alarm text is a user definable
string that will be send together
with other information as alarm
trap
There are different ways to send an
alarm trap. You can send an alarmemail, SMS (additional HW required).
SITE can also send traps to a SNMP
trap host.
There is also the option to call a
certain application or shell script once
the alarm was raised.
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
60/61
Keynote SIGOS GmbH 2008, page 59
alarm email example
8/9/2019 1 SIGOS SITE 2 1 0 User ReportingAndAlarmingV2
61/61
Keynote SIGOS GmbH 2008, page 60
w o r k s h o p
Chart showing the average duration of all testcases run in the last two weeks.
Chart showing the number of testruns for each testcase (TCName) run in the last
two weeks.
Chart showing the number of passed, failed and incomplete testruns for each
testcase (TCName) run in the last two weeks.