Upload
servicenow
View
57
Download
0
Embed Size (px)
Citation preview
© 2016 ServiceNow All Rights ReservedConfidential © 2016 ServiceNow All Rights ReservedConfidential
Establishing Good Reporting Practices and Effective Management Dashboards
© 2016 ServiceNow All Rights Reserved 2Confidential
Seven Deadly Measurement Sins
Selection Collection Reporting Usage
An IT-Centric View of IT Performance: Lack of Senior Business involvement in measurement selection
Measures That Don’t Matter: No Link between metrics and Business / IT Strategy
No Common Ground: Lack of standard metrics definitions complicates aggregation
Overreliance on Tools: Reliance primarily of reports available in the existing tools
Lack of Drill-Down Capability and Metric Context
Too Many Metrics
No Individual Impact or Consequence
Source: CIO Working Council Research - www.executiveboard.com
1
3
4
5
6
7
2
Nature of Problem
Strategic
Tactical
© 2016 ServiceNow All Rights Reserved 3Confidential
Three Basic Service Measurements• Service Measurement is really about providing the results of the IT service as the
customer experiences the service. This would be an end-to-end experience.• For services, there are three basic measurements that most organizations utilize:
– Availability of the service– Reliability of the service– Performance of the service
Other Measures: Financial, Quality, Customer Satisfaction
© 2016 ServiceNow All Rights Reserved 4Confidential
CSR vs. KPI vs. Activity Metric• CSF – Critical Success Factor
– A key element of a service or process which must be in place to achieve the goal
• KPI – Key Performance Indicator– A specific measurable attribute related to a critical success factor
• Activity Metric– A general measure element of a process activity
Need to consider both Lead & Lag KPIs
© 2016 ServiceNow All Rights Reserved 5Confidential
Three Types of Metrics
These metrics are the results of the end-to-end service. Component metrics are used to compute the service metrics.
• Technology Metrics
• Process Metrics
• Service Metrics
These metrics are often associated with component and application-based metrics such as performance, availability, etc.
These metrics are captured in the form of CSFs, KPIs and activity metrics for the service management processes. These metrics can help determine the overall health of a process. Four key questions that KPIs can help answer are around quality, performance, value and compliance of following the process. CSI would use these metrics as input in identifying improvement opportunities for each process.
© 2016 ServiceNow All Rights Reserved 6Confidential
Example Incident Management CSF and KPICSFMaintain IT Service Quality
KPI #1Targeted Percentage Of Incidents Resolved Within Target Times By Priority
Why is it important and what it tells us about the operation?
Focused on Incidents in the proper order, improved escalation within the Incident Management process, and should have improved customer satisfaction.
How can it be measured and analyzed?
Reports against priority model and defined resolution times defined in SLAs.
The frequency for which it should be measured?
Weekly to start and then monthly. For critical customers it can be even more frequently.
How to present and distribute results?
Charts and graphs, distribute with analysis, what led to positive/negative results, and plan of action to take.
How the analysis of results can drive continuous improvement efforts
Identify potential weaknesses in priority model, escalations, and need for OLAs, or enforcement of SLAs. Could point to need to renegotiate SLAs.
How targets for performance can be set and revised
Negotiated with the customer for inclusion within the SLA. Reviewed as defined within the SLA.
© 2016 ServiceNow All Rights Reserved 7Confidential
IT Strategy and Cascading MeasuresCore Business Measures
Revenue
Market Share
Profit
ROE
IT Core Strategic MeasuresFinance
Innovate
InternalCustomer
Balanced Scorecard
Enterprise Operating Model
IT ProcessesITSM, Development, Project Mgmt., Security, Architecture, Operations, etc.
IT Value Chain (Service Organization)
IT Operating Model / COBIT
Business Value
IT Goals
IT CapabilityMeasures
IT ProcessMetrics
© 2016 ServiceNow All Rights Reserved 8Confidential
Balanced Scorecard Approach to Measure BenefitsFinancial Customer
Innovation Internal
Improve Quality of Service
Improve Management Control
Employ New Technology
Reduce Cost
Example
KPI Categories Value Quality Performance Compliance
In order to improve a Service or Service Management process, you must look at it from different perspectives
© 2016 ServiceNow All Rights Reserved 9Confidential
Process KPI Categories• Getting the whole picture:
– Value: Is what we are doing making a difference?– Quality: How well are we doing it?– Performance: How fast or slow are we doing it?– Compliance: Are we doing it?
A single measure may contain or cover more than one category. This, in itself is not an issue. However, understand that, when this occurs, the success criteria for this measure is more difficult to satisfy.
© 2016 ServiceNow All Rights Reserved 10Confidential
Choosing KPIs• Ideally, to measure a process, at least one KPI per category should be chosen
to provide a balanced perspective. However, due to the difficulty of measurement or tool limitation, a process management staff may find it necessary to limit what is measured according to what category is the most important to the objective of the process.
1. Establish the core objective of the process.2. Evaluate which category is of the highest priority to achieve the process objective.3. Define measures according to the categories which are appropriate to achieve the overall
process objective.
© 2016 ServiceNow All Rights Reserved 11Confidential
Building a Measurement Grid1. Define the measure.2. Determine the KPI category.3. Establish the policy and target. (Target will change with process maturity.)
4. Determine the tool or medium to realize the report.5. Define the output format (graph, data, etc.).6. Define distribution list and report frequency.
Category Measure Policy Target ToolValue, Quality # of incidents resolved
outside of SLAAll incidents to be resolved within SLA
90% of the time Incident Module
© 2016 ServiceNow All Rights Reserved 12Confidential
Example KPIs and CategoriesProcess Core Objective Example Core KPIs Category
Incident Restore service degradations to expected level ASAP.
# of Incidents by category, priority and resolution type by LOB.# of Incidents restored within SLA Targets.
ValuePerformance
Problem Identify systemic Infrastructure Errors and eliminate them to minimize impact and improve availability.
# of Problems identified and root cause determined with solution or workaround.# of Repeat Incidents by category trending downwards.
QualityValue
Change Handle changes efficiently while minimizing impact to service delivery.
# of Changes by type / category / group / customer. (emergency changes trending down).# of Changes that have resulting Incidents, or fail and have to be backed out% of changes submitted within lead times
QualityValuePerformance
Config. Identify / control / manage IT resources within a Configuration Management Database.
% of CMDB data population and accuracy vs. actual, according to scope.% Growth or Change by CI type over an elapsed time period.
ComplianceQuality
Release Ensure production readiness, quality and authorization of new or modified CIs and their planned deployment.
# of Releases by type that satisfy release management criteria when submitted to Change.
QualityCompliance
© 2016 ServiceNow All Rights Reserved 13Confidential
Example KPIs and CategoriesProcess Core Objective Example Core KPI Category
SLM Define servicesAgree on level, scope, quality, performanceMonitor and Manage
% Score of customer satisfaction survey trends up over time.% of SLA reviews that are completed within 5 days of agreement expiry
ValuePerformance
Avail. Define and plan for service availability to meet or exceed stated business requirements through process, technology and people resource planning and implementation.
% of service availability within SLA negotiated requirements. % of service design packages which contain availability management requirements
ValueCompliance
Capacity Current and future resources are greater than or equal to demand, but excess is planned.
% of components the breach tolerance thresholds in correspondence to planned capacity levels for components and complete IT systems.
Quality
ITSCM Recover IT systems to normal state in an alternate way after a disaster within an expected timeframe.
% of systems that fail recovery test.Time to execute test of plan and recover IT services in a contingency state against expected targets.
QualityPerformance
Finance Plan for and deliver IT Services within a forecasted budget against actual cost.
% deviation of forecasted vs. actual cost of IT services within defined tolerance limits.(% of Deviation $ of Deviation)
Compliance
© 2016 ServiceNow All Rights Reserved 14Confidential
Service Measurement ModelIT Scorecard
or Balanced Scorecard
Component 1 Measure
Rolled up Service Measurement Results
Key Performance
Indicators
Service Dashboard
Real time information
Point in time information
Component 2 Measure
Component 3 Measure
Service Scorecard
© 2016 ServiceNow All Rights Reserved 15Confidential
Service Management Model for Processes
Activity Measure One = number of urgent
changes
Activity Measure Two = number of failed
urgent changes
Activity Measure Three = Number of unauthorized failed
changes
Process Key Performance
Indicators
Process High Level Goal
Overall Service Management Process Scorecard
Example for Change Management = reduce the number of failed changes
Example = Improve Service Quality
© 2016 ServiceNow All Rights Reserved 16Confidential
Dashboard Key Performance IndicatorsProcess KPI/Description Type Progress Indicator
Incident Tickets Resolved within target time Value Meets / Exceeds Target Times
Incident % of Incidents Closed – First Call Performance Service Desk only – Target is 80%
Incident Abandon Rate Service Desk with ACD. 5% or less goal (after 24 seconds)
Incident Count of Incidents submitted bySupport Group
Compliance Consistency in number of incidents – investigation is warranted for 1) rapid increase which may indicate Infrastructure investigation, 2) rapid decrease which may indicate compliance issues.
Problem % of repeated Problems over time Quality Problems that have been removed from the infrastructure and have re-occurred. Target: Less than 1% over a 12 month rolling time frame.
Problem % Root Cause with permanent fix Quality Calculated from Problem Ticket . Start date to permanent fix found. This may not include implementation of permanent fix. Internal Target: 90% of Problems – within 40 days External Target: 80% of Problems – within 30 days Internal = BMO internal External = 3rd Party/vendor
Problem % and number of Incidents raised to Problem Management Compliance Sorted by Infrastructure (Internal and External ) and Development (Internal and External).
Change % of RFCs successfully implemented without back out or issues Quality Grouped by Infrastructure / Development
Change % of RFCs that are emergencies Performance Sort by Infrastructure or Development – and by Emergency Quick fix (service down) or Business requirement
SACM Number of CI additions or Updates Compliance Configuration Item Additions or updates broken down by group – CMDB / Change Modules
SACM Number of Records related to CI Performance Number of associations grouped by process
Release % of Releases using exceptions Value Exceptions are criteria deemed Mandatory – identify by groups
Release % of Releases by-passing process Compliance Identify groups by passing Release Process
Capacity Action Required Value Number of services that require action vs.Total number of systems
Capacity Capacity Related Problems Quality Number of Problems caused by Capacity issues sorted by group
© 2016 ServiceNow All Rights Reserved 17Confidential
Executive ITSM Dashboards
© 2016 ServiceNow All Rights Reserved 18Confidential
Incident Process Owner Dashboard
© 2016 ServiceNow All Rights Reserved 19Confidential
Service Desk Worker Dashboards
© 2016 ServiceNow All Rights Reserved 20Confidential
Executive Roll-up of Dashboards
Processes Health
Incident Target Times for Resolution Critical/High not met
Problem 50% fewer problem tickets raised – no correlation
with number of incident tickets.
Change No Concerns
Config Updates to CMDB to be broken down by group. Fewer Incidents associated.
Release Low compliance to process
KPIs were defined by and agreed to by the process owners.
In some cases, the new Service Management Tool was required to collect the metrics. The KPIs collected were updated in February. Data being collected is based on February/March Data.
SUMMARY: The chart on the right denotes status of KPI’s received and evaluated.
Follow Up / Investigate Status – Does not necessarily mean there is an issue.
No Concerns Follow Up /Investigate Target Times Exceed
© 2016 ServiceNow All Rights Reserved 21Confidential
CSI RegisterProcess Issue Improvement Activity Assignment Due Date
Incident
1) Updates to documentation 1) Formalise/document procedure enhancements 1) Process Owner April 30/03
Problem
1) Knowledge Database does not exist. 1) Create Knowledge Database 1) Process Owner 1) February 12No Revised date provided.
Change
1) Change Management Documentation. 2) Change Management – documentation under investigation3) MAC Process
1) Finalise documentation2) Being Updated3) Being Formalised
1) Process Owner 1) April 302) April 303) April 30
Config.
1) Lack of Associations between Logical and Physical CI’s
2) Post project CIs
1) 18 Critical systems identified – Pilot system is in database. Further design and population required.
2) Additional CI’s outside scope of project to be added
1) Process Owner 1) Jan 8
2) Not defined
Release
1) Compliance – to process2) Definitive Software Library (s)are not being updates/used. Evidence of updates to the DSL not evident.
1) Low Compliance through process – Implemented ‘RFC Requirements’2) Review/Investigate COE/Infra/Development DSL usage
1) Process Owner 1) March 302) March 30
CAPACITY
1) Area data collection procedure/method not documented
1) Each team to document and detail steps for data collection 1) Process Owner 1) April 30