KEY PERFORMANCE INDICATORS IN HOSPITALS

From Measurement and Reporting to Actual Improvement and Action

Fadi El-Jardali, MPH, PhD.

March 2016 DAY 1

Introduction, expectations and overview Learning Outcomes

Understand the concepts of measurement, reporting and using data from KPIs to inform decisions

and actual improvement plans and actions

Understand and apply a criteria to short list critical and strategic indicators

Apply KPI measurement and reporting steps including identifying, measuring, validating,

analyzing, and reporting data

Apply skills to identify barriers and enablers for effective measurement, reporting and

improvement in hospitals

Develop action plans and improvement plans based on data from KPIs

Develop skills to apply strategies to implement improvement plans

Understand the development and applications of balanced scorecard system in Hospitals

Develop skills to use benchmarks in monitoring performance

Develop skills to apply evidence-based management in making decisions and improvement plans

Develop skills to become evidence based hospitals and evidence-based health professionals

CONCEPTS OF MEASUREMENT AND REPORTING KEY PERFORMANCE INDICATORS (KPIS)

Measurement for Improvement

• Measurement is important to determine whether changes that are believed to lead to improvements in quality do in fact result in improvements.

You can’t manage what you can’t measure

“Trying to improve performance without measurement is wasteful since we will be uncertain about what to improve, and unable to assess whether our actions made a difference!”

Dr. G. Ross Baker, University of Toronto

The Foundation for Organizational Effectiveness

• The importance of recognizing the difference between

reporting indicators and measuring performance

• Measurement and analysis are critical to the effective

management of the organization and to a fact-based system

for improving healthcare and operational performance

• Measurement is essential to managing and improving

organizational performance and results

• That is why, managers must see performance – not simply

performance indicators – as the end result of their efforts

• Performance measurement to be viewed as a process

Importance of measurement for improvement

•The aim of improvement endeavours in healthcare is to make services better. That might be safer (less errors, infections, falls), more effective (delivering care that is based on science -neither over or under treating), more efficient (less waste), more person-centred (caring, compassionate, fitting with patient/family requests), equitable or timely

Measurement for improvement asks questions like:

1. What does "better" look like?

2. How will we recognise better when we see it?

3. How do we know if a change is an improvement?

Measurement System • How will you know if the change is an

improvement?

• A performance measurement system should be

the first step of implementation, not only

because of its role in describing, monitoring,

and communicating current reality and

progress toward the vision, but also because it

provides information about the influence of

improvement interventions on the behavior of

the overall system

Data and measurement is required: • To plan for improvement

• For testing changes

• For tracking compliance

• For determining outcomes

• For monitoring long term progress

• To make improvement visible and tell an improvement story

The Act of Measurement

• The act of measurement of something signals that that thing is seen as a priority.

• However, the aim should be improvement rather than measurement

Measurement for Improvement

Measures, Indicators, Targets

Measures (The “What”)

•Quantitative or qualitative attributes that must be measured in order to

determine whether the expected results are being achieved

•E.g. patient experience

Indicators ( The “How”)

•The quantification or qualification of a performance measure

•E.g. percentage of patients reporting positive experience

Targets (The “How Much”)

•Specific quantitative or qualitative goals against which actual outputs or

outcomes will be compared

•E.g. 90% patients positive experience

•Targets imply a desired goal that may be more ambitious than a

standard

An Indicator is…

• A measurement tool, which is rate-based or defined as an event

• A guide to monitor and evaluate the quality of client care and

service, clinical support services, and organizational functions

• An aid to make continuous improvements

• A numerical measure

• Based on quantitative and qualitative data collection

• Expressed in various ways (rates, averages, proportions, etc.)

What are Indicators used for?

Track improvement activities

Pinpoint areas for improvement

Measure day-to-day operations

Provide strategic directions

Compare performance to an established norm (e.g.

benchmarking)

Provide evidence for compliance to standards

Reflect achievement of positive outcomes

Common Questions

• What indicators are being used?

• Why were they selected (rationale)? By whom?

• How is the information from the indicators used for

quality improvement?

• How are the results shared within your

organization?

Key Components of an Indicator

Key Questions

• What is the event or observation being tracked?

(Numerator)

• What is the population base from which the

indicator is calculated? (Denominator)

• Over what period will the event or observation be

tracked? (Time period)

Indicators are Rate-based

# of individuals readmitted to the hospital

Total # of individuals discharged from hospital (Time period)

Example

Formula Numerator

Denominator (Time Period)

Types of Measures for Consideration when Selecting Indicators

• When choosing performance indicators, balance 3 types of indicators:

• Structure measures (i.e. # of physicians on staff; staff turnover; age of the radiology equipment, etc..)

• Process measures: activities that occur between patients and providers – what is done to the patient (i.e. % Visits for which a list of current medications is documented; Preventative care activities such as mammography and

immunization)

• Outcomes measures: changes in the clinical status – what happened to the patients (# of women who get a mammogram is a process measure, while the number of women who die form breast cancer is an outcomes measure)

Structural Measures

• Indicators which:

• Describe the type and amount of resources used to deliver programs and

services

• Relate to resource allocation (staff, clients, money, supplies, equipment and

buildings)

• Example: Staff turnover

Example:

Number of terminations from the 1st of the month through the last day of the month

Average headcount for the month

Process Measures

• Indicators which:

• Provide a measure of the activities and tasks undertaken to achieve

program or service objectives

• Relate to resource utilization

• Example: Patients with an Acute stroke and who are non- ambulatory

should start receiving DVT prophylaxis before the end of hospital day 2

Example

from Kuwait

Non- ambulatory ischemic stroke patients who had DVT

prophylaxis initiated before the end of hospital day 2

Ischemic stroke patients who are non- ambulatory at the end of

hospital day 2

Outcome Measures

• Indicators which:

• Measure changes in clients’ / patients’ health status

• Relate to results attributable to preceding care and service

Number of patients readmitted to ICU within 48 hours of

discharge from ICU

Total number of ICU discharges

Example

from Kuwait

Measures

• Outcome measures reflect the impact on a patient and demonstrate the end result of doing things (including making changes that you predict will be improvements). Examples are mortality, hospital acquired infection or falls rates.

• Process measures reflect the things that you do (processes) and how systems are operating. Commonly process measures show how well (e.g. % compliance with protocol) you are delivering a change that you want to make. Examples are % of hand-washing opportunities taken or % of patients with possible sepsis who received antibiotics within an hour of assessment.

Balancing measures

• Balancing measures show whether unintended consequences have been introduced elsewhere in the system. For example the aim of an improvement might be to reduce the number of hypoglycaemic episodes in those with diabetes who are inpatients in general surgery.

• As a balancing measure you might wish to assess whether the number of hyperglycaemic episodes goes up. Another common balancing measure is readmission rate when measuring length of stay as an outcome

The hierarchy of measurement reporting (Lloyd & Caldwell 2007)

Focus on Kuwait

•Clinical indicators measuring performance on Major Disease Categories (e.g., Appendectomy, Stroke, Heart Failure, Maternity, ICU, etc…)

•Indicators reflect process and outcome measures mainly

•Lack of Structure measures

•Lack of overall perspective on performance of healthcare organization:

• Lack of indicators on patient experience, staff satisfaction, turnover rate, financial performance, etc..

STEP- BY- STEP GUIDE TO

DEVELOP INDICATORS FOR

QUALITY IMPROVEMENT

A Guide to Start from ….

Phase I – Educate the Organization

Phase II – Find the Right Things to

Measure

Phase III – Build a Data Collection

Strategy

Phase IV – Build a Performance Report

Phase V – Refine and Adjust

Data Collection

Templates

List of Program

Indicators

Performance

Report

Update the

process

Common

Understanding

CRITERIA TO SHORTLIST CRITICAL AND STRATEGIC INDICATORS

Phase II – Find the right things to Measure

Determining the right Indicators occurs in two steps:

Step 1. Generate a preliminary list of possible quality

measures and indicators

Step 2. Identify the “critical few” indicators that will

be used to measure performance in the short

to medium term (This will be discussed

shortly).

Who is the primary intended

audience?

•The level of detail in the report varies depending on the audience

•Policymakers

•Healthcare managers

•Patients

What is the aim of

measuring indicators?

•Suitability, usefulness, and impact of indicators will depend on clarity about the aims of measurement.

•Healthcare system performance assessment

•Benchmarking against peers for quality improvement

•Measuring alignment with strategic plan

•Governance

How many indicators?

•Too few → Risk of oversimplification and neglect of unmeasured aspects of care

•Too many → Confusion or apathy

What sort of indicators?

•Balance between structure/ outcome/ process indicators

•General indicators reflecting common services and programs

What are the data sources

for indicators?

•Optimal use of existing data → Cost effective, no added burden, IT system in place, data and methodology well understood.

•Clinical Audit → rich sources of data, assess the process (evidence- based criteria) and/ or outcome of care (by comparison with others).

•Data quality → rigorous checks for quality, ongoing monitoring of data quality is important for interpretation and driving improvements

Main Questions to Guide Indicator Selection

How to Determine “the Critical Few” Indicators

Specific – eliminates ambiguity, shows relevance

Linked – clear (cause and effect) linkages to other

indicators

Reliable – scientifically and statistically sound, provides an

appropriate degree of accuracy

Available – data is easily accessible or there is a low level of

effort to collect and analyze

Understandable – data can be easily grasped by various

audiences.

Literature Review: Selection Criteria • Literature review on the most common selection criteria used in national

performance assessment experiences and other quality measurement

initiatives:

Common Criteria Common Definitions References

Feasible The indicator is easy to measure in terms of:

• Data availability; or

• Minimal burden of data collection; or

• Minimal costs of data collection.

1,2,3,4,5,8,9,1

0,11,13,14,15

Important • The indicator is important for policy and regulation; or

• The indicator is important for accreditation; or

• The indicator can help inform senior management goals,

organizational goals, and strategies.

4,7,10,13

Utility for making

improvement

• The indicator can help identify opportunities for improvement; or

• The indicator can help contribute to the delivery of care.

2,3,4,6,8, 9,

10,

11,12,13,14

Risk for adverse

incentives • The indicator has potential to create perverse effects (i.e., gaming,

data manipulation, or inaccurate reporting)

8

Appropriateness to

National Health

Strategy

• The indicator can be used to direct the implementation of the

National Health Strategy.

• The indicator can be used to direct the implementation of priorities

set at the MOH level.

10

Literature Review: Selection Criteria

1. York’s BSC for public health performance (Weir et al., 2009)

9. OAC Balanced Scorecard (Pink et al., 2001)

2. Common wealth Fund’s International Working Group on quality indicators. (Zellerino et al., 2009)

10. Pakistan tertiary care hospital BSC (Rabbani et al., 2010)

3. OECD Health Care Quality Indicators (Mattke et al. 2006

11. Safety improvement for patients in Europe (Kristensen et al. 2009)

4. Lessons from the OECD Health Care Quality Indicators (Kelly et al. 2006)

12. Raleigh and Foot (2010)

5. World Health Organization (WHO) Performance Assessment Tool for Quality Improvement in Hospitals (PATH) (Groene et al. 2008)

13. OECD Health Care Quality Indicator Project, The expert panel on primary care prevention and health promotion, Quality indicators for international benchmarking of mental health care (Hermann et al. 2009, MARSHALL et al. 2006)

6. India clinical indicators for the secondary health system

14. CIHI http://www.cihi.ca/cihi-ext-portal/pdf/internet/chrp_faq_pdf_en

7. A balanced scorecard for health services in Afghanistan

15. Performance indicators for public mental healthcare: a systematic international inventory (Lauriks et al. 2012)

8. Dutch Hospital public performance indicators on patient safety and clinical effectiveness (Berg et al., 2005)

• Literature sources used to extract selection criteria:

Lessons for Healthcare Managers

• Select indicators and design a measurement system that is linked to and aligned with their organization’s goals, business strategy and customer and stakeholder requirements

• Use a varied and balanced set of indicators to ensure that one area of performance is not unintentionally excelling at the expense of others

EXERCISES & GROUP WORK

KPI MEASUREMENT AND REPORTING STEPS: INCLUDING IDENTIFYING, MEASURING, VALIDATING, ANALYZING, AND REPORTING DATA

Reminder

A Guide to Start from ….

Phase I – Educate the Organization

Phase II – Find the Right Things to

Measure

Phase III – Build a Data Collection

Strategy

Phase IV – Build a Performance Report

Phase V – Refine and Adjust

Data Collection

Templates

List of Program

Indicators

Performance

Report

Update the

process

Common

Understanding

Phase III – Build a Data Collection Strategy

Phase III is made up of two steps:

Step 1. Develop a Data Collection Strategy

Step 2. Complete Data Collection Templates for

Chosen Indicators.

The aim of this Phase is to determine in a step-by-step approach what is required to report regularly on each of

the indicators that has been selected.

Step 1

Develop a Data Collection Strategy

For each indicator, it is important to clarify key points such as:

individuals responsible for data collection and reporting

data source(s)

data availability

timeline and resource requirements to initiate data collection.

QualityandAccreditationDirectorate

IndicatorID IndicatorSet IndicatornameG-2 Genericindicator Cancelledoperations

IndicatorStatementPercentageofelectiveoperationscancelledonthedayof,orafter,admission

NumeratorstatementThenumberoflastminutecancelledoperationsfromoperationlists.

NumeratorexcludedpopulationNone

DenominatorstatementTotalnumberofscheduledelectiveoperationsintheoperationlists.

DenominatorexcludedpopulationCasualtyTheatreoperations,Day-casesurgery

Rationale TypeofindicatorThecancellationofoperationsatshortnoticeisdistressingandinconvenientforpatients.Italsohasanegativeimpactontheutilizationofoperatingtheatres

inthehospital,whichleadstomassivelossesinmoney,timeandeffort.

Process,rate-based

Datereportedas

Percentage

Improvementreportedas

Decreaseinpercentage

Sampling

None

Riskadjustment

None

Suggestedanalysis

None

Definitions SubcategoriesLastminutecancelledoperations:operationscancelledonthedaypatientsareduetoarrive,afterarrivaltohospital,oronthedayoftheiroperation/surgery.

UnitSurgicalprocedureOrderofoperationinlistPreoperativelengthofstay

Reasonforcancellation

Frequencyofanalysis

Quarterly

Datasource

Incidentreportform(AQAD-IR-CO)–tobefilledoutbytheOperatingTheatreHeadNurseandSurgeonorAnaesthetist,andtobecollectedbythehealthinformationspecialistsfromAccreditationandQualityAssuranceDirectorate.

Createdon6/12/2011Revisedon15/1/2013

Sample: Data collection form

from Kuwait

Employee surveys

Qualitative Data

(on-going consultations,

exit interviews, feedback,

etc.)

Administrative Data

(absenteeism, turnover,

unfilled positions, training

investments, etc.)

Data Sources and Triangulation

DATA COLLECTION

Collect Data

•How did you collect the data?

Is the data reliable?

• Degree to which data are accurate and consistent across repeated measures

• Is data being collected in the same way?

Analyze and refine

• Does the indicator provide you with useful information for your work plan?

• Does the indicator help you identify opportunities for quality improvement?

• Does the indicator need to be revised?

Data Analysis and Reporting

Data Rich and Information Poor

• While immersed in data and reporting, healthcare managers

face the risk of being “data rich and information poor” about

how their unit, department or organization is actually

performing

Performance Measurement Data for Better Decision Making

• Whether performance measurement data are being used by

management, governance and stakeholders for better decision

making

• The true value associated with performance measurement will

only be realized when it can be shown conclusively that

tangible improvements in care are being engendered as a

byproduct of the measurement, and that managers and key

stakeholders are making data driven decisions.

Reporting of Results

• Share results with

• Other departments/units in your organization

• Management

• Present the information in a clear and interesting way, using an appropriate tool

Picturing the Data

• There are many valuable tools for interpreting and presenting

data

• Pie chart, histogram, run chart, flow chart, tables

• These tools can improve decision making, trigger attention and

direct improvement actions

Interpreting Data

• Key questions

• What are the trends or patterns in the data?

• What do the trends and patterns signify?

• Are there other factors involved?

• Are there opportunities for improvement?

• What is the benchmark?

Yes

No

No response

Pie Chart % patients who answered Yes for the question “Would you recommend this

hospital to your family and friends?”

Indicator:

% patients who answered Yes for the question “Would you recommend

this provider’s office to your family and friends?”

10% (15/150)

5% (8/150)

85% (127/150)

Run Chart

The Data-to-Action Cycle

KNOWLEDGE

• Study processes in

detail

• Benchmark results

• Redesign processes

based on best

practices

ACTION

• Publish results

• Use technology to

automate processes

and give feedback.

INFORMATION

• Answer clinical and

financial questions

• Choose

opportunities for

improvement

DATA

• Gather data

regarding costs,

quality, satisfaction

for key care

processes

Month:

Med errors:

April

14

Med errors:

Jan. Feb. Mar. Apr. May June July Aug. Sept. Oct. Nov.

Data on April Medication Errors for Unit 12BG

Information on Medication Errors for the Year

15

10

5

0

Decisions Have Been Taken To Improve the Medication Delivery Process

Medication Errors

Translating Data into Information

• Linking information to improvement

• Producing more reports will only improve outcomes if the information can be easily accessed and effectively used by clinicians and managers in health care organizations and governments officials making decision about findings and program developments

Generating Information

The purpose of data analysis is insight. The problem with our information

age was succinctly stated by Daniel Boorstein (Wheeler 1993):

“information is random and miscellaneous, but knowledge is orderly and

cumulative.”

Before information can be useful it must be analyzed, interpreted, and

assimilated.

VALIDATING DATA

Validating data

Review and verify the collected data:

• Have the results been reported in detail as requested, including stratification

and time periods?

• Do results from national sources match those reported from sources of

international comparisons?

• If not, do methodological differences account for the different results, and

can these be reconciled?

Validating data • Are results reported for previous years consistent?

• Historical results should be revised based on changes in

methodology.

• Where results have been restated note and explain the difference.

• Are there outliers in the historical trends or in the stratified results?

• Outliers should be reviewed with the facilities that reported the

results, explanations should be noted and/or corrections should be

made.

Validating data

• It is important to be explicit about data limitations and

understand the strengths and limitations of the data when

making concrete use of data.

• Identify issues of data availability, quality, coverage,

fragmentation and timeliness.

• For this reason, it is useful to report on the lessons learned and

options for strengthening the health information system.

Validating data Audits

• Examination of records (clinical, financial, personnel, etc.) and observations of or interviews with providers to verify contents and/or check accuracy of data and assess issues of under-performance.

• The process for conducting audits is presented in three phases: 1. before the audit, 2. during the audit, and 3. after the audit.

• Each phase comprises key steps to guide the audit team in conducting thorough, systematic, and consistent checks of data.

Key audit steps

PhaseI.Beforetheaudit

• Preparetheauditteam• Selecthealthcareproviders

• Selecttheindicators

• Selecttherepor ngperiod

• Determinethetypeandobjec vesoftheaudit

• Prepareforthevisit

PhaseII.Duringon-sitevisit

• Ini atetheon-siteprocess• Assessqualityofdatacollec onandrepor ng

• Assessissuesofunder-performance

• Concludetheon-sitevisit

PhaseIII.A ertheaudit

• Completeauditoutputs

PHASE I. BEFORE THE AUDIT

Prepare the audit team

•Two to four monitoring and evaluation professionals.

Select healthcare organizations, indicators, and reporting period

•Organizations that report non-compliance or under-performance

• Visits may be announced or unannounced, as deemed appropriate

by the audit team.

•Visits may assess quality of data collection and reporting

•Visits may assess compliance with remedial steps following issues

of non-compliance or under-performance

PHASE II. DURING THE ONSITE VISIT

Guiding framework on focus areas of the audit:

Datacollec onandrepor ng

-Aretherestandarddata-collec onandrepor ngformsthataresystema callyused?

-Isdatarecordedwithsufficientprecision/detailtomeasurerelevantindicators?

-Aresourcedocumentskeptandmadeavailableinaccordancewithawri enpolicy?

Datamanagement

-Doescleardocumenta onofcollec on,aggrega onandmanipula onstepsexist?

-Aredataqualitychallengesiden fiedandaremechanismsinplaceforaddressingthem?

-Arethereclearlydefinedandfollowedprocedurestoiden fyandreconcilediscrepancies?

-Arethereclearlydefinedandfollowedprocedurestoperiodicallyverifysourcedata?

Healthcarequalityimprovementandsafety

-Whatisthehealthcareorganiza on’sapproachtoqualitymanagementandsafety?

-Whataretheinvestmentsinresourcesorstaff,performancemeasurementsystems,andqualityimprovement?

-Aretherestepstakentoimprovequalityandpa entandstaffsafety?

-Areperformancemeasureresultsusedtoiden fypriori esforimprovement?

Availability, meliness,andcompletenessofdata

-Isthereanyindica onthatsourcedocumentsaremissing?

-Areallavailablesourcedocumentscomplete?

-Arethereproceduresinplacetopreventrepor ngerrors?

Integrityandreliabilityofdata

-Arethereanydifferencesbetweenthereporteddataanddatafromsourcedocuments?

-Arethereanydifferencesbetweenthereporteddataandotherdata-sources?

I. Interviews:

1. Are there standard data-collection and reporting forms that are systematically used?

2. Is data recorded with sufficient precision/detail to measure relevant indicators?

3. Are source documents kept and made available in accordance with a written policy?

4. Does clear documentation of collection, aggregation and manipulation steps exist?

5. Are data quality challenges identified and are mechanisms in place for addressing them?

6. Are there clearly defined procedures to identify and reconcile discrepancies in reports?

7. Are there clearly defined procedures to verify source data?

Assess quality of data collection and reporting

II. Documentation review

1. Check availability and completeness of source documents and ensure that all the completed source documents fall within the reporting period being audited.

• Is there any indication that source documents are missing? If yes, determine how this might have influenced the reported numbers.

• Are all available source documents complete? If no, determine how this might have influenced the reported numbers.

2. Verify that procedures are in place to prevent reporting errors.

Assess quality of data collection and reporting

III. Trace and verify data reported

1. Recount the numbers from available source documents

2. Compare the source numbers to the reported numbers

3. Identify reasons for any differences.

• Possible reasons for discrepancies could include simple data entry or arithmetic errors.

IV. Cross-checks

• Perform feasible cross-checks of the reported data with other data-sources, if possible (e.g. inventory records, laboratory reports, registers, etc.).

Assess quality of data collection and reporting

Assess issues of under-performance

Interview staff responsible for quality monitoring and

improvement and other staff related to the selected indicators

Discuss:

a. The healthcare organization’s approach to quality

management

b. Establishing a culture of safety

c. Identifying, managing, and addressing safety risks

d. Investing in resources or staff, performance measurement

systems, quality improvement initiatives.

e. Staffing, provider safety and work/life issues

Guiding tool for audit on quality of data collection and reporting

PHASE III. AFTER THE AUDIT

The audit team will provide an assessment of the status of the organization following the audit:

• Passed with no major issues to flag

• Passed but with minor issues to discuss with the organization

• Major issues are flagged that require follow up with the organization

INTERPRETING DATA

Interpreting the data

• Collect and organize indicator results.

• Review and verify results.

• Acknowledge caveats and data limitations.

• Present a summary of key findings and data issues for each indicator.

• Report on lessons learned and opportunities for strengthening

information systems.

• These actions should be performed with regular contacts and direct

support from the data holders and stakeholders from the health

system.

Interpreting the data and organizing and writing the report

• To build a complete picture, three levels should be brought

together: the individual indicators, the performance

dimensions and the overall health system.

Individual indicators

• The first component is interpretation of the individual

indicators in context, identifying reference points, trends and

relationships with other indicators:

• How have results changed over time?

• Are the results where they should be?

• How much do the results vary across regions or

subpopulations?

How have results changed over time?

What historical trend data is available?

• There may be no historical results for recent indicators.

• Results from different methodologies cannot be compared

directly.

• Limitations in available historical data should be noted in the

analysis.

How quickly would results be expected to change?

• Results for measures of health status, such as inpatient

mortality, probably change more slowly than results related to

processes, such as changes in length of hospital stay.

Are the results where they should be? Have targets been established for the indicators?

•Results can be compared directly to this target. Synergies can be

achieved with the National Health Strategy when possible.

What has been achieved in other countries’ health systems?

•A set of comparator health systems (regional or international)

could provide some insights into what might be achieved, and

where there are opportunities to improve the health system in

the country.

•Careful, context-specific comparison and explanation of the

reasons for variations are important for benchmarking.

Setting references for benchmarking

How high to set the bar?

•The identification of international, regional, or national reference points is a

critical exercise.

•It is helpful to consider which countries (or regions) might offer the most

relevant experience rather then focusing exclusively on the countries with the

greatest similarity to one’s own.

•Combining elements from the different reference groups is the best way to

make a sound comparison and, more importantly, to identify opportunities

for improving the health system.

•This will be further discussed on Day 2.

Interpreting the data and organizing and writing the report

Overall picture

•The relationships among the performance dimensions is used to

paint the “big picture” of overall healthcare performance.

•The big picture should again refer to key policy/ strategy

questions.

Interpreting the data and organizing and writing the report

Summary points

•Assess achievements on each indicator with regard to evolution over time,

comparisons with targets or international reference points.

•Bring together all indicators within a dimension and complementary sources

of information to build a comprehensive picture for each dimension.

•Develop the “big picture” and “tell the story” of the performance of the

system/ organization as a whole by linking all dimensions within the

performance framework.

GUIDELINES FOR ACTION Examples from Regional Initiatives

In-patient Mortality Rate Guidelines for action Scenarios Action

Rates higher than

benchmarks

o Identify outliers for example by length of stay, gender, age and co-morbidities,

diagnosis, cause of death

o Investigate reasons for high rates:

o Are deaths preventable?

o Are they related to shortcomings in healthcare or healthcare settings?

o Are they linked to non- compliance with clinical guidelines?

o Compare figures across hospitals and derive lessons from hospitals with low rates

o Implement action plan for improvement

o Continue monitoring rates closely

Rates lower than

benchmarks

o Lower rate is preferred

o Investigate very low rates as they may indicate early discharges or transfers rather

than high quality of care

Rates at/or slightly

vary from

benchmark

o Continue monitoring rates closely

o Identify outliers, if any

Readmission for same Diagnosis within 30 days of discharge Guidelines for action

Scenarios Action

Rates higher than

benchmarks

o Identify outliers for example by gender, age, and diagnosis of all patients.

o Investigate reasons for high rates:

o Are they related to shortcomings in healthcare or healthcare settings?

o Are they linked to non- compliance with clinical guidelines and the

assessment of the readiness of patients for discharge?

o Compare figures across hospitals/ divisions and derive lessons from

settings with low rates

o Implement action plan for improvement

o Continue monitoring rates closely

Rates lower than

benchmarks

o Lower rate is preferred

Rates at/or slightly vary

from benchmark

o Continue monitoring rates closely

o Identify outliers, if any

Focus on Kuwait

• MOH reports indicators back to providers in a “dashboard”

specific to each Clinical Area.

Example: Appendectomy Indicators, Hospital X

Current Results

Range in 6 MOH Hospitals (Min- Max)

National Result

Compared to National Performance

Less than 12-Hours Interval between Admission & Appendectomy (H)

85% 4 - 90% 84%

Normal Pathology (L)

20% 4- 30% 9.8%

Perforated Appendicitis (L)

6% 4- 30% 13.9%

Missed Appendicitis (L)

3% 0- 14% 4.1%

Antibiotic Prophylaxis (H)

40% 14- 50% 47.1%

Post Appendectomy Complications (L)

3% 0- 11% 4.5%

Focus on Kuwait: Challenges

• Need to better report on a set of balanced indicators to provide

“overall picture” on performance of healthcare organization/

health system Need for a Balanced Scorecard (will discuss

on Day 2)

• Better comparisons to international/ regional benchmarks or

targets is needed

• Limited information on context to better understand

performance of healthcare providers

Discussion