Strategies for health system performance comparison: some international experience

Peter C. SmithEmeritus Professor of Health Policy

Imperial College London

Health system performance comparison

• Why comparison?• Some European efforts to date• What does HSPA look like?• Challenges and barriers• Conclusions

Tallinn Charter, WHO Europe, 2008 “We, the member states, commit

ourselves to:

o Promote shared values of solidarity, equity and participation ...

o Invest in health systems, and foster investment across sectors that influence health ...

o Promote transparency and be accountable ...o Make health systems more responsive ...o Engage stakeholders ...o Foster cross-country learning and

cooperation ...o Ensure that health systems are prepared and

able to respond to crises ...”

A European Union priority• Letter from Jean-Claude Juncker, President of the

European Commission, to incoming Commissioner for Health and Food Safety:

“… developing expertise on performance assessments of health systems, drawing lessons from recent experience, and from EU-funded research projects to build up country-specific and cross-country knowledge which can inform policies at national and European level.”

Brussels, 1 November 2014

The universal role of performance information

• … to enable actors throughout the system to make better decisions

• … measuring and reporting performance offers one of the most powerful instruments for incentivizing and facilitating performance improvement.

Three stages of HSPA• What aspects of performance to include:

– Reporting framework– Selection criteria– Practical limitations

• How to report that performance:– At what level of organization– How much aggregation / detail– How to adjust for contextual differences– Media and dissemination tools

• What to do with the results:– Who is intended to take action?– What incentives do they have to take action?– How is such action facilitated (eg benchmarking clubs)?

Conceptual Framework for Portuguese HSPA Report

The various stated goals of HSPA• Armenia: Enhance stewardship; Accountability; Transparency; Identify

policy priorities.• Belgium: Transparency and accountability; Comparisons with other

countries; Performance monitoring over time.• England: Performance management of public sector organizations.• Estonia: Enhance accountability; Enhance stewardship; Provide a

monitoring scheme for the National Health Plan.• Kyrgyzstan: Monitor progress and impact of health sector programmes;

Accountability to donors; Identify potential policy problem areas.• Portugal: Accountability; Inform policy.• Turkey: Provide a monitoring and evaluation scheme for the Health

Transformation Program; Transparency and accountability; Support the development of evidence-based policy-making; Guide governmental policy development; Identify policy priority areas.

World Health Organization, Case studies on health system performance assessment. A long-standing development in Europe, 2012, Copenhagen: World Health Organization Regional Office for Europe.

Some differences in focus

• Health (PT) vs health services (NL)• Between country (EE) vs within country (SE)• Aggregate outcomes (PT) vs distribution (BE) • Policymakers (planning) (GB) vs accountability

and transparency (NL)• Trends over time (NL) or cross-sectional

international comparison (BE)

Belgian Health Care Knowledge Centre (KCE), Performance of the Belgian health system report 2012.

Performance of the Belgian health system 2012: continuity of care

Eight enemies of valid comparison• ‘You cannot measure what we are trying to achieve.’ (eg mental

health)• ‘Our objectives go beyond what you are trying to measure.’(eg

quality of life)• ‘The data you are using are of poor quality and cannot be relied on.’• ‘There are external factors that influence our performance that you

have not taken account of.’ (eg low income population)• ‘The risk adjustment methods you have used are inadequate.’• ‘There is huge uncertainty in the reported measures.’• ‘The data you are using are out of date.’• ‘We are unique and cannot be compared with other institutions.’

EQ-5D: A Generic Quality of Life Measure

Towards patient-reported outcome measures (PROMS)

• Mandated in England since 2009– Hip replacement– Knee replacement– Hernia repair– Varicose veins

• Health status measured just before surgery and 3/6 months afterwards

• Independent data collection• Also relevant for routine population health

monitoring

HSPA: some leadership responsibilities• Development of a clear conceptual framework and a clear

vision of the purpose of performance measurement;• Seizing information technology possibilities;• Prompting and sustaining clinical involvement and leadership;• Mandating data collection mechanisms; • Information assurance and governance;• Development of analytic devices and capacity to help

understand the data;• Development of appropriate data presentational methods;• Dissemination and securing attention;• Stimulating action in response to performance measures;• Proper evaluation of performance measurement instruments.

Health System Performance Comparison

• Examines rationale for international comparison

• State of the art in six performance domains

• Current practice and resources

• Future prospects

20http://www.euro.who.int/en/who-we-are/partners/observatory/studies/health-system-performance-comparison.-an-agenda-for-policy,-information-and-research