19
The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance Global Expert Team 9 May Gwyn Bevan

The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Embed Size (px)

Citation preview

Page 1: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework

Targets & Performance ManagementWorld Bank’s Public Sector Performance

Global Expert Team 9 May

Gwyn Bevan

Page 2: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Models of delivery of public services

Knights / Knaves

Econs / Humans

Prospect theory

Trust & Altruism (T&A)

Knights Humans Reward failure?

Hierarchy & Targets (H&T)

Knights & Knaves

Econs Sack staff?

Public Ranking (PR) Knights Humans Reputation?

Choice & Competition (C&C)

Knights & Knaves

Econs Winners & losers?

Page 3: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

UK’s natural experiment: before & after devolution (1999)

NHSs Schools

Before After Before After

EnglandT&A

H&T

PR

C&C

PRWales T&A C&C

PR

C&C

Page 4: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

H&T+PR: ‘naming & shaming’ spotlight + prospect theory

Staff in zero-rated hospital ‘devastating …hit right

down to workforce -- bad reports usually hit senior management upwards

nurses demand changing rooms .. because being accosted in streets’

Source: Mannion et al (2005)

‘dirty dozen’ zero-rated CEOs(English NHS)

Page 5: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Hospital waiting times: T&A H&T+PR

Numbers waiting elective admissions (England) (‘000s)

H&T+PR

Page 6: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Hospital waiting times: T&A

0

10

20

30

40

50

60

70

1997 1999 2001 2003 2005

England Wales

No/‘000 waiting > 3 months GP 1st outpatient appointment

Page 7: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Hospital waiting times. Natural experiment: T&A vs H&T+PR

051015202530354045

1997 1999 2001 2003 2005

England Wales

No/‘000 waiting > 3 months GP 1st outpatient appointment

H&T+PR

Page 8: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Ambulances:T&A

45

50

55

60

65

70

75

80

1999 2001 2003 2005 2007

England Wales

Source: Bevan & Hamblin (2009)

% responses life-threatening emergencies < 8 minutes Target

E & W

Page 9: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Ambulances. Natural experiment: T&A vs H&T+PR

45

50

55

60

65

70

75

80

1999 2001 2003 2005 2007

England Wales

Source: Bevan & Hamblin (2009)

% responses life-threatening emergencies < 8 minutes H&T+P

RTarget E

Target W

Page 10: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

35

45

55

65

1994 1996 1998 2000 2002 2004 2006 2008

England Wales

% pupils achieving > 5 good grades at 16

Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years

Schools: C&C+PR

Page 11: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

35

45

55

65

1994 1996 1998 2000 2002 2004 2006 2008

England Wales

Schools. Natural experiment: C&C+PR vs C&C

Note: data are missing for 1998, 2001 & 2005 & these have been estimated as the mean values from adjacent years

Wales stops PR

% pupils achieving > 5 good grades at 16

Page 12: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

H&T & PR: design

Residual domain R

Gages on P

No measures on P

Tin openers on P

Selected domain P

Page 13: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

H&T & PR : problems of gaming

Selected domain T: incentives apply to measures M[T]

Excluded domain N: unimportant or cannot be measured

Holmstrom & Milgram (1991)

Page 14: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

H&T & PR: types of gaming

N? Worsen

performance? M[T]?

Hit the target & miss the point

Misreporting & cheating

CollusionTarget M[T]

Domain N

Domain T

Page 15: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

For discussion

Theory of contingency mix of models

T&A performance measurement no gaming ineffective

PR + H&T powerful gaming Design to counter gaming

C&C choice attractive in principle Characteristics influence effectiveness

Page 16: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

For discussion C&C: Requirements

Universities Schools Hospitals

Money follows choice Supply-side flexibility Information Transaction costs Management freedom

Page 17: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

COFEPRIS: Commission for Protection against Sanitary Risks

9,000 registrations (value $1bn) required 150 registered in 2010

Page 18: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

Elements for transformation of failing organisations

English NHS COFEPRIS

Spotlight effect

Naming & shaming in media

Inform media of failure

‘Broken windows’

Waiting times Registrations

Prospect theory

Sack ‘failing’ CEOs Sack underperforming middle managers

Rewards Name elite organisations

Promote young able managers

Page 19: The Challenge of Designing ‘Good Enough’ Performance Measures & Results Framework Targets & Performance Management World Bank’s Public Sector Performance

COFEPRIS: Sanitary registrations

Source: Mikel Arriola Peñalosa (2012) Federal Commissioner for the Protection against Sanitary Risks