Upload
edward-donald-ellis
View
214
Download
0
Embed Size (px)
Citation preview
Self Review Training
24 June 2010
Performance Improvement Framework
Central
Agencies DPMC SSC Treasury
2PIF challenge: Showing value to a sceptical public (or reviewer)
-10%
0%
10%
20%
30%
40%
50%
99/00 00/01 01/02 02/03 03/04 04/05 05/06 06/07 07/08 HYEFU 08/09
Ch
an
ge
in R
ea
l Pri
ce
or
Sta
ff
Financial Year
Mythological Ministry of MagicMission: Making Money Materialise
Staff Growth (FTE)
Personnel Expense per Employee (FTE)
Personnel Expenses (ex. ACC)
Total Operating Expenses
Central
Agencies DPMC SSC Treasury
#1.TechnicalEfficiency
#2.Allocative Efficiency
#3.Cost
Effectiveness
VALUE FOR MONEY
(Best Result for NZers)
Value for Money is hard to assess directly … but poor results at ANY corner of the ‘triangle’ shows under-performance
Central
Agencies DPMC SSC Treasury
#1. PRICE, QUANTITY & STANDARDS (TECHNICAL EFFICIENCY)
Effect of Flying Hours on Maritime Patrol Prices
-20%
-10%
0%
10%
20%
30%
40%
50%
60%
70%
80%
FY 98/99 FY 99/00 FY 00/01 FY 01/02 FY 02/03 FY 03/04 FY 04/05 FY 05/06 FY 06/07
Financial Year
Rea
l cha
nge
in p
rice
or
outp
ut
Total flying hours
Real cost per flying hour
Real cost (Personnel)
Real cost (Total Expenses)
Gulf of Oman (Output D16 - 659 extra hours)
1. How did prices of big outputs change?
2. Did output volume or quality change proportionately?
3. Did quality change?4. Did o’heads grow?5. Can we show
citizens actually benefit from this?
‘Did new spending actually increase or improve output ?’
Central
Agencies DPMC SSC Treasury
#2. DID OUTPUT GET TO THE PROBLEM AREA, & EFFECT CHANGE
Large Proportion of New Prison Space Used for Less Serious or Less Risky Offenders
-200
-150
-100
-50
0
50
100
150
200
250
300
Ext
ra P
riso
ners
('03
vs
'07)
Seriousness Score
Reoffending Risk
Seriousness Score 198 251 218 137 63 -1 15 104 17
Reoffending Risk -157 155 127 85 99 81 129 164 118 247
" 0 " " 1 " " 2 " " 3 " " 4 " " 5 " " 6 " " 7 " " 8 " " 9 " " 10 "
Can we link extra $$$ & output to priority groups, areas, needs & opportunities?
Who has / is the problem? Who got interventions or $$$? Who missed out?
How well do allocative rules really work?
Can we deliver more in areas that will benefit more, & limit waste?
Allocative Efficiency ‘Can we get better results without spending more ?’
Central
Agencies DPMC SSC Treasury
#3. INCREASED PRICE (OR OUTPUT) IMPROVES CORE OUTCOMES
1. Did end outcomes improve much? 2. Did gains mirror / equal spending?3. Did gains precede or lag changes?4. Intermediate outcomes improved?
‘Are results good, bad or simply indifferent ?’
Look for diminishing returns.Scale & timing ($ vs. outcome)?
Investing to reduce crime …
-20%
-10%
0%
10%
20%
30%
Ch
an
ge
Re
lati
ve
to
FY
01
/02
Real Funding Violence
Drugs & AntiSocial Dishonesty
ERO Approach - Context
ERO evaluates the performance of schools and early childhood services
ERO’s self-review was undertaken as a pilot review for the PIF (July 2009)
The context for ERO’s self-review was “the alignment of services and supporting infrastructure to better support Government priorities for education”
This organisation-wide context meant that ERO tested all aspects of the framework
Brent McPherson- June 2010
ERO Approach - Team
The team:
was appointed by the Chief Executive
had knowledge about the business and governance
had access to information
had evaluation skills
Level Position
(tier 2) National Manager Corporate Services
(tier 2) National Manager Review Services (Central)
(tier 3) Strategy & Performance Manager
(tier 3) Manager Information Services
(tier 3) Chief Finance Officer
(tier 3) Manager Standards & Contracts
Brent McPherson- June 2010
ERO Approach - Method
Critical areas identified in the framework were allocated between review members - in pairs (allocated areas were independent to current portfolios)
Each pair recorded their assessment, action points & evidence in a template (to justify their judgement)
The team synthesised results & discussed judgements (moderation)
The report (and recommendations) was discussed with the Senior Management team
Brent McPherson- June 2010
Benefits to ERO
The self-review: reinforced a commitment to all staff to
promote a culture of continuous improvement reinforced ERO’s own credibility
(ERO promotes evaluation capacity externally) identified a set of recommendations for action
(the recommendations have been accepted by SMT and integrated into the work programme)
encouraged the Chief Executive to think about ERO’s future approach to internal evaluation
Brent McPherson- June 2010
Why Do A Self-Review?
respond to Govt priorities (ie how to “maximise the value received from its education services”)
promote continuous improvement through a formal and structured approach
promote accountability – and a balance of internal and external review (ERO ‘s approach to schools)
prepare for a formal review review the organisation (do it in chunks?) basis for engaging with the Minister (PIAs, SOI)
Brent McPherson- June 2010
Issues and ChallengesTo improve the value of the self-review, next time ERO would….. plan aheadschedule at a time appropriate to ERO(and set time aside to conduct the self-review) engage managers and staff in the planning (and engage the senior management team throughout) identify priorities – to reduce time spent on areas where there are high levels of comfortincorporate a greater field component - to test the cascade of initiatives to the field (ERO is a field agency)develop an associated communications plan
Brent McPherson- June 2010
Issues and Challenges
using a moderation process to synthesise the results
ERO removed bias by:
ensuring that the views expressed were backed up by supporting evidence
ensuring that review team members did not review their own portfolio
Brent McPherson- June 2010
Issues and Challenges
ensure the team has access to evaluation skills
Other recommendations/challenges:
plan for conflicting opinion
treat it as a formal review (ready the documents)
what is the current culture of the agency?
Brent McPherson- June 2010