Embed Size (px)
Citation preview
Building Government-wide
Performance M&E Systems: Lessons from Global Good Practices
International Seminar: Improving Public Policy in Brazil through Results
Sao Paulo, September 29, 2015
Kathrin A. Plangemann - The World Bank
Que coisa?
2
What is a Government-wide Performance M&E System?
A Government-wide (GW), integrated performance M&E (PM&E) system is a:
policy-based;
set of aligned cross-cutting management instruments;
with a combination of tools, capacities and incentives;
that allows governments;
to achieve results;
to meet key national priorities;
strengthening public sector effectiveness and efficiency, transparency and accountability.
3
• Effectiveness: Disappointing results – nationally and sectorally from poor service- delivery.
• Efficiency- Insufficient value for money.
• Transparency: Lack of access to information
• Accountability- consequences for non-performance.
• Culture of doing things the way they have always been done, as opposed to culture of continuous improvement.
• Focus on activities without assessing their results or impact.
•Monitoring and reporting for compliance rather than for improvement.
• Poor programme planning, weaknesses in setting indicators and targets, weak logic models / theories of change.
•Weaknesses with design of data measurement and collection and data processes.
• Lack of reengineering of plans and business processes based on analysis of data.
• Evidence-based planning and decision making not sufficiently valued.
• Poor responsiveness and accountability to the citizens.
4
What are the problems that PM&E aims to address?
Objectives of GW PM&E Systems
Support national and sectoral planning, help implement the national goals.
Support budget decision-making.
Design policies, programs and projects and enhance their effectiveness.
Assist sector ministries/agencies in their management and performance drive.
Strengthen accountability and transparency.
5
Benefits of an effective GW PM&E System
Improved policy-making and implementation.
Enhanced government program performance.
Increased budget efficiency.
Greater coordination.
Performance-driven management culture.
Strengthened transparency and accountability.
6
7
PM&E Framework: Key Principles
Comprehensiveness: Government-wide coverage and integration with other public sector management systems.
Demand-driven approach: Focus on monitoring and evaluating what matters, on information that adds value for users.
Simplicity: Flexible, user-friendly system, building on existing M&E practices, promoting the use of M&E tools and information.
Credibility: Strong quality assurance, use of vetted data and rigorous processes to enhance transparency.
Accountability: Strengthening individual and institutional accountability, by strengthening internal and external/upward and downward accountability.
WBG Collaboration on GW PM&E
PM&E priority and cutting-edge reform area for countries, rich or poor.
Small, but rising global community of pioneers working on innovative public reform areas such as PM&E towards transformational change.
WBG is working on public sector performance and service-delivery globally across regions and countries.
Good practices: Canada, Mexico, Colombia, Chile, Australia, South Africa; also other emerging good practices at national/subnational levels (e.g. Malaysia, Indonesia).
8
Findings from Global WBG Collaboration
Good practice systems:
Integration: PM&E as key part of public sector reform for improved service-delivery.
Supply side: good quality data, effective systems, integrated tools and capacities for PM&E.
Demand side: strong internal and external demand for PM&E, with multiple champions, types of users and adequate incentives.
Utilization: Use of PM&E findings for policymaking, with budget and service-delivery implications reflects sustainability.
9
Institutional Leadership
10
Leadership/Champion: Presidency? MoF? MoP? NPC? Separate Ministry? Separate Agency? Intergovernmental body? Strategy Office? Statistics Office?
Advisory body, if any: Private sector? civil society? Academia? IFIs?
Staffing: Profiles? Experience? Numbers? Assignment Length? Incentives?
Coordination/Implementation mechanisms:
Cabinet? Steering Committee? Implementation Fora? Working Groups? Performance agreements?
11
Institutional Architecture: Options
Define government goals (e.g. Vision, NDP, presidential goals, government program, budget …).
Build an effective performance framework around it.
Design a cascading hierarchy of indicators from government goals, policies, programs to projects.
Make indicator development as much demand-driven as possible, with central quality assurance.
Ensure adequate use of theory of change.
Decide upfront on the use of particular PM&E tools to monitor and evaluate it.
Hold institutions accountable for delivery.
12
PM&E Alignment with Government Goals
Collaboration across Government
13
PM&E within the broader PSM Agenda
Usually M&E systems not developed from scratch - typically, governments have existing PM&E systems which may or may not have had traction.
While PM&E can be seen as a “system” in its own right, its success depends greatly on the broader institutional context in which it is created.
Do they create a demand for the performance information a M&E system can deliver?
Planning System
Budgeting System
HR Management
15
Public S ector P erformance
Enabling Environment
PB
P
PB
B
PM
S
PM
E
Performance Agenda
PM&E and National Planning Systems
Impacts
Outputs
Inputs
Outcomes
Strategic Objectives
Programs
Vision, NDP
Sub- Programs
Initiatives
Activities/Projects
Impact Indicators
Output/Efficiency/ Process Indicators
Outcome Indicators
16
Planning: The Malaysian System
Vision 2020: To achieve a self-sufficient industrialized nation by the year 2020 (stated in 1991).
The Government Transformation Programme (GTP) and the New Economic Model (NEM) built a framework of what has to be done to lift the economy towards a higher growth.
Nationwide policy planning document: Tenth Malaysia Plan 2011-2015 elaborated by Economic Planning Unit. Translates the GTP and NEM into action (policy direction and programs).
The 10thMP is built on five key strategic thrusts, with each of them entailing specific commitments and outputs to be achieved, each underpinned by the respective indicators.
M&E done among others by PERMANDU, a delivery unit.
17
18
Impact evaluation
Has the intervention had
impact at outcome and
impact level, and why
DESIGN
Design evaluation
Does the theory of
change seem strong?
Economic Evaluation
What are the
cost-benefits? Implementation
evaluation
- what is
happening and
why
Diagnostic
what is the underlying situation
and root causes of the problem
Expenditure Review (NT)
What are the activities, are
they cost-effective?
South Africa: Types of evaluations related to national planning
19
•What caused the changes?
•What direct/indirect, external/internal factors led to successful (or not) results?
STRATEGIC
Are we doing the right things?
•Is program implementation following anticipated paths?
•Are promised activities taking place as planned?
OPERATIONAL
Are we doing things right?
•Are there best practices or lessons learned that can be applied?
LEARNING
Are there better ways of doing
them?
PM&E and PBB: Key questions
20
PM&E and the Budget
Developing a program budget
classification
1 2 3 4 5 6 8 7
Developing performance
information and IT Systems
Integration of performance
information (PI) into budget preparation
Integrating accounting and cost information
systems with PI
Aligning policy objectives and medium term expenditure frameworks -
MTEF
Relaxing of inputs and administrative
controls
Setting incentives for managerial
performance
9
Implementing change management
strategies
Defining medium term policy
objectives, targets and indicators
10
Holding agencies
accountable for results
21
1 Budget Preparation
Budget execution
Budget Control Budget discussion and approval
Budget Cycle
Diagnostic Categories
I
Budget Planning II
Programmatic structure of the budget
Financial Accountability
2
3
IV program implementation
Performance information management
1 Budget Preparation
Budget execution
Budget discussion and approval
Budget Cycle
Reform Areas I
Budget Planning II
III V
2
3
4
IV
VI
Budget discussion
and approval
Budget execution and
VII
PM&E as a Key Reform towards HIC Status
Budget Monitoring and
Evaluation
PM&E and PMS
PM&E:
Institutional performance.
Focus on M&E of a cascading hierarchy of targets of policies, programs, projects.
Focus from input to outcomes/impact.
Consequences for policy-making?.
PMS:
Individual performance.
Targets at individual, unit/organizational level, aligned with sector/national priorities.
Focus level mainly on inputs/outputs.
Consequences for individual?
22
Different approaches towards performance management - with few direct linkages to PM&E: Only in a few OECD countries (UK, Denmark) is
agencies’ performance directly reflected in pay or performance bonus of senior management.
Recent research in OECD: recruitment & retention of capable staff & performance linked promotions are more important than performance contracts & performance-related pay.
In other countries, use of individual performance agreements & appraisal systems to link organizational goals to performance, influencing future career, but often suboptimal effectiveness.
23
OECD: PM&E and PMS
PPS/PBRS – Integrated Personnel Performance System/Performance Based Reward System: link employee performance to organizational performance, and the latter to the delivery of national goals (e.g. NDP): Effective recruitment and retention of staff. Strong staff engagement in the organization’s
mission. Well-designed incentives for staff to perform as
well as “opportunities to perform”. Tailored training and capacity building, ensuring
alignment to job needs and career development, annually reporting against results.
High quality performance dialogues with staff and effective follow-up.
24
Example: PM&E and PMS:
The Performance Management Pyramid
National
Dev. Plan
Strategic Plan
Business Plan
Performance Management
GOVERNMENT (Policies)
STRATEGIC (Program)
OPERATIONAL (Projects)
INDIVIDUAL (inputs/ outputs)
Ind
ivid
ual
O
utc
om
es
De
par
tam
en
tO
utc
om
es
Min
istr
eri
al
Ou
tco
me
s N
atio
nal
O
utc
om
es
Individual performance
Institutional performance
Carrots Sticks Sermons
Conduct “How are we doing” team meetings
Highlight good/bad results (using M&E)
High-level statements of endorsement
Awards or prizes for managing for results
Set performance targets
Awareness-raising seminars
Staff incentives, e.g. recruitment, promotion
Require performance “exception reporting”
Pilot rapid evaluations to demonstrate usefulness
Output or outcome-based performance triggers
Include information on results when appraising managers
Highlight examples of useful, influential M&E
Source: Mackay, K. 2007 How to Build M&E Systems to Support Better Government, World Bank.
26
PM&E Incentives
SA: Evaluation Incentives
Sticks • Once the evaluation is
selected – departments must follow the system
• Evaluation results will go to Cabinet, to Parliamentary Portfolio Committees and be made public
• Departments do not manage the evaluation but Steering Committee
Carrots • Part-funding the
evaluations • Providing training • Chance to participate
in ETWG – e.g. selecting evaluations
• Award for best evaluation
• Exposure - presenting at conferences
• Systems based on international good practice
Sermons • Cabinet endorsing
Policy and Plans • PM&E Minister
supporting • Emphasising that
learning not punitive • Study tours to give
message independently
27
1. Good design is key-takes time: strategic incrementalism vs. big bang.
2. Flexible dynamic approach – need for legal basis?
3. Key roles played by powerful M&E champion(s): Centrally-driven, by capable institution, yet demand-sensitive , persuasive and empowering.
4. Solid diagnostic crucial to inform and customize strategy design (“readiness assessment”).
5. Focus on M&E strategy and framework, ideally around national priorities –e.g. NDP/budget.
28
Lessons Learned for PM&E Systems
6. Supply-side: strengthening data and systems.
7. Demand-side : design and regularly adjust incentives framework.
8. Change management and risk management strategy, e.g. create quick wins early on, cycles of momentum, reporting harmonization and evaluation pilots.
9. Institutionalization: non-executive bodies, citizens.
10. Utilization, budget implications and integration into performance cycle for improved service-delivery as ultimate success factors.
29
Lessons Learned for PM&E Systems (2)
WHY ARE WE DOING THIS?
30
