The PEFA Indicators – How are they being used

Actionable Governance Indicators Course - April 29th, 2010

Frans RonsholtPEFA Secretariat

2

Characteristics and roll-out of the PEFA / PFM Performance Measurement Framework

What may PEFA Assessments be used for?

Content

PEFA indicators characteristics PFM Performance Measurement Framework

– 28 performance indicators + 3 donor indicators– Evidence based, rated on 4-point ordinal scale– PFM Performance Report with standard format

Launched in June 2005 – multi-agency initiative Application is decentralized

– Country teams/stakeholder groups decide if, when and how to implement the assessment work

PEFA Secretariat – Neutral body - supports users, monitors application

3

PFM Links to Development Goals

PFM system performance

Budgetary Outcomes

Fiscal / Exp Policy Goals

Dev Goals

MDGs, PRSP, Political Manifesto

Budget deficit, Sector allocations, Investment, Debt

ratio, Tax burden etc

Fiscal discipline, Strategic allocation,

Operational efficiency

PEFA Framework

Other influencing factors

Evolution of Number of Assessments

6

Regional Coverage of PEFA AssessmentsApril 2010

AFR EAP ECA LAC MENA SAR Other Total0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Not Covered

Planned

Commenced

Substantially Completed

Regions

Perc

enta

ge o

f Reg

ion

Cove

red

7

Data coverage– 175 assessments done in 110 countries– 31 repeat assessments in 27 countries

Increasingly used for Sub-National government– 27 SN assessments in 14 countries

Wide stakeholder involvement– WB & EC leading 85% of assessments, 25 other agencies

involved– Government leadership/self-assessment increasing, but not

yet norm Public access to reports

− Only 56% of final reports in public domain− Links provided at www.pefa.org

PEFA Framework adoption

Assessment in HICs & MICs - examples OECD/HICs

– Norway (national), Switzerland (Canton/state) Large Upper MICs

– Brazil, Turkey, Belarus, (Russian Federation), South Africa

Large Lower MICs– India, Kazakhstan, Ukraine, Morocco, Egypt,

Thailand, Philippines, Indonesia, Colombia, Peru

8

9

What can countries use the PEFA assessments for?

Harmonize information needs for all stakeholders

Inform PFM reform formulation Monitor results of reform efforts Aid allocation and operational decisions Cross-country comparison

Harmonize information needs Create a shared view of PFM system

performance among all major stakeholders at country level – for various purposes

Reduce transactions costs of analytical work Provide a starting point for subsequent,

collaborative work on PFM reform and capacity building

10

Implement PFM reforms

Recommend PFM reform measures

Identify main PFM

weaknesses

High level performance

overview

Coverage of PFM-PR in Reform Cycle

Investigate underlying

causes

Formulate PFM reform

program

Identify main PFM

weaknesses

Recommend PFM reform measures

PFM-PR

Inform PFM reform formulation (1)

• PEFA report is one of several inputs – Identification of main strengths and weaknesses – and potential

impact on budgetary outcomes – Other factors: political economy, culture, constitution/legal,

resources, capacity at entry

• Ownership means government decisions on priorities– Government to consider all factors in deciding priorities– Allow ample space for government’s prioritization in the reform

dialogue with International Agencies

• Do not use Indicator scores simplistically – A low score is not sufficient justification for reform – Other factors: relative importance of subject, complexity

/timeframe for improvement, interdependence with other elements

Inform PFM reform formulation (2)• Complementary analysis to PEFA may be required

– Detailed analysis of underlying causes needed for formulation of detailed action plan

– Limit such analysis to priority areas – Drill-down tools – some exist, others under development

• Guidance on using reports for reform formulation – How to establish if a reform program is ‘credible’– Challenge to develop general approach and toolkit to

help government/donor teams identify priorities/sequencing– Such an approach could strengthen country ownership of

reform and coordination of donor support– Work in progress

Monitor results of reform efforts Schedule full repeat assessments – every 3-5

years Select a few indicators to monitor more

frequently Incorporate into the M&E component of the

PFM reform program (Kenya, Zambia)

Incorporate as monitoring tool in CAS (Bangladesh)

PEFA indicators being used for PFM reform program evaluations (IEG, Multi-donor evaluations)

14

Aid allocation & operational decisions Help to define main system weaknesses & related

safeguards for use of country systems PEFA indicators used by many agencies as input to

– fiduciary risk assessments (WB/CIFA, DFID, KfW, AFD) – aid allocation instruments (WB/CPIA, NL track record)

Do not use PEFA performance ratings as conditionality for disbursements– activity indicators may be more suitable– such measures to be under direct government control

15

Cross-country comparison Regional peer learning events

– Eastern Europe, West Africa, Caribbean, Pacific– Suitable for countries that share key characteristics– Often arranged in collaboration with IMF TACs

Research– AFTFM/Brookings study identifying regional performance

characteristics and reform trends– ODI/di Renzio study – correlating country characteristics

to PFM performance

16

17

Issues in country comparison• Comparison of two countries must be

done very cautiously: - Technical definitions may be different- Need to carefully read each report to understand

performance differences behind the scores.- Consider country context, ensure comparison of

like with like

• Comparing the scores alone can be misleading

18

Comparing groups of countries

• Aggregation requires three decisions

– Conversion from ordinal to numerical scale

– Weighting of indicators (generally and country specific)

– Weighting of countries (for country cluster analysis)

19

Comparing groups of countries

• No scientifically correct or superior basis for deciding conversion and weights exists– Each user takes those decisions on individual opinion– PEFA program does not endorse any particular

method

• In case aggregation is desired:– Be transparent on aggregation methods used– Discuss reasons for choice– Sensitivity analysis to illustrate impact on findings

Discussion