24
INNOVATIONS IN GOVERNANCE MEASUREMENT Public Accountability Mechanisms (PAM) Initiative Francesca Recanatini & Stephanie E. Trapnell April 26, 2013 1

Innovations in governance measurement

  • Upload
    devon

  • View
    26

  • Download
    0

Embed Size (px)

DESCRIPTION

Innovations in governance measurement. Francesca Recanatini & Stephanie E. Trapnell April 26, 2013. Public Accountability Mechanisms (PAM) Initiative. Outline. Introductions Objectives and potential outputs ISPMS, AGI, and behavioral dimensions of measurement - PowerPoint PPT Presentation

Citation preview

Page 1: Innovations in governance measurement

INNOVATIONS IN GOVERNANCE MEASUREMENT

Public Accountability Mechanisms (PAM) Initiative

Francesca Recanatini & Stephanie E. TrapnellApril 26, 2013

1

Page 2: Innovations in governance measurement

Outline

IntroductionsObjectives and potential outputs ISPMS, AGI, and behavioral dimensions of measurement

Public Accountability Mechanisms De jure and De facto

“Big ideas” for today

2

Page 3: Innovations in governance measurement

Objectives of the Workshop

To discuss innovations in the measurement of “de facto” aspects of governance focusing on: the gaps in existing data coverage, the strengths and weaknesses of recent

data collection methodologies, and the sustainability of these efforts.

If possible, to identify a “core” set of governance areas and related indicators (both existing and potential)

3

Page 4: Innovations in governance measurement

Outputs4

Menu of governance indicators and efforts

Draft note on methodologies Annotated bibliography of sources

Technical expert group (TEG)

Page 5: Innovations in governance measurement

A proposal and a new way forward

ISPMS, AGI, and behavioral impacts5

Page 6: Innovations in governance measurement

The challenge

To measure performance of institutions, and in particular, public sector institutions

To identify areas for institutional reform To evaluate the impact of change in

policies at the country and local level

6

Page 7: Innovations in governance measurement

Complementary sets of indicators

Measurement of the performance of Public Sector Management (PSM) systems in the middle of the public sector results chain.

Measurement of the way in which the government is held to account through political and non-executive institutions and directly by the public.

7

Indicators of the Strength of Public Management Systems (ISPMS)

Actionable Governance Indicators (AGIs)

Page 8: Innovations in governance measurement

Public sector results chain within a broader governance environment

8B

roader

Govern

ance

Envir

onm

ent B

roader G

overn

ance

Enviro

nm

ent

Page 9: Innovations in governance measurement

Moving forward: Utility criteria + feasibility

Criterion Definition

1. Action-worthy We know (or strongly believe) that they contribute to results

2. Actionable They are amenable to government action and project interventions

3. Behavioral Focus on function, not form

4. Replicable Generated transparently and can be reproduced by others

…and 2 “feasibility” criteria.

Criterion Definition

1. Country coverage

Available for a minimum of 20 countries

2. Time series Collected repeatedly

Page 10: Innovations in governance measurement

Selected Examples

1. “Does the executive’s budget or any supporting budget documentation present expenditures for the budget year that are classified by administrative units?” (Open Budget Survey)

2. Does a legislative committee hold public hearings on the individual budgets of central government administrative units in which testimony from the public is heard? (Open Budget Survey)

3. Tax payers access to information on tax liabilities and administrative procedures (PEFA)

10

Page 11: Innovations in governance measurement

De jure vs De facto

An illustration: Public Accountability Mechanisms

11

Page 12: Innovations in governance measurement

Public Accountability Mechanisms (PAM)de jure indicators

Financial disclosure (interests, assets,

income)217 indicators

2008, 2012

Freedom of information

37 indicators2010

Conflict of interest restrictions

124 indicators2011

Immunity protections56 indicators

2012

12

Page 13: Innovations in governance measurement

Outputs of de jure data13

Library of laws Analytical publications on the design of

public accountability mechanisms Data, including qualitative and quantitative

datasets, country profiles, and descriptive statistics

Country reports on enabling governance environment

….But how do we know what happens in practice?

Page 14: Innovations in governance measurement

Freedom of information systems in practice:

public sector management functions

1.1 Facilities 1.2 Data/Records 1.3 Human

resources 1.4 Financial 1.5 Policy

2.1 Demystification*

2.2 Responsiveness

2.3 Appeals 2.4 Proactive

Disclosure

14

1.0 Administrative Functions

2.0 Disclosure Functions

* clarity about government processes, rules, and decisions

Page 15: Innovations in governance measurement

Freedom of information systems:

Immediate impacts/Intermediate outcomes

15

3.1 Public engagement: Extent to which the public understands, believes in, and engages with the freedom of information process.

3.2 Government commitment: Extent to which the government supports the freedom of information regime, including efforts to establish participatory decision-making

3.3 Administrative culture: Extent to which the bureaucratic culture has shifted from principle of secrecy to openness.

3.4 Operational efficiency: Extent of improvement in operations and decision-making within the organization.

Page 16: Innovations in governance measurement

How do you measure impacts in a governance “ecosystem”

16

Participation of civil society organizations and citizens Accountability and enforcement by parliament, judiciary,

police, supreme audit institution and the ombudsman Political will that enables meaningful change, including

protection from retribution, lack of clientilism, and an environment that maintains appropriate incentives for civil servants

Foundational support mechanisms found in public sector systems, such as freedom of information frameworks, conflict of interest rules, financial disclosure practices, and human resources and policy management.

Page 17: Innovations in governance measurement

Methodologies matter17

Surveys: validation issues, response problems Expert assessment: time-consuming, with

degrees of subjectivity Household surveys: costly, time-consuming,

logistical difficulties Economic/social indicators: collected by

whom? National statistics agencies, multi-laterals, NGOs, donors….

Outcome mapping/process tracing: lengthy, involved, requires local capacity

Page 18: Innovations in governance measurement

So where do we start?18

Methodology

IndicatorsConcepts

Page 19: Innovations in governance measurement

“Big ideas” for today19

Actionability and action-worthiness Change in behavior and not in function Sustainable measurement practices Fuzzy concepts and methodologies Practical applications of data Challenges in de facto measurement

Page 20: Innovations in governance measurement

Some emerging trade-offs (or complementarities?)

Global focus versus sub-national focus? Existing data or new data? Focused on public sector or broad

governance issues? Perception-based versus objective data?

Or both? Using a single tool or multiple tools? Detailed focus on a specific case/country

20

Page 21: Innovations in governance measurement

Some emerging issues

Areas of focus? Tax administration Human resources Rule of law

Looking forward Clarify the focus of the effort and

link to a clear research question Integrate the issue of sustainability Benchmarking Understanding strengths of

different types of respondents Clear definition of the context Dissemination and wide use of the

data (and need for a strategy) New set of tools for certain

phenomena (high level corruption)

21

Page 22: Innovations in governance measurement

PAM website: www.agidata.org/pamAGI data portal: www.agidata.orgMetrics & Accountability: http://go.worldbank.org/H1K725TJV0

Thank you!22

Page 23: Innovations in governance measurement

23

Inputs Outputs

Rules (Institutiona

l Arrangeme

nts)

Organizational

Capacities

Governance System

Performance

Outcomes

The Missing Middle

Resources invested in projects to deliver its outputs. Examples: Funding, contracts, materials

Goods and services produced by the project.Examples: Surveys and Trainings conducted, Laws revised, Agencies established

Desired state of well-being -- a set of conditions, experiences or behaviors – that is the goal for change or improvement.Examples: Maternal/infant mortality rates, Standardized test scores of K-8 students

Actionable Governance Indicators (AGIs) focus

on specific and narrowly-defined

aspects of governance.

Page 24: Innovations in governance measurement

24