Embed Size (px)
DESCRIPTION
Citation preview
The use of evidence in developing countries:
Experiences and Challenges
Miguel Szekely, October 11, 2010
3ie
3ie aims to improve the lives of people in the developing world by supporting the production
and use of better evidence of what works, when, why and for how much
Why are we still asking this question?
• 21st Century
• Decades (centuries) of policy interventions
• The fact is that we don’t have powerful answers (in general)
• Why is it necessary to have a 3ie?
Evaluation in other areas
• Building a bridge (stands and resists, or breaks down)
• Economic management (growth, employment, inflation, exchange rate stability)
Tangible results observable and evident in the short run
Constituencies can vote depending on measurable tangible results
Evaluation in developing policy (education, poverty alleviation)
• Problems with measuring welfare
• Results take a long time to manifest and to be observed
• Difficult to find who to blame (Toyota, Spaceship vs teacher)
• Large bureaucracies become constituencies and feel threatened
Contents:
1. The need for improving policy impact in developing countries
2. Experiences from Mexico in Social Development and Education
3. Thinking about the problem
4. Ways forward
Contents:
1. The need for improving policy impact in developing countries
2. Experiences from Mexico in Social Development and Education
3. Thinking about the problem
4. Ways forward
Normally you would expect ….
Evidence-based
Policy designImplementation
Evaluation
Monitoring
EvidenceDecisions
Improvement
Changes
Usually you see….
Implementation
Decisions
Improvement
Changes
Unfortunately…
• Very seldom are policy decisions taken, based on scientific (or other) evidence
• Has been like this for years in important areas such as education, (health) and poverty alleviation programs
• Good opportunity for 3ie
Contents:
1. The need for improving policy impact in developing countries
2. Experiences from Mexico in Social Development and Education
3. Thinking about the problem
4. Ways forward
4 experiences
• Poverty measurement
• Social program evaluation
• Standardized tests in Education
• Evaluation of school principals
4 regularities
• Ignored
• Angry
(people take it personal)
• Scared
• Use
Contents:
1. The need for improving policy impact in developing countries
2. Experiences from Mexico in Social Development and Education
3. Thinking about the problem
4. Ways forward
Very similar to principal-agent problem
Politicians Evaluators
Utility =
• Get reelected• Re-appointed• Keep the boat
afloat
Utility =
• Academic purity
• Publications• Prestige
Improvement design and maximizing policy
impact
1. Different Marginal Propensity to “consume and produce” evaluations
Politicians Evaluators
High MPc due to need to show results (deliver to constituencies, and little time to reap benefits)
Low MPc due to time needed to increase quality of academic output (probability of publication)
2. Different expected returns from evaluating
Politicians Evaluators
Uncertain results, uncertain political capitalization (only through positive impact)
Publication depends on methodological and theoretical rigorousness, not on positive or negative effects
3. Different priorities
Politicians Evaluators
• Political feasibility• Economic feasibility• Social feasibility
• Academic rigorousness• Production of knowledge
Contents:
1. The need for improving policy impact in developing countries
2. Experiences from Mexico in Social Development and Education
3. Thinking about the problem
4. Ways forward
How can we get closer (aligning incentives/cultural change)
Politicians Evaluators
• Link budget to results• Link hiring and firing to results• Institutionalize evaluation
practice (de-link from political cycles)
• Offer evaluations of a set of instruments to choose
• Evaluation NOT the final product (implementing improvement is)
• Pay for implementation strategies', not only for evaluation
• Incentives on successful implementation (3ie)
• Quicker ways of disseminating results
Critical role for the media
