Overlooked Links in the Results Chain

Vinod Thomas

Director-General and Senior Vice President

Xubei Luo

Senior Economist

Independent Evaluation Group

World Bank Group

June 2nd, 2011

When vital links are overlooked…

• Focusing only on relief and rehabilitation and not on prevention costs millions of lost lives and livelihood

• School access without learning hurts millions of children and their success in the marketplace

• Misplaced emphasis on liberalization and self-regulation without regulatory frameworks contributed to the global financial crisis

• Repeating successful water projects in the face of groundwater and coastal zone crises drowns development results

Crucial links in the results chain

►Focus on the right results

►Measure results right

►Use findings creatively to improve results

Focus on the right results

1. The urgent can drive out the important►One-off natural disaster responses limit

capacity for reacting in the future

• Almost half of the countries borrowing from the World Bank for disaster response did not mention disaster prevention in their development plans

►Mitigation and prevention can greatly reduce loss of life and damages

• The benefit-cost ratios of prevention range from 1.5 to 5.7

2. Project- and country-level results differ

► Projects and country programs are rated against different objectives

• No fixed relationship between results frameworks at the project- and country-level

• Outcome objectives at the project level are not the same as those at the country level

► Satisfactory project outcomes do not add up to satisfactory country program outcomes

• Chad-Cameroon pipeline met project objectives but not Chad’s country objectives of poverty reduction and capacity building

3. Results are linked across sectors

► Infrastructure

• Better roads contribute to higher school enrollment and better health outcomes

• Rural electrification improves life quality

► Health

• Mother’s education affects her children’s health

• Poor sanitation and hygiene can wipe out benefits of health projects

► Public-private partnership

• Public and private sector inputs combine to improve crop production

Measure results right

4. Composite indicators may mislead

► Indicators should be built on sound premises

• Emphasizing costs but not benefits of regulations can lead to the wrong prescriptions for regulatory reform

► Clusters and weights can have a critical effect

• Countries with better governance ratings may not gain from a larger weight on governance in IDA resource allocation

► Adding, rating, or ranking can over simplify reality

• Adding indices of different dimensions can obscure actual results

• Rigor can be compromised by rescaling from cardinal to ordinal values

5. Intermediate outcomes do not ensure desired final results

►Program achievement will be hard to assess if the desired results are not spelled out

► The results chain is complex from school enrollment to learning• Tanzania: increase in enrollment vs. decline in secondary

education learning outcomes

• India: increase in teacher attendance vs. same test scores

► The new WB education strategy rightly focuses on accountability and results

6. Averages can mask targeted outcomes

►Rural electrification• Bangladesh: the poorest 40% rural households accounted for 17% of

total electrified rural households

►Nutrition• Ethiopia: free food distribution improved children’s weight-for-height z

score in richer households but not in poorer households

►Microfinance• Philippines: microfinance benefited men and richer entrepreneurs

more than women and the poor

► Social funds and community-driven development• Benin: community contribution requirement created hardship for the

poor

Use findings creatively to improve results

7. Applying lessons for the future

► Repeating successful projects may not be enough in a changing environment• Water accessibility vs. water scarcity concerns

– China: large-scale irrigation projects

• Road investment vs. environmental concerns – Bangladesh: three-wheeled taxis

► Innovative strategies are required to meet future needs• Water: coastal zone management, pollution reduction, and

groundwater conservation

• Transport: programmatic, cross-cutting, and multi-sectoral approaches

8. Missed opportunities

► Resolve apparent conflicts among policy objectives• Energy subsidies

• Protected forest areas

► Overcome institutional constraints and limitations• MIGA’s Convention on risk insurance

• WBG’s safeguard policies

► Reduce information constraints• Cost-Benefit Analysis

9. Timing can make a big difference

►Learn faster what works and focus on

results at the right time

• Mexico: evaluation of Progresa, Oportunidades

• Philippines: early childhood development program

evaluation

►Translate evaluative lessons into

development results • Tailored messages in various formats to reach the

audience

• Collaboration with clients and stakeholders

Conclusions

► Enormous value can be added when evaluators recognize and emphasize crucial factors

► Pay-offs can be high when evaluation frameworks encourage innovation and risk taking and when findings and recommendations are followed up

Thank you!

Improving Development Results Through Excellence in Evaluation

www.ieg.worldbankgroup.org