Plan

© Plan

Post-Intervention Studies

Presentation delivered to the conference on Perspectives on Impact Evaluation, Cairo 1 April 2009

By Junaid Habib and Irko Zuurmond (irko.zuurmond@plan-

international.org)

© Plan

Context

•Founded in 1937 as a child-sponsorship organisation

•Geographical scope:• Programmes in 49 countries in Africa, Americas, Asia• Fund-raising through 17 National Organisations

•Expanding programme scope: child health, education, livelihood, water and sanitation, ….. child protection, child participation

•Changing programme approach: from a needs-based to a rights-based approach –increased emphasis on underlying processes of change

•Long-term presence in the community/district

•Budget: approx US$600 million

© Plan

Increased pressure to assess programme effectiveness Increased institutional commitment

•Development and adoption of a Programme Effectiveness Framework

•A basket of initiatives to assess programme effectiveness using multiple methodologies and multiple sources of information

•One of the new, proposed initiatives are post-intervention studies

•Considered an essential component to strengthen future programme design, in particular sustainability

© Plan

Sponsorship: phase-out process

•Standard protocol in place for phase-out from communities/areas/districts

•Phase-out process starts 18-24 months prior to phase out and includes justification for phase out.

•Reasons include reaching of agreed development targets (e.g. immunisation levels, primary school enrolment, etc)

•Once phased-out, there is no systematic process of going back to the community/area/district

Evaluation gap

© Plan

Proposal for introduction ofPost-Intervention Studies

•Aim: To assess Plan’s contribution to long-term changes and document lessons learnt.

•Objectives: Each post-intervention study will have its context specific objectives, including the analysis and documentation of the lasting results (positive / negative), lessons learnt and factors of sustainability.

© Plan

In practical terms

•To what extent have programme outcomes been maintained after phase-out?

•What are the contributing and impeding factors for sustaining (positive) programme outcomes?

© Plan

Methodology

• No fixed methodology

• However, a quasi-experimental study design is proposed. To include a credible counterfactual analysis:

• A carefully selected control (or comparison) group: E.g. neighbouring community or a physically distant community but with identical socio-economic indicators and comparable characteristics – essential

• Pre-and-post intervention comparison of both groups - desirable

© Plan

Phase-in Phase-out Post-intervention

10-15 yrs 3-5 yrs

Factual (with Plan programme)

Counter-factual

?

?

© Plan

Where are we?

•Concept paper developed and circulated

•Overall organisational buy-in into the concept of post-intervention studies

•Participation of Country Offices is on voluntary basis volunteers have been identified

© Plan

Next steps

•Assess ‘evaluability’ narrow down eligible countries

•Selection of two pilot countries among those who have volunteered

•In the first instance, the emphasis will be on developing a sound process and methodology/ methodologies, rather than having a representative sample of countries.

•Depending on the feasibility of the two pilots, scale-up the approach across Plan.