Lessons from the Field - Impact Evaluation of SDN Projects:

A Water and Sanitation Project in Paraguay

Maria Angélica Sotomayor (LCSUW)

and Luis A. Andrés (LCSSD)

Tuesday February 26, 2008Time: 12:15 to 5:30 PMRoom: I2-220

Rural Water and Sanitation in Paraguay

From the 4th Rural W&S Project to

…a New 5th Intervention

Rural Water and Sanitation Projects (I to IV) in PY

Project/approval

US$Million

Targets Results Comments

1st RW12/78

6.0 42 systems 47 systems First dedicated rural W&S project financed by the Bank (included hygiene education)

2nd RWSS 06/81

11.8 49 systems 51 systems Continued the BIRF I approach plus sanitation

3rd RWSS09/92

23.0 170 systems+ 15 indig.

177 systems

+ 15 indig.

Included a pilot to serve indigenous population (poorest of the poor)

4th RWSS08/97–12/07

40.0 330 systems

+ 35 indig.

496 systems

+ 92 indig.

PPP for larger schemes (1st OBA in water in the world), Scaled up indigenous, sewerage pilot

TOTAL 80.8 591 systems

+ 50 indig.

771 systems

+ 107 ind.

More than 800,000 people served, successful pilots and all systems operational

A Successful Project?

30 year sustainability test No system collapsed All targets met or exceeded Innovative pilots: first OBA scheme in

the water sector ever.. Satisfied clients/beneficiaries…

… ….NO DATA TO PROVE IT!!.NO DATA TO PROVE IT!!

How to move forward?

Prepare a solid M&E system –

IMPACT EVALUATION Results framework Indicators Baseline Design features: randomized

intervention -

The New Project

PDO: Improve quality of life of rural population

Indicator: Reduce the morbidity rate (diarrhea, parasitosis) related to water and basic sanitation in rural population

Goals / purpose

Sustainable access to potable water and sanitation services in the population of up to 10.000 inhabitants # of people served with potable water. # of people served with sanitation.

Strengthen the management capacity of the community based organizations responsible for rural W&S service delivery Increase xx% of Water Boards that honor financial

responsibilities Increase xx% of Water Boards joining Associations of Water

Boards # of Water Boards that adopt measures to improve

accountability

Components Investments

Rural water PSP Indigenous systems Individual sanitation solutions Simplified sewerage

Hand-washing Institutional strengthening

Central Agency - SENASA Water boards Associations of water boards

Outcomes

Do improvements in water access (and sanitation) in rural (and urban) areas in Paraguay:

reduce the incidence of water related illness (diarrhea, etc)? For any specific group?

improve the nutritional level of the children? decrease infant mortality? change intra-household behavior such as time

usage, access to education, and/or productivity?

Identification strategy

Random assignment?Very hard to do this in INF’s interventions (mainly due to

engineering constrains);Unit of observation are the communities rather than the

HHs;Self selection: local communities have to be eligible,

prepare a project, apply for funds, and commit 15% of the project value (in kind and cash);

But: ~200 communities (already selected) will be intervened in 4 years and the government has no capacity to start everywhere at the same time…

Identification strategy (cont.)

So? What we agreed to do… We will randomize the entrance of these communities (50 each

year): those that will enter last will be the control group for those that will enter first [“internal control group”];

Other (external) control groups? Pipeline of projects for the (potential) 6th loan. In principle, these

communities have similar characteristics => some of them may be an external CG;

Projects intervened in the 4th loan (assumption: trend for those intervened in the 4th loan is similar to the counterfactual);

Matching with those communities that did not apply (not self selected) to study the selection bias…

Where we are…

Building local capacities: Team members and staff from the government attended the regional

workshop on IE a year ago => they know (and accept) the commitments for this IE;

Working with Fundición Desarrollo (a local think tank); This was discussed with different stake holders.

Proper project preparation: A project already has a (draft) design; This design will be included in the PAD; We are working together with a professor from Northwestern University; Fundraising: for doing the data collection (it is very likely to get TF to

complement the project’s budget).

…Thanks!