Upload
trinhdang
View
229
Download
2
Embed Size (px)
Citation preview
SNV WASH Project in Ethiopia Final impact evaluation
Commissioned by
SNV Netherlands
October 2015
Project number 2529
Aidenvironment
Barentszplein 7
1013 NJ Amsterdam
The Netherlands
+ 31 (0)20 686 81 11
www.aidenvironment.org
Project number 2529 ii
Contents
Abbreviations and definitions iv
Executive Summary v
1. Introduction 1
2. Methodology 3 2.1 Baseline study 3 2.2 Overview methodology 4 2.3 Survey methods, sampling and analyes 5 2.3.1 User and facility level: households and schools 5 2.3.2 Water schemes 7 2.3.3 Capacity building efforts 8 2.4 Analysis of result pathways 10 2.5 Limitations of the methodology 10
3. Results 12 3.1 Households 12 3.1.1 Access to safe water 12 3.1.2 Access to improved sanitation 13 3.1.3 Incidence of diarrhea and analysis 15 3.1.4 Access to information and training on WASH 16 3.1.5 Additional analysis on explaining factors 17 3.2 Schools 18 3.2.1 Access to safe water in schools 18 3.2.2 Access to improved sanitation in schools 19 3.2.3 Functionality of WASH club and analysis 19 3.2.4 WASH related school management and analysis 20 3.2.5 School attendance and analysis 21 3.3 Water schemes 21 3.3.1 Functionality of water schemes 21 3.3.2 Performance of the WASH committees 23 3.3.3 Preventive and Operational capacity training by SNV 23 3.3.4 Comparison with baseline situation 24 3.4 Capacity building 25 3.4.1 Capacity building interviews and focus group discussions per woreda 25 3.4.2 Interviews at regional level 31 3.4.3 Interviews at the Hawassa TVETC 33 3.4.4 Interviews at UNICEF – Hawassah branch 34 3.4.5 Conclusions from the regional level interviews 35
4. Analysis and conclusions 36 4.1 Summary of main findings 36 4.1.1 Comparison of woredas 36 4.1.2 School level findings 37 4.2 Analysis of impact pathways 39 4.2.1 Impact pathway 1. Household access to improved sanitation 39 4.2.2 Impact pathway 2. Schools access to water and improved sanitation 40
Project number 2529 iii
4.2.3 Impact pathway 3. Improved management of water schemes 42 4.2.4 Impact pathway 4. Capacity building of institutions responsible for WASH 44 4.3 Evaluation questions 45
Annex 1: Theory of change (reconstructed by the evaluation team) 51
Annex 2: Evaluation Questions (extract from ToR) 53
Project number 2529 iv
Abbreviations and definitions
AKVO Akvo (organization in the Netherlands)
CLTSH Community Led Total Sanitation and Hygiene
FGD Focus Group discussion
GDL guided distance learning
GLOWS Guided Learning on water and Sanitation
MDG Millennium Development Goals
M&E Monitoring & Evaluation
MHM Menstrual Hygiene Management
MIS Management Information Systems
NGO Non Governmental Organisation
O&M Operations and Maintenance
POM Preventive Operations and Maintenance
SNNPRS Southern Nations, Nationalities and People Regional State
SNV Netherlands Development Organisation
SPSS Statistical Packages for Social Scientists
TTLM Teaching Training learning Materials preparation
TVETC Technical vocational education training center
UAP Universal Action plan
UNICEF United Nations Children’s Fund
VIP ventilated improved pit
WASH water sanitation and hygiene
Definitions
Improved sanitation facility: one that hygienically separates human excreta from human contact.
(WHO definition). The improved sanitation facilities include flush/pour flush (to piped sewer
system, septic tank, pit latrine), ventilated improved pit (VIP) latrine, pit latrine with slab, and
composting toilet.
Unimproved sanitation facilities include: public or shared latrine, open pit latrine or bucket latrine
Secure sanitation facility: sanitation facility with secured privacy (in schools)
Traditional sanitation facility: open field defecation or traditional latrine facilities that are pit
latrines without a slab.
Project number 2529 v
Executive Summary
Background
In the period of 2007 to 2012, SNV implemented a WASH project in 6 woredas (districts) of the
SNNPRS (Southern Nations, Nationalities and People Regional State) in Ethiopia. The project aimed
at increasing access to water supply, sanitation services, and improved sanitary practice in the
communities and at schools. The project introduced the CLTSH (Community led Total Sanitation and
Hygiene) approach in the 6 woredas. The project chose a model of close collaboration with, and
capacity building of, the local government offices, focused at strategic planning, improved service
delivery, community involvement, monitoring and evaluation. Aidenvironment carried out an impact
evaluation of this project in 2015, with the purpose to assess the continuing impacts of the SNV
WASH project, especially the improved access to improved water and sanitation for the target groups
as well as the effects on the capacities of the government functionaries in the WASH sector.
Methodology
The impact evaluation focused at household level, school level, water schemes and capacities of
institutions within targeted woredas and regional level. Mixed methods were used including
structured and semi-structured surveys, in-depth interviews and focus group discussions. The
household and schools surveys focused on access to safe water, access to improved sanitation and
information and training on WASH. The surveys were translated in local language and use was made
of digital data collection. In total 1450 household interviews were conducted, selected from 58
kebeles (rural municipalities) in the 6 woredas. In total 50 schools were selected and 100 school
interviews (50 pupils and 50 teachers) were conducted, equally distributed over the 6 woredas.
Survey data were analysed using SPSS, descriptive statistics and inferential analysis. In total 50 water
schemes were visited, with surveys to determine functionality. To evaluate capacity building effects of
relevant staff (within the WASH, education and health sectors) at woreda level, the 5C (5 capabilities)
approach of evaluating capacity building was used as a basis for interviews, using a balance score
card system specifically developed for this study. The study did not include control groups for
households, schools or water schemes. The study did include two non-targeted woredas, as control
groups, to make a comparison in capacities between staff within targeted and non-targeted woredas.
To establish the main changes in time, the study compared the current values with the baseline
values (from 2007). This posed some challenges because the baseline data and the approach used
were not well defined. Also, a reliable midterm measurement survey was missing. To draw
conclusions on the contribution by the SNV project, the quantitative and qualitative results were used
to assess whether defined impact pathways (based on the project’s theory of change) could explain
the changes in time, including the relative contribution by the SNV project and other (external)
factors.
Findings
In the analysis of access to water, in line with the SNV baseline survey we used three criteria: a) the
type of main water source and its safety (by interview and observation), b) the travel distance or
travel time to the safe main water source being less than 1.5 km or travel time not being more than 30
minutes, and c) the access of clean water storage. When taking into account these three criteria,
access to water for households in the project area has declined from 53% in 2008 to 46.4% in 2015.
Currently travel time was more than 30 minutes for 40.7% of the respondents, which is still better
than the average of 62% for rural areas in Ethiopia, according to the Ethiopia Demographic and
Health survey of 2011. In the analysis of access to improved sanitation, we adopted four criteria, in
line with the baseline survey: a) the presence of an improved latrine facility; b) cleanliness of the
latrine, c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.
Project number 2529 vi
When taking into account these criteria, we found that an increase of access to improved sanitation
within households from 27% to 41.4%. Looking at the various factors that influence access to
sanitation (according to the impact pathway), we conclude there is a significant relation and plausible
contribution by the SNV project to the improvement in access to improved sanitation. There is also a
significant correlation between access to safe water and access to improved sanitation for households,
suggesting that access to safe water influences improved sanitation. The decline in the incidence of
diarrhoea is highly significant (from 52% to 17%), and can be partly explained by the improved access
to improved sanitation.
Concerning the schools, access to water has significantly improved (from 7% to 22%, or 52% when
water sources beyond the school premises are also considered). In the schools 94% have a latrine
facility, but access to improved sanitation declines to 28% if we also take into account aspects of
cleanliness and safety. This can be explained by limited access to water within the school compounds
or limited management by teachers of WASH within schools (only 26% scores very good). Most
student-based WASH clubs (72%) are functional. We cannot conclude there is a significant relation
by the SNV project to access to improved sanitation in schools but a contribution by the SNV project
is plausible based on evidence from interviews and focus group discussions.
To determine functionality of water schemes, three criteria were used: water discharge, water quality
and waiting time. It was found that for the majority (72%) of water schemes in the project area
functionality has improved. The SNV project has contributed to the improved functionality of water
schemes in the project area, by training on operations & maintenance and supporting local WASH
committees. For instance, the time it takes before a reported failure is repaired has considerably
reduced. However, for 70.6% of the water scheme users, waiting time at the water scheme was more
than 30 minutes (with a maximum of 5 hours), reducing their functionality. We conclude there is a
significant contribution by the SNV project to the improved functionality of water schemes, based on
evidence from interviews and focus group discussions This also means that access to water would
probably have been lower (than 46.4%, see above) if SNV had not contributed to improved
functionality of water schemes. The remaining low functionality of water schemes is mainly due to
low volumes of water from existing water schemes in relation to high and increasing water demand,
with underlying causes of relatively few new water schemes being constructed or low water volume
per water scheme.
Opinions of woreda staff are positive with respect to the services being provided by the SNV project.
At woreda level, strategic planning, service delivery, community involvement and monitoring have
much improved. The targeted woreda’s have better capacities for WASH strategic planning and
WASH service delivery than control woreda’s. The model of training-on-the–job (GLOWS) has been
very effective in terms of improving service delivery. The response rate to repairs has increased. Use
of the Management Information System (MIS) has failed due to lack of support and continuity at
regional level. We conclude there are positive changes in capacities of woreda level staff to manage
WASH related aspects and plausible evidence for a significant contribution by the SNV project.
With respect to sustainability of the results, most capacity building results will sustain, such as the
integrated strategic planning approach (education, health and WASH) and the improved service
delivery. Also, most local water schemes are well maintained with WASH committees (88%),
including fees being charged. Among the targeted woreda’s, there is some evidence that the targeted
woreda’s are better able to access funds from donors for investment objectives, presumably because
of their improved planning and funding requests. Nevertheless, funds for investments in new water
schemes and large-scale maintenance of existing water schemes remain insufficient. The GLOWS
approach is reported to be costly and still not formally recognised, and therefore uncertain to be
continued. Altogether, there are insufficient recurrent financial inputs to the WASH sector in the
Project number 2529 vii
region, e.g. through fees or taxes. From an environmental angle, it is worrisome that water boreholes
have to go deeper and yet cannot supply sufficient water for the whole population in the region. This
may lead to depletion of underground water resources. More attention could be given to rainwater
capture and/or underground storage. There may also be need for collaboration and integration with
natural resources and watershed management for improving vegetation coverage around water
sources so as to improve water recharge.
Thus, there are remaining challenges, which are mainly related to external factors, including the need
for institutional support to use the MIS and continue the use of GLOWS, additional hardware
especially on new water schemes, operations and maintenance of large water schemes to increase
water discharge, and improved motivation of staff and improved skills on specific tasks.
Conclusions and recommendations
The approach taken by SNV in the WASH sector is innovative by focusing on capacity building of
local public institutions, on awareness raising and change of practices, involving training institutes
and networking. This focus by SNV on the ‘software’ was part of a programme partnership with
UNICEF that focused on the ‘hardware’. Overall, this evaluation shows that the SNV approach has
generated significant results, but the ‘hardware’ component now seems to constitute the main
constraint. The contribution by SNV is positive in relation to change of behaviour on sanitation and
hygiene at household level, improved maintenance of water schemes, build up of capacities at woreda
level, among the communities, WASH committees and WASH clubs at schools. Also, there is
increasing private sector involvement in the WASH sector. Although the funding from national
sources to the WASH sector has improved for the 6 targeted woreda’s, funds for investments in new
water schemes and large-scale maintenance of existing water schemes remain insufficient.
Considering the remaining challenges, there is a potential role for SNV to play as follows:
Scaling the successes achieved in the sanitation sector, by capitalizing on the significant correlation
between access to improved sanitation for households and the support and training provided;
Strengthen the WASH monitoring and evaluation system within SNV and with local partners,
including proper establishment of a baseline measurement, and including a technology to monitor
water discharge of water systems and waiting time;
Strengthen a joint learning and evaluation system and assure continuity in follow-up or refresher
training, to assure further improvement and/or to avoid fall-back;
Take the lead in proactively engaging in advocacy and lobby to narrow the existing gap on demand
for safe water, by lobbying for budget increases to develop additional water facilities and
developing more sustainable solutions ;
Further strengthen private sector engagement in the WASH sector;
Pay more attention to rainwater capture, underground water recharge and storage and watershed
management, to avoid water depletion and to increase access to water from sustainable sources.
Project number 2529 1
1. Introduction
The SNV WASH project
In the period of 2007 to 2012, SNV implemented a WASH project in 6 woredas (districts) of the
SNNPRS (Southern Nations, Nationalities and People Regional State) in Ethiopia. The project aimed
at increasing access to water supply, sanitation services, and improved hygiene practice in the
communities and at schools. The project introduced the CLTSH (Community led Total Sanitation and
Hygiene) approach in the selected woreda’s. To assure sustainability of its efforts, the project chose a
model of close collaboration with, and capacity building of, the local government offices. Capacity
development focused at community needs assessment, planning and introducing a Management
Information Systems (MIS). It supported the woreda’s to develop their WASH strategic plan aligned
with the UAP (Universal Access Plan) at national level. Another element of the project was to
improve Operations and Maintenance (O&M) of selected water schemes. To do so, within each
woreda the technicians of the WASH team were mobilised and trained on O&M aspects. In each
woreda, water schemes were supported on O&M through the local WASH committees (committee
responsible for O&M of the water scheme) to properly manage user fees by users, keep the water
scheme clean, install a fence and establish effective drainage.
At schools the approach of multi-stakeholder inquiry was applied. This a form of action research to
raise awareness and then make a plan to improve WASH performance. To do so, SNV selected 10-15
schools in each woreda. SNV also introduced the MHM (Menstrual Hygiene Management) for girls at
schools (introduction of low-cost pads). Within the selected schools, SNV also introduced the
establishment of WASH clubs and stimulated these clubs to come together and develop an action
plan.
SNV also worked with the local vocational college (TVETC) to improve their curriculum and develop
practical skills of their graduates in operating and maintaining water and sanitation facilities. This
was later on expanded with an approach of guided distance learning (GDL), which was formulated as
GLOWS (Guided Learning on water and Sanitation).
Based on the above project headlines, the evaluation team reconstructed the theory of change (see
Annex 1) and a number of impact pathways (see chapter 4). The theory of change distinguishes
between a range of ‘software’ or capacity building activities, and assumes that either the ‘hardware’ is
already available or that resources for ‘hardware’ are made available by other partners. The
partnership with UNICEF would assure that hardware (water schemes) could be provided where this
was deemed necessary and useful. The functionality of water schemes is an important element of the
theory of change. The assumption was also that upscaling would take place as a result of increased
awareness, increased finance allocation for WASH hardware provision by the district, and improved
capacities within woredas to manage the existing water schemes.
This impact evaluation study
Aidenvironment has been commissioned the assignment to carry out an impact evaluation of this
project. The purpose of the evaluation documented in this report is to assess the continuing impacts
of the above SNV WASH project. It will include assessing the effects on the capacity and ways of
working of the government functionaries in the WASH sector as well as any lasting effects on
improved access to improved water and sanitation for the target groups in these woredas. The
findings will be used both for accountability to the donors and other stakeholders, and for
organisational learning to improve SNV’s practice in WASH and of working with local governments
in particular.
Project number 2529 2
The scope of the evaluation can be characterized as follows:
focus on the WASH project in 6 woreda’s of the SNNRPS region in Ethiopia.
focus on the OECD/DAC criteria impact, effectiveness and sustainability.
cover the period from the start of the project until now. The effort will be to assess the changes that
the project brought and whether those changes have been sustained until now or not.
focus on the 6 project woreda’s in the SNNP region and comparison areas/ woredas selected for the
comparison of assessment in the SNNP region.
A set of research questions was formulated that form the basis for this evaluation (see Annex 2).
This report
An inception report has been delivered in February 2015. To do so, Aidenvironment studied the
available documentation and had a 3-day workshop in Ethiopia (28-30 January), together with the
local consultants and relevant SNV staff, to prepare the impact study. The fieldwork was carried out
in March-April. Data were received in May and this draft report was prepared in June 2015.
Following feedback by the Steering Committee a final version of this report was prepared in August
2015.
This report is organized by the following chapters:
Chapter 2: methodology followed for the field data collection and analysis of the data
Chapter 3: results of the data collection and analyses, for household survey, school survey, water
scheme survey and capacity building analysis separately
Chapter 4: conclusions, validation of impact pathways and responses to the main evaluation
questions.
There are a series of appendices in a separate document that provide all the basic data.
Project number 2529 3
2. Methodology
2.1 Baseline study
According to the ToR the SNV WASH project conducted in 2007 a baseline study as well as in 2011 a
midterm evaluation study. During the inception phase we reviewed the quality of these studies.
Baseline study
In 2007 an extensive baseline study was executed. The main characteristics of the baseline study
executed in 2007 are the following:
Baseline survey undertaken in the selected 6 woredas, the Kebele and major urban centres are the
basic unit for the survey in each Woreda
The basic units for data collection and observations are the households, the institutions, Kebele
WaSH/Water scheme committees, Woreda WaSH team and Town WaSH Teams.
The data were collected by local capacity builders, Woreda WASH teams, Kebele baseline survey
teams and interviewers at the Woreda, Kebele, institutions and water source levels.
Both quantitative and qualitative data were collected by interviewing of household members and
informants from institutions based on a structured questionnaire and also by observations.
50 households were sampled randomly as respondents per Kebele.
For all schools, Health centers, Markets and drinking Water sources surveys were done.
A team of 3 interviewers each interviewing 7-8 households in a day.
A Kebele baseline survey team consisting of 1 man and 1 woman was trained at each Kebele
In each Woreda and Kebele, a focus group discussion was held with the Woreda WaSH team and
Kebele WASH/water scheme committees to collect relevant WaSH information.
During verification of the baseline data collected in 2007 several woredas found the data did not
correspond to their perception of the reality of the field. Especially, access to sanitation data were
found to be too positive. This was assumed to be mainly the result of the fact that many households
construct latrines but do not use them, or have constructed very simple latrines because this was a
requirement to be able to receive food aid. These latrines were not used and therefore cannot be
qualified as ‘access to sanitation’. Apparently enumerators too easily provided a positive qualification
of access to sanitation if only a simple latrine had been produced.
Therefore, in 2008 a verification survey was carried out, with the following approach:
Adopt both observation and interviewing of households and institutions in the Kebeles. In total 58
kebeles were selected from the 6 woredas (see in below table)
Out of 50 households interviewed in the previous survey, 25 were selected randomly and re-
interviewed. Each data collector interviewed 5 units (households and or institutions) per day.
Female teachers collected the data and fewer questions were asked. Female teachers were preferred
because they were presumed to be more credible and therefore unlikely to lie about the WaSH
status and they could easily win the confidence of the household members and therefore be allowed
to observe cleanliness in the households and water storage containers. The teachers signed an
agreement with SNV to provide accurate data.
A Woreda employee not associated with the WaSH activities supervised the data validation process.
The verification process generated new and more reliable data, leading to corrections on the previous
set of baseline data. The impact evaluation team does not fully understand how the validated data
were combined with the initial survey data to result in a final value, which is the reason why we took
as the baseline the data from the verification survey (see relevant tables in chapter 3).
Project number 2529 4
As verified during the inception phase, according to SNV the verified access to water baseline data
have taken into account:
distance to the water points (at most 1.5 km)
cleanliness of the water storage containers
As verified during the inception phase, according to SNV the verified access to sanitation data have
taken into account:
use of the latrines (if a latrine is not used, it falls in category of open defecation)
cleanliness of the latrines (if latrines are unclean it would assume open defecation).
absence of faeces in the compound.
Midterm evaluation 2011
During the inception phase it was found that SNV did not carry out a midterm evaluation survey.
However, surveys have also been carried out by the Water bureau of the SNNPRS and consultants
have made use of these data in case study and other reports for SNV. For instance, a report for SNV
entitled ‘report on six SNV supported woreda’s in SNNPRS’ makes use of the 2008/09 and 2010/11
inventory of water access by the SNNPRS (mentioning that these are unpublished data). However,
there are different reasons why these data cannot be used:
we do not have access to the survey sheets and do not know which questions were asked and how
the data had been interpreted and analysed;
the data for 2011 vary in almost every available document; moreover, the 2011 data in the formal
report of this survey (which only appeared in 2014) are all different from the data mentioned in the
case study reports and show outliers that cannot be correct (this is even admitted in the report).
There is also available a Demographic and Health Survey (DHS) from 2011. However, this survey
does not provide data at woreda level. We did make use of the DHS survey questionnaire to compile
our own questionnaire.
2.2 Overview methodology
The evaluation focused at four different levels of evaluation: household user level; facilities (school)
level (schools), water schemes level and local government capacities level. In terms of facilities, only
schools were assessed, because at the level of markets and health facilities no significant effects are
expected. A variety of methods was used to collect data to assess the impact of SNV’s WASH-project
in Ethiopia: structured and semi-structured surveys, in-depth interviews and focus group discussions
(FGD). Table 1 provides an overview of the methods used at the three evaluation levels.
It should be emphasized that the study did not include control groups for households, schools or
water schemes. On the contrary, the study did include control groups at woreda level. This was done
according to the ToR and is understandable for two main reasons. First, as direct effects of the SNV
WASH project we expect results at the level of local woreda agencies, reason why controls are
included. According to the theory of change capacity building will result in effects at household,
school and water scheme levels. However, most likely these effects will expand beyond the focus
woredas, also in view of the fact that other NGOs may have interventions that directly target
households, schools and water schemes. Thus, controls at woreda level are in line with the fact that
we want to test the specific SNV WASH theory of change. The second reason is pragmatic, being the
high costs that would be related to control groups at household, school and water scheme levels.
Project number 2529 5
Table 1. Overview of methodology
User level (households)
and facilities (schools)
Water schemes Institutional capacity
building efforts
Type of
method
Before-after comparison:
comparing baseline study
data with impact data
Mixed methods:
quantitative survey and
qualitative FGD
Surveys based on baseline
survey, for comparison
Contribution analysis
based on impact
pathways
In-depth interviews and
FGDs
Interview scheme based
on questionnaire baseline
study
Contribution analysis
based on impact
pathways
Control group approach:
selection of 2 Woredas for
comparison
In-depth interviews and
FGDs
Interview scheme based
on questionnaire baseline
study
Contribution analysis
based on impact
pathways
For each of the 4 levels mentioned in table 1, we will describe briefly the methodology implemented.
2.3 Survey methods, sampling and analyes
2.3.1 User and facility level: households and schools
Survey construction
For household and school level, separate surveys were constructed: a structured survey for household
level interviews and a semi-structured survey for school level interviews. Both surveys have been
developed in collaboration with SNV Ethiopia and local consultants.
The household surveys focused on 3 topics: access to safe water, access to sanitation and awareness,
received support and information. The school level surveys has separate sections for students and
teachers, covering as topics: access to safe water, access to improved sanitation, membership of
WASH-club, management of WASH-facility (teachers only) and school attendance (teachers only).
Note that these surveys were designed as based upon the baseline survey formats in order to allow for
before (baseline)-after comparisons. Both surveys can be found in appendix 1. Both surveys have
been translated into local language and have been uploaded in the AKVO tool to allow for digital data
collection.
Data collection
The surveys for households and schools were conducted in the period between March 15th and April
7th, 2015. The household and school surveys were conducted by a team of 10 local enumerators, in
collaboration with and supervised by the 2 local consultants. All local enumerators and consultants
were trained extensively beforehand on the understanding of the questionnaire, the use of the AKVO
tool, and the ethical criteria to be respected. The enumerators and consultants also received training
in using the AKVO tool, a tool for data collection and analysis. A survey coordinator was present
during data-collection to support the team, to ensure sampling procedures were followed and to
perform quality checks on data collected.
As four different local languages are spoken in the 6 surveyed woredas, local translators were
deployed to translate the local language to Amharic. These local translators were selected from each
surveyed village, taking into account a) their ability to speak the local language, b) their ability to
speak the Amharic language and c) neutrality and prevention of bias by making sure the translators
did not work for the kebele administration and/ or WASH-sector.
Project number 2529 6
Households: sampling
In total 1450 Household interviews were conducted using the AKVO -tool and included in the
analysis.1 Due to technical reasons the data of 24 interviews were not saved into the tool and could
therefore not be included in the analysis. These 24 interviews are from different villages, so did not
create a bias. In total, data of 1426 interviews are available for analysis.
The 1426 households were randomly selected from 58 kebeles (rural municipalities), situated in 6
woreda’s (districts). To ensure comparability, the woreda’s and kebeles that were selected for the
survey are identical to those selected in the baseline study. Selection of villages within the kebeles was
made at random: for each of the 58 kebeles a list of villages was made and, using the random
numbers-method, 1 village per kebele was selected. Likewise, the selection of households within the
selected villages was made at random: per village a list of households was prepared and the random
numbers method was used to select 30 households per village. Note that at average there are about 10
villages per kebele. The number of households per kebele generally varies between 40 and 70.
Schools: sampling
In total 50 schools were selected and 100 school interviews (50 pupil interviews and 50 teacher
interviews) were conducted, using the AKVO tool. For 2 woreda’s a list of SNV-targeted schools was
available. In these woreda’s we selected all targeted schools for the survey (8 schools in Misrak
Badewacho and 5 schools in Kedida Gamella). The selection of the remaining 37 schools in the other
4 woreda’s took place by randomly selecting schools from the villages where the household surveys
were carried out. School characteristics (such as first cycle primary, second cycle primary, secondary,
preparatory) were included in the sampling procedure, to ensure different types of schools were
included in the sample.
Data Cleaning and Analysis
The data submitted to the server for storage were accessed on AKVO dashboard and inspected by the
survey coordinator for quality check on daily basis throughout the data collection period.
At the end of the data collection for the quantitative household and school surveys, the 1426 data
forms for the household survey and 100 data forms for school survey submitted to the dashboard
were exported into EXCEL for making ready to import into SPSS (Statistical Packages for Social
Scientists). The three EXCEL data sets (one for household survey, and two for teachers and students
surveys in the school) were converted into SPSS files for cleaning, processing and analysis. Then the
household, student and teacher surveys were used for generating quantitative insights which were
interpreted and analysed on apparent correlations between parameters.
In analyzing the quantitative data in SPSS, descriptive statistics and inferential analysis were carried
out. Descriptive statistics, like percentages and ratios, were calculated for describing the key
statistical results. Chi-square and correlation tests were done for assessing association between
selected dependent variables (which are incidence of diarrheal diseases) and the
independent/explanatory variables like access to safe drinking water and improved sanitation.
Note that in the analysis of criteria for access to water, in line with the SNV baseline survey we
introduced three criteria: a) an observation is made of the safety of the main water source, b) travel
distance to the safe main water source is less than 1.5 km, and c) people have the possibility of clean
water storage. We also looked at waiting time because this could be a more reliable indicator than
distance to the nearest water source.
1 To calculate the number of household interviews, the following formula for determining sample size was used: n = ((z2 x p (1-p)) /d2,
where z2 stands for confidence level of 90%, p stands for prevalence of WASH indicators, d stands for acceptable difference/ margin of
error of 5%.
Project number 2529 7
In the analysis of access to improved sanitation, we also adopted four criteria, in line with the
baseline survey, as follows: a) observation that there is an improved latrine facility; b) cleanliness of
the latrine, c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.
Note that an improved latrine facility for MDG monitoring is defined as one that hygienically
separates human excreta from human contact. The improved sanitation facilities include flush/pour
flush (to piped sewer system, septic tank, pit latrine), ventilated improved pit (VIP) latrine, pit latrine
with slab, and composting toilet. This compares with open field defecation or traditional latrine
facilities that are pit latrines without a slab.
2.3.2 Water schemes
For assessing impact on the water schemes, focus group discussions (FGD) were held. The checklist
of questions for the FGDs is based on the baseline survey and has been constructed in collaboration
with SNV Ethiopia and the local consultants. The FGD checklist can be found in appendix 1.4.
In total 50 Water schemes have been visited, consisting of boreholes, shallow wells, hand dug wells
and springs. Of the 50 water schemes that were surveyed, 25 were constructed before 2007 (when the
project started) and 25 were constructed after 2007. Also, SNV support that was provided to these
water schemes was a selection criterion, so that half of the old and new water schemes received
maintenance support from SNV and the other half did not. Thus we have 4 categories:
12 constructed before 2007 and no SNV operation and maintenance support received
13 constructed before 2007 and SNV operation and maintenance support received
12 constructed in or after 2007 and no SNV operation and maintenance support received
12 constructed in or after 2007 and SNV operation and maintenance support received
The focus group discussions were carried out with users and WASH committees (WASHCO’s). Based
on the questionnaire results, the water schemes and WASH committees were assessed and classified
in terms of their current functionality (of water schemes) and operations and maintenance
performance of the WASH committees.
Functionality of water schemes
To determine functionality, we developed 3 criteria to assess water schemes: water discharge, water
quality and waiting time. We defined three levels of water scheme functionality, as follows.
Functionality level of
water schemes
Criteria
Good Discharge volume = good or low / less during dry season, and
Water quality= good or moderate / lower during rainy season, and
Waiting time = <30 min
Partly Discharge volume = low / less during dry season, and
Water quality = moderate or lower during rainy season, or
Waiting time = >30 min
Poor Discharge volume = not functional, or
Water quality = low
Please note that in the above classification of functionality we may deal with cumulative (and) criteria
or with facultative (or) criteria. The criterion of waiting time more than 30 minutes was introduced
Project number 2529 8
because of the very high proportion of users that have to wait long to be able to collect water. In the
literature, a waiting time of more than 30 minutes is commonly considered as being too long.2
Performance of the WASH committees
To determine performance, WASH committees were assessed on 7 criteria: presence of WASH
committee, satisfaction of users about WASH committee, charge of fees, fees used for Operation &
Monitoring (O&M) and satisfaction of fee use, incidence of conflict and resolution of conflicts. Using
these criteria, we defined 2 levels of performance, as follows.
Performance of
WASH committee
Criteria
Good Presence of WASH committee
Satisfied with WASH committee, or partly satisfied
Fees charged
Fees used for O&M
Satisfied with use of fee
No incidence of conflict or resolution of conflicts
Poor Absence of WASH committee
Users not satisfied with WASH committee
Conflicts existent and/or not solved
2.3.3 Capacity building efforts
Survey construction
The checklist of questions for evaluating capacity building has been developed in collaboration with
SNV Ethiopia and the local consultants. As a basis we took the capacity building interventions that
took place during the SNV project period, and the 5C approach of evaluating capacity building (as
also being adopted by SNV). The survey checklists can be found in appendix 1.4.
Separate surveys were designed for the 4 different levels where interviews were held. To assess the
impact of capacity building efforts, focus group discussions (FGD) and in-depth interviews were
performed at 4 levels.
Level Method Number
Woredas (8 woredas, of which 6
targeted and 2 control woredas)
- Individual in-depth interview
- FGD WASH team
- 3–4 per district
- 1 per district
Regional authorities Individual in-depth interviews 3–4
TVETC Individual in-depth interviews 1 -2
UNICEF Individual in-depth interviews 1 -2
Sampling method
In total 8 woredas were visited, including the 6 SNV-targeted woredas and 2 control woredas. The 2
control woredas (Baloso Bombe and Angecha) were selected on the basis of the following criteria:
presence of water schemes a bit comparable to the target woredas;
near / not too far from the target woredas (for reasons of logistics and comparison);
no past interventions by SNV, no major interventions by other NGOs.
2 Queuing time at a water source is no more than 30 minutes. See :
http://www.spherehandbook.org/en/water-supply-standard-1-access-and-water-quantity/
Project number 2529 9
Appendix 2.2 provides names of consulted respondents, through individual interviews or FGDs.
Scoring capacity building
In order to classify the wealth of information generated on capacity building by using the surveys, we
developed a balance score system to assess capacity building efforts on 7 aspects, in line with the 5C
approach of evaluating capacity building efforts as being used and promoted by SNV:
Strategic planning of WASH sector
Finance for WASH sector
WASH service delivery
Gender aspects in WASH
Communication and private sector participation in WASH setor
WASH database and monitoring and evaluation system
Partnerships for support to the WASH sector.
The balance score card makes an assessment of the information for each of the above 7 aspects, as
indicated in the below assessment matrix.
Current situation score
(1= lowest score)
‘What has changed’ score
(a=lowest)
Strategic
planning
There is no strategic plan (1)
Strategic plan follows the Growth and
Transformation Plan (GTP), but little or no
special attention for WASH (2)
There is a strategic plan with special
attention for WASH (3)
No improved focus on WASH in strategic
plan (a)
Improved design of strategic plan but no
improved result (yet) or lessons learned
(b)
Improved design and implementation of
strategic plan, possibly leading to improved
WASH services (c)
Financing Low priority (awareness) and no adequate
budget for WASH and high dependency on
donors (1)
There is high priority and budget for
WASH but not yet sufficient, dependency
on donors (2)
There is high priority and adequate budget
for WASH ( 3)
No improvement of budget and / or
priority (a)
Improved priority (awareness) but no
increased budget (b)
Increased budget and improved priority
for WASH (c)
WASH
delivery
No or limited WASH services are provided,
or are not of good quality (1)
WASH services are provided but do not
fully meet demand client (2)
WASH services are provided in terms of
coverage and utilization, are adequate in
terms of coverage and quality needs of
client (3)
No change or no improvement in service
delivery capacities (a)
Improved capacity of staff but does not
translate into improved services (b)
Improved capacity of staff to deliver
WASH services and improved services
delivered (in terms coverage and quality)
(c)
Gender and
youth aspects
Limited or no involvement in WASH and
no specific service delivery for women,
youth and vulnerable groups(1)
Good involvement of women ,youth and
vulnerable groups, OR specific service
delivery is provided ( 2)
No change in gender, youth and vulnerable
social groups inclusion, or situation has
become worse (a)
Improved gender, youth and vulnerable
group inclusion OR improved gender ,
youth , vulnerable group focus / priority (b)
Project number 2529 10
2.4 Analysis of result pathways
The results of the three levels of evaluation were analysed and interpreted to assess whether the
identified impact pathways could be validated or not. To do so we combined the quantitative data (of
the household and school surveys and the water schemes) and compared these with the qualitative
data and scores on capacity building based on the FGDs and individual interviews, to draw
conclusions whether impact pathways have been realized or not.
Combining quantitative and qualitative scores is a challenge when using a mixed approach. In our
study, we have given scores to the capacity building results according to the 5C approach (see
previous section) in order to be able to match the scores with quantitative results per woreda.
However, we found the variation between woredas to be minimal. Also, we looked at specific
qualitative information to support or disqualify quantitative data.
2.5 Limitations of the methodology
The above methodology has been worked out as part of the inception report and submitted to the
Steering Committee that was assigned for this impact evaluation (with representatives of the donor,
SNV and an impact evaluation research institute). It has been approved with some minor
adjustments that were incorporated in the final version.
In spite of the rigorous method, the impact evaluation met the following limitations:
Good involvement of women, youth
vulnerable groups AND specific WASH
service delivery is provided for women and
youth (3)
Improved inclusion of gender aspects in
WASH activities and service delivery ( c)
Community
and private
sector
participation
No or little participation of community in
WASH (1)
Communities participate in WASH but
there is not sufficient capacity (fees, skills,
human) (2)
Community (and private sector if
applicable) participates in WASH,
expresses their own demands for
support(3)
No improvement in community
participation, in terms of water payment
and maintenance (a)
Improved community participation, but sill
insufficient to meet maintenance needs (b)
Improved participation community,
enough to meet maintenance needs (c)
Data base,
monitoring
and evaluation
There is no management information
system (MIS) or data base on WASH and
no resources for WASH monitoring (1)
There is a MIS or data base on WASH but it
is not used or not up-to-date (2)
There is a MIS and up to date data base on
WASH (3)
No improvement or change in WASH data
base and monitoring (a)
Introduction or involvement of staff in
M&E activities or MIS, but no follow-up (b)
Improved involvement of staff in WASH
M&E activities or MIS, adequate resources
(c)
Partnerships No partners active in the field that
contribute to WASH strategy (1)
Some partners active in the field that
contribute to WASH strategy (2)
Many partners active in the field that
contribute to WASH strategy (3)
Partners have left the area (a)
The level of support has not changed (b)
Level of support for WASH strategy has
increased (c)
Project number 2529 11
One is the fact that the impact evaluation results were supposed to be compared with baseline data.
The baseline data appeared to be difficult to interpret, in spite of a verification survey of which the
final results are more reliable. Also, the criteria for assessment of access to water and access to
improved sanitation are not always clear. Also, a midterm assessment of the project results was
considered too unreliable. As a result, uncertainties remain with respect to the comparison between
baseline and impact data, and reliable trends were not always easy to establish.
In the design of the impact evaluation method we used as much as possible the same criteria as had
been used during the baseline survey. However, as noted above, in some cases the exact criteria
were not celar, e.g. to determine access to improved sanitation. Also, some adjustments had to be
made in order to stay in line with national (Ethiopia) and international standards. Therefore, the
methodology was not exactly the same as had been applied during the baseline survey.
There are no controls at household or school level. However, we believe that having controls at this
level would have been costly and there were no baseline data for control woredas (reason why it
was not part of the ToR).
The matching of quantitative and qualitative data remains a challenge but we believe to have
developed an approach to generate conclusions that make best use of both sources of information.
Ideally, one would either carry out a qualitative survey and then test resulting insights through
quantitative surveys, or the other way round carry out a quantitative survey and validate the
findings by a qualitative survey. We have done neither because the quantitative and qualitative
surveys were done in parallel (at the same time). Thus, for instance, we did not validate through
focus group discussions the quantitative finding that WASH clubs or WASH management capacity
at school did not make a difference in access to sanitation at schools.
Lastly, it remains difficult to fully attribute observed changes or differences to the SNV WASH
project, due to the fact that there are many intervening NGOs as well as government services. Over
the last few years, both have been also been active in the WASH sector. However, it has been
possible to draw firm conclusions on the contribution by SNV to the observed changes, which in
most cases are very positive.
Project number 2529 12
3. Results
3.1 Households
3.1.1 Access to safe water
Access to safe water is first of all based on the statement by the respondent if there is access to a
water source. We then included in the analysis of access to safe water the following three criteria, in
line with the SNV baseline survey:
a) an observation is made to judge the safety of the main water source,
b) travel distance to the safe main water source is less than 1.5 km, and
c) an observation is made if people have the possibility of clean water storage.
When just considering the presence of a water source (looking neither at distance nor at the
possibility of safe storage) 88% of the respondents have access to safe water. When using the above
criteria only 46% of the respondents have access to safe water (Table 2). These respondents have
access to water that is safe, relatively nearby (under 1.5 km distance) and that can be stored safely.
Almost 54% of the respondents do not have access to safe water. When looking at access to safe water
without taking into account the possibility of safe storage, 53% of the respondents report having
access to safe water. Travel distance to collect water from safe main source is the variable which has
the highest influence on household’s access to safe drinking water. Among households who collected
water from safe (protected) sources, close to one-half (47%) travelled over 1.5 km for getting water
from these safe main sources, which cannot be considered as ‘access to safe water’. We found that for
40.7% of the respondents travel time to a water scheme was more than 30 minutes, while for 70.6%
of the respondents waiting time at the water scheme was more than 30 minutes.
Table 2: Access to safe water in the 6 woredas together, according to three criteria.
Access to
safe
water
Access to safe
main water source
(no criteria)
Access to safe source
(with first two criteria)
Access to safe water
(with all three criteria)
Count (%) Count (%) Count (%)
Yes 1255 88.1 667 53.2 659 46.4
No 169 11.9 586 46.8 760 53.6
Total 1424 100.0 1253 100.0 1419 100.0
It is interesting to observe that the proportion of households with a source of water at more than 1.5
km distance (47%) is comparable to the proportion of households that have to travel more than 30
minutes (40.7%).
When looking at woreda-level, results show that access to clean water, varies among woredas.
Highest is 70% in Misrak Badewacho having access to safe water, compared to the lowest score of
17% in Demboya. Results for these and other woredas are summarized in Table 3, with indication of
the baseline values and the conclusions with respect to trends.
Project number 2529 13
Table 3: Baseline (2008) and current access to safe water in the 6 woredas, with trend
Woreda
Baseline
2008
Access to safe main
water source (2015)
Access to safe water
(all 3 criteria) (2015)
Trend 2008-
2015
(none/3 criteria) (%) Count (%) Count (%)
Alaba 50 325 89.0 74 20.4 Up / down
Boloso Sore 20 170 85.0 104 52.3 Up / up
Demboya 60 86 57.3 26 17.3 down / down
Kedida Gamella 30 182 90.5 139 69.5 Up / up
Misrak B. 80 237 94.8 175 70.3 Up / down
Shashego 80 255 98.8 141 54.7 Up / down
All 53 1255 88.1 659 46.4 Up / down
Given the uncertainties in the baseline survey, there are now two options for comparison between
baseline and impact survey data. The first option is that of comparing the baseline data with the
simple ‘access to safe main water source’ scoring (assuming that the baseline survey did not take into
account distance to water and clean storage). In this case, 5 of the 6 woredas show an improvement,
and only in Demboya there is a slight decline. The second option is that of comparing the baseline
with the more realistic data on access to water (assuming that the baseline data took into account
distance to water and clean storage). In this case, there is a decline in 4 woredas and an improvement
in 2, and an average decline in the totality of 6 woredas. Given our understanding of the baseline
survey, we assume that the second option is most likely the correct one, thus there is a deterioration
in most of the woredas.
3.1.2 Access to improved sanitation
Access to improved sanitation is not only based on the statement by the respondent that there is a
sanitation facility. We also included in the analysis four additional criteria, as adopted by SNV in the
baseline survey: a) observation that there is an improved latrine facility; b) cleanliness of the latrine,
c) regular use of the latrine, and d) cleanliness (no feces remains) in the compound.
The results show that, first of all, while 72% of the respondents perceive their latrine facility as
improved,observations showed that many of these cannot be considered as improved (e.g. pit latrine
without slab), so that based on this first criterion only 49% has access to a improved latrine facility
(Table 4).
Table 4: Perception and access to improved sanitation in the 6 woredas together.
Access to
improved
sanitation
Perception and awareness to
improved sanitation
Actual access to improved
latrine facility (observation)
Count Percent (%) Count Percent (%)
Yes 1031 72.4 703 49.4
No 393 27.6 721 50.6
Total 1424 100.0 1424 100.0
When applying the other 3 criteria, 41% of the respondents should be considered as having access to
improved sanitation (table 5). These respondents have access to a latrine facility that is improved,
Project number 2529 14
clean, regularly used and used by everyone so that the compound is clean. Thus, the compound would
also be open defecation free (ODF). Based on the four criteria it must be concluded that almost 59%
of the respondents do not have access to improved sanitation.
Table 5: Access to improved sanitation in the 6 woredas together, according to three criteria.
Access to
improved
sanitation
Observation of
safe latrine
Including
cleanliness
Including regular use and
cleanliness compound
Count (%) Count (%) Count (%)
Yes 703 49.4 629 44.2 589 41.4
No 721 50.6 795 55.8 835 58.6
Total 1424 100.0 1424 100.0 1424 100.0
When looking at woreda-level, results show that access to improved sanitation varies among
woredas. Highest is 57% in Kedida having access to improved sanitation, compared to the lowest
score of 30% in Demboya. Results for these and other woredas are summarized in Table 6, with
indication of the baseline values and the conclusions with respect to trends.
Table 6: Baseline (2008) and current access to improved sanitation in the 6 woredas, with trend
Woreda Baseline
(a)
(2008)
(%)
Improved
sanitation (1
criteria) (2015)
Baseline
(b)
(2008)
Improved
sanitation (4
criteria) (2015)
Trend
(safe / safe &
clean)
Count (%) (%) Count (%)
Alaba 50 138 37.8 20 117 32.1 down / up
Boloso Sore 50 105 52.5 10 79 39.5 up / up
Demboya 60 60 40.0 30 45 30.0 down / stable
Kedida 90 125 62.2 40 114 56.7 down / up
Misrak B. 60 126 50.4 20 110 44.0 down / up
Shashego 70 149 57.8 40 124 48.1 down / up
All 63 703 49.4 27 589 41.4 Down / up
Given the uncertainties in the baseline survey, there are now two options for comparison between
baseline and impact survey data. The first option is that of comparing the baseline data (baseline (a),
column 2) and current data (columns 3 and 4) for the improved sanitation indicator that only took
into account the first criterion (assuming that the baseline survey did not take into account
cleanliness aspects and regular use). In this case, 5 of the 6 woreda’s show a decline, and only in
Boloso Sore there is a slight increase. The second option is that of comparing the improved sanitation
baseline value (baseline (b), column 5) with the improved sanitation indicator that took into account
all four improvement criteria (columns 6 and 7) (assuming that the baseline survey took into account
aspects of cleanliness and usage). In this case, there is an improvement in 5 of the 6 woredas and one
woreda where there is no change. Given our understanding of the baseline survey, we assume that
the second option is most likely the correct one, thus we observe a significant improvement in most
cases.
Project number 2529 15
3.1.3 Incidence of diarrhea and analysis
The survey also looked at the incidence of diarrhea. It appears that 17% has suffered from diarrhea
during the last year, and 83% has not. Looking at the incidence of diarrhea during the last 2 weeks, of
those who suffered from diarrhea in the last year, 32% suffered from diarrhea during the last 2
weeks. The baseline survey in 2008 gives an average value of 52% that suffered from diarrhea, it is
not clear during which time range. In any case, whether within one year or whether within the last 2
weeks, the incidence of diarrhea has strongly declined.
We now explore the relation between the incidence of diarrhea and access to safe water (overview
tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5% level of significance)
confirm that there is a significant relationship between access to safe drinking water and incidence of
diarrhea over the last one year (Table 7). The result corroborates that the households who did not
have access to safe drinking water were more likely to be affected by diarrheal diseases in the last one
year than households who had access to safe and clean water. Furthermore, the result enables to
draw a conclusion that over the last one year, households who accessed safe and clean water were less
exposed to contracting diarrheal diseases as compared to those who had no access to safe and clean
water. However, the statistical tests show no significant relation with the incidence of diarrhea over
the last 2 weeks.
Table 7. Relationship between access to safe water and incidence of diarrhea (absolute frequencies)
Access to safe water
Yes No
Incidence of
diarrhea over last
year*
Yes 91 154
No 568 606
Total 659 760
Incidence of
diarrhea in last 2
weeks**
Yes 30 48
No 61 106
total 91 154
* Relationship between access to safe water and incidence of diarrhea over last year is significant:
Chi-square (df) = 1, p <.05.
** Relationship between access to safe water and incidence of diarrhea over last 2 weeks is not
significant: Chi-square (df) = 1, not significant
Table 8. Relationship between access to improved sanitation and incidence of diarrhea (absolute
frequencies)
Access to improved sanitation
Yes No
Incidence of
diarrhea over last
year*
Yes 83 162
No 506 673
Total 589 835
Incidence of
diarrhea in last 2
weeks**
Yes 21 57
No 62 105
Total 83 162
* Relationship between access to improved sanitation and incidence of diarrhea over last year is
significant: Chi-square (df) = 1 , p <.05.
** Relationship between access to improved sanitation and incidence of diarrhea over last 2 weeks is
significant: Chi-square (df) = 1, p<.05.
Project number 2529 16
We now explore the relation between the incidence of diarrhea and access to improved sanitation
(overview tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5% level of
significance) show that there is a statistically significant relation between incidence of diarrhea in the
last one year and access to improved sanitation (Table 8). There is a similar relation between
incidence of diarrhea in the 2 weeks and access to improved sanitation.
We can conclude that households with access to improved sanitation were less likely to be exposed to
diarrheal disease over the last one year and during the last two weeks, in contrast to those households
without access to improved sanitation. The fact that both relations were found with the incidence of
diarrhea in the last one year and the last two weeks suggest a more fine-grained and significant
relation with access to improved sanitation than with access to water.
3.1.4 Access to information and training on WASH
The survey also looked at whether households had received access to information and training
support on WASH facilities. We found that 74% of the respondents received both information and
training support, 9% received either of the two and 8% none at all. This information or training
support indicator was not captured during the baseline survey.
We now explore the relation between the scores on information and training support on WASH and
access to safe water (overview tables see appendix 3.1). The statistical tests (Chi-square and
correlation at 5% level of significance) indicate that there is no significant relationship between
access to safe drinking water and the receipt of information or support on WASH.
We now explore the relation between the scores on information and support on WASH and improved
sanitation (overview tables see appendix 3.1). The statistical tests (Chi-square and correlation at 5%
level of significance) confirm that there is statistically significant relation between household’s access
to improved sanitation and the information and training support they obtained on WASH (Table 9).
It means that households who have got information and training support on WASH are more likely to
access improved sanitation than their counterparts who have not got any information and support.
Table 9. The relationship between scores on information and training support on WASH and access
to improved sanitation (absolute frequencies).
Access to information and training support on WASH
no Either Both
Access to safe
drinking water*
Yes 53 127 479
No 67 116 577
Total 120 243 1056
Access to
improved
sanitation**
Yes 23 89 477
No 97 156 582
Total 120 245 1059
* Relationship between access to safe drinking water and access to information / training support on
WASH is not significant: Chi-square (df) = 2 , ns.
** Relationship between access to improved sanitation water and access to information/ support on
WASH is significant: Chi-square (df) = 2, p <.05.
We conclude that provision of information and support on WASH to households is related to access
to improved sanitation. On the contrary, information and support on WASH is not related to access
to safe water. This may be explained by the fact that access to safe drinking water is not so much a
Project number 2529 17
matter of behavioral change rather than the hardware (wells, boreholes, …) being available close to
communities and households and also being maintained and being operational.
3.1.5 Additional analysis on explaining factors
Additional analyses were carried out to identify factors explaining some of the patterns that we
found. This was particularly useful to do for the household survey results, given the large data set and
availability of information on household characteristics. For the school survey, this was not
considered very relevant, as the data set is much smaller and there is also considerable variability
between schools (e.g. school size, location, history, type of school) while many of these factors were
not included in the survey.
The following statistical analyses were carried out for the household survey data, with corresponding
results.
Statistical analysis of relation between access to safe drinking water and sex of head of household:
no significant relation was found
Statistical analysis of relation between access to safe drinking water and age of head of household:
no significant relation was found
Statistical analysis of relation access to safe drinking water and family size: no significant relation
was found
Statistical analysis of relation between access to improved sanitation and sex of head of household:
no significant relation was found
Statistical analysis of relation between access to improved sanitation and age of head of household:
here we find a statistical relation (see table 10). The result shows that the households headed by
young score better than households headed by aged ones for changing their attitude and behavior
for accessing improved sanitation. In other words, the aged heads of households may lag behind to
change their attitude and behavior for accessing improved sanitation due to long-standing
traditional beliefs on support of defecating openly or continuing o use the traditional unimproved
latrine facilities.
Table 10. The relation between access to improved sanitation and age of head of household
Access to improved
sanitation
Age of head of household, in years
18 – 30 30 – 64 65 and above Total
Have access * 208 / 44.7% 363 / 40.6% 18 / 27.7% 589 / 41.4%
No access * 257 / 55.3% 531 / 59.4% 47 / 72.3% 835 / 58.6%
Total 465 / 100% 894 / 100% 65 / 100% 1424 / 100%
* Relationship between access to improved sanitation and age of head of household is significant:
Chi-square (df) = 2 , p<0.5.
Statistical analysis of relation between access to improved sanitation and family size: here we find a
statistical relation (see table 11). The result shows that the households with large family size are
better than those households with small ones for changing their attitude and behavior for accessing
improved sanitation. In other words, the households with large family may have better opportunity
for getting a wide range of WASH information through different channels on benefits of having
improved sanitation for changing the attitude and behavior of the household for switching from
traditional beliefs and thinking to access improved sanitation. Large families may have better
resources especially labor for constructing / building improved latrine facility than their
counterparts with small family size.
Project number 2529 18
Table 11. The relation between access to improved sanitation and size of family
Access to
improved
sanitation
Family size, number of members
Total 2 or less 3–5 6-7 8-10 > than 10
Have access * 20 / 37.0% 181 / 36.9% 188 / 37.3% 175 / 51.8% 25 / 58.1% 589 / 41.4%
No access * 34 / 63.0% 309 / 63.1% 311 / 62.3% 163 / 48.2% 18 / 41.9% 835 / 58.6%
Total 54 / 100.0% 490 / 100% 499 / 100% 338 / 100% 43 / 100% 1424 / 100%
* Relationship between access to improved sanitation and family size is significant: Chi-square (df) =
4, p<0.5.
Overall, the additional analyses show that access to water cannot be explained by factors related
household characteristics, but access to improvd sanitation does show relations with head of
household and family size, but not for sex of head of household. This is due to the fact that sanitation
is a facility/service at household level that can be created directly by households, while the water
supply is a collective facility/service, its development depends often on external funding.
3.2 Schools
3.2.1 Access to safe water in schools
According to the results of the surveys, out of 50 schools there are 11 with access to safe water. Table
12 shows the results with respect to access to safe water in the school compounds, as distributed per
per woreda.
Table 12: Baseline (2007) and current access to safe water in schools in the 6 woredas, with trend
Woreda
Baseline (%)
(2007)
Access to safe water in schools (2015) Trend
Positive / total per Woreda (%)
Alaba 2 1 / 13 7.7 Up
Boloso Sore 6 1 / 8 12.5 Up
Demboya 4 2 / 6 33.3 Up
Kedida Gamella 15 2 / 5 40.0 Up
Misrak B. 14 2 / 8 25.0 Up
Shashego 0 3 / 10 30.0 Up
All 7 11 / 50 22.0 Up
We observe that 22% of the schools have access to safe water within the school compound in 2015,
which is considerably higher than in 2007 (7% only).
Our survey also found that of those that do not have access to safe water within the school compound
(39 out of 50), 15 have access to safe water outside the school, meaning that water is being fetched
outside the school. The remaining 24 schools that do not have access to safe water within the school
compound, also do not have access to safe water outside the school compound. This means that 48%
of the schools surveyed (24 schools) do not have access to water within or outside the school.
Project number 2529 19
3.2.2 Access to improved sanitation in schools
With respect to access to improved sanitation, we look at safe latrine facilities and whether these are
well maintained and clean, using the following three criteria: a) observation that there is an improved
latrine facility; b) cleanliness of the latrine, c) security / privacy of the latrine. Table 13 gives an
overview. Table 14 compares the baseline valued of 2007 with the current values, per woreda.
Table 13: Access to improved sanitation in schools, in different categories according to criteria
Access to
improved
sanitation
Access to improved
sanitation (1 criteria)
Access to improved
sanitation (2 criteria)
Access to improved
sanitation (3 criteria)
Count (%) Count (%) Count (%)
Yes 47 94.0 14 28.0 13 26.0
No 3 6.0 36 72.0 37 74.0
Total 50 100.0 50 100.0 50 100.0
Table 14: Baseline (2007) and current access to improved sanitation in schools in 6 woredas
Woreda
Baseline
value (%)
(2007)
Access to improved
sanitation (1 criteria)
Access to improved
sanitation (3 criteria)
Trend
Safe/ safe&clean
Count (%) Count (%)
Alaba 54 10 76.9 3 23.1 Up / down
Boloso Sore 35 8 100.0 2 25.0 Up / down
Demboya 41 6 100.0 1 16.7 Up / down
Kedida Gamella 65 5 100.0 1 20.0 Up /down
Misrak B. 56 8 100.0 3 37.5 Up / down
Shashego 40 10 100.0 3 30.0 Up / down
All 49 47 94.0 13 26.0 Up / down
Due to uncertainties with respect to the baseline values, there are now two different options for
comparison. The first option is when we compare data on improved sanitation with the baseline value
(assuming that the baseline survey did not take into account aspects of cleanliness and security). In
this case, we observe a considerable improvement (positive trend). This may be explained by the fact
that almost all schools nowadays have established improved latrine facilities, which was not the case
in 2007. The second option is when comparing the baseline value with the indicator on improved
sanitation (assuming that the baseline survey took into account aspects of cleanliness and usage). In
this case, we observe a downward trends in all cases. This may be due to the fact that many schools
do not have access to safe water within the school compound so that the sanitation facilities are not
well maintained. In addition, as almost all schools don’t have hired cleaners for cleaning toilets and
they use students to do so, and in some schools students are supposed to clean toilets as punishment
when they violate school regulations, it can hardly be expected that toilets are kept clean at all times.
3.2.3 Functionality of WASH club and analysis
Of the 50 surveyed school, 46 (92%) has a WASH club available. However, only 36 (72%) of the
schools have a functional WASH club. Functionality was defined on the basis of the fact that the
WASH club members come together at least once a month. We analyzed whether there is any relation
between functionality of the WASH club and access to safe water or access to improved sanitation
Project number 2529 20
within schools. The statistical analysis (Chi-square and correlation at 5% level of significance) did not
show any significant relation (Table 15).
Table 15. The relationship between functionality of WASH-clubs and access to safe drinking water
and improved sanitation (absolute frequencies).
Functionality of WASH-club
Not functioning functioning
Access to safe
drinking water*
Yes 4 7
No 10 29
Total 14 36
Access to
improved
sanitation**
Yes 3 10
No 11 26
Total 14 36
* Relationship between access to safe drinking water and functioning of WASH-clubs is not
significant: Chi-square (df) = 1 , ns.
** Relationship between access to improved sanitation water and functioning of WASH-clubs is not
significant: Chi-square (df) = 1 , ns.
3.2.4 WASH related school management and analysis
Among the 50 surveyed schools, we also asked 8 questions to school management level to assess their
capacity of managing the WASH facilities. The eight questions have been grouped for rating the level
of management capacity as: poor (if the answers for two of the eight questions are positive); good (if
answers to three to five questions are positive); and very good (if answers for at least six of eight
questions are positive). Note that in 4 cases teachers responded ‘don’t know’ for some of the
management related questions, these were removed from the analysis.
Out of the 46 responses, 4 (9%) were considered to have poor WASH management capacity, 29 (63%)
fairly good, and 13 (26%) had very good capacity.
We analysed whether there is any relation between school WASH management capacity and access to
safe water or access to improved sanitation within schools. The statistical analysis did not show a
significant relation (Table 16).
Table 16. The relationship between WASH-management (by school management) and access to safe
drinking water and improved sanitation (absolute frequencies).
WASH-management by school management
poor good Very good
Access to safe
drinking water*
Yes 0 8 2
No 4 21 11
Total 4 29 13
Access to
improved
sanitation**
Yes 0 5 3
No 4 24 10
Total 4 29 13
* Relationship between access to safe drinking water and WASH-management by school
management is not significant: Chi-square (df) = 2 , ns.
** Relationship between access to improved sanitation water and functioning of WASH-clubs is not
significant: Chi-square (df) = 2 , ns.
Project number 2529 21
3.2.5 School attendance and analysis
Among the 50 surveyed schools, we also asked in the survey if school attendance had improved in the
last 6 years. Out of the 50 schools, 47 (94%) gave a positive response, 3 (6%) gave a negative
response.
We analysed whether there is any relation between improved school attendance during the last 6
years and access to safe water or access to improved sanitation within schools. The statistical analysis
(Chi-square and correlation at 5% level of significance) did not show a significant relation (Table 17).
Table 17. The relation between improved school attendance during the last 6 years and access to
safe water or access to improved sanitation within schools
Improved school attendance
No yes
Access to safe
drinking water*
Yes 1 10
No 2 37
Total 3 47
Access to
improved
sanitation**
Yes 0 8
No 3 39)
Total 3 47
* Relationship between access to safe drinking water and improvement of school attendance is not
significant: Chi-square (df) = 1 , ns.
** Relationship between access to improved sanitation water and improved school attendance is not
significant: Chi-square (df) = 1 , ns.
3.3 Water schemes
3.3.1 Functionality of water schemes
The functionality of water schemes is based in 3 criteria: water discharge volume, water quality and
waiting time. We first present the findings on these 3 criteria.
Water discharge
The classification of water discharge volume per water scheme and per woreda is presented in Table
18. It shows that:
8% of the water schemes no water was coming out of the water point
12% of the water scheme had low water discharge or less water discharge during the dry seasons
80% of the water schemes had good water discharge.
Table 18: Functioning of water discharge per woreda
Water
discharge
Total Kedida
gamela
Alaba Misrak
Badawacho
Demboya Shashego Boloso
Sore
no 8% 57% 0% 29% 0% 10% 9%
moderate 12% 0% 0% 0% 20% 0% 0%
yes 80% 43% 100% 71% 80% 90% 91%
Total 100% 14% 10% 14% 20% 20% 22%
Project number 2529 22
Water quality
The classification of water quality according to the respondents to the survey gives the following
results:
14% stated that water quality is poor (of which 4 cases – 8% - are obvious because of no discharge)
8% stated that water quality is less during the rainy season
78% stated that the water quality is good
Waiting time
The classification of waiting time per water scheme and per woreda is presented in Table 19. It shows
that:
Of all water schemes surveyed, for 38% the waiting time is 30 minutes or less, which is within our
criteria of a functional water source. Even though a large part of the water schemes were reported
to have good water discharge (80%), 63% of these schemes have a waiting time of > 30 minutes
which we considered as ‘partly functional’.
At the far end of the waiting time, 18% of the water schemes have a waiting time of more than 5
hours.
Looking at the waiting time for water schemes within woredas, waiting time is by far lowest in
Boloso Sore (all water schemes 30 minutes or less), and by far longest in Alaba (60% with more
than 5 hours waiting).
Table 19: Waiting time per water scheme, per woreda
Woreda % of
surveyed
schemes
Waiting time
30 min or
less
30 min-
2hr
2-5 hr > 5 hrs
Kedida Gamela 8% 33% 67% 0% 0%
Alaba 13% 0% 40% 0% 60%
Misrak Badawacho 13% 40% 0% 20% 40%
Demboya 20% 0% 50% 38% 13%
Shashego 23% 22% 22% 44% 11%
Boloso Sore 25% 100% 0% 0% 0%
Total 100% 38% 25% 20% 18%
Functionality
Based on the classification of the water schemes to assess functionality (see section 2.2.2) we found
the following results (Table 20). For easy comparison of relative scores per woreda, we attributed
values to each of the 3 categories: functional receives a score of 2, partly functional a score of 1, not
functional a score of 0. Table 17 shows that:
Few water schemes (24%) are fully functional, most (62%) are partly functional;
The highest scores on functionality of water schemes is found in Boloso Sore (1.37), followed by
Shashego (1.10); the lowest score is found in Kedida (0.88). All the others are in between with
almost the same score (1.0).
The most discriminating factor for water schemes to be partly functional is waiting time. There are
also 3 cases with a low discharge volume or no water during the dry season as well as having poor
water quality and a long waiting time. There are 3 cases with moderate or lower water quality
during the rainy season.
Among the water schemes that are not functional (7 cases or 14%), there are 4 cases with no
discharge volume, and there are 3 cases with poor water quality.
Note that several woreda’s report that water schemes are functional, but then observe that water
discharge is very low – we have integrated this finding in the functionality scores.
Project number 2529 23
Table 20: Functionality of water schemes
Functionality
water schemes
Total Kedida
gamela
Alaba Misrak
Badawacho
Demboya Shashego Boloso
Sore
Functional (2) 24% 29% 0% 29% 0% 20% 55%
Partly functional (1) 62% 43% 100% 43% 100% 70% 27%
Not functional (0) 14% 29% 0% 29% 0% 10% 18%
(score) 0.883
1.00 1.01 1.00 1.10 1.37
3.3.2 Performance of the WASH committees
Based on the classification of the WASH committees to assess performance (see section 2.2.2) we
have found the following results (Table 21).
Table 21: Performance of WASH Committees
Performance
WASH
Committees
Total Kedida
gamela
Alaba Misrak
Badawacho
Demboya Shashego Boloso
Sore
Good 88% 88% 100% 100% 82% 100% 90%
Poor 12% 14% 0% 0% 30% 10% 9%
We can observe the following:
By far most WASH committees (88%) are performing well, as based on our set criteria. This means
the members are satisfied with the WASH committee, fees are charged, fees are used for O&M,
members are satisfied with the fees, and if there are conflicts, these are resolved;
Among the not performing WASH committees (12%), in 2 cases there was no WASH committee, in
2 cases the users were not satisfied with the WASH committee and in 2 cases no fees were charged
and conflicts did occur.
In terms of conflicts, in total 19 water schemes did not have any conflicts, in all other cases there
have been minor or major conflicts but these have been resolved (25) or not (4).
3.3.3 Preventive and Operational capacity training by SNV
During the survey we also asked for the perception of the users and WASH committee on the
progress on operation and maintenance of the water scheme during the last few years (Table 22). In
general almost 70% of the water schemes have made progress on Operation and Maintenance. More
than half of the progress has been made since 2008 and 16% continuously make progress.
As indicated in section 2.2.2, the 50 surveyed water schemes were evenly distributed over 4 distinct
categories: those constructed before and after 2007, and those with and without SNV training on
operations and maintenance received. Table 20 shows the relation between the perceived progress
and training on preventive operations and maintenance (POM) received with these 4 categories. We
did not carry out statistical analyses but compared to what extent the water schemes within a certain
category of progress in operations and maintenance are distributed over these 4 categories of water
schemes. It seems that the frequencies over the 4 different categories of water schemes is generally
3
For example the score for Kedida is composed as follows: (29*2) + (43*1) + (29*0) / 100 = 0.88
Project number 2529 24
evenly distributed. Of the WASH committees that observed progress in POM, 56% did receive
training by the SNV project. Looking at the WASH committees that do not observe any progress, a
slight minority (43%) received SNV training, while the majority are older schemes.
Table 22: Progress in terms of Operations and Maintenance
Progress on
Operation and
Maintenance
Total Distribution of water schemes with different levels of operations
and maintenance over the 4 categories
Before 2007
No SNV POM
training
Before 2007
+ SNV POM
training
After 2007
No SNV POM
training
After 2007
+ SNV POM
training
Progress since
2008-2015
54% 19% 26% 26% 30%
Progress before
2008
2% 0% 0% 0% 100%
Continuous
progress
16% 25% 13% 25% 38%
No progress 28% 29% 36% 29% 7%
It can be concluded that if progress is made on POM there is no clear relation with ‘age’ of the water
scheme and whether SNV training has been provided on operations and maintenance issues. It must
be observed that many other agencies also provide training on operations and maintenance.
Table 23 shows the relation between the performance of the WASH Committee and the 4 categories
of water schemes (POM training and ‘age’). The distribution of scores is equally distributed over the
four categories of water schemes, but the number of not performing WASH committees is rather
small. It suggests that there is no clear relation between the performance of the WASH Committees
and preventive operational and maintenance training having been provided by SNV or by others.
Table 23: Relation between performance WASH Commitees and POM training
Performance
WASH
committee
Total Distribution of water schemes with different levels of
performance of WASH committee over the 4 categories
Before 2007
No SNV POM
training
Before 2007
+ SNV POM
training
After 2007
No SNV POM
training
After 2007
+ SNV POM
training
Good 88% 27% 27% 20% 25%
Poor 12% 17% 17% 35% 35%
3.3.4 Comparison with baseline situation
We will now compare the current situation with the baseline measurement of water schemes. For the
baseline situation, an assessment was made of three parameters:
The functionality of water schemes – probably implying whether there is water discharge
The management of water schemes – probably implying whether there is a WASH committee
managing the water scheme
The recovery of O&M costs by user fees – probably implying whether user fees are being paid.
Table 24 shows a comparison between the first two parameters and our findings.
Project number 2529 25
Table 24: A comparison of baseline (2007) and current (2015) information on water schemes and
WASH committees
Functionality
water schemes
and WASH Co
Kedida
gamela
Alaba Misrak
Badawacho
Demboya Shashego Boloso
Sore
2015 Functional 29% 0% 29% 0% 20% 55%
Partly functional 43% 100% 43% 100% 70% 27%
Not functional 29% 0% 29% 0% 10% 18%
Poor performing
WASH Co
14% 0% 0% 30% 10% 9%
2007 Not functional 14% 39% 33% 21% 53% 25%
No / poor
management
49% 50% 73% 52% 47% 40%
No cost recovery 35% 32% 97% 34% 57% 100%
From the comparison we can draw the following conclusions:
The proportion of non functional water schemes has declined between 2007 and 2015 for all
woredas, except for Kedida. However, it should be noted that functionality in the 2015 survey was
defined as no discharge (4 cases) or poor water quality (3 cases), whereas in the baseline survey
this probably only refers to no discharge. What has caused the improvement for Kedida is unclear.
The proportion of no or poor management or no cost recovery in 2007 is always (much) higher
than the proportion of poorly performing WASH committees in 2015. This means there is a
considerable improvement in terms of the performance of WASH committees.
3.4 Capacity building
As explained in section 2.2.3, the impacts on capacity building were evaluated at four different levels:
Woredas (8 woredas, of which 6 targeted and 2 control woredas), regional authorities
TVETC and UNICEF. These will now be treated subsequently.
3.4.1 Capacity building interviews and focus group discussions per woreda
As indicated in section 2.2.3, the capacity building assessments were analysed using a balance score
card approach, using a 3-point scoring system for current status (A, B, C), and trend over the last few
years (1, 2, 3) – see appendix 6.4 for detailed narratives of each woreda. The list of persons who were
interviewed or who took part in the focus group discussions is listed in appendix 2.2.
Part of the interview and focus group discussion was to ask for the contribution by SNV to any of the
observed changes. In some cases the answers were not affirmatively positive or negative, in which
case we noted unclear. We also noted some relevant quotes that characterize the situation. Following
are the results for each of the 8 woreda’s separately, Tables 25-30 including the 6 targeted woreda’s
and then Tables 31-32 the 2 control woreda’s.
Project number 2529 26
Table 25: Overview of capacity building results Woreda Boloso Sore
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes Important improvements but now tendency to
retreat back and concerns about future
Financing 2 C yes Budget increased from 25,000 birr to 500,000 birr.
Awareness and priority has increased, but budget is
not enough.
Service delivery 2 C yes Response rate much improved, but too few staff to
manage 300 water schemes, currently only 2 staff on
maintenance, suggest a new structure
Incidence of diarrhea declined
Increasing population so demands
Gender aspects 3 C Not clear Gender participation has much improved last 7 years.
Percentage of women in kebele WASH committee at
least 50%. Much improved
Community
participation +
private sector
2 C yes Water schemes managed by communities
Private sector involvement of artisans
Improved functionality of water schemes
Data base +
monitoring
2 C yes Staff used MIS initially only
Needs specific budget for M&E WASH
Partnerships 2 B? Not clear Several NGOs have shifted their priority from WASH
to agriculture and food security
Table 26: Overview of capacity building results Woreda Misrak Badewacho
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes We used to have plans that were abstract, now much
better, with budget and responsibilities
Learned to carry out a good problem analysis
Financing 3 C yes Due to SNV training on fundraising we are able to
prepare our own projects
WASH budget has become top priority
UNICEF selected this woreda as a model
WASH budget still not enough
Service delivery 2 C yes Non functionality of water schemes has declined
from 32% to 12%
There is mainly transport shortage: 1-2 motorbikes
for all WASH staff
GLOWS has been very useful
Gender aspects 3 C partly Specially improved women and girls participation in
schools
Community
participation +
private sector
3 C partly There is a cooperative for spare parts supported by
another NGO, this has greatly contributed to
functionality of water schemes
Data base +
monitoring
2 C yes Staff uses MIS occasionally
The budget for M&E is very limited
Partnerships 3 C Not clear Support by NGOs is increasing in providing ‘full
packages’ e.g. schools with toilets and water supply
Project number 2529 27
Table 27: Overview of capacity building results Woreda Kedida Gamella
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes Efforts to scale up the good practices to more
schools and households
Financing 2 C yes We have an operational budget only and it has
increased, but not for construction, here we depend
on donors
Several new schemes constructed recent years
Service delivery 2 C partly We lack electric and mechanical engineers, this
should be supplied by the region
The staff positions are not attractive
Gender aspects 3 C yes Gender participation has focused at schools
Collaboration with departments of women &
children affairs
Community
participation +
private sector
2 C Not clear There are traders who supply spare parts
There are currently 4 cooperatives producing slabs
and one enterprise supporting sanitation in schools
Data base +
monitoring
2 B yes Staff only used MIS once
Not enough staff and budget
Partnerships 2 ? Not clear
Table 28: Overview of capacity building results Woreda Shashego
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes Important improvement in collaboration within
woreda between health, education and water
Financing 2 C yes Budget and priority has increased but is not enough
for construction
The contribution by donors remains very important
Several new schemes constructed recent years
Service delivery 2 C yes The spare-parts shop has been very useful, provided
by another NGO
Budget and staff for small mechanics is OK, but for
large mechanical issues it is missing
Gender aspects 3 C yes We follow the SNV WASH manual for women
participation
Women now feel ashamed if there is no latrine in
their household
Fetching water has changed from an open storage to
a closed jerry-can
Community
participation +
private sector
2 B unclear There are traders who supply spare parts
Two active artisans had to stop their business
because the woreda water office did not
acknowledge
Data base +
monitoring
2 C yes Staff used MIS only once
M&E is not a priority and there are no funds, but
some updates are made
Partnerships 3 B? Not clear Several NGOs are busy in WASH
Project number 2529 28
Table 29: Overview of capacity building results Woreda Alaba
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes Important improvements in planning, especially by
collaboration between 3 core sectors
However, water coverage remains far too low
Financing 2 C yes Budget from the government has increased to 3
million birr, but inadequate for new constructions
mainly.
Several new schemes constructed recent years
Service delivery 2 C yes Response rate much improved, but few staff
Functionality of water schemes much improved
Gender aspects 3 C Not clear Gender awareness and participation has much
improved.
However, there remains a serious water shortage,
reason why girls cannot go to school in some cases.
Community
participation +
private sector
2 C yes Much improved but some communities require
more support
Nor private sector providing spare parts
Data base +
monitoring
2 C yes Staff used MIS initially only
Data collection on WASH is continuing, we are up-
to-date with the data base
We receive lump sum for WASH, not specific M&E
Partnerships 2 B? Not clear
Table 30: Overview of capacity building results Woreda Damboya
Aspect
Scores Contribution
SNV
Quotes Status Trend
Strategic plan 3 C yes Important improvements but coverage is still
insufficient in large part of the woreda and for
several schools
Financing 2 C yes Donors provide software but the hardware is still
missing
Especially a problem for kebeles in mountain areas
Service delivery 2 B yes Response rate for maintenance improved
Plumbing and electronic experts are missing
On-the-job training has been good
There has been good training but recently declined
again
Gender aspects 3 C yes Gender participation in WASH committees has
much improved
Community
participation +
private sector
2 C yes Much more ownership developed among
communities, now reached 80-90%
Yet, some CBOs still do not have enough capacity
Data base +
monitoring
2 C yes Staff trained on MIS but did not use it
Data base on water schemes is up-to-date
No specific budget on M&E
Partnerships 2 B Not clear Several NGOs are not anymore active in the WASH
sector
Project number 2529 29
Table 31: Overview of capacity building results Woreda Boloso Bombe (control woreda)
Aspect
Scores
Quotes Status Trend
Strategic plan 2 B No integrated woreda WASH strategic plan, but separate plans for
the 3 different units, no WASH sector coordination
This year the integration will take place
Access to water has improved in recent years
Financing 2 B Awareness on WASH is low, e.g. compared to malaria and
HIV/AIDS
Budget for WASH is only 90,000 birr, it slightly increased this year
No specific government budget for sanitation, this is all covered by
a donor.
Service delivery 2 B No on-the-job training on WASH
Quite a lot of community awareness raising activities took place on
hygiene and sanitation
CLTS approach has been adopted
Most households and schools now do have latrines, but their use
and maintenance is low and awareness is low
Lack of spare parts
Gender aspects 3 C Gender participation has improved last years.
Percentage of women in kebele WASH committee at least 50%.
Women and youth have priority in service delivery
Community
participation +
private sector
2 B Community participation in WASH has increased considerably,
especially since 2012
No private sector involvement
Data base +
monitoring
2 C No training on MIS
Training received on monthly data collection water scheme
functionality, which is currently being done
Very limited budget, staff and logistics for M&E
Partnerships 2 B Apart from government, 3 NGOs active in WASH sector
Project number 2529 30
Table 32: Overview of capacity building results Woreda Angecha (control woreda)
Aspect
Scores
Quotes Status Trend
Strategic plan 2 B No integrated woreda WASH strategic plan, but separate plans for
the 3 different units, no WASH sector coordination
This is the situation in spite of support by AfD
Initiatives were taken to make kebeles ODF, but this initiative has
been retreated
Financing 2 B Priority and awareness on WASH is low, thus budget is also low
Budget is just enough for one hand-dug well per year
There is a high dependency on donors, both for maintenance and
for constructing new water schemes
Service delivery 2 B Since 2011 there has not been any additional water scheme
constructed, the communities are forced to do this by themselves
Received on-the-job training on WASH
Community awareness raising activities took place on hygiene and
sanitation
CLTS approach has been adopted
Most households and schools have latrines, their use and
maintenance is low and awareness is low; there is no proper system
for maintenance of water schemes
Lack of spare parts
Gender aspects 2 B Training on gender and development, but not on gender and WASH
Community
participation +
private sector
2 B Ownership and participation by community in WASH has increased
No private sector involvement
Data base +
monitoring
2 B Training on MIS was received but it was not used
M&E is only done for newly constructed water schemes
The data base for WASH sector is not updated
Limited budget, staff and logistics for M&E
Partnerships 2 B Since 2011 very little support by NGOs
Conclusions from comparing targeted and control woreda’s
Both the scores for current situation and the score for improvement trends during the last few
years are more positive for the 6 targeted woreda’s than for the 2 control woreda’s.
Most significant are the following conclusions and differences per aspect:
Strategic planning: the targeted woreda’s much appreciate integrated strategic planning
(involving health, education and water sectors). The SNV WASH project has contributed
significantly. The control woreda’s are aware of the risk of fragmented planning and also want to
move to a more integrated approach.
Finance: the targeted woreda’s have been able to attract more government and donor funding, as
a result of better planning and support in fundraising. The SNV WASH project has contributed
significantly. However, funds are still insufficient especially for large scale constructions and
maintenance of water schemes. Control woreda’s seem to remain more dependent on donor
funding.
Service delivery: the targeted woreda’s have been able to give more training on WASH awareness
raising as well as POM aspects, reason why water schemes are stated to be well maintained and
more functional than in the past. Also, at household level communities better maintain their
latrines. The response rate to repairs has increased. The SNV WASH project has contributed
significantly. However, service delivery on maintenance of large water schemes and large-scale
Project number 2529 31
mechanical issues remains problematic even in targeted woreda’s because of limited staff, budget
and logistics. In the control woredas less new water schemes seem to have been constructed, and
there are more concerns about their maintenance and sustainability.
Gender awareness: there does not appear to be much difference between the targeted and
control woreda’s, even though some targeted woreda’s specifically mention training provided by
the SNV project and the use of their manual. Important has been the new regional regulation for
women to be nominated for leadership positions and at least 50% women in WASH COs and
other WASH activities.
Community participation and private sector involvement: in the targeted woreda’s communities
show sense of ownership and participation. Also, there are initiatives of private sector, e.g. in
spare part provision. The SNV WASH project has contributed significantly, although other NGOs
also play an important role. In the control woredas there are more concerns about community
ownership and less private sector involvement.
Monitoring and evaluation. Training on the MIS has not been followed up by further support,
and is therefore not anymore used. While some woredas state that the data base on WASH is up-
to-date, others say they have insufficient staff to maintain the data base.
Partnerships: most targeted woreda’s have many partners willing to invest in the WASH sector,
although some seem to have shifted their attention to agriculture and food security. This might
be explained by the fact these woreda’s have better and more integrated WASH strategic plans
and funding strategy. The control woredas have less donor and NGO support.
3.4.2 Interviews at regional level
From the interviews at regional level, the following main points can be extracted (the full interview
narratives are found in appendix 6.3).
1. Regional WASH Strategic Planning
In the past, efforts were scattered but there is now a joint approach by the ministries of water,
health and education. Under the integrated (‘one WASH’) approach there is more efficiency and
effectiveness in services. In the coming five years schools through the woreda education office
schools are entitled to construct their own latrines and water points.
For the education sector school improvement project, water, sanitation and hygiene is one of the
components and WASH is one of the quality indicators for schools.
The majority of schools have latrines constructed from locally available materials. Access to water
at schools remains challenging; sanitation and hygiene is difficult where there is limited access to
water. Schools without access to water are encouraged to look for alternative sources of water - like
collecting water from nearby springs, store using water tanks, use roof catchments for water
collection etc. There will be more roof catchment techniques to collect and store water.
A major change has been that from only construction to work in changing the behavior of the
community, in order to avoid that new water schemes are misused or not maintained.
During 2010-2014 the regional water coverage showed some improvement and reached 67%. But
the sector still faces a number of challenges:
limited number and low capacity contractors for water resource development,
limited number of private sector involved,
low government encouragement and motivation of staff, e.g. the WASH sector needs hydrologist,
geologist, electro mechanics.
There is need to change the structure of the water sector at region, zonal and woreda levels, using a
cascade structure, with three core processes (teams):
Water resources study and design
Project number 2529 32
Water schemes construction supervision
Drinking water schemes equipments maintenance and administration
2. WASH sector Financing
Both Governments annual budget allocation and donors support are annually increasing
No budget used to be allocated before for sanitation and hygiene, this has much improved.
There remains limited capacity both at Zonal and Woreda levels. Human resources management is
a critical factor.
Water schemes construction are also planned for institutions-schools and health posts/centers.
3. WASH service delivery
About 95% constructed their own latrines though it might not be according to the expected
standard; there is a move from open field, to traditional pit latrine, to improved latrine.
There is need for continuous training at woreda level WASH support unit so as to enhance WASH
delivery services.
The majority of woreda staff are graduates from TVETC- water supply and sanitation, electro
mechanical, and water quality departments.
Before GLOWS intervention, the graduates were theory dominated and the majority of teachers
also lack practical aspects. With GLOWS and RiPPLE this has much improved, the regional water
bureau actively participated in design, training and joint assessment of the impact of the GLOWS
and reported to SNV. There were graduates who fear to go to rural water schemes, but after the
GLOWS intervention, the graduates were more motivated.
4. Community and private sector participation
Community participation: An important change has been made to encourage active
participation of the community. However, sometimes this does not work, possible reasons are:
No proper community mobilization and consultation is done,
Low level of expertise on community mobilization.
No proper local ownership is created.
In schemes where capacity building was done, O&M can be handled by the community but not for
major maintenance.
In schemes where construction was done without ownership development, communities claim
support even for simple issues.
Private sector participation: Private sector participation has much increased. Currently there
are groups who produce latrines from stone and supply slabs, and trained local artisans, but there
remains the need to organize them. There is work on spare parts supply; however it was found that
spare-parts sale only does not generate much benefit. Alternatives are being developed, such as:
cooperative supply of spare parts by allocating one shelf in their sales shop;
developing a town water service;
established outlet shop with some seed money supply;
working with traders for fast moving goods.
Recently the initiative has been taken to work on Guidelines for Private sector involvement in
Maintenance and spare parts supply.
5. Gender aspects:
Now it is recognized that women play a great role in water handling and utilization.
There is a new regional regulation for women to be nominated for leadership positions and at least
50% women in WASCOs and other WASH activities design and implementation.
The education sector gives attention to the school girls’ hygiene and sanitation.
There is a Gender Focal Unit under the Regional Water Bureau.
Project number 2529 33
6. Data base, M&E
On MIS: there was a gap from regional side in continuity. After the responsible person in the region
for MIS WASH left, no new person was assigned. The use of the software faced some challenges.
There is a gap regarding joint evaluation and lessons learned with SNV, except for GLOWS. It
would be good to have an opportunity for joint review on progress, challenges and sharing of
lessons to scale up the good practices of the SNV WASH interventions.
At regional level there was responsible staff for WASH inventory. But after the person left, no
responsible staff or unit was assigned to take over regional WASH inventory.
Recently, in collaboration with RiPPLE a regional WASH resource centre was established.
There are staff, financial and logistics constraints to conduct regular M&E. There is a need to build
more capacity at woreda levels.
7. Partnership
Main partners and active currently: UNICEF, World Bank, AfD bank, JICA, SNV and RiPPLE.
Trends in support: The donors support is increasing from year to year.
More support is required on the hardware parts for WASH in schools; even though for hygiene and
sanitation more software is needed, it is better if donors also give attention to the hardware.
In the region, currently there are more than 15 partners working on hygiene and sanitation, but still
there are 53 woredas without any donor. In the future if all donors come in one system of support
there is possibility to utilize more efficiently the available support.
3.4.3 Interviews at the Hawassa TVETC
From the interviews at the TVETC, the following main points can be extracted (the full interview
narratives are found in appendix 6.4).
Vocational training:
WASH training was introduced in 2010 to the water technology department through SNV.
About 21 graduates annually and in total 150 graduates benefited since the vocational training
project was introduced (for five years) for 7 woredas.
At initial stage it was as GDL one curriculum for testing. Later 7 curriculums were developed.
The curriculum is still in use, some additional improvement has been made:
Water treatment was added on the module
POM was also added to the existing WASH training curriculum, for one year but now discontinued.
Strengths of the SNV project:
SNV is seen as one of the strategic partners
SNV contributed to Teaching Training learning Materials preparation
Within a short period the water related curriculum was considerably improved
This contributed in making our college to reach higher level
This was a good input for improvement of the water department.
It gave direction for the water technology department
It helped to change the teaching from theory dominated to practical
Weaknesses:
At initial stage of implementation there was coordination problem but later it was solved in close
consultation with SNV and college administration.
Different types of materials including training inputs are required when implementing the
curriculum.
Project number 2529 34
At woreda level there is low level of awareness and attention.
In some woredas there is an absence of graduates in the water sector.
Distance learning (GLOWS): was introduced in 2011. It is estimated over three years about 70
graduates benefited from the GLOWS project. The trainees were nominated by the regional water
bureau not by the college. Graduate employment places: woreda water offices, irrigation offices,
different industries including sugar factories etc. Its strengths are:
The approach is an improvement as compared to GDL
It helped to know the graduates performance as a feedback
The practical orientation in teaching is of learning significance
It filled the skill gaps of TVETC graduate experts on jobs in water sector
It helped to motivate the experts on the jobs
It includes to trace the graduates and identify their gaps
It created linkage with the community to work with and set the direction of the college
However, the application of GLOWS is costly. Vehicles, DSA, professional fee is required for the team
field supervision, e.g. per annum 4 supervisions are supposed to be conducted. Skilled experts are
required for training and field supervision. Without donors support it is difficult to implement.
GLOWS is still being applied, but the application depends on the available budget. It can’t be applied
by itself. No information about the government plan to finance the project.
Another challenge the project faced was accreditation. The graduates want the certificate to get
recognition by their employers in promotion to higher level. This requires government commitment.
There has been discussion on this issue but it is still not resolved.
The finance is managed by the college and the college provides training as well as pays the college
staff for the training they provided for the trainees coming from different organizations. However,
the service user organizations should pay for the services provided. It is better if SNV gives the
trainees finance for the college water sector utilization rather than paying for trainees.
Improved graduates performance: GLOWS served the college to know its graduates situation in
the field. During implementing GLOWS the college teaching staff got an opportunity and was able to
observe the gaps and performance of its graduates. After WASH and GLOWS the graduates’
performance has highly improved both in technical capacity and motivation for more works in the
community.
Other TVETCs that adopted GLOWS are: Soda and Dila TVTECs from SNNPR, TVETCs in Oromia
and Amhara regions.
3.4.4 Interviews at UNICEF – Hawassah branch
From the interviews at UNICEF, the following main points can be extracted (the full interview
narratives are found in appendix 6.5).
Strategic partnership with SNV. SNV and UNICEF collaboration started some years back, with a
new agreement made in 2011. Advantages of the collaboration are:
Mutual complementarity; SNV introduce has an added value in WASH interventions.
SNV working on the software component has advantage with regard to sustainability of the
hardware.
Disadvantage: There is a joint relation to finance and capacity, a small number of woredas was
reached, there is need in the future to identify ways to replicate.
Project number 2529 35
Improvement in collaboration. A national level study shows that 25-40% water schemes are
non-functional. It has been shown that this can be improved if the water schemes are managed and
owned by the community. The collaboration with SNV has generated as results:
Schemes become stronger in financial management
POM training brought change in functionality of schemes.
Effective elements: Software and hardware components feed each other. Missing one of the
components doesn’t bring change. SNV working on software component is a great change.
WASH service delivery. There is some improvement in both the water coverage and utilization,
according to the national inventory that was done in 2012.
Up scaling effectiveness: There are a number of factors determining the potential for scaling up
government and partners’ capacity. A major problem is high staff turnover.
Community and private sector participation
Water: maintenance of water schemes is still minor in majority of the target woredas , but there is
an increase in contribution of finance which is an indicator of increasing ownership.
Sanitation: nowadays in most places a household latrine exists; it may need to be improved.
Schools: there is still a problem in terms of access to water supply hardware.
Private sector participation: The private sector plays an important role in WASH. It is possible to
work both software and hardware components, e.g. in small schemes construction, sanitation
facilities, spare-parts supply. Hardware parts can be supplied by local private sectors like artesian
and cooperatives in the community.
3.4.5 Conclusions from the regional level interviews
From the above interviews and focus group discussions, the following main conclusions emerge:
A change during the last years has been one from the focus at constructions towards behavioural
change on sanitation mainly. The woredas have more budget to spend on water schemes and
sanitation and hygiene. As a result the population is more aware, have latrines, and these are better
maintained. Most households now have latrines, these are not always the improved ones.
The integrated approach (education, health and sanitation) to WASH has approved. It has led to a
focus at water & sanitation within schools.
Access to water within schools remains a challenge and schools are encouraged to use alternative
sources, such as collecting from nearby springs, storage tanks and rainwater harvesting.
Service delivery by public agencies on WASH has much improved and the practical training
(GLOWS) has contributed to expertise and staff motivation. However, there are still complaints
about high levels of staff turn-over, staff motivation and lack of qualified hydrologists, geologists
and electro mechanics. These are external factyprs beyond the reach of donors such as SNV.
The collaboration with SNV has shown that operations and maintenance of water schemes can be
much improved (at national level 25-40% of water schemes are not functional). However,
maintenance of deep bore hole water systems is still a challenge.
The private sector plays an increasing role in the WASH sector. It constructs latrines and provides
spare parts for water schemes, but also does not address major maintenance issues.
The management information system (MIS) is not being used due to poor continuity by regional
level and problems with software.
At regional level it is felt that continuity and follow-up evaluation and learning by SNV is missing.
More (donor) support is required on the hardware aspects of WASH, especially for schools.
UNICEF mentions that too few woredas were reached in terms of hardware.
Project number 2529 36
4. Analysis and conclusions
4.1 Summary of main findings
4.1.1 Comparison of woredas
We now present an overview of the main findings per woreda, with the aim to recognize relations and
patterns. Table 33 shows subsequently:
The current survey (2015) findings on access to water per woreda and the most likely trend
emerging from the comparison with the 2008 baseline values (see for uncertainties with respect to
the baseline survey values previous chapter table 3);
The current survey (2015) findings on access to improved sanitation per woreda and the most trend
emerging from the comparison with the 2008 baseline values (see for uncertainties with respect to
the baseline values previous chapter table 6);
The functionality of water schemes, using an index score
The functionality of WASH Co performance, using an index score.
Table 33: Data and trends on main findings at household level and with respect to water schemes.
Woreda
Access to safe water
Access to improved
sanitation
Functionality
water scheme
(index score)
Performance
WASH Co’s
(index score) % 2015* Trend 2008* % 2015 Trend 2008
Alaba 20.4 Down 32.1 up 1.00 1.00
Boloso Sore 52.3 up 39.5 up 1.37 0.91
Demboya 17.3 Down 30.0 stable 1.00 0.70
Kedida 69.5 Up 56.7 up 1.01 0.86
Misrak B. 70.3 Down 44.0 up 1.01 1.00
Shashego 54.7 Down 48.1 up 1.10 0.90
All 46.4 down 41.4 up 1.10 0.88
* Values in 2015 take into account three criteria including distance to the nearest water scheme and
clean water storage, most likely similar to the 2008 baseline survey
Following are conclusions and explanations when comparing the different values per woreda:
Access to water has declined in most woredas, except in Boloso Sore and Kedida,
from an average of 53% in 2008 to 46.4% in 2015. We assume that the baseline survey also
took into account distance (not more than 1.5 km) or travel time to the water scheme (not more
than 30 minutes). When doing so in this survey, average access to water is 46.4%, which is the most
correct and realistic value. Access to water has improved in Boloso Sore and in Kedida. Access to
water when taking into account distance to water is especially low in Alaba and Demboya. Due to
great distance to water, currently travel time was more than 30 minutes for 40.7% of the
respondents, significantly affecting access to water.
According to the Ethiopia Demographic and Health survey of 2011, 13% of households reported
having water on their premises, 36% in rural areas have water within 30 minutes walking, and 62%
in rural areas have to spend more than 30 minutes walking. This means that, when looking at
distance to nearest water scheme, the data in the survey area of our study are somewhat better.
Access to improved sanitation has improved in most woredas, from an average of
27% to 41.4%. In our survey 72.4% of the respondents state that they have access to improved
Project number 2529 37
sanitation, but when observing the latrine facility it turns out that for only 49.4% of the
respondents the latrine is of an improved type, and when taking into account cleanliness the value
comes to 41.4%. When comparing this with the 2008 baseline on improved sanitation, there is an
improvement in 5 of the 6 woredas and only in Demboya there is no change.
There is a correlation between access to safe water and access to improved
sanitation. Figure 1 shows the significant relation between access to safe water and access to
improved sanitation. This relation is in conformity with the results of focus group discussions
which indicate that access to water influences whether latrine facilities can be properly used.
Figure 1: relation between access to safe water and access to improved sanitation in 6 woredas.
There is a relation between access to water (and access to improved sanitation) and functionality of
water schemes. Functionality of water schemes shows variation between woredas, with Boloso Sore
scoring highest and Alaba and Demboya scoring lowest (table 25). The performance of WASH Co’s
also shows variation, with Alaba and Misrak Badewacho scoring highest, and Demboya scoring
lowest. We observe low scores for all indicators at Demboya. Alaba also has low scores on all
indicators except for WASH Co performance. This suggests that poor functionality of water
schemes leads to low scores on access to water (and thus also to access to improved sanitation).
4.1.2 School level findings
Table 34 gives an overview of the findings on access to water and access to improved sanitation for
schools, per woreda, with trends on the comparison between the 2008 baseline and current survey.
Table 34: Data and trends on main findings in schools.
Woreda
Access to safe water
(2015)
Access to improved
sanitation (2015)
(%) Trend (%) Trend*
Alaba 7.7 up 23.1 Up / down
Boloso Sore 12.5 up 25.0 Up / down
Demboya 33.3 up 16.7 Up / down
Kedida Gamella 40.0 up 20.0 Up / down
Misrak B. 25.0 up 37.5 Up / down
Shashego 30.0 up 30.0 Up / down
All 22.0 up 26.0 Up / down
y = 0,3785x + 23,786
0
10
20
30
40
50
60
0 20 40 60 80
Access to safe and clean sanitation
Access to water
Project number 2529 38
* Before and after the slash: without / with criteria of maintenance, regular use and security,
respectively
Following are conclusions and explanations when comparing the different values per woreda:
Access to water has considerably improved for schools in all woredas, although the average is still
rather low (increase from 7% to 22.0%). Access to water implies that there is a water tap or source
within the school compound. Several schools also have access to water outside the school
compound (30%). This means that in total 48% of the schools surveyed do not have access to water
within or outside the school.
Almost all schools have a latrine facility (94%) but the proportion of latrine facilities that is also
clean is much lower (26.0%). When we compare data on safe latrine facility with the baseline value,
we can observe a considerable improvement (positive trend). This may be explained by the fact that
almost all schools nowadays have improved latrine facilities. However, when we look at the
indicator on improved sanitation (taking into account cleanliness and usage) the situation is less
positive and comparing the baseline value (49%) with current value (26%) shows a decline. This
may be a consequence of the fact that many schools do not have access to safe water within the
school compound so that maintenance of the sanitation facilities may be difficult, but it may also be
due to lack of supervision by school teachers. We suspect that the 2008 baseline data did not
adequately take into account cleanliness of the latrines, so that the decline may be less than
suggested.
It is uncertain whether there is a relation between access to water and access to
improved sanitation within schools. The survey data do not show a significant relation
between access to water and improved sanitation within schools. This finding seems to be in
contrast with the results from focus group discussions, which suggest that many schools have
latrine facilities but cannot use them properly because they do not have access to water. However,
the absence of this relation may be due to the fact that schools that do not have access to water
within the school premises, are able to acquire access to water from outside the school or acquire
water through trucks and storage in water tanks. Also, the numbers are small so that a significant
relation will be difficult to acquire.
Project number 2529 39
4.2 Analysis of impact pathways
In the inception phase, impact pathways were identified, with indicators and assumptions to be
validated. We now analyse to what extent these impact pathways can be validated by our quantitative
data and qualitative insights from focus group discussions.
4.2.1 Impact pathway 1. Household access to improved sanitation
Input Output Outcomes Impact
Ob
jectives / iIn
dica
tors
1. Households trained on CLTS, by woreda staff
2. HH received support and information on water & sanitation 3. Awareness raised among HH water users on improved sanitation and safe hygiene practices
4. HH construct sanitation facilities 5. HH maintain sanitation facilities and use these in a clean and safe way 6. HH access to improved sanitation
7. Reduced incidence of diarrhea and other water-borne diseases
Assu
mp
tion
s
a. Woreda staff capacitated on sanitation by the SNV project
b. HH are accessible and open to receive information and support in WASH
c. Access to water for households d. Households have means to construct sanitation facilities
Figure 2: Results on impacts pathway for household access to improved sanitation
Changes between baseline (2008) and current (2015) [numbers and letters refer to Figure 2]
1. Woreda water and sanitation (WASH) and health officers raised awareness on the need for
WASH using the Community Led Total Sanitation (CLTS) approach. All households have been
reached in 6 woredas.
2. Over the last 6 years 74.1% received both information and support on WASH, 9% received either
of the two and 8% none at all.
3. Currently, 72.4% are aware of the need of improved sanitation.
4. Currently, 49.4% have improved latrine facilities.
5. Currently, 44.2% have improved latrine facilities and also keep them clean.
6. Currently, 41.4% have improved latrine facilities, keep them clean and use them regularly so that
their compound and surroundings are clean (‘open defecation free’). This is a considerable
improvement as compared to the baseline in 2008 (27%).
7. Currently, the incidence of diarrhea over the last year is 17%. This is a considerable
improvement from 52% in 2007.
Assumptions
a. Woreda staff was capacitated by SNV, as well as other NGOs (but SNV was most important)
b. All households can be reached by woreda extension staff.
c. Of surveyed households, 46.4% have access to safe water. This is most likely a decline as
compared to the baseline in 2008 (53%), which is mainly the result of long distances to, and
waiting times at water schemes.
d. Construction of improved latrines is possible even for low-income households.
We conclude that the situation has improved in many respects. But what can we conclude about the
contribution by the project and the causality of these changes?
A+
C+
B+ D+
Project number 2529 40
Causality and contribution analysis [letters refer to items in Figure 2]
A. There is a positive relation between woreda staff capacitated to provide CLTS extension (a) and
their performance in providing services on WASH to households (1). This was found when
comparing the results of capacity building among woredas targeted by the project with the
control / non-targeted woredas. The comparison shows that the targeted woredas score
consistently better on WASH related capacities, as a result of better (on-the-job) training mainly.
B. There is a significant positive relation between access to improved sanitation (2) and information
and training support received on WASH (6). This causality is supported by the results of FGDs,
supporting the usefulness of a shift to awareness raising and behavioural changes.
C. There is a significant positive relation between access to improved sanitation (6) and access to
safe water (c). This causality is supported by the results of FGDs, and will result mainly from the
fact that water schemes have been better maintained.
D. There is a significant positive relation between access to improved sanitation (6) and incidence of
diarrhea (7). This causality is supported by the results of FGDs. The relation cannot be fully
understood by the actual presence of latrines but also by the improved behavior of users.
The causality B is further supported by the analysis of underlying factors, showing that there is a
significant positive relation between access to improved sanitation and young age of household head
(younger age scoring better). This suggests that young family heads more easily change their
behavior. There is also a positive relation with family size, suggesting that larger families change their
behavior more easily. There are no significant relations between access to water and household
characteristics, suggesting that the main causal factors are beyond the household.
We conclude there are significant correlations and plausible underlying causal explanations (based
on interviews and/or focus group discussions) for linkages between elements of this impact chain.
This implies that this impact pathway is validated: the contribution by the SNV project is plausible.
.
4.2.2 Impact pathway 2. Schools access to water and improved sanitation
Input Output Outcomes Impact T
arg
ets / ind
icato
rs
1. School teachers and pupils trained on CLTS 2.WASH clubs are formed and supported
3. Among school teachers and pupils increased awareness on clean and safe water use and sanitation facilities 4. School management capacity to support WASH in schools 5. WASH clubs are functional
6. Schools construct latrine facilities 7. Schools assure sanitation facilities are managed in a clean and safe way
8. Improved school attendance
Assu
mp
tion
s a. Woreda staff capacitated on sanitation training schools
b. Schools can be reached to raise awareness on WASH
c. Access to water at school compounds
Figure 3: Results on impact pathway for schools access to water and improved sanitation
Changes between baseline (2008) and current (2015) (numbers and letters refer to Figure 3]
A+
D?
B?
C?
Project number 2529 41
1. Woreda water and sanitation (WASH) and health officers raised awareness among school
students and teachers on the need for WASH. Many schools have been reached in 6 woredas.
2. In addition, at all schools WASH clubs have been formed.
3. Over the last 6 years 74.1% received both information and training support on WASH, 9%
received either of the two and 8% none at all.
4. Of the surveyed schools, 63% have fairly good and 26% has very good WASH management
capacity among the teachers.
5. Currently, 72% of schools have WASH clubs that are functional.
6. Currently, 94% of the schools have a latrine facility, which is higher than the baseline value of
49%. This can be explained by the fact that many latrine facilities have been constructed
following integration of the education sector in WASH.
7. Currently, 28% of the schools have latrine facilities that are also clean and safe, which is lower
than the baseline value of 49%. This can be explained by the fact that very few schools have
access to water within their compounds.
8. Currently, in 94% of the schools, attendance has improved over the last 6 years.
Assumptions
a. Woreda staff was capacitated by SNV, as well as other NGOs (but SNV was most important)
b. All schools can be reached by woreda extension staff.
c. Access to water within the school compounds is 22%. This is a considerable improvement
compared to the baseline situation (7%). Several schools also have access to water outside the
school compound (30%). Woreda’s claim higher levels of access to water by schools than 22%,
which is probably due to the fact that external sources of water (beyond the school) are
included.
We conclude that the situation has improved in many respects. But what can we conclude about the
contribution by the project and the causality of these changes?
Causality and contribution analysis [letters refer to items in Figure 3]
A. There is a positive relation between woreda staff capacitated to provide WASH extension (a) and
the capacities at schools on WASH (1). This was found when comparing the results of capacity
building among woredas targeted by the project with the control / non-targeted woredas. The
comparison shows that the targeted woredas score consistently better on WASH related
capacities, and was supported by the results of interviews at regional level.
B. There is not a significant relation between WASH management capacity in schools (4) and access
to water or access to improved sanitation within schools (7). This may be explained by results
from the FGDs at regional level, revealing that access to water within schools remains the
dominant problem, even if schools use alternatives to access water.
C. There is not a significant relation between the functionality of the WASH clubs (5) and access to
water or access to improved sanitation within schools (7). The explanation is similar as above.
D. There is not a significant relation between school attendance (8) and access to water or access to
improved sanitation within schools (7). However, the results of FGDs provide plausible evidence
that there is such a relation in several woredas.
We conclude there are no significant relations and underlying causal explanations (based on
interviews and/or focus group discussions) for linkages between elements of the impact chain. This
implies that this impact pathway is not validated. However, a contribution by the SNV project is
plausible, given the available evidence from interviews and focus group discussions.
Project number 2529 42
4.2.3 Impact pathway 3. Improved management of water schemes
Input Output Outcomes Impact
Ta
rgets / in
dica
tors
1.Local WASH committees are supported and trained
2. Awareness raised among water users and WASH co on operation and maintenance 3. WASH Co’s are performing well and water users pay fees
4. Water schemes are well maintained. 5. Water schemes remain functional
6. Improved access to water for water users
Assu
mp
tion
s
a. Woreda staff capacitated on O&M b. Water schemes are constructed
c. WASH Co’s receive support by woreda officers d. Water users have capacity to pay
e. Water scheme can be maintained in time f. Water reserves, rainwater and climate change
Figure 4: Results on impact pathway for improved management of water schemes
Changes between baseline (2008) and current (2015) (numbers and letters refer to Figure 4]:
1. SNV has directly supported the training of WASH committees, and woredas have trained more.
All water schemes that were surveyed have WASH committees.
2. The survey showed that WASH committees are well aware of the need for operations and
maintenance.
3. By far most WASH committees (88%) are performing well, as based on our set criteria. This
means the members are satisfied with the WASH committee, fees are charged, fees are used for
O&M, members are satisfied with the fees, and if there are conflicts, these are resolved.
4. Of the WASH Co’s, 72% stated that operations and maintenance of the water schemes had
improved over the last years.
5. Based on the classification of the water schemes to assess functionality, it appears that few water
schemes (24%) are fully functional, most (62%) are partly functional. The most discriminating
factor for being partly functional is waiting time. Of all water schemes, 63% have a waiting time
of > 30 minutes which is considered as ‘non- functional’, of which 18% have a waiting time of
more than 5 hours. This means that even if the water scheme provides water, and the water is of
good quality, the volume of water discharge is often very low. This finding is confirmed by results
of the FGDs, especially those at regional level. Note that several woreda’s report that water
schemes are functional, but then observe that water discharge is very low – we have integrated
this finding in the functionality scores.
6. Of surveyed households, 46.4% have access to safe water, and thus have water available to use
the latrines. This is most likely a decline as compared to the baseline in 2008 (53%), which is
mainly the result of long distances to, and waiting times at water schemes.
Assumptions
a. Woreda staff was capacitated by SNV, as well as other NGOs (but SNV was most important)
b. Few new water schemes have been constructed (information from woredas and UNICEF)
c. Woredas provide services to WASH Co’s, but expertise is missing on hydrology, geology and
electro mechanics
d. Users have little capacity to pay for water use
e. The maintenance of water schemes has much improved during the last time, the time it takes
before a reported failure is repaired has considerably reduced (results of FGDs)
f. We do not have information on water reserves and rainfall data, but given the low water
discharge of many water schemes it appears that reserves are low.
A+
B? C? D?
Project number 2529 43
g. All schools can be reached by woreda extension staff.
We conclude that the situation has improved in many respects, but many water schemes remain
partly functional because of the long waiting times due to poor relation between water discharge and
water demand. Poor water discharge may be due to limited water reserves and recharge from rainfall.
The number of water schemes remains too low. But what can we conclude about the contribution by
the project and the causality of these changes?
Causality and contribution analysis [letters refer to items in Figure 4]
A. There is a positive relation between woreda staff capacitated to provide WASH extension (a) and
local WASH committees (1).
B. We did not find a relation between performance of the WASH Co’s (3) and whether POM training
had been provided by the SNV project (1), or with the ‘age’ of the water scheme.
C. We did not find a relation between perceived progress in maintenance (4) and performance of
the WASH committees (3).
D. We did find plausible evidence that higher functionality of water schemes (5) leads to more
access to water (6). The main determining factor is probably the waiting time at the water
scheme.
We conclude there are no significant relations for linkages between elements of the impact chain.
The interviews and results of FGDs show that this is mainly due to the following main factors. First,
many water schemes provide too low volumes of water in relation to water demand, which affects
their functionality. Second, too few new water schemes have been constructed. Third, there is the fact
that the lifespan of facilities is 15 -20 years and old facilities cannot serve current populations growth.
In absolute numbers the coverage increases but not in terms of % of demand being served.
Project number 2529 44
4.2.4 Impact pathway 4. Capacity building of institutions responsible for WASH
Input Output Outcomes Impact
Ta
rgets / in
dica
tors
1. Training of woreda and regional WASH officers 2. On-the-job training of WASH graduates
3. Awareness raised among Woreda and regional officers 4. Improved skills of WASH graduates
5. Improved WASH planning 6. Improved financial resources for WASH in woredas 7. Improved service delivery on WASH 8. Improved community participation
9. Improved access to water
Assu
mp
tion
s
a. Woreda staff motivated to improve WASH capacities b. Regional staff motivated to improve WASH capacities
c. Woreda and Region have capacity & budget for WASH d. Woreda and Region have staff available for WASH
Figure 5: Results on impact pathway for capacity building of institutions responsible for WASH
Changes between baseline (2008) and current (2015) (numbers and letters refer to above figure 5]:
1. SNV has directly supported the training of Woreda and regional level WASH officers.
2. SNV has supported Awassa TVETC and GLOWS on-the-job training of WASH graduates, this has
reportedly been much appreciated.
3. The results of FGDs show increased awareness at woreda and regional level on WASH related
aspects.
4. The results of interviews show evidence of improved skills of WASH graduates.
5. The results of FGDs show improved WASH planning skills within woredas targeted by the
project, as compared to control woredas. The SNV WASH project has contributed significantly.
The control woreda’s are aware of the risk of fragmented planning and also want to move to an
integrated approach.
6. The results of FGDs show that targeted woreda’s have been able to attract more government and
donor funding, as a result of better planning and support in fundraising. The SNV WASH project
has contributed significantly. However, funds are still insufficient especially for large scale
constructions and maintenance. Control woreda’s seem to remain more dependent on donor
funding.
7. The results of FGDs show that targeted woreda’s have been able to give more training on
awareness raising as well as POM aspects, reason why water schemes are stated to be well
maintained and more functional than in the past, and at household level communities better
maintain their latrines. The response rate to repairs has increased. The SNV WASH project has
contributed significantly. However, service delivery on maintenance of large water schemes and
large-scale mechanical issues remains problematic even in targeted woreda’s because of limited
staff, budget and logistics. In the control woredas less new water schemes seem to have been
constructed, and there are concerns about their maintenance and sustainability.
8. The results of FGDs show that in targeted woreda’s communities show sense of ownership and
participation. Also, there are initiatives of private sector, e.g. in spare part provision. The SNV
WASH project contributed significantly. In the control woredas there are more concerns about
community ownership and less private sector involvement.
Project number 2529 45
9. We do not have data on access to water in control woredas so this comparison cannot be made.
Assumptions
a. Woreda staff was found to be motivated to be involved in the SNV training, but there are also
indications that currently staff is not sufficiently motivated and turn-over is high. This may be
due to limited funds, means of transport, and low investments in new water schemes.
b. At regional level staff motivation was not evaluated. However, regional staff on MIS has left
which is the main reason for lack of continuity on monitoring using MIS.
c. The budget for WASH investments is reportedly too low.
d. There is shortage of staff on specific subjects related to WASH, being geologist, hydrologist and
electro-mechanics. These issues mainly relate to the construction of new water schemes and
maintenance of large water schemes.
We conclude that the situation has improved in many respects, but capacities remain low as related
to the construction of new water schemes and the maintenance of large water schemes. This may
largely explain why access to water remains too low.
We do not have additional information on causal relations in the above scheme.
We conclude there is plausible evidence for positive changes and relations in the above impact chain.
However, there are remaining challenges which are mainly related to the following factors:
Need for additional hardware, especially new water schemes, operations and maintenance of large
water schemes, especially to increase water discharge;
Need for improved motivation of staff, probably mainly due to available funding, skills and
mechanics;
Continuity in follow-up, joint learning and evaluation of progress by SNV.
4.3 Evaluation questions
We now answer the evaluation questions based on the above analyses. To do so, a summary and
selection has been taken from the set of evaluation questions (see Annex 2).
Impact
When taking into account distance to the water scheme (not more than 1.5 km) or travel time to the
nearest water scheme (30 minutes), as well as cleanliness of water storage, access to water for
households in the project area has declined from 53% in 2008 to 46.4% in 2015. Currently travel
time was more than 30 minutes for 40.7% of the respondents, which is still better than the average
of 62% for rural areas in Ethiopia, according to the Ethiopia Demographic and Health survey of
2011. Most likely, without the project access to water would have been lower because we found that
of the majority (72%) of water schemes in the project area functionality has improved.
The project has made a significant contribution to access to improved sanitation within
households, which has increased from 27% to 41.4%. We conclude there is a significant relation and
plausible contribution by the SNV project to the improvement in access to improved sanitation (see
impact pathway figure 2). For instance, there is a positive relation with access to information and
training support on WASH, and also with younger age of the household head. This shows that
behavioural change on sanitation and hygiene has taken place, especially among younger people.
There is also a significant correlation between access to safe water and access to improved
sanitation for households, suggesting that access to safe water influences improved sanitation.
Project number 2529 46
The decline in the incidence of diarrhoea is highly significant (from 52% to 17%), and is most
significantly related to the improved access to improved sanitation within households. It cannot be
excluded that other factors also play a role, but improved sanitation has certainly contributed to
less incidence of diarrhoea.
Concerning the schools, access to water in their own premises has significantly improved (from 7%
to 22%). The majority of schools do not have access to water in their own premises, but many have
access to water beyond the school (30% of the schools). Schools are encouraged to use alternative
sources of water, such as collecting from nearby springs, storage tanks and rainwater harvesting.
Within the schools 94% have a latrine facility, but the score of access to improved sanitation
declines to 28% if we also taken into account aspects of cleanliness and safety, which seems lower
than the baseline value of 49% (but it is uncertain to what extent these criteria were used during
the baseline survey). Low access to improved sanitation (including its use, cleanliness and safety)
could be explained by low access to water within the school compounds as well as limited
management by teachers of WASH within schools (only 26% scores very good). Most student-based
WASH clubs (72%) are functional.
We have not been able to conclude that there is a significant relation by the SNV project to access to
improved sanitation in schools (see impact pathway figure 3), but a contribution by the SNV
project is plausible, given the available evidence from interviews and focus group discussions.
For the majority (72%) of water schemes in the project functionality has improved. The SNV
project has contributed to this improvement, by training on operations & maintenance and
supporting local WASHCO’s. However, for 70.6% of the respondents waiting time at the water
scheme was more than 30 minutes. Based on the long waiting time at water schemes, we classify
the majority of schemes (62%) as partly functional. We have not been able to conclude that there is
a significant relation by the SNV project to improved functionality of water schemes (see impact
pathway figure 4), but a contribution by the SNV project is plausible, given the available evidence
from interviews and focus group discussions.
The interviews and results of FGDs show that low functionality of water schemes is mainly due to
the following main factors. First, many water schemes provide too low volumes of water in relation
to water demand, which affects their functionality. Second, too few new water schemes have been
constructed. Third, there is the fact that the lifespan of facilities is 15 -20 years and old facilities
cannot serve current populations growth. In absolute numbers the coverage increases but not in
terms of % of demand being served.
With respect to the contribution by SNV, this is positive in relation to change of behaviour on
sanitation and hygiene at household level, improved maintenance of water schemes, build up of
capacities at woreda level, among the communities, WASH committees and WASH clubs at
schools.
It seems that the main remaining problem is the fact that in the 6 woredas and in the region as a
whole, available water is too low in relation to the needs of the local populations. The number of
new water schemes constructed has been too limited in recent years, because funds and capacities
for constructing new water schemes are limited, and also the water reserves may be limited. Wells
are already very deep.
Effectiveness
Opinions of woreda staff are positive with respect to the services being provided by the project.
At woreda level, strategic planning, service delivery, community involvement and monitoring have
much improved. Gender involvement has improved due to implementation of the national policy
mainly. The targeted woreda’s have better capacities for WASH strategic planning and WASH
service delivery than control woreda’s. These are systemic changes that will remain after the
project. Other (non targeted) woreda’s claim that they would like to receive similar support as they
see the advantages of these changes.
Project number 2529 47
The model of training-on-the–job (GLOWS) has been very effective in terms of improving service
delivery. The response rate to repairs has increased. The GLOWS training and the SNV WASH
project contributed significantly.
Use of the MIS has failed due to lack of support and continuity at regional level. In a general sense,
monitoring remains a weak point. The recent regional WASH survey is of mediocre value and the
data could not be used for comparison purposes.
There are a number of external factors that influence staff performance within woredas and
regional WASH agencies, being high levels of staff turn-over, staff motivation and lack of qualified
hydrologists, geologists and electro mechanics. Although financial resources available for WASH
have increased, and are higher in targeted woreda’s than control woreda’s, there is still insufficient
funding available.
We conclude there is plausible evidence for positive changes and a contribution by the SNV project
to improved capacities on WASH in woreda’s (see impact pathway figure 5). However, there are
remaining challenges which are mainly related to the following factors:
Need for additional hardware, especially new water schemes, operations and maintenance of
large water schemes, especially to increase water discharge;
Need for improved motivation of staff, related to available funding, skills and mechanics;
Continuity in follow-up, joint learning and evaluation of progress by SNV.
Sustainability
From an institutional angle, several of the capacity building changes at woreda level are structural
(systemic) changes that will last. For instance, there is wide acknowledgment of the usefulness of
integrated strategic planning (education, health and WASH). Also, there is firmly established
insight that for improvement of sanitation and hygiene the change of behaviour is at least as
important as the hardware. Service delivery has improved and private sector initiatives are
emerging, which seem to be processes that will continue.
From a local financial and maintenance angle, most local water schemes are well maintained with
WASH committees (88%) performing well, as based on our set criteria: the members are satisfied
with the WASH committee, fees are charged, fees are used for O&M, members are satisfied with the
fees, and if there are conflicts, these are resolved. Also, the time it takes before a reported failure is
repaired has considerably reduced.
From a regional financial angle, the funding from national sources to the WASH sector has
improved for the 6 selected woreda’s. Also, there is some evidence that the targeted woreda’s are
better able to access funds from donors for investment objectives, presumably because of their
improved planning and funding requests. The control woreda’s have less donor and NGO support.
Nevertheless, funds for investments in new water schemes and large-scale maintenance of existing
water schemes remain insufficient. Lastly, although WASH training of graduates will most likely
remain, the GLOWS approach is reported to be costly and still not formally recognised, and
therefore uncertain to be continued. Altogether, there are insufficient recurrent financial inputs to
the WASH sector in the region, e.g. through fees or taxes.
From an ownership point of view, in the targeted woreda’s there is more community ownership
than in the control woreda’s (e.g. through the WASHCo’s).
From a private sector point of view, there is more private sector involvement than in the control
woreda’s, which will also contribute to sustainability as it shows that a business case may be there
(e.g. for spare parts of water schemes maintenance).
From an environmental angle, it is worrisome that water boreholes have to go deeper and yet
cannot supply sufficient water for the whole population in the region. This may lead to depletion of
underground water resources. More attention could be given to rainwater capture and/or
underground storage. There may also be need for collaboration and integration with natural
resources and watershed management for improving vegetation coverage around water sources so
as to improve water recharge.
Project number 2529 48
Lessons learnt and recommendations
The approach taken by SNV in the WASH sector is innovative by focusing on capacity building of
local public institutions, on awareness raising and change of practices, involving training institutes
and networking. This focus by SNV on the ‘software’ was part of a programme partnership with
UNICEF that focused on the ‘hardware’. Overall, this evaluation shows that the SNV approach has
generated significant results, but the ‘hardware’ component now seems to constitute the main
constraint.
The original project document did not reflect a clear theory of change that shows how the different
components of the project, together with expected partnerships with UNICEF and regional
institutes, were expected to improved WASH performance in the region. This theory of change was
reconstructed as part of this impacts evaluation.
Capacity building by the SNV project at woreda level has been effective, but exchange events,
evaluation and learning for continuous improvement has been felt missing.
A significant proportion of households need to travel long distances for accessing safe drinking
water coupled with waiting long for fetching water. These are indications for the changes on
attitude and behaviour of households for using safe water over times. It is important that all WASH
and other development actors in the region strengthen their concerted efforts to ensure that safe
water sources are adequately and close to the communities and households as per the national
standards. The reduction in travel and waiting times for accessing safe water may enable water
collectors for having more time for development or productive activities.
As still a significant proportion of households incorrectly perceives their available latrine facilities
as improved ones, there is a need to enhance the awareness and education campaigns through
CLTS for enabling that households and communities have the correct perception. To this effect, it is
recommended to continue the awareness raising and follow up by health extension workers as they
are entry points to reach the communities and households.
Although there are positive effects at school level on improving skills and knowledge on WASH,
there is still insufficient management of WASH aspects by school teachers and insufficient access to
water for schools. Therefore, the governmental and non-governmental WASH actors in the region
may need to strengthen their partnership and collaboration for improving access to safe water and
improved sanitation in schools.
There are remaining challenges with a potential role for SNV to play as follows:
Strengthen the WASH monitoring and evaluation system within SNV and with local partners,
including clear and undisputable ways of defining and monitoring access to water and access to
improved sanitation. Reliable surveys must be carried out once in a few years to assess whether
progress is being made. SNV could support these surveys and provide quality assurance.
Underlying a good monitoring system is the need to define an integrated theory of change or
strategy for the WASH sector, with included software and hardware components, and linkages to
the education and health sectors. The theory of change should include insight in root causes of the
currently remaining problems in the sector.
SNV could introduce methods and technologies to monitor the discharge of water schemes (e.g. by
sensors) and not only whether they are functional (i.e. providing water or not).
It is recommended to share the insights of this evaluation with the responsible WASH staff at
woreda and regional levels, as part of a learning and evaluation exercise, and to assure continuous
improvement. There is also need for continuity in follow-up or refresher training, to assure further
improvement and avoid fall-back.
SNV could take the lead in proactively engaging in advocacy and lobby with the governmental and
non-governmental WASH actors who engage in hardware development, to narrow the existing gap
on demand for safe water and its supply as per the national standards of accessing within 1.5 km
and waiting time of less than 30 minutes, by addressing defined root causes.
SNV could further strengthen private sector engagement in the WASH sector;
Project number 2529 49
SNV could consider stimulating young family heads who have adopted improved sanitation
facilities and sanitation and hygiene behaviour to convince or support other households.
More attention could be given to rainwater capture and/or creating underground water storage, as
well as collaboration and integration with natural resources and watershed management for
improving vegetation coverage around water sources so as to improve water recharge.
Page/of 51/1
Annex 1: Theory of change (reconstructed by the evaluation team)
Outcomes Outputs
Awareness raised at household and school levels, organization of WASH Committees
At Woreda / Kebele level: WASH plans are made Plans are implemented Monitoring of WASH
occurs
Impact
Improved access to
safe and clean water
Installed new water schemes
Maintenance of existing water
schemes
Upscaling of water schemes
Graduates with experience
find jobs in WASH sector
Improved WASH skills of graduates
from TVETCs
Enabling national policies
SNNPR have financial means
Assumptions
Capacities built of SNNPR water resources bureau
Hardware provided (e.g. piped water systems, sanitation facilities) (UNICEF?)
Capacities built of graduates from TVETC: guided learning of graduates
Capacities built at Woreda and Kebele level, strategic planning, MIS, CLTS
SNNPR support Woreda / Kebele to
develop WASH plans, aligned to UAP
Institutions supported at national
level – UAP with WASH priorities?
National level UAP integrates WASH
priorities
New water schemes installed,
protected and supported
Water schemes maintained
Improved management of water
schemes, use of safe water,
management of sanitation facilities
Reduced incidence of water
related diseases / diarrhoea
Donors finance hardware
Coordination between NGOs
Installed sanitation facilities,
households and schools
Maintenance of sanitation
facilities Upscaling of
sanitation facilities
Improved access to
safe & clean sanitation
Software provision:
Woreda and Kebele have
financial and human
resources available
Page/of 53/2
Annex 2: Evaluation Questions (extract from ToR)
The evaluation questions are largely based on the DAC/OECD criteria for evaluating development
assistance
Impact
Conduct a household survey and a facility survey in the project area. Compare the progress on key
project impact indicators – compare the baseline conducted in 2007 to the survey conducted in
2011, and compare them to the survey to be conducted in 2014. Comment on factors that
influenced the achievement of impacts. In particular, assess the impact on the following key
indicators:
Proportion of households with access to water supply within 1.5km distance
Proportion of households with access to safe latrine
Number of schools with access to improved water supply
Number of schools with access to safe latrine in schools
Presence and status of water and sanitation facilities in the household
Hand washing practice and Hygiene behaviour
Incidence of diarrhoea
Assess attribution of impacts to the efforts made under this project. What other factors apart from
the project’s interventions influenced the achievement of these impacts?
Assess the change in hygiene behavior among the community compared to the past.
Effectiveness
Assess the model of capacity building that the project followed:
Assess the long term effects of capacity building on improvements in functional drinking water
and sanitation facilities.
Assess the long term effects of capacity building on key operational aspects of the woredas such
as planning, provision and maintenance of water and sanitation facilities. Are the practices in the
project area different than in non-project areas? Are the needs assessments strategies and the
planning tools that were introduced by the project still operational? Are the partners still using
the key approaches that the project introduced?
Assess the capacity building model that this project used and comment on effectiveness of the
strategies employed in this project.
What is the view of the local functionaries of the support provided by the project? Did they find it
useful and if yes, how so? What are the main reasons for the continuance or discontinuance of
the practices that project introduced?
Assess whether the changes introduced by the project were institutionalized (systemic changes)
in the working of the local government? If so how? If not, why not?
What is the view of the local functionaries of the support provided by the project? Did they find it
useful and if yes, how so? What are the main reasons for the continuance or discontinuance of the
practices that project introduced?
Assess the prevalence of non-functional water schemes, and the number of water schemes not
recovering O&M costs. Comment on the reasons for non-operation.
Assess if the vocational colleges are still using the curriculum changes that SNV helped bring? Are
those changes institutionalized including the distance education for the technicians in the field?
How did the setting up of MIS system helped with the achievement of the project objectives? Is the
MIS system implemented by the project still operational? And is it being used for planning and
monitoring purposes?
Project number 2529 54
Sustainability
Assess the lasting impact of the project as evidenced by the continuation of project benefits brought
about by the project, continuation of productive WASH strategies, and institutionalization of
changes/innovations/strategies introduced by the project.
Lessons learnt
What lessons can be drawn regarding sustainability based on project elements that were still
functioning and those not functioning anymore?
What elements of SNVs approaches were more (or less) effective in contributing to envisaged
impacts?
What could have worked better?