Embed Size (px)
Citation preview
1
Conducting disability inclusive baseline assessments for community-level WASH Projects
Clare Hanley and Hanna Goorden, CBM Australia in partnership with World Vision and
WaterAid
May 2016
2
Contents
List of Acronyms ...................................................................................................................................... 2
Executive Summary ................................................................................................................................. 3
Introduction ............................................................................................................................................ 6
Methodology ........................................................................................................................................... 7
Partnership between CBM, WaterAid and World Vision ........................................................................ 8
The WASH projects ................................................................................................................................. 8
Baseline data collection in WASH projects ............................................................................................. 9
Key lessons on disability inclusive data collection ................................................................................ 12
Discussion.............................................................................................................................................. 23
List of Acronyms CS WASH Fund Civil Society WASH Fund
DPO Disabled People’s Organisation
DFAT Department of Foreign Affairs and Trade
GESI Gender Equity and Social Inclusion
M&E Monitoring and Evaluation
NGO Non-Government Organisation
PNG Papua New Guinea
SDGs Sustainable Development Goals
WASH Water, Sanitation and Hygiene
WGSS Washington Group Short Set of Questions on Disability
Further information
This report has been produced in partnership with WaterAid and World Vision. For further
information or to provide feedback please contact:
Cover Photo credit: WaterAid staff conducting an interview with a person with disability as part of
the disability situational assessment in PNG. WaterAid/CBM 2015
CBM Australia – Clare Hanley Technical Advisor – Disability Inclusion Email: [email protected] WaterAid Australia - Chelsea Huggett Equity, Inclusion and Rights Advisor Email: [email protected]
CBM Australia – Hanna Goorden Quality Officer: Monitoring & Evaluation Email: [email protected] World Vision Australia - Andrew Jalanski Civil Society WASH Program Manager Email: [email protected]
3
Executive Summary
This report documents the lessons learnt and outcomes of collecting disability data in five water,
sanitation and hygiene (WASH) projects, with the aim of learning and therefore strengthening
disability inclusive data collection processes in future projects. With the adoption of the Sustainable
Development Goals, which aim for universal access by 2030, awareness of the need for WASH
programs to reach and benefit everyone, everywhere, is growing. Many programs are now
proactively seeking to reach people with disabilities and collecting disability inclusive data is critical
in setting the foundations for disability inclusive WASH. Global frameworks such as the SDGs also call
for disaggregated data by disability to monitor progress towards the Global Goals and to ensure
programs are leaving no one behind. Since 2014, CBM has partnered with World Vision and with
WaterAid to support disability inclusion within five Civil Society WASH Fund Projects (two projects in
Papua New Guinea (WaterAid & World Vision) and one each in Timor-Leste (WaterAid), Sri Lanka
(World Vision) and Zimbabwe (World Vision).
Key Lessons on Disability Inclusive Data Collection in WASH
1. Including people with disabilities in data collection teams resulted in positive outcomes for people
with disabilities, their communities & other stakeholders, but needs to be resourced appropriately
The WASH projects all included people with disabilities in data collection teams (rather than merely
as subjects) to some degree, generally by establishing partnerships with Disabled People’s
Organisations (DPOs). This was found to increase the confidence of people with disabilities, and
challenge negative attitudes within communities, government and implementing partners. It also
increased the quality of the data collected by encouraging people with disabilities to speak during
consultations. It did however require investment of additional time and resources to broker
relationships and build trust, strengthen the capacities and confidence of data collectors and plan
the logistics. In Zimbabwe for instance, a research team of people with diverse impairments had to
be transported to project sites over rough terrain which proved challenging.
2. Collecting data on people with disabilities in communities needs to be done sensitively
The term ‘disability’ can be understood differently across cultures and communities and in some
communities it is not widely used or understood. In the WaterAid projects in PNG and Timor-Leste,
project staff found it more useful to talk about difficulties with aspects of functioning (as
recommended by the Washington Group) as opposed to talking about ‘disability’. Increasing the
visibility of people with disabilities by including them in community consultations for the first time
also needs to be closely monitored to ensure that any potential backlash for challenging social
norms is addressed.
3. Household surveys need to collect individual level data about access to WASH in order for that
data to be disaggregated by disability. However, inclusion of the Washington Group Short Set of
(WGSS) questions is also useful for identifying households which include people with disabilities
who can then be followed up throughout the project.
The SDGs and some donors require monitoring data to be disaggregated by disability, sex and age to
assist in determining who does and does not benefit from development programs. This was one of
4
the common objectives of including the WGSS questions in the baseline surveys of the WASH
projects. However, the fact that surveys collected information at a household level only (generally
from the head of household) rather than individual level, meant that individual differences in access
to WASH within households were not identified, and some data could not be fully disaggregated.
Finding people with disabilities within communities is often a key challenge reported by NGOs
undertaking development projects. It is therefore noteworthy that while in most cases including the
WGSS questions did not enable disaggregation of data because of the structure of the survey, they
were generally effective in identifying households which included people with disabilities. People
with disabilities identified could then be invited to participate in qualitative data collection processes
or targeted for inclusion throughout the project.
4. There are challenges with but also unexpected findings from using the WGSS questions in surveys
While the WGSS questions are often promoted as being easy to use and proved useful, all NGOs and
their local partners reported some challenges in using the questions for the first time as they were
intended and with analysing the results. Pressure to complete the baseline quickly and with limited
resources meant that in some cases insufficient time was spent explaining to NGOs and data
collectors why the WGSS questions were recommended and training them on their use. As a result in
some cases the questions were not translated appropriately or were changed without sufficient
testing. Deciding on an appropriate ‘cut-off’1 to use to classify people ‘with’ and ‘without’ disabilities
and using this consistently was also challenging. For WASH projects, it was also found that there
may be advantages in using a more sensitive ‘cut-off’ (e.g. also including people who answer ‘some
difficulty’ to at least one question). This is because people with minor difficulties performing tasks in
fact reported significant difficulties accessing WASH and were often still expected to undertake
WASH work for the household.
5. A combination of quantitative and qualitative data collection processes produces the most useful
baseline data on people with disabilities
While the quantitative data helped to identify people with disabilities that faced WASH access issues
(who), collection of qualitative data on the situation of people with disabilities was critical to enable
an understanding of the type of difficulties people with disabilities were experiencing in accessing
WASH (why) and developing strategies to address these (how). Projects which completed qualitative
data collection specifically with people with disabilities ended up with richer baseline information
about barriers to WASH, than those who only included people with disabilities in general WASH
community planning processes. Standard community consultation tools, such as pocket voting and
community mapping could be used for disability specific consultations with only minor adaptations.
6. Collecting baseline data on people with disabilities led to more inclusive WASH programming and
had a range of positive outcomes
Collecting disability inclusive baseline data prompted a range of changes to project planning and
inclusive programming. This have resulted in construction of accessible WASH infrastructure,
1 Each question includes four response categories including ‘no-difficulty’, ‘some difficulty’, ‘a lot of difficulty’ and ‘cannot do at all’. The recommended ‘cut-off’ for determining disability status in a census is answering “a lot of difficulty’ or ‘cannot do at all’ to at least one question.
5
successful use of the baseline data for advocacy and training purposes with stakeholders including
government, and involvement of people with disabilities in local government decision-making
processes. Baseline data collection was able to inform projects most effectively when it was
completed early in project implementation, the implementing team had ownership of the findings,
data was analysed effectively, and collection and analysis of data was recognised as a tool to inform
project implementation rather than as a stand-alone activity for monitoring and evaluation.
Discussion
The above learnings demonstrate that in many ways, inclusive baseline data collection processes lay
the foundations for inclusive practice. They can empower people with disabilities, establish
relationships between DPOs and other program partners, raise awareness of access challenges,
challenge negative attitudes and identify people with disabilities for direct follow up and inclusion.
There are however a few major areas where further analysis is recommended. The first relates to
the current design of household surveys used by WASH projects to gather baseline information,
which was found to be inadequate to allow full disaggregation of data by disability and therefore
understanding the extent to which people with disabilities differed in their access to WASH in
comparison to people without disabilities. Further analysis is required to determine the best tools
for exploring differences in access to WASH within the household within quantitative surveys.
Secondly, given the time and budget constraints which come with any community development
project, program implementers need to carefully consider the balance of qualitative and
quantitative data that should be collected, based on an analysis of what information is required to
inform disability inclusion within that context. Quantitative data can be used to powerfully highlight
inequalities in access and measure progress of some indicators towards inclusion. However, there is
a danger that analysis stops at identifying people with disabilities and that not enough attention is
given to identifying and understanding the situation of people with disabilities more broadly, and
developing strategies and solutions to address this, which can impact on the effectiveness of
planned WASH interventions.
6
Introduction
It is estimated that 15 per cent of the world’s population have a disability, with prevalence higher in
low income countries.2 Although all people have the right to access water, sanitation and hygiene
(WASH), people with disabilities often face additional barriers to accessing WASH, compared to
people without disabilities, and have often been unintentionally excluded from WASH projects. To
achieve universal and sustained access to WASH, programs must overcome these barriers. Over the
last few years and with the focus of the Sustainable Development Goals (SDGs) on universal access
to WASH, awareness of the need for WASH programs to reach and benefit everyone has been
growing. Many programs are now proactively seeking to reach people with disabilities.
Collecting baseline data in WASH programs can provide critical information about the demographics
of a community, existing WASH facilities and practices, and a baseline from which to measure
change. Ensuring this data collection process is inclusive of people with disabilities is critical in
setting the foundations for a disability inclusive WASH program. It provides a systematic way to
identify people with disabilities in communities (a precursor for inclusion), to identify potential
difficulties experienced in accessing WASH services and strategies to address these, and to enable
evaluation over time as to whether inclusion strategies have been successful.
The focus of the SDGs on the need for poverty reduction initiatives to reach all people also has
implications for data collection. It increases the need for greater disaggregation of data (e.g. by sex,
age and disability) to measure intra-community and intra-household level inequalities. It has also
resulted in increased pressure on WASH actors to capture information that helps monitor
elimination of inequalities in access to WASH. Collecting data on people with disabilities presents
some unique challenges. Unlike determining age or sex, identifying disability is complex. Due to
stigma and different cultural understandings of the term ‘disability’, directly asking ‘do you have a
disability?’ has been shown to result in very low and likely underreported rates of identification.
CBM Australia is an international Christian development organisation committed to improving the
quality of life of people with disabilities in the poorest countries of the world. CBM has built
considerable expertise in supporting mainstream development organisations to strengthen
approaches to disability inclusive programming. Since 2014, CBM has been partnering with both
World Vision and WaterAid to support disability inclusion within their five Civil Society WASH Fund3
Projects in partnership with local Disabled People’s Organisations (DPOs) (two projects in Papua New
Guinea and one each in Timor-Leste, Sri Lanka and Zimbabwe). While the support provided has
focused on disability inclusion across the project as a whole, a key initial component of this has been
supporting the baseline data collection process to ensure it is disability inclusive. This has involved
providing guidance on data collection methodologies, developing survey tools and designing and co-
facilitating disability-specific situational analyses where these are planned, as well as raising
awareness, building capacities of and facilitating connections between partners, people with
2 WHO and World Bank (2011) World Report on Disability, http://www.who.int/disabilities/world_report/2011/en/. 3 The Civil Society WASH Fund ($103m over five years (2013-18)) is an initiative of the Australian Aid Program and aims to increase access to safe water, sanitation and hygiene in developing countries. It funds 13 Civil Society Organisations involved in implementing 29 projects.
7
disabilities, DPOs and data collectors (with and without disability) in disability inclusive data
collection and analysis.
To learn from these experiences, and particularly given a focus on disability data in these settings is
relatively new, CBM has led the process of documenting the experiences of collecting disability data
in these five project settings. This report shares the main findings, with the aim of learning from the
approaches and therefore strengthening disability inclusive data collection processes in future
programming.
Methodology
Information for this analysis was gathered from a range of sources. Firstly, a workshop was held
during a Civil Society WASH Fund learning event in July 2015 to reflect on the disability inclusive
baseline data collection approaches. This was facilitated by CBM with WaterAid, Plan, World Vision
and the Australian WASH consultant involved in some of the baseline processes and enabled
organisations to share their experiences and lessons learnt. After this event, questions to further
reflect on these experiences were defined, and semi-structured interviews were completed for each
project with CBM’s Technical Advisors, the Gender and Inclusion Officer at WaterAid Australia and
the Civil Society WASH Program Manager of World Vision Australia. Input was also sought from key
staff members and partners of WaterAid and World Vision in-country who were involved in the data
collection processes in the four countries, such as M&E, Gender and Social Inclusion Officers within
the projects, and DPO members. Finally some desk-based analysis was completed, which involved
reviewing key documents, data collection tools and guidance notes as well as the baseline finding
reports. Based on the individual interview notes for each project, key themes/lessons and good
practices were identified and summarised.
8
Partnership between CBM, WaterAid and World Vision
In 2014, CBM Australia developed two separate partnerships to support disability inclusion within
Civil Society WASH Fund Projects – one with WaterAid (a WASH-specific NGO) to support WASH
projects in Papua New Guinea and Timor-Leste, and another with World Vision Australia (a child-
focused NGO) to support WASH projects in PNG, Sri Lanka and Zimbabwe. Within these two
partnerships, World Vision and WaterAid are the lead implementing agencies, bringing their
extensive WASH expertise and a partnership approach to working with local organisations,
communities and governments. CBM Australia has played the role of disability advisor, supporting
the NGOs to develop disability inclusive approaches and practices within their WASH projects
including facilitating involvement and partnership with local DPOs. CBM has also facilitated
knowledge sharing and learning on disability inclusion across organisations involved in the CS WASH
Fund, for instance through facilitating a CS WASH Fund Learning Event.
Both WaterAid and World Vision commenced this work building on previous successful programs
and (some shared) experiences with regards to disability inclusion, for instance through working
together with organisations including CBM on the Australian Disability and Development
Consortium, disability inclusive pilot programs, and training for staff.
The WASH projects
This report analyses the experiences of these organisations in incorporating disability into baseline
data collection processes within five WASH projects implemented in urban, peri-urban and rural
settings. A brief background and description of each project is provided below.
WaterAid Projects
In PNG, WaterAid aims to improve the health and quality of life of the poor and vulnerable in the
Sepik region (East and West Sepik provinces) through accessible water, clean sanitation and
improved hygiene practices. Program implementation is strengthened through advocacy and sector
capacity building activities in Port Moresby and with the provincial and lower level government and
stakeholders.
In Timor-Leste, WaterAid has led a rural WASH program for ten years, and has a strong commitment
to piloting and implementing disability inclusive approaches in WASH. This project is focussed on
delivering inclusive and sustainable WASH to 36 additional communities in rural and remote areas of
Liquica and Manufahi districts. As in PNG, the project in Timor-Leste also includes significant
advocacy and sector capacity building activities in Dili and with the district government and
stakeholders.
9
World Vision Projects
In Northern Sri Lanka, the World Vision Project is targeting the growing population and most
vulnerable groups (including female headed households and people with disabilities) in two Divisions
and 10 villages. It aims to improve coordination, capacity and governance amongst WASH actors to
sustain services, knowledge and adoption of safe, contextually and environmentally appropriate
hygiene practices and equitable access to safe water and sanitation. Peri-urban and rural areas
within the post-conflict Northern Province of Sri Lanka have been severely impacted by prolonged
infrastructure underinvestment and have, as a result, limited and distant safe water sources, as well
as a high degree of water contamination.
In Zimbabwe, the World Vision project improves equitable access to safe water and sanitation in the
rapidly growing, high density urban and peri-urban areas of the Municipality of Gwanda, and
Cowdray Park and Robert Sinoyka in Bulawayo City Council. These areas are typically underserviced
by WASH facilities. Residents continue to experience poor water access, limited sanitary facilities
and display poor sanitation practices. World Vision, in collaboration with local government and other
stakeholders, will sustainably improve WASH infrastructure and practices through delivery of
enhanced water and sanitation facilities as well as hygiene awareness and education campaigns to
drive behaviour change.
In PNG, World Vision works with remote subsistence villages/communities which are generally
accessible only by river, in Western Province of Papua New Guinea. These communities have been
adversely impacted by prolonged underinvestment in WASH, resulting in very low access to safe
WASH facilities. Hygiene is a particular concern and diseases such as cholera and tuberculosis have
had significant and ongoing impact. Civil society is under-developed, representing an opportunity to
build governance, capacity and other social capital, for sustainable improvement in WASH. The
project aims to increase access to safe water and improve sanitation and hygiene behaviours for
target communities, schools and health clinics.
Baseline data collection in WASH projects Collecting baseline data is an activity undertaken by many WASH programs to inform program
planning, implementation and for monitoring and evaluation purposes. Some WASH organisations
use household surveys to collect quantitative information regarding household demographics and
current WASH access and practices. This is often complemented by qualitative information collected
through community meetings, focus groups and key informant interviews.
Given the particular challenges described above with identifying people with disabilities, the UN
Washington City Group on Disability Statistics has developed and tested a set of six questions (called
the Washington Group Short Set of Questions on Disability (WGSS)) that ask about difficulties with
performing basic functions, rather than about disability directly. Whilst the WGSS Questions were
originally designed for use in national censuses, in the absence of alternative methods, they are also
increasingly being integrated into other data collection processes. Thus far however, there is limited
10
documented evidence as to the usefulness of the questions in these contexts, or the most
appropriate ways to use them.
The Washington Group Short Set (WGSS) of Questions on Disability
Due to stigma and different cultural understandings of the term disability, identifying disability is
complex and cannot reliably be identified through asking questions such as ‘do you have a
disability?’
The UN Washington City Group on Disability Statistics developed and tested a group of six questions
(known as the Washington Group Short Set (WGSS)) that can be used in surveys to identify people at risk
of disability. The questions ask how much difficulty a person has performing basic actions (walking,
seeing, hearing, understanding, self-care and communication). Response categories range from ‘no
difficulty’ to ‘cannot do at all’.
The WGSS questions and further information are available at www.washingtongroup-disability.com
Methodologies of collecting baseline information about people with
disabilities All five WASH projects conducted baseline household surveys and included the WGSS questions in
these to some extent. Organisations also incorporated disability-specific questions into other data
collections tools and processes, such as interviews, focus groups and infrastructure accessibility
audits. In some cases, a disability-specific situational analysis was completed. The following provides
a brief overview of the processes used in each project.
WaterAid PNG In PNG, WaterAid conducted a household survey which included the WGSS questions. WaterAid and
implementing partners then visited project sites and worked with local water committees to confirm
and finalise a list of people with disabilities. WaterAid then invited people with disabilities to
participate in a disability-specific situational analysis, conducted in partnership with CBM and the
DPO. During this two-day workshop, modified versions of WaterAid’s standard participatory tools
were used to identify challenges people with disabilities in the project areas experience in accessing
WASH.
WaterAid Timor-Leste
In Timor-Leste, WaterAid conducts a household survey each time they commence activities in a new
community. Initially this directly asked if a household member had a disability. This process was
revised in July-August 2014 to instead include WGSS questions. If a person with disability is
identified, WaterAid then conducts a follow up visit to the family to refer to disability services. The
WGSS questions have also been used in engagement with key informants (i.e. village Chiefs).
World Vision Sri Lanka
In Sri Lanka, World Vision conducted a household survey which included the WGSS Questions. If
these questions identified a person with disability, additional follow up questions specific to water,
sanitation and hygiene practices were asked for each person with disability identified. Additional
questions were also included to identify who in the household had a disability (e.g. sex, age). A
11
qualitative disability assessment is also being undertaken during 2016 using a combination of focus
groups and interviews to gain further information on WASH practices.
World Vision Zimbabwe
In Zimbabwe, World Vision conducted a household survey which included the WGSS questions and
an additional disability-specific question regarding whether adaptations had been made to WASH
facilities to make them more accessible to those identified at risk of disability. Disability questions
were also included in some key informant interviews. A qualitative disability assessment (focus
groups and interviews) was then undertaken with people with and without disabilities identified
during the household survey and using DPO contacts. This was done in partnership with CBM and
the DPO. Following this, the DPO also completed accessibility audits of existing community
infrastructure and made recommendations for improvements.
World Vision PNG
In PNG, World Vision conducted a household survey which included the WGSS questions and an
additional disability-specific question regarding whether adaptations had been made to WASH
facilities to make them more accessible. When a household member with a disability was identified,
they were asked whether they could be contacted again, however follow-up didn’t necessarily take
place. WASH Assessments were completed for schools, which included questions about the number
of children with disabilities in schools and an assessment of whether infrastructure was physically
accessible. Questions about disability were also asked during community discussions to gain an
understanding of community attitudes towards people with disabilities. A monitoring visit in 2016
included focus group discussions with people with disabilities in several communities. In 2016,
interviews will be conducted with people with disabilities identified during the baseline survey to
gain further information about WASH needs.
12
Key lessons on disability inclusive data collection
1. Including people with disabilities in data collection teams resulted in
positive outcomes for people with disabilities, their communities and
other stakeholders, but needs to be resourced appropriately
The WASH projects all included people with disabilities as active participants (rather than merely as
subjects) in data collection processes to some degree, generally by establishing partnerships with
DPOs. In Zimbabwe and Sri Lanka, people with disabilities were recruited and trained as data
collectors to conduct household surveys alongside people without disabilities. In Timor-Leste,
WaterAid also employed people with disabilities in their core staff. In PNG, World Vision partnered
with a DPO who provided input into the household survey questions and a DPO representative
participated in the process of piloting the questions and training data collectors. WaterAid in PNG
involved DPO representatives in qualitative data collection by co-facilitating the disability situational
analysis workshop.
Including people with disabilities in data collection increases their confidence and can have
immediate outcomes in communities by challenging negative attitudes about their capacity
All projects reported positive outcomes from having people with disabilities involved in conducting
baseline data collection, particularly with regards to challenging negative attitudes and raising
awareness about disability (evident in Timor-Leste, Zimbabwe and PNG). For people with
disabilities, it increased their sense of self-worth, confidence, and knowledge of WASH and
development projects more generally. This also increased their standing in the community. For
example, in Sri Lanka, people with disabilities recruited as data collectors reported that prior to
being involved in the project they had lost their confidence and thought they had nothing to offer
their communities. Being involved as data collectors gave them back their dignity.
Involving Disabled People’s Organisations resulted in new forms of collaboration with NGOs and
governments, helping to change their attitudes and approaches towards people with disabilities
In Sri Lanka, involving the DPO strengthened its relationship with the government and changed
attitudes of government employees as they witnessed people with disabilities conducting their own
research and presenting findings confidently. This also helped start to change the government’s
attitude towards people with disabilities, seeing them as active change agents. In Zimbabwe, being
involved in the baseline data collection helped strengthen the DPO’s relationship with World Vision
Zimbabwe. This helped staff members see the potential of involving people with disabilities in the
project and led to a more formal partnership arrangement. DPOs have also used the qualitative data
collected for the baseline to provide training to senior municipal staff and other stakeholders,
thereby also influencing their attitudes.
Involving people with disabilities in data collection also increased the quality of data collected
In the WaterAid PNG project the DPO representative led introductory activities during workshops,
which helped participants with disabilities have confidence to introduce themselves and discuss
issues related to WASH access. She also helped NGO staff become familiar with how to interact with
13
people with disabilities, and was able to assist with data analysis by providing a unique
interpretation of the results from her perspective as a person with disability. In Timor-Leste, a
WaterAid Timor-Leste staff member with disability was able to model inclusion by effectively
supporting households with inclusion strategies and making referrals to disability services. Including
people with disabilities in data collection and analysis in Zimbabwe meant that the findings could be
tested and contextualised for accuracy.
Including people with disabilities as data collectors also requires additional time and resources
While including people with disabilities clearly had significant benefits, it also presented logistical
challenges and required additional time and resources. In particular, logistics associated with
transporting a team with diverse impairments over rough terrain was reported as a challenge in both
Zimbabwe and Sri Lanka. In the World Vision PNG project, this restricted the extent to which people
with disabilities were directly involved in data collection as the communities visited were very
remote and difficult to access.
Sufficient time and resources also needed to be dedicated to brokering relationships, building trust
and confidence among organisations, training DPO representatives in data collection methodologies
and ensuring they understand why certain methodologies are being used. While this was also the
case for other NGO staff, in many cases people with disabilities were participating in data collection
for the first time and required additional support.
It was also sometimes a challenge for DPOs to navigate undertaking this work in addition to their
existing work priorities, particularly given this was a new area of work which required investment of
time and energy into learning new concepts and skills. Implementing partners can assist DPOs to
consider their role in data collection by instigating a discussion on how they want to be involved and
encouraging them to also be mindful and realistic about existing workloads.
Recommendation 1: Involve people with disabilities/DPO representatives in data collection
processes as much as possible, not just as ‘subjects’ of data collection.
Recommendation 2: Allow sufficient time to discuss and sensitise DPOs to the methodologies used
and provide training for data collectors to undertake their role.
Recommendation 3: Think through how data collectors with disabilities will travel to project sites,
taking account of time needed to collect and drop off people, and additional space required in
vehicles for support people and assistive devices. Note that even in contexts where public transport
is available, this may not be accessible to enumerators with disabilities and transport may need to
be provided.
Recommendation 4: Work with DPOs to develop a realistic and clear understanding of their
involvement, role and responsibilities. Where needed formalise this through a clear Memorandum
of Understanding.
14
2. Collecting data on people with disabilities in communities needs to be
done sensitively
Disability is a complex and evolving concept4 which is understood differently across cultures and
communities. In many communities, people with disabilities are not ’classified’ as a separate group
and the word ‘disability’ is not widely used or understood. Instead, people may have difficulty doing
certain tasks because they are ‘elderly’ or for another reason. In the WaterAid PNG project and in
Timor-Leste, project staff found it more useful to talk about difficulties with aspects of functioning
(as recommended by the Washington Group) when working with the community as opposed to
talking about ‘disability’, which would potentially create a whole new group in the communities
which previously did not exist.
Increasing the visibility of people with disabilities by including them in community consultations for
the first time also needs to be closely monitored to ensure that any backlash from communities for
challenging social norms is addressed. For example, in Zimbabwe, community members wrongly
blamed people with disabilities as a reason for a project changing direction and 'punished' them by
excluding them from community funeral contributions. In another example, DPOs warned that
parents bringing children with disabilities to a meeting for the first time may be ridiculed for being
'cruel' by bringing children out in public. Such issues can be addressed through adequate awareness
raising, but it is important to identify these issues through strong monitoring. This will help to
ensure that people with disabilities will not be further marginalised within their communities as a
result of the inclusive WASH project.
Recommendation 5: Be aware of the language you are using when talking about disability within
communities, to avoid creating new terminology or grouping people in stigmatising ways. Referring
to ‘difficulties’ with doing certain tasks rather than using the word ‘disability’ when engaging with
communities may help.
Recommendation 6: Aim to consult both people with disabilities as well as the wider community and
be clear about the purposes of consultations. In doing so, it is important to provide “safe spaces” for
people with and without disabilities to bring up and discuss issues and community dynamics or
beliefs around WASH use which may impact on the project. Close monitoring of consultation
processes are vital to identifying issues as they arise.
4 UN Convention on the Rights of Persons with Disabilities.
15
3. Household surveys need to collect individual level data about access to
WASH in order for that data to be disaggregated by disability. However,
inclusion of the WGSS questions is also useful for identifying households
which include people with disabilities who can then be followed up
throughout the project.
All projects included the WGSS questions on disability in their household surveys. Most projects also
added an additional question regarding difficulty using hands, as this was seen as being of particular
significance to WASH tasks such as collecting water and hand washing and is not included in the
standard WGSS questions.
The WGSS questions were generally effective in identifying people with disabilities in communities,
although there were variations.5 In Timor-Leste, implementing partners reported they were able to
find more people with disabilities using this method than previous methods which involved directly
asking about disability. However, they also found that trust is a key factor and this needs time to be
established. In some cases families did not disclose having a family member with a disability (even
when the WGSS questions were asked) until during project implementation once the organisation
had demonstrated what the program was doing and that it would deliver results. In PNG, World
Vision found very low rates of disability using the recommended cut-off, so decided to also include
those who identified as having ‘some difficulty’ performing certain tasks. In contrast, WaterAid in
PNG found very high rates of disability however identified that this may have been due to a
misconception that community members would receive some benefit in the project if they identified
as having a disability.
The SDGs and some donors require data to be disaggregated by disability, sex and age wherever
possible, to assist in determining who does and does not benefit from development programs and
encourage a focus on universal access. This was one of the common objectives of including the
WGSS in the baseline surveys of the WASH projects. However, the fact that surveys collected
information about households (generally with the head of household) rather than collecting
individual-level data about household members, meant that individual differences in access to
WASH within households were not identified and data could not be fully disaggregated by disability
(or age and sex). The surveys could generally only identify whether a household included a person
with disability and whether a household had access to WASH, rather than identifying whether a
particular person with disability within the household had access to WASH. In some cases, surveys
also did not identify how many people within a household had a particular functional difficulty, or
the age/sex of those people which prevented data disaggregation.
5 WGSS questions were designed to be asked at an individual level. When asked to the head of the household this requires someone else to make a judgement about whether another person has difficulty seeing/hearing/ remembering etc. Further testing and analysis is required to determine the potential impact this may have on the reliability of the questions.
16
Sri Lanka: Capturing individual level data on disability and access to WASH within a household
survey
The baseline survey in Sri Lanka successfully captured information on disability and access to WASH
by including some individual level questions in the survey. In this questionnaire, an initial screening
question was asked to the head of the household ‘does anyone in this household have any
difficulties in seeing, hearing, walking, remembering, self-care, communicating, or in using hands?’
Respondents were then asked how many people in the household experienced any difficulties (in
total)6 and were then asked the WGSS questions about each person, along with their age and
gender. Additional questions were also included as to whether and how people with difficulties
accessed water and sanitation facilities and if they required assistance. While this methodology still
did not enable full disaggregation of survey data by disability (as the remainder of the survey was
only asked at household level) it did capture a far greater level of detail on the WASH needs of
people with disabilities within the household and provided a solid baseline on which to monitor
disability inclusion.
Recommendation 7: With the Sustainable Development Goals’ renewed focus on access to water
and sanitation for all, WASH implementing organisations should reconsider their approach to
collecting baseline information to better identify and seek to monitor inequalities to accessing WASH
within households, rather than only collecting household level information.
Household surveys were an important mechanism to identify people with disabilities for inclusion
in qualitative data collection processes
Finding people with disabilities within communities is often a key challenge reported by NGOs
undertaking development projects, as social norms, stigma and the physical environment often
mean that people with disabilities remain hidden within households. While using the WGSS
questions alone did not enable full disaggregation of data, the fact that they were generally effective
in identifying households which included people with disabilities was very useful for projects and
opened up a range of opportunities for their inclusion in the WASH projects. For example, in most
projects, people with disabilities identified during household surveys were then invited to participate
in other data collection processes such as interviews, focus groups and community meetings. One
key challenge in this regard was using an identifier that enabled the household to be located again
after the survey. In Zimbabwe, following up people with disabilities identified during the initial
survey proved to be a challenge due to the urban/peri-urban context where there was no
systematised numbering of households. In other settings this was not a significant concern.
People with disabilities were also recruited to be involved in qualitative processes through DPO
networks, however this sometimes led to a skewed representation of different impairment types (if
for example DPOs had more networks amongst people with mobility and vision impairment rather
than psychosocial impairments). This further highlights the benefits of using the WGSS questions to
identify households with people with disabilities in addition to DPO networks to get a more
representative sample of people with disabilities for inclusion in qualitative processes.
6 This question may result in ‘over-reporting’ disability as it refers to those with ‘any difficulty’ in functioning rather than just those with ‘a lot of difficulty’ or that ‘cannot (see/hear etc) at all’.
17
Recommendation 8: Even where disaggregation of data will not be possible, the WGSS questions
should be included in household surveys for the purposes of identifying households with people with
disabilities for future follow-up. In doing so, ensure processes are used during surveys to enable
identification of households where people with disabilities have been identified. Undertaking follow-
up analysis at the same /similar time as when the survey is being undertaken will help to avoid
difficulties in finding households.
Recommendation 9: If relying primarily on DPO contacts to find people with disabilities within
communities, consider what impairment types are/are not represented and aim for a broad
representation. This may require contacting a range of DPOs or using additional key informants.
4. There were challenges with but also unexpected findings from using the
WGSS questions in surveys
In most countries, this was the first time the NGOs and implementing partners had used the WGSS
questions in a household survey, and while they are often promoted as being easy to use, in reality it
turned out to be quite complex. All NGOs reported some challenges in using the questions as they
were intended and in analysing the results. An overview of some of the challenges that NGOs
encountered is provided below.
Finding the time and resources to build understanding of the rationale of the WGSS questions and
follow testing and translating protocols
The WGSS questions have been developed to overcome particular challenges and cultural
sensitivities in gathering data on people with disabilities, and are the result of extensive testing to
ensure they reliably identify people at risk of disability. This means the exact wording of each
question has been chosen very carefully. As words and concepts are understood differently across
cultures and can be difficult to translate, the Washington Group recommends the WGSS questions
are translated by a qualified/experienced translator and cognitively tested prior to their use. While
this is best practice, in reality, development organisations are often pressed for time and funding
and are under pressure to get their baselines completed quickly. For the WASH projects, this meant
translation and testing protocols could not always be followed.
Additional time also needed to be invested in ensuring organisations using the questions understood
the rationale behind their development, as well as the risks of changing the questions without
proper testing. In some cases when this was not addressed sufficiently, changes were made to the
wording of the questions without adequate testing or additional questions were added. In Sri Lanka,
the DPO completed a parallel disability survey using an alternative approach to identifying disability
as they did not believe the WGSS questions were effective. Spending additional time with the DPO
to build their understanding of the approach could have prevented this duplication of effort.
Balancing the need to ensure consistent use of the WGSS questions with the preferences of data
collectors regarding survey tools and guidance materials
While the Washington Group has recommended the WGSS questions are used in a certain way to
maximise their effectiveness, this needed to be balanced with the preferences of data collectors. For
18
example, in Zimbabwe, data collectors expressed a strong preference for the survey tool to be
provided in English for them to translate on the spot, as they were not accustomed to reading in the
local language. While this was not ideal (as it is likely to have produced many inconsistencies in the
way questions were translated), it was important to provide survey tools to the data collectors in a
format they were comfortable with. Similarly, in Timor-Leste, it was not considered appropriate for
data collectors to carry large amounts of paper given the distances travelled within communities. In
this circumstance it was difficult to provide sufficient guidance to help data collectors implement the
WGSS questions effectively.
Selecting and using a ‘cut-off’ to determine disability status
The baseline surveys of the WASH projects were all structured differently. Some surveys were
designed so that the responses to the WGSS questions would trigger asking additional questions
once a person had been identified as having a disability. While this enabled more specific
information to be captured around the situation of people with disabilities, it also required data
collectors to interpret responses to the WGSS during data collection, which in some cases became a
point of confusion. This is because the WGSS questions are not ‘yes/no’ but instead include four
response categories (no difficulty, some difficulty, a lot of difficulty, cannot do at all). Those using
the questions are required to select a ‘cut-off’ to determine disability status. For large/national level
surveys this is generally recommended to be when a person answers ‘yes - a lot of difficulty’ or
‘cannot do at all’ to at least one question; however it can also be when a person answers ‘yes-some
difficulty’ if programs are interested to identify people with more minor functional difficulties.
While the cut-off can generally be selected during data analysis, it must be clear to data collectors
when additional disability specific questions should be ‘triggered’ during the survey. In one project
for example data collectors were not clearly instructed on what cut-off to use and unintentionally
used different cut-offs in different project locations, which meant the additional questions on
disability were not asked consistently.
Benefits of using a more sensitive cut-off rate for the WGSS questions
The projects found that there may be a need to change the ’cut off’ to be more sensitive in WASH
projects as people with minor difficulties performing tasks in fact reported significant difficulties
accessing WASH. In PNG for example, WaterAid unintentionally classified people who reported
‘some difficulty’ to one of the WGSS questions as people with disabilities, meaning that people with
relatively “minor” disabilities were included in their disability analysis. While this was unplanned, it
lead to revealing findings as people who would not traditionally have been thought of as having a
disability reported a range of substantial difficulties accessing WASH. This was particularly the case
for women who were still culturally expected to undertake WASH work, and undertake caring and
other household duties despite the difficulties they faced. These women reported spending
significantly longer undertaking WASH tasks (particularly collecting water) because they were not
able to carry the same quantity of water as others, took longer to travel to the water source and had
to make more return trips. Similarly, by also consulting with carers of people with disabilities, new
and unique insights were gathered on their specific needs which can inform project design and
implementation.
19
Using the language of the WGSS questions to talk about disability in communities also helps
capture difficulties faced by elderly people
In Timor-Leste, using the WGSS questions enabled implementing partners to also capture
information about elderly people who were facing difficulties accessing WASH. Previously their
needs may have been missed as they may not have identified as people with disabilities, instead
considering their impairments to be simply a consequence of ageing. This highlights the benefits of
focusing on difficulties with functioning rather than talking directly about disability.
Recommendation 10: Ensure adequate time is spent discussing and explaining why the WGSS
questions are recommended for use in baseline surveys, to increase buy-in of local DPOs,
implementing partners and data collectors. This should highlight the importance of not changing the
questions unnecessarily and of appropriate translation and testing.
Recommendation 11: Prior to use, provide face to face training for data collectors on how to ask the
WGSS questions, why they are being asked and how they will be expected to interpret the questions
(if relevant).
Recommendation 12: If data collectors are required to interpret the WGSS questions during the
survey in order to ‘trigger’ additional questions, ensure the more sensitive cut-off is used (e.g.
answering ‘some difficulty’ to at least one question) so that people with minor functional difficulties
are also asked about their access to WASH. Where possible, also state this cut-off on survey
templates to remind data collectors.
5. A combination of quantitative and qualitative data collection processes
produces the most useful baseline data on people with disabilities
Even in Sri Lanka, where additional questions were included in the household survey and the WGSS
questions were asked at an individual level, this only resulted in a clear picture of who did/did not
have access to WASH rather than what, why, or how best to address the barriers faced. Collection of
qualitative information on the experiences of people with disabilities in accessing WASH facilities
was critical to enable an understanding of the type of difficulties people with disabilities were
experiencing in accessing WASH (and therefore strategies to address these).
All projects completed (or plan to complete) some type of qualitative disability assessment as part of
the baseline assessment. In Zimbabwe, this included interviews and focus groups with people with
disabilities and accessibility audits of WASH infrastructure. In the PNG World Vision project, disability
questions were included in community mapping activities and in WASH infrastructure audits. In
Timor-Leste, a follow up is done with people with disabilities identified through the survey to
understand what support they need and refer them to services where possible and to assist with
further monitoring. In the PNG WaterAid project, a participatory community workshop was held for
people with disabilities.
20
Projects which completed qualitative data collection specifically with people with disabilities
ended up with richer baseline information about barriers to WASH
Where most projects completed specific (separate) qualitative processes for people with disabilities,
World Vision PNG included questions about disability within standard community mapping
processes. These aimed to gather information about community attitudes towards people with
disabilities and access to WASH. The data gathered through these discussions was useful in
understanding broad community attitudes but did not provide sufficient information on barriers to
accessing WASH on which to adapt project designs. World Vision PNG therefore also held additional
interviews directly with people with disabilities in some communities to supplement this
information.
Standard community consultation and planning tools can be used with people with disabilities
with minor adaptations
In PNG, WaterAid used a slightly modified version of their existing participatory WASH planning tools
which partners and communities were already familiar with for their disability situational analysis.
While some additional disability-specific questions were added, overall, these were found to be
appropriate to use with men, women and children with disabilities and were successfully combined
with focus group discussions, interviews and observation techniques (to go into more depth on the
identified issues). The exception to this was for some image-based participatory tools used for
children with disabilities where some further adjustments would need to be made to fully cater for
children with vision impairments.
In Zimbabwe and PNG, World Vision adapted standard infrastructure audits of school WASH facilities
to include a question on accessibility. While these did not assess accessibility in great detail, they
provided an initial indication of accessibility on which to base further analysis. In Zimbabwe, this was
supplemented with a targeted accessibility audit which was undertaken of community infrastructure
by the DPO. This successfully raised awareness of what was lacking in school and as a result, two
schools have now completed refurbishments beyond just WASH facilities, and it is expected that
more children with mobility issues can now attend school. While this produced a good outcome, it is
recommended that this type of activity is included in the initial baseline visit to schools instead of as
a separate activity.
21
Participatory tools used during the disability situational analysis with Water Aid in PNG
Community mapping: brought out particular issues for people with disabilities with regard to access to water holes, location of toilets etc and practices around sanitation and hygiene.
Timeline exercise: gave insights in how a typical day looks like, and what challenges people with disabilities may typically experience during the day to access and participate in WASH activities.
Story telling for children using standard WASH picture cards: brought out issues related to access to WASH for children with disabilities, WASH practices of the children themselves and their parents, gender roles, their own attitudes towards these roles and how they perceived themselves as children with disabilities compared to other children. It also brought out their personal priorities in relation to WASH and community participation.
6. Collecting baseline data on people with disabilities led to more inclusive
WASH programming and had a range of positive outcomes
In addition to being a source of data to monitor progress, the process of collecting baseline data has
in itself been a catalyst for projects to increase their commitment to disability inclusion and has led
to a range of outcomes. For example:
In the PNG World Vision project, some accessible toilets have been constructed; while as a
result of implementing the Healthy Islands approach in their communities, people have
began to consider the needs of people with disabilities in broader community activities
addressing hygiene behaviour.
In Timor-Leste, people with disabilities have been involved in deciding where community
WASH infrastructure will be located, support has been given to households to problem solve
accessibility issues, and materials have been distributed to assist households to construct
accessible latrines. WaterAid has now been asked to share its experience using the WGSS
questions by speaking at the working group on the National Census data process. It has also
been able to draw on this experience to influence the Government’s national rural WASH
program (BESIK).
In the PNG WaterAid project, an advocacy booklet is currently being prepared to share with
water committees.
In Sri Lanka, the baseline helped to identify which people with disabilities had no access to
latrines and enabled the project to assist those people to apply for them.
22
Zimbabwe: Baseline data prompts a disability inclusive approach
In Zimbabwe, data from the baseline and the accessibility audits have prompted the construction of
accessible public toilets, and two schools have refurbished latrines with accessibility features.
Information from the baseline assessment is also being used in training workshops with council
workers and WASH sector staff to influence policy and planning. Furthermore, Gender and Social
Inclusion (GESI) Champions have been established in each community to strengthen representation
at community meetings and identify households who have a person with a disability to facilitate
access to disability services and advocate for government support. GESI Champions are now being
consulted by the Local Authority on disability and gender issues beyond the WASH project. The Local
Authority is even planning to replicate the GESI model to other communities beyond the project
area, as they have found it an excellent way to consult with people with disabilities.
While all projects have seen benefits from conducting disability inclusive baselines, a range of factors
have influenced the extent to which baseline information informed project design and
implementation.
Baseline data collection needs to be conducted early and seen as a tool to guide project
implementation
Timing of baseline data collection was a big factor in the extent to which it could inform project
design. For some projects, delays in conducting the baseline data collection (sometimes beyond the
project’s control), analysing the data and sharing the findings with implementing partners limited
the extent to which it has informed project implementation. For example, in Sri Lanka there was a
need to ensure local government personnel were given the opportunity to contribute to the tools
and methodologies used. In some cases, baseline surveys were also considered stand-alone activities
rather being seen as a source of data to inform projects. Encouraging and actively supporting
projects to conduct baselines data collection early and see the value of baseline data as a project
planning tool would assist with strengthening its use in the future.
Analysis of baseline data related to disability needs to be resourced appropriately
Analysis of both quantitative and qualitative data related to disability is complex and was also
reported to be a challenge. In some cases this was because the actual questions asked did not
provide the type of information sought from the baseline (e.g. the WGSS questions were not
coupled with other questions in a way that allowed for data disaggregation) whereas in others,
further training and support was needed to analyse and interpret the information effectively.
Recommendation 13: Conduct baseline data collection and share findings as early in the project as
possible, and support implementing partners to think through what the findings mean for project
implementation.
Recommendation 14: Plan for how baseline data will be analysed before it is collected and ensure
adequate training and support is provided to implementing partners and DPOs to facilitate this.
23
Discussion
Benefits of disability inclusive baseline data collection
Overall, conducting disability inclusive baseline data collection produced a broad range of benefits to
projects on multiple levels. The action learning approach used for the baseline processes led to
greater disability awareness and capacity among NGO and partner staff members. This led to
increased confidence to talk about disability rights in a more open manner, particularly when
meeting with community leaders, and awareness of the need to consider people with disabilities in
WASH programming. At a community level, the baselines helped to identify people with disabilities,
including groups that may have been left out previously, such as people with minor disabilities and
older people, but who turned out to face significant access issues. Baseline data collection processes
also helped to challenge negative attitudes of community members, NGOs and government
stakeholders towards people with disabilities.
Finally, use of the findings resulted in more inclusive programming and improved project and
development outcomes. These included the construction of accessible WASH infrastructure,
successful use of the baseline data for advocacy and training purposes with stakeholders including
government, and involvement of people with disabilities in local government decision-making
processes.
Despite these positive outcomes, conducting disability inclusive data collection was not without its
challenges and required additional resourcing in terms of time and budget, particularly as
organisations learned new tools and approaches. Furthermore, there are a few major areas where
further analysis is recommended.
Need to explore alternative methods for measuring intra-household inequalities
The design of household surveys used by the WASH implementing partners collected information
primarily at a household level (e.g. whether the household has a latrine). This was found to be
inadequate to disaggregate data on access to WASH by people with disabilities and therefore
understand the extent to which inequity may occur within households. Capturing only household-
level data assumes that everybody within a household experiences poverty in the same way and has
access to the same resources, whereas we know that gender, disability, age and other family
dynamics impact on this. If development partners are serious about ensuring WASH programming
reaches everyone, it is vital to reflect more on how household surveys can better collect information
on individuals within households and differences in access to WASH. Further analysis is required to
determine the best methods of achieving this, acknowledging the significant expense associated
with undertaking large surveys or other data collection processes. This should include exploration of
other measures such as the Individual Deprivation Measure (IDM) developed by the International
Women’s Development Agency7 and the Rapid Assessment of Disability (RAD) developed by the
University of Melbourne’s Nossal Institute for Global Health and Centre for Eye Research Australia8.
7 See https://www.iwda.org.au/assets/files/IDM-Poster_digital.pdf
8 The RAD survey consists of household and individual questionnaires. Each head of household is invited to complete the household questionnaire, while individuals residing in the household are then invited to complete the individual questionnaire with the interviewer that explores the situation of people with disabilities that can be included in household surveys when a person with disability is identified.
24
Balancing qualitative and quantitative methods
Secondly, given the time and budget constraints which come with community development projects,
program implementers need to carefully consider the balance of qualitative and quantitative data
that should be collected, based on an analysis of what information is required to inform disability
inclusion within that context. Quantitative data can be used to powerfully highlight inequalities in
access and measure progress of some indicators towards inclusion. However, there is a danger that
analysis stops at identifying people with disabilities and too much emphasis is placed on ‘classifying
people’ as with or without disability which may create further separation and marginalisation of
people with disabilities within communities by highlighting difference. Sufficient attention needs to
be given to identifying and understanding the situation of people with disabilities more broadly, and
the barriers they may face in accessing WASH services. A strong analysis of the situation of people
with disabilities will enable development of strategies and solutions to address challenges in
accessing WASH that are contextually appropriate.
