Capstone
Report A quantitative and
qualitative analysis of the
World Bank Community
Monitoring Project in
Burkina Faso
Saman Amir, Brittany Heyer, Obaid Khan,
Sarute Vithoontien
THE GEORGE WASHINGTON UNIVERSITY
ELLIOTT SCHOOL OF INTERNATIONAL AFFAIRS
INTERNATIONAL DEVELOPMENT STUDIES
MAY 15, 2017
1
TABLE OF CONTENTS
Executive Summary ................................................................................................... 2
Introduction ................................................................................................................ 3
Country Context ............................................................................................................................................ 3
Project Overview ........................................................................................................................................... 4
Research Purpose ...................................................................................................... 6
Methodology ................................................................................................................ 7
Literature Review: ..................................................................................................... 8
Quantitative Component: ....................................................................................... 11
Sampling Strategy for the Community Monitoring Program: Cluster-Randomized Controlled Trial ......... 11
Quantitative Analysis .................................................................................................................................. 13
Establishing Baseline Equivalence for the Treatment and Control Group ............................................... 13
Using Difference-in-Differences when Samples are Randomized ........................................................... 16
Models ........................................................................................................................................................ 17
Comparison of Program Impact on Child Sickness Across Regions ......................................................... 22
Comparison of Program Impact on Child Health Across Gender ............................................................. 22
Qualitative Findings ................................................................................................ 23
Community Monitoring Project-Specific Themes ....................................................................................... 23
Contextual Themes ..................................................................................................................................... 26
Limitations ................................................................................................................ 33
Conclusion ................................................................................................................. 34
Recommendations .................................................................................................... 35
Bibliography .............................................................................................................. 36
Appendix .................................................................................................................... 42
Interview Guides ......................................................................................................................................... 42
Models ........................................................................................................................................................ 53
2
EXECUTIVE SUMMARY
The World Bank Independent Evaluation Group and Development Impact Evaluation, along with
their implementing partner in Burkina Faso, the Higher Institute of Population Sciences (ISSP),
collaborated in September 2010 to implement a community monitoring pilot project to improve
health and education service delivery. This novel approach was adopted to capture the spillover
effects of community monitoring across health and education sectors.
This report presents evidence from a literature review on different facets of citizen engagement,
quantitative analysis of child health outcomes as a result of the Community Monitoring Project,
and qualitative analysis of social accountability and inclusion based on field research in Burkina
Faso. The impact of community monitoring programs can be mediated by multiple contextual
factors. This warrants comprehensive research to explore mechanisms and pathways through
which community monitoring can positively impact service delivery outcomes.
Rigorous evaluations of citizen engagement projects should be utilized not only to generate
credible evidence on the effects on ultimate outcomes, but also to unpack the causal chain to
identify the intermediate outputs that link the intervention to changes in short-term and long-term
outcomes. Comprehensive evaluation can help identify enabling and impeding mediating factors
and provide insightful lessons for future citizen engagement projects. Building on the social
accountability and inclusion components from the inception of the project design can also
amplify the effectiveness of citizen engagement projects by including diverse and marginalized
voices.
KEY FINDINGS
The Project had an impact on child health outcomes, specifically the likelihood of getting sick.
However, the impact differed across region and gender. Further analysis of the Project’s impact
on immediate and intermediate outcomes can help unravel the causal pathways for the
effectiveness of the Project in improving child health outcomes.
Participants framed the Project’s success as community sensitization, empowerment and
improved self-efficacy.
Participants identified inadequate inclusion of women and other marginalized segments of
population as a shortcoming of the Project.
Budget constraints and lack of capacity at different levels of government were cited as the main
impediments to effective service delivery.
Participants emphasized the instrumental role of NGOs, civil society organizations, donors, and
the media and protests to improve social accountability.
3
INTRODUCTION
COUNTRY CONTEXT
Despite steady economic growth and more than a decade of reforms and sound macroeconomic
policies, persistently high rates of rural poverty undermine development efforts in Burkina Faso
(World Bank 2013). With a gross per capita income of $615, approximately 40% of the
population lives below the poverty line (World Bank 2015). Burkina Faso ranks 183 out of 188
on the United Nations Development Programme’s Human Development Index (2015),
registering one of the lowest adult literacy rates in the world (36%), and alarmingly high infant
and maternal mortality rates (60.9 per live births and 371 per 100,000 live births, respectively).
Gender inequality is also pervasive in Burkina Faso, as gendered differences in human
development indicators illustrate. 6% of adult women have reached at least a secondary
education, compared to 11.5% of their male counterparts (UNDP 2015).
Nevertheless, improvements in human development have been made: the gross enrollment rate at
primary level education rose from 57% in 2005 to 87% in 2014, and the poverty rate fell slightly
between 2009 and 2014: 46% to 40.1% (World Bank 2015).
In addition to improvements in quality of life, changes in the country’s political context are cause
for a positive outlook. Burkina Faso has undergone a process of decentralization and
democratization since the 1990s, culminating in a popular democratic uprising in 2014 in
response to a proposed constitutional amendment to suspend presidential term limits (Freedom
House 2016). However, decentralization remains a challenge for the Government of Burkina
Faso, as local government structures struggle to fulfill their social services mandates (World
Bank 2013).
The figure below presents the decentralized framework of the Government of Burkina Faso
today, beginning with local village’s health facility and school management bodies, COGES
(Comité de Gestion), and parent-teacher associations, APE and AME (Association de Parents
d’Elèves and Association de Mères Éducatrices). To facilitate service delivery, these local
structures work primarily with the village municipal council and communal governments, and
then the regional and central governments.
Figure 1.1: Decentralization framework
The World Bank Community Monitoring for Better Health and Education Service Delivery
Project (CMP) complements and supports the Government’s decentralization efforts by
empowering communities to oversee local development and service delivery.
4
PROJECT OVERVIEW
The Burkina Faso Community Monitoring for Better Health and Education Service Delivery
Project (CMP) seeks to improve the quality and quantity of health and education services by
empowering and building the capacity of individuals and communities to demand better services.
The primary intervention is a community scorecard mechanism, which provides the community
with the means to monitor and evaluate education and health services, improving accountability
and transparency of service providers.
The process is intended to be community driven; in each village, the community is trained by
local non-governmental organizations (NGOs)1
in using community scorecards to define
evaluation criteria for education and health facilities and identify service delivery issues. The
community and service providers then work together to develop community action plans with
strategies to address these issues. Implementation of these action plans is routinely monitored by
the community using the scorecards.
This process is meant to engage multiple local, regional, and national stakeholders in public
health and education service provision. The different tiers of the national and regional
government are first involved through sensitization and consultation frameworks to create buy-
in. The communities are expected to make desired changes on their own or with assistance from
the local COGES, APE and AME bodies, as well as the village development committees under
the village municipal council that are tasked with overseeing and promoting local development.
The higher tiers of national and regional actors are the last resort for changes that fall outside of
the scope of villagers and local government.
The underlying theory of change is that this monitoring, coupled with public praising or shaming
through the dissemination of scores, will motivate service providers to implement action plans
and improve performance, which will yield greater improvements in health and education
outcomes. Additionally, the CMP is expected to induce greater participation of households in the
management of community affairs and, potentially, improve social capital.
1 ISSP, the central implementing partner for the pilot project, was responsible for identifying local non-
governmental organizations (NGOs) to facilitate trainings and implementation of the CMP in the selected
communes.
5
Figure 1.2: CMP Theory of Change
The project posits that the community and providers care—or can be induced to care—about
better health and schooling outcomes. Available resources should be applied in greater quantity
or with greater efficiency, and that the degree of commitment among relevant actors is sufficient
to mobilize those resources.
To generate credible evidence on whether any changes in outcomes are attributable to the
Project, the CMP was designed as an experimental impact evaluation piloted by World Bank
Independent Evaluation Group (IEG), in collaboration with the Development Impact Evaluation
(DIME). The evaluation design is a cluster-randomized controlled trial with health facilities and
schools randomly assigned to treatment or control groups in three of Burkina Faso’s thirteen
administrative regions.
Preliminary analysis of the Project has been conducted by IEG through focus groups and a
process evaluation (de Briey 2015). The George Washington University (GWU) team aims to
build upon the existing evaluative work and integrate learning from the various evaluation modes
through field research and empirical analysis of panel data. The team enlarged the scope and
focus of the Project’s evaluation by examining the CMP within the broader political and social
context of Burkina Faso.
6
RESEARCH PURPOSE
To supplement the existing program evaluation, the team analyzed the effectiveness of the
intervention on health outcomes. In order to understand the channels through which these
outcomes materialized, the team explored the mechanisms of citizen engagement and social
accountability underpinning the theory of change.
The GWU team sought to answer the following key questions:
1. What are the treatment effects of the CMP on child health outcomes?
2. Are there gendered differences in the Program’s outcomes or mechanisms?
3. What contextual factors influenced the Program’s effectiveness?
4. What are the perceptions of social accountability and citizen engagement among
Burkinabe?
5. How can community monitoring interventions at the grassroots level be
complemented with social accountability at higher tiers of government?
6. How can social inclusion be better integrated with community monitoring
interventions to improve voice and empowerment across the community?
7
METHODOLOGY
The George Washington University team conducted quantitative and qualitative research to
answer the research questions.
Using the child health data collected by IEG, the quantitative component included multivariate
regression and linear probability models, within the difference-in-difference approach, to isolate
the Community Monitoring Project’s effects on child health outcomes. To ensure consistency
and unbiasedness, the models added relevant covariates.
The GWU team’s qualitative research was primarily informed by a field visit to Burkina Faso.
The team conducted 13 key informant interviews with Project stakeholders, including NGOs,
local government officials, youth groups, World Bank staff, and members of the CMP Pilot
Committee who oversaw Project implementation. A focus group discussion was also organized
in a beneficiary commune, consisting of members of the local COGES, APE, and AME.
Interview protocols were designed after a thorough survey of relevant literature, prior qualitative
research, the CMP process evaluation and program documents, and consultation with World
Bank staff. The interview recordings were transcribed, coded, and analyzed to aggregate
common and divergent themes.
Finally, the team synthesized the quantitative and qualitative findings to formulate a narrative
assessing the CMP’s effectiveness, and developed recommendations for implementation and
scaling up of similar interventions in Burkina Faso.
8
LITERATURE REVIEW:
Prior to a thorough engagement with the social accountability approaches involved in the
Community Monitoring Project, we briefly review what social accountability entails. Ackerman
defines social accountability as “a pro-active process by which public officials inform about and
justify their plans of action, their behavior and results and are sanctioned accordingly”
(Ackerman 2005, 2). Per the World Bank, social accountability refers to the “extent and
capability of citizens to hold the state accountable and make it responsive to their needs” (2012).
Other scholars support this definition with the emphasis on primary engagement of citizens and
civil society in demanding actions from public officials (Brinkerhoff and Wetterburg 2016;
Grandvoinnet et al. 2015; Malena and McNeil 2010). Note that these definitions are holistic and
not restricted to elections, which many scholars discard as the key tenet of accountability
(Ackerman 2005; Geddes, 1994; Varshney, 1999; Stokes, 2003). For the purpose of this paper,
our team considers the social accountability definitions from the World Bank and Ackerman as
conceptual frameworks.
Fox (2014) divides social accountability into dual tactical and strategic approaches. Tactical
approaches rely merely on greater access to information to affect change, whereas multi-pronged
strategic approaches that combine citizen engagement with governmental reforms offer more
promise. Using this lens, the CMP could be seen to follow a strategic design and therefore
exhibit potential to influence what Fox terms ‘voice’ and ‘teeth’ (state responsiveness).
Literature emphasizes the paramount importance of contextual factors in assessing community
driven social accountability processes. O’Meally (2013) prioritizes understanding of context in
social accountability interventions by chalking out six instrumental contextual factors: political
society, civil society, state society relations, intra-society relations, and inter-elite relations.
Scholars also point to decentralization as a crucial contextual factor. A systematic literature
review on community empowerment, accountability, and education outcomes by the Department
for International Development (DFID) and International Initiative for Impact Evaluation (3ie)
focuses on decentralization as a key factor for the effectiveness of community empowerment
programs (Westhorpe et al. 2014). Other than the direct effect of decentralization on community
empowerment, the study mentions that decentralization provides a conducive context for
collective action. Campbell Collaboration’s systematic review undertakes a meta-analysis of
quasi-experiments and randomized controlled trials to verify the effectiveness of CMPs in terms
of corruption, access and quality of service delivery outcomes’ (Molina et al. 2016). The study
addresses the reduced incentive for politicians to respond to citizens’ needs if the needs are not a
contributing factor to the electoral strategy. The policy prescription that stems from these studies
is the enforcement of mutual accountability to amplify the effect of community-driven
interventions and ensure sustainability. Devolution of power and responsibilities, coupled with
social accountability at every level of the government, can help empower communities and the
government to pursue improvements in health and education outcomes.
9
Social capital also emerges as a pivotal factor particularly due to its relevance to CMP’s theory
of change. Evidence from a community-driven development program in Morocco on social
capital reveals that intervention increased the willingness of members of the community to
contribute to public goods, but had a negative effect on trust (researchers believe that this may be
due to tensions arising from competition between communities for CDD funding), and no impact
on altruism (Tu Chi Nguyen and Matthias Rieger 2017).
In terms of the impact of social accountability and community-based monitoring on service
delivery, a randomized field experiment on community-based monitoring of public primary
health care providers in Uganda stands out. Bjorkman and Svensson (2007) find that the use of
community scorecards to rate health service providers and community meetings to discuss
subsequent action plans significantly increased the quality and quantity of health care provision,
exhibited by an in increase in weight of infants, reduction in children deaths, increase in service
utilization, and positive behavioral change in service provision. A variation in treatment intensity
across districts allows the authors to infer that there is a significant relationship between the
degree of community monitoring and health utilization and outcomes, consistent with the
community-based monitoring mechanism.
Facilitation of women’s groups in conjunction with some health system strengthening could also
improve health outcomes, particularly reduction in maternal and neonatal mortality, and change
in health seeking behavior (Manandhar et al. 2004; Tripathy et al. 2010). Per the International
Development Coordinating Group’s review on community monitoring interventions, outcomes
regarding access to services were more definitive (Molina et al. 2016). Although employing only
information campaigns did not have a statistically significant effect on healthcare service
utilization in the short-run, running information and scorecard campaigns in tandem had positive
effects on utilization in both the short and medium-run (Molina et al. 2016). Furthermore, the
systematic review notes that community monitoring interventions had a positive short-term
impact on immunizations; however, medium-term effects were reportedly smaller and less
accurate.
Literature finds that community engagement and decentralization in school based management
tends to increase access to schooling, but a positive effect on learning outcomes and improved
accountability are not forthcoming. Results from a randomized experiment in Kenya show that
the use of school committees to monitor teachers improved teacher attendance and student
performance (Duflo, Dupas and Kremer 2008). The USAID Nigeria Northern Education
Initiative aimed to improve school performance management by encouraging civil society
participation and participatory evaluation through collection and dissemination of school report
cards (Brinkerhoff and Wetterberg 2016). In contrast to the experiment in Kenya, Healey’s
assessment of the project revealed little improvement in service delivery because powerful state
actors ignored the enhanced voice and demands from the relatively powerless civil society
(Brinkerhoff and Wetterberg 2016).
10
Scholars also evaluate social accountability’s equivocal effect on social inclusion and addressing
inequality. Gaventa and Barrett (2010: 46) highlight the potential of social accountability in
promoting social cohesion and cite the example of the use of forum theater in Nigeria to bring
the issues of community hierarchies based on wealth and ethnicity to the public fore. Mohanty
(2007: 85), however, describes a complete lack of agency among women recruited into
watershed management committees merely for procedural concerns, with limited awareness of
their membership in the committee. Corbridge et al (2005: 149) observed that participatory social
accountability approaches employed in village education committees disproportionately helped
wealthier groups.
Although social accountability initiatives often encompass citizen engagement, evidence on
citizen engagement’s impact deserves a special mention. Baldwin et al. (2016) evaluate the
impact of community-based development on citizen empowerment in Eastern Ghana. The
authors found that the leadership and civic commitment skills taught in the intervention
workshops led to improved perceptions of the village leadership quality and increased political
participation and representation at the district level. However, the results show that the program
did not translate into increased local government investment and did not improve socioeconomic
outcomes in the areas of literacy and education, health and nutrition, environment, food security,
and agricultural production. The World Bank’s report, ‘Making Politics Work for Development:
Harnessing Transparency and Citizen Engagement’, documents experimental evidence from
different countries on enabling factors that were adopted to strengthen transparency and citizen
engagement (World Bank 2016). The report highlights that ‘political accountability’, citizens
holding political institutions accountable, is as important for development as ‘social
accountability, whereby citizens can improve service delivery through collective action without
necessarily involving political institutions. USAID’s review of the effectiveness of citizen
engagement instruments in improving health service delivery and health outcomes draws
attention to socioeconomic and political context, and emphasizes that the impact of citizen
engagement on health outcomes can be amplified if the intervention is part of a broader package
of services (Edstrom 2015).
11
QUANTITATIVE COMPONENT:
SAMPLING STRATEGY FOR THE COMMUNITY MONITORING PROGRAM: CLUSTER-
RANDOMIZED CONTROLLED TRIAL
IEG employed a cluster-randomized controlled design as a sampling strategy to implement the
Community Monitoring Project. Three of the country’s thirteen regions, namely Cascades,
Central Plateau and Sahel, were selected to be included in the Project. For each region, three
communes were randomly selected as implementation sites for the treatment health facilities, and
three communes were randomly selected for the control health facilities. In each commune, two
health facilities were randomly selected to be assigned to the treatment or control arms.
Treatment and control school groups were also randomly assigned if they were within proximity
to a treated health facility to capture the indirect effects across health and education sectors.2
This effectively meant that 36 health facilities and 36 schools were randomly assigned to the
treatment or control arm in total, with 18 schools and 18 health facilities in each arm.
Finally, there were 36 villages allocated for the community monitoring intervention for treatment
and control schools. 168 villages were designated for treatment and control health facilities, with
36 in the direct treatment villages where the health facilities were located, and another 132 in the
catchment areas of health facilities. The additional 132 villages with no health facilities were
randomly selected to be included within the vicinity of treatment and control health facilities (66
each). The selection of additional villages was to capture the indirect effects of the intervention
over health facilities that were in proximity to the treatment health facilities as compared to the
control health facilities. The step-by-step sampling strategy is shown below in Figure 2.1.
2 The treatment and control schools were nested in the catchment area of treatment health facilities only, and not in
the control health facilities (as shown below in Figure 2.2.)
12
Figure 2.1 IEG Sampling Strategy
13
QUANTITATIVE ANALYSIS
To isolate the effect of the community monitoring program on health outcomes, we employed a
difference-in-difference analysis with multivariate regression and linear probability models. Each
model explored treatment effect on different health outcomes of interest (child sickness in the
past 30 days and child body mass index). The core model is demonstrated below:
CONCEPT MODEL:
𝑌ℎ𝑒𝑎𝑙𝑡ℎ_𝑜𝑢𝑡𝑐𝑜𝑚𝑒 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒 + 𝛽𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡(𝑥1) + 𝛽(𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝜇𝑖
The main variable of interest 𝛽(𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡∗𝑡𝑖𝑚𝑒)
is the treatment effect represented by the
interaction between the community monitoring program and time the effect of the monitoring
program across time. The first two models (Model 1 and Model 2) highlighted the effect of the
monitoring programs implemented in villages with health centers (direct and indirect treatment).
The third model observed the effect of monitoring programs in schools that were nested within
the treatment villages (villages with health centers). Finally, we compared the direct treatment
effects on health outcomes across the three regions where the monitoring program was
implemented (effect of monitoring programs in villages with health centers in each region on
child sickness).
ESTABLISHING BASELINE EQUIVALENCE FOR THE TREATMENT AND CONTROL GROUP
To determine if the treatment and control groups for health facilities were similar after
randomization in terms of covariates and outcomes of interest, we tested the statistical
significance of the difference between average proportions across treatment and control groups at
baseline. The difference in average proportions between the treatment and control groups were
tested at village level. As expected, there was no significant difference in the baseline
characteristics between the treatment and control groups where the community monitoring
intervention was directly implemented (see table 1). Table 2 exhibits baseline similarity between
treatment and control villages that were in proximity of the health facilities where the
intervention was directly implemented. The only exceptions were the proportion of male children
and the proportion of households with a personal link to the village chief, which were
significantly different in treatment and control groups.
14
TABLE 1.
Baseline Children Characteristics at Village Level for Treatment and Control Group (Direct)
Treatment Control Control-Treatment
Proportion of village meeting participation 0.90 0.92
0.02
(0.04)
Proportion of male children 0.53 0.54
0.004
(0.02)
Proportion of HHs receiving APE/AME/COGES services
0.37 0.44
0.07
(0.05)
Proportion of HHs with personal link with village chief
0.93 0.92 -0.01
(0.04)
Proportion of HHs with personal link with APE member
0.70 0.69 -0.01
(0.07)
Proportion of Children Sick in Past 30 days 0.11 0.07
-0.04
(0.02)
Children’s BMI z-score -0.45 -0.52
-0.07
(0.14)
Standard errors in parentheses. Significantly different than zero at 99 (***), 95 (**), and 90 (*) percent
confidence.
15
TABLE 2.
Baseline Children Characteristics at Village Level for Treatment and Control Group (Indirect)
Treatment Control Control-Treatment
Proportion of village meeting participation
0.94 0.93 -0.01
(0.017)
Proportion of male children
0.53 0.51 -0.017*
(0.01)
Proportion of HHs receiving APE/AME/COGES services
0.40 0.32 -0.08
(0.05)
Proportion of HHs with personal link with village chief
0.89 0.95 0.06***
(0.02)
Proportion of HHs with personal link with APE member
0.60 0.59 -0.01
(0.04)
Proportion of Children Sick in Past 30 days
0.066 0.061 -0.005
(0.009)
Children’s BMI z-score 0.34 -0.59 -0.93
(0.68)
Standard errors in parentheses. Significantly different than zero at 99 (***), 95 (**), and 90 (*) percent
confidence.
16
USING DIFFERENCE-IN-DIFFERENCES WHEN SAMPLES ARE RANDOMIZED
Considering the random assignment of households to treatment and control and the demonstrated
baseline equivalence, concerns on the use of difference-in-difference may arise. If households
are assigned to treatment and control groups at random, then differences between groups across
time should be attributed to the treatment. However, on the off-chance that randomization was
done inadequately or the groups were different by pure chance, then our covariates would control
for these differences and ensure precision in our estimates. We have used Miguel and Kremer’s
approach in the deworming randomized experiment to use difference-in-differences analysis to
corroborate the regression results (2004). The following section details the model specifications
and treatment effects on health outcomes; child sickness, and child body mass index.
Treatment Types (independent variables of interest) in our analysis include:
1) CMP implemented within village health centers (direct treatment);
2) CMP implemented in villages within proximity to villages with treated and/or controlled
health centers (indirect treatment); and
3) CMP implemented in schools located within villages with health centers (school
treatment).
The following section presents our regression results once covariates were added to the core
specification.
17
MODELS
Model 1a. Monitoring Program in Villages with Health Centers (direct treatment) and Child
Sickness
Model 1 identifies the relationship between the monitoring program in village health centers and
the likelihood that a child within a village household would be sick within the past thirty days.
The model takes the following form:
𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝜇𝑖
To control for potentially confounding variables, we installed the following covariates:
1. Region: Cascades, Central Plateau, and Sahel
2. Child Gender
3. Membership in a local village committees
4. Household’s personal links to the community village chief: close relative, other relative,
extended family/clan, close friend, friend of a friend, known person, none, the village
chief himself/herself
5. Household’s participation in the village meetings
6. Households’ personal links to local village committees: close relative, other relative,
extended family/clan, close friend, friend of a friend, known person, none, the village
chief himself/herself
The confounding variables were chosen based on the CMP process evaluation and the team’s
qualitative research. Regional and gender variation in the program’s effectiveness were
frequently cited as a concern in the Process Evaluation, and these concerns were corroborated by
our field research (de Briey 2015). Household’s personal links to community village chief and
village committees and membership in local village committees were included in the model to
accommodate for any elite capture or exclusion of poor or ethnic minorities, another concern
expressed in the Process Evaluation and corroborated by field research. These concerns are
discussed in greater depth in the qualitative findings.
The entirety of Model 1a. takes the following form:
𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3)
+ 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6)
+ 𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Where the dependent variable 𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 is a dichotomous variable that takes on the values 0 and
1 (1 = sick within the past 30 days, 0 = not sick within the past 30 days). 𝛽𝑡𝑖𝑚𝑒 is the time
variable that equals 0 at baseline and 1 at end-line. 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) is also a
dichotomous variable that equals 1 if the community monitoring program was implemented
18
directly (within villages with health centers), and equals 0 for control villages (villages with
health centers that did not have a community monitoring program). 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)
is the time-
treatment interaction variable; our independent variable of interest. Dichotomous variable of
𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 designates the child’s gender, 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒 represents membership within a
village committee and 𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀 indicates a household’s participation in village meetings.
We also applied this model to our second health outcome of interest (child’s body mass index);
this model takes the following form:
Model 1b.
𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) +
𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +
𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Model 1a. Results:
The community monitoring program in village health centers had a negative and statistically
significant (10% significance level) effect on the likelihood that a child within a village
household has been sick within the past 30 days. Children in villages with treatment health
centers were 6.34% less likely to be sick within the past 30 days (relative to children in villages
with health centers that did not receive a monitoring program) when controlling for region, child
gender, household membership in village committees, households’ personal links to village
chief, household participation in village meetings, and households’ personal links to village
committees. Furthermore, children from the Central Plateau region were 9.44% (1% significance
level) less likely to be sick within the past 30 days relative to children living in the Cascade
Region, holding all other factors constant. Similarly, children living within the Sahel region were
also less likely to be sick within the past 30 days relative to children living in the Cascades
Region (6.93% less likely; 1% significance level), assuming ceteris paribus. Child gender,
membership in village committees, households’ personal link to village chief, household
participation in village meetings, and households’ personal links to village committees did not
have statistically significant effects on child sickness.
Model 1b. Results:
The community monitoring program in village health centers did not have a statistically
significant effect on child body mass index; however, this model did reveal that children from
the Central Plateau and Sahel regions had lower body mass index for their age relative to
children in the Cascades region (Children in Central Plateau were 27.5% less likely to be sick
within the past 30 days relative to children in the Cascades; Children in Sahel were 35.4% less
likely to be sick within the past 30 days relative to children in the Cascades). (Note: body mass
index variable coded as body mass index for age z-scores. Full model can be found in the
Appendix).
19
Model 1 Interpretation:
The results from the two models present a contradiction with a significant drop in child sickness
but no profound effect on child BMI. Limitation of the BMI metric offers a strong explanation.
BMI is used to measure excess weight rather than excess body fat (which is what we are
interested in), and, as such, it is unable to identify whether the excess weight is attributed to fat
or muscle mass and bone density. Another plausible explanation is that many health
improvements from the program may have been produced from changes outside the health
center. For instance, communal building of wells through the program could diminish the
likelihood of child sickness by reducing the physical stress borne by children through strenuous
and labor intensive water collection.
We now proceed to our second model which compares the effect of community monitoring
programs (on the same two health outcomes in Model 1) on villages that did not have a health
center but were within proximity to a village with a health center that received the community
monitoring program (indirect treatment villages) relative to villages that also did not have a
health center but were within proximity to villages with health centers but did not receive the
monitoring program treatment (indirect control villages). The purpose of Model 2 is to identify
whether there are any spillover effects of the community monitoring program on nearby villages
with no health centers.
Model 2a.
𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑖𝑛𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +
𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +
𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Model 2b.
𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑖𝑛𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +
𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +
𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Model 2a. Results:
Model 2a. found that the community monitoring program had a negative and statistically
significant effect (at the 5% significance level) on the likelihood of a child being sick within the
past 30 days. Children within indirectly treated villages were 3.67% less likely to be sick within
the past 30 days relative to children within indirectly controlled villages (after controlling for
20
location, child gender, membership in village committees, personal links with village chief,
participation in village meetings, and personal links with village committees). Note that among
indirectly treated/controlled villages, households with members that had a role in village
committees had a higher probability of having a child sick within the past 30 days (1.23% more
likely; 10% significance level). This is not to say that having membership ties with village
committees increases the chances of a child being sick but that these households may be more
likely to report their child’s health status relative to households with no members having a role in
village committees.
Model 2b. Results:
Model 2b. looked at relationship between indirect treatment and whether children’s body mass
index, again we did not find a statistically significant relationship between the two.
Model 2 interpretation:
Model 2 results for both health outcomes are similar to those from Model 1. The indirect
treatment had a negative and statistically significant effect on child sickness but had no
statistically significant effect on child body mass index. The results in Model 2 conform to our
expectation of indirect treatment bringing less profound outcomes in contrast with the direct
treatment. However, the key takeaway point is that the community monitoring program in
villages with health centers did have a spillover effect on villages without health centers nearby.
Model 3 aims to identify whether community monitoring programs in schools influenced child
health outcomes. Here we looked at differences between schools with and without the
community monitoring program in villages with health centers.
Model 3a.
𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑠𝑐ℎ𝑜𝑜𝑙 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑠𝑐ℎ𝑜𝑜𝑙𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +
𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +
𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Model 3b.
𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑠𝑐ℎ𝑜𝑜𝑙 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑠𝑐ℎ𝑜𝑜𝑙𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +
𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +
𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖
Model 3 Results and Interpretation:
Community monitoring programs in schools did not have a statistically significant effect on any
of the two child health outcomes. The community monitoring program serves as a performance
litmus test to public service providers in the institution in which it was implemented. Here we
speculate that it is unlikely that a monitoring program in schools, where it is probable that only
21
school administrators are present, would induce changes in health outcomes rather than academic
outcomes.
Table 3. below briefly summarizes our regression results for our outcomes of interest before and
after controlling for covariates.
TABLE 3.
Regression Outputs with No Control and All Controls
Dependent Variable No Controls All Controls
Child Sick in Past 30 days (Direct Health Treatment)
-0.0613* -0.0634*
Child BMI (Direct Health Treatment)
-0.0597 -0.0699
Child Sick in Past 30 Days (Indirect Health Treatment)
-0.0370** -0.0367**
Child BMI (Indirect Health Treatment)
-0.734 -0.667
Child Sick in Past 30 Days (School Treatment)
-0.0101 -0.00680
Child BMI (School Treatment) -0.108 -0.149
*** p<0.01, ** p<0.05, * p<0.1
22
COMPARISON OF PROGRAM IMPACT ON CHILD SICKNESS ACROSS REGIONS
Comparing the effect of direct treatment across regions on whether the child has been sick in the
past 30 days (Note: values obtained by running regression Model 1a. with a restriction on
“regions”), we found that the treatment in health centers in Central Plateau and Sahel did not
have a significant effect on whether the child was sick in the past 30 days. However, the
treatment effect was significant in Cascades at the 5% level. The likelihood of the child getting
sick in the Cascades region decreased by 16% over the course of intervention.
We also compared the effect of indirect treatment across regions on whether the child has been
sick in the past 30 days. Here, we found that the effect of indirect treatment in health centers did
not have a significant effect in on whether the child was sick in the past 30 days in Cascades or
Central Plateau. This indicates that there were no significant spillover effects of the treatment on
child sickness in proximate villages within these two regions. However, the treatment effect was
significant in the Sahel region at 5% level. The likelihood of the child getting sick (in the past 30
days) in the Sahel region decreased by 8.11% over the course of intervention.
COMPARISON OF PROGRAM IMPACT ON CHILD HEALTH ACROSS GENDER
We found statistically insignificant direct treatment effects for the children’s BMI for both boys
and girls in the study. However, we also found that the program had a significant effect in
lowering the likelihood of the boys being sick in the past 30 days, but no significant effect on
girls being sick the past 30 days. Similarly, indirect treatment effects for the children’s BMI were
insignificant for both boys and girls, but the program had a significant indirect effect in lowering
the likelihood of the boys being sick, but not girls. The differential impact of the program on
child sickness based on gender might point to a broader problem of intra-household gender
inequality.
23
QUALITATIVE FINDINGS
To reiterate the research purpose, the GWU team’s quantitative research was restricted to
answering the following questions:
What are the treatment effects of the CMP on child health outcomes?
Are there gendered differences in the Program’s outcomes or mechanisms?
This section will highlight key qualitative findings with reference to the following pivotal
research questions:
What contextual factors influenced the Program’s effectiveness?
What are the perceptions of social accountability and citizen engagement among Burkinabe?
How do social accountability approaches fit into the larger political and historical context of
Burkina Faso? How can community monitoring interventions at the grassroots level be
complemented with social accountability at higher tiers of government?
How can social inclusion be better integrated with community monitoring interventions to
improve voice and empowerment across the community?
COMMUNITY MONITORING PROJECT-SPECIFIC THEMES
SUCCESS
We received unequivocal support for success of the Project from participants, but the extent of
this success varied. The participants’ perception of success was not grounded in a concrete
evaluation of success, rather on field visits, interaction with the treatment communes, and review
of monitoring reports.
When discussing the Project’s success,
participants noted improvements in infrastructure
in schools and health centers, community
awareness on accountability and citizen’s rights,
and behavior change. Behavior change, or a
change in “attitudes and mentality”, was
frequently cited by participants as a metric of
success for the Project. Prior to the Project,
citizens believed their ability to affect change in
service delivery was limited, but the CMP
engendered an understanding that community
members can work collectively to make
improvements without over relying on the local
government. This newly found self-efficacy was reflected in building walls around schools,
installing solar panels, increasing school enrolment, improving teachers’ housing, etc. Although
“The community was able to assess
problems in health and education.
Project wasn’t successful in making
the connection between making the
problems identified at community
level and action taken by
authorities. It’s one thing to have
the village action plan, but it’s
another thing to have those plans
enforced/implemented.”
24
the process evaluation undertaken by the World Bank highlighted the many improvements made
in communes (de Briey 2015), the emphasis on behavior change and self-efficacy was not
revealed until the GWU team’s interviews. Despite these positive changes undertaken by the
communities themselves, the participants were mindful of the inability of communities to
accomplish more substantive changes, such as the training or hiring of health workers, which
required greater input from the government. Thus, the development of community action plans
by both members of the community and service providers did not result in full implementation of
the plan. This discrepancy is further detailed in the following sections.
REGIONAL VARIATION
The process evaluation and quantitative component of this evaluation revealed that the Project
saw the greatest success in Cascades, followed by Central Plateau and the Sahel (de Briey 2015).
Our field research substantiated the finding that success varied by region, but the order of
regional success is less certain. Participants mostly considered their native regions most
successful. Although earlier findings suggested that Cascades performed the best, participants
would often consider Cascades and Central Plateau at par. In the process evaluation, the Sahel
region was considered to have achieved limited success, but a representative of a partner NGO in
the region thought the Project was a great success. We hypothesize that these discrepancies may
be originating from limited access to the other regions where the Project was implemented, as
well as circumstantial evidence, such as observational or anecdotal evidence. Most importantly,
participants alluded to variation across communes within the same region, which was not
captured in the process evaluation, and cannot be
ascertained by quantitative analysis. Participants would
often point out how one commune performed better than
others. These claims of variation across communes are more
credible, as they are based on field visits and interaction
with people from the various communes, in addition to
interviews with members of the CMP Pilot Committee who
oversaw CMP implementation.
According to participants, these variations were a result of differences in social context,
frequency of consultation meetings, engagement of local actors, quality of the implementing
NGO, and prior exposure to development programming. A further exploration of these factors
can add value to future projects in this area.
COMMUNITY SCORE CARDS
The community scorecard (CSC) was the mechanism employed in this Project to mobilize
communities to diagnose and address problems in education and health service delivery. There
was consensus that CSC was an effective, easy to use, and participatory tool that brought
service providers and users together. Two partner NGOs claimed that CSC was so useful that
they continue to use it in similar contexts. However, it must be noted that most of the
OUR FIELD RESEARCH
SUBSTANTIATED THE FINDING
THAT SUCCESS VARIED BY
REGION, BUT THE ORDER OF
REGIONAL SUCCESS IS LESS
CERTAIN.
25
interviewees used the CSC for the first time under the CMP. Also, the favorable view of the CSC
can be treated merely as perception since very few interviewees had used other tools and lacked
a reference point to compare the CSC with.
CHALLENGES
An exploration of the challenges of the Project corroborated findings from the process evaluation
and further exposed the crucial constraint of the lack of leverage amongst local communities
with higher levels of government. As highlighted by IEG’s qualitative research, procurement
delays, a short project timeline incompatible with behavior change processes, and a lack of
capacity among implementing partners were again cited by participants as challenges (de Briey
2015).
This finding provides an impetus for a deeper inquiry into social accountability processes, not
just at the local service delivery level, but at the higher levels of government. Following sections
delve into this further.
FUNDING
There was a discrepancy between the World Bank’s conceptualization and the perception of
some stakeholders that were interviewed about project financing. Although the intention of this
Project was to make communities self-reliant to improve health and education service delivery,
some interviewees expected the World Bank to finance the implementation of community action
plans. This demand had its origins in difficulty of mobilizing financial resources to realize the
action plans. There was also confusion about the Project’s funding scheme. For example, a
government agency tasked with monitoring the implementation of the action plans expected to
be compensated for their field visits, while ISSP considered the field visits to be the budgetary
responsibility of the agency. Other funding constraints cited include an extension of the Project’s
timeline without a commensurate cost extension, and delays in delivering funding to
implementing NGOs.
SUSTAINABILITY
Concerns about the CMP’s sustainability were raised during our study. Although participants
cited the increased self-efficacy in solving problems in the commune as the key success of the
Project, many doubted that these changes could be sustained because of a cessation of the
consultation frameworks and sensitization meetings that reinforced the citizen engagement and
social accountability mechanisms. Only one participant expressed greater understanding of the
“…But if they need a health agent in the center, this is now the national level responsibility to
bring the health agent in this center. At this level it slows down. At the regional level,
national level, it’s true that they have a pilot committee for the project, but this committee
does not have full means to respond to all the needs of the population.”
26
Project’s conceptual framework by suggesting that the gains could last if the existence of
community groups was capitalized by well-intentioned community leaders.
CONTEXTUAL THEMES
This section details the contextual themes relevant to the Project, with a focus on social
accountability at both the local service delivery unit and higher levels of government.
SOCIAL ACCOUNTABILITY CHANNEL
The channel of accountability in Burkina Faso’s government is understood by most participants
in the same way as detailed by the Project’s conceptual framework, with hierarchical linkages
originating from the village, to the communal, regional, and national levels. In discussing local
governments, most participants understood the channel of accountability to originate from APE,
AME, and COGES, the organizations that serve as the first point of contact, to the municipal
councils, and then to the mayors at the communal level of government.
Considering these apparent linkages between the different levels of government, it is essential to
explore the state-citizen connection to gain greater insight into how social accountability unfolds.
Varied perceptions of the government’s responsiveness to citizens’ needs emerged. For instance,
some participants viewed a new decree providing free healthcare at public health facilities to
children between ages of zero and five as a sign of increased commitment of the government. In
contrast, some participants disputed this claim when highlighting the government’s massive
shortcomings in equipping health facilities and the long processing times for fulfillment of
requests. The participants were generally cognizant of the resource constraints behind the
shortcomings, but some considered a lack of political will and informational asymmetry as
culpable factors.
The service providers attributed these shortcomings to severe resource constraint and inadequate
resource allocation from the central government.
Variation in mayors’3 perception of social accountability mechanisms also provides a glimpse
into the attitudes of public officials. One mayor was open to social accountability mechanisms
like the CMP because it facilitated greater contact with the citizens he is tasked with serving,
3 The head of the communal level of government, the mayor, is indirectly elected by the municipal council. The
communal level of government is responsible for the management of most basic socioeconomic infrastructure: rural
roads, health facilities, schools, water and sanitation.
27
whereas the other mayor welcomed the CMP because people took initiative to address challenges
and would not bother him for menial tasks.
CENTRAL GOVERNMENT
With respect to social accountability at the higher levels, sluggishness in the central government
bureaucracy was unanimously asserted by participants. Barring one participant, there was no
reference to any internal mechanism for social accountability. The unique reference pointed out
the poor functioning of that social accountability channel, but the exact nature of that channel
requires greater research. This failure was understood partially in the context of resource
constraints and lack of political will. Participants understood and acknowledged the resource
constraints faced by their government and the country’s over reliance on foreign aid. One local
government official revealed seeking assistance from French donors after several unanswered
requests by the central government for water pumps in Zitenga. To resolve this situation,
additional connections across the government hierarchy via greater communication were
suggested. The CMP was viewed as a crucial step in that direction. Another government
representative advocated for instating independent monitoring committees at each level of
government, citing an ongoing UNICEF project as an example.
Most participants acknowledge that channeling citizen engagement to foment social
accountability is an atypical concept in Burkina Faso, but there was consensus among the
participants on the need for building social accountability as a means of improving the
government’s responsiveness. Many contextually-appropriate suggestions detailed below form
the backbone of our findings on social accountability. It is important to acknowledge that many
of these suggestions emerged out of the participants’ experience of the popular uprising in 2014.
The forces that were said to have catalyzed the popular uprising were cited as actionable
recommendations. The relevance and importance of the popular uprising to the Burkinabes
manifests in the participants’ understanding of events or experiences as pre- and post- uprising.
Strong Civil Society Organizations
Local NGOs and civil society organizations, such as religious groups and youth groups, were
viewed as a potent force in triggering social accountability. Participants pointed out that the
presence of local NGOs in a community puts pressure on local governments to perform.
Indeed, the involvement of local organizations in the development of community action plans
was a means of inducing social accountability by encouraging transparency, and the role of
implementing NGOs in the CMP was considered the distinguishing factor for success of the
Project between treatment and control by one participant. Many participants attributed the
success of the popular uprising in 2014 to a strong civil society, further underscoring the
perception of these organizations as drivers of social accountability.
Donors/External actors
Participants expressed that donors could play an integral part in engendering social
accountability in Burkina Faso. The conditions that their money comes with imply greater
28
power to create opportunities for social inclusion through an emphasis on grassroots
participation and more transparency.
Media
The role of media, or radio, in particular, was emphasized as a useful tool for social
accountability. Media can be used to call out the local government for its inaction or
communicate concerns which may not otherwise reach or influence governors. Radio is
considered the best medium in Burkina Faso because the high rate of illiteracy precludes the
saturation of print media.
Taxes
Resource constraints of the local government was often seen as a basis for many of its
shortcomings. Tax evasion in communities was cited as a serious concern. Interviewees
proposed building awareness and mobilizing communities to pay taxes to address this
resource deficiency and provide a direct channel of accountability.
Dissemination of Information
Participants also cited greater circulation of the national and regional development plans as
an impetus for inclusion of communal and village concerns, as well as a greater
understanding of the government’s priorities.
Protest
Protest was considered a powerful tool to effect change. This was demonstrated by the
success of the 2014 popular uprising. Consequently, strikes have become more frequent
throughout the country in protest of government policies, wages, etc.
Decentralization
The findings on local government and central government’s shortcomings were closely
linked with the decentralization process in Burkina Faso. The interviewees pointed out that
the decentralization process in the country is still new, and needs improvement. Although
some participants criticized the government, interviewees unanimously demonstrated an
understanding of the resource constraints of the government. Inefficient transfer of resources
from the central to the local government, and an inadequate devolution of power were also
emphasized. Poor service delivery in health and education was deemed a manifestation of the
flawed decentralization process. In contrast, participants called attention to APE and COGES
as the functioning positive elements of decentralization, with APE performing better than
COGES.
Role of NGOs and Donors
The role of NGOs was largely seen in a positive light. They were regarded as integral actors
in ensuring social accountability. A majority of participants highlighted that NGOs could
ensure accountability by checking whether the local government’s promised improvements
for the community were made. Additionally, NGOs were viewed as mediators between
communities and governments. The NGOs had the influence to raise important issues as their
29
voice could reach the highest level of government, contrary to communities who could not
access the national government directly.
In most cases, NGOs were applauded for complementing government actions. For instance,
NGOs were cited as sensitizing communities to increase service utilization, such as
increasing school attendance or visits to the health center. A mayor appreciated an NGO’s
role in being able to raise awareness of the importance of paying taxes in his commune.
However, a representative from the implementing NGO in the Sahel raised the concern that
there was very little contact between local government and NGOs. The local government
often lacked the will to work in concert with the NGOs. This finding signified some regional
variation in performance of local government and the relationship between NGOs and
governments. However, a deeper exploration into this possible regional variation is beyond
the scope of this study.
There was strong agreement on the need for donors in Burkina Faso. The stakeholders
appreciated the financial resources and technical expertise donors provided. The donors were
particularly seen to be instrumental in encouraging social accountability by conditioning how
the money would be spent and making aid contingent on involvement of local populations.
Some stakeholders expressed concerns and suggested that external donors needed to ensure
transparency and prevent elite capture by giving grants in public forums.
SOCIAL INCLUSION
Inclusion emerged as a central theme in our qualitative research because evaluation of the CMP
and its inclusion of all members of communities is insufficient. It must be noted that some of the
“The role of the organizations and NGOs in communities is social
checking—social accountability, and checking activities.
Everyone now is ready to ask the mayor when we need the
school. When they build a school, they will now check to see if
they bring people—it is not just the building that is needed. So
we can see what we can do for people so that people can to
come to that school. If some people say that they need a health
center and they get the building or a health agent, now NGOs
have to go check out if they are sick; if there is a health
campaign they should do in this community. NGOs should raise
the awareness of everyone to hold those accountable at all
levels of the community. The role of the NGOs also is to raise
awareness of taxes in communities…”
30
participants were not able to make specific comments on the Project’s inclusion, but relied on
other experiences in their community as reference.
Some participants expressed concerns on
class-based or kinship-based exclusion in the
Project and general community decision-
making processes. A participant conveyed
that wealthier groups who have emigrated
from the village still tend to exert ample
influence in decision-making processes in
their villages of origin. In contrast to this
general view of communal decision-making, a
participant from an implementing partner
voiced his concern that the CMP did not
account for the possible differences in
participation based on socioeconomic classes,
and may have excluded some groups because of an over-reliance on the local bodies of COGES
and APE. Representatives of some local NGOs and a youth group highlighted that membership
of these local structures may be inclusive on paper, but kinship ties play an important role in
membership. As the leadership of COGES and APE is elected, the majority ethnic group in the
community is likely to control these organizations. A participant cited that in a village of
majority Mossi people, only Mossi people will be part of these organizations and the local
government.4 This practice appeared to be pervasive, affecting women’s groups and youth
associations, as well. However, an implementing partner suggested that ethnic exclusion did not
afflict the Project because the implementation communities were largely homogenous. Another
implementing partner asserted that people at the grassroots level were not represented in the
higher tiers of government, as adequate representation from local communities was not sought.
Variations in responses between implementing NGOs and stakeholders, and the local
government was seen. The local government stakeholders would often maintain the use of an
inclusive Project strategy, whereas implementing NGOs would be quick to acknowledge the
Project’s gaps in inclusion.
4 The Mossi people are the largest ethnic group in Burkina Faso, primarily residing in the central regions of Burkina
Faso.
“... Other minority groups must be included too in
village development council. We can have six
organizations in a village (which are supposed to
have 2 leaders per organization), but we see only
1 leader per organization. Even youth association
members have ties to village chief. Even women
association leaders are wives/daughters of village
chief.”
31
GENDER INCLUSION
Gender inclusion emerged as a strong contextual factor within the CMP and greater Burkina
Faso. Our discussion on gender inclusion revolved around CMP mostly as it was the participant's
primary reference point. In our investigation of female involvement in the Community
Monitoring Project, a simple question inquiring if the Project was inclusive would mostly
generate a solid “yes”. Further probing on inclusion of women would then result in a
contradictory response revealing that women were not active participants in meetings. This
contradiction is possibly rooted in a flawed perception of inclusion, equated with participation
with little regard for voice and agency. More inquiry into whether women were vocal in CMP
meetings or adequately voiced their concerns would reveal this discrepancy. After multiple
probes, some participants pointed out that even if women were given a platform, they would lack
the courage to speak. Study participants considered restrictive gender norms to be the key
problem. The norms highlighted in our study are listed below:
● Women need to seek husband permission.
● Women’s involvement in public affairs is looked down upon by in-laws.
● Women do not have the right to speak in the Mossi culture.
● Women were considered inferior to men. Women have internalized this value and
therefore step back in matters of public or communal decisions.
● Women fear participation.
This finding substantiated our earlier assertion of a lack of agency precluding women from equal
participation in the CMP. A mayor pointed out that more women came to him with concerns
because they were the most vulnerable members of the community. This lack of agency would
then lead to a vicious cycle, where less attention or understanding of women’s issues would be
reflected in poorer health and education outcomes amongst women.
Many participants alluded to the lack of an even playing field for women and a strong
internalization of restrictive norms by women themselves. Project stakeholders had some
suggestions to encourage women’s participation and agency. These recommendations are
highlighted below. Note that we do not endorse these recommendations, but are merely stating
the stakeholders’ opinions.
● Gender sensitization trainings before the start of projects to encourage women and get
men on board.
● Gender segregation of consultation groups could also give women more power to voice
concerns.
● Use of women leaders in community.
32
33
LIMITATIONS
DEPENDENT VARIABLE SELECTION
Our dependent variables of interest were health outcomes; however, the treatment was
not a health treatment. As such, there may have been intermediate outcomes “in-
between” which were not analyzed. Therefore, we are unable to identify the channels
through which the treatment was able to improve health outcomes such as child sickness.
EXTERNAL VALIDITY
Our analysis of the treatment effects on child health outcomes cannot be generalized to
the regions and communes that were not selected for the study, given that our field
research revealed regional and communal variations.
LIMITED ACCESS TO PROJECT BENEFICIARIES
Limited access to beneficiaries meant that we could not verify the results for the Project’s
effectiveness from those directly affected.
SOCIAL DESIRABILITY BIAS
Because our sample for the field research comprised primarily of stakeholders involved
in the Project’s management in some capacity, there was a tendency to overstate the
Project’s success.
34
CONCLUSION
We cannot unequivocally conclude that the Burkina Faso Community Monitoring Project was
successful in achieving its intended outcomes. Our quantitative analysis merely presents
evidence of the Project’s effectiveness on dual child health outcomes of child sickness and BMI,
but does not capture the causal chain through which these outcomes were achieved. Our field
research alludes to the intermediate outcomes by highlighting the Project’s marked contribution
to improved self-efficacy and empowerment of communities; however, these changes may not be
sustainable outside of the Project’s framework.
Regardless of the level of success of the CMP, some glaring shortcomings were identified. The
participation of women in the Project was lacking, with little improvement in women’s voice and
agency. Societal power hierarchies may have excluded poor and ethnic minorities, leading to
elite capture of the process. A lack of responsiveness from a resource-constrained government,
and a strong disconnect between the higher levels of government and citizens, impeded effective
service delivery.
By and large, there was a consensus on the need to encourage social accountability and citizen
engagement processes. To accomplish these processes, the participants recommended the use of
a strong civil society, pressure on the government from local NGOs and donors to perform,
dissemination of information through media, wider tax base, and a more effective and devolved
decentralization process.
35
RECOMMENDATIONS
The Project’s theory of change needs to clearly highlight the problems that the intervention aims
to address with the inputs, immediate outputs, and intended short-term and long-term outcomes.
Identifying the specific problem to focus on for each sector is important because the policy
prescriptions that stem out of each problem are different. Specific social accountability indicators
are necessary to capture the theory of change more effectively.
Citizen engagement that triggers demand-side accountability processes has to go in tandem with
building state capacity to interact with and respond to these new accountability pressures. We
advise introducing community monitoring as part of a broader package of services, well-
integrated with the national development framework of Burkina Faso to engender state capacity
and buy-in.
Community monitoring interventions require a greater timeline to accommodate for slow
behavioral and attitudinal changes, and their impact on service delivery outcomes.
Social inclusion must be built into the design of community monitoring interventions from
inception. An ad hoc treatment later in the project cycle risks negating the project outcomes and
gains in other areas. The impact of citizen engagement projects depends on the inclusivity of all
the voices in the target population.
36
BIBLIOGRAPHY
Ackerman, J.M. (2005). Social Accountability in the Public Sector: A Conceptual Discussion.
Washington, D.C.: World Bank.
Baldwin, Kate, et al. "Does Community-Based Development Empower Citizens? Evidence from
a Randomized Evaluation in Ghana." J-PAL, Oct. 2016, Accessed 25 Jan. 2017.
Brinkerhoff, Derick W., and Anna Wetterberg. "Gauging the Effects of Social Accountability on
Services, Governance and Citizen Empowerment." Public Administration Review 76, no. 2
(2015): 274-86. Accessed May 12, 2017. doi:DOI: 10.1111/puar.12399.
Corbridge, S., G. Williams, M. Srivastava, and R. Veron. 2005. Seeing the State: Governance
and Governmentality in India. Cambridge: Cambridge University Press.
Edstrom, Judith. "Engaging Citizens in Health Service Delivery: A Review of the Literature."
Partnership for Transparency Fund, USAID, 31 Aug. 2015, ptfund.org/engaging-citizens-
in-health-service-delivery-a-literature-review/.
Fox, Jonathan. 2014. Social Accountability: What Does the Evidence Really Say?
Global Partnership for Social Accountability Working Paper no. 1, World
Bank.
Geddes, B. 1994. Politician’s Dilemma: Building State Capacity in Latin America. Berkeley:
University of California Press.
37
“Grandvoinnet, Helene; Aslam, Ghazia; Raha, Shomikho. 2015. Opening the Black Box : The
Contextual Drivers of Social Accountability. New Frontiers of Social Policy;. Washington,
DC: World Bank. © World Bank.
https://openknowledge.worldbank.org/handle/10986/21686 License: CC BY 3.0 IGO.”
Healey, F. Henry, Derick W. Brinkerhoff, and Jana C. Hertz. "Social Accountability in
Education: The Northern Education Initiative in Nigeria." In Governance and Service
Delivery: Practical Applications of Social Accountability across Sectors., edited by Anna
Wetterberg. (RTI Press Publication No. BK-0019-1609). Research Triangle Park, NC: RTI
Press. DOI: 10.3768/rtipress.2016.bk.0019.1609
"Making Politics Work for Development: Harnessing Transparency and Citizen Engagement."
Open Knowledge Repository, openknowledge.worldbank.org/handle/10986/24461.
Mohanty, R (2007) ‘Gendered Subjects, the State and Participatory Spaces: The Politics of
Domesticating Participation in Rural India’, in A. Cornwall and V. Schattan Coelho (eds),
Spaces for Change? The Politics of Participation in New Democratic Arenas, London and
New York: Zed
Bjorkman, M., and J. Svensson. 2007. “Power to the People: Evidence from a Randomized
Field Experiment of a Community-Based Monitoring Project in Uganda.” Policy Research
Working Paper 4268, World Bank, Washington, DC.
38
de Briey, Valerie. Projet D’appui Au Suivi Evaluation Participatif Des Projets De La Banque
Mondiale Dans Les Secteurs Education Et Sante Au Burkina Faso. Washington DC:
Independent Evaluation Group at World Bank Group, 2015.
Duflo, E., P. Dupas, and M. Kremer. 2008. “Peer Effects, Pupil-Teacher Ratios, and Teacher
Incentives: Evidence from a Randomized Evaluation in Kenya.” Online Working Paper
CCPR 055-08, California Center for Population Research, University of California, Los
Angeles.
Freedom House. 2016. “2016 Freedom in the World - Burkina Faso.”
https://freedomhouse.org/report/freedom-world/2016/burkina-faso.
Gaventa, J., and G. Barrett. 2010. “So What Difference Does It Make? Mapping the Outcomes
of Citizen Engagement.” IDS Working Paper 347, Institute of Development Studies,
Brighton, United Kingdom.
Malena, C., and M. McNeil. 2010. “Social Accountability in Africa: An Introduction.” In
Demanding Good Governance: Lessons from Social Accountability Initiatives in Africa,
edited by C. Malena and M. McNeil. Washington DC: World Bank.
Manandhar, D. S., D. Osrin, B. P. Shrestha, N. Mesko, J. Morrison, K. M. Tumbahangphe, and
S. Tamang. 2004. “Effect of a Participatory Intervention with Women’s Groups on Birth
Outcomes in Nepal: Cluster Randomised Controlled Trial.” Lancet 364: 970–79.
39
Miguel, Edward, and Michael Kremer. "Worms: Identifying Impacts on Education and Health in
the Presence of Treatment Externalities." Econometrica 72, no. 1 (2004), 159-217.
doi:10.1111/j.1468-0262.2004.00481.x.
Molina, Ezequiel, Laura Carella, Ana Pacheco, Guillermo Cruces, and Leonardo Gasparini.
"Community monitoring interventions to curb corruption and increases access and quality
of service delivery in low-and middle-income countries: a systematic review." November
2016. Accessed January 25, 2017.
file:///C:/Users/sarut_000/Downloads/0150_IDCG_Molina_Community_Monitoring_Final
Tripathy, P., N. Nair, S. Barnett, R. Mahapatra, J. Borghi, S. Rath, and S. Rath. 2010. “Effect of
a Participatory Intervention with Women’s Groups on Birth Outcomes and Maternal
Depression in Jharkhand and Orissa, India: A Cluster-Randomised Controlled Trial.”
Lancet 375: 1182–92.
Nguyen, T., and M. Rieger. 2014. “Community-driven development and social capital: evidence
from Morocco.” EUI Working Paper MWP 2014/02
O’Meally, S. 2013. “Mapping Context for Social Accountability: A Resource Paper.”
Background for this book, World Bank, Social Development Department, Washington,
DC.
40
Stokes, S (2003) “Perverse Accountability: Monitoring Voters and Buying Votes.” Paper
presented at the 2003 Annual Meeting of the American Political Science Association,
August 31.
United Nations Development Programme. 2015. “Burkina Faso Human Development
Indicators.” http://hdr.undp.org/en/countries/profiles/BFA.
Varshney, Ashutosh (1999) “Democracy and Poverty,” paper for the Conference on World
Development Report 2000, organized by the U.K. Department for International
Development, and the Institute of Development Studies, Sussex, Caste Donnington,
England, August 15-16, 1999.
Westhorp, Gill, Bill Walker, and Patricia Rogers. 2012. Under What Circumstances
Does Enhancing Community Accountability and Empowerment Improve Education
Outcomes, Particularly for the Poor? A Realist Synthesis. London: University of
London, Institute of Education, Social Science Research Unit, Evidence for
Policy and Practice Information and Co-ordinating Centre.
Wetterberg, A., Brinkerhoff, D., & Hertz, J. (Eds.) (2016). Governance and service delivery:
Practical applications of social accountability across sectors. (RTI Press Publication No. BK-
0019-1609). Research Triangle Park, NC: RTI Press. DOI:
10.3768/rtipress.2016.bk.0019.1609
41
World Bank. 2016. “Burkina Faso Overview, World Bank, Washington, DC, October 11.
http://www.worldbank.org/en/country/burkinafaso/overview.
World Bank. 2013. “Burkina Faso - Country partnership strategy for the period FY2013-2016.”
World Bank, Washington DC.
http://documents.worldbank.org/curated/en/937251468005970222/Burkina-Faso-
Country-partnership-strategy-for-the-period-FY2013-2016
World Bank. 2012. “Strengthening Governance, Tackling Corruption: The World Bank’s
Updated Strategy and Implementation Plan.” World Bank, Washington, DC, March 6.
http://siteresources.worldbank.org/PUBLICSECTORANDGOVERNANCE/Resources/
285741-1326816182754/GACStrategyImplementationPlan.pdf
42
APPENDIX
INTERVIEW GUIDES
Interview Guide 1: Beneficiaries/ citizens in the field
Citizens Engagement:
1. Do you think your local government (name the lowest tier) is responsive to your
community’s needs? (Note: response should not be dichotomous)
a. Why or why not?
2. Who is allowed to do the following?
a) Own land
b) Speak to public officials
c) Speak in community meetings.
d) Vote
e) Seek jobs
3. Can everyone talk to government officials? Can all kinds of men talk to government
officials? What are the exceptions?
4. Can women talk with government officials? If not, why? Are there any exceptions?
5. What are the organizations that bring people together? (Probe: Community based
organizations, cooperatives, kinship groups, religious organizations, etc)
How do they interact with government on behalf of the community?
6. How are decisions made within your community? For instance, if a community is faced
with a problem, like a dysfunctional health center and/or school, who is responsible to make
decisions about dealing with such issues? What is the process for such decisions?
Probe: Who makes the key decisions like these? (Is it a group of people or a single
individual?)
7. What types of people speak or participate in community meetings? Why?
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
8. What types of people do not speak or participate in community meetings? Why not?
43
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
9. Does your community have people from many different ethnic backgrounds? If so, how
does that affect your ability to work together or to come to an agreement?
NGOs, CSOs and Multilaterals:
1. How can communities work together with local government bodies to improve their
health or education?
2. Do you think that local governments should be held responsible for their performance
by the community or their constituents? Have you seen this in your community? Give examples.
3. What impact do you think your/ the local government has in improving healthcare or
education? Has the progress made so far been satisfactory?
4. What impact do you think the NGOs have had in improving healthcare or education? Has
the progress made so far been satisfactory? (probe: local or international)
5. Do you think social accountability at higher tiers of government can increase the
effectiveness of community monitoring interventions in Burkina Faso?
a. Why or why not?
6. What are the strong points and challenges for the local and central government to
coordinate health and education service delivery? Why or Why not?
7. Do you think that multilateral institutions (e.g. World Bank) can help the communities in
holding service providers responsible in any way? Suppose that you are in local clinic or CSPS,
the health attendants are rarely present, how can multilaterals help you to improve their
performance?
44
8. Do you think that local government, multilateral institutions and local NGOs should work
together to improve health and education service delivery?
9. How should the government, multi-lateral institutions and local NGOs work together to
improve health and education service delivery in Burkina Faso?
10. Can you think of a case where decentralization and community empowerment worked
well to improve health and education service delivery? Give examples. (Probe using a context-
specific example.)
11. In your opinion, how can social accountability approaches include all citizens? (Probe:
Which approaches come to mind such as community score cards, social audits, grievance
redressal).
45
Interview Guide 2: NGOs, CSOs, Multilaterals
Citizens Engagement:
1. Do you believe the local government in Burkina knows about your concerns about poor
health or education? And do they address those concerns as you wish?
2. Who is allowed to do the following?
a) Own land
b) Speak to public officials
c) Speak in community meetings.
d) Vote
e) Seek jobs
3. Can everyone talk to government officials? Can all kinds of men talk to government
officials? What are the exceptions?
4. Can women talk with government officials? If not, why? Are there any exceptions?
5. What are the organizations that bring people together? (Probe: Community based
organizations, cooperatives, kinship groups, religious organizations, etc.)
How do they interact with government on behalf of the community?
6. How are decisions made within your community? For instance, if a community is faced
with a problem, like a dysfunctional health center and/or school, who is responsible to make
decisions about dealing with such issues? What is the process for such decisions?
Probe: Who makes the key decisions like these? (Is it a group of people or a single
individual?)
7. What types of people speak or participate in community meetings? Why?
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
46
8. What types of people do not speak or participate in community meetings? Why not?
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
9. Does your community have people from many different ethnic backgrounds? If so, how
does that affect your ability to work together or to come to an agreement?
NGOs, CSOs, Multilaterals:
1. Do you believe the local government in Burkina Faso is generally responsive in terms of
service delivery such as an unfit health center or a school?
2. How can communities work together with local government bodies to improve their
health or education?
3. Do you think that local governments should be held responsible for their performance
by the community or their constituents? Have you seen this in your community? Give examples.
4. What impact do you think your/ the local government has in improving healthcare or
education? Has the progress made so far been satisfactory?
5. What impact do you think the NGOs have had in improving healthcare or education? Has
the progress made so far been satisfactory? (probe: local or international)
6. How do you see the role of NGOs or CSOs like yourself in improving health and
education by directly involving communities?
7. In your opinion, what have NGOs done to assist local government perform their duties in
improving health and education?
47
8. Do you think social accountability at higher tiers of government can increase the
effectiveness of community monitoring interventions in Burkina Faso?
b. Why or why not?
9. What are the strong points and challenges for the local and central government to
coordinate health and education service delivery? Why or Why not?
10. Do you think that multilateral institutions (e.g. World Bank) can help the communities
in holding service providers responsible in any way? Suppose that your in local clinic or CSPS,
the health attendants are rarely present, how can multilaterals help you to improve their
performance?
11. Do you think that local government, multilateral institutions and local NGOs should
work together to improve health and education service delivery?
12. How should the government, multi-lateral institutions and local NGOs work together to
improve health and education service delivery in Burkina Faso?
13. Can you think of a case where decentralization and community empowerment worked
well to improve health and education service delivery? Give examples. (Probe using a context-
specific example.)
14. Do you think social accountability approaches are generally inclusive? (NGO
specific). Why or why not?
15. In your opinion, how can social accountability approaches include all citizens?
(Probe: Which approaches come to mind such as community score cards, social audits, grievance
redressal).
48
Interview Guide 3: ISSP staff or members of the CMP Pilot Committee
Citizens Engagement:
1. Do you believe the local government in Burkina knows about your concerns about poor
health or education? And do they address those concerns as you wish?
2. Who is allowed to do the following?
a) Own land
b) Speak to public officials
c) Speak in community meetings.
d) Vote
e) Seek jobs
3. Can everyone talk to government officials? Can all kinds of men talk to government
officials? What are the exceptions?
4. Can women talk with government officials? If not, why? Are there any exceptions?
5. What are the organizations that bring people together? (Probe: Community based
organizations, cooperatives, kinship groups, religious organizations, etc.)
6. How do they interact with government on behalf of the community?
7. How are decisions made within your community? For instance, if a community is faced
with a problem, like a dysfunctional health center and/or school, who is responsible to make
decisions about dealing with such issues? What is the process for such decisions?
Probe: Who makes the key decisions like these? (Is it a group of people or a single
individual?)
8. What types of people speak or participate in community meetings? Why?
49
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
9. What types of people do not speak or participate in community meetings? Why not?
Probe: (young or older men, ethnicity, older and younger women, socio economic status)
10. Does your community have people from many different ethnic backgrounds? If so, how
does that affect your ability to work together or to come to an agreement?
NGOs, CSOs, Multilaterals:
1. Do you believe the local government in Burkina Faso is generally responsive in terms of
service delivery such as a unfit health center or a school?
2. How can communities work together with local government bodies to improve their
health or education?
3. Do you think that local governments should be held responsible for their performance
by the community or their constituents? Have you seen this in your community? Give examples.
4.What impact do you think your/ the local government has in improving healthcare or
education? Has the progress made so far been satisfactory?
5. What impact do you think the NGOs have had in improving healthcare or education? Has
the progress made so far been satisfactory? (probe: local or international)
6. How do you see the role of NGOs or CSOs like yourself in improving health and
education by directly involving communities?
50
7. In your opinion, what have NGOs done to assist local government perform their duties in
improving health and education?
8. Do you think social accountability at higher tiers of government can increase the
effectiveness of community monitoring interventions in Burkina Faso? Why or why not?
9. What are the strong points and challenges for the local and central government to
coordinate health and education service delivery? Why or Why not?
10. Do you think that multilateral institutions (e.g. World Bank) can help the communities in
holding service providers responsible in any way? Suppose that your in local clinic or CSPS, the
health attendants are rarely present, how can multilaterals help you to improve their
performance?
11. Do you think that local government, multilateral institutions and local NGOs should work
together to improve health and education service delivery?
12. How should the government, multi-lateral institutions and local NGOs work together to
improve health and education service delivery in Burkina Faso?
13. Can you think of a case where decentralization and community empowerment worked
well to improve health and education service delivery? Give examples. (Probe using a context-
specific example.)
14. Do you think social accountability approaches are generally inclusive? (NGO specific).
Why or why not?
15. In your opinion, how can social accountability approaches include all citizens? (Probe:
Which approaches come to mind such as community score cards, social audits, grievance
redressal).
51
ISSP-specific questions
1. Do you think the CMP project was successful?
a. Why or why not?
2. Do you think the CMP was more successful in some regions as opposed to the other?
Why or why not?
3. Did you think the CSC was a good approach and why was it used as opposed to other
approaches? Why or why not?
4. What did you think were the most pressing challenges in the implementation of the action
plans?
5. Do you think the project can continue without funding from the World Bank? Why or why
not?
6. Did the project try to enable certain excluded groups to participate and voice their
concerns? What can be done about this going forward?
7. Whose concerns are still not heard?
8. How could the project have been improved? Give examples
9. Do you think the project would be effective in other regions of Burkina Faso? Why or
why not?
10. How would you change this project if and when implementing in other regions of
52
Burkina?
Interview Guide 4: Focus Group Discussion - COGES, AME, APE
1. Can you please describe what your organization does?
2. Do you believe your organization representative of all members of the community?
3. Do you believe your organization is able to serve your community well?
4. How does your organization learn about citizens’ concerns regarding health and
education facilities?
5. What improvements have you been able to make at schools and health facilities?
6. Are there any improvements you would have liked to make but have been unable to?
7. What do you do if there is an issue at education or health facilities that you cannot
resolve?
8. Does your organization meet with the local government structures?
Probe: village development committee, municipal council, and mayor
9. Is the local government responsive to your organization’s requests and concerns?
10. Is the central government responsive to your organization’s requests and concerns?
11. How can the local government support your organization in fulfilling its mandate?
12. How can the central government support your organization in fulfilling its mandate?
13. Does your organization work with NGOs to facilitate education and health service
delivery?
14. How can NGOs support your organization in fulfilling its mandate?
53
MODELS
Model 1a.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls
p_time 0.0920*** 0.0940*** 0.0942*** 0.0939*** 0.0962*** 0.0983*** 0.0984***
(0.0244) (0.0245) (0.0245) (0.0246) (0.0258) (0.0263) (0.0262)
csps_dir 0.0281 0.0262* 0.0258 0.0258 0.0265 0.0270 0.0254
(0.0212) (0.0153) (0.0154) (0.0154) (0.0165) (0.0166) (0.0164)
time_treat -0.0613* -0.0680* -0.0677* -0.0669* -0.0685* -0.0683* -0.0634*
(0.0344) (0.0340) (0.0340) (0.0340) (0.0348) (0.0350) (0.0350)
Plateau-Central -0.0966*** -0.0967*** -0.0971*** -0.0947*** -0.0957*** -0.0944***
(0.0173) (0.0173) (0.0176) (0.0175) (0.0177) (0.0179)
Sahel -0.0715*** -0.0713*** -0.0713*** -0.0689*** -0.0697*** -0.0693***
(0.0237) (0.0238) (0.0237) (0.0241) (0.0243) (0.0248)
p_child_sex -0.0164 -0.0166 -0.0155 -0.0155 -0.0153
(0.0116) (0.0115) (0.0115) (0.0115) (0.0115)
p_ape_ame_coges_b -0.00517 -0.00747 -0.00887 -0.00764
54
(0.0166) (0.0169) (0.0164) (0.0172)
Other relative 0.0502 0.0511 0.0520
(0.0376) (0.0374) (0.0383)
Extended family/clan -0.0134 -0.0126 -0.00416
(0.0233) (0.0228) (0.0222)
Close Friend -0.0398 -0.0389 -0.0401
(0.0329) (0.0332) (0.0353)
Friend of a friend -0.0640 -0.0595 -0.0696
(0.0574) (0.0581) (0.0698)
Known person -0.00249 -0.00168 -0.00493
(0.0204) (0.0202) (0.0199)
none -0.00329 0.000245 0.00317
(0.0209) (0.0219) (0.0219)
itself 0.000101 -0.000459 -0.00203
(0.0483) (0.0479) (0.0451)
NA -0.0790*** -0.0796*** -0.0887***
(0.0243) (0.0248) (0.0314)
Other -0.0542 -0.0539 -0.0528
(0.0528) (0.0518) (0.0517)
55
p_village_meeting 0.0165 0.0142
(0.0207) (0.0207)
Other relative 0.0278
(0.0553)
Extended family/clan -0.0852**
(0.0361)
Close friend 0.0145
(0.0290)
Friend of a friend 0.0437
(0.0868)
Known person 0.0244
(0.0170)
none -0.00637
(0.0184)
itself -0.00270
(0.0211)
Other 0.846***
(0.0348)
Constant 0.0746*** 0.134*** 0.158*** 0.160*** 0.159*** 0.144*** 0.140***
56
(0.0113) (0.0179) (0.0249) (0.0264) (0.0289) (0.0297) (0.0333)
Observations 2,903 2,903 2,903 2,903 2,903 2,903 2,903
R-squared 0.011 0.027 0.028 0.028 0.030 0.030 0.036
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Model 1b.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls
p_time 0.175 0.154 0.155 0.151 0.182 0.185 0.143
(0.135) (0.131) (0.131) (0.129) (0.114) (0.114) (0.110)
csps_dir 0.0427 0.0297 0.0298 0.0289 0.0297 0.0285 0.0179
(0.123) (0.107) (0.107) (0.107) (0.112) (0.111) (0.108)
time_treat -0.0597 -0.0754 -0.0754 -0.0695 -0.0814 -0.0743 -0.0699
(0.184) (0.171) (0.171) (0.173) (0.173) (0.173) (0.173)
57
Plateau-Central -0.266** -0.266** -0.267** -0.263** -0.269** -0.275**
(0.114) (0.114) (0.115) (0.114) (0.111) (0.110)
Sahel -0.368*** -0.369*** -0.368*** -0.345*** -0.351*** -0.354***
(0.116) (0.116) (0.116) (0.121) (0.117) (0.112)
p_child_sex -0.0114 -0.0113 -0.0144 -0.0166 -0.0158
(0.0884) (0.0885) (0.0896) (0.0889) (0.0888)
p_ape_ame_coges_b -0.0262 -0.0315 -0.0386 -0.0546
(0.0743) (0.0778) (0.0768) (0.0742)
Other relative 0.0974 0.0997 0.0931
(0.219) (0.217) (0.215)
Extended family/clan -0.0975 -0.0897 -0.0632
(0.160) (0.156) (0.135)
Close friend -0.0630 -0.0522 -0.0628
(0.276) (0.281) (0.269)
Friend of a friend 0.425 0.457 0.725*
(0.402) (0.385) (0.390)
Known person -0.0710 -0.0680 -0.0571
(0.0834) (0.0843) (0.0876)
none -0.247 -0.232 -0.244
58
(0.173) (0.173) (0.178)
itself -0.201 -0.205 -0.217
(0.427) (0.428) (0.392)
NA -1.291*** -1.291*** -1.276***
(0.257) (0.256) (0.246)
Other -0.163 -0.151 -0.132
(0.987) (1.003) (1.015)
p_village_meeting 0.0743 0.0546
(0.109) (0.104)
Other relative -0.215
(0.358)
Extended family/clan 0.0482
(0.335)
Close Friend 0.163
(0.224)
Friend of a Friend -0.611
(0.426)
Known Person -0.0375
(0.105)
59
none 0.0266
(0.111)
itself 0.229
(0.167)
Constant -0.548*** -0.312*** -0.295* -0.284 -0.240 -0.299 -0.273
(0.0813) (0.102) (0.165) (0.171) (0.179) (0.224) (0.215)
Observations 1,695 1,695 1,695 1,695 1,695 1,695 1,695
R-squared 0.002 0.013 0.013 0.013 0.017 0.018 0.022
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
60
Model 2a.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls
p_time 0.0885*** 0.0884*** 0.0884*** 0.0886*** 0.0874*** 0.0876*** 0.0874***
(0.0158) (0.0159) (0.0159) (0.0159) (0.0159) (0.0161) (0.0164)
csps_indir 0.00724 0.00399 0.00399 0.00307 0.00356 0.00360 0.00286
(0.00821) (0.00773) (0.00773) (0.00775) (0.00773) (0.00776) (0.00804)
time_treatindir -0.0370** -0.0364* -0.0364* -0.0364* -0.0378** -0.0378** -0.0367**
(0.0184) (0.0185) (0.0185) (0.0185) (0.0185) (0.0184) (0.0185)
Plateau-Central -0.0354** -0.0354** -0.0350** -0.0368** -0.0368** -0.0378***
(0.0142) (0.0142) (0.0140) (0.0143) (0.0143) (0.0141)
Sahel -0.0157 -0.0157 -0.0150 -0.0145 -0.0144 -0.0144
(0.0143) (0.0143) (0.0141) (0.0145) (0.0145) (0.0144)
p_child_sex 0.000794 0.000737 0.000450 0.000433 0.000299
(0.00467) (0.00467) (0.00470) (0.00470) (0.00467)
p_ape_ame_coges_b 0.0120* 0.0134* 0.0132* 0.0123*
(0.00696) (0.00708) (0.00721) (0.00723)
Other relative -0.00932 -0.00925 -0.0131
61
(0.0163) (0.0162) (0.0159)
Extended family/clan 0.00741 0.00736 0.0129
(0.0105) (0.0105) (0.0110)
Close friend 0.0151 0.0152 0.0168
(0.0209) (0.0209) (0.0211)
Friend of a Friend 0.0326 0.0326 0.0365
(0.0398) (0.0399) (0.0394)
Known Person 0.0223** 0.0223** 0.0200**
(0.00860) (0.00862) (0.00886)
none 0.0103 0.0105 0.0127
(0.0126) (0.0129) (0.0130)
itself 0.0289 0.0287 0.0298
(0.0215) (0.0214) (0.0216)
NA -0.0165 -0.0165 -0.0187
(0.0168) (0.0170) (0.0171)
Other -0.0104 -0.0104 -0.0180
(0.0391) (0.0392) (0.0391)
p_village_meeting 0.00259 0.000943
(0.0107) (0.0109)
62
Other relative 0.0659**
(0.0267)
Extended family/clan -0.0351**
(0.0145)
Close friend -0.00214
(0.0177)
Friend of a friend -0.0410
(0.0293)
Known person 0.0194*
(0.0109)
none -0.000930
(0.00821)
itself 0.00391
(0.0137)
NA -0.0749***
(0.0118)
Other -0.00731
(0.111)
Constant 0.0617*** 0.0858*** 0.0846*** 0.0802*** 0.0741*** 0.0716*** 0.0699***
63
(0.00596) (0.0140) (0.0153) (0.0156) (0.0166) (0.0196) (0.0202)
Observations 13,362 13,362 13,362 13,362 13,362 13,358 13,358
R-squared 0.013 0.015 0.015 0.016 0.017 0.017 0.019
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Model 2b.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls
p_time 0.181** 0.211** 0.211** 0.210** 0.0892 0.111 0.118
(0.0740) (0.0816) (0.0815) (0.0812) (0.130) (0.118) (0.111)
csps_indir 0.724 0.735 0.733 0.727 0.705 0.703 0.705
(0.541) (0.574) (0.573) (0.573) (0.561) (0.560) (0.561)
time_treatindir -0.734 -0.734 -0.733 -0.733 -0.683 -0.667 -0.667
(0.540) (0.530) (0.529) (0.529) (0.497) (0.490) (0.489)
64
Plateau-Central -0.241** -0.240** -0.245** -0.309*** -0.303*** -0.310***
(0.117) (0.117) (0.119) (0.107) (0.108) (0.112)
Sahel 0.169 0.168 0.165 0.0299 0.0388 0.0319
(0.427) (0.426) (0.426) (0.340) (0.345) (0.338)
p_child_sex -0.0380 -0.0383 -0.0257 -0.0288 -0.0285
(0.307) (0.306) (0.295) (0.297) (0.292)
p_ape_ame_coges_b 0.0655 0.0985 0.0772 0.0959
(0.328) (0.301) (0.313) (0.268)
Other relative -0.246 -0.239 -0.209
(0.289) (0.288) (0.258)
Extended family/clan -0.241 -0.240 -0.224
(0.240) (0.240) (0.193)
Close Friend -0.353 -0.337 -0.343*
(0.242) (0.244) (0.201)
Friend of a Friend -0.467* -0.460* -0.424
(0.270) (0.267) (0.274)
Known Person -0.150 -0.147 -0.141
(0.186) (0.186) (0.134)
None 1.316 1.331 1.302
65
(1.414) (1.427) (1.363)
Itself 0.0703 0.0488 0.0522
(0.238) (0.224) (0.162)
NA -1.067 -1.065 -1.039
(0.727) (0.726) (0.714)
Other -0.654** -0.638** -0.627***
(0.270) (0.286) (0.234)
p_village_meeting 0.221 0.245
(0.206) (0.245)
Other relative -0.514
(0.431)
Extended family/clan -0.0731
(0.336)
Close friend -0.0636
(0.340)
Friend of a friend 0.0756
(0.419)
Known person -0.117
(0.357)
66
none 0.0395
(0.414)
itself -0.0756
(0.372)
NA -0.644
(0.829)
Other -0.230
(0.421)
Constant -0.573*** -0.557** -0.499 -0.515 -0.411 -0.616 -0.607**
(0.0453) (0.263) (0.502) (0.571) (0.472) (0.417) (0.260)
Observations 7,510 7,510 7,510 7,510 7,510 7,510 7,510
R-squared 0.000 0.001 0.001 0.001 0.002 0.002 0.002
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
67
Model 3a.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls
p_time 0.0777** 0.0765** 0.0765** 0.0774** 0.0736** 0.0679** 0.0701**
(0.0308) (0.0313) (0.0313) (0.0308) (0.0326) (0.0333) (0.0338)
school 0.0166 0.0141 0.0142 0.0127 0.00949 0.0113 0.00906
(0.0205) (0.0187) (0.0186) (0.0189) (0.0183) (0.0187) (0.0173)
time_treatschool -0.0101 -0.0103 -0.0103 -0.0100 -0.00970 -0.00868 -0.00680
(0.0390) (0.0396) (0.0396) (0.0397) (0.0409) (0.0414) (0.0411)
Plateau-Central -0.0411* -0.0412* -0.0407* -0.0424* -0.0421* -0.0443*
(0.0223) (0.0223) (0.0222) (0.0236) (0.0237) (0.0238)
Sahel -0.0269 -0.0268 -0.0262 -0.0197 -0.0213 -0.0218
(0.0249) (0.0249) (0.0246) (0.0245) (0.0239) (0.0242)
p_child_sex 0.00305 0.00288 0.00113 0.00115 0.000370
(0.0116) (0.0115) (0.0117) (0.0117) (0.0116)
p_ape_ame_coges_b 0.0114 0.00967 0.0126 0.0166
(0.0148) (0.0156) (0.0159) (0.0158)
Other relative -0.0674** -0.0735*** -0.0797***
68
(0.0254) (0.0243) (0.0236)
Extended family/clan -0.0202 -0.0219 -0.0183
(0.0250) (0.0254) (0.0264)
Close friend -0.0454 -0.0497 -0.0453
(0.0323) (0.0333) (0.0372)
Friend of a friend 0.00963 0.00389 0.00505
(0.0968) (0.0881) (0.0838)
Known person -0.00195 -0.00563 -0.00707
(0.0198) (0.0202) (0.0196)
none -0.0368 -0.0387 -0.0441
(0.0274) (0.0273) (0.0284)
itself -0.0475 -0.0464 -0.0564
(0.0426) (0.0407) (0.0431)
NA -0.0432 -0.0582 -0.0705*
(0.0363) (0.0363) (0.0356)
Other -0.0349 -0.0415 -0.0433
(0.0688) (0.0637) (0.0637)
p_village_meeting -0.0570** -0.0562**
(0.0249) (0.0257)
69
Other relative 0.0678
(0.0560)
Extended family/clan -0.0162
(0.0374)
Close friend -0.0127
(0.0354)
Friend of a friend 0.0750
(0.0834)
Known person 0.00745
(0.0185)
none 0.0185
(0.0165)
itself -0.0119
(0.0249)
Other 0.204
(0.276)
Constant 0.0693*** 0.0943*** 0.0897*** 0.0845** 0.105*** 0.158*** 0.152***
(0.0173) (0.0272) (0.0304) (0.0311) (0.0326) (0.0380) (0.0399)
70
Observations 2,676 2,676 2,676 2,676 2,676 2,672 2,672
R-squared 0.014 0.017 0.017 0.017 0.021 0.024 0.028
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
Model 3b.
(1) (2) (3) (4) (5) (6) (7)
VARIABLES No Controls 1 Control 2 Controls 3 Controls 34 Controls 5 Controls 6 Controls
p_time 0.319 0.297 0.300 0.295 0.253 0.246 0.277
(0.272) (0.275) (0.282) (0.281) (0.248) (0.261) (0.299)
school -0.108 -0.0479 -0.0458 -0.0540 -0.0419 -0.0439 -0.0171
(0.173) (0.140) (0.142) (0.147) (0.150) (0.149) (0.157)
time_treatschool -0.108 -0.127 -0.131 -0.121 -0.120 -0.119 -0.149
(0.318) (0.313) (0.321) (0.324) (0.287) (0.288) (0.294)
2.p_region -0.188 -0.189 -0.199 -0.224 -0.224 -0.237
(0.211) (0.210) (0.222) (0.232) (0.232) (0.228)
3.p_region -0.395 -0.394 -0.406 -0.514 -0.514 -0.462
(0.332) (0.336) (0.358) (0.366) (0.367) (0.364)
71
p_child_sex 0.0290 0.0295 0.0473 0.0475 0.0520
(0.158) (0.158) (0.160) (0.160) (0.158)
p_ape_ame_coges_b 0.0607 0.0432 0.0484 0.120
(0.156) (0.147) (0.155) (0.154)
Other relative -0.105 -0.107 -0.204
(0.218) (0.216) (0.225)
Extended family/clan 0.0504 0.0495 -0.0842
(0.173) (0.173) (0.148)
Close Friend -0.499 -0.505 -0.626
(0.399) (0.386) (0.397)
Friend of a friend 0.473*** 0.475*** 0.772***
(0.134) (0.136) (0.236)
Known Person -0.00790 -0.0126 -0.149
(0.213) (0.201) (0.194)
None 0.490** 0.484** 0.398**
(0.175) (0.176) (0.172)
itself 0.445 0.452 0.387
(0.424) (0.422) (0.371)
NA -0.414 -0.419 -0.344
72
(0.300) (0.291) (0.282)
Other 0.521 0.509 0.484*
(0.312) (0.306) (0.242)
p_village_meeting -0.0489 0.00403
(0.203) (0.215)
Other relative 0.00253
(0.373)
Extended family/clan 0.691
(0.450)
Close friend -0.0676
(0.439)
Friend of a Friend 0.375
(0.404)
Known person 0.460*
(0.245)
none 0.495**
(0.223)
itself 0.258
(0.221)
73
Other 0.462
(0.707)
Constant -0.482*** -0.308* -0.353 -0.365 -0.370 -0.323 -0.667
(0.0610) (0.178) (0.365) (0.348) (0.353) (0.447) (0.554)
Observations 989 989 989 989 989 989 989
R-squared 0.008 0.014 0.014 0.014 0.024 0.024 0.039
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1