Capstone

Report A quantitative and

qualitative analysis of the

World Bank Community

Monitoring Project in

Burkina Faso

Saman Amir, Brittany Heyer, Obaid Khan,

Sarute Vithoontien

THE GEORGE WASHINGTON UNIVERSITY

ELLIOTT SCHOOL OF INTERNATIONAL AFFAIRS

INTERNATIONAL DEVELOPMENT STUDIES

MAY 15, 2017

1

TABLE OF CONTENTS

Executive Summary ................................................................................................... 2

Introduction ................................................................................................................ 3

Country Context ............................................................................................................................................ 3

Project Overview ........................................................................................................................................... 4

Research Purpose ...................................................................................................... 6

Methodology ................................................................................................................ 7

Literature Review: ..................................................................................................... 8

Quantitative Component: ....................................................................................... 11

Sampling Strategy for the Community Monitoring Program: Cluster-Randomized Controlled Trial ......... 11

Quantitative Analysis .................................................................................................................................. 13

Establishing Baseline Equivalence for the Treatment and Control Group ............................................... 13

Using Difference-in-Differences when Samples are Randomized ........................................................... 16

Models ........................................................................................................................................................ 17

Comparison of Program Impact on Child Sickness Across Regions ......................................................... 22

Comparison of Program Impact on Child Health Across Gender ............................................................. 22

Qualitative Findings ................................................................................................ 23

Community Monitoring Project-Specific Themes ....................................................................................... 23

Contextual Themes ..................................................................................................................................... 26

Limitations ................................................................................................................ 33

Conclusion ................................................................................................................. 34

Recommendations .................................................................................................... 35

Bibliography .............................................................................................................. 36

Appendix .................................................................................................................... 42

Interview Guides ......................................................................................................................................... 42

Models ........................................................................................................................................................ 53

2

EXECUTIVE SUMMARY

The World Bank Independent Evaluation Group and Development Impact Evaluation, along with

their implementing partner in Burkina Faso, the Higher Institute of Population Sciences (ISSP),

collaborated in September 2010 to implement a community monitoring pilot project to improve

health and education service delivery. This novel approach was adopted to capture the spillover

effects of community monitoring across health and education sectors.

This report presents evidence from a literature review on different facets of citizen engagement,

quantitative analysis of child health outcomes as a result of the Community Monitoring Project,

and qualitative analysis of social accountability and inclusion based on field research in Burkina

Faso. The impact of community monitoring programs can be mediated by multiple contextual

factors. This warrants comprehensive research to explore mechanisms and pathways through

which community monitoring can positively impact service delivery outcomes.

Rigorous evaluations of citizen engagement projects should be utilized not only to generate

credible evidence on the effects on ultimate outcomes, but also to unpack the causal chain to

identify the intermediate outputs that link the intervention to changes in short-term and long-term

outcomes. Comprehensive evaluation can help identify enabling and impeding mediating factors

and provide insightful lessons for future citizen engagement projects. Building on the social

accountability and inclusion components from the inception of the project design can also

amplify the effectiveness of citizen engagement projects by including diverse and marginalized

voices.

KEY FINDINGS

The Project had an impact on child health outcomes, specifically the likelihood of getting sick.

However, the impact differed across region and gender. Further analysis of the Project’s impact

on immediate and intermediate outcomes can help unravel the causal pathways for the

effectiveness of the Project in improving child health outcomes.

Participants framed the Project’s success as community sensitization, empowerment and

improved self-efficacy.

Participants identified inadequate inclusion of women and other marginalized segments of

population as a shortcoming of the Project.

Budget constraints and lack of capacity at different levels of government were cited as the main

impediments to effective service delivery.

Participants emphasized the instrumental role of NGOs, civil society organizations, donors, and

the media and protests to improve social accountability.

3

INTRODUCTION

COUNTRY CONTEXT

Despite steady economic growth and more than a decade of reforms and sound macroeconomic

policies, persistently high rates of rural poverty undermine development efforts in Burkina Faso

(World Bank 2013). With a gross per capita income of $615, approximately 40% of the

population lives below the poverty line (World Bank 2015). Burkina Faso ranks 183 out of 188

on the United Nations Development Programme’s Human Development Index (2015),

registering one of the lowest adult literacy rates in the world (36%), and alarmingly high infant

and maternal mortality rates (60.9 per live births and 371 per 100,000 live births, respectively).

Gender inequality is also pervasive in Burkina Faso, as gendered differences in human

development indicators illustrate. 6% of adult women have reached at least a secondary

education, compared to 11.5% of their male counterparts (UNDP 2015).

Nevertheless, improvements in human development have been made: the gross enrollment rate at

primary level education rose from 57% in 2005 to 87% in 2014, and the poverty rate fell slightly

between 2009 and 2014: 46% to 40.1% (World Bank 2015).

In addition to improvements in quality of life, changes in the country’s political context are cause

for a positive outlook. Burkina Faso has undergone a process of decentralization and

democratization since the 1990s, culminating in a popular democratic uprising in 2014 in

response to a proposed constitutional amendment to suspend presidential term limits (Freedom

House 2016). However, decentralization remains a challenge for the Government of Burkina

Faso, as local government structures struggle to fulfill their social services mandates (World

Bank 2013).

The figure below presents the decentralized framework of the Government of Burkina Faso

today, beginning with local village’s health facility and school management bodies, COGES

(Comité de Gestion), and parent-teacher associations, APE and AME (Association de Parents

d’Elèves and Association de Mères Éducatrices). To facilitate service delivery, these local

structures work primarily with the village municipal council and communal governments, and

then the regional and central governments.

Figure 1.1: Decentralization framework

The World Bank Community Monitoring for Better Health and Education Service Delivery

Project (CMP) complements and supports the Government’s decentralization efforts by

empowering communities to oversee local development and service delivery.

4

PROJECT OVERVIEW

The Burkina Faso Community Monitoring for Better Health and Education Service Delivery

Project (CMP) seeks to improve the quality and quantity of health and education services by

empowering and building the capacity of individuals and communities to demand better services.

The primary intervention is a community scorecard mechanism, which provides the community

with the means to monitor and evaluate education and health services, improving accountability

and transparency of service providers.

The process is intended to be community driven; in each village, the community is trained by

local non-governmental organizations (NGOs)1

in using community scorecards to define

evaluation criteria for education and health facilities and identify service delivery issues. The

community and service providers then work together to develop community action plans with

strategies to address these issues. Implementation of these action plans is routinely monitored by

the community using the scorecards.

This process is meant to engage multiple local, regional, and national stakeholders in public

health and education service provision. The different tiers of the national and regional

government are first involved through sensitization and consultation frameworks to create buy-

in. The communities are expected to make desired changes on their own or with assistance from

the local COGES, APE and AME bodies, as well as the village development committees under

the village municipal council that are tasked with overseeing and promoting local development.

The higher tiers of national and regional actors are the last resort for changes that fall outside of

the scope of villagers and local government.

The underlying theory of change is that this monitoring, coupled with public praising or shaming

through the dissemination of scores, will motivate service providers to implement action plans

and improve performance, which will yield greater improvements in health and education

outcomes. Additionally, the CMP is expected to induce greater participation of households in the

management of community affairs and, potentially, improve social capital.

1 ISSP, the central implementing partner for the pilot project, was responsible for identifying local non-

governmental organizations (NGOs) to facilitate trainings and implementation of the CMP in the selected

communes.

5

Figure 1.2: CMP Theory of Change

The project posits that the community and providers care—or can be induced to care—about

better health and schooling outcomes. Available resources should be applied in greater quantity

or with greater efficiency, and that the degree of commitment among relevant actors is sufficient

to mobilize those resources.

To generate credible evidence on whether any changes in outcomes are attributable to the

Project, the CMP was designed as an experimental impact evaluation piloted by World Bank

Independent Evaluation Group (IEG), in collaboration with the Development Impact Evaluation

(DIME). The evaluation design is a cluster-randomized controlled trial with health facilities and

schools randomly assigned to treatment or control groups in three of Burkina Faso’s thirteen

administrative regions.

Preliminary analysis of the Project has been conducted by IEG through focus groups and a

process evaluation (de Briey 2015). The George Washington University (GWU) team aims to

build upon the existing evaluative work and integrate learning from the various evaluation modes

through field research and empirical analysis of panel data. The team enlarged the scope and

focus of the Project’s evaluation by examining the CMP within the broader political and social

context of Burkina Faso.

6

RESEARCH PURPOSE

To supplement the existing program evaluation, the team analyzed the effectiveness of the

intervention on health outcomes. In order to understand the channels through which these

outcomes materialized, the team explored the mechanisms of citizen engagement and social

accountability underpinning the theory of change.

The GWU team sought to answer the following key questions:

1. What are the treatment effects of the CMP on child health outcomes?

2. Are there gendered differences in the Program’s outcomes or mechanisms?

3. What contextual factors influenced the Program’s effectiveness?

4. What are the perceptions of social accountability and citizen engagement among

Burkinabe?

5. How can community monitoring interventions at the grassroots level be

complemented with social accountability at higher tiers of government?

6. How can social inclusion be better integrated with community monitoring

interventions to improve voice and empowerment across the community?

7

METHODOLOGY

The George Washington University team conducted quantitative and qualitative research to

answer the research questions.

Using the child health data collected by IEG, the quantitative component included multivariate

regression and linear probability models, within the difference-in-difference approach, to isolate

the Community Monitoring Project’s effects on child health outcomes. To ensure consistency

and unbiasedness, the models added relevant covariates.

The GWU team’s qualitative research was primarily informed by a field visit to Burkina Faso.

The team conducted 13 key informant interviews with Project stakeholders, including NGOs,

local government officials, youth groups, World Bank staff, and members of the CMP Pilot

Committee who oversaw Project implementation. A focus group discussion was also organized

in a beneficiary commune, consisting of members of the local COGES, APE, and AME.

Interview protocols were designed after a thorough survey of relevant literature, prior qualitative

research, the CMP process evaluation and program documents, and consultation with World

Bank staff. The interview recordings were transcribed, coded, and analyzed to aggregate

common and divergent themes.

Finally, the team synthesized the quantitative and qualitative findings to formulate a narrative

assessing the CMP’s effectiveness, and developed recommendations for implementation and

scaling up of similar interventions in Burkina Faso.

8

LITERATURE REVIEW:

Prior to a thorough engagement with the social accountability approaches involved in the

Community Monitoring Project, we briefly review what social accountability entails. Ackerman

defines social accountability as “a pro-active process by which public officials inform about and

justify their plans of action, their behavior and results and are sanctioned accordingly”

(Ackerman 2005, 2). Per the World Bank, social accountability refers to the “extent and

capability of citizens to hold the state accountable and make it responsive to their needs” (2012).

Other scholars support this definition with the emphasis on primary engagement of citizens and

civil society in demanding actions from public officials (Brinkerhoff and Wetterburg 2016;

Grandvoinnet et al. 2015; Malena and McNeil 2010). Note that these definitions are holistic and

not restricted to elections, which many scholars discard as the key tenet of accountability

(Ackerman 2005; Geddes, 1994; Varshney, 1999; Stokes, 2003). For the purpose of this paper,

our team considers the social accountability definitions from the World Bank and Ackerman as

conceptual frameworks.

Fox (2014) divides social accountability into dual tactical and strategic approaches. Tactical

approaches rely merely on greater access to information to affect change, whereas multi-pronged

strategic approaches that combine citizen engagement with governmental reforms offer more

promise. Using this lens, the CMP could be seen to follow a strategic design and therefore

exhibit potential to influence what Fox terms ‘voice’ and ‘teeth’ (state responsiveness).

Literature emphasizes the paramount importance of contextual factors in assessing community

driven social accountability processes. O’Meally (2013) prioritizes understanding of context in

social accountability interventions by chalking out six instrumental contextual factors: political

society, civil society, state society relations, intra-society relations, and inter-elite relations.

Scholars also point to decentralization as a crucial contextual factor. A systematic literature

review on community empowerment, accountability, and education outcomes by the Department

for International Development (DFID) and International Initiative for Impact Evaluation (3ie)

focuses on decentralization as a key factor for the effectiveness of community empowerment

programs (Westhorpe et al. 2014). Other than the direct effect of decentralization on community

empowerment, the study mentions that decentralization provides a conducive context for

collective action. Campbell Collaboration’s systematic review undertakes a meta-analysis of

quasi-experiments and randomized controlled trials to verify the effectiveness of CMPs in terms

of corruption, access and quality of service delivery outcomes’ (Molina et al. 2016). The study

addresses the reduced incentive for politicians to respond to citizens’ needs if the needs are not a

contributing factor to the electoral strategy. The policy prescription that stems from these studies

is the enforcement of mutual accountability to amplify the effect of community-driven

interventions and ensure sustainability. Devolution of power and responsibilities, coupled with

social accountability at every level of the government, can help empower communities and the

government to pursue improvements in health and education outcomes.

9

Social capital also emerges as a pivotal factor particularly due to its relevance to CMP’s theory

of change. Evidence from a community-driven development program in Morocco on social

capital reveals that intervention increased the willingness of members of the community to

contribute to public goods, but had a negative effect on trust (researchers believe that this may be

due to tensions arising from competition between communities for CDD funding), and no impact

on altruism (Tu Chi Nguyen and Matthias Rieger 2017).

In terms of the impact of social accountability and community-based monitoring on service

delivery, a randomized field experiment on community-based monitoring of public primary

health care providers in Uganda stands out. Bjorkman and Svensson (2007) find that the use of

community scorecards to rate health service providers and community meetings to discuss

subsequent action plans significantly increased the quality and quantity of health care provision,

exhibited by an in increase in weight of infants, reduction in children deaths, increase in service

utilization, and positive behavioral change in service provision. A variation in treatment intensity

across districts allows the authors to infer that there is a significant relationship between the

degree of community monitoring and health utilization and outcomes, consistent with the

community-based monitoring mechanism.

Facilitation of women’s groups in conjunction with some health system strengthening could also

improve health outcomes, particularly reduction in maternal and neonatal mortality, and change

in health seeking behavior (Manandhar et al. 2004; Tripathy et al. 2010). Per the International

Development Coordinating Group’s review on community monitoring interventions, outcomes

regarding access to services were more definitive (Molina et al. 2016). Although employing only

information campaigns did not have a statistically significant effect on healthcare service

utilization in the short-run, running information and scorecard campaigns in tandem had positive

effects on utilization in both the short and medium-run (Molina et al. 2016). Furthermore, the

systematic review notes that community monitoring interventions had a positive short-term

impact on immunizations; however, medium-term effects were reportedly smaller and less

accurate.

Literature finds that community engagement and decentralization in school based management

tends to increase access to schooling, but a positive effect on learning outcomes and improved

accountability are not forthcoming. Results from a randomized experiment in Kenya show that

the use of school committees to monitor teachers improved teacher attendance and student

performance (Duflo, Dupas and Kremer 2008). The USAID Nigeria Northern Education

Initiative aimed to improve school performance management by encouraging civil society

participation and participatory evaluation through collection and dissemination of school report

cards (Brinkerhoff and Wetterberg 2016). In contrast to the experiment in Kenya, Healey’s

assessment of the project revealed little improvement in service delivery because powerful state

actors ignored the enhanced voice and demands from the relatively powerless civil society

(Brinkerhoff and Wetterberg 2016).

10

Scholars also evaluate social accountability’s equivocal effect on social inclusion and addressing

inequality. Gaventa and Barrett (2010: 46) highlight the potential of social accountability in

promoting social cohesion and cite the example of the use of forum theater in Nigeria to bring

the issues of community hierarchies based on wealth and ethnicity to the public fore. Mohanty

(2007: 85), however, describes a complete lack of agency among women recruited into

watershed management committees merely for procedural concerns, with limited awareness of

their membership in the committee. Corbridge et al (2005: 149) observed that participatory social

accountability approaches employed in village education committees disproportionately helped

wealthier groups.

Although social accountability initiatives often encompass citizen engagement, evidence on

citizen engagement’s impact deserves a special mention. Baldwin et al. (2016) evaluate the

impact of community-based development on citizen empowerment in Eastern Ghana. The

authors found that the leadership and civic commitment skills taught in the intervention

workshops led to improved perceptions of the village leadership quality and increased political

participation and representation at the district level. However, the results show that the program

did not translate into increased local government investment and did not improve socioeconomic

outcomes in the areas of literacy and education, health and nutrition, environment, food security,

and agricultural production. The World Bank’s report, ‘Making Politics Work for Development:

Harnessing Transparency and Citizen Engagement’, documents experimental evidence from

different countries on enabling factors that were adopted to strengthen transparency and citizen

engagement (World Bank 2016). The report highlights that ‘political accountability’, citizens

holding political institutions accountable, is as important for development as ‘social

accountability, whereby citizens can improve service delivery through collective action without

necessarily involving political institutions. USAID’s review of the effectiveness of citizen

engagement instruments in improving health service delivery and health outcomes draws

attention to socioeconomic and political context, and emphasizes that the impact of citizen

engagement on health outcomes can be amplified if the intervention is part of a broader package

of services (Edstrom 2015).

11

QUANTITATIVE COMPONENT:

SAMPLING STRATEGY FOR THE COMMUNITY MONITORING PROGRAM: CLUSTER-

RANDOMIZED CONTROLLED TRIAL

IEG employed a cluster-randomized controlled design as a sampling strategy to implement the

Community Monitoring Project. Three of the country’s thirteen regions, namely Cascades,

Central Plateau and Sahel, were selected to be included in the Project. For each region, three

communes were randomly selected as implementation sites for the treatment health facilities, and

three communes were randomly selected for the control health facilities. In each commune, two

health facilities were randomly selected to be assigned to the treatment or control arms.

Treatment and control school groups were also randomly assigned if they were within proximity

to a treated health facility to capture the indirect effects across health and education sectors.2

This effectively meant that 36 health facilities and 36 schools were randomly assigned to the

treatment or control arm in total, with 18 schools and 18 health facilities in each arm.

Finally, there were 36 villages allocated for the community monitoring intervention for treatment

and control schools. 168 villages were designated for treatment and control health facilities, with

36 in the direct treatment villages where the health facilities were located, and another 132 in the

catchment areas of health facilities. The additional 132 villages with no health facilities were

randomly selected to be included within the vicinity of treatment and control health facilities (66

each). The selection of additional villages was to capture the indirect effects of the intervention

over health facilities that were in proximity to the treatment health facilities as compared to the

control health facilities. The step-by-step sampling strategy is shown below in Figure 2.1.

2 The treatment and control schools were nested in the catchment area of treatment health facilities only, and not in

the control health facilities (as shown below in Figure 2.2.)

12

Figure 2.1 IEG Sampling Strategy

13

QUANTITATIVE ANALYSIS

To isolate the effect of the community monitoring program on health outcomes, we employed a

difference-in-difference analysis with multivariate regression and linear probability models. Each

model explored treatment effect on different health outcomes of interest (child sickness in the

past 30 days and child body mass index). The core model is demonstrated below:

CONCEPT MODEL:

𝑌ℎ𝑒𝑎𝑙𝑡ℎ_𝑜𝑢𝑡𝑐𝑜𝑚𝑒 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒 + 𝛽𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡(𝑥1) + 𝛽(𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝜇𝑖

The main variable of interest 𝛽(𝑡𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡∗𝑡𝑖𝑚𝑒)

is the treatment effect represented by the

interaction between the community monitoring program and time the effect of the monitoring

program across time. The first two models (Model 1 and Model 2) highlighted the effect of the

monitoring programs implemented in villages with health centers (direct and indirect treatment).

The third model observed the effect of monitoring programs in schools that were nested within

the treatment villages (villages with health centers). Finally, we compared the direct treatment

effects on health outcomes across the three regions where the monitoring program was

implemented (effect of monitoring programs in villages with health centers in each region on

child sickness).

ESTABLISHING BASELINE EQUIVALENCE FOR THE TREATMENT AND CONTROL GROUP

To determine if the treatment and control groups for health facilities were similar after

randomization in terms of covariates and outcomes of interest, we tested the statistical

significance of the difference between average proportions across treatment and control groups at

baseline. The difference in average proportions between the treatment and control groups were

tested at village level. As expected, there was no significant difference in the baseline

characteristics between the treatment and control groups where the community monitoring

intervention was directly implemented (see table 1). Table 2 exhibits baseline similarity between

treatment and control villages that were in proximity of the health facilities where the

intervention was directly implemented. The only exceptions were the proportion of male children

and the proportion of households with a personal link to the village chief, which were

significantly different in treatment and control groups.

14

TABLE 1.

Baseline Children Characteristics at Village Level for Treatment and Control Group (Direct)

Treatment Control Control-Treatment

Proportion of village meeting participation 0.90 0.92

0.02

(0.04)

Proportion of male children 0.53 0.54

0.004

(0.02)

Proportion of HHs receiving APE/AME/COGES services

0.37 0.44

0.07

(0.05)

Proportion of HHs with personal link with village chief

0.93 0.92 -0.01

(0.04)

Proportion of HHs with personal link with APE member

0.70 0.69 -0.01

(0.07)

Proportion of Children Sick in Past 30 days 0.11 0.07

-0.04

(0.02)

Children’s BMI z-score -0.45 -0.52

-0.07

(0.14)

Standard errors in parentheses. Significantly different than zero at 99 (***), 95 (**), and 90 (*) percent

confidence.

15

TABLE 2.

Baseline Children Characteristics at Village Level for Treatment and Control Group (Indirect)

Treatment Control Control-Treatment

Proportion of village meeting participation

0.94 0.93 -0.01

(0.017)

Proportion of male children

0.53 0.51 -0.017*

(0.01)

Proportion of HHs receiving APE/AME/COGES services

0.40 0.32 -0.08

(0.05)

Proportion of HHs with personal link with village chief

0.89 0.95 0.06***

(0.02)

Proportion of HHs with personal link with APE member

0.60 0.59 -0.01

(0.04)

Proportion of Children Sick in Past 30 days

0.066 0.061 -0.005

(0.009)

Children’s BMI z-score 0.34 -0.59 -0.93

(0.68)

Standard errors in parentheses. Significantly different than zero at 99 (***), 95 (**), and 90 (*) percent

confidence.

16

USING DIFFERENCE-IN-DIFFERENCES WHEN SAMPLES ARE RANDOMIZED

Considering the random assignment of households to treatment and control and the demonstrated

baseline equivalence, concerns on the use of difference-in-difference may arise. If households

are assigned to treatment and control groups at random, then differences between groups across

time should be attributed to the treatment. However, on the off-chance that randomization was

done inadequately or the groups were different by pure chance, then our covariates would control

for these differences and ensure precision in our estimates. We have used Miguel and Kremer’s

approach in the deworming randomized experiment to use difference-in-differences analysis to

corroborate the regression results (2004). The following section details the model specifications

and treatment effects on health outcomes; child sickness, and child body mass index.

Treatment Types (independent variables of interest) in our analysis include:

1) CMP implemented within village health centers (direct treatment);

2) CMP implemented in villages within proximity to villages with treated and/or controlled

health centers (indirect treatment); and

3) CMP implemented in schools located within villages with health centers (school

treatment).

The following section presents our regression results once covariates were added to the core

specification.

17

MODELS

Model 1a. Monitoring Program in Villages with Health Centers (direct treatment) and Child

Sickness

Model 1 identifies the relationship between the monitoring program in village health centers and

the likelihood that a child within a village household would be sick within the past thirty days.

The model takes the following form:

𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝜇𝑖

To control for potentially confounding variables, we installed the following covariates:

1. Region: Cascades, Central Plateau, and Sahel

2. Child Gender

3. Membership in a local village committees

4. Household’s personal links to the community village chief: close relative, other relative,

extended family/clan, close friend, friend of a friend, known person, none, the village

chief himself/herself

5. Household’s participation in the village meetings

6. Households’ personal links to local village committees: close relative, other relative,

extended family/clan, close friend, friend of a friend, known person, none, the village

chief himself/herself

The confounding variables were chosen based on the CMP process evaluation and the team’s

qualitative research. Regional and gender variation in the program’s effectiveness were

frequently cited as a concern in the Process Evaluation, and these concerns were corroborated by

our field research (de Briey 2015). Household’s personal links to community village chief and

village committees and membership in local village committees were included in the model to

accommodate for any elite capture or exclusion of poor or ethnic minorities, another concern

expressed in the Process Evaluation and corroborated by field research. These concerns are

discussed in greater depth in the qualitative findings.

The entirety of Model 1a. takes the following form:

𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) + 𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3)

+ 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6)

+ 𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Where the dependent variable 𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 is a dichotomous variable that takes on the values 0 and

1 (1 = sick within the past 30 days, 0 = not sick within the past 30 days). 𝛽𝑡𝑖𝑚𝑒 is the time

variable that equals 0 at baseline and 1 at end-line. 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) is also a

dichotomous variable that equals 1 if the community monitoring program was implemented

18

directly (within villages with health centers), and equals 0 for control villages (villages with

health centers that did not have a community monitoring program). 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)

is the time-

treatment interaction variable; our independent variable of interest. Dichotomous variable of

𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 designates the child’s gender, 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒 represents membership within a

village committee and 𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀 indicates a household’s participation in village meetings.

We also applied this model to our second health outcome of interest (child’s body mass index);

this model takes the following form:

Model 1b.

𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑑∗𝑡𝑖𝑚𝑒)(𝑥2) +

𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +

𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Model 1a. Results:

The community monitoring program in village health centers had a negative and statistically

significant (10% significance level) effect on the likelihood that a child within a village

household has been sick within the past 30 days. Children in villages with treatment health

centers were 6.34% less likely to be sick within the past 30 days (relative to children in villages

with health centers that did not receive a monitoring program) when controlling for region, child

gender, household membership in village committees, households’ personal links to village

chief, household participation in village meetings, and households’ personal links to village

committees. Furthermore, children from the Central Plateau region were 9.44% (1% significance

level) less likely to be sick within the past 30 days relative to children living in the Cascade

Region, holding all other factors constant. Similarly, children living within the Sahel region were

also less likely to be sick within the past 30 days relative to children living in the Cascades

Region (6.93% less likely; 1% significance level), assuming ceteris paribus. Child gender,

membership in village committees, households’ personal link to village chief, household

participation in village meetings, and households’ personal links to village committees did not

have statistically significant effects on child sickness.

Model 1b. Results:

The community monitoring program in village health centers did not have a statistically

significant effect on child body mass index; however, this model did reveal that children from

the Central Plateau and Sahel regions had lower body mass index for their age relative to

children in the Cascades region (Children in Central Plateau were 27.5% less likely to be sick

within the past 30 days relative to children in the Cascades; Children in Sahel were 35.4% less

likely to be sick within the past 30 days relative to children in the Cascades). (Note: body mass

index variable coded as body mass index for age z-scores. Full model can be found in the

Appendix).

19

Model 1 Interpretation:

The results from the two models present a contradiction with a significant drop in child sickness

but no profound effect on child BMI. Limitation of the BMI metric offers a strong explanation.

BMI is used to measure excess weight rather than excess body fat (which is what we are

interested in), and, as such, it is unable to identify whether the excess weight is attributed to fat

or muscle mass and bone density. Another plausible explanation is that many health

improvements from the program may have been produced from changes outside the health

center. For instance, communal building of wells through the program could diminish the

likelihood of child sickness by reducing the physical stress borne by children through strenuous

and labor intensive water collection.

We now proceed to our second model which compares the effect of community monitoring

programs (on the same two health outcomes in Model 1) on villages that did not have a health

center but were within proximity to a village with a health center that received the community

monitoring program (indirect treatment villages) relative to villages that also did not have a

health center but were within proximity to villages with health centers but did not receive the

monitoring program treatment (indirect control villages). The purpose of Model 2 is to identify

whether there are any spillover effects of the community monitoring program on nearby villages

with no health centers.

Model 2a.

𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑖𝑛𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +

𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +

𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Model 2b.

𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑖𝑛𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑖𝑛𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +

𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +

𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Model 2a. Results:

Model 2a. found that the community monitoring program had a negative and statistically

significant effect (at the 5% significance level) on the likelihood of a child being sick within the

past 30 days. Children within indirectly treated villages were 3.67% less likely to be sick within

the past 30 days relative to children within indirectly controlled villages (after controlling for

20

location, child gender, membership in village committees, personal links with village chief,

participation in village meetings, and personal links with village committees). Note that among

indirectly treated/controlled villages, households with members that had a role in village

committees had a higher probability of having a child sick within the past 30 days (1.23% more

likely; 10% significance level). This is not to say that having membership ties with village

committees increases the chances of a child being sick but that these households may be more

likely to report their child’s health status relative to households with no members having a role in

village committees.

Model 2b. Results:

Model 2b. looked at relationship between indirect treatment and whether children’s body mass

index, again we did not find a statistically significant relationship between the two.

Model 2 interpretation:

Model 2 results for both health outcomes are similar to those from Model 1. The indirect

treatment had a negative and statistically significant effect on child sickness but had no

statistically significant effect on child body mass index. The results in Model 2 conform to our

expectation of indirect treatment bringing less profound outcomes in contrast with the direct

treatment. However, the key takeaway point is that the community monitoring program in

villages with health centers did have a spillover effect on villages without health centers nearby.

Model 3 aims to identify whether community monitoring programs in schools influenced child

health outcomes. Here we looked at differences between schools with and without the

community monitoring program in villages with health centers.

Model 3a.

𝑌𝑐ℎ𝑖𝑙𝑑𝑠𝑖𝑐𝑘 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑠𝑐ℎ𝑜𝑜𝑙 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑠𝑐ℎ𝑜𝑜𝑙𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +

𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +

𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Model 3b.

𝑌𝑐ℎ𝑖𝑙𝑑𝑏𝑚𝑖 = 𝛽0 + 𝛽𝑡𝑖𝑚𝑒(0,1) + 𝛽𝑚𝑜𝑛𝑖𝑡𝑜𝑟𝑖𝑛𝑔 𝑝𝑟𝑜𝑔𝑟𝑎𝑚 𝑠𝑐ℎ𝑜𝑜𝑙 𝑑𝑖𝑟𝑒𝑐𝑡(𝑥1) + 𝛽(𝑚𝑝𝑠𝑐ℎ𝑜𝑜𝑙𝑑𝑖𝑟∗𝑡𝑖𝑚𝑒)(𝑥2) +

𝛽𝑟𝑒𝑔𝑖𝑜𝑛(𝑥3) + 𝛽𝑐ℎ𝑖𝑙𝑑 𝑔𝑒𝑛𝑑𝑒𝑟 (𝑥4) + 𝛽𝑀𝑒𝑚𝑏𝑒𝑟𝐶𝑜𝑚𝑚𝑖𝑡𝑡𝑒𝑒(𝑥5) + 𝛽𝑙𝑖𝑛𝑘𝑠𝐶ℎ𝑖𝑒𝑓(𝑥6) +

𝛽𝑃𝑎𝑟𝑡𝑖𝑐𝑖𝑝𝑎𝑡𝑒𝑉𝑀(𝑥7) + 𝛽𝑙𝑖𝑛𝑘𝑠𝑉𝐶(𝑥8) + 𝜇𝑖

Model 3 Results and Interpretation:

Community monitoring programs in schools did not have a statistically significant effect on any

of the two child health outcomes. The community monitoring program serves as a performance

litmus test to public service providers in the institution in which it was implemented. Here we

speculate that it is unlikely that a monitoring program in schools, where it is probable that only

21

school administrators are present, would induce changes in health outcomes rather than academic

outcomes.

Table 3. below briefly summarizes our regression results for our outcomes of interest before and

after controlling for covariates.

TABLE 3.

Regression Outputs with No Control and All Controls

Dependent Variable No Controls All Controls

Child Sick in Past 30 days (Direct Health Treatment)

-0.0613* -0.0634*

Child BMI (Direct Health Treatment)

-0.0597 -0.0699

Child Sick in Past 30 Days (Indirect Health Treatment)

-0.0370** -0.0367**

Child BMI (Indirect Health Treatment)

-0.734 -0.667

Child Sick in Past 30 Days (School Treatment)

-0.0101 -0.00680

Child BMI (School Treatment) -0.108 -0.149

*** p<0.01, ** p<0.05, * p<0.1

22

COMPARISON OF PROGRAM IMPACT ON CHILD SICKNESS ACROSS REGIONS

Comparing the effect of direct treatment across regions on whether the child has been sick in the

past 30 days (Note: values obtained by running regression Model 1a. with a restriction on

“regions”), we found that the treatment in health centers in Central Plateau and Sahel did not

have a significant effect on whether the child was sick in the past 30 days. However, the

treatment effect was significant in Cascades at the 5% level. The likelihood of the child getting

sick in the Cascades region decreased by 16% over the course of intervention.

We also compared the effect of indirect treatment across regions on whether the child has been

sick in the past 30 days. Here, we found that the effect of indirect treatment in health centers did

not have a significant effect in on whether the child was sick in the past 30 days in Cascades or

Central Plateau. This indicates that there were no significant spillover effects of the treatment on

child sickness in proximate villages within these two regions. However, the treatment effect was

significant in the Sahel region at 5% level. The likelihood of the child getting sick (in the past 30

days) in the Sahel region decreased by 8.11% over the course of intervention.

COMPARISON OF PROGRAM IMPACT ON CHILD HEALTH ACROSS GENDER

We found statistically insignificant direct treatment effects for the children’s BMI for both boys

and girls in the study. However, we also found that the program had a significant effect in

lowering the likelihood of the boys being sick in the past 30 days, but no significant effect on

girls being sick the past 30 days. Similarly, indirect treatment effects for the children’s BMI were

insignificant for both boys and girls, but the program had a significant indirect effect in lowering

the likelihood of the boys being sick, but not girls. The differential impact of the program on

child sickness based on gender might point to a broader problem of intra-household gender

inequality.

23

QUALITATIVE FINDINGS

To reiterate the research purpose, the GWU team’s quantitative research was restricted to

answering the following questions:

What are the treatment effects of the CMP on child health outcomes?

Are there gendered differences in the Program’s outcomes or mechanisms?

This section will highlight key qualitative findings with reference to the following pivotal

research questions:

What contextual factors influenced the Program’s effectiveness?

What are the perceptions of social accountability and citizen engagement among Burkinabe?

How do social accountability approaches fit into the larger political and historical context of

Burkina Faso? How can community monitoring interventions at the grassroots level be

complemented with social accountability at higher tiers of government?

How can social inclusion be better integrated with community monitoring interventions to

improve voice and empowerment across the community?

COMMUNITY MONITORING PROJECT-SPECIFIC THEMES

SUCCESS

We received unequivocal support for success of the Project from participants, but the extent of

this success varied. The participants’ perception of success was not grounded in a concrete

evaluation of success, rather on field visits, interaction with the treatment communes, and review

of monitoring reports.

When discussing the Project’s success,

participants noted improvements in infrastructure

in schools and health centers, community

awareness on accountability and citizen’s rights,

and behavior change. Behavior change, or a

change in “attitudes and mentality”, was

frequently cited by participants as a metric of

success for the Project. Prior to the Project,

citizens believed their ability to affect change in

service delivery was limited, but the CMP

engendered an understanding that community

members can work collectively to make

improvements without over relying on the local

government. This newly found self-efficacy was reflected in building walls around schools,

installing solar panels, increasing school enrolment, improving teachers’ housing, etc. Although

“The community was able to assess

problems in health and education.

Project wasn’t successful in making

the connection between making the

problems identified at community

level and action taken by

authorities. It’s one thing to have

the village action plan, but it’s

another thing to have those plans

enforced/implemented.”

24

the process evaluation undertaken by the World Bank highlighted the many improvements made

in communes (de Briey 2015), the emphasis on behavior change and self-efficacy was not

revealed until the GWU team’s interviews. Despite these positive changes undertaken by the

communities themselves, the participants were mindful of the inability of communities to

accomplish more substantive changes, such as the training or hiring of health workers, which

required greater input from the government. Thus, the development of community action plans

by both members of the community and service providers did not result in full implementation of

the plan. This discrepancy is further detailed in the following sections.

REGIONAL VARIATION

The process evaluation and quantitative component of this evaluation revealed that the Project

saw the greatest success in Cascades, followed by Central Plateau and the Sahel (de Briey 2015).

Our field research substantiated the finding that success varied by region, but the order of

regional success is less certain. Participants mostly considered their native regions most

successful. Although earlier findings suggested that Cascades performed the best, participants

would often consider Cascades and Central Plateau at par. In the process evaluation, the Sahel

region was considered to have achieved limited success, but a representative of a partner NGO in

the region thought the Project was a great success. We hypothesize that these discrepancies may

be originating from limited access to the other regions where the Project was implemented, as

well as circumstantial evidence, such as observational or anecdotal evidence. Most importantly,

participants alluded to variation across communes within the same region, which was not

captured in the process evaluation, and cannot be

ascertained by quantitative analysis. Participants would

often point out how one commune performed better than

others. These claims of variation across communes are more

credible, as they are based on field visits and interaction

with people from the various communes, in addition to

interviews with members of the CMP Pilot Committee who

oversaw CMP implementation.

According to participants, these variations were a result of differences in social context,

frequency of consultation meetings, engagement of local actors, quality of the implementing

NGO, and prior exposure to development programming. A further exploration of these factors

can add value to future projects in this area.

COMMUNITY SCORE CARDS

The community scorecard (CSC) was the mechanism employed in this Project to mobilize

communities to diagnose and address problems in education and health service delivery. There

was consensus that CSC was an effective, easy to use, and participatory tool that brought

service providers and users together. Two partner NGOs claimed that CSC was so useful that

they continue to use it in similar contexts. However, it must be noted that most of the

OUR FIELD RESEARCH

SUBSTANTIATED THE FINDING

THAT SUCCESS VARIED BY

REGION, BUT THE ORDER OF

REGIONAL SUCCESS IS LESS

CERTAIN.

25

interviewees used the CSC for the first time under the CMP. Also, the favorable view of the CSC

can be treated merely as perception since very few interviewees had used other tools and lacked

a reference point to compare the CSC with.

CHALLENGES

An exploration of the challenges of the Project corroborated findings from the process evaluation

and further exposed the crucial constraint of the lack of leverage amongst local communities

with higher levels of government. As highlighted by IEG’s qualitative research, procurement

delays, a short project timeline incompatible with behavior change processes, and a lack of

capacity among implementing partners were again cited by participants as challenges (de Briey

2015).

This finding provides an impetus for a deeper inquiry into social accountability processes, not

just at the local service delivery level, but at the higher levels of government. Following sections

delve into this further.

FUNDING

There was a discrepancy between the World Bank’s conceptualization and the perception of

some stakeholders that were interviewed about project financing. Although the intention of this

Project was to make communities self-reliant to improve health and education service delivery,

some interviewees expected the World Bank to finance the implementation of community action

plans. This demand had its origins in difficulty of mobilizing financial resources to realize the

action plans. There was also confusion about the Project’s funding scheme. For example, a

government agency tasked with monitoring the implementation of the action plans expected to

be compensated for their field visits, while ISSP considered the field visits to be the budgetary

responsibility of the agency. Other funding constraints cited include an extension of the Project’s

timeline without a commensurate cost extension, and delays in delivering funding to

implementing NGOs.

SUSTAINABILITY

Concerns about the CMP’s sustainability were raised during our study. Although participants

cited the increased self-efficacy in solving problems in the commune as the key success of the

Project, many doubted that these changes could be sustained because of a cessation of the

consultation frameworks and sensitization meetings that reinforced the citizen engagement and

social accountability mechanisms. Only one participant expressed greater understanding of the

“…But if they need a health agent in the center, this is now the national level responsibility to

bring the health agent in this center. At this level it slows down. At the regional level,

national level, it’s true that they have a pilot committee for the project, but this committee

does not have full means to respond to all the needs of the population.”

26

Project’s conceptual framework by suggesting that the gains could last if the existence of

community groups was capitalized by well-intentioned community leaders.

CONTEXTUAL THEMES

This section details the contextual themes relevant to the Project, with a focus on social

accountability at both the local service delivery unit and higher levels of government.

SOCIAL ACCOUNTABILITY CHANNEL

The channel of accountability in Burkina Faso’s government is understood by most participants

in the same way as detailed by the Project’s conceptual framework, with hierarchical linkages

originating from the village, to the communal, regional, and national levels. In discussing local

governments, most participants understood the channel of accountability to originate from APE,

AME, and COGES, the organizations that serve as the first point of contact, to the municipal

councils, and then to the mayors at the communal level of government.

Considering these apparent linkages between the different levels of government, it is essential to

explore the state-citizen connection to gain greater insight into how social accountability unfolds.

Varied perceptions of the government’s responsiveness to citizens’ needs emerged. For instance,

some participants viewed a new decree providing free healthcare at public health facilities to

children between ages of zero and five as a sign of increased commitment of the government. In

contrast, some participants disputed this claim when highlighting the government’s massive

shortcomings in equipping health facilities and the long processing times for fulfillment of

requests. The participants were generally cognizant of the resource constraints behind the

shortcomings, but some considered a lack of political will and informational asymmetry as

culpable factors.

The service providers attributed these shortcomings to severe resource constraint and inadequate

resource allocation from the central government.

Variation in mayors’3 perception of social accountability mechanisms also provides a glimpse

into the attitudes of public officials. One mayor was open to social accountability mechanisms

like the CMP because it facilitated greater contact with the citizens he is tasked with serving,

3 The head of the communal level of government, the mayor, is indirectly elected by the municipal council. The

communal level of government is responsible for the management of most basic socioeconomic infrastructure: rural

roads, health facilities, schools, water and sanitation.

27

whereas the other mayor welcomed the CMP because people took initiative to address challenges

and would not bother him for menial tasks.

CENTRAL GOVERNMENT

With respect to social accountability at the higher levels, sluggishness in the central government

bureaucracy was unanimously asserted by participants. Barring one participant, there was no

reference to any internal mechanism for social accountability. The unique reference pointed out

the poor functioning of that social accountability channel, but the exact nature of that channel

requires greater research. This failure was understood partially in the context of resource

constraints and lack of political will. Participants understood and acknowledged the resource

constraints faced by their government and the country’s over reliance on foreign aid. One local

government official revealed seeking assistance from French donors after several unanswered

requests by the central government for water pumps in Zitenga. To resolve this situation,

additional connections across the government hierarchy via greater communication were

suggested. The CMP was viewed as a crucial step in that direction. Another government

representative advocated for instating independent monitoring committees at each level of

government, citing an ongoing UNICEF project as an example.

Most participants acknowledge that channeling citizen engagement to foment social

accountability is an atypical concept in Burkina Faso, but there was consensus among the

participants on the need for building social accountability as a means of improving the

government’s responsiveness. Many contextually-appropriate suggestions detailed below form

the backbone of our findings on social accountability. It is important to acknowledge that many

of these suggestions emerged out of the participants’ experience of the popular uprising in 2014.

The forces that were said to have catalyzed the popular uprising were cited as actionable

recommendations. The relevance and importance of the popular uprising to the Burkinabes

manifests in the participants’ understanding of events or experiences as pre- and post- uprising.

Strong Civil Society Organizations

Local NGOs and civil society organizations, such as religious groups and youth groups, were

viewed as a potent force in triggering social accountability. Participants pointed out that the

presence of local NGOs in a community puts pressure on local governments to perform.

Indeed, the involvement of local organizations in the development of community action plans

was a means of inducing social accountability by encouraging transparency, and the role of

implementing NGOs in the CMP was considered the distinguishing factor for success of the

Project between treatment and control by one participant. Many participants attributed the

success of the popular uprising in 2014 to a strong civil society, further underscoring the

perception of these organizations as drivers of social accountability.

Donors/External actors

Participants expressed that donors could play an integral part in engendering social

accountability in Burkina Faso. The conditions that their money comes with imply greater

28

power to create opportunities for social inclusion through an emphasis on grassroots

participation and more transparency.

Media

The role of media, or radio, in particular, was emphasized as a useful tool for social

accountability. Media can be used to call out the local government for its inaction or

communicate concerns which may not otherwise reach or influence governors. Radio is

considered the best medium in Burkina Faso because the high rate of illiteracy precludes the

saturation of print media.

Taxes

Resource constraints of the local government was often seen as a basis for many of its

shortcomings. Tax evasion in communities was cited as a serious concern. Interviewees

proposed building awareness and mobilizing communities to pay taxes to address this

resource deficiency and provide a direct channel of accountability.

Dissemination of Information

Participants also cited greater circulation of the national and regional development plans as

an impetus for inclusion of communal and village concerns, as well as a greater

understanding of the government’s priorities.

Protest

Protest was considered a powerful tool to effect change. This was demonstrated by the

success of the 2014 popular uprising. Consequently, strikes have become more frequent

throughout the country in protest of government policies, wages, etc.

Decentralization

The findings on local government and central government’s shortcomings were closely

linked with the decentralization process in Burkina Faso. The interviewees pointed out that

the decentralization process in the country is still new, and needs improvement. Although

some participants criticized the government, interviewees unanimously demonstrated an

understanding of the resource constraints of the government. Inefficient transfer of resources

from the central to the local government, and an inadequate devolution of power were also

emphasized. Poor service delivery in health and education was deemed a manifestation of the

flawed decentralization process. In contrast, participants called attention to APE and COGES

as the functioning positive elements of decentralization, with APE performing better than

COGES.

Role of NGOs and Donors

The role of NGOs was largely seen in a positive light. They were regarded as integral actors

in ensuring social accountability. A majority of participants highlighted that NGOs could

ensure accountability by checking whether the local government’s promised improvements

for the community were made. Additionally, NGOs were viewed as mediators between

communities and governments. The NGOs had the influence to raise important issues as their

29

voice could reach the highest level of government, contrary to communities who could not

access the national government directly.

In most cases, NGOs were applauded for complementing government actions. For instance,

NGOs were cited as sensitizing communities to increase service utilization, such as

increasing school attendance or visits to the health center. A mayor appreciated an NGO’s

role in being able to raise awareness of the importance of paying taxes in his commune.

However, a representative from the implementing NGO in the Sahel raised the concern that

there was very little contact between local government and NGOs. The local government

often lacked the will to work in concert with the NGOs. This finding signified some regional

variation in performance of local government and the relationship between NGOs and

governments. However, a deeper exploration into this possible regional variation is beyond

the scope of this study.

There was strong agreement on the need for donors in Burkina Faso. The stakeholders

appreciated the financial resources and technical expertise donors provided. The donors were

particularly seen to be instrumental in encouraging social accountability by conditioning how

the money would be spent and making aid contingent on involvement of local populations.

Some stakeholders expressed concerns and suggested that external donors needed to ensure

transparency and prevent elite capture by giving grants in public forums.

SOCIAL INCLUSION

Inclusion emerged as a central theme in our qualitative research because evaluation of the CMP

and its inclusion of all members of communities is insufficient. It must be noted that some of the

“The role of the organizations and NGOs in communities is social

checking—social accountability, and checking activities.

Everyone now is ready to ask the mayor when we need the

school. When they build a school, they will now check to see if

they bring people—it is not just the building that is needed. So

we can see what we can do for people so that people can to

come to that school. If some people say that they need a health

center and they get the building or a health agent, now NGOs

have to go check out if they are sick; if there is a health

campaign they should do in this community. NGOs should raise

the awareness of everyone to hold those accountable at all

levels of the community. The role of the NGOs also is to raise

awareness of taxes in communities…”

30

participants were not able to make specific comments on the Project’s inclusion, but relied on

other experiences in their community as reference.

Some participants expressed concerns on

class-based or kinship-based exclusion in the

Project and general community decision-

making processes. A participant conveyed

that wealthier groups who have emigrated

from the village still tend to exert ample

influence in decision-making processes in

their villages of origin. In contrast to this

general view of communal decision-making, a

participant from an implementing partner

voiced his concern that the CMP did not

account for the possible differences in

participation based on socioeconomic classes,

and may have excluded some groups because of an over-reliance on the local bodies of COGES

and APE. Representatives of some local NGOs and a youth group highlighted that membership

of these local structures may be inclusive on paper, but kinship ties play an important role in

membership. As the leadership of COGES and APE is elected, the majority ethnic group in the

community is likely to control these organizations. A participant cited that in a village of

majority Mossi people, only Mossi people will be part of these organizations and the local

government.4 This practice appeared to be pervasive, affecting women’s groups and youth

associations, as well. However, an implementing partner suggested that ethnic exclusion did not

afflict the Project because the implementation communities were largely homogenous. Another

implementing partner asserted that people at the grassroots level were not represented in the

higher tiers of government, as adequate representation from local communities was not sought.

Variations in responses between implementing NGOs and stakeholders, and the local

government was seen. The local government stakeholders would often maintain the use of an

inclusive Project strategy, whereas implementing NGOs would be quick to acknowledge the

Project’s gaps in inclusion.

4 The Mossi people are the largest ethnic group in Burkina Faso, primarily residing in the central regions of Burkina

Faso.

“... Other minority groups must be included too in

village development council. We can have six

organizations in a village (which are supposed to

have 2 leaders per organization), but we see only

1 leader per organization. Even youth association

members have ties to village chief. Even women

association leaders are wives/daughters of village

chief.”

31

GENDER INCLUSION

Gender inclusion emerged as a strong contextual factor within the CMP and greater Burkina

Faso. Our discussion on gender inclusion revolved around CMP mostly as it was the participant's

primary reference point. In our investigation of female involvement in the Community

Monitoring Project, a simple question inquiring if the Project was inclusive would mostly

generate a solid “yes”. Further probing on inclusion of women would then result in a

contradictory response revealing that women were not active participants in meetings. This

contradiction is possibly rooted in a flawed perception of inclusion, equated with participation

with little regard for voice and agency. More inquiry into whether women were vocal in CMP

meetings or adequately voiced their concerns would reveal this discrepancy. After multiple

probes, some participants pointed out that even if women were given a platform, they would lack

the courage to speak. Study participants considered restrictive gender norms to be the key

problem. The norms highlighted in our study are listed below:

● Women need to seek husband permission.

● Women’s involvement in public affairs is looked down upon by in-laws.

● Women do not have the right to speak in the Mossi culture.

● Women were considered inferior to men. Women have internalized this value and

therefore step back in matters of public or communal decisions.

● Women fear participation.

This finding substantiated our earlier assertion of a lack of agency precluding women from equal

participation in the CMP. A mayor pointed out that more women came to him with concerns

because they were the most vulnerable members of the community. This lack of agency would

then lead to a vicious cycle, where less attention or understanding of women’s issues would be

reflected in poorer health and education outcomes amongst women.

Many participants alluded to the lack of an even playing field for women and a strong

internalization of restrictive norms by women themselves. Project stakeholders had some

suggestions to encourage women’s participation and agency. These recommendations are

highlighted below. Note that we do not endorse these recommendations, but are merely stating

the stakeholders’ opinions.

● Gender sensitization trainings before the start of projects to encourage women and get

men on board.

● Gender segregation of consultation groups could also give women more power to voice

concerns.

● Use of women leaders in community.

32

33

LIMITATIONS

DEPENDENT VARIABLE SELECTION

Our dependent variables of interest were health outcomes; however, the treatment was

not a health treatment. As such, there may have been intermediate outcomes “in-

between” which were not analyzed. Therefore, we are unable to identify the channels

through which the treatment was able to improve health outcomes such as child sickness.

EXTERNAL VALIDITY

Our analysis of the treatment effects on child health outcomes cannot be generalized to

the regions and communes that were not selected for the study, given that our field

research revealed regional and communal variations.

LIMITED ACCESS TO PROJECT BENEFICIARIES

Limited access to beneficiaries meant that we could not verify the results for the Project’s

effectiveness from those directly affected.

SOCIAL DESIRABILITY BIAS

Because our sample for the field research comprised primarily of stakeholders involved

in the Project’s management in some capacity, there was a tendency to overstate the

Project’s success.

34

CONCLUSION

We cannot unequivocally conclude that the Burkina Faso Community Monitoring Project was

successful in achieving its intended outcomes. Our quantitative analysis merely presents

evidence of the Project’s effectiveness on dual child health outcomes of child sickness and BMI,

but does not capture the causal chain through which these outcomes were achieved. Our field

research alludes to the intermediate outcomes by highlighting the Project’s marked contribution

to improved self-efficacy and empowerment of communities; however, these changes may not be

sustainable outside of the Project’s framework.

Regardless of the level of success of the CMP, some glaring shortcomings were identified. The

participation of women in the Project was lacking, with little improvement in women’s voice and

agency. Societal power hierarchies may have excluded poor and ethnic minorities, leading to

elite capture of the process. A lack of responsiveness from a resource-constrained government,

and a strong disconnect between the higher levels of government and citizens, impeded effective

service delivery.

By and large, there was a consensus on the need to encourage social accountability and citizen

engagement processes. To accomplish these processes, the participants recommended the use of

a strong civil society, pressure on the government from local NGOs and donors to perform,

dissemination of information through media, wider tax base, and a more effective and devolved

decentralization process.

35

RECOMMENDATIONS

The Project’s theory of change needs to clearly highlight the problems that the intervention aims

to address with the inputs, immediate outputs, and intended short-term and long-term outcomes.

Identifying the specific problem to focus on for each sector is important because the policy

prescriptions that stem out of each problem are different. Specific social accountability indicators

are necessary to capture the theory of change more effectively.

Citizen engagement that triggers demand-side accountability processes has to go in tandem with

building state capacity to interact with and respond to these new accountability pressures. We

advise introducing community monitoring as part of a broader package of services, well-

integrated with the national development framework of Burkina Faso to engender state capacity

and buy-in.

Community monitoring interventions require a greater timeline to accommodate for slow

behavioral and attitudinal changes, and their impact on service delivery outcomes.

Social inclusion must be built into the design of community monitoring interventions from

inception. An ad hoc treatment later in the project cycle risks negating the project outcomes and

gains in other areas. The impact of citizen engagement projects depends on the inclusivity of all

the voices in the target population.

36

BIBLIOGRAPHY

Ackerman, J.M. (2005). Social Accountability in the Public Sector: A Conceptual Discussion.

Washington, D.C.: World Bank.

Baldwin, Kate, et al. "Does Community-Based Development Empower Citizens? Evidence from

a Randomized Evaluation in Ghana." J-PAL, Oct. 2016, Accessed 25 Jan. 2017.

Brinkerhoff, Derick W., and Anna Wetterberg. "Gauging the Effects of Social Accountability on

Services, Governance and Citizen Empowerment." Public Administration Review 76, no. 2

(2015): 274-86. Accessed May 12, 2017. doi:DOI: 10.1111/puar.12399.

Corbridge, S., G. Williams, M. Srivastava, and R. Veron. 2005. Seeing the State: Governance

and Governmentality in India. Cambridge: Cambridge University Press.

Edstrom, Judith. "Engaging Citizens in Health Service Delivery: A Review of the Literature."

Partnership for Transparency Fund, USAID, 31 Aug. 2015, ptfund.org/engaging-citizens-

in-health-service-delivery-a-literature-review/.

Fox, Jonathan. 2014. Social Accountability: What Does the Evidence Really Say?

Global Partnership for Social Accountability Working Paper no. 1, World

Bank.

Geddes, B. 1994. Politician’s Dilemma: Building State Capacity in Latin America. Berkeley:

University of California Press.

37

“Grandvoinnet, Helene; Aslam, Ghazia; Raha, Shomikho. 2015. Opening the Black Box : The

Contextual Drivers of Social Accountability. New Frontiers of Social Policy;. Washington,

DC: World Bank. © World Bank.

https://openknowledge.worldbank.org/handle/10986/21686 License: CC BY 3.0 IGO.”

Healey, F. Henry, Derick W. Brinkerhoff, and Jana C. Hertz. "Social Accountability in

Education: The Northern Education Initiative in Nigeria." In Governance and Service

Delivery: Practical Applications of Social Accountability across Sectors., edited by Anna

Wetterberg. (RTI Press Publication No. BK-0019-1609). Research Triangle Park, NC: RTI

Press. DOI: 10.3768/rtipress.2016.bk.0019.1609

"Making Politics Work for Development: Harnessing Transparency and Citizen Engagement."

Open Knowledge Repository, openknowledge.worldbank.org/handle/10986/24461.

Mohanty, R (2007) ‘Gendered Subjects, the State and Participatory Spaces: The Politics of

Domesticating Participation in Rural India’, in A. Cornwall and V. Schattan Coelho (eds),

Spaces for Change? The Politics of Participation in New Democratic Arenas, London and

New York: Zed

Bjorkman, M., and J. Svensson. 2007. “Power to the People: Evidence from a Randomized

Field Experiment of a Community-Based Monitoring Project in Uganda.” Policy Research

Working Paper 4268, World Bank, Washington, DC.

38

de Briey, Valerie. Projet D’appui Au Suivi Evaluation Participatif Des Projets De La Banque

Mondiale Dans Les Secteurs Education Et Sante Au Burkina Faso. Washington DC:

Independent Evaluation Group at World Bank Group, 2015.

Duflo, E., P. Dupas, and M. Kremer. 2008. “Peer Effects, Pupil-Teacher Ratios, and Teacher

Incentives: Evidence from a Randomized Evaluation in Kenya.” Online Working Paper

CCPR 055-08, California Center for Population Research, University of California, Los

Angeles.

Freedom House. 2016. “2016 Freedom in the World - Burkina Faso.”

https://freedomhouse.org/report/freedom-world/2016/burkina-faso.

Gaventa, J., and G. Barrett. 2010. “So What Difference Does It Make? Mapping the Outcomes

of Citizen Engagement.” IDS Working Paper 347, Institute of Development Studies,

Brighton, United Kingdom.

Malena, C., and M. McNeil. 2010. “Social Accountability in Africa: An Introduction.” In

Demanding Good Governance: Lessons from Social Accountability Initiatives in Africa,

edited by C. Malena and M. McNeil. Washington DC: World Bank.

Manandhar, D. S., D. Osrin, B. P. Shrestha, N. Mesko, J. Morrison, K. M. Tumbahangphe, and

S. Tamang. 2004. “Effect of a Participatory Intervention with Women’s Groups on Birth

Outcomes in Nepal: Cluster Randomised Controlled Trial.” Lancet 364: 970–79.

39

Miguel, Edward, and Michael Kremer. "Worms: Identifying Impacts on Education and Health in

the Presence of Treatment Externalities." Econometrica 72, no. 1 (2004), 159-217.

doi:10.1111/j.1468-0262.2004.00481.x.

Molina, Ezequiel, Laura Carella, Ana Pacheco, Guillermo Cruces, and Leonardo Gasparini.

"Community monitoring interventions to curb corruption and increases access and quality

of service delivery in low-and middle-income countries: a systematic review." November

2016. Accessed January 25, 2017.

file:///C:/Users/sarut_000/Downloads/0150_IDCG_Molina_Community_Monitoring_Final

.pdf

Tripathy, P., N. Nair, S. Barnett, R. Mahapatra, J. Borghi, S. Rath, and S. Rath. 2010. “Effect of

a Participatory Intervention with Women’s Groups on Birth Outcomes and Maternal

Depression in Jharkhand and Orissa, India: A Cluster-Randomised Controlled Trial.”

Lancet 375: 1182–92.

Nguyen, T., and M. Rieger. 2014. “Community-driven development and social capital: evidence

from Morocco.” EUI Working Paper MWP 2014/02

O’Meally, S. 2013. “Mapping Context for Social Accountability: A Resource Paper.”

Background for this book, World Bank, Social Development Department, Washington,

DC.

40

Stokes, S (2003) “Perverse Accountability: Monitoring Voters and Buying Votes.” Paper

presented at the 2003 Annual Meeting of the American Political Science Association,

August 31.

United Nations Development Programme. 2015. “Burkina Faso Human Development

Indicators.” http://hdr.undp.org/en/countries/profiles/BFA.

Varshney, Ashutosh (1999) “Democracy and Poverty,” paper for the Conference on World

Development Report 2000, organized by the U.K. Department for International

Development, and the Institute of Development Studies, Sussex, Caste Donnington,

England, August 15-16, 1999.

Westhorp, Gill, Bill Walker, and Patricia Rogers. 2012. Under What Circumstances

Does Enhancing Community Accountability and Empowerment Improve Education

Outcomes, Particularly for the Poor? A Realist Synthesis. London: University of

London, Institute of Education, Social Science Research Unit, Evidence for

Policy and Practice Information and Co-ordinating Centre.

Wetterberg, A., Brinkerhoff, D., & Hertz, J. (Eds.) (2016). Governance and service delivery:

Practical applications of social accountability across sectors. (RTI Press Publication No. BK-

0019-1609). Research Triangle Park, NC: RTI Press. DOI:

10.3768/rtipress.2016.bk.0019.1609

41

World Bank. 2016. “Burkina Faso Overview, World Bank, Washington, DC, October 11.

http://www.worldbank.org/en/country/burkinafaso/overview.

World Bank. 2013. “Burkina Faso - Country partnership strategy for the period FY2013-2016.”

World Bank, Washington DC.

http://documents.worldbank.org/curated/en/937251468005970222/Burkina-Faso-

Country-partnership-strategy-for-the-period-FY2013-2016

World Bank. 2012. “Strengthening Governance, Tackling Corruption: The World Bank’s

Updated Strategy and Implementation Plan.” World Bank, Washington, DC, March 6.

http://siteresources.worldbank.org/PUBLICSECTORANDGOVERNANCE/Resources/

285741-1326816182754/GACStrategyImplementationPlan.pdf

42

APPENDIX

INTERVIEW GUIDES

Interview Guide 1: Beneficiaries/ citizens in the field

Citizens Engagement:

1. Do you think your local government (name the lowest tier) is responsive to your

community’s needs? (Note: response should not be dichotomous)

a. Why or why not?

2. Who is allowed to do the following?

a) Own land

b) Speak to public officials

c) Speak in community meetings.

d) Vote

e) Seek jobs

3. Can everyone talk to government officials? Can all kinds of men talk to government

officials? What are the exceptions?

4. Can women talk with government officials? If not, why? Are there any exceptions?

5. What are the organizations that bring people together? (Probe: Community based

organizations, cooperatives, kinship groups, religious organizations, etc)

How do they interact with government on behalf of the community?

6. How are decisions made within your community? For instance, if a community is faced

with a problem, like a dysfunctional health center and/or school, who is responsible to make

decisions about dealing with such issues? What is the process for such decisions?

Probe: Who makes the key decisions like these? (Is it a group of people or a single

individual?)

7. What types of people speak or participate in community meetings? Why?

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

8. What types of people do not speak or participate in community meetings? Why not?

43

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

9. Does your community have people from many different ethnic backgrounds? If so, how

does that affect your ability to work together or to come to an agreement?

NGOs, CSOs and Multilaterals:

1. How can communities work together with local government bodies to improve their

health or education?

2. Do you think that local governments should be held responsible for their performance

by the community or their constituents? Have you seen this in your community? Give examples.

3. What impact do you think your/ the local government has in improving healthcare or

education? Has the progress made so far been satisfactory?

4. What impact do you think the NGOs have had in improving healthcare or education? Has

the progress made so far been satisfactory? (probe: local or international)

5. Do you think social accountability at higher tiers of government can increase the

effectiveness of community monitoring interventions in Burkina Faso?

a. Why or why not?

6. What are the strong points and challenges for the local and central government to

coordinate health and education service delivery? Why or Why not?

7. Do you think that multilateral institutions (e.g. World Bank) can help the communities in

holding service providers responsible in any way? Suppose that you are in local clinic or CSPS,

the health attendants are rarely present, how can multilaterals help you to improve their

performance?

44

8. Do you think that local government, multilateral institutions and local NGOs should work

together to improve health and education service delivery?

9. How should the government, multi-lateral institutions and local NGOs work together to

improve health and education service delivery in Burkina Faso?

10. Can you think of a case where decentralization and community empowerment worked

well to improve health and education service delivery? Give examples. (Probe using a context-

specific example.)

11. In your opinion, how can social accountability approaches include all citizens? (Probe:

Which approaches come to mind such as community score cards, social audits, grievance

redressal).

45

Interview Guide 2: NGOs, CSOs, Multilaterals

Citizens Engagement:

1. Do you believe the local government in Burkina knows about your concerns about poor

health or education? And do they address those concerns as you wish?

2. Who is allowed to do the following?

a) Own land

b) Speak to public officials

c) Speak in community meetings.

d) Vote

e) Seek jobs

3. Can everyone talk to government officials? Can all kinds of men talk to government

officials? What are the exceptions?

4. Can women talk with government officials? If not, why? Are there any exceptions?

5. What are the organizations that bring people together? (Probe: Community based

organizations, cooperatives, kinship groups, religious organizations, etc.)

How do they interact with government on behalf of the community?

6. How are decisions made within your community? For instance, if a community is faced

with a problem, like a dysfunctional health center and/or school, who is responsible to make

decisions about dealing with such issues? What is the process for such decisions?

Probe: Who makes the key decisions like these? (Is it a group of people or a single

individual?)

7. What types of people speak or participate in community meetings? Why?

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

46

8. What types of people do not speak or participate in community meetings? Why not?

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

9. Does your community have people from many different ethnic backgrounds? If so, how

does that affect your ability to work together or to come to an agreement?

NGOs, CSOs, Multilaterals:

1. Do you believe the local government in Burkina Faso is generally responsive in terms of

service delivery such as an unfit health center or a school?

2. How can communities work together with local government bodies to improve their

health or education?

3. Do you think that local governments should be held responsible for their performance

by the community or their constituents? Have you seen this in your community? Give examples.

4. What impact do you think your/ the local government has in improving healthcare or

education? Has the progress made so far been satisfactory?

5. What impact do you think the NGOs have had in improving healthcare or education? Has

the progress made so far been satisfactory? (probe: local or international)

6. How do you see the role of NGOs or CSOs like yourself in improving health and

education by directly involving communities?

7. In your opinion, what have NGOs done to assist local government perform their duties in

improving health and education?

47

8. Do you think social accountability at higher tiers of government can increase the

effectiveness of community monitoring interventions in Burkina Faso?

b. Why or why not?

9. What are the strong points and challenges for the local and central government to

coordinate health and education service delivery? Why or Why not?

10. Do you think that multilateral institutions (e.g. World Bank) can help the communities

in holding service providers responsible in any way? Suppose that your in local clinic or CSPS,

the health attendants are rarely present, how can multilaterals help you to improve their

performance?

11. Do you think that local government, multilateral institutions and local NGOs should

work together to improve health and education service delivery?

12. How should the government, multi-lateral institutions and local NGOs work together to

improve health and education service delivery in Burkina Faso?

13. Can you think of a case where decentralization and community empowerment worked

well to improve health and education service delivery? Give examples. (Probe using a context-

specific example.)

14. Do you think social accountability approaches are generally inclusive? (NGO

specific). Why or why not?

15. In your opinion, how can social accountability approaches include all citizens?

(Probe: Which approaches come to mind such as community score cards, social audits, grievance

redressal).

48

Interview Guide 3: ISSP staff or members of the CMP Pilot Committee

Citizens Engagement:

1. Do you believe the local government in Burkina knows about your concerns about poor

health or education? And do they address those concerns as you wish?

2. Who is allowed to do the following?

a) Own land

b) Speak to public officials

c) Speak in community meetings.

d) Vote

e) Seek jobs

3. Can everyone talk to government officials? Can all kinds of men talk to government

officials? What are the exceptions?

4. Can women talk with government officials? If not, why? Are there any exceptions?

5. What are the organizations that bring people together? (Probe: Community based

organizations, cooperatives, kinship groups, religious organizations, etc.)

6. How do they interact with government on behalf of the community?

7. How are decisions made within your community? For instance, if a community is faced

with a problem, like a dysfunctional health center and/or school, who is responsible to make

decisions about dealing with such issues? What is the process for such decisions?

Probe: Who makes the key decisions like these? (Is it a group of people or a single

individual?)

8. What types of people speak or participate in community meetings? Why?

49

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

9. What types of people do not speak or participate in community meetings? Why not?

Probe: (young or older men, ethnicity, older and younger women, socio economic status)

10. Does your community have people from many different ethnic backgrounds? If so, how

does that affect your ability to work together or to come to an agreement?

NGOs, CSOs, Multilaterals:

1. Do you believe the local government in Burkina Faso is generally responsive in terms of

service delivery such as a unfit health center or a school?

2. How can communities work together with local government bodies to improve their

health or education?

3. Do you think that local governments should be held responsible for their performance

by the community or their constituents? Have you seen this in your community? Give examples.

4.What impact do you think your/ the local government has in improving healthcare or

education? Has the progress made so far been satisfactory?

5. What impact do you think the NGOs have had in improving healthcare or education? Has

the progress made so far been satisfactory? (probe: local or international)

6. How do you see the role of NGOs or CSOs like yourself in improving health and

education by directly involving communities?

50

7. In your opinion, what have NGOs done to assist local government perform their duties in

improving health and education?

8. Do you think social accountability at higher tiers of government can increase the

effectiveness of community monitoring interventions in Burkina Faso? Why or why not?

9. What are the strong points and challenges for the local and central government to

coordinate health and education service delivery? Why or Why not?

10. Do you think that multilateral institutions (e.g. World Bank) can help the communities in

holding service providers responsible in any way? Suppose that your in local clinic or CSPS, the

health attendants are rarely present, how can multilaterals help you to improve their

performance?

11. Do you think that local government, multilateral institutions and local NGOs should work

together to improve health and education service delivery?

12. How should the government, multi-lateral institutions and local NGOs work together to

improve health and education service delivery in Burkina Faso?

13. Can you think of a case where decentralization and community empowerment worked

well to improve health and education service delivery? Give examples. (Probe using a context-

specific example.)

14. Do you think social accountability approaches are generally inclusive? (NGO specific).

Why or why not?

15. In your opinion, how can social accountability approaches include all citizens? (Probe:

Which approaches come to mind such as community score cards, social audits, grievance

redressal).

51

ISSP-specific questions

1. Do you think the CMP project was successful?

a. Why or why not?

2. Do you think the CMP was more successful in some regions as opposed to the other?

Why or why not?

3. Did you think the CSC was a good approach and why was it used as opposed to other

approaches? Why or why not?

4. What did you think were the most pressing challenges in the implementation of the action

plans?

5. Do you think the project can continue without funding from the World Bank? Why or why

not?

6. Did the project try to enable certain excluded groups to participate and voice their

concerns? What can be done about this going forward?

7. Whose concerns are still not heard?

8. How could the project have been improved? Give examples

9. Do you think the project would be effective in other regions of Burkina Faso? Why or

why not?

10. How would you change this project if and when implementing in other regions of

52

Burkina?

Interview Guide 4: Focus Group Discussion - COGES, AME, APE

1. Can you please describe what your organization does?

2. Do you believe your organization representative of all members of the community?

3. Do you believe your organization is able to serve your community well?

4. How does your organization learn about citizens’ concerns regarding health and

education facilities?

5. What improvements have you been able to make at schools and health facilities?

6. Are there any improvements you would have liked to make but have been unable to?

7. What do you do if there is an issue at education or health facilities that you cannot

resolve?

8. Does your organization meet with the local government structures?

Probe: village development committee, municipal council, and mayor

9. Is the local government responsive to your organization’s requests and concerns?

10. Is the central government responsive to your organization’s requests and concerns?

11. How can the local government support your organization in fulfilling its mandate?

12. How can the central government support your organization in fulfilling its mandate?

13. Does your organization work with NGOs to facilitate education and health service

delivery?

14. How can NGOs support your organization in fulfilling its mandate?

53

MODELS

Model 1a.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls

p_time 0.0920*** 0.0940*** 0.0942*** 0.0939*** 0.0962*** 0.0983*** 0.0984***

(0.0244) (0.0245) (0.0245) (0.0246) (0.0258) (0.0263) (0.0262)

csps_dir 0.0281 0.0262* 0.0258 0.0258 0.0265 0.0270 0.0254

(0.0212) (0.0153) (0.0154) (0.0154) (0.0165) (0.0166) (0.0164)

time_treat -0.0613* -0.0680* -0.0677* -0.0669* -0.0685* -0.0683* -0.0634*

(0.0344) (0.0340) (0.0340) (0.0340) (0.0348) (0.0350) (0.0350)

Plateau-Central -0.0966*** -0.0967*** -0.0971*** -0.0947*** -0.0957*** -0.0944***

(0.0173) (0.0173) (0.0176) (0.0175) (0.0177) (0.0179)

Sahel -0.0715*** -0.0713*** -0.0713*** -0.0689*** -0.0697*** -0.0693***

(0.0237) (0.0238) (0.0237) (0.0241) (0.0243) (0.0248)

p_child_sex -0.0164 -0.0166 -0.0155 -0.0155 -0.0153

(0.0116) (0.0115) (0.0115) (0.0115) (0.0115)

p_ape_ame_coges_b -0.00517 -0.00747 -0.00887 -0.00764

54

(0.0166) (0.0169) (0.0164) (0.0172)

Other relative 0.0502 0.0511 0.0520

(0.0376) (0.0374) (0.0383)

Extended family/clan -0.0134 -0.0126 -0.00416

(0.0233) (0.0228) (0.0222)

Close Friend -0.0398 -0.0389 -0.0401

(0.0329) (0.0332) (0.0353)

Friend of a friend -0.0640 -0.0595 -0.0696

(0.0574) (0.0581) (0.0698)

Known person -0.00249 -0.00168 -0.00493

(0.0204) (0.0202) (0.0199)

none -0.00329 0.000245 0.00317

(0.0209) (0.0219) (0.0219)

itself 0.000101 -0.000459 -0.00203

(0.0483) (0.0479) (0.0451)

NA -0.0790*** -0.0796*** -0.0887***

(0.0243) (0.0248) (0.0314)

Other -0.0542 -0.0539 -0.0528

(0.0528) (0.0518) (0.0517)

55

p_village_meeting 0.0165 0.0142

(0.0207) (0.0207)

Other relative 0.0278

(0.0553)

Extended family/clan -0.0852**

(0.0361)

Close friend 0.0145

(0.0290)

Friend of a friend 0.0437

(0.0868)

Known person 0.0244

(0.0170)

none -0.00637

(0.0184)

itself -0.00270

(0.0211)

Other 0.846***

(0.0348)

Constant 0.0746*** 0.134*** 0.158*** 0.160*** 0.159*** 0.144*** 0.140***

56

(0.0113) (0.0179) (0.0249) (0.0264) (0.0289) (0.0297) (0.0333)

Observations 2,903 2,903 2,903 2,903 2,903 2,903 2,903

R-squared 0.011 0.027 0.028 0.028 0.030 0.030 0.036

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Model 1b.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls

p_time 0.175 0.154 0.155 0.151 0.182 0.185 0.143

(0.135) (0.131) (0.131) (0.129) (0.114) (0.114) (0.110)

csps_dir 0.0427 0.0297 0.0298 0.0289 0.0297 0.0285 0.0179

(0.123) (0.107) (0.107) (0.107) (0.112) (0.111) (0.108)

time_treat -0.0597 -0.0754 -0.0754 -0.0695 -0.0814 -0.0743 -0.0699

(0.184) (0.171) (0.171) (0.173) (0.173) (0.173) (0.173)

57

Plateau-Central -0.266** -0.266** -0.267** -0.263** -0.269** -0.275**

(0.114) (0.114) (0.115) (0.114) (0.111) (0.110)

Sahel -0.368*** -0.369*** -0.368*** -0.345*** -0.351*** -0.354***

(0.116) (0.116) (0.116) (0.121) (0.117) (0.112)

p_child_sex -0.0114 -0.0113 -0.0144 -0.0166 -0.0158

(0.0884) (0.0885) (0.0896) (0.0889) (0.0888)

p_ape_ame_coges_b -0.0262 -0.0315 -0.0386 -0.0546

(0.0743) (0.0778) (0.0768) (0.0742)

Other relative 0.0974 0.0997 0.0931

(0.219) (0.217) (0.215)

Extended family/clan -0.0975 -0.0897 -0.0632

(0.160) (0.156) (0.135)

Close friend -0.0630 -0.0522 -0.0628

(0.276) (0.281) (0.269)

Friend of a friend 0.425 0.457 0.725*

(0.402) (0.385) (0.390)

Known person -0.0710 -0.0680 -0.0571

(0.0834) (0.0843) (0.0876)

none -0.247 -0.232 -0.244

58

(0.173) (0.173) (0.178)

itself -0.201 -0.205 -0.217

(0.427) (0.428) (0.392)

NA -1.291*** -1.291*** -1.276***

(0.257) (0.256) (0.246)

Other -0.163 -0.151 -0.132

(0.987) (1.003) (1.015)

p_village_meeting 0.0743 0.0546

(0.109) (0.104)

Other relative -0.215

(0.358)

Extended family/clan 0.0482

(0.335)

Close Friend 0.163

(0.224)

Friend of a Friend -0.611

(0.426)

Known Person -0.0375

(0.105)

59

none 0.0266

(0.111)

itself 0.229

(0.167)

Constant -0.548*** -0.312*** -0.295* -0.284 -0.240 -0.299 -0.273

(0.0813) (0.102) (0.165) (0.171) (0.179) (0.224) (0.215)

Observations 1,695 1,695 1,695 1,695 1,695 1,695 1,695

R-squared 0.002 0.013 0.013 0.013 0.017 0.018 0.022

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

60

Model 2a.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls

p_time 0.0885*** 0.0884*** 0.0884*** 0.0886*** 0.0874*** 0.0876*** 0.0874***

(0.0158) (0.0159) (0.0159) (0.0159) (0.0159) (0.0161) (0.0164)

csps_indir 0.00724 0.00399 0.00399 0.00307 0.00356 0.00360 0.00286

(0.00821) (0.00773) (0.00773) (0.00775) (0.00773) (0.00776) (0.00804)

time_treatindir -0.0370** -0.0364* -0.0364* -0.0364* -0.0378** -0.0378** -0.0367**

(0.0184) (0.0185) (0.0185) (0.0185) (0.0185) (0.0184) (0.0185)

Plateau-Central -0.0354** -0.0354** -0.0350** -0.0368** -0.0368** -0.0378***

(0.0142) (0.0142) (0.0140) (0.0143) (0.0143) (0.0141)

Sahel -0.0157 -0.0157 -0.0150 -0.0145 -0.0144 -0.0144

(0.0143) (0.0143) (0.0141) (0.0145) (0.0145) (0.0144)

p_child_sex 0.000794 0.000737 0.000450 0.000433 0.000299

(0.00467) (0.00467) (0.00470) (0.00470) (0.00467)

p_ape_ame_coges_b 0.0120* 0.0134* 0.0132* 0.0123*

(0.00696) (0.00708) (0.00721) (0.00723)

Other relative -0.00932 -0.00925 -0.0131

61

(0.0163) (0.0162) (0.0159)

Extended family/clan 0.00741 0.00736 0.0129

(0.0105) (0.0105) (0.0110)

Close friend 0.0151 0.0152 0.0168

(0.0209) (0.0209) (0.0211)

Friend of a Friend 0.0326 0.0326 0.0365

(0.0398) (0.0399) (0.0394)

Known Person 0.0223** 0.0223** 0.0200**

(0.00860) (0.00862) (0.00886)

none 0.0103 0.0105 0.0127

(0.0126) (0.0129) (0.0130)

itself 0.0289 0.0287 0.0298

(0.0215) (0.0214) (0.0216)

NA -0.0165 -0.0165 -0.0187

(0.0168) (0.0170) (0.0171)

Other -0.0104 -0.0104 -0.0180

(0.0391) (0.0392) (0.0391)

p_village_meeting 0.00259 0.000943

(0.0107) (0.0109)

62

Other relative 0.0659**

(0.0267)

Extended family/clan -0.0351**

(0.0145)

Close friend -0.00214

(0.0177)

Friend of a friend -0.0410

(0.0293)

Known person 0.0194*

(0.0109)

none -0.000930

(0.00821)

itself 0.00391

(0.0137)

NA -0.0749***

(0.0118)

Other -0.00731

(0.111)

Constant 0.0617*** 0.0858*** 0.0846*** 0.0802*** 0.0741*** 0.0716*** 0.0699***

63

(0.00596) (0.0140) (0.0153) (0.0156) (0.0166) (0.0196) (0.0202)

Observations 13,362 13,362 13,362 13,362 13,362 13,358 13,358

R-squared 0.013 0.015 0.015 0.016 0.017 0.017 0.019

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Model 2b.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls

p_time 0.181** 0.211** 0.211** 0.210** 0.0892 0.111 0.118

(0.0740) (0.0816) (0.0815) (0.0812) (0.130) (0.118) (0.111)

csps_indir 0.724 0.735 0.733 0.727 0.705 0.703 0.705

(0.541) (0.574) (0.573) (0.573) (0.561) (0.560) (0.561)

time_treatindir -0.734 -0.734 -0.733 -0.733 -0.683 -0.667 -0.667

(0.540) (0.530) (0.529) (0.529) (0.497) (0.490) (0.489)

64

Plateau-Central -0.241** -0.240** -0.245** -0.309*** -0.303*** -0.310***

(0.117) (0.117) (0.119) (0.107) (0.108) (0.112)

Sahel 0.169 0.168 0.165 0.0299 0.0388 0.0319

(0.427) (0.426) (0.426) (0.340) (0.345) (0.338)

p_child_sex -0.0380 -0.0383 -0.0257 -0.0288 -0.0285

(0.307) (0.306) (0.295) (0.297) (0.292)

p_ape_ame_coges_b 0.0655 0.0985 0.0772 0.0959

(0.328) (0.301) (0.313) (0.268)

Other relative -0.246 -0.239 -0.209

(0.289) (0.288) (0.258)

Extended family/clan -0.241 -0.240 -0.224

(0.240) (0.240) (0.193)

Close Friend -0.353 -0.337 -0.343*

(0.242) (0.244) (0.201)

Friend of a Friend -0.467* -0.460* -0.424

(0.270) (0.267) (0.274)

Known Person -0.150 -0.147 -0.141

(0.186) (0.186) (0.134)

None 1.316 1.331 1.302

65

(1.414) (1.427) (1.363)

Itself 0.0703 0.0488 0.0522

(0.238) (0.224) (0.162)

NA -1.067 -1.065 -1.039

(0.727) (0.726) (0.714)

Other -0.654** -0.638** -0.627***

(0.270) (0.286) (0.234)

p_village_meeting 0.221 0.245

(0.206) (0.245)

Other relative -0.514

(0.431)

Extended family/clan -0.0731

(0.336)

Close friend -0.0636

(0.340)

Friend of a friend 0.0756

(0.419)

Known person -0.117

(0.357)

66

none 0.0395

(0.414)

itself -0.0756

(0.372)

NA -0.644

(0.829)

Other -0.230

(0.421)

Constant -0.573*** -0.557** -0.499 -0.515 -0.411 -0.616 -0.607**

(0.0453) (0.263) (0.502) (0.571) (0.472) (0.417) (0.260)

Observations 7,510 7,510 7,510 7,510 7,510 7,510 7,510

R-squared 0.000 0.001 0.001 0.001 0.002 0.002 0.002

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

67

Model 3a.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 4 Controls 5 Controls 6 Controls

p_time 0.0777** 0.0765** 0.0765** 0.0774** 0.0736** 0.0679** 0.0701**

(0.0308) (0.0313) (0.0313) (0.0308) (0.0326) (0.0333) (0.0338)

school 0.0166 0.0141 0.0142 0.0127 0.00949 0.0113 0.00906

(0.0205) (0.0187) (0.0186) (0.0189) (0.0183) (0.0187) (0.0173)

time_treatschool -0.0101 -0.0103 -0.0103 -0.0100 -0.00970 -0.00868 -0.00680

(0.0390) (0.0396) (0.0396) (0.0397) (0.0409) (0.0414) (0.0411)

Plateau-Central -0.0411* -0.0412* -0.0407* -0.0424* -0.0421* -0.0443*

(0.0223) (0.0223) (0.0222) (0.0236) (0.0237) (0.0238)

Sahel -0.0269 -0.0268 -0.0262 -0.0197 -0.0213 -0.0218

(0.0249) (0.0249) (0.0246) (0.0245) (0.0239) (0.0242)

p_child_sex 0.00305 0.00288 0.00113 0.00115 0.000370

(0.0116) (0.0115) (0.0117) (0.0117) (0.0116)

p_ape_ame_coges_b 0.0114 0.00967 0.0126 0.0166

(0.0148) (0.0156) (0.0159) (0.0158)

Other relative -0.0674** -0.0735*** -0.0797***

68

(0.0254) (0.0243) (0.0236)

Extended family/clan -0.0202 -0.0219 -0.0183

(0.0250) (0.0254) (0.0264)

Close friend -0.0454 -0.0497 -0.0453

(0.0323) (0.0333) (0.0372)

Friend of a friend 0.00963 0.00389 0.00505

(0.0968) (0.0881) (0.0838)

Known person -0.00195 -0.00563 -0.00707

(0.0198) (0.0202) (0.0196)

none -0.0368 -0.0387 -0.0441

(0.0274) (0.0273) (0.0284)

itself -0.0475 -0.0464 -0.0564

(0.0426) (0.0407) (0.0431)

NA -0.0432 -0.0582 -0.0705*

(0.0363) (0.0363) (0.0356)

Other -0.0349 -0.0415 -0.0433

(0.0688) (0.0637) (0.0637)

p_village_meeting -0.0570** -0.0562**

(0.0249) (0.0257)

69

Other relative 0.0678

(0.0560)

Extended family/clan -0.0162

(0.0374)

Close friend -0.0127

(0.0354)

Friend of a friend 0.0750

(0.0834)

Known person 0.00745

(0.0185)

none 0.0185

(0.0165)

itself -0.0119

(0.0249)

Other 0.204

(0.276)

Constant 0.0693*** 0.0943*** 0.0897*** 0.0845** 0.105*** 0.158*** 0.152***

(0.0173) (0.0272) (0.0304) (0.0311) (0.0326) (0.0380) (0.0399)

70

Observations 2,676 2,676 2,676 2,676 2,676 2,672 2,672

R-squared 0.014 0.017 0.017 0.017 0.021 0.024 0.028

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1

Model 3b.

(1) (2) (3) (4) (5) (6) (7)

VARIABLES No Controls 1 Control 2 Controls 3 Controls 34 Controls 5 Controls 6 Controls

p_time 0.319 0.297 0.300 0.295 0.253 0.246 0.277

(0.272) (0.275) (0.282) (0.281) (0.248) (0.261) (0.299)

school -0.108 -0.0479 -0.0458 -0.0540 -0.0419 -0.0439 -0.0171

(0.173) (0.140) (0.142) (0.147) (0.150) (0.149) (0.157)

time_treatschool -0.108 -0.127 -0.131 -0.121 -0.120 -0.119 -0.149

(0.318) (0.313) (0.321) (0.324) (0.287) (0.288) (0.294)

2.p_region -0.188 -0.189 -0.199 -0.224 -0.224 -0.237

(0.211) (0.210) (0.222) (0.232) (0.232) (0.228)

3.p_region -0.395 -0.394 -0.406 -0.514 -0.514 -0.462

(0.332) (0.336) (0.358) (0.366) (0.367) (0.364)

71

p_child_sex 0.0290 0.0295 0.0473 0.0475 0.0520

(0.158) (0.158) (0.160) (0.160) (0.158)

p_ape_ame_coges_b 0.0607 0.0432 0.0484 0.120

(0.156) (0.147) (0.155) (0.154)

Other relative -0.105 -0.107 -0.204

(0.218) (0.216) (0.225)

Extended family/clan 0.0504 0.0495 -0.0842

(0.173) (0.173) (0.148)

Close Friend -0.499 -0.505 -0.626

(0.399) (0.386) (0.397)

Friend of a friend 0.473*** 0.475*** 0.772***

(0.134) (0.136) (0.236)

Known Person -0.00790 -0.0126 -0.149

(0.213) (0.201) (0.194)

None 0.490** 0.484** 0.398**

(0.175) (0.176) (0.172)

itself 0.445 0.452 0.387

(0.424) (0.422) (0.371)

NA -0.414 -0.419 -0.344

72

(0.300) (0.291) (0.282)

Other 0.521 0.509 0.484*

(0.312) (0.306) (0.242)

p_village_meeting -0.0489 0.00403

(0.203) (0.215)

Other relative 0.00253

(0.373)

Extended family/clan 0.691

(0.450)

Close friend -0.0676

(0.439)

Friend of a Friend 0.375

(0.404)

Known person 0.460*

(0.245)

none 0.495**

(0.223)

itself 0.258

(0.221)

73

Other 0.462

(0.707)

Constant -0.482*** -0.308* -0.353 -0.365 -0.370 -0.323 -0.667

(0.0610) (0.178) (0.365) (0.348) (0.353) (0.447) (0.554)

Observations 989 989 989 989 989 989 989

R-squared 0.008 0.014 0.014 0.014 0.024 0.024 0.039

Robust standard errors in parentheses

*** p<0.01, ** p<0.05, * p<0.1