Growth Week 2011: Ideas for Growth Session 5 - Governance, Accountability and Political Economy

Reshaping institutions: Evidence on external aid

and local collective action

Katherine Casey, Stanford University Rachel Glennerster, Abudl Latif Jameel Poverty Action Lab, MIT

Edward Miguel, University of California, Berkeley September 2011

Motivation (1) • Many scholars agree that institutions are important determinants of

economic development and that a history or hierarchical and noncompetitive institutions are associated with poor performance. – E.g. Banerjee and Iyer, 2004

• However, there is limited consensus on exactly what the “right” institutions are, and even less evidence on how to “improve” existing institutions in poor countries.

• Measuring institutional performance is challenging: --Subjective measures are prone to “halo effects” (Olken 2009) -- Institutions are multi-faceted, leaving open the risk of data mining or “cherry-picking” of results consistent with prior beliefs. -- Institutions are themselves affected by economic performance

2 IGC, September 2011 Reshaping Institutions

Motivation (2) • A key question is whether it is possible for foreign aid donors to

transform institutions in less developed countries? (Is it even desirable?)

• Among donors today, arguably the most popular strategy to promote accountability, competence and inclusion of under-represented groups in local government institutions is “community driven development” (CDD). Billions of dollars in donor funding per year.

• Influential evidence from India that “imposing” greater representation of minority groups can create a demonstration effect and empower minorities in the longer term

-- Beamen et al 2009


Motivation (3) • In this paper we evaluate one attempt to transform local

institutions in post-war Sierra Leone. We exploit a randomized experiment to assess CDD impacts on local public goods and local institutions.

• We develop new, objective institutional performance measures, and employ a pre-analysis plan to eliminate data mining.

• We show that without the preanalysis plan we could have cherry picked data to tell two very different stories: that CDD improved institutions, and that it undermined social capital.


Why might Sierra Leone’s institutions warrant reform?

• Legacy of bad governance and corruption in the formal system – Brief period of post-Independence (1961) stability and growth – President Siaka Stevens abolished local government (1972) and

banned rival political parties (1978), abysmal public services – Brutal civil war (1991-2002)

• The traditional system is (also) dominated by elder male elites – 149 Paramount Chiefs rule for life; come from hereditary ruling

houses; and control land, labor and the judiciary outside the capital – Women are not eligible for chieftaincy in most of the country

• Scholars point to seeds of unrest in social divisions and inequalities – Anger at the failings of the corrupt ruling regime (Richards 1996) – Inequality between chief and subject, including capricious fines,

coerced labor and unpopular land allocations (Keen 2003) – Disenfranchisement of women and youth from decision-making


What does CDD aim to do? • Financial grants for local public goods, small enterprise development

– The "GoBifo" Project ("Move Forward") we study in Sierra Leone gave $4,667 to communities in 3 tranches (~$100 per household)

• Training and facilitation to build durable local collective action capacity (6 months of intensive contact spread out over 4 years) – Forms a representative Village Development Committee to

promote democratic decision-making – Helps communities agree on a medium-term development plan – Establishes bank accounts and transparent accounting procedures

• Requirements to increase participation of marginalized groups – Women were co-signatories on the community bank accounts – Recorded how actively women, youths (18-35 years) participated – Women and youths managed own projects, e.g. labor groups


Elements of the study • Randomized experiment produces rigorous evidence on causal

impacts with a relatively large sample (236 villages, 2,832 households), and extended time frame (2005 to 2009).

• Combines survey data with three "structured community activities" (SCAs) that unobtrusively observe communities post-program: – responding to a matching grant opportunity, – making a communal decision, – managing a public asset.

• Hypothesis document agreed in 2004 of key objectives of program

• Analysis follows a pre-analysis plan based on hypothesis document, to avoid data mining.


8 U.C. Berkeley September 2011 Reshaping Institutions


Local public goods construction projects • The distribution of community projects by sector was:

– Infrastructure (43%) - e.g., community centers, primary schools – Agriculture/livestock (40%) - e.g., seed multiplication, goats – Skills training, small business (17%) - e.g., carpentry, soap-making


Data collection • Household survey panel (male, female, youth, non-youth respondents) • Field supervisor direct assessments of local public goods quality. • Village focus group discussions with local leaders. • A novel component - structured community activities (SCAs):

– Matching grant: communities received six vouchers that could be redeemed with a co-pay at a local building materials store (max value $300). A direct measure of collective action capacity.

– Communal choice: communities were presented with two equally valued assets (batteries vs. salt) and enumerators observed ensuing deliberations, recording the number of male/female and youth/elder speakers as measures of participation and influence.

– Managing an asset: communities were given a large tarpaulin, useful as an agricultural drying floor or roofing material. Focus on elite capture in a surprise follow-up visit 5 months later.


Pre-analysis plan • Tying the researcher’s hands with a pre-analysis plan limits

selective presentation (“cherry-picking”) of results (Leamer 1974, 1983), and produces appropriately sized statistical tests (Anderson 2008).

• Randomized experiments may not lead to more reliable results, if not all research results are published. Registering pre-analysis plans in a public archive can also limit publication bias.

• Over the past decade, pre-analysis plans for experimental studies (and some observational studies) have become standard in medical research, but this is one of the first economics studies with one.


Pre-analysis plan • Before the program began in 2005, the research and project teams

together agreed to a set of hypotheses about GoBifo impacts.

• Before analyzing any endline data, we submitted the exact list of outcome and explanatory variables under each hypothesis, and econometric specifications, to the J-PAL project registry. – Main plan submitted in August 2009 and a supplement concerning

outcomes from the second follow-up survey in March 2010, during the respective periods of data entry and reconciliation.

• Project impacts are determined by the mean treatment effect across

all outcomes under a given hypothesis (Kling and Leibman 2004). – Results for all 318 pre-specified outcomes in web appendix.


Econometric specifications • Basic model for outcomes with post-program data only: Yc = β0 + β1Tc + Xc′Γ + Wc′Π + εc

– Yc is outcome in community c (HH data averaged by village) – Tc is an indicator for GoBifo treatment – Xc is a vector of community-level controls (results are robust to

their exclusion); Wc are ward fixed effects – εc is an idiosyncratic error term

• Panel specification where longitudinal data available: Yc = β0 + β1(Tc * POSTt) + β2Tc + β3POSTc + Xc′Γ + Wc′Π + εc

– POSTt is an indicator for the post-program period (2009)


Hypothesized CDD impacts • 1) Participation requirements on women and youth aim to decrease

their disutility of participation – Requirements automatically translate into higher participation

during project implementation • 2) Grants subsidize construction and training, reduce the marginal

costs of public goods and start up of community enterprises during the program, leading to more construction and economic activity

• 3) The increase in community participation, plus establishment of committees, plans and bank accounts, reduces the fixed coordination costs of achieving collective action – If the organizing institutions are durable, the beneficial effects

should persist into the post-program period – If women and youths learn-by-doing or their participation exerts

positive demonstration effects on others, shifting social norms, this could trigger a sustained improvement in participation


Overview of results • Outcome family A: Positive infrastructure or hardware and activity

– Village-level structures and tools to manage development projects were established (e.g. bank accounts)

– Finances were disbursed with little leakage (<13% discrepancies) – Increases in the stock and quality of local public goods – Increases in household assets and village-level market activity

• Outcome family B: Zero impact on “software” / “institutions” – No impacts on participation in decision-making – No sustained increase in collective action capacity – No change in the “voice” of women and young men – Apparent “capture” of new organizations by chiefly authorities – Example of communal farms: established but low participation


17

Baseline mean for controls

T-C difference at baseline

N

(1) (2) (3)Panel A: Community CharacteristicsTotal households per community 46.76 0.30 236

(3.67)Distance to nearest motorable road in miles 2.99 -0.32 236

(0.36)Index of war exposure (range 0 to 1) 0.68 -0.01 236

(0.02)Historical legacy of domestic slavery (range 0 to 1) 0.36 0.03 236

(0.06)Average respondent years of education 1.65 0.11 235

(0.13)Panel B: Selected Outcomes from "Hardware" Family AProportion of communities with a Village development committee (VDC) 0.55 0.06 232

(0.06)Proportion visited by Ward Development Committee (WDC) member in past year 0.15 -0.01 228

(0.05)Proportion of communities with a functional primary school 0.41 0.08 230

(0.06)Average household asset score -0.06 0.11 235

(0.08)Proportion of communities with any petty traders 0.54 -0.01 226

(0.06)Panel C: Selected Outcomes from "Software" Family BRespondent agrees that chiefdom officials can be trusted 0.66 -0.01 235

(0.02)Respondent agrees that Local Councillors can be trusted 0.61 0.00 235

(0.02)Respondent is a member of credit / savings group 0.25 -0.03 235

(0.02)

Table 1: Baseline (2005) Comparison between Treatment and Control Communities


Hypotheses by family GoBifo Mean Effect

(std. error)Family A: Development Infrastructure or "Hardware" Effects

Mean Effect for Family A (Hypotheses 1, 2 and 3; 37 total outcomes) 0.352**(0.030)

H1: GoBifo creates functional development committees (7 outcomes) 0.687**(0.062)

H2: GoBifo increases the quality and quantity of local public services infrastructure (16 outcomes) 0.164**(0.040)

H3: GoBifo improves general economic welfare (14 outcomes) 0.399**(0.047)

Table 2: Summary of GoBifo Program Impacts by Research Hypothesis and Outcome Family

19

Row Mean in Controls

Treatment Effect

Standard Error

N Specification

1 0.458 0.341** (0.077) 467 Panel2 0.212 0.156* (0.070) 462 Panel3 0.617 0.296** (0.048) 221 Cross section4 0.081 0.706** (0.045) 226 Cross section

10 Primary School 0.462 -0.007 (0.050) 464 Panel11 Grain drying floor 0.237 0.104 (0.066) 459 Panel12 Traditional midwife post 0.079 0.175** (0.035) 235 Cross section13 Latrine 0.462 0.210** (0.059) 234 Cross section14 Community center 0.212 0.241** (0.063) 469 Panel15 0.292 -0.156+ (0.081) 460 Panel

25 2.432 0.719* (0.344) 225 Cross section26 4.449 0.560* (0.240) 236 Cross section27 -0.170 0.212* (0.090) 471 Panel28 2.835 0.158+ (0.094) 471 Panel29 0.061 0.119** (0.018) 235 Cross section30 746.94 -21.773 (73.069) 236 Cross section

Total goods on sale of 10Household asset scoreHousehold asset quintileAttended skills trainingIncome from top 3 cash earning sources (in 1,000 Leones)

Panel E: Hypothesis 3 - Full Sample OutcomesTotal petty traders in village

Community bank account

Panel C: Hypothesis 2 - Full Sample OutcomesExistence of functional local public good in the community:

Community took a proposal to an NGO or donor for funding

Table 3: Illustrative Treatment Effect Estimates for Selected Family A Outcome Measures

Outcome variable

Panel A: Hypothesis 1 - Full Sample OutcomesVillage development committeeVisit by WDC memberVillage development plan

20

Structured Community Activity (SCA) Outcome: Mean for Controls

Treatment Effect

Standard Error

N

(1) (2) (3) (4)Panel A. Collective Action and the Building Materials VouchersGoBifo Mean Effect for SCA #1 (13 outcomes in total) 0.00 -0.057 (0.053) 236Proportion of communities that redeemed any vouchers at the building material supply store 0.54 -0.01 (0.06) 236Average number of vouchers redeemed at the store (out of six) 2.95 0.11 (0.35) 236Proportion of communities that held a meeting after the research team left to discuss what to do with the vouchers 0.98 -0.05* (0.02) 231

Panel C. Community Use of TarpaulinGoBifo Mean Effect for SCA #3 (18 outcomes in total) 0.00 -0.032 (0.045) 236Proportion of communities that held a meeting after the research team left to discuss what to do with the tarp 0.98 -0.03 (0.02) 233Proportion of communities that stored the tarp in a public place 0.06 0.06 (0.04) 225Proportion of communities that had used the tarp by the follow up visit (5 months after receipt) 0.90 -0.08+ (0.04) 233Given use of the tarp, proportion of communities that used the tarp in a public way 0.86 0.02 (0.05) 161

21

Structured Community Activity (SCA) Outcome: Mean for Controls

Treatment Effect

Standard Error

N

(1) (2) (3) (4)

Panel B. Participation in Gift Choice DeliberationGoBifo Mean Effect for SCA #2 (32 outcomes in total) 0.00 0.005 (0.036) 236Duration of gift choice deliberation (in minutes) 9.36 1.60 (1.13) 225Total adults in attendance at gift choice meeting 54.51 3.50 (3.20) 236Total women in attendance at gift choice meeting 24.99 1.99 (1.68) 236Total youths (approximately 18 to 35 years old) in attendance at gift choice meeting 23.57 2.10 (1.38) 236Total number of public speakers during the deliberation 6.04 0.24 (0.40) 236

Total number of women who spoke publicly during the deliberation 1.88 -0.19 (0.22) 236Total number of youths (approximately 18 to 35 years old) who spoke publicly 2.14 0.23 (0.24) 236

Proportion of communities that held a vote during the deliberation 0.10 0.07 (0.04) 236

Illustrating the risk of “cherry-picking” (A) • Given our large number of outcome measures (318 in all), it is

possible to selectively present one subset of outcomes for which CDD had a “positive” impact on institutions, and a second subset of outcomes that show the opposite impact.

• Interpretation A: institutions deteriorated: – Heavy emphasis on participation led to “meeting fatigue”

which translated into poor management and political disaffection and disengagement


IGC< September 2011 Reshaping Institutions 23

Row Survey question Mean for controls

Treatment effect

Standard error

N Hypothesis

(1) (2) (3) (4) (5)Panel A: Institutions Deteriorated

1 Did you attend any meeting to decide what to do with the tarp after our team left your community (not the original gift meeting)? 0.812 -0.037+ 0.021 236 H5

2 Everybody in the village had equal say in deciding how to use the tarp

0.509 -0.106+ 0.058 232 H5

3 Correctly able to name what the tarp was used for or what the community's plan is for using the tarp

0.589 -0.08+ 0.048 236 H9

4 Has anyone in this community ever used the tarp? (verified by supervisor physical assessment)

0.897 -0.079+ 0.044 233 H4

5 Supervisor asks to see the tarp at second round follow-up visit: can the community show you the tarp?

0.836 -0.116* 0.051 232 H5

6 [Given not a member of the VDC] would you like to be a member of the VDC?

0.361 -0.043* 0.021 236 H10

7 Is the current (or acting) village chief/Headman less than 35 years old?

0.044 -0.038+ 0.023 229 H12

8 Did you vote in the local government election (2008)? 0.851 -0.036* 0.016 236 H10

Table 5: Alternative Interpretations

Illustrating the risk of “cherry-picking” (B)

• Interpretation B: positive institutional spillovers from GoBifo – Experience with GoBifo encouraged communities to

incorporate more democratic procedures into community decision making this created space for new leaders, and incited more engagement and activity including more involvement in womens groups and training of community teachers.

• Illustrates the clear value of having a pre-analysis plan in place.


IGC, September 2011 Reshaping Institutions 25

Panel B: Institutions Improved9 [Given community teachers at the school children in the

community attend] were the community teachers ever trained? 0.471 0.122+ 0.066 173 H4

10 Are you a member of any women's groups (general)? 0.235 0.060** 0.021 236 H811 Did anyone take minutes (written record of what was said) at the

most recent community meeting?0.295 0.140* 0.063 227 H5

12 Enumerator record of whether a vote occurred during the gift choice deliberation

0.097 0.069+ 0.042 236 H5, H6

13 Respondent does not choose a chiefdom official or elder in response to "who had the most influence over how the tarpaulin is used or whether to keep it in storage?"

0.543 0.058* 0.029 236 H6

14 Correctly able to name the year of the next general elections 0.192 0.038* 0.018 236 H915 Respondent agrees with "Responsible young people can be good

leaders" and not "Only older people are mature enough to be leaders"

0.762 0.038* 0.017 236 H6, H12

16 Proportion of female members of the VDC 0.209 0.066+ 0.037 151 H10

Row Survey question Mean for controls

Treatment effect

Standard error

N Hypothesis

(1) (2) (3) (4) (5)

Table 5: Alternative Interpretations

Robustness checks • Were there threats to the research design?

– Complete compliance with treatment group assignment – Baseline balance on observables across T/C groups

• Did control communities benefit from spillovers? – GoBifo operated at the ward level as well, so targeting was

possible. However, treatment households are if anything more likely to report direct benefits from ward project (15% versus 6%).

– There are roughly an equal number of positive and negative coefficients on the POST indicator in our outcome equations.

• Are our measures simply too blunt to detect subtle changes? – Large and diverse number of outcomes for each hypothesis, 318 in

all. Consistent results across different data collection methods: HH surveys, direct observation, focus group discussions, and SCAs.


Conclusion: GoBifo results and questions • The CDD project was a reasonable mechanism for delivering local scale

public goods in Sierra Leone. It gave women more voice temporarily yet did not lead to lasting changes in local collective action, village institutions, gender inclusion, or social norms.

• Open questions for future research: – Did increased community participation during the project facilitate

local public goods provision? In other words, would the positive Family A outcomes have materialized in the absence of activities designed to promote community involvement?

– If not problems with coordination and participation, what is holding communities back from engaging in more collective action?

– Would a project that explicitly attempted to weaken traditional chiefly authorities have had greater impact?


Our results in context • The comparative advantage of the World Bank and similar external

donors may lie more in building development hardware than in instigating sustainable social change. Our results are positive on delivering concrete improvements in a challenging setting.

• Setting up new organizations may be insufficient to promote social change since they can be co-opted by elites (Bardhan 2002; Gugerty and Kremer 2008), here, the chiefs.

• Giving marginalized groups formal authority (i.e. political reservations for women in India, Beamen et al. 2009) may be more effective than indirect interventions like CDD that hope to shift informal institutions, especially when existing authorities are strong (chiefs in Sierra Leone).

• As our results concern one program in one country, general implications are clearly speculative. More research is still needed to identify the precise interventions to reshape institutions to enhance collective action capacity while promoting accountability and inclusion.


END • Extra slides follow.



Democratic Institutions and Collective Action Capacity:Results from a Field Experiment in Post-Conflict Liberia

James Fearon, Macartan Humphreys, and Jeremy WeinsteinStanford and Columbia Universities

September 20, 2011

Summary: The question.

I DFID, USAID, World Bank, and other major donors spendmany billions each year on “Community DrivenDevelopment” and “Community Driven Reconstruction”projects.

I A common, core goal is to improve local governance byintroducing elected development councils to select andmanage projects.

I In various degrees, CDD/CDR projects also hope to improvematerial welfare.

I Does it work?

Summary: What we did.

I We worked with International Rescue Committee to randomlyassign a CDR project (funded by DFID) to 42 of 83communities in northern Liberia in Sept 2006.

I After program, measured impact on

1. material welfare and opinions about governance, etc, withsurveys (March/April 2008),

2. a community-wide public goods game to assess impact onactual behavior.

Summary: Main findings (1).

I CDR had no apparent impact on material welfare.

I But we find

1. Consistent evidence of CDR impact on attitudes expressed infollow-up survey. eg, support for dem process, inclusion ofmarginalized, levels of participation, trust in leaders.

2. CDR communities contributed significantly more to raise fundsfor a small development project they selected (our publicgoods game). ⇒ CDR improved collective action capacity.

I Opposite results to Casey et al. Why? (speculative)

I No econ impact: IRC project forbid income-generatingprojects, was almost all small scale construction of buildings.GoBifo was mainly economic projects.

I coll action impact: We gave comm’s a decision problemperhaps closer to problems encountered in CDR program.

Summary: Main findings (2).

Mechanisms. Why/how did this CDR program increase collectiveaction capacity? (more speculative)

I No evidence that CDR made people more public spirited,caused more popular projects to be selected, improved commsanctioning capacity, increased trust in NGOs.

I Evidence that CDR improved community leadersability/inclination to mobilize community.

I (maybe) CDR program ⇒ greater leadership experience withsimilar organizational problems ⇒ learning by doing

I If so, then CDR worked via ↑ “leadership capital” more thanby changing institutions, authority, and d-making practices.

I good, but less durable than institutional change?

Outline

1. A CDR program in Lofa County, Liberia.

2. Measurement strategies, including a public goods game.

3. Results on community contributions in game.

4. Mechanisms: Why did CDR have a positive impact?

5. Conclusions, policy implications.

The CDD/CDR program model

I CDR and CDD: Set up community-level institutions todemocratically select and oversee development projects.

I Process (in our case):

I Initial sensitization of communities, formation of “advisory boards.”I Election of Community Development Councils (CDCs).I Participatory process to select quick impact ($1-2k) and larger

development project (∼ $13k).I CDCs oversee tendering of projects, implementation, and

maintenance.

I Goals: Several, but mainly to improve community capacityfor collection action in supplying or demanding publicgoods. Also, post-conflict reconciliation.

CDR implementation in Lofa County

I Program implemented by IRC in two districts of Lofa County,Liberia (Voinjama and Zorzor). Very hard hit by wars.

I Over 400 villages clustered into 83 similarly sized‘communities’ for the purposes of CDR programadministration.

I After baseline data collected, equally deserving/reachablevillages randomly assigned to 42 treatment and 41 controlgroups in a public lottery attended by local chiefs and elders.

I CDR program runs October 2006 to March 2008.

I Community projects mainly emphasized small scaleconstruction (latrines, renovation of town halls, schools,clinics, guesthouses).

Measurement strategy

I Panel survey of about 1600 households. Data collected beforeprogram (March/April 2006) and after (March/April 2008).

I Representative sample of Voinjama and Zorzor districts.I Survey covered material welfare, political attitudes, collective

action, political efficacy, etc.

I Behavioral “real life” public goods game in 83 treatment andcontrol communities (July-September 2008).

I Goal to observe collective action capacity and politicalpractices six months after end of CDR program.

I Conducted in “hub” towns (largest village among the set ofvillages assigned to each community – avg about 140households, or 800-1000 people).

Game protocol

I Community meeting at which community members toldcommunity could receive up to $420 to spend on adevelopment project. Money received depends on:

I Community must complete form indicating how the moneywould be spent and which three people would handle the funds(“comm reps”).

I How much money a random sample of 24 people contributedto the project in a community-wide public goods game.

I One week later, team returns to village, collects form, samples24 households, plays the public goods game, publicly countsthe contributions, announces total, and provides the money tothe three community reps.

Game protocol in more detail

I 83 communities participated in the game.

I In half, only women were selected to play the game; in theother half, household participants were equally dividedbetween men and women (randomly assigned gendertreatment).

I 24 randomly selected indivs played the game in eachcommunity, choosing how much of 300LD ($4.75) tocontribute to the public good. Indiv decision made in private.

I 12 indivs had contributions multiplied by 2, other 12 by 5(randomly assigned interest rate treatment).

I Surveys conducted with each indiv after s/he played.

I Additional surveys were completed with each of the threecommunity representatives and the town chief.

Distribution of treatments

CDR intervention total comm’sgender composition control treatment (participants)

mixed groups (12m/12w) 20 22 42 (1008)

women only (24w) 21 20 41 (984)

total comm’s 41 (982) 42 (1008) 83 (1992)(participants)

Notes: In all communities, 12 players were randomly assigned to havea high interest rate and 12 to low (equal m/w in mixed).

Advantages of behavioral game as msm’t strategy

I Measure CDR effect on behavior in a cooperation problemrather than attitudes expressed in a survey. CDR could changesurvey responses w/o changing capacity or inclination to act.

I Can observe means by which communities select reps anddecide on community project (“democratic process”).

I Was entirely disconnected from CDR program, implementedby an established Liberian NGO.

I Implementers generally did not know which communities weretreatment and control.

I Not a lab experiment on indiv’s, but a real-world problem ofcollective action to raise money for community benefit.

Getting there

Getting there

Community meeting

Game day meeting

Game day meeting

Results: Contribution levels

I 82 communities successfully completed the exercise (turned ina form, selected project, chose reps, had 24 households playgame).

I Contribution rates very high overall in the public goods gameI Max communities could earn was 25,200LD:

I Average payout = 20,020 (80%), median = 20,850 (83%).I Min was 10,900; max was 25,000.

indiv contrib n %

300 1327 67.4200 201 10.2100 231 11.7

0 209 10.6

payout/25200

Fre

quen

cy

0.4 0.5 0.6 0.7 0.8 0.9 1.0

05

1015

Results: Main treatment effects

CDR program Gender comp. (allW) Interest Rate% max avg contrib % max avg contrib (5× vs 2×)

Control 75.8 226 74.3 223 226

ATE 6.3 17 9.8 27 18

p-value .02 .03 .001 .003 0

Notes:

I CDR and gender comp treatment effects are estimated at thecommunity level, interest rate at indiv level.

I p values are two-tailed calculated by randomization inference forCDR and gender and with clustered se’s for interest rate.

I Avg contributions are in Liberian Dollars.

Heterogeneous treatment effect

I Large CDR effect in mixed gender groups, nothing in all Wcomm’s. Robust to conditioning on Quarter.

% of max payoutmixed all W

no CDR 67.5 85.7

CDR 82.2 82.8

Community averages.

I Not explained by different behavior of men and women:

mixed all W

women 221 248(504) (960)

men 226(504)

Avg contributions (n).

Possible mechanisms: Why did CDR work?

1. Direct effects on

1.1 ‘Econ’ value for public good: eg, better project selection, orless graft by leaders.

1.2 ‘Social value’ for public good: eg, weight indiv’s put oncommunity welfare vs cash.

1.3 Indiv costs/benefits of contributing per se: eg, income effect,more sanctioning, desire to please NGO, or democraticlegitimacy effect.

1.4 Coordination of expectations (if aspect of coordination game).

2. Or, indirect effect on these via mobilization activities bycommunity leaders. Maybe CDR increased experience withsolving similar coll action problems.

Mechanisms evidence: Direct effects.

Effects of CDR Treatment on Intermediate Variables

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

Women only

Mixed

All

τ̂τ̂|Quarter

τ̂τ̂|Quarter

τ̂τ̂|Quarter

Satisfaction with Projects Trust in leaders No Anonymity Social Desirability Dem Procedure Expectations Accuracy

Economic Value Contribution Value Coordination

Correlation between Intermediate Variables and Contributions (Non Experimental)

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

●

●

0 100−100 50−50

Women only

Mixed

All

bb|CDR

bb|CDR

bb|CDR

Satisfaction with Projects Trust in leaders No Anonymity Social Desirability Dem Procedure Expectations Accuracy

Economic Value Contribution Value Coordination

Mechanisms evidence: Mobilization.

Effects of CDR Treatment on Mobilization Variables

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

●

●

0 0.2−0.2 0.4−0.4

Women only

Mixed

All

τ̂τ̂|Quarter

τ̂τ̂|Quarter

τ̂τ̂|Quarter

Meetings Contact Contact (Elite reports) Knowledge Leader experience Past CDC member

Correlation between Mobilization Variables and Contributions (Non Experimental)

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

●

●

0 50−50 25−25

Women only

Mixed

All

b

b|CDR

b

b|CDR

b

b|CDR

Meetings Contact Contact (Elite reports) Knowledge Leader experience Past CDC member

Conclusions: Material welfare

I Little or no evidence that CDR program in Liberia increasedmaterial welfare. Casey, Glennerster, Miguel find materialwelfare impact of GoBifo CDR program in Sierra Leone, butno effect on collective action capacity, d-mking processes.

I Why the difference? Possibly bec CDR in Liberia disallowedprojects for income generation, whereas about 57% of GoBifowere this sort.

Conclusions: Collective action

I Clear evidence that in mixed comm’s, CDR caused greatermobilization efforts which plausibly explain better performancein game.

I Suggests that CDR worked by mechanism of learning bydoing: increasing human resources/experience via with relatedproblems.

I Mobilization also occurs in all W, but not affected by CDRtreatment. Why?

I H: CDR program was deliberately mixed gender. Maybecomm’s used different networks of people to mobilize for game,using women’s structures less affected by CDR in all W.

I Mixed evidence for this: More “lady chiefs” are com rep’s in allW, but more (self reported) leaders of “women’s actiongroups” in mixed.

Conclusions: Policy implications.

Possible policy implications (1): From the study/paper.

I CDR approach can work to increase community coll. actioncapacity.

I However, if mechanism is “leadership capital” rather thandeeper institutional change, may not be large or stronglypersistent.

I If CDD/CDR aid projects focus on setting up and workingthrough gender-inclusive CDCs, but culture has tradition ofgender-specific organization for coll action, may only help collaction in relations with outsiders.

Conclusions: Policy implications

Possible policy implications (2): Some personal impressions.

I Aid programs are one-size-fits-all templates. Some keyassumptions of this CDR program wrong about this specificcontext.

1. Post-conflict reconciliation bet. Mandingo and Loma sorelyneeded, yes, but mainly inter- not intra-community/village.

2. Despite terrible war and displacement, social capital really highin these communities, and baseline attitudes about localdemocracy apparently strongly supportive.

I Big-donor RFP cycle and thematic lending ⇒ little incentivesfor NGOs to invest in location-specific knowledge, tailoring ofprograms to local context.

Community-Driven Development in Sierra Leone and Liberia

Discussion at IGC Growth Week September 20, 2011

Eric Werker

Harvard Business School

Random

• Unintended and adverse consequences

• Committing to hypotheses and publishing all results an amazing step

The dilemma of decentralization, liberally

• The DRC is decentralized • But, in a post-conflict fragile

state, the central government is by definition weak – projects little power or

services into outlying areas • Are “youths” (18-35)

disempowered? – Brutal civil war as social

revolution

“Underdevelopment has become dangerous. This reinterpretation has become associated with a radicalization of development. Indeed, the incorporation of conflict resolution and societal reconstruction within aid policy – amounting to a commitment to transform societies as a whole – embodies this radicalization. Such a project, however, is beyond the capabilities or legitimacy of individual Northern governments.”

(Mark Duffield, Global Governance and the New Wars)

In Sierra Leone, the local institutions were part of a broader decentralization occurring under GoSL initiative, and created one extra level of administration.

Weak local institutions? • Local institutions seemed quite strong on their own; examine

controls • Sierra Leone

– 98% of communities held a meeting to decide what to do with the vouchers, tarp

– 1 in 9 people speak to decide what to do with the gift, one third of whom women

– 86% of villages used the tarp for public purposes • Liberia

– Over 50% of respondents involved in at least three community organizations

– Large majority has attended a community meeting in the last 6 months, and has spoke

– 50% more community-focused in public goods games than typical lab experiments

Radically Ambitious • IRC : “This programme aims to empower local communities to be

drivers and owners of their own recovery through community governance structures that stress choice and accountability…”

• CGM : “created new structures designed to facilitate local development” (GoBifo) “…to increase social capital in grassroots communities… in order to promote more inclusive and effective development.”

On the cheap? SL: $10/p/y for 3.5 years in capacity building, $16/p/y in grants Lib: $8/p/y for 2 years in grants, 1-3 projects

Do we get behavioral change from CDD?

• SL: Money talks… • No, but we get institutional shells and stuff and income. • Fantastic

– What’s the ERR of the investment? • LIB: Talking the talk, walking the walk (sometimes) • A marginal improvement when mixed gender games,

plus survey response changes, but no stuff or income.

“We like the GoBifo approach because it gives us confidence and helps us improve on other activities that are appropriate for human development... everybody in GoBifo communities is always involved in planning and development. We manage and take ownership of the projects upon completion.” (chairman of a VDC)

Measuring up the tests • On the whole, extremely innovative and exciting, so the

following will be unhelpful • Tested on pooling equilibria?

– Lib: average 79% of payout goes to community – SL:

• SCA3: 86% of villages used tarp in public way • SCA1,3: 98% of all communities discussed what to do

– So what does treatment on the treated imply? • What is implied by better institutions?

– SL SCA2: longer deliberation on whether to choose batteries or salt? More youth in attendance? Voting?

Measuring up the tests • On the whole, extremely innovative and exciting, so the

following will be unhelpful • Tested on pooling equilibria?

– Lib: average 79% of payout goes to community – SL:

• SCA3: 86% of villages used tarp in public way • SCA1,3: 98% of all communities discussed what to do

– So what does treatment on the treated imply? • What is implied by better institutions?

– SL SCA2: longer deliberation on whether to choose batteries or salt? More youth in attendance? Voting?

• Measuring social capital? – SL SCA1: not clear this requires more than a tiny coalition of

elite

So what happened? • IRC Liberia

– Community-based projects now work with existing local governance structures

– M&E is straightforward baseline and indicator monitoring surveys




• MPEA – Little information on work done 2006-08

because process of NGO reporting was not yet well established




• MPEA – Little information on work done 2006-08

because process of NGO reporting was not yet well established

• Multiple community development funds are being established – Inconsistent organizing principles, rocky

start

Mismatch • In post-conflict situation, “building” institutions early and

“working with” them later creates a mismatch – Govt not at the point, while its institutions are to be worked

with, to have learned lessons immediately prior – Sierra Leone paper examining institutions with intent and

commitment to permanently revive • This limits the potential impact, if the institutionalizing

learning channel is cut off, to: – Temporary material relief, or social stability – Permanent material improvement, or shift in habits

• Formal institutions could allow communities more practice with the processes, to actually build institutions, and enjoy the cultural shifts that might come with more practice.