Evidence on Conditional Cash Transfers in Education

Laura Poswell, J-PAL Africa

IGC WorkshopMaputo 6 July 2015

Successful partnerships for evidence-based policy-making: Lessons from J-PAL

2

Overview

Who is J-PAL Measuring impact: Why randomise Evidence from randomized evaluations

on conditional cash transfers for education Mexico (PROGRESA) Colombia Malawi

Lessons and nuances

3

Overview

Who is J-PAL Measuring impact: Why randomise Evidence from randomized evaluations

on conditional cash transfers for education Mexico (PROGRESA) Colombia Malawi

Lessons and nuances

4

J-PAL: Over 110 affiliated researchers and 600 projects in 64 countries

6

J-PAL’s Work in Education

165 completed and ongoing evaluations in 36 countries

7

Overview

Who is J-PAL Measuring impact: Why randomise Evidence from randomized evaluations

on conditional cash transfers for education Mexico (PROGRESA) Colombia Malawi

Lessons and nuances

8

How should we increase secondary school participation?

Address health issues (malnutrition, intestinal parasites?)

Lower the costs of schooling to families?

Inform parents of the returns to secondary schooling?

9

Intervention

Time

Att

en

dan

ce

Counterfactual

Impact

How to measure impact

Time

Att

en

dan

ce

Counterfactual

ImpactIntervention

How to measure impact

11

Counterfactual

Counterfactual: Would have happened in the absence of the policy?

Problem: Will never observe what would have happened

How to measure the causal impact of a policy?

1. B

ase

line

3. Intervention

3. Comparison

4. E

ndlin

e

2. Randomassignment

Randomised Evaluations in practice

Members of the groups are statistically identical

thus any change can be attributed to the program itself.

Key advantage of randomised evaluations

14

Partnerships for randomised evaluations

Implementing Partner (Government, NGOs, Business)

Research team (Academics, fieldwork team)

Question – Intervention + Evaluation – Results – Scale up

Start early together

15

Overview

Who is J-PAL Measuring impact: Why randomise Evidence from randomized evaluations

on conditional cash transfers for education Mexico (PROGRESA) Colombia Malawi

Lessons and nuances

16

Conditional Cash Transfers

Cash transfers to poor families conditional on children’s health and educational investments School attendance Regular medical check ups

Can address short term liquidity constraints longer term savings constraints

Aim Break the intergenerational cycle of poverty through

encouraging investments in children’s education & health

17

Evidence from randomised evaluations on CCTs in education

Conditional cash transfer (CCT) programs widely tested, consistently effective at increasing participation Over 30 countries have some form of CCT Program

Impacts on learning less clear

Differential impacts? More or less effective for the most marginal?

Other impacts beyond education?

Can careful design improve effectiveness?

18

PROGRESA : landmark program and evaluation

Context of Program: Beginning of Mexico’s Economic Crisis in 1994 Poor and marginalised populations lagged

behind in terms of education, health, and nutrition.

Group of cabinet officials with presidential support want to introduce conditional cash transfers for the poorest. Faced some objections Pilot and then a

rigorous evaluation

19

PROGRESA

PROGRESA program details Included education, nutrition, and health conditions. Two-stage targeting mechanism (geographic,

household) Ave. transfer amount = 22% of the monthly income of

beneficiary families.

Evaluation integrated into programme design Evaluation at scale planned from the outset Budget and administration constraints Randomised phase-in design (320 localities receive

program in first 2 years, remaining 186 in year 3)

20

PROGRESA evaluation outcomes

Education outcomes Increased enrolment rates especially for children transitioning to junior

secondary school (11.1 percentage points) And especially for girls (14.8 percentage points)

Lower dropout rates Reduced child labour

No significant impacts on learning outcomes

References: Shultz (2001), Behrman et al 2001)

21

PROGRESA-OPORTUNIDADES

Implications of Rigorous Evaluation Credibility

People believed the results – international recognition Made it politically robust – new government in 2000

continues and expands the program 2.6 million families by 2000

Precedence Set a new high standard for the design and

conduct of social policy - early involvement of government & researchers

Evaluation recommendations lead to new questions for further program improvements

CCTs introduced in more and more countries and many rigorous evaluations done

22

Columbia StudyTiming and size of payout

Structure of most CCT programs is surprisingly similar: Families receive money (often every 2 months) when their

children meet specified monthly attendance targets Usually +- 80%

In 2005 the Secretary of Education of the City of Bogota decided to pilot a new CCT programme to: Prevent drop out from secondary school Encourage matric graduation & tertiary enrollment

Pilot in 2 of 12 localities in the city for 1 year Results to inform the design for city-wide program

References: Barrera-Osorio et al. (2011)

23

Columbia study: 3 programs tested

Intervention Regular Transfers

Conditions Additional Transfers

1 Standard CCT 30,000 pesos (US$15)/month

80 % attendance /month

 

2 Savings CCT 20,000 pesos (US$10)/month

80 % attendance /month

100,000 pesos (US$50) at enrollment time of next school year

3 Graduation CCT

20,000 pesos (US$10)/month

80 % attendance /month

Graduated from secondary school

600,000 pesos (US$300) immediate payment with proof of enrolment in higher education;

otherwise, payment delayed by one year

24

Colombia study: Attendance

Despite reducing the amount for regular payments, the saving and graduation CCTs attendance >= standard CCT

25

Colombia study: Re-enrollment Savings & graduation CCTs on re-enrolment >

standard CCT

26

Colombia study: Tertiary enrollment

Savings & graduation CCTs tertiary enrollment but standard CCT does not (not statistically significant)

27

Columbia study: Other effects? Savings CCT

Especially effective at improving re-enrollment of the poorest students & those most at risk of dropping out

+- 12 percentage point

Eligibility rules & Unintended consequences: Families on average did not enrol all eligible

children And if they were registered, but not selected,

they attended school less than a sibling who was selected

28

Transfer size?

In Colombia – the savings CCT reduced the monthly transfer by 30% from the standard CCT. Did not result in lower monthly attendance than the standard CCT

CCT study in Malawi Randomly varied CCT amounts in Malawi:

$1/month to girls as effective as $2-5, similar results for transfers to parents

29

Conditional vs Unconditional? Malawi study – first study to test CCT vs

UCT Considers effects on education and

sexual behaviour of adolescent girls Zomba district, 2007 - 2009 Sample size – nearly 3,000 schoolgirls

aged 13 – 22 CCT - 80 % attendance /month UCT – no conditions

References: Baird et al (2011)

30

Malawi study – Education Results

Enrollment UCTs - 5% higher than control CCTs – 11% higher

Attendance UCTs – no significant impact CCTs - 8% points higher than control

Learning (English, Maths, Cognitive tests) UCTs – no significant difference CCTs – significant improvements in all

31

Malawi study – Other Results Marriage/ Pregnancy

Married after 2 years Control: 18% married UCT: 10% CCT: no significant difference

Pregnant during the program Control: 25% UCT: 18% CCT: no significant difference

UCT effect appears to be driven by cash to school drop outs as opposed to CCT where no cash if drop out

32

In rural Burkina Faso CCT vs UCT study for children 7 – 15 , Very low enrollment - +- 50% UCTS = CCTs in erollment for non-marginal children

(boys, older & higher ability children)

CCTS > enrollment for marginal children (girls, younger children, lower ability children)

“Labeled” cash transfers as effective as CCTs in Morocco

Mixed Results on Importance of Conditionality

References: Akresh et al (2013), Benhassine et al (2014)

33

Overview

Measuring impact: Why randomise Who is J-PAL Evidence from randomized evaluations

on conditional cash transfers for education Mexico (PROGRESA) Colombia Malawi

Lessons and nuances

34

Lots of lessons, many nuances Many are supportive Some conflict When thinking through your own design:

Try to understand the evidence And what might be most applicable to your

context Ask new questions important to you based

on work already done Work with partners from early on

35

36

J-PAL’s Post Primary Education Initiative

To promote policy-relevant research on secondary, tertiary, and vocational education in developing countries

Step 1: Review paper Highlighted gaps in the literature Identified policies that that should be given the

highest priority for future research

37

J-PAL’s Post Primary Education Initiative

Research questions Pedagogy The role of information communication

technology The design of teacher incentives Strategies to include and support

disadvantaged students The role of private schools Mechanisms to increase the demand for

schooling among students and their parents (CCTs, information interventions)