Evaluation: Experiments, Matching and D-in-D

Evaluation: Experiments, Matching and D-in-D

Lecture 5

Agenda

• Work our way through a bit of Kremer and duFlo …– Experiments– Matching– DD

• Migration, Mothers and Money …

Migration, Mentoring and Mothers: The Effect of Migration on Children’s

Educational Performance in Rural China

Scott Rozelle and Xinxin ChenStanford University

QQ Huang (U. of Minn.) and Linxiu Zhang (CCAP)

July 2007

Introduction/Motivation

Migration is one of the main ways of alleviating poverty in developing countries

Migration itself, however, is not costless.

• For example: There may be an adverse effect of migration on the educational achievement of the children of migrants (McKenzie et al. on Mexico)

Overall Increase in Off-farm Work

0%

20%

40%

60%

80%

100%

1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000

Year

off-farm busy season part time farm only

In 2000: 45% of rural labor force have jobs off the farm … more than 80% of households have at least 1 person working off the farm

In 1980: only 4% worked full time off the farm

51%

2005

Migration-fastest growing segment

0%

4%

8%

12%

16%

20%

Year

Per

cent

of T

otal

Wor

kfor

ce

Migrant Wage Earners

Self employed

TVE/Local wage earning local

migrant

Summary: Migration in China

• Migration is rising fast, surpassing 100 million individuals (deBrauw et al., 2002)

• Migrants also are moving further away from home and leaving for a longer period of time (Rozelle et al., 1999).

• Most of China’s migration is by individuals instead of entire households, in most cases the school-aged children of the migrant parents are being left.

• Work in China: Migration higher income / poverty alleviation

Introduction/Motivation

Migration is one of the main ways of alleviating poverty in developing countries

Migration itself, however, is not costless.

• For example: There may be an adverse effect of migration on the educational achievement of the children of migrants (McKenzie et al. on Mexico)

Results from current literature

• School performance of the migrant children is being adversely affected by migration since parental care falls with migration (Wang and Wu, 2003; Tan and Wang, 2004; Li, 2004; Zhou and Wu, 2004).

• These results are all based on casual observation

• Are they true?

• Is there anything about migration that can offset this effects?

The Trade-off:

Parents can stay at home and can take care of their kids …

But often good jobs are hard to find and people are very poor … sometimes they can

not supply their kids with warm clothes and enough nutrition

Although migrants to China’s cities typically earn more money than those left in the

village (Giles and deBrauw / Park and Du)

Parents are often far from their kids …

… and, their children are left behind … sometimes with their grandparents … and sometimes at boarding schools …

Objectives Examine the effect of migration activities of men

and women on the educational performance of their children.

– Compare the distribution of children’s scores for different types of rural households and describe how the grades vary over time.

– Examine whether migration negatively affects the school grades of rural children.

– Explore how migration will affect children’s educational performance in different types of households in terms of wealth or demographic composition.

Limitations / Contribution

• Only one region of China • Counterfactual limited … do not observe

children that migrate with their parents …• May be mainly looking at correlations

• Still say something about what happens to grades of children of migrants when they leave them in the countryside

Data• A data set collected in 2006, with information of

changes in school performance of children before and after their parents outmigrated.

• 1649 fifth grade students in 36 primary schools in 6 counties in Shaanxi province

• Random sample of towns and schools within the counties and classes within the schools … but surveyed ALL students within each class …

The Sample (6 counties)

Two Key Variables

• Grades of school achievement – math and Chinese language scores – scores from 2001/2 (first grade) to 2005/6 (fifth grade)

• From records kept by students + schools – standardized scores (second term scores)

[all scores from standardized tests corrected by joint grading panel of teachers]

• Migration status– migration histories of each parent between 2002 and 2006

Time lines of academic calendars from 2001/2 to 2005/6

200620042002 2003 20052001

Grade 1 second term scores


Never Migrant Households

200620042002 2003 20052001



New Migrant Households

New migrant = parents were both home in 2002; at least one or both parents outmigrated by 2005

Different types of new migrant households by migration status

Any Parent Migrated households: households in which both parents lived at home in 2002 and at least one parent –either the father; mother or both parents—outmigrated in 2006;

Father Migrated Only (or mother-stayed-at-home) households: households in which only the father outmigrated in 2006 but was at home in 2002;

Father Migrated (Unconditional ) households: households where the father was at home in 2002 but outmigrated in 2006 (including households in which the mother was either at home or not at home in 2006);

Mother Migrated Only (or father-stayed-at-home) households: households where only the mother outmigrated in 2006 but was at home in 2002; Mother Migrated (Unconditional) households where the mother was at home in 2002 but outmigrated in 2006 (including households in which the father was either at home or not at home in 2006);

Both Parents Migrated households: households where both parents were at home in 2002, but outmigrated in 2006.

Never Migrant

households.

New Migranthouseholds

Migration status in 2006

Migration Status in 2002

(1)

Number of

Migrants/

Non-migrants

in 2002 a

(2)

Father Migrated Only

(mother stayed home)

(3)

Mother Migrated Only

(father stayed home)

(4)

Both Parents

Migrated

(5)

Return migrants

(rows 1-3) / Never

Migrant (row 4)

(1)

Father Migrated Only

(mother stayed home)

149

94 d

55 c

(2)

Mother Migrated

Only (father stayed

home)

18

9 d

9 c

(3)

Both Parents

Migrated

69

7

5

40 d

17 c

(4)

New Migrants (col. 2,

3 and 4) / Never

Migrant (col. 5)

1358

131 b

35 b

54 b

1138

(5)

Total number of

households

1594

232

49

94

1219

131 35 54 1138

Patterns of Migration for Sample Households in China, 2002 and 2006

Migration status, 2002

Migration status, 2006

Never migrants, 2002

Never migrants, 2006

1594

Total sample size

Figure 1 Average Yearend Test Scores in China, 2002 and 2006

73.4074.54 73.97

70.1972.38

71.29

50.00

55.00

60.00

65.00

70.00

75.00

80.00

Chinese Math Average

2002

2006

School Performance

The average grades of all children fell between 2002 and 2006 … one interpretation is that grades fell as more migration … alternative

explanation: fifth grade is graded “harder” than first grade?

Figure 2. Differences in Yearend Test Scores between First Grade Students from Migrant and Non-migrant Households in Rural China, 2006

70.87

69.95

72.01

70.57

68.50

69.00

69.50

70.00

70.50

71.00

71.50

72.00

72.50

Never Migrant Both ParentsMigrated

Father Migrated(Unconditional)

Mother Migrated(Unconditional)

The effect of migration on school performance seems to be complicated and that care needs to be exercised

in any interpretation. If one went out and found a family in 2006 in which neither parent outmigrated in 2002 and both parent outmigrated in 2006 … and compared to Never Migrant … might be inclined to blame migration!

Figure 2. Differences in Yearend Test Scores between First Grade Students from Migrant and Non-migrant Households in Rural China, 2006

70.87

69.95

72.01

70.57

68.50

69.00

69.50

70.00

70.50

71.00

71.50

72.00

72.50

Never Migrant Both ParentsMigrated

Father Migrated(Unconditional)

Mother Migrated(Unconditional)

The effect of migration on school performance seems to be complicated and that care needs to be exercised

in any interpretation.

Figure 3. Kernel Density Plots of Distributions of Average Test Scores in Never Migrated Households and Both Parents Migrated Households,

2002 and 2006

0.0

1.0

2.0

3.0

4D

ensi

ty

0 20 40 60 80 1002002 Average Test Scores(both Chinese and Math)

Both Parents Migrated

Never Migrant0

.01

.02

.03

.04

Den

sity


Both Parents Migrated

Never Migrant

the gap was narrowing

Figure 4. Kernel Density Plots of Distributions of Average Test Scores in Households that Vary by Wealth Category and Household Composition in China, 2002 and 2006

0.0

1.0

2.0

3.0

4D

en

sity


Poorer Households

Wealthier Households

0.0

1.0

2.0

3.0

4D

en

sity


Poorer Households

Wealthier Households

Panel A. Average Test Scores by Wealth Category

0.0

1.0

2.0

3.0

4D

en

sity


Households with no Siblings

Households with Siblings

0.0

1.0

2.0

3.0

4D

en

sity


Households with no Siblings

Households with Siblings

Panel B. Average Test Scores by Household Composition

Methodology (1)

Difference in Difference (DID)

Model (1), Restricted & Unadjusted: ΔScorei = α + δMIGi + εi

Model (2), Unrestricted & Unadjusted: ΔScorei = α +δMIGi +γScore_02i + εi

Model (3), Restricted & Adjusted: ΔScorei = α +δMIGi +βXi + εi,

Model (4), Unrestricted & Adjusted: ΔScorei = α +δMIGi +γScore_02i +βXi + εi

where, i is an index for the student, ΔScorei is the change of the second term score of

student i between 2002 and 2006 (that is the final grade from the fifth grade minus the final grade from the first grade); MIGi is the treatment variable (which makes δ the parameter of

interest). Finally, the term Xi is a vector of covariates that are included to capture the

characteristics of students, parents and households and also includes a set of 12 town indicator or dummy variables.

Model #2: Difference-in-differences

• Compares before-after changes of participants vs. before-after change of non-participants

• Any common trends get differenced out.

• Limitation: only common trends between two groups get differenced out– We control for base value of Yi and observables

i T i D i i i D iDD iY T D T D X

Our equation of choice (the full model)

• Model (4), • Unrestricted & Adjusted:

ΔScorei = α +δMIGi +γScore_02i +βXi + εi

• Propensity Score Matching (PSM)----Basic matching

----Multi-dimensional matching

• Difference in Difference Matching (DDM)----Basic matching

----Multi-dimensional matching

Methodology (2)

Results

• DD results• PSM results• DDM results

Table 2. DD Regression Results Analyzing the Effect of Migration on School Performance of Students in China

Dependent Variable = Changes in Second Term Test Scores between 2002 and 2006 (ΔScore)

(1) (2) (3) (4)

Treatment Variable (MIGi)b Restricted &

Unadjusted

Unrestricted &

Unadjusted

Restricted &

Adjustedc

Unrestricted &

Adjustedc

(1) Any_Parent_Migrated 3.183 2.327 2.169 1.164

(3.72)*** (3.03)*** (2.58)** (1.65)*

Characteristics of the students in 2002

-0.460 -0.627 (2)

Student score in the second term in

2002 (Full score is 100) (14.93)*** (18.04)***

0.826 -0.383 (3)

Gender dummy (=1 if male and 0 if

female) (1.28) (0.75)

Age of the student in 2002 (Years) 0.097 -1.322 (4)

(0.26) (4.39)***

-2.754 1.168 (5)

Cadre dummy (=1 if the student was

a student cadre in 2002 and 0 if not) (3.83)*** (1.93)*

-1.051 -0.972 (6)

Mentor dummy (=1 if the student

had a mentor in 2002) (0.99) (1.26)

0.438 0.443 (7)

Sibling dummy (=1if the student had

no siblings in 2002) (0.55) (0.71)

Characteristics of the parents in 2002

-0.066 -0.053 (8) Age of the father (Years)

(0.85) (0.85)

-0.200 -0.044 (9)

Level of education of the father

(Years of schooling) (1.06) (0.35)

0.114 0.274 (10)

Level of education of the mother

(Years of schooling) (0.77) (2.39)**

Characteristics of the household in 2002

0.031 0.037 (11)

Size of total household land holding

in 2002 (mu) (0.36) (0.57)

0.078 0.251 (12)

Number of household members in

2002 (Person) (0.25) (1.01)

0.056 -0.037

(13)

House value dummy (=1 if the

house is worth more than 5000

yuan) (0.08) (0.07)

(14) Number of Observations 1575 1575 1549 1549

(15) R-squared 0.01 0.27 0.10 0.43


Treatment Variable (MIGi)

(1) (2) (3) (4)

Restricted & Unadjusted

Unrestricted &

Unadjusted

Restricted & Adjusted

Unrestricted & Adjusted

Any_Parent_Migrated 3.183 2.327 2.169 1.164

(3.72)*** (3.03)*** (2.58)** (1.65)*




(1) (2) (3) (4)

Treatment Variable (MIGi)b Restricted &

Unadjusted

Unrestricted &

Unadjusted

Restricted &

Adjustedc

Unrestricted &

Adjustedc

(1) Any_Parent_Migrated 3.183 2.327 2.169 1.164

(3.72)*** (3.03)*** (2.58)** (1.65)*

Characteristics of the students in 2002

-0.460 -0.627 (2)

Student score in the second term in

2002 (Full score is 100) (14.93)*** (18.04)***

0.826 -0.383 (3)

Gender dummy (=1 if male and 0 if

female) (1.28) (0.75)

Age of the student in 2002 (Years) 0.097 -1.322 (4)

(0.26) (4.39)***

-2.754 1.168 (5)

Cadre dummy (=1 if the student was

a student cadre in 2002 and 0 if not) (3.83)*** (1.93)*

-1.051 -0.972 (6)

Mentor dummy (=1 if the student

had a mentor in 2002) (0.99) (1.26)

0.438 0.443 (7)

Sibling dummy (=1if the student had

no siblings in 2002) (0.55) (0.71)

Characteristics of the parents in 2002

-0.066 -0.053 (8) Age of the father (Years)

(0.85) (0.85)

-0.200 -0.044 (9)

Level of education of the father

(Years of schooling) (1.06) (0.35)

0.114 0.274 (10)

Level of education of the mother

(Years of schooling) (0.77) (2.39)**

Characteristics of the household in 2002

0.031 0.037 (11)

Size of total household land holding

in 2002 (mu) (0.36) (0.57)

0.078 0.251 (12)

Number of household members in

2002 (Person) (0.25) (1.01)

0.056 -0.037

(13)

House value dummy (=1 if the

house is worth more than 5000

yuan) (0.08) (0.07)

(14) Number of Observations 1575 1575 1549 1549

(15) R-squared 0.01 0.27 0.10 0.43

Highest R-square:

> 0.40

Table 3. DD Regression Results Analyzing the Effect of Migration on School

Performance of Students in China by Household’s Migration Status

Dependent Variable = Changes in Second Term Test Scores

between 2002 and 2006 (ΔScore)

(1) (2) (3) (4)


b Restricted &

Unadjusted

Unrestricted

& Unadjusted

Restricted &

Adjustedc

Unrestricted

& Adjustedc

3.183 2.327 2.169 1.164 (1)

Any_Parent_migrated,

(3.72)*** (3.03)*** (2.58)** (1.65)*

4.634 3.812 3.630 2.356 (2)

Father_Migrated_Only

(mother stayed home) (4.27)*** (4.09)*** (3.45)*** (2.73)***

3.812 2.879 2.984 1.508 (3)

Father_Migrated

(Unconditional) (4.10)*** (3.52)*** (3.24)*** (1.98)**

0.839 0.156 -0.861 -0.121 (4)

Mother_Migrated_Only

(father stayed home) (0.45) (0.08) (0.45) (0.07)

0.903 0.444 -0.147 -0.541 (5)

Mother_Migrated,

( Unconditional) (0.73) (0.37) (0.12) (0.48)

1.367 0.615 1.040 -0.536 (6) Both_parents_migrated

(0.79) (0.38) (0.58) (0.35)

Table 4. PSM and DDM Estimators and the Effect of Migration on the School Performance of Students in Rural China, 2002 and 2006

Propensity Score Matching

Difference-in-Difference

Matching Treatment Variable c d

Average Treatment

Effect for the Treated

t-value/

z-value b

Average Treatment


t-value/

z-value b

(1) (2)

(1a) Basic Matching 1.16 (1.02) 0.31 (0.28) Any_parent_migrated

(1b) Multi-dimensional Matching 1.57 (1.60) 2.12 (1.86 )*

(2a) Basic Matching 2.04 (1.36) 1.12 (0.77) Father_Migrated_Only

(mother stayed home) (2b) Multi-dimensional Matching 3.59 (2.96 ) *** 3.12 (1.93 )**

(3a) Basic Matching 1.57 (1.20) 2.35 (1.93)** Father_migrated,

(Unconditional) (3b) Multi-dimensional Matching 2.19 (2.04 ) *** 2.52 (1.99 )***

(4a) Basic Matching -0.63 (-0.22) -1.1 (-0.39) Mother_Migrated_Only

(father stayed home) (4b) Multi-dimensional Matching -0.94 (-0.43) 1.93 (0.58)

(5a) Basic Matching -0.45 (-0.26) -1.51 (-0.88) Mother_migrated

(Unconditional) (5b) Multi-dimensional Matching -0.46 (-0.32) 0.82 (0.48)

(6a) Basic Matching -0.22 (-0.09) -0.56 (-0.23) Both_parents_migrated

(6b) Multi-dimensional Matching -0.28 (-0.13) 0.97 (0.43)



Difference-in-Difference

Matching Treatment Variable c d

Average Treatment


t-value/

z-value b

Average Treatment


t-value/

z-value b

(1) (2)

(1a) Basic Matching 1.16 (1.02) 0.31 (0.28) Any_parent_migrated

(1b) Multi-dimensional Matching 1.57 (1.60) 2.12 (1.86 )*

(2a) Basic Matching 2.04 (1.36) 1.12 (0.77) Father_Migrated_Only

(mother stayed home) (2b) Multi-dimensional Matching 3.59 (2.96 ) *** 3.12 (1.93 )**

(3a) Basic Matching 1.57 (1.20) 2.35 (1.93)** Father_migrated,

(Unconditional) (3b) Multi-dimensional Matching 2.19 (2.04 ) *** 2.52 (1.99 )***

(4a) Basic Matching -0.63 (-0.22) -1.1 (-0.39) Mother_Migrated_Only

(father stayed home) (4b) Multi-dimensional Matching -0.94 (-0.43) 1.93 (0.58)

(5a) Basic Matching -0.45 (-0.26) -1.51 (-0.88) Mother_migrated

(Unconditional) (5b) Multi-dimensional Matching -0.46 (-0.32) 0.82 (0.48)

(6a) Basic Matching -0.22 (-0.09) -0.56 (-0.23) Both_parents_migrated

(6b) Multi-dimensional Matching -0.28 (-0.13) 0.97 (0.43)




Difference-in-Difference Matching

ATT t-value/ z-value

ATT t-value/ z-value

Any_parent_migrated (1a) Basic Matching

1.16 (1.02) 0.31 (0.28)

(1b) Multi-dimensional Matching　

1.57 (1.60) 2.12 (1.86 )*

Summary

• There is no evidence that migration in our sample of households has hurt school performance.

• In fact, when the father outmigrates (either by himself or with others), migration appears to have a small, positive effect on the school performance of migrant children.

Heterogeneous effects

• Heterogeneous Effects from Wealth

Model (5): ΔScorei = α +δ1MIGi +δ2*MIGi *poor+γScore_02i +βXi + εi,

• Heterogeneous Effects from Household Composition

Model (6): ΔScorei = α +δ1MIGi +δ2*MIGi *nosibling+γScore_02i +βXi + εi,

Table 5. DD Regression Results with Heterogeneous Effects from Wealth and Household Composition


Panel A

Heterogeneity Effects from Wealtha

Panel B

Heterogeneity Effects from Household compositionb

Treatment Variable(MIGi) a Treatment Variable(MIGi) b

2.397

1.118

Any_Parent_Migrated (2.82)***

Any_Parent_Migrated (1.28)

-2.271 0.195 Any_Parent_Migrated * Poor

(1.79)* Any_Parent_Migrated * Nosibling

(0.15)

2.958

2.028


(mother stayed home) (2.83)***


(mother stayed home) (1.87)*

-1.170 0.965 Father_Migrated_Only*Poor

(0.72) Father_Migrated_Only*Nosibling

(0.57)

2.668

1.516

Father_Migrated

(Unconditional) (2.85)***

Father_Migrated

(Unconditional) (1.60)

-2.139 0.028 Father_Migrated * Poor

(1.54) Father_Migrated * Nosibling

(0.02)

2.285

-0.828


(Father stayed home) (1.59)


(Father stayed home) (0.40)

-3.783 1.680 Mother_Migrated_Only* Poor

(1.28) Mother_Migrated_Only* Nosibling

(0.48)

1.349

-0.403

Mother_Migrated


Mother_Migrated


-3.369 -0.174 Mother_Migrated * Poor (1.62)

Mother_Migrated * Nosibling (0.08)

1.720

0.155

Both_Parents_Migrated (0.87)

Both_Parents_Migrated (0.08)

-3.982 -1.457 Both_Parents_Migrated * Poor

(1.38) Both_Parents_Migrated* Nosibling

(0.48)

Conclusion• We can reject the hypothesis that migration harms the

grades of their children .

• In fact, the migration of some migrant households has a statistically significant and positive effect on the performance of the children.

• There is neither a systematically different effect of migration between the children of more wealthy and less wealthy households nor between the children from families that have one and more than one child.

Policy implications

• It is not that migrants do not need to have special attention in education … their grades are lower (but they are always lower) … increased education will raise their productivity (other studies) …

• Point of our paper: migration by itself does not cause this … although we have not identified that exact mechanism, may be that the income effect of migration is offsetting the parental care effect

• So should build better schools … have high quality boarding facilities … increase mentoring inside schools (e.g., by small classes) … and promote the admittance of rural students to urban schools (for low or no tuition) … but, don’t do it because believe migration leads to lower education achievement … there is no evidence from our sample

Documents

Evaluation: Experiments, Matching and D-in-D