Upload
pedro-cango
View
233
Download
0
Embed Size (px)
Citation preview
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
1/44
www.worldbank.org/hdchiefeconomist
TheWorldBank
HumanDevelopment
Network
Spanishmpact
!valuation"und
www.worldbank.org/hdchiefeconomist
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
2/44
DIFFERENCE
S
This material constitutes supporting material for the #mpact !valuation in $ractice# book. This additional material is made freel% but please acknowledge its use as follows& 'ertler( $. ).* +artine,( S.( $remand( $.( -awlings( . B. and hristel +. ). 0ermeersch(1232( mpact !valuation in $ractice& 4ncillar% +aterial( The World Bank( Washington D 5www.worldbank.org/ieinpractice6. The content of this presentation re7ects the views of the authors and not necessaril% those of the World Bank.
Technical Track Session III
IN
&PANELDATA
DIFFERENCE
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
3/44
Structure of this session
When do we use Di8erences9in9Di8erences: (Dif-in-Dif or DD)
!stimation strateg%& ; wa%s to lookat DD
!
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
4/44
When do we use DD3
We can=t alwa%s randomi,e!.g. !stimating the impact of a >past? program
4s alwa%s( we need to identif%o which is the group a8ected b% the polic% change
(treatment), ando which is the group that is not a8ected (comparison)
We can tr% to @nd a >natural e
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
5/44
! wa"s to looks atDD
1
n a Bo< 'raphicall%
n a-egression
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
6/44
The box#rou$ a%ected "
the $olic" chan'e(treatment)
#rou$ that is not
a%ected " the$olic" chan'e(comparison)
4fter theprogram start
Y1 (ui) | Di=1 Y1 (ui) | Di=0
Before theprogram start
Y0 (ui) | Di=1 Y0 (ui) | Di=0
(Y1|D=1)-(Y0|
D=1)
(Y1|D=0)-(Y0|
D=0)
DD=(Y1|D=1)-(Y0|D=1)! - (Y1|D=0)-(Y0|
D=0)!
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
7/44
Graphically
Dutcome
0ariable
Y0 | Di=1
Y1| Di=0Y0 | Di=0
TE2 TE3 Time
!nrolled
Not
enrolled
!stimated
4T!
Y1 | Di=1
DD=(Y1|D=1)-(Y
0|D=1)! - (Y
1|D=0)-(Y
0|
=
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
8/44
Re'ression (for ) ti*e$eriods+
1
0
1
0
1 0 1 0
.1( 1) .1( 1) .1( 1).1( 1)
( | 1) ???
( | 1) ???
( | 0) ???
( | 0) ???
( ( | 1) ( | 1)) ( ( | 0) ( | 0)) ???
it i i it
i i
i i
i i
i i
i i i i i i i i
Y t D t D
E Y D
E Y D
E Y D
E Y D
DD E Y D E Y D E Y D E Y D
= + = + = + = = +
= =
= =
= =
= =
= = = = ==
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
9/44
Re'ression (for ) ti*e$eriods+
1 1
0 0
1 1
0
.1( 1) .1( 1) .1( 1).1( 1)
( | 1) .1 .1 .1.1 ( | 1)
( | 1) .0 .1 .0.1 ( | 1)
( | 0) .1 .0 .1.0 ( | 0)
( | 0) .0 .1
it i i it
i i i i
i i i i
i i i i
i i
Y t D t D
E Y D E D
E Y D E D
E Y D E D
E Y D
= + = + = + = = +
= = + + + + = = + + +
= = + + + + = = +
= = + + + + = = += = + + 0
1 0 1 0
.0.0 ( | 0)
( ( | 1) ( | 1)) ( ( | 0) ( | 0))
( )
i i
i i i i i i i i
E D
DD E Y D E Y D E Y D E Y D
+ + = =
= = = = =
= +
=
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
10/44
If we ha,e *ore than) ti*e $eriods-'rou$s.
1 1
where is the intensity of the treatment
in group in p
.1( ) .
eriod
)
.
1( .T I
it it
it
itY t
D D
i
i D
t
= =
= + = + = + +
We use a regression with @
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
11/44
Identi/cation in DD
The identi@cation of the treatment e8ectis based on the inter9temporal variationbetween the groups.
I"e" hanges in the outcome variable Y
over time( that are speci@c to thetreatment groups.
I"e")umps in trends in the outcome
variable( that happen onl% for thetreatment groups( not for the comparisongroups( e
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
12/44
Warnin'sDD/ @
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
13/44
Warnin's
DD attributes an% di8erences in trendsbetween the treatment and controlgroups( that occur at the same time asthe intervention( to that intervention.
f there are other factors that a8ect thedi8erence in trends between the two
groups( then the estimation will bebiasedG
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
14/44
Violation of Equal TrendAssumption
Dutcome
0ariable
TE2 TE3Time
!nrolled
Notenrolled
Y0 | Di=1
Y1 | Di=0Y0 | Di=0
!stimate
d mpact
Y1 | Di=1
Bias
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
15/44
Sensiti,it" anal"sis for di%0in0di%$erform a >placebo? DD( i.e. use a >fake?
treatment groupo !
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
16/44
Fre1uentl" occurrin' issuesin DD$articipation is based in di8erence in outcomes prior
to the intervention. !.g. >As&ene'ter dip selectioninto treatment in7uence b% transitor% shocks on pastoutcomes 54shenfelter( 3JKL* ha% et al.( 122 6.
f program impact is heterogeneous across individual
characteristics( pre9treatment di8erences inobserved characteristics can create non9paralleloutcome d%namics 54badie( 1226.Similarl%( bias would occur when the si,e of theresponse depends in a non9linear wa% on the si,e of
the intervention( and we compare a group with hightreatment intensit%( with a group with low treatmentintensit%When outcomes within the unit of time/group arecorrelated( S standard errors understate the st.
dev. of the DD estimator 5Bertrand et al.(
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
17/44
E2a*$le 3
Schoolin' and laor *arketconse1uences of school
constructionin Indonesia. e,idence fro* an
unusual $olic" e2$eri*ent
Esther Du4o5 6ITA*erican Econo*ic Re,iew5 Se$t )773
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
18/44
Research 1uestionsSchool
infrastructure
!ducational
achievement
!ducationalachievement:
Salar% level:
What is the economicreturn on schooling:
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
19/44
Pro'ra* descri$tion3JK;93JKL&The ndonesian government builtA3(222 schools e.ui/a'ent to one sc&oo' per00 c&i'dren et%een and 12 3ears o'd
The enrollment rate increased from AJ toLbetween 3JK; and 3JKL
The number of schools built in each region
depended on the number of children out ofschool in those regions in 3JK1( before thestart of the program.
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
20/44
Identi/cation of thetreat*ent e%ect
B% regionThere is variation in the number of schools receivedin each region.
There are 1 sources of variations in theintensit% of the program for a given individual&
B% ageo
hildren who were older than 31 %ears in 3JK1did not bene@t from the program.o The %ounger a child was 3JK1( the more it
bene@ted from the program Obecause she spentmore time in the new schools.
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
21/44
Sources of data
3JJ population census.ndividual9level data on&o birth dateo 3JJ salar% levelo
3JJ level of educationThe intensit% of the building programin the birth region of each person inthe sample.
Sample& men born between 3J2 and3JK1.
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
22/44
A /rst esti*ation of thei*$actSte$ 3. Let8s si*$lif" the $role*and esti*ate the i*$act of the$ro'ra*9
We simplif% the intensit% of the
program& highor low
o oungcohort of children whobene@tted
o ldercohort of children who did notbene@t
We simplif% the groups of childrena8ected b% the program
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
23/44
Let8s look at the a,era'e ofthe
outco*e ,ariale :"ears ofschoolin';Intensit" of the 79!? 793) DD(797@+
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
24/44
Intensit" of the
3193K5older cohort6
L.21 J.M 039!
793) DD(797@+
Let8s look at the a,era'e ofthe
outco*e ,ariale :"ears ofschoolin';
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
25/44
Placeo DD(Cf9 $9>@5 Tale !5 $anel
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
26/44
Ste$ ). Let8s esti*ate this with are'ression
.( . ) .( . )
education level of person i in
region j in cohort k
1 if the person was born in a region with a
high program intensity
1 if the pe
ijk j k j i j i ijk
ijk
j
i
S c P T C T
with
S
P
T
= + + + + +
=
=
= rson belongs to the young cohort
dummy variable for region j
cohort fi!ed"effectdistrict of birth fi!ed"effect
error term for person in region in cohort
j
k
j
ijk
C
i j k
=
==
=
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
27/44
Ste$ !. Let8s use additionalinfor*ation
#e will use the intensity of the program in each region$
where % the intensity of building activity in region j
% a vector of regional characteristi
.( . ) .( . ) = + + + + +
j
j
j j i j iijk k ijk
P
C
S c P T C T
&' &'
& &
cs
#e estimate the effect of the program for each cohort separately$
where a dummy variable for belonging to cohort i
.( . ) = ==
= + + + + + i
j j i j iijk k l l ijk l l
d
S c P d C T
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
28/44
Pro'ra* e%ect $ercohort
l
4ge in 3JKM
For y B De$endent ,ariale B
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
29/44
For y B De$endent ,ariale BSalar"
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
30/44
Conclusion-esults& "or each school built per 3222
students*o The average educational achievement
increase b% 2.319 2.3J %earso
The average salaries increased b% 1.A O.M
+aking sure the DD estimation isaccurate&o 4 placebo DD gave 2estimated e8ecto Ise various alternative speci@cationso heck that the impact estimates for each
age cohort make sense.
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
31/44
E2a*$le )
Water for Life.The I*$act of the Pri,atiationof
Water Ser,ices on Child6ortalit"
Seastin #aliani5 ni,ersidad de San AndrsPaul #ertler5 C
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
32/44
Chan'es in water ser,icesdeli,er"
3703T"$e of $ro,ision*ethods Nu*er of*unici$alities
4lwa%s public 3JA;J.K
4lwa%s anot9for9pro@tcooperative
3M; 1L.J
onvertedfrom public to private
3;L1K.J
4lwa%s private 3 2.1
No information 3A ;.1
Total ==377
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
33/44
Figure1: Percentage of Municipalities with Privatized
Water Systems
0
5
10
15
20
25
30
1990 1991 1992 1993 1994 1995 1996 1997 1998 1999
Year
%ccum
ulate
id f
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
34/44
se outside factors todeter*ine who $ri,aties
The political part% that governed themunicipalit%o "ederal( $eronist % $rovincial parties&
allowed privati,ationo -adical part%& did not allow privati,ation
Which part% was in power/whether the
water got privati,ed did not depend on&o ncome( unemplo%ment( ineCualit% at the
municipal levelo -ecent changes in infant mortalit% rates
i
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
35/44
it it it t i ity( d) * + ,
infant mortality rate in munic. in year
dummy variable that takes value 1 if
municipality has private water provider in year
vector of covariates
= + + + +
=
=
=
it
it
it
where
y i t
dI
i t
x
x
t
i
fi!ed time effect
fi!ed municipality effect
=
=
Re'ression
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
36/44
DD lt P i ti ti
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
37/44
DD results. Pri,atiationreduced infant *ortalit"
S iti it l i
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
38/44
Sensiti,it" anal"sis3
1
heck that the trends in infant mortalit% were
identical in the two t%pes of municipalitiesefore privati,ationo ou can do this b% running the same eCuation(
using onl% the %ears before the intervention O thetreatment e8ect should be ,ero for those %ears
o "ound that we cannot rePect the null h%pothesisof eCual trends between treatment and controls(in the %ears before privati,ation
heck that privati,ation onl% a8ects mortalit%through reasons that are logicall% related towater and sanitation issues."or e
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
39/44
I*$act of $ri,atiation on death fro*,arious causes
DD on co**on su$$ort
Pri ati ation has a lar'er e%ect
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
40/44
Pri,atiation has a lar'er e%ectin $oor and ,er" $oor
*unici$alities than in non0$oor*unici$alities6unici$alities
A,era'e*ortalit" $er
3775 1990
Esti*atedi*$act
chan'ein
*ortalit"Non9poor .3 2.32 F
$oor K.3 92.KAKQQQ 932.K
0er% poor J.MA 91.13MQQQ 91;.M
C l i
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
41/44
Conclusion
$rivati,ation of water services is associatedwith a reduction in infant mortalit% of 9K.
Ising a combination of methods( we found that&
The reduction of mortalit% is&o Due to fewer deaths from infectious and
parasitic diseases.o Not due to changes in death rates from reasons
not related to water and sanitation
The largest decrease in infant mortalit%occurred in low income municipalities.
R f
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
42/44
References4badie( 4. 51226. >Semiparametric Di8erence9in9Di8erences!stimators?( 4e/ie% o #conomic 5tudies( K1.
4shenfelter( . 53JKL6& >!stimating the !8ect of Training$rograms on !arnings(? *&e 4e/ie% o #conomic and 5tatistics,A2( 3. pp. MK9K"
ha%( Ren( +c!wan( $atrick and +iguel IrCuiola 51226& >The
central role of noise in evaluating interventions that use testscores to rank schools(? American #conomic 4e/ie%( J( pp.31;K9L.
'ertler( $aul 5122M6& >Do onditional ash Transfers mprovehild Health: !vidence from $-'-!S4=s ontrol -andomi,ed
!Schooling and abor +arket onseCuences ofSchool onstruction in ndonesia& !vidence "rom an Inusual$olic% !Water forife& The mpact of the $rivati,ation of Water Services on hild+ortalit%(?6ourna' o 7o'itica' #conom3, 0olume 33;( pp. L;9312"
Bertrand( +.( Du7o( !. and S. +ullainathan 5122M6. >How muchshould we trust di8erences9in9di8erences !stimates:(?8uarter'3 6ourna' o #conomics.
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
43/44
Thank JouThank Jou
7/24/2019 21.Technical Dif in Dif Premand Holla ENG PP
44/44
K & AK & A