1

Is there a systematic way to assess regression studies? (SW Chapter 9)

2

A Framework for Assessing Statistical Studies: Internal and External Validity

3

Threats to External Validity

4

Threats to Internal Validity

5

1. Wrong functional form

6

2. Omitted variable bias

7

3. Errors-in-variables bias

8

Errors-in-variables bias

9

10

Errors-in-variables bias

11

Potential solutions to errors-in-variables bias

12

4. Sample selection bias

13

Example #1: Mutual funds

14

Example #2: Returns to education

15

Potential solutions to sample selection bias

16

5. Simultaneous causality bias

17

Simultaneous causality bias

18

Potential solutions to simultaneous causality bias

19

Internal and External Validity When the Regression is Used for Forecasting

20

Applying External and Internal Validity: Test Scores and Class Size

21

The Massachusetts data: summary statistics

22

23

24

25

26

Predicted effects for a class size reduction of 2

27

28

Summary of Findings for Massachusetts

29

Comparison of estimated class size effects: CA vs. MA

30

Summary: Comparison of California and Massachusetts Regression Analyses

31

Remaining threats to internal validity in the test score/class size example?

32

Spot the Endogeneity?

• Effect of police on crime …

• Estimating the effect of police on crime …

• Source: Levitt (American Economic Review, 1997)

33

Spot the Endogeneity?• Estimating the effect of police on crime …

34

Spot the Endogeneity?• Estimating the effect of guns on crime …

• Source: Ayres and Donohue (Stanford Law Review, 2003)

35

36

37

38

39

Spot the Endogeneity?• Estimating the effect of studying on GPA …

• Source: Stinebricker and Stinebricker (BE Journal of Analysis & Policy, 2008)

40

Spot the Endogeneity?• Estimating the effect of school start time on academic

performance …

• Source: Carrell, Maghakian, and West (American Economic Journal: Economic Policy, 2011)

41

42

Spot the Endogeneity?• Estimating the effect of national income on democracy

…

• Source: Acemoglu, Johnson, Robinson, and Yared (American Economic Review, 2008)

43

Spot the Endogeneity?• Estimating the effect of competition on student

achievement …

• Source: Hoxby (American Economic Review, 2000)

44

Spot the Endogeneity?• Estimating the effect of teen drinking on HS

completion/attending college …

• Source: Dee & Evans (Journal of Labor Economics, 2003)