S I M O N M C D O N N E L L

C O L I N C H E L L M A N

G I L J A E L E E

D A V I D C R O O K

O F F I C E O F P O L I C Y R E S E A R C H

T H E C I T Y U N I V E R S I T Y O F N E W Y O R K

The impact of Hurricane Sandy on students at the City University of New

York

1

Hurricane Sandy 2

October 29th 2012

Over 40 New Yorkers died

CUNY:

4 campuses damaged

10 campuses used for displaced NYers

Classes disrupted for a week

Hardship withdrawals

Questions

How many students were impacted?

Who were they?

Where were they?

Did students who lived in the surge zone do worse as a result of being exposed the storm?

Were key measures of academic progress worse for students living in the surge zone relative to similar students?

3

Outline

Our research questions

Some related literature

Data & Methodology

PSM Groups:

DiD

Additional regressions analysis

Results

Fall 2011 FTF -> Fall 2012 V peers

Conclusions & next steps

4

Literature - Potential Disruptions to Academic Progress

Examples Type of Disruption

Type of Impact Academic Non-Academic

Academic

Stopping out associated with lower

likelihood to graduate with a BA

Increased education funding following

tsunami in Indonesia (2004)

Non-Academic Increased financial strain from student

loans repayment after dropping out

Changes in voting patterns following

Hurricane Andrew (1992)

5

Many factors in student success:

High school prep., full-time, gender, race, family history

Natural disasters are historic, disruptive events

Can be seen as disruptions in students’ academic and non-academic lives

Literature - Potential Disruptions to Academic Progress

Examples Type of Disruption

Type of Impact Academic Non-Academic

Academic

Stopping out associated with lower

likelihood to graduate with a BA

Increased education funding following

tsunami in Indonesia (2004)

Non-Academic Increased financial strain from student

loans repayment after dropping out

Changes in voting patterns following

Hurricane Andrew (1992)

6

Disasters: Hurricane Katrina (Herlihy and Phillips, 2009), Indian ocean tsunami (Pelupessy et al), school shootings etc.

Family disruptions (Ver Ploeg, 2002)

Drugs/alcohol (Arria et al, 2013)

Little on higher ed. outcomes

Data

Student-level data from Office of Institutional Research and Assessment at CUNY;

The Primary Land Use Tax Lot Output (PLUTO) geospatial dataset;

Inundation flood maps developed by FEMA;

American Community Survey and decennial census data from the U.S. Census Bureau.

Bad identification of impacted students?

Too inclusive?

7

Initial Results

Fall 2012 snapshot:

17,000 of 225,000 NYC CUNY students (undergrads and grads) in surge zone – 8% of all students

Brooklyn (50%), Manhattan (20%), Queens (18%), Staten Island (8%), and the Bronx (3%).

These numbers were used to reach out to potentially impacted students

8

Methodology 1

PSM – similar to CUNY ASAP methodology

Nearest neighbor, caliper (.20 SD), non-replacement

One Cohort over time

Fall 2011 FTF Retained to Fall 2012

GPA Diff (Semester 3- Semester 1)

Credits Withdrawn Diff (Semester 3 – Semester 1)

Big caveat:

Retained to fall 2012 – more resilient students

Fall 2012 students – similar results…

9

Fall 2011 Students Retained to Fall 2012

Associate Students Baccalaureate Students

Non-Surge Zone Surge Zone Non-Surge Zone Surge Zones

Total Students (fall 2011) 19,958 1,665 8,745 766

Total Students (retained to fall 2012)

12,915

(64.7%)

1,015

(60.9%)

7,599

(86.9%)

658

(85.9%)

10

Methodology 2

DiD as Average Treatment Effect on the Treated

ACADi represents the difference in GPA score and credits withdrawn between semesters 1 and 3 for each student i.

ATT = ∑ i (ACADNon-Surge*Semester3 - ACADNon-Surge*Semester1) - ∑ i (ACADSurge*Semester3 - ACADSurge*Semester1 )

Additional regression analysis on matched groups

Same as surge with additional controls

Boro, college and major fixed effects

11

Methodology 3

Group membership (AA & BA Separately):

SURGEit = β0 + β1SEDit + β2ECOLit + β3CITit + β5HOODit + β5BOROit + εit

SED: NYCHA, race, gender, educationally disadvantaged, Pell status, dependency status

ECOL: No delay, fulltime, credits attempted (sem. 1), SEEK/CD, remediation

CIT: citizenship status

HOOD: neighborhood age, gender, earnings, education, race, building stock

BORO: Boro fixed effects

12

PSM Results

Fall 2011 FTF Retained to Fall 2012

AA:

1,015 student in Surge zone (7.3%)

799 matched to a nearest neighbor (12,915 Non-surge)

BA:

658 student in Surge zone (7.9%)

536 matched to a nearest neighbor (7,599 N0n-surge)

Bias reduction for almost all IV’s & balanced samples

13

Selected Descriptives: Associate

Variable Unmatched/ Matched

Treated (Surge)

Control (Non-Surge) % Bias

Bias Reduction

In NYCHA BBL U 0.203 0.056 44.7

M 0.173 0.173 0 100

No Pell Disbursed U 0.294 0.254 8.9

M 0.305 0.289 3.7 58.9

Educationally Disadvantaged U 0.737 0.771 -7.8

M 0.737 0.762 -5.8 25.5

No delay after high school U 0.808 0.769 9.5

M 0.809 0.796 3.1 67.7

Participant in SEEK/CD Program U 0.035 0.030 3.3

M 0.036 0.036 0 100

In Need of Any Remediation U 0.637 0.665 -5.8

M 0.633 0.647 -2.9 49.8

Age as of FTF date U 19.656 19.920 -6.5

M 19.568 19.835 -6.5 -1.2

Asian female U 0.047 0.082 -14

M 0.049 0.044 2 85.4

Asian male U 0.065 0.088 -9

M 0.065 0.071 -2.4 73.7

Black female U 0.194 0.161 8.7

M 0.204 0.194 2.6 69.9

Black male U 0.130 0.122 2.5

M 0.140 0.126 4.1 -64.4

Hispanic female U 0.141 0.218 -20.1

M 0.139 0.133 1.6 91.8

Hispanic male U 0.090 0.161 -21.5

M 0.093 0.100 -2.3 89.4

14

Fall 2011 FTF Fall 2012: Associate

Variable Sample Treated Controls Difference S.E. T-stat

Difference in GPA Sem. 3 – Sem. 1 Unmatched -0.303 -0.171 -0.132 0.063 -2.09

ATT -0.303 -0.163 -0.140 0.071 -1.96

GPA: Semester 3 Unmatched 2.173 2.260 -0.087 0.059 -1.48

ATT 2.171 2.289 -0.118 0.066 -1.80

GPA: Semester 1 Unmatched 2.476 2.431 0.045 0.053 0.85

ATT 2.474 2.452 0.021 0.060 0.35

Diff. in Credits W/D Sem. 3 – Sem. 1 Unmatched 1.057 0.898 0.159 0.158 1.01

ATT 1.052 0.922 0.130 0.178 0.73

Credits Withdrawn: Semester 3 Unmatched 1.768 1.558 0.211 0.150 1.41

ATT 1.766 1.560 0.206 0.167 1.23

Credits Withdrawn: Semester 1 Unmatched

0.712 0.660 0.052 0.090 0.58

ATT 0.714 0.638 0.076 0.098 0.78

15

Fall 2011 FTF Fall 2012: Baccalaureate

Variable Sample Treated Controls Difference S.E. T-stat

Difference in GPA Sem. 3 – Sem. 1 Unmatched -0.152 -0.148 -0.004 0.053 -0.07

ATT -0.159 -0.115 -0.043 0.063 -0.69

GPA: Semester 3 Unmatched 2.844 2.877 -0.033 0.058 -0.58

ATT 2.841 2.937 -0.096 0.068 -1.41

GPA: Semester 1 Unmatched 2.996 3.026 -0.030 0.045 -0.66

ATT 2.999 3.052 -0.053 0.053 -0.99

Diff. in Credits W/D Sem. 3-Sem. 1 Unmatched 0.461 0.417 0.044 0.158 0.28

ATT 0.473 0.280 0.192 0.185 1.04

Credits Withdrawn: Semester 3 Unmatched 0.906 0.896 0.010 0.146 0.07

ATT 0.905 0.787 0.118 0.166 0.71

Credits Withdrawn: Semester 1 Unmatched

0.445 0.479 -0.034 0.090 -0.38

ATT 0.432 0.507 -0.074 0.107 -0.69

16

What does this mean?

GPA Values - Current

Grade Undergraduate

A+ 4.0

A 4.0

A- 3.7

B+ 3.3

B 3.0

B- 2.7

C+ 2.3

C 2.0

D 1.0

F 0

WU 0

FIN 0

FAB 0

17

Additional Regression Analysis

Academic Outcome (AA & BA Separately):

ACADit = β0 + β1SURGEit + β2SEDit + β3ECOLit + β3CITit + β5HOODit +

β5HSit + β5FXit + εit

GPA Differences and Credit Withdrawn Differences Changed Major between semester 1 and semester 3

Testing impact of additional boro, college and major fixed effects

18

Regression Results: Fall 2011 FTF Fall 2012 (Associate)

Associate Students: GPA DIFFERENCE (Sem. 1 – Sem. 3)

Model 1: BORO FX

Model 2: Model 3: Model 4: Model 2A: Model 3A: Model 4A:

Lived in Surge -0.134** -0.168**

-0.194*** -0.198***

-0.183** -0.207*** -0.211***

SED + + + + + +

ECOL + + + + + +

CIT + + + + + +

N’HOOD + + + + + +

HS + + + + + +

Boro FX + + + + +

College FX + + + +

Major FX + + +

N 1,598 1,467 1,467 1,467 1,467 1,467 1,467

R2 0.005 0.074 0.080 0.081 0.087 0.093 0.094

19

Regression Results: Fall 2011 FTF Fall 2012 (Associate)

Associate Students: CREDITS W/D DIFFERENCE

Model 1: BORO FX

Model 2: Model 3: Model 4: Model 2A: Model 3A: Model 4A:

Lived in Surge 0.155 0.243 0.305* 0.316* 0.252 0.341** 0.334**

SED + + + + + +

ECOL + + + + + +

CIT + + + + + +

N’HOOD + + + + + +

HS + + + + + +

Boro FX + + + + +

College FX + + + +

Major FX + + +

N 1,598 1,467 1,467 1,467 1,467 1,467 1,467

R2 0.007 0.068 0.062 0.073 0.088 0.093 0.095

20

Conclusions

Small, and in many cases, insignificant impact on our students (so far)

But more resilient students are… more resilient.

Interventions by CUNY and other actors?

Suspension of classes? Outreach?

Our sample more resilient than average?

Perhaps our students are resilient – look at 9/11 for instance

Bad identification of impacted students?

Too inclusive?

Work with administrators etc to get a more nuanced sample

Look at longer time frame to confirm treatment and control groups follow same path

21

Next Steps

Should we include a cross sectional of the fall 2012 entering cohort?

Where should we aim this paper at?

22