Lauren Schudde The University of Texas at Austin & CAPSEE

Judith Scott-Clayton Teachers College & CAPSEE

The Effects of Satisfactory Academic Progress Requirements on Pell Grant Recipients

April 6–7, 2017 | Washington, DC

Evidence from Two States

• Results from two CAPSEE papers: – Pell Grants as Performance-Based Scholarships? An

Examination of Satisfactory Academic Progress Requirements in the Nation’s Largest Need-Based Aid Program (Schudde & Scott-Clayton, 2014, 2016)

– Performance Standards in Need-Based Student Aid (Scott-

Clayton & Schudde, 2016)

2

Federal Eligibility for Need-Based Aid

• Initially based on financial need; no consideration of prior achievement

• Renewal requires meeting “Satisfactory Academic Progress” (SAP) requirements

• Applies to all federal aid recipients—we focus on Pell program

Federal SAP Guidelines

• Institutions must assess SAP for federal aid recipients using: – “Qualitative Standard”: 2.0 or equivalent by the end of second

academic year – “Quantitative Standard”: Minimum percentage of work successfully

completed – Maximum timeframe: Cannot exceed 150% of published length of

undergraduate program in credits

• Flexibility in how institutions implement, but most use: – Cumulative 2.0 GPA and 2/3 ratio of credits attempted vs. completed

Source: CFR 668.34; Information for Financial Aid Professionals handbook, ch.1, vol.1

4

First-Year Students with GPA<2.0, by Pell Receipt: NPSAS 2011-2012

5

24%

15%

25%

15%

19%

8%

21%

12%

0%

5%

10%

15%

20%

25%

30%

Public 4-Year Private Nonprofit 4-Year Public 2-Year Private For-Profit 2-Year

Perc

enta

ge o

f Firs

t-Ye

ar S

tude

nts

Pell Recipients

Non-PellRecipients

Source: Schudde & Scott-Clayton, 2016

Performance Requirements in Need-Based Aid: What Roles does SAP Policy Serve?

Performance Requirements in Need-Based Aid: What Roles does SAP Policy Serve?

• 1. Financial incentive for academic effort early in college – Not tested on federal aid prior to our work on SAP and Pell students – Research on performance-based scholarships and promise programs:

merit requirements can improve student progress and achievement (Barrow, Richburg-Hayes, Rouse, & Brock, 2014; Barrow & Rouse, 2013; Patel & Valenzuela, 2013; Scott-Clayton, 2011)

Performance Requirements in Need-Based Aid: What Roles does SAP Policy Serve?

• 2. Signal performance expectations to students – The 2.0 cumulative GPA standard aligns with requirements for

graduation – Informs students they are below requirements and allows them to

adjust decisions • “Adjusting decisions” may mean improving performance or

discouraging from persistence

Performance Requirements in Need-Based Aid: What Roles does SAP Policy Serve?

• 3. Improve efficiency of federal aid – Minimize program spending on students with low likelihood of

graduation – Concentrates federal supports on students with higher likelihood of

success

Research Questions and Methods

How does SAP impact students?

• How many students fail to meet SAP? – Which requirements do they fail? – How do Pell students compare to students without federal aid?

• What is the impact of SAP standards on persistence, transfer, and degree attainment?

11

Data

• Administrative data from two states – 3 year follow-up for outcomes

• Both focus on first-time CC enrollees – State community college system 1 (State 1)

• 49 colleges with equivalent SAP policies • Fall cohorts 2002-2007: ~147,000 (43,000 Pell recipients)

– State community college system 2 (State 2) • 20 community colleges • Fall cohorts 2004-2010: ~113,000 (52,000 Pell recipients)

12

Outcomes of Interest

• Short-term outcomes – Persistence into second year, GPA in first-term of second year – Behavioral responses to being warned about SAP status (not loss of

aid, yet)

• Longer-term outcomes – Credit and degree completion at end of third year, – Behavioral response to warning and consequences of losing aid (for

some, but not all) students—drop out

13

Methods

• Descriptive Statistics – Patterns of SAP failure for GPA and credit standards

• Leverage 2.0 GPA cutoff and two approaches: – Regression Discontinuity (RD) – Difference-in-Differences Approach (DD)

14

Methods

• Regression Discontinuity (RD) – Use GPA at end of year 1 to examine “discontinuity” at the cutoff – Compare subsequent enrollment and performance of Pell recipients

above and below 2.0 cutoff

15

Methods

• Difference-in-Differences Approach (DD) – Compares patterns above and below 2.0 cutoff for Pell recipients and

non-recipients

– Determines effect of falling below cutoff for Pell recipients by comparing to students who are not subject to SAP standards

16

Methods

17

Pell Non-Pell

.463 Below

.578 Above

-.115 Diff for Pell

Difference of the Difference: -.115 -.075= -.040

.561 Below

.636 Above

-.075 Diff for non-Pell

Persistence into Year 2 Estimate

Results

18

Distribution of Pell Entrants by Overall SAP and Enrollment Status Over Time: State 1

19

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Fall, Y1 Spring, Y1 Fall, Y2 Spring, Y2 Fall, Y3 Spring, Y3

Perc

ent o

f Pel

l Rec

ipie

nts

Term

Enrolled, Failing SAP Enrolled, Passing SAP Not Enrolled

Based on Schudde & Scott-Clayton, 2016—results from State 1

Fall to Fall Persistence by First-Year GPA and Pell Status: State 1

20

0%

10%

20%

30%

40%

50%

60%

70%

80%

1.10

1.20

1.30

1.40

1.50

1.60

1.70

1.80

1.90

2.10

2.20

2.30

2.40

2.50

2.60

2.70

2.80

2.90

Perc

ent o

f Fal

l Ent

rant

s Stil

l Enr

olle

d

First-Term GPA

Pell

No Pell

Based on Schudde & Scott-Clayton, 2016—results from State 1

Impacts of First-year SAP Failure on GPA, Fall of Second Year: State 2

21

0.07*

0.03

0

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

RD (Near 2.0) DD (Incl. GPA as low as 1.0)

GPA

Poi

nt Im

prov

emen

t RD (GPAs near 2.0) DD (GPAs as low as 1.0)

Impacts of First-year SAP Failure on Enrollment, Fall of Second Year: State 2

22

-1 pp

-8 pp*** -9

-8

-7

-6

-5

-4

-3

-2

-1

0RD (Near 2.0) DD (Incl. GPA as low as 1.0)

Perc

enta

ge P

oint

Red

uctio

n in

En

rollm

ent

RD (GPAs near 2.0) DD (GPAs as low as 1.0)

Impacts on Enrollment and Certificate Completion at End of Year 3: DD, State 2

23

-5 pp***

-2 pp***

-6

-5

-4

-3

-2

-1

0Enrollment Certificate Completion

Perc

enta

ge P

oint

Dec

line

Impacts on Credits Attempted and Earned at End of Year 3: DD, State 2

24 -3.17***

-1.12***

-3.5

-3

-2.5

-2

-1.5

-1

-0.5

0Total Credits Attempted Total Credits Earned

Redu

ctio

n in

Cre

dits

Att

empt

ed/E

arne

d

Discussion

• Efficiency vs. Equity – Makes aid more efficient – To the detriment of the neediest students

• Primary effect of SAP policy = punitive, not formative – Limits students access to aid – Does not incorporate adequate intervention to improve

skills/performance

• Anecdotal evidence from administrators and aid officers: students do not learn about SAP until they lost aid – Are warning systems working? How can we optimize them? 25

Implications and Future Inquiry

• Proactive, early communication—before end of first term—could help students stay on track

• Improved coordination between administrative offices – Academic advisors and aid officers must communicate to stay

informed about academic progress and aid status – Need stronger strategies to reach students and keep them informed

• SAP failure is quite high in four-year colleges as well – This is a widespread problem – Need more research to understand its impact in various settings

26

Visit our website at capseecenter.org

You can also follow us on Twitter at @capsee and like us on Facebook.

Center for Analysis of Postsecondary Education and Employment

Teachers College, Columbia University

525 West 120th Street, Box 174, New York, NY 10027

TEL: 212.678.3091 | capsee@columbia.edu

CAPSEE is funded through a grant (R305C110011) from the Institute of Education Sciences, U.S. Department of Education.