Upload
hector-simon
View
219
Download
0
Embed Size (px)
Citation preview
Does Remediation Help All Students?
How the Effects of Postsecondary Developmental Courses Vary by
Background and Ability
Bridget Terry LongAngela BoatmanHarvard Graduate School of Education
IES ConferenceJune 2010
Florida Remediation Studies Developmental courses at the community colleges All schools use the same placement instrument
and have the same cutoff for placement Florida Administrative Dataset: Sample: All
first-time, degree-seeking CC students who began in Fall 1997, 1998, 1999, or 2000 (over 100,000 records)
Remedial and Developmental Postsecondary Courses
Lack of academic prep is a significant barrier to success –
Nationally approx. 40% of 1st year students are placed into college remediation (55-60% at CCs)
How should this problem be addressed?
Juan Carlos Calcagno and B.T. Long – "The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting &
Noncompliance“ and “Does Remediation Help All Students? The Heterogeneous Effects of Postsecondary Developmental Courses"
Regression Discontinuity: The Intuition
Beth and Becky are observationally similar
Compare the outcomes of Beth & BeckyCompare the outcomes of Beth & Becky
Beth and Becky both take the College Placement Test (CPT)
Beth scores just above the cut-off score
Becky scores just below the cut-off score
Beth to college-level courses Becky to remediation
Endogenous Sorting:
Beth retests to place out of remediation
Crossover: Beth takes
remediation anyway
No Show: Becky never
enrolls in remediation
Negative Effects on Passing First College-Level Course
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.066(0.008)
Passing First College-Level Course
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.025(0.004)
2 yr Degree Completion
3035
4045
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = 1.527(0.447)
Total Credits Earned
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.009(0.008)
Fall-to-Fall Retention
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.016(0.004)
Transfer to 4 yr
2025
3035
40
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -1.751(0.467)
Total College-Level Credits Earned
Placed in Development
al Reading
Placed in College Level
Passing the First College-Level Course by Reading CPT Score at the
Discontinuity
Positive Effects on Total Credits Earned
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.066(0.008)
Passing First College-Level Course
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.025(0.004)
2 yr Degree Completion
3035
4045
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = 1.527(0.447)
Total Credits Earned
0.2
.4.6
.8
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.009(0.008)
Fall-to-Fall Retention
0.2
.4
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -0.016(0.004)
Transfer to 4 yr
2025
3035
40
-50 -40 -30 -20 -10 0 10 20 30CPT Score Relative to Reading Cutoff
Estimated Discontinuity = -1.751(0.467)
Total College-Level Credits Earned
Total Credits Earned by Reading CPT Score at the Discontinuity
Placed in Development
al Reading
Placed in College Level
Florida Remediation Studies Being assigned to a developmental course appears
to increase total credits completed for students on the margin of passing out of the requirement
Does not increase the completion of college-level credits or eventual degree completion.
Remediation might promote early persistence but does not necessarily help students on the margin of passing the cutoff to make long-term progress
What about students of different backgrounds? The effects were: More positive for female (compared to male),
older (compared to younger), and LEP (compared to non-LEP)
More negative for low-income student (compared to non-Pell-eligible)
Does remediation work for students with far less
preparation? Past studies (FL, OH, TX) focus on students on
the margin of needing remediation -- they do not investigate the effects of remediation on students who are extremely under-prepared (i.e. don’t have an appropriate control group)
The Tennessee Case Multiple cutoffs and changes in placement
policy over time Investigate effects of different levels of remediation using multiple RDs
Sample: Fulltime students under 21 who began at a TN public college or university in the fall of 2000“Does Remediation Work for All Students? How
Remedial and Developmental Courses Affect Students With Different Levels of Academic Preparation” – A.
Boatman and B.T. Long
Remediation in Remediation in TennesseeTennessee
ACT/SAT score
determines which
placement exam (if
any) Take Take COMPASS COMPASS Arithmetic Arithmetic
examexam
Take Take COMPASS COMPASS Algebra Algebra examexam
Take Take DevelopmentDevelopmental Algebra Ial Algebra I
Go to Go to College-College-
level level coursescourses
Take Take Remedial Remedial ArithmeticArithmetic
Take Take DevelopmentDevelopmental Algebra IIal Algebra II
COMPASS Arithmetic test
COMPASS Algebra II test
CollegeCollegeLevelLevel
Devel. Devel. Alg IIAlg II
Devel. Devel. Alg IAlg I
Remed.Remed.Arith.Arith.
Score 0-29Score 0-29Score 30-100Score 30-100
Score 0-27Score 0-27Score 28-49Score 28-49Score 50-100Score 50-100
RD #1: Developmental Algebra II vs. College-level
COMPASS Arithmetic test
COMPASS Algebra II test
CollegeLevel
Devel. Alg II
Devel. Alg I
Remed.Arith.
Score 0-29Score 30-100
Score 0-27Score 28-49Score 50-100
Total Credits after 3 years
College Credits after 3 years
Completed any Degree or Certif.
FULL SAMPLE Assigned to the Lower-level
-1.4753 (2.6086)
-6.4105** (2.6360)
-0.2333* (0.1297)
Observations 490 490 490 FOUR-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
-0.9504 (3.3765)
-4.8965 (3.4042)
-0.1176 (0.1687)
Observations 263 263 263 TWO-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
-2.1376 (4.1381)
-8.3325** (4.1780)
-0.4397** (0.2080)
Observations 227 227 227
RD #1: Effects of Recommendation to Develop. Algebra II –vs– College-Level Math
Credits Accumulated Over Three Years
5
15
25
35
45
55
65
1 2 3 4 5 6
Mea
n C
redi
ts C
ompl
eted
Semester After Initial Enrollment
Total Credits - College-Level
Total Credits - Develop Alg II
College Credits - College-level
College Credits - Develop Alg II
Diff in College Credits
Not much Difference in Total Credits
4245
4851
5457
6063
66N
umbe
r of
Cre
dits
-6 -5 -4 -3 -2 -1 0 1 2 3 4 5 6COMPASS Score Relative to Cutoff
Developmental Algebra II vs. College-levelCumulative College Credits After Three Years
RD #1: Effects of Recommendation to Develop. Algebra II –vs– College-Level Math
Negative Effects in
Cumulative College Credits
RD #3: Remedial Arithmetic vs. Develop Algebra I
COMPASS Arithmetic test
COMPASS Algebra II test
CollegeLevel
Devel. Alg II
Devel. Devel. Alg IAlg I
Remed.Arith.
Score 0-29Score 30-99Score 30-99
Score 0-27Score 0-27Score 28-49Score 50-100
Total Credits after 3 years
College Credits after 3 years
Completed any Degree or Certif.
FULL SAMPLE Assigned to the Lower-level
0.0524 (1.7409)
-3.0366* (1.5681
-0.1227 (0.1009)
Observations 1279 1279 1279 FOUR-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
-3.2922 (3.0729)
-5.7550** (2.9051)
-0.2475+ (0.1597)
Observations 469 469 469 TWO-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
2.2181 (2.0909)
-1.2294 (1.8187)
-0.0140 (0.1335)
Observations 810 810 810
Math Developmental/Remedial Education
Total College Credits Completed after 3 years
RD#1: College-
Level vs. Develop. Algebra II
RD #2: Develop. Algebra II vs.
Develop. Algebra I
RD #3: Develop. Algebra I vs.
Remedial Arithmetic FULL SAMPLE
Assigned to the Lower-level
-6.4105** (2.6360)
-6.8230*** (2.5605)
-3.0366* (1.5681)
Outcome Mean 54.23 54.23 36.44 Observations 490 1562 1279
FOUR-YEAR INSTITUTIONS ONLY Assigned to the Lower-level
-4.8965 (3.4042)
-5.2957 (5.9278)
-5.7550** (2.9051)
Outcome Mean 61.73 61.73 51.33 Observations 263 697 469
TWO-YEAR INSTITUTIONS ONLY Assigned to the Lower-level
-8.3325** (4.1780)
-8.1737*** (2.7701)
-1.2294 (1.8187)
Outcome Mean 46.34 46.34 28.73 Observations 227 865 810
Smaller in MagnitudeSmaller in Magnitude
College-Level vs. Developmental vs. Remedial Writing
RD #2: Develop. vs. Remedial Writing
Total Credits Completed after 1 year
1st to 2nd Semester
Persistence
Still Enrolled in Year 3
Total Credits
after 3 yrs
College Credits
after 3 yrs
Completed Degree or
Certif.
Assigned to the Lower-level
0.4092 (0.6548)
0.0664 (0.1524)
0.2197+ (0.1378)
3.9792+ (2.7198)
-0.4107 (2.5282)
0.3057* (0.1623)
FOUR-YEAR INSTITUTIONS ONLY Assigned to the Lower-level
-1.0377 (0.7709)
-0.2335 (0.2544)
0.0713 (0.1970)
-0.6326 (4.0214)
-3.2463 (3.9283)
0.2226 (0.2011)
TWO-YEAR INSTITUTIONS ONLY Assigned to the Lower-level
1.2481 (0.9836)
0.2150 (0.2002)
0.3446* (0.1927)
6.9964* (3.6959)
1.7769 (3.3119)
0.3913 (0.2530)
CollegeComposition
Developmental Writing
Remedial Writing
Score 68-100 Score 0-27Score 28-67COMPASS
English test
Grade in first college-level course MATHEMATICS WRITING
College-Level vs.
Develop. Education (RD #1)
Upper Develop. vs.
Lower Develop. (RD #2)
Develop. vs.
Remedial Education (RD #3)
College-Level vs.
Develop. Education (RD #1)
Develop. vs.
Remedial Education (RD #2)
FULL SAMPLE Assigned to the Lower-level
0.1710 (0.1565)
-0.0068 (0.2027)
-0.0352 (0.1634)
0.0342 (0.0998)
0.5389*** (0.1598)
Observations 335 803 375 782 522 FOUR-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
0.2169 (0.2025)
0.3306 (0.4126)
0.1068 (0.2462)
0.2791* (0.1464)
0.1954 (0.1803)
Observations 185 372 183 315 231 TWO-YEAR INSTITUTIONS ONLY
Assigned to the Lower-level
0.1451 (0.2422)
-0.1503 (0.2353)
-0.0617 (0.2128)
-0.1438 (0.1356)
0.8960*** (0.2765)
Observations 150 431 192 467 291
Not causal analysis, but interesting estimates…
Summary of Tennessee Results Generally negative effects if placed in a
lower math class, but the size of the effect is smaller farther down the test score distribution
Similar effects found for Developmental and Remedial Reading
In Writing, the effects are negative at the college/developmental threshold but positive at the developmental/remedial threshold
Positive effects on first college-level grade (though these results are not causal)
Implications and Remaining Questions
Remedial and Developmental courses do not affect all types of students equally (by background, ability level, or college level)
What is the best way to offer remediation? Characteristics of strong remedial programs?
What are the effects of limitations states impose on remedial course-taking (e.g., only at CCs, time limits)?
Early Placement Testing – a preventative measure?
Contact Information
Prof. Bridget Terry Long, Ph.D.Harvard Graduate School of Education, NCPR, and [email protected]
Angela Boatman, Doctoral CandidateHarvard Graduate School of [email protected]