CLEP Me Out of Here: The Impact of Prior Learning Assessments on

College Completion

Angela Boatman, Vanderbilt UniversityMichael Hurwitz, College BoardJason Lee, University of Georgia

Jonathan Smith, Georgia State University

Motivation

• College completion rates are low– 60% among four-year colleges– 20% among two-year colleges– Over last 20 year, ~31 million left college with no

degree (Shapiro et al., 2014)

• Improving completion focus of policy, research, and interventions– Some success (e.g. ASAP- doubling AAs, ~$5,400/stud)– Typically null to modest impacts

This Paper

• Estimates impact of College Level Examination Program (CLEP) on degree completion– $85 competency based exam– Accepted at nearly 3,000 institutions

• Use a RDD on over 800k exam takers– Numerous military members, non-traditional

aged, and homeschoolers

Main Findings

• Improved AA rates for two-year students– Overall - 5.7 p.p. (17.3%) – Military members – 8.6 p.p. (18.1%)– Older than 24 – 4.8 p.p. (19.5%)– Homeschoolers – 9.1 p.p. (62.8%)

• Improved BA rates for four-year students– Overall – 1.5 p.p. (2-3 percent)

Literature/Contributions• Large literature on college completion

– Impacts are larger than most (in two-year)– Cheaper than most (e.g., CUNY ASAP)

• Course requirements/credit– Remediation (e.g., Boatman and Long, 2018)– AP (e.g., Smith et al., 2017), dual enrollment– PLA/competency based models (e.g., Klein-Collins &

Wertheim, 2013)

• Under researched groups

CLEP Background• Credit-by-exam program that validates student knowledge and

translates learning into college credit• Serves college students, high school students, military service

members, adults learners, and others• 2,900 colleges grant credit for CLEP exams• 1,900 test centers administer CLEP• 170,618 exams administered in 2016-17

• Non-military: 122,746

• Military: 47,872• More than eight million exams taken since 1967

CLEP BackgroundComposition and Literature• American Literature • Analyzing and Interpreting

Literature • College Composition• College Composition Modular• English Literature • Humanities

Science and Mathematics• Calculus• College Algebra • Precalculus• College Mathematics• Biology • Chemistry • Natural Sciences

World Languages• French Language • German Language • Spanish Language

History and Social Sciences• American Government• History of the United States I• History of the United States II• Human Growth and Development• Introduction to Educational

Psychology• Principles of Macroeconomics• Principles of Microeconomics• Introduction Psychology• Introductory Sociology• Social Sciences and History• Western Civilization I• Western Civilization II

Business• Information Systems and

Computer Applications • Principles of Management • Financial Accounting• Introductory Business Law • Principles of Marketing

CLEP Background• American Council on Education (ACE) recommends credit for

all CLEP exams• Standing faculty committees oversee ongoing test

development, shape content, review data, set exam policies– Nation-wide surveys of faculty conducted regularly

• More than 600 faculty contribute to the development of and standard-setting for CLEP examinations

• All exams go through multi-step validation process• Items are relevant to purpose of the exam• Items are fair and lead to a decision• Items are non-biased for all types of test takers • Items are rigorously field tested prior to going live• Developed by subject matter experts with requisite experience• Content is fair for the length of the exam• The exam does what it is supposed to do• Ongoing review of item and test performance

CLEP Background

15%

19%

12%16%

10%

18%

10%

Education Level

High School Student

High School Graduate

College Freshman

College Sophomore

College Junior

College Senior

College Graduate

11%

39%27%

11%

12%

Age

Under 18

Age 18-22

Age 23-29

Age 30-35

Age 36 and older

Data• Over 800k unique CLEP takers between 2008-

2015• Observe exam scores, location of exam, basic

demographics• Merged with college enrollment data from the

NSC• Credit-granting scores obtained from Annual

Survey of Colleges• BRAND NEW: Merged with Labor Market Data

from TransUnion

Exam Freq. Percent Avg. Score Std. Dev.Spanish Language 206,118 23.78 68.91 10.10Analyzing and Interpreting Lit 92,897 10.72 57.86 9.80College Algebra 54,936 6.34 49.48 12.49College Mathematics 44,996 5.19 52.86 12.25College Composition 42,868 4.95 54.11 5.82College Composition Modular 31,786 3.67 55.35 6.75Introductory Sociology 31,639 3.65 53.56 7.37Principles of Management 27,883 3.22 49.42 10.79History of United States I 25,225 2.91 51.52 10.66Introductory Psychology 21,838 2.52 56.81 9.72Biology 21,233 2.45 53.23 10.04English Composition 19,933 2.30 46.48 11.39Humanities 19,806 2.28 49.87 8.79Human Growth and Development 19,247 2.22 53.79 7.81Info Systems and Computer Appl 18,680 2.15 53.69 11.15French Language 18,512 2.14 65.04 11.90Freshman College Composition 17,477 2.02 57.07 11.02American Government 17,029 1.96 47.30 11.77History of United States II 15,391 1.78 49.85 9.83English Composition with Essay 14,813 1.71 51.65 10.68Natural Sciences 13,567 1.57 48.31 9.57Principles of Marketing 11,873 1.37 55.92 10.79Social Sciences and History 8,914 1.03 48.90 10.77Precalculus 6,914 0.80 52.87 10.48German Language 6,846 0.79 64.78 14.86American Literature 6,678 0.77 45.89 12.58Western Civilization I 6,436 0.74 54.48 9.01Principles of Microeconomics 6,112 0.71 52.79 13.86Principles of Macroeconomics 5,954 0.69 53.40 13.12Calculus 5,447 0.63 53.18 12.38Chemistry 5,211 0.60 46.35 11.49Intro to Educational Psych 4,744 0.55 50.50 12.26English Literature 4,116 0.47 49.85 10.34Western Civilization II 4,034 0.47 50.49 8.63Introductory Business Law 3,881 0.45 47.05 12.59Financial Accounting 3,859 0.45 49.41 12.87

Descriptive Statistics

Mean Std. Dev. Mean Std. Dev.Background Variables

Asian 0.046 0.209 0.054 0.225Black 0.094 0.291 0.109 0.311Hispanic 0.239 0.426 0.096 0.295White 0.556 0.497 0.672 0.469Other 0.057 0.231 0.060 0.238Missing Race 0.008 0.091 0.009 0.095Female 0.493 0.500 0.472 0.499Age 26.684 9.219 26.662 9.328Military Member 0.265 0.441 0.321 0.467Home Schooled 0.034 0.180 0.034 0.181

CLEP VariablesNumber of CLEP Exams 1.577 1.440 1.655 1.427First CLEP Score 56.989 12.772 51.064 5.618Average of All CLEP Scores 56.726 12.334 51.100 5.838Distance Between CLEP Score and 50 6.989 12.772 1.064 5.618CLEP Score Greater Than or Equal to 50 0.707 0.455 0.608 0.488

Outcome VariablesAny Degree 0.485 0.450 0.464 0.499Bachelor's Degree 0.346 0.476 0.332 0.471Associate's Degree 0.184 0.388 0.169 0.375

Full Sample (obs = 866,893)

Analytic Sample (bandwidth of 10, obs

= 449,092)

Where are the CLEP Test-Takers

Methodology – RDD

• Primary Specification:

𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑂𝑖𝑖𝑖𝑖 = 𝛽𝛽0 + 𝛽𝛽1𝐶𝐶𝐶𝐶𝑂𝑂𝐶𝐶𝐶𝐶𝑂𝑂𝑖𝑖𝑖𝑖 + 𝛽𝛽2𝐷𝐷𝐶𝐶𝐷𝐷𝑂𝑂𝑖𝑖𝑖𝑖 + 𝛽𝛽3𝐶𝐶𝐶𝐶𝑂𝑂𝐶𝐶𝐶𝐶𝑂𝑂𝑖𝑖𝑖𝑖 ∗ 𝐷𝐷𝐶𝐶𝐷𝐷𝑂𝑂𝑖𝑖𝑖𝑖 + 𝑋𝑋𝑖𝑖𝑖𝑖 + 𝜀𝜀𝑖𝑖𝑖𝑖

• 𝐷𝐷𝐶𝐶𝐷𝐷𝑂𝑂𝑖𝑖𝑖𝑖 is one of three running variables– Distance from credit granting score of first college

enrolled (primary)– Distance from 50– Distance from credit granting score at testing center

college

Methodology – Identification• Intuition – comparing two students with:

– Identical characteristics (observable and unobservable)– Nearly identical CLEP scores (e.g., 49 and 50)– But differ in receipt of credit from CLEP– How do their outcomes differ?

• Identifying assumptions (testable)– Students are in fact on average identical around threshold – No strategic behavior around the credit granting scores

• Can interpret 𝛽𝛽1 as the causal impact of receiving credit granting score from CLEP– LATE – only the impact for those around the threshold (usually 50)

Methodology – Details• Only use first exam

– Eliminates endogenous retaking

• Local linear regression– IK Bandwidth (usually between 7-10 points)– Triangular kernel

• Test sensitivity of specification– Variety of bandwidths– Variety of kernels– Including covariates– Second order polynomials of forcing variable

Testing RDD Assumptions

• Is there gaming by students (or scorers) around threshold?– Density tests

• Are students identical on observables across threshold? – Run RD using student characteristics as outcome– Use aggregate measure of predicted CLEP score– Examine graphically

0.0

1.0

2.0

3

-50 0 50 100

Dens

ity

Distance from Credit Granting Score at First College Enrolled

Density of Observations

Forcing Variable=

Distance from 50

Two-Year Four-Year First College Enrolled Testing CenterDensity -0.004 -0.040 0.001 -0.008 -0.007Above Threshold (0.009) (0.021) (0.011) (0.011) (0.010)CovariatesAsian -0.001 0.003 -0.003 -0.003 -0.001

(0.002) (0.004) (0.002) (0.002) (0.002)Black -0.002 -0.005 0.000 -0.004 -0.001

(0.003) (0.006) (0.003) (0.003) (0.003)Hispanic -0.003 -0.007 -0.003 -0.003 0.001

(0.002) (0.006) (0.003) (0.003) (0.003)White 0.011*** 0.009 0.013*** 0.016*** 0.007

(0.004) (0.009) (0.005) (0.005) (0.005)Other -0.005** -0.002 -0.008*** -0.007*** -0.007***

(0.002) (0.004) (0.002) (0.002) (0.002)Missing Race 0.001 0.002 0.000 0.001 0.001

(0.001) (0.002) (0.001) (0.001) (0.001)Female 0.001 -0.009 0.000 -0.000 -0.005

(0.004) (0.009) (0.005) (0.005) (0.005)Age 0.030 -0.238 -0.147 -0.412 -0.216

(0.244) (0.571) (0.294) (0.276) (0.298)Military Member -0.004 0.021** -0.007 0.002 0.002

(0.004) (0.009) (0.005) (0.004) (0.004)Home Schooled 0.003** 0.004 0.003* 0.002 0.003**

(0.001) (0.004) (0.001) (0.002) (0.002)Currently in High School -0.001 0.008 -0.002 0.002 -0.002

(0.003) (0.007) (0.003) (0.003) (0.003)High School Graduate -0.006* 0.011 -0.010*** -0.003 -0.002

(0.003) (0.007) (0.004) (0.003) (0.004)Enrolled in College 0.000 -0.012 0.004 -0.001 -0.002

(0.004) (0.009) (0.005) (0.005) (0.005)College Graduate 0.004 -0.006 0.004 -0.001 0.006**

(0.002) (0.005) (0.003) (0.002) (0.003)Predicted CLEP Score -0.004 -0.026 0.003 -0.047 0.027

(0.043) (0.044) (0.043) (0.050) (0.048)Observations 301,538 56161 191555 214869 215600

Covariate Balancing and Density Tests

Forcing Variable is Distance Between CLEP Score and Minimum Credit

Granting Score at:

First Took CLEP in College Where Level

is:

Quick Detour: Idea GenesisEarning a Credit-Granting AP Score Increases the Probability of On-Time BA Completion

Discontinuity of 2 percentage points

Main Result - AA

Discontinuity ≈ 5 percentage points

Main Result - BA

Discontinuity ≈ 1-1.5 percentage point

Earning a credit-granting CLEP score decreases the time to graduation among students who earn degrees.

Between CLEP Date and Associates

Degree

Between CLEP Date and Bachelors

Degree

Between Initial Enrollment and

Associates Degree

Between Initial Enrollment and

Bachelors Degree

Associates Degree

Within 3 Years

Bachelors Degree Within 6

Years(1) (2) (1) (2)

Above Threshold -79.266*** -27.862*** -53.927*** -21.870** 0.022*** 0.006(10.297) (8.401) (13.803) (9.839) (0.003) (0.004)

Observations 66,014 113,419 75,164 113,419 270,837 343,874Optimal Bandwidth 10.342 8.138 12.728 8.823 8.805 11.041Control Mean 758.884 687.961 1,121.102 1,408.261 0.125 0.363

Above Threshold -96.108*** 29.813 -81.159*** 55.991 0.035*** -0.021***(17.688) (23.454) (28.964) (37.933) (0.006) (0.008)

Observations 21,035 14,246 22,824 11,437 83,789 47,980Optimal Bandwidth 9.610 12.613 10.825 9.850 15.126 7.315Control Mean 743.306 1,047.199 1,346.414 1,556.302 0.185 0.170

Above Threshold -75.970*** -29.229*** -54.814*** -28.198*** 0.018*** 0.013***(11.824) (8.762) (16.968) (9.527) (0.003) (0.004)

Observations 48,912 92,690 43,190 113,345 217,605 238,408Optimal Bandwidth 12.333 7.806 10.519 9.621 8.867 9.325Control Mean 766.838 647.552 1,006.052 1,391.608 0.109 0.412

Table 4: Impact of Receiving CLEP Credit on Time-to-Degree

Notes: Heteroskedasticity robust standard errors are in parentheses (* p<.10 ** p<.05 *** p<.01). Each estimate comes from a local linear regression with a triangular kernel of the outcomes regressed on a forcing variable representing the distance between the student's first CLEP score and minimum credit-granting threshold, an indicator variable for whether the forcing variable is greater than or equal to 0 (i.e. eligible for credit) and interaction of these two terms.

Panel 1: All Students

Panel 2: Started at two-year College

Panel 3: Started at four-year college

Time to Degree in Days

Earning a credit-granting CLEP score on the first exam influences subsequent CLEP exam-taking

Earning a credit-granting CLEP score on the first exam influences subsequent CLEP exam-taking

Retook First CLEP

Took Subsequent (Different) Exam

Total Subsequent (Different Exams)

Total Credit-Granting CLEP Scores

Forcing VariableAbove Threshold -0.091*** 0.030*** 0.091*** 0.958***

(0.003) (0.004) (0.010) (0.009)

Observations 177,637 286,742 309,906 177,637Optimal Bandwidth 5.043 9.349 10.112 5.958Control Mean 0.092 0.267 0.502 0.316

Table 5: Impact of Receiving CLEP Credit on CLEP Retaking/Additional CLEP Test-Taking

Distance from College-Specific Minimum Credit-Granting Threshold

Heterogeneous Impacts

Results: Impact on Associate’s Degree Completion

32.9%

47.4%

14.5%

30.7%37.4%

5.7 pp

8.6 pp

9.1 pp

5.0 pp

7.3 pp

0%

10%

20%

30%

40%

50%

60%

Overall Military Home-Schooled Hispanic Non-TraditionalAge(>=25)

Asso

ciat

e's

Degr

ee C

ompl

etio

n Ra

te

Impact fromCLEP Credit-Granting ScoreBaseline Rate

Results: Impact on Bachelor’s Degree Completion

45.9%

23.5% 24.9%

44.3%48.5%

1.2 pp

2.6 pp0.0 pp

3.1 pp2.6 pp

0%

10%

20%

30%

40%

50%

60%

Overall Military Home-Schooled Hispanic Non-TraditionalAge(>=25)

Bach

elor

's De

gree

Com

plet

ion

Rate

Impact from CLEPCredit-Granting ScoreBaseline Rate

Results: Impact on Bachelor’s Degree Completion, by CLEP Subject

44.6% 44.5%51.6% 51.4%

3.3 pp 2.3 pp

5.6 pp

14.4 pp

0%

10%

20%

30%

40%

50%

60%

70%

Literature and Arts Overall Science andMath

Biology Financial Accounting

Bach

elor

's De

gree

Com

plet

ion

Rate

Impact from CLEPCredit-Granting Score

Baseline Rate

Is College Enrollment Driving Completion Findings? No!

Hot off the press: Impacts of CLEP Credit on Labor Market Outcomes

$1,235($370)

Hot off the press: Impacts of CLEP Credit on Labor Market Outcomes

No Discontinuity

Conclusion• CLEP has large impacts on AA completion

– Modest impacts on BA completion

• Passes any cost/benefit analysis– Under conservative assumption on returns to degree, 50

fold return on $85 cost– DoD subsidy pays off to servicemen

• Are there unobserved costs?– Early CLEP literature concerned that students miss out on

college course work– Is purpose of college to certify or create human capital?