Back to the Basics:Back to the Basics:

Saul GeiserUniversity of California, Berkeley

Saul GeiserUniversity of California, Berkeley

In Defense of Achievement (and Achievement Tests)

in College Admissions

In Defense of Achievement (and Achievement Tests)

in College Admissions

““Readiness” for college: Two viewsReadiness” for college: Two views

• Achievement: performance in college-preparatory subjects, mastery of curriculum content

• Ability: generalized reasoning and problem-solving skills

UC findingsUC findings

Admissions criteria that tap student mastery of curriculum content, such as high-school grades and achievement tests, are stronger predictors of success in college and fairer to low-income and minority applicants than tests of general reasoning such as the SAT.

Underrepresented minorities as a percent Underrepresented minorities as a percent of new UC freshmen, 1995 to 1998of new UC freshmen, 1995 to 1998

Correlation of SAT I, SAT II and HSGPACorrelation of SAT I, SAT II and HSGPAwith socioeconomic factorswith socioeconomic factors

Family Income

Parent’s Education

High SchoolRank

SAT I Reasoning .32 .40 .40

SAT II Subject Tests .23 .29 .33

High School GPA .04 .06 .01

% of underrepresented minority applicants % of underrepresented minority applicants by SAT I vs. HSGPA decilesby SAT I vs. HSGPA deciles

SAT I HSGPA

Top decile 4% 9%

9th 6% 11%

8th 7% 13%

7th 9% 14%

6th 12% 16%

5th 15% 17%

4th 18% 19%

3rd 22% 20%

2nd 29% 23%

Bottom decile 45% 28%

HSGPA

SAT I SAT IISchoolRank SES

BioScience .35 .10 .23 .05 .03

PhysSci/Math .33 -.02 .25 .04 .03

SocSci/Hum .31 .08 .17 .04 .07

Other .32 .05 .21 .05 .07

ALL FIELDS .31 .03 .21 .06 .06

Relative weight of different admissions Relative weight of different admissions factors in predicting freshman GPAfactors in predicting freshman GPA

Sample: New UC freshmen in 1996-1999 who completed first year. N = 68,602.

HSGPA

SAT I SAT IISchoolRank SES

BioScience .22 .06 .14 .04 .00

PhysSci/Math .25 -.01 .22 .07 .03

SocSci/Hum .17 -.03 .07 .07 .09

Other .18 -.10 .10 .09 .09

ALL FIELDS .20 -.03 .12 .08 .07

Relative weight of different admissions Relative weight of different admissions factors in predicting 4-year graduationfactors in predicting 4-year graduation

Sample: New UC freshman in 1996-1999. N = 74,618.

Minority participation in AP/honors classes Minority participation in AP/honors classes among California college-bound seniorsamong California college-bound seniors

# AP/HonorsSubjects Taken

UnderrepresentedMinority %

None 32.3%

1 - 4 30.5%

5 - 8 23.7%

9 or more 17.3%

HSGPA SAT I

SAT IISchoolRank SES

# AP Classes

AP Exams

BioScience .21 .07 .14 .01 .05 .01 .19

PhysSci/Math .29 .03 .08 -.02 .05 .06 .22

SocSci/Hum .26 .08 . 09 .06 .08 .02 .11

Other .23 .07 .06 .04 .09 .01 .15

ALL FIELDS .24 .03 .08 .03 .08 .02 .15

Relative weight of different admissions Relative weight of different admissions factors in predicting cumulative UC GPAfactors in predicting cumulative UC GPA

Sample: New UC freshmen in 2002 who completed two years. N = 14,976.

Achievement tests and UC outreachAchievement tests and UC outreach

• Alignment with K-12 curriculum

• Diagnostic function

• Message to students

UC policy changes after Prop 209UC policy changes after Prop 209

• 1998: Increased weight for HSGPA and SAT II, reduced weight for SAT I scores in statewide eligibility index

• 2001: Introduced Top 4% Plan and Comprehensive Admissions Review

• 2002: Adopted policy favoring achievement tests over tests of general reasoning or aptitude

Underrepresented minorities as a percent Underrepresented minorities as a percent of new UC freshmen, 1995 to 2007of new UC freshmen, 1995 to 2007

Restoring the role of high-school recordRestoring the role of high-school record

• Increase emphasis on high-school grades and class rank over admissions tests

• Eliminate “bonus point” for Advanced Placement classes except where students take and pass AP exams

Limitations of New SAT as Limitations of New SAT as an achievement testan achievement test

• Less curriculum-based than other achievement tests

• Remains norm-referenced

• Limited diagnostic value

• Prediction is no better than old SAT

An expanded role for subject testsAn expanded role for subject tests

• Most content-intensive of all nationally available assessments

• Align with and reinforce classroom instruction

• Best predictors, after high-school record, of student performance in college

Back to the basicsBack to the basics

• Criterion-referenced vs. norm-referenced assessment

• Accent on achievement and mastery of foundational subjects

Additional slides for Q & Additional slides for Q & AA

Rothstein, J. (2004). “College performance predictions and the SAT." Journal of Econometrics, volume 121, 297-317.

“The results here indicate that the exclusion of student background characteristics from prediction models inflates the SAT’s apparent validity, as the SAT score appears to be a more effective measure of the demographic characteristics that predict [college grade-point average] than it is of preparedness conditional on student background. … [A] conservative estimate is that traditional methods and sparse models [i.e., those that do not take into account demographic characteristics] overstate the SAT’s importance to predictive accuracy by 150 percent.”

1996 1997 1998 1999 1996-1999

Predictor Variables/Equations:

(1) HSGPA 17.0% 16.7% 14.7% 12.9% 15.4%(2) SAT I 13.8% 10.8% 12.2% 14.2% 13.3%(3) SAT II 16.4% 14.4% 15.6% 16.4% 16.0%(4) SAT I + SAT II 16.7% 14.4% 15.6% 16.8% 16.2%(5) HSGPA + SAT I 21.9% 20.1% 19.2% 20.4% 20.8%(6) HSGPA + SAT II 23.0% 21.7% 21.1% 21.5% 22.2%(7) HSGPA + SAT I + SAT II 23.2% 21.7%* 21.1%* 21.9% 22.3%

SAT I increment: [(7)-(6)] 0.2% 0.0% 0.0% 0.4% 0.1%

* SAT I not statistically signif icant in prediction equation; all other variables are statistically signif icant at <.01 level.

Percent of Variance in UC Freshman GPA Explained by HSGPA, SAT I and SAT II Scores

HSGPASAT IM + V

SAT IIWriting

SAT IIOther

APExams

SchoolRank SES

Asian .28 -.03 .13 .05 .23 .03 .00

Black .20 -.09 .05 .06 .21 .13 .06

Latino .28 .05 .13 .05 .07 .08 .05

White .32 -.04 .14 .05 .17 .05 .06

Relative weight of different admissions factors Relative weight of different admissions factors in predicting freshman GPA by race/ethnicityin predicting freshman GPA by race/ethnicity

Sample: New UC freshmen in 1998-2001 who completed first year. N = 57,377.

HSGPA

SAT I SAT IISchoolRank SES

Berkeley .31 -.05 .24 .03 .09

Davis .35 .06 .24 .09 .03

Irvine .28 .08 .15 .03 .04

Los Angeles .32 .00 .20 .07 .08

Riverside .34 .11 .11 -.02 .02

San Diego .33 .06 .19 .06 .07

Santa Barbara .39 .06 .18 .05 .07

Prediction weights for 4Prediction weights for 4thth year GPA by campus year GPA by campus

Sample: New 1996-1999 UC freshmen completing 4 years. N = 58,539.

Mean SD Mean SD Mean SD

SAT I V 585.4 100.1 591.5 92.2 1.0% -7.9%

SAT I M 617.9 95.8 629.1 88.1 1.8% -8.0%

SAT II W 567.5 104.1 574.5 97.3 1.2% -6.5%

SAT II M 599.0 101.4 612.0 94.0 2.2% -7.3%

SAT II 3rd 606.6 111.8 616.7 105.4 1.7% -5.7%

SAT I * 1203.4 173.6 1220.6 156.9 1.4% -9.6%

SAT II WM ** 1166.5 180.8 1186.5 165.8 1.7% -8.3%

SAT II *** 1773.0 261.8 1803.2 239.4 1.7% -8.6%

HSGPA 3.75 0.49 3.86 0.41 2.9% -16.3%

* Composite of SAT I V + SAT I M** Composite of SAT II W + SAT II M*** Composite of SAT II W + SAT II M + SAT II 3rd

N = 144,082 N = 58,064

Freshmen - Applicants

Means and Standard Deviations of SAT Scoresfor UC Applicants and Enrolled Freshmen

Fall 1997 through Fall 1999

Applicants Enrolled Freshmen% Difference,

Racial/Ethnic Composition of CA High SchoolsBased on School Average SAT I Scores

21%

46%

62%

74%

83%

79%

54%

38%

26%

17%

0% 25% 50% 75% 100%

Bottom Quintile

Quintile 2

Quintle 3

Quintile 4

Top Quintile

White, Asian and Other Black, Latino and American Indian

School HSGPA SAT I SAT IIAPI Quintile

5 (high) .33 -.01* .204 .32 .01* .203 .29 .03* .252 .28 .07 .221 (low ) .25 .12 .18

All Schools .27 .07 .23

* Not statistically signif icant at <.01 level.

Standardized Regression Coefficients for HSGPA, SAT I and SAT II Scoresby School API Quintile, 1996-1999

Regression equation: UCGPA = HSGPA + SAT I + SAT II

HSGPA SAT I

SAT IISchoolRank SES

# AP Classes

AP Exams

1998 Cohort .28 -.04 .10 .04 .07 .04 .24

1999 Cohort .28 -.01 .07 .08 .08 .03 .22

2000 Cohort .30 .03 . 05 .05 .08 .03 .21

ALL YEARS .29 .00 .07 .05 .08 .03 .22

Relative weight of different admissions Relative weight of different admissions factors in predicting freshman GPAfactors in predicting freshman GPA

Sample: New UC freshmen who completed first year. N = 41,116.

HSGPA SAT I SAT IISchoolRank SES

BioScience .34 .11 .23 .05 .02

Phys/Math .36 .02 .23 .05 .02

SocSci/Hum .33 .07 .20 .04 .08

Other .34 -.03 .23 .07 .07

ALL FIELDS .33 .04 .20 .05 .06

Relative weight of different admissions factors Relative weight of different admissions factors in predicting cumulative 4-year UC GPAin predicting cumulative 4-year UC GPA

Sample: New 1996-1999 UC freshmen completing 4 years. N = 58,539.