J U L I A N F E R N A L DU C S A N T A C R U Z

C A I R 2 0 1 1

From Tracking to Analyzing to Action: Using Data to Improve

Retention at UC Santa Cruz

Context

WASC Accreditation Process Redesign Off-site review of retention and graduation rates for all

institutions, including data templates, contextualizing narrative, and some form of benchmarking or comparisons.

UCSC theme in last reaccreditation review Provost’s “5 for 15” Goals for Undergraduate

Education Increase first year retention by 3% Reduce attrition after second year to under 10%

“Tracking”

Annual Report Latest Statistics Trends Breakouts

Race/Ethnicity; Gender; Residency; College; SAT; HS GPA External Benchmarking

UC campuses; Division I; Public 4 years

“Contextualizing and Analyzing”

More refined comparison groups UC campuses, Carnegie Very High Research, AAU non-med

Regression Based Benchmarking: Predicted vs. Actual Based on HERI study of 262 4-years

More refined analysis of sub-group populations Predicted vs. actual by breakout groups

National Student Clearinghouse Predictive Modeling using Logistic Regression

Recent Rates and Trends

2004 cohort: campus high 6-year graduation 74% Six year rates have been slowly but steadily

improving since we began tracking them. Six year graduation rates track 1 and 2 year retention

rates pretty closely. One year retention rates had been improving for over

a decade but have leveled off at 89% since 2004. Without intervention our graduation rates are likely

to level off into the foreseeable future.

External Benchmarking: Select Comparators

2006 2007 2008 2001 2002 2003

UC Santa Cruz 89 88 89 68 72 73

UC Mean 92 92 93 79 80 80Carnegie Very High Research 90 90 91 77 77 78AAU without medical schools 91 91 91 79 79 80

1 Year Retention 6 Year Graduation

Internal Benchmarking

Cohort Entry Year Predicted Actual

1998 66 691999 67 702000 67 702001 66 682002 65 71

1998‐02 66 70

Predicted vs. ActualUCSC  6 Year Gaduation Rates by Cohort Year 1998‐2002 Freshmen Cohorts

Internal Comparisons by Race/Ethnicity

External Comparisons by Race/Ethnicity Gaps

2003 Cohort:  6‐Year Graduation Rates

%  UCSC gap %  UCB gap %  UCD gap %  UCI gap %  UCLA gap %  UCR gap %  UCSD gap %  UCSB gap

Tota l 73 90 80 82 89 65 84 80

Men 45% 74 1 46% 88 ‐2 46% 76 ‐4 51% 80 ‐2 40% 87 ‐2 48% 63 ‐2 45% 84 0 42% 79 ‐1

Women 55% 73 0 54% 92 2 54% 83 3 49% 84 2 60% 90 1 52% 67 2 55% 85 1 58% 81 1

White, non‐Hispanic 51% 74 1 29% 89 ‐1 42% 81 1 24% 79 ‐3 34% 90 1 20% 60 ‐5 36% 84 0 54% 82 2

Black, non‐Hispanic 3% 69 ‐4 4% 73 ‐17 2% 70 ‐10 2% 78 ‐4 3% 81 ‐8 7% 66 1 1% 70 ‐14 2% 75 ‐5

Hispanic 15% 70 ‐3 11% 84 ‐6 8% 67 ‐13 11% 71 ‐11 15% 81 ‐8 22% 60 ‐5 9% 71 ‐13 16% 74 ‐6

Asian/Pacific Islander 19% 75 2 44% 94 4 41% 83 3 51% 85 3 42% 92 3 43% 70 5 43% 88 4 16% 84 4

Amer. Ind./Alaska Native 1% 72 ‐1 0% 69 ‐21 1% 81 1 0% 73 ‐9 0% 88 ‐1 0% 27 ‐38 0% 86 2 1% 80 0

Race/Ethnicity Unknown 11% 72 ‐1 9% 88 ‐2 5% 76 ‐4 10% 84 2 5% 88 ‐1 7% 64 ‐1 10% 86 2 10% 79 ‐1

Nonresident Alien 0% 63 ‐10 3% 90 0 2% 84 4 1% 81 ‐1 1% 86 ‐3 1% 67 2 0% 97 13 1% 63 ‐17

*  Source:  IPEDS ‐ http://nces.ed.gov/ipeds/datacenter/

6‐Year Graduation Rate Gaps by UC Campus 

Internal Benchmarking by Race/Ethnicity

Cohort Entry Year Predicted Actual

American Indian/Alaskan Native 67 68Asian/Pacific Islander 67 71Black, Non‐Hispanic 57 66

Hispanic 54 66White, Non‐Hispanic 69 71

Predicted vs. ActualUCSC  6 Year Gaduation Rates by Race/Ethnicity 1998‐2002 Freshmen Cohorts

Internal Benchmarking by Academic Preparation

Cohort Entry Year Predicted Actual

High Quintile 80 764th 72 733rd 65 702nd 60 70

Low Quintile 55 67

Predicted vs. ActualUCSC  6 Year Gaduation Rates by HS GPA Quintiles 1998‐2002 Freshmen Cohorts

Internal Benchmarking by Academic Preparation

Cohort Entry Year Predicted Actual

High Quintile 74 704th 70 713rd 67 712nd 64 70

Low Quintile 58 68

Predicted vs. ActualUCSC  6 Year Gaduation Rates by SAT Quintiles 1998‐2002 Freshmen Cohorts

So where do they go?

Top Graduating Transfer Institutions of the 2004 Cohort

UC San Diego - 30UC Davis - 28

UCLA - 22UC Berkely - 19

UC Santa Barbara - 14SF State - 10

San Jose State - 9USC - 6

UC Riverside - 6

Was that just an anomaly?

N= 2,350 N= 2,869 N= 2,939 N= 3,168 N= 3,353

Graduated from: n % n % n % n % n %

Santa Cruz 1,649 89.8% 2,006 69.9% 2,011 68.4% 2,250 71.0% 2,444 72.9%

Berkeley 11 0.5% 21 0.7% 25 0.9% 17 0.5% 17 0.5%Davis 19 0.8% 34 1.2% 30 1.0% 21 0.7% 18 0.5%Los Angeles 15 0.6% 25 0.9% 27 0.9% 27 0.9% 15 0.4%Riverside 1 0.0% 6 0.2% 10 0.3% 8 0.3% 4 0.1%San Diego 23 1.0% 17 0.6% 16 0.5% 25 0.8% 14 0.4%Santa Barbara 15 0.6% 15 0.5% 23 0.8% 26 0.8% 13 0.4%Irvine 8 0.3% 14 0.5% 8 0.3% 15 0.5% 8 0.2%missing 0 0.0% 0 0.0% 1 0.0% 0 0.0% 0 0.0%Other UC 92 3.9% 132 4.6% 140 4.8% 139 4.4% 89 2.7%

any UC 1,741 74.1% 2,138 74.5% 2,151 73.2% 2,389 75.4% 2,533 75.5%

UCSC Systemwide Graduation Rates by UC Campus: 1999-2003 Freshmen Cohorts

2001 cohort 2002 cohort1999 cohort 2000 cohort 2003 cohort

Predictive Modeling: Data and Methods

Four Outcomes First to second year retention

2005-2009 freshmen cohorts Second to third year retention

2004-2008 cohorts Third to fourth year retention (or graduation)

2003-2007 cohorts Fourth year to graduation within six years

2001-2004 cohorts

Predictive Modeling: Data and Methods

Predictor Variables Academic Performance

UCSC GPA (ranges: <2.0, 2.0-2.49, 2.5-2.99, 3.0-3.49, >3.5) Academic Preparation

High School GPA; SAT Scores (quintiles) Socio-Demographic Characteristics

Race/Ethnicity; Gender; First Generation College Status Experiences and Attitudes

UCUES 2010 Factors: (satisfaction, sense of belonging, academic engagement and disengagement, self-assessed gains in academic and social competencies, and perceptions of campus climate)

Predictive Modeling: Data and Methods

Logistic Regression to estimate three models at each of the four time points. Model 1:

UCSC GPA only Model 2:

UCSC GPA; Academic Preparation; Socio-Demographics Model 3:

UCSC GPA; Academic Preparation; Socio-Demographics; UCUES factors

Sample weighted by retention, UCSC GPA, gender to control for non-response bias.

First to Second Year Retention

Regression Models Year 1 to Year 2

Model 1 Model 2 Model 31st year GPA <2.0 0.07*** .06*** 0.06***

2.0-2.49 0.70*** .64*** 0.42*2.5-2.99 1.09 1.05 1.133.0-3.49 (ref) - ->=3.5 0.83* 0.86 0.88

Sociodemographic CharacteristicsGender Male 1.31***

Female (ref) -

Race/ethnicity American Indian 0.94Hispanic 1.37***African American 1.54*Asian American 1.45***International 0.74White (ref) - 0.57*

First generation 5.77**Not first generation -

Interaction 1st generation*Belonging 0.63***UCUES factors Belonging 2.27**

Financial concern 0.81*0.145 0.154 0.30017083 15882 833

*p<.05; **p<.01; ***p<.001

R2 (Nagelkerke)N of respondents

refe

renc

e gr

oup

Second to Third Year Retention

Regression Models Year 2 to Year 3

Model 1 Model 2 Model 32nd year GPA <2.0 0.04*** 0.03*** 0.06***

2.0-2.49 0.48*** 0.35*** 0.572.5-2.99 1.24** 1.10 0.953.0-3.49 (ref) - - ->=3.5 0.93 1.02 0.58

Academic PreparationHS GPA (quintiles) 1st (lowest) 0.72*** 0.28*

2nd 0.75** 0.703rd 0.83* 0.554th 0.74** 0.35*5th (ref) - -

Race/EthnicityAmerican Indian 1.64Hispanic 1.34** 0.20**African American 1.16Asian American 1.01 1.17International 0.54 0.07White (ref) - -

Interaction terms (GPA 2.0-2.49)*Hispanic 1.61*(GPA 2.0-2.49)*African American 3.96*(GPA 2.0-2.49)*Asian American 1.90***(GPA 2.5-2.99)*Asian American 1.94***(GPA >=3.5)*Hispanic 0.56**(GPA >=3.5)*Asian American 0.51* 0.23*HSGPA 1st quint*Hispanic 11.53**HSGPA 4th quint*Hispanic 6.11*

UCUES factors Belonging 1.32***Career Orientation 0.74***R2 (Nagelkerke) 0.12 0.135 0.22N of respondents 15,283 13,572 973

*p<.05; **p<.01; ***p<.001

Third to Fourth Year Retention (or Graduation)

Regression Models Year 3 to Year 4

Model 1 Model 2 Model 33rd year GPA <2.0 0.04*** 0.03*** low n

2.0-2.49 0.24*** 0.22*** 0.17**2.5-2.99 0.88 0.77* 0.503.0-3.49 (ref) - - ->=3.5 0.88 1.07 0.29*

Academic PreparationSAT (quintiles) 1st (lowest) 1.92***

2nd 1.69***3rd 1.64***4th 1.30*5th (ref) -

Race/EthnicityAmerican Indian 1.25 -Hispanic 1.07 0.05*African American 1.15 -Asian American 1.81** 0.08*International 1.25White (ref) - -

Interaction terms (GPA >=3.5)*Asian American 0.49*

UCUES factors Gains in Non-quant. skills 1.35*Curricular foundations for reasoning 1.34*Quantitative Skills 0.98

Interaction terms Quantitative skills*Asian American 2.38*Curriculum foundations*Hispanic 3.70*R2 (Nagelkerke) 0.098 0.113 0.415N of respondents 12961 11696 597

*p<.05; **p<.01; ***p<.001

Fourth Year to Six Year Graduation

Regression Models Year 4 to 6 Year Graduation

Model 1 Model 2Final GPA <2.0 0.00*** 0.00***

2.0-2.49 0.06*** 0.05***2.5-2.99 0.38*** 0.34***3.0-3.49 (ref) - ->=3.5 4.39*** 6.29***

Academic PreparationSAT (quintiles) 1st (lowest) 2.87***

2nd 2.15***3rd 1.71***4th 1.38*5th (ref) -

Sociodemographic CharacteristicsRace/Ethnicity American Indian 0.6

Hispanic 0.81African American 0.87Asian American 1.67***International 1.6White (ref) -

First Generation 0.63***Unknown if 1st gen 0.71**Not first generation (ref) -

Interaction terms (GPA >=3.5)*Hispanic 0.29**R2 (Nagelkerke) 0.293 0.314N of respondents 9603 8468

*p<.05; **p<.01; ***p<.001

Implications

Early identification and intervention for low academic performance

Opportunities for challenges for the highest prepared and highest achieving students

Getting students who make it to the upper division “across the finish line”