2012 Innovations Conference March 4, 2012

Designing an Effective Intervention System for At-risk Students:

A proactive, dynamic, and data-informed approach

2012 Innovations ConferenceMarch 4, 2012

Cheoleon Lee & Pamela Wallentiny

Who is in our audience?

1 2 3 4 5 6

17% 17% 17%17%17%17%1. Administrators2. Faculty3. Advisor4. Student services

staff5. Counselor6. Other

Does your campus have an early warning system in place?

1 2 3

33% 33%33%1. Yes2. No3. In progress

How do you identify at-risk students?

1 2 3 4

25% 25%25%25%1. Early alert system2. Academic policy3. All of the above4. None

Topics Background Review of Academic Standing Policy Connecting Campus Resources Analysis of At-Risk Group Data

Who are At Risk?Predictors of At-Risk Group

Outcomes AssessmentAdvising makes difference? Tutoring makes difference?

Summary and Discussion

Background• Need new academic standing policy (2007) Implemented (2009)• Research and best practices

Theories of Retention & Student Success Astin (1985): I-E-O Model & Theory of Involvement Pascarella (1985): General Model for Assessing Change Bean (1990): Psychological Model of Retention Tinto (1993): Theory of Student Departure

Retention is a function of how the student and the institution interact with one another.Identification : Early Alert Intervention : Student advising, counseling, & other student services

Seidman’s (1996) retention formula*R = E(Id) + (E + I + C)(Iv)

“Retention equals early identification & early, intensive, and continuous intervention.”

*Seidman, A. (1996, Spring). Retention Revisited: RET = E Id + (E + I + C)Iv. College and University, 71(4), 18-20.

FA09 Before After

Based on Term GPA < 2.0 Cum. GPA & Cum. Attempted Credits

Levels GoodProbation

Suspension

GoodWarning Probation

Suspension

Point of Intervention Suspension Warning

Academic Standing Policy Makeover

Satisfactory Academic Progress(Good Standing) under new policy

Cum GPA of at least 2.0 If a student’s Cum GPA is below 2.0, the standing is

determined by both attempted credits and Cum GPA:

Attempted Credits Minimum CUM GPA12 – 15 1.0016 – 30 1.5031 – 40 1.75

41 or greater 2.00

Students receive letter, followed up by phone call

Advising Process

Students receive letter Students receive letter

Warning

Students completes Self assessment

Required to meet with anAcademic Success

Coordinator

Recommendations made

Hold flag on record untilmeeting with Academic

Success Coordinator

Probation

If enrolled in more than 8 credits, then call placed by

Academic Advisor

Meet with Academic Advisor

Recommendations made

All classes dropped ifRegistered for > 8 credits

Suspension

First suspension, opportunity to file a written appeal

One semester suspension Or, appeal approved for

probation

Second suspension, noopportunity to file awritten appeal andrequired to serve

a one year suspension

Connecting Campus Resources• Academic & Transfer Advising• Counseling & Career Services• Disability Support Services• Financial Aid Services• Tutoring: Learning Assistance Center (LAC)• Step-Up• Peer Mentor Program• Silas Craft Collegians Program• Children’s Learning Center• Use of Technology

Designing an Effective Early Intervention System for At-risk Students

Study Historical Data

Identify Patterns & Predictors of At-risk

Perform Simulation with different Intervention Points

Pilot Study

Outcomes Assessment

Implement Intervention Programs:(Academic Standing Policy, Early Alert Program, Advising, Tutoring, Counseling)

At-Risk Group Trend Over TimeProportion of Warning formula applied cases

(First-time Full-time)

FA06 (N=189)

SP07 (N=20)

FA07 (N=217)

SP08 (N=24)

FA08 (N=201)

SP09 (N=27)

FA09 (N=207)

SP10 (N=12)

FA10 (N=235)

0%

5%

10%

15%

20%

25%

18% 18%21% 21%

18% 17% 17%

10%

19%

“Fresh Load” of At-Risk Cases

Incl. (PT) 191 20 219 24 204 28 215 12 238

At-Risk Group Trend Over TimeProportion of Warning formula applied cases

(All Students)

FA06 (N=828)

SP07 (N=690)

FA07 (N=899)

SP08 (N=776)

FA08 (N=899)

SP09 (N=889)

FA09 (N=1026)

SP10 (N=1005)

FA10 (N=1188)

0%

10%

20%

12%10%

12%11% 11%

12% 12% 12% 12%

“Advising Work load/term : new + old cases (Base figure)”

FA09 Academic Standing

(N=8,778)

PROB

SUSP

*Unknown

DSTN

DEAN

WARN

GOODP

GOOD

0 10 20 30 40 50 60

.9

2.9

3.9

5.1

5.5

8.6

19.5

53.5

(All Students FALL 2009)

* Unknown: Mostly ‘N’ (audit) grades with missing or zero Term GPA

%

At -

Risk

Retention (% came back in SP2010)

by FA2009 Standing

SUSP (42/257)

WARN (412/759)

GOODP

GOOD

DSTN

PROB (61/76)

DEAN

Total

0% 10% 20% 30% 40% 50% 60% 70% 80% 90%

16%

54%

56%

75%

79%

80%

85%

69%

% Came back in SP2010

(N=6,022 : Enrolled in Both FA09 & SP10)

At-R

isk

Comparison of Formula

FA09 Standing(new policy)

FA09 Term GPA Below 2.0

FA09 Cum GPA Below 2.0

Warning Formula Applied Cases***

Unknown 49 166 12DEAN* 1DSTN** 9 5GOOD 1164 529 16

GOODP 511 487 6PROB 75 73SUSP 256 255 242WARN 688 751 672Total 2668 2273 1062

* DEAN: This student had 3.7 Term GPA, 1.85 Cum GPA, and 36 Cum Attempted Credits **DSTN status is based on Term GPA of 3.5 or higher (part-time)*** Strictly based on the formula. Slightly different from the final standing.

Good

Sta

ndin

gAt

Ris

k

Scenario 1 Scenario 2 Scenario 3

Who are at-risk?What are the predictors?

Comparison of “At-Risk” vs. “Good Standing” 8 Terms: FA06~SP10 (First-Time Full-Time)

First-Time Full-TimeFA06 thru SP10 (8 terms)

Formula based At-Risk (n=897)

Good Standing (n=4000)

% Male 60% 50%

% African American 32% 20%

Top 5 majors (no difference)

General Studies, Accounting/Bus. Adm., Undeclared, Nursing, Engineering

No Post Secondary degree 82% 80%

Academic Goal Associate deg/Transfer (43%/46%)

Associate deg/Transfer (43%/47%)

Personal Goal XFER to 4yr (68%) XFER to 4yr (65%)

Math Placement 78% Dev Ed 60% Dev Ed

Writing Placement 37% Dev Ed 31% Dev Ed

Reading Placement 33% Dev Ed 30% Dev Ed

Financial Aid Rcvd 25% 26%

10

12

18

6

Diff.

Predictors of being At-Risk Group among First-Time Full-time Students

(Multivariate Analysis)

Asian

Black

Hispanic

Other Race

Male

Math Dev Ed

Writing Dev Ed

Reading Dev Ed

Fin. Aid Rcvd

Age

FTFTAt Risk

(Warning)Whi

te: r

efer

ence

cat

egor

y N=4,791 FTFT students 8 terms (FA06-SP10)1.6***

1.8***

1.5*

2.2***

.97***

*** : sig at .001** : sig at .01* : sig at .05

Odds Ratio

Demographics WARN (n=759)

PROB (n=76)

SUSP (n=257)

Gender %Male (43%) 55% 66% 60%

Race

Asian (12%) 11% 22% 15%African Am. (25%) 34% 24% 27%Native Am. (0.6%) 1% 1% .3%Hispanic (5%) 7% 5% 6%White (51%) 40% 43% 44%Other (7%) 7% 4% 7%

Academic Load Full-Time (39%) 62% 24% 42%

Dev Ed Placement

Math 67% 67% 72%Writing 35% 38% 43%

Reading 30% 33% 35%

Characteristics of FA09 “At-Risk” Group(n=1,092)

-Cont.

Registration Status

Attending Another College/Institution

.3%

First-Time College (21%) 31%

Previous Postsecondary 6%

Re-admit 4% 7% 4%

Returning (64%) 59% 93% 96%

Advising

Self Assessment 42% 4%

Met Academic SupportFaculty

24%

Seen by Advising 21%

Met Advising Staff/Faculty 43%

Retention (Enrolled in SP2010)54% 80% 16%

Demographics WARN (n=759)

PROB (n=76)SUSP (n=257)

Characteristics of FA09 “At-Risk” Group -Cont. (n=1,092)

Analysis of Self-Assessment Data(318 of 759 Warning Cases)

Listening to Students’ Voices

Self Assessmenthttp://www.howardcc.edu/students/academic_support_services/retention_services/academic_warning_form.html

Analysis of FA09 Self-Assessment DataReasons for being unsuccessful

Communication/language Issues

Don't like school

Class too difficult

Transportation problems

Changed major one or more times

Helath problems

Other Explain further 2

Financial problems

Late for class three or more times

Uncertain about major/not sure about career goals


High School study habits didnt work

Wait until last minute to do schoolwork

Absent from class three or more times

Pressure, stress, anxiety

0% 10% 20% 30% 40% 50% 60%

2%

8%

10%

12%

12%

14%

14%

15%

18%

21%

23%

30%

43%44%

48%

(N=318)


Dont like school


Class too difficult


Helath problems

Financial problems









0% 10% 20% 30% 40% 50% 60%

MaleFemale

Analysis of FA09 Self-Assessment DataReasons for being unsuccessful by Gender

(N=318)


Dont like school


Class too difficult


Helath problems

Financial problems









0% 10% 20% 30% 40% 50% 60% 70%

WhiteNon-White

Analysis of FA09 Self-Assessment DataReasons for being unsuccessful by Race

(N=313)

Outcomes Assessment

Does advising make difference?

Does tutoring make difference?

Does advising make difference? Retention Cum. GPA Change

Retention Analysis (Warning cases)Retention Self-Assessment

Yes No

FA09-SP10 100% 32%

FA09-FA10 63% 16%

FA10-SP11 87% 48%

Retention Met with Advisor

Yes No

FA09-SP10 100% 19%

FA09-FA10 61% 8%

FA10-SP11 86% 35%

Advi

sing

Mean Cum GPA Change(Warning cases)Mean Cum. GPA Change

Self-Assessment Sig

Yes No

FA09-SP10 +0.29 +0.17 sig

FA09-FA10 +0.52 +0.47 ns

FA10-SP11 +0.20 +0.17 ns

Mean Cum. GPA Change

Met with Advisor Sig

Yes No

FA09-SP10 +0.28 +0.09 Sig

FA09-FA10 +0.53 +0.32 ns

FA10-SP11 +0.21 +0.05 sig

Advi

sing

Advising Makes Difference!

• Higher Retention Rates• Bigger Cum. GPA improvements

Does tutoring make difference?

Retention Cum. GPA Change

Retention AnalysisRetention Overall (All 9,568 Students)

Tutoring (n=1,127)

No Tutoring(n=8,441)

FA10-to-SP11 78% 69%

FA10-to-FA11

Retention Warning Cases Only (n=774)

Tutoring (n=88)

No Tutoring(n=686)

FA10-to-SP11 72% 61%

FA10-to-FA11

Tuto

ring

Mean Cum GPA Change


Tutoring (Warning Cases only) Sig

Yes (n=61)

No (n=414)

FA10-to-SP11 +0.05 +0.03 ns

FA10-to-FA11


Tutoring (All Students enrolled in both FA10 & SP11)

Sig

Yes(n=868)

No (n=5,776)

FA10-to-SP11 +.013 +.007 ns

FA10-to-FA11

Tuto

ring

Tutoring doesn’t seem to be making difference yet.

It’s Your Turn…

Build your own intervention system for at-risk students.

1) Carefully remove all the pieces from your packet.

2) Design your system by placing the pieces on the template.

3) Draw arrows to connect the pieces to show the flow of the system.

Early Intervention System for “At-risk” Students

Early Alert

Academic Warning

Counseling

Advising

Tutoring

Academic Standing

“Retention equals early identification & early, intensive and continuous intervention.”

Asse

ssm

ent

Assessment

Asse

ssm

ent

Discussion & Suggestions

• Questions?• Observations?• Suggestions/Recommendations?• Ideas to share?• Policy Implications?

Contact Info.• Cheoleon Lee, Ph.D.

Associate Director of Institutional Research, PRODHoward Community College443-518-4289, [email protected]

• Pamela Wallentiny, M.Ed.Retention SpecialistHoward Community College443-518-4144, [email protected]

mailto:[email protected]



Self Assessment Survey

Documents

2012 Innovations Conference March 4, 2012