Upload
aron-bailey
View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Dale R. Tampke – Dean, Undergraduate Studies, University of North [email protected]
Developing and Implementing a Web-Based Early Alert System
Where we’re headed today…
Our Context - UNT Early Alert as a Concept Project Scope (the tech-y part) Building Advocacy Functionality
End-user Responder
Data from 2011-12 (and what we’ve learned so far)
System improvements
University of North Texas - UNT Main campus –
Denton, TX Enrollment
35,754 total headcount
23,756 undergraduates
Moderately selective SAT 1105 ACT 23.4
11 Colleges/Schools Degrees
97 Bachelor’s 101 Master’s 48 Doctoral
Faculty 988 FT 519 PT
Median Class Size - 28
A bit more about UNT
Gender Female (56.0%)
Ethnicity White (62.2%) African American (13.2) Latino (12.8) Asian (5.5) Native American (0.7) Non-resident Alien (4.7)
Over 80% from <100 mi
25% Pell eligible 49% first-generation Students admitted
into colleges and schools
Mandatory two-day summer orientation
FTIC retention rate – 75.6% (2011 cohort)
Six-year graduation rate – 49.4% (2005 cohort)
Please respond to the following:
Describe your institution:A. Public or PrivateB. Two-year or Four-yearC. Small (999 and below), Medium
(1,000 – 4,999), Large (5,000 – 24,999), Mega (25,000 and up)
D. Residential or commuterE. Urban or rural
The Early Alert concept
Grounded in literature on undergraduate retention Student behavior can predict attrition Early intervention can change outcomes
First efforts were course-centered Poor performance Excessive absences
(Think “mid-term” grades)
Early Alert progresses…
Expansion to campus-wide availability Include psycho-social concerns Web front end E-mail back end Authentication varies Integration varies
A common issue: How many faculty use the system?
Our idea:
Integrate with student information system
We could build it ourselves Start with a focus on faculty (make it
easy for them) Designate a central receiver of the data Expand beyond “academic” issues Have a ready referral Begin a personal, caring conversation
Here’s a question:
What stakeholders would you need to include to implement an
Early Alert system on your campus?
Building Advocacy
Include stakeholders Students (8 from office staffs) Faculty (12 from Arts and Sciences) Academic Advisors (10 from all colleges) Student Services (15 areas) IT
Get feedback at the conceptual stage Be ready to adopt a good idea Create a faculty test group
Things to ask (examples)
Issues that affect student performance User access to the system Information a user would need to know
about a student How and whether to inform the student of
the alert Security and permissions Real time or batch processing Reporting (programmed, ad hoc, or both?)
Aspects of the system
Secure – authentication required Campus wide access Easy for faculty to use
Menu-driven Minimal information about the student needed
Ability to inform referred student via e-mail Timely Real-time ad hoc query capability Nightly reporting Completed in six weeks by one programmer
A question…
What student issues would be included in a drop-down menu on
an Early Alert system at your campus?
Reasons for Referral(what’s on the drop down menu)
Poor class attendance Poor performance on
quizzes/exams Poor performance on
writing assignments Does not participate in
class Difficulty completing
assignments Difficulty with reading Difficulty with math Sudden decline in
academic performance Concerns about their major
College adjustment issues
Financial problems Physical health
concerns Mental health concerns Alcohol or substance
use concerns Roommate difficulty Disruptive behavior Absent from work Student needs veterans
assistance Other concerns (text
box)
How Early Alert works
EARS 1.0 (early alert referral system) is available from the on-line class roll
Instructors of record receive an e-mail reminding them of EARS at the beginning of the term
Accessed through the faculty portal (The “Faculty Center”)
Nightly report delivered to a central office (Student Academic Readiness Team – START)
Follow up within one day of receiving
Other features
Relationship to student Professor, instructor Teaching assistant, teaching fellow Academic Advisor Mentor Department administrator Campus Employer Club, organization advisor
“I have had a conversation with the student”
Send a copy of the referral to the student (via e-mail)
Another question…
How would access to alert records be determined on your campus?
Consider academic advisors, student services staff, faculty,
clerical staff, others?
Accessing Early Alert
From the Faculty Center in the Student Information System
To the class roster…
From the class roster…
To the Early Alert form…
After the referral is made…
Review report every morning Real-time e-mail prompt to sender Morning report
Includes following information Demographics Student ID Faculty member’s name Course Reason(s) for referral
Follow-up – Routing alerts
First responders – Routine referrals Residence hall staff Course Achievement Assistants (peers)
More serious issues Academic Readiness Advisors Academic Advisors CARE team Counseling, Health Center
EARS is not designed for urgent situations
More follow-up – The student experience
Caring conversation (no scolding)
Emphasize mattering Resources Self-efficacy Focus on academic success Follow-up2 (we need to get better at this)
Descriptive data from academic year 2010-11
EARS Data from UNT
Alert frequency during the term
A28-S3 S4-10 S11-17 S18-24 S25-O1 O2-8 O9-15 O16-22 O23-29 O30-N5 N6-12 N13-19 N20-26 N27-D3 D4-10 D11-17
14
4953
49
133
59
11
101
30
7
29
4 2 2 1 2
Fall 2011: Alerts by Week (n=546)
Alert frequency during the term
J15-21 J22-28 J29-F4 F5-11 F12-18 F19-25 F26-M3 M4-10 M11-17 M18-24 M25-31 A1-7 A8-14 A15-21 A22-28 A29-M5 M6-12
1711
59
107
31
147
172
82
57
713
2433
13
1 1 1
Spring 2012: Alerts by Week (n=776)
First reasons for alerts
Attendance Issues Academic Issues Behavioral Issues Other Issues
575605
26
116
2011-12: Alerts by Reason
Demographic data
309; 23%
12; 1%
58; 4%
231; 17%
32; 2%11; 1%
669; 51%
Alerts by Ethnicity: 2011-12(n=1322)
Af-Amer Am-Ind
As-Pac Hispanic
Non-Res Other
White
Gender
Female48%
Male52%
Alerts by Gender: 2011-12(n=1322)
Annual Totals
2008-9 2009-10 2010-11 2011-12 2012 (fall only)
0
200
400
600
800
1000
1200
1400
553
882 920
1322
618
Annual Alert Totals(2008-present)
Analysis from Fall 2008 (pilot year)
Outcomes data
Outcomes
Literature suggests early intervention impacts: Student success Student persistence/progression
Fall GPA Spring re-enrollment Use a within-group comparison No useful “control” group
Findings
Success and Persistence
Fall GPA – 1.39 Cumulative GPA –
1.94 Persistence –
70.2%
Course Grade Distribution
A’s – 3.4% B’s – 5.9% C’s – 11.9% D’s – 12.3% F’s – 43.0% I’s – 1.3% Drops – 21.7%
Contact types (frequencies)
Faculty E-mail notice only – 42.0% Personal – 8.2% Both – 3.5% None – 46.3%
Academic Readiness E-mail notice only – 65.9% Personal (phone, response from student, meeting) – 34.1%
Outcomes by contact type
Fall GPAPersistence (% re-enrolling)
Faculty
E-mail only 1.19 62.6
Personal 2.17 85.7
Both 2.07 77.8
None 1.39 73.7
START
E-mail only 1.26 67.9
Personal 1.64 74.7
Some statistics
Personal Contact Mean Term GPA Significance
Faculty
Yes (n=25) 2.15
No (n=213) 1.30 F=11.894, p<.001
START
Yes (n=60) 1.63
No (n=158) 1.26 F= 5.436, p<.021
Outcomes by Contact Type by Reason
(Attendance)
Attendance (n=144)
Fall GPAPersistence (% re-enrolling)
Faculty
E-mail only 0.83 53.1
Personal 1.96 100.0
Both 1.77 80.0
None 1.34 71.2
START
E-mail only 1.06 62.3
Personal 1.48 73.7
Outcomes by Contact Type by Reason
(Performance)
Performance (n=74)
Fall GPA
Persistence (% re-enrolling)
Faculty
E-mail only 1.90 83.3
Personal 1.88 100.0
Both 2.48 100.0
None 1.52 80.0
START
E-mail only 1.58 82.4
Personal 1.88 85.0
EARS 2.0
System Improvements
Making the system better – EARS 2.0
Available to all staff via web portal Immediate e-mail communication
To referrers To service providers To students
Real-time referral based on alert type Improved outcome tracking using
workflow Batch uploads (at-risk students)
From the staff portal…
New responder screen…
Responder notes…
Responders can add an infinite number of “Alert Notes” to track conversations / referrals they have made for each student. Each note will be time / date stamped and include Advisors’ EUID and name.
Assessment data…
Advisor / Responder contacts student
Advisor / Responder creates notes / adds additional notes.
Advisor / Responder “completes” Alert only if student completes prescribed intervention.
COTS Early Alert Offerings
SunGard Course Signals (Purdue) - http://www.sungardhe.com/signals/
Hobson’s Early Alert system - http://www.hobsons.com/products/earlyAlert.php
Starfish Early Alert - http://www.starfishsolutions.com/sf/solutions/earlyalert.html
Datatel Retention Alert - http://www.datatel.com/products/products_a-z/student-retention-software.cfm
EducationDynamics Early Alert - http://www.educationdynamics.com/Retain-Students/Early-Alert-Systems.aspx
EBI MAPWorks - http://www.map-works.com/
Sinclair Community College -http://www.sinclair.edu/support/success/ea/
What we’ve learned
1. Including faculty in the design was critical2. Linking to class roll, self-populating made it easier for
faculty to use3. Faculty generally focus on course-related issues4. Personal faculty contact is the most effective follow-up5. E-mail contact by itself is not effective6. Some positive effect on success and persistence based
on type of contact7. Timing of alert has no apparent effect on success or
persistence8. Tracking confirmed contacts needs improvement9. EARS is not a “large class” solution
Resources
Bowen, E., Price, T., Lloyd, S., & Thomas, S. (2005). Improving the quantity and quality of attendance data to enhance student retention. Journal of Further and Higher Education, Vol. 29 (4), 375-385.
Eimers, M. (2000). Assessing the impact of the early alert program. AIR 2000 Annual Forum Paper. (ERIC Document Reproduction Service No. ED446511) Retrieved February 28, 2009, from ERIC database.
Fischman, J. (2007, October 29). Purdue uses data to identify and help struggling students. Chronicle of Higher Education Online, Retrieved May 15, 2009 from http://chronicle.com/daily/2007/10/530n.htm.
Geltner, P., & Santa Monica Coll., CA. (2001). The characteristics of early alert students, Fall 2000. (ERIC Document Reproduction Service No. ED463013) Retrieved February 28, 2009, from ERIC database.
Hudson, W. (2006). Can an early alert excessive absenteeism warning system be Effective in retaining freshman students? Journal of College Student Retention, Vol. 7(3-4), 217- 226.
More references
Kelly, J. & Anandam, K. (1979). Computer enhanced academic alert and advisement system. (ERIC Document Reproduction Service No. ED216722) Retrieved February 23, 2009, from ERIC database.
Richie, S. & Hargrove, D. (2005). An analysis of the effectiveness of telephone intervention in reducing absences and improving grades of college freshmen. Journal of College Student Retention, Vol. 6(4), 395-412.
Tampke, D. (2013). “Developing, implementing, and assessing an early alert system,” Journal of College Student Retention, 15 (1), in press.
The Hanover Research Council. (May 2008). Intrusive advising and large class intervention strategies: A review of practices. Washington, DC: Author.
Wasley, P. (2007, February 9). A secret support network. Chronicle of Higher Education, 53(23), A27.
Thank you for your participation!
Dale R. Tampke
Dean, Undergraduate Studies
University of North Texas