Embed Size (px)
Citation preview
Backtracking Alumni: Using Institutional Research to Improve
Student Outcomes
Dr. Faye R. Jones
Dr. Marcia A. Mardis
Florida State University
Supported in part by NSF 1700581
(2017-2020)2019
Today’s Agenda
1. Alumni Tracking
FITC Example
Promising Practices
1. Alumni Backtracking
Goal-Path Theory
Scenario: Lisa’s Journey
Demo (Illustration Video)
Implications and Limitations
1. Next Steps and Further Research2. Q&A
Alumni Tracking
Alumni TrackingFoundational Work
• Florida IT Career (FITC) Alliance (funded by Florida Board of Governors; $1.9M, 2 years)
• Focus on BS in IT, CS, CE in North Florida State Colleges and Universities
• 2013 – 2018 (5 years of reporting)
• Increase enrollment, retention, and completion rates
• Monitor and improve completer outcomes within a year of graduation:
• In-field employment
• Salary
• Extent to which students remain in FL
Alumni Tracking (FITC) Questions
• What percentage of alumni in CS, CE, and IT are employed in-field within a year of obtaining their undergraduate degree?
• What are the salaries of alumni in these majors after they have graduated from these programs?
• Where (geographically) are students employed after graduation? To what extent are these students employed in state (i.e., Florida) after graduation?
• Define a successful completer.
• What outcomes should a successful graduate from your program achieve (at a minimum) as it relates to employment?
• The vision…
• guides survey questionnaire creation
• defines “successful employment”
• prioritizes contexts of institution and state
• Where would you expect most of your graduates to go (geographically)?
• Are they more likely to go public or private sector?
• What have you exposed them to in order to justify these conclusions?
• Can you link your curriculum to these outcomes?
• What other sorts of services and activities do you offer to support these outcomes?
Alumni Tracking (FITC) Strategic Planning
Alumni Tracking (FITC)Internal Institutional Considerations
• Obtain IRB approval
• Identify institutional policies that govern student/alumni surveys (Policies on alumni outreach).
• Minimize student response fatigue
• Who else sends alumni surveys? When? How often?
• Leverage appropriate institutional data sources for most current student contact information
• Watch for nicknames, maiden names, common names, etc.
• Graduation list is updated daily. Keep original list.
• Take note of double majors – will you double count them?
• Consider variables to be collected longitudinally.
• Define a collection window
• Keep Data Analysis in mind. Determine supplemental data to be used (embed before, merge after, both).
• Ensure you have an IR specialist on team
Alumni Tracking (FITC) Data Collection Pre-planning
Alumni Tracking (FITC) Data Collection Embed/Merge Examples
• Embed the source
• Embed Supplemental Data• Student ID
• Merge• Demographic info
• Performance Indicators
Alumni Tracking (FITC) Alumni Survey QuestionsEmployment
• Please check the box(es) that describe(s) your current status.
❑I have a full-time job.
❑I have a part-time job.
❑I am not working because I choose to be unemployed.
❑I am not working, but not by choice.
❑I am not working (no reason provided).
• My job is located in (City) , (State) . (Note: We added international location option).
• My job is in a field related to my bachelor’s degree. Please check ◻ Yes or ◻ No
• My total annual salary is:
❑$16,000 or less
❑$16,001 to $32,000
❑$32,001 to $48,000
❑$48,001 or more
Survey questions provided by FL Board of Governors
• Source 1: Surveys -Individual emails• Primary and secondary emails from multiple sources• Challenges
• Survey fatigue
• Survey timing (multiple attempts by others to contact student)
• Those without a job are least likely to respond
• Source 2: Phone Calls • Challenges:
• Disconnected numbers
• Uncomfortable calls (e.g., disgruntled students, esp. if unemployed).
• Less likely to provide salary and person information over the phone.
• More likely to get partial responses.
• Source 3: Social Media• Requires fast response from alumni• Script• Facebook
• limit before blocking
• Requires ongoing connections with student beyond graduation
• FITC Population• Cohort 1 (CH1) (2013-14) = 258 alumni in CS, CE, IT; Cohort 2 (CH2)
(2014-15) = 312; Total CH1 & CH2 = 570
Collection
Method
Response
Rate (%)
Unreachable
(%)
Survey
Invitation
(Primary
emails)
114 (20%) 114 (20%)
Bounced
Survey
Invitation
(secondary
emails)
29 (5%) 103 (18%)
Phone Calls 29 (5%) 97 (17%)
Social
Media
153 (27%) 46 (8%)
Total
Response
325 (58%)
Alumni Tracking (FITC) Data CollectionOverlapping Sources
Qualtrics Insight Platform and Social Media
Q1 – What percentage of CS, CE, and IT BS completers are employed after a year?
• 325 responded to the survey
• 308 (or 94%) reported employment information.
• Of the 308 employed alumni
• 292 or 95% were employed full-time
• 16 or 5% were employed part-time.
• 87% were working in a field related to their major
• 2% were self-employed.
• 11% of responders indicated that they were enrolled in college.
Alumni Tracking Results (FITC)
Q2 – What are alumni salaries?
35% (n=107) $48,001 - $64,000
29% (n=89) $32,001 - $48,000
14% (n=43) $64,001- $80,000
10% (n=31) $16,001 - $32,000
4% (n=12) $16,000 or less.
The two highest salary categories, $80,001-$100,000
and $100,001 or more each had 4% of respondents
(n=12 each).
Figure shows, graduates of the three programs tend to
secure jobs with salaries in excess of $32,000 per year.
Note: 88% went into the private sector.\Responders (N=306)
Alumni Tracking Results (FITC)
Alumni Tracking Results (FITC)
Q3 – Where are our students employed?
• 80% (n=230) of graduates are
employed on the east coast
• 19% (n=55) employed in mid
or western U.S.
• 1% (n=2) employed overseas
Maps created with Google Maps
Alumni Tracking Results (FITC)
Q3 – In-state employment location by major
Figure shows 55% (or 158 out of 287) all FSU CE,
IT, and CS alumni were employed in Florida.
• highest concentrations (n=132) employed in
central and south Florida.
• Of those living in North Florida (n=26), 95%
(n=24) of them resided in a major city (e.g.,
Jacksonville, Pensacola, or Tallahassee), or in a
tourist city such as Panama City.
• remaining 2 graduates live in small towns or rural
locations.
Q3b. Are they in Florida?
Backtracking Alumni Pathways
= Influencer
=Hindrance
=Critical Juncture
Deeper into Path-Goal Theory
Formal Learning
(coursework)
Informal Learning
(Clubs, Mentors)
Experiential
Learning
(Internships, Jobs)
Goal 1
Student
Outcome
Graduation
Goal 2
Completer
Outcome
Higher Ed/
Employment
…but in REVERSE!
Backtracking Alumni
Supplement
quantitative
data with in-
depth
participant
experiences
gained
through
interviews.
Interview to
saturation.
Individual Pathway Scenario: Sample Interview Data
Completion: Graduating with a 3.2 GPA and an IT AS, Lisa interviewed at Municode and because her
communication skills were so strong, became a Network Design scrum leader.
Semester 8 (9 credits): Critical Juncture: Lisa failed her certification exam and is discouraged. Hindrance:
The college’s career counselors didn’t offer her any alternatives. Desperate, Lisa reaches out to former
Technical Writing teacher. He encourages her to sit for the exam again, but advises Lisa to take a
Advanced Networking course first (Influencer), along with Hardware Fundamentals and Storytelling, for
relaxation and fun. He also called his contact at Municode (Influencer). His contact was eager to keep
Lisa at Municode because she was a great intern.
Semester 7 (9 credits): Influencer: Lisa is a paid intern at Municode, while completing her final courses,
including Network Topology, the course which was supposed to prepare her for the Cisco Certified
Design Associate certification. Influencer: She has learned time management strategies from Raul and
his peer support team and is able to balance her internship and two other courses.
Semester 6 (12 credits): With a loan from her mother, Lisa got a scooter. The money she saved on gas
went into her certification fee fund. Lisa completed her Network Fundamentals course, along with three
other courses. Critical Juncture: Unable to work, Lisa connected with the student pantry and emergency
fund to tide her over until she could begin working again.
Semester 5 (0 credits): Lisa participates in a local “job hop” (Influencer) led by her Technical Writing instructor, which included a visit to Municode. Lisa liked the work environment and saw herself there. Influencer: Lisa’s instructor introduced her to a manager who said that all Municode employees had relevant certifications. Lisa also enrolled in Network Fundamentals. Critical Juncture Lisa’s car broke down and she had a hard time getting to school. (Hindrance): She took an Incomplete in Network Fundamentals.
Semester 4 (6 credits): Lisa interned at Ghostly, a local cybersecurity firm. She found the security work boring, but realized she had a flair for network design. She took the Technical Writing course and earned a 3.0! She credited her great instructor. Influencer: Lisa reported that with faculty support she became confident enough to start writing cover letters and a resume!
Semester 3 (8 credits): With Raul’s support, Lisa re-took her English course and passed. She also took a CAD course and realized why algebra is important!
Semester 2 (8 credits): Lisa took remedial algebra courses and, culminating in a 2.6 GPA, but she failed her English course, despite tutoring. Critical Juncture! Lisa’s college connected her with Raul, a peer writing coach (Influencer).
Semester 1 (8 credits): Lisa was dual-enrolled while in a high school and earned 8 credits and a Certified Ethical Hacker (CEH) certificate, but she remembered neglecting writing and algebra skills in high school (Hindrance).
Individual Pathway Scenario: Sample Interview Data
What does this look like in practice?For each participant, aggregate:
1. Layer 1: Institutional Research and College Data (IR Data)
2. Layer 2: Completer and Alumni Data (Surveys, 6 mo - 1yr after graduation)
For selected participants, integrate:
1. Layer 3: Pathway Explanatory Model (Interview to Saturation)
Results allow you to identify:
● Recurring critical junctures, including weak spots in curriculum
● Effective and needed student supports
● Placement of soft skills opportunities within student experiences
● The interplay between formal and informal learning experiences
http://www.youtube.com/watch?v=eHz8vhLKR4c
Next Steps / Future Research
Next Steps for Alumni Tracking Research
• Longitudinal Alumni Tracking• Use of alumni data to investigate and evaluate academic program efficacy• Post-graduation completer outcomes are becoming THE performance
metric for assessing program and student success (not just program completion).
• Automate IR dashboards to inform inquiry and share with others
• Use data to fuel change and improve decision-making
• Leverage various social media platforms to bolster alumni outreach and participation (RR)
• Use data to inform local economic contribution ROI models
• Not many CCs have time or expertise to do this…how do we build capacity?
Office of Alumni Employment Tracking Support Services (OAETSS)
Recognize
Institutional
Factors
Recognize
Student
Employment
Factors
Recognize
Program
Factors
Program and
student
employment
Goals and
Expectations
Office of
Alumni and
Employment
Tracking
Support
Services
(OAETSS)
1. Stress
Employability
Factors
And Student
Goals
2. Integrate Social
Media (e.g.,
LinkedIn) into
Coursework
learning
Outcomes
3. Conduct
Regular
Student and
Employer
Data Collection
and
Analysis
4. Promote
Student Employer
Networking
5. Other
considerations
Tracking of
Employment
Information
Upon
Graduation or
Program
Departure
External
Data
Sources
Integrate IR
sources
Alumni
Employment
Database
(AEDB)
Strategies
to Update
and Expand
Student and
Employment
Data
Reports from
Alumni
Employment
Database
1. Update
Student
Employment
Goals
2. Reassess
Success
Strategies of the
Office of Alumni
Employment
3. Revise
Curriculum as
Needed
4. Reduce
Institutional and
Program Barriers
Affecting Tracking
5. Other
considerations
Feedback
Questions?
This research was supported in part by NSF 1304382, 1700581, and the
Florida Board of Governors TeAMs Grant Program
Dr. Marcia A. Mardis
Professor & Associate Dean, CCI
Dr. Faye R. Jones
Faculty Researcher, CCI
