ATIM FINAL IMPACT REPORT BRIEFING
April 18, 2017
1
Deborah Kogan and Hannah Betesh
Today’s Presentation 2
Evaluation Overview and Context
Impact Study Sample
Impact Study Findings on Training
Impact Study Findings on Employment and Earnings
Implications for WIOA
Q & A
Overview of ATIM: Project Tested Evidence-Based Innovations
3
Industry Driven
Regional Manufacturing Partnerships
Accelerated Occupational
Skills Training
Industry
Recognized Credentials
Work-Based Training
Opportunities
Individualized Training Plans
and Case Management
Overview of ATIM: The State Played A Strong Leadership Role
4
Specification of the ATIM Program Model
Establishment of State Agency Partnerships with IDES, IMA and ICCB
Selection and Funding of Regional Consortia (with IDES and ICCB)
Development of centralized MIS for Applications, Intake, and Progress Monitoring
Overview of ATIM: Development of Regional Partnerships
5
Coordination of Activities Between Participating Local Areas
Enhanced Partnerships between Employers, Education Providers, and Workforce System
Coordinated Outreach and Intake of Applicants
Creation of New Accelerated Training Opportunities
ATIM Evaluation Overview 6
Implementation Study
• Implementation Progress and Fidelity: 2013 - 2015 Data from two rounds of site visits Stakeholder survey
Outcome Study
• Participant Training and Employment Outcomes 2013 -2016 Data from ATIM tracking system and
DCEO administrative records Data from Illinois DES
Impact Study
• Differences in outcomes between ATIM participants and the control group 2013 - 2016 Enrollment and completion of
training Completion of employer-
recognized credentials Rates of employment Quarterly earnings during
quarters following RA
Impact Study Sample 7
Enrollment Totals 8
90
204
88 112 20
514
42
93
25 60
4
224
Central Collar Counties North Central Northern Stateline Southwest Total
Study Participants, Overall and By Region
ATIM Participants Control Group
Key Characteristics 9
Demographics
Mostly white and male
Average age: 40 years old
Education and Employment
History
Nearly all had prior employment
history, half in manufacturing
Over half had education beyond
a high school diploma
Barriers to employment
Most did not self-report barriers to
employment
Most common = criminal record
Training 10
ATIM Participants Had Higher Rates of Training Enrollment and Completion
11 95.6%
[VALUE]
23.7% 19.7%
0.0%
20.0%
40.0%
60.0%
80.0%
100.0%
Enrolled in training Completed training
Training Enrollment and Completion
ATIM Participants Control Group
71.9%***
51.9%***
ATIM Participants Earned More Certificates
12
0.5
2.5
Control Group
ATIM Participants
Average Number of Certificates Earned Following Random Assignment
21.0%
76.1%
Control Group
ATIMParticipants
Attained a Certificate Following Random Assignment
55.1%***
2***
Limited Uptake of Work-Based Training 13
2.8%
1.6%
7.0%
Internships
Work Experience
On-the-Job Training(OJT)
Percent of Total ATIM Participants Enrolled in Work-Based Training
Impacts on Employment and Earnings 14
Training-Related Employment 15
According to the ATIM Tracking System…
• 71% of ATIM participants obtained employment at the end of the program
• 63% of those participants found training related employment
ATIM Participants Had Higher Rates of Employment Two Years After RA
16
0
10
20
30
40
50
60
70
80
-8 -7 -6 -5 -4 -3 -2 -1 0 1** 2 3* 4 5** 6** 7** 8
Empl
oym
ent (
%)
Quarter Relative to Random Assignment
Program Group Control Group
ATIM Led to Higher Earnings 17
0
1,000
2,000
3,000
4,000
5,000
6,000
7,000
-8 -7 -6 -5 -4 -3 -2 -1 0 1** 2 3 4 5 6*** 7** 8
Ave
rage
Ear
ning
s ($
)
Quarter Relative to Random Assignment
Program Group Control Group
Implications for WIOA 18
ATIM Anticipated Key Features Emphasized in the WIOA Legislation
19
WIOA
Sectoral Focus
Career Pathways to
Jobs of Today and Tomorrow
Employer Demand-
Driven Orientation
Work-Based Learning
Data Driven
Implications of ATIM Experience 20
Key Features of ATIM Model
Fidelity of ATIM Implementation Implications for WIOA
Sectoral Industry-driven Training +
Focus on promising sectors, but be prepared for economic downturns in targeted sectors.
Accelerated Training Schedules
Some regions were more successful than others
Strengthen partnerships with community colleges and increase their commitment to offering short-
term stackable credentials.
Contextualized Basic Skills “On Ramps”
- Not enough time in grant
period for sequenced services
Recognize that individuals with basic skills gaps may need longer to complete occupational skills
training
Case Management by Multiple Partners
- Agency and employer partners not interested
Negotiate partner roles to add value for participating partners
Individualized Employment/ Training Plans
+
Encourage design of services to match individualized assessed customer needs and
interests rather than a one-size-fits-all approach
State-level Participant Tracking System
+ Need time and resources for detailed training of local staff on standardized data system; greatly
increases transparency of results at state, regional and local level
Limitations of ATIM Evaluation 21
Effectiveness of ATIM model for participants with recent employment experience and skills
transferable to manufacturing
Less evidence about whether model works
for individuals with less formal education/ work
experience
Issues for Future Research 22
• What particular features of sector-focused training initiatives make the most difference in bringing about the desired program impacts?
• What program models work best for low-income individuals with serious employment barriers, including limited basic skills?
• What are the most effective models for employer engagement? • To what extent are positive program impacts sustained or
increased over time? • How can programs promote employment retention and further
career advancement, once individuals are placed in their initial jobs after training?
Questions? 23