Embed Size (px)
Citation preview
CONTINUOUS IMPROVEMENT AND PERFORMANCE ACCOUNTABILITY TEXAS
STYLE: A QUALITY MODEL FOR PROGRAM MANAGEMENT
Anson Green
Carrie Tupa
Lori Slayton
Adult Education and Literacy
Texas Workforce Commission
BEFORE YOU CAN IMPROVE… WHAT DOES GOOD LOOK LIKE?
THE TEXAS WORKFORCE COMMISSION
• Title I Adult and Dislocated Worker
and Youth Programs
• Title II Adult Education and Literacy
• Title III Wagner-Peyser
• Title IV Vocational Rehabilitation
Services (September 1)
TRANSITION OF THE ADULT EDUCATION AND LITERACY PROGRAM – SUNSET COMMISSION FINDINGS
“Making the TWC responsible for adult education would better position the State to effectively ensure that adult Texans have the basic skills necessary to succeed in the workplace.” pg. 26
Alignments would support needs of shared customers
Better support performance for shared customers
Facilitate more seamless customer services
Reduce challenges associated with sharing test data and other student information
SENATE BILL 307
SB 307
(2) …develop the mechanism and guidelines for the coordination of comprehensive adult education and related skill training services for adults with other entities…
HISTORICAL LENS…
•Priorities were EFL gains, high school equivalency obtainment
•Secondary priorities were transitions to postsecondary/employment
•Workforce integration represented through special projects/initiatives
BIG STATE, BIG GOALS
From the draft Adult Education and Literacy strategic plan
TWC Adult Education will have 20,000 students enrolled in career pathways programs by the year
2020.
Under this goal, career pathways programs include:
Integrated education and training
Workplace literacy
Skills for internationally trained professionals
WHAT IS OUR ACCOUNTABILITY STRUCTURE UNDER WIOA?
•Focus on skill gains that lead to something
•Increased currency on transitions to postsecondary education and training and employment
These things take time to show in the data!
THE TEXAS AEL ACCOUNTABILITY SYSTEM TWC Program
Support Specialists
TRAIN Tex PD System
TWC Division of Operational
Insight
Texas Educating Adults
Management System (TEAMS)
TWC Sub-recipient Monitoring
TWC Board and Special Initiatives
Contracts
THE TEXAS AEL ACCOUNTABILITY SYSTEM TWC Program
Support Specialists
TRAIN Tex PD System
TWC Division of Operational
Insight
Texas Educating Adults
Management System (TEAMS)
TWC Sub-recipient Monitoring
TWC Board and Special Initiatives
Contracts
1) Oversight
2) Guidance
3) Vision
Oversight
Ongoing/
”Just-in-Time”
• Program data-entry into TEAMS
• TEAMS data corrections/ error checking
Weekly/ Bi-weekly
• General “status” check of all program health
Monthly
• Local programs validate data
• Program Management Reports
• Inter-departmental meeting -Monthly Performance Report (MPR)
Yearly
• Desk reviews
• On-sight reviews
• Target setting
• Performance funding allocations
Multi-year/Long-term
• Contract terms in rule
• Competitive procurement
PROGRAM MANAGEMENT REPORTS
TEXAS EDUCATING ADULTS MANAGEMENT SYSTEM (TEAMS)
THERE
I FIXED IT
THERE
I FIXED IT
CANNED REPORTS IN TEAMS (54) Average Daily Attendance Non Validated Classes
GED Count with Educational
Functioning Level
Curriculum by Grant
Recipient, Sites,
Participants and Teachers
(DL)
Table X: Outcomes for
Adults in Correctional
Education Programs
Table IVC: Educational
Gains and Attendance for
Participants in Distance
Education
Classes with Functioning
Level Participant Achievements of
Follow-up Measures Participant Address List
Participants by Educational
Functioning Level and
Contact Hour Range (DL)
Table XI: Secondary
Outcome Measures (Self
Reported Data)
Table V: Core Follow-up
Outcome Achievement
Participant Contact Hours
By School District (LEA) and
Funding Source
Participant Count and
Hours By Education Level Participant Address Mailing
Labels Site and Provider Address
Mailing Labels
Table XI - Part 2 -
Achievements of Program
Participants (Self Reported
Data)
Table VA: Core Follow-up
Outcome Achievement for
Participants in Distance
Education
Data Sign Off Status Participant Count and
Hours By Separation
Reason
Participant Count and
Hours By Site Staff Address List
Table XIII: Core Follow-up
Outcome Achievement for
Prior Reporting Year and
for Unintended Outcomes
Table VI: Participant Status
and Program Enrollment
Participants by Contact
Hour Range and Level
Completion (DL)
Participant Counts By
Ethnicity and Gender Participant Roster
Staff Address Mailing
Labels Table XIV: Local Grantees
by Funding Source
Table VII: Adult Education
Personnel by Function and
Job Status
Performance Report Participant Count and
Hours By Funding Source,
Site and Class
Participants Tested with
Less Than Minimum Hours of
Instruction
Staff Count By Employment
Status Table II: Participants by
Ethnicity, Gender, and Age Participants Funded By
TANF
Individual Waiting List Staff and Participants
Count by Grant Recipient Participant Roster With
Latest Assessments
Adult Education Staff
Development Report (In-
Service & Pre-Service)
Table III: Participants by
Age and Functioning Level TANF - Welfare to Work
Initiatives in Texas
Organization Waiting List Participants Enrolled and
Level Completed with
Proxy Hours
Participants with Less than
12 Contact Hours Staff Roster Main
Table IV: Educational Gains
and Attendance by
Educational Functioning
Level
GED Participant Matches Participant Gains
Educational Gains and
Attendance for Pre- and
Posttested Participants
(Table IV and IVB
Combined)
Staff Without
Qualifications
Table IVA: Educational
Gains and Attendance by
Educational Functioning
Level, with Completion
Breakdown
GED Practice Test Participants Achieved GED
by Age and Functioning
Level
Participants with Proxy
Hours
Table I: Participants by
Ethnicity, Gender, and
Educational Functioning
Level
Table IVB: Educational
Gains and Attendance for
Pre- and Posttested
Participants
PROGRAM STATUS SHEET (PROGRAM REPORT CARD)
MONTHLY PERFORMANCE REPORT MEETING
Finance Provides financial analysis and alerts of issues
Adult Ed Program
Staff
Workforce Programs
Division Director
Provides agency leadership
perspective
Division of Operational
Insight
Contracts Provides contractual/ compliance lens
Provides the “detail lens”/ program
perspective
Provides the data
Guidance
Ongoing/
”Just-in-Time”
• AEL Letters
• Technical assistance bulletins
• Webinars
• OCTAE Guidance
Weekly/ Bi-weekly
• All-grantee conference call
Monthly
• Program management reports
Yearly
• Business meetings (2-3 per year)
• Symposiums
• TWC Annual Conference
Multi-year/Long-term
• Strategic Plan
PROGRAM STATUS SHEETS
•Easy to see at-a-glance what the expectation is
•Help diagnose weaknesses
EFL GAIN ANALYSIS
Summary
DOS 1 2 3 4 5 6 1 2 3 4 5 6 Total
Language 0 0 0 2 0 1 3
Reading 0 0 2 1 1 1 5
Math 0 0 4 8 1 1 14
Oral 0 0 0 0 0 0 0
Literacy 0 0 0 0 0 0 0
Listening 15 2 0 0 0 0 17
Reading 8 2 0 0 0 0 10
Writing 66 43 15 0 0 0 124
Speaking 0 0 0 0 0 0 0
AB
E/
AS
E
ABE/ASE Levels ESL Levels
BEST/BEST Plus
TABE CLAS-E
ES
L
EFL GAIN ANALYSIS
Summary
DOS 1 2 3 4 5 6 1 2 3 4 5 6 Total
Language 0 0 0 2 0 1 3
Reading 0 0 2 1 1 1 5
Math 0 0 4 8 1 1 14
Oral 0 0 0 0 0 0 0
Literacy 0 0 0 0 0 0 0
Listening 15 2 0 0 0 0 17
Reading 8 2 0 0 0 0 10
Writing 66 43 15 0 0 0 124
Speaking 0 0 0 0 0 0 0
AB
E/
AS
E
ABE/ASE Levels ESL Levels
BEST/BEST Plus
TABE CLAS-E
ES
L
Summary
DOS 1 2 3 4 5 6 1 2 3 4 5 6 Total
Language 0 0 0 2 0 1 3
Reading 0 0 2 1 1 1 5
Math 0 0 4 8 1 1 14
Oral 0 0 0 0 0 0 0
Literacy 0 0 0 0 0 0 0
Listening 15 2 0 0 0 0 17
Reading 8 2 0 0 0 0 10
Writing 66 43 15 0 0 0 124
Speaking 0 0 0 0 0 0 0
AB
E/
AS
EABE/ASE Levels ESL Levels
BEST/BEST Plus
TABE CLAS-E
ES
L
EFL GAIN ANALYSIS
Summary
DOS 1 2 3 4 5 6 1 2 3 4 5 6 Total
Language 0 0 0 2 0 1 3
Reading 0 0 2 1 1 1 5
Math 0 0 4 8 1 1 14
Oral 0 0 0 0 0 0 0
Literacy 0 0 0 0 0 0 0
Listening 15 2 0 0 0 0 17
Reading 8 2 0 0 0 0 10
Writing 66 43 15 0 0 0 124
Speaking 0 0 0 0 0 0 0
AB
E/
AS
E
ABE/ASE Levels ESL Levels
BEST/BEST Plus
TABE CLAS-E
ES
L
Summary
DOS 1 2 3 4 5 6 1 2 3 4 5 6 Total
Language 0 0 0 2 0 1 3
Reading 0 0 2 1 1 1 5
Math 0 0 4 8 1 1 14
Oral 0 0 0 0 0 0 0
Literacy 0 0 0 0 0 0 0
Listening 15 2 0 0 0 0 17
Reading 8 2 0 0 0 0 10
Writing 66 43 15 0 0 0 124
Speaking 0 0 0 0 0 0 0
AB
E/
AS
EABE/ASE Levels ESL Levels
BEST/BEST Plus
TABE CLAS-E
ES
L
Requires more analysis:
Is this issue systemic, or is it
targeted to an individual
instructor?
THE TEXAS AEL ACCOUNTABILITY SYSTEM TWC Program
Support Specialists
TRAIN TEX PD System
TWC Division of Operational
Insight
Texas Educating Adults
Management System (TEAMS)
TWC Sub-recipient Monitoring
TWC Board and Special Initiatives
Contracts
Program Specialist
TRAIN Tex
PD Specialist
Vision
Ongoing/
”Just-in-Time”
• AEL Letters
• Technical assistance bulletins
• Webinars
• OCTAE Guidance
Weekly/ Bi-weekly
• All-grantee conference call
Monthly
• Program management reports
Yearly
• Business meetings (2-3 per year)
• Symposiums
• TWC Annual Conference
Multi-year/Long-term
• Strategic Plan
• AEL provider performance workgroup
“A vision was never
accomplished simply through
requirements.”
Johan Uvin
Department of Education
TWC’S DIVISION OF OPERATIONAL INSIGHT
• Does agency-wide performance and accountability, including target development and negotiations
TWC’S DIVISION OF OPERATIONAL INSIGHT
• Does agency-wide performance and accountability, including target development and negotiations
TWC’S DIVISION OF OPERATIONAL INSIGHT
• Does agency-wide performance and accountability, including target development and negotiations
• Un-biased (believe us!)
• Understands what’s happening at the federal level
• Participates in federal workgroups for all agencies
WHEN THINGS GO…
THE TECHNICAL ASSISTANCE PLAN PROCESS
Sub-recipient
monitoring
reports
Performance
reports/timely
data entry
Fiscal reports
Objective
Criteria
+
What is the risk
to the agency
or public?
Technical
Assistance Plan
Issued
THE TECHNICAL ASSISTANCE PLAN
THE TECHNICAL ASSISTANCE PLAN
Based on a set of clear issues
Examples: Failure to enter data on-time
Did not meet enrollment target
THE TECHNICAL ASSISTANCE PLAN
Contains a set of clear benchmarks
Issue Benchmark
Failure to enter data on-
time
Complete six consecutive
monthly data validations
on-time with less than 5%
error rate (requiring
invalidation)
Did not meet enrollment
target
Meet four consecutive
months of meeting monthly
enrollment benchmark
THE TECHNICAL ASSISTANCE PLAN
• Contains mutually agreed upon strategies to meet benchmarks
Issue Benchmark Strategies
Failure to enter data
on-time
Complete six
consecutive monthly
data validations on-
time with less than 5%
error rate (requiring
invalidation)
• Create standard
operating
procedure for data-
entry – DUE DATE
• Re-train data-entry
staff within 2
months
BEYOND THE TECHNICAL ASSISTANCE PLAN
Technical Assistance
Plan
Corrective Action Plan
Intent to sanction
Sanctions
BEYOND THE TECHNICAL ASSISTANCE PLAN
Technical Assistance
Plan
Corrective Action Plan
Intent to sanction
Sanctions
Getting here typically indicates a very severe systemic issue
Professional Development
Center
Texas A & M University
Career Pathways Expansion
Houston Community College
Public Library AEL
Expansion
Texas State Library and Archives Commission
Assessment and Standards Initiative
Texas State University
Focus on the Basics Reading and Math
Institutes Region 6
Distance Learning
Mentor Initiative
College of the
Mainland
Northside ISD
PD and Program Integration Efforts with Non-Profits
Literacy Texas
OneStar
Foundation
Adult Education and Literacy Grantees
Texas Workforce Commission
TRAIN TEX TRAINING, RESOURCE AND INNOVATION NETWORK FOR TEXAS
WHAT WORKS FOR TEXAS?
Vision
Ongoing/
”Just-in-Time”
• AEL Letters
• Technical assistance bulletins
• Webinars
• OCTAE Guidance
Weekly/ Bi-weekly
• All-grantee conference call
Monthly
• Program management reports
Yearly
• Business meetings (2-3 per year)
• Symposiums
• TWC Annual Conference
Multi-year/Long-term
• Strategic Plan
• AEL provider performance workgroup
Big/innovative performance idea
from TWC
Performance workgroup
determines: Will this work?
Big/innovative performance idea
from TWC
Performance workgroup determines:
Will this work?
TWC “tries it out”
QUESTIONS?
Anson Green
Director
Carrie Tupa
Strategic Planning and Accountability Coordinator
Lori Slayton
Program Specialist, Professional Development & Special Projects
![Session 4 Engagement, Continuous Improvement, and Accountability CLAS Training [ADD DATE] [ADD PRESENTER NAME] [ADD ORGANIZATION NAME]](https://img.pdfslide.net/doc/110x75/56649eaa5503460f94baff02/session-4-engagement-continuous-improvement-and-accountability-clas-training.jpg)
