FY2007 Federal Performance for Managers1June 2006 FY2007 Federal Performance for Managers Overview and Discussion of Performance Measures for FY07

June 2006 FY2007 Federal Performance for Managers 1

FY2007 Federal Performancefor Managers

Overview and Discussion of

Performance Measures for FY07

FY2007 Federal Performance for Managers 2June 2006

FY2007 Performance MeasuresTechnical Assistance Session for Managers

Agenda

Performance Measures At-A-Glance

Performance Reporting Cohorts

Key Definitions You Need to Know

Performance Measures Methodologies Who’s In/Out of the Measure How the Measures are Calculated

Resources


FY2007 Performance Measures At-A-Glance

Performance Measures

Adult Measures Youth

AdultsDislocatedWorkers

Trade NEG VeteransLabor

ExchangeOlderYouth

YoungerYouth

AllYouth

Entered Employment Rate

Retention Rate

Average Earnings

Employment & Certificate Rate

Earnings Gain

Skill Attainment Rate

Youth Diploma Rate

Attainment Degree/Certificate Placed in Employment/ Education Rate

Literacy/Numeracy Gain


FY2007 LABOR EXCHANGE, VETERAN AND TITLE IPERFORMANCE MEASURES COHORTS

Performance Measures Report Cohort

Entered Employment Rate(Adult, DW, OY, LX)

Exit Cohort10/01/2005 – 09/30/2006

Retention Rate(Adult, DW, OY, YY, LX)

Exit Cohort04/01/2005 – 03/31/2006

Average Earnings(Adult, DW, LX)

Exit Cohort04/01/2005 – 03/31/2006

Employment/Certificate Rate(Adult, DW, OY)

Exit Cohort10/01/2005 – 09/30/2006

Earnings Gain(OY)

Exit Cohort04/01/2005 – 03/31/2006

Skill Attainment Rate(YY)

Participation/Goals Cohort04/01/2006 – 03/31/2007

Youth Diploma Rate(YY)

Exit Cohort04/01/2006 – 03/31/2007

Attainment of Degree/ Certificate Rate(All Youth)

Exit Cohort10/01/2005 – 09/30/2006

Placed in Employment/ Education Rate(All Youth)

Exit Cohort10/01/2005 – 09/30/2006

Literacy and Numeracy Gain(All Youth)

Participation/Test Cohort07/01/2006 – 06/30/2007


FY2007 Performance Measures

A Few Key Definitions Who is a Participant?

What about Self-Service?

What is a Program Exit?

What is the Exit Quarter?

What is an Exclusionary Exit?

What is Supplemental Employment Data?


A Few Key Definitions…

Participant - A participant is an individual who is determined eligible to participate in the program and receives a service funded by the program in either a physical location (One-Stop Career Center or affiliate site) or remotely through electronic technologies. Self Service Participant

For Labor Exchange, we will begin counting as self-service participants, those who receive only self-service or informational services whether through the virtual career center (i.e. MJQ) or at a physical career center.

Labor Exchange self-service participants will be counted in performance calculations.



Program Exit - The term “program exit” means a participant does not receive a service funded by the program or funded by a partner program for 90 consecutive calendar days and is not scheduled for future services.

Operational Parameters:

Title I participants will be reported as exiters according to the exit date entered by local staff on the Basic tab of MOSES. Local operators are responsible for assuring that participants that receive no service in 90 days are exited. Labor Exchange participants who receive no service for 90 days will be reported as exiters.

To the extent possible, it is the responsibility of local staff to assure that a program exit is not entered until all services are completed (e.g. Labor Exchange, NEG, etc.). This will assure that the participant enters the exit cohort for all programs on a common exit date.



Exit Quarter - Represents the calendar quarter in which the date of exit is recorded for the individual.

Quarter 1 Quarter 2 Quarter 3 Quarter 4

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC


Example-QuartersA Few Key Definitions…

1st Quarter after exit

Quarter 1 Quarter 2 Quarter 3 Quarter 4

JAN FEB MAR APR MAY JUN JUL AUG SEP OCT NOV DEC

1st, 2nd and 3rd Quarters After Exit

2nd Quarter after exit

3rd Quarter after exit

Exit Quarter



Exclusionary Exit: A participant in any of the following categories either at the time of exit or during the three quarter measurement period following the exit quarter, will be excluded from all performance measures:

Institutionalized Health/Family Medical Care Deceased Reservist Called to Active Duty Youth Relocated to a Mandated Program Invalid SSN



Supplemental Employment Data: Career Centers may use supplemental sources of data to document a participant’s entry and retention in employment for those participants not covered by wage records; e.g. self-employment, federal jobs, etc.

Operational Parameters:

Supplemental employment data is entered on MOSES via the Services/Employment Information tab. All supplemental data and methods must be documented and are subject to audit.


FY07 Adult Performance Measures(Labor Exchange, Veterans, Trade, Title I Adults, DW, NEG)

■ Entered Employment Rate■ Retention Rate■ Average Earnings

■ Employment & Certificate Rate(Title I Adults and Dislocated Workers)


Entered Employment RateOf those who are not employed at the date of participation:

# of adults who are employed in thefirst quarter after the exit quarter

divided by

# of adults who exit during the quarter

FY07 AdultPerformance Measures…


= 73%

180 adults employed 1st qtr. after exit – 15* = 165

240 total exiters – 15* = 225

*These 15 exiters were already employed at participation and are excluded from the entire calculation.

FY07 Adult Performance Measures…

[ADULT ENTERED EMPLOYMENT RATE]



Of those who are employed in thefirst quarter after the exit quarter:

# of adults who are employed in both thesecond and third quarters after the exit quarter

divided by

# of adults who exit during the quarter

Employment Retention Rate


= 81%

145 adults employed in both 2nd and 3rd qtr after exit

180 adults exited during the quarter

Of those who are employed in the 1st quarter after exit:

Exiters who were employed at time of participation are included.

FY07 Adult Performance Measures…

[ADULT EMPLOYMENT RETENTION RATE]



Of those adults who are employed in thefirst, second and third quarters* after the exit quarter:

Total earnings in the second plus the total earningsin the third quarters after the exit quarter

divided by

# of adult participants who exit during the quarter

Adult Average Earnings

* via wage match only


= $15,909

Summed wages from quarters 2 and 3 after exit = $1,225,000

77 exiters

FY07 Adult

Performance Measures…

[ADULT AVERAGE EARNINGS]

Of those adults who are employed in the 1st, 2nd AND 3rd qtr after the exit quarter:


Employment & Credential Measure(Adults and Dislocated Workers)

Of those who receive training services:

# of participants who were employed in the first quarterafter the exit quarter and received a credential/certificate by the

end of the third quarter after the exit quarter

divided by

# of participants who exit during the quarter

FY07 Adult PerformanceMeasures…


= 66%

105 were both employed 1st qtr after exit AND received a credential/certificate by end of the 3rd quarter after exit

160 exiters during the quarter

Of the Adults and Dislocated Workers who receive training services,

FY07 Adult PerformanceMeasures…

[ADULT EMPLOYMENT & CERTIFICATE RATE]


Carmen enrolled as a dislocated worker in September 2005. She was still working when she registered, but had received notice of layoff. She exited in June 2006. She starts a new job in September 2006. What is her performance outcome for Entered Employment?

Carmen's performance outcome for Entered Employment is positive because she was employed during the first quarter after exit, July-September. Remember Notice of Layoff or Military Separation means “unemployed” at participation.

Carmen's performance outcome for Entered Employment is positive because she was employed during the first quarter after exit, July-September. Remember Notice of Layoff or Military Separation means “unemployed” at participation.

FY07 Adult PerformanceMeasures…Entered Employment Rate Quiz

Positive ExcludedNegative


Ben completed an A+ Certification training and earned his certificate while enrolled in WIA. He exits with a job in April, but loses the job in June and does not find another until October. What is Ben’s performance outcome for the Employment and Credential measure?

Ben's performance outcome for the Employment and Credential measure is negative because he was not employed during his first quarter after exit, July-September.

Ben's performance outcome for the Employment and Credential measure is negative because he was not employed during his first quarter after exit, July-September.

FY07 Adult PerformanceMeasures…Empl/Credential Rate Quiz



Julie is an adult. She exits in December and does not find work until April. She keeps the job through September. What is her performance outcome for Employment Retention?

Julie is excluded from the Employment Retention Measure because she was not employed during her first quarter after exit, January-March.

Julie is excluded from the Employment Retention Measure because she was not employed during her first quarter after exit, January-March.

FY07 Adult PerformanceMeasures…Employment Retention Quiz



FY07 Adult PerformanceMeasures…Average Earnings Quiz

John is a dislocated worker. He exits in March and finds a new job in July making about $38,000/year and keeps on working. What is John’s performance outcome for Average Earnings measure?

John is excluded from the Average Earnings measure because he was not employed during his first quarter after

exit.(April – June)

John is excluded from the Average Earnings measure because he was not employed during his first quarter after

exit.(April – June)

Excluded Positive Negative


FY07 Older YouthPerformance Measures

■ Entered Employment Rate■ Retention Rate■ Earnings Gain■ Employment & Certificate Rate


FY07 Older YouthPerformance Measures…

Of those Youth who are not employed at the date of participationand who are either not enrolled in post-secondary education or

advanced training in the first quarter after the exit quarter orare employed in the first quarter after the exit quarter:

# of youth who are employed in the first quarter after the exit quarter

divided by

# of youth who exit during the quarter

Entered Employment Rate


FY07 Older Youth PerformanceMeasures…

= 68%

105 youth employed 1st qtr. after exit – 4* - 7** = 94

150 exiters – 4* - 7** = 139

*These 4 youth were excluded from the entire calculation because they were employed at registration.

**These 7 youth are not employed and are enrolled in advanced training or post secondary education in the quarter after exit, they are excluded.

[OLDER YOUTH ENTERED EMPLOYMENT RATE]


Of those youth who are employed in the first quarter after the exit quarter and who are either not enrolled in post-secondary

education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter:

# of youth who are employed in the third quarter after the exit quarter

divided by

# of participants who exit during the qtr


Employment Retention Rate



= 82%

75 youth employed in 3rd qtr. after exit

99 youth employed in 1st qtr. after exit – 7* = 92

*These 7 youth are excluded because they are enrolled in post secondary education or advanced training and not employed in 3rd quarter after exit.

Of those youth who are employed in the first quarter after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter:

[OLDER YOUTH EMPLOYMENT RETENTION RATE]


Earnings Gain MeasureOf those youth who are employed in the first qtr after the exit quarter

and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are

employed in the third quarter after the exit quarter:

Post-Earnings in the [second + third qtrs] after the exit qtr minusPre-Earnings in the [second + third qtrs] prior to the participation qtr

divided by

# of participants who exit during the quarter




= $2381

$950,000 - $550,000 = $400,000

240 exiters - 60 * - 12** = 168

*These 60 customers were not employed in the 1st quarter after exit, so they are excluded.

**Supplemental data was used to verify employment for these 12 customers, so they are excluded.

Of those youth who are employed in the first qtr after the exit quarter and who are either not enrolled in post-secondary education or advanced training in the third quarter after the exit quarter or are employed in the third quarter after the exit quarter:

[OLDER YOUTH EARNINGS GAIN MEASURE]


Employment/Education & Credential Measure

# of older youth participants who were either employed, inpost-secondary education or in advanced training in the first qtr after the exit qtr and received a credential by the

end of the third qtr after the exit qtr

divided by

# of older youth participants who exit during the qtr




= 63%

120 w/credential by the 3rd quarter after exit - 25* = 95

150 exiters

*These 25 were not employed or enrolled in advanced training or post secondary education in the quarter after exit.

[OLDER YOUTH EMPL/CREDENTIAL RATE]


FY07 Older Youth PerformanceMeasures… Entered Employment Quiz

Excluded Negative

Adam, an Older Youth, entered the WIA youth program last year without a job. He exited January 5th with a job. April 3rd he lost his job. What is Adam's outcome for the Entered Employment measure?

Adam's outcome for the Entered Employment measure is positive because he was employed during his first quarter after exit, April-

June.

Adam's outcome for the Entered Employment measure is positive because he was employed during his first quarter after exit, April-

June.

Positive


Miriam, an Older Youth, exits with a job on October 15. She leaves that job December 15, finds another one April 7, and keeps it through the following September. What is her outcome for Employment Retention?

FY07 Older Youth PerformanceMeasures… Employment Retention Quiz

Miriam's outcome for Employment Retention is excluded because she was not employed during her first quarter after exit, January-March.

Miriam's outcome for Employment Retention is excluded because she was not employed during her first quarter after exit, January-March.



Tanya, an Older Youth, entered WIA and completed a GED. She exits in June and travels (jobless) through the summer before beginning classes for her Associate's Degree in October. What is her outcome for the Credential measure?

FY07 Older Youth PerformanceMeasures…Empl/Educ & Credential Quiz


Tanya's outcome for Credential is negative because she was not

employed during her first quarter after exit, July-September.

Tanya's outcome for Credential is negative because she was not

employed during her first quarter after exit, July-September.


FY07 Younger YouthPerformance Measures

■ Skill Attainment Rate■ Diploma or Equivalent Attainment■ Retention Rate


Skill Attainment Rate

Of all in-school youth and any out-of-school youthassessed to be in need of basic skills, work readiness skills, and/or occupational skills:

Total # of goals attained

divided by

Total # goals set

FY07 Younger YouthPerformance Measures…



= 83%

125* Youth Goals with a Status of “Attained”

150 Youth Goals set

*These goals are required to have the status changed to “Attained” within one (1) year of being set to be include in the numerator.

[YOUNGER YOUTH SKILL ATTAINMENT RATE]


Diploma or Equivalent Attainment Rate

Of those who participate in the programwithout a diploma or equivalent:

# of participants who attained a secondary school diploma orequivalent by the end of the first quarter after the exit qtr

divided by

# of participants who exit during the qtr(except those still in secondary school at exit)




= 71%

60 Youth attained Diploma or Equivalent = 60

105 Youth exiters – 20* = 85

*These 20 Younger Youth were still in secondary school at exit, and they are excluded.

Of those who participate in the program without a diploma or equivalent and did not exit as Still in secondary school at Exit :

[YOUNGER YOUTH DIPLOMA/EQUIVALENCY RATE]


Retention Rate

# of participants who were found in either post-secondary education, advanced training, employment, military service or a qualified

apprenticeship in the third qtr after the exit qtr

divided by

# of participants who exit during the qtr(except those still in secondary school at exit)




= 71%

85 were retained in 3rd qtr after exit

120 Youth Exiters

[YOUNGER YOUTH RETENTION RATE]

Of those younger youth who did not exit as Still in Secondary School at Exit:


Tamika exits from WIA in December. She is enrolled in high school and expects to attain her diploma in June. What is her outcome on the Diploma/Equivalent Attainment measure?

FY07 Younger Youth PerformanceMeasures… Diploma or Equiv Rate Quiz

Tamika's outcome on the Diploma/Equivalent Attainment

measure is excluded because she is in secondary school at exit.

Tamika's outcome on the Diploma/Equivalent Attainment

measure is excluded because she is in secondary school at exit.



Javier exits WIA on September 23. He gets a job right away and works through April 4, when he suddenly decides to quit. What is his outcome on the Retention measure?

FY07 Younger Youth PerformanceMeasures… Retention Quiz

Javier's outcome on the Retention measure is positive because he was in a positive outcome (employment) during his third quarter after exit,

April-June.

Javier's outcome on the Retention measure is positive because he was in a positive outcome (employment) during his third quarter after exit,

April-June.

Excluded Negative Positive


FY07 All YouthPerformance Measures

■ Placement in Employment or Education■ Attainment of a Degree or Certificate■ Literacy and Numeracy Gains


Attainment of a Degree or Certificate

Of those enrolled in education (at the date ofparticipation or at any point during the program):

# of youth participants who attain a diploma, GED or certificate by the end of the third quarter after the exit quarter

divided by

# of youth participants who exit during the quarter

FY07 All YouthPerformance Measures…



= 77%

120 Youth attained a diploma or certificate by 3rd qtr after exit

156 Youth exiters in Education

Of those enrolled in education (at the date of participation or at any point during the program):

[YOUTH ATTAINMENT OF DEGREE OR CERTIFCATE]


Placement in Employment or Education

Of those who are not in post-secondary education or employment (including the military) at the date of participation:

# of youth participants who are in employment (including military) or enrolled in post-secondary education and/or advanced training/

occupational skills training in the first quarter after the exit quarter

divided by

# of youth participants who exit during the quarter




= 77%

142 Youth in qualifying outcomes in 1st qtr. after exit

185 Youth Exiters

Excluded from this measure are those that were employed, in the military, or in post-secondary education at date of participation.

[YOUTH PLACEMENT IN EMPLOYMENT OR EDUCATION]


Literacy and Numeracy Gains

Of those out-of-school youth who are basic skills deficient(and enroll on or after July 1, 2006):

# of youth participants who increase one or moreeducational functioning levels

divided by

# of youth participants who have completed a year in the program (i.e., one year from the date of first youth program service) plus the # of youth participants who exit before completing a year in the program



High Adult Secondary Education

Low Adult Secondary Education

High Intermediate Basic Education

Low Intermediate Basic Education

Beginning Basic Education

Beginning ABE Literacy

Tests such as TABE, CASAS and ABLE provide numerical scores that correspond with one of six Adult Basic Education educational functioning levels.

Adult Basic Education (ABE) Educational Functioning Levels



High Advanced ESL

Low Advanced ESL

High Intermediate ESL

Low Intermediate ESL

Beginning ESL

Beginning ESL Literacy

Tests such as CASAS, Oral BEST, BEST Plus and Literacy BEST provide numerical scores that correspond with one of six ESL educational functioning levels.

English As A Second Language (ESL) Ed Functioning Levels




= 56%

68 Youth who advance one or more educational levels

100 Youth who completed a year + 22 who exited before completing a year = 122

[YOUTH LITERACY AND NUMERACY GAIN]


Youth #2

EFL 1

EFL 2

EFL 3

EFL 4

EFL 5

Youth #1 Youth #3 Youth #4

Examples of EFL ImprovementPre-test to post-test, within one program year




Placement Measure

CertificateMeasure

Literacy Gain Measure

Yes

Yes

In In OutISY—has a summer job

In In OutISY--earns HS diploma but nothing else

In In InUE/OSY/BSD-- increases basic skills & earns GED

In Out OutOSY, Not BSD -- gets job

Out Out InEmployed OSY/BSD--increases basic skills

In In InOSY/BSD--starts GED but drops out; no skill gain

Youth Performance Measures Different Cohorts

NEGATIVEPOSITIVE


ADDITIONAL RESOURCES

Workforce Issuance 06-35 - Performance Review Tool

TEGL 17-05: Common Measures Policy for theEmployment and Training Administration (ETA)

http://wdr.doleta.gov/directives/corr_doc.cfm?DOCN=2195

PPV/SPRA Performance Enhancement Project On-Line Tutorials

http://www.spra.com/PEP/online_intro.shtml

(not 100% up to date for FY2007)

Documents

FY2007 Federal Performance for Managers1June 2006 FY2007 Federal Performance for Managers Overview and Discussion of Performance Measures for FY07