MWIB Performance Accountability SubCommittee

Report on Governance and Coordination of the Workforce Development System

Draft October 31, 2008

I. Introduction

The Economic Stimulus Act of 2006 created the Performance Standards and Workforce

Accountability Task Force. Among other responsibilities, the legislation directed the Task Force

to “design and conduct an evaluation and analysis of the present governance and coordination

of workforce development agencies and programs in the commonwealth”. The legislation

defined the goals of the study as follows: “The goals of the study shall include assisting citizens

of the commonwealth in making better use of the state’s workforce development system,

defining clearer lines of responsibility and accountability, and analyzing the management of the

system in an effort to both improve service delivery and supplement the resources available for

education and training.”

In 2007, in consultation with stakeholder groups throughout the Commonwealth, the Patrick

Administration identified three key priorities for workforce development:

Closing the Skills Gap

Enhancing the Youth Pipeline

Building the Capacity of the Workforce Development System

In December 2007 the Task Force filed a report with the Legislature that provided a summary of

the Task Force’s work and recommendations for subsequent activities. In January 2008 the

Performance Accountability SubCommittee of the Massachusetts Workforce Investment Board

assumed the responsibilities of the Task Force. This report addresses the SubCommittee’s

findings and recommendations associated with two responsibilities related to governance and

coordination:

a. Conduct a qualitative research study to identify the governance and coordination issues

associated with the workforce development system’s capacity to meet the needs of key

user groups.

b. Build capacity for performance management, including:

identifying objectives for performance management,

1

defining the key attributes and making recommendations for the development and

implementation of a shared performance information system, and

developing an estimate of resource (staff, software, hardware) and

legislative/regulatory requirements for implementing the recommendations for a

shared information system.

These recommendations were developed on the basis of interviews with end users and key

informants in four regions of the Commonwealth, workforce development system staff in five

states, and a literature review. The recommendations also reflect the Patrick Administration’s

key priorities for workforce development and incorporate findings and recommendations of the

August 2008 report of the Adult Basic Education/English for Speakers of Other Langugages

Committee of the Massachusetts Workforce Investment Board. The Executive Office of Labor

and Workforce Development and the Commonwealth Corporation provided staffing support to

the Performance Accountability SubCommittee, including conducting research activities.

Section II of the report provides an overview of end users’ objectives for the Commonwealth’s

workforce development system.

Section III outlines end users’ and key informants’ suggestions for investment or improvement.

Section IV provides information on practices and structures in other states that are relevant to

the objectives and suggestions for improvement identified by end users and key informants.

Section V addresses objectives, attributes, costs and benefits of a shared performance

management system.

Section VI provides recommendations related to governance and coordination.

Section VII provides recommendations related to a shared performance management system.

Appendix A provides information on the research methodology and information sources and

Appendix B provides examples of collaboration and coordination practices in the four regions

that support the objectives identified by end users and key informants.

2

II. End Users’ Objectives for the Commonwealth’s Workforce Development System

To identify objectives for the Commonwealth’s workforce development system, research staff

interviewed end-users1 representing the following groups: Low skilled adults who are

unemployed or underemployed, unemployed or underemployed immigrants, adults with

disabilities2, at-risk youth, and employers.

These individuals were identified by staff providing workforce development services in

community based organizations, career centers and community colleges in four workforce

regions: Brockton, Franklin Hampshire, North Shore, and Worcester. Research staff conducted

end user interviews with job, education, and occupational skills seekers primarily through focus

groups. Staff gathered employer input through one-on-one interviews and surveys.

Research staff asked end users to identify their goals for seeking and participating in workforce

development services. They provided the following responses:

• Low skilled adults sought services to improve their chances to get a job or find a better

job. To this end, they also identified the need to develop specific occupational skills and

to gain threshold educational credentials such as a high school diploma or GED. They

articulated a clear understanding that the jobs that provide economic opportunity

demand increasingly higher levels of academic and occupational skills.

• Unemployed and underemployed immigrants who were participating in ESOL services

viewed English-language proficiency not as the objective but as a means to an end –

preparation for work or for further education or skills training that would provide them

improved economic opportunity.

• At-risk youth reported their desire to finish their GED or gain some other education

credential that would allow them to gain access to post-secondary education. They also

reported that they were seeking the opportunity to develop a specific occupational skill or

1 The study was developed in the context of the Commonwealth’s skills gap and was specifically designed to address issues associated with governance and coordination. Therefore, the research did not include end-users who are using the workforce development system but are not likely to need significant assistance, either in finding work or in finding qualified workers, from multiple components of the system.

2 A small number of adults identified themselves as individuals with disabilities. Research staff did not conduct focus groups with any discrete groups of individuals who identified themselves as having a disability.

3

trade in order to find a job. Some reported that they were engaged in programs because

of the opportunity to secure an internship.

• A significant number of end users in all three groups voiced concern about their lack of

basic computer skills and identified this as a barrier to success. As businesses

increasingly use on-line job application and applicant screening technologies, job

seekers are finding that basic computer skills are essential, even in the job search

process.

• Employers were seeking assistance in finding applicants who will not only meet their

hiring qualifications but will be successful employees.3 The skill needs identified by

these employers ranged from entry-level4 to technical. Some employers also identified

the desire for assistance in developing a pipeline (or career pathway) to meet not just

their immediate hiring needs but their longer term workforce needs.

Research staff also asked end-users to identify features or services that are essential to their

success in achieving their objectives. They provided the following responses:

Job, education and skill seeking end users want:

programs that are accessible (location, cost, hours, ease of entry5) and staff who are

knowledgeable and can offer individual attention, and

someone to help them figure out how to make the best use of their time and resources.

Employer end users want:

program staff who are knowledgeable about their industry or company,

time and cost-efficient interactions, and

3 Employers defined successful employees in several ways, including retaining their employment with the firm and growing their skills and career with the firm.

4 The Commononwealth’s Executive Office of Labor and Workforce Development is investigating options for offering a “basic work readiness certificate” to job seekers. Several states (including Oklahoma and Georgia) have established mechanisms for certifying workplace basic skills.

5 Ease of entry included: no wait for service or for program start-up and no complicated enrollment or application paperwork or processes.

4

local and state program and system staff to communicate and coordinate services and

resources with each other.

Job, education and occupational skill seekers and employers all identified computer and

technological literacy as foundational skills and want the system to support their development of

these skills.

III. End Users’ and Key Informants’ Suggestions for Investment or Improvement

Research staff asked end users and key informants to suggest investments or improvements

that would increase the effectiveness of the workforce development system. They identified the

following:

Job, education and occupational skill seeker end users suggested (in addition to the system

features identified in Section II):

• more accessible information about choices and options for pursuing career paths and

help in planning the courses they need to take to achieve their career goal,

• better coordination between agencies and programs to tie together education, training,

and placement,

• staff with more knowledge about the courses and schools related to specific

occupational goals,

• use of volunteers to help expand staff capacity,

• more one-on-one assistance and tutoring,

• the opportunity to develop basic computer skills,

• assistance in preparing to fill out on-line job applications, and

• help with childcare, transportation, housing, and training and education costs

(scholarships and stipends).

Employer end users did not offer additional suggestions beyond those identified in Section II.

5

Managers and staff of local workforce development agencies and programs suggested:

more investment in staff training (related to labor market dynamics, experience and

training requirements for occupations, and information about community resources) and

in technology,

the authority and flexibility to manage funding streams (and eligibility and participation

requirements) from multiple state agencies that are all targeted for the same customer

group(s),

resources (workshops, materials) that help job and training seeking customers

understand occupations, skill requirements, and career paths,

facilitated opportunities for staff from local agencies and organizations to develop

working relationships and share information about resources and practices,

programs that provide participants with the opportunity to combine work and learning

over a longer period of time,

more investment in programs and services for at-risk youth and the authority to expand

eligibility requirements,

more resources for learning disability assessments,

expand use of public school classroom and training facilities on nights and weekends,

more training programs that are offered at night and on weekends,

more investment in GED and alternative diploma programs,

a distance learning6 entry point for individuals who are on a wait list for ESOL and pre-

GED, and

programs that integrate academic and occupational skills training.

IV. Other States’ Practices and Structures That Could Support End Users’ Objectives

6 The Department of Elementary and Secondary Education (Adult and Community Learning Services) is currently investigating distance learning models.

6

As part of the qualitative research study on governance and coordination staff conducted

a review of selected states to review how workforce development programs and

agencies are governed and how they coordinate delivery of services to their customers.

Highlights of observations from other states are included below and they broadly fall

under the four themes of (1) current structure of workforce development programs, (2)

recent history and plans for re-organization, (3) coordination of programs and services,

and (4) local and state roles.

Michigan

Structure. The major workforce programs are organized as follows:

The primary workforce agency is the Department of Labor and Economic Growth

(DLEG), which includes WIA activities, adult education and literacy, vocational

rehabilitation, apprenticeship, community colleges and Wagner Peyser labor

exchange services.

The Department of Education is responsible for secondary career and technical

education, which recently moved back from DLEG.

Michigan Economic Development Corporation is the primary funder of employer

based customized training.

Reorganization History and Plans. A few years ago Michigan re-organized under

Governor Engel to combine economic development and workforce development into

DLEG. Subsequently economic development was spun out as a quasi-governmental

agency – the Michigan Economic Development Corporation. In January 2008 a major

reorganization was announced within DLEG which re-oriented responsibilities around

two types of customers – services to workers and services to employers, as opposed to

the traditional alignment by funding source.

Coordination. The recent re-organization is aimed at better coordination of services to

the two types of customers. The services to employers functions combine

regional/sector strategies and employer training and recruitment needs. The services to

individuals functions combine lifelong learning, veterans services, acceleration of re-

employment and services to Spanish speaking and migrant workers.7

7 Michigan Department of Labor and Economic Growth, “State Announces New Bureau of Workforce Transformation Unprecedented re-design will drive change to a system of lifelong learning; Radically restructures workforce efforts.”Michigan

7

State and Local Roles. Historically Michigan has had significant decentralization, with

most WIA, employment services and TANF funding in the control of local workforce

boards and services being provided through one stop career centers. Community

colleges are also independent with significant local autonomy. With the recent No

Worker Left Behind initiative, the state is trying to maintain local flexibility and decision

making while pushing a broader statewide point of view. The state has historically kept

the WIA discretionary and incentive funds for new initiatives.

North Carolina

Structure. The major workforce programs are organized as follows:

Workforce development programs are in the Department of Commerce (Commerce),

where the Division of Workforce Development (DWD) manages the JobLink centers

(the state’s one-stop career centers), the WIA programs and incumbent worker and

sector projects and provides staff support to the North Carolina Commission on

Workforce Development, which is the statewide workforce board.

North Carolina Community College System (NCCCS), manages the adult basic

education/ESOL programs, post-secondary vocational/technical education,

customized training programs and career readiness certificate programs.

The Department of Health and Human Services (DHHS) manages vocational

rehabilitation programs;

The Employment Security Commission (ESC) provides labor exchange services.

The Department of Public Instruction manages secondary vocational and technical

education.

TANF services are offered through county departments of social services.

Reorganization History and Plans. The Division of Employment of Training was

integrated into Commerce in the late 1990s. The Division has since been renamed the

Division of Workforce Development. The Commission on Workforce Development was

previously in the Governor’s Office and was moved to the Department of Commerce.

Department of Labor and Economic Growth Press release, January 18, 2008; http://www.michigan.gov/dleg/0,1607,7-154--183738--,00.html (accessed October 20, 2008).

8

Coordination. The workforce development programs are well aligned with economic

development. Local workforce boards also tend to be aligned with local economic

development agencies. In 2007 the Commission on Workforce Development issued

revised chartering criteria for the state’s JobLink system, based on Baldridge Quality

Award principles, to ensure consistent and quality service throughout the state. The

charter provides the minimum standards for the system. The Department of Commerce,

through the Division of Workforce Development, has a memorandum of understanding

(MOU) with ESC, NCCCS, and DHHS to coordinate services through the state’s JobLink

system. Within DWD business services are coordinated as the incumbent worker

programs. The sector programs and the rapid response team are all part of the

business service team. The state workforce board (Commission) has played a very

active role recently in spearheading these coordination efforts.

State and Local Roles. While at the state level the Commission on Workforce

Development has defined minimum standards for the state’s one-stop career center

system, local workforce development boards manage the centers locally. The key state

agencies have memorandum of understanding on co-location, staffing, branding, cost

allocation, etc. and the local workforce development boards are required to execute a

local MOU. The local boards are responsible for coordination and for oversight locally

and have the flexibility and the autonomy to engage partners and stakeholders locally

based on local and regional needs.

Pennsylvania

Structure. Workforce programs are spread among seven cabinet level departments:

The Department of Labor & Industry (L&I) is the key workforce development agency

that manages WIA programs, CareerLink – the state’s one stop career center system

- vocational rehabilitation services and supports the state workforce board.

The Department of Public Welfare manages TANF.

The Pennsylvania Department of Education (PDE) includes secondary and post-

secondary career and technical education, community and technical colleges, and

adult basic and literacy education.

9

The Department of Community and Economic Development (DCED) funds WEDnet,

providing free training to qualified businesses.

Reorganization History and Plans. In 2003 Governor Rendell appointed a Deputy

Secretary for Workforce Development in the Department of Labor & Industry to

coordinate workforce strategy. The Deputy Secretary also serves on the Economic

Development Cabinet. In 2005 Governor Rendell launched Job Ready PA, an industry-

led training strategy involving industry partnerships to address skills gaps in 20 targeted

industry clusters. This initiative highlights how Pennsylvania has organized recent

workforce development efforts to meet the dual goals of employers and individuals

through a focus on sector strategies and the development of industry-led partnerships

with workforce development agencies. Job Ready PA and the focus on employer-led

partnerships are also seen as an opportunity to tie labor exchange services with

employer-led initiatives. While there is cabinet level coordination there is concern that

many of the programs still operate as silos within the individual agencies. To address

this issue the Deputy Secretary for Workforce Development and the state WIB recently

commissioned a study to provide an analysis of the state’s workforce system and

recommendations for improvement.8 The report’s recommendations include greater

systemic oversight through the elevation of the function of workforce development as a

separate office in the Governor’s office; raising system visibility, better integration and

coordination of state agencies by consolidating training and adult education resources

and activities into one agency and one fund; and improved local service delivery through

the statewide CareerLink system.

Coordination. The Deputy Secretary for Workforce Development chairs the inter-agency

Human Capital Task Force, which is a sub-committee of the Economic Development

Cabinet. Pennsylvania has recently also received multiple WIRED and industry

partnership grants that speak to the coordination of economic development with

workforce development and the important role of sector strategies. The linkage with

economic development has also helped to provide a focus for workforce development to

address the needs of employers and targeted industries. Industry partnerships are

required to work with the local WIB to ensure alignment with the strategic focus of the 8 Pennsylvania Economy League, State Office, “Restructuring Pennsylvania’s Workforce System: Lessons from Pennsylvania’s Stakeholders and Other States and Recommendations for Improvement.” The Pennsylvania State Workforce Investment Board, 2008.

10

WIB. Coordination with community and technical colleges and economic development

agencies is encouraged through the high-performance board initiative. There have been

recent efforts, motivated by cost pressures, to undertake cost-sharing of CareerLink

infrastructure costs between WIA, vocational rehabilitation, welfare and adult basic

education funding streams. As mentioned in the previous section, however, there is

ongoing concern that most programs continue to operate as silos and there is need for

greater coordination and consolidation both at the state level as well as in local service

delivery.

State and Local Roles. Pennsylvania has been cited as integrating at the local level

through policy, technical assistance and training provided to all local staff.9 The state is

also providing guidance to local WIBs through the High Performance WIB Standards.

While there have been improvements there is concern that at the local level there is an

“exceptionally large number of local partners with different points of view and missions”

that presents the local WIBs with a considerable challenge.10 The role of the local WIB in

the state’s one stop career center system is also not well defined and the existence of a

“variety of structures” for the local WIBs also acts as a barrier to efficient and

streamlined operations.

Washington

Structure. In Washington there are multiple cabinet level departments responsible for

workforce development programs:

The Superintendent of Public Instruction, who is responsible for the K-12 education

system, is directly elected.

The Employment Security Department (ESD) is responsible for employment

services, WIA, trade adjustment assistance, employment services for TANF, and

WIA.

The Department of Social and Health Services manages TANF and vocational

rehabilitation.

9 Melissa Mack, “Strategies for Integrating the Workforce System: Best Practices in Six States.” September 2006. Social Policy Research Associates, Oakland, CA.

10 Pennsylvania Economy League, State Office, “Restructuring Pennsylvania’s Workforce System: Lessons from Pennsylvania’s Stakeholders and Other States and Recommendations for Improvement.” The Pennsylvania State Workforce Investment Board, 2008. P 19

11

The Higher Education Coordinating Board coordinates universities and colleges.

The State Board for Community and Technical Colleges is responsible for adult

education and basic skills, post-secondary technical education, worker retraining and

literacy programs.

Re-organization History and Plans. The Washington Training and Education

Coordinating Board (WTECB), a powerful workforce board, was created in the early

1990s in response to a ‘blue ribbon’ commission to ensure consistent program outcomes

for performance accountability and coordination of service delivery. The state

administration at that time focused on coordination of programs and services through a

strategy of using WTECB to identify key performance measures while leaving local

operators the flexibility to develop local solutions in response to local needs. WTECB

oversees 18 programs; the board members include the Superintendent of Public

Instruction, the Commissioner of ESD, and the Executive Director of the State Board for

Community and Technical Colleges. The Executive Director of WTECB reports directly

to the Governor. WTECB plays a critical role in coordinating statewide workforce

development activities, evaluating programs, certifying local boards, and maintaining

eligible provider lists. Washington has also benefited from almost 15 years of ongoing

effort in performance accountability in workforce development across all government

agencies and programs.

Coordination. Effective coordination at the level of one stop career centers is taking

place in selected regions but not in all regions. Individuals typically get the services they

need, sometimes through co-enrollment or joint enrollment requiring coordination of

service delivery. For the recently announced strategic plan for workforce development, a

key objective is a “functionally integrated service delivery system that measurably

improves the employability of customers.”11 One of the key strategic opportunities for the

next two to four years, identified in the strategic plan, is improved coordination between

workforce development and economic development through industry clusters or industry

skills panels. Coordination is being driven by the workforce board, including through

inter-agency staff committees. There is also an executive order requiring cost sharing at

the one stop career centers among state agencies and programs.11 Washington Training and Education Coordinating Board, “High Skills High Wages: 2008-2018, Washington’s Draft Strategic Plan for Workforce Development,” (Olympia, Washington, 2008) http://www.wtb.wa.gov/Activities_HighSkills.asp (accessed, October 20, 2008)

12

State and Local Roles. About half the career centers are run by local WIBs while most

of the rest are operated by ESD. There is coordination at the state level through the

state workforce board and also through the application of common set of performance

and accountability measures across the major workforce programs. However

application at the local levels is not consistent across workforce regions.

Virginia

Structure. Virginia’s workforce development system is currently undergoing significant

redesign, which began in 2006. The Governor began by creating a position of Senior

Advisor on Workforce Development (later changed to the Chief Workforce Development

Officer). Subsequently (in early 2008) the Governor created a Workforce Development

sub-cabinet (see below). The Secretary of Education’s responsibilities include

secondary and post-secondary career and technical education. The Virginia Community

College System has recently assumed responsibility for one stop career centers, WIA

services and adult education/family literacy. The Secretary for Health and Human

Services is responsible for TANF and vocational rehabilitation services.

Reorganization History and Plans. The Chief Workforce Development Officer has been

leading efforts to review the current system and determine how it may need to be re-

organized. In 2006 the Governor issued an executive order to implement a Workforce

Development Strategic Plan with the primary goal of streamlining and integrating policy

and services for business, workers and job seekers. In 2008 the Governor issued

another executive order to establish “the Workforce Sub Cabinet ….to ensure Cabinet

collaboration regarding workforce development.”12 The sub-cabinet includes the Chief

Workforce Development Officer, the Secretaries of Administration, Commerce and

Trade, Education, Finance, Health & Human Resources and Public Safety, the Deputy

Chief of Staff, the Director of Policy, a staff member assigned to workforce from the

Governor’s Policy Office, the Deputy Workforce Advisor, the Superintendent of Public

Instruction, the Virginia Community College Chancellor, and the Executive Director of

12 Executive Order 61, Chief Workforce Development Officer Coordination of Workforce Development and Accountability,

Cross-Cabinet Collaboration and the Role of the Virginia Community College System at http://www.governor.virginia.gov/Initiatives/ExecutiveOrders/2008/EO_61.cfm.

13

the State Council of Higher Education. As part of the same executive order the Governor

has also given responsibility for WIA to the community colleges. The executive order

also requires the support of all agencies in the “development and implementation of a

seamless and integrated One Stop workforce service delivery system” and the creation

of a performance accountability system.

Coordination. The governor first appointed a Senior Advisor on Workforce Development

to serve as the Chief Workforce Development Officer on the Governor’s behalf and

coordinate all workforce development across Secretariats. As described in the previous

section, the Governor then created a Workforce Development sub-cabinet. A formal

memorandum of understanding has also been executed among the secretariats in the

workforce development cabinet regarding the participation of various state agencies in

the one stop career system and sharing infrastructure costs.

State and Local Roles. As a result of the executive order a formal memorandum of

understanding has been executed, giving local WIBs the responsibility for coordinating

the one-stop career center, but also mandating the participation of various state

agencies and funding streams in at least one one-stop career center in each WIB and

requiring the allocation and sharing of infrastructure costs.

14

Key Conclusions from Other States

Structure. In all states workforce development programs and services are distributed

across multiple cabinet secretariats and departments. At a minimum, each state has:

a workforce department that manages the one stop centers and WIA programs

and services,

an education department, with responsibility for K-12, secondary and post-

secondary career and technical education, and adult/family literacy, and

a department for human services or social services or public welfare, with

responsibility for TANF.

Workforce development is located within an economic department in only one of the

states we surveyed, North Carolina, where workforce development is part of the

Department of Commerce. Michigan had in the past combined workforce development

and economic development in one agency, but subsequently separated them. (The

Michigan economic development agency still manages incumbent worker programs.)

Washington has a goal to develop strong links between economic development and

workforce development. Many states are working with key industry clusters or

partnerships or panels for determining business education and skills needs and skill

gaps. Michigan, Pennsylvania and Washington all work with industry clusters or groups

to understand and address their workforce development needs.

Reorganization History and Plans. The focus of reorganization activity in the workforce

development system in various states is designed to address two kinds of issues:

multiple programs, spread across many state agencies, and

coordination of delivery services for key customers.

Recent proposals for reorganization in Pennsylvania and the recent efforts in Virginia are

designed to address coordination among multiple programs and agencies. Virginia has

recently reorganized by shifting WIA operations to community colleges and mandating

participation, presence and cost sharing by multiple agencies and programs at the one

stop career centers in each workforce region. Pennsylvania is in the process of

determining how best to coordinate workforce development programs and funding both

at the state and local levels.

15

At same time Virginia, together with Michigan, North Carolina, and Washington are

attempting to improve service delivery by focusing on individual needs and employer

needs. The individual is usually a job-seeker or an incumbent worker. Sometimes there

is also an explicit focus on a third segment - youth - to prepare them for work or college.

Michigan has recently reorganized agencies and staff responsibilities around serving

individuals and employers. It seems that the reorganization of state agencies and

programs does not necessarily result in better coordinated delivery of services.

Michigan, which has more workforce development programs in a single agency than

other states, is still grappling with how to break down silos within the single workforce

development agency and to coordinate service delivery to key customer segments.

Coordination. All states seem to be grappling with the challenges of coordinating multiple

state agencies and programs both at the state level and at the local level. Recent

reorganization efforts in Virginia and Michigan are motivated by improved coordination,

and Washington has an explicit goal of improving coordination as part of its ten year

strategic plan. No clear picture emerges about how best to “coordinate” the delivery of

services at the local level, or to coordinate or integrate programs and agencies at the

state level. Virginia recently developed a “workforce development” sub-cabinet;

Pennsylvania utilizes an interagency group.

Some states are focusing on the adoption of performance accountability standards and

consistent outcomes for training and education programs across various workforce

development and education agencies and programs. Washington, Virginia, and

Pennsylvania have explicit goals related to developing, improving and implementing

performance accountability through the establishment of common outcome measures.

In Washington the workforce board is responsible for coordinating workforce

development activities among various agencies/programs at the state level. The board

publishes performance reports on major workforce programs and also provides

comments and recommendations on various agency/program budgets to the Governor

and the legislature.

16

State and Local Roles. There is considerable variation in state and local roles, among

different states and also within a state among different agencies. A number of

coordination efforts are focused on state level agencies and programs, although

Virginia’s recent executive order requires coordination at the local WIBs and one stop

career centers, and cost sharing among various state agencies and programs in the

local one-stop career centers. Pennsylvania has negotiated cost sharing at the state

level and is exploring coordination both at the state level and locally. In North Carolina,

coordination at the state level among agencies is addressed through a memorandum of

understanding and also through defining minimum standards for local areas. However,

local areas and workforce boards determine the type and extent of coordination among

different agencies and programs at the local level. Washington has a strong workforce

board that leads coordination efforts at the state level and also makes extensive use of

performance data. Local boards have the responsibility of ensuring service delivery at

the local level. In summary, in order to support effective use of resources, many states

are understaking state-level coordination efforts among statewide agencies on broad

parameters, including resource allocation and performance management. At the same

time, states are still working to develop models that support local accountability for

coordinating services and local planning and decision making to ensure that resources

address local needs.

V. Objectives, Attributes, Costs and Benefits of a Shared Performance Management

System

The Performance Accountability SubCommittee developed the following objectives and

attributes for the Commonwealth’s workforce development performance management

system:

1. Supporting managers in local organizations and state agencies in:

understanding the impact of publicly funded programs,

making resource allocation decisions, and

aligning the investment of workforce development resources

2. Supporting local managers and staff in facilitating planning and service coordination

for customer groups that are served by multiple agencies and funding streams.

17

3. Providing state and local managers with timely information about progress toward

achieving program goals, such as skill gains, credential acquisition, enrollment in

postsecondary education, employment, employment retention, placement wages, or

earnings increases.

4. Supporting trend analysis to identify the need for redirection of resources or new

investments.

5. Facilitating analysis to support program improvement, including analysis of program

impact and effectiveness for specific populations such as at-risk youth, low skilled

unemployed or underemployed adults, or persons with disabilities. Understanding

and tracking the experience and outcomes for customers who are served by multiple

programs either simultaneously or over a several-year period.

6. Aligning goals across the workforce development system to facilitate a clear

understanding of the objectives for investing in programs and services.

Program staff conducted a review of the workforce development performance management

systems in the states of Florida and Washington. These states were selected because they

offer two substantially different models that address many of the objectives and attributes

identified by the Performance Accountability Sub-Committee.

The following describes the features and costs of each of these state’s performance

management system.

Florida’s Performance Management System

Features

The program is administered by the Florida Education and Training Placement Information

Program (FETPIP) which is part of the Florida Department of Education. FETPIP’s location

within the Department of Education partly addresses concerns related to the Family

Education and Privacy Rights Act (FERPA) because this allows information on individuals

enrolled in education programs to remain within the Department of Education. FETPIP staff

18

has developed protocols to meet FERPA requirements, which allow for the use, analysis

and reporting of aggregate data for the purpose of program improvement and research.

Individual records are not released unless a waiver is obtained from the Department of

Labor. The assessment of whether the waiver is applicable is made by a “Data Request

Review” team comprising staff of the Florida Department of Education.

The program collects and analyzes data about individuals completing or exiting the following

components of the workforce development system:

Universities

Community colleges

School districts

Selected private vocational schools, colleges and universities

Welfare Transition Services

Workforce Investment Act (WIA)

Corrections System

Farm Worker Jobs and Education programs

Specialized and longitudinal studies

A Florida statute (section 1008.39) requires programs and agencies (including independent

or private colleges and universities) to provide the data to FETPIP so that it may report on

outcomes. The purpose of the analyses is to disseminate information concerning the

educational histories, placement and employment, enlistments in the United States armed

services, and other measures of success of former participants in state educational and

workforce development programs. A key purpose is to facilitate comparison of “Employment

and/or education outcomes of a training program to others” 13 in the following manner:

13 http://www.fldoe.org/fetpip/method.asp FETPIP Methodology, “What is the Information Used For”, [Florida Department of Education, Florida Education & Training Placement Information Program (FETPIP), not dated], http://fldoe.org/fetpip/method.asp (accessed October, 17, 2008)

19

1. “Employment results can be examined in terms of the training programs that feed

them.

2. Program outcomes can be compared by race, sex, age or income level.

3. Earnings can be compared across various education levels

4. The level of public assistance can be compared between graduates, dropouts and

others.”

There are four major elements in the analysis:

1. Program records of individual participant “exiters” are provided once a year (quarterly

for selected programs).

2. Participant program records are matched with employment (wage) records from

Florida, from other states covered by the Wage Record Interchange System (WRIS),

and from federal employment records. This analysis provides data on employment

and earnings outcomes; FETPIP makes estimates of hourly wages.

3. Participant program records are matched with enrollment data from public school

district postsecondary programs, community colleges, all public four year higher

education institutions, and all private colleges and universities.

4. Participant program records are matched with data from the Department of

Corrections (for those incarcerated or under community supervision) and with

records of individuals receiving public assistance (TANF and Food Assistance).

Other purposes include performing analyses to support state and federal accountability and

outcome information and to better prepare students and counselors of career information.

Additional analyses may be requested by any of the state agencies/programs or also by private

researchers. An annual report on outcomes is published by FETPIP every year. The most

recent published report is the annual outcomes report, published in December 2007, covering

Fall 2006 data.

Costs

The annual operating costs of FETPIP are $600,000 and are provided through general

appropriation funds. The statute requires cost recovery for access to Florida employment

20

(wage) records, which requires FETPIP to reimburse the Florida Department of Labor for the

use of employment records. Other data sets are provided free of cost. FETPIP has 3 positions

funded through the state’s workforce agency (Agency for Workforce Innovation). The remainder

are funded by DOE with two positions supplemented by Perkins funding. Equipment and/or

upgrades are paid for in proportion to the funding of positions.

Washington State’s Performance Management System

Features

The performance management system is administered by the Washington Training and

Education Coordinating Board (WTECB) which is the state’s workforce investment board. The

12 largest workforce programs are included.

The purpose of the performance management system is to “report the results of workforce

development and to recommend areas for improvement.”14 There are seven “desired

outcomes” for the state workforce development system established by WTEC:

1. Employment of former participants in the third quarter after leaving the program for

adult programs,

2. Employment or enrollment in further education in the third quarter after leaving youth

programs,

3. Median annualized earnings of former participants in the third quarter after leaving

the programs,

4. The percentage of program participants achieving the appropriate skills against a

desired education or skill credential,

5. Employer satisfaction with participants and participant satisfaction with programs and

services, and

6. Return on investment to the taxpayer due to the net impact on tax revenues and

social welfare payments, and return on investment to the participant due to the net

impact on participant earnings and employer provided benefits.

14 Washington State Workforce Training and Education Coordinating Board, “2006 Workforce Training Results,” Olympia, WA.

21

The following are the major elements of the analysis:

Program records of individual participants leaving one of the 12 programs during a

particular program year.

Participant program records are matched with employment (wage) records from

Washington, Oregon, Idaho and federal employment records. This analysis provides

data on employment, and earnings outcomes for participants who are employed in

organizations covered by these employment data. Wage record analysis is contracted

out every 5 years. The current vendor is the state Community College data center.

A Telephone Survey of a sample of participants in the 12 workforce development

programs is conducted every 2 years to get information on services received as well as

to get information on employment and earnings. The current vendor is Washington State

University.

A Telephone Survey of employers who hired new employees from one of the 12

workforce development programs is conducted every 2 years. The most recent vendor

was the Northwest Research Group.

Program participant records are matched with enrollment data from community and

technical colleges, all public four year higher education institutions, private career

schools licensed by the workforce board, and apprenticeships overseen by the

Department of Labor and Industries. This includes record matches with the National

Student Loan Clearinghouse for out-of-state enrollment in higher education. As of this

date, private in-state four year institutions and out-of-state institutions are not included.

A Return on Investment Study is conducted every 4 years. This study is contracted to

the Upjohn Institute in Kalamazoo, MI. In this study net impacts (earning and benefits)

to participants are estimated for the period since they left the program as well through

age 65 by comparing their outcomes with similar individuals who did not participate in

the programs. Also estimated are increased tax revenues due to increased participant

earnings and compared to program costs.

A bi-annual report of workforce development system results incorporating the results of

all elements listed above and an analysis and interpretation of the results is prepared by

the staff of the Workforce Board.

22

Costs

Analysis of program records (every year, for completion and credentials) is undertaken

by each relevant agency.

Matching records with National Student Loan Clearinghouse records = $15,000.

Matching program records with employment (wage) records = $70,000 (for state report).

Agencies pay on a per participant (or social security number) basis for any additional

analysis that they request.

Participant survey (every two years) = $180,000

Employer survey (every two years) = $150,000

ROI study (every four years) = $270,00015

In addition there is a cost of 2-2.5 FTE incurred by the Workforce Board for managing

these various contracts and writing the bi-annual report.

Conclusions

It is clear from the details of the system in Florida and Washington that two different strategies

are at work but both are based on the key principle of using employment or wage records as

well as post-secondary education records to determine two key outcomes:

1. whether participants leaving any programs have gained and retained employment, and

at what earnings level, and

2. if they have enrolled in further education.

These outcomes can be consistently measured across programs and agencies and different

user groups by matching programs’ administrative records with

employment or wage records, and

higher education enrollment records.

Florida has chosen the path of undertaking all the data matching in a single agency and leaving

the interpretation of the data and analysis to the individual agencies or programs. Washington

has consolidated the data matching operation with a single vendor, but also focused on

additional measures such as:

15 The true cost of the analysis is likely to be higher than this, as it is likely subsidized by the Upjohn institute.23

customer satisfaction – both for individual participants and for employers,

net impact analysis to determine benefit to individuals, and

return on investment for both the individuals and for the public.

Washington also utilizes the WTECB staff to provide detailed analysis and interpretation of data.

Washington is also using additional regression models to understand and explain variation

among the target populations and regions.

Florida, while it provides limited analysis, has extensive coverage of different types of

employers, including federal and state employment, and of different types of post-secondary

institutions – including private colleges. Florida also tests for incidence of incarceration and for

use of public assistance in different groups.

24

VI. Recommendations Related to Governance and Coordination

These recommendations were developed on the basis of interviews with end users and key

informants in four regions of the Commonwealth, workforce development system staff in five

states, and a literature review. The recommendations also reflect the Patrick Administration’s

key priorities for workforce development:

Closing the Skills Gap

Enhancing the Youth Pipeline

Building the Capacity of the Workforce Development System

and incorporate findings and recommendations of the August 2008 report of the Adult Basic

Education/English for Speakers of Other Langugages Committee of the Massachusetts

Workforce Investment Board.

1. Create a coordinating body with state-level policy making authority to integrate state and

federal funds that support locally provided workforce development services.

(Alternatively, expand the mission and membership of the coordinating body

recommended by the MWIB in the August 2008 Report of the Adult Basic

Education/English for Speakers of Other Languages Committee.)

To be co-chaired by the Secretary of Labor and Workforce Development and the

Secretary of Education, with required participation from the Departments of

Transitional Assistance, Elementary and Secondary Education (including ACLS

and Vocational Education), and Higher Education, and the Massachusetts

Rehabilitation Commission.

The objective would be to support integrated budgeting and service planning at

the local level, facilitating local decision-making, service design and

accountability that reflect local needs, delivery systems and labor market

dynamics while addressing federal performance and service requirements.

Begin by piloting integrated budgeting and service planning in regions in which

the workforce investment board has achieved certification as a High Performing

Workforce Investment Board. The Workforce Boards would play a convening

role but local or regional stakeholders (such as DTA office directors, DOE-funded

25

ABE providers, Career Center directors, business leaders, labor organizations,

community college presidents, school superintendents, local government) must

participate fully in the planning process and commit to implementation. Regions

would be required to integrate local workforce development resources (such as

WIA vouchers or CDBG funds).

Local plans would include identification of targeted sectors and occupations that

provide opportunity for employment and advancement and would address how

the local and state resources would be used to prepare targeted populations for

these opportunities.

2. Conduct pilot process in 4 regions to streamline eligibility requirements and processes

for “high need” end users served by key funding streams.

High Performing Workforce Board regions would be invited to apply to participate

in the pilot.

Establish a team in each region made of up state level staff and staff of local

workforce development organizations and agencies. The teams’ objective would

be to identify opportunities to combine or streamline eligibility requirements and

process. Local staff would lead the process, but would have state-level

commitment to address restrictions and requirements as needed.

3. Provide local and state managers and staff with professional development in the

following areas:

Local labor market supply and demand trends

Career paths, training and eligibility requirements for demand occupations

Other local and state agency program resources

4. Develop resources for job, education and skills training seeking customers that provide

accessible information about demand occupations, skills and training requirements, and

career paths.

5. Support local managers and staff (adult basic education providers, youth vendors,

workforce investment board staff, career center directors, DTA office directors, skills

26

training vendors and other workforce development system organizations) in developing

performance management analysis capacity.16

6. Develop a prioritized resource investment plan.

Identify priorities for future investments, redirected resources, or leveraging

underutilized resources.

7. Conduct a technology assessment and develop a technology investment plan for all

components of the workforce development system.

16 Several agencies have invested or are currently investing resources to support managers in conducting program and performance analysis. Examples include the Department of Elementary and Secondary Education’s investment in COGNOS, the Executive Office of Labor and Workforce Development’s investment in wage record analysis, and the Department of Higher Education’s investment in National Student Clearinghouse records analysis.

27

VII. Recommendations Related to Developing an Integrated Performance Management

System.

In the current budget environment, it is clear that the level of initial and ongoing investments

made in Washington and Florida to support integrated performance management is not feasible

in Massachusetts. However, there are some steps that can be undertaken now to prepare for a

time when there may more resources available for this purpose. These recommendations are

designed to provide a framework for future investments that will achieve the objectives identified

by the Massachusetts Workforce Investment Board Performance Accountability SubCommittee

for the Commonwealth’s workforce development performance management system:

1. Supporting managers in local organizations and state agencies in:

understanding the impact of publicly funded programs,

making resource allocation decisions, and

aligning the investment of workforce development resources

2. Supporting local managers and staff in facilitating planning and service coordination for

customer groups that are served by multiple agencies and funding streams.

3. Providing state and local managers with timely information about progress toward

achieving program goals, such as skill gains, credential acquisition, enrollment in

postsecondary education, employment, employment retention, placement wages, or

earnings increases.

4. Supporting trend analysis to identify the need for redirection of resources or new

investments.

5. Facilitating analysis to support program improvement, including analysis of program

impact and effectiveness for specific populations such as at-risk youth, low skilled

unemployed or underemployed adults, or persons with disabilities. Understanding and

tracking the experience and outcomes for customers who are served by multiple

programs either simultaneously or over a several-year period.

28

6. Aligning goals across the workforce development system to facilitate a clear

understanding of the objectives for investing in programs and services.

The recommendations also reflect the Patrick Administration’s key priorities for workforce

development:

Closing the Skills Gap

Enhancing the Youth Pipeline

Building the Capacity of the Workforce Development System

1. File legislation that enables the use of educational records from secondary and post-

secondary systems for the purpose of research and program improvement. The

legislation could also explicitly require workforce development programs to share data

and track key outcomes. One of the key purposes of this legislation would be to address

concerns related to the Family Education and Privacy Rights Act (FERPA).

2. Consider expanding the use of wage records to determine employment, employment

retention, and earnings outcomes for participants in all publicly funded workforce

development programs and educational programs at the secondary and post-secondary

level. This function could be provided by the Department of Workforce Development,

another state agency, or could be contracted to a vendor.

3. Explore developing the capacity to match and share records of “high need” individuals

(those served by multiple programs and funding streams, either simultaneously or over

several years) to facilitate improved coordination of service delivery and impact analysis.

This would include a protocol for addressing privacy and confidentiality concerns.

4. Explore developing the capacity to periodically (as in the case of the State of

Washington) undertake a cost benefit analysis based on net impact analysis, which in

turn, would be based on the use of employment (wage) records to determine participant

earnings.

5. Develop cost projections for initial investment and ongoing costs of an integrated

performance management system. Consider, at a minimum, the following key costs:

29

The cost of managing data matching between program administrative records and

employment (wage) records in Massachusetts and from other states and federal

employment records to determine employment and earnings of participants.

The cost of data matching between basic education, employment and skills training

programs and post-secondary education institutions to determine the number and

percentage of participants going on to and completing post-secondary education.

The cost of conducting customer satisfaction surveys of program participants and of

employers.

30