OSHPD WET 5-Year Plan: 2013 Needs Assessment & 2008-2013 Evaluation Initial Findings Resource Development Associates December 13, 2013 Kevin J. Wu, MPH

OSHPD WET 5-Year Plan:2013 Needs Assessment & 2008-2013 Evaluation Initial Findings

Resource Development Associates

December 13, 2013Kevin J. Wu, MPH

Linda A. Hua, PhD

2

Topics

County-Reported Assessments Educational Capacity Program Evaluation Forthcoming Q&A

3

Highlights

County-Reported Assessments

4

County-Reported Assessments:Overview

Purpose: To identify key needs in the public mental health workforce including: Users of Public Mental Health System Shortages Hard-to-Fill, Hard-to-Retain Declining Needs/Needs Met Diversity Needs Language Needs State Administered WET Program

Need/Participation Analyses by:

Statewide MHSA WET Region County Size

5

County-Reported Assessments:Data Sources

OSHPD-led 2013 County-Reported Needs Assessment

MHSA Annual Updates OSHPD-led Community Forums, Surveys,

Key Informant Interviews RDA-led County Needs Follow-Up Survey

6

County-Reported Assessments:Hard-to-Fill & Hard-to-Retain Positions

Clinical Nurse Specialist

Psychiatrist, Child/Adolescent

Psychiatric Mental Health Nurse Practitioner

Marriage and Family Therapist

Licensed Clinical Social Worker

Psychiatrist

0 5 10 15 20 25 30 35 40 45 50

8

7

12

19

22

23

6

1

2

24

Highest Need Other Needs

Count of Reported Hard-to-Fill, Hard-to-Retain

Occu

pa

tio

na

l C

ate

go

ry

7

County-Reported Assessments:Workforce Shortages

Substance Abuse Counselor


Psychiatric Mental Health Nurse Practitioner



Psychiatrist

0 5 10 15 20 25 30 35 40 45 50

13

4

17

24

23

17

11

2

4

30


Count of Workforce Shortages

Occu

pa

tio

na

l C

ate

go

ry

8

County-Reported Assessments:Diversity Needs

Socio-Economic Status

Physical/Mental Abilities

Sexual Orientation

Age

Gender

Other

Language

Race/Ethnicity

0 10 20 30 40 50 60 70 80

2

3

2

12

34

4

10

17

17

17

26

24

37


Count of Reported Diversity Needs

Typ

e o

f D

ive

rsit

y N

ee

d

9

County-Reported Assessments:MHSA WET Program Participation Rates by County Size

Stipend Program

Physician Assistant Residency Program

Psychiatric Residency Program

Working Well Together

MHLAP 0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

11%

0% 0%

56%

69%

36%

0%

27%

73% 73%

58%

42% 42%

83% 83%

Small Medium Large

Pe

rce

nt

of

Cou

nti

es

County SizesSmall (<200K people)

Medium (200-800K people)

Large (>800K people)

10

County-Reported Assessments:Key Findings – Public Mental Health Users Users of the public

mental health system are predominantly located in the southern part of the state, more likely to be of a minority race/ethnicity, and more likely to be adults.

11

County-Reported Assessments:Key Findings – Public Mental Health Workforce Needs Counties’ identification of

their workforce shortages is consistent with their hard-to-fill, hard-to-retain positions.

Psychiatrists are of highest need in the public mental health system, but educational graduation information regarding psychiatrists has not been available.

The Superior Region, small counties, and medium counties identified persons with bilingual capabilities as workforce shortages and hard-to-fill, hard-to-retain positions more often than any other MHSA region or large counties.

Most counties did not identify any declining workforce needs or needs met, except for the Southern Region counties.

Of those counties that identified declining workforce needs or needs met, non-licensed mental health staff members were most frequently noted.

Workforce race/ethnicity diversity needs identified by the counties coincide with the race/ethnicity compositions of the public mental health user populations across the state’s MHSA Regions.

12

County-Reported Assessments:Key Findings – Public Mental Health Workforce Needs Workforce language

diversity needs identified by the counties are reflective of the race/ethnicity compositions of the public mental health user populations across the state’s MHSA Regions.

The Southern Region (including Los Angeles County) has a workforce that is largely meeting the language needs of its Hispanic/Latino public mental health system user population.

Counties’ designated positions for consumers and/or family members are largely reserved for peer and administrative/clerical positions, not provider/professional positions, and do not come with set wages.

13

County-Reported Assessments:Key Findings – Statewide-Administered WET Programs

Statewide-WET programs are mostly being utilized by large counties.

The WET Stipend Program’s participation across the MHSA regions reflects the state’s distribution of graduates with mental health-related degrees or certificates.

The WET Psychiatric Residency Program is underutilized compared to the counties’ desire for psychiatrists.

The WET Mental Health Loan Assumption Program (MHLAP) is utilized by most counties, especially the large ones.

The WET Physician Assistant Program is being utilized in few counties throughout the state, but is consistent with the low rate of physician assistant graduates being produced across the state.

Los Angeles County has the highest number of graduates with mental health-related degrees or certificates, but it does not participate in the WET Psychiatric and Physician Assistant Residency Stipend Programs.

14

County-Reported Assessments:Limitations

Variability in consistency of County-Reported Assessments data

Burden of data collection and reporting on county agencies

Unclear if counties reported on behalf of contractors as well

15

Highlights

Educational Capacity

16

Educational Capacity:Overview

Purpose: To identify the pipeline of future mental health providers

17

Educational Capacity: Data Sources

Integrated Postsecondary Education Data System (IPEDS)

California Postsecondary Education Commission (CPEC)

OSHPD-led Educational Institutions Survey

18

Educational Capacity:Graduates in All Mental Health Disciplines Projections

1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 20090

5000

10000

15000

20000

25000

f(x) = 124.272727272727 x + 3498.54545454545R² = 0.934080528842549

f(x) = 535.090909090909 x + 10573.1818181818R² = 0.940893740904704

f(x) = 660.672727272727 x + 14074.5090909091R² = 0.946974700976847

Male Graduates Linear (Male Graduates) Female GraduatesLinear (Female Graduates) Total Graduates Linear (Total Graduates)

2014

19

Educational Capacity:Distribution of Mental Health Graduates by Discipline (1999-2009)

Clinical Psychology7%

Counseling Psychology

4%Educational Psychology

0%Marriage and

Family Therapy2%

Other2%

Psychiatric Technician

2%

Psychology63%

School Psy-chology

1% Social Work 13%

Substance Abuse/Addiction Counseling

4%

198,424 total graduates

between 1999-2009

20

Educational Capacity:Distribution of Schools and Graduates

Educational Institutions Graduates

21

Educational Capacity:Limitations

IPEDS data has incomplete reporting CPEC student pipeline data is truncated

at 2009 Low response rates to RDA’s educational

institutions survey

22

Highlights

Program Evaluation

23

Program Evaluation:Approach

Evaluated program impact by actions and direct outcomes: General Capacity Cultural & Linguistic Competency Formal Education Structure & Curricula People with Lived Experience in the Workforce Filling Gaps in the Five Regions

Data gathered via multiple methods: Baseline: 2008 needs assessment Progress: 2013 progress reports Impact: 2013 needs assessment & RDA

interviews and county survey

24

Program Evaluation:General CapacityHardest-to-fill/retain

positions in 2008# MHLAP

Participants

Relevant State-Administered WET Programs

# Particip

antsTotal #

2009-2013Psychiatrist, General 228 Psychiatric

Residency25 253


884 Stipend 1,838* 2,722


1,251 Stipend 474 1,725

Licensed Supervising Clinician

0 -- 0 0


0 Psychiatric Residency

Unknown --

Registered Nurse 0 -- 0 0CEO or Manager above Direct Supervisor

0 -- 0 0

Psychiatric or Family Nurse Practitioner

1 Stipend 92 92

Licensed Clinical Psychologist

0 Stipend 232 232

Analysts, tech support, quality assurance

0 -- 0 0

Family Member Support Staff

0 WWT Unknown Unknown

Clinical Nurse Specialist 0 -- 0 0Psychiatrist, Geriatric 0 Psychiatric

ResidencyUnknown Unknown

Consumer Support Staff 9 WWT Unknown UnknownPositions not identified as hard to fill/retain in 2008

Physician Assistant 0 Song Brown 1,382 517TOTAL SERVED: MHLAP:

2,363Stipend & Residency: 1,680

Overall: 4,043

*Includes Stipend Program for Social Work participants from beginning of program in 2005.

25

Program Evaluation:General Capacity

State-Administered WET Program n

Average

Rating

% Rating the Program Somewhat or Very

EffectiveMHLAP 26 3.42 92%Social Worker Stipend Program

21 2.67 67%

MFT Stipend Program 20 2.55 70%PMHNP Stipend Program 17 2.24 53%Psychiatric Residency Program

19 1.68 26%

Clinical Psychologist Stipend Program

17 1.59 18%

Song-Brown Residency for Physician’s Assistants

12 1.08 0%

Overall -- 2.32 --

Counties were asked to rate the effectiveness (impact) of each of the state-administered programs in meeting their WET needs

Scale: 1 (not at all effective) - 4 (very effective)

26

Program Evaluation:Cultural & Linguistic Competency

Baseline 2008 Progress 2013Licensed Direct

% Difference from target

Licensed Direct

% Difference from Target

Caucasian/White 6,938 54% -20.6% 15,650 51% -18.00%Hispanic/Latino 2,417 19% +19.0% 6,558 21% +16.70%African American/ Black 1,072 8% +9.0% 2,538 8% +9.30%

Asian/Filipino/ Pacific Islander 1,342 10% -3.4% 3,252 11% -4.00%

American Indian/ Alaska Native 110 1% +0.1% 261 1% -0.10%

Multiple/Other 1,076 8% -4.2% 2,637 8% -3.90%Total 12,955 100% -- 30,896 100% --

27


Non-English Language

Number of public sector staff

needed in 2008

Adjusted estimate of need

Number served by statewide

WET programs

% of need potentially

met by statewide

WET programs

Spanish 6,092 12,671 1,708 13%Chinese

513 1,067109

10%Other Asian 974 20,26 283 14%Other 221 460 319 69%Total 7,800 16,224 2,419 15%

Linguistic Capacity of 2008 Workforce and WET Progress toward Targets

28


Program Impacts

n

Not at all

effective

Somewhat

effective

Very effectiv

e Don't know

How effective have state-administered WET programs been in increasing the cultural and linguistic competency of the workforce in your county?

26 12% 50% 35% 4%

How effective have state-administered WET programs been in increasing the diversity of the workforce in your county so that the workforce is more representative of the population served in terms of ethnicity, cultural tradition, religion, LGBT identification, etc.?

26 27% 39% 19% 15%

29

Program Evaluation:Formal Education Structure & Curricula

Institutions reported many courses that highlighted Use of evidence-based practices Cultural competency across gender, race,

religion, sexual orientation, etc. Principles of wellness and recovery among

adults and resiliency among youth Several Regional Partnerships, in

collaboration with local colleges and agencies, are developing PMH professional core competencies

30

Program Evaluation:People with Lived Experience

Working Well Together has developed the following: Assessment tools for agencies to recognize where

they are in terms of engaging consumers and family members in the workforce,

Curricula for training individuals who identify as consumers and family members, and trainings to preparing the workforce for employing consumers and family members

Toolkit for recruiting, hiring and retaining employees with lived experience within the public mental health workforce,

White paper on how to successfully employing people with lived experience within public mental health

Peer Certification standards and recommendations for a statewide plan

31

Program Evaluation:People with Lived Experience

Working Well Together Accomplishments:

62 Networking calls and Webinars combined

Region TA Visits Trainings Participants

Central 39 6 159

Southern 40 13 343

Superior 29 7 79

Bay Area 51 8 104

Total 159 34 685

32

Insert matrix

Program Evaluation:Regional Partnerships Evaluation

32

Regional

Partner-ship

Cultural

competenc

y trainin

gs

Curric. focuse

d efforts with

colleges

High school mental health career pathw

ay

Core competencie

s projec

t

Move towar

d recove

ry focus

in WET

Programs

targeting the under-served

Explicit

stigma

reduction

efforts

First respon

der training and MH First Aid

trainings

Central

X X X X X X X X

Greater Bay Area

X X X X X

Los Angeles

X X X

Southern

X X X X X

Superior

X X

33

Program Evaluation:Recommendations for Future Practice

Ensure that program offerings correspond to current workforce needs

Ensure that the pipeline to employment is considered so that programs can succeed in placing all graduates in the public mental health system workforce

Ensure a strategic and consistent approach to consumer and family member workforce development

34

Program Evaluation:Recommendations for Future Evaluation

Track participation consistently Track LGBT and consumer/family

member identification Track progress of Regional Partnerships

with more consistent tools Monitor curricula with checklist

35

Program Evaluation:Limitations

Baseline Data Limitations: Only 28 counties included in baseline report Cannot disaggregate (e.g., by county,

rural/urban/suburban, etc.) Very limited in baseline information pertaining to

two outcome areas: Formal Education Structure and Curricula and Filling Gaps in Five Regions

Progress Data Limitations: Inconsistent reporting from programs No consistent measures for looking at progress in

Formal Education Structure and Curricula and Filling Gaps in Five Regions

36

Program Evaluation:Limitations

Impact Data Limitations: Voluntary participation resulted in poor

response rate Baseline survey data were only available for

28 counties Impact survey data were only available for

26 counties Only 12 counties were represented in both

baseline and impact county lists 16 counties for which no data were collected at

all

37

Program Evaluation:Strengths of the Programs

The vast majority of the work that the statewide-administered WET programs have been conducting aligns with needs identified at the beginning of the process

There are significant program components in place to build the general capacity of the workforce

Programs have been very successful in recruiting and engaging people of color and people with non-English language proficiency

Curriculum changes have been noted that demonstrate alignment with MHSA principles

Trainings, TA visits and webinars focused on increasing the participation of people with lived experience have been offered to counties in all regions

The Regional Partnerships have engaged in work specific to the needs of their regions

38 Forthcoming

39

Forthcoming Work

Workforce Projections Integrating controls to identify most important

influences in supply trends Acquiring and analyzing demand-side data to

identify trends amongst consumers of public mental health services

Gap Analysis Identify any anticipated shortages and surpluses

over the next five years Occupational type, prescribing authority, location,

demographic needs (where data is available)

40

Workforce Projections:Data Sources

Supply: Current Workforce National Provider Identification (NPI) Data California Professional Boards

Demand for Public Mental Health Services Client & Service Information (CSI)

41

Workforce Projections:Limitations

No perfect data source with exact counts of individuals currently in public mental health workforce

Limited number of variables Time-intensive process of accessing

Client Service Information (CSI) data

42

Strengths of Overall Project

Reviewed extensive amount of county-reported data

Utilizing multiple data sources to inform needs assessment, evaluation, and projections

OSHPD support in prioritizing best possible data sources

43

Thank you!

Questions?

Documents

OSHPD WET 5-Year Plan: 2013 Needs Assessment & 2008-2013 Evaluation Initial Findings Resource Development Associates December 13, 2013 Kevin J. Wu, MPH