Embed Size (px)
Citation preview
3/6/2014
1
Improving Wraparound Facilitator
Competency through Observation
and Coaching: Challenges and
Lessons Learned
Amy Neumeyer, MPH
Monica Hampton, LMSW
Goal of this Presentation
• Share our experience with developing and implementing an observation competency tool that measures the skills of the facilitator through shadowing
2
3/6/2014
2
Detroit Wayne Mental Health Authority
• Division of Wayne County Department of Health and Human Services that manages specialty services for consumers with or at risk for serious emotional disturbance (SED), severe mental illness (SMI), developmental disabilities (DD), substance abuse, and MIChild beneficiaries.
• 18 Community Mental Health (CMH)providers serve Children
• 10 Providers offer Wraparound
3
Organization of Service Delivery
Detroit Wayne Mental Health
Authority (DWMHA)
Manager of Comprehensive
Provider Network (MCPN)
Providers Providers
Manager of Comprehensive
Provider Network (MCPN)
Providers Providers
4
3/6/2014
3
Wayne County, Michigan
5
Represents location of provider agency within Wayne County
Children Served through DWMHA
6–12 36%
13-17 56%
0-5 9%
AGE
ETHNICITY
AFRICAN AMERICAN/BLACK
CAUCASIAN/WHITE
OTHER
HISPANIC/LATINO
59%
34%
7.5%
4%
Female37.6%
Male 62.4%
GENDER
14,777
15,371 15,381
15,906
16,335
13,500
14,000
14,500
15,000
15,500
16,000
16,500
FY08-09 FY09-10 FY10-11 FY11-12 FY12-13
CHILDREN SERVED BY FISCAL YEAR
6
3/6/2014
4
Wraparound Client Characteristics 551 youth served FY 12-13
Female40.6%
Male 55.4%
*4% missing
50% of parent/caregivers were unemployed when youth began services 45% of youth live in single parent homes 5.1% of youth live in foster homes
21.4 20.6
8.5
18.4 17.1
6.0
3.5 2.6
18.5
15.6
7.7
16.6 16.0
3.9 3.5 1.8
0
5
10
15
20
25
School Role Home Role Community Role Behavior toOthers
Moods/Emotions Moods/Self Harm Substance Abuse Thinking
Sco
re (
0-3
0)
Subscale
Average CAFAS Subscale Scores at Intake
WA
AllProgs
Average Age: 12.8 years
7
Wraparound Supervisory Hierarchy
8
DWMHA WA
Coordinator WA
Supervisors Facilitators Families
1 10 34 551
3/6/2014
5
9
Problems
• Wraparound was not being implemented the same way at all 10 provider agencies
• Auditor concerns
• Outcomes were varying by agency and Facilitator
• Families exiting from services without proper transition
▫ Only 27.2% have successful transition (2012)
10
3/6/2014
6
Outcome Data Average: Successful Grads Other Exit
Total Initial 105 111.43
Total Exit 52.5 99
Score Diff 52.5 10
Improvement- Any Indicator
87.5 39.3
No Severe 75 17.9
Clinical Meaningful 62.5 28.6
(Fiscal Year 2012-2013)
Behaviors- Type of Exits Successful Exit Other Exit
Initial Exit Initial Exit
Has NOT used drugs 76.9 89.5 65.5 71
Has NOT hurt themselves on purpose
94.1 100 81.6 78.1
Has NOT threatened to hurt self/others
69.2 73.7 55.2 45.2
Has NOT hurt others on purpose
83.3 85 57.9 60
Charged with Crime 0 3.4 2.6 6.3
Probation Violation 0 0 15.8 25.8
Outcome Data
(Fiscal Year 2012-2013)
3/6/2014
7
Problems
• High staff turnover
• No formal standard training for Facilitators
▫ 3 day training in Lansing
▫ Any additional training depends on agency, supervisor
• Varying support from Supervisors
▫ Supervisor responsibilities range from only supervising Wraparound to Children’s Program Manager
13
Problems
• Fidelity Monitoring
▫ Lack funding/resources for rigorous fidelity monitoring, or independent program evaluations
▫ 2010- Michigan’s Fidelity Tool
Scores didn’t accurately reflect practice
Families lack clear understanding of Wraparound model
14
3/6/2014
8
15
Why Competency Tool?
• Starting point for strengths and weaknesses of Facilitators
• Vehicle for conversation to take place between Coordinator, Wraparound Supervisor and Facilitator
• “Coaching” model vs. “Training” model
• Allowed us to see where we needed to implement guidelines
16
3/6/2014
9
17
Development
• Worked with national Wraparound consultant, Wraparound Supervisors, DWMHA, Michigan Department of Community Health-Wraparound Coordinator and MCPNs to develop tool
• Based on Wraparound Principals, Phases of Wraparound and what to expect in each phase
• Key features
▫ Two subscales: Core Capacity and Core Understanding
18
3/6/2014
10
Two Subscales
• Core Capacity-Knowledge of Wraparound
• Core Understanding-Can explain Wraparound and do the process
19
Sample Question from Competency
Tool: Planning Phase
Core Capacity
• Can run a meeting using Wraparound inputs (strengths, needs) that follows Wraparound steps (team mission, needs, strategies, strengths, outcomes)
Core Understanding
Verbalizes the importance of Wraparound inputs and how these will be highlighted
20
3/6/2014
11
Implementation • Supervisor Role
▫ Shadow Facilitator quarterly
▫ Complete “Core Capacity” section of Competency Tool
▫ Provide regular supervision to Facilitator
• Coordinator Role
▫ Attends any Wraparound event- CFT meetings, school meetings, face to face, etc.
▫ Coordinator observes, takes notes, completes “Core Understanding”
▫ Meets with Facilitator and Supervisor to debrief, give feedback, discuss the meeting, etc.
21
Evolution over 3 years
• Year 1: Development and pilot testing of tool
• Year 2: Quarterly shadowing of Facilitators, rate only phase observed
▫ Supervisor complete “Core Capacity”
▫ Coordinator complete “Core Understanding”
• Year 3: Quarterly shadowing of Facilitators, rate all phases
22
3/6/2014
12
Competency Tool Scoring
• Scoring
▫ “Meets” = 2
▫ “Somewhat meets” = 1
▫ “Does not meet” =0
• Average scores for item, subscale, phase
• Percent scores created from Total Score/48
▫ 24 questions worth 2 points a piece
23
Year 2 Results: Total Scores
1.65 1.65 1.76
1.42
1.73
0
0.5
1
1.5
2
Average Total Score by Phase
24
N varied by phase due to supervisor/coordinator ability to shadow all phases
3/6/2014
13
25
Year 2 Results: Subscale Scores
1.8
1.5
1.8
1.5
1.7 1.8 1.8
1.0
0
0.5
1
1.5
2
Capacity Understanding Capacity Understanding Capacity Understanding Capacity Understanding
Engagement Planning Implementation Transition
Ax
is T
itle
Average Subscale Scores by Phase
N varied by phase due to supervisor/coordinator ability to shadow all phases
Report Card Compares with Other
Agencies and DWMHA Wraparound Competency Scale Report Card
AGENCY:
FY11-12
Core Capacity Core Understanding
Low High System YOU Low High System YOU
Engagement .67 1.0 .88 .83 .33 .83 .63 .78
Planning .50 1.0 .92 1.0 .67 .83 .78 .83
Implementation .50 1.0 .81 .79 .33 1.0 .77 .89
Transition .50 1.0 .90 1.9 .33 1.0 .75 1.0
Total .79 1.0 .89 .91 .83 .88 .85 .85
26
3/6/2014
14
Wraparound is not always Wraparound.
27
Lessons Learned
#1: Observations Reveal Model Infidelity
• Facilitators are doing other things and calling it Wraparound; baking, making blanket, doing projects (stones and building with popsicle sticks)
• Building Child and Family teams is a challenge for Facilitators-even when they are right in the home
• Seeing things repeated that were pointed out as not being model fidelity
28
3/6/2014
15
Lessons Learned #2: Shadowing and Coaching is very time
consuming
• Observing 30+ Facilitators on a quarterly basis, in all four phases, takes careful planning and coordination
• Coordinator, not Facilitator, picks family and time for shadowing
• It’s possible to complete both scales through one observation and discussion with Facilitator
29
Year 2 Observations
30
3/6/2014
16
Lessons Learned #3: Supervisor Background is Important
• Significant difference in Supervisors based on background
▫ Clinical vs. Non-Clinical
• Best Supervisor: Has Wraparound experience
• Hard to implement this process with unsupportive Supervisors
• Focus on Supervisor training
31
Lessons Learned #4: Observations Practice Standards
• Observing how Wraparound is implemented across the county allowed for development of Practice Standards
• Minimum requirements for each phase
• Focus on increasing referrals for Medicaid B3 services
32
3/6/2014
17
Lessons Learned #5: Improved coordination with MCPNs
• Holding quarterly meeting
• Coordinate shadowing with audits to create a more complete picture
• Coordinator and one MCPN are working together to share feedback from both perspectives
• Working toward expanding with second MCPN
33
Lessons Learned #6: Pushback From Facilitators
• Some Facilitators unreceptive to process
• More experienced Facilitators had biggest problem
• Lack of understanding of the process and purpose
▫ Relied on Supervisors to explain to their staff
34
3/6/2014
18
Facilitator Feedback
• Satisfaction with process about 50/50
• Pros:
▫ Most mentioned were specific activities, methods, tools that were noted by Supervisor/Coordinator
▫ Affirming of abilities; increased confidence in abilities as a Facilitator
• Cons:
▫ Felt insulted, targeted
▫ Shadowing too frequent, prefer Supervisor
35
Lessons Learned
#7: Reevaluate Process based on Outcomes
After being a part of the coaching process…
1=Strongly Disagree 7=Strongly Agree
My understanding of the Wraparound model has increased. 3.94
My skills as a Wraparound facilitator have increased. 3.89
I feel more confident in my abilities as a Wraparound facilitator. 3.94
I am more satisfied in my position than I was before the coaching began. 3.61
I facilitate my Wraparound families in different ways. 4.16
How similar was feedback received from Supervisor/Wraparound Coordinator? 6.94 on scale of 1-10
36
3/6/2014
19
37
Moving forward
• Focus on training Supervisor
• Rating reliability between Supervisors and Coordinator
• Focus on training in area of Transition Phase
• Coordinated audit/shadowing schedules
• Look at competency scores, audit scores, fidelity scores, and outcomes for individual facilitators
38
3/6/2014
20
39
Thank you!
Monica Hampton
Amy Neumeyer
40
Contact Us:
