Embed Size (px)
Citation preview
Anthony Salerno, PhDJune 16, 2020
Supervisory Best Practices Series: The Supervisor's Role in Promoting
Data Guided Decision Making
Agenda
‣ The Decision Making Process
‣ Data to Drive Quality
‣ Types of Data: Process and Outcome
‣ The Primacy of Client Self-Report
‣ Data Representations and Tools
How do we make decisions‣ Instruction/directives/mandates from senior level
leadership, governmental authorities, payer requirements•Sometimes we have no choice (maybe on how to fulfill a
directive)‣ Our own biases (whatever feels right to us based on
our personality, perceptions and experiences)“We don’t see things as THEY are but as WE are”
~Anais Nin
3
How do we make decisions‣ Asking people we trust or who know things• Experts, consultants‣ Go to the literature (maybe Science will help)‣ What I remember from my training and professors‣ What we have been doing that seems OK (maintain status
quo)‣ What I think my team members will accept‣ What I think my boss wants to hear‣ Ask friends or family‣ Find out what my counterparts are doing‣ Attending webinars (maybe I’ll learn something)‣ Use data
4
High Performance Teams Pay Attention to ResultsAsk yourself:• Do we have a way of knowing how we are doing?• Are we able to keep track of critically important outcomes?• Do we keep track of our accomplishments and areas that have not
been successful?• What are the important outcomes? How will we know? Who decides?• Do we regularly review our goals and progress in a way that is
specific and measurable?• Do we express dissatisfaction with data that falls short of our
expectations and goals?• Do team members discuss areas of performance they are concerned
about?
5
Inattention to Results
‣ If the team leader/supervisor doesn’t attend to results/outcomes, it communicates that results don’t matter, and…
• it becomes difficult to hold people accountable for behavior that detracts from the goals and aims of the organization
6
Data is emotional: Where are you?
7
Limitations: Data is only one part of the decision-making process
8
Poll: How often does data play a major role in the decisions you make as a supervisor?
A. All the time
B. Much of the time
C. Sometimes
D. Not much
9
Poll: What best describes your access to data to make decisions as a supervisor?A. I have access to all the data I need
B. I have much data available to me
C. I have some data available to me
D. Just a little
E. Absolutely none
10
Chat
What kind of data would you most want to have that is difficult to get?
11
Purpose of Measurement: Improve Quality‣ What is Quality?• Doing what you intended to do in the first place‣ What are Quality Indicators?• Safety• Efficiency & Cost Management• Coordination• Integration• Equity• Efficacy• Effectiveness• Timeliness• Continuity• Appropriateness• Access
12
Major Goal of Data: Knowing your Population (Describing who you serve)‣ Who are your customers? • Gender• Religion• Ethnicity• Religion• Race• Diagnoses (Problem areas)• Education• Current and Medication history• Age• ACE’s score• Resiliency/strengths measure• Residential situation• Income and resource sufficiency• Substance use• Chronic health conditions• Retention in treatment (attendance)• Services received• Employment status• Degree of Loneliness (social supports, family, friends)
13
Performance Indicators: Using Data to understand a process, function or result and using data to find problems and measure Improvement‣ It’s hard to change what you can’t measure‣ It’s hard to know how you’re doing without data and
information‣ It’s hard to get others to change their behavior without
data and information‣ It’s hard to understand the parts of the system that
need to change without data and information◦What to keep doing
◦What to stop doing
◦What to begin doing
14
Dr. Salerno’s “not so terrific” response to a question early on about dataQuestion: How is your day hospital doing?
Answer: “Great, nothing bad has happened in over a week! Staff are showing up, clients are showing up and no major disasters…. I deserve a raise!”
15
Data Elements: Performance Indicators‣ Process: what you do to achieve the ultimate and critical
outcomes
‣ Outcomes: what really matters - the final most desired goal - the purpose of the process!
‣ Quantitative data: type of numerical value to be used to express the indicator (percentage, rate, number of occurrences, etc.)
‣ Qualitative data: Focus groups, interviews, surveys involving written feedback
16
Types of Data
• % of people who stop smoking• Number of days a person is
homeless• % of hospitalizations per 100
patients• Change in psychological and
functional improvement scores
• How confident clients are that they will prevent relapse?
• What clients believe has been most helpful?
• Staff view of an evidence-based practice?
• % of people screened for SU during intake
• Attendance data• Turnover data• Time from intake to treatment
• What clients think and feel about the SU screening process?
• Reasons for poor attendance/early termination
• Why do staff leave?PRO
CES
S
OU
TCO
ME
QUANTITATIVE QUALITATIVE
17
Performance Indicators:Process Indicators‣ These indicators help you monitor and track the
degree to which you are implementing your improvement plan as you wanted‣ Practice fidelity is an example of a process indicator‣ Monitoring process improvements is critical to ensure
the evaluation of outcome indicators
18
Process Indicator ExampleImplementation of a trauma screening process•Quantitative: ◦ Trauma screening implemented at a specified time, place and person◦ Total number of clients screened per day◦ Number/% who screen positive (defining what is meant by positive)◦ Number/% who screen negative (defining what is meant by negative)
•Qualitative:◦ Response and feedback from the client about the process◦ Feedback from staff involved in the process◦ Feedback re: the clarity, time demands, burden of implementation,
interference with other key processes, team adherence to process
19
Process Indicator ExampleImplementation of the trauma related assessment process•Quantitative: ◦ Total number of clients who screen positive who receive a more in-depth
assessment◦ Number who have a positive assessment◦ Number who have a negative assessment
•Qualitative:◦ Response and feedback from the client◦ Feedback from staff involved in the process◦ Feedback re: the clarity, time demands, burden of implementation,
interference with other key processes, team adherence to process
20
Performance Indicator ExampleIntervention to address trauma related concerns•Quantitative: ◦ Total number of clients who agree and attend at least one individual/group
trauma focused service◦ Total number who agree initially but do not attend any sessions◦ Total number of individual/group sessions attended by client
•Qualitative:◦ Response and feedback from the client via interviews, focus groups◦ Feedback from staff involved in the process◦ Feedback re: the clarity, time demands, burden of implementation,
interference with other key processes, team adherence to process
21
Performance Indicators: OutcomesThe degree to which the provision of a trauma specific service improves the client’s high priority whole health goals. Quantitative Examples:
◦ Mechanical indicators (BMI, Weight, Blood Pressure, weight circumference) and◦ Blood chemistry indicators (A1C, Cholesterol, other physiological measures pertinent to the
selected cohort)◦ PHQ-9 Score◦ ACEs Score◦ Psychological improvement score◦ Goal attainment score◦ Outcome Questionnaire
Qualitative Examples:◦ Response and feedback from the client◦ Feedback from staff involved in the process◦ Feedback re: the clarity, time demands, burden of implementation, interference with other key
processes, team adherence to process
22
Will there be a need for data related to Covid-19 and the disruptions it has caused?
‣ There is a lot of talk about re-imagining behavioral healthcare as well as general healthcare services ‣ In what way can data help? What do we need to know?Maybe….‣ % of missed appointments (kept appointments)
compared to pre Covid-19‣ % of clients who prefer virtual appointments‣ % of clients living alone with little to no social
connections‣ % of clients requiring emergency psychiatric care‣ Increase in referrals compared to pre Covid-19
23
The Importance of Self-Report: Patient-Centered
Care
“The ultimate judge of the quality of ourwork is the patient, end of story.”
—Don Berwick
24
Patient Outcomes‣ Patients' views of their physical and behavioral health
status have typically not been addressed outside of clinical trials
‣ Yet the ultimate measure of health system performance is whether it helps people…..
• recover from an acute or chronic health condition; a persisting mental health and/or substance use problem and/or the negative impact of adverse life experiences
•make progress towards personally meaningful goals
Personal reports of one’s perception and experience is one of the critical ways to gauge success.
25
Use of PROMS in Clinical Practice‣ In the U.S., PROMs are in the early stages of
development for use in clinical practice, as opposed to research.
‣ They have been most widely used to monitor conditions that rely on patients' reports (rather than diagnostic tests), such as depression.
‣One major goal of electronic records is to integrate PROMs.
26
Displaying Data
‣Personal Dashboards (outcomes)
‣Population Health Measures (outcomes)
‣Process Indicators
27
INDIVIDUAL CLIENT DASHBOARD
28
Population Health100 ClientsTracking Improvement
0%
10%
20%
30%
40%
50%
60%
70%
80%
Q1 Q2 Q3 Q4
A1c normal Cholesterol norm BMI normal
29
Population Health100 ClientsTracking Improvement
0%
10%
20%
30%
40%
50%
60%
70%
80%
Q1 Q2 Q3 Q4
EmploymentPositive mood Medication Adherence
30
31
Challenges Faced by Organizations‣ Capacity to Collect Data, Exchange Information, and
Monitor Population Health‣ Capability•Can the organization use an electronic health record
(EHR) or other methods to collect individual and practice-level data that allows them to identify, track, and segment the population?•Does the organization have a reliable system for
collecting data that supports aggregation of data, information sharing, and identification of high-risk populations?
32
Use of Data as Part of Continuous Quality
Improvement
33
0
5
10
15
2 0
2 5
30
35
Time
How Improvement Efforts Usually Happen: The Deviation Approach
Relapse rates, Hospitalization and emergency visits,Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
Our Program
34
0
5
10
15
2 0
2 5
30
35
Time
StudyProblem
Quality Improvementwith Deviation Based Approaches
Relapse rates, Hospitalization and emergency visits,Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
35
0
5
10
15
2 0
2 5
30
35
Time
Fix theProblem
Quality Improvement: Reduce/Eliminate Deviation
Relapse rates, Hospitalization and emergency visits, Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
36
0
5
10
15
2 0
2 5
30
35
Time
How Change Happens: The Benchmarking Approach
Relapse rates, Hospitalization and emergency visits,Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
Our Program
Their Program
37
Benchmarking:The Turnover Problem by the Numbers: NYS Turnover and Vacancy rates (2015)
‣ 126 behavioral health community provider agencies responded to the survey. A total of 14,449 full and part-time behavioral health positions were identified by respondents.
Area Turnover Rate Vacancy RateStatewide 34% 14%Central New York 28% 12%Hudson River Region 32% 14%Long Island 41% 20%New York City 42% 20%Western New York 25% 11%
Source: Mental Health Association in New York State Inc.
38
0
5
10
15
2 0
2 5
30
35
Time
How Change Happens:The Benchmarking Approach
Relapse rates, Hospitalization and emergency visits,Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
Our Program
Their Program
39
0
5
10
15
2 0
2 5
30
35
How Continuous Quality Improvement Happens: The Proactive Approach
Time
Study the
Process
Relapse rates, Hospitalization and emergency visits, Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover, burnout/compassion fatigue
40
0
5
10
15
2 0
2 5
30
35
Time
Redesignthe Process
Quality Improvement with Continuous Quality Improvement
Relapse rates, Hospitalization and emergency visits,Missed appointments, Drop (pushed) outs, Poor health measures (risk levels), Negative incidents, Days homeless, Unemployment rates, poor clinical outcomes, staff turnover , burnout/compassion fatigue
41
0
0.25
0.5
0.75
1
1.25
1.5
1.75
2
2.25
2.5
2.75
3
3.25
3.5
3.75
4
••
• •
•
•
••
Staff Kept % physical % filled % on Med Trauma CompassionRetention Appointments Exams prescription Assist Tx. Screening Satisfaction
Aim of Quality Improvement: It’s All About Moving Your Dot: Sample of Possible Quality Performance Indicators
42
Graphic Representation of the Improvement
Starting with a Baseline ClientPerception of Care and Environment
0
0.5
1
1.5
2
2.5
3
Consumer Screening Workforce Practice Recurrence Community
Average Scoress
Safety Comfort Friendliness Accessibility Choice Involvement
43
Benchmarking Example: What has Changed for Clients?Statewide Utah OQ/YOQ Data 2009 – 2013 by MH/SA Center
32%
24%
0%5%
10%15%20%25%30%35%40%45%50%
2009 2010 2011 2012 2013
Percentage of Clients ShowingStatistically Significant Improvement on the OQ/YOQ
Weber H.S.Center 1Center 2Center 3Center 4Center 5Statewide Ave.
44
45
1. I get along well with others3. I feel no interest in things4. I feel stressed at work/school5. I blame myself for things7. I have thoughts of ending my life10. I feel fearful11. After heavy drinking, I need a drink the next morning to get going. (If you do not drink, mark “never”)12. I find my work/school satisfying13. I am a happy person15. I feel worthless17. I have an unfulfilling sex life18. I feel lonely19. I have frequent arguments20. I have trouble getting along with friends and close acquaintances31. I am satisfied with my life23. I feel hopeless about the future24. I like myself28. I am not working/studying as well as I used to29. My heart pounds too much
Developed by Michael J. Lambert, Ph.D. and Gary M. Burlingame, Ph.D.© Copyright 1996 OQ Measures LLC. All Rights Reserved. License Required For All Uses.For More Information Contact: OQ MEASURES LLC E-MAIL: [email protected]: WWW.OQMEASURES.COMTOLL-FREE: 1-888-MH SCORE (1-888-647-2673)
Outcome Questionnaire Items (Sample)
46
Customer Service by Practitioner
47
20181614121086420
A EB C D
Practitioners
Additional Tools to Display and Use Data:Pareto Chart: In Order of Max to Min‣ Customer
Service Ratings by Client
‣ Missed Appointments
‣ Billable Hours
‣ Time to First Contact
‣ Involves Family
48
People (staff, leadership, client factors)Equipment/materials
Procedures/PracticesPolicies
Health Disparities• Utilization of services
(over/under)• Retention in services• Outcomes• Satisfaction with
services• Practices
Fishbone (ISHIKAWA) Diagram
49
Who Messed Up?
How bad we got to hurt the guy so he don’t screw up again?
Who’s gonna take care of it?
FISHHEAD DIAGRAM (Carmine “thefish” Managia)
oThe Blame Game Approach to Quality Improvement
50
Flowcharts: Diagnosing a problem and finding asolution based on data
The terminator symbol marks the starting or ending point of the system. It usually contains the word "Start" or "End."
A box can represent a single step ("add two cups of flour"), or an entire sub-process ("make bread") within a larger process.
A printed document or report.
A decision or branching point. Lines representing different decisionsemerge from different points of thediamond.
51
Workflow addressed: In this workflow, we are considering the patient who is already receiving behavioral health services at the provider agency and is eligible for enrollment in an integrated care program. We are ensuring that the patient sees the primary care provider as soon as possible.
Intake > EXISTING Behavioral Health patients > Getting Patient to Primary Care Exam or Primary Care Appointment
Beha
vior
al H
ealth
Pro
fess
iona
lN
urse
Car
e M
anag
er/N
urs
e Pr
actit
ione
r
Workflows for Vital Signs, PC
appt
Has PC and active patient?
Wants to see PC?
PC available now?
Walk patientto PC office to be seen
now
Obtain consents, PC contact data, request CCR
Walk pt toNurse CareManager /Practitioner
No
Yes
Yes Yes
No No
Workflow to monitor patient
interest
NCM/Pavailable?
Yes
No
BehavioralHealth WFfor PCP appt.
BH patient is eligible.
See PBHCIProject PCP visit follow
up WF
See PBHCIProject PCP appt follow
up WF
52
Services Authorization WorkflowService authorization is a process for requesting/reviewing, approving and monitoring patient use of services. When this is integrated into the business process workflows related to clinical services, practice management and financial management, it supports sustainability without interfering with service delivery or the patient/clinician relationship.
Role ActivityClinician
Clinician supervisor
Financial Manager
Yes
No
Based on the assessment and treatment plan, request authorization for services
Authorize?
Requiresfiscal review?
Review request re: service authorization policies and procedures
Deny, return to clinician
Modify request ?
Close request
No
No
Yes
Yes
Review request re: service authorization policies and procedures
No Approve, returnto clinician
Initiate and monitor delivery of services
Deny, return to clinicianAuthorize?
53
Q&A
54
Upcoming WebinarsSupporting Children Coping with Grief
and Loss During the COVID-19 PandemicWednesday, June 17th
1-2:30 PM
Applying Psychological First Aid During the COVID-19 Crisis
Monday, June 221-2 PM
55
Contact Us…Anthony Salerno, PhD
For questions about CTAC trainings, [email protected]
Please complete the feedback survey when you leave the webinar!
www.ctacny.org
56
