Upload
gwenda-rogers
View
222
Download
2
Tags:
Embed Size (px)
Citation preview
Achieving Our MissionAchieving Our Mission
The Role of CQI in Public Safety
Kimberly Gentry Sperber, Ph.D.
Efforts To DateEfforts To Date
• “What Works” Literature– Principles of Effective Interventions– Growing evidence based on individual
program evaluations and meta-analyses
• Continuing Gap Between Science and Practice– Few programs score as satisfactory on
CPAI/CPC
Importance of CPAI/CPC CriteriaImportance of CPAI/CPC Criteria
• Differences in recidivism rates based on CPAI scores:– Scores of 0-49% demonstrated 1.7% reduction
compared to comparison group.– Scores of 50-59% demonstrated 8.1% reduction.– Scores of 60-69% demonstrated 22% reduction.
What are implications for ODRC standards for correctional programs?
Why IsnWhy Isn’’t t ““ItIt”” Working? Working?Latessa, Cullen, and Gendreau (2002)Latessa, Cullen, and Gendreau (2002)
• 4 common failures of correctional programs:– Failure to use research in designing programs– Failure to follow appropriate assessment and
classification practices– Failure to use effective treatment models– Failure to evaluate what we do
What Happens When We DonWhat Happens When We Don’’t Measure Ourselves?t Measure Ourselves?
The Role of Fidelity
Washington State ExampleWashington State Example(Barnoski, 2004)(Barnoski, 2004)
• For each program (FFT and ART), an equivalent comparison/control group was created
• Felony recidivism rates were calculated for each of three groups, for each of the programs• Youth who received services from therapists
deemed ‘competent’• Youth who received services from therapists
deemed ‘not competent’• Youth who did not receive any services (control
group)
Functional Family Therapy Results: Functional Family Therapy Results: % New Felony% New Felony
139 6
2519
11
3227
17
0
10
20
30
40
50
60
70
6 Months 12 Months 18 Months
FFT Not Competent
Control group
FFT Competent
Results calculated using multivariate models in order to control for potential differences between groups
Project GreenlightProject Greenlight
• Short-term prison-based reentry program in New York– CBT Skills Training– Employment Services– Housing Services– Drug Education and Awareness– Family Counseling– Practical Skills Training– Community-Based Networks– Familiarity With Parole– Individualized Release Plans
Project Greenlight BenefitsProject Greenlight Benefits
• Participants received more service referrals
• Participants reported more contacts with community services after release
• Participants demonstrated significantly more familiarity with parole conditions
• Participants were more positive about parole
But Did It Work?But Did It Work?
17.214.4 13
34.1
26.824.2
0
5
10
15
20
25
30
35
Arrests at 6 Months Arrests at 12Months
Greenlight
No Services
Existing TransServices
What Went Wrong?What Went Wrong?
• Violation of the risk principle– Ceased use of risk assessment instrument when staff
deemed process too cumbersome
• Violation of the need principle– All offenders received same services whether
needed or not
• Violation of the fidelity principle– Staff modified delivery of the CBT curriculum
(shortened the duration, increased frequency, increased class size)
• Differential staff competence– Certain case managers produced worse outcomes
UC Halfway House/CBCF Study in Ohio: A UC Halfway House/CBCF Study in Ohio: A Look at Fidelity Statewide (2002)Look at Fidelity Statewide (2002)
• Average Treatment Effect was 4% reduction in recidivism
• Lowest was a 41% Increase in recidivism• Highest was a 43% reduction in recidivism
• Programs that had acceptable termination rates, had been in operation for 3 years or more, had a cognitive behavioral program, targeted criminogenic needs, used role playing in almost every session, and varied treatment and length of supervision by risk had a 39% reduction in recidivism
2010 UC Halfway House/CBCF Study in Ohio:2010 UC Halfway House/CBCF Study in Ohio:Adherence to CBT in Groups and Changes in RecidivismAdherence to CBT in Groups and Changes in Recidivism
-2
-1
0
1
2
3
4
5
6
7
-1 0 1
What Do We Know About What Do We Know About Fidelity?Fidelity?
• Fidelity is related to successful outcomes (i.e., reductions in recidivism, relapse, and MH instability).
• Poor fidelity can lead to null effects or even iatrogenic effects.
• Fidelity cannot be assumed• Fidelity can be measured and
monitored.
Bringing Mission to the Bringing Mission to the ForefrontForefront
Becoming a Mission-Driven Becoming a Mission-Driven OrganizationOrganization
• Knowing the mission statement• What does it mean to you?• What does it mean for your role?• Who is responsible for fulfilling the
agency’s mission statement?• How can QA/QI help us accomplish
our mission statement?
Providing Managed ChangeProviding Managed Change
• Achieving and maintaining quality is not accidental.
• It requires a deliberate effort on the part of everyone.
Achieving QualityAchieving Quality
• Responsibility for quality falls on both the organization and the individual.
• The individual and the organization should be linked in a formal framework designed to continually improve quality.
Quality Assurance (QA)Quality Assurance (QA)
• Retrospective review process• Emphasis on regulatory and
contract compliance• Catching people being bad leads to
hide and seek behavior
Continuous Quality Continuous Quality Improvement (CQI)Improvement (CQI)
• CQI is a prospective process• Holds quality as a central priority within the
organization• Focus on customer needs; relies on feedback
from internal and external customers• Emphasizes systematic use of data• Not blame-seeking• Trust, respect, and communication• Move toward staff responsibility for quality,
problem solving and ownership of services
Focus on Causes Focus on Causes notnot SymptomsSymptoms
• Focus on processes/systems rather than individuals or specific errors
• Identification of risk points and their contribution to the problem
• Identify changes in these processes that reduce risk of re-occurrence
Objectives of CQIObjectives of CQI
• To facilitate the Agency’s mission• To ensure appropriateness of services• To improve efficiency of services/processes• To improve effectiveness of directing
services to client needs• To foster a culture of learning• To ensure compliance with funding and
regulatory standards
Common ElementsCommon Elements
• Documentation Review• Performance Indicators
– Process vs. Outcomes
• Satisfaction – Client, Employee, Referral Source/Stakeholders
• Protocols for Data Use• Infrastructure
Focused Discussion:Focused Discussion:Why Examine Documentation?Why Examine Documentation?
• Clinical/Service Implications– Documentation is not separate from service delivery.– Did the client receive the services he/she needed?
• Operational Implications– Good documentation should drive decision-making.– Means of communication
• Risk Management Implications– If it isn’t documented, it didn’t happen.– Permanent record of what occurred in the program/unit.
• Source of Staff Training • Reflection of the provider and organization’s
competency:– EBP– Outcome of care
Action PlansAction Plans
• Plan of correction• Proactive approach to problem-
solving• Empowers staff• Using objective data to inform
decision making
Who Creates Action Plans?Who Creates Action Plans?
• Anyone and everyone can create action plans
• Focus should be on who has knowledge or expertise to contribute
• Focus should not be on the person’s title
Minimum RequirementsMinimum Requirements
• Buy-in from staff at all levels of the organization
• Sufficient resources allocated for staff training
• Sufficient resources allocated for staff to participate in the process– Meetings– Data collection, reporting, use
• Sufficient information systems
Why Invest the Resources?Why Invest the Resources?
What makes this effort too compelling to ignore?
A CEOA CEO’’s Perspectives Perspective
• Why invest in CQI?– Because it’s the right thing to do!– Better for clients (i.e., better outcomes)
• Mission-driven– Increased staff satisfaction– Increased staff retention– Improved referral source satisfaction– More business for related projects– Outcomes to sell to business community
and other payers– Demonstrates fiscal responsibility (i.e.,
effective use of dollars)
Strategic Use of CQI DataStrategic Use of CQI Data
• CQI data used to provide testimony before legislature
• CQI data and infrastructure used to secure new contracts and grants
• CQI data used in newsletters, media relations, levy campaigns, etc.
• CQI data used to negotiate programmatic changes with stakeholders
Performance ExamplesPerformance Examples• Men’s halfway houses employment with
benefits– 33% to 67% over 5 years
• Changes in HIT over time per strategic plan– 57.4% to 72% over 5 years
• CCC’s employment indicator performance– <10% to 75% over 5 years
How do these changes in performance link to mission?
Top 10 Practices for Reducing Top 10 Practices for Reducing Recidivism in Drug CourtsRecidivism in Drug Courts
(Carey & Finigan, 2013)(Carey & Finigan, 2013)
1. Program caseload less than 252. Participants have greater than 90 days
clean before graduation3. Judge spends average of 3 minutes or
more per person during status review hearings
4. Treatment communicates with court via email
5. Representative from treatment attends drug court team meetings
Top 10 Practices for Reducing Top 10 Practices for Reducing Recidivism in Drug CourtsRecidivism in Drug Courts
(Carey & Finigan, 2013)(Carey & Finigan, 2013)
6. Review of the data/program stats has led to modification in operations
7. Representative from treatment attends court sessions
8. Drug court allows non-drug charges9. Law enforcement is member of drug court
team10.Results of program evaluations have led
to modification in operations
Top 10 Practices for Increasing Cost Top 10 Practices for Increasing Cost Savings in Drug CourtsSavings in Drug Courts
(Carey & Finigan, 2013)(Carey & Finigan, 2013)
1. Review of the data/stats has led to modifications in operations
2. Results of program evaluations have led to modification in operations
3. Sanctions are immediate4. Defense attorney attends drug court
team meetings5. Must have a job or be in school to
graduate
Top 10 Practices for Increasing Cost Top 10 Practices for Increasing Cost Savings in Drug CourtsSavings in Drug Courts
(Carey & Finigan, 2013)(Carey & Finigan, 2013)
6. Representative from treatment attends court sessions
7. Team members are given copy of sanctioning guidelines
8. Drug test results back in 48 hours or less9. Drug tests collected at least 2x/week
during first phase10.Law enforcement attends court sessions
NPC Research on Drug CourtsNPC Research on Drug Courts
37%
16%
0%
10%
20%
30%
40%
YesN=20
NoN=15
Perc
ent I
mpr
ovem
ent i
n O
utco
me
Cost
s*
* "Percent improvement in outcome costs" refers to the percent savings for drug court compared to business-as-usual
Drug Court Uses Evaluation Feedback to Make Modifications
The Role of QA/QI in Community CorrectionsThe Role of QA/QI in Community Corrections(based on UC Halfway House and CBCF study)(based on UC Halfway House and CBCF study)
6
1
0
2
4
6
8
Internal QA No Internal QA
% C
hang
e in
Rec
idiv
ism
Bottom LineBottom Line
• Many programs are implementing evidence-based practices with little evidence of strong fidelity.
• Result is an ongoing gap between science and practice.
• This gap often results in null or even iatrogenic effects.
• Correctional organizations have a responsibility to ensure effective services.
• Responsibility for EBP needs to be aligned at all levels – administration, management, line staff.
• Need to focus on creating formal infrastructure to support EBP and mission achievement
Continuous Quality Continuous Quality ImprovementImprovement
Questions & Answers