Upload
jase-certain
View
219
Download
0
Embed Size (px)
Citation preview
Evidence-based Application of Evidence-based Treatments
Peter S. Jensen, M.D.President & CEO
The REACH InstituteREsource for Advancing Children’s Health
New York, NY
Effect Sizes of PsychotherapiesEffect Sizes of Psychotherapies
00.10.20.30.40.50.60.70.80.9
1
Smith &Glass,1977
Shapiro&
Shapiro,1982
Casey &Berman
Weisz etal., 1987
Kazdinet al.,1990
Weisz etal., 1995
Weisz etal, 1995
Mea
n E
ffec
t S
izes
Weisz et al., 1995
Children & AdolescentsAdults
University
“Real World”
Three Levels:
Child & Family Factors:
e.g., Access & Acceptance
Provider/Organization Factors:
e.g., Skills, Use of EB
Systemic and Societal Factors:
e.g., Organiz., Funding Policies
Barriers vs. “Promoters” to Barriers vs. “Promoters” to Delivery of Effective Services Delivery of Effective Services
(Jensen, 2000)(Jensen, 2000)
“Effective” Services
Efficacious Treatments
0.5
1
1.5
2
2.5
0 100 200 300 400
Assessment Point (Days)
Av
erag
e S
core
CC-NOMEDS
CC-MEDS
BEH
MED
COMB
Key Differences, MedMgt vs. CC:
Initial Titration
Dose
Dose Frequency
#Visits/year
Length of Visits
Contact w/schools
Teacher-Rated InattentionTeacher-Rated Inattention
(CC Children Separated By Med Use)(CC Children Separated By Med Use)
Would You RecommendWould You Recommend Treatment? (parent) Treatment? (parent)
Medmgt Comb Beh
Not recommend 9% 3% 5%
Neutral 9% 1% 2%
Slightly Recommend 4% 2% 2%
Recommend 35% 15% 24%
Strongly recommend 43% 79% 67%
Key ChallengesKey Challenges
Policy makers and practitioners hesitant to implement change
Vested interests in the status quo Researchers often not interested in promoting
findings beyond academic settings Manualized interventions perceived as difficult
to implement or too costly Obstacles and disincentives actively interfere
with implementation
Key ChallengesKey Challenges
Interventions implemented but “titrate the dose”, reducing effectiveness
“Clients too difficult”, “resources inadequate” used to justify bad outcomes
Research population “not the same” as youth being cared for at their clinical site
Having data and “being right” neither necessary nor sufficient to influence policy makers
The Good and the Bad: Effectiveness of Interventions The Good and the Bad: Effectiveness of Interventions by Intervention Type by Intervention Type
05
101520253035
Positive Negative
Davis, 2000
No. of Interventions demonstrating positive or negative/inconclusive change
Little or No Effect (Provider & Little or No Effect (Provider & Organization-focused) :Organization-focused) :
Educational materials (e.g., distribution of recommendations for clinical care, including practice guidelines, AV materials, and electronic publications)
Didactic educational meetings
Bero et al, 1998
Effective Provider & Organizational Effective Provider & Organizational Interventions:Interventions:
Educational outreach visits Reminders (manual or computerized) Multifaceted interventions Sustained, interactive educational meetings
(participation of providers in workshops that include discussion and practice)
Bero et al, 1998
Implications re: Changing Provider Implications re: Changing Provider BehaviorsBehaviors
• Changing professional performance is complex - internal, external, and enabling factors
• No “magic bullets” to change practice in all circumstances and settings (Oxman, 1995)
• Multifaceted interventions targeting different barriers more effective than single interventions (Davis, 1999)
• Little to no theory-based studies
• Consensus guidelines approach necessary, but not sufficient.
• Lack of fit w/HCP’s mental models
Additional PerspectivesAdditional Perspectives
Messenger of equal importance as the message Trusted Available Perceived as expert/competent
Adult Learning Models Tailored to learner’s needs Learner-defined objectives Hands-on, with ample opportunities for practice Sustained over time Skill-oriented Feedback Attention to Maintenance and sustaining change
Dissemination and Adoption of Dissemination and Adoption of New Interventions New Interventions
Dissemination and Adoption of Dissemination and Adoption of New Interventions New Interventions
Source: Backer, Liberman, & Kuehnel (1986) Dissemination and Adoption of Innovative Psychosocial Interventions. Journal of Consulting and Clinical Psychology, 54:111-118; Jensen, Hoagwood, & Trickett (1997) From Ivory Towers to Earthen Trenches. J AppliiedDevelopmental Psychology
Sustained Interpersonal contact
Organizational support
Persistent championship of the intervention
Adaptability of the intervention to local situations
Availability of credible evidence of success
Ongoing technical assistance, consultation
Science-based Plus Necessary “-abilities” Science-based Plus Necessary “-abilities”
• Palatable• Affordable• Transportable• Trainable• Adaptable, Flexible• Evaluable • Feasible• Sustainable
Models for Behavior Change:
(Jaccard et al, 2002)
The Theory of Reasoned Action (Fishbein & Ajzen, 1975)
Self-efficacy Theory (Bandura, 1977)
The Theory of Planned Behavior (Ajzen, 1981)
Diffusion of Innovations (Rogers, 1995)
Influences on Provider BehaviorInfluences on Provider Behavior
Patient & Family Factors:
•Stigma
•Adherence
•Negative attitudes
•Rapport, engagement
Provider Factors:
•Knowledge, training
•Self-efficacy
•Time pressures
•Fear of litigation
•Attitudes & beliefs
•Social conformity
•Lack of information
Economic Influences:
•Compensation
•Reimbursement
•Incentives
Systemic & Societal Factors:
•Organizational standards
•Staff support/resistance
•Staff Training
•Funding policy
Prescribing Practices
First, Use an Atypical vs. TypicalFirst, Use an Atypical vs. Typical
Favor/Unfavor
Easy/Hard
Improve/No
Agree/Disagree
543210-1-2-3-4-5
2
323
362
Descriptives (n=19)
Min/Max Mean(SD)
Favor/Unfavor 0/5 3.73(1.61)
Easy/Hard -1/5 4.16(1.64)
Improve/No 0/5 2.84(1.57)
Agree/Disagree 0/5 4.05(1.27)
First Use Atypical--AdvantagesFirst Use Atypical--Advantages
Advantages Count Percent of Responses
Avoids typicals' side effects 13 59.1%
Better patient approval/compliance 5 22.7%
Atypicals effective in treating aggression 2 9.1%
Other (i.e. looks better politically) 2 9.1%
Total responses 22 100.0%
First Use Atypical – DisadvantagesFirst Use Atypical – Disadvantages
Disadvantage Count Percent of Responses
Typicals may work better for some patients 6 23.1%
Avoids atypicals' side effects 6 23.1%
If need to sedate patient, typicals may be better 6 23.1%
More is known about typicals in kids 4 15.4%
Can not be administered as IM’s 3 11.5%
Other 1 3.8%
Total responses 26 100.0%
First Use Atypical—ObstaclesFirst Use Atypical—Obstacles
Obstacle Count Percent of Responses
Cost 5 23.8% More data supporting typicals 5 23.8%Patient history of non-response to atypicals 4 19.1% Patient resistance 3 14.3%Less available 2 9.5% Other 2 9.5% Total responses 21 100.0%
Favor/Unfavor
Easy/Hard
Improve/No
Agree/Disagree
543210-1-2-3-4-5
2
202
2912
Limit the Use of Stat’s & P.R.N.’s
Descriptive Statistics (n=19)
Min/Max Mean (SD)
Favor/Unfavor -5/5 2.63(2.89)
Easy/Hard -5/5 -0.38(3.22)
Improve/No -2/5 2.44(1.92)
Agree/Disagree -2/5 3.86(1.81)
Limit Stat‘ & P.R.N.’s -- AdvantagesLimit Stat‘ & P.R.N.’s -- Advantages
Advantage Count Percent of Responses
Other (i.e avoids traumatizing patient, 6 27.3% Avoids unnecessary medication 5 22.7% Avoids unnecessary side effects 4 18.2% Allows doctor to better understand patient’s condition 4 18.2% Patient learns techniques they can apply in ‘real life’ 3 13.6% Total responses 22 100.0%
Limiting Stat’s & P.R.N.'s — Limiting Stat’s & P.R.N.'s — DisadvantagesDisadvantages
Disadvantage Count Percent of Responses
Possible safety risk to patient and others 9 2.9%
Other (i.e. does not address biological factors 6 28.6%
Difficult for staff, who may feel less in control 4 19.0%
May need to rapidly sedate patient 2 9.5%
Total responses 21 100.0%
Limiting Stat’s & P.R.N.'s--Limiting Stat’s & P.R.N.'s--ObstaclesObstacles
Obstacle Count Percent of Responses
Safety 8 33.3%Other (i.e.patient belief
that p.r.n.’s condone behavior; 5 20.8%Staff resistance 4 16.7% Patient too aggressive 4 16.7% Staff availability and training 3 12.5%
Total responses 24 100.0%
Favor/Unfavor
Easy/Hard
Improve/No
Agree/Disagree
543210-1-2-3-4-5
25
Monitor Side Effects
Descriptives (n=19)
Min/Max Mean(SD)
Favor/Unfavor 3/5 4.57(.69)
Easy/Hard -2/5 2.94(2.4)
Improve/No 1/5 4.0(1.15)
Agree/Disagree 3/5 4.68(.58)
Use Standardized Scales for Side Use Standardized Scales for Side Effects -- AdvantagesEffects -- Advantages
Advantage Count Percent of Responses
Helps captures side effects you might otherwise miss 8 27.6%Other (i.e. increases patient compliance; improves 6 20.7%
communication between doctors; helps assess severity of side effects)
Provides objective measure 4 13.8%Keeps doctors’ focus on side effects 4 13.8%Determines drug effectiveness for specific symptoms 4 13.8%Enables doctor to track side effects over time 3 10.3% Total responses 29 100.0%
Use Standardized Scales for Side Use Standardized Scales for Side Effects--DisadvantagesEffects--Disadvantages
Disadvantage Count Percent of Responses
Doctor may ignore side effects not on scale 3 27.3%
May minimize importance of clinical evaluations 3 27.3%
Other (i.e. may make patient more aware of side effects) 3 27.3%
Methodological problems (i.e. inter-rater reliability) 2 18.2%
Total responses 1 100.0%
Scales for Side Effects--ObstaclesScales for Side Effects--Obstacles
Obstacle Count Percent of Responses
Time 8 25.0% Scales are complicated/require training 6 18.7%Instrument availability 5 15.6% Other (i.e. staff resistance; instrument availability; 5 15.6%
cost) Administrative barriers 3 9.4% Laziness 3 9.4%Clinician resistance 2 6.3% Total responses 32 100.0%
Self-Efficacy Beliefs
Expected - Values
Behavioral Intention
Normative Beliefs
OBSTACLES Cognition, Habits & Automatic Processes, Knowledge, Behavioral Skills, Decisional Style, Behavioral Salience
Mental Contrasting
Implementation Intentions
Behavior
Expected Effects
Possible Effects
INTERVENTION
New Models for Behavior Change: TMC, TII (Gollwitzer, Oettingen, Jaccard, Jensen et al, 2002; Perkins et
al., 2007)
Mental Contrasting/Implementation Mental Contrasting/Implementation IntentionsIntentions
1. Use mental contrasting to strengthen behavioral intentions:
“What are the advantages or positive consequences associated with the use of Guideline X”
2. Identify Obstacles:“What gets in the way of implementing guideline X”3. Form Implementation Intentions to overcome
obstacles:“If I encounter obstacle Y, then I will X.”
Track Target SymptomsTrack Target Symptoms
Favor/Unfavor
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Pre-Intervention Post-Intervention
Descriptive Statistics (n=4)
1/5 3.5(1.9)0/4 1.8(1.7)1/4 2.5(1.3)3/5 4.3(1.0)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/Max Mean(SD)
Descriptive Statistics (n = 4)
1/5 3.0(1.6)-3/1 -0.5(1.9) 2/3 2.8(0.5) 3/4 3.3(0.5)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/MaxMean(SD)
Favor/Unfavor
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Descriptive Statistics (n = 4)
4/5 4.8(0.5) -5/5 3.3(2.9) 4/5 4.8(0.5) 5/5 5.0(0.0)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/MaxMean(SD)
Descriptive Statistics (n=4)
5/5 5.0(0.0)1/5 3.5(1.9)5/5 5.0(0.0)5/5 5.0(0.0)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/Max Mean(SD)
Use A Conservative Dosing StrategyUse A Conservative Dosing Strategy
Favor/Unfavor
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Favor/Unfavor
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Pre-Intervention Post-Intervention
Limit the Use of P.R.N.sLimit the Use of P.R.N.s
Favorable/Unfavorabl
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Descriptive Statistics (n=4)
3/5 4.5(1.0)0/4 2.0(1.8)1/5 3.3(1.7)4/5 4.8(0.5)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/Max Mean(SD)
Descriptive Statistics (n = 4)
-3/5 2.5(3.8)-5/5 -0.8(4.2) 2/5 3.8(1.5) 3/5 4.5(1.0)
Favor/UnfavorEasy/HardImprove/No ImproveAgree/Disagree
Min/MaxMean(SD)
Favor/Unfavor
Easy/Hard
Improve/No Improve
Agree/Disagree
543210-1-2-3-4-5
Pre-Intervention Post-Intervention
Intention to Use Guidelines in the Intention to Use Guidelines in the Next Month (n=4)Next Month (n=4)
Guideline Pre-Intervention Post-Intervention
Track Target Symptoms 4.6(2.89) 8.25(2.1)
Conservative Dosing Strategy
8.8(1.30) 10.00(.0)
Limit P.R.N. 5.6(3.64) 8.75(.96)
Track Side Effects 9.6(.89) 8.75(1.5)
Three Levels:
Child & Family Factors:
e.g., Access & Acceptance
Provider/Organization Factors:
e.g., Skills, Use of EB
Systemic and Societal Factors:
e.g., Organiz., Funding Policies
Barriers vs. “Promoters” to Barriers vs. “Promoters” to Delivery of Effective Services Delivery of Effective Services
(Jensen, 2000)(Jensen, 2000)
“Effective” Services
Efficacious Treatments
CLINIC/COMMUNITY INTERVENTION CLINIC/COMMUNITY INTERVENTION DEVELOPMENT AND DEPLOYMENT MODEL DEVELOPMENT AND DEPLOYMENT MODEL
(CID) (Hoagwood, Burns & Weisz, 2000)(CID) (Hoagwood, Burns & Weisz, 2000) Step 1: Theoretically and clinically-informed construction, refinement, and manualizing of the protocol within the context of the practice setting where it is ultimately to be delivered
Step 2: Initial efficacy trial under controlled conditions to establish potential for benefit
Step 3: Single-case applications in practice setting with progressive adaptations to the protocol
Step 4: Initial effectiveness test, modest in scope and cost
Step 5: Full test of the effectiveness under everyday practice conditions, including cost effectiveness
Step 6: Effectiveness of treatment variations, effective ingredients, core potencies, moderators, mediators, and costs
Step 7: Assessment of goodness-of-fit within the host organization, practice setting, or community
Step 8: Dissemination, quality, and long-term sustainability within new organizations, practice settings, or communities
Partnerships & Collaborations Partnerships & Collaborations in Community-Based Research in Community-Based Research
Why Partnerships? partnerships -- not with other scientists per se, but with experts
of a different type -- experts from families, neighborhoods, schools, in communities.
Only from these experts can we learn what is palatable, feasible, durable, affordable, and sustainable for children and adolescents at risk or in need of mental health services
“Partnership” - changes in typical university investigator - research subject relationship
Practice – based Research Networks Bi-directional learning
Traditional approach research question posed, building on theory and body of
previous research logical next step in elegant chain of hypotheses, tests, proofs,
and/or refutations isolation of variables from larger context; limit potential
confounds and alternative explanations of findings study designed, investigator then looks for “subjects” who will
“recipients of the bounty” cannot answer questions about sustainability unidirectional blind to issues of ecological validity
Partnerships & Collaborations Partnerships & Collaborations in Community-Based Research in Community-Based Research
Alternative (collaborative) approach expert-lay distinction dissolved both partners bring critical expertise to research agenda
research methods and technical expertise from the university investigator
systems access and local-ecological expertise from the community collaborator
so-called “confounds” can provide useful “tests” of the feasibility, durability, and generalizability of the intervention
hence, importance of replication improved validity of knowledge obtained?
Partnerships & Collaborations Partnerships & Collaborations in Community-Based Research in Community-Based Research
The REACH Institute….The REACH Institute….Putting Science to WorkPutting Science to Work
- Problem area identification
- Bring key “change agents” and gatekeepers to the table (federal or state partners, consumer and professional organizations)
- Identify “actionable” knowledge among experts and “consumers”
- Identify E-B QI procedures that are feasible, sustainable, palatable, affordable, transportable
- Consumer and stakeholder “buy-in” & commitment to E-B practices
- Dissemination via partners across all 3 system levels - “with an edge” (policy/legislative strategy with relevant federal/state partners)- Training and TA/QI intervention;
all sites eventually get intervention.
- Monitoring/fidelity
- Report preparation
- Results fed back into Step II.
Step I Step II
- Site recruitment and preparation within “natural replicate” settings
- Tool preparation, fidelity/monitoring
- ”Skimming the cream,” first taking those sites most ready
Step III
Step IV
Design ConsiderationsDesign Considerations
“Begin with the end in mind” – CID model Enemy of the good is the perfect: raise the floor, not the ceiling “Randomized encouragement trials” vs. randomized controlled
trials Quality Improvement group vs. TAU
How does one know the necessary ingredients of change? Attention – Expectations – Hawthorne effects? Measure them Attention dose, time in treatment? Measure them Measure change processes
Assuring fidelity to model? Measure it Ensure therapeutic relationship…and measure it Ensure family buy-in and therapist buy-in. Measure it
Need for two controls? TAU, attention control group
Overcoming Challenges: Overcoming Challenges: A Motivational ApproachA Motivational Approach
Change implementation strategies based on motivational approaches - William Miller
Practice what you preach Express empathy
to challenges of policy makers and practitioners in implementing change with population
Develop discrepancy between ideal and current Success of evidence-based treatment must be
explainable, straightforward, simply stated, meaningful
Overcoming Challenges: Overcoming Challenges: A Motivational ApproachA Motivational Approach
Avoid argumentation Clinician scientists credible to policy makers
and community-based practitioners Avoid overstating the case and “poisoning
the well” Roll with resistance
Develop strategies for engagement, prepare for possible resistance
Foundation of Collaborative EffortsFoundation of Collaborative Efforts
Researcher driven
Research retains
Research skills designated as primary
One-wayUnbalanced
Continual suspicion
Shared; equal investment
Recognition of contribution by community member &
& researchers
Open; opportunities to discuss & resolve conflict
Belief in the good faith of partners; room for mistakes
Fairly distributed
Goals
Power
Skills
Communication
Trust
Degrees of collaborationDegrees of collaboration
Focus groups
Community Advisors or Advisory
Board
Community partners as paid
staffCollaboration
(+) identificationof pressing community/familyneeds(+) definition ofacceptable researchprojects or serviceinnovations
(+) provides ongoing input regarding various stages ofresearch process
(+) collaboration regarding implementation of project(+) access to researchers to provide guidance asobstacles encountered
(+) co-creationco-implementationco-evaluationco-dissemination
Points of Collaboration in the Points of Collaboration in the Research ProcessResearch Process
Study Aims Research design& sampling
Measurement& Outcomes
Procedures (recruit, retain, data
collectionImplementation Evaluation Dissemination
Definedcollaboratively
OR
Advice sought
OR
Researcherdefined
Decision madejointly
OR
Researchereducateson methods &advice sought
OR
Methods pre-determined
Defined withinpartnership
OR
Advicesought
OR
Researcherdefined
Shared responsibility (e.g. communityto recruit, researchstaff to collect data)
OR
Designed withinput
OR
Designed byresearchers
Projects areco-directed
OR
Researcherstrain community members asco-facilitators
OR
Research staffhired for project
Plans for analysisco-created to ensurequestions of bothcommunity & researchers answered
OR
Community members assist in interpretationof results
OR
Researchers analyzedata
Members ofpartnershipdefine disseminationoutlets
OR
Members of community fulfill co-author& co-presenter roles
OR
Researchers present at conferences &publish
The REACH Institute….The REACH Institute….Putting Science to WorkPutting Science to Work
- Problem area identification
- Bring key “change agents” and gatekeepers to the table (federal or state partners, consumer and professional organizations)
- Identify “actionable” knowledge among experts and “consumers”
- Identify E-B QI procedures that are feasible, sustainable, palatable, affordable, transportable
- Consumer and stakeholder “buy-in” & commitment to E-B practices
- Dissemination via partners across all 3 system levels - “with an edge” (policy/legislative strategy with relevant federal/state partners)- Training and TA/QI intervention;
all sites eventually get intervention.
- Monitoring/fidelity
- Report preparation
- Results fed back into Step II.
Step I Step II
- Site recruitment and preparation within “natural replicate” settings
- Tool preparation, fidelity/monitoring
- ”Skimming the cream,” first taking those sites most ready
Step III
Step IV