Evidence-Based Clinical Reasoning in Children's Mental Health Services Bruce F. Chorpita Eric L. Daleiden CHARPP Conference: Real Data in Real Time September

Evidence-Based Clinical Evidence-Based Clinical Reasoning in Children's Reasoning in Children's Mental Health ServicesMental Health Services

Bruce F. ChorpitaBruce F. Chorpita

Eric L. DaleidenEric L. Daleiden

CHARPP Conference: Real Data in Real TimeCHARPP Conference: Real Data in Real Time

September 10, 2007September 10, 2007

Acknowledgments

Acknowledgments

Acknowledgments

Acknowledgments

The Story of JohnThe Story of John

A Tale of Two CulturesA Tale of Two Cultures

This is John.

PremisePremise

As researchers attempt to As researchers attempt to implement multiple EBTs, they implement multiple EBTs, they come across the practitioner’s come across the practitioner’s realityreality

This was my experience with the This was my experience with the CAMHD Practice Development CAMHD Practice Development SectionSection

Practitioner ConcernsPractitioner Concerns

Fixed contentFixed content Fixed intensityFixed intensity Fixed lengthFixed length Single target Single target

approachapproach ReplacementReplacement

Empty cell Empty cell problemproblem

Crowded cell Crowded cell problemproblem

Expiration Expiration problemproblem

Aarons (2004); Addis & Krasnow (2000); Addis, Wade, & Hatgis (2004); Chorpita, Daleiden, & Weisz (2005); Kimhan & Chorpita (2006); Persons (1995)

Throw out the Bath Water:Throw out the Bath Water:Keep the BabyKeep the Baby

Move away from the idea that Move away from the idea that “Problem A gets Treatment B”“Problem A gets Treatment B”

We decided to map the decisions We decided to map the decisions made at the system level – not just made at the system level – not just the IF-THEN of treatment selectionthe IF-THEN of treatment selection

Goal: To build an Evidence Based Goal: To build an Evidence Based SystemSystem

Where should we treat the client?Where should we treat the client?

ServiceSetting

How should we treat the client?How should we treat the client?

TherapeuticPractices

ServiceSetting

Are we providing quality service to the Are we providing quality service to the client?client?


ServiceSetting

TreatmentIntegrity

Is the client getting better?Is the client getting better?


ServiceSetting

Who should treat the client?Who should treat the client?


ServiceSetting

How should we manage the treatment?How should we manage the treatment?


ServiceSetting

Treatment programs formalize these elementsTreatment programs formalize these elements


ServiceSetting

Selecting a program structures other Selecting a program structures other decisionsdecisions

Tx Program Selection


ServiceSetting

Selecting a program structures other Selecting a program structures other decisionsdecisions



ServiceSetting

Multisystemic Therapy (MST) Example

Home & Community Based

TAMS & SAMS

Instrumental &Ultimate Outcomes

MST

4 – 5 Members

Team Supervisor

Cross-Team Supervisor

Family Therapy, etc.


Valid alternatives to deciding by Valid alternatives to deciding by program?program?


ServiceSetting

How should we make decisions?How should we make decisions?



ServiceSetting

Treatment Team

Supervision


GeneralServicesResearch



ServiceSetting

Treatment Team

Supervision

Evidence-Based Services Model


Case-SpecificHistorical

Information

CausalMechanismResearch



ServiceSetting

Treatment Team

Supervision

Individualized Case Conceptualization Model


LocalAggregateEvidence


Information



ServiceSetting

Treatment Team

Supervision

Practice-Based Evidence Model

The Full System ModelThe Full System ModelGeneralServicesResearch



Information




ServiceSetting

The Phases of EvidenceThe Phases of Evidence

1.1. Data: Discretely identifiable unitsData: Discretely identifiable units

2.2. Information: Data in a context that provides it Information: Data in a context that provides it meaningmeaning

3.3. Knowledge: Information helpful to decision-Knowledge: Information helpful to decision-makingmaking

4.4. Wisdom: Knowing when to apply our Wisdom: Knowing when to apply our knowledgeknowledge

c.f., Speigler, I. (2000). Knowledge management: A new idea or a recycled concept? Communications of the Association for Information Systems, 3, 1 – 23.

The Phases of Evidence: The Phases of Evidence: ExampleExample1.1. Data: 70Data: 70

2.2. Information: 70Information: 70ºº F F

3.3. Knowledge: It is warm enough to wear shorts.Knowledge: It is warm enough to wear shorts.

4.4. Wisdom: I am giving a professional talk today, Wisdom: I am giving a professional talk today, so my knowledge that it is warm enough to so my knowledge that it is warm enough to wear shorts is irrelevant to my attire.wear shorts is irrelevant to my attire.

The Phases of Evidence: The Phases of Evidence: Managing Evidence BasesManaging Evidence Bases

1.1. Data: Define and capture relevant dataData: Define and capture relevant data

2.2. Information: Organize and analyze into Information: Organize and analyze into meaningful units for consumersmeaningful units for consumers

3.3. Knowledge: Deliver the information to the Knowledge: Deliver the information to the decision-making situationdecision-making situation

4.4. Wisdom: Prioritize use of the knowledge basesWisdom: Prioritize use of the knowledge bases

General Services Research:General Services Research:Turning Data into Knowledge ITurning Data into Knowledge I

Meta Analysis of Children’s Treatment Meta Analysis of Children’s Treatment ResearchResearch– Over 300 studies, 60 yearsOver 300 studies, 60 years– Largest meta-analysis to dateLargest meta-analysis to date– Made dynamic for providersMade dynamic for providers

Results: AnxietyResults: Anxiety

Best SupportBest Support CBTCBT EducationEducation ExposureExposure ModelingModeling

Good SupportGood Support Assertiveness TrainingAssertiveness Training Cognitive Behavior Cognitive Behavior

Therapy and Therapy and MedicationMedication

Cognitive Behavior Cognitive Behavior Therapy with ParentsTherapy with Parents

HypnosisHypnosis Play TherapyPlay Therapy RelaxationRelaxation

Results: TraumaResults: Trauma

Best SupportBest Support CBTCBT

Good SupportGood Support Cognitive Behavior Cognitive Behavior

Therapy with ParentsTherapy with Parents Play TherapyPlay Therapy

Results: AttentionResults: Attention

Best SupportBest Support Contingency Contingency

ManagementManagement Parent Management Parent Management

TrainingTraining Self Verbalization Self Verbalization Behavior Therapy and Behavior Therapy and

MedicationMedication

Good SupportGood Support BiofeedbackBiofeedback Contingency Contingency

ManagementManagement EducationEducation Management Training Management Training

and Problem Solvingand Problem Solving Physical ExercisePhysical Exercise Relaxation and Physical Relaxation and Physical

Exercise Exercise Social Skills and Social Skills and

Medication Medication

Results: DepressionResults: Depression

Best SupportBest Support CBTCBT CBT and MedicationCBT and Medication

Good SupportGood Support Client Centered Client Centered

TherapyTherapy Cognitive Behavior Cognitive Behavior

Therapy with ParentsTherapy with Parents Interpersonal TherapyInterpersonal Therapy Play TherapyPlay Therapy Relaxation Relaxation

Results: ConductResults: Conduct

Best SupportBest Support Assertiveness TrainingAssertiveness Training Cognitive Behavior Cognitive Behavior

TherapyTherapy Contingency Contingency

ManagementManagement ExposureExposure ModelingModeling Parent Management Parent Management

TrainingTraining Social Skills Social Skills

Good SupportGood Support Anger ControlAnger Control Client Centered TherapyClient Centered Therapy Communication SkillsCommunication Skills Functional Family TherapyFunctional Family Therapy Multidimensional Treatment Multidimensional Treatment

Foster CareFoster Care Multisystemic TherapyMultisystemic Therapy Parent Management Training and Parent Management Training and

Problem SolvingProblem Solving Physical ExercisePhysical Exercise Problem SolvingProblem Solving Rational Emotive TherapyRational Emotive Therapy RelaxationRelaxation Transactional Analysis Transactional Analysis

How do we get this knowledge to the How do we get this knowledge to the treatment team?treatment team?

GeneralServicesResearch



Information




ServiceSetting

Treatment Team

Supervision

ClientProgress

TreatmentIntegrity

General Services Research:General Services Research:Turning Data into Knowledge IITurning Data into Knowledge II

Common Elements approachCommon Elements approach– Identified components of evidence based Identified components of evidence based

practicespractices– Generated profiles using a decision tree Generated profiles using a decision tree

approachapproach

Is there a different Is there a different level of analysis?level of analysis?

Protocol Protocol Protocol

Family


IncredibleYears

PCITDefiant Children

Parent Training


Protocol Protocol Protocol

Family

PracticeElement

PracticeElement

PracticeElement

PracticeElement

PracticeElement

PracticeElement


Incredible Years

PCITDefiant

Children

Parent Training

Commands CommandsAttending

Time Out

Rewards

Time Out

These are “practice elements.

”

ExampleExample

Attending

Objectives: to increase the amount of positive attention provided to the child, even if the child has misbehaved

at other times during the day

to teach the caregiver to attend to positive behaviors

to promote the child’s sense of self-worth

Steps:

Provide rationale Emphasize the importance of providing positive attention to the child. Elicit the caregiver’s opinion about how attention affects behavior and

people’s motivation to do a good job. Have the caregiver describe his or her best and worst “managers”

and the caregiver’s motivation to work for each. Lead the caregiver to recognize that how he or she was treated

affected the caregiver’s desire to work. Discuss how the child’s behavior may be affected by the caregiver’s

behavior towards the child and how the child’s desire to behave can be increased by improving the caregiver-child relationship.

Set aside one-on-one time for caregiver and child

Encourage the caregiver to set aside a block of time (e.g., 10 minutes) each day devoted to joining the child in an activity the child has chosen.

Teach caregiver to provide positive and descriptive commentary

Show the caregiver how to demonstrate sincere interest in the child’s activities while they are playing.

Instruct the caregiver to provide enthusiastic descriptive (e.g., “You are drawing a tree”) and/or positive (e.g., “I like the way you stacked the blocks”) commentary and praise regarding the child’s behavior.

Encourage caregiver to engage in child’s activity

Suggest that the caregiver become actively involved in the play activity by imitating the child’s behavior in order to demonstrate approval.

Restrict criticism, questions, and commands

It is important that the child lead the activity; that is, the caregiver should refrain from making suggestions, asking questions, and criticizing the child.

Allow the child to use his or her imagination (e.g., coloring the green or making up new rules to a game) without caregiver input about the “correct” way to do things.

Anticipate difficulties When the procedure is initially implemented, the child may engage in negative behavior that characterizes the usual caregiver-child interaction. When this occurs, the caregiver should:

consistently ignore negative behavior by looking away; refrain from scolding the child so as to avoid providing negative

attention for misbehavior; end one-to-one time if disruptive behavior continues or is

dangerous. Over time, however, it is expected that consistent positive attending will result in decreased negative behavior and increased positive caregiver-child interactions.

Attending

Use This When:

To improve the quality of the caregiver-child relationship.

Practitioner Guide

For CaretakerFor Caretaker

Exposure

Relaxation

Cognitive

Psychoeducational-Child

Modeling

Self-Monitoring

Therapist Praise/Rewards

Psychoeducational-Parent

Self-Reward/Self-Praise

Maintenance/Relapse Prevention

Problem Solving

Tangible Rewards

Relationship/Rapport Building

Assertiveness Training

Praise

Social Skills Training

Guided Imagery

Communication Skills

Educational Support

Response Prevention

Differential Reinforcement

Goal Setting

Play Therapy

Emotional Processing

Natural and Logical Consequences

Parent Coping

Monitoring

Peer Pairing

Stimulus Control or Antecedent Man.

Time Out

Behavioral Contracting

Activity Scheduling

Catharsis

Family Therapy

Hypnosis

Supportive Listening

Individual Therapy for Caretaker

PracticeWise Evidence Based Services Practice Element ProfileAnxiety Levels I and II –

September 05, 2007

0.0 0.2 0.4 0.6 0.8 1.0

Proportion of Studies (n = 91)

PracticeWise Evidence Based Services Practice Element ProfileTrauma Levels I and II –

September 05, 2007


CognitiveExposure

Psychoeducational-ChildRelaxation

Maintenance/Relapse PreventionPsychoeducational-Parent

Assertiveness TrainingCommunication Skills

ModelingProblem Solving

Emotional ProcessingGoal Setting

Relationship/Rapport BuildingSelf-Monitoring

Tangible RewardsDifferential Reinforcement

Natural and Logical ConsequencesParent Coping

PraiseMonitoring

Play TherapyStimulus Control or Antecedent Man.

Time Out

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

PracticeWise Evidence Based Services Practice Element Profile

Attention Levels I and II – September 05, 2007


Praise

Problem Solving

Tangible Rewards



Time Out

Commands

Modeling

Monitoring

Relaxation


Differential Reinforcement

Response Cost


Educational Support


Biofeedback/Neurofeedback


Parent Coping

Guided Imagery

Insight Building

Goal Setting

Natural and Logical Consequences

Peer Pairing


Self-Monitoring

Talent or Skill Building

Attending


PracticeWise Evidence Based Services Practice Element Profile

Depression Levels I and II – September 05, 2007


Cognitive

Psychoeducational-Child

Maintenance/Relapse Prevention

Activity Scheduling

Problem Solving

Self-Monitoring

Goal Setting


Relaxation





Talent or Skill Building

Guided Imagery

Modeling



Assertiveness Training

Insight Building

Parent Coping


Tangible Rewards

Crisis Management

Exposure

Play Therapy

Supportive Listening

0.0 0.2 0.4 0.6 0.8 1.0

0.0 0.2 0.4 0.6 0.8 1.0

PracticeWise Evidence Based Services Practice Element ProfileConduct Levels I and II –

September 05, 2007


PraiseTangible Rewards

Problem SolvingTime Out

CommandsPsychoeducational-Parent

ModelingMonitoringCognitive

Differential ReinforcementCommunication Skills

Goal SettingResponse Cost

Therapist Praise/RewardsSocial Skills Training

Maintenance/Relapse PreventionBehavioral Contracting

Natural and Logical ConsequencesAttending

Self-MonitoringRelationship/Rapport Building

Stimulus Control or Antecedent Man.Parent Coping

Family TherapyPsychoeducational-Child

Educational SupportPeer Pairing

RelaxationTalent or Skill Building

Functional AnalysisMarital Therapy

Assertiveness TrainingFamily Engagement

Insight BuildingCrisis Management

Individual Therapy for CaretakerSelf-Reward/Self-Praise

ExposurePlay Therapy

Guided ImageryInterpretation

Activity SchedulingSupportive Listening

Line of Sight SupervisionCase Management

The Full System ModelThe Full System ModelGeneralServicesResearch



Information




ServiceSetting

Case Specific History:Case Specific History:Turning Data into Knowledge IIITurning Data into Knowledge III

Clinical DashboardClinical Dashboard– ProgressProgress– PracticesPractices

Individual Case Supervision Form Case Number: 6

Age: 12 Diagnosis: Generalized Anxiety Disorder Gender: M

Child

Mother

Father

Other

Progress Measure:

Fear rating

Rewards

Commands

Time Out

Praise

Problem Solving

Parent Monitoring

Response Cost

Ignoring/DRO

Stimulus Control/Antecedents

Attending/Directed Play

Modeling

Cognitive

Parent Psychoeducation

Self-Monitoring

Relaxation

Exposure

Maintenance

Psychoeducation

Activity Scheduling

Skill Building

Social Skills

Self Monitoring

Other

Other

Other

Other

Other

Other

Other

Other

Days Since First Event

0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

A Foray into WisdomA Foray into Wisdom

ClinicalProgress?

Clinical Dashboard:Progress Pane

Continue plan until goals met

Significant concerns?

Critical Incidents & Complaints

Reports, etc.

Consult with specialists as

needed

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane

Prob. w/ TxSelection?

TreatmentIntegrity?

Therapy Protocols,Dashboards, EBS DB,

ConsultationConsider adding consultation or

training supports

Identify barriers and revise plan

Options1. Increase supports2. Change Intervention3. Further Consultation4. Add intervention

EBS DB, Local Best Practices, Tx Team

New Case?

no

yes Select Evidence-Based Service (EBS)

EBSDatabaseStart


ClinicalProgress?





Reports, etc.


needed

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, EBS DB


TreatmentIntegrity?



training supports




New Case?

no


EBSDatabaseStart


ClinicalProgress?





Reports, etc.


needed

yes

no

no

yes

no

yes

no

yes



TreatmentIntegrity?



training supports




New Case?

no


EBSDatabaseStart



Child

Mother

Father

Other

Progress Measure:

Fear rating

Rewards

Commands

Time Out

Praise

Problem Solving

Parent Monitoring

Response Cost

Ignoring/DRO



Modeling

Cognitive


Self-Monitoring

Relaxation

Exposure

Maintenance

Psychoeducation

Activity Scheduling

Skill Building

Social Skills

Self Monitoring

Other

Other

Other

Other

Other

Other

Other

Other


0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100


ClinicalProgress?





Reports, etc.


needed

yes

no

no

yes

no

yes

no

yes



TreatmentIntegrity?



training supports




New Case?

no


EBSDatabaseStart

This tells you the treatment types that work for this problem.

This tells you the practice elements associated with those treatment types.



Child

Mother

Father

Other

Progress Measure:

Fear rating

Rewards

Commands

Time Out

Praise

Problem Solving

Parent Monitoring

Response Cost

Ignoring/DRO



Modeling

Cognitive


Self-Monitoring

Relaxation

Exposure

Maintenance

Psychoeducation

Activity Scheduling

Skill Building

Social Skills

Self Monitoring

Other

Other

Other

Other

Other

Other

Other

Other


0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

Do the practices fit the problem?

A Bit of Practice WisdomA Bit of Practice Wisdom

ClinicalProgress?





Reports, etc.


needed

yes

no

no

yes

no

yes

no

yes

Clinical Dashboard:Practice Pane, PW DB


TreatmentIntegrity?

Therapy Protocols,Dashboards, PW DB,


training supports



PW DB, Local Best Practices, Tx Team

New Case?

no


PWDatabaseStart



Child

Mother

Father

Other

Progress Measure:

Fear rating

Rewards

Commands

Time Out

Praise

Problem Solving

Parent Monitoring

Response Cost

Ignoring/DRO



Modeling

Cognitive


Self-Monitoring

Relaxation

Exposure

Maintenance

Psychoeducation

Activity Scheduling

Skill Building

Social Skills

Self Monitoring

Other

Other

Other

Other

Other

Other

Other

Other


0 10 20 30 40 50 60 70 80 90 100

0

1

2

3

4

5

6

7

8

0 10 20 30 40 50 60 70 80 90 100

Select new practices?

Seek Consultation?Seek Consultation?

Results in HawaiiResults in HawaiiFrom Daleiden, Chorpita, Arensdorf, Donkervoet, & Brogan (2006)From Daleiden, Chorpita, Arensdorf, Donkervoet, & Brogan (2006)

Youth Receiving Hospital-Based Residential Services

96

80

56

30 28

37

0

20

40

60

80

100

FY1999 FY2000 FY2001 FY2002 FY2003 FY2004 FY2005

Fiscal Year

Yo

uth

per

Qu

arte

r (M

ean

)

Avoid Hospital Residential Services for Conduct Avoid Hospital Residential Services for Conduct Disorders: MST InitiativeDisorders: MST Initiative

MSTBegan

EBS TaskForce Began

33

17

51

9587 90

0

20

40

60

80

100F

Y19

96

FY

1997

FY

1998

FY

1999

FY

2000

FY

2001

FY

2002

FY

2003

FY

2004

Fiscal Year

Acc

epta

ble

Rat

ing

(C

om

ple

x %

)

Complexes Meeting Quality Standards for System Performance

Quality Reviews: System PerformanceQuality Reviews: System Performance

Quality Dimension Examples: Functional Assessment Service Coordination & Transition Long-term view Caregiver Supports Service Plan & Implementation Effective Results Service Array & Integration Monitoring & Modification

48 48

61

8074

8589

94 94

0

20

40

60

80

100F

Y19

96

FY

1997

FY

1998

FY

1999

FY

2000

FY

2001

FY

2002

FY

2003

FY

2004

Fiscal Year

Acc

epta

ble

Rat

ing

(C

ase

%)

Cases Rated as Acceptable in Child Status

Quality Reviews: Child StatusQuality Reviews: Child Status

Quality Dimension Examples: Learning Progress Community Home Personal Responsibility Caregiver Functioning Safety/Personal Well-being Child/Family Satisfaction Emotional/Behavioral Well-being

Rate of Improvement?Rate of Improvement?

CAFAS 8-Scale Total

-1.1

-1.5-1.3 -1.3

-1.4-1.7 -1.8 -1.8

-2.3 -2.3-2.6 -2.6

-3.0

-2.5

-2.0

-1.5

-1.0

-0.5

0.0

2002.12002.22002.32002.4 2003.12003.22003.32003.4 2004.12004.22004.32004.4

Fiscal Quarter

Ch

ang

e p

er M

on

th(M

+/-

SE

)

Final Effect Size for Change = .07/mo, .84/yr

Getting Better at Getting Them BetterGetting Better at Getting Them Better

Expected Rate of Improvement Across Time

0

30

60

90

120

150

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

Months Since Intake

CA

FA

S 8

-Sca

le T

ota

l (M

ean

)

End of2004

Start of2002

127

97 9991

149

119

0

20

40

60

80

100

120

140

160

2001

2002

2003

2004

2001

2002

2003

2004

2001

2002

2003

2004

2001

2002

2003

2004

2001

2002

2003

2004

2001

2002

2003

2004

Ave

rag

e S

core

HospitalResidential

CommunityResidential

TherapeuticGroupHome

TherapeuticFosterHome

CAFAS 8-Scale Total Scores within 45-days of Admission to Specific Service

MultisystemicTherapy

IntensiveIn-Home

Greater Impairment Treated at Less Greater Impairment Treated at Less Restrictive Levels of CareRestrictive Levels of Care

Cost per OutcomeCost per Outcome

$4,640$3,535

$2,087$1,080 $654 $564

$12,477

$9,325

$6,828

$0

$2,000

$4,000

$6,000

$8,000

$10,000

$12,000

$14,00020

02

2003

2004

2002

2003

2004

2002

2003

2004

Exp

en

dit

ure

s (U

S$)

CALOCUSLevel of Care

(0 - 5)

ASEBAParent CBCL

T-Score

CAFAS8-Scale Total

(0 - 240)

Service Expenditures per Unit of Improvement(Annual Cost per Youth / Average Annual Rate of Improvement)

This is John now.

Thank You!Thank You!Questions?Questions?

Documents

Evidence-Based Clinical Reasoning in Children's Mental Health Services Bruce F. Chorpita Eric L. Daleiden CHARPP Conference: Real Data in Real Time September