7/28/2011

1

Jacqueline Sparks, Ph.D.Associate Professor

University of Rhode Island

TEACHING OUTCOME-INFORMED PRACTICE

Integrating CDOI &University Graduate Training jsparks@uri.edu

Colleagues

Department of Human Development & Family Department of Human Development & Family Studies – University of Rhode Island

Couple & Family Therapy Program

Jerome Adams Tiffani Kisler Jacqueline Sparks Dale Blumen

7/28/2011

2

Our Program

Students accrue 500 direct client contact

S d i 1 l l i h l

AAMFT Accredited

Students intern 1 year at local agencies, schools, residential treatment, or hospital

Students carry a caseload at on-campus clinic

2-year, full-time, 60 credits

Supervision provided by approved supervisors

Where We’re Headed

Identifying and connecting with stakeholder interests

Cultivating allies

First steps

Integration (curricula, clinical training & supervision, evaluation)

Tools

Results

Maintaining & Building

7/28/2011

3

Stakeholders

Program (faculty & students)Program (faculty & students)

Clients (served by our clinic)

Department & University

Communityy

Professional Field

Program Interests

Train effective practitioners

Create a unifying training model that conforms with core program philosophy

Provide optimum service to clients

Provide practice-based research opportunities

Offer new options for community network providers Offer new options for community network providers

Develop a competitive profile as a forward leaning, innovative program

Align with University and the field’s interests in outcome-based learning.

7/28/2011

4

Train Effective Practitioners

Clients have largely been overlooked as teachers

Th f l h h Therapists performing poorly can improve when they routinely integrate client feedback into their work

A feedback system would give us a field-tested method for helping trainees with poorer outcomes improve.

CDOI Fit with Program Philosophy

Collaborative & Strengths-based – Teaching students to routinely ask for and respond to client feedback embodies routinely ask for and respond to client feedback embodies core values related to collaboration and a strengths focus

Integrative – CDOI as a centerpiece for student integration of diverse treatment models

Commitment to diversity –With client voices guiding With client voices guiding clinical work, trainees could learn from and better assist persons whose social and cultural locations differed from their own

7/28/2011

5

Improve Clinic Services

Drop outs are a significant problem in the delivery of mental health services, averaging at least 47% (Wierzbicki & Pekarik, 1993)

Training clinics have more premature terminations and fewer successful outcomes than in other outpatient settings (Callahan et al., 2009)

Our clinic serves financially disadvantaged clients –

i i f li f provision of quality care of particular concern

CDOI could serve as a vehicle for improving the effectiveness of our clinic services

Practice-Based Research Opportunities

MFT research and practice historically have been disconnected if not iewed as in opposition disconnected, if not viewed as in opposition

Clinics operate “in the dark,” and students absorb the message that continuous assessment of services is not important

CDOI would instill in students a sense of the ethics and utility of a client – service system loop

CDOI would supplement our ongoing research and client satisfaction protocols

7/28/2011

6

Creating a Competitive Profile

Interested in

To the University of Rhode Island Couple and Family Therapy

Program Information Interview

attracting and recruiting top students from across the country and abroad.

CDOI offered an i

WELCOME

opportunity to showcase our program as unique and exceptional.

Community Interests

Larger systems of care serving children and families in our area moving to measurably identify treatment success or failure

Desire for University ll b i i collaborative service

partnerships that introduce innovative, effective, and efficient new models

7/28/2011

7

Department & University Interests

Desire for exceptional, empirically-based scholarship

Interest in objective, outcome-based models of evaluating student learning

Interest in meaningful service and connection with local Interest in meaningful service and connection with local communities, state agencies, and statewide initiatives

Emphasis on attracting a diverse student body and faculty and offering a diversity-infused curriculum

Interest in global engagement and partnership

Interests of the Field

Calls for the provision of “safe, effective, patient-centered, ti l ffi i t d it bl ” h lth i l di timely, efficient, and equitable” health care, including mental health (Committee on Quality of Health Care in America, 2001, pp. 7-8; President’s New Freedom Commission on Mental Health, 2003)

Mental and behavioral health organizations attempt to define effective, safe care (e.g., see APA Presidential Task define effective, safe care (e.g., see APA Presidential Task Force on Evidence-Based Practice, 2006)

The AAMFT defines core competencies and calls for shift from input-oriented to outcome-based education.

7/28/2011

8

Fit with Core Competencies

Domain 4: Therapeutic Interventions Match treatment modalities and techniques to clients’ needs Match treatment modalities and techniques to clients needs,

goals, and values. 4.3.1 Facilitate clients developing and integrating solutions to

problems. 4.3.16 Modify interventions that are not working to better fit

treatment goals. 4.3.10 Evaluate treatment outcomes as treatment progresses. 4.4.3 Evaluate treatment outcomes as treatment progresses. 4.4.3 Evaluate clients’ reactions or responses to interventions.

4.4.4 Evaluate clients’ outcomes for the need to continue, refer, or

terminate therapy. 4.4.5

Input-Oriented Clinical Training

Training in model h itechniques

Emphasis on use of EBT without client input

Skills training

Emphasis on accrued Emphasis on accrued clinical hours

Intensive supervision (often self-report)

7/28/2011

9

CompetenceCompetence====

EffectivenessEffectiveness

Input to Outcome-Based Training

Not Related to OutcomeNot Related to Outcome Related to OutcomeRelated to Outcome

Skills training

Supervision

Hours of experience

Feedback protocols for improving counselor efficacy

Reese, R. J., Usher, E. L., Bowman, D., Norsworthy, L., Halstead, J., Rowlands, S. et al. (2009). Using client feedback in psychotherapy training: An analysis of its influence on supervision and counselor self-efficacy. Training and Education in Professional Psychology, 3(3), 157-168.Whorthen, V. E.. & Lambert, M. J. (2007). Outcome oriented supervision: Advantages of adding systematic client tracking to supportive consultations. Counseling and Psychotherapy Research, 7(1), 48-53.

7/28/2011

10

Recruiting the Team

Once CDOI-based training is connected to the varying interests and the empirical literature, support follows. . .

Cultivate the CDOI Garden

E l i th t CDOI d t Explain that CDOI does not replace a preferred view or treatment approach

Work collaboratively to create a team that has a shared mission f i l t ti of implementation

Identify projects and responsibilities

7/28/2011

11

Choose Measures

W h (ORS) (CORS) (SRS) We chose (ORS) (CORS) (SRS) (CSRS)

Met standards of practicality for everyday clinical use without sacrificing validity and

li bilitreliability.

Brevity and face validity of the instruments facilitate administration

Choose Tracking System

We are using ASIST

Data is entered into ASIST (either in the room on a laptop or after the session)

ASIST automatically calculates and graphically depicts a trajectory of change and target benchmark

7/28/2011

12

Integration

Begin at applicant interview

S di f

How URI CFT Training Is Unique

Summer reading for incoming students

Students systematically gather formal feedback from clients at each session to guide treatment (Outcome Management [OM]).

Duncan, B. (2011). What therapists want: It’s Duncan, B. (2011). What therapists want: It’s certainly not money or fame. certainly not money or fame. Psychotherapy Psychotherapy Networker, May/June, 40Networker, May/June, 40--43, 47, 62.43, 47, 62.Duncan, B. (2011). Opening a path: From what is Duncan, B. (2011). Opening a path: From what is to what can be. to what can be. Psychotherapy Networker, Psychotherapy Networker, May/June 46May/June 46--4747

Pre-Practicum

The Heroic Clients, Heroic Agencies Manual serves as a basic text

Students receive hands on training in CDOI

Students role play introducing, p y g,integrating the measures and being informed by client feedback

Available for download at Available for download at www.heartandsoulofchange.com www.heartandsoulofchange.com

7/28/2011

13

Theory Course

Second course focuses on postmodern to present

H i Cli t i i t d d i th d h lf f th Heroic Client is incorporated during the second half of the second course

Students learn current evidence for feedback, common factors, and understand evidence based practice as currently defined by APA

Learn and evaluate the principles and application of practice-based evidence with an emphasis on current research and debates in the field and as an integrative theory in family therapy.

Practicum

CDOI provides a ready- made structure

S d l h f ll i l Students learn how to respectfully, yet persistently, request and respond to client feedback

Students learn to identify problems and address them before clients drop out

Clients come to trust not only their therapist’s only their therapist s desire to learn their views, but the significance of their own perspectives to treatment success.

7/28/2011

14

Supervision

Therapists are required to bring graphs, measures, d ASIST i l f ll i i

Supervisors use information from the measures to structure the supervisory

and ASIST signals for all supervision

p yconversation, including requests for specific video data

Supervision Decision Points

Supervisors can prioritize at- Supervisors can prioritize atrisk cases that need immediate attention

Helps to generate conversations about different approaches that may better approaches that may better fit a given client’s preferences and expectations.

7/28/2011

15

Identifying and Amplifying Change

Visual trajectories help trainees, supervisors, and clients demarcate change and plan for termination

Supervisors have a tool for privileging the client’s unique lived experience over theory

Shorter lengths of stay with fewer cases extending beyond 4 months where no measurable change has occurred.

Including the Client in Supervision

Supervisor/supervisee conversations are now more intimately connected to client experiences

Clients’ voices become the central focus of supervision conversations and planning.

7/28/2011

16

Live Supervision

Live supervision using one-p gway mirror gives supervisor real time information

Supervisor can make more targeted suggestions for in-session process and homework

Live supervision using CDOI Live supervision using CDOI has resulted in more productive supervisory focus and more efficient and successful sessions

Evaluation

Students conduct a self-assessment at the end of each semester of practicumsemester of practicum

The Student Clinical Self-Assessment Report requires students to analyze their outcome data for % of clients reaching benchmark, average alliance score, length of stay, and % of cases defined as drop-out

From this assessment and supervisor From this assessment and supervisor evaluation, students set goals for their next practicum

Evaluations are directly connected to outcomes identified by clients.

7/28/2011

17

Evaluation

Student caseload ratings of outcome and alliance h ld b f di ishould not be sources of grading or promotion

Grading reflects students’ engagement in the process and willingness to learn from clients

Our mantra is: “There is no There is no negative feedback, only negative responses to it.”

Expanding the Learning

Cultivate knowledge and appreciation of CDOI at community internship sites

Connect CDOI to site needs related to outcome evaluation

Invite site staff and supervisors to

community internship sites Encourage students to conduct CDOI-based research at

sites

Invite site staff and supervisors to trainings and roundtable discussions

Introduce the goal of requiring students to track all outcomes as part of their training and evaluation

7/28/2011

18

School-based Site Research

ORS/CORS assessments of 72 students, aged 6-18

f Students referred to Ledyard Youth Services by guidance counselors, school psychologists, teachers, and parents between 2009 and 2011

Anger, poor communication with family members, interpersonal difficulties with peers, bullying, depression, poor academic performance,

and substance abuse

ORS, 38 youth; CORS 34 children

Results – Ledyard Youth Services

Youth and children experienced significant gains in pre-post analysis (average mean increased 7 3 points pre post analysis (average mean increased 7.3 points ORS and 5.0 on CORS)

At or above clinical cutoff scores for ORS increased from 39.5% to 81.6%; CORS, 50% to 79.4%

Combined ORS and CORS sample 93% p %reached benchmark

7/28/2011

19

Additional Sites Adopting

Large multiple service agency for at-g p g yrisk children and families integrating CDOI throughout

Regional modified wrap-around program instituting CDOI into service delivery in the community

Office-based community mental health agency and residential treatment program open to training; several CDOI practitioners on staff and interning

7/28/2011

20

7/28/2011

21

7/28/2011

22

Client Feedback on ORS Use

Sample: 9 adults and 2 16-year old boys

1 family, 3 individuals, 2 unmarried couples

Caucasian, African American, mixed race

7, somewhat helpful; 2 extremely helpful; , p ; y p ;1, unhelpful; 1, neither helpful nor unhelpful

81.8% somewhat or extremely helpful

7/28/2011

23

Qualitative Responses ORS

Extremely helpful: “it gave a good basis of how h k h d d h I f l b ” d “ the week had gone and how I felt about it,” and “a tool for better family living”

Somewhat helpful: “feeling good,” “fairly and consistently positive,” “good visual tool,” “good jumping off point for the session,” “I don’t always recall where we are or how we progressed”

Neither helpful nor unhelpful: “it was nice to learn about my personal progress but charting us against what is ‘normal’ feels counter-intuitive”

Client Feedback on SRS

6 of 11 responses (54.6%), somewhat helpful

f f 3 of 11, extremely helpful

1, unhelpful; 1, neither helpful nor unhelpful

81.9%, somewhat or extremely helpful

7/28/2011

24

Qualitative Responses SRS

Somewhat or Extremely Helpful: Importance of letting the therapist know how the are feeling letting the therapist know how they are feeling about the session; “made me consider the things we went over in a bit more depth”

Neither Helpful nor Unhelpful: “trying to rate on the spot gave me no time to reflect on the session. F l ti d t ith For example, sometimes one does not agree with what a therapist might say or advice until on can process it”

3 and 6-Month Follow-Up

69 clients contacted 3 and/or 6 months post therapy

25 l 44 f l d 5 67 LOS 9 07 25 male, 44 female; aged 5 to 67; mean LOS 9.07 sessions

69 completed ORS last session; 36, 3 months; 38, 6 months

ORS was administered telephonically

7/28/2011

25

3 and 6-Month Follow-up (n = 69)

ORS1: 22.33 ORS2 31.00 ORS3 31.01 ORS4 30.12

Clinical Outcomes (208 cases)

Therapist n = 31

4 ½ -year period

Av. # of sessions: 7

Av. LOS 3 mos.

% of clients reaching benchmark 71%

7/28/2011

26

Student Feedback

CDOI : Gives a conceptual and practical framework to t bli h l f t d fidestablish early comfort and confidence

Provides practical guidance for structuring the session

Provides a format for clients to guide treatment

Provides an incentive for expanding beyond one’s comfort zone and for learning new skillsone s comfort zone and for learning new skills

Helps to minimize guesswork regarding whether progress is being made or a case may be at risk

Provides a format for learning about children’s views

Expanding One’s Comfort Zone

7/28/2011

27

Counselor Education at a Crossroad

I found the paper compelling (JMFT Associate Ed.)

I believe this is a watershed article that will have a lasting impact on how supervision is done in the future—both in mft and in other mental health professions. (JMFT Reviewer)

The authors have done a stellar job on an important piece that I believe will have a lasting impact on the field (JMFT Reviewer)

Sparks, J. A., Kisler, T. A., Adams, J. F., & Blumen, D. G. (in press). Teaching accountability: Using client feedback to train family therapists.Journal of Marital and Family Therapy.