Management Practices in Addiction Treatment Programs

K. John McConnell, PhD

Oregon Health & Science University

Supported by a grant from NIDA 1R01DA020832

Two research threads

Substance Abuse:– A call for research on the “business” of addiction treatment

(Kimberly & McLellan, JSAT 2006)

Economics:– Typically treat “management quality” as a fixed effect– New research to measure management practice (Bloom &

Van Reenen, QJE 2007) Innovative telephone survey to “score” management practice Strongly correlated with firm performance (>700 firms in 4

countries)

Study Objective

Adapt management practice survey tool to addiction treatment programs

Are better management practices associated with better client treatment?

Study Sample

Programs from the NIATx 200 project– NIDA 1R01DA020832– Approximately 200 agencies in 5 States (MA, MI, NY, OR,

WA) randomized to different levels of support (coaching, interest circles, learning sessions, and combinations of all 3)

We use data on 147 surveys by 3 interviewers– ~20 programs not interviewed because they were part of a

multi-site agency– Excluded 5 programs with different interviewers– Excluded 5 with interview scored as “not reliable”– Excluded 20 with missing data

Steps to measure management practices

1. Developing management practice scoring– Adapted scorecard for 14 questions on 4 areas:

Client intake and retention Quality improvement & data monitoring Program targets Incentives & employee management

– 60 minute phone interview of program executive sponsors 2. Obtaining unbiased responses

– Interviews run by 3 MPH level research assistants – “Double-blind”:

Interviewers do not know program performance Interviewees are not informed (in advance) they are scored

– “Open-ended” question format

(13) Promoting high performersa. Tell me about your promotion system.b. What about poor performers – do they get promoted more slowly? Are there any examples you

can think of?c. How would you identify and develop (i.e. train) your star performers?d. If two people both joined the agency 5 years ago and one was much better than the other

would he/she be promoted faster?

Scoring grid: Score 1

People are promoted primarily upon the basis of tenure

Score 3

People are promoted upon the basis of performance

Score 5

We actively identify, develop and promote our top performers

Examples: An agency based on an individual’s commitment to the company measured by experience.

An agency has no formal training program. People learn on the job and are promoted based on their performance on the job.

At one agency each employee is given a red light (not performing), amber light (doing well and meeting targets) a green light (consistently meeting targets; very high performer)

Analysis

Negative binomial, clustered std. err. Outcome: Time to treatment

– Days from first phone call to first appointment Pseudo-patient phone calls by NIATx researcher Caller identified herself as NIATx researcher Average of 7 calls made to each program

Independent variables– Average z-score of management practices across 14 questions

z-score removes scaling issues between questions– Employee FTE– State & interviewer fixed effects– Day of week for pseudo-patient phone call– Month of survey

Results

Distribution of management practice scores for 147 addiction treatment programs

Double-scored interviews(N = 14)

Model results

Main analysis:– Time-to-treatment

1081 observations on 147 programs Management score significantly associated with time-to-treatment (P

<0.01) Sub-analyses

– 7 of 14 management practices were correlated with better time to treatment

– Of the 4 areas, 3 were associated with better time-to-treatment Client intake and retention Quality improvement & data monitoring Program targets

– One was not Incentives & employee management ?

Limitations

Endogeneity Double scoring vs. test-retest Programs selected from NIATx 200 project

Concluding thoughts

Management practices can be measured and are associated with client treatment

Are they associated with financial outcomes? Do employee incentives matter or are substance

abuse treatment programs “different”? NIATx aims for process improvement; can we design

interventions to improve management practices?

Thank you

University of Wisconsin– David Gustafson, PI– Andy Quanbeck– Alice Pulvermacher– Jay Ford– Anna Wheelock– Renee Hill

Oregon Health & Science University

– Dennis McCarty– Kim Hoffman– Marie Shea– Gretchen Luhr– Susan Rosenkranz– Traci Rieckmann

Stanford University– Nick Bloom

Research supported by NIDA R01DA020832