Linking Multiple Administrative Data Sets to Measure Outcomes for Behavioral Health

Services

Oklahoma Department of Mental Health and Substance Abuse Services

Tracy Leeper, Grant Projects Manager

Funded by CSAT

Focused on Administrative Data

All Adults Who Receive SA

Treatment Services in a Given Year

Two Year Pre & Post Follow-Up Periods

Determine Costs Associated with Outcomes

Outcomes Substance Abuse Treatment Outcomes Monitoring System

Administrative Data

Ongoing & Sustainable

Empirically-Based & Objective

Less Resource Intensive

Longitudinal

Flexible for Populations & Time Frames

Unobtrusive

No Direct Measures for Certain IndicatorsNo Direct Measures for Certain Indicators Little Drug Use InformationLittle Drug Use Information

Level of Functioning or SymptomotologyLevel of Functioning or Symptomotology

Indirect MeasuresIndirect Measures Look at Contribution to Society Versus Cost to

Society

Employed, Paying Taxes

Decrease in Jail and Inpatient Days

Administrative Data

Sources of Data

DMHSAS - State-funded AOD & Mental Health

DPS – DUI Convictions

DOC – Incarcerations, Probation & Parole

OTC – Household Income

OESC – Wage & Unemployment Benefits

Sources of Data

OSDH - Mortality Data

OHCA – Medicaid Claims

OSBI – Arrests/Dispositions

DHS – TANF, Food Stamps, Child Welfare

Legislation

Interagency Pilot Projects

Quid Pro Quo

Good Ol’ Boy Approach

Getting “Buy-In”

Leave Plenty of Time for:Executing Agreements

After Administrative “OK,” Still Need MIS Staff Buy-InDiscussionsWorking Out DetailsMust Have a Working Relationship with Other

Agencies’ MIS Staff

Getting “Buy-In”

Confidentiality Concerns

42 CFR

HIPAA

Interagency Agreement

IRB Approval

42 Codified Federal Regulation (CFR)

Medical Emergencies Research Activities Audit and Evaluation Activities

Subchapter A, Part 2Confidentiality of Alcohol and Drug

Abuse Patient RecordsSubpart D – Disclosures without Patient Consent

Health Insurance Portability and Accountability Act of 1996 (HIPAA)

C. General Risk. (§ 164.506)Covered entities would be able to use or disclose an individual’s protected health information without authorization for treatment, payment, and health care operations.

B. Definitions. (§160.103 and § 164.504)16. Health Care Operations.- Conducting quality assessment and improving activities, including evaluating outcomes, and developing clinical

guidelines.

Interagency Agreement

I. Purpose

II. Effective Dates

III. Confidentiality

IV. Transfer of Data

V. Specify Persons Working with Data

VI. Modification or Amendment

VII. Signatures

Data SecurityHandling and Storage

Receive Data In A Variety Of Ways

Tapes, CDs, Disks, FTP, E-Mail

Dedicated File Server

Limited Users

Secure Location

Separate Identifiers and Outcome Data

Cleaning & Preparing the Data(Good News, Bad News)

Data Sets Are Not Well Documented

Many Variables Are Defined Differently

Data Formats Change Periodically (i.e.,

Constantly)

Cleaning & Preparing the Data(Good News, Bad News)

Recode the Variables

01/01/1999 vs. 01-01-99

vs. 990101

Unduplicate the Records

Allow for Aliases

Format Names

Exact (All-Or-Nothing)

Probabilistic

Combination of the Two

Overlap Procedure (PPE)

Linking The Data

Methods:

Block by Sex and DOB Components

Sex & Month & Year

Sex & Day & Year

Sex & Month & Day

Assign Weights to All Identifying

Variables

Linking The Data

Positive & Negative Weights

Jaro, Matthew. Statistics in Medicine, Vol. 14, 491-498

(1995).

Determine Threshold of Likelihood

Gray Area – Manual Inspection

Matching Algorithm Changes Depending on

Available Variables

Weighting The Data

Perfect Match = 69 pts.

SSN (all digits) = 22.55

SSN (8 digits) = 14.51 pts.

SSN (7 digits) = 6.07 pts.

Weighting The Data

or -2.38 pts.

Last or Maiden Name = 9.58 or –3.62 pts.

First Name = 6.69 or –3.27 pts.

Middle Initial = 3.65 pts.

Date of Birth = 6.22 pts.

Weighting The Data

Data SecurityHandling and Storage

Determine What Type of Results

You Want Before You Start What Type of Analysis Do the Data Allow?

What Do the Stakeholders Want?

What Type of Data Structure Do You Need?

Large Data SetsNeed Lots of Hard Drive Space & Memory

Need Appropriate Software

Treatment Completers: “Planned Discharge” & more than 5 units of Service

Treatment Non-completers: No “Planned Discharge” & more than 5 units of Service

Minimally Treated: 5 or fewer units of Service

Detox Only: Detoxification Services Only

Work Closely with Participating Agencies –

Disclose Results To Them

Think of Political Ramifications

for All Agencies Involved

Explicitly State What You Want the

Audience to Get from the Results

Disseminating The Results

Allow Much More Time Than You Think At Each Step

Be Flexible – Willing to Start Over Or Live With It

Meet With Other MIS Staff

Be Willing to Evolve With Technology and Experience

Lessons Learned Along The Way

Managed Care vs. Traditional System

Mental Health Court Involvement

PACT/ACT Clients

Persons with Co-Occurring

Special Interest Groups

Case-Mix Analysis

Cost Analysis

Interstate Cooperative Study

Parity – Race, Gender, Geography

Additional Uses

EVALUATING OUTCOMES OF CLIENTS RECEIVING BOTH SUBSTANCE ABUSE AND

MENTAL HEALTH TREATMENT

Oklahoma Department of Mental Health

and Substance Abuse ServicesBecki Moore and Tracy Leeper

To determine whether outcomes for clients with

dual treatment differ from those of clients receiving

mental health or substance abuse treatment only

To demonstrate the feasibility and utility of using

administrative data for different populations and

outcomes

Purpose

Methods

Study GroupThree Cohorts

mental health treatmentsubstance abuse treatmentdual treatment

Adults, 18 years of age or older, admitted to treatment during FY97 (N=28,104)

Administrative Data

Two-year Pre- and Post-Treatment Study Periods

Methods

Sources of Data

DMHSAS - State-funded SA & MH

OSDH - Mortality

DOC - Incarcerations, Probation &

Parole

OSBI - Arrests

OESC - Wages

Distribution of FY97 Clients (N=28,104)

53.1

39.9

7.0

0

10

20

30

40

50

60

Mental Health SubstanceAbuse

MH & SA

% o

f C

lien

ts

Mental Health

Substance Abuse

MH & SA

N=14,920 N=11,207 N=1,977

Clients

Cohort

Clients

Clients with Dual Treatment by Diagnosis(N=1,977)

05

1015202530

Schizophrenia OtherPsychotic

MoodDisorders

Other Non-psych.

Other

Diagnosis

% o

f C

lie

nts

Alcohol Abuse

Substance Abuse

Deferred

Mortality

2.5

3.8

2.9

4.1

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

MentalHealth

SubstanceAbuse

MH & SA Oklahoma*

Cohort

% o

f C

oh

ort

Mental Health

Substance Abuse

MH & SA

Oklahoma*

* Crude two-year state rate (1997, 1998)

Percent of Clients Who Died During the Two Years Following FY97 Admissions Age-Adjusted to the State Rate

Average Age at Death by Cohort

71.7

40.6

57.5

43.2

01020304050607080

MentalHealth

SubstanceAbuse

MH & SA GeneralPopulation

Cohort

Ae

at D

eath

Mental Health

Substance Abuse

MH & SA

General Population

Mortality

Average Time Till Death During the Two Years Following FY97 Admissions

351.4359.3 347.8

050

100150200250300350400

Mental Health SubstanceAbuse

MH & SA

Cohort

Day

s T

il D

eath

Mental Health

Substance Abuse

MH & SA

Mortality

Cause of Death during Two Years Following FY97 Admission

91 5

69

10 6

19

72

815

78 6

29

84 1 2 111

40

13

37

81

0102030405060708090

Accidents Alcohol orDrug

Related

Infectious& Parasitic

Disease

NaturalCauses

Suicide Other

Cause of Death

% o

f D

eath

s

Mental Health

Substance Abuse

MH & SA

OK 1998-1999

Mortality

Percent of Clients Employed at Least One Quarter During the Two Years Following FY97

Admission

45.5

63.8

58.1

40455055606570

Mental Health Substance Abuse MH & SA

Cohort

Per

cen

t

Mental Health

Substance Abuse

MH & SA

Employment

Average Number of Quarters Worked During the Two Years Following FY97 Admission for Persons with a

Post Wage

4.54.84.9

0

2

4

6

8

Mental Health Substance Abuse MH & SA

Cohort

Qu

arte

rs

Mental Health

Substance Abuse

MH & SA

Employment

Median Annual Wages During the Two Years Following FY97 Admission for Persons with a

Post Wage

$5,975

$8,703$8,108

$0$2,000

$4,000$6,000

$8,000$10,000

Mental Health SubstanceAbuse

MH & SA

Cohort

Med

ian

Wag

es

Mental Health

Substance Abuse

MH & SA

Employment