Fixed and Random Effects Econometric

Modeling for Provider Profiling

Stephen T. Parente, Ph.D.University of Minnesota,

Carlson School of Management, Department of Healthcare Management

August 12, 2002

Overview• Provider Profiling Objectives, Language

& Examples• Rationale for Multivariate Models for

Profiling• Identifying Persistent Treatment

Patterns– Exploratory analysis (using random effects)– Confirmatory analysis (using fixed effects)

• Managed Care Results• Modeling Strategies & Summary

Provider Profiling Objectives

• Feedback (to consumer, employers & providers)

• Surveillance & performance assessment

• Focused provider education• Convey financial incentives to

providers• Justification for punitive action

Common Profiling Applications

• HEDIS (Health Employer Data Information Set) ‘Report Cards’ for providers & plans

• Track variation in health care use and cost • Quality of care profiles• Identifying providers for risk-adjusted

prospective capitation payments• Financial incentive report card• Variation in access to care

Language of Profiling

• Unit of Analysis• Unit of Observation• Case-mix adjustment• Risk-adjustment• Resource Use• Practice Variation• Physician vs. Provider vs. Practice

vs. Specialty

Provider Profile Fuel - Claims Data

A Primer• Strengths of Claims Data– Inexpensive to use (RE $$/patient record)– Increasingly standardized across many

populations– Good breadth of information

• Weaknesses of Claims Data– ‘Shallow’ on clinical detail– Can’t identify why a procedure was done,

only what was done.

Profile Examples in Use Today

State of Current Provider Profiles

• Produce simple descriptive statistics– Counts– Means– Variance

• Some case-mix/risk-adjustment• Some statistical significance testing• Used in stealth to detect cost & use outliers• Bluntly used to ‘educate’ and

‘communicate’.

Using Multivariate Regression

to Profile Providers• Approach– Use linear/logistic regression to control for

case-mix and provider attributes when patient is the unit of analysis.

• Uses– Plan specific analysis of patient and provider

factors affecting quality of care.

• Special Data Requirements– Need extremely good provider data along with

claims and membership data.

What’s the Point of Profiling?

• ‘Chance’ of detecting an inappropriate cost outlier at the tail of a distribution.

• ‘Chance’ of detecting poor quality care– Overuse of services– Underuse of services

• ‘Chance’ of detecting physicians gaming the system to induce demand in a fee-for-service reimbursement environment or minimize/moderate demand in a capitated environment.

• It would be good to have a method to improve your chances.

Reduced Form Econometric Model of Provider Treatment

Choices• Yij = B0 + C*cij + P*pj + uj + eij

Where:Yij =patient/provider medical outcome or treatment

i=ith patientJ=jth providerij=patient/provider combined setCij=vector of consumer (patient) factors

Pj=vector of provider attributes

Uj =physician-specific error structure

eij=physcian/patient error structure

Identifying an Outcome/Treatment

• Using Claims:– Medical treatment

• Physician procedure codes (CPT4/HCPCS)• Inpatient surgical codes (ICD-9)• Inpatient DRGs (diagnosis related groups)• Institutional revenue center codes

– Inpatient discharge status• Went home/transferred/died

– Treatment costs• Allowed charges (provider payment + patient

copayment)• Actual costs to treat (much harder to get)

Intervention / Control Variables• Using Provider Data:

– Financial incentive (e.g., capitation, fee-schedule)– Provider characteristics (e.g., specialty, med school)

• Using Membership Data:– Patient demographics– Health plan options

• Using Claims Data:– Case-mix/severity/comorbidity– Episode events & timing

• Using ‘Custom’ Plan-specific Data:– Disease management– Patient outreach

Why Use Random or Fixed Effects Models for Provider

Profiling?• RE/FE explicitly identifies the systematic

persistence of an independent variable on a dependent variable.

• Provides a convenient approach to:– Detect the extent of provider-level

persistence in treatment decisions– Control for idiosyncratic provider

characteristics driving the individual medical treatment decisions that sum to the health economy.

In Manager Speak…

• Use Two approaches to improve profiling:

1. 10,000 foot level: • ID a list of ‘target’ medical cost/utilization/outcome

performance measures. (e.g., 80 HEDIS measures) • Generate a PRIORITY SCORE reflecting targets

where physicians are the greatest contributing factor to variation in meeting a target.

2. 500 foot level:• Take top 5 performance level targets with highest

priority score and identify which physicians are contributing to the variance.

• When identifying physicians, control for patient case-mix and other demand factors that should not affect medical treatment choice.

Statistical Application

• 10,000’ Level: Use random effects to identify a population effect of physicians (as a group) persistence of treatment choice/outcomes.

• 500’ Foot Level: Use fixed effects to identify individual physician effects in the persistence of treatment choice/outcomes.

• Control for patient, hospital, group practice and physician characteristics.

Populations Used for AnalysisRochester (New York) Blue Cross Blue Shield

Insurance DataCholecystectomy48.7% laparoscopic 3,262 patients68 physicians16 hospitals6 surgical group

practices1990-1992

First-time deliveries22.4% cesarean7,151 patients112 physicians11 hospitals7 group practices1987-1992

Modeling Specifications Used

• Logistic binomial regression with random effects.– 12 points of quadrature.– Mixed model of physician random effects

combined with hospital and group practice fixed effects.

• GLS with random effects for linear fit estimates.

• Hausman test to identify variables unaffected by random effects specification.

Additional Physician Characteristics Used

• Age • Gender • Years in practice• Medical school & residency• Academic appointment• Peer review participant• Malpractice history

Using RE to Show Persistent Treatment Patterns

Population Level - Cholecystectomies

0 20 40 60

+MDcharacterics

+GroupPractice

+Hospitals

+Case-mix

+Year &Quarter

Age &Gender

Total %varianceexplainedMD %varianceexplained

• Y=1 (lap/chole), 0 open• Graph shows share of

variance explained by different attributes.

• Share of MD variance explained decreases as additional variables are added.

• % variance explained does not improve dramatically as more variables are added.

• Physician variance appears correlated to other factors.

Using RE to Show Persistent Treatment Patterns

Population Level - Deliveries

0 10 20 30

+MDcharacterics

+GroupPractice

+Hospitals

+Case-mix

Age, year &quarter

Total %varianceexplainedMD %varianceexplained

• Y=1(c-section), 0 vaginal• Graph shows share of

variance explained by different attributes.

• Share of MD variance explained decreases as additional variables are added.

• % variance explained does not improve dramatically as more variables are added.

• Physician variance appears correlated to other factors.

Was Random Effects an Appropriate Specification?

• Hausman (1978) specification test for examining the equality of the coefficients estimated by fix and random effects.

• Results: – Deliveries: Prob>chi2 = 0.3889– Cholecystectomies: Prob>chi2 =

Identifying Persistent Treatment Patterns

Provider Level - Cholecystectomies

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1 4 7 10 13 16 19 22 25 28 31 34 37 40 43 46 49 52 55

Marginal Propensity of Each Physician to Use a laparoscopic procedure, controlling for case-mix, group practice, hospital, and secular trends

Fixed Effects Results

Identifying Persistent Treatment Patterns

Provider Level - Deliveries

-0.5

-0.4

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77 81 85 89 93

Marginal Propensity of Each Physician to Use a C-section,controlling for case-mix, group practice, hospital, and secular trends

Fixed Effects Results

A Checklist of Profiling Issues - I

___ Data• Are all necessary variables present?• Can the data be manipulated easily?• Is the data generalizable to other populations?

___ Methods• What is the denominator for the analysis?• Is this a one-time approach or is the code re-

usable?• Is case-mix adjustment necessary?• What statistical methods will be employed?

A Checklist of Profiling Issues - II

___ ‘Research’ Questions• What is the best possible outcome you hope

to achieve?• What is the worse case scenario for the

effort involved?• Are there unintended consequences to your

approach?• Are you providing information that allows

providers to game the system?

Modeling Strategies

• When profiling multiple performance measures, use random effects to prioritize provider profiling and gain further insight.

• If physician persistence for treatment choices remains high, conduct a physician-fixed effects model to identify variation in order to develop an intervention to change practices or patient demand.

Summary

• Results show how random effects and fixed effects can be used to improve provider profiles.

• Employing strategies will generate improved accuracy for provider identification for intervention programs.

• RE then FE strategy may lead to quick surveillance of many other measures to detect new activities to be profiled.