Upload
brent
View
43
Download
0
Tags:
Embed Size (px)
DESCRIPTION
The Relationship between Performance on Medicare’s Process Quality Measures and Mortality: Evidence of Correlation, not Causation. Andrew Ryan, Doctoral Candidate. Acknowledgements. Support Agency for Healthcare Research and Quality Jewish Healthcare Foundation Dissertation Committee - PowerPoint PPT Presentation
Citation preview
Schneider Institute for Health Policy,The Heller School for Social Policy and Management,Brandeis University
The Relationship between Performance on The Relationship between Performance on Medicare’s Process Quality Measures and Medicare’s Process Quality Measures and Mortality: Evidence of Correlation, not CausationMortality: Evidence of Correlation, not Causation
Andrew Ryan, Doctoral Candidate
2
Acknowledgements
• Support• Agency for Healthcare Research and
Quality
• Jewish Healthcare Foundation
• Dissertation Committee• Stan Wallack
• Chris Tompkins
• Deborah Garnick
• Kit Baum
3
Outline
• Measurement of quality in health care• Literature on relationship between
process and outcome measures in health care
• Methods• Results• Policy implications
5
Measurement of quality in health care
• Quality measurement • Central to public quality reporting and pay for performance
• Process measures• Assess whether “what is known to be ‘good’ medical care has been
applied” (Donabedian, 1966)• Process performance measurement has dominated quality
measurement• Hospital Compare• CMS/Premier Hospital Quality Incentive Demonstration
• Why?• Providers deemed to have control over performance• Greater statistical power• Provide “actionable” information for improvement (Birkmeyer et al.
2006; Mant 2001) • Fear of inappropriately labeling hospitals as “bad” due to random
variation in outcomes• Fear that use of outcome measures will lead to avoidance of higher risk
patients
6
Potential problems with process measures• A process measure might:
• not appropriately represent a process that is positively associated with patient health
• represent a process that is positively associated with patient health but has been rendered obsolete by advances in clinical practice (Porter and Teisberg 2006)
• Complicated nature of inpatient care may make process measures an inadequate proxy for quality due to limited scope
• Implementation of process measurement may alter the relationship between the observed process measures and patient outcomes
• measurement error• enhanced record keeping, or gaming• multitasking
7
CMS/ Joint Commission Starter Set Process Measures• Hospital Compare
• hospital care• voluntary, internet-based public quality reporting • implemented by CMS in 2003• After low rates of voluntary reporting in 2003, CMS made hospitals’
2004 payment update conditional on reporting for 10 of the 17 indicators-- reporting increased dramatically.
• CMS/Joint Commission Starter Set measures• AMI
• aspirin at arrival/discharge• β blocker at arrival/discharge• use of ACE inhibitor
• Heart failure• use of ACE inhibitor• assessment of left ventricular function
• Pneumonia• oxygenation assessment• timing of initial antibiotics• pneumococcal vaccination
Exception reporting: hospitals have discretion to exclude patients from calculation of process performance
8
Research Question: Relationship of Process Measures to Outcome
For AMI, heart failure, and pneumonia:
1) Are the CMS/Joint Commission process measures correlated with hospital mortality?
2) Are the CMS/Joint Commission process measures causally related to hospital mortality?
9
Effects of exception reporting
• Greater exception reporting may improve process performance
• Association between process performance and mortality may be weaker for hospitals that report more exceptions
• Supplemental analysis to examine:• Relationship between exception reporting and
process performance• Effect of exception reporting on relationship
between process performance and mortality
10
Previous research finds association between processes and outcomes
• Association between process measures and mortality for AMI, heart failure, pneumonia• Bradley et al. 2006; Eagle et al. 2005; Fonarow
et al. 2007; Granger et al. 2005; Luthi et al. 2004; Luthi et al. 2003; Peterson et al. 2006; Werner and Bradlow 2006; Jha et al. 2007
• Limitations of studies• Employed cross-sectional designs
• Results may be confounded by unobserved factors
11
Data and Methods
12
Data
• Medicare fee-for-service inpatient claims and denominator files (2004-2006)• Primary diagnoses for which beneficiaries were
admitted• Secondary diagnoses, demographics, and type
of admission for risk adjustment• Discharge status to exclude transfer patients • Outcome measure: 30-day mortality
• 2006 Medicare hospital characteristics • 2004-2006 Hospital Compare data
• Hospital process performance
13
Composite process quality measure• Defined as the z score of the weighted sum of z-
scores for process measures corresponding to each condition
• Transform each process measure by the z-score to avoid bias • Bias could result from a positive correlation between
the likelihood of reporting on a measure and performance on that measure.
• Transform the sum of the weighted z-scores by the z-score• Facilitates interpretation
• Compute for all hospitals reporting a denominator of at least 10 patients for at least one measure
14
Risk adjustment of 30-day MortalityAMI, Heart Failure, Pneumonia
• Hospital-level observed / expected• Expected: estimated from patient-level logit
models where mortality was regressed on:• Age, gender, race• Elixhauser comorbidities (Elixhauser et
al.1998)• Type of admission (emergency, urgent,
elective)• Season of admission
15
Two Specifications for Relationship of Process Measures to Mortality
#1: Pooled cross section
#2: Add hospital Fixed Effects
Then consider impact of hospitals’ exceptions reporting for both specifications
16
Analysis Analysis #1:Pooled Cross Section
ln (RA Mortality jkt)= b0 + b1 Z j + b2 year t +
b3 Z j * year t + δ1 Process jkt + δ2 Process2 jkt + e jkt
Where:• j is indexed to hospitals, k is indexed to condition (AMI or
heart failure), t is indexed to year (2004-2006)• Z is a vector of hospital characteristics (bed size, ratio of
residents / average daily census, urbanicity, ownership, % of Medicare admissions)
• Process is the composite quality measure• year is a vector of dummy variables for 2005 and 2006
17
Analysis: Analysis: #2 Hospital Fixed Effects
ln (RA Mortality jkt)= b0 + b2 year t + b3 Z j * year t + δ1 Process jkt + δ2 Process2 jkt + h j + e jkt
• Where h is a vector of hospital-specific fixed effects
• The inclusion of hospital-specific effects controls for unobserved time-invariant factors at the hospital level (e.g. physician skill/experience, technology) that may confound the relationship between processes of care and outcomes
18
Standard error specification
• Multiple observations from the same hospitals over time gives rise to group-level heteroskedasticity • Cluster-robust standard errors are estimated
(Williams 2000)• Hospital-level RA mortality rates vary in their
precision as a result of the number of patients in the denominator of the calculation • Analytical weights (Gould 1994), based on the
average number of patients in the denominator of hospitals' RA mortality calculation over the two study years, are employed
19
Results
20
Descriptive statistics
Hospital characteristics 2004 2005 2006 Between hospital
s.d.
Within hospital
s.d.
AMI
n 2,986 2,925 2,886 - -
Risk-adjusted 30-day mortality (mean)
16.6% 16.3% 16.5% 7.3% 7.1%
Composite 1 (mean) 0.83 1.07 1.13 .99 .39
Heart failure
n 3,276 3,225 3,219 - -
Risk-adjusted 30-day mortality (mean)
11.7% 11.1% 11.1% 3.4% 2.8%
Composite 1 (mean) 0.81 1.11 1.18 .93 .34
Pneumonia
n 3,261 3,233 3,230 - -
Risk-adjusted 30-day mortality (mean)
10.1% 10.1% 9.9% 3.4% 3.0%
Composite 1 (mean) 0.64 1.13 1.26 .93 .45
Note: table includes data from hospitals that are included in at least one of the regression modelsNote: data from individual process measures is included if hospitals report at least 10 in the measure denominatorNote: composite measures are scaled so that the mean of each measure across all hospitals across all years is 1
21
Marginal effects of process performance on risk-adjusted mortality
Description AMI Heart failure Pneumonia
∂y/∂x n R2 ∂y/∂x n R2 ∂y/∂x n R2
#1:Pooled cross section
-9.5%*** (1.0)
8,696 .09 -2.1%** (0.9)
9,630 .04 -2.1%*** (0.7)
9,673 .03
#2: Hospital fixed effects
0.2%(1.4)
8,549 .02 0.7% (1.3)
9,546 .03 0.3% (1.0)
9,596 .04
*** p<0.01, ** p<0.05, * p<0.1
Note: hospital controls include ownership, bed size, teaching status, urbanicity, and ratio of residents to average daily census
Note: Marginal effects are evaluated at median of process composite
Note: Marginal effects are multiplied by 100 to facilitate interpretation
Note: Robust standard errors in parentheses
22
Effects of exception reporting
• Not associated with process performance for composite measures for any diagnosis• Higher exception reporting was associated with
a substantial and significant increase in one process performance measure: ACE inhibitor for AMI
• Did not moderate effect of process performance in expected direction
• Did not explain null results from FE model
23
Summary of findings• Pooled cross section models showed CMS/Joint
Commission process performance measures are correlated with patient 30-day RA mortality for AMI, pneumonia, and heart failure• Consistent with recent evidence
• However, hospital fixed effects models showed no evidence of an association between process performance and mortality for any diagnosis• Suggests that CMS/Joint Commission measures are
not causally related to mortality• Conflicts with recent evidence• Unobserved improvements in record keeping may be
responsible for lack of casual relationship• Higher rate of exceptions does not appear to increase
measured process quality nor affect relationship between process performance and mortality
24
Implications for payment system
• Correlation between process performance and mortality supports utility of process measures for public reporting • Steer patients toward higher quality
providers• Absence of a causal relationship casts
serious doubt on the utility of current process performance measures as a metric for hospital quality improvement