Upload
gabriel-burgess
View
217
Download
1
Tags:
Embed Size (px)
Citation preview
Development and Evaluation of CMS-HCCConcurrent Risk Adjustment Models
Presented byEric Olmsted, Ph.D.Gregory Pope, M.S.John Kautter, Ph.D.RTI International
Presented atAcademy HealthJune 26, 2005
411 Waverley Oaks Road ■ Suite 330 ■ Waltham, MA 02452-8414
Concurrent Risk AdjustmentIntroduction
OverviewRisk Adjustment/HCC ModelConcurrent v. Prospective
Project Goals and Challenges
Model Development
Model Evaluation
Summary and Conclusion
OverviewRisk Adjustment Introduction
Population Risk Adjustment:The process by which the health status of a
population is taken into consideration when setting capitation rates or evaluating patterns or outcomes of practice
Risk adjustment is used to create “apples to apples” comparisons Risk adjustment removes the effect of health
status differences Reduces or eliminates the problem of selection
OverviewRisk Adjustment Model
Model calibrated on 5% national sample of Medicare fee-for-service beneficiaries
Expenditures are regressed on HCC (& demographic) risk markers to estimate incremental impact of each diagnosis on expenditures
Annualized Expenditures = Σαi + Σβi + Єi
αi = demographic markers
βi = HCC markers
OverviewHCC Model
Full model contains 184 HCCs
CMS-HCC model contains 70 HCCs CMS-HCCs:
Cover a broad spectrum of health disorders Have well-defined diagnostic criteria Exclude highly discretionary diagnoses Include conditions with significant expected health expenditures
Demographic Markers Age, Gender, Medicaid, & Originally Disabled Status Ensure means for demographic populations correctly estimated
OverviewConcurrent vs. Prospective
Prospective risk adjustment uses current year diagnoses to predict next year’s expendituresChronic conditions are more important
Concurrent risk adjustment uses current year diagnoses to predict this year’s expendituresAcute conditions are more important
OverviewConcurrent vs. Prospective
AMI:Prospective Coefficient = $1,838Concurrent Coefficient = $12,211
63% of HCC coefficients with >$1,000 difference
R-squared:Concurrent - .4811Prospective - .0981
Project Goals
Concurrent Risk Adjustment Project Goals:Develop payment model for Pay-for-Performance
demonstrationDevelop model for use in profiling physiciansMake model consistent with prospective CMS-HCC
model that is being used for MA payment, and its data collection requirements
Improve prediction across the spectrum of patient cost
Concurrent Modeling Challenges
Applied standard HCC modelResulted in negative predictions and coefficients
Concurrent HCC coefficients fit high-cost beneficiariesThis forces age-sex coefficients down and they
sometimes become negativeAge-sex coefficients reflect the average beneficiaryNegative age-sex coefficients can lead to negative
predictions
Model Challenges Standard Regression
-$2,000
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
0 0.5 1 1.5 2 2.5 3
Risk Score
Exp
end
itu
res
($)
Beneficiaries
Regression Line
Model ChallengesSplit Sample Regression
-$2,000
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
0 0.5 1 1.5 2 2.5 3
Risk Score
Exp
end
itu
res
($)
High-CostBeneficiaries
Low-CostBeneficiaries
Low-Cost Regression
High-Cost Regression
Model ChallengesRegression through the Origin
-$2,000
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
0 0.5 1 1.5 2 2.5 3
Risk Score
Exp
end
itu
res
($)
High-CostBeneficiaries
Low-CostBeneficiaries
Regression Line
Model ChallengesNonlinear Regression
-$2,000
$0
$2,000
$4,000
$6,000
$8,000
$10,000
$12,000
$14,000
0 1 2 3
Risk Score
Exp
end
itu
res
($)
High-CostBeneficiaries
Low-CostBeneficiaries
Project GoalsModel Selection
Criteria for Model SelectionAvoid negative predictions, which lack face validityAvoid negative coefficientsMaintain correct age-sex means to prevent age
and sex selection by providersPrefer simple models to complex modelsSelect model with good ‘performance’ among
model evaluation measures
Model DevelopmentSample Statistics
1.4 million FFS Medicare beneficiaries with mean expenditures of $5,214
Beneficiaries with at least one CMS-HCC represent 61% of the population, but provide 94% of all Medicare expenditures
Model Development Standard Models
Full HCC Model184 HCCs & demographics
CMS-HCC Model70 HCCs & demographics
Interaction and Topcoding ModelsCreated disease and demographic interactions
to tease out high-expense beneficiariesCreated topcoded models to reduce impact of
outliers
Model DevelopmentAlternative Models
Nonlinear Models Log model Square root model
Split Sample Models Designed separate models for populations with different expected
expenditures Community/Institutional High Cost/Low Cost HCC Catastrophic HCC Multi-stage models including two-part and four-part logit models Simple two-stage model with demographic multipliers Segmentation
Model EvaluationStandard Model Results
Full HCC model suffers not only from 30% negative predictions, but also contains negative HCC coefficients
CMS-HCC model explains 92% of the variation that the Full HCC model explains CMS-HCC model eliminates negative HCC
coefficientsCMS-HCC model has only 10% negative predictions
Interaction and Topcoding ModelsDid not sufficiently reduce negative predictions
Model EvaluationAlternative Model Results
Nonlinear ModelsLog model and square root model did not produce
reasonable predictions
Split Sample ModelsSplitting sample by community/institutional did not eliminate
negative predictionsSplitting sample by disease burden eliminated negative
predictions
Model EvaluationMeasures of Model Performance
R2 within .04 for all modelsR2 did not differentiate models
Predictive Ratio = Average of model’s predictions
Average of actual expenditures
Where each of the two averages is taken over the individuals in the subgroup Predicted expenditure deciles Number of HCCs for a beneficiary
Model EvaluationPredictive Ratios by Expenditure Percentile
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
Percentile Predicted Expenditures
Pre
dic
ted
/Act
ual
Exp
end
itu
res
CMS-HCC Model
Multiplier Model
High Cost/LowCost Model
Four Part Model
Model EvaluationPredictive Ratios by Expenditure Percentile
0.00
0.20
0.40
0.60
0.80
1.00
1.20
1.40
Percentile Predicted Expenditures
Pre
dic
ted
/Act
ual
Exp
end
itu
res
CMS-HCC Model
Catastrophic/Non-CatastrophicModel
Two-Stage SampleSegmentationModel
Topcoded at$50,000 Model
Model EvaluationPredictive Ratios by # of HCCs
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
Observed Number of HCCs
Pre
dic
ted
/Act
ual
Exp
end
itu
res
CMS-HCCModel
MultiplierModel
High-Cost/Low-Cost Model
Four PartModel
Model EvaluationPredictive Ratios by # of HCCs
0.40
0.50
0.60
0.70
0.80
0.90
1.00
1.10
1.20
Observed Number of HCCs
Pre
dic
ted
/Actu
al E
xp
en
dit
ure
s
CMS-HCCModel
Catastrophic/Non-CatastrophicModel
Two-StageSampleSegmentationModel
Concurrent Model EvaluationModel Summary
High Cost & Catastrophic Models performs well Some face validity problems with splitting HCCs into “high-cost”
and “low-cost” Still has negative predictions
Four Part Model also performs well Computationally advanced and hard to interpret intuitively No negative predictions
Sample Segmentation Model performs very well Also computationally advanced
Two-Stage Multiplier Model performs adequately No face validity problems
Concurrent Model EvaluationConclusion
Nonlinearities cause difficulties in concurrent risk adjustment model calibration Negative coefficients and predictions
These difficulties can be addressed with: Nonlinear models Split sample models
But nonlinear/split sample models add complexity Difficult to estimate Difficult to interpret Adds instability
Two-Stage Multiplier Model Good face validity, avoids negative coefficients and predictions Simpler to estimate and interpret