Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
SCIOinspire Corp Proprietary & confidential. Copyright 2008
September, 2008
So You’ve Got a Predictive Modeling ToolCongratulations! Now What?
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• Ian Duncan, FSA MAAA
President, Solucia Consulting
• Kate Hall, ASA, MAAA.
Vice President, Solucia Consulting.
Faculty
2
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• Have you thought about how you are going to use it?
• Is your data source optimal?
• Predictive Modeling Project Planning for ROI
• Care Management
• Underwriting.
• Implementation and Automation.
• How about going back to test whether results are as predicted?
Agenda Topics
3
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Models and Uses
4
• Numerous uses. Models are not necessarily optimal for each use.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
5
Why do it? Potential Use of Models
Program Management Perspective
Identifying individuals at very highrisk of an event (death, LTC, disability, annuity surrender, etc.).
Identify management opportunities and determine resource allocation/ prioritization.
Reimbursement
Predicting (normalized) resource use in the population. Reimbursement by Episode. Reimbursement by risk level.
Program Evaluation
Predicting resource use based on condition profile.Trend Adjustment.
Provider Profiling
Profiling of provider
Efficiency Evaluation
Provider & health plan contracting
Actuarial, Underwriting
Calculating new business and renewal premiums
SCIOinspire Corp Proprietary & confidential. Copyright 2008
6
Optimal Models
Reimbursement
Predicting (normalized) resource use in the population. Reimbursement by Episode. Reimbursement by risk level.
Program Evaluation
Predicting resource use based on condition profile.Trend Adjustment.
Any model needs to be stable over time. Episode Treatment Groups.Risk scoring models.
Risk Score models
Keys to successful model implementation:• Stability over time;• High correlation between risk score and $’s;• Independence between risk score and any intervention that may be applied;
SCIOinspire Corp Proprietary & confidential. Copyright 2008
7
Why do it? Potential Use of Models
Program Management Perspective
Identifying individuals at very highrisk of an event (death, LTC, disability, annuity surrender, etc.).
Identify management opportunities and determine resource allocation/ prioritization.
Program Management Perspective
• Risk Scoring models;• Gaps-in-care models;• Self-reported risk factor models;• Intervenability assessment models.
Keys to successful model implementation. All the things on the prior slide, plus:
• In care management, program managers frequently use more than one model. Which model’s predictions are used in which situation (which one trumps?) is not a simple problem.
• How do you dynamically incorporate new targets and terminate old ones? How do you convert predictive model targets into a set of algorithms to apply real-time to your data?
SCIOinspire Corp Proprietary & confidential. Copyright 2008
8
Why do it? Potential Use of Models
Provider Profiling
Profiling of provider
Efficiency Evaluation
Provider & health plan contracting
Actuarial, Underwriting
Calculating new business and renewal premiums
Episode Treatment Groups.
Risk Scores.
Risk Scores
Predicted Costs
Self-reported conditions (HRAs)
For Provider profiling, the biggest issue will be assembling a credible database with adequate volumes of consistent provider data.
For Underwriting, many techniques have been developed to address the interaction between cost, data timeliness and predicted outcomes.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
8 Simple Rules for Dating your Model:
1. Know what you are getting into – plan the project’s desired outcomes.
2. Know the existing workflow into which the model will fit, and plan any changes in workflow that will result from the new model.
3. Evaluate data sources carefully.
4. Evaluate model(s) against a known objective, as well as against “Business As Usual”.
Implementing a Model
9
SCIOinspire Corp Proprietary & confidential. Copyright 2008
8 Simple Rules for Dating your Model:
5. Make sure you understand the results of the model evaluation, and have a plan to optimize if inadequate.
6. Look for ways to automate the new workflow (the point of a model is to replace human intelligence, so don’t let the humans get in the way).
7. Pilot.
8. Evaluate outcomes: did the new system produce the high-risk targets that it was predicted to? How many of the targets were new/not found by existing methods? For an underwriting or provider reimbursement project, how did the results compare with the prior method? What can be done to enhance the results?
Implementing a Model (contd.)
10
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Model Evaluation Examples
11
The next few slides are examples of projects in which we have evaluated a model(s) for implementation.
These are all examples of situations in which it was necessary to “Re-frame” the original predictive modeling question in order to understand the value of the model.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
12
How well does the model perform?
All Groups
0
20
40
60
80
100
120
140
-100%
+
-90%
to -99
%-80
% to
-89%
-70%
to -79
%-60
% to
-69%
-50%
to -59
%
-40%
to -49
%-30
% to
-39%
-20%
to -29
%
-10%
to -19
%0%
to -9
%0%
to 9%
10%
to 19
%20
% to
29%
30%
to 39
%40
% to
49%
50%
to 59
%60
% to
69%
70%
to 79
%80
% to
89%
90%
to 99
%
Analysis 1: all groups. This analysis shows that, at the group level, prediction is not particularly accurate, with a significant number of groups at the extremes of the distribution.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
13
How well does the model perform?
NodePREDICTED
Average Profit
PREDICTED Number in
Node
PREDICTED Number in Node
(Adjusted)
ACTUAL Number in
nodeACTUAL
Average Profit
Directionally Correct (+ or -)
Predicted to be
Profitable1 (3.03) 70 173 170 (0.60) 2 0.19 860 2,122 2,430 0.07 3 (0.20) 2,080 5,131 6,090 (0.06) 4 0.09 910 2,245 2,580 0.10 5 (0.40) 680 1,678 20 0.02 6 (0.27) 350 863 760 0.16 7 0.11 650 1,604 1,810 0.04 8 0.53 190 469 470 (0.01) 9 (0.13) 1,150 2,837 2,910 0.03
10 0.27 1,360 3,355 3,740 0.04 11 0.38 1,560 3,849 3,920 (0.07) 12 0.08 320 789 830 0.08 13 0.06 12,250 30,221 29,520 0.02 14 0.27 2,400 5,921 6,410 0.21 15 (1.07) 540 1,332 1,320 (0.03) 16 0.07 10,070 24,843 24,950 (0.08) 17 (0.33) 1,400 3,454 3,250 (0.10) 18 0.11 4,460 11,003 11,100 0.08 19 (0.13) 1,010 2,492 2,100 (0.11)
42,310 104,380 104,380 0.005 6 red13 green 11 nodes
SCIOinspire Corp Proprietary & confidential. Copyright 2008
14
Underwriting Decision-making
Underwriting Decision Total Profit Average Profit per
Case
Cases Written
Accept all cases as rated. 557.5 0.005 104,380
Accept all cases predicted to be profitable; reject all predicted unprofitable cases.
1,379.4 0.016 87,760
Accept all cases predicted to be profitable; rate all cases predicted to be unprofitable +10%.
2,219.5 0.021 104,380
Accept all cases for which the directional prediction is correct.
2,543.5 0.026 100,620
Accept all cases for which the directional prediction is correct; rate predicted unprofitable cases by +10%
3,836.5 0.038 100,620
Accept all cases for which the directional prediction is correct.
2,540.8 0.025 101,090
SCIOinspire Corp Proprietary & confidential. Copyright 2008
15
0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% 90.0%
100.0%
99 96 93 90 87 84 81 78 75 72 69 66 63 60 57 54 51 48 45 42 39 36 33 30 27 24 21 18 15 12 9 6 3 0Model Percentile
Percent of Members w/ Hospitalization Identified
Model 2 Model 1
Lift Chart – Comparison between Two models
Care Management - Analysis
SCIOinspire Corp Proprietary & confidential. Copyright 2008
16
Background
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
99 98 97 96 95 94 93 92 91 90 89 88 87 86 85 84 83 82 81 80Model Percentile
Percent of Members w/ Hospitalization Identified
Model 2 Model 1
Lift Chart – Comparison between Two models
SCIOinspire Corp Proprietary & confidential. Copyright 2008
The importance of Intervenability
17
Prediction is Not Enough
For example, are these conditions equally intervenable?
NAME LABEL CONDITION GROUP CCDXG060 10.01 colon cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG061 10.02 rectal cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG062 10.03 oth/unspec ca of digest organs/per Cancer Breast/Prostate/Colorectal/Other CancerDXG063 10.04 melanoma Cancer Breast/Prostate/Colorectal/Other CancerDXG064 10.05 breast cancer, age 45+ Cancer Breast/Prostate/Colorectal/Other CancerDXG065 10.06 cancer of uterus Cancer Breast/Prostate/Colorectal/Other CancerDXG066 10.07 cancer cervix/fem genital organs Cancer Breast/Prostate/Colorectal/Other CancerDXG067 10.08 prostate cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG068 10.09 cancer testis/male genital organs Cancer Breast/Prostate/Colorectal/Other CancerDXG069 10.10 ca bladder/ureter/urethra/oth urin Cancer Breast/Prostate/Colorectal/Other CancerDXG070 10.11 cancer of kidney and renal pelvis Cancer Breast/Prostate/Colorectal/Other CancerDXG071 10.12 cancer of the eye Cancer Breast/Prostate/Colorectal/Other CancerDXG072 10.13 thyroid/endocrine ca/exc adrenal/p Cancer Breast/Prostate/Colorectal/Other CancerDXG073 10.14 other/ill-defined site cancer Cancer Breast/Prostate/Colorectal/Other CancerDXG078 10.19 breast cancer, age < 45 Cancer Breast/Prostate/Colorectal/Other Cancer
High Cost does not equal High Opportunity.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Segmentation and Operational Model
Population Segment Interventions
ICMIntensive Case Mgmt: High dollar cases (>$50,000 over 3 months excluding those with PCC) Specific diagn
Case Management Condition Management Wellness Coaching
CC Chronic Conditions Asthma, COPD, CAD, CHF, Diabetes
ECElective/Other Manageable Conditions Maternity Depression Osteoarthritis Hips/Backs etc.
NTHCNon targeted health conditions (lower prevalence, lower cost and less modifiable conditions) -- those with claims beyond positive utilization for conditions other than those defined in other segments
ARAt risk on the basis of HRA -- those with no claims other than preventive
but with defined risks on basis of HRA .
AW Apparently well on basis of HRA with no claims other than preventive Targeted messaging highlighting resources to stay healthy
NUNonusers: No HRA or claims-based conditions
.Targeted messaging urging HRA and positive utilization
Wellness Coaching for those indicated on basis of HRA (High risk in correctable areas: Nutrition, Weight Reduction, Stress Mgmt, Physical Activity, Smoking Cessation w ith high motivation and high confidence)
Condition Management Wellness Coaching
Wellness Coach
Health Coach (RN)
Health Coach RN
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Care Management Program Planning
19
SCIOinspire Corp Proprietary & confidential. Copyright 2008
20
Step 1 The intake coordinator receives a daily admit notification.
Step 2For each patient the intake coordinator completes a brief admission survey.
Step 3ProGuide combines the admission survey with historical patient information and assigns the patient a risk ranking in real time on nurse C.M. task-list.
High Risk
Medium Risk
Low Risk
Step 4The intervening nurse contacts admitting hospital on behalf of high and medium risk patients. Tasks are prioritized according to Risk ranking.
High Risk
Medium Risk
Data Data WarehouseWarehouse
Implementation
Data Data WarehouseWarehouse
SCIOinspire Corp Proprietary & confidential. Copyright 2008
In future, integrated Systems are key to success
Electronic Patient/Provider Record
Transactional Systems Service-Level Data
Rx Claims
Medical Claims
Lab Values
Pre-authorization
Surveys
Demographics
*Actuarial Functionality
Price Programs
Plan Intervention Programs
Price Guarantees
Model Inter-vention Pgm
Analyze FinancialOpportunity
Analyzer
Workflow System
Reporting
RemoteReplication
Manage Patients
CM Database
Evaluation
Reconciliation
OutcomesManagementIdentification
21
SCIOinspire Corp Proprietary & confidential. Copyright 2008
22
But so too are non-traditional functions
• Customer interaction: ability to attract and enroll patients;• Financial projections and guarantees: Care Management is
increasingly being sold as a Financial Product, rather than a clinical or administrative product;
• Designing incentives/disincentives to steer patients and change behavior.
• Data/information about best-practices, gaps and quality providers.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
23
Care Management is evolving
“The trend that we are beginning to see with our large health plan clients is a re-examination of the outsourced model of DM that has been prevalent for the last 5 years in favor of an “insourced” or “assembled” strategy. Health plans are questioning the economics of outsourced DM and the silo effect of stand-alone DM programs. DM companies able to effectively address health plans’ concerns with cost, program integration and seamless member management, as well as more effectively engaging their provider networks, will be better positioned as this trend evolves.”
Ian Duncan, quoted in Disease Management News, February 10, 2006.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
24
Predictive modeling can’t solve…
• How to integrate into health plan/provider processes;
• Involving the provider;• Timeliness of information;• How to integrate the medical record.
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• Long lead time needed - - - due to renewal notification requirements & claims lag.
• Means little data available for first renewal – need to supplement predictive model information.
• Large group may use the predictive model results as an adjustment to the renewal – time lag could be longer.
• Rx data has less lag so incorporating up to date Rx data may have a benefit.
Unique circumstances of underwriting
25
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• The loss ratio method or the build up method for renewals for a block of business is a reliable and proven method to assess the overall needed rate increase.
• Allocating the needed rate increase among small groups is an opportunity to introduce predictive modeling.
• Rather than using the predicted cost as an absolute use it to stratify groups and apply the rate increase to groups based on where their risk falls compared to the mean for the block.
Predictive Modeling & Renewal Rating
26
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Small Group Example
• Determine the average risk score for your in-force business
• Calculate the average rate up for your in- force business
• Find where the threshold is for best rate.
Predictive Modeling & Renewal Rating
27
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• New Business Underwriting & Predictive Modeling
• Example:
Predictive Modeling & New Business Rating
28
SCIOinspire Corp Proprietary & confidential. Copyright 2008
What did we do?
Stage 1
• Convert written condition responses from the underwriting questionnaires into identifiable DxCG conditions.
• This was an extensive manual process (although automatable in the future).
• Some degree of subjectivity was involved, though no more than in the standard underwriting process.
New Business Underwriting Project
29
SCIOinspire Corp Proprietary & confidential. Copyright 2008
What did we do?
Stage 2
• Convert DxCG conditions into DCG condition-based scores: automatic process.
• Calculate DxCG age/gender scores. • Summarize at a group level.• Calculate Relative Risk Score (RRS) as condition-based
score divided by age/gender score. We are trying to isolate the deviation from the age/gender norm (because the differences due solely to age/gender are accounted for in the premium).
New Business Underwriting Project
30
SCIOinspire Corp Proprietary & confidential. Copyright 2008
What did we do?
Stage 3
• Compare the RRS by group based on the predictive model and implied RRS from the underwriting formula to the actual first year experience.
A couple of notes:• Children not included in the analysis - insufficient data
to generate predictive values;• Some conditions not mapped – may require original
source document to understand the written conditions;• Need to further develop the “with
complications”/severity information.• Need to incorporate information contained in the drug
data.
New Business Underwriting Project
31
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Current manual approach and predictive modeling approach initially produce similar results.
Given the similar results:• The data limitations of the automated approach;• The potential for refinements to the data collection
and• The potential for refinements to the model
Give the opportunity to improve the accuracy of the underwriting process AND reduce the manual effort.
Preliminary Results
32
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Ranking of Calculated Underwriting LoadManual Loss Ratio* Percentile Ranking Below 15th
Between 15th & 25th
Between 25th & 50th
Between 50th & 75th
Between 75th & 90th Over 90th Grand Total
Correct Grouping
Under Predicted
Over Predicted
Below 15th 48 15 27 27 7 8 132 36.4% 63.6%
Between 15th & 25th 19 13 33 16 4 2 87 14.9% 21.8% 63.2%
Between 25th & 50th 30 29 68 49 27 16 219 31.1% 26.9% 42.0%
Between 50th & 75th 14 15 57 67 45 21 219 30.6% 39.3% 30.1%
Between 75th & 90th 8 10 21 37 35 21 132 26.5% 57.6% 15.9%
Over 90th 13 5 13 23 14 20 88 22.7% 77.3%
Grand Total 132 87 219 219 132 88 877
Ranking of Relative Risk ScoreManual Loss Ratio* Percentile Ranking Below 15th
Between 15th & 25th
Between 25th & 50th
Between 50th & 75th
Between 75th & 90th Over 90th Grand Total
Correct Grouping
Under Predicted
Over Predicted
Below 15th 49 15 24 22 14 8 132 37.1% 62.9%
Between 15th & 25th 15 13 26 17 9 7 87 14.9% 17.2% 67.8%
Between 25th & 50th 31 24 73 48 26 17 219 33.3% 25.1% 41.6%
Between 50th & 75th 15 22 58 61 33 30 219 27.9% 43.4% 28.8%
Between 75th & 90th 13 7 25 40 32 15 132 24.2% 64.4% 11.4%
Over 90th 9 6 13 31 18 11 88 12.5% 87.5%
Grand Total 132 87 219 219 132 88 877
* Incurred claims divided by “manual” premium (actual without rate up).
Preliminary Results
33
SCIOinspire Corp Proprietary & confidential. Copyright 2008
• Self-reported data
• FSA/HSA Disbursement data
• Rx data from a service
• Other consumer data
• Other?
Possibilities
34
SCIOinspire Corp Proprietary & confidential. Copyright 2008
Discussion
35
Slide Number 1Slide Number 2Slide Number 3Slide Number 4Why do it? Potential Use of ModelsOptimal ModelsWhy do it? Potential Use of ModelsWhy do it? Potential Use of ModelsSlide Number 9Slide Number 10Slide Number 11How well does the model perform?How well does the model perform?Underwriting Decision-makingSlide Number 15Slide Number 16Slide Number 17Segmentation and Operational Model Care Management Program PlanningSlide Number 20In future, integrated Systems are key to successBut so too are non-traditional functionsCare Management is evolvingPredictive modeling can’t solve…Slide Number 25Slide Number 26Slide Number 27Slide Number 28Slide Number 29Slide Number 30Slide Number 31Slide Number 32Slide Number 33Slide Number 34Slide Number 35