Upload
hilda-warren
View
29
Download
0
Tags:
Embed Size (px)
DESCRIPTION
More Practical Metrics for Standardizing Health Outcomes in Effectiveness Research. John E. Ware, Jr., PhD, Professor and Chief Division of Measurement Sciences, Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA - PowerPoint PPT Presentation
Citation preview
More Practical Metrics for Standardizing Health Outcomes in Effectiveness Research
John E. Ware, Jr., PhD, Professor and Chief
Division of Measurement Sciences, Department of Quantitative Health Sciences, University of Massachusetts Medical School, Worcester, MA
Track A - Patient Reported Outcome Measurement and Comparative Effectiveness Research to Reform: Achieving Health System Change
AHRQ 2009 Annual Conference, Bethesda MD September 13-16, 2009
2
What is the Relationship Between Health Care Expenditures &
Outcomes?
Expenditures for Health Care ($)
HealthOutcome
3
More Health Care is Not Always Better
HealthOutcome
“Flat of the Curve”
Expenditures for Health Care ($)
Health Insurance Experiment Revealed:
4
When the Same Outcome Costs More,
HealthOutcome
Payers & Consumers:
Expenditures for Health Care ($)
Want to Pay Less
5
Who is Most Vulnerable with Aggressive Cost Containment?
HealthDecline
Expenditures for Health Care ($)
Cost Containment
WellWell offYoung
Health Insurance Experiment (HIE)(1974-1981)Medical Outcomes
Study (MOS)(1986-1990)
Most vulnerable inthe MOS:• Chronically ill• Elderly• Poor• Non- white
6
4-Year Physical Health Outcomes Favored FFS > HMO for Chronically-
Ill Medicare in the MOS
Source: Ware, Bayliss, Rogers et al., JAMA, 1996; 276:1039-1047
These percentages, better & worse would be only about 5% due to measurement error
7
When Outcomes Vary at the Same Price
Expenditures for Health Care ($)
HealthOutcome
Payers & ConsumersWant the Best Outcomes
8
To Compare Health Care EffectivenessWe Need Health Outcomes “Rulers”
Expenditures for Health Care ($)
Better
Same
Worse
HealthOutcome 4
6
7
1
2
3
5
9
Continuum of Disease-specific and Generic Health Measures
9
Specific Symptoms
Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995
(1) (2) (3) (4)
1010
Specific Symptoms
Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995
(1) (2) (3) (4)
dd
Spirometry
Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all
Shortness of Breath
Continuum of Disease-specific and Generic Health Measures
1111
Specific Symptoms
Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995
(1) (2) (3) (4)
dd
Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all
How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely
Spirometry Shortness of Breath
Respiratory-specific
Continuum of Disease-specific and Generic Health Measures
1212
Specific Symptoms
Adapted from: Wilson and Cleary, JAMA, 1995 Ware, Annual Rev. Pub. Health, 1995
(1) (2) (3) (4)
dd
Over the last 4 weeks I have had shortness of breath Almost every day Several days a week A few days a month Not at all
How much did your lung/respiratory problems limit your usual activities or enjoyment of everyday life? Not at all A little Moderately Extremely
In general, would you say your health is…
Excellent Very good Good Fair Poor
Spirometry Shortness of Breath
Respiratory-specific
Generic
Continuum of Disease-specific and Generic Health Measures
13
Specific Symptoms
(1) (2) (3) (4)
There is More to the Continuum
14
Prediction and Risk Management:
PROs are among the Best Predictors
Health-Related QOL (HR-QOL)
Future health
Inpatient expenditures
Outpatient expenditures
Job loss
Response to treatment
Return to work
Work productivity
Mortality
(3) (4)
What Do We Need for Comparative Effectiveness
Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation
15
Physical activity limitationsSymptoms of psychological distressPhysical well-beingLife satisfactionEmotional behaviorRole disability due to physical problemsPsychological well-beingGeneral health perceptionsPhysical mobilityRole disability due to emotional problemsSatisfaction with physical conditionSocial activities with friends/relatives
16
Content of Widely-Used Patient-Reported Outcome Measures
Source: Adapted from Ware, 1995
Reported health transition
SIP = Sickness Impact Profile (1976)HIE = Health Insurance Experiment surveys (1979)NHP = Nottingham Health Profile (1980)QLI = Quality of Life Index (1981)COOP = Dartmouth Function Charts (1987)DUKE = Duke Health Profile (1990)MOS FWBP = MOS Functioning and Well-Being
Profile (1992)
MOS SF-36 = MOS 36-Item Short-Form Health Survey (1992)
QWB = Quality of Well-Being Scale (1973)EUROQOL = European Quality of Life Index (1990)HUI = Health Utility Index (1996)SF-6D = SF-36 Utility Index (Brazier, 2002)
Psychometric Utility RelatedSIP HIE NHP COOP DUKE MOS
FWBPMOSSF-36
QWB EURO-QOL
HUI SF-6DCONCEPTS
Physical functioning Social functioning Role functioning Psychological distress Health perceptions (general) Pain (bodily) Energy/fatigue Psychological well-being Sleep Cognitive functioning Quality of life
PROMIS = Patient Reported Outcomes Measurement Information System
= Quality of Well-Being Scale (1973)
PROMIS
What Do We Need for Comparative Effectiveness Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation17
What Do We Need for Comparative Effectiveness Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation 18
“Ceiling Effect”
19Ware JE, Jr, et al. Med Care. 2000;38:1173-82.
Skewed 5-Item Headache Pain Measure
“Ceiling Effect”
r = 0.536N = 1016
Dynamic 5-Item HeadachePain Measure
r = 0.938N = 1016
CriterionScore
CriterionScore
A Practical Solution in 1999: Computerized Dynamic Health
Assessment
3 SD units
NoNoDisabilityDisability
IRT/CAT will spawn a new generation of
static tools
What Do We Need for Comparative Effectiveness Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation 20
Cri
teri
on
Cri
teri
on
VASVAS
What Do We Need for Comparative Effectiveness Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation 21
22
Practical Solution in 2000:Cross-Calibration of Headache Pain Disability Measures
Note: Direction of scoring shown with arrows Source: Ware, Bjorner & Kosinski, Medical Care, 2000
Scales 20 30 40 50 60 70
HDI 16 43 73 91 98 100
HIMQ 74 53 31 17 8 2
MIDAS 58 28 5 1 0 0
MSQ 31 53 79 92 96 99
DYNHA-5 (+) 23 32 41 51 58 66
Theta (θ) [Best Possible Estimate]
23
52 centimeters = 20.5 inches
and Public-Private Partnerships That Meetthe Needs of Research and Business
We Need the Health Equivalent of a Two-Sided Tape Measure
What Do We Need for Comparative Effectiveness Research?
• Outcomes that matter to patients
• Practical measures
• Coverage of a wide range
• Greater precision
• Comparability of scores
• Ease of interpretation24
What do the results mean?
25
PRO Validation Must be Comprehensive
• Diagnosis
• Disease severity
• Responders
• Treatments
MeasuresIn
Question
GoldStandard
• Work productivity• Costs of care• Mortality• Self- evaluated
health
OtherMeasures& Methods
• Diagnosis
• Disease severity
• Responders
• Treatments
• Work productivity• Costs of care• Mortality• Self-evaluated
health
GoldStandard
OtherMeasures& Methods
Adapted from: Ware JE, Jr. and Keller SD: Interpreting general health measures, in: Quality of Life and Pharmacoeonomics in Clinical Trials. Philadelphia, PA: Lippincott-Raven Publishers; 1995: Chapter 47.
Causes Consequences
26
What Do Differences in Treatment Effectiveness Mean?
30 40 50
Chronic Lung Disease
Physical Component Summary (PCS)
DiabetesType II
Congestive Heart Failure
Average Adult
Asthma After
Rx
Asthma Before
Rx
Treatment
Average Well Adult
50% reduction in disease burden 33% reduction in hospitalization Substantial increase in work productivity Subsequent cost savings
Matching Methods to Applications:“Choosing the Right Horse for the Course”
• Population monitoring
• Group-Level outcomes monitoring
• Patient-level measurement/management
27
28
1
2
3
4
5
6
7
1
3
5
7
Single-Item
1
2
3
4
5
Multi-ItemScale
PopulationMonitoring
Group-LevelOutcomesMonitoring
Patient-LevelManagement
“Item Pool” (CAT Dynamic)
7
6
Most Functionally Impaired
NoisyIndividual
Classification
Very AccurateIndividual
Classification
Matching Methods to Applications
Solutions
• Improved psychometrics (Item response theory – IRT)
• Computerized adaptive testing (CAT) software
• The Internet (and other connectivity)
29
Business Week. November 26, 2001.
30
First, Construct Better Metrics
Source: Business Week 11/26/01
• Comprehensive Item “Pools”• IRT Cross Calibration of Items
ADLSIPFIM
Physical Functioning (PF)
PF-10
+ =
NEWPF
% @ Ceiling:
1980 “PF Ruler” > 75% @ Ceiling
1990 “PF Ruler” > 30% @ Ceiling
2008 “PF Ruler” < 3 % @ Ceiling
31
RheumatoidArthritis
Physical Function (PF), Mean = 50 Source: Rose M, Bjorner JB, Becker J, Fries JF and Ware JE. Evaluation of a preliminary physical function item bank supported expected advantages of the Patient-Reported Outcomes Measurement Information System (PROMIS). Journal of Clinical Epidemiology, 2008, 61, 17-33.
6.0
5.0
4.0
3.0
2.0
1.0
0 10 20 30 40 50 60 70 80
StandardError
0.75
0.90
0.95
Reliability
PF “Criterion”(Item Bank)
PF-10 (“Static”)
PF-2 (“Static”)
PF CAT-10
PF-1 (“Static”)
Precision Varies Across “Static” and Dynamic Forms and Across Score Levels
32
CAT
2nd Solution, Assess Health Dynamically
Patient scores
here
CAT = Computerized Adaptive Testing
33
What are the Advantages of Dynamic Assessments?
• More accurate risk screening • Reliable enough to monitor individual
outcomes• Brevity of a short form –
90% reduction in respondent burden• Elimination of “ceiling” & “floor” effects • Can be administered using various data
collection technologies• Markedly reduced data collection costs• Monitor data quality in real time
35
Reference – Headache Impact: MS Bayliss, JE Dewey, R Cady etal., A.Study of the Feasibility of Internet Administration of a computerized health survey: The Headache Impact Test (HIT), Quality of Life Research, 2003, 12: 953-961
References – Asthma Control: Nathan RA, Sorkness CA, Kosinski M et al., “Development of the Asthma Control Test: A survey for assessing asthma control. Journal of Allergy and Clinical Immunology. 2004;113: 59-65.
www.amIhealthy.com www.asthmacontroltest.com
/
3rd Solution: The Internet
September 15, 2009
More Practical Metrics for Standardizing Health Outcomes in Effectiveness Research ([email protected])
Conclusions
• Patient-reported outcomes (PROs) are very useful
• Standardization of concepts & metrics is enabling comparisons across treatments & settings
• Increasing widespread use proves that more practical tools will be adopted
• Promising technological advances include: item response theory (IRT), computerized adaptive testing (CAT) and Internet-based data capture