Upload
tana-hughes
View
17
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Methodological Issues in Physician-Level Measurement of Clinical Quality. Elizabeth A. McGlynn, Ph.D. June 26, 2006. Information About Individual Physicians’ Performance is Increasingly Sought. - PowerPoint PPT Presentation
Citation preview
Methodological Issues in Physician-Level Measurement
of Clinical Quality
Elizabeth A. McGlynn, Ph.D.June 26, 2006
McGlynnAcadHealth-2 06/26/06
Information About Individual Physicians’ Performance is Increasingly Sought
• Health plans believe they can save money through differential payments to physicians (pay for performance)
• Employers believe they can save money through increasing consumer cost-sharing (consumer directed health plans)
• Medical groups believe they can negotiate higher rates or market share by demonstrating better performance (tiered networks, rate increases)
• Consumers are likely to demand information on performance as the share they pay for health care increases (public release)
McGlynnAcadHealth-3 06/26/06
What Is Being Measured?
McGlynnAcadHealth-4 06/26/06
Data Sources for Measuring Quality
• Available sources include:– Administrative (claims) data– Manual abstraction of medical records– Surveys of patients– Inspection of office practice– Extraction of data from electronic medical
records– Board certification/Maintenance of certification
• Each of these sources has strengths and weaknesses• No single source is adequate to address all
questions
McGlynnAcadHealth-5 06/26/06
Most Existing Approaches to Measuring Physician Performance Use Claims Data
• Data are readily available and impose less burden on providers
• But they have some significant problems– Generally available one payer at a time– Information availability driven by the benefit
package and the ways coding systems are used– Some confounding of physician practice
patterns with patient behavior
• Pressure to deliver answers driving widespread use of these methods
McGlynnAcadHealth-6 06/26/06
Current Approaches to Quality Measurement
• “Leading indicators”– One measure at a time
• Condition-specific aggregates/composites– Multiple measures on the same population with
the same health problem
• Comprehensive cross-condition measures– Patient as the unit of analysis
McGlynnAcadHealth-7 06/26/06
Examples of Where These Approaches Are Currently Used
Approach Use
Leading indicators Pay for performance
Public reporting
Tiered networks
Disease composites Recognition programs
Maintenance of certification
Comprehensive aggregates Not in widespread use
McGlynnAcadHealth-8 06/26/06
What You Measure May Affect the Conclusions You Draw
0 20 40 60 80 100
Cardiology
Family Practice
Endocrinology
InternalMedicine
% of recommended care delivered
HbA1c DM Overall QA Tools Overall
McGlynnAcadHealth-9 06/26/06
Some Challenges in Measuring Physician Performance
McGlynnAcadHealth-10 06/26/06
Physicians See Multiple [Different] Patients
MD1
PT3
PT2
PT1
So, representing the variety of practice matters:Case Mix Adjustment
MD2
PT5
PT4
PT3
McGlynnAcadHealth-11 06/26/06
A Market Basket of Indicators May Be Necessary to Reflect the Variety of Practice
0%
20%
40%
60%
80%
100%
Cardio
logy
Endocrin
ology
Family
Pra
ctic
e
Inte
rnal
Med
icin
e
OB-GYN
% o
f el
igib
le e
ven
ts
Afib
CAD
CHF
Diabetes
Headache
Hypertension
Hyperlipidemia
Pneumonia
Prenatal
Preventive
UTI
Other
McGlynnAcadHealth-12 06/26/06
Patients See Multiple Providers
PT1
MD3
MD2
MD1PT2
PT3
PT4
PT5
PT6
PT9
PT8
PT7
So, determining who is “responsible” mattersAttribution
Hosp A
Hosp B
McGlynnAcadHealth-13 06/26/06
Information Rarely Available to Link Patients to Physicians a Priori
• As gatekeeper models decline, no clear assignment of patients to a physician exists
• Algorithms are used to “assign” patients to physicians– Done most frequently in economic profiling– Basis is majority of dollars or visits
• We are experimenting with other rules:– First eligible provider seen in study period– Provider “triggering” eligibility for indicator
• Critical to reality test assignments
McGlynnAcadHealth-14 06/26/06
Physicians Have Multiple Contracts
Medicare
MD3
MD2
MD1PacifiCare
Humana
Wellpoint
United
Aetna
Medicare
Medicare
Anthem
So, putting the pieces together matters:Aggregation
McGlynnAcadHealth-15 06/26/06
Few Physicians Can Be Evaluated Using Single Indicators from One Payer
0 10 20 30 40 50
Cardiology
Family Practice
Endocrinology
Internal Medicine
% of MDs with >10 eligibilities
HbA1c DM Overall QA Tools Overall
McGlynnAcadHealth-16 06/26/06
Physicians Practice in Different Systems
So, understanding the organizational context matters:Fair comparisons
McGlynnAcadHealth-17 06/26/06
Little Routine Information Available on Physician Practice Setting
• Taking organizational context into account is challenging because of data limitations
• Using location may be misleading– Shared space vs. shared practice
• Rationale for constructing scores at group level:– Increase sample size– Demonstrate value of integrated medical groups– Avoid scores at the physician level
• Relatively little known about within vs. between group variation
McGlynnAcadHealth-18 06/26/06
Categorizing Physician-Level Results• Many applications of physician-level scoring
require using results to categorize physicians– In/out of network– In/out of performance bonus– Tiering
• We prefer statistical testing to straight cut-points• Applied this to the three different approaches to
MD-level scoring– Test performance compared to the mean– Use 95% confidence interval around each
provider’s score– Those with scores significantly below average
were assigned to the low performance category
McGlynnAcadHealth-19 06/26/06
Different Methods Will Result in Different Category Assignments
0% 20% 40% 60% 80% 100%
HbA1c
DMComposite
QA ToolsOverall
% of internists in category
Not rated
1 star
2 stars
3 stars
McGlynnAcadHealth-20 06/26/06
Different Results Under Different Systems Likely To Produce Challenges from MDs
0% 20% 40% 60% 80% 100%
% of internists
Agree
DM>QAT
DM<QAT
McGlynnAcadHealth-21 06/26/06
Summary
• A number of methodological issues arise in creating quality scores at the physician level
• We need to better understand the implications of these methodological choices
• Because the data on which the scores are based were not intended for this purpose, feedback loops and data quality improvement are essential
• But, the world isn’t going to wait for us to get the methods perfect…
McGlynnAcadHealth-22 06/26/06
This Train Is Headed Your Way!