Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues

HIP Health Plans/Medical & Quality Informatics 1

Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues

Randall K. Spoeri, Ph.D.

Vice President

Medical and Quality Informatics

HIP Health Plans

New York, NY


Coverage• Measurement and Quality in Managed Care

• Some Statistical and Computing Issues– Measure Definitions– Data Availability and Quality– Risk Adjustment– Analytical/Interventional Use of Results– Predictive Modeling

• Thoughts About the Future


About HIP Health Plans• A managed care company with membership

of over 1 million in metropolitan NYC and FL

• Has a long history of over 50 years of service to NYC residents

• 2000 revenues budgeted at $2 Billion

• Has a strong commitment to quality measurement and improvement


Measurement and Quality in Managed Care

• Managed care has experienced rapid expansion in recent years

• With that expansion, concerns have been expressed about the quality of care delivered

• In response, quality management efforts have relied heavily on performance measurement and accreditation review



• The National Committee for Quality Assurance (NCQA) has been the major provider of accreditation reviews and performance measurement standards

• The Health Plan Employer Data and Information Set (HEDIS) is the NCQA-sponsored collection of standardized performance measures widely used by Managed Care Organization (MCOs)



• With the growth of managed care and the increasing focus on quality and performance measurement, MCOs have begun to expand their staff to include medical and quality informatics personnel



• Skills of Medical and Quality Informatics (MaQI) staff include: statistics and statistical computing, epidemiology and public health, psychology, and economics

• Demand is high for this type of individual, not only in MCOs, but also in allied areas such as hospital organizations and pharmaceutical companies



• Statisticians and computing professionals can make major contributions to the health care industry, especially managed care

• The opportunities are many, as are the challenges


Some Statistical and Computing Issues

• Measure Definitions

• Data Availability and Quality

• Risk Adjustment

• Analytical/Interventional Use of Results

• Predictive Modeling


Measure Definitions

• In order that comparisons can be made among health plans, it is essential that measure definitions be uniform, unambiguous, and precise

• In the managed care world, HEDIS is the standard set of performance measures used by MCOs


Measure Definitions

• The recently announced HEDIS 2001 will have 52 measures across 7 domains of care: Effectiveness of Care, Access/Availability of Care, Satisfaction with the Experience of Care, Health Plan Stability, Use of Services, Informed Health Care Choices, and Health Plan Descriptive Information


Measure Definitions

• Example measures: Childhood Immunizations, Breast Cancer Screening, Beta Blocker Treatment After a Heart Attack, Practitioner Turnover, Cesarean Section Rate

• Developing specifications (i.e., detailed definitions) for these measures is an arduous task, often involving statistical issues


Measure Definitions

• Some statistical issues include: – Measure reliability/validity– Data source (administrative or medical record)– Complete enumeration or sample (or hybrid)– Sampling methodology– Sample size

• Technical Panels were convened by NCQA to advise on these and other issues


Data Availability and Quality

• Encounter data are administrative data supplied by a provider (i.e., a physician) to a health plan on outpatient visits by the plan’s members

• The incentives to supply these data range from none to weak, to strong: monetarily or contractually or both



• Consequently, encounter data repositories are often incomplete and frequently of poor quality

• Many physicians don’t see or understand the importance of this encounter data, and view it as yet another disruption imposed by MCOs



• The movement from paper-based systems for submission of encounter data, to electronic ones has promise

• The ever-expanding use of encounter data for such things as physician performance measurement (so-called “provider profiles”) has begun to garner physician attention



• Federal and state mandates may also help to improve the completeness of encounter data

• But even when data repositories are complete, there remain major questions about the quality of the data contained therein

• Encounter, claim, pharmacy, lab, and other health care data are still of uneven quality



• One approach, taken by NCQA, in connection with the quality of HEDIS data is to require that the complete system for this data’s production be audited (i.e., externally reviewed and verified): structure, process and outcome



• Other techniques used in data quality assurance include the examination of source code used for administrative data pulls, medical record reviews to measure the accuracy of administrative data counterparts, and medical record re-review to assess the accuracy of primary chart abstraction



• With the increasing reliance of MCOs on data and measurement, through complex data structures like data warehouses, the completeness and accuracy of the information becomes paramount

• Statisticians and IS/IT professionals need to work with clinicians and administrators to continuously improve these data systems


Risk Adjustment

• When comparing entities such as hospitals and physicians, it is important that an adjustment be made for differences in the mix of patients being treated

• For example, it would be inappropriate to compare one hospital providing mostly obstetrical care in the suburbs to an inner city trauma center hospital using mortality


Risk Adjustment

• Similarly, comparing a physician seeing predominantly young patients to another physician seeing a large proportion of Medicare patients is not appropriate when looking at pharmacy usage

• Many of the measures used to assess hospital and physician performance need risk adjustment, a leveling of the field based upon differing patient characteristics


Risk Adjustment

• Characteristics used generally include age, gender, diagnosis and procedure, as well as comorbidities, for example

• One commonly used approach to risk adjust performance statistics is to use the concept of “observed and expected”


Risk Adjustment

• The observed result could be a hospital’s average length of stay for a certain surgical procedure

• The expected result would be that predicted using epidemiological or statistical methods for the patient population undergoing the surgical procedure at that hospital


Risk Adjustment

• Statistical issues include methodology selection, model calibration, and explication

• Popular methods include multiple and logistic regression and survival analytic (Bailey-Makeham)

• Calibration issues often surround data: timeframe, geography, availability and quality, breadth


Risk Adjustment

• Explication: Users of the data often become suspicious of methods they don’t understand (e.g., complex statistical methods), even when the methods are sound

• Computing issues arise in building and handling the enormous databases common in managed care (e.g., encounters)


Risk Adjustment

• Computing challenges also arise out of building and using complex risk adjustment models, or even in using models developed by others (i.e., commercial vendors)

• Nevertheless, these issues and challenges provide opportunities for statisticians, computing professionals, and clinically prepared individuals, working together


Analytical/Interventional Use of Results

• The best measurements and measurement systems are of no value unless they are followed by action and behavior change

• Some analyses prompt action and intervention by their very nature…an example follows using real HIP data


HIP Health Plan of New York Surgery Referrals As Percent of Total Specialty Referrals by PCP

Medicare Risk Population, 7/98 - 6/99

SI 999999

CB 888888

KB 777777

QL 666666

NY 555555SI 444444

KB 333333CB 222222

NY 000000QL 111111

0%

10%

20%

30%

40%

50%

0 100 200 300 400 500 600Total Number of Specialty Referrals by PCP

Perc

en

t o

f S

urg

ery

Refe

rrals

Note: 1). Excludes outliers. An outlier is defined as having total referrals of greater than 600 or the surgery referral rate of greater than 50%. 2). The rates are for members who have been continuously enrolled with HIP for 12 months and who have been with the current PCP for the last 3 months. 3). The observation represents a PCP's rate. If a PCP's rate falls outside of the upper 95% confidence limit, he/she is shown with an identifier. The identifier is a combination of the PCP's Medical Group and his /her provider ID. Codings for Medical Groups are: NY=New York, CB=Central Brooklyn, KB=Kingsboro, QL=Queens/Long Island, SI=Staten Island.

Upper 95% Confidence Limit



• Most analyses, however, require an intervention strategy

• For example, learning that an MCO has a low rate of childhood immunizations could prompt a variety of actions



• Example interventions:– Reminder to Primary Care Physician– Reminder to Pediatrician– Reminder to Parents– Reminder stickers for medical chart– Promotional brochures– Educational forums


Predictive Modeling

• Most MCOs have undertaken disease management initiatives for high prevalence/high cost conditions, such as asthma, diabetes, congestive heart failure and hypertension

• As part of these efforts, predictive modeling is sometimes undertaken to identify high risk individuals with a given disease


Predictive Modeling

• The goals are to:– Avoid costly ER visits or hospitalizations– Treat individuals at higher risk earlier in the

disease process, and monitor them more closely– Reduce the risk of adverse outcomes– Improve satisfaction and quality of life


Predictive Modeling

• Some of the methodologies used include– Multiple Regression (Linear and Nonlinear)– Logistic Regression– Neural Networks– Time Series

• Many of the disease management vendors that have come on the scene tout predictive modeling as a value-added benefit


Predictive Modeling

• Many MCOs do some of this work themselves or in partnership with disease management vendors

• There is a need, however, to assure that the statistical methodologies are being properly utilized, and that the data used to both build and apply the models be accurate

• Results to date have been encouraging


Thoughts About the Future

• MCOs will continue to expand their performance measurement and improvement activities

• Accreditors (e.g., NCQA), Federal and State Agencies (e.g., HCFA and DOH/DOI), and purchasers/employers are driving much of this expansion



• The use of statistical methods in MCO quality management will increase

• Administrative data systems must improve to greatly reduce the need for medical chart abstraction

• Opportunities for statisticians with a bent toward computing will abound in MCOs



• “The future ain’t what it used to be” - Yogi Berra

Documents

Measuring and Improving Quality in Managed Care: Some Statistical and Computing Issues