MbsRwiMiatrrtro

liphadCn

eg

pma

FIrhft

Journal of the American College of Cardiology Vol. 53, No. 10, 2009© 2009 by the American College of Cardiology Foundation ISSN 0735-1097/09/$36.00P

Public Reporting of Quality MeasuresWhat Are We Trying to Accomplish?

Robert M. Califf, MD, MACC, FAHA,* Eric D. Peterson, MD, MPH, FACC, FAHA†

Durham, North Carolina

ublished by Elsevier Inc. doi:10.1016/j.jacc.2008.10.056

trmP

ttRsitpptcemdophag

oppststwfRocscuccp

ore than a decade has passed since we first commented onoth the promise and potential pitfalls of “cardiovascularcorecard medicine” (1,2). In this issue of the Journal,esnic and Welt (3) report on their real-world experiencesith public outcomes reporting for percutaneous coronary

ntervention (PCI) at Brigham and Women’s Hospital inassachusetts. Their findings raise a number of fascinating

ssues that deserve careful consideration by cardiologists,dministrators, researchers, policy makers, payers, and pa-ients. Although some might see their work as an attempt toationalize a mediocre performance rating, we believe theireport serves a nobler purpose—it is the proverbial canary inhe coal mine, warning of serious challenges inherent inelying upon in-hospital mortality as a meaningful measuref quality of care.Public reporting of mortality outcomes in medicine has a

ong and complicated history, beginning, perhaps, with thell-fated Dr. Earnest Codman. Dr. Codman, also from arestigious Boston institution, daringly proposed to reportis hospital’s mortality results, believing that this wouldttract informed consumers and would soon become stan-ard practice for the country (4). Unfortunately for Dr.odman, his hospital shortly went out of business andobody followed his example.Eighty years after the inglorious end of Dr. Codman’s

xperiment, believing that outcomes reporting would soonain widespread acceptance, we predicted that:

The convergence of concern about costs of medical care, the avail-ability of large amounts of clinical outcome data in computerizeddatabases, and dramatic advances in the methods of assessing factorsrelated to outcome have ushered in a new era of accountability forphysicians, hospital and health care systems (1).

Although our proposal (like Dr. Codman’s) was a bitremature, a decade later, public reporting of hospitalortality data is now a reality for all U.S. cardiologists. In

ddition to the statewide activities discussed in this article,

rom the *Duke Translational Medicine Institute and the †Duke Clinical Researchnstitute, Durham, North Carolina. Dr. Califf does not have specific industryelationships on this topic; a complete list of industry relationships can be found atttp://www.dcri.duke.edu/research/coi.jsp. Dr. Peterson has received research supportrom Bristol-Myers Squibb/Sanofi, Schering-Plough, and Merck/Schering; addi-

aional disclosures can be found at http://www.dcri.duke.edu/research/coi.jsp.

Manuscript received October 14, 2008; accepted October 21, 2008.

he Center for Medicare and Medicaid Services (CMS)outinely reports 30-day hospital mortality rates for acuteyocardial infarction and heart failure, and will soon addCI mortality and heart failure readmission rates (5).As we enter this new era of accountability, it is essential

hat the health services research community investigate bothhe intended and unintended consequences of these policies.esnic and Welt (3) contribute to this evaluation by

crutinizing the public reporting process for PCI mortalityn Massachusetts. Using their own hospital as an example,he investigators describe the challenges in benchmarkingerformance for acute PCI outcomes. They point out thatresent-day PCI mortality rates reflect relatively rare eventshat occur most commonly among patients who arrive at theatheterization laboratory with a multifactorial profile ofxtreme illness (e.g., in cardiogenic shock, with acuteyocardial infarction, or with other major comorbid con-

itions). They also note that current risk-adjustment modelsften fail to fully capture these complexities, and theyrovide convincing evidence that public profiling efforts mayave the unintended result of encouraging clinicians tovoid those high-risk patients most likely to receive thereatest benefit from the procedure.

Is the public release of PCI mortality data good or bad forur field? We might conclude that it is both, depending inart on the goal of such reporting. An oft-stated reason forublic reporting programs is to inform consumers’ choices,o that they can select a quality provider and hospital forheir PCI procedure. This rationale, however, falls flat foreveral reasons. First, mortality is a crude measure to applyo situations characterized by very low event rates, or forhich the ability to predict the outcome depends upon

actors that are poorly collected by current systems. Asesnic and Welt (3) note, PCI mortality occurred mostften when it was performed under urgent or emergentonditions (and therefore, in circumstances in which con-umer choice is moot). Under elective conditions, mortalomplications related to PCI are exceedingly rare, making itnlikely that provider quality can be differentiated fromhance background events. For example, an earlier studyoncluded that under plausible conditions, up to 90% of trueoor PCI performers could be missed by a profiling system,

nd 60% to 70% of physicians identified as poor quality

pr2epn

riiWtbpaQtoroe

stwsioorwhPvNsrmwc

cada“pYcriawgsA

daa

spmftettp

schirafptit

nicpm

iaict

tptvo

RCMBd

R

832 Califf and Peterson JACC Vol. 53, No. 10, 2009Public Reporting of Quality Measures March 10, 2009:831–3

roviders might be falsely labeled, simply because of purelyandom variation (6). Additionally, public mortality data are

or more years out of date when reported. Becausestimates for hospitals also tend to vary markedly by year,ast outcomes information, like stock market warnings, mayot reflect current or future performance.On a more positive note, provider profiling efforts can in fact

einforce clinicians’ and administrators’ interest in the qualitymprovement (QI) process. Again, Resnic and Welt’s paper (3)llustrates this effect. It is debatable whether Brigham and

omen’s Hospital would have assigned 3 full-time employeeso support PCI clinical data collection and QI efforts had it noteen for state mandates or the impending public release oferformance data. Additionally, although the detailed reviewfforded to each PCI mortality event is laudable, such intensiveI was in part motivated by the desire to demonstrate flaws in

he provider profiling system rather than purely to identifypportunities for preventing errors. Thus, although the publicelease of outcomes data may not necessarily serve the purposef informing the public, it can nonetheless supply the strongxternal motivation needed to pursue internal QI activities.

Given that public reporting of death rates is likely here totay, what can we do to make the best use of this informa-ion? Resnic and Welt (3) make several good suggestions,hich we would like to highlight and augment: First, we are

keptical that any amount of tuning can resolve the difficultssue of predicting death after PCI with sufficient accuracy,r can overcome statistical challenges arising from the lawsf small numbers. Although we do not support the publicelease of potentially misleading PCI mortality information,e do support state and national efforts that would mandateospitals to collect and compare clinical information onCI, such as that required in Massachusetts and offeredoluntarily by the American College of Cardiology (ACC)–ational Cardiovascular Data Registry (7). Armed with

uch data, we would further encourage all institutions tooutinely review any deaths occurring during PCI in aanner similar to that outlined by Resnic and Welt (3),ith the main goal of identifying opportunities for future

are improvement.In place of the public release of procedure-related out-

omes, we would argue for better and more completenalysis of condition-specific outcomes (i.e., acute myocar-ial infarction and heart failure). For example, by examiningll patients with MI, one can lessen concerns of providercase selection creep,” as both patients with and without arocedure will ultimately be included in the denominator.et in order to pursue such a course, we will need improved,

omprehensive data, since current provider profiling effortsely on administrative databases that are notoriously limitedn terms of the completeness and accuracy needed for riskdjustment (8). Instead of implementing flawed profilingith inadequate administrative data, we again would sug-est expanding participation in national clinical registries,uch as those run by the ACC and the American Heart

ssociation. In addition to providing incentives or man-

ates for participation, they could also provide support foruditing registries for completeness and accuracy and, ide-lly, for including longitudinal outcomes information.

We would also argue that any outcomes reporting systemhould be augmented with data on care process and appro-riateness. Assuring that PCI patients receive optimaledical therapy may be as important as the procedure itself

or long-term health outcomes, as demonstrated by recentrials (9). Thus, reporting on the routine use of thesevidence-based secondary prevention therapies may increaseheir ubiquity, which might potentially save many more liveshan reporting on small differences in the quality of therocedure itself could.Similarly, as Resnic and Welt (3) acutely point out,

election of patients likely to benefit from the procedure isritical. Close monitoring of procedural appropriateness canelp us avoid unnecessary risk to patients lacking good

ndications for a given procedure, while simultaneouslyeducing the associated financial costs. Procedural appropri-teness criteria for PCI are being developed by the ACC toacilitate such measurement, and digital technology nowermits remote oversight of angiograms to confirm ana-omic findings and allow peer feedback. But again, externalncentives must be in place before institutions will be willingo routinely collect and report these data.

Better education of the public and of provider commu-ities on how to interpret and make use of this information

s essential. We also need better methods for summarizingomplex, multidimensional information on care structure,rocesses, and outcomes into composite measures that areeaningful and reflect their intended purpose and values (10).Finally, professional regulations are needed to prevent

nappropriate marketing of comparisons of quality data thatre linked with advertisements promoting false or overstatedmpressions. Even in an era of public transparency andompetition, the main role of quality assessment should beo promote QI, not increase market share.

In summary, Resnic and Welt (3) have done all of us inhe cardiology community a favor by “singing” about theroblems with the application of PCI mortality risk scoreso assess comparative quality. As with any quality tool, thealue of the measurement will be determined by the wisdomf those who use it.

eprint requests and correspondence: Dr. Robert M. Califf, Vicehancellor for Clinical Research, Director, Duke Translationaledicine Institute, Duke University Medical Center, Campus

ox 3850, Durham, North Carolina 27710. E-mail: rob.califf@uke.edu.

EFERENCES

1. Topol E, Califf R. Scorecard cardiovascular medicine. Its impact andfuture directions. Ann Intern Med 1994;120:65–70.

2. Califf RM, Jollis JG, Peterson ED. Operator specific outcomes: A callfor professional responsibility. Circulation 1996;93:403–6.

1

K

833JACC Vol. 53, No. 10, 2009 Califf and PetersonMarch 10, 2009:831–3 Public Reporting of Quality Measures

3. Resnic FS, Welt FGP. The public health hazards of risk avoidanceassociated with public reporting of risk-adjusted outcomes in coronaryintervention. J Am Coll Cardiol 2009;53:825–30.

4. Codman E. A Study in Hospital Efficiency. Boston, MA: Privatelyprinted, 1916.

5. Center for Medicaid and Medicare Services. Technical specificationsfor proposed ACE demonstration quality measures requested in theacute care demonstration application. Available at: http://www.hospitalcompare.hhs.gov/Hospital/Search/Welcome.asp. Ac-cessed October 13, 2008.

6. Thomas JW, Hofer TP. Accuracy of risk-adjusted mortality rate as ameasure of hospital quality of care. Med Care 1999;37:83–92.

7. National Cardiovascular Data Registry. Available at: http://

www.ncdr.com/WebNCDR/Common/. Accessed October 6, 2008. c

8. Jollis JG, Ancukiewicz M, DeLong ER, Pryor DB, Muhlbaier LH,Mark DB. Discordance of databases designed for claims paymentversus clinical information systems. Implications for outcomes re-search. Ann Intern Med 1993;119:844–50.

9. Boden WE, O’Rourke RA, Teo KK, et al. Optimal medical therapywith or without PCI for stable coronary disease. N Engl J Med 2007;356:1503–16.

0. O’Brien SM, DeLong ER, Dokholyan RS, Edwards FH, PetersonED. Exploring the behavior of hospital composite performance mea-sures: an example from coronary artery bypass surgery. Circulation2007;116:2969–75.

ey Words: percutaneous coronary intervention y quality assurance y

ardiogenic shock.