Graduate Medical Education

Benefits and Costs of Pay for Performance asPerceived by Residents: A Qualitative StudySarah Augustine, MD, Renee H. Lawrence, PhD, Pratibha Raghavendra, MD,and Brook Watts, MD, MS

Abstract

PurposeTo describe and interpret pay-for-performance (P4P) systems as perceivedby internal medicine residents to furtherdevelop and improve curricula that relateP4P measures to quality improvementinitiatives.

MethodIn 2008–2009, the authors conducted aqualitative study in which 97 internalmedicine residents completed amandatory survey soliciting their views ofthe advantages and disadvantages ofP4P. The authors analyzed responses toidentify and categorize emergentthemes.

ResultsEighty-two residents (85%) notedadvantages. Analysis of these responsesindicated that P4P improves overalldelivery of quality care by enablingquality care and by motivating providersto improve or provide quality care.Analysis also indicated that P4P enablescontemplative and knowledgeenhancement (e.g., by promotingreflection) and has potential impactsboth on physicians’ delivery of bettercare (e.g., by facilitating vigilance andcloser follow-up) and on the care deliveryprocess (e.g., by increasingpay/satisfaction). Eighty-seven residents(90%) indicated disadvantages. Theemergent themes reflected P4P’s impactson patient perceptions (e.g., by

decreasing patient satisfaction andaccess), role in compromising clinical care(e.g., by fostering abuse/gaming andcompromising focus, care, and safety),impacts on resources and efficiency, andimpacts on providers that mayundermine morale.

ConclusionsResidents’ reported advantages anddisadvantages were often in directopposition to each other (e.g., P4Penables quality care but alsocompromises focus, care, and safety).These opposing responses form acontinuum that the authors believe willrequire providers to perform a balancingact to practice successfully in a P4Penvironment.

Pay for performance (P4P), which hasshown success in the business world, hasemerged at the forefront of discussionson U.S. health care reform over the past

several years. The interest in adoptingP4P in the health care sector is based onthe belief that P4P programs are a meansto reduce medical errors, optimizequality of care, and control escalatinghealth care costs.1 In most P4P programs,physicians or physician groups reporttheir performance on predefined qualityindicators in order to receivesupplemental reimbursement above theirbaseline earnings. Inherent in P4P is theassumption that financial rewards willencourage providers to improve theirperformance on certain qualityindicators, ultimately translating intodirect improvement in the quality of careprovided. On the basis of thisassumption, P4P is viewed as aligningwith quality improvement (QI)initiatives.2

The arguments for adopting P4P inhealth care are based on several perceivedadvantages. P4P is viewed as amechanism that would ensure thatproviders and institutions are rewardedfor the quality of care provided ratherthan reimbursed according to thetraditional, capitation-based system. P4Pis also seen as a means to ensurestandardization of care regardless of the

setting in which care is provided. Inaddition, giving providers incentives toimprove patient care across settings mayimprove coordination of care forcomplex patients with multiple providers.Improvements in coordination of careare expected to lead to improvement inelectronic databases and registrydevelopment.3,4

Given that many desired health outcomesare long-term goals and most P4Pprograms are in their infancy, it may notbe surprising that the literature related toP4P has focused primarily on theunintended negative consequences ofinstituting P4P.5–7 Such unintendedconsequences include targeting “high-yield” medical problems (e.g., achievingtarget hemoglobin A1c �7%) at theexpense of other “lower-yield” ones (e.g.,addressing patient concerns related topolypharmacy); increasing exceptionreporting whereby the sickest patients ina population are referred out to removethem from the denominator8; increasingtime spent on documentation rather thandirect patient care; and treating toachieve numbers (e.g., achieving bloodpressure �140/90 in 75% of a patientpopulation) rather than to meet

Dr. Augustine is associate chief of medicine,Department of Medicine, Louis Stokes Cleveland VAMedical Center, and assistant professor, Departmentof Medicine, Case Western Reserve University Schoolof Medicine, Cleveland, Ohio.

Dr. Lawrence is research health scientist, VeteransAffairs Health Services Research and DevelopmentCenter for Implementation Practice and ResearchSupport, and Department of Medicine, Louis StokesCleveland VA Medical Center, Cleveland, Ohio.

Dr. Raghavendra is quality scholar, VA NationalQuality Scholars Fellowship Program, Louis StokesCleveland VA Medical Center, Cleveland, Ohio.

Dr. Watts is director, Medical Quality Improvement,Department of Medicine, Louis Stokes Cleveland VAMedical Center, and assistant professor, Departmentof Medicine, Case Western Reserve University Schoolof Medicine, Cleveland, Ohio.

Correspondence should be addressed to Dr. Augustine,Department of Medicine, Louis Stokes ClevelandDepartment of Veterans Affairs Medical Center, 10701East Boulevard (111W), Cleveland, OH 44106;telephone: (216) 791-3800, ext. 3620; fax: (216)231-3289; e-mail: sarah.augustine@va.gov.

Acad Med. 2010;85:000–000.First published onlinedoi: 10.1097/ACM.0b013e3181fa7aec

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afm

Academic Medicine, Vol. 85, No. 12 / December 2010 1

<zdoi;10.1097/ACM.0b013e3181fa7aec> • <zjs;Graduate Medical Education> • <zjss;Graduate Medical Education>

individual patient preferences or increasepatient satisfaction.1,3 Other unintendedconsequences include changing thepatient–physician dynamic duringroutine office visits, threatening thephysician’s ability to practiceautonomously,9 and shifting resourceallocation with the intent to designsystems geared to meet measures ratherthan individual patients’ needs.10

Concerns also have been raised regardingthe potential worsening of health caredisparities. P4P incentives tend to placeadditional financial stressors on healthcare facilities that cater to primarilyunderserved minorities by rewardingonly those institutions that demonstrateattainment of predefined benchmarksrather than rewarding those that showincremental improvements in meetingquality measures. Safety net institutionsoften are able to document improvementin quality measures but fall short of thepredefined benchmarks as they havefewer resources to allocate towardmeeting the quality measures.3,11

Although the literature addressesunintended consequences of P4P, it lacksstudies that assess physician trainees’perceptions of P4P and explore theirviews of how P4P affects the delivery ofcare to their patients. By gaining insightinto residents’ perceptions of P4P, webelieve that medical educators may bebetter equipped to develop curricula thatincorporate instruction on aligning P4Pincentives with improved quality of care.As many practice environments haveadopted P4P, we believe that it isnecessary for residents to understand theissues involved and the ways they canfacilitate the alignment of P4P and QIpractices to ensure that they enjoycontinued success in their chosen fields.We are not aware of any curricula withformal instruction on these issues, whichis an important gap to close given thatP4P continues to grow in scope. Thus, tofurther develop and improve ourcurriculum for internal medicine (IM)residents, we undertook a qualitativestudy to describe and interpret P4P as aphenomenon experienced by physiciantrainees.

Method

Setting and participants

This study was conducted at the LouisStokes Cleveland VA Medical Center

(VAMC), a teaching affiliate ofUniversity Hospitals (UH) Case MedicalCenter. All IM residents in the combinedUH/VAMC residency program rotatethrough a four-week ambulatory careblock at the VAMC during each of theirthree years of residency training. Since2006, residents rotating through thisblock have participated in a practice-based learning and improvement (PBLI)curriculum. In June 2008, this PBLIcurriculum was modified to requireresidents to attend two didactic sessions(conducted during the first and thirdweeks of the block) targeted towardintroducing the discipline of QI.Residents at all levels of trainingparticipated jointly in the curriculum.We focused our study on IM residents atour own institution to help furtherdevelop our evolving local PBLIcurriculum.

Data collection and analysis

During the 11, four-week block rotationsfrom July 2008 through June 2009, 97 IMresidents were required to complete amandatory survey at the beginning of thefirst PBLI session. The survey includedthe following question about P4P: “Whatare the advantages and disadvantages ofpay for performance?” We did not defineor describe P4P before distributing thesurvey or on the form because we wantedto assess the range of residents’familiarity with the concept via theirresponses (including “not sure” and “noidea”), irrespective of the sources of theirperceptions (e.g., faculty, popular press,academic readings).

No personal identifying information wasincluded on the survey. The localinstitutional review board for humansubjects reviewed the protocol andmaterials and determined that the studywas exempt.

Analyses were conducted separately foradvantages and disadvantages tocomprehensively identify emergentthemes. Consistent with qualitativeapproaches, responses were reviewed andpreliminarily coded during datacollection.12,13 To ensure as exhaustive alisting as possible (i.e., to maximizesaturation for this group), we collecteddata until no new themes emerged duringtwo consecutive blocks of data.12,13

Responses were parsed into units ofinformation that were analyzed initially

by one author (R.H.L.), who reviewedand iteratively sorted data to identifythemes. Through the iterative process,key themes that emerged provided thefirst level of interpretation and served toreduce the residents’ responses intomeaningful units of data. The emergentthemes were analyzed to determinewhether they represented advantages ordisadvantages of P4P and were thencategorized. These themes weresubsequently grouped by one author(R.H.L.) into a scheme that provided abasis for understanding P4P as perceivedby residents, an approach consistent withthe constant comparative method.12,14 Toensure completeness and accuracy of thethemes and groupings, the lead author(S.A.) reviewed the analyses anddiscussed them with the initial coder(R.H.L.) to achieve consensus. Thisrevised version was reviewed anddiscussed by all of the authors to achieveconsensus.

Once the emergent themes wereestablished, we tallied the total number ofresponses coded into each theme todetermine the prevalence of specificresident perceptions related to P4P. Thisinformation might serve as a startingpoint when considering future revisionsto our PBLI curricula.

Results

Advantages of P4P

Eighty-two of the 97 trainees (85%)provided advantages of P4P, yielding 127units of information for coding. Of these,three units were excluded because theinformation provided was insufficient tounderstand the response well enough tocode (“look at outcomes,” “doctor willspend more time,” “decreased fines”).Further, one other advantage was notlegible and could not be used. The 15residents who were not included in thisanalysis wrote such responses as “notsure,” “don’t know,” “no idea,” or “?,” orprovided no advantages.

Table 1 summarizes the 13 themes (123units) that emerged regarding P4Padvantages, by frequency of occurrence,and provides representative examples ofresidents’ responses. The two themes thatemerged with the greatest frequency werethat P4P enables quality care (29 units)and that it motivates providers toimprove care or provide quality care (28units). These themes, though similar,

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 20102

T1

Table 1Emergent Themes and Examples for Pay for Performance (P4P) AdvantagesProvided by 82 Internal Medicine Residents, July 2008–June 2009*

Emergent themes (no. of units) Representative responses

Enables quality care (29) • “Ensures patients are getting appropriate care.”• “Improve outcomes/improve patient care.”• “Increased quality of care.”• “Higher-quality care.”• “Ensures quality care is being performed.”• “Ensures patients are getting appropriate care.”

...................................................................................................................................................................................................................................................................................................................Motivates to improve care/provide quality care (28) • “Incentive for increased productivity and performance.”

• “Stimulate health care workers for better quality of care.”• “Money is always a good incentive.”• “Advantages are that physicians would have higher incentive to get patients’ numbers

at a better goal.”• “Good incentive for PCP versus procedure-based physicians.”

...................................................................................................................................................................................................................................................................................................................Increases compliance/adherence to guidelines (11) • “Pushes MD to be up to guidelines and follow them and apply them to patient to

improve care.”• “Increased compliance with guidelines.”• “Increased education about new guidelines incorporated into quality improvement

initiatives and widely disseminated by payments.”• “Encourages physician compliance to standards of care practices.”• “Forces docs to adhere to care standards and guidelines.”• “Increases the likelihood that patient will receive best practice guidelines.”

...................................................................................................................................................................................................................................................................................................................Helps ensure standardized, uniform level of care (11) • “Increased education about new guidelines incorporated into quality improvement

initiatives and widely disseminated by payments�.”• “Promotes equality.”• “Uniform care.”• “Standardize care.”• “Better standard of care—all patients getting clinical reminders, screening test.”• “Eliminate conflict of interest.”

...................................................................................................................................................................................................................................................................................................................Encourages focus on prevention and process goals (10) • “More patients on appropriate meds.”

• “Better heart disease care.”• “More patients will be on ASA.”• “Meeting process goals increases.”• “Better preventative care.”• “More age-appropriate screening.”

...................................................................................................................................................................................................................................................................................................................Facilitates vigilance/aggressive/closer follow-up (6) • “More vigilant provider—better care for patient.”

• “Closer follow-up to patient compliance.”• “More close adherence to postop/postprocedure care.”• “Encourages better patient follow-up.”• “Aggressive therapy by provider when it has shown to benefit patients.”• “May increase number of patients who reach treatment goals by physicians being

more aggressive.”...................................................................................................................................................................................................................................................................................................................Reduces costs/inefficiencies (6) • “Saves cost.”

• “May work to decrease cost of medical care.”• “Reduces overall health care costs by minimizing inpatient stay.”• “Adequate use of resources.”• “Potential improvement in outcomes for money spent on health care.”• “Only advantage is to objectify and produce data/goals with which you can define

efficiency �“...................................................................................................................................................................................................................................................................................................................Increases pay/satisfaction (5) • “Higher reimbursement.”

• “Increased pay.”• “Extra money.”• “Satisfaction among physicians in terms of health care services.”• “Perhaps a form of recognition of one’s skills � not certain.”

...................................................................................................................................................................................................................................................................................................................Promotes reflection (5) • “Force you to think.”

• “Work smarter.”• “Prompts physician to really think about what will decrease hospital stay, etc.”• “Capitalistic motivation for improving yourself.”• “� freeing up MDs’ time to be thoughtful about their own practices.”

...................................................................................................................................................................................................................................................................................................................Increases awareness of guidelines (3) • “Pushes MD to be up to guidelines �”

• “Forces practitioners to know guidelines.”• “Increased education about new guidelines incorporated into quality improvement

initiatives and widely disseminated by payments.”(Continues)

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 2010 3

address very different aspects of thepotential advantages of P4P. The firstfocuses on P4P’s ensuring quality care;the other focuses more on the rewardaspect. The remaining emergent themesrepresent more specific considerationsthat residents offered to explain why theybelieved P4P leads to improving thequality of care.

Figure 1 graphically represents the majorcategories of themes regarding theadvantages of P4P. All responses seemedto fit under an overarching category ofimproving delivery of quality care,particularly the two themes related toenabling quality of care and motivatingproviders to improve care or providequality care. The remaining themes,

which differentiate the means by whichP4P improves the delivery of quality care,are grouped under the threesubcategories that emerged. The firstsubcategory relates to morecontemplative or knowledgeenhancement themes (e.g., promotesreflection, increases physician awarenessof gaps). The second subcategory relates

Table 1(Continued)

Emergent themes (no. of units) Representative responses

Increases physician awareness of gaps (3) • “Awareness that things need to get done.”• “Make physicians more aware of problem areas and need for improvement.”• “Highlights important areas of treatment or evaluation that might otherwise be

overlooked.”...................................................................................................................................................................................................................................................................................................................Decreases error (3) • “Safer for patients.”

• “Possible decrease in errors.”• “Less error in patient care.”

...................................................................................................................................................................................................................................................................................................................Changes provider–patient relationship (3) • “Satisfaction among patients in terms of health care services.”

• “Creates incentive to encourage patients to participate in their own care.”• “Good communication.”

* Themes are listed from the most frequent to least frequent. Ninety-seven residents at Louis Stokes Cleveland VAMedical Center responded to a mandatory written survey, which asked them, “What are the advantages anddisadvantages of pay for performance?” Units refer to individual responses; some residents provided severalresponses to each question. ASA indicates acetylsalicylic acid (aspirin); PCP, primary care provider. Ellipsesindicate that text has been omitted because it was not directly related to the theme.

Advantages of P4P

Improves delivery of quality care

Theme: Theme:Theme:

• Enables quality care

Theme:

• Motivates to improve care/provide quality care

Enables contemplative and knowledge enhancement

Potential impacts linked to the care delivery process

Potential impacts on physicians’ delivery of better care

Themes:

P t fl tiThemes:

E f

Themes:

I / ti f ti• Promotes reflection

• Increases physician awareness of gaps

• Increases awareness of guidelines

• Encourages focus onprevention and process goals

• Facilitates vigilance/aggressive/closer follow-up

• Increases compliance/adherence

• Increases pay/satisfaction

• Changes provider-patientrelationship

• Reduces costs/inefficienciesg Increases compliance/adherence

to guidelines

• Decreases error

• Helps ensure standardized,uniform level of care

Figure 1 Categorization of emergent themes related to advantages associated with pay for performance as perceived by physician trainees at LouisStokes Cleveland VA Medical Center, July 2008 to June 2009. Themes were mentioned by 82 of 97 internal medicine residents responding to amandatory written survey question (“What are the advantages and disadvantages of pay for performance?”).

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 20104

F1

Table 2Emergent Themes and Examples of Pay for Performance (P4P) DisadvantagesMentioned by 87 Internal Medicine Residents, July 2008 to June 2009*

Emergent themes (no. of units) Representative responses

Compromises focus, care, and safety (25) • “But decline in other nonmeasured parameters; leading to overall poorer patient care.”• “But may lead to losing sight of overall patient care in favor of obtaining ’good numbers.’”• “Concentrating on meeting &lquote;performance’ expectations could result in decreased

quality elsewhere.”• “Goal-centered rather than patient-centered.”• “Narrow focus3 forget about big picture and just try to meet specific outcomes.”• “More documentation for those giving care so things more likely to be missed.”• “Overly concerned with process rather than outcomes.”• “Not all patients are compliant and despite our best efforts, a patient may not do well.”• “Medicine is very dependent on a team approach; therefore, if others fail to perform

their duties, it may affect those that are performing at their best level but are stilldependent on the outcomes of their team members.”

...................................................................................................................................................................................................................................................................................................................Imposes incomplete/unfair equation (24)† • “May not be looking at adequate factors to evaluate performance.”

• “Not all patients are compliant and despite our best efforts, a patient may not do well.”• “Medicine is very dependent on a team approach; therefore, if others fail to perform

their duties, it may affect those that are performing at their best level but are stilldependent on the outcomes of their team members.”

• “Doesn’t take into account patient contributions to not reaching therapeutic target.”• “Some things we can’t control/timeliness of imaging, outpatient behavior.”• “Assumes not reaching goals are determined by physician.”• “Somehow there is an assumption that the only thing affecting the patient outcome is

certain things that physician does.”• “Patient population differences affect physician pay.”• “Difficult/impossible to be fair.”• “Disparity in terms of specialist location.”

...................................................................................................................................................................................................................................................................................................................Simplifies variability and penalizes individualizedapproaches (21)

• “Big disadvantage3 does not take into account individual variation in responses totreatment.”

• “Health care professionals more interested in favorable outcomes on paper thanindividualizing patient care, e.g., patient with sprained shoulder from trauma getting anEKG in UCC.”

• “Difficult to implement simple mechanisms because of the variability of patientpopulations served.”

• “They don’t look at contraindications to certain therapeutic interventions, but penalize ifnot followed.”

• “Doesn’t take into account complexity of patients.”• “Makes physicians stick to guidelines but guidelines are for specific patients while in

reality each patient is an individual case and the guidelines will not apply to him. So,sticking to guidelines and pay for performance is wrong in some cases.”

• “Benchmarks that don’t take into account patient’s individual physiologies or mental/financial/cognitive concerns.”

...................................................................................................................................................................................................................................................................................................................Encourages selective recruiting and retention (14) • “May discourage care for sick/noncompliant patients.”

• “May cause doctors to avoid patients who have history of noncompliance.”• “Doctor may drop noncompliant/patient with multiple comorbidities from their

practice.”• “Physician selects for more compliant patient to care for.”• “Noncompliant patients/difficult patients would become an even greater burden3 no

one would want to take care of them.”• “Deters taking on care of complex, time-consuming patients with lower likelihood of

good outcomes.”• “Dissuades provider from taking on difficult patients.”• “‘Picking’ patients that are adherent.”• “May encourage institutions to refuse care to patients who do not comply or meet goals.”

...................................................................................................................................................................................................................................................................................................................Increases costs and resources (9) • “May be more costly (United Kingdom study).”

• “May not be cost-effective.”• “More expensive.”• “Not cost-effective.”• “More patient visits.”• “Increase resource utilization.”

...................................................................................................................................................................................................................................................................................................................Increases documentation and time/decreases efficiency (8) • “Need to do more paperwork.”

• “More documentation for those giving care so things more likely to be missed.”• “Decreased efficiency.”• “Time consuming.”• “Increased time constraints.”• “Need to meet a certain complexity of care which sometimes is time consuming.”

(Continues)

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 2010 5

to themes concerning P4P’s potentialimpacts on how physicians deliver care(e.g., facilitates vigilance/aggressive/closerfollow-up; helps ensure standardized,uniform level of care). The thirdsubcategory includes themes about thecare delivery process, such as the impacton pay/physician satisfaction, theprovider–patient relationship, and costand efficiency.

Disadvantages of P4P

Eighty-seven of the 97 trainees (90%)provided at least one disadvantage, yielding

a total of 143 units of information forcoding. Of these units, four were excludedbecause there was insufficient informationto understand the response well enough tocode it (“fines,” “researching what properquality of care is,” “less money tocaregivers,” and “different variablesmeasured”). The 10 trainees not includedin this analysis wrote either “not sure,”“don’t know,” “I have no idea,” “?,” orprovided no disadvantages.

Table 2 summarizes the 16 emergentthemes (139 units), ordered by frequency

of occurrence, and includesrepresentative examples of residents’responses. The three most frequentlyoccurring themes were that P4Pcompromises focus, care, and safety (25units), imposes incomplete/unfairequation (24 units), and simplifiesvariability and penalizes individualizedapproaches (21 units). Two of thesethemes (compromises focus, care, andsafety; and simplifies variability andpenalizes individual approaches) are theopposites of the most frequently citedadvantages (enables quality care and

Table 2(Continued)

Emergent themes (no. of units) Representative responses

Increases inappropriate use of treatments/resources (7) • “Increased burden on health care system by more medication prescribed, unnecessarymed use, increased side effects from medication”

• “Increase in medication use when not warranted.”• “Performing a service (not needed, i.e., not standard of care) in order to get paid more.”• “Sometimes even providing unnecessary treatments.”• “People may order every lab under the sun.”• “Also �encouraging� over aggressive treatment.”• “May be using unnecessary meds/resources and office visits for patients.”

...................................................................................................................................................................................................................................................................................................................Does not guarantee quality care or benefits to patient (6) • “Pay increase that may or may not benefit patient.”

• “Sets a defined end point, which is often not maximizing quality of care.”• “Process-based measures don’t always accurately reflect quality care.”• “Results of increased patient quality of life may not improve despite all measures taken.”• “Measures don’t necessarily result in better care.”• “Arbitrary measure.”

...................................................................................................................................................................................................................................................................................................................Fosters abuse/gaming (5) • “Abuse may occur.”

• “Gaming.”• “Perverse incentives and �?� strong incentive to game the system.”• “Abuse potential3 physicians will ‘figure out how to get paid’.”• “Trying to improve your numbers rather than treat patients as individuals.”

...................................................................................................................................................................................................................................................................................................................Promotes false/inaccurate documentation (5) • “Misuse documentation.”

• “Maybe ‘falsifying’ data in chart.”• “Misleading “reports3 reports good outcomes only3 may be factious reports as it has

happened in countries that heavily rely on reporting achievement.”• “Encourages number fudging.”• “Increased ‘chart fixing’.”

...................................................................................................................................................................................................................................................................................................................Sends wrong signals/motivation (4) • “Tacky.”

• “Money should not be the stimulus for improvement.”• “Financial incentive.”• “Does not provide any incentive for doctors to do more work.”

...................................................................................................................................................................................................................................................................................................................Diminishes art of medicine (3) • “Removing physician’s judgment.”

• “Loss of ‘art of medicine’.”• “Subtracts from the art of medicine.”

...................................................................................................................................................................................................................................................................................................................Increases stress (2) • “Stress on performer.”

• “More pressure on physicians.”...................................................................................................................................................................................................................................................................................................................Undermines trust and collegiality (2) • “Distrust.”

• “Friction between colleagues.”...................................................................................................................................................................................................................................................................................................................Decreases patient satisfaction and access (2) • “Patient feels that they are not getting the best care/attention.”

• “Patient being in trouble to afford their health care.”...................................................................................................................................................................................................................................................................................................................Encourages guideline nonadherence (1) • “It will incite MDs sometimes to override guidelines.”

* Themes are listed from the most frequent to least frequent. Ninety-seven residents at Louis Stokes Cleveland VAMedical Center responded to a mandatory written survey, which asked them, “What are the advantages anddisadvantages of pay for performance?” Units refer to individual responses; some residents provided severalresponses to each question. Arrows appeared in residents’ handwritten comments. �?� indicates a word ismissing that the authors were unable to decipher; EKG, electrocardiograph; UCC, urgent care center.

† The theme “imposes incomplete/unfair equation” refers to providers being held responsible for processes oroutcomes not entirely within their control.

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 20106

T2

motivates to improve/provide qualitycare).

Figure 2 illustrates the four categories ofthemes related to disadvantages. Therewas no overarching category fordisadvantages; rather, all categoriesseemed to be of relatively equalimportance. The first category, whichrelates to P4P’s potential impacts onpatient perceptions, includes the themethat such a system decreases patientsatisfaction and access. The secondcategory relates to potential impacts onclinical care by compromising care. Thiscategory includes two main themes, eachwith its own set of subthemes. Differentaspects of the first main theme, that P4Pfosters abuse/gaming, are reflected insubthemes such as promotesfalse/inaccurate documentation andencourages selective recruiting andretention. Further differentiation of thesecond main theme, that P4Pcompromises focus, care, and safety, isprovided by subthemes such as simplifiesvariability and penalizes individualizedapproaches, and increases inappropriateuse of treatments/resources.

The third category of emergent themesreflects P4P’s potential impacts onresources and efficiency. Finally, thefourth category includes themes relatedto P4P’s potential impacts on providerperceptions, with a focus onundermining morale (e.g., imposesincomplete/unfair equation; diminishesthe art of medicine). Given theconceptual relevance, the theme ofincreasing inappropriate use oftreatments/resources was included underboth the third and fourth categories.

Discussion

Although an increasing number ofarticles are being published regardingP4P’s potential advantages andunintended negativeconsequences,1,5,7,8,10 we believe our studyis the first to solicit data about theadvantages and disadvantages of such asystem from its future users, that is,physicians in training. Our IM traineesmentioned many of the advantages (e.g.,enables quality care, increasespay/satisfaction) and disadvantages (e.g.,fosters gaming strategies, decreases

patient satisfaction) expressed bypracticing physicians in previousresearch.3 Our data also raised additionalnuances related to P4P, such as its role inincreasing providers’ knowledge ofguidelines by making them more awareof problems and the need forimprovement. By obtaining andcategorizing trainees’ perceptions of P4P,we believe we can help medical educatorsdevelop more effective PBLI curriculathat will help trainees understand howP4P incentives and quality measures canhelp bring about improved patientoutcomes and assess quality of care, whilemaking trainees aware of the unintendedconsequences that have been previouslyreported. It is our hope that residents willultimately be able to view QI in thebroader context of P4P and qualitymeasure development.

Interestingly, unlike practicing physiciansin the existing literature, the residents inour sample did not explicitly identify P4Pas a direct contributor to a potentialdecline in the provider–patientrelationship.1 It is possible that traineesdid not identify this perceived

Disadvantages of P4P

Potential direct impacts on clinical care: Potential impacts on Potential impacts onP t ti l i t

Disadvantages of P4P

Potential direct impacts on clinical care:Compromising care

Potential impacts onresources and

efficiency

Potential impacts onprovider perceptions:

Undermining morale

Potential impacts onpatient

perceptions

Theme:

•Decreases patient

Theme:

• Fosters abuse/

Themes:

• Imposes incomplete/Themes:

• Increases costs and

Theme:

• Compromises•Decreases patientsatisfaction and access

Fosters abuse/gaming

(Sub)themes:

• Promotes false/inaccurate

Imposes incomplete/unfair equation

• Increases stress

• Sends wrong signals/motivation

• Increases costs andresources

• Increases documentationand time/decreases

Compromisesfocus, care, andsafety

(Sub)themes:

• Does not guaranteedocumentation

•Encourages guideline non-adherence

•Encourages

• Undermines trust and collegiality

• Diminishes artof medicine

decreasesefficiency

• Increasesinappropriate use of treatments/ resources

gquality care or benefits to patient

• Simplifies variability and penalizes individualized•Encourages

selectiverecruitingand retention

resourcesapproaches

• Increasesinappropriate use of treatments/resources

Figure 2 Categorization of emergent themes related to disadvantages associated with pay for performance as perceived by physician trainees atLouis Stokes Cleveland VA Medical Center, July 2008 to June 2009. Themes were mentioned by 87 of 97 internal medicine residents responding to amandatory written survey question (“What are the advantages and disadvantages of pay for performance?”). Italics are used to indicate a theme thatappears in more than one category.

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 2010 7

F2

disadvantage given the transient natureand inherent discontinuity of theirresidency clinics.15,16 However, traineesdid mention other considerations thatmay indirectly affect this relationship,such as encouraging selective recruitingand retention and holding physiciansaccountable for patient noncomplianceby imposing an incomplete or unfairequation.

It is worth noting that the most commonemergent themes related to theadvantages and disadvantages of P4Pseem to be in direct opposition: Residentsnoted that P4P enables quality care, butthey also indicated that it compromisesfocus, care, and safety. Similarly,residents’ concern that P4P increases theburden on the health care system byincreasing the use of arguablyunnecessary medications or proceduresmust be weighed against their insight thatP4P can be used to increase physicians’adherence to established guidelines.

An additional pair of opposing themesemerged: the disadvantage of penalizingproviders for adopting an individualapproach to care versus the advantage ofhelping ensure that a standard level ofcare is delivered to all patients inaccordance with guidelines. Someresidents commented that, under a P4Psystem, physicians lose the ability to carefor patients on an individual basis andfind it difficult to care for a medicallycomplex patient whose presentation maynot adhere to one care plan or whosemultiple disease processes lead toconflicting guideline recommendations.These comments directly contradictresidents’ comments citing P4P as ameans for ensuring that all patientsreceive at least a minimum standard ofcare according to published guidelines.

The opposing consequences of andtension inherent in a P4P system havebeen previously noted in the scenario ofthe provider who wants to “do the rightthing” for an individual patient yetrealizes this course of treatment may notalign with established P4P criteria andunderlying financial considerations.1 If aprovider feels a conflicting financialmotivation to practice according to suchrigid P4P criteria, ultimately the art ofmedicine is at risk of being compromised.This potential tension betweenmotivators was reflected in a significantnumber of residents’ comments.

Residents noted that P4P serves asadditional motivation for physicians toprovide quality care to their patients, andthey also relayed concerns that P4Pprovides financial incentives for betterdocumentation but not necessarily bettercare. Not surprisingly, some researchsuggests that the P4P approach to caredelivery will create incentives to “game”the system in ways that may cause truehealth care quality to decline even whenmeasured indicators might showimprovement1—another concern that isreflected in our data.

By summarizing the categories of theemergent themes, Figures 1 and 2 helpclarify both the positive and unintendednegative consequences of P4P systems.When these categories are consideredwith other emerging lists and groupingsof P4P issues,1,3 it is clear that a potentialmodel for monitoring progress andtrade-offs is evolving. The directopposition of the advantages anddisadvantages as perceived by ourtrainees suggests that these oppositionscan be viewed as forming a continuum tohelp us better understand the issues andhow to balance them.

Accordingly, important issues to considerare where the balance currently lies andhow best to train physicians to influencethis balance to maximize quality of care.Despite the increasing momentum forP4P implementation, few initiatives havebeen undertaken to introduce theconcept and its practical implications tophysicians in training. By providingresidents with an effective foundationthat emphasizes the importance of QIand measurement development, we hopeto establish a curriculum that will engagethem in actively developing a P4P systemthat swings the pendulum towardachieving the goals of QI—that is, asystem that will enable care rather thancompromise the focus of care. Asdiscussed in an earlier publication,17 forresidents to “buy in” to the concept ofQI, they must understand thebackground behind its development andthe implications of practicing in a systemwhere QI is not integrated into routinehealth care delivery. In a similar vein,residents need to understand thebackground behind the development ofP4P, the pitfalls and advantages noted inprevious trials of such a reimbursementplan, and how embracing theadvantageous end of the P4P continuum

can improve the quality of the health carethey deliver.

There are several limitations of this study.Our results may not be generalizable to alarger audience given that we used aconvenience sample of IM residentsrotating through their ambulatorycurriculum in one residency programwithin one site. It is possible that thesaturation we achieved only relates to thisparticular sample. The survey wasmandatory, however, so that we couldensure that we would obtain input fromthe full spectrum of residents. Moreover,given that the findings are consistent withand expand the existing literature, webelieve our sampling approach provideda solid foundation for determining thatthere is a need to include formaldiscussions about P4P in curricula. Ourfindings therefore provide a mechanismto help integrate and shape thosediscussions.

In addition, we purposely limited thesurvey to IM residents so that we couldbe sure of consistent faculty support forthe curriculum and that all residents inthe program would rotate through thecurriculum. Lack of faculty support inother services, time constraints, andscheduling issues remain significantbarriers to instituting a PBLI curriculumon a broader scale. However, thechallenges associated with practicing in aP4P environment are not unique to IM,and expansion of our curriculum to otherservices would be worthwhile.

Further, responses to the survey wereanonymous, which did not allow us toanalyze results by level of training.Residents at different levels of trainingmay have had more or less previousexposure to the concept of P4P. Had weaccounted for this in our analysis, wemay have found responses to be skewedbased on level of training. Inclusion ofsuch analysis would be elucidative infuture iterations of our curriculum.

Finally, there was no comparison group,but including such a group would nothave changed the structure of thefindings. Although a comparative analysismay be a potentially interesting next step,our interest with this study was toidentify resident perceptions to developour curriculum to engage trainees inmerging the concepts of P4P andimproved quality of care. Finally, we hadno way of clarifying the sources of

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 20108

residents’ perceptions. Linking and/oridentifying the various sourcesinfluencing trainee perceptions would bedifficult.

P4P is a complex, system-level factor thathas the potential to influence decisionsand practices in multiple ways, somemore favorable than others. Moreover, itis not likely to go away in the near future,and, as such, it is medical educators’responsibility to encourage opendiscussions about P4P. Next steps formedical education include formallyintegrating P4P discussions into thecurriculum. On the basis of ourpreliminary findings, we plan to expandthe scope of our current PBLI curriculumto include broader discussions related tothe development of performancemeasures and their unintendedconsequences. We plan to use ourcategorization of perceived P4Padvantages and disadvantages and theconcept of a continuum along which theylie to help motivate residents to exploreways in which they can shift the focus ofP4P to providing quality care. Next stepsfor research include comparing differentgroups’ perceptions of P4P (e.g., faculty,administrators, other types of health caretrainees), conducting interviews in whichparticipants are asked specifically aboutdefinitions and sources of their beliefsabout P4P, and evaluating the impact of acurriculum integrating P4P discussionson participants’ perceptions of P4P.

Conclusions

Using the knowledge gleaned from ourstudy, we anticipate building a PBLIcurriculum that enhances residents’understanding of QI and P4P, offersthem a framework that reveals how thetwo concepts are related, and providesthem with tools to identify and develop

their own QI projects. As residentsdevelop a project that ultimately leads toimproved outcomes, we hope that theywill begin to see the link betweenimproved quality of care and the way thatP4P measures are designed to achieve thisimprovement. We believe that exposingresidents to the advantages anddisadvantages of P4P early in theirtraining will foster their engagement inhelping to define solid quality measuresthat will ultimately allow them toestablish the important balance betweenachieving quality measures and practicingthe “art of medicine.”

Funding/Support: None.

Other disclosures: None.

Ethical approval: The local institutional reviewboard for human subjects reviewed the protocoland materials and determined that this study isexempt.

Disclaimer: The views expressed in this article arethose of the authors and do not necessarilyrepresent the views of the Department ofVeterans Affairs.

Previous presentations: Material previouslypresented at the April 2009 Society of GeneralInternal Medicine National Meeting, EducationalInnovations Poster Session, Miami Beach,Florida.

References1 Dolinar RO, Leininger SL. Pay for

performance means compliance-based care.J Am Med Dir Assoc. 2006;7:328 –333.

2 Stulberg J. The physician quality reportinginitiative—a gateway to pay for performance:What every health care professional shouldknow. Qual Manag Health Care. 2008;17:2–8.

3 Committee on Redesigning Health InsurancePerformance Measures, and PerformanceImprovement Programs, Board on HealthCare Services. The promise of pay forperformance. In: Rewarding ProviderPerformance: Aligning Incentives inMedicare. Washington, DC: NationalAcademies Press; 2007:32–59.

4 Glickman SW, Peterson ED. Innovativehealth reform models: Pay-for-performanceinitiatives. Am J Manag Care. 2009;15(10 suppl):S300 –S305.

5 Weber DO. The dark side of P4P. PhysicianExec. 2005;31:20 –25.

6 Petersen LA, Woodard LD, Urech T, Daw C,Sookanan S. Does pay-for-performanceimprove the quality of health care? AnnIntern Med. 2006;145:265–272.

7 Rosenthal MB, Frank RG, Li Z, Epstein AM.Early experience with pay-for-performance:From concept to practice. JAMA. 2005;294:1788 –1793.

8 Vesely R. An uneven paying field? Studiesindicate pay-for-performance can exacerbatedisparities in access. Mod Healthc. 2008;38:32–33.

9 McDonald R, Roland M. Pay for performancein primary care in England and California:Comparison of unintended consequences.Ann Fam Med. 2009;7:121–127.

10 Pines JM, Hollander JE, Lee H, Everett WW,Uscher-Pines L, Metlay JP. Emergencydepartment operational changes in responseto pay-for-performance and antibiotic timingin pneumonia. Acad Emerg Med. 2007;14:545–548.

11 Karve AM, Ou FS, Lytle BL, Peterson ED.Potential unintended financial consequencesof pay-for-performance on the quality of carefor minority patients. Am Heart J. 2008;155:571–576.

12 Strauss A, Corbin J. Basics of QualitativeResearch: Techniques and Procedures forDeveloping Grounded Theory. ThousandOaks, Calif: SAGE Publications; 1998.

13 Creswell J. Qualitative Inquiry and ResearchDesign: Choosing Among Five Traditions.Thousand Oaks, Calif: SAGE Publications;1998.

14 Glaser B, Strauss A. The Discovery ofGrounded Theory. Chicago, Ill: Aldine; 1967.

15 Feddock CA, Hoellein AR, Griffith CH, et al.Are continuity clinic patients less satisfiedwhen residents have a heavy inpatientworkload? Eval Health Prof. 2005;28:390 –399.

16 McBurney PG, Gustafson KK, Darden PM.Effect of 80-hour workweek on continuity ofcare. Clin Pediatr (Phila). 2008;47:803–808.

17 Watts B, Augustine SA, Lawrence RH.Teaching quality improvement in the midstof performance measurement pressures:Mixed messages? Qual Manag Health Care.2009;18:209 –216.

balt1/zvk-acm/zvk-acm/zvk01210/zvk4675-10z wasifk S�3 9/29/10 17:47 Art: ACM202256 Input-afmGraduate Medical Education

Academic Medicine, Vol. 85, No. 12 / December 2010 9

JOBNAME: AUTHOR QUERIES PAGE: 1 SESS: 1 OUTPUT: Tue Sep 28 15:19:47 2010/balt1/zvk�acm/zvk�acm/zvk01210/zvk4675�10z

AUTHOR QUERIES

AUTHOR PLEASE ANSWER ALL QUERIES 1