Industrial and Organizational Psychology ... · Additional services for Industrial and Organizational Psychology: ... Borman(1977)arguedthatundersomecircumstances,itmightbepos-

Industrial and Organizational PsychologyhttpjournalscambridgeorgIOP

Additional services for Industrial and OrganizationalPsychology

Email alerts Click hereSubscriptions Click hereCommercial reprints Click hereTerms of use Click here

Getting Rid of Performance Ratings Genius or Folly A Debate

Seymour Adler Michael Campion Alan Colquitt Amy Grubb Kevin Murphy Rob Ollander-Krane andElaine D Pulakos

Industrial and Organizational Psychology Volume 9 Issue 02 June 2016 pp 219 - 252DOI 101017iop2015106 Published online 04 July 2016

Link to this article httpjournalscambridgeorgabstract_S1754942615001066

How to cite this articleSeymour Adler Michael Campion Alan Colquitt Amy Grubb Kevin Murphy Rob Ollander-Krane andElaine D Pulakos (2016) Getting Rid of Performance Ratings Genius or Folly A Debate Industrialand Organizational Psychology 9 pp 219-252 doi101017iop2015106

Request Permissions Click here

Downloaded from httpjournalscambridgeorgIOP IP address 12821012919 on 12 Aug 2016

Industrial and Organizational Psychology 9(2) pp 219ndash252 June 2016Copyright copy 2016 Society for Industrial and Organizational Psychology doi101017iop2015106

Focal Article

Getting Rid of Performance Ratings Genius orFolly A Debate

Seymour AdlerAon Hewitt

Michael CampionPurdue University

Alan ColquittEli Lilly

Amy GrubbFBI

Kevin MurphyColorado State University

Rob Ollander-KraneGap Inc

Elaine D PulakosCEB

Despite years of research and practice dissatisfaction with performance appraisal isat an all-time high Organizations are contemplating changes to their performancemanagement systems themost controversial of which is whether to eliminate perfor-mance ratings The pros and cons of retaining performance ratingswere the subject ofa lively standing-room-only debate at the 2015 Society for Industrial and Organiza-tional Psychology conference in Philadelphia (Adler 2015) Given the high interestin this topic this article recaps the points made by the panelists who participatedin the debate The arguments for eliminating ratings include these (a) the disap-pointing interventions (b) the disagreement when multiple raters evaluate the sameperformance (c) the failure to develop adequate criteria for evaluating ratings (d)

Seymour Adler AonHewitt NewYork NewYorkMichael Campion Krannert Schoolof Management Purdue University Alan Colquitt Eli Lilly Indianapolis Indiana AmyGrubb FBI Washington DC Kevin Murphy College of Natural Sciences Colorado StateUniversity RobOllander-KraneGap Inc San Franciso California ElaineD Pulakos CEBArlington Virginia

This article grew out of a presentation at the 30th Annual Conference of the Societyfor Industrial andOrganizational Psychology in Philadelphia Pennsylvania (April 23 2015)titled ldquoGetting Rid of Performance Ratings Genius or Follyrdquo (S Adler Chair) Authorshiporder is alphabetical

Correspondence concerning this article should be addressed to Seymour Adler AonHewitt 199 Water Street New York NY 10038 E-mail seymour_adleraoncom

220 seymour adler et al

the weak relationship between the performance of ratees and the ratings they receive(e) the conflicting purposes of performance ratings in organizations (f) the inconsis-tent effects of performance feedback on subsequent performance and (g) the weakrelationship between performance rating research and practice in organizations Thearguments for retaining ratings include (a) the recognition that changing the ratingprocess is likely to have minimal effect on the performance management process asa whole (b) performance is always evaluated in some manner (c) ldquotoo hardrdquo is noexcuse for industrialndashorganizational (I-O) psychology (d) ratings and differentiatedevaluations have many merits for improving organizations (e) artificial tradeoffsare driving organizations to inappropriately abandon ratings (f) the alternativesto ratings may be worse and (g) the better questions are these How could perfor-mance ratings be improved and are we conducting the entire performance man-agement process properly The article closes with questions organizational membershave found useful for driving effective performance management reform

Background to the Debate (Pulakos)Despite years of research and practice aimed at improving the performanceappraisal and performance management process in organizations dissat-isfaction with the process is at an all-time high More than 90 of man-agers employees and human resource (HR) heads feel that their perfor-mance management processes fail to deliver the results they expected andmany view their current processes as ineffective andor inaccurate (Corpo-rate Leadership Council 2012) But even beyond this study after study hasshown that one important component of performance managementmdashthatis the performance reviewmdashis dreaded Formal performance reviews arenot only perceived to be of little value they also can be highly demotivat-ing to even the highest performing employees (Aguinis Joo amp Gottfredson2011 Culbertson Henning amp Payne 2013 Rock 2008) Performance ap-praisal and performance management are not synonymous but there is of-ten a strong relation between the two Although performance managementsystems often incorporate more frequent and informal feedback it is com-mon in many organizations to retain and to rely heavily on annual reviewsof performance Pulakos and OrsquoLeary (2011) have argued that the key rea-son traditional performance management approaches have failed to live upto their promise of enabling performance is that the mechanics of formalsystemsmdashhow ratings are done the documentation required how goals aresetmdashare inconsistent with the goal of providing frequent credible and use-ful feedback about performance The changing nature of work these days(eg semiautonomous teams remote work freelancing temporary worketc) also suggests it is time to rethink our fundamental assumptions aboutapproaches to performance management

One of the main goals of performance appraisal is and always hasbeen simply to achieve high performance by enabling managers to guide

getting rid of performance ratings 221

employees to increasingly higher levels of productivity and by motivatingemployees to do their very best We have somehow managed to create justthe opposite however Most employees donrsquot find performance appraisalsto be valuable or motivating Instead they find performance appraisals andperformance management systems frustrating too bureaucratic and oftennot relevant to their jobs Managers spend a lot of time on formal perfor-mance management activities that they likewise believe add little value TheCorporate Leadership Council found that managers and employees spendabout 210 and 40 hours respectively per year on performance appraisal andperformance management activities such as formal goal-setting processesmidyear and year-end reviews and often extensive rating and calibrationprocesses For a 10000-person organization with average salaries this timeplus the other costs associated with performance management like HR timetraining costs and software can cost over $35 million dollars annually Thisis a very big investment only to disengage employees and potentially under-mine performance

Within the past 2 years a number of companies have begun contem-plating changes to their performance management systems aimed at gain-ing more value and impact on performance In fact we cannot remembera time when so many companies have been uniformly focused on makingsimilar disruptive changes to a major talent systemmdashall at the same timewhich is a testament to the ineffectiveness of and dissatisfaction with cur-rent performance management processes However this has also generatedconsiderable debate about how performancemanagement can and should bechanged to gain the most impact on performance Many organizations havebegun changing aspects of their formal performance management systemsand introducing training and change management to drive more effectiveperformance management behavior (eg real-time feedback more agile ex-pectations and goals more collaboration etc) Where companies are strug-gling more is in deciding what aspects of their formal systems to changemdashwith the most controversial and challenging decision surrounding whetherto eliminate or retain formal (usually annual) performance ratings Severalhigh profile companies have opted to eliminate annual performance ratingsentirely such as Eli Lilly Adobe and Gap Inc but most companies are tak-ing smaller steps such as introducing more goal-setting and performancecheck-ins to drive more regular expectation setting and feedback

The focus on performance ratings in particular is noteworthy espe-cially considering the main point in recent thought pieces on performancemanagement reform (eg Pulakos Mueller-Hanson Arad amp Moye 2015Pulakos amp OrsquoLeary 2011) which is that over 25 years of research haveshown that formal system changes have no discernable impact on drivingmore effective performance management behavior or performance (DeNisi

amp Smith 2014) This may be because formal performance management pro-cesses that incorporate or rely on annual performance ratings drive bothmanagers and employees to focus on rating outcomes rather than on howto actually improve and drive effectiveness on the job Based on the researchliterature and our collective experience engaging in performance manage-ment reform over decades it is not realistic to think that any intermittentactivity-driven evaluation systemwill necessarily compel more effective per-formance management behavior or ultimately improved job performanceThe pros and cons of retaining performance ratings were the subject of alively standing-room-only debate at the 2015 Society for Industrial and Or-ganizational Psychology conference in Philadelphia Given the high interestin this topic the remainder of this article will recap the points made by thepanelists who participated in the debate and advanced a position to keep orto eliminate ratings The participants in the debate included both academicsand practitioners well-known authors in performance management highlyexperienced consultants and people implementing performance manage-ment systems in large organizations in both the private and the public sectorThey were allocated to each side of the debate to balance the range of per-spectives They were selected for their knowledge and interest in the topicnot because they inherently support one side of the debate or the otherThe side for getting rid of performance ratings included Alan Colquitt ofEli Lilly Kevin Murphy of Colorado State University and Rob Ollander-Krane of Gap Inc The side for retaining performance ratings included Sey-mourAdler ofAonHewittMichael Campion of PurdueUniversity andAmyGrubb of the FBI The debate was moderated by Elaine Pulakos of CEB whoset the background for the debate and provided closing remarks

Get Rid of Performance Ratings (Colquitt Murphy amp Ollander-Krane)There is a long history of research on performance rating and performanceappraisal (for reviews see Bernardin amp Beatty 1984 DeCotiis amp Petit 1978DeNisi 2006 DeNisi Cafferty amp Meglino 1984 Ilgen amp Feldman 1983Landy amp Farr 1983 Milkovich ampWigdor 1991 Murphy amp Cleveland 19911995Wherry ampBartlett 1982) and although different reviews highlight dif-ferent strengths and weakness of the methods that are used in organizationsto measure job performance via rating scales or other similar measures itis fair to say that none of these reviews leads to the conclusion that perfor-mance rating is particularly successful either as a tool for accurately mea-suring employee performance or as a component of a broader program ofperformancemanagement Nearly a century of research on performance ap-praisal (see for example Austin amp Villanova 1992) suggests that there is alongstanding history of problems with performance rating and little reasonto believe that these problems will be solved in the foreseeable future The

conclusion that performance rating is not working is not solely an academicone there is evidence of widespread dissatisfaction with performance ratingand related techniques in organizational settings many large organizations(eg Accenture Deloitte Microsoft Gap Inc Eli Lilly) have abandoned orsubstantially curtailed their use of performance appraisal (Culbert amp Rout2010 Cunningham 2014)

Most organizations continue to use performance appraisals and rat-ings and there is little sign this is changing at least in the short term (seeLawler Benson amp McDermott 2012) Their use affects the distribution ofrewards and other outcomes for millions of employees in organizations Thevast majority (89) of companies link compensation decisions to perfor-mance ratings (Mercer 2013) Performance ratings are the basis for pay-for-performance systems in most organizations most organizations ldquopay forperformance ratingsrdquo It is fair to say that tens if not hundreds of billions ofdollars in compensation and rewards are riding on the backs of performanceratings These ratings can have long and lasting effects on employeesrsquo livesand careers in organizations affecting staffing promotion and terminationdecisions as well as affecting access to other development opportunities Thisis serious business and ratings donrsquot measure up

The task of accurately evaluating someonersquos performance is difficult ifnot impossible It requires a supervisor to observe the performance of an-other employee over the course of a year and to collect reports of othersrsquoobservations of the same employee The supervisor usually with little or noformal training then sifts sorts analyzes weighs and aggregates this infor-mation to make a judgment about the employeersquos performance Supervisorsmust also exclude fromconsideration other irrelevant information about thisindividual and any other judgments that may have been made about the in-dividual in the past and they must suspend any biases or tendencies theypossess while making this judgment Sounds easy enough right Judges inprofessional sports for whom this is their lifersquos work have a far simpler task(eg count the number of punches each boxer lands) and yet they frequentlydisagree with each other (see Stallings amp Gillmore 1972) The point of thisis what we are asking managers to do is virtually impossible to do well espe-cially with the frailties of human beings as measurement instruments

Performance Appraisal A Failed ExperimentIn this section we will argue that it is time to treat performance rating inorganizations as a failed experiment Despite decades of work on meth-ods of improving performance ratings and hundreds of research articlesthere is little evidence of meaningful improvement in performance ratingsIt is always possible that some future intervention might lead to substan-tial improvements but after nearly a century of dashed hopes and unmet

expectations we believe it is time to call a halt to this sorry business Scien-tists rarely like to give up on a problem they have struggled with for so longbut if the ship is sinking and it is also on fire it is probably time to head forthe lifeboats

A comprehensive review of all of the shortcomings of performance rat-ings and of the failed efforts to improve ratings is beyond the scope of thisarticle but the broad outline of the problems with performance rating andour lack of success in resolving these problems can be summarized underseven headings (a) the disappointing interventions (b) the widespread dis-agreement when multiple raters evaluate the same performance (c) the fail-ure to develop adequate criteria for evaluating ratings (d) the weak relation-ship between the performance of ratees and the ratings they receive (e) theconflicting purposes of performance rating in organizations (f) the incon-sistent effects of performance feedback on subsequent performance and (g)the weak relationship between performance rating research and practice inorganizations

Disappointing InterventionsThe most common suggestion for improving performance rating in organi-zations has been to improve rating scales In particular substantial effortshave been devoted over several decades to making rating scales clearer andmaking ratings more reliable and valid by adding valid and detailed behav-ioral information to rating scales (eg behaviorally anchored rating scalesSmithampKendall 1963 behavior observation scales LathamampWexley 1977Murphy Martin amp Garcia 1982) The payoff for this effort was judged tobe so meager by Landy and Farr (1980) both of whom had themselves pub-lished numerous articles on the development of behavior-based rating scalesthat they called for a moratorium on further rating scale research Subse-quent reviews (eg Murphy amp Cleveland 1995) suggested that behavior-based rating scales might have advantages in terms of their face validity andacceptance by users but there is little evidence that the substantial body ofresearch on the development and use of these scales made performance rat-ings in organizations more reliable valid or useful

The secondmost common intervention has been rater training This re-search has followed two main themes First numerous studies have reliedon the strategy of telling raters about so-called rating errors (eg leniencyrange restriction halo) and urging them not to commit these error Secondnumerous studies have developed variations of frame of reference trainingdesigned to ensure that raters adopt a common frame of reference regardingtarget performance dimensions and performance levels (see for exampleBernardin amp Buckley 1981 Bernardin amp Pierce 1980 Day amp Sulsky 1995Pulakos 1984) On the whole both types of training ldquoworkrdquo in some sense

If you tell raters not to give ratings that are too high or too low they are a bitmore likely to follow your advice if you give raters a clearer idea of what theperformance dimensions mean and what different performance levels looklike they will show a bit more agreement in their evaluations However nei-ther variation on rater training has been successful in markedly improvingratings in organizations (Murphy amp Cleveland 1991 1995)

Murphy and Cleveland (1995) noted that the interventions that weremost commonly used to improve performance ratings were based on thehypothesis that raters lacked the knowledge or the tools to accurately eval-uate performance They suggested that these interventions were probablybased on a faulty diagnosis of the shortcomings of performance appraisal inorganizations As we note in a later section a substantial body of researchsuggests that there was little real evidence that raters lack the ability to rateaccurately and that the likely causes of the failure of performance appraisalare related to ratersrsquo motivation and goals rather than to the ability of ratersto perform this task

Disagreement Among RatersOne way to improve ratings might be to collect evaluations of performancefrom multiple raters rather than relying on the judgment of any single raterUnfortunately there is consistent evidence that raters do not agree in theirevaluations of ratees Disagreement is probablymore substantial when ratersdiffer in terms of their relationshipwith the persons to be rated (eg supervi-sors vs peers) but even when raters are at the same level in an organizationthey often do not agree in their evaluations (Facteau amp Craig 2001 Har-ris amp Schaubroeck 1988 Heneman 1974 Murphy Cleveland amp Mohler2001 Viswesvaran Ones amp Schmidt 1996) There have been intense dis-agreements about precisely how disagreements among raters should be in-terpreted and about the more general implications of these disagreementsfor the psychometric quality of ratings (eg Murphy ampDeShon 2000 OnesViswesvaran amp Schmidt 2008) but there is little controversy about the factthat raters do not show the level of agreement onemight expect from for ex-ample two different forms of the same paper-and-pencil test Viswesvaran etal (1996) suggest that performance ratings obtained from two separate setsof raters in organizations are likely to show correlations in the 50s hardlythe level one would expect if ratings were in fact good measures of the per-formance of ratees

In theory it is possible to obtain ratings from multiple raters and poolthem to eliminate some types of interrater disagreement However in prac-tice this is often difficult Rating systems that obtain information from mul-tiple raters (eg 360 degree feedback programs) often sample raters fromdifferent levels of the organization or raters from outside of the organization

(eg customers) and there are good reasons to believe that differences inroles and perspectives will introduce systematic disagreements among ratersthatmight not be readily resolved by simply pooling ratings (Murphy Cleve-land amp Mohler 2001)

Weak Criteria for Evaluating RatingsSuppose N raters each evaluate the performance of k subordinates How doyou know whether the ratings they provide do a good job measuring theperformance of the people being rated If multiple raters evaluate the sameratees interrater agreement measures provide one criterion and as notedabove the data regarding interrater agreement are far from encouraging Themore common approach has been to evaluate performance ratings in termsof the presence or absence of one or more so-called ldquorater errorsrdquo

Seventy-five years ago Bingham (1939) identified what would come tobe called ldquohalo errorrdquo in ratings Other rater errors including leniency orseverity and range restriction were soon identified There is a substantialresearch literature that analyzes rater error measures particularly measuresof halo (Balzer amp Sulsky 1992 Cooper 1981a 1981b Murphy 1982 Mur-phy amp Anhalt 1992 Murphy amp Balzer 1989 Murphy Jako amp Anhalt 1993Nathan amp Tippins 1989 Pulakos Schmitt amp Ostroff 1986) On the wholerater error measures are deeply flawed criteria Different measures of haloleniency and the like do not necessarily agree More important all of thesemeasures are based on completely arbitrary assumptions about how perfor-mance is distributed Suppose for example that I give ratings of 4 to 5 whileyou give ratings of 2 or 3 This could mean that I am lenient that you aresevere or that the people who work for me are performing better than thepeople who work for you are and in principle there might be no way to de-termine which explanation is correct Suppose that I rate my subordinateson four performance dimensions (eg planning written communicationattention to detail and receptiveness to customer feedback) and that the av-erage intercorrelation among the ratings I give is 50 Is this too high or is ittoo low How do you know how high these correlations should be MaybeI am committing halo error or maybe the people who are good at planningreally are also good at oral communication Outside of artificial laboratoryenvironments there is usually no a priori way of determining whether thecorrelations among ratings is too high too low or just right

Perhaps the most important critique of halo error measures as criteria isthe paucity of evidence that ratings that do not exhibit what we judge to behalo leniency or the like are better measures of job performance than rat-ings that are suffused with ldquorater errorsrdquo Earlier we noted that one popularform or training instructed raters not to make halo errors not to be lenientand so forth It is not clear that removing halo leniency and the like makes

performance ratings better measures of performance and it is even plausi-ble that efforts to suppress these so-called errors make performance ratingsworse (Murphy 1982 Murphy Jako amp Anhalt 1993)

Borman (1977) argued that under some circumstances it might be pos-sible to assess the accuracy of performance ratings The prototypical ratingaccuracy studywould collect ratings under standardized conditions inwhichall raters viewed the same performance (eg using videotaped vignettes) andin which the pooled judgments of multiple experts could be used as a crite-rion for evaluating accuracy There is a vibrant literature dealing with themeasurement and interpretation of rating accuracy (eg Murphy amp Balzer1989 Sulskyamp Balzer 1988) and in general this literature leads to three con-clusions (a) Rater error measures are virtually useless as indices of ratingaccuracy (b) there are multiple ways of defining accuracy in rating and thedifferent accuracy indices are often essentially unrelated to one another and(c) outside of tightly controlled experimental settings it is prohibitively dif-ficult to measure accuracy directly in the field

In sum there are two classes of measures we can use in field settings toevaluate ratings rater agreement measures and rater error measures Rateragreement measures tell a sorry story but it is one whose full implicationsare not fully understood (MurphyampDeShon 2000 Viswesvaran et al 1996)whereas rater error measures have no clear value as criteria

Contextual Effects on RatingsAlthough the most widely cited conclusion of Landy and Farrrsquos (1980) re-view of performance appraisal research was that rating scales probably donot have a large effect on the quality of rating data perhaps amore importantconclusion was that there are a number of variables other than job perfor-mance that have a clear influence on performance ratings (see also Murphy2008 Scullen Mount amp Judge 2003) For example a number of authors(eg Grey amp Kipnis 1976 Murphy amp Cleveland 1991 1995) have suggestedthat contextual factors ranging from broad societal factors (eg the state ofthe economy and the labor market) to the organizational context in whichratings are collected (eg the climate and culture of the organization) arelikely to affect performance ratings Other authors (eg Aguinis amp Pierce2008 Ferris Munyon Basik amp Buckley 2008 Judge amp Ferris 1993) havenoted the importance of the social context of rating such as social powerinfluence leadership trust social exchange group dynamics negotiationcommunication and other issues

One of the nonperformance determinants of performance ratings thathas beenmost extensively studied can best be described as the political use ofrating (Cleveland ampMurphy 1992 Longenecker Sims amp Gioia 1987 Mur-phy amp Cleveland 1995 Tziner amp Murphy 1999 Tziner Prince amp Murphy

1997) Research on the political aspects of performance appraisal suggeststhat raters pursue a variety of goals when completing performance appraisalsand that these goals substantially influence the ratings they give (MurphyCleveland Skattebo amp Kinney 2004) Murphy and Cleveland (1995) sug-gest that these goals include (a) task performance goalsmdashusing performanceratings to influence the subsequent performance of ratees (b) interpersonalgoalsmdashusing appraisal to maintain or improve interpersonal relations be-tween the supervisor and the subordinates (c) strategic goalsmdashusing ap-praisal to increase the supervisorrsquos andor the workgrouprsquos standing in theorganization and (d) internalized goalsmdashgoals that are the results of ratersrsquobeliefs about how he or she should evaluate performance (eg some raterswant to convey the image of being a hard case with high performance stan-dards like the professor who announces at the beginning of a class thathe or she rarely gives ldquoArdquo grades) On the whole the goals raters pursuetend to push those raters in the direction of the ldquoLake Woebegone Effectrdquo(ie giving high ratings to just about all employees) High ratings will max-imize rewards (thereby potentially maximizing motivation) will preservegood relationships between supervisors and subordinates and will makethe supervisor look good (a supervisor whose subordinates are perform-ing poorly will seem ineffective) In most organizations there are few con-crete rewards for giving accurate ratings and few sanctions for giving inflatedratings

In our experienceworkingwith performance appraisal in real-world set-tings a number of reasons are often cited by supervisors or managers forsystematically manipulating the ratings they give These are summarized inTable 1

Conflicting PurposesRatings are used for many purposes in organizations (eg to determine pro-motions and salary increases to evaluate developmental needs to serve ascriteria for validating tests and assessments to provide documentation for le-gal purposes) and these purposes often come into conflict (Cleveland Mur-phy amp Williams 1989 Murphy amp Cleveland 1995) Fifty years ago MeyerKay and French (1965) noted the incompatibility of using performance ap-praisal simultaneously to make decisions about rewards and to provide use-ful feedback and suggested that the different uses of performance appraisalshould be separated by creating different evaluative mechanisms for rewardsversus feedback Murphy and Cleveland (1995) suggested that uses of per-formance appraisal to highlight differences between people (eg salary pro-motion validation) are fundamentally at odds with uses of appraisal to high-light differences within persons (eg identifying developmental strengthsand weaknesses)

Table 1 Commonly Cited Reasons To Manipulate Performance Ratings

Overratebull Get more money for employeesbull Encourage employeesbull Avoid painful outcomes for himherself by not rating someone low (eg the workrequired by a disciplinary process)bull Avoid hurting employees now or in the future by giving them low ratingsbull Avoid confrontation with an employee by giving him or her a rating lower than he orshe expectbull Get rid of a bad employee so he or she is marketable elsewhere within the companybullMake themselves look good Low ratings might imply the supervisor is not doing agood jobbull Get employees access to opportunities that are dependent on higher performanceratings (eg development opportunities)

UnderratebullMake an example out of an employee A supervisor may rate employees lower thanthey deserve to punish them or teach them a lessonbull Get rid of an employee Supervisors may rate someone lower than he or she deservessimply to create the necessary documentation and evidence to fire someone theydonrsquot likebull Send a message A supervisor may rate an employee lower simply to send themessage that the employee should leave

Cleveland et alrsquos (1989) survey suggested that most organizations usedperformance appraisals for multiple conflicting purposes and that this at-tempt to satisfy incompatible goals with performance appraisal was onesource for the continuing evidence of dissatisfaction with performance ap-praisal in organizations Some goals might be compatible For example useof performance appraisal for salary administration is not necessarily incom-patible with using performance appraisal as a tool for validating selectioninstruments (both uses focus on differences between ratees) However evenwhen there is no underlying conflict among uses the particular dynamics ofone use of appraisals (eg for determining raises) might limit the value ofthe appraisal process for other uses (eg the range restriction that is likelywhen most ratees receive high ratings might limit the value of appraisals ascriteria for validating selection tests)

Feedback Is Not Accepted or Acted OnOne of the core assumptions of virtually all performance management pro-grams is that if people receive feedback about their performance they willbe both motivated and empowered to improve (assuming that appropriateorganizational supports and incentives are in place to support this improve-ment) This assumption does not hold up well at least in the context of

performance appraisal There is evidence that employees dislike giving orreceiving performance feedback (ClevelandMurphyampLim 2007) The per-formance feedback they receive is often inconsistent and unreliable (Murphyet al 2001) Even if feedback is reasonably accurate the effects of feedbackare inconsistent sometimes improving subsequent performance sometimesmaking things worse and sometimes having little discernible effect (Klugeramp DeNisi 1996)

Accurate feedback requires accurate evaluations of performance andthere is little good reason to believe that performance appraisals in organi-zational settings provide this sort of accuracy However even if performancefeedback were accurate there is little reason to believe that it would be con-sistently accepted and acted on It is well known that peoplesrsquo self-ratings ofperformance are consistently higher than ratings from supervisors or subor-dinates (Harris amp Schaubroeck 1988 Heneman 1974)

Differences in self-ratings versus othersrsquo ratings are not a shortcomingof performance appraisal per se but rather a reflection of broadly relevantprocesses in the way we understand our own behavior versus the behaviorof others For example there is a fundamental attribution error (Ross 1977)that makes us likely to attribute our own successes to internal factors (egskill effort) and our own failures to external ones (eg luck lack of oppor-tunity) but to make the opposite attributions when others succeed or fail Inother words we are likely to take credit when we succeed and to avoid blamewhen we fail but to see othersrsquo success and failure in starkly different termsmaking it likely that we will view our own performance favorably even whenwe have in fact performed poorly As a result feedback that is unbiased andaccurate will sometimes seem harsh and unfair

Performance appraisal and performance management systems are of-ten built on assumptions that may not be warranted The first assumptionis that employees want to be rated Managers assume employees want toldquoknowwhere they standrdquo and they assume this means employees want someform of rating or relative comparison This makes sense People comparethemselves with others This is the basis for social comparison equity andgaming theory (see Adams 1965 Bryson amp Read 2009 Festinger 1954)However this doesnrsquot mean that people want to be compared with others bythird parties It is also true that people generally think they are above average(Meyer 1980) so most feel they will benefit from these ratings and com-parison Although employees may say they want to be rated their actionssuggest otherwise Barankay (2011) showed that a majority (75) of peoplesay they want to know how they are rated (or ranked in this case) but whengiven the opportunity to choose between a job where their performance wasrated against others and one where it was not most choose the work withoutratings

The second assumption is that performance ratings motivateemployeesmdashthose getting poor ratings are motivated to improve and thosegetting top ratings aremotivated to keep them This assumption comes fromlabor economics and tournament theory (see Connelly Tihanyi Crook ampGangloff 2014) Research suggests this assumption does not always hold upAlthough some research finds ratings improve motivation and performance(see Blanes i Vidal amp Nossol 2009) other research suggests they canhurt both motivation and performance especially in collaborative workenvironments and when ratings are tied to differential rewards (see egCasas-Arce ampMartinez-Jerez 2009) On balance ratings seem to hurt morethan they help

Performance feedback represents the ultimate losendashlose scenario It isextremely difficult to do well and if it was done well the recipients would belikely to dismiss their feedback as inaccurate and unfair It is no wonder thatsupervisors and subordinates alike approach performance appraisal with amix of skepticism and unease

Weak ResearchndashPractice RelationshipsDecades of research and hundreds of articles chapters and books deal-ing with performance appraisal seem to have little impact on the way per-formance appraisal is done or the way information from a performanceappraisal is used in organizations (Banks amp Murphy 1985 Ilgen Barnes-Farrell ampMcKellin 1993) Banks andMurphy (1985) suggest that this is duein part to a lack of fit between the questions that are of greatest interest toresearchers and the problems practitioners need to solve For example a sig-nificant portion of the performance appraisal studies published in the 1980sand 1990s were concerned with cognitive processes in appraisal (eg atten-tion allocation recognition and recall of performance-related informationDeNisi 2006) For psychologists these are very important and interestingquestions but they are not questions that are necessarily relevant to the prac-tical application of performance appraisal in organizations It is worth notingthat the lack of a solid researchndashpractice relationship is not solely the resultof academic disengagement from the realities of the workplace Murphy andCleveland (1995) presented a list of practical applications and research-basedsuggestions for improving performance appraisal and there is little evidencethat these were ever taken up and applied by practitioners

One potential explanation for the failure of performance appraisal re-search to address the problems of performance appraisal is that these prob-lems may be more intractable than we want to admit Several researchers(eg Banks amp Murphy 1985 Murphy amp Cleveland 1995) have suggestedthat the shortcomings of performance appraisal are not an indication thatraters cannot effectively evaluate job performance but rather an indication

that raters and more generally organizations do not want to solve the prob-lem For example organizations do not appear to do anything to rewardsupervisors who do a good job with performance appraisal or to sanctionthose who do a bad job Accuracy in performance appraisal is likely to makesubordinates unhappy anddemotivated and it has the potential tomake sub-ordinates look bad Indeed if a managerrsquos job is to help get the most out ofhis or her subordinates a case can bemade that performance appraisal (evenat its best) interferes with that job and that a smart manager will do all he orshe can to subvert and ignore the appraisal system (Murphy amp Cleveland1995)

Institutional theory provides some insight into the difficulty of chang-ing performance appraisal and performance management practices in or-ganizations by means of academic research Performance management andrating practices get embedded and become hard to change These practicesget institutionalized in organizations and companies copy and adopt thembecause they are ldquothe standardrdquo Companies also imitate and copy them inorder to achieve legitimacy (Pfeffer 2007 Scott 1995) Social relationshipscreated among organizational members (especially senior leaders who aremembers of each otherrsquos boards) also contribute to this conformity and im-itation behavior General Electricrsquos (and Jack Welchrsquos) adoption of forcedranking systems in the 1990s is a good example

Additional Negative Consequences of RatingsFinally there are several additional potential drawbacks of ratings that werenot mentioned above First ratings sometimes bring us unpleasant newsabout diversity differences (seemeta-analysis of race differences byMcKayampMcDaniel 2006) Second they are the focus of many lawsuits on topics suchas promotion compensation and terminations Third they are often nothelpful when you really need them such as in terminations and downsizingwhen you find they have not sufficiently documented poor performance orhave too much skew to be useful for decision making

Get Rid of Ratings but Tread CarefullyMost companies stay with ratings because they donrsquot know what elseto do and because ratings are inputs to so many other processes espe-cially rewards How will organizations distribute rewards and make pro-motion and termination decisions without performance ratings Rest as-sured these are all solvable problems These decisions can all be madewithout performance ratings and will probably be better without themHowever it is also important to realize there are also some significantphilosophical questions wrapped up in a decision to abandon perfor-mance ratings that are beyond the scope of this article Most organizations

have pay-for-performance philosophies and see themselves asmeritocraciesIn most companies performance ratings are the primary way of determin-ing merit Well over 90 of companies tie pay to performance to at leastsome extent and 82 link individual performance to compensation (Mer-cer 2013) How will organizations determine merit without ratings Howdo organizations design performance-based rewards systems without indi-vidual performance ratings The answer is they donrsquotmdashat least not individ-ually based reward systems Companies should not abandon performanceratings without wrestling with these issues There are effective solutions tothese problems as well but we will save these arguments for another day(or article)

Why Getting Rid of Performance Ratings Is a Bad Idea (Adler Campion ampGrubb)Performance evaluation is a complex difficult and often unpleasant part ofmanagement but is essential in some form if an organization wants to be ameritocracy In this section we argue that eliminating performance ratingsis probably not the solution for meaningfully addressing the widespread dis-satisfaction with performance management We start by clarifying the pointof the debate Then we argue that performance is always rated in someman-ner and being difficult to do is not an excuseWe then summarize themeritsof ratings (no pun intended) and the value of differentiated evaluations Wesuggest that artificial tradeoffs are driving organizations to abandon ratingsWe then describe the alternatives to performance ratings most of which areeven less pleasant Finally we provide some suggestions for the improvementrather than the abandonment of performance ratings

Performance Ratings Performance ManagementLetrsquos start with a clarifying observation on the terms of this debate becausesome of the issues described above have ranged outside the scope Whenaddressing the question of using or discarding performance ratings withinthe larger context of performance management and all that ails it it maystand out as odd (and possibly unproductive see Landy amp Farr 1980) tohave the conversation focused on just a single element of performance man-agement the use of performance ratings There are a myriad of complex andinterwoven issues that need untangling and restringing when it comes tothe broader and more significant task of effectively managing an employeersquosperformance (eg Banks ampMurphy 1985 Curtis Harvey amp Ravden 2005)But instead our field seems to have focused on perhaps the least meaningfulelement of performance management but one that is most comfortable forI-O psychologistsmdashmeasurement also known as the ratings A prime illus-tration of this disproportionate focus is the extensive summary of the 75-year

history of empirical literature on ratings presented above by the ldquoantiratingsrdquoside

To be sure HR professionals leaders and I-O psychologists all widelyagree that howperformancemanagementmdashincluding the assignment of per-formance ratingsmdashis practiced within the business environment is badly inneed of improvement However we also need to recognize that fundamen-tally the goal of an organization is to perform well and that this goal neces-sarily requires the individuals within the organization to perform well Theevaluation of individual performance then becomes an indispensable com-ponent of measuring and achieving organizational success To us this logichowever painful is inescapable

Letrsquos be clear then about the center of the debate This is not aboutwhether there is value in core aspects of performance management (eggoals regular constructive feedback development focus etc) We all agreewith that The issue is whether we should continue to attempt to collect apsychometrically meaningful quantitative performance index of some sort

Performance Is Always EvaluatedWhether called ratings standards or judgments leaders and employeesmake decisions and evaluations about the performance skills attributes andassets of those around them all the time Judgments that employees ldquodonrsquothave itrdquo ldquohave itrdquo or ldquoknock it out of the parkrdquo are made every day in or-ganizations Dozens of conceptualizations of what can and should be mea-sured within an individual performance system have been put forth in theliterature and in practice (five- four- three- two-point rating scales compe-tency based production based objective based task based behavior basedetc eg Benson Buckley amp Hall 1988 Bommer Johnson Rich Podsakoffamp MacKenzie 1995 Borman 1979 Catano Darr amp Campbell 2007 Fayamp Latham 1982 Goffin Jelley Powell amp Johnston 2009 Jelley amp Goffin2001 Johnson 2001 Latham Fay amp Saari 1979 Thompson amp Thompson1985 Tziner amp Kopelman 2002) Irrespective of the manner in which evalu-ations happen judgments aremade and decisions about rewards and roles inthe future are based on those judgments For corporate success these judg-ments should be grounded in something tangible and documented and ide-ally something that furthers the performance of the organization as a wholeThe organization needs to have a common language common perceptionsof what is important and a starting point of where the population is in dif-ferent skills achievements and attributes in order to manage individual andcollective performance forward If there is no starting point metric how canan organization gauge its progress These judgments standards and evalu-ations are all essentially what we mean by the term ldquoratingsrdquo whether or notthey are labeled as such

For those organizations that promote a ldquodevelopment culturerdquo and feelthey need to move away from using ratings to do so we remind them thatjudgments of where people are in their professional growth are still beingmade Development conversations start with a picture of where employeesare now where they are going and the actions needed to close that deltaThe ldquowhere they are nowrdquo is a judgment on some evaluation scale Althoughassigning or being assigned a ldquoratingrdquomay be uncomfortable for some (RockDavis amp Jones 2014) the science of behavioral change based on control the-ory also tells us that some discomfort some feedback on the gap betweenthe ldquonowrdquo state and the desired future state is required to stimulate behaviorchange and development (Carver Sutton amp Scheier 2000) Indeed in or-ganizations with a genuine development culture detailed evaluative ratingsfeedback is not the threatening beast as characterized in recent popular pressIn organizations with a genuine developmental culture employeesmdashandespecially high-potential employees and particularly Millennialsmdashmanagetheir own development and proactively seek out feedback on areas of growthand improvement Those areas can only be identified through the measure-ment of performance evaluations against established standards a processotherwise known as rating

Reflecting the ubiquity and inescapability of evaluative judgmentsof performance we should recognize that manymdashperhaps mostmdashorganizations that have trumpeted their ldquodaringrdquo elimination of ratingshave actually done no such thing Many continue to have managers en-ter a number into the performance management system to represent arecommended percentage increase in salary or a recommended percent-of-target-bonus award These entries are intended to quantitatively reflect prioryear performancemdashwe would call that activity ldquoratingrdquo Other organizationsclaiming to have eliminated ratings continue to use some sort of traditionalrating scale but have only eliminated imposing a forced distribution dueto employee reactions (Blume Baldwin amp Rubin 2009 Chattopadhayayamp Ghosh 2012 Schleicher Bull amp Green 2008) Others have simplifiedtheir rating process typically reducing the number of dimensions rated(eg fewer individual goals rated a one-item global rating of the ldquohowrdquoof performance in lieu of rating a long list of individual competencies)Still others have simply changed the wording of the dimensions ratedFor example some organizations have substituted the term ldquooverall im-pactrdquo for ldquooverall performancerdquo One more creative example is Deloittersquosrecent adoption of a three-item scale (a fourth item measures promo-tion potential) instead of more traditional performance rating formats(Buckingham amp Goodall 2015) A sample item is ldquoGiven what I knowof this personrsquos performance I would always want him or her on myteamrdquo Many of these organizations have made laudable improvements in

aspects of performance managementmdashbut they surely did not ldquoget rid ofratingsrdquo

ldquoToo Hardrdquo Is No Excuse for I-O PsychologyMany in the field seem to have turned the conversation about performancemanagement into a lament that ldquoperformance is too hard tomeasurerdquo whichtherefore leads to the suggestion that we abandon the science and practiceof defining and measuring what ldquoperforming wellrdquo means To this we saydifferential psychology is and always has been a coremdasharguably the coremdashdiscipline of our field (Chamorro-Premuzic 2011) Many (most) I-O psy-chologists spend their careers attempting to quantify how people differ inattributes as varied as knowledge skill engagement safety orientation nar-cissism andmdashyesmdashjob performance We have a well-developed set of psy-chometric principles and tools that allow us to evaluate and improve the re-liability and validity of our measures

Admittedly job performance is a complex construct (CampbellMcCloyOppler amp Sager 1993) Its definition varies across organizational contextsand roles There are almost as many ways that job performance has beenmeasured as there are jobs (Aguinis 2013) When assessed the psychome-tric characteristics of measures of job performance range widely with mostmeasures showing weak-to-moderate reliability and construct-oriented va-lidity this is well documented in the prior section of this article But a historyof measurement challenges is not an excuse for abandoning the differentialpsychology of performance for the ldquoarmchairrdquo assignment of idiosyncraticvague qualitative descriptors of performance Giving up on the effort to im-prove the differential measurement of performance should simply not be anoption for either I-O scientists or practitioners Note also that for many rolesacross a large segment of the workforce including for those in governmentand other organizations there is actually no real option of abandoning rat-ings whatever their flaws ratings are mandated by law or regulation (egcivil servicemerit-principle requirements) or traditional practice (eg thinkteacher evaluations) It is therefore critical that our field continues to exploresupportable ways to rate individual performance

What Are the Merits of RatingsAre there anymerits to having a psychometric index of job performance Al-though not perfect there is evidence supporting the reliability and validityof performance ratings For example meta-analytic research shows at leastmodest reliability for performance ratings (Rothstein 1990 Viswesvaran etal 1996) if ratings can be pooled across many similarly situated raters itshould be possible to obtain quite reliable assessments Despite the currentdebate there is probably general agreement among I-O psychologists that

performance ratings if done well share at least some meaningful true scorevariancewith actual job performance If collected for research purposes per-formance ratings have been shown to be a useful criterion in thousands ofvalidation studies of selection procedures in the history of our profession(Schmidt amp Hunter 1998) and are recommended in our testing principles(Society for Industrial and Organizational Psychology 2003)

Applied psychometrics is what I-O psychologists are good at and itbrings great value in other domains As examples consider how well it helpsus measure the knowledge and skills of candidates in personnel selectionor measure the attitudes and beliefs of employees in engagement surveysWould we really be better off not to at least attempt to generate a psychome-tric index to add to the performance management system Would we reallybe better off with just unreliable qualitative data

Performance rating that depends on the subjective judgments of one ormore raters is not necessarily the preferredmethod of evaluating job perfor-mance it is the only feasible method for attacking this problem Objectivemeasures (eg absenteeism production counts) are almost always deficientcriteria (Landy amp Farr 1980) Frequent informal feedback something that isoften suggested by proponents of performance management (eg Aguinis2013) would be difficult to validate in any systematic way Establishing thejob relatedness of informal feedback would be difficult and establishing thefairness and freedom from bias of an informal evaluation system would beeven more daunting Like it or not performance rating by supervisors or bysome other source that is knowledgeable about the job (eg peers) remainsour best bet for obtaining useful measures of performance

The Value of Differentiated EvaluationsResearch has consistently shown that differentiating rewards and recogni-tion based on quantified performance differences actually contributes to en-hanced individual and organizational performance Four closely related linesof research have consistently shown the following

Strong performers are differentially attracted to organizations that rec-ognize individual contributions (eg Menefee amp Murphy 2004) For in-stance Trank Rynes and Bretz (2002) demonstrated that high achievershave a stronger preference for organizations that apply individual ratherthan group pay systems emphasize praise and recognition for individualaccomplishment and employ fast-track promotion practices for top per-formers Not surprisingly strong performers consistently choose organi-zations that have a reputation for pay-for-performance systems over fixedsalary compensation systems (eg Cadsby Song amp Tapon 2007)

Strong performers will leave organizations that do not differentially rewardstrong performance (eg Allen amp Griffeth 2001) As an example TrevorGerhart and Boudreau (1997) showed that low salary growth is related tohigh turnover among top performers relative to low performers

Weak performers will leave organizations that strongly reward high per-formance (eg Lazear 1986) Corresponding to the finding regarding lowsalary growth Trevor Gerhart and Boudreau (1997) showed that highsalary growth is related to higher turnover among low performers but tolow turnover among top performers

Weak performers are more likely to stay with an employer when pay-for-performance differentiation is weak (eg Harrison Virick amp William1996) For exampleWilliams and Livingstone (1994) showed that the rela-tionship between performance and turnover is significantly weaker whenrewards are not linked to performance than when they are linked to per-formance

Recent CEBCorporate Leadership Council (2014) research based on a sam-ple of over 10000 showed that Millennials are particular motivated by rela-tive rather than absolute performance feedback comparedwith the broaderemployee population This research shows that in contrast to the myth thatcharacterizes Millennials as more collaborative they are actually more com-petitive and have a strong desire for performance evaluation systems thatprovide clear differentiation from peers One conclusion then is that theimportance of performance differentiation may indeed grow as Millennialsbecome the core segment of the working population

Artificial Tradeoffs Are Driving Organizations To Abandon RatingsIf the benefits ofmeasuring differences in performance and rewarding differ-entially based on those performance differences are so clear why have somesmall number of organizations abandoned the attempt to measure individ-ual differences in performance We believe this recent trend (if there is one)stems at least in part from a set of falsely dichotomized choices

Quantification versus humanization There is a belief that attaching a rat-ing to the quality of an individualrsquos job performance somehow dehuman-izes that individual reducing him or her to a number That indeed is a riskHow often have we heard employees referred to as ldquohersquos a 2rdquo or ldquoshersquos a 4rdquoinstead of the accurate and appropriate ldquoher performance last year was a 4it exceeded the established goals and expectationsrdquo Quantification can bedehumanizing but does not have to be Managers need to learn and applythe language of effective performance management indeed the impact offeedback on performance improvement is strongest when that feedback isfocused on performance and on partnering with the employee on the means

of enhancing performance rather than on labeling performance as an inher-ent quality of the performer The fact that rating labels are misused is nota reason to avoid ratings There is a large research literature in our field onhow to improve reactions to performance evaluation (eg Bobko amp Colella1994 Cawley Keeping amp Levy 1998 Dipboye amp Pontbriand 1981 Dorf-man Stephan amp Loveland 1986 Giles ampMossholder 1990 Ilgen PetersonMartin amp Boeschen 1981 Pichler 2012 Silverman ampWexley 1984)

Bad performance management versus no performance ratings In manymdashperhaps even mostmdashorganizations performance management practices arenot well-implemented as both sides of this debate concede As argued bythe antiratings side above appraisal forms are often overengineered in anattempt to satisfy multiple organizational objectives (eg rewards develop-mental planning succession legal and regulatory compliance etc) Man-agers are neither selected nor adequately trained to manage performancein many instances (Murphy 2008) Rating guidelines are often confusingpoorly understood and inconsistently applied across raters and withinraters across ratees Organizations at times require ratings to fit distributioncurves which may help reduce leniency (Scullen Bergey amp Aiman-Smith2005) but may not represent the true distribution of performance (AguinisampOrsquoBoyle 2014 but cf Beck Beatty amp Sackett 2014) All these flaws can beand in many organizations are being addressed (see the example of Cargillin Pulakos et al 2015) There is no reason to add to bad performance man-agement practice by eliminating a core area of managerial accountability

Ratings versus conversations The value of active ongoing constructivefeedback performance strategy and coaching conversations between man-agers and employees has been long recognized (Meyer et al 1965) No onedisputes that However there is no reason to believe that eliminating sys-tematic contextually bounded and documented performance evaluationswill result in an increase in the frequency or quality of those conversationsWhere is the evidence that the only reason managers do not conduct morefrequent more engaging and more impactful conversations is that they aretoo busy filling out appraisal forms Opponents of ratings cite a recent KornFerry study that indicated across companies the competency consistentlyrated lowest (67 of 67 on the Lominger list) is growing talent (Rock et al2014) It would seem to be naiumlve to think that relieved of the burden ofratings and without the ldquocrutchrdquo of a structured feedback tool managerswill somehow overcome this weakness and consistently engage in positiveand impactful conversations Indeed one importantmechanism for assuringthat quality ongoing performance conversations occurmdashand acknowledgingMurphyrsquos (2008) argument that it is more a matter of manager motivationthan skillmdashis by setting goals as well as evaluating and rewarding managersfor how effectively they manage the performance of their own subordinates

Feedback versus motivation Feedbackmdashif provided destructivelymdashcanindeed be a demotivator (Rock 2008) Poorly communicated feedback isan aspect of poor leader performance Effective feedback is motivating Re-search has shown that feedback can stimulate the development of more ef-fective performance strategies Brockner (1988) has shown that people differin behavioral plasticity and that although some do have difficulty internaliz-ing feedback others demonstrate meaningful behavioral change when pro-videdwith feedback Further opponents of ratings are fond of citingDweckrsquos(2006) mindset work in support of eliminating ratings However that re-search actually shows that it is not performance feedback per se that is theissue it is the interpretation of that feedback that is key Thosewith a learningmindset benefit from performance feedback it is just that they use that feed-back constructively to improve their skills and ultimately their performanceEffective performance management requires the constructive communica-tion of specific actionable feedback not the avoidance of feedback

In other words we are arguing that organizations can have it allmdashif theyapply our science and effective performance management practice Man-agers can quantify individual differences in performance and treat their em-ployees as partners in performance management They can design adminis-tratively simple but impactful performance management practices and trainmanagers to apply those practices to drive better individual and collectiveperformance They can learn to validly assess performance and based onthose assessments engage employees in frequent constructive dialogues inways that are empowering motivating and inspiring to improve perfor-mance and employee engagement

What Are the Alternatives to Performance RatingsLet us consider the consequences of living without ratings Consider the po-tential answers to questions such as the following

Will performance measurement be more precise without a rating Thisseems unlikely because the onlymeasurementwill be narrative data whichare inherently less reliable

Will merit-based management be improved With no measure of meritother than narrative comments it is hard to imagine how merit-basedmanagement like pay increases and promotions would be improved

Will employees be more motivated to work hard Antagonists of ratingsargue that eliminating ratings will indeed be more motivating but that isyet an untested hypothesis It is certainly true that most employees prefernot to be rated (Cleveland et al 2007) but achieving higher ratings (andavoiding low ones) is very motivational

Will lawsuits be easier to defend Without any measure of performanceit would likely be more difficult to defend HR decisions based on per-formance However the existence of diversity differences in performanceratings is sometimes used in an attempt to establish a claim of discrimi-nation In addition if the organization does not use the performance rat-ings consistently and fairly to determine HR decisions this might createmore of a liability than a defense So the answer to this question may beuncertain

Howwould wemake compensation decisions Across-the-board increasesserve little value in motivating higher performance Step increases basedon time in grade are similarly limited Standard of living increases are theleast desirable way to allocate pay increases because they imply that in-creases are somehow required of the organization to keep pace with thecost of living and not related to individual performance Organizations canonly give increases when there is a change of job or responsibilities Thisis not uncommon especially in small organizations but it not as useful forimproving performance because of the limited availability of higher leveljobs Finally but by no means the last possibility management could usesubjective judgment based on the narrative-only performance appraisalThis would maintain the appearance of a link to performance but wouldlikely be difficult to justify to employees getting smaller pay increases

Howwouldwemake promotiondecisionsWe could rely on selection toolsand not past performance However past performance in the same com-pany in similar jobs is often considered to be an excellent predictor of per-formance after promotion (Beehr Taber amp Walsh 1980 DeVaro 2006)and employees have a strong expectation that past performance shouldbe a factor considered in promotion (De Pater Van Vianen Bechtoldt ampKlehe 2009)We could just use seniority likemost union contracts requireThis may be viewed as fair by some employees especially in environmentswith low trust in management but it does not ensure that the highest per-formers or the most skilled are promoted We could rely on successionmanagement systems which refer to management planning discussions ofpromotion potential but the results of these are usually not shared withemployees and thus do not serve the purposes of motivating performanceor ensuring fairness perceptions We could use promotion panels to readthe narrative appraisals and recommend promotions This is common insome settings especially in the public sector and has many advantagessuch as considering a wide range of inputs multiple decision makers anda consistent process but this is very costly in terms of management timeFinally to complete this nonexhaustive list we could qualitatively considerpast performance based on the narrative-only appraisals Again this wouldmaintain the appearance of a link to performance but would likely be

difficult to justify to the employees who are not promotedwithout anymet-ric of performance

The Path ForwardPerhaps the better question than whether or not to get rid of ratings is thisHow could performance ratings be improved Many choices of rating scalesand expected distributions of ratings might help depending on the contextand culture A recent trend with considerable merit is the formal use of ldquocal-ibrationrdquo which refers to meetings among managers for the purpose of jus-tifying and comparing the proposed performance ratings of employees toensure their accuracy and comparability across business units (Catano et al2007 Church 1995 DeNisi amp Pritchard 2006) Another increasing trendis competency modeling which involves (in part) the explicit establishmentof behavioral descriptions of performance expectations at each rating levelfor important job competencies (M A Campion et al 2011 Catano et al2007 Halim 2011) Yet another trend worth considering is the use of 360feedback as an input to performance management (Bracken amp Rose 2011M C Campion Campion amp Campion 2015 Gilliland amp Langdon 1998Latham Almost Mann amp Moore 2005 Reilly amp McGourty 1998) Mul-tirater 360 feedback in the form of either formal ratings or informal inputmay be especially relevant as work becomesmore team based less hierarchi-cal andmore customer focused as other changes result in additional partieshaving performance information on an employee and aswork arrangementschange (eg remote work) such that the manager has less of an opportunityto observe the work of the employee It has many potential advantages suchas increased reliability reduced bias and reduced leniency (eg AguinisGottfredson amp Joo 2013 Antonioni 1994 Flint 1999 London ampWohlers1991 Smither London amp Reilly 2005) Simplifying the cognitive demandsand reducing through simplification of the process and forms the motiva-tional barriers for managers is also likely to be helpful (Efron amp Ort 2010)Finally performance review panels might be used which would have simi-lar advantages and disadvantages as promotion panels (Catano et al 2007Church 1995 Gilliland amp Langdon 1998 Kozlowski Chao amp Morrison1998 Vance Winne amp Wright 1983 Werner amp Bolino 1997)

In our view it is the consequences of leaders assigning and communi-cating evaluative judgments expressed as ratings that matter the most notthe ratings themselves Leaders are the front line of performance manage-ment The stronger the promotion process to select leaders who value de-velopment and the stronger the formal and informal support for leaders tohelp thembetter develop their people the stronger the performancemanage-ment will be within the organization The more employees are rewardedmdashformally and informallymdashfor evaluating performance diligently and (on the

employee side) for actively seeking and incorporating feedback the moreperformance management will strongly contribute to organizational perfor-mance The potential impact of I-O psychology on performance manage-ment practice is not limited to improved measurement We have the capac-ity for enhancing the full lifecycle of performance-relevant behaviors andinforming organizations how to build their performance culture The fullgamut of interconnected human capital activities can reinforce that cultureleadership development employee development selection and promotionand rewards and compensation Therefore letrsquos stop engaging in the self-deceptive thinking that getting rid of ratings is the key to improving perfor-mance management

Moreover before we toss out ratings as the villain we should ask our-selves the broader and much more important question Are we conduct-ing the entire performance management process properly Our perspec-tive is that the ratings issue is just representative of deeper problemsmdasha performance management process that is poorly designed and imple-mented Our profession has accumulated extensive advice on how to de-sign and implement performance management (eg Posthuma amp Campion2008 Posthuma Campion amp Campion 2015) often summarized as ldquobestpracticesrdquo which if followed correctly may well moot the discussion aboutratings

A DilemmaIn the end the narrow ratings issue may be a dilemmamdasha choice betweenundesirable alternatives However that is not new to I-O psychologists Infact helping make these hard choices is what we do Consider the fact thatturned-down candidates are not happy andwould prefer to do awaywith ourtests and selection systems Nonpromoted employees would prefer to get ridof our promotion procedures Employees receiving smaller pay increases willdisagreewithwhatever systemwas used So shouldwe abandonperformanceratings because managers or employees do not like them and because theyare difficult to do well Will we be better off

ClosingIf managers and employees engaged in effective day-to-day performancemanagement behavior as needed in real time there should be less ifany need for formal performance management systems including for-mal performance ratings Some managers in fact do this and realizehigh team and individual performance as a result in spite of the formalperformance management system However not all or even most man-agers regularly engage in effective performance management behavior es-pecially given that the formal system often gets in the way and drives

ineffective performance management behavior and reactions Althoughorganizations can abandon ratings and even their formal performancemanagement systems entirely many are not ready to consider such extremesteps Furthermore many organizations feel the need to maintain evalua-tions of record for legal and other purposes The merits of these positionshave been argued above but what remains is a practically more importantquestion How can organizations make the right decisions about perfor-mance management reform including the question of ratings to best mit-igate negative impact on effective performance management behavior andperformance

The concept of performance management is squarely aimed at helpingindividuals and organizationsmaximize their productivity through enablingemployees to perform to their potential To achieve this performance needsto be managed with three critical goals in mind

Enable employees to align their efforts to the organizationrsquos goals Provide guideposts to monitor behavior and results and make real-timeadjustments to maximize performance

Help employees remove barriers to success

Each aspect of the performance management process should be de-signed to efficiently and directly impact one ormore of these goals Many or-ganizations seeking to improve their performance management approacheswant to start with questions such as ldquoshould we have ratingsrdquo or ldquoshould weuse a forced distribution and if so what should the cutoff percentage be forthe lowest ratingrdquo These are the wrong questions to ask at the start The bet-ter questions to start with are these What are the critical outcomes we wantto achieve and how can we best ensure employees deliver against key goalsand outcomes Framed from this perspective there is no right answer to theratings question It really is ldquoIt dependsrdquo based on the organizationrsquos goalsstrategies maturity trust openness to change management philosophy andother contextual factors Taking a broader and more strategic approach ver-sus a narrow view focused simply on ratings will help keep the focus on whatthe performance management system needs to attain holistically This willrequire answering key questions that matter most in deciding on the rightperformance management strategy for each situation We close with thosequestions organizational members have found most useful for driving effec-tive performance management reform

What business problem(s) are we trying to solve Drive organizational strategy Reduce unnecessary process and costs Increase engagement and performance

Use formal evaluations to justify differential decisionsbull What do we want to evaluate and reward Individual or team performance or both Behaviors outcomes both or otherbull How do we view our employees and howmuch do we need to compete fortalentbull What proportion of an employeersquos total compensation do we put at riskbull How much do we really rely on ratings for decision making How much do our ratings actually align with our decisions Do we need a number to make decisions Why or why notbull Howmature and ready is our organization to remove formal performancemanagement steps and processbull How effective are we at the daily behaviors that drive performancebull Are the answers the same for all units jobs and work in the organization

ReferencesAdams J S (1965) Inequity in social exchange In L Berkowitz (Ed) Advances in experi-

mental social psychology (Vol 2 pp 267ndash299) New York NY Academic PressAdler S (2015 April) Getting rid of performance ratings Genius or folly In S Adler

(Chair) Symposium presented at the 30th Annual Conference of the Society for In-dustrial and Organizational Psychology Philadelphia PA

Aguinis H (2013) Performance management (3rd ed) Upper Saddle River NJ Pear-sonPrentice Hall

Aguinis H Gottfredson R K amp Joo H (2013) Avoiding a ldquomerdquo versus ldquowerdquo dilemmaUsing performance management to turn teams into a source of competitive ad-vantage Business Horizons 56 503ndash512 Retrieved from httpmypageiueduhaguinispubshtml

Aguinis H Joo H ampGottfredson R K (2011)Whywe hate performancemanagementmdashand why we should love it Business Horizons 54 503ndash507 Retrieved fromhttpmypageiueduhaguinispubshtml

Aguinis H amp OrsquoBoyle E Jr (2014) Star performers in twenty-first century organizationsPersonnel Psychology 67 313ndash350

Aguinis H amp Pierce C A (2008) Enhancing the relevance of organizational behavior byembracing performancemanagement research Journal of Organizational Behavior 29139ndash145

Allen D J amp Griffeth R W (2001) Test of a mediated performancendashturnover relation-ship highlighting the moderating roles of visibility and reward contingency Journal ofApplied Psychology 86 1014ndash1021

Antonioni D (1994) The effects of feedback accountability on upward appraisal ratingsPersonnel Psychology 47 349ndash356

Austin J T amp Villanova P (1992) The criterion problem 1917ndash1922 Journal of AppliedPsychology 77 836ndash874

Balzer W K amp Sulsky L M (1992) Halo and performance appraisal research A criticalexamination Journal of Applied Psychology 77 975ndash985

Banks C amp Murphy K (1985) Toward narrowing the researchndashpractice gap in perfor-mance appraisal Personnel Psychology 38 335ndash345

Barankay I (2011) Rankings and social tournaments Evidence from a crowd-sourcing ex-periment (Working paper) Philadelphia PA University of Pennsylvania

Beck J W Beatty A S amp Sackett P R (2014) On the distribution of job performance Therole of measurement characteristics in observed departures from normality PersonnelPsychology 67 531ndash566

Beehr T A Taber T D amp Walsh J T (1980) Perceived mobility channels Criteria forintraorganizational job mobility Organizational Behavior and Human Performance26 250ndash264

Benson P G Buckley R M amp Hall S (1988) The impact of rating scale format on rateraccuracy An appraisal of themixed standard scale Journal ofManagement 14(3) 415ndash423

Bernardin H J amp Beatty R W (1984) Performance appraisal Assessing human behaviorat work Boston MA Kent

Bernardin H J amp Buckley M R (1981) Strategies in rater training Academy of Manage-ment Review 6 205ndash212

Bernardin H J amp Pierce E C (1980) Effect of rater training Creating new response setsand decreasing accuracy Journal of Applied Psychology 65 60ndash66

Bingham W V (1939) Halo invalid and valid Journal of Applied Psychology 23 221ndash228

Blanes i Vidal J amp Nossol M (2009) Tournaments without prizes Evidence from personnelrecords Paper presented at the CEP Labour Market Workshop Centre for EconomicPerformance London School of Economics and Political Science London UnitedKingdom

Blume B D Baldwin T T amp Rubin R S (2009) Reactions to different types of forceddistribution performance appraisal systems Journal of Business Psychology 24 77ndash91

Bobko P amp Colella A (1994) Employee reactions to performance standards A review andresearch propositions Personnel Psychology 47 1ndash29

Bommer W H Johnson J L Rich G A Podsakoff P M ampMacKenzie S B (1995) Onthe interchangeability of objective and subjective measures of employee performanceA meta-analysis Personnel Psychology 48 587ndash605

Borman W C (1977) Consistency of rating accuracy and rating errors in the judgment ofhuman performance Organizational Behavior and Human Performance 20 238ndash252

BormanW C (1979) Format and training effects on rating accuracy and rater errors Jour-nal of Applied Psychology 64(4) 410ndash421

Bracken D W amp Rose D S (2011) When does 360-degree feedback create behaviorchange And how would we know it when it does Journal of Business amp Psychology26 183ndash192

Brockner J (1988) Self-esteem at work New York NY Lexington BooksBryson B amp Read J L (2009) Total engagement Using games and virtual worlds to change

the way people work and businesses compete Boston MA Harvard Business PressBuckingham M amp Goodall A (2015 April) Reinventing performance manage-

ment Harvard Business Review Retrieved from httpshbrorg201504reinventing-performance-management

Cadsby C B Song F amp Tapon F (2007) Sorting and incentive effects of pay for perfor-mance An experimental investigation Academy of Management Journal 50 387ndash405

Campbell J P McCloy R A Oppler S H amp Sager C E (1993) A theory of performanceIn N Schmitt ampW C Borman (Eds) Personnel selection in organizations (pp 35ndash70)San Francisco CA Jossey-Bass

Campion M A Fink A A Ruggeberg B J Carr L Phillips G M amp Odman R B(2011) Doing competencies well Best practices in competency modeling PersonnelPsychology 64 225ndash262

Campion M C Campion E D amp Campion M A (2015) Improvements in performancemanagement through the use of 360 feedback Industrial and Organizational Psychol-ogy Perspectives on Science and Practice 8 85ndash93

Carver C Sutton S amp Scheier M (2000) Action emotion and personality Emergingconceptual integration Personality and Social Psychology Bulletin 26 741ndash751

Casas-Arce P amp Martinez-Jerez F A (2009) Relative performance compensation con-tests and dynamic incentivesManagement Science 55 1306ndash1320

Catano V M DarrW ampCampbell C A (2007) Performance appraisal of behavior-basedcompetencies A reliable and valid procedure Personnel Psychology 60 201ndash230

Cawley B D Keeping L M amp Levy P E (1998) Participation in the performance ap-praisal process and employee reactions A meta-analytic review of field investigationsJournal of Applied Psychology 83(4) 615ndash633

Chamorro-Premuzic T (2011) Personality and individual differences London UKBlackwell

Chattopadhayay R amp Ghosh A K (2012) Performance appraisal based on a forced distri-bution system Its drawbacks and remedies International Journal of Productivity 61(8)881ndash896

Church A H (1995) From both sides now performance appraisals Political tools or effec-tive measures Industrial and Organizational Psychologist 33 57ndash65

Cleveland J N ampMurphy K R (1992) Analyzing performance appraisal as goal-directedbehavior In G Ferris amp K Rowland (Eds) Research in personnel and human resourcesmanagement (Vol 10 pp 121ndash185) Greenwich CT JAI Press

Cleveland J N Murphy K R amp Lim A (2007) Feedback phobia Why employees donot want to give or receive it In J Langan-Fox C Cooper amp R Klimoski (Eds) Re-search companion to the dysfunctionalworkplaceManagement challenges and symptoms(pp 168ndash186) Cheltenham UK Edward Elgar

Cleveland J N Murphy K R amp Williams R (1989) Multiple uses of performance ap-praisal Prevalence and correlates Journal of Applied Psychology 74 130ndash135

Connelly B L Tihanyi L Crook T R amp Gangloff K A (2014) Tournament theoryThirty years of contests and competitions Journal of Management 40 16ndash47

CooperW H (1981a) Conceptual similarity as a source of illusory halo in job performanceratings Journal of Applied Psychology 66 302ndash307

Cooper W H (1981b) Ubiquitous halo Psychology Bulletin 90 218ndash244Corporate Leadership Council (2012) Driving breakthrough performance in the new work

environment (Catalog No CLC4570512SYN) Washington DC CEBCorporate Leadership Council (2014) The millennial myth Three strategies for effectively

managing millennials in the workforce Washington DC CEBCulbert S A amp Rout L (2010) Get rid of the performance review How companies can

stop intimidating start managingmdashand focus on what really matters New York NYBusiness Plus

Culbertson S S Henning J B amp Payne S C (2013) Performance appraisal satisfactionThe role of feedback and goal orientation Journal of Personnel Psychology 12 189ndash195

Cunningham L (2014) In big move Accenture will get rid of annual performance reviewsand ranking The Washington Post Retrieved from httpwwwwashingtonpostcom

blogson-leadershipwp20150721in-big-move-accenture-will-get-rid-of-annual-performance-reviews-and-rankingstid=pm_pop_b

Curtis A B Harvey R D amp Ravden D (2005) Sources of political distortions in perfor-mance appraisals Appraisal purpose and rater accountability Group amp OrganizationManagement 30(1) 42ndash60

Day D V amp Sulsky L M (1995) Effects of frame-of-reference training and informationconfiguration on memory organization and rating accuracy Journal of Applied Psy-chology 80 158ndash167

DeCotiis T amp Petit A (1978) The performance appraisal process A model and sometestable propositions Academy of Management Review 3 635ndash646

DeNisi A S (2006) A cognitive approach to performance appraisal New York NY Rout-ledge

DeNisi A S Cafferty T amp Meglino B (1984) A cognitive view of the performance ap-praisal process A model and research propositions Organizational Behavior and Hu-man Performance 33 360ndash396

DeNisi A S amp Pritchard R D (2006) Performance appraisal performance managementand improving individual performance A motivational frameworkManagement andOrganization Review 2(2) 253ndash277

DeNisi A amp Smith C E (2014) Performance appraisal performance management andfirm-level performance Academy of Management Annals 8 127ndash179

De Pater I E Van Vianen A E Bechtoldt M N amp Klehe U C (2009) Employeesrsquo chal-lenging job experiences and supervisorsrsquo evaluations of promotability Personnel Psy-chology 62(2) 297ndash325

DeVaro J (2006) Strategic promotion tournaments and worker performance StrategicManagement Journal 27 721ndash740

Dipboye R L amp Pontbriand R (1981) Correlates of employee reactions to performanceappraisals and appraisal systems Journal of Applied Psychology 66(2) 248ndash251

Dorfman P W Stephan W G amp Loveland J (1986) Performance appraisal behaviorsSupervisor perceptions and subordinate reactions Personnel Psychology 39 579ndash597

Dweck C (2006)Mindset New York NY Random HouseEfron M amp Ort M (2010) One-page talent management Cambridge MA Harvard Busi-

ness PressFacteau J D amp Craig S B (2001) Are performance appraisal ratings from different rating

sources comparable Journal of Applied Psychology 86 215ndash227Fay C H amp Latham G P (1982) Effects of training and rating scales on rating errors

Personnel Psychology 35 105ndash116Ferris G R Munyon T P Basik K amp Buckley M R (2008) The performance evaluation

context Social emotional cognitive political and relationship components HumanResource Management Review 18 146ndash163

Festinger L (1954) A theory of social comparison processesHuman Relations 7 117ndash140Flint D H (1999) The role of organizational justice inmulti-source performance appraisal

Theory-based applications and directions for research Human Resource ManagementReview 9(1) 1ndash20

GilesW F ampMossholder K W (1990) Employee reactions to contextual and session com-ponents of performance appraisal Journal of Applied Psychology 75(4) 371ndash377

Gilliland S W amp Langdon J C (1998) Creating performance management systems thatpromote perceptions of fairness In J W Smither (Ed) Performance appraisal State ofthe art in practice (pp 209ndash243) San Francisco CA Jossey-Bass

Goffin R D Jelley B R Powell D M amp Johnston N G (2009) Taking advantage of so-cial comparisons in performance appraisal The relative percentile method HumanResource Management 48(2) 251ndash268

Grey R J amp Kipnis D (1976) Untangling the performance appraisal dilemma The in-fluence of perceived organizational context on evaluative processes Journal of AppliedPsychology 61 329ndash335

HalimA H (2011) Validation on behavioral competencies of a performance appraisal toolJournal of Psychological Research 6(2) 281ndash290

Harris M M amp Schaubroeck J (1988) A meta-analysis of self-supervisory self-peer andpeer-subordinate ratings Personnel Psychology 41 43ndash62

Harrison D A Virick M amp William S (1996) Working without a net Time perfor-mance and turnover under maximally contingent rewards Journal of Applied Psychol-ogy 81 331ndash345

Heneman H G (1974) Comparisons of self- and superior ratings of managerial perfor-mance Journal of Applied Psychology 59 638ndash642

Ilgen D R Barnes-Farrell J L amp McKellin D B (1993) Performance appraisal processresearch in the 1980s What has it contributed to appraisals in use Organizational Be-havior and Human Decision Processes 54 321ndash368

Ilgen D R amp Feldman J M (1983) Performance appraisal A process focus In L Cum-mings amp B Staw (Eds) Research in organizational behavior (Vol 5 pp 141ndash197)Greenwich CT JAI Press

Ilgen D R Peterson R B Martin B A amp Boeschen D A (1981) Supervisor and subor-dinate reactions to performance appraisal sessions Organizational Behavior and Hu-man Performance 28 311ndash330

Jelley B R amp Goffin R D (2001) Can performance-feedback accuracy be improved Ef-fects of rater priming and rating-scale format on rating accuracy Journal of AppliedPsychology 86(1) 134ndash144

Johnson J W (2001) The relative importance of task and contextual performance dimen-sions to supervisor judgments of overall performance Journal of Applied Psychology86(5) 984ndash996

Judge T A amp Ferris G R (1993) Social context of performance evaluation decisionsAcademy of Management Journal 51 709ndash732

Kluger A N amp DeNisi A S (1996) The effects of feedback interventions on performanceA historical review meta-analysis and a preliminary feedback intervention theoryPsychological Bulletin 119 254ndash284

Kozlowski S W Chao G T amp Morrison R F (1998) Games raters play Politics strate-gies and impression management in performance appraisal In J W Smither (Ed)Performance appraisal State of the art in practice (pp 163ndash208) San Francisco CAJossey-Bass

Landy F J amp Farr J L (1980) Performance rating Psychological Bulletin 87 72ndash107Landy F J amp Farr J L (1983) The measurement of work performance New York NY Aca-

demic PressLatham G P Almost J Mann S ampMoore C (2005) New developments in performance

management Organizational Dynamics 34(1) 77ndash87Latham G P Fay C H amp Saari L M (1979) The development of behavioral observation

scales for appraising the performance of foremen Personnel Psychology 32 299ndash311Latham G ampWexley K (1977) Behavioral observation scales Journal of Applied Psychol-

ogy 30 255ndash268

Lawler E E Benson G S amp McDermott M (2012) Performance management and re-ward systemsWorldatWork Journal 21(4) 19ndash28

Lazear E P (1986) Salaries and piece rates Journal of Business 59 405ndash431London M amp Wohlers A J (1991) Agreement between subordinate and self-ratings in

upward feedback Personnel Psychology 44 375ndash390Longenecker C O Sims H P amp Gioia D A (1987) Behind the mask The politics of

employee appraisal Academy of Management Executive 1 183ndash193McKay P F ampMcDanielM A (2006) A reexamination of black-whitemean differences in

work performance More data more moderators Journal of Applied Psychology 91(3)538

Menefee J amp Murphy R (2004) Rewarding and retaining the best Compensation strate-gies for top performers Benefits Quarterly 20 13ndash20

Mercer (2013) Global performance management survey report Executive summaryRetrieved from httpwwwmercercomcontentdammercerattachmentsglobalTalentAssess-BrochurePerfMgmtpdf

Meyer H H (1980) Self-appraisal of job performance Personnel Psychology 33 291ndash295

Meyer H H Kay E amp French J (1965) Split roles in performance appraisal HarvardBusiness Review 43 123ndash129

Milkovich G T amp Wigdor A K (1991) Pay for performance Washington DC NationalAcademy Press

Murphy K R (1982) Difficulties in the statistical control of halo Journal of Applied Psy-chology 67 161ndash164

Murphy K R (2008) Explaining the weak relationship between job performance and rat-ings of job performance Industrial and Organizational Psychology Perspectives on Sci-ence and Practice 1 148ndash160

Murphy K R amp Anhalt R L (1992) Is halo error a property of the rater rates or thespecific behaviors observed Journal of Applied Psychology 77 494ndash500

Murphy K R amp Balzer W K (1989) Rater errors and rating accuracy Journal of AppliedPsychology 74 619ndash624

Murphy K R amp Cleveland J N (1991) Performance appraisal An organizational perspec-tive Needham Heights MA Allyn amp Bacon

Murphy K R amp Cleveland J N (1995) Understanding performance appraisal Social or-ganizational and goal-oriented perspectives Newbury Park CA Sage

Murphy K R Cleveland J N amp Mohler C (2001) Reliability validity and meaningful-ness of multisource ratings In D Bracken C Timmreck amp A Church (Eds) Hand-book of multisource feeback (pp 130ndash148) San Francisco CA Jossey-Bass

Murphy K R Cleveland J N Skattebo A L amp Kinney T B (2004) Raters who pursuedifferent goals give different ratings Journal of Applied Psychology 89 158ndash164

Murphy K R amp DeShon R (2000) Interrater correlations do not estimate the reliabilityof job performance ratings Personnel Psychology 53 873ndash900

Murphy K R Jako R A amp Anhalt R L (1993) Nature and consequences of halo errorA critical analysis Journal of Applied Psychology 78 218ndash225

Murphy K R Martin C amp Garcia M (1982) Do behavioral observation scales measureobservation Journal of Applied Psychology 67 562ndash567

Nathan B R amp Tippins N (1989) The consequences of halo ldquoerrorrdquo in performance rat-ings A field study of the moderating effect of halo on test validation results Journal ofApplied Psychology 74 290ndash296

Ones D S Viswesvaran C amp Schmidt F L (2008) No new terrain Reliability and con-struct validity of job performance ratings Industrial and Organizational PsychologyPerspectives on Science and Practice 1 174ndash179

Pfeffer J (2007) A modest proposal How we might change the process and product ofmanagerial research Academy of Management Journal 50 1334ndash1345

Pichler S (2012) The social context of performance appraisal and appraisal reactions Ameta-analysis Human Resource Management 51(5) 709ndash732

Posthuma R A amp Campion M A (2008) Twenty best practices for just employee perfor-mance reviews Compensation amp Benefits Review 40(1) 47ndash55

Posthuma R A Campion M C amp Campion M A (2015) A taxonomic founda-tion for evidence-based management research on employee performance managementManuscript submitted for publication

Pulakos E D (1984) A comparison of rater training programs Error training and accuracytraining Journal of Applied Psychology 69 581ndash588

Pulakos E D Mueller-Hanson R A Arad S amp Moye N (2015) Performance manage-ment can be fixed An on-the-job experiential learning approach for complex behaviorchange Industrial and Organizational Psychology Perspectives on Science and Practice8(1) 51ndash76

Pulakos E D amp OrsquoLeary R S (2011) Why is performance management so broken In-dustrial and Organizational Psychology Perspectives on Science and Practice 4(2) 146ndash164

Pulakos E D Schmitt N amp Ostroff C (1986) A warning about the use of a standard de-viation across dimensions within ratees to measure halo Journal of Applied Psychology71 29ndash32

Reilly R R ampMcGourty J (1998) Performance appraisal in team settings In J W Smither(Ed) Performance appraisal State of the art in practice (pp 244ndash275) San FranciscoCA Jossey-Bass

Rock D (2008) SCARF A brain-based model for collaborating with and influencing oth-ers NeuroLeadership Journal 1 1ndash9

Rock D Davis J amp Jones B (2014 August) Kill your performance ratings Strategy +Business Retrieved from httpvwwwstrategy-businesscomarticle00275

Ross L (1977) The intuitive psychologist and his shortcomings Distortions in the attri-bution process In L Berkowitz (Ed) Advances in experimental social psychology (Vol10 pp 173ndash220) New York NY Academic Press

Rothstein H R (1990) Interrater reliability of job performance ratings Growth to asymp-tote level with increasing opportunity to observe Journal of Applied Psychology 75322ndash327

Schleicher D J Bull R A amp Green S G (2008) Rater reactions to forced distributionrating systems Journal of Management 35 899ndash927

Schmidt F L ampHunter J E (1998) The validity and utility of selectionmethods in person-nel psychology Practical and theoretical implications of 85 years of research findingsPsychological Bulletin 124 262ndash274

Scott W R (1995) Institutions and organizations Thousand Oaks CA SageScullen S E Bergey P K amp Aiman-Smith L (2005) Forced distribution rating systems

and the improvement of workforce potential A baseline simulation Personnel Psychol-ogy 58 1ndash32

Scullen S E MountM K amp Judge T A (2003) Understanding the latent structure of jobperformance ratings Journal of Applied Psychology 85 956ndash970

Silverman S B amp Wexley K N (1984) Reaction of employees to performance appraisalinterviews as a function of their participation in rating scale development PersonnelPsychology 37 703ndash710

Smith P C amp Kendall L M (1963) Retranslation of expectations An approach to theconstruction of unambiguous anchors for rating scales Journal of Applied Psychology47 149ndash155

Smither J W London M amp Reilly R R (2005) Does performance improve followingmultisource feedback A theoretical model meta-analysis and review of empiricalfindings Personnel Psychology 58 33ndash66

Society for Industrial and Organizational Psychology (2003) Principles for the validationand use of personnel selection procedures Bowling Green OH Author

Stallings W M amp Gillmore G M (1972) Estimating the interjudge reliability of the Ali-Frazier fight Journal of Applied Psychology 56 435ndash436

Sulsky L M amp Balzer W K (1988) Meaning and measurement of performance rating ac-curacy Some methodological and theoretical concerns Journal of Applied Psychology73 497ndash506

Thompson D E amp Thompson T A (1985) Task-based performance appraisal for blue-collar jobs Evaluation of race and sex effects Journal of Applied Psychology 70(4) 747ndash753

Trank C Q Rynes S L amp Bretz R D Jr (2002) Attracting applicants in the war for tal-ent Differences in work preferences among high achievers Journal of Business andPsychology 16 331ndash345

Trevor C O Gerhart B amp Boudreau J W (1997) Voluntary turnover and job perfor-mance Curvilinearity and the moderating influences of salary growth Journal of Ap-plied Psychology 82 44ndash61

Tziner A ampKopelman R E (2002) Is there a preferred performance rating formatAnon-psychometric perspective Applied Psychology An International Review 51(3) 479ndash503

Tziner A amp Murphy K (1999) Additional evidence of attitudinal influences in perfor-mance appraisal Journal of Business and Psychology 13 407ndash419

Tziner A Prince B amp Murphy K (1997) PCPAQmdashThe questionnaire for measuringthe perceived political considerations in performance appraisal Some new evidenceregarding its psychological properties Journal of Social Behavior and Personality 12189ndash200

Vance R J Winne P S amp Wright S E (1983) A longitudinal examination of rater andratee effects in performance ratings Personnel Psychology 36 609ndash620

Viswesvaran C Ones D S amp Schmidt F L (1996) Comparative analysis of the reliabilityof job performance ratings Journal of Applied Psychology 81 557ndash574

Werner J M amp Bolino M C (1997) Explaining US courts of appeals decisions involvingperformance appraisal Accuracy fairness and validation Personnel Psychology 50 1ndash24

Wherry R J amp Bartlett C J (1982) The control of bias in ratings A theory of rating Per-sonnel Psychology 35 521ndash555

Williams C R amp Livingstone L P (1994) Another look at the relationship between per-formance and voluntary turnover Academy of Management Journal 37 269ndash298

Background to the Debate (Pulakos)

Get Rid of Performance Ratings (Colquitt Murphy amp Ollander-Krane)

Performance Appraisal A Failed Experiment

Disappointing Interventions

Disagreement Among Raters

Weak Criteria for Evaluating Ratings

Contextual Effects on Ratings

Conflicting Purposes

Feedback Is Not Accepted or Acted On

Weak Research-Practice Relationships

Additional Negative Consequences of Ratings

Get Rid of Ratings but Tread Carefully

Why Getting Rid of Performance Ratings Is a Bad Idea (Adler Campion amp Grubb)

Performance Ratings Performance Management

Performance Is Always Evaluated

ldquoToo Hardrdquo Is No Excuse for I-O Psychology

What Are the Merits of Ratings

The Value of Differentiated Evaluations

Artificial Tradeoffs Are Driving Organizations To Abandon Ratings

What Are the Alternatives to Performance Ratings

The Path Forward

A Dilemma

Closing

References

Industrial and Organizational Psychology 9(2) pp 219ndash252 June 2016Copyright copy 2016 Society for Industrial and Organizational Psychology doi101017iop2015106

Focal Article

Getting Rid of Performance Ratings Genius orFolly A Debate

Seymour AdlerAon Hewitt

Michael CampionPurdue University

Alan ColquittEli Lilly

Amy GrubbFBI

Kevin MurphyColorado State University

Rob Ollander-KraneGap Inc

Elaine D PulakosCEB

Despite years of research and practice dissatisfaction with performance appraisal isat an all-time high Organizations are contemplating changes to their performancemanagement systems themost controversial of which is whether to eliminate perfor-mance ratings The pros and cons of retaining performance ratingswere the subject ofa lively standing-room-only debate at the 2015 Society for Industrial and Organiza-tional Psychology conference in Philadelphia (Adler 2015) Given the high interestin this topic this article recaps the points made by the panelists who participatedin the debate The arguments for eliminating ratings include these (a) the disap-pointing interventions (b) the disagreement when multiple raters evaluate the sameperformance (c) the failure to develop adequate criteria for evaluating ratings (d)

Seymour Adler AonHewitt NewYork NewYorkMichael Campion Krannert Schoolof Management Purdue University Alan Colquitt Eli Lilly Indianapolis Indiana AmyGrubb FBI Washington DC Kevin Murphy College of Natural Sciences Colorado StateUniversity RobOllander-KraneGap Inc San Franciso California ElaineD Pulakos CEBArlington Virginia

This article grew out of a presentation at the 30th Annual Conference of the Societyfor Industrial andOrganizational Psychology in Philadelphia Pennsylvania (April 23 2015)titled ldquoGetting Rid of Performance Ratings Genius or Follyrdquo (S Adler Chair) Authorshiporder is alphabetical

Correspondence concerning this article should be addressed to Seymour Adler AonHewitt 199 Water Street New York NY 10038 E-mail seymour_adleraoncom

ogy 30 255ndash268