Due Process in Performance Appraisal: A Quasi-experiment ...€¦ · It seems fair to conclude that many organizational appraisal systems have failed to realize their full potential

Due Process inPerformance Appraisal:A Quasi-experiment inProcedural Justice

M. Susan TaylorUniversity of MarylandKay B. TracyGettysburg CollegeMonika K. RenardIVest Virginia UniversityJ. Kline HarrisonWake Forest UniversityStephen J. CarrollUniversity of Maryland

® 1995 by Cornell University.0001 -8392/95/4003-0495/$! .00.

Portions of this paper were presented atthe annual meeting of the Academy ofManagement, Atlanta, 1993. The authorsgratefully acknowledge the comments ofAngelo DeNisi, Mary Konovsky, PaulHanges. and three anonymous reviewerson earlier versions of this manuscript.

This field experiment extended research on proceduraljustice by examining effects of a due-processperformance appraisal system on reactions of bothemployees and managers. Employee-manager pairs wererandomly assigned to either a due-process appraisalsystem or the existing system. Even though due-processemployees received lower evaluations, they displayedmore favorable reactions: perceived system fairness,appraisal accuracy, attitudes toward the system,evaluations of managers, and intention to remain withthe organization. Managers also responded positively,reporting greater ability to resolve work problems,satisfaction with the system, job satisfaction, and lessdistortion of appraisal results to further their ownself-interests."

In many organizations performance appraisal systems remainone of the great paradoxes of effective human resourcemanagement. On one hand, appraisal systems can providevaluable performance information to a number of criticalhuman resource activities, such as the allocation of rewards,e.g., merit pay, promotions; feedback on the developmentand assessment of training needs; other human resourcesystems evaluation, e.g., selection predictors; andperformance documentation for legal purposes (Cleveland,Murphy, and Williams, 1989). In the abstract, appraisalsystems seem to offer much potential for enhancing theeffectiveness of human resource decisions and for satisfyingemployees' need for performance feedback (llgen. Fisher,and Taylor, 1979). On the other hand, there is evidence thatappraisal systems are a practical challenge to the academicswho often design them and to the managers and employeeswho must use them. As Banks and Murphy (1985: 335)noted: "Organizations continue to express disappointment inperformance appraisal systems despite advances in appraisaltechnology. Appraisal reliability and validity still remain majorproblems in most appraisal systems, and new (andpresumably improved) appraisal systems are often met withsubstantial resistance. In essence, effective performanceappraisal in organizations continues to be a compelling butunrealized goal." This negativity is echoed by practitioners inthe private and public sectors (George, 1986; Meyer, 1991).It seems fair to conclude that many organizational appraisalsystems have failed to realize their full potential contributionto organizational effectiveness.

Such a conclusion raises questions about why appraisalsystem effectiveness remains an elusive goal. Oneexplanation has been offered by Folger, Konovsky, andCropanzano (1992), who observed that appraisal systemshave traditionally been designed and implemented around a"test" metaphor that treats performance disagreementsbetween managers and employees as disputes over themost accurate view of reality, in which truth can bemeasured against some precise, consistent standard.Appraisers become "truth seekers who record objectivereality using reliable and valid measures" (p. 3). Theunderlying assumptions of the test metaphor becomequestionable, however, when applied to performanceappraisal. For example, work settings are assumed to permit

495/Administrative Science Quarterly, 40 (1995): 495-523

the reliable and valid measurement of objectiveperformance, but increasing numbers of employees nowwork in service jobs, where objective results are unavailable,or in groups, where individual performance results aredifficult to measure. Further, rather than assessingperformance objectively and accurately, raters' evaluationsare often subjectively biased by cognitive and motivationalfactors (Longenecker, Gioia, and Sims, 1987; DeNisi andWilliams, 1988), Finally, supervisors often apply verydifferent standards to employee performance, resulting ininconsistent, unreliable, invalid evaluations across theorganization (Folger, Konovsky, and Cropanzano, 1992). Inlight of these questionable assumptions, Folger, Konovsky,and Cropanzano (1992) suggested that performanceappraisals may be viewed more appropriately as disputesover the allocation of outcomes such as merit pay,promotion, or status. They argued that a due-processmetaphor, consistent with theories of procedural justice,may better address the appraisal situation than does the testmetaphor.

Applying the Due-process Metaphor toPerformance Appraisal

Due process of law, a right guaranteed under the Fifth andFourteenth Amendments of the United States Constitution,is intended to ensure individuals fair treatment whencharged with legal violations. There are three essentialfeatures of due process: (1) adequate notice—thatindividuals be held responsible for obeying laws only whenthey have been published or otherwise communicated andfor satisfying only those charges explicitly presented; (2) afair hearing—that all relevant evidence to a proposedviolation be presented and considered and that chargedparties be given the opportunity to provide commentary; and(3) judgment based on evidence—that judicial decisions befree from external pressures, personal corruption, and moreevident sources of bias (Folger, Konovsky, and Cropanzano,1992),

Folger, Konovsky, and Cropanzano (1992) have developedthree characteristics of a due-process appraisal system.Adequate notice in the context of performance appraisalrequires organizations and their agents to publish, distribute,and explain performance standards to employees, to discusshow and why such standards must be met, and to providefor regular and timely feedback on performance. Fair hearingrequires a formal review meeting in which an employee isinformed of a tentative assessment of his or herperformance and how it was derived by his or her manager,who should have a familiarity with the employee'sperformance based on sufficiently frequent observation ofthe individual's work or work product. Employees are alsopermitted to challenge this assessment and to provide theirown commentary by conducting and presenting aself-appraisal. Finally, fair hearing requires that employeesreceive training in the appraisal process to ensure that theypossess the knowledge needed to challenge assessmentsperceived to be unfair. Judgment based on evidencerequires the organization and its agents to applyperformance standards consistently across employees,

496/ASQ, September 1995

Due Process

without yielding to external pressure, corruption, or personalprejudice. Evaluations should reflect the organization'sefforts to use principles of honesty and fairness, andemployees should be given an explanation of perfornnanceratings and of reward allocations and be provided with anopportunity for discussion. Further, the appraisal systemshould be based on the best technology available in formcontent, format, and appraiser training to minimize bias.Thus the due-process appraisal system provides employeeswith substantial input into the appraisal process.

Relationship of Due-process Appraisal System toProcedural Justice Theories

Procedural justice research emphasizes the process throughwhich decisions are made in conflicts of interests as theprimary determinant of disputing parties' perceptions offairness (Leventhal, 1976; Folger, 1977; Thibaut and Walker,1978; Lind and Tyler, 1988; Tyler and Lind, 1992),Researchers generally have proposed two theoreticalexplanations for the psychological processes underlyingprocedural justice effects: (1) instrumental control and (2)relational concerns. The instrumental control explanation hasbeen best articulated by Thibaut and Walker (1978), whoargued that disputing parties or those affected by aninstitutional or organizational decision want control over theconflict resolution or decision process because they view itas instrumental to improving their own outcomes or theirrelationship with the other party in the dispute. Parties takea short-term perspective. Control over process assures eachparty that a third party who will resolve the dispute or makethe decision will receive sufficient information about theparty's contributions and outcomes to resolve the dispute ormake the decision equitably. Conversely, Lind and Tyler(1988), proponents of the relational expianation, argued thatconcerns about dispute resolution or decision-makingprocedures arise because people care about their long-termsocial relationship to the authorities or institutions thatemploy the procedures. Such procedures have importantimplications for individuals' feelings of self-worth and groupstanding. Because the procedures are viewed asmanifestations of basic process values in the institution ororganization, they take on value in and of themselves, notsimply because they promote the attainment of goalsoutside the process. Thus procedures reflecting a positive,full-status relationship are viewed as fair, while thoseindicative of a negative or low-status position within theinstitution are judged unfair (Tyler and Lind, 1992). To date,both the instrumental control and the relational explanationshave been supported by several empirical studies,stimulating Lind and Tyler (1988) to conclude that both areaccurate but incomplete explanations of the psychology ofprocedural justice.

The due-process metaphor of performance appraisal isconsistent with prior theoretical models of procedural justice,whether based on the instrumental control or relationalexplanation, Thibaut and Walker's (1978) instrumental controlmodel, for example, emphasized disputing parties'opportunity to present arguments supporting their case as acentral determinant of their perceptions of fairness,


Due-process appraisal systems provide considerableopportunity for employees and organizational representativesto present their views throughout the appraisal process,Leventhal and his associates (Leventhal, 1976; Leventhal,Karuza, and Fry, 1980) proposed that six procedural rulesaffect individuals' judgments of fairness, largely forinstrumental reasons. All six rules are present in a •due-process appraisal system: (1) the consistencyrule—maintaining consistency in performance standards overtime and among employees, resulting from the due-processappraisal system's requirement that performanceexpectations be discussed and recorded at the beginning ofthe period; (2) the bias-suppression rule—constrainingself-interest by discussing performance expectations anddiscrepancies at the final review; (3) the accuracyrule—training managers and employees to recordperformance accurately throughout the period and use thisrecord to prepare and justify performance evaluations; (4)the correctability rule—instructing managers to listen to theemployee's case and change the evaluation if appropriate;(5) the representativeness rule—discussing concerns of theemployee and manager throughout each stage of theprocess; and (6) the ethicality rule—using procedures thatare compatible with existing moral and ethical standards,

Tyler (1989), working from a relational perspective, proposedthat three factors of the authority or institution-personrelationship—standing, neutrality, and trust—are particularlystrong determinants of perceptions of fairness. Informationabout individuals' standing, defined as status or recognitionin the group, is thought to be conveyed by theirinterpersonal treatment during social interactions. Rudetreatment conveys that the group or authority regards theindividual as being of low status, and disrespect forindividuals' rights communicates that their interests areunlikely to be protected. Elements of standing seem presentin both the adequate-notice and judgment-based-on-evidencecomponents of the due-process metaphor for performanceappraisal. Neutrality concerns the authority or institution'stendency to create a neutral playing field on which allaffected parties will benefit fairly from the application of fairdecision processes. Neutrality is consistent with thejudgment-based-on-evidence component of the due-processmetaphor. Finally, trust is the individual's belief that theauthority or institution intends to treat people in a fair andreasonable way. It seems likely that all three connponents ofthe due-process metaphor serve to enhance individuals'feelings of trust toward the organization and agents chargedwith implementing the performance appraisal system.

Finally, Folger and Bies (1989) have proposed sevensuggestions for managers to assure proisedural justice,based on both the instrumental control and relationalliteratures: (1) adequate consideration of employees' pointsof view, similar to Thibaut and Walker's (1975) control overprocess; (2) bias suppression; (3) consistency of standardsacross employees, as proposed by Leventhal (1976); (4)timely feedback about decision outcomes; (5) managers'truthfulness in communications with employees, oneelement of Leventhal's (1976) ethicality rule; (6) treating


Due Process

employees with courtesy and civility, as proposed in Biesand Moag's (1986) development of interactional justice; and(7) providing employees with reasonable justification for anoutcome decision. The due-process metaphor thus appearsto be consistent with existing theories and models ofprocedural and interactional justice.

Just as the fairness of the procedures associated withorganizational decisions ranging from layoffs to pay allocationhave been shown to affect employees' reactions directly(Folger and Konovsky, 1989; Moorman, 1992), so areappraisal systems based on theories of procedural justiceexpected to affect the attitudes and behaviors of theemployees and managers who use them. Although oftenignored, employees' and managers' reactions to appraisalsystems are essential to successfully attaining at least threepurposes of appraisals—employee development, rewardallocation, and iegal documentation (Murphy and Cleveland,1991). Employees' reactions to the fairness and accuracy ofthe appraisal system may affect their motivation to correctweak performance or develop unused potential. Similarly,managers' reactions to performance appraisals are anecessary, if not sufficient condition to collect the relativelyunbiased performance information needed for rewardallocation or legal documentation. Thus, from a practicalperspective, examining how due-process appraisal systemsaffect reactions of employees and managers may providevaluable information about whether these systems cancontribute to organizational effectiveness.

Employees' Reactions to Due-process Appraisal Systems

There is considerable evidence that appraisal systems oftenviolate employees' due process by providing them withinfrequent and relatively general performance feedback(llgen. Fisher, and Taylor, 1979), allowing them little inputinto the appraisal process, and knowingly introducing biasinto performance ratings (Bernardin and Villanova, 1986;Longenecker, Gioia, and Sims, 1987). The three features ofdue-process systems—adequate notice, fair hearing, andjudgment based on evidence—should both increaseindividuals' perceptions of instrumental control over theappraisal process and enhance their sense of self-worth andgroup standing.

Although no previous study has examined the effects of afull due-process appraisal system on employees' reactions,prior research has investigated the impact of one or twocomponents. Studying a sample of managerial employees,Greenberg (1986) found that two-way communication duringthe review, opportunities to rebut evaluation (fair-hearingfeature), and the consistent application of a prioriperformance standards (adequate notice) were significantlyrelated to individuals' perceptions of the fairness of theappraisal. Similarly, in a laboratory setting, Kanfer et al.(1987) reported that subjects permitted to provideinformation about their own performance to their manager(fair hearing) perceived the appraisal to be fair. Using asurvey methodology, Fulk, Brief, and Barr (1985) reportedthat engineering employees perceived their evaluations asaccurate and fair when their managers were more


knowledgeable and trustworthy, and thus freer from bias(judgment based on evidence). Therefore, we hypothesized:

Hypothesis 1: Employees evaluated under a due-process appraisalsystem will report greater fairness of the appraisal system andgreater accuracy in their own appraisal than those evaluated by anappraisal system providing less due process.

Due-process features should also increase employees'satisfaction with the appraisal system (process satisfaction)and with their performance rating (outcome satisfaction).Thibaut and Walker's (1978) research clearly demonstratedthat people prefer to have as much control over the decisionprocess as they can, particularly when they cannot controlthe decision itself. Employees working under a due-processappraisal system should have their preferences for greaterprocess control met to a greater extent than do thoseworking under a more traditional system and express greatersatisfaction with the appraisal process (Locke, 1976).Correspondingly, satisfaction with the appraisal processshould be directly related to satisfaction with one'sperformance rating. Under a due-process system,performance ratings reflect not only the level of evaluationbut also one's positive group standing. Thus a high ratingwill be even more satisfying because the employee realizesthat it has high integrity, while a low rating will tend to beoffset by the realization that one is well regarded enough tobe treated fairly. As Lind and Tyler (1988: 207) concluded intheir extensive review of the procedural justice literature, "Inmost situations, procedural justice judgments lead toenhanced satisfaction" with both the process and theoutcome.

The procedural justice-satisfaction relationship is supportedby several correlational studies of performance appraisal.Dipboye and de Pontbriand (1981) reported that discussingperformance objectives and plans (adequate notice),providing employees with the opportunity to state their sideof the issues (fair hearing), and using job-relevantperformance dimensions in the appraisal (judgment based onevidence) were related to employees' favorable reactionstoward the appraisal system, while the first two systemcharacteristics were also related to their satisfaction withtheir rating (e.g.. Burke and Wilcox, 1969; Nathan, Mohrman,and Milliman, 1991). Therefore, we hypothesized:

Hypothesis 2: Employees evaluated under a due-process appraisalsystem will report greater satisfaction with the appraisal systemand greater satisfaction with their appraisal score than thoseevaluated by an appraisal system providing less due process.

Heightened satisfaction with a due-process appraisal systemand with the appraisal score should generalize to employees'overall job satisfaction. Consistent with the relationalexplanation of procedural justice effects, a due-processappraisal system should enhance employees' feelings ofseif-worth and their feeling of positive standing within theorganization. Consequently, individuals' job satisfactionshould increase and their overall attitude toward their workand their job situation (Locke, 1976) should improve underthe due-process appraisal system. This prediction isconsistent with findings in the legal arena, where proceduraljustice has been shown to increase individuals' overall


Due Process

satisfaction with the legal experience (Lind and Tyler, 1988).Further, in an organizational setting, Moorman (1992) foundthat the perceived justice of employers' decision-making anddecision-implementation procedures was positively related toemployees' job satisfaction.

More relevant to the performance appraisal process, Kanferet al. (1987) found that subjects who were permitted toprovide their manager with information about theirperformance reported greater satisfaction with their jobassignments. Similar results were found in two correlationalfield studies. Burke and Wilcox (1969) found that employeeswho reported relatively open communications with theirmanager about performance appraisal also had higher jobsatisfaction, while Nathan, Mohrman, and Milliman (1991)reported that employees' opportunity to participate in theappraisal review was positively related to satisfaction withtheir work. Therefore, we hypothesized:

Hypothesis 3: Employees evaluated under a due-process appraisalsystem will report greater job satisfaction than those evaluated byan appraisal system providing less due process.

Similarly, the procedural justice effects of a fairerperformance appraisal system are expected to generalize toemployees' evaiuations of their managers and to theirsatisfaction with their employing organization. According tothe relational explanation of procedural justice, individuals areconcerned with the long-term social relationships withauthorities and institutions. The implementation of adue-process appraisal system should reaffirm employees'self-worth and convey that the organization and participatingagents hold them in high standing and are committed toprotecting their rights in the context of organizationaldecisions. As Lind and Tyler (1988: 209) concluded in theirextensive review of the procedural justice literature,"Judgments of procedural justice enhance the evaluation ofauthorities and institutions." This effect has been supportedby correlational research in the legal arena and theorganizational context. Alexander and Ruderman (1987)reported that procedural justice was significantly related togovernment employees' evaluations of management.Similarly, McFariin and Sweeney (1992) found a proceduraljustice main effect on employees' evaluation of theirmanager. Therefore, we hypothesized:

Hypothesis 4: Employees evaluated under a due-process appraisalsystem will evaluate their manager more favorably and reportgreater satisfaction with their employing organization than thoseevaluated by a system providing less due process.

Beyond attitudinal effects, performance improvement is aprimary goal of appraisal in many organizations, andparticipation behaviors such as staying in the organizationmay have a strong impact on organizational effectiveness.Thus it is important to determine whether due-processappraisal systems positively affect critical work behaviors.Work by Tyler and Lind (1992) suggests that behavioraleffects may result because procedural justice increases thelegitimacy of the implementing organization in employees'eyes and thereby their willingness to comply with the goalsof the organization for high performance. Nevertheless,reviews of the procedural justice literature (Lind and Tyler,


1988) have found that behavioral changes result lessconsistently from increased procedural justice than doattitudinal effects. Although Earley (1984) reported thatallowing people to voice opinions about performance goalsimproved their subsequent performance in both lab and fieldsettings, Earley and Lind (1987) found that significantperformance effects emerged only in the laboratory, not inthe field. Similarly, Nathan, Mohrman, and Milliman (1991)found that the opportunity to participate in their performancereview was only marginally related to employees'subsequent performance improvements. Two studies,however, have tested and supported the negativerelationship between perceived procedural justice andemployees' intentions to leave the organization (Alexanderand Ruderman, 1987; Tyler and Schuller, 1990).

Several researchers have speculated that the findings havebeen mixed because behaviors are multiply determined byability, motivation, and opportunity (Lind and Tyler, 1988);thus increased compliance alone may not be sufficient tobring about increased performance. At the low-complianceend of the spectrum, employees may be unwilling tojeopardize their employment security by lowering theirperformance even though they feel little inclination tocomply with organizational goals (Greenberg, 1986).Greenberg (1986) also argued that the effect of proceduraljustice on withdrawal behaviors such as absenteeism orturnover is likely to be less constrained than its effects onperformance. In light of these proposals, behavioral effectsseem more likely to emerge from procedurally just appraisalsystems when the dependent variable examined is relativelyunconstrained by other factors such as ability. Therefore, inthis study we investigated the effects of a due-processappraisal system on employees' motivation to improveperformance, rather than actual changes in performance overtime and on their intentions to remain a member of theorganization. We hypothesized:

Hypothesis 5: Employees evaluated under a due-process appraisalsystem will report greater motivation to improve their performanceand stronger intentions to remain organizational members thanemployees evaluated by a system providing less due process.

Managers' Reactions to Due-process Appraisal Systems

Managers' reactions to performance appraisal systems areeven more important for system effectiveness thanemployees' because of managers' greater power to distortappraisal results and sabotage its reward, developmental,and documentation purposes. Nevertheless, relatively fewstudies of the appraisal process have examined the reactionsof managers (for exceptions, see Mount, 1983; Dorfman,Stephan, and Loveland, 1986). Similarly, managers' reactionsto procedural justice interventions are quite important, sincethey may either safeguard or sabotage fair policies andprocedures through their interactions with employees. Thus,knowing how managers react to the implementation ofenhanced procedural justice systems is critical to anunderstanding of the way such systems develop and aremaintained in organizations. Yet the focus of proceduraljustice research on managers to date has been largely


Due Process

limited to their role as third parties in disputes betweenpeers or subordinates (Karambayya and Brett, 1989),

Pfeffer (1981) expressed doubt that multiconstituentsystems of justice, similar to the due-process systemsdiscussed here, can succeed, because (1) those with morepower are unable to move beyond their own parochialconcerns to make decisions that are fair to those with lesspower; and (2) those with less power possess feweranalytical resources, bargaining skills, and information toprotect their own interests, Pfeffer's (1981) proposalsreceived indirect support from previous studies showing thatmanagers view cost, efficiency, and informality as moreimportant criteria for organizational dispute resolutionprocedures than fairness, while employees take the oppositeview (Lissak and Sheppard, 1983; Valley, 1989), Otherresearch has shown that managers frequently distortappraisal results to further their own self-interests(Longenecker, Gioia, and Sims, 1987), For example, deflatingan appraisal rating can send a message that an employee isnot wanted in the manager's department, while ratinginflation may occur in an attempt to make the manager andthe department look good. Thus due-process systems maywell provoke negative reactions from managers bydecreasing the efficiency of dispute resolution proceduresand constraining their ability to distort the results of theappraisal.

It is also true, however, that managers' own performance ishighly dependent on the efforts of those who work forthem. Consequently, it also might be argued that they willreact favorably toward a due-process appraisal system undertwo conditions. First, the system must not negatively affectmanagers' own reward contingencies in the organization,such as merit pay or promotions. Second, constraints ontheir power to distort appraisal results must be offset bymore valued gains in the long term. One outcome that mightoffset the constraints placed on managers under adue-process system is an increased ability to resolve workproblems that result from ineffective communication withemployees about work processes and priorities.Considerable research (Graen, 1976) has documented thehigh incidence and negative impact of discrepant roleperceptions between employees and managers. Whereasappraisal systems traditionally have been shown to strainmanagers' relationships with their employees (Bernardin andVillanova, 1986; Murphy and Cleveland, 1991), due-processappraisal systems inform employees of managers'performance expectations from the very beginning, provideopportunities for open exchanges about factors that mayimpede employees' ability to meet expectations, and provideongoing performance feedback. Further, becausedue-process systems are also expected to yield favorableemployee attitudes toward appraisal (hypothesis 2), as wellas increase the resolution of unit work problems,due-process appraisal systems should also increasemanagers' satisfaction with the appraisal system. Therefore,we hypothesized:

Hypothesis 6: Managers who evaluate employees using adue-process appraisal system will report greater resolution of work


problems and express higher satisfaction with the performanceappraisal system than those evaluating with a system providing lessdue process.

A second gain managers receive from implementingdue-process appraisal systems may be their increasedlegitimacy in the eyes of employees. Prior research suggeststhat, as the organizational agents responsible forimplementing the procedurally just appraisal system,managers will gain increased legitimate power, i.e., ability towin employee acceptance of work goals and rules (Tyler andLind, 1992). Thus managers' ability to influence employees'behavior without relying on the distortion of appraisal resultsor coercion should increase. Since influencing others'behavior through the use of legitimate power is assumed tobe more pleasant and easier than using manipulation orcoercive power for these ends, managers should experiencegreater satisfaction with their job and job situation:

Hypothesis 7: Managers who evaluate employees using adue-process appraisal system will report greater job satisfactionthan those who evaluate with a system providing less due process.

In the implementation of a due-process system, managersshould also distort appraisal results less to achieve their owninterests, because this implementation will increase thelegitimacy of the organization in their own eyes as well.Since managers may benefit less than employees from adue-process system, their reactions may be closer to thoseof observers of just procedures than of participants.Nevertheless, Thibaut and Walker (1978) found thatobservers also expressed a preference for and greatersatisfaction with dispute resolution procedures that provideddisputing parties with a high degree of control over thedecision process. Tyler and Lind (1992) have proposed thatthe organization implementing a due-process appraisalsystem will gain increased legitimacy in the eyes of itsemployees. Combining their work with the findings onobservers' reactions (Thibaut and Walker, 1978), it seemsthat the implementation of a due-process system also mayenhance the organization's legitimacy in the eyes ofmanagers and thereby make them more willing to complywith its goals, which in this case include increasingprocedural justice in performance appraisal for employees.Information about how managers determine that thedistortion of appraisal results is inconsistent withorganizational goals is provided by research on managers'distortion of appraisal results. Longenecker, Gioia, and Sims(1987) reported that organizational leaders cued managersabout the organization's tolerance for distortion of appraisalresults. As one manager put it (Longenecker, Gioia, andSims, 1987: 186): "At some places the PA (performanceappraisal) process is a joke—just a bureaucratic thing thatthe manager does to keep the IR (industrial relations) peopleoff his back. At the last couple of places where I've workedthe formal review process has been taken really seriously;they train you how to conduct a good interview, how tohandle problems, how to coach and counsel. You see thethings (appraisal) reviewed by your boss and he is seriousabout reviewing your performance in a thorough manner. . , ,I guess the biggest thing is that people are led to believe


Due Process

that it is a management tool that works; it's got to start atthe top,"

Thus by implementing a due-process system of appraisal,complete with training for employees and supervisors, anorganization is expected to signal to managers that there willbe little tolerance for behavior that distorts appraisal results.The organization's enhanced legitimacy from implementing aprocedurally just appraisal system should increase managers'willingness to comply with its goals for appraisal:

Hypothesis 8: Managers who evaluate employees using adue-process appraisal system will report less distortion of appraisalratings to serve their own self-interests than those who evaluatewith a system providing less due process.

We examined the eight hypotheses developed above in afield experiment covering a three-month period. The studymakes several contributions to the existing literatures onprocedural justice and performance appraisal. First, it can bedistinguished from other tests of enhanced proceduraljustice effects in organizations by its field-experiment design,the breadth of the procedural justice manipulation, theassessment of both specific and global job attitudes, and alonger follow-up period. Second, this research extended thestudy of procedural justice to include the reactions ofmanagers, a population largely unexamined in the proceduraljustice literature to date. Finally, this study makes a valuablepractical contribution to the performance appraisal literatureby determining whether the design of evaluation systemsaround the due-process metaphor may lead to favorablereactions by managers and their employees. If so,due-process appraisal systems seem likely to achieve thedevelopmental, reward allocation, and documentationpurposes of appraisal and thereby to enhance organizationaleffectiveness,

METHOD

Overview

This research was conducted in a government agency (1,400employees) of a northeastern state. The majority ofemployees (some 900) administered unemployment benefitsand provided job-search services, while the remainder wereresponsible for building the state's economy throughcultivating new business. Most employees heldadministrative (section head), clerical (typist, clerk) orprofessional (lawyer, auditor, administrator) positions.Approximately 20 percent of the workforce worked inmanagerial positions.

Organizational leaders requested that the researchersdevelop, pilot, and revise a new performance appraisalsystem for the department, stipulating that they must do sowhile incorporating a high level of employee participation inthe development process. The participation requirementprovided a unique opportunity to discover what aspects ofthe current performance appraisal system employeesconsidered fair and unfair and to tailor those aspects into anexperimental field study to test the due-process metaphor ofperformance appraisal,


Employee participation was achieved in several ways. First,we held random interviews, and then administered adepartment-wide attitude survey to determine reactionstoward the existing appraisal system and preferences for anew one. Two representative employee committees (total N= 40) were formed to assist in system design, pilot testing,and revision. Finally, we randomly selected representativegroups of employees and managers in the department toparticipate in pilot testing the appraisal system. Only twomembers of the employee committees participated in thepilot testing. The data reported here are from the pilotproject.

Interviews with a representative sample of employeesrevealed that they genuinely desired a performance appraisalsystem that would discriminate among high and lowperformers and provide them with feedback about theirperformance ("I haven't had an appraisal since I've beenhere"; "No one ever tells you how you are doing aroundhere,"). The interviews also indicated that the existingappraisal system, developed by the state personnel officemore than a decade earlier, was disliked and distrusted byvirtually everyone. Based on a relatively general, graphicrating scale that included three performancedimensions—quality of work, quantity of work, and jobattitudes—the existing system generated perceptions ofinequity ("The appraisal system is not fair; it does notdistinguish between high and low performers."), inflation("Everybody gets a satisfactory rating or above."),insensitivity ("My manager sends my evaluation throughinter-office mail without marking it confidential, so otheremployees open it before I do."), and uselessness ("No oneever looks at performance ratings when making salary orpromotion decisions.").

The agency provided a strong test of the due-processsystem because employees' and managers' dissatisfactionwith the existing appraisal system was accompanied by anextremely pessimistic attitude toward the likelihood ofpositive change ("Nothing ever changes around here; what awaste of money to fund work on the appraisal system; whynot put those resources in more pay for the employees?").The organizational climate was one of dissatisfaction,distrust, and pessimism, providing no assurance that eitheremployees or managers would be any more receptive to anew appraisal system than they were to the existing one.

The design for the study was a field experiment usingexperimental and control groups of employees and theirmanagers, with pre- and post-study measures of thedependent variables. The independent variable was thedue-process appraisal system.

Sample

We selected representative, matched pairs (111) ofemployees and their managers from the organization'spersonnel files and invited them to participate in the pilotproject. Employees and managers were similar in age (mean= 43,86 and 44,91, respectively), job tenure (mean = 9,52and 8,40), and organizational tenure (mean = 12,17 and16.11). Most employees (58.5 percent) were female; most


Due Process

managers, male (58 percent). Over 60 percent of employeesand 72 percent of managers had at least some collegeeducation.

We paired managers with only one employee, although theymay have supervised several in the organization. Randomassignment placed 61 pairs into the experimental group and50 pairs into the control group. Fifty-one pairs in theexperimental group and 41 pairs in the control group beganthe study, a beginning participation rate of 84 percent and 82percent, respectively. Because the pilot project ran into thesummer and the experimental design was longitudinal, welost participants from both groups. Forty-two employees and40 managers in the experimental condition, and 21employees and 29 managers in the control conditioncompleted the study, resulting in final participation rates of83, 78, 51, and 79 percent, respectively.

Final sample sizes revealed a higher dropout rate in theemployee control group than in the experimental one.Discussions with participants suggested that the higherdropout rate in the control group resulted from their lowerlevel of involvement in the pilot project: They were asked toparticipate in the pilot project by using the existing system,an appraisal system they had already been exposed to manytimes before .and knew that they did not like.

Follow-up analyses revealed no significant differences in thedemographic characteristics (i,e,, age, education, gender, jobor organizational tenure) or pre-survey reactions of thoseparticipants who completed the study and those whodropped out. Further, analysis of the pre-survey measurescompleted for the study revealed only one significantdifference between stayers and leavers. Employeecontrol-group dropouts were significantly more negative thancontrol-group stayers on the pre-survey measure ofperceived appraisal accuracy. These findings suggest thatthose who dropped out did not differ in a meaningful wayfrom those who remained, although there is no way todetermine definitively the effects of the differential dropoutrate. If anything, they were initially somewhat more negativeabout the accuracy of the existing appraisal system that theywere asked to use during the pilot study and therefore mightbe expected to have been somewhat more negative at itsconclusion.

Procedure

Employees and managers selected for the pilot projectreceived a letter from the department head encouragingthem to participate. Individuals who agreed received apre-survey identified by a code number and returned it to theresearchers' university address. Those not returning thesurvey after ten days were given one to two reminders bytelephone.

Experimental group participants were scheduled into trainingsessions during their work day. We trained managers firstand instructed them to begin the performance period assoon as the employee with whom they had been paired wastrained. We asked managers to meet with their employee onat least three occasions—to discuss performance


expectations at the beginning of the pilot period, to givefeedback midway through the pilot period, and to conductthe formal appraisal session. If we did not receive formsverifying that these meetings had been held, within ten daysof the scheduled date, we contacted the managers by phoneand reminded them to conduct the meetings.

Control group managers and employees conducted theappraisal process "as they normally did" and began theappraisal period at the same point as the experimentalgroup. We contacted them at the end of the three-monthperiod and reminded them to conduct a formal appraisalusing the organization's existing appraisal form.

All participants received a post-study questionnaire assessingtheir perceptions of and reactions to the performanceappraisal system under which they worked. We gave themup to two reminders by phone if we did not receive thesurvey within ten days after the scheduled final review time.

Independent Variable—Due-process PerformanceAppraisal System

The due-process appraisal system developed for thedepartment included a new appraisal form, process, manual,and training for managers and employees. The training,conducted in multiple groups of 15-20 participants, used thenew appraisal forms and relied heavily on experientiallearning, It included video-taped behavioral modelsillustrating desired behaviors, role-play exercises, andindividual feedback. Differences between the measures andtreatments given to control and experimental groups aresummarized in Table 1. Table 2 shows the features of thenew system, which incorporated elements from all threedue-process features, as recommended by Folger, Konovsky,and Cropanzano (1992).

Table 1

Treatments and Measures in Experimental and Controi Groups*

Employees Managers

Treatment/Measure

Pre-survey questionnaireFirst training sessionSecond training sessionNew appraisal formNew appraisal manualThree-month performance

periodFinal performance evaluationPost-survey questionnaire

Dueprocess

XX0XX

XXX

Control

X0000

XXX

Dueprocess

XXXXX

XXX

Control

X0000

XXX

• X = present, 0 = absent.

We collected measures of the due-process variables fromemployees and managers on the pre- and post-surveys toprovide a manipulation check. Table 3 shows these variables,as well as a sample item from each scale, the total numberof items used in the scale, and where appropriate,Cronbach's alpha coefficient, indicating the internal


Due Process

Table 2

Content of Due-process Appraisal Systenti*

XXXX0

XX

0XXX

0X

0

XXX

XXXXX

XX

XX00

XX

X

0XX

System feature Employees Managers

Adequate noticeTraining to develop standardsTraining to communicate/clarify standardsExpectation setting meetingTraining to negotiate standardsTraining to give feedbackTraining to participate in mid-study feedback

sessionMid-study feedback session

Fair hearingTraining to encourage two-way

communication in final reviewTraining on how to use formTraining to conduct self-appraisalConduct self-appraisal

Judgment based on evidenceTraining to sample representative performanceTraining to keep performance diaryTraining to solicit performance information

from employeesTraining to make supervisor aware

of accomplishmentsAppraisal form fitted to jobManual on appraisal process

• X = present, 0 = absent.

consistency of the scale. Except where noted, responseoptions for survey items used a 5-point Likert scale, rangingfrom "strongly disagree" (1) to "strongiy agree" (5), Thescore for a variable was the mean of participants' responsesto the items included on a scale, and unless noted, a highscore indicated a high level of the variable assessed.

In addition, we kept records that we used to validateresponses to some of the due-process variable measures,such as training sessions attended and expectation meetingsheld. Correlations between participants' self-reports andarchival records were high (iow to high ,80s).

Measurement of Check Variables

We included several items in the surveys to control forvariables that were not of primary interest to the study butappeared likely to affect the dependent measures:managers' reward contingencies, performance ratings, andtwo measures of sensitization to the study.

Managers' reward contingencies were included to checkwhether unintended negative changes occurred that mighthave offset the effects of the due-process appraisal system.We measured four reward contingencies with scalesdeveloped by Napier and Latham (1986) to assess managers'outcome expectancies for conducting valid performanceappraisals, A central question asked managers to assess thelikelihood that, if they took the time required to evaluateemployees' performance accurately, reviewed the ratingswith employees, and turned them in to the Personnel Office,(1) their rewards would increase; (2) their rewards would


Table 3

Measurement of Due Process and Check Variables

VariableNo. ofItems Sample item Alpha

Due process variables—Employees

Fair hearingFormal performance review

meeting

Conducted self-appraisal

Adequate noticeInput to process

Performance expectationmeeting

Supervisor communication ofperformance expectation

Frequency of feedback fromsupervisor

Held mid-study feedbackmeeting

Judgment based on evidenceValidity of appraisal form

Did you meet with your supervisor at the end of the PilotProject to review how he/she evaluated your performanceduring the Pilot Proiect? (yes/no)

Did you complete a self-appraisal by evaluating your ownperformance on the same appraisal form your supervisorused to evaluate you? (yes/no)

During the Pilot Project I basically had no say about the .76performance expectations that were set for me.(Reverse-scored)

Did you meet with your supervisor at the beginning of the JobExcellence Pilot Project to discuss expectations for yourperformance during the next three months of the project?(yes/no)

During the Pilot Project my supervisor let me know what .75he/she expected of me before evaluating my performance.

During the Pilot Project my supervisor often told me how I .85was doing on the job.

Did you meet with your supervisor about midway through Pilot -Project to discuss your performance? (yes/no)

The form used to evaluate my performance during the Pilot .83Project measures things that are important for my job.

Supervisor's honesty

Employee check variables

Sensitization variables:Training satisfaction

Performance recognition

Due process variables—Managers

6

CM

C

M

My supervisor was very honest with me.

(Company name) provides training that helps me do a betterjob.

(Company name) recognizes good performance.

.89

.61

.64

Fair hearingFinal performance review

meeting

Employee self-appraisal

Adequate noticeEmployee input to process

Performance expectationmeeting

Communication of performanceexpectation

Frequency of feedback

Held mid-study feedbackmeeting

Judgment based on evidenceValidity of appraisal form

Honesty with employee

Did you meet with your assigned employee at the end of thePilot Project to review how you evaluated his/herperformance during the Project?

Did your employee complete a self-appraisal by evaluatinghis/her own performance on the same appraisal form youused to evaluate him/her?

The last time I evaluated my employee's performance, I askedhim/her to tell me how well he/she thought he/she haddone.

Did you meet with your assigned employee at the beginning ofthe Job Excellence Pilot Project to discuss expectations forperformance during the next three months?

I let my employee know what I expected of him/her before Ievaluated his/her performance at the end of the PilotProject.

I often told my employee how he/she was doing on the jobduring the Pilot Project.

Did you meet with your assigned employee halfway throughthe Pilot Project to discuss his/her performance? (yes/no)

The form used to evaluate my employee's performance duringthe Pilot Project doesn't really measure what he/she doeson the job. (Reverse-scored)

I am very honest with my employees.

.76

.76

.83

.63


Table 3 {continued}

Due Process

VariableNo, ofItems Sample item Alpha

Managers' check variables

Sensitization variables;Training satisfaction

Performance recognition

Demoralization variabie;Self-esteem

Reward contingencies;My rewards decrease

My rewards increaseUnit productivity decreaseUnit productivity increase

2 (Company name) provides training that can help my employees .52do a better job here at (company name). (Reverse-scored)

2 (Company name) recognizes good performance. ,81

4 On the whole, I am satisfied with myself. .83

2 If I took the time required to evaluate my employees' .90performance accurately, reviewed the ratings with them, andturned them in to the company Personnel Office, it wouldhurt my chances of getting a raise.

2 It would help me get a raise, .894 The productivity of my unit would decrease. ,702 The productivity of my unit would increase, .90

decrease; (3) unit productivity would increase; or (4) unitproductivity would decrease (see Table 3),

Performance ratings. Employees' performance ratings wereexamined to assure that any favorable changes in thereactions of experimental-group employees and managerscould not be accounted for by the fact that experimental-group employees received higher ratings. Two items weremeasured to determine if there were differences in theperformance ratings between the experimental and controlgroups. These were the manager's report, assessed on thefinal survey, of the employee's performance rating, rangingfrom "deficient" (1) to "excellent" (5), and the final score onthe appraisal form managers completed and returned to us.Since the due-process form was based on a 5-point scaleand the existing appraisal form on a 3-point scale, wetransformed scores on the two different forms to makethem comparable.

Sensitization variables. We included two variables todetermine whether positive reactions from the due-processgroup might be caused by the fact that the pre-studyquestionnaire sensitized them to the study hypotheses,which they then consciously may have attempted tosupport. Because the due-process appraisal system had noimpact on training for improved job performance or theorganization's recognition of good performance in the periodof the pilot study, employees' and managers' satisfactionwith employees' opportunities to receive training that wouldimprove their job performance and their perception of theorganization's recognition of good performance (see Table 3)were not expected to change as a result of the pilot project.

Dependent Variables

Dependent variables were measured by scales of items toassess employees' and managers' reactions (attitudes)toward the appraisal system. Although we used established


nneasures whenever possible, we had to develop somescales specifically for the study. Scale items typically used a5-point Likert scale, ranging from "strongly disagree" (1) to"strongly agree" (5), The scales for employees andmanagers, a representative item from each scale, the totalnumber of items in the scale, and where relevant,Cronbach's alpha coefficient for the scale are shown inTable 4,

Table 4

Measurement

Variable

of Dependent Variable Scales

No, ofItems

Employees

Sample item Alpha

Faimess of appraisal

Accuracy of appraisal

Satisfaction with appraisal system

Rating satisfaction

Satisfaction with jobEvaluation of supervisorSatisfaction with organizationMotivation to improve

Intention to remain

I don't think my performance appraisal during the Pilot Projectwas a fair one. (Reverse-scored)

My performance evaluation from the Pilot Project tended to be .74pretty accurate,

I am satisfied with the way my performance evaluation was .60done during the Pilot Project.

I was very satisfied with the results of the performance ,64evaluation I received during the Pilot Project,

All in all, I am satisfied with my job. .74My supervisor is a good manager, ,89I generally look forward to coming to work at (company name), .74My last performance appraisal did not motivate me to do a ,71

good job, (Reverse-scored)I intend to keep working at (company name) for at least the

next three years.

Managers

Decreased work problems

Satisfaction with appraisal system

Satisfaction with jobAppraisal distortion

2 We have fewer work problems in my department because of .75the performance evaluation system.

4 I am satisfied with the way performance appraisals were done .61in the Pilot Project.

3 All in all, I am satisfied with my job. .799 During the Pilot Project, I evaluated my employee higher than .90

he/she actually performed to keep my unit from looking badto people outside. (Reverse-scored)

Employees. Employees' reactions included the perceivedfairness of the system, the perceived accuracy of theirappraisal, their satisfaction with the appraisal system, theirsatisfaction with the performance rating, their jobsatisfaction, an evaluation of their manager, satisfaction withtheir employing organization, and two variables intended as.behavioral surrogates—motivation to improve performanceand intention to remain in the organization during theupcoming three-year period. Fairness of appraisal wasassessed with a single item asking whether or not theappraisal received during the pilot project was a fair one.Accuracy of appraisal was assessed with two itenns askinghow accurate the pilot project appraisal was and whetherthe evaluation showed how they "really performed" on thejob. Satisfaction with appraisal system was assessed withfour items asking whether the organization should changethe appraisal system, whether there were fewer workproblems in the unit as a result of the systenn, whetheremployees were satisfied with the way the organization did


Due Process

appraisals, and whether having appraisals was a waste oftime. Satisfaction with performance rating received in thepilot project was assessed with two items asking aboutemployees' satisfaction with the results of the appraisal andwith "how they did" on the last appraisal. Evaluation ofmanager was assessed by six items asking about theirmanagers' people skills, knowledge of their job, competenceat managing, etc. Satisfaction with job and organization wereassessed with three items each from the short form of theIndex of Qrganizational Reactions (IQR), a satisfaction surveydeveloped for the Sears Corporation that has been found tobe reliable and valid across many different organizationalsettings (Dunham and Smith, 1979). Motivation to improveas a result of the appraisal feedback received was assessedwith two items asking employees if the appraisal motivatedthem to do a good job and made them want to work hardduring the upcoming year. Intent to remain was assessedwith a single item that asked the extent to which employeesintended to continue working at the department for the nextthree years. Intercorrelations between the dependentvariables for employees were generally moderate, rangingfrom .10 to .74 (appraisal satisfaction with appraisal,accuracy). They are shown in Tabie 5.

Table 5

Correlation Matrix of Empioyees' Dependent Variables*

Variable

1. Fairness of appraisal

2. Accuracy of appraisal

3. Satisfaction with system

4. Satisfaction with rating

5. Satisfaction with job

6. Evaluation of supervisor

7. Satisfaction with organization

8. Motivation to improve

9. Intent to remain

• p < .05; • • p < .01.

Mean 2 3

3 86 .73** .52**(1.01)3.58 .55~(.91)

3.38(.61)

3.78(1.04)3.98(.67)

3.76(.81)

3.73(.71)

3.47(.87)

3.94(.89)

• Standard deviations are in parentheses. Degrees of freedom

4

.66*

5

.21

.74~ .28*

.44- .12

range

.21

from 65 to

6

.38-

.53-

.18*

.44-

.52-

51.

7

.15

.24

.24

.18

. 5 1 -

.32*

8

.32*

.35**

.28*

.21

.32*

.32*

.11

9

. 4 1 ~

.12

.15

.10

.37-

.25*

.13

.20

-

Managers. Managers' reactions assessed for the studyincluded the increased resolution of work problems,satisfaction with the appraisal system, job satisfaction, andlevel of distortion of appraisal results. Table 4 showsadditional descriptive information for these variables.Decreased work problems was assessed by two itemsasking the extent to which the appraisal facilitated solvingwork problems and improved communications withemployees. Satisfaction with the appraisal system wasassessed with the same four items used for employees. Jobsatisfaction was measured with the same three items from


the lOR scale used for employees. Appraisal distortion, theextent to which managers tended to distort employees'performance appraisal ratings to further their ownself-interests, rather than to assess employees' performanceaccurately, was measured by a nine-item scale developedfrom the research of Longenecker, Gioia, and Sims (1987).The items asked about the extent to which managerstended to inflate or deflate performance ratings for a varietyof different purposes. These included (1) inflation becausethe unit would look bad to outsiders; the rating wouldbecome a permanent part of the individual's file; statesalaries were low enough without making people lose meritmoney; conflict with the employee would likely result; andthe employee's recent performance was better than his orher earlier performance; and (2) deflation to get employeesback on track; remind them who is boss; and get them tothink about leaving the unit because they were not welcomeby the manager. A final item asked the extent to which themanager's own boss tended to distort appraisals. This scalewas reverse-scored so that a high score reflected lessappraisal distortion. Intercorrelations between the dependentvariables assessed for managers were of moderate size,ranging from .07 to .68 (decreased work problems andsatisfaction with appraisal system). These relationships areshown in Table 6.

Table 6

Correlation Matrix of Managers' Outcome Variables*

Variable

1. Decreased work problems

2. Satisfaction with system

3. Satisfaction with job

4. Appraisal distortion

• p < .05; -p< .01.

Mean

2.75(1.04)3.26(.74)2.99(.55)

4.24(.58)

1 2

.67** .24

.28*

3

.07

.68-

.52 -

_

• Standard deviations are in parentheses. Degrees of freedom range from 67to 62.

Analyses

We performed MANCOVAs, using the pre-survey measuresas covariates, to check the effectiveness of the due-processappraisal manipulation. Separate ANCOVAs, using thepre-survey measure of the relevant dependent variable as acovariate, were performed to test hypotheses.

RESULTS

Due-process Manipulation Checks

MANCOVAs performed on the due-process manipulationchecks provided strong evidence that the manipulation wasperceived as intended by both employees and managers(Wilks X.10.44 = -33, p < .00; and Wilks X.io.,42 = 28, p <.00, respectively). Table 7 shows the univariate ANCOVAs.


Due Process

Table 7

ANCOVAs on Due Process iVIanipulation Checics

VariableControlmean

Employees

Fair hearingFinal review session held*Employee conducted self-appraisal

Adequate noticeEmployee input to processPerformance expectation meetingSuper\/isor communicated

performance expectationsFrequency of feedbackMid-study feedback meeting held*

Judgment based on evidenceValidity of appraisal formSupervisor's honesty

14.005.00

3.0912.00

2.483.054.00

2.683.59

Managers

Fair hearingFinal review session held*Employee conducted self-appraisal*

Adequate noticeEmployee input to processPerformance expectation meeting*Supervisor communicated

performance expectationsFrequency of feedbackMidway feedback meeting held*

Judgment based on evidenceValidity of appraisal formSupervisor's honesty

•p< .05; - p < .01.

19.008.00

3.4216.00

3.263.388.00

2.543.68

* Variable is dichotomous; mean = number ofused instead of F.

Dueprocessmean

40.0031.00

3.7033.00

3.453.37

33.00

3.533.92

38.0027.00

3.9838.00

4.043.50

29.00

3.714.07

d.f.

11

1,431

1, 441,501

1,461,52

11

1, 491

1, 441, 491

1,481,51

yes responses; x^

F

15.43-19.37-

9.20-6 . 6 1 -

9.89-1.43

15.61-

4.46*4.20*

10.18-8.54-

6.47-14.44-

11.02—5.61*

13.64-

22.05-5.62*

• statistic

Due-process-group employees reported significantly higherperceptions than the control group on all due-processnnanipulation check variables except the frequency offeedback received. Due-process-group managers reportedsignificantly higher perceptions on all due-process variablesthan did control-group managers.

Sensitization Check VariablesNo significant differences were found between control- anddue-process-group employees on either their satisfactionwith training opportunities (M,, = 3.07, M p̂ = 3.16; F, 53 =.75, n.s.) or the extent to which they perceived that theorganization recognized good performance (Mg = 2.96, M^p= 2.96; FT 50 = 001' i^s)- Nor were significant differencesfound for nianagers on the satisfaction with training variable(Me = 3.02, Mdp = 3.07; F^^o = .046, n.s.) or theirperceptions of trie organization's recognition of goodperformance (M,, = 3.21, M p̂ = 3.00; F, 51 = .73, n.s.).Since these two variables tended to display lower internalconsistency reliabilities than did the majority of variablesused in the study—possibly because participants had no


clear frame of reference for responding to them—theattenuation formula was used to determine whether any ofthe four relationships would be significant under conditionsin which the measures had perfect reliability (Nunnally,1978), Once again, none of the relationships was found tobe significant. Thus, results failed to provide any support forthe idea that study findings were biased by participants'sensitization to the treatment.

Performance Ratings

Analysis of variance results showed that control-groupperformance ratings were significantly greater than those ofthe due-process group on both the rating category assessedon the final survey and on the appraisal form itself(managers' reports: M^ = 4.29; M^p = 3,84; f , gg = 3,05,p < ,05; appraisal form: Mp = ,82, M p̂ = ,29; Fi 63 =65,08, p < ,00), These findings are not surprising, given theexisting appraisal system's reputation for rating inflationversus the judgment-based-on-evidence component of thedue-process appraisal system. Once again, however, thepreliminary results support the idea that more favorablereactions by due-process employees were caused by thedue-process appraisal system, not by other factors such ashigher performance ratings.

Managers' Reward Contingencies

We examined managers' perceptions of their rewardcontingencies to ensure that a perceived negative effect ofthe due-process appraisal system did not constrain theirfavorable reactions to the system. Only one of the fourcontingencies showed even a tendency to differ for thecontrol and due-process groups. In this case,experimental-group managers felt it was less likely thatproviding accurate performance appraisals would decreasethe productivity of their work units (p < ,10), Thus, ifanything, the due-process manipulation slightly increasedmanagers' motivation to evaluate their employees'performance accurately and would not be expected toconstrain their favorable reactions to the system.

Tests of Hypotheses

Employees' reactions. Table 8 shows the results of testinghypotheses 1-5, on employees' reactions to the appraisalsystem. Hypothesis 1 predicted that due-process-groupemployees would perceive the appraisal system as beingfairer, and their own appraisal as being more accurate, thanwould control-group employees. This hypothesis receivedstrong support, with significant effects on both dependentvariables. Hypothesis 2, predicting a positive impact onemployees' satisfaction with the due-process appraisalsystem and with their appraisal score, was also supported.Due-process-group employees expressed significantlygreater satisfaction with the appraisal system and with theirappraisal rating than did control-group employees, despitethe fact that the overall mean appraisal rating for thedue-process group was significantly lower than that of thecontrol group. But hypothesis 3, predicting thatdue-process-group employees' would report greater jobsatisfaction than the control group, was not supported,


Due Process

Table 8

ANCOVAs on Ennpioyees' Outcome Variabies

Variable

Fairness of appraisalAccuracy of appraisal

Control Experimentagroupmean

3.463.25

Satisfaction with appraisal system 3.15Satisfaction with ratingSatisfaction with jobEvaluation of managerSatisfaction with organizationMotivation to improveIntention to remain

• p < . 1 0 ; - p < .05.

3.444.073.593.773.383.73

* Degrees of freedom vary due to missing values

groupmean

3.963.673.463.973.963.923.663.554.05

1 on some

1

d.f.*

1, 501,481, 501, 501,521,521,521, 501, 53

scales.

F

3.07-3 . 0 1 -3.42-3 . 6 1 -

.434.20-

.43

.492.31*

Hypothesis 4, which proposed that due-process-groupemployees would evaluate their managers more favorablyand report greater satisfaction with the employingorganization than control-group employees, received onlypartial support. While due-process-group employees didevaluate their managers significantly more favorably than didthe control group, the two groups did not differ significantlyin their satisfaction with the employing organization. Finally,hypothesis 5, which predicted that due-process-groupemployees would report a greater increase in theirmotivation to improve performance and their willingness toremain in the organization than those in the control group,also received partial support. Due-process-group employeesdid not express significantly greater motivation to improveperformance than did those in the control group, but theydid report a greater tendency to remain organizationalmembers.

Managers' reactions. Table 9 reports the results of testinghypotheses 6 through 8, on managers' reactions to thedue-process appraisal system. Hypothesis 6, which predictedthat due-process-group managers would report greaterresolution of unit work problems and greater satisfactionwith the appraisal system than would control-groupmanagers, received strong support. Hypothesis 7, predictingthat due-process-group managers would express significantlygreater job satisfaction than would control-group managers.

Table 9

ANCOVAs on Managers' Dependent Variabies

Variable

Control Experimentalgroup groupmean mean d.f.*

Decreased work problemsSatisfaction with appraisal systemSatisfaction with jobAppraisal distortiont

2.343.092.874.03

3.023.503.014.35

1,1,1,1,

48485148

5.05*3.59*3.83*2.98*

• p < .05.* Degrees of freedom vary due to missing values on some scales.t Variable is reverse-scored.


was also supported. Finally, hypothesis 8—thatdue-process-group managers would display less tendency todistort appraisal results to further their own self-intereststhan wouid those in the control group—was also supported,

DISCUSSION

This field experiment tested the impact of a due-processperformance appraisal system (Folger, Konovsky, andCropanzano, 1992) on employees' and managers'perceptions of fairness and on their attitudinal and behavioralreactions over a three-month period. Overall, resultsprovided fairly strong support for the effects of adue-process appraisal system on the reactions of employeesand managers.

The research makes at least three contributions to theexisting literature on procedural justice, two theoretical andone practical: (1) extending research on the determinants ofprocedural justice judgments (Lind and Tyler, 1988) to adue-process intervention and providing additional support forthe external validity of procedural justice effects; (2)examining managers' reactions to an intervention increasingprocedural justice for their employees at the expense oftheir own freedom to distort appraisal results; and (3)investigating the impact of a due-process appraisal systemon the reactions of employees and managers to their jobsand their organization.

Readers should consider these contributions in light of threepotential limitations. First, the public sector organizationstudied is one that places more emphasis on employmentstability than on pay and employee development. This factmay have heightened the salience of the due-processmanipulation for employees and strengthened its favorableeffects. Second, the employee control group displayed agreater dropout rate than did the due-process group.Although analyses of demographics and pre-surveymeasures suggest that this did not bias the results, thispossibility cannot be completely ruled out. Finally, it may bethat the study's pre-post research design may havesensitized participants to the purpose of the research or thatdifferences in the treatment of the due-process and controlgroups may have demoralized control group members.Analyses of the check variables, however, suggest thatneither sensitization nor demoralization were majordeterminants of the results presented here. Having identifiedpotential weaknesses in the research, we now examine itscontributions in greater depth.

Extension of Procedural Justice Research

Employees in the due-process condition of this studyperceived greater accuracy and fairness in the appraisalsystem and greater satisfaction with appraisals than didthose in the control group, even though they receivedsignificantly lower performance evaluations. Thus it appearsthat due-process features, including elements of proceduraland interactional justice, can increase employees' sense offairness about organizational processes (Lind and Tyler,1988),


Due Process

Surprisingly, the due-process appraisal system did not affectbroader job and organizational attitudes for employees,although such results have emerged from previous research,both lab experiments and field studies (Burke and Wilcox,1969; Lissak, 1983; Kanfer et al., 1987; Nathan, Mohrman,and Milliman, 1991; Moorman, 1992). It seems likely that inactual work settings, employees' reactions to their jobs andorganization are affected by many different human resourceprocedures, including performance appraisal. Employees inthis study may have been waiting to see whether changes inthe performance appraisal system would lead to other joband organizational changes.

It is also noteworthy that procedural justice effects onbehavioral variables, such as the motivation to improve,were nonexistent in this study, despite the fact that thevariables were chosen to minimize potential contaminantssuch as ability and opportunity. This finding strengthens Lindand Tyler's (1988) tentative conclusion that performanceincrements from enhanced procedural justice have been lessconsistent in cases of performance evaluation.Due-process-group employees, however, did tend to reportstronger behavioral intentions to remain in the organizationthan did the control group, a finding that replicates resultsfrom two earlier studies (Alexander and Ruderman, 1987;Tyler, 1990). In light of growing evidence of proceduraljustice effects on organizational citizenship behaviors(Konovsky and Folger, 1991; Moorman, 1992), we concludethat increased procedural justice in organizations is morelikely to affect the display of discrete behaviors reflectingloyalty and commitment to the organization than ongoing,day-to-day job performance.

Finally, due-process-group employees evaluated theirsupervisor's competence more favorably than didcontrol-group employees, despite receiving significantlylower evaluations. This result supports Tyler's (1990)proposal that a managerial strategy based on proceduraljustice will be more effective in maintaining employees'loyalty and commitment toward the work organization andits representatives than one based on distributive justice oroutcome favorability. A quote from one of thedue-process-group employees indicates that improvedcommunication may be responsible for more favorableevaluations of supervisors: "I like discussing theexpectations about what you have to do on your job aheadof time because supervisors tend not to tell you this. Withthe new method, you don't have any surprises, unlike withthe current system, and it tends to get rid of some of thepolitics."

Future research and theorizing are needed to integrate thegrowing number of conceptual models specifying thedeterminants of procedural justice. As noted in theintroduction, the due-process metaphor examined here isconsistent with different, yet similar formulations presentedby Thibaut and Walker (1975, 1978), Leventhal (1976), andTyler and Lind (1992). But, because clear statements of thesimilarities and differences in these models do not appear inthe literature, it is increasingly difficult to determine theconsistency of findings across different procedural justice


manipulations and to evaluate critically the contribution ofexisting theory.

Managers' Reactions to Increased Procedural Justice

Previous research on procedural justice has largely ignoredmanagers' reactions to human resource systems thatsafeguard employees' interests. This is unfortunate because,without managers' support and cooperation, it is unlikely thatemployees can experience true due process in organizations(Longenecker, Gioia, and Sims, 1987), Some researchers(e,g,, Pfeffer, 1981) have argued that multiconstituentdecision-making systems, such as the due-process onepresented here, cannot succeed because of differences inthe power, information, and negotiation skills of the twoparties. Others (e.g,, Folger, Konovsky, and Cropanzano,1992), however, have noted that a due-process system doesnot require the full equalization of power, only that managersremain open to employee input and be responsive to thereasoned questioning of standards and evaluations. Further,managers seem likely to benefit considerably from such asharing of power, since their own effectiveness is heavilydependent on the long-term efforts of their employees.

In this study, due-process-group managers reported greatersatisfaction with the appraisal system, a greater ability toresolve work problems, and less distortion of appraisalresults than did control-group managers. These resultssupport Folger, Konovsky, and Cropanzano's (1992)contention that managers can move beyond their ownself-interests to increase the procedural justice within theperformance appraisal process, and that, when doing so,they and the organization will experience many benefits.Nevertheless, two observations about this study seemappropriate. First, this research addressed concerns aboutunequal negotiation skills and information between theparties by training both in how to negotiate disagreementsand how to collect and document performance information.This training undoubtedly helped to overcome obstacles thattypically make the sharing of power difficult in organizations.Second, in this study, the procedural justice intervention didnor negatively affect managers' reward contingencies ordirectly threaten their self-interests; such effects would havemost certainly undermined the working of the due-processsystem.

Future research is needed to examine the conditions underwhich the more powerful organizational members, such assupervisors, are willing to accept and support proceduraljustice interventions that constrain their ability to determineunilaterally the outcome of work disputes. Such knowledgewill be very important to designers of organizationalprocedural justice systems. As Ury, Brett and Goldberg(1989: 74) stated, "Why should teachers participate in thedesign of a procedure that will make it easier for students toraise complaints about their grades? , . , Understandably,parties who are 'winning' with the current system may beunwilling to help change it," In this study, managers werewilling to help change the system and came to prefer it overthe previous system. As one manager put it, "I like thesystem and think it is a good one. It forces a manager to


Due Process

spend time giving employees the feedback they need andhelps them to communicate needed changes in prioritiesand performance expectations. Initially, having to setexpectations made my job harder but a lot ofmisunderstandings came out in our initial expectation settingmeeting and clearing these up then made my job easier."

This research supports the viability of the due-processmetaphor (Folger, Konovsky, and Cropanzano, 1992) as amechanism for increasing the contribution of performanceappraisal systems to organizational effectiveness. Thesystem studied here increased perceived appraisal accuracy,decreased rating inflation, and generated favorable reactionsfrom both managers and employees, despite requiringsignificantly more time and effort from them. These resultsare quite exciting, given that both research and practicesuggest that favorable reactions to performance appraisalsystems are rare. A due-process approach seems likely tofacilitate the design of appraisal systems that more fullyachieve their potential contributions to employeedevelopment, reward allocation, and legal documentation.Rather than elusive, the goal of designing an effectiveappraisal system could even be a realistic one.

REFERENCES

Alexander, Sheldon, and MarianRuderman1987 "The role of procedural and

distributive justice inorganizational behavior."Social Justice Research, 1:117-198.

Banks, Christine G., and Kevin R.Murphy1985 "Toward narrowing the

research-practice gap inperformance appraisal."Personnel Psychology, 38:335-345.

Bernardin, H. John, and PeterVillanova1986 "Performance appraisal." In

E. A. Locke (ed.). Generalizingfrom Laboratory to FieldSettings: 43-62. Lexington,MA: D. C. Heath.

Bies, Robert J., and J. S. Moag1986 "Interactional justice:

Communication criteria offairness." In R. J. Lewicki,B. H. Sheppard, and M. H.Bazerman (eds.). Research onNegotiations in Organizations,1: 43-55. Greenwich, CT: JAIPress.

Burke, Ronald J., and D. S. Wilcox1969 "Characteristics of effective

employee performance reviewand development interviews:Replication and extension."Personnel Psychology, 25:291-305.

Cleveland, Jeanette N., Kevin R.Murphy, and Richard E. Williams1989 "Multiple uses of

performance appraisal:Prevalence and correlates."Journal of Applied Psychology,74: 130-135.

DeNisi, Angelo S., and Kevin J.Williams1988 "Cognitive approaches to

performance appraisal." In G.Ferris and K. Rowland (eds.).Research in Personnel andHuman ResourceManagement, 6: 109-155.Greenwich, CT: JAI Press.

DIpboye, Robert L., and Rene dePontbriand1981 "Correlates of employee

reactions to performanceappraisals and appraisalsystems." Journal of AppliedPsychology, 60: 736-741.

Dorfman, Peter, W. G. Stephan,and J. Loveland1986 "Performance appraisal

behaviors: Supervisorperceptions and subordinatereactions." PersonnelPsychology, 39: 579-597.

Dunham, Randall B., and Frank J.Smith1979 Organizational Surveys.

Glenview, IL: Scott-Foresman.

Earley, P. Christopher1984 "Informational mechanisms of

participation Influencing goalacceptance, satisfaction andperformance." Unpublisheddoctoral dissertation.University of Illinois,Champaign.

Earley, P. Christopher, and E.Allan Lind1987 "Procedural justice and

participation in task selection:The role of control inmediating justice judgments."Journal of Personality andSocial Psychology, 52:1148-1160.

Folger, Robert1977 "Distributive and procedural

justice: Combined impact of'voice' and improvement onexperienced inequity." Journalof Personality and SocialPsychology, 35: 108-119.

Folger, Robert, and R. J. Bies1989 "Managerial responsibilities

and procedural justice."Employee Responsibilities andRights Journal, 2: 79-90.

Folger, Robert, and Mary A.Konovsky1989 "Effects of procedural and

distributive justice onreactions to pay raisedecisions." Academy ofManagement Journal, 32:115-130.


Folger, Robert, Mary A, Konovsky,and Russell Cropanzano1992 "A due process metaphor for

performance appraisal." InB. M. Staw and L. L.Cummings (eds.), Research inOrganizational Behavior, 13:129-177. Greenwich, CT: JAIPress.

Fulk, Janet, Arthur P. Brief, andStephen H, Barr1985 "Trust-in-supervisor and

perceived fairness andaccuracy of performanceevaluations." Journal ofBusiness Research, 13:299-313.

George, Jim1986 "Appraisal in the public

sector: Dispensing with thebig stick." PersonnelManagement, 18: 32-35.

Graen, George1976 "Role-making processes

within complexorganizations." In M. D.Dunnette (ed.). Handbook ofIndustrial and OrganizationalPsychology: 1201-1245.Chicago: Rand McNally.

Greenberg, Jeraid1986 "Determinants of Perceived

Fairness of PerformanceEvaluation." Journal ofApplied Psychology, 71:340-342.

Ilgen, Daniel R,, Cynthia D, Fisher,and M, Susan Taylor1979 "Consequences of individual

feedback on behavior inorganizations." Journal ofApplied Psychology, 64:349-371.

Kanfer, Ruth, John Sawyer, P,Christopher Earley, and E, AllanLind1987 "Fairness and participation in

evaluation procedures: Effectson task attitudes andperformance." Social JusticeResearch, 1: 235-249.

Karambayya, Rekha, and JeanneM, Brett1989 "Managers handling disputes:

Third-party roles andperceptions." Academy ofManagement Journal, 32:687-704.

Konovsky, Mary A,, and RobertFolger1991 "The effects of procedural

and distributive justice onorganizational citizenshipbehavior." A.B. FreemanSchool of Business WorkingPaper Series, 91-HRMG-Ol,Tulane University.

Leventhal, Gerald S,1976 "Fairness in social

relationships." In J. W.Thibaut, J. T. Spence, andR. C. Carson (eds.).Contemporary Topics in SocialPsychology: 211-240.Morristown, NJ: GeneralLearning Press.

Leventhal, Gerald S,, J, Karuza,and W, R, Fry1980 "Beyond fairness: A theory of

allocation preferences." In G.Miikula (ed.). Justice andSocial Interaction: 167-218.New York: Springer-Verlag.

Lind, E, Allan, and Tom R, Tyler1988 The Social Psychology of

Procedural Justice. New York:Plenum.

Lissak, Robert I,1983 "Procedural fairness: How

employees evaluateprocedures." Unpublisheddoctoral dissertation.University of Illinois,Champaign.

Lissak, Robert I,, and Blair H,Sheppard1983 "Beyond fairness: The

criterion problem in researchon dispute intervention."Journal of Applied SocialPsychology, 13: 45-65.

Locke, Edwin A,1976 "The nature and causes of job

satisfaction." In M. D.Dunnette (ed.), The Handbookof Industrial andOrganizational Psychology:1297-1350. Chicago: RandMcNally.

Longenecker, Clinton 0,, DennisA, Gioia, and Henry P, Sims1987 "Behind the mask: The

politics of employeeappraisal." Academy ofManagement Executive 1(3):183-193.

McFarlin, Dean B,, and Paul D,Sweeney1992 "Distributive and procedural

justice as predictors ofsatisfaction with personal andorganizational outcomes."Academy of ManagementJournal, 35: 626-637.

Meyer, Herbert1991 "A solution to the

performance appraisalfeedback enigma." Academyof Management Executive5(1): 68-76.

Moorman, Robert H,1992 "Relationship between

organizational justice andorganizational citizenshipbehaviors: Do fairnessperceptions influenceemployee citizenship?"Journal of Applied Psychology,76: 845-B55.

Mount, M, K,1983 "Comparisons of managerial

and employee satisfactionwith a performance appraisalsystem." PersonnelPsychology, 36: 99-110.

Murphy, K, R,, and Jeanette N,Cleveland1991 Performance Appraisal: An

Organizational Perspective.Needham Heights, MA: Allynand Bacon.

Napier, Nancy K,, and Gary P,Latham1986 "Outcome expectancies of

people who conductperformance appraisals."Personnel Psychology, 39:827-837.

Nathan, Barry R,, Allan M,Mohrman, and John Milliman1991 "Interpersonal relations as a

context for the effects ofappraisal interviews onperformance and satisfaction:A longitudinal study."Academy of ManagementJournal, 34: 352-369.

Nunnally, Jum C,1978 Psychometric Theory. New

York: McGraw-Hill.

Pfeffer, Jeffrey1981 Power in Organizations.

Marshfield, MA: Pitman.

Thibaut, John W,, and LaurensWalker1975 Procedural Justice: A

Psychological Analysis.Hillsdale, NJ: Eribaum.

1978 "A theory of procedure."California Law Review, 66:541-566.

Tyler, Tom R,1989 "The psychology of procedural

justice: A test of the groupvalue model." Journal ofPersonality and SocialPsychology, 57: 333-344.

1990 "Using procedures to justifyoutcomes: Managing conflictand allocating resources inwork organizations." DisputeResolution Research CenterWorking Paper, KelloggGraduate School ofManagement, NorthwesternUniversity.


Due Process

Tyler, Tom R., and E. Allan Lind Tyler, Tom R., and Randall Ury, William L., Jeanne M. Brett,1992 "A relational model of Schuller and Stephen B. Goldberg

authority in groups." In Mark 1990 "A relational model of 1989 Getting Disputes Resolved.Zanna (ed.). Advances in authority in work San Francisco: Jossey-Bass.Experimental Social organizations: The psychology valley Kathleen LPsychology, 25. 115-191. of Procedural justice." i989 "Rank and relationship effectsNew York: Academic Press. Unpublished manuscript, . ^^ procedural choice in

Amencan Bar Foundation, ^^^^..^^ management."

Dispute Resolution ResearchCenter Working Paper,Kellogg Graduate School ofManagement, NorthwesternUniversity.


Documents

Due Process in Performance Appraisal: A Quasi-experiment ...€¦ · It seems fair to conclude that many organizational appraisal systems have failed to realize their full potential