Rater perceptions, satisfaction and performance ratings

Rater perceptions, satisfaction and performance ratings

Cheri Ostroff* Indusirral Relatiotir Center. Ilnrverrity of Mmnetuta, 5 3 7 Manajiement~Ec~nomtrr Uldji< 27 I 19th Aiieniie J.

Minneupo/r$, MN 55455. USA

This study investigated relationships between raters’ perceptions of the purpose of rating ( a s opposed to objective purpose), norms about the rating process, satisLiction and rating outcomes. Sixty-four managers of 340 employees completed a survey of pcrccp- tions of purpose, norms and satisfaction. Performance rating data for subordinates of these managers were culled from files. Results indicated that raters’ perceptions and sat- isfacrion were significantly rclated to halo, Icnieocy and range restriction in ratings. Implications of these findings are discussed.

Models of performance appraisal processes and systems indicate that rater and ratee characteristics, interactions among raters and ratees, rating formats, raters’ cognitive processes, the rating process and rating context all influence rating behaviour and outcomes of ratings (e.g. DeNisi, Cafferty & Meglino, 1984; Feldman, 1981; Ilgen & Feldman, 1981; Landy & Farr, 1980; Murphy & Cleveland, 1991). Recently, i t has also been suggested that the rater’s definition of the performance appraisal situation will influence performance appraisal behaviour, including evaluations given, feedback given to others, withholding information, biasing, and so forth (Mohrman & Lawler, 1983). For example, i t has been suggested that raters’ perceptions of the uses of performance ratings and the norms regarding ratings influence rating behaviour and rating outcomes such as leniency and halo in ratings (Cleveland, Murphy & Williams, 1989; Landy & Farr, 1983; Murphy C(C Cleveland, 1991). It is likely that raters’ perceptions of the Performance appraisal context will result in systematic differences in the ratings provided.

Similarly, interest in the issue of affect in the rating process has been increasing (Dipboye, 1985; Dobbins & Russell, 1986; Murphy & Cleveland, 1991). Raters’ moods, temperament, dispositions, satisfaction, or other affective responses are believed to influence the ratings provided. For example, raters who are in good mood or in a positive affective state may be more lenient in their ratings (Williams, Alliger & Pulliam, 1988). However, little empirical attention has been devoted to examining whether raters’ definitions of the appraisal situation and affect are related to rating behaviours and rating outcomes. This study attempts to investigate the relationships between three categories of rater perceptions of the appraisal situation (perceived purpose, acceptance and norms),

* Rcqucsts frir rcprinrs.

346 Cheri Ostrojf rater job satisfaction, and rating outcomes (leniency and discrimination within and between people).

Rater perceptions

Mohrman & Lawler (1983) proposed a model of performance appraisal behaviour in which the organizational context and performance appraisal system influence the individual’s definition of the performance appraisal situation, which ultimately determines performance appraisal behaviour. The real existence of situational elements does not guarantee that they will be perceived and be part of the individual’s definition of the performance appraisal system. Similarly, the absence of situational elements does not guarantee that they will not be perceived. That is, raters’ perceptions may not fit the objective reality of the situation.

Raters may vary the ratings they provide depending on the performance appraisal context and on their goals and perceptions of the rating system and process (Murphy & Cleveland, 1991). Studies of raters’ perceptions of the rating context and system have focused primarily on factors such as frequency or degree of job knowledge that affect perceptions of the fairness and accuracy of the system, or perceptions of ratees such as feelings of likability (e.g. Cardy & Dobbins, 1986; Dipboye & depontbriand, 1981; Landy, Barnes & Murphy, 1978). Little work has examined raters’ perceptions of the purpose of appraisal, norms and feelings of acceptance and fairness as they relate to rating outcomes.

Rating purpose Performance appraisals provide information relevant to a number of organizational goals and purposes including salary increases, promotion recommendations, training and development and performance feedback (Jacob, Kafrey & Zedeck, 1980; Landy & Farr, 1983). Some of these may be explicitly defined by the organization, while others may be implicitly defined by the rater (Murphy & Cleveland, 1991). It is well known that purpose of ratings affects the way in which observers attend to, encode, store and retrieve information (Murphy, Philbin & Adams, 1989; Murphy, Garcia, Kerkar, Martin & Balzer, 1982; Williams, DeNisi, Blencoe & Cafferty, 1985; Williams, DeNisi, Meglino & Cafferty, 1986; Zedeck & Cascio, 1982). In addition, purpose of appraisal has been related to rating outcomes such as leniency and halo in ratings. Research has consistently demon- strated that ratings are more favourable or lenient and show less discrimination among ratees (range restriction) when the purpose of the appraisal is administrative (e.g. salary decisions, promotion decisions) than for research, feedback or other purposes (e.g. Bernardin, Orban & Carlyle, 1981; Dobbins, Cardy & Truxillo, 1988; McIntyre, Smith & Hassett, 1984; Zedeck & Cascio, 1982).

Raters may purposely distort their ratings based on the purpose or use of the ratings (Cleveland et af., 1989). Further, ratings are likely to reflect only some aspects of the judgemental process, such as differences between individuals or individual strengths and weaknesses (Murphy & Cleveland, 1991). Evidence suggests that performance ratings are often used for both within and between individual purposes (Cleveland et a/., 1989) and that when this is the case, raters tend to focus on only one set of comparisons (Cleveland, Morrison & Bjerke, 1986). Ratings that are recorded by supervisors will be driven by uses to which these ratings are applied. When the purpose is for salary administration or pro-

Rater perceptions 347

motion decisions, raters are likely to focus on overall performance of employees, rather than focus on the relative strengths and weaknesses of an individual as when the purpose is developmental feedback.

Nearly all of the studies involving rating purpose have either manipulated the purpose variable in a lab setting, or have compared different organizations or departments with a different rating purpose. Further, research has been restricted to a consideration of a single explicit purpose, typically contrasting ratings made for research or feedback purposes with those made for administrative purposes. In addition to the explicit purpose of appraisals in organization, ratings may be driven by implicit purposes as defined by the rater (Murphy & Cleveland, 1991; Landy & Farr, 1983). Thus it was hypothesized that systematic differences in ratings would emerge based on differences in raters’ perceptions of the purpose of ratings. More specifically, perceptions of administrative purpose should lead to more favourable ratings. Ratings made when the perception is for feedback purposes should exhibit less leniency and more discrimination between and within individuals. Perceptions of purpose for documentation or goal identification should lead to ratings with less leniency, more within-individual discrimination and less between- individual discrimination.

Norms Raters’ perceptions about the norms which govern rating behaviours may also influence rating behaviour (Bernardin et al., 1981; Mohrman & Lawler, 1983; Murphy & Cleveland, 1991). Here, norms refer to beliefs about which rating behaviours are acceptable and unacceptable. For example, some raters may believe that the expectation is to provide honest accurate appraisals, whereas other raters may believe ‘political’ factors operate, such as ratings should be given so as to maximize ratee rewards, or all ratees should receive high ratings (Longenecker, Sims & Gioia, 1987).

Managers may be more likely to manipulate employees’ ratings when political influences within the organization are operating. The social context or norms which develop surrounding rating behaviour may result in managers distorting ratings such as giving higher ratings to maximize merit increases or deflating ratings to shock subordinates into increasing performance. However, when the culture or expectations about performance ratings are such that the rating process is taken seriously and managers and subordinates trust the system, then ratings are less likely to be manipulated (Longenecker et al., 1987). For example, raters expressing trust in the appraisal process were less lenient in their ratings than those expressing lower trust (Bernardin et al., 1981). Further, in a survey of managers, Longenecker et al. found that managers most often inflate ratings for ‘political’ reasons. Thus, it was hypothesized that systematic differences in ratings would emerge based on the perceived norms regarding the rating process. Raters who believe that honest norms prevail will exhibit less leniency, while those who believe political norms prevail will exhibit more leniency and less between- and within-individual discrimination in ratings.

Reactions

Rater reactions to the performance appraisal system are important considerations in the effectiveness of the system. Raters who have negative reactions or feel the system is unfair

348 Cheri Ostrofl

are not likely to devote serious time and attention to their ratings and may be more likely to distort their ratings (Murphy & Cleveland, 1991). Most research on reactions has focused on the factors that contribute to the perceptions of fairness and acceptance of the appraisal system, such as frequency of appraisals, formality of the system, degree of job knowledge, relevance of performance dimensions, and participation (e.g. Dipboye & depontbriand, 1981; Landy et a/., 1978; Waldham, Bass & Einstein, 1987). Little work has focused on the relationship between raters’ acceptance and perceptions of fairness and the perceived purpose of ratings or rating outcomes. Thus, i t was hypothesized that raters’ acceptance and perceptions of fairness of the system would be positively related to perceptions of honest norms and negatively related to perceptions of political norms. Further, it was hypothesized that systematic differences in rating outcomes would emerge such that a greater degree of rater acceptance and perceived fairness would be related to less leniency and more discrimination between and within individuals.

Affect Recently, there has been a growing interest in the issue of affect in the rating process (Dipboye, 1985; Murphy & Cleveland, 1991). It is likely that the rater’s affective state, mood, temperament or general disposition will influence the rating process and rating outcomes. For example, a rater who is in a good mood at the time of the evaluation may give more positive ratings than one who is in a foul mood. Or, raters who have a positive disposition may evaluate performance differently than raters with a more negative disposition (Murphy & Cleveland, 1991).

Most performance appraisal research which has investigated affect has focused on liking and indicates that raters are more likely to give more favourable evaluations to individuals they like (e.g. Cardy & Dobbins, 1986; Dobbins & Russell, 1986). Positive affect has also been related to a greater degree of halo (Williams & Keating, 1987) and leniency in ratings, whereas negative affect has been related to harsher ratings (Williams eta/., 1988).

The precise explanation for such effects is unclear. While i t is likely that affect influences evaluations directly, it is also likely that affect may change the cognitive processes of raters, which in turn influence rating outcomes (Murphy & Cleveland, 1991). There is evidence that people in a positive affective state are more likely to classify more persons or objects into a given category than persons in a negative affective state (Sinclair, 1988). That is, positive affect may lead to ‘sorting’ more people into higher performance categories and to making fewer distinctions within and between people about their performance levels.

One type of affective variable which has not received much research in this context is overall job satisfaction. It is likely that raters’ overall affective responses to their jobs will influence the ratings they provide. Given the findings of previous studies (e.g. Sinclair, 1988; Williams et a/., 1988; Williams & Keating, 1987), i t was hypothesized that a greater degree of job satisfaction would be related to more leniency and less discrimination within (halo) and between individuals (range restriction) in ratings.

Research focus Raters’ perceptions, beliefs and definition of the rating situation are likely to influence

Rater perceptions 349

their rating behaviour (Mohrman 8r Lawler, 1983; Murphy & Cleveland, 1991). A study which examines rater perceptions of purpose or norms within a single organization in which the explicit purpose is stated to be the same across workgroups is limited in that the objective ‘reality’ is fixed. However, raters may differently perceive different purposes and norms. Given the fixed organizational purposes and norms, the variance in perceptions is likely to be small. Thus, the strength of the relationships between perceptions and rating outcomes may be attenuated when a single organization, as opposed to multiple organizations which differ in their rating purposes and norms, are investigated. However, if differences in perceptions are significantly related to rating outcomes in expected ways when the objective purpose does not vary, then we may conclude that perceptions do in fact influence rating outcomes, although the magnitude of the effects is most likely underestimated.

To summarize, the purpose of this research was to investigate the relationship between raters’ perceptions of rating purpose, their beliefs about the norms surrounding the rating process, their feelings of acceptability and fairness of the system, and the rating outcomes of leniency and the degree to which they discriminate within people (halo) and between people (range restriction) in their ratings. Specifically, i t was hypothesized that perceptions of administrative purpose would be related to a greater degree of leniency in ratings; perceptions of feedback purpose would be related to a lesser degree of leniency, halo and range restriction; and, perceptions of documentation and goal identification would be related to a lesser degree of leniency and halo, and more range restriction. With respect to rater perceptions about the norms of the rating process, i t was hypothesized that perceptions of honest norms would be related to a lesser degree of leniency in ratings, and perceptions of political norms would be related to a greater degree of leniency, halo and range restriction. Raters would believe the ratings process is fair and acceptable and would exhibit less leniency, halo and range restriction in their ratings. Finally, it was hypothesized that positive relationships between job satisfaction and leniency, halo and range restriction would be found.

Met hod Procediire A survey about perceptions of the performance appraisal process was distributed through company mail to 72 managers of 379 employees. They were asked to complete the survey and return it directly to the researcher in a postage-paid return envelope. In addition, performance ratings for the subordinates of these managers (which were completed one to three weeks prior to the survey) were culled from files.

In this organization, performance ratings are collected twice a year. The primary LISCS for performance appraisals are administrative decisions and developmentd feedback. This dual purpose may allow for managers to perceive one purpose to be of greater importance than another. Yearly salary increases are based directly on the ratings provided. In addition, supervisors provide feedback to employees about their performance. During the feedback session, supervisors and subordinates are expected to set new goals and objec- tives to be accomplished during the year, and rcview progress in meeting the past year’s goals.

Part icip~znts The sample consisted of managers from two departments (distribution and information systems) at a large mid-western recail chain. LJsable data were collected for 64 managers and performance ratings were available for 340 subordinatcs of these managers. Most (95 per cent) of the managers were white and 73 per cent were

350 Cheri Ostroff

male. They averaged 8.5 years (SD = 6.3) of work experience at the company and 3.4 (SD = 2.5) years in their current position.

Measures The performance appraisal survey was designed to measure managers’ perceptions of the performance appraisal system, including purpose, usefulness of the information, acceptability, fairness and norms regarding rating behavior.

Perreiuedpurpose. Four categories of rating purpose and usefulness of rating information were assessed with 14 items based on Cleveland et al. (1989). The administrative category was assessed with four items including determining salary, merit increases, promotion decisions, and which employees are not performing well. The feedback and development category was assessed with five items including providing feedback to employees about their performance, identifying strengths and weaknesses, and determining which individuals need training. The documentation/legal category consisted of two items about providing documented evidence of performance and decisions, and meeting legal guidelines for documenting performance. The goal category was assessed with two items including identifying new departmental goals and determining if departmental goals are met. The managers were asked to read each of the 14 items, and first indicate the extent to which performance ratings at the company are used for this purpose and second, to indicate whether the performance ratings were effective in providing useful information about that area. Ratings were made on a five-point Liken-type scale ranging from no or little extent (1) to a very great extent ( 5 ) . Item responses were averaged to obtain eight scale scores-four scale scores for perceived purpose of appraisal (administration, feedback, documentation, goals) and four for providing useful information.

The correlations between like dimensions of purpose and information provided were quite high, ranging from .7 1 to .86; thus commensurate dimensions of purpose and information provided were averaged to yield four dimensions. A factor analysis confirmed the four dimensions of purpose. Intercorrelations among the dimensions were fairly low, ranging from .2O to .34. Internal consistency reliabilities were .SO for administration, .84 for feedback, .83 for documentation and 3 5 for goals.

Reactions One item measured the acceptability of the performance appraisal system. Raters were asked to indicate on a five-point scale the extent to which they found the appraisal system very unacceptable (1) to very acceptable ( 5 ) . In addition, raters indicated the extent to which they felt the performance appraisal system was fair, using a five-point scale ranging from no extent (1) to a great extent ( 5 ) .

Norms Two categories of norms regarding rating behaviour were assessed with eight items. All items were framed in the form of departmental expectations, i.e. in this department, it is expected that . . . . Three items assessed ‘honest’ norms, such as employees will receive honest and accurate feedback about performance, performance ratings will be taken seriously by both employees and management. Five items measured ‘political’ norms, including managers give employees higher evaluations than they deserve to maximize rewards, and evaluations are based on how much the employee needs the rewards. Raters indicated their agreement with each expectation using a five-point scale ranging from strongly disagree (1) to strongly agree (5). Internal consistency reliabilities were .65 for honest norms and .62 for political norms. The correlation between the two dimensions was - . 5O.

Satis faction Overall job satisfaction was measured with five items based on Hackman & Oldham (1975). A five-point Likert-type scale ranging from strongly disagree (1) to strongly agree (5) was used for all items. Internal consistency reliability was .86.

Rater perceptions 35 1

Rating outcomes

The performance rating forms consisted of seven dimensions, but the dimensions differed slightly by department. A definition of each dimension was provided on the form, and examples of performance were provided for the high, middle and low scale points for each dimension. Raters made ratings for each dimension using a seven-point scale ranging from unsatisfactory (1) to superior (7). The ratings were first standardized within each of the two departments due to the differences in format and to account for any potential overall departmental differences. An overall performance measure was computed by taking the average across dimensions for each ratee.

Three rating outcome variables were derived for each rater. For each manager, leniency (Saal, Ilowney & Lahey, 1980) was measured by averaging the overall ratings across subordinates. High scores indicate more lenient ratings. Halo or the extent of discrimination within individuals was assessed by first standardizing the ratings within each rater for each dimension, then taking the standard deviation across rating dimensions for each ratee and finally averaging across ratees for each manager (Pulakos, Schmitt & Ostroff, 1986). Higher scores indicate less halo or more discrimination within individuals. Range restriction (Saal et a/., 1980) or discrimination between ratees was computed by taking the standard deviation for the overall ratings across ratees within the manager’s group. Intercorrelations among the rating outcome variables were low, ranging from -.07 to - .12.

Results

The means, standard deviations and intercorrelations among all measures are presented in Table 1. It was first hypothesized that systematic differences in rating outcomes would emerge based on raters’ perceptions about the purpose of ratings. Perceived purpose of ratings was significantly related to several rating outcome variables. As expected, when raters perceived the purpose to be more administrative, they were more likely to be lenient in their ratings. However, contrary to expectations, perceptions of a feedback purpose were related to a greater degree of leniency and perceptions of a goal purpose were related to less leniency in ratings. A lesser degree of halo in ratings (more discrimination within individuals) was significantly related to perceived purposes other than administration, as expected. Finally, raters who were more likely to perceive the purpose as documentation or goal identification were more likely to show range restriction or show less discrimination among ratees, in support of the hypothesis.

Consistent with the hypothesis, raters who felt the system was fair and acceptable exhibited less halo in their ratings. However, in contrast to expectations, raters who felt the system was fair and acceptable were more likely to provide more favourable ratings (leniency). Raters who believed the norms about rating behaviour were honest tended to be more lenient and exhibited more halo in their ratings while those who felt the norms were more political in nature exhibited less halo and more range restriction (i.e. more within-people and less between-people discrimination).

General affective feelings about the job were also purported to influence rating outcome variables. In support of the hypothesis, raters who were more satisfied overall with their job were more likely to be lenient in their ratings and discriminate less between people (more range restriction). The results for halo were not significant.

Several correlations among perceptions and beliefs are worth mentioning. Perceived purpose of rating was significantly related to beliefs about the acceptability and fairness of the rating system. In particular, raters who perceived the purpose to be administrative or feedback to a greater extent were more likely to feel that the system was an acceptable and fair one. Perceived purpose was also related to beliefs about rating norms. Perceptions

Table 1. Means, standard deviations and correlations of perceptions, satisfaction and rating outcomes (N = 64)

Purpose Administration Feedback Documentarion Goals

Acceptance Fairness

Honest Polirical

Affecr Satisfaction

Raring ourcomes Leniency Halo Range restriction

Reactions

Norms

Mean SD _ _ _ ~ -

3.37 0.58 3.21 0.56 2.82 1.00 2.67 0.91

2.67 0.92 2.45 0.95

4.03 0.53 2.02 0.56

3.87 0.58

0.04 0.62 0.41 0.22 0.88 0.43

Admin Feed Docum Goal

.80

.34 .84

.29 .20 .83

.11 .31 2 3

.35 .21 -.12

.27 .30 -.13

.23 .46 -.03 -.14 -.17 .04

.14 .30 -.12

.23 .21 .04 -.l8 .21 .29

.08 -.02 -.21

.85

-.lo - .08

-.08 .28

.20

-.21 .26

- .20

Accept Fair Honesr Poliric Sat Lenienr Halo Restricr ~- ~~ ~ ~

- .64

.31 -.22

.16

.30 -.30

.24

.48 .65 -.33 -.50 .62

. l I .24 -.06 .86

.20 .25 -.18 .23 -

.31 -.01 .08 - -.28 -.32 -.02 -.O2 -.2O - 2 1 -.12 p.69 -

Note. Ratings were standardized by department before computing rating outcome variables. Low scores for halo and range restriction indicate greater amounts of these variables. Diagonal elements are alpha coefficients. Correlations greater then .20 are significant arp 5 .05.

Rater perceptions 3 5 3

of administrative or feedback purposes were significantly related to perceptions of honest norms regarding the rating process, while raters who perceived the purpose to be one of goal identification and achievement felt the norms and expectations regarding appraisals were more political in nature. Norm beliefs were also significantly related to acceptabil- i t y and fairness of the system. More specifically, more honest norms were related to more acceptance and perceived fairness of the appraisal system; raters who felt the norms were more political were less likely to feel the appraisal process was acceptable and fair.

Discussion

This study investigated the relationships between raters’ perceptions about the purpose of appraisal, norms regarding rating behaviours, reactions to the appraisal system, job satisfaction and rating outcomes. Systematic differences in rating outcomes were generally related to perceived purpose of ratings in expected ways, while results regarding norms of the rating process and acceptability of the system were weaker and were often contrary to expectations.

Relationships between perceived purpose of ratings and rating outcomes indicated that when the purpose was perceived to be administrative, raters were more lenient in their ratings. Surprisingly, perceptions of feedback as the purpose were also related to leniency in ratings. This may be due to the fact that, in this organization, both administration and feedback were the explicit purposes of appraisals. Hence, raters may have focused more on one purpose, such as administration (Cleveland et a/., 1986) when making their ratings. Results regarding halo in ratings were consistent with expectations. Less halo was evi- denced the more the perceived purpose was feedback, documentation or goal identification. Similarly, when raters believed the purpose was focused more on documentation and goal identification, they did not discriminate as much between employees in their overall performance.

One explanation for these results is that rating errors reflect raters’ conscious decisions to distort ratings based on their perceptions of rating purpose (Murphy c(r Cleveland, 1991). For example, i t has been suggested that discrepancies between raters’ judgements about subordinates’ level of performance and the actual ratings managers provide may be due to the fact that raters use the performance appraisal as a tool to achieve their own goals. For example, leniency in ratings has been attributed to the fact that raters want to avoid negative consequences associated with giving poor ratings or because they do not know how to counsel employees who receive poor ratings (e.g. Bernardin & Buckley, 1981; Latham, 1986; Napier c(r Latham, 1986). Thus there may not be a link between performance ratings and true performance. Katers’ beliefs about the purpose of rating may be one factor which causes raters to ignore performance and make decisions based on other factors (Murphy & Cleveland, 1991).

On the other hand, the rating process may not always be rational and may not always reflect conscious decisions on the part of the rater. The small amount of previous work regarding norms or the rating environment suggests that the more raters trust the appraisal system, the less lenient they are (Bernardin et a/., 1981) and the more political the culture about appraisals, the more likely raters are to distort ratings (Longenecker et a/ . , 1987). Similarly, i t would be expected that raters who find the system acceptable and fair would be less lenient and more likely to make distinctions within and between

354 Cheri Ostroff

people. Surprisingly, the results found here were generally in opposite directions. It is plausible that these measures of norms and reactions represented a more general affective response of raters and that ratings may be distorted simply due to the affective state of the rater. That is, raters who believed in the system, and felt the norms were to be accurate and honest in appraisals, had a more general positive affective response to the rating process. Raters’ affective states have been shown to be related to rating behaviour and studies indicate that a more positive affect state is related to greater leniency and less discrimination between people in ratings. The findings from this study relating overall satisfaction to rating outcomes also support this notion.

It is also possible that the political and honest norm measures did not capture the political influences on ratings. The mean score for the honest norm measure was quite high (coupled with a low standard deviation), while the mean for the political measure was low. Hence, i t may be that the raters simply provided ‘socially desirable’ responses to these measures, even though political factors were indeed operating. For example, Longenecker et al. (1 987) found that ratings tend to be distorted upward for political reasons, particularly when the purpose of appraisal is for administrative uses. Thus, it may be that the findings here for perceived administrative purpose are capturing, in part, political influences regarding the appraisal process. More research is needed to fully test this notion.

Findings from this study support the notion that raters’ own perceptions about the purpose have an effect on their rating behaviours. What is interesting in this study is that the explicit purpose of ratings did not vary; however, raters did vary in their implicit purposes and systematically differed in their rating behaviour based on their perceptions of the purpose. Although the majority of the correlations were somewhat low in magnitude, this is to be expected given the fact that explicit purpose did not vary, variation in perceived purpose was fairly small, and perceived purposes closely matched explicit purposes. Future work should investigate relationships between perceived purpose and rating outcomes across organizations which vary in their explicit purposes. In this case, stronger relationships may be found. Similar arguments can be applied to findings for different norms about the rating process. Different results may emerge across different organizations with different norms and purposes.

Further, in organizations, ratings are often used for multiple purposes, rather than for one purpose (Cleveland et al., 1989) and the purposes of the appraisal often involve multiple and conflicting goals (Kane & Lawler, 1979), such as for making promotion-related decisions and also as a tool for guiding employee development. In such a case, raters must make distinctions between individuals (for promotion) and within individuals (for development). This is likely to be a source of conflict for raters (Cleveland et al., 1989). In this study, there were multiple explicit purposes for ratings which could have caused conflicts for raters, It has been suggested that the ratings collected for multiple purposes may be different than those collected for any single purpose (Cleveland et al., 1989; Murphy & Cleveland, 1991). Therefore, more work is needed to further investigate the effect of multiple explicit and implicit purposes versus single purposes on rating behaviour.

It should also be noted that the rating outcome variables, although similar to those used in other studies (e.g. Bernardin et al., 1981; Dobbins et al., 1988) are not without problems. In particular, it is possible that these measures, particularly leniency and range restriction, partially reflect real differences in workgroup performance, as opposed to

Rater perceptions 355 ‘error’. Likewise, the reliabilities of the norm variables were somewhat low, and the reliabilities of the fairness and acceptable measures are not known as these variables were each assessed with one item. Thus, the results of the study should be interpreted in light of these issues. Further, in this study the rating format was a behaviourally anchored scale. Future work should investigate the extent to which the findings of this study generalize to other types of rating formats. Future work should also include additional aspects of appraisal purpose. For example, in some companies, managers’ assessments of their subordinates are scrutinized by their superiors and used to to evaluate managers’ performance. Thus, the appraisal takes on an additional purpose in such a case and managers may distort their ratings based on this purpose. Further, it is important to study how much of an effect raters’ perceptions have on rating outcomes as opposed to other rating context effects such as workgroup characteristics, frequency of ratings and climate.

References

Bernardin, H . J . & Buckley, R. W. (1981). Strategies in rater training. Aca&y of Management Review, 6, 205-212.

Bernardin, H . J., Orban, J. & Carlyle, J. (1981). Performance ratings as a function of trust in appraisal and rater individual differences. Proceedings of the 41sr Annual Meeting of the Academy of Manafieement, 311-314.

Cardy, R. L. & Dobbins, G. H . (1986). Affect and appraisal accuracy: Liking as an integral dimension in evaluating performance. Psychology, 7 1, 672-678.

Cleveland, J. N., Morrison, R. & Bjerke, D. (1986). Rater intentions in appraisal ratings: Malevolent manip- ulation o r functional fudging. Presented at the First Annual Conference of the Society for Industrial and Organizational Psychology, April, Chicago.

Cleveland, J. N, Murphy, K. R. &Williams, R. E. (1989). Multiple uses ofperformance appraisal: Prevalence and correlates. Journal of Applied Psychology, 74, 130-1 35.

DeNisi, A. S., Cafferty, T. & Meglino, B. (1984). A cognitive view of the performance appraisal process: A model and research implications. Organizational Behavior a n d Human Performance, 33, 360-396.

Dipboye, R. L. (1985). Some neglected variables in research on discrimination in appraisals. Academy of Managmrent Review, 10, 1 l ( ~ 1 2 7 .

Dipboye, R. L. & depontbriand, R. (1981). Correlates of employee reactions to performance appraisals and appraisal systems. Journal of Applied Psychnlogy, 66, 248-25 1.

Dobbins, G. H., Cardy, R. L. & Truxillo, D. M. (1988). The effects of individual differences in stereotypes of women and purpose of appraisal on sex differences in performance ratings: A laboratory and field study.

Journal of Applied Psychology, 73, 5 5 1-558. Dobbins, G. H. & Russell, J. M. (1986). The biasing effect of subordinate likableness on leaders’ reactions to

poor performers: A laboratory and a field study. Persnnnel Psychofogy, 39,759-777. Feldman, J. M. (1981). Beyond attribution theory: Cognitive processes in performance appraisal. Journal of

Applied Psychology, 66, 127-148. Hackman, J. R. & Oldham, F. R. (1975). Development of the job diagnostic survey. Journal of Applied

Psychology, 60, 159-1 70. Ilgen, D. R. & Feldman, J. M. (1983). Performance appraisal: A process focus. In L. L. Cummings & B. M.

Staw (Eds), Research in Organizational Behavior. Greenwich, C T JAI Press. Jacobs, R., Kafrey, D. & Zedeck, S. (1980). Expectations of behaviorally anchored rating scales. Personnel

Psychology, 33, 595-640. Kane, J. S. & Lawler, E. E. 111 (1979). Performance appraisal effectiveness: Its assessment and determinants.

In B. M. Staw (Ed.), Research in Organizational Behaviw. Greenwich, C T JAI Press. Landy, F. J., Barnes, J. L. & Murphy, K. R. (1978). Correlates of perceived fairness and accuracy of performance

appraisals. Journal of Applied Psychology, 63, 75 1-754.

356 Cheri Ostroff

Lanrly, F. J. & Farr, J. L. (1980). Performance rating. Psychological Bufletin, 87, 72-107. Landy, F. J. & Farr, J . L. (1983). ?‘he Meusuretnent of Work Perfhrmance: Methods, Theory and AppLicutions. New

York: Academic Press. Latham, G. (1986). Job performance and appraisal. In C. Cooper & I. Robertson (Eds), Intwnationul Review (4.

1 ndi&rial and Orgunizutional Psychology. C h iches te r, England: Wi ley. Longenccker, C. O., Sims, € I . P. & Gioia, I). A. (1987). Behind the mask: the politics ofemployee appraisal.

Academy ofManagenient Executive, 1, 183-193. Mclntyre, K . M., Smith, D. & Hassett, C. E. (1084). Accuracy of performance ratings as affected by rater

training and perceived purpose of rating.Journul o/ApplIedP.ryl.hoIo~y, 69, 147-1 56. Mohrman, A. M., Jr & Iawler, E. E. I11 (1983). Motivation and performance appraisal behavior. In F. Landy,

S. Zedeck & J. Cleveland (Eds), PrvJi,rtnuncr Meusurenzent and Theory. I3illsdale, NJ: Erlbaum. Murphy, K. R . & Cleveland, J. N . (1991). Perfownunre Appraisal: An Organizutionul Perspective. Boston: Allyn

& Bacon. Murphy, K. R., Garcia, M., Kerkar, S., Martin, C . & Balzer, W. K. (1982). Relationships between observa-

tional accuracy and accuracy in evaluating pcrformance.Jowrnuf of Applird Psychology, 67, 320-325. Murphy, K. K . , Philhin, T. A. & Adams, S. R. (1989). Effect of purpose of observation on accuracy of

immediate and delayed performancr rarings. Orpnizurionaf Behatiior und tlimm Decision Proce.rses, 43, .3 3 6-3 54.

Napier, N. d Latham, G . ( I 986). Outcome expectancies of people who conduct performance appraisals. Personnel Psychology, 4 1, 5 17-5 35.

Piilakos, E. D., Schmitt, N . & Ostroff, C. (1986). A warning about the use of a stanrlarrl deviation across dimensions to measure halo.Journal of Applird Prychology. 7 1, 29-32.

Saal, F. E., Downey, R. G. & Lahey, M. A. (1980). Kating the ratings: Assessing the quality of rating data. PsycholoKicul Bidletin, 88, 4 13-428.

Sinclair, R.C. (1988). Mood, categorization, breadth, and performance appraisal: The effects of order of information and affective state on halo, accuracy, information retricval and evaluations. Organizutionuf Behazhr and Hiinun Decision Prmemr, 42, 22-46,

Waldham, D. A., Bass, B. M. & Einstein, W. 0. (1987). Leadership and outcomes of the performance appraisal process.Joimrul of Oiwpational Pqdmlogy, 60, 177-186.

Williams, K. J., Alliger, G . M. & Pulliam, R. (1988). Rater affect and performance ratings: Evidence for the moderating effects of rater perceptions. Presented at the Third Annual Conference of the Society of Industrial and Organizational Psychologists, April, Dallas.

Williams, K. J., DeNisi, A. S., Blencoe, A. G. & Cafferty, T. P. (1985). The role ofappraisal purpose: Effects of purpose on information acquisition and utilization. Organizationuf Behuuior and Hurnun Pwfhrttzunce, 35, 3 1 4-3 39.

Williams, K. J., DeNisi, A. S., Mcglino, B. M. & Caffcrty, T. P. (1986). Initial decisions and subsequent performance ratings.journul of Applied Psychology, 71, 189-195.

Williams, K. J. & Keating, C. W. (1987). Affect and che processing of performance appraisal. Presented at the Second Annual Conference of the Society for Industrial and Organirational Psychologists, April, Atlanta.

Zetleck, S. & Cascio, W. F. (1982). Performance appraisal decisions as a function of rater training and purpose of the appraisal. Journuf ofApplied Psychology, 67, 752-758.

Rereiued 16 July 1992; revised ver.rion received 5 .Junuury I093

Documents

Rater perceptions, satisfaction and performance ratings