When Workers Rate the Boss

When Workers Rate the Boss

By J. C . RUPE, Human Resources Research Center, USAFl

SUMMARY

T H E evaluation of personnel in relation to the success of their organizations is an important and well recognized problem. Much study has been made of the rating of the great body of workers, but what constitutes a good or poor executive has had little if any serious attention as a scientific problem. This is not to say that there is not a mass of literature regarding the executive nor that it is not of value. However, nearly the whole of it is the product of experience on the job and casual observation. Such writings may very well be valid, but as science they have, at best, the status of plausible hypotheses to be tested rather than the status of being offered as proved formulae for making poor executives good and good ones better.

The present study is one attempt to examine presumably relevant aspects of executives. It is perhaps unique in that it is concerned with a “worm’s-eye” view of executives-an appraisal of their characteristics by subordinates who know them and directly experience their administrative procedures. This information for the executive about himself may be used for self-improvement or for the improvement of understanding between himself and his employees.

J . C . Rupe has been a project director in the Job Analysis Division of the Technical Training Research Laboratory of the Human Resources Research Center, Chanute Air Force Base, Illinois, since receiving his Ph.D. from Purdue University in February, 1960. He was an instructor i n psychology at Carnegie Institute of Technology for two years and taught at Purdue while completing his degree. He i s a member of the American Psychological Association, Division of Industrial and Business Psychology.

This is a summary of a doctoral dissertation accomplished under the direction of H. H. Remmers and on file in the library at Purdue University. Its original title was “Some Psychological Dimensions of Business and Industrial Executives.”

271

272 J. C. RUPE

The Purdue Rating Scale for Administrators and Executives2 was used to measure executives’ effectiveness. A further purpose of the study was to evaluate this scale and to compare the findings with a prior study conducted with academic administrators of institutions of higher learning. (4) The results indicate that it is possible to measure the traits of business and industrial executives by means of a subordinate-executive rating scale. The Purdue Rating Scale for Administrators and Executives has demonstrated this possibility with acceptably high reliabilities and satisfactory evidence of validity. Halo effect does not appear to be of major importance. Two factors common to various traits of business and industrial executives can be measured. These factors, as determined by this study, are (1) social responsibility to subordinates and society and (2) executive achievement. As measured by this scale, the similarities between executives and academic administrators appear to be much greater than their dissimilarities.

This scale does not give all the answers that might be desired concerning executive ability. It does provide an executive who wishes it an honest and anonymous appraisal of his effectiveness as seen by his subordinates. It permits him to compare these appraisals with those of other executives. It provides a means of arriving at information which will highlight areas of strength and weakness which are difficult to survey in any other way.

INTRODUCTION A survey of the literature indicates that most interest has

been in the identification of universal traits of leadership or executive ability. Such identification is not the purpose of this paper, but rather it seeks a means of accurate evaluation of the leader in terms of his interaction with other persons within his sphere of influence. Such an evaluation may or may not be of direct usefulness to the individual’s superiors. For reasons dis-

2 Published by the Division of Educational Reference, Personnel Evaluation Re- search and Service, Purdue University, Lafayette, Indiana.

WHEN WORKERS RATE THE BOSS 273

cussed later, the executive’s rating by his subordinates may be withheld from his superiors, but it may be of inestimable value to the individual himself and indirectly to his group by enabling him to identify and to do something about the areas of strength and weakness in his total make-up.

The weight of the evidence reviewed would seem to justify the following conclusions :3

1. Executive leadership is not the mere possession of some combination of traits. It is rather a working relationship among members of a group striving toward mutual goals. Success is determined on the basis of many of the attitudes and values generally accepted by middle-class American society.

2. The accurate measurement of the attitudes of the working group would do much to indicate the effectiveness of its leadership.

3. Rating scales are probably most commonly used in atti- tude measurement but, as usually used in business and industrial evaluation, they have suffered from errors due to “halo” effect and too few raters being qualified by training, actual knowledge of the work of the ratee, and sufficient time in which to do the rating. 4. As demonstrated by teacher and academic administrator

evaluation by students and subordinates, the usual errors in the use of rating scales are minimized. The results attained may be used by the ratee for self-improvement and/or, by appro- priate educative means, the development of better understanding of him and his procedures among his raters.

Bittner (1) reports that people prefer to be rated by their supervisors rather than by their subordinates. No executive, of course, can avoid being rated. His colleagues constantly judge him. His only choice is whether or not he wants to know what these ratings are. Many hypotheses may be drawn for this preference to be rated by supervisors. The ratee has been judged by his superiors, parents, teachers, etc., been advised

3 For the review of the literature see the original dissertation on file in the Purdue University Library.

274 J. C. RUPE

of these evaluations, and become accustomed to the practice. There are many reasons, however, why more valuable information may be gained by deliberately measuring the attitudes of the subordinates. The amount of the executive’s time on the job spent with supervisors is far less than that spent with his employees. He will tend consciously to conduct himself so as to favorably impress his boss, thus assuring, in his own mind, a good rating, but his unconscious behavior traits will often be demonstrated for his employees. He may be so unaware of these traits that he cannot identify some of the ways in which he is either positively or negatively affecting his employees. They also tend to show their most agreeable sides to him, clouding his accurate estimate of their attitudes toward him and preventing his identification and evaluation of these traits. The question, “What do they really think of me?” may be very threatening to the individual with basic feelings of insecurity. Whether such persons should be in executive posi- tions is beside the point. Most people have such feelings in some areas. The point is, however, that if subordinate ratings are to be used most beneficially, considerable caution must be exercised in the form of information to the persons concerned as to their purpose and the use to be made of them. This must be done if feelings of job security are not to be completely undermined by suspicions that the big boss is trying to spy or to clean house. Assurance must be given that this is not the case and that results are to be exclusively for the ratee for such use as may seem desirable to him.

Education of the raters in the program is also necessary. It is reasonable to assume that anonymity must be assured to avoid the same suspicions and to obtain honest judgments.

If properly conducted, subordinate-superior ratings can make a definite contribution to all concerned. When an executive shows his subordinates that he is interested in their honest opinions of him for his own improvement and convinces them that he has no ulterior motives, improved morale and more hearty cooperation toward attaining mutual goals are most likely to result.


PROCEDURE The Purdue Rating Scale for Administrators and Executives

by H. H. Remmers and R. L. Hobson was used for this study. It is composed of 36 items, each of which is arranged with five possible quantitative responses. Five represented the greatest amount of the characteristic and one the least. The items are grouped into ten logical groups as follows:

I Intellectual Balance I1 Emotional Balance

I11 Leadership IV Planning

VI Capacity for Work V Use of Funds

VII Accomplishment

IX Public Relations VIII Relations with Subordinates

X Social Responsibility A guarantee of anonymity was made to each executive who

participated. Assurance was given that his ratings would be released to him alone and that any publication of the results of the study would be in such form as to make identification of any ratee impossible. This was assumed most likely to en- courage more active participation in the project.

Each executive distributed a rating scale, an IBM mark- sensing card, and an electrographic pencil to each subordinate who was to rate him. A stamped envelope, addressed to the investigator, was also provided. The rater did not put his own name on the rating materials and, since he mailed them directly, his anonymity was also assured.

It was not expected that a random sample of the popula- tion of executives could be obtained. Voluntary participation was requested. This resulted in a self-selected sample. The findings, then, are descriptive of this particular group of executives who wished to know how they were rated by their subordinates.

The sample used resulted from a solicitation by mail and personal contact. It was composed of 133 executives rated by

276 J. C. RUPE

702 subordinates who were associated with eight organizations from widely scattered points throughout the United States. These were a large national bank, a wholesale and re- tail merchandising institution, a manufacturer of structural glass, a manufacturer of airplane parts and accessories, a manufacturer of machine equipment, a manufacturer of valves and castings, a manufacturer of chains, and a manufacturer of chemical products. The executives ranged in rank from presidents down to, but not including, foremen.

The 133 executives were rated by from one to seventeen direct subordinates, with a mean number of 5.3 raters. Nineteen had insufficient or incomplete responses and these were not used in the statistical analysis of the scale.

When the ratings had been received, the mark-sensing cards were processed and tabulations were prepared by IBM equipment. Average ratings on each item for each executive were then calculated and percentile norms on each item computed from them. Table 1 presents these norms.

A profile sheet was then prepared so that each executive could draw his profile and see from it those traits in which he was relatively strong or weak in comparison with other executives. A report of the ratings as well as the average rating on each item was included with this profile sheet, and this, with a covering letter of explanation, was sent to each executive.

A statistical evaluation of the scale was made and a factor analysis performed. A comparison of the results of the analysis was made with those of a prior study conducted by Hobson (4) with academic administrators of institutions of higher learning.

STATISTICAL CHARACTERISTICS OF THE SCALE ReZiabiZity of the Items. For obvious reasons it was not con-

sidered possible in this situation to use a test-retest method for determining the extent to which an executive might expect to obtain about the same score on subsequent ratings. The method chosen for determining reliability was analogous to the common “split-half” technique. Each test for each executive was,


TABLE 1 Percentile Norms on the Purdue Rating Scale f o r Administrators and Executives

ITEM NO.

1 2

3 4 5 6

7 8 9

10 11 12 13 14

15 16 17

18 19 20

21 22 23

24 25 26 27 28 29 30 31 32 33

34

35 36

0

3.0 3.0

3.0 3.0 2.0 2.8

2.0 2.6 2.7

2.5 2.7 3.0 2.8 2.8

3.0 2.0 2.0

3.3 2.5 3.0

3.0 3.0 3.0

2.4 3.0 2.7 3.0 3.1 2.0 3.3 2.6 3.3 2.4

2.8

2.8 3.0

10

3.9 4.0

3.8 4.0 2.7 3.3

2.9 3.8 3.5

3.6 3.7 3.5 3.8 3.8

4.0 4.0 3.4

3.8 3.7 4.0

4.0 4.0 4.0

3.0 4.2 3.4 3.5 4.0 3.7 4.0 3.8 4.2 3.5

4.0

3.5 4.0

20

4.0 4.3

4.0 4.2 2.9 3.7

3.3 4.0 3.9

3.8 3.8 3.8 3.9 4.0

4.3 4.1 3.7

4.1 3.9 4.1

4.1 4.1 4.1

3.3 4.4 3.7 3.8 4.3 4.0 4.1 4.0 4.3 3.8

4.2

3.8 4.2

30

4.2 4.4

4.1 4.3 3.0 3.8

3.5 4.2 4.0

4.0 3.9 4.0 4.0 4.2

4.4 4.3 3.9

4.5 4.0 4.3

4.3 4.3 4.3

3.7 4.5 3.9 4.0 4.6 4.2 4.3 4.2 4.5 3.9

4.3

3.9 4.4

PEBCENTILES

40

4.2 4.5

4.2 4.4 3.3 4.0

3.7 4.3 4.3

4.2 4.3 4.1 4.2 4.3

4.5 4.4 4.0

4.6 4.3 4.4

4.4 4.3 4.4

3.8 4.7 4.0 4.1 4.7 4.3 4.5 4.3 4.6 4.0

4.4

4.0 4.5

50

4.3 4.6

4.3 4.5 3.5 4.3

3.8 4.5 4.4

4.3 4.4 4.3 4.3 4.4

4.6 4.5 4.2

4.7 4.4 4.5

4.5 4.5 4.5

4.0 4.8 4.2 4.3 4.8 4.5 4.7 4.4 4.7 4.1

4.5

4.2 4.7

60

4.4 4.7

4.4 4.6 3.7 4.4

4.0 4.5 4.5

4.4 4.5 4.4 4.4 4.5

4.7 4.6 4.3

4.8 4.5 4.6

4.6 4.6 4.7

4.0 4.9 4.3 4.4 4.9 4.6 4.8 4.5 4.8 4.3

4.7

4.3 4.8

70

4.5 4.8

4.5 4.7 3.8 4.5

4.1 4.7 4.6

4.5 4.7 4.5 4.5 4.7

4.8 4.7 4.4

4.9 4.7 4.7

4.7 4.7 4.8

4.3 4.9 4.4 4.5 4.9 4.8 4.8 4.7 4.9 4.4

4.8

4.4 4.9

80

4.6 4.9

4.7 4.8 4.0 4.6

4.3 4.8 4.7

4.8 4.8 4.7 4.7 4.8

4.9 4.8 4.5

5.0 4.8 4.9

4.8 4.8 4.9

4.5 5.0 4.7 4.7 5.0 4.8 4.9 4.8 5.0 4.5

4.9

4.5 5.0

90

4.7 5.0

4.8 4.9 4.3 4.8

4.5 4.9 4.8

4.9 4.9 4.8 4.8 4.9

5.0 4.9 4.9

5.0 5.0 5.0

5.0 5.0 5.0

4.8 5.0 4.8 4.8 5.0 5.0 5.0 5.0 5.0 4.8

5.0

4.8 5.0 -

100

5.0 5.0

5.0 5.0 4.8 5.0

5.0 5.0 5.0

5.0 5.0 5.0 5.0 5.0

5.0 5.0 5.0

5.0 5.0 5.0

5.0 5.0 5.0

5.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0 5.0

5.0

5.0 5.0

278 J. C. RUPE

in effect, split in half and the two halves were paired in obtaining the coefEcient of reliability. Each item on the scale had then to be considered a “test” in itself and the raters the “items,” in the usual sense, on that test. This would result in a coefficient of reliability for each item rather than of the scale as a whole, which would then become a battery of 36 tests. Since each executive had different raters, this would be like having different items on a test. It follows, then, that equivalent, not identical, forms for each “test” were administered, for it was assumed that each subordinate had an equivalent opportunity to make judgments of his superior. Also, there was a different equivalent form for each executive who was rated. In view of the influence exerted by each executive in the selection, training, and direction and continued em- ployment of his subordinates, it would seem that this basic assumption was reasonable.

In the problem of reliability of rating scales we are usually concerned with the stability of the rating of the individual rater. Such is not the only question in this instance, however. The averaged ratings have been taken as the measure of an executive on a trait; consequently it is the stability of the averaged ratings with which we are concerned.

Because of a future need for coefficients based on a constant number of raters, the largest possible number of executives with a uniform number of raters were chosen from the data. Four randomly selected ratings for 114 executives were used with the odds paired with the evens. This is admittedly a small number of raters, but it is recognized as good administrative practice in upper levels not to have too large a number of immediate subordinates to supervise.

The Pearson product-moment method of correlation was used in obtaining the coefficients of reliability. The Spearman-Brown prophecy formula (5) was used to step up the reliabilities from two to four and, for comparison with Hobson’s study (4), to ten and twenty raters. Academic administrators ap- parently do have more immediate subordinates than do business and industrial executives.


10 Raters

1 10 20 5

.76

For two raters the range of coefficients was from .24 to .81, with a median r of .40. For four raters the range was .38 to 39, with a median of .58. For 10 raters it was .61 to .96, with a median of .76 and for 20 it was .76 to .98, with a median of 36. These data are presented in Table 2 where the comparison can be made with those of the prior study.

In order to determine whether these coefficients could have occurred by chance from uncorrelated variables, Fisher's (3) method of testing the significance of an r was used. This gave assurance that they represented true relationships.4

20 Raters ~~

9 24 3

.86

TABLE 2

Tabulation of Magnitudes of Reliability Coeflcients for Each Item

10. Raters

4 14 9 7

YIAGNllVDt

.900-1.ooo

.800- .a99

. 7 w .799

.600- .699

.50- .599

. 4 w .499

.300- .399

.2W .299 Median r

20' Raters

15 15 6

~~

2 Raters 4 Raters --I-

7 10 14 4

.40

1 l l 3

10 17 3 2

.58

2

.go* .88*

* Data from Hobson (4).

Combined Item Reliubiilities. It is helpful in evaluating a rating scale, even for rough descriptive purposes, to have a single measure of reliability rather than 36. In order to obtain such a measure, the z' transformation (5 ) was used to obtain a measure of central tendency. Based on two raters the result- ing correlation was .43, and stepped up to 20 raters it was .88. Hobson reported .89 for this number of raters.

Validity of the Items. The validity of a test consists of the relationship between the scores on the test and those on some criterion. The criterion must be some acceptable measure of the characteristic which the test is supposed to measure. In the

'For a complete presentation of all tables of statistical data see the original dissertation.

280 J. C. RUPE

absence of an acceptable, preferably objective, measure to serve as a criterion as is found here, the investigator may rely upon the combined subjective judgment of a group of individuals competent to judge the characteristic. For each item the relationship found to exist between these combined judgments and the ratings would result in a coefficient of validity.

Those individuals assumed to be best able to make competent judgments in this study were the subordinates of the executives, and their judgments were in the form of the ratings given on each item. No better criterion for the judgment of a subordinate’s rating of his superior could be found than the judgment of that subordinate himself. It will be recalled that the purpose of the scale was to measure the effectiveness of the executive as seen by his immediate employees. Therefore, the characteristic is the judgment of the subordinate, and the rather unusual situation exists here that the reliability and validity coefficients of the items are identical. Any question that might arise here would probably have to do with the frankness or honesty of the rater and it would seem that this must be controlled through the manner of the administration of the scale and the maintenance of anonymity.

Validity of the Scale as a Unit . It is important to know the validity of the items, but it is also important to determine the validity of the entire scale when a combined score is obtained.

The scale was designed to measure the effectiveness of an executive, and the general morale of the staff seems to be a reasonable criterion of such effectiveness. If the staff itself judges its morale to be high, the executive is presumably op- erating effectively; but if it is low, he is probably failing in his performance of his job. The frequency of mention of this item by executives themselves indicates something of the importance they attach to it.

Item 33 of the rating scale reads: “The general morale of his staff: (5 ) Exceptionally high, (4) Good, (3) Fair, (2) Poor, (1) Very low.,, This item was selected as the criterion for validity of the scale as a unit.


Hobson (4) reported an insignificant difference between the results of adding raw scores, the average rating on each item, and adding the z-score conversions. This was investigated elab- orately by a comparison of these data with Hobson's and the results were in accord.5 Thus the problem of the items being automatically weighted in proportion to their variabilities if raw scores are added (6) proved not to be significant.

The combination of raw scores correlated with the raw scores on Item 33, corrected for overlap, resulted in a validity coefficient of .72. It appears that the battery of items has a respect- able validity when staff 'morale as judged by the staff itself is used as a criterion. Though this is a validity coefficient, in this instance validity and reliability have an unusual syno- nymity already discussed. For the sake of comparison this coefficient based on four raters was stepped up by use of the Spear- man-Brown prophecy formula to see what might be expected had 20 raters been used by each executive. This resulted in a. coefficient of .93.

The Halo Eflect. It is desirable to know the extent of influence of the halo effect upon a rating device which is in- tended to rate more than one trait. This is the tendency to rate an individual consistently on all traits as he is rated on a single trait. An examination of the raw data indicated some tendency for this to occur but not enough for it to assume major importance. In other words, most raters seemed to discrim- inate between one trait and others in the ratings they gave.

Another way can be used to demonstrate the slight degree of influence of the halo effect. If it were present to any important extent , the item intercorrelations would be consistently high. Table 3 presents these intercorrelations. It can be seen that they range from .lo0 to B O O ; and though they are not completely normal as to distribution, they are not radically skewed toward the lower end. The median r is .53.

A third indication in this respect is apparent. If halo effect were complete, then in a factor analysis only one factor could

5 For the detailed investigation see the original dissertation.

282 J. C. RUPE

obviously appear. That this is not so will be shown in the factor analysis of these data.

The Factors of the Scale. The Thurstone centroid method of factor analysis was performed as described by Peters and Van Voorhis (5 ) . This method is characterized by interpreting the data in terms of overlapping group factors. The Pearson product-moment method was used for obtaining the item intercorrelations based on the averaged ratings of the four raters of each executive selected for the reliability study.6 It is based on an assumption of rectilinearity of regression, as is the Thur- stone multiple-factor analysis, and it gives a more accurate representation of the data than do other common forms of linear correlation.

TABLE 3 Tabulation of Item Intercorrelations According to Magnitudes

.700-.799 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.6W.699. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.500-.599 . . . .

.400-.499. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.300-.399. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.200-.299 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

.100-.199.. . . . . . . . . . . . . . . . . . . . . . . . . . . Median Item Intercorrelation . . . . . . .

31 148 193 157 66 28 7

.529

As a result of this analysis two factors emerged. Sixty-three per cent of the variance measured is attributable to Factor A and 37 per cent to Factor B. Factor A . In Table 4, the 10 items which contain the heaviest and purest loadings on Factor A have been listed, together with their loadings on each factor. It may be noted that Item 29, “Displays unwarranted favoritism to some subordinates,” scored in reverse, is a pure measure of Factor A, since the loading on Factor B is zero. Item 24, “Compliments and thanks his subordinates appropriately and sincerely,” is almost a pure measure. These two items appear to be key items in naming Factor A.

6 The correlational matrix, consisting of 630 intercorrelations, is shown in the original dissertation, Table 7, pp. 4141d.


“A”

.78

.77

.71

.81

.76

.77

.79

.77

.67

.64

The presence of Item 29 may cause logical difficulties unless it is remembered that it was scored in a reverse fashion to the other items. For purposes of Table 4, Item 29 may be read as: “Does not display unwarranted favoritism to some subordinates. ’ ’

All but two of the other items shown in Table 4 seem chiefly directed toward the executive’s dealings with his subordinates. Item 35, “Attempts to orient his work to the welfare of society at large”; and Item 34, “Promotes public relations,” are related to the larger society. orientation of an executive’s work

“B”

.oo

.15

.21

.24

.25

.30

.32

.34

.20

.22

~~

TABLE 4 Items Contributing Most Heavily to the Naming of Factor A

ITEM

29 Displays unwarrantgd favoritism to some subordinates. . . . . 24 Compliments and thanks his subordinates appropriately

32 Is just and considerate in discharging subor 8 Uses democratic procedures wherever possible. . . . . . . . . . . . . 28 Is honest and dependable in dealings with subordinates. . . . .

is work to the welfare of society at

33 The general f his staff.. . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Recognizes and rewards meritorious achievements of his

subordinates. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 Welcomes differences in viewpoints.. ......................

and sincerely.. . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

34 Promotes public relations. . . . . . . . . . . . . . . . . . . . . . . . . . .

for the welfare of society is probably an extension of the same outward-going characteristics involved in fair dealings with his own smaller society. Concern for public opinion, whether it be the smaller public of the group of subordinates or the larger body politic, has long been considered good business. It appears, however, that responsibility to the public is some- what secondary to concern for the immediate subordinates in the eyes of those subordinates.

Hobson (4) in his factor analysis of the data on academic administrators extracted a third factor which he called “Demo-

284 J. C. RUPE

ITEM

Possesses specific knowledge in his own field.. . . . . . . . . . . . . . Selects equipment wisely.. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Possesses general knowledge . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Welcomes additional responsibilities. . . . . . . . . . . . . . . . . . . . . . . The important work of his organization is completed.. . . . . Has knowledge of pertinent details of his subordinates’

work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Has adequate self-confidence . . . . . . . . . . . . . . . . . . . . . . . . . Makes plans carefully and adequately.. . . . . . . . . . . . . . . . . . . .

The essential work of his organization gets done on time. .

cratic Orientation.” The items he used for naming this were heavily loaded with Factor A and dealt with teamwork, democratic procedures, the welfare of others, and public relations. Items 6, 8,26, 34, and 35 fit into this area. It would seem that the present analysis tended to combine Hobson’s Factor A, which he called “Fairness to Subordinates,” and his Factor C, “Democratic Orientation.” It may well be that this indicates a difference between the academic administrator and the business and industrial executive. The former may be able to be fair to his own group without being oriented toward the

FACTOR WOADINGS

“B” “A” _______

.70 .35

.69 .49

.62 .39

.61 .46

.59 .47

.58 .46

.55 .42

.55 .21

.66 .54

TABLE 5 Items Contributing Most Heavily to the Naming of Factor B

I

-

2 16

1 22 19 23 14

4 10 12

larger group, but the latter may find it more necessary for business success to be concerned with both.

The trend of the items seemed to be justness, honesty, de- pendability, appreciation, consideration, and democratic tend- encies expressed toward subordinates principally but also toward society. It was decided therefore to name this factor “Social Responsibility for Subordinates and Society.”

Ninety-one per cent of the variance measured by these 10 items is attributable to Factor A and 9 per cent to Factor B. Factor B. The summary of information about the items contributing to Factor B is given in Table 5 . I t is apparent that


there are no items as clearly pure as there were for Factor A. Four items stand out as being reasonably separated from it. These are: Item 2, “Possesses specific knowledge in his own field”; Item 16, “Selects equipment wisely”; Item 1, “Possesses general knowledge”; and Item 22, “The essential work of his organization gets done on time.” These items seem to be related to an all-round grasp of the requirements of an executive’s work and ability to accomplish it.

Study of items 10, 19, and 4, shown in the table, inject some emphasis on the executive’s recognition of his own abilities and a willingness to use them. The weighting of all items with Factor A suggests that considerateness for subordinates and society has a real relationship with executive accomplishment.

In Factor B three aspects of executive requirements appear to emerge: the “knowing” with respect to his work and himself, the “planning,” and the “doing.” Knowledge and planning have much to do with accomplishment; so this factor was named “Executive Achievement.” This was in complete har- mony with Hobson.

Sixty-six per cent of the variance measured by these 10 items is attributable to Factor B and 34 per cent to Factor A.

Summary and Interpretation of the Factor Analysis. It is apparent from the size of the factor loadings that these two factors do not account for all that the scale is measuring but that they do provide some understanding of the scale. They appear on the basis of actual differences in the way executives are viewed by their subordinates. Real meaning can be attached to those differences.

It seems entirely credible that an executive may be considerate of his employees and socially oriented without attaining executive achievement. It also appears that he may succeed to a considerable degree-possessing knowledge, self-assurance, planning, and creativeness-without being entirely susceptive to the welfare of either the immediate or the larger group.

The history of industrial growth compounds these conclusions. Great accomplishment has been achieved by the in-

286 J. C. RUPE

dustrial barons of the past-but the great labor movement developed. More modern growth through paternalism to the recognition of the need for equality of bargaining may be reflected by the weighting of the items of Factor B and Fac- tor A. Executive achievement may be possible without considerateness of others, but the relationship indicated by this analysis surely echoes the philosophy of its importance in the long run.

CONCLUSIONS In the data which have been gathered and processed, there

are numerous implications about executives and their traits. Some of these judged important by the investigator are presented, together with factual findings of the study, in the following conclusions :

1. It is possible to measure the traits of business and industrial executives by means of a subordinate-executive rating scale. The Purdue Rating Xmle for Administrators and Ex- ecutives has demonstrated this possibility with acceptably high reliabilities and satisfactory evidence of validity. The 36 coefficients based on four raters range from .38 to .89, with a median of .58. When stepped up by use of the Spearman- Brown Prophecy Formula to 10 raters, a number of raters judged reasonably available to most executives, all 36 coefficients are above .60, 31 are above .70 and 11 are above .80 with a median of .76. Because of the technique used in determining reliability, it is possible on sound logical grounds to consider reliability and validity of the items as synonymous. Validity of the scale as a unit with an internal measure of staff morale as a criterion yields a coefficient of .72.

2. A study of the influences of automatic weighting, due to different item variabilities, was investigated. It is concluded that for this scale total scores obtained by averaging raw scores do not differ significantly from those obtained by a standard-score method.

3. The halo effect is not found to be of major importance. 4. Two factors common to various traits of business and in-


dustrial executives can be measured. These factors, as determined by this study, are (1) social responsibility to subordinates and society and (2) executive achievement. These factors seem to represent the two-fold responsibilities of an executive to the people in his immediate and larger society and to his job and employers.

5. It appears that an executive, as perceived by his subordinates, may succeed in executive achievement with only a mod- erate degree of concern for his subordinates and the public. He may, on the other hand, be considerate of his employees and be public spirited without being able to succeed as an executive. Responsibility to the public seems to play a some- what minor role to dealing fairly with subordinates.

6. The factor which is most important in determining staff morale is social responsibility to subordinates and society.

7. The freedom from display of unwarranted favoritism to some subordinates appears to be quite unrelated to executive achievement but highly related to social responsibility to subordinates and society.

8. The scale may be used to measure characteristics of any type of business or industrial executive and at any level down to but not including foremen. (No foremen were rated in this study.) Because of a lack of significant differences among total scale scores, it appears that the same norms may be used for all of them.

9. As measured by this scale, the similarities between business and industrial executives and academic administrators appear to be much greater than their dissimilarities. This is so much so that the scale may be profitably used by both and a single set of combined norms could be used. Inconclusive differences seem to appear in what may be slightly greater degree of frankness of rating by academic personnel and in a greater separation of concern for public mindedness.

DISCUSSION What the executive does with the information he receives

is entirely within his domain. At this time it seems best psy-

288 J. C . R U P E

chologically that this should be so. For such information concerning the executives to be given to the head of an organization would probably be viewed as a threat and create considerable fear and insecurity. The individual can, however, make as much or as little use of this information as he wishes. If deficiencies seem apparent to him, he may choose to be his own clinician, or he may wish to call upon the professional services of those who specialize in such consulting. (2)

Leadership is no longer considered an inborn characteristic. It is an aspect of social effectiveness and consists of certain habits and skills and can be learned. The Purdue Rating Scale for Administrators and Executives is an instrument for provid- ing information concerning those areas where new or additional learning and practice are needed.

As with any other instrument, it is possible to misuse this one. By a forced administration or a careful selection of specific raters instead of all those qualified, almost any results an executive might desire could be obtained. The morale of his staff might very easily be materially affected by such methods. As with any other scientific tool, it should be administered by personnel trained in its use so that both ratee and rater may understand its possibilities for their mutual gain. The valid truth may be painful at times even as it sometimes rewards. The path toward mature consciousness and social effectiveness is not easy and probably has no greater obstacle than a lack of understanding of self. One source of such understanding lies in those who work closely with and for an individual. One must be mindful of the fact that this source is itself pressed by personal responsibilities and insecurities in the face of the power and prestige of the one to be rated. Thus it requires special techniques to draw from it that sometimes painful type of information without which the individual cannot achieve his fullest potentialities.

REFERENCES 1. BITTNER, R. H. Developing an industrial merit rating procedure. PERSONNEL

PSYCHOLOGY, 1948, 1, 403-32.


2. FLORY, C. D. , AND JANNEY, J. E. Psychological services to business leaders. Journal of Consulting Psychology, 1946, 18, 115-19.

3. GUILFORD, J. P. Psychometric Methods. New York: McGraw-Hill Book Company, 1936.

4. HOBSON, R. L. Some psychological dimensions of academic administrators. Studies in Higher Education L X X I I I , Division of Educational Reference, Purdue Uni- versity, 1950.

5. PETERS, C. C., AND VAN VOORHIS, W. R. Statistical Procedures and their Mathe- matical Bases. New York: McGraw-Hill Book Company, 1940.

6. TIFFIN, J., AND MUSSER, W. Weighting merit rating items. Journal of Applied Psychology, 1942,26, 575-83.

Documents

When Workers Rate the Boss