Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S. CJBE Conference July 28-30, 2006 A Qualitative Exploration of the Relationship between Performance

Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S.

CJBE Conference

July 28-30, 2006

A Qualitative Exploration of the Relationship between Performance Appraisal and the

Development of Inspired Business Leaders

CJBE - July 28-30, 2006 Sillup/Porth - 2

Key Findings • PA seen as a way to gain recognition for performance and as a link

to becoming an IBL• Experienced (mean 16.6 years) respondents

– Were successful under their PA System - “exceeded” their objectives– Some problems with bias - none as raters but two when being rated

• Not enough time to implement– Vast mast majority (15 of 16 respondents) indicated that they did not

have enough time to implement the PA system• No formal training

– None of the respondents received formal training about the PA systems they were using

• Must go beyond objective measure to develop IBLs– PA System must also introduce the employee to aspects of

performance that go beyond the objective measures evaluated in results-oriented PA systems


Research Questions

• Does implementation of an effective Performance Appraisal (PA) system help to develop Inspired Business Leasers (IBLs)?

• What are the key impediments to using a PA system? This includes impediments before and/or during implementation?

• How can implementation be improved? • What are the implications of this research for the

development of IBLs?

The author wishes to acknowledge the financial support from the Saint Joseph’s University Pedro Arrupe Center for Business Ethics and the intellectual guidance from John McCall and Stephen J. Porth


What Is Performance Appraisal (PA)?

• Systematic description of the job-relevant strengths and weaknesses of an employee or an employee team

• Combines two processes, observation and judgment, to assess job performance on the basis of objective and subjective indices– Objective indices are appealing because they

measure a quantifiable endpoint but often measure factors beyond an employee’s control or outcomes of behavior (e.g., number of units sold)

– Subjective indices measure the job performance behavior (e.g., what did it take to sell the units)


PA and Its Influence on IBLs

• PA can improve employees’ work performance by helping them to realize their full potential as IBLs

• PA accomplishes this by providing information to employees and managers for use in making work-related decisions

• Because PA is an inexact human process, it may only effectively evaluate how employees meet objective performance measures

– e.g., assessing the number of units sold

• PA may fail to evaluate attributes that lead to improved performance

– e.g., integrity or other attributes especially important to the development of IBLs


For a Positive Influence on IBLs

• PA must take steps to ensure preparation for implementation and implementation addresses the following areas:

– Setting fair performance standards– Meeting requirements of effective PA systems– Assessing the types of PA systems/determining which one to

use– Identifying who should rate performance– Overcoming judgmental biases in rating– Establishing training programs for raters– Determining how often PA should be done– Understanding how PA fits with Total Quality Management


Research Method

• Structured interview over 30-45 minutes to assess how the shortcomings of PA potentially impact the development of IBLs

• 10 questions - 9 designated areas and 1 open-ended question:1. Does your PA system have clear PS?2. How well does your PA system meet the requirements of an effective

PA system?3. What type of PA system does you company use?4. Who rates PA?5. Are you aware of any biases in rating?6. Have you had “rater” training? When?7. How often is PA conducted?8. Is your company doing some variation of TQM? If so, how does PA fit

with TQM?9. Have you encountered any barriers to PA?10. Are there any other factors that help/hinder implementation of PA?

Full research instrument available in back-up slides


Results – Profile of Respondents

Companies Number of Respondents (Gender)

Departments/Years Experience

Aetna 4 (3F, 1M) Marketing = 2 (7, 10)HR = 2 (20, 18)

IBM 2 (2F) HR = 1 (22)CS = 1 (11)

J&J 6 (1F, 5M) Finance = 2 (15, 17)Marketing = 2 (19, 23)Sales = 2 (9, 15)

Wyeth 4 (2F, 2M) R&D = 1 (24)RA = 2 (16, 20)Finance = 1 (20)

Total 16 (8F, 8M) Marketing = 4 Finance = 3HR = 3 Sales = 2 RA = 2CS = 1 R&D = 1


Analysis of Structured Interviews

• Revealed use of results-oriented PA systems on an annual basis• Similar responses from the four companies for both the male and

female respondents– Summary on table in next slide

• Top-down support was identified as most important for the successful implementation of the PA system

• No matter the department, all respondents had their PA conducted by their immediate managers and used multi-rater input (360-feedback)

• 10 of the 16 used self-appraisal as preparation for PA• 2 different responses

– Wyeth for TQM because they were the only company with a respondent from Manufacturing, the only department implementing TQM

– Aetna where four customers were asked for feedback.


Results – Frequency of ResponsesInterview Questions Responses as Rater Responses as Ratee

1.) Clear PS Yes = 16 Yes = 16

2.) Meets requirements of effective PA system

Yes = 16 Yes = 16

3.) Type of PA system Behavior-Oriented = 0Results-Oriented = 16 with non-objective performance attributes included

Behavior-Oriented = 16Results-Oriented = 16

4.) Who rates PA Immediate Manager = 16Peers = 0Direct Reports = 0Self-appraisal = 10; employee preparation for PA at J&J and WyethCustomers = 4 at AetnaMulti-Rater = 16; 360˚ Computer Monitoring = 0

Immediate Manager = 16Self-Appraisal = 10 as preparation Multi-Rater = 12

5.) Bias in rating None observed = 16 No = 16 but two suspected bias in their reviews

6.) Rater training Yes = 14; informal from their managersNo = 2; new hires at company

N/A

7.) How often conducted Annually = 16 with mid-year update Annually = 16 with mid-year update

8.) TQM fit with PA Yes = 2; Manufacturing at Wyeth Yes =1; Manufacturing at Wyeth

9.) Behavioral barriers No = 16; all noted the importance of top-down support

No = 16

10.) Other factors Not enough time = 15 Not enough time = 15

Number responses ≠ number respondents N = 16 N = 16


Results Summary• Taken as a whole, the interviews were positive

– Regarding PA as a way to gain recognition for performance– Suggesting it as a link to becoming an IBL

• None of the respondents gave me their individual PA rating but they said they “exceeded” the annual objectives

• None of the respondents suspected bias when they were raters but two suspected bias (political) when they were being rated by their managers

• Vast mast majority (15 of 16 respondents) provided unsolicited responses to Question #10 (Other) and indicated that they did not have enough time to implement the PA system

• None of the respondents received formal training about the PA systems they were using– 2 received no informal training because they were new hires with

considerable experience in conducting PA– Those interviewed were experienced (mean of 16.6 years)

• Most inexperienced respondent had 7 years experience


Must Go beyond Effective PA System to Inspire IBLs

• Results reinforce that effective implementation of a PA system is an optimal way to assess an employee’s performance but……………………………………………

• Development must also introduce the employee to aspects of performance that go beyond the objective measures evaluated in results-oriented PA systems if the PA system is to help develop IBLs

• These include the performance attributes added to complement the behavior-oriented PA systems being implemented by the companies in this study


Performance Attributes Added to Behavior-Oriented PA Systems

• Examples of this can viewed in three of the four companies participating in this study

– Johnson & Johnson– IBM– Aetna

• They integrated concerns for employees’ rights into the very fabric of their PA systems and corporate cultures

• Despite more recent ethical concerns (e.g., IBM connected to the Holocaust)

– Both companies receive strong employee approval for their policies on company attitude surveys


Allowing Enough Time to Implement PA System is Necessary for Success

• Enough time to implement a PA system seems to be a determinant for the success of the system

• Furthermore, when time is constrained– Raters revert to familiar objective measures, such as financial

metrics, and “tell and sell” approach rather than an information exchange

– Raters also become more vulnerable to biases, (e.g., recency effect)

• Although most companies require that PA results be discussed with employees, they do not consider PA as an individual objective

– Consequently they miss an opportunity to help improve their PA system and to establish the basis for development of IBLs


Implementation Practices - Need to be Continued and Started

Continue:• Annual reviews with a

mid-year update– Feedback has maximum impact

when given as close as possible to the action

• Self-appraisal is very useful

– Employees who prepare are more satisfied with the PA process and more likely to meet their personal objectives

• Encourage Participation– Managers welcome employees’

ideas

Start:• Training for Raters

– Emphasize how to observe behavior more accurately and fairly and not on how to rate or not rate employees’ performance.

• Set PA as an Objective– Objectives that are set emphasize

the importance of meeting the objective


Implications for Further Research

• More companies and more raters/ratees from different departments with less experience

– Provide the basis for quantitative assessment of these qualitative observations

• Larger, more robust dataset would enable other interesting observations

– e.g., how managers who “exceed” their annual objectives rate their employees

• Consider how PA systems are working globally – Differences between the US and Europe and, especially,

Pacific-Rim countries

N.B. Contributors to the content of this paper include employees of Aetna, IBM, Johnson & Johnson and Wyeth as well as

Susan Givens-Skeaton for her input and introduction to Wayne Casio’s research and Christopher Field for his critical review.


Connectedness of PA System, Top Management and IBLs

PA System for top management

Inspired Business Leaders

Balanced Scorecard

•No “correct” PA system to stimulate development of IBLs but one alternative that enables a business to “manage by values” is the Balanced Scorecard


Balanced Scorecard Creates Environment for IBLs

Financial MeasuresOwners and Stakeholders’

View

Customer Perspective

Customers’ View

Future PerspectiveContinue

Internal Perspective

Employees’ View

Balanced Scorecard

Adaptation of Source: S. Porth, Strategic Management, A Cross-Functional Approach, 2006


Thank You Questions


Back-Up Slides


Research Instrument

1.) Does your PA system have clear PS?2.) How well does your PA system meet the requirements of an effective PA

system?– Relevance – identify what makes a difference– Sensitivity – determine effective/ineffective performance– Reliability – platform for consistent judgment– Acceptability – have the support of those using it– Practicality – easy to understand

3.) What type of PA system does you company use?– Behavior-Oriented

• Narrative essay• Ranking• Checklist• Critical incidents• Graphic rating scale

– Results-Oriented• Management By Objectives• Working Plan and Review


Research Instrument (continued)

4.) Who rates PA?– Immediate Manager– Peers– Direct Reports– Self-appraisal– Customers– Multi-Rater (360˚-feedback)– Computer Monitoring

5.) Are you aware of any biases in rating?– Leniency and severity– Central tendency– Halo effect– Contrast error– Recency error

6.) Have you had “rater” training? When?7.) How often is PA conducted?


Research Instrument (concluded)

8.) Is your company doing some variation of TQM? If so, how does PA fit with TQM?

9.) Have you encountered any barriers to PA?

– Lack of top-down support

– Lack of employee buy-in

– Political barriers

– Interpersonal barriers

10.) Are there any other factors that help/hinder implementation of PA?

Documents

Stephen J. Porth, Ph.D. for George P. Sillup, Ph.D., M.S. CJBE Conference July 28-30, 2006 A Qualitative Exploration of the Relationship between Performance