Assessment Center Essentials

Kevin R. MurphyDepartment of Psychology

Pennsylvania State University, USA

What Defines an Assessment Center

• Assessment– Of groups (multiple candidates)– By groups (observed by multiple assessors)– Using Behavioral Exercises (e.g., group

discussions, in-basket)– Integration (discussion, statistical, both)

• Guidelines for Assessment and Development Centres in South Africa, 4th Ed.

Are These Features Essential?• This talk poses the question of what makes

particular features essential and whether some features of ACs really are essential

• I intend to pose the question, but not provide the answer– You are the AC experts, and are better situated to

answer these questions– However, there is some value in thinking each of

these through

What Makes the “Essentials” Essential?

• Of groups– Why is this essential?• Opportunities to observe group behavior, interaction

with others• Probably the least essential of the “essentials”

What Makes the “Essentials” Essential?

• By groups– Why is this essential?• Reliability, adequate coverage, different perspectives

might yield unique insights

What Makes the “Essentials” Essential?

• Behavioral exercises– Why is this essential?• Behavior reveals things that self-reports, task

performance (e.g., ability tests) does not• People who have the right declarative and procedural

knowledge might not be able to do the things that are required on the job

What Makes the “Essentials” Essential?

• Integration– Why is this essential?• Different observers will have different information and

different perspectives

Could We Get By Without Some of the Essentials?

• In groups– Group exercises are not as big a part of current ACs

as they once were

– Assessment in groups can lead to confounds between individual and group effects. My strengths and weaknesses may look different depending on the characteristics of others participating in my AC

Could We Get By Without Some of the Essentials?

• By groups– This is an empirical question

– The conditions under which the pooled judgments of several assessors, each of which may have seen different behavioral samples are likely to be better than the judgments of one good assessor who has seen it all are complex to evaluate

Could We Get By Without Some of the Essentials?

• Behavioral Exercises– This is also an empirical question

• There is some evidence that behavioral exercises provide both unique measurement and better measurement of key competencies but empirical support is not strong

• Well-selected batteries of tests might provide much of the same information, and where tests and ACs differ in the information they provide, the evidence that the AC information is better (as opposed to different) is thin

Could We Get By Without Some of the Essentials?

• Integration– This is a complicated empirical question• It depends on the starting point

– Do raters disagree, how much, about what?

• Consensus judgments that are influenced by one strong judge might be worse if strength of conviction is not correlated with accuracy of assessment

Would it Still be an Assessment Center?

• AC with one candidate?– Why not?• Original OSS design made extensive use of

confederates• Use of confederates or virtual groups would provide

better control of a range of variables• There are some ACs that do not deal with groups of

candidates, although there are cost-effectiveness issues when the number of candidates assessed is small

Would it Still be an Assessment Center?

• AC with one assessor?– Why not?• One good assessor might be preferable to several less

good ones

Would it Still be an Assessment Center?

• AC without behavioral exercises?– Why not?• There is some evidence that behavioral exercises

provide both unique measurement and better measurement of key competencies but empirical support is not strong

• Well-selected batteries of tests might provide much of the same information, and where tests and ACs differ in the information they provide, the evidence that the AC information is better (as opposed to different) is thin

Would it Still be an Assessment Center?

• AC without integration?– Why not?

• If “integration” refers to judgmental/clinical integration, some ACs pretty much function this way already– Statistical integration is increasingly common, especially if AC is used

mainly for prediction

– The evidence in favor of statistical over judgmental integration has been around for over 50 years, and still seems compelling

– The argument that clinical integration is superior when AC is used for diagnosis/feedback is just that – an argument

So What IS Essential?

• Multiple dimensions/competencies• Multiple measures• Information is put together in a defensible way• Behavioral Exercises?

– This is still an under-researched question– AC proponents should gather the sorts of evidence that

would convince a skeptic– If we don’t need behavioral exercises, do we really need

assessors?

Behavioral Exercises: Show me the Money!

• The best way to convince the skeptics is to demonstrate:– Candidates and organizations make better decisions with

behavioral information than they would make without it– Does behavioral information allow you to do a better job

developing yourself, developing others, selecting candidates for some position, etc?

– What is the comparison standard?