Principles for the Validation andUse of Personnel Selection Procedures

Fourth Edition

Society for Industrial and Organizational

Psychology, Inc.

2003

principles.qxd 10/3/2003 9:43 AM Page i

This document is an official policy statement of the Society for Industrialand Organizational Psychology (Division 14 of the American Psychologi-cal Association) and was approved as policy of the American Psychologi-cal Association (APA) by the APA Council of Representatives in August2003. Copies are available from the Society Administrative Office. Toorder, please visit the SIOP Web site at www.siop.org.

Society for Industrial Organizational Psychology520 Ordway AvenueBowling Green OH 43402419-353-0032www.siop.org

2003 by the Society for Industrial and Organizational Psychology,Inc. All rights reserved. No part of this document may be reproducedwithout written permission of the publisher.

ii

principles.qxd 10/3/2003 9:43 AM Page ii

Committees

Ad Hoc Committee on the Revision of thePrinciples for the Validation and Use of Personnel Selection Procedures

Advisory Panel on the Revision of thePrinciples for the Validation and Use of Personnel Selection Procedure

iii

Marcia M. Andberg, Ph.D.Marcia Andberg Associates

Steven H. Brown, Ph.D.LIMRA, Inc.

Wayne J. Camara, Ph.D.The College Board

Wanda J. Campbell, Ph.D.Edison Electric Institute

Donna L. Denning, Ph.D.City of Los Angeles

P. Richard Jeanneret, Ph.D., ChairJeanneret and Associates, Inc.

Jerard F. Kehoe, Ph.D.Selection & Assessment Consulting

James L. Outtz, Ph.D.Outtz and Associates

Paul R. Sackett, Ph.D.University of Minnesota

Mary L. Tenopyr, Ph.D.Consultant

Nancy T. Tippins, Ph.D.Personnel Research Associates

Sheldon Zedeck, Ph.D.University of California at Berkeley

Herman Aguinis, Ph.D. University of Colorado at Denver

Winfred Arthur, Jr., Ph.D.Texas A & M University

Philip Bobko, Ph.D. Gettysburg College

John C. Callender, Ph.D. The Procter & Gamble Company

Fritz Drasgow, Ph.D. University of Illinois

Joyce C. Hogan, Ph.D. Hogan Assessment Systems

Leaetta M. Hough, Ph.D. The Dunnette Group, Ltd.

Frank J. Landy, Ph.D. SHL Landy Jacobs, Inc.

Kenneth Pearlman, Ph.D. Consultant

Robert A. Ramos, Ph.D.Ramos & Associates

Ann Marie Ryan, Ph.D. Michigan State University

Frank L. Schmidt, Ph.D. University of Iowa

Neal W. Schmitt, Ph.D. Michigan State University

principles.qxd 10/3/2003 9:43 AM Page iii

Table of Contents

Committees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .iiiForeword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .viiiIntroduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

Statement of Purpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1Principles as Guidance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .2

Selection Procedures Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3Overview of the Validation Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

Sources of Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Evidence Based on the Relationship Between Scores on Predictorsand Other Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5Content-Related Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .6Evidence Based on the Internal Structure of the Test . . . . . . . . . . . .6Evidence Based on Response Processes . . . . . . . . . . . . . . . . . . . . . .6Evidence Based on Consequences of Personnel Decisions . . . . . . .7

Planning the Validation Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Existing Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .8Proposed Uses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .9Requirements of Sound Inference . . . . . . . . . . . . . . . . . . . . . . . . . . .9Feasibility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

Analysis of Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10Purposes for Conducting an Analysis of Work . . . . . . . . . . . . . . . .10Level of Detail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .11

Sources of Validity Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13Evidence of Validity Based on Relationships with Measures of OtherVariables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .13Criterion-Related Evidence of Validity . . . . . . . . . . . . . . . . . . . . . . . .13

Feasibility of a Criterion-Related Validation Study . . . . . . . . . . . .14Design and Conduct of Criterion-Related Studies . . . . . . . . . . . . .14Criterion Development . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .16Choice of Predictor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17Choice of Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18Data Analysis for Criterion-Related Validity . . . . . . . . . . . . . . . . .18

Evidence for Validity Based on Content . . . . . . . . . . . . . . . . . . . . . . .21Feasibility of a Content-Based Validation Study . . . . . . . . . . . . . .21Design and Conduct of Content-Based Strategies . . . . . . . . . . . . .22Defining the Content Domain . . . . . . . . . . . . . . . . . . . . . . . . . . . .22Choosing the Selection Procedure . . . . . . . . . . . . . . . . . . . . . . . . .23Procedural Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24Evaluating Content-Related Evidence . . . . . . . . . . . . . . . . . . . . . .25

Evidence of Validity Based on Internal Structure . . . . . . . . . . . . . . . .25

iv

principles.qxd 10/3/2003 9:43 AM Page iv

Generalizing Validity Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27Transportability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27Synthetic Validity/Job Component Validity . . . . . . . . . . . . . . . . . . . . .27Meta-Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28

Fairness and Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31Fairness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32

Predictive Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32Measurement Bias . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .33

Operational Considerations in Personnel Selection . . . . . . . . . . . . . . . . .35Initiating a Validation Effort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .35

Defining the Organizations Needs, Objectives, and Constraints .35Communicating the Validation Plan . . . . . . . . . . . . . . . . . . . . . . . .37

Understanding Work and Worker Requirements . . . . . . . . . . . . . . . . .38Strategies for Analyzing the Work Domain and Defining WorkerRequirements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38Considerations in Specifying the Sampling Plan . . . . . . . . . . . . . .38Documentation of the Results . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

Selecting Assessment Procedures for the Validation Effort . . . . . . . .38Review of Research Literature . . . . . . . . . . . . . . . . . . . . . . . . . . . .39Psychometric Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39Administration and Scoring Considerations . . . . . . . . . . . . . . . . . .39Format and Medium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39Acceptability to the Candidate . . . . . . . . . . . . . . . . . . . . . . . . . . . .40Alternate Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41

Selecting the Validation Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41Fit to Objectives, Constraints, and Selection Procedures . . . . . . .41Individual Assessments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

Selecting Criterion Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42Performance-Oriented Criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . .42Other Indices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42Relevance and Psychometric Considerations . . . . . . . . . . . . . . . . .43

Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43Pilot Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44Match Between Data Collection and Implementation Expectations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44

Confidentiality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44Quality Control and Security . . . . . .