chapter 8 Makeing Judgments and Decisions

1

Chapter 8 Making Judgments and Decisions P189

• Intuitive Predictions• Judgment Aids• Utility Analysis

chapter 8 Makeing Judgments and Decisions

2

JUDGMENTS OF VALIDITY

• Evaluating a Measure – Without statistical validation – judge job

relatedness based on sequential questions:• Is trait related to performance (link KSA -> task)?• is the construct a valid measure of the KSA?• Give an example

– 12 questions to ask (p190)– Be prepared to answer the questions for a

measure you may use

chapter 8 Makeing Judgments and Decisions

3

MANAGERIAL USE OF ASSESSMENTS

• What do managers want?– Fill vacancy asap– Want the best people– Want it cheap

• What do managers know about testing?– Little; no training in psychometrics or test theory– Nothing about construct validity– Misinformed on test validity– Often distrust testing– Some overestimate the validity of tests (especially when

over-sold by test vendors (e.g. DISC, MBTI)

4

JUDGMENTS AS PREDICTIONS AND DECISIONS

• What is “common sense”, “intuition?” – Give some examples.– A thought question -

• Statistical v. judgmental Prediction– What are the pitfalls with each?– Which is generally superior? (Meehl, ‘54)

• Prediction and Decision Without Statistics– Usually the most important jobs have few people

• Why is this a problem for using statistics?• How do you deal with this?

• Combining Information for Overall Assessment• Average assessments/use common scale

chapter 8 Makeing Judgments and Decisions

chapter 8 Makeing Judgments and Decisions

5

JUDGMENT AIDS• Clarifying Judgments to Be Made

– JA is essential! –it makes management focus on what it will take to do the job• Procedural Planning

– Make a detailed protocol and document it • Why is this a good thing to do?

• Developing Assessment Scales– Use the same scale or different scales?– A structured interview?

• Expectancy Charts– What is a theoretical expectancy chart?– How is it used?– Give an example

• Expectancy Graphs– Use selection ratio and possible cut scores –

• to illustrate selection costs to show validity impact (for managers to understand)

• Utility Analysis– Brogden (‘49), Cronbach & Gleser (‘57)– Show relative pay off (average dollar pay off)– Caveat (GIGO)

• Concluding Comments – Judgments must be rational and include all infromation available