1

What does a test score of 85 (out of 100) indicate?

When you get a test back in class, what is one of the first questions that you ask?

0 10 20 30 40 50 60 70 80 90 100

Test Score Distribution

*

* *

* *

*

* *

* **

* *

* **

* *

* *

*

* *

* **

* *

* **

* *

* *

*

* *

* *

*

* *

* *

Low Variability

2

40 45 55 60 70 75 80 90 100

Test Scores

40 45 55 60 70 75 80 90 100

Test Scores

Positively Skewed

Distribution

Negatively Skewed

Distribution

3

-4 -3 -2 -1 Mean +1 +2 +3 +4

Central Tendency

a) Mode (most frequent score)

b) Mean (average score; [EX/N])

c) Median (midpoint of scores)

Variability (Spread in Scores)

a) Range (lowest to highest score)

b) Standard Deviation

c) Variance

Normal Curve

4

X

10

20

30

40

50

-20

-10

0

10

20

Computation of Standard Deviation & Variance

x

(EX/N) = 30 (Mean)

EX = 150

Test Scores

Deviation scores (scores

minus the mean

x2

400

100

0

100

400

Squared deviation scores

EX2 = 1000 (Sum of the squared deviation scores)

EX2/N = 200 (the variance or s2)

200 = 14.14 (standard deviation)s2 = standard deviation or s

Mean of the sum of the squared deviation scores

5

-4 -3 -2 -1 Mean +1 +2 +3 +4 Test Score

13.59% 34.13% 34.13% 13.59%

0.13% 2.14%2.14% 0.13%

Number of Cases

Z score

T score

CEEB score

Deviation IQ (SD = 15)

Stanine

Percentile

-4 -3 -2 -1 0 +1 +2 +3 +4

10 20 30 40 50 60 70 80 90

200 300 400 500 600 700 800

55 70 85 100 115 130 145

4% 7% 12% 17% 20% 17% 12% 7% 4%

1 2 3 4 5 6 7 8 9

1 5 10 20 30 40 50 60 70 80 90 95 100

Relationships Among Different Types of Test Scores in a Normal Distribution

6

7

Test Score – Mean

Standard DeviationZ SCORE =

~ Standard Score ~

Fathers Height (in inches)

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Son’s Height

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

**

* * *

* * *

* * *

* * * *

* *

* *

* *

* * *

* * *

* **

* *

* * *

* * *

* * *

**

* * *

* * *

* *

* *

* * *

8

Correlation --- Some Key Concepts

a) Consists of a set of ordered pairs

b) Indicates both the magnitude and direction of the relationship between variables

c) Range is from -1.0 to + 1.0

9

Widely Reported Correlations

Smoking and lung cancer

Media violence and aggression

Condom use and sexually transmitted HIV

Passive smoking and lung cancer at work

Lead exposure and children’s IQNicotine patch and smoking cessation

Calcium intake and bone massHomework and academic achievementAsbestos and laryngeal cancerSelf-examination and breast cancer

-.2 -.1 0 .1 .2 .3 .4

From Bushman, B.J., & Anderson, C. A. (2001). Media violence and the American public: Scientific facts versus media misinformation, American Psychologist, June/July, 477-489.

Test Scores

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

**

* * *

* * *

* * *

* * * *

* *

* *

* *

* * *

* * *

* **

* *

* * *

* * *

* * *

**

* * *

* * *

* *

* *

* * *

Positive Correlation

11

Absenteeism (in hours)

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

**

* * *

* * *

* * *

* * *

* * *

* *

* *

* * *

* * *

* **

* *

* * *

* * *

* * *

**

* * *

* * *

* *

* *

* * *

Negative Correlation

12

Test Scores

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

* * *

* * *

* * *

* * *

* *

* * *

* *

* *

* * *

* * *

*

* * * * *

* * *

* *

* * *

* * *

* *

**

**

* * *

* * *

Significant Correlation

Poor

Good

Fail Pass

Correct Acceptances

False Rejections

Correct Rejections

False Acceptances

13

Test Scores

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

**

* * *

* * *

* * *

* * *

* * *

* *

* *

* * *

* * *

* **

* *

* * *

* * *

* * *

** *

* *

* * *

* *

* *

* * *

**

* * *

* * *

* * *

* * *

* *

**

**

* * *

* * *

* * *

* * *

* * *

No Correlation

Poor

Good

Fail Pass

Correct Acceptances

False Rejections

Correct Rejections

False Acceptances

14

15

Test Scores

Job Performance

Scores

100

50

0

100

50

0

Test Scores

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

* * *

* * *

* * *

* * *

* *

* * *

* *

* *

* * *

* * *

*

* * * * *

* * *

* *

* * *

* * *

* *

**

**

* * *

* * *

Poor

Good

Fail Pass

Correct Acceptances

Correct Rejections

False Acceptances

Significant Correlation

False Rejections

Effect of raising cutoff score?

Test Scores

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85

Job

Perform

ance

85

84

83

82

81

80

79

78

77

76

75

74

73

72

71

70

69

68

67

66

65

64

63

62

61

60

* * *

* * *

* * *

* * *

* *

* * *

* *

* *

* * *

* * *

*

* * * * *

* * *

* *

* * *

* * *

* *

**

**

* * *

* * *

Poor

Good

Fail Pass

Correct Acceptances

Correct Rejections

False Acceptances

Significant Correlation

False Rejections

Effect of lowering cutoff score?

Observation

Statement of the Problem (Research Question)

Design Study

Measurement (Collect Data)

Basic Steps in Research

Statistical Analysis

Interpretation (Conclusion)

• State Hypotheses

• Use/Generate a Theory

18

~ Tips for Choosing a Research Topic ~

• Read, read, read (e.g., journal articles) to generate research ideas

• Pick a topic that interests you (it will be more fun to do)

• Be realistic (pick a topic that can be accomplished in a reasonable time frame)

• Improve on prior research (e.g., “limitations of present study,” “suggestions for future research”)

• Minimize problems, time delays in data collection phase of research

Basic Scales of Measurement

1) Nominal (Categorization or classification)

• Yes/No, True/False

• Male/Female

2) Ordinal (Ranking)

• 1st, 2nd, 3rd

1) Interval (Equal intervals exist between points on the scale)

_______ _______ _______ _______ _______

1 2 3 4 5

• Ratio (an absolute zero point exists)

2) Kelvin scale of temperature

3) Time

4) Height, Weight

Measurement: The assignment of numerals to events or objects according to certain rules

20

• Limit collection of categorical data

Age

0 - 1819 – 2526 – 3536 – 4546 – 5556 – 6585 & Above

Income

0 ------ 10,00010,001 – 25,000 25,001 – 35,00035,001 – 50,00050,001 – 75,00075,001 – 100,000100,000 & Above

Age in Years: _______

Income: ____________

~ I-O Research ~Measurement

~ I-O Research ~Measurement (cont.)

Yes _____

No __________ _____ _____ _____ _____

1 2 3 4 5 Highly HighlyDisagree Agree

• Limit collection of dichotomous data

~ I-O Research ~Measurement (cont.)

• Restrict possibility of missing data

1.2.3.4.5.

4849 50

Scale Questions

Missing

Missing

Computed score for scale or subscales containing questions #5

and #48 will also be missing

~ Key Terms ~

Independent variable (IV) (predictor): One that can be manipulated or used to predict scores on the dependent variable

Dependent variable (DV) (criterion): The variable of interest; the one you are attempting to understand or affect

• SAT, ACT scores used to predict success in college

• Interview scores used to predict performance in a job

IVs DVs

24

Design Options

Control Realism

Laboratory Experiment

• Manipulate independent variable

• Precise measurement of dependent variable

Case study

• Detailed information

• Low generalizability

Naturalistic observation

Field study

Survey research

25

~ I-O Research ~

Interesting fact: Substantial amount of I-O studies are non-experimental (about 50%)

Overall Point:

Best for research to be driven by theories and problem-solving approaches not by methodology/statistics

• Much research efforts in I-O focus on rather trivial questions that can be studied with “fancy” techniques

• Bulk of research has limited applied significance

Some Pre-Experimental Designs

Static Group ComparisonX O

O

One-Shot Case Study

X O

X = Treatment or Intervention

O = Observation or Collection of Data

One-Group Pretest-Posttest Design

O X O

27

Math

Pretest

55

64

44

33

28

63

48

38

46

47

Math

Posttest

56

66

46

38

29

63

50

40

48

47

English

Pretest

33

35

43

36

20

60

40

31

52

64

English

Posttest

35

37

47

36

21

62

40

31

56

66

6-week training program between tests

Did the program work to increase scores?

28

% increase

100

90

80

70

60

50

40

30

20

10

0

Math English

“Lying” with numbers

29

An organization reports that accidents have decrease substantially since they began a drug testing program. In 1995, the year before drug testing, the number of accidents was 50. In 1996, the year testing began, the amount dropped to 40. In 1997, the year after drug testing the number of accident dropped to 29. What do you make of this?

1995 Drug Testing

55

50

45

40

35

30

25

20

15

10

5

1997

*

*

*

30

65

60

55

50

45

40

35

30

25

20

15

Given the illustration below, now what do you make of the effectiveness of the drug testing program?

1992 1993 1994 1995 1996 1997 1998 1999 2000

*

*

* *

*

**

*

*

31

Multiple Time-Series DesignO1 O2 O3 O4 O5 X O6 O7 O8 O9 O10

O1 O2 O3 O4 O5 O6 O7 O8 O9 O10

Some Quasi-Experimental Designs

O1 O2 O3 O4 O5 X O6 O7 O8 O9 O10

Time-Series Design

O X O

O O

Non-Equivalent Control-Group Design

32

R O X O

R O O

R indicates randomization

Pretest-Posttest Control Group Design

R X O

R O

Posttest-Only Control Group Design

Some True Experimental Designs

33

34

~ Basic Ethics in Research ~

Informed Consent

Right to Privacy

Anonymous, confidential treatment of data

Protection from Deception (cost-benefit assessment)

Debriefing e.g., (explanation of the purpose of the study)

50

45

40

35

30

25

20

15

10

5

0

J F M A M J Jul Aug S O N D

At first glance, anything happening here?

35

10

9

8

7

6

5

4

3

2

1

0

J F M A M J Jul Aug S O N D

How about now?

36