1

Chapter 7: The Experimental Research Strategy

Manipulating the IV Controlling Extraneous Variance

Holding Extraneous Vars Constant Between Subjects Designs Within Subjects Designs

Multiple-Group Designs Quantitative IVs Qualitative IVs

Factorial Designs Summary

2

Experiment: Characteristics

Manipulation of IV Hold other vars constant Participants in all conditions are equivalent

Personal attributes (on average) Any variables relating to the DV Usually done by

random ASSIGNMENT to conditions (random selection is an external validity issue)

Why?

3

Statistics

Descriptive v. inferential Parametric

Partition vars into ratio of treatment/error Non-parametric

No assumptions about the distributions

4

Manipulation of IV

Conditions of the IV Experimental and control conditions Comparison Conditions

Additional Control and Comparison Conditions Hypothesis testing Ruling out specific alternative explanations

Characteristics of a good manipulation Construct validity Reliability Strength Salience

5

Manipulation of IV

Conditions of the IV Experimental and control conditions

Equivalence of ? Allows you to rule out nonspecific treatment effects Any differences between the conditions other than treatment Similar to placebo effects

Comparison Conditions How does comparison group differ from control?

It doesn’t

Additional Control and Comparison Conditions Hypothesis testing

(Bransford &Johnson, ’72) Why three conditions? No context, context before, context after

Ruling out specific alternative explanations (Alloy, Abramson, & Viscusi, ’81) added control conditions

Neutral mood, role-play to mood state-> demand

6

Manipulation of IV (con’t)

Characteristics of a good manipulation Construct validity

Use manipulation check (e.g. Mood from essay writing) Debrief interview; include in DV; pilot testing Is it sensitive enough? Are Ps attending to IV?

Reliability Automate instructions; detailed scripts

Strength Realistic level (for external validity, and mundane realism),

Salience Make sure they notice it

7

Manipulation of IV (con’t)

Using multiple stimuli IV Stimulus: person, object, event

Examples from your project?

Use only one stimulus for a condition E.g. training program to increase cooperation

What would possible stimuli be? Avoid confounding: stimulus person (multiple char)

Physical char; personal char

8

Manipulations (con’t)

Controlling Extraneous Variance External (keep environment; time same) Internal to P (more difficult)

Random assignment Ps > conditions Use homogenous sample Repeated measures (within subjects)

Between subjects designs To ensure group equivalence

1. Simple random assignment of Ps 2. Matched random assignment

9

Between-Subjects Designs

Simple random assignment (most used) How does this help to ensure group equivalence?

Individual differences (error variance) is randomly distributed across all conditions

How does Kidd &Greenwald’s (’88) do this? What individual difference variable that may affect the outcome

is randomly distributed across conditions? Memorization skill (does not differentially affect group

means)

Is it ok to use “quasi-random” assignment? What the hell is that?!!!!

10

Between-Subjects Designs

If random assignment doesn’t guarantee group equivalence, what can help? (why doesn’t it?) Matched random assignment can!

What are some Variables to match on? Categorical v. continuous vars

Which ones are more difficult to match on? Compare gender and IQ

Which need a pretest? Any downside to pretesting? Does the pretest variable need to be related to the DV?

11

Within-Subjects DesignsPs participate in each condition

Advantages Control individual differences (Perfect match)

What does this do? Reduce error (random) variance

Fewer Ps needed (increased power) Disadvantages

Order effects Practice effects Carryover Sensitization

E.g. Wexley et al. (’72) what was the problem? Demand effects

12

Within-Ss Controls

Order effects Counterbalancing Latin Square

Basic v. balanced What’s the difference? = Sequence v. order What’s a washout period? Differential order effect (Table 7-4)

Sensitization / demand characteristics Don’t use repeated measures

Order effects can be of theoretical interest Build into the experiment

13

Multiple Group Designs

Quantitative IVs Linear relationships

What is an e.g. of a linear IV for your project? Positive / negative / curvilinear?

What is the minimum levels necessary for quantitative? Why? 3… 2 points can only define a straight line DeJong et al. (’76); Feldman & Rosen (’78); Whitley (’82) What happened?

Qualitative IVs Give an e.g. of a qualitative IV for your project

14

Multiple-Group Designs

Interpreting the Results One way ANOVA

Post hoc or Contrasts (Planned comparisons) What’s the difference? A priori (Before=contrasts) v. Post hoc (After)

Compare omnibus F with focused F tests What is the benefit of a priori?

15

INCREASING THE NUMBER OF LEVELS OF AN INDEPENDENT VARIABLE

Provides more information about the relationship than a two level design

Curvilinear Relationship Inverted-U

Comparing Two or More Groups I.E. How dogs, cats, and birds as opposed to dogs

alone have beneficial effects on nursing home residents

16

LINEAR VERSUS POSITIVE MONOTONIC FUNCTIONS

17

LINEAR VERSUS POSITIVE MONOTONIC FUNCTIONS

18

Factorial Designs

Nature of Factorial Designs Describing them

2X2 (how many factors? Levels? Conditions? 2 factors, 2 levels each = 4 conditions

4X2 2 factors, 4 and 2 levels= 8 conditions

2X3X2 3 factors, 2, 3, & 2 levels =12 conditions

Information provided Main effects (how many in each example above?) Interactions (how many 2 way; three way?) What did Platz & Hosch (’88) find?

What caused the interaction to occur?

19

Factorial Designs

Displaying interactions Which is clearer? Line or bar graph? (fig 7-5) Convert from table of means to graph

(fig 7-6, p. 208 -209)

Interpreting interactions Main effects, interactions, both? Theory driven? (a priori v. post hoc)

20

Factorial Designs: Forms

Forms of Factorial Designs Between & Within-Subjects Designs

Between: Each subject participates in only one condition Within: Each subject participates in all conditions Mixed: Each subject participates in more than one condition

Platz & Hosch (’88) Store clerk (between) could it be within? Customer (within) could it be between?

Manipulated & Measured IVs Manipulated IV: true experimental design Measured IV: correlational aspect of design Caveat: Don’t dichotomize when not needed

21

Factorial Designs: Forms

Design Complexity Factors and levels (already discussed) How many Ps needed for Between design

With 10 per condition? 2X3?

60 Ps 3X4X2?

240 Ps

22

INCREASING THE NUMBER OF INDEPENDENT VARIABLES:

FACTORIAL DESIGNS

23

INCREASING THE NUMBER OF INDEPENDENT VARIABLES:

FACTORIAL DESIGNS

24

INCREASING THE NUMBER OF INDEPENDENT VARIABLES:

FACTORIAL DESIGNS

25

Uses of Factorial Designs

Testing Moderator Hypotheses Moderator: changes the effects of IV

E.g. Platz & Hosch (’88) race of clerk Use of ANCOVA & MR

Detecting Order Effects Table 7-6

Top: main for condition; no main for order; no interaction Middle: main for condition; no main for order; interaction Bottom: main for condition & order; interaction

26

Blocking on Extraneous Vars Including it as an IV Ps are grouped on extraneous var and tested by

ANOVA as a factorial Blocking reduces the error term (fig 7-9)

Caveat: Remember that the blocking var cannot be explained as cause

27

Experimental Strategy:Summary

Manipulating the IV Controlling Extraneous Variance

Holding Extraneous Vars Constant Between Subjects Designs Within Subjects Designs

Multiple-Group Designs Quantitative IVs Qualitative IVs

Factorial Designs Summary