Quiz 1 next Monday & Tuesday - University of Queenslanduqwloui1/stats/3010 for post... · 2018-08-02 · 2 Quiz 1 next Monday & Tuesday preparation • review Lectures 1 Æ4 • organize

1

11

psyc3010 lecture 4psyc3010 lecture 4

factorial betweenfactorial between--Ps ANOVA III:Ps ANOVA III:higherhigher--order ANOVAorder ANOVA

last lecture: factorial between-Ps ANOVA II(following up significant effects)

next lecture: power analyses and blocking designs

22

Quiz 1 next Monday & Tuesdaypreparation• review Lectures 1 4• organize your notes so you know where to find information• look at quiz tips + complete the online practice questions • complete the online Practice Quiz on Blackboard• ?s welcome before the quiz. During the quiz, I will not be

available.

taking the quiz• opens @ 9am Sunday (27/3), closes @ 9pm Monday (28/3)• no time restrictions + can return to active quiz• can submit quiz only once, and must do so by closing date • work alone

2

33

last lecture this lecturelast lecture: – following up significant main effects and

interactions in 2-way factorial ANOVA

this lecture:– higher-order factorial designs

(“complex ANOVA”)

Next lectures:– March 28th guest lecture by Joanne Brown on

power analyses & Blocking designs.– I return in Week 6.

44

topics for this week topics for this week introduction to higherintroduction to higher--order designsorder designs

omnibus tests in 3omnibus tests in 3--way factorial ANOVAway factorial ANOVAmain effects, 2main effects, 2--way interactions, 3way interactions, 3--way interactionsway interactions

overview of followoverview of follow--up tests in 3up tests in 3--way ANOVAway ANOVAfollowing up main effects, 2following up main effects, 2--way interactions, way interactions, and 3and 3--way interactionsway interactions

following up 3following up 3--way interactions way interactions -- detailsdetailssimple interaction effectssimple interaction effectssimple simple simplesimple effectseffectssimple simple simplesimple comparisonscomparisons

3

55

higher-order factorial designs:an introduction

66

higherhigher--order factorial designsorder factorial designsmore than 2 independent variables (factors)allow for designs with higher external validity

world is often more complicated than a 2 x 3

EXAMPLE EXAMPLE –– predicting driving performancepredicting driving performance

•• a number of variables could have an important influence:a number of variables could have an important influence:* age (young, old)* age (young, old)* amount of alcohol drunk (0, 1, or 5 drinks)* amount of alcohol drunk (0, 1, or 5 drinks)* gender (men, women)* gender (men, women)

how do these factors influence driving performance,independently and/or interactively?

4

77

•• 2 (age)2 (age) x x 3 (alcohol)3 (alcohol) x x 2 (gender)2 (gender)between-subjects design

• 12 cells [calculated as for 2-way factorial design]

menmenno alc 1 drink 5 drinksno alc 1 drink 5 drinks

womenwomenno alc 1 drink 5 drinksno alc 1 drink 5 drinks

oldold

youngyoung

notation for highernotation for higher--order designsorder designs

88

main effects:main effects:–– differences between marginal means of one differences between marginal means of one

factor factor (averaging over levels of other factors)(averaging over levels of other factors)

effects in highereffects in higher--order designsorder designs

5

99

main effect of gendermain effect of gender

menmen

(averaged across (averaged across alcohol and age)alcohol and age)

womenwomen

(averaged across (averaged across alcohol and age)alcohol and age)

main effects in highermain effects in higher--order designsorder designs

1010



twotwo--way interactions:way interactions:–– whether the effect of one factor is the whether the effect of one factor is the

same at every level of another factor same at every level of another factor (averaging over levels of a third factor)(averaging over levels of a third factor)


6

1111

age by alcohol interactionage by alcohol interaction

no alc 1 drink 5 drinksno alc 1 drink 5 drinks((averaged across averaged across men & women)men & women)

oldold

youngyoung

22--way intxs in higherway intxs in higher--order designsorder designs

1212

0100200300400500600700800

no alcohol 1 drink 5 drinks

Alcohol consumption

Driv

ing

perf

orm

ance

old

young

1. compare line slopes – are lines are (not) parallel? (to check for interaction)

2. compare averages of red line and blue line (to check for main effect of factor represented by lines)

3. compare average for each column (to check for main effect of factor represented on X axis)

graph for 2-way interaction

7

1313

1414



twotwo--way interactions:way interactions:–– examines whether the effect of one factor is examines whether the effect of one factor is

the same at every level of another factor the same at every level of another factor (averaging over levels of a third factor)(averaging over levels of a third factor)

threethree--way interaction:way interaction:–– examines whether the twoexamines whether the two--way interaction way interaction

between two factors is the same at every between two factors is the same at every level of the third factorlevel of the third factor


8

1515

age x alcohol x gender age x alcohol x gender (no averaging)menmen

no alc 1 drink 5 drinksno alc 1 drink 5 drinkswomenwomen

no no alcalc 1 drink 5 drinks1 drink 5 drinks

oldold

youngyoung

3-way intx in higher-order designs

does a 2-way interaction hold at each level of the 3rd factor?

1616

Females

0

200

400

600

800

1000

No drink 1 drink 5 drinks

Alcohol consumption

Driv

ing

perf

orm

ance

old

young

Males

0

200

400

600

800

1000

no alcohol 1 drink 5 drinks

Alcohol consumption

Driv

ing

perf

orm

ance

old

young

1. plot 2-way interactions withineach level of the third factor

2. check if pattern for 1st graph (simple interaction of AB at C1) is different from 2nd graph (simple interaction of AB at C2) -- If graphs are

not same pattern, there is a 3-way interaction

difficult to interpret other 2-way interactions from graphs, let alone main effects

The more complex design, the more rely on statistical tests not eyeballing

graphs for 3-way interactions

9

1717

partitioning variance in partitioning variance in a 3a 3--way ANOVAway ANOVA

e

main effectsvariance due to αα

variance due to ββ

variance due to γγ

2-way interactionsvariance due to αβαβ

variance due to βγβγ

variance due to αγαγ

3-way interactionvariance due to αβγαβγ

error/residual

variance due to ee

1818

structural models in factorial ANOVA

22--way factorial design:way factorial design:

XXijkijk = = μμ. + . + ααj j + + ββk k + + αβαβjkjk + + eeijkijk

33--way factorial design:way factorial design:

XXijklijkl ==

μμ. + . + ααjj + + ββk k + + γγll + + αβαβjkjk + + βγβγklkl + + αγαγjljl + + αβγαβγjkljkl + + eeijklijkl

10

1919

degrees of freedomdegrees of freedomdfdftotal total = N = N -- 1 1

dfdfJ J = j = j -- 11dfdfK K = k = k -- 11dfdfL L = l = l -- 11

dfdfJK JK = (j = (j -- 1)(k 1)(k -- 1)1)dfdfJL JL = (j = (j -- 1)(l 1)(l -- 1)1)dfdfKL KL = (k = (k -- 1)(l 1)(l -- 1)1)

dfdfJKLJKL = (j = (j -- 1)(k 1)(k -- 1)(l 1)(l -- 1)1)

dfdferror error = N = N -- jkljkl

regardless of # of factors in between-groups design, df for a factor always = # of levels - 1

df for an interaction = product of df for factors involved

df for error =(N - # cells) or (n -1) x (# cells)

2020

33--way factorial designs:way factorial designs:omnibus tests omnibus tests

11

2121

yet another (quasi) experimentyet another (quasi) experiment

Reinforcement Sensitivity TheoryReinforcement Sensitivity Theory

do people learn better from a particular type of do people learn better from a particular type of reinforcement: reward versus punishment?reinforcement: reward versus punishment?

impulsive personality impulsive personality learn well from reward learn well from reward but not punishmentbut not punishment

anxious personality anxious personality learn well from punishment learn well from punishment but not rewardbut not reward

gender differences are also possible gender differences are also possible

2222

yet another (quasi) experimentyet another (quasi) experimentDV:DV: speed of responses in reaction time (RT) tasktime (RT) task

3 independent factors:3 independent factors:•• reinforcement typereinforcement type

-- reward for fast responses reward for fast responses -- punishment for slow responses, punishment for slow responses, -- control condition (no reward or punishment) control condition (no reward or punishment)

•• personality typepersonality type-- anxious personalityanxious personality-- impulsive personalityimpulsive personality

•• gendergender-- malemale-- femalefemale

12

2323

7 omnibus tests baby!7 omnibus tests baby!

–– 3 main effects3 main effects-- reinforcement (reward, punishment, none) reinforcement (reward, punishment, none) -- personality (impulsive, anxious)personality (impulsive, anxious)-- gender (male, female)gender (male, female)

–– 3 two3 two--way interactionsway interactions (a.k.a. first(a.k.a. first--order interactions)order interactions)-- reinforcement x personalityreinforcement x personality-- reinforcement x genderreinforcement x gender-- personality x genderpersonality x gender

–– 1 three1 three--way interactionway interaction (a.k.a. second(a.k.a. second--order interaction)order interaction)-- reinforcement x personality x genderreinforcement x personality x gender

yet another (quasi) experimentyet another (quasi) experiment

2424

data and cell totals / meansdata and cell totals / means(full layout)(full layout)

Males

Personality Rew None Pun

Impulsive 310 355 490320 350 495330 360 485

Total 960 1065 1470Mean 320 355 490

Anxious 485 450 310490 455 320495 445 330

Total 1470 1350 960Mean 490 450 320

Reinforcement

Females

Personality Rew None Pun

Impulsive 310 450 490320 455 486330 445 480

Total 960 1350 1456Mean 320 450 485

Anxious 485 345 310480 350 320490 355 330

Total 1455 1050 960Mean 485 350 320

Reinforcement

13

2525

dftotal = N - 1 = 36 - 1 = 35

dfP = p - 1 = 2 - 1 = 1dfG = g - 1 = 2 - 1 = 1dfR = r - 1 = 3 - 1 = 2

dfPG = (p - 1)(g - 1) = 1 x 1 = 1dfRG = (r - 1)(g - 1) = 2 x 1 = 2 dfPR = (p - 1)(r - 1) = 1 x 2 = 2dfPRG = (p - 1)(g - 1)(r - 1) = 1 x 1 x 2 = 2

dferror = N - prg = 36 - 2 x 3 x 2 = 36 – 12 = 24

degrees of freedomdegrees of freedomregardless of # of factors in between-groups design, df for a factor = # levels - 1

df for an interaction = product of df for factors involved

df for error =(N - # cells) or (n -1) x (# cells)

2626

Mathematics is the language with which Mathematics is the language with which God has written the universe.God has written the universe.

-- Galileo Galilei, physicist and Galileo Galilei, physicist and astronomer (1564astronomer (1564--1642)1642)

14

2727

summary table (from SPSS)summary table (from SPSS)Tests of Between-Subjects Effects

Dependent Variable: RT

51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709

53.778 1 53.778 1.075 .310

168516.722 2 84258.361 1684.232 .000

.056 2 .028 .001 .999

9538.778 1 9538.778 190.670 .000

19015.056 2 9507.528 190.045 .000

1200.667 24 50.028198383.889 35

Source

ReinforcementPersonalityGenderReinforcement xPersonalityReinforcement x GenderPersonality x GenderReinforcement xPersonality x GenderErrorTotal

Type III Sumof Squares df Mean Square F Sig.

2828

no significant main effectsno significant main effectsTests of Between-Subjects Effects


51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709

53.778 1 53.778 1.075 .310

168516.722 2 84258.361 1684.232 .000

.056 2 .028 .001 .999

9538.778 1 9538.778 190.670 .000

19015.056 2 9507.528 190.045 .000

1200.667 24 50.028198383.889 35

Source



15

2929

significant 2significant 2--way interaction: way interaction: personality x reinforcementpersonality x reinforcement

Tests of Between-Subjects Effects


51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709

53.778 1 53.778 1.075 .310

168516.722 2 84258.361 1684.232 .000

.056 2 .028 .001 .999

9538.778 1 9538.778 190.670 .000

19015.056 2 9507.528 190.045 .000

1200.667 24 50.028198383.889 35

Source



3030

0100200300400500600

Rew Neut Pun

Task condition

Mea

n re

actio

n tim

e

Imp

Anx

significant 2significant 2--way interaction: way interaction: personality x reinforcementpersonality x reinforcement

effect of at least one of the two factors is different at different levels of the other factorignoring (averaging across) the third factor

16

3131

significant 2significant 2--way interaction:way interaction:personality x genderpersonality x gender



51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709

53.778 1 53.778 1.075 .310

168516.722 2 84258.361 1684.232 .000

.056 2 .028 .001 .999

9538.778 1 9538.778 190.670 .000

19015.056 2 9507.528 190.045 .000

1200.667 24 50.028198383.889 35

Source



3232

360370380390400410420430

Imp Anx

Personality

Mea

n re

actio

n tim

e

Men

Women

significant 2significant 2--way interaction:way interaction:personality x genderpersonality x gender

effect of at least one factor is different at different levels of the other factorignoring (averaging across) the third factor

17

3333

significant 3significant 3--way interaction:way interaction:reinforcement x personality x genderreinforcement x personality x gender



51.722 2 25.861 .517 .6037.111 1 7.111 .142 .709

53.778 1 53.778 1.075 .310

168516.722 2 84258.361 1684.232 .000

.056 2 .028 .001 .999

9538.778 1 9538.778 190.670 .000

19015.056 2 9507.528 190.045 .000

1200.667 24 50.028198383.889 35

Source



3434

so there’s this 3so there’s this 3--way interactionway interactionmeans that a two-way interaction is different at different levels of the third factor

Simple interaction of AxB is for C1 than simple interaction for C2, etc..

But which simple interaction to look at ? Could mean one or more of the following:

personality X genderpersonality X gender 2-way interaction is different across levels of reinforcement

reinforcement X personalityreinforcement X personality 2-way interaction is different across levels of gender

reinforcement X genderreinforcement X gender 2-way interaction is different across levels of personality

18

3535

following up following up significant omnibus effects significant omnibus effects

in a 3in a 3--way ANOVA: way ANOVA: overview of all testsoverview of all tests

3636

following up main effectsfollowing up main effectsin 3in 3--way factorial ANOVAway factorial ANOVA

just as in a 2-way ANOVA, a significant omnibus main effect must be interpreted– e.g. what is the effect of Factor A, ignoring

(averaging over) Factor B and Factor C?

if the factor has > 2 levels, we don’t know exactly where the differences are

we then use main effect comparisons(with t-tests or linear contrasts), exactly as we did in 2-way ANOVA

19

3737

following up omnibus 2following up omnibus 2--way interactionsway interactionsin 3in 3--way factorial ANOVAway factorial ANOVA

just as in a 2-way ANOVA, a significant omnibus 2-way interaction must be interpreted– e.g., is the effect of Factor A different at different

levels of Factor B, and vice-versa? (ignoring Factor C)

we then test simple effects (with the F test), exactly as we did in 2-way ANOVA

if you find a significant simple effect for a factor with > 2 levels, you follow it up with simple comparisons (with t-tests or linear contrasts), exactly as we did in 2-way ANOVA

3838

following up a 3following up a 3--way interaction way interaction in 3in 3--way factorial ANOVA:way factorial ANOVA:

overview of the stepsoverview of the stepssimple interaction effects– F tests

if simple interaction effects are significant, follow up with simple simple effects– F tests

if simple simple effects are significant with > 2 levels, follow up with simple simple comparisons– t-tests and linear contrasts

20

3939

following up a following up a 33--way interaction, part I:way interaction, part I:simple interaction effectssimple interaction effects

4040

so there’s this 3so there’s this 3--way interactionway interaction

need to focus your investigation1) go back to theory and hypotheses2) conduct follow-up analyses to test predictions

this could mean one or more of the following:personality X genderpersonality X gender 22--way interaction way interaction is different across levels of is different across levels of reinforcementreinforcement

reinforcement X personalityreinforcement X personality 22--way interaction way interaction is different across levels of is different across levels of gendergender

reinforcement X genderreinforcement X gender 22--way interaction way interaction is different across levels of is different across levels of personalitypersonality

21

4141

simple interaction effectssimple interaction effects• simple interaction effects break down the 3-way

interaction into a series of 2-way interactions at each level of the third factor

WHY WOULD WE DO THIS?• this gives a first close-up look at where the differences

between cell means might be• once we know this, we can follow up these simple 2-way

interactions further to figure out where the differences are(simple simple effects & simple simple comparisons / contrasts)just as we follow up an interaction in a 2-way design

in a 3-way design there are three potential follow-up steps (compared to two in a 2-way design)

4242

the graphs depicting the 2 x 2 x 3 interaction between gender, personality, and reinforcement provide a visual representation of the simple interaction effects we would conduct: here, the simple personality x reinforcement interaction at each of the two levels of gender

GENDER: 2.00 Female

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

GENDER: 1.00 Male

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

Males Females

22

4343

GENDER: 2.00 Female

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

GENDER: 1.00 Male

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

omnibus 2-way interaction in a 3-way design: ignores levels of the third factor

tests the 2-way P x R interaction with data averaged across gender

(i.e., ignoring the third factor)

simple 2-way interactions in a 3-way design: test the 2-way interaction at each level of the third factor

tests the 2-way P x R interaction for each gender group separately(i.e., at each level of third factor)

4444

don’t mix up your designsdon’t mix up your designssimple interaction effects in a 3-way design look like a series of 2-way ANOVAs at each level of the third factor(e.g. a P x R ANOVA for men, then a P x R ANOVA for women)But the F ratios calculated for these tests are not the same– simple interaction (a) uses pooled error term; (b) is conducted after significant 3-way interaction is observed

omnibus P x R 2-way interaction in 2-way designs:• separate P x R studies for men and women (e.g., at different times, in different recruiting contexts – confounds?)• MSerror taken from each 2-way omnibus ANOVA table

(different tables for men and women, so different MSerror)

simple P x R 2-way interaction in a 3-way design:• is a follow-up test for 3-way interaction (P x R x G)• men and women were sampled as part of the same study• MSerror taken from 3-way omnibus ANOVA table

23

4545

Source SS df MS F p

PR at G1 95725.00 2 47862.50 956.72 0.000PR at G2 91806.78 2 45903.39 917.56 0.000

Error 1200.67 24 50.03

critical F at alpha=.05 (2,24) = 3.40

summary table for P x Rsimple interaction effects

includes F tests for all P x R simple interaction effects (i.e., the P x R simple interaction effect at each level of the gender factor–men and women)

4646

Source SS df MS F p

PR at G1 95725.00 2 47862.50 956.72 0.000PR at G2 91806.78 2 45903.39 917.56 0.000

Error 1200.67 24 50.03


these are the SS values for the effects

degrees of freedom for a simple interaction effect are the df for the

associated interaction df = dfPR (2 - 1)(3 - 1) = 2

SSerror term (and df) is taken from the omnibus 2 x 2 x 3 ANOVA

mean squares and F values calculated

as usual

indicates that the personality x reinforcement interaction is significant for males and females

follow-up tests will identify where the differences are

24

4747

following up a 3following up a 3--way interaction,way interaction,part II: simple part II: simple simplesimple effects &effects &simple simple simplesimple comparisonscomparisons

4848

simplesimple simple effectssimple effectssimple effects after an omnibus 2simple effects after an omnibus 2--way way interaction examine the effect of factor A interaction examine the effect of factor A at each level of factor Bat each level of factor B

simple simple simplesimple effects effects are like simple effects are like simple effects except that they examine except that they examine the effect of factor A the effect of factor A at each level of factor B, at each level of factor C at each level of factor B, at each level of factor C (i.e., within each combo of B & C)(i.e., within each combo of B & C)

Simple Simple simplesimple effects differ from oneeffects differ from one--way way ANOVAs because they use the ANOVAs because they use the MSMSerrorerror from the from the omnibus ANOVA table as the error term omnibus ANOVA table as the error term

25

4949

simple simple effect # 1:personality at each level of reinforcement (separately for men and women)

results: for both men and women, the effect of personality at each level of reinforcement was significant (although opposite under neutral reinforcement!)

GENDER: 2.00 Female

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

GENDER: 1.00 Male

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

Males Females

5050

Source SS df MS F p

P at R1 at G1 43350.00 1 43350.00 866.52 0.000P at R2 at G1 13537.50 1 13537.50 270.60 0.000P at R3 at G1 4330.50 1 4330.50 86.56 0.000

P at R1 at G2 40837.50 1 40837.50 816.30 0.000P at R2 at G2 15000.00 1 15000.00 299.83 0.000P at R3 at G2 41002.7 1 41002.66 819.60 0.000

Error 1200.67 24 50.03


R1 = reward, R2 = neutral, R3 = punishmentG1 = men, G2 = women

summary tablesummary tablesimple simple simplesimple effects of effects of personalitypersonality,,

at each level of reinforcement, at each level of reinforcement, at each level of genderat each level of gender (males and females)(males and females)

26

5151

simple simple effect # 2:reinforcement at each level of personality (separately for men and women)

results: for both men and women, the effect of reinforcement at each level of personality was significant

GENDER: 2.00 Female

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

GENDER: 1.00 Male

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

Males Females

5252

Source SS df MS F p

R at P1 at G1 48350.00 2 24175.00 483.23 0.000R at P2 at G1 47400.00 2 23700.00 473.74 0.000

R at P1 at G2 45483.56 2 22741.78 454.58 0.000R at P2 at G2 46350.00 2 23175.00 463.24 0.000

Error 1200.67 24 50.03

critical F at alpha=.05 (2,24) = 3.40BUT reinforcement factor has > 2 levels: how do we know which cell means are significantly different from each other?

P1 = impulsive, P2 = anxiousG1 = men, G2 = women

summary tablesummary tablesimple simple simplesimple effects of reinforcement, effects of reinforcement,

at each level of personality, at each level of personality, at each level of gender at each level of gender (males (males and and females)females)

27

5353

simplesimple simple comparisonssimple comparisons

Like simple comparisons (or contrasts) except we Like simple comparisons (or contrasts) except we compute for each level of a third factorcompute for each level of a third factor

formulae from Lecture 3 can be used:formulae from Lecture 3 can be used:

abdferror −=nMSaLt

errorj∑=

2

∑= jj XaL

5454

GENDER: 2.00 Female

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

GENDER: 1.00 Male

Reinforcement

PunNeutRew

500

400

300

Personality

Imp

Anx

Males Females

some possible comparisons…

R1 and R2 vs R3 at P1 for G1 R2 vs R3 at P2 for G2

28

5555

Males Rew None Pun

Impusivity 320 355 490

Contrast 1 1 -1 0Contrast 2 1 1 -2

Anxiety 490 450 320


Reinforcement

simple simple comparisons simple simple comparisons for for reinforcementreinforcement at at

each level of each level of personalitypersonality (for (for malesmales) )

5656

calculations for impulsivity contrast 1calculations for impulsivity contrast 1

06.6

350.03)0)1(1(

00.35222

−=+−++

−=t

L = 2(66.88) – 1(66.88) – 1(35.63) = -35.63

t’α=.05 (24) = 2.39

(with Bonferroni adjustment for 2 comparisons)

L = 1(320) – 1(355) + 0(490) = -35.00

Males Rew None Pun

Impusivity 320 355 490


Anxiety 490 450 320


Reinforcement

29

5757

and so on for and so on for

–– impulsivity contrast 2impulsivity contrast 2–– anxiety contrast 1anxiety contrast 1–– anxiety contrast 2anxiety contrast 2–– then all four contrasts then all four contrasts

for femalesfor females

5858

oh, the followoh, the follow--up tests you’ll doup tests you’ll do

omnibus testsomnibus tests–– 7 (3 main effects, 3 two7 (3 main effects, 3 two--way interactions, 1 threeway interactions, 1 three--way interaction)way interaction)

simple interaction effectssimple interaction effects–– 2 (personality x reinforcement at each level of gender)2 (personality x reinforcement at each level of gender)

simple simple simplesimple effectseffects–– 10 (6 for personality at each level of reinforcement for men & women, 10 (6 for personality at each level of reinforcement for men & women,

4 for reinforcement at each level of personality for men & women)4 for reinforcement at each level of personality for men & women)

simple simple simplesimple comparisonscomparisons–– 8 (2 comparisons for each personality condition for men & women) 8 (2 comparisons for each personality condition for men & women)

total = 27 tests! (in this study)total = 27 tests! (in this study)

30

5959

but what if you don’t want to?but what if you don’t want to?conducting an exhaustive set of followconducting an exhaustive set of follow--up tests for higherup tests for higher--order factorial designs can inflate order factorial designs can inflate familywisefamilywise error rateerror rate27 tests from previous slide: each has Type27 tests from previous slide: each has Type--1 error of .051 error of .05

means a means a familywise alpha of 27 *.05 = 1.35 (!)

ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends entirely upon your research predictionsdepends entirely upon your research predictions

in our case we had (implicitly) predicted the personality x in our case we had (implicitly) predicted the personality x reinforcement interaction, and we were going to see if this reinforcement interaction, and we were going to see if this interaction was the same for males and females:interaction was the same for males and females:

•• impulsive impulsive reward not punishmentreward not punishment•• anxious anxious punishment but not rewardpunishment but not reward•• possible gender differencespossible gender differences

6060

steps for following up steps for following up a 3a 3--way interactionway interaction

is 3-way significant?

NO

calculate simple interaction effects at each level of least

important factor or according to hypotheses

YES

NOYES

does the factor have >2 levels?

NO YESconduct tests

for simple simple

comparisons

STOP

STOP

STOP

STOP

is A x B at C1 or C2

significant?

Conduct simple simple effects of key IV (A or

B). Are they significant?

NO

STOP

YES

31

6161

summaryin a 3-way ANOVA, you have main effects and 2-way interactions (like a 2-way ANOVA) plus a 3-way interaction

the follow-up tests for omnibus main effects and two-way interactions are like in a 2-way ANOVA –main effect comparisons, simple effects, simple comparisons

following up a 3-way interaction can be very complex and time-consuming

highlights the need for analyses to be driven by hypotheses

6262

Reporting Reporting some sources suggest that once you find a some sources suggest that once you find a

significant interaction you should ignore the significant interaction you should ignore the lower lower order effects order effects

––main main effects have been effects have been “qualified”“qualified” by the interaction by the interaction ––22--way interaction qualified by 3way interaction qualified by 3--wayway

this this is because the is because the higher order interactions higher order interactions maymayrequire you to change the interpretation given by the require you to change the interpretation given by the lower effect alone lower effect alone

––Partly depending on whether it’s a Partly depending on whether it’s a disordinaldisordinal interactioninteractionultimatelyultimately, there is no simple rule:, there is no simple rule:

––what what you report depends you report depends upon upon your research predictions your research predictions ––Usually, if Usually, if you predict you predict an effectan effect, then report that , then report that effect effect (and any follow(and any follow--up testsup tests))

32

6363

Reporting our 2 way Reporting our 2 way designdesign

“Results indicated a significant main effect of consumption, F(2,42) = 20.07, p<.001, ω2 = .34. Linear contrasts with a Bonferroni adjustment for 2 comparisons indicated that creativity ratings were significantly lower after 2 or 4 pints than after consuming no alcohol, t’(42) = 2.91, p<.05 (Ms = 63.75, 55.63), and were lower after 4 pints than after 2 pints, t’(42) = 5.63, p<.05 (Ms = 64.69, 46.56). There was no significant main effect for distraction, indicating that creativity ratings for distracted participants’ limericks (M = 56.46) were not significantly different from those for controls (M = 60.21), F(1,42) = 2.03, p = .16, ω2 = .01. There was, however, a significant interaction between consumption and distraction, indicating that the effect of consumption was different for distracted and control participants, F(1,42) =11.91, p<.001, ω2 = .20. The interaction is depicted in Figure 1.”NB Interaction needs following up in results section (simple effects + simple comparisons if nec.).Discuss: although the predicted main effect of alcohol consumption was significant, the direction of the effect was contrary to hypotheses: alcohol lowered creativity ratings. Also the predicted effect of distraction was not significant.

I haven’t put effect sizes in for the follow-up comparisons / contrasts; most do nowadays esp. if report Fs.

6464

reporting simple reporting simple effects effects it is preferable to not report all sets of simple effects, for 2 it is preferable to not report all sets of simple effects, for 2

reasons:reasons:a)a) the more simple effects we calculate, the greater our risk of making a the more simple effects we calculate, the greater our risk of making a

type 1 error (see Howell, p.436)type 1 error (see Howell, p.436)b)b) usually both sets of simple effects will communicate usually both sets of simple effects will communicate similarsimilar

information information -- redundancyredundancy–– so, in our case we would want to report so, in our case we would want to report eithereither the simple effects of the simple effects of

distraction (at each level of consumption) distraction (at each level of consumption) oror the simple effects of the simple effects of consumption (at each level of distraction)consumption (at each level of distraction)

ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends depends entirelyentirely upon your research predictions.upon your research predictions.

–– Let’s say we Let’s say we specifically predicted that “specifically predicted that “the effect of consumption on the effect of consumption on creativity ratings will be stronger for distracted participants than for creativity ratings will be stronger for distracted participants than for controls”. Therefore, we would want to report the simple effects for controls”. Therefore, we would want to report the simple effects for consumptionconsumption (and associated simple comparisons / contrasts)(and associated simple comparisons / contrasts)

33

6565

reportingreporting

“. . .To follow up the significant two-way interaction, the simple effects of consumption were analysed at each level of distraction. There was a significant simple effect of consumption for distracted participants, F(2,42) = 31.36, p<.001, ω2 = .56, but not for controls, F(2,42) = 0.61, p = .546, ω2 = .00. The significant simple effect of consumption for distracted participants was followed up with Linear contrasts using a Bonferroni adjustment for 2 comparisons. These indicated that, for distracted participants, creativity ratings were lower after 2 or 4 pints than after consuming no alcohol, t’(42) = 4.52, p<.001 (Ms = 66.88, 51.26), and also lower after 4 pints than after 2 pints, t’(42) = 6.86, p<.001 (Ms = 66.88, 35.63).”

I haven’t put effect sizes in for the follow-up comparisons / contrasts ; most do now . Effect sizes for simple effects are also required. NB if you calc w2 for controls it works out to -.01 – a meaningless % (% cannot be negative), so set to zero. Another reason some prefer to report eta2.

Discuss: the hypothesis was confirmed that the effect of consumption on hypothesis was confirmed that the effect of consumption on creativity will be stronger for distracted participants than for controls.creativity will be stronger for distracted participants than for controls.

6666

"Writing is easy; all you do is sit staring at "Writing is easy; all you do is sit staring at a blank sheet of paper until the drops of a blank sheet of paper until the drops of blood form on your forehead." blood form on your forehead." ---- Gene Gene FowlerFowler

"There's nothing to writing. All you do is "There's nothing to writing. All you do is sit down at a typewriter and open a vein." sit down at a typewriter and open a vein." ---- Red SmithRed Smith

34

6767

Reporting threeReporting three--way designsway designsconducting an exhaustive set of followconducting an exhaustive set of follow--up tests for up tests for higherhigher--order factorial designs can inflate familywise order factorial designs can inflate familywise alpha (and is very tedious!)alpha (and is very tedious!)

ultimately, there is no simple rule:ultimately, there is no simple rule: what you report what you report depends depends entirelyentirely upon your research predictionsupon your research predictions–– in our case we had (implicitly) predicted the Personality x in our case we had (implicitly) predicted the Personality x

Reinforcement interaction, and we were going to see if this Reinforcement interaction, and we were going to see if this interaction was the same for males and femalesinteraction was the same for males and females

•• people with an impulsive personality learn well from reward people with an impulsive personality learn well from reward but not punishment, and people with an anxious personality but not punishment, and people with an anxious personality learn well from punishment but not reward.learn well from punishment but not reward.

•• Possible gender differences not well understood.Possible gender differences not well understood.–– hence our write up might have gone something like thishence our write up might have gone something like this

6868

reportingreporting“The predicted interaction was significant, “The predicted interaction was significant, FF(2, 24) = (2, 24) = 1684.23, 1684.23, pp<.001<.001, but this was qualified by 3, but this was qualified by 3--way interaction way interaction among personality, reinforcement, and gender among personality, reinforcement, and gender FF(2, 24) = (2, 24) = 190.67, 190.67, pp<.001<.001. Simple interaction analyses revealed the . Simple interaction analyses revealed the personality x reinforcement interaction was significant for personality x reinforcement interaction was significant for both males, both males, FF(2, 24) = 956.72, (2, 24) = 956.72, pp<.001<.001, and females, , and females, FF(2, (2, 24) = 917.56, 24) = 917.56, pp<.001<.001. The simple simple effects of . The simple simple effects of personality were then analysed for each level of gender and personality were then analysed for each level of gender and reinforcement, and Table 1 presents the relevant means. reinforcement, and Table 1 presents the relevant means. For both genders, as predicted, under punishment anxious For both genders, as predicted, under punishment anxious participants were faster than impulsive participants, participants were faster than impulsive participants, FFs > s > 819.58, 819.58, psps<.001,<.001, while under reward impulsive participants while under reward impulsive participants were faster than anxious participants, were faster than anxious participants, FFs > 816.29, s > 816.29, psps<.001<.001. . However, in the neutral reinforcement condition the gender However, in the neutral reinforcement condition the gender difference emerged: impulsive males performed difference emerged: impulsive males performed better better than than anxious males anxious males FF(1, 24) = 270.60, (1, 24) = 270.60, pp<.001 (Ms = 355, 450)<.001 (Ms = 355, 450), , while impulsive females performed while impulsive females performed worseworse than anxious than anxious females, females, FF(2, 24) = 299.83, (2, 24) = 299.83, pp<.001<.001 (Ms = 450, 350)(Ms = 450, 350).”.”

I haven’t put effect sizes. These would be required for all tests these days.

35

6969

Table 1. Mean reaction time as a function of Table 1. Mean reaction time as a function of personality, reinforcement, and gender. personality, reinforcement, and gender.

Personality TypePersonality TypeImpulsiveImpulsive AnxiousAnxious

Reinforcement:Reinforcement:PunishmentPunishment

WomenWomen 485.33485.33aa 320.00320.00aaMenMen 490.00490.00aa 320.00320.00aa

NoneNoneWomenWomen 450.00450.00aa 350.00350.00aaMenMen 355.00355.00aa 450.00450.00aa

RewardRewardWomenWomen 320.00320.00aa 485.00485.00aaMenMen 320.00320.00aa 490.00490.00aa

Note. Subscripts within the row indicate significant simple Note. Subscripts within the row indicate significant simple simple effects of personality.simple effects of personality.

Most experimental journals would also want to see standard deviations.

7070

Education: That which discloses to the wise Education: That which discloses to the wise and disguises from the foolish their lack of and disguises from the foolish their lack of understanding. understanding. --Ambrose Bierce, writer Ambrose Bierce, writer (1842(1842--1914)1914)

36

7171

readingsreadings

between-Ps ANOVA III: higher-order designs (this lecture)Field (3rd ed): review Chapter 12Field (2nd ed): review Chapter 10Howell (all eds): Chapter 13 (pp 446-453)

power analyses and blocking designs (next lecture)Field (2nd or 3rd ed): no new readingsHowell (all eds): Chapter 8

Documents

Quiz 1 next Monday & Tuesday - University of Queenslanduqwloui1/stats/3010 for post... · 2018-08-02 · 2 Quiz 1 next Monday & Tuesday preparation • review Lectures 1 Æ4 • organize