PSY 1950Fixed and Random Effects

October 20, 2008

Preamble• Midterm

– Review– Room

• Homework• Simple effects

– Simple main effects– Simple interaction effects

The Embarrassing Footnote• With fixed effects analysis, one can’t generalize beyond measured levels of factor– e.g., the influence of expert communicators

The Embarrassing Footnote

• Fixed effects and random effects analyses treat different variables as randomly sampled– For FE, randomly sampled variable is subject

– For RE, randomly sampled variable is factor

• Fixed effects and random effects analyses use different error term (i.e., the denominator in an F-ratio) – For FE, error term is within-group variance– For RE, error term is interaction term

• Why?– If interactions are present, random sampling of levels introduces additional variability that MSwithin does not capture

Random Effects

Example• 4 College x 3 Test

Test is fixed factor, college is random factor

College Test1 Test2 Test3 mean1 28 12 20 202 12 28 20 203 12 28 20 204 12 28 20 20

mean 16 24 20 20

College Test1 Test2 Test3 mean1-250 28 12 20 20

251-500 12 28 20 20mean 20 20 20 20

Ffixed effect = MStest /MSwithin

Frandom effect = MStest /MStest x college

Fixed vs. Random Effects• Generalization

– FE: no generalization beyond measured levels

– RE: generalization beyond measured levels

• Selection of levels– FE: nonrandom– RE: random

• Interest in levels– FE: focused (e.g., planned/post-hoc tests)

– RE: general• Replication

– FE: same levels– RE: different levels

The Weak Test of Generality• RE analyses sacrifice power for generality– Reduction in F-ratio– Reduction in df

• One on hand… fixed effects– Power, but no generality

• On the other hand… random effects– Generality, but no power

Minimizing the Dilemma• RE model• Huge main effect• Small interaction effect• Many levels of random factor

• FE model• Representative levels of ordered

factor• e.g., age, angle of rotation

• Exhaustive levels• e.g., gender

Item analyses• Prof. Snedeker: “Psycholinguists do one thing which is different from most areas of psychology. We do all of our analyses twice: once with subject as the random variable (averaging across items), and once with item as the random variable (averaging across subjects). The goal is to understand whether the results generalize both to the population of possible participants and to the population of possible items (words, sentences etc). I don't expect the stats course to cover this (though it might help the students grasp the notion of a random variable)”

t-test is Special Case of ANOVA (k=2)

Contrast Weighting w/ Zero• With odd number of groups, contrast weights for some trends require weight of zero– e.g., linear trend w/ 3 groups: -1, 0, 1

a1 a2 a3-1 0 1

M1 M2 M3

2 3 4

a1M1 a2M2 a3M3

-2 0 4

ANOVA Effect Size: Eta

Advantages: conceptual simplicityDisadvantages: biased, depends on other factors/effects, depends on design/blocking

Advantages: does not depend on other factors/effects

Disadvantages: biased, conceptually complexity, depends on design/blocking

ANOVA Effect Size: Beyond Eta

• Omega-squared (2) and partial omega-squared (partial 2)– Not biased estimators of population effect size

– Better than eta for inferential purposes

• Generalized eta and omega– cf. Bethany’s presentation– Correct/control for research design

•Independent measures ANOVA and dependent measures ANOVA designs that investigate the same effect produce comparable effect sizes

ANOVA Assumption #1• Normality of sampling distribution of means– Not normality of raw sample data– Not normality of population– CLT says that sampling distribution of means is normal if:•Population is normal•Sample size is large (>30)

ANOVA Assumption #2• Independence of errors

– Group– Time/sequence– Space

ANOVA Assumption #3• Homogeneity of variances

– Population variances, not sample variances