8/04/2011
1
11
psyc3010 lecture 7psyc3010 lecture 7
analysis of covariance (ANCOVA)analysis of covariance (ANCOVA)
last lecture: correlation and regressionnext lecture: standard MR & hierarchical MR
(MR = multiple regression)
22
announcementsannouncementsquiz 2 quiz 2 –– correlation and regressioncorrelation and regression–– to be completed online May 15 and 16to be completed online May 15 and 16–– assesses material taught in Lectures 6, 7, 8, and 9assesses material taught in Lectures 6, 7, 8, and 9–– Lecture 10 will be a review of regression topics Lecture 10 will be a review of regression topics plus plus fun fun
material on mediation and indirect effectsmaterial on mediation and indirect effects–– practice questions and quiz will be posted on practice questions and quiz will be posted on
BlackboardBlackboard
assignment 2 assignment 2 –– Multiple RegressionMultiple Regression–– due on May 23 (Week 12)due on May 23 (Week 12)–– will learn all skills and concepts by Week 9will learn all skills and concepts by Week 9–– all files on Blackboard next week (Week 8)all files on Blackboard next week (Week 8)
8/04/2011
2
FeedbackFeedbackProsPros
Repetition, revision, elaboration Repetition, revision, elaboration helpful ||||| ||||| ||helpful ||||| ||||| ||Clear, Made it easy ||||| ||||Clear, Made it easy ||||| ||||Funny, Fun ||||| ||||Funny, Fun ||||| ||||Interactive, Engaging, ? breaks Interactive, Engaging, ? breaks good ||||| ||good ||||| ||Slides great, prepared ||||| |Slides great, prepared ||||| |Good pace |||| Good pace |||| Thorough, detailed ||||Thorough, detailed ||||Enthusiasm |||Enthusiasm |||Detailed summaries helpful | ; Detailed summaries helpful | ; Flow charts great | ; Great Flow charts great | ; Great examples | ; examples | ; LectopiaLectopia great |great |Lecture slides up early |; Caring | Lecture slides up early |; Caring | Lecture ended early |Lecture ended early |
ImproveImproveGo slower ||||| ||Go slower ||||| ||More review, repetition needed ||||| More review, repetition needed ||||| Make week 1 lecture slides Make week 1 lecture slides available early |||available early |||Trouble printing ||Trouble printing ||Explanations unclear ||Explanations unclear ||More entertainment ||More entertainment ||Jargon frustrating, boring ||Jargon frustrating, boring ||More practice questions ||More practice questions ||Changing terms/notation Changing terms/notation frustrating |frustrating |Assumed knowledge |Assumed knowledge |More real life examples |More real life examples |Key concepts slide at start |Key concepts slide at start |End early |End early |Spread slides out |Spread slides out |Use harder questions |Use harder questions |Simple Simple simplesimple comparisons / comparisons / effects exasperating |effects exasperating |
33
44
last lectures last lectures this lecturethis lecture2 lectures ago:2 lectures ago:–– importance of maximising power in research importance of maximising power in research
(maximising likelihood of correctly detecting effects that (maximising likelihood of correctly detecting effects that exist in the population, and rejecting Hexist in the population, and rejecting H00))
–– how blocking designs can increase powerhow blocking designs can increase power
last lecture:last lecture:–– correlation (association between two variables) and correlation (association between two variables) and
regression (association + prediction)regression (association + prediction)
this lecture:this lecture:–– an additional strategy to maximise power: an additional strategy to maximise power:
Analysis of Covariance (ANCOVA)Analysis of Covariance (ANCOVA)
8/04/2011
3
55
topics covered in this lecture topics covered in this lecture
review of blocking designs and introduction toanalysis of covariance (ANCOVA)
1st use of ANCOVA: reduce error variance
2nd use of ANCOVA: adjust treatment means
structural model & assumptions of ANCOVA
why ANCOVA is controversial
66
review of blocking review of blocking designs and designs and introduction to ANCOVAintroduction to ANCOVA
8/04/2011
4
77
how blocking designs can helphow blocking designs can helpproblemproblem = there is a lot of unexplained variance (“error = there is a lot of unexplained variance (“error variance”) in your 1variance”) in your 1--way experiment, and so the effect of way experiment, and so the effect of your focal IV is not statistically significantyour focal IV is not statistically significantsolutionsolution = add a 2= add a 2ndnd IV that you know will explain additional IV that you know will explain additional variance (based on previous research)variance (based on previous research)adding this adding this controlcontrol or or concomitantconcomitant variable changes the variable changes the design of your study (now a 2design of your study (now a 2--way factorial)way factorial)22ndnd IV explains additional systematic variance, and so now IV explains additional systematic variance, and so now there is less unexplained / residual / error variancethere is less unexplained / residual / error varianceincreased chance that your focal IV has a significant effect: increased chance that your focal IV has a significant effect:
error
treat
MSMSF =
ideally the same as in your 1-way design
ideally smaller than in your 1-way design
88
ancova ancova –– analysis of covarianceanalysis of covariancehas the same goal as blocking but works has the same goal as blocking but works differently:differently:–– blocking works at the blocking works at the level of designlevel of design –– the the
reduction in the size of the error term is a reduction in the size of the error term is a consequence of including a factor that explains a consequence of including a factor that explains a good proportion of variance in the DVgood proportion of variance in the DV
–– With ancova the error term is adjusted With ancova the error term is adjusted statisticallystatistically
8/04/2011
5
99
“remind me what is covariance?”“remind me what is covariance?”
variancevariance is the tendency for scoresis the tendency for scoresto vary around some mean value to vary around some mean value coco--variancevariance is the tendency for two is the tendency for two scores to vary scores to vary togethertogether–– If a participant’s score on one variable deviates from the mean, If a participant’s score on one variable deviates from the mean,
the score on the other (covarying) variable also deviates the score on the other (covarying) variable also deviates –– positive covariance = both deviate in the same direction positive covariance = both deviate in the same direction [ Review [ Review
pp. 236pp. 236--8 (Howell 68 (Howell 6thth ed.)ed.) if nec]if nec]
a a covariatecovariate is like the control variable used for is like the control variable used for blocking, with a couple of differences:blocking, with a couple of differences:–– the covariate is a the covariate is a continuouscontinuous variable and treated as such (i.e., variable and treated as such (i.e.,
participants are not matched at discrete levels)participants are not matched at discrete levels)–– in ancova, the covariate is used to remove error from both the in ancova, the covariate is used to remove error from both the
error term error term andand treatment effecttreatment effect
( ) 1/2
−−∑ XXij
( )( )1−
−−∑ ZZXX
1010
Analysis of CovarianceAnalysis of Covariance——ANCOVAANCOVA
Originally a technique for analysing Originally a technique for analysing experiments and removing nuisance experiments and removing nuisance variationvariationAttempt to Attempt to reduce error termreduce error term by measuring by measuring another variable and estimating its another variable and estimating its parametersparameters–– if the variable affects the DV and it is not part if the variable affects the DV and it is not part
of the statistical model for the ANCOVA, the of the statistical model for the ANCOVA, the variable is in the unmeasured ‘error’variable is in the unmeasured ‘error’
8/04/2011
6
1111
Ho H1
μo μ1 α
power
β
1212
Ho H1
μo μ1 α
power
β
Use ANCOVA to reduce error (just like with
blocking)
8/04/2011
7
1313
ANCOVAANCOVAAll forms of ANOVA can be performed with a All forms of ANOVA can be performed with a covariate (or several)covariate (or several)A covariate is another IV/predictor in the modelA covariate is another IV/predictor in the model–– but continuous (ordered scores, not discrete groups)but continuous (ordered scores, not discrete groups)
Can reduce Can reduce errorerror termterm——ifif it is related to the DVit is related to the DVif unrelated you lose DF (lose power) without if unrelated you lose DF (lose power) without compensatory reduction in error (i.e., bad tradecompensatory reduction in error (i.e., bad trade--off) off)
1414
Uses of ANCOVAUses of ANCOVA
1.1. To control unwanted variation that would To control unwanted variation that would otherwise inflate the error with which we otherwise inflate the error with which we test our models (classical usage)test our models (classical usage)
2.2. To control for group differences, esp. in To control for group differences, esp. in the analysis of clinical trials or other the analysis of clinical trials or other pre/post designs (controversial, see pre/post designs (controversial, see Howell 16.5)Howell 16.5)
8/04/2011
8
1515
OnewayOneway ANCOVA structural modelANCOVA structural model
Covariate is just another source of varianceCovariate is just another source of varianceUse the term Use the term βΖβΖijij because of continuous because of continuous nature;nature;Score on DV goes up or down depending on Score on DV goes up or down depending on score on Zscore on ZImplicitly, we have specified Implicitly, we have specified no interactionno interactionbetween covariate and the IVbetween covariate and the IV–– the presence of such an interaction is a violation of the presence of such an interaction is a violation of
ANCOVA assumptionsANCOVA assumptions•• stats software normally provides output to check as a defaultstats software normally provides output to check as a default•• Howell includes interaction in the modelHowell includes interaction in the model
XXijij = = μ + μ + ααjj + + βΖβΖijij + + εεijij Error
X (the DV) for participant I in Jth group
Grand mean 1st IV – factor A – group j
2nd IV – score on variable Z multiplied by a fixed weight (beta)
1616
XXijkijk = = μμ + + ααj j + + ββk k + + αβαβjkjk + + eeijkijk
the structural modelsthe structural models
XXijij = = μμ + + ααj j + + ββZZij ij + e+ eijkijk
Factorial ANOVA model
One-way ANCOVA model
XXijij = = μμ + + ττjj + e+ eijijOne-way ANOVA model
βs are not the same!!
8/04/2011
9
1717
how ANCOVA reduces error variancehow ANCOVA reduces error variancecovariatecovariate = another IV or predictor in the model= another IV or predictor in the model–– but continuous (ordered scores, not discrete groups)but continuous (ordered scores, not discrete groups)
if the covariate is associated with the DV:if the covariate is associated with the DV:–– this relationship accounts for some systematic this relationship accounts for some systematic
variance unexplained by the focal IV variance unexplained by the focal IV –– accounting for this systematic variance reduces the accounting for this systematic variance reduces the
amount of unexplained variance in the designamount of unexplained variance in the design
–– A smaller error term because we’ve A smaller error term because we’ve partionedpartioned out out the variance due to the covariate means an increase the variance due to the covariate means an increase in statistical power in testing the effect of the focal in statistical power in testing the effect of the focal IV (just as when using blocking designs)IV (just as when using blocking designs)
1818
an example research studyan example research study
comparison of driving performance with three different car sizes: are smaller cars easier to handle?
easily addressed using 1-way ANOVA:
– DV: handling rating after 10 laps on set course– 3 performance cars are compared:
• BMW Z3 (small)• Subaru WRX (medium)• Ford GTP (large)
different groups of drivers used for each condition
8/04/2011
10
1919
μ1 μ2μo
• results show lots of overlapping variance
• this indicates a large error term
• this results in low power
to reduce the variance, we could identify a a covariate –which past research tells us is related to the DV
in this case:driving experience
handling
2020
driving experience
hand
ling
8/04/2011
11
2121
driving experience
hand
ling
2222
handling
mean handling scores
for each car
8/04/2011
12
2323
driving experience
hand
ling
mean handling scores for each car
juxtaposed with actual scores
on handling and driving experience
2424
2jijerror )XX(SS ∑ −=
in a regular ANOVA, SSerror = the sumof the squared
deviations of scores around their group
meansdriving experience
hand
ling
8/04/2011
13
2525
a lot of the variance is due to the relationship between experience and handling –we can remove this from the error term first
driving experience
hand
ling
2626
reduce error by computing pooled error term based on deviations around each group’s regression slope
driving experience
hand
ling
slope of regression line describes avg. covariance between the two variables
8/04/2011
14
2727
2)(_ ∑ −= jijerror XXSSUnadjusted
driving experience
hand
ling
the DV scores are not clustered around the mean based on random chance alone
they vary systematically (based on relationship with covariate)
unadjusted error includes
the chance variance + covariance
2828
adjusted error
driving experience
hand
ling
Estimate covariate’s effects with a regression line Calculate error as deviation from the Yhat instead of
the mean YIf the covariate is related to the DV, the regression
line is a better “anchor” around which scores cluster (smaller error)
8/04/2011
15
2929
As an adolescent I aspired to lasting fame, I As an adolescent I aspired to lasting fame, I craved factual certainty, and I thirsted for a craved factual certainty, and I thirsted for a meaningful vision of human life meaningful vision of human life -- so I so I became a scientist. became a scientist. This is like becoming This is like becoming an archbishop so you can meet girls. an archbishop so you can meet girls. ---- Matt Cartmill, anthropology professor Matt Cartmill, anthropology professor and author (1943and author (1943-- ))
3030
how does that do anything how does that do anything different different from blockingfrom blocking??
at this stage it does notat this stage it does not–– the effects of the covariate are subtracted from the the effects of the covariate are subtracted from the
error term, making it smaller error term, making it smaller –– The covariate is a more powerful way to do this if the The covariate is a more powerful way to do this if the
control variable is continuous, but it’s conceptually control variable is continuous, but it’s conceptually the samethe same
the next thing the next thing ancovaancova does is quite different does is quite different –– treatment means treatment means are adjusted to account for differences on are adjusted to account for differences on
the covariate the covariate –– random random assignment to IV conditions normally prevent assignment to IV conditions normally prevent
differences in covariate means (confounds should be designed differences in covariate means (confounds should be designed out)out)
–– But in case covariate does differ across groups, ANCOVA But in case covariate does differ across groups, ANCOVA effectively effectively partials outpartials out the effects of the covariate from the the effects of the covariate from the focal IV as well as the error termfocal IV as well as the error term
8/04/2011
16
3131
ANCOVA adjusts treatment means (DV)ANCOVA adjusts treatment means (DV)
why we would do this
if focal IV affects DV scores there is a significant difference among treatment means between the levels of the IV ☺if covariate also differs between levels of focal IV which variable explains difference in DV treatment means? confound!we care about the effect of the focal IV, not the effect of the covariateANCOVA teases apart the effects of the covariate ANCOVA teases apart the effects of the covariate and the IV by asking the question:and the IV by asking the question:“would the focal IV have an effect on the DV “would the focal IV have an effect on the DV if all participants were equivalent on the if all participants were equivalent on the covariate?”covariate?”
3232
How ANCOVA adjusts treatment means How ANCOVA adjusts treatment means on the DVon the DV
problem: participants in each level of focal IV also differ in their scores on the covariate variable
solution: Calculate the overall covariate mean. We assume this is the population mean. In an unconfounded population, all groups of the focal IV are assumed have this covariate mean.For your sample, if a group’s mean is different on the covariate than the overall covariate mean, that is a confound.Adjust the group’s mean on the DV to be what it would be if the group’s covariate mean were the overall covariate mean, by using the regression lineusing the regression line
8/04/2011
17
3333
driving experience
hand
ling
3434
in this case, there are no differences between the groups on the covariate, as you would expect, given random assignmentdriving experience
hand
ling
8/04/2011
18
3535
in this case, the 3 conditions have different covariate means
confound:what’s causing the difference in DV scores?
driving experience
hand
ling
mean scores meet on regression line
3636
1. calculate overall covariate mean
2. adjust DV scores according to regression line
3. test group main effect using adjusted meansdriving experience
hand
ling
shift DV scores to new point on regression line
8/04/2011
19
3737
here, observe a larger effect for car if adjust means so each group has average driving experience
driving experience
hand
ling
3838
logic of ANCOVAlogic of ANCOVAadjusted treatment means assume that covariate means are the same at each level of the focal IVthus, any differences in the adjusted treatment means can be attributed to the focal IV only
“would groups differ on the DV “would groups differ on the DV ifif they were they were equivalent on the covariate?”equivalent on the covariate?”refines error termrefines error term by subtracting variation that is by subtracting variation that is predictable from covariatepredictable from covariate–– larger adjustment when covariatelarger adjustment when covariate--DV relationship DV relationship
is strongis strong
refines treatment effectrefines treatment effect to adjust for any to adjust for any systematic group differences on covariate that systematic group differences on covariate that existed before experimental treatmentexisted before experimental treatment
8/04/2011
20
3939
comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA
Tests of Between-Subjects Effects
Dependent Variable: ATTRACT
231.780 2 106.890 .710 .5062263.330 15 150.8902477.110 17
SourceCarErrorTotal
Type III Sumof Squares df Mean Square F Sig.
DV = handling
effect is not significant1-way ANOVA
4040
Tests of Between-Subjects Effects
Dependent Variable: ATTRACT
213.780 2 106.890 4.180 .0521933.780 2 966.890 37.830 .000
99.550 4 24.890 .970 .469230.000 9 25.560
2477.110 17
SourceCarExperienceCar x ExperienceErrorTotal
Type III Sumof Squares df Mean Square F Sig.
reduction of error term from 150.89
to 25.56
blocking, using factorial ANOVA
DV = handling (block on experience – e.g., no training, some training, professional)
comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA
effect is marginally significant
8/04/2011
21
4141
Tests of Between-Subjects Effects
Dependent Variable: ATTRACT
252.040 2 126.020 8.697 .0031833.780 1 1833.780 126.555 .000202.880 14 14.490
2477.110 17
SourceCarRegressionErrorTotal
Type III Sumof Squares df Mean Square F Sig.
1-way ANCOVA
DV = handling (experience as a continuous scale, included as a covariate)
reduction of error
term from 150.89 to
14.49
increase in treatment effect from 106.89 to 126.02
effect is now significant!
comparison of results using 1-way ANOVA, blocking, & 1-way ANCOVA
4242
structural model and structural model and assumptions of ANCOVAassumptions of ANCOVA
8/04/2011
22
4343
ANCOVA vs blocking
blockingblocking–– conceptually simplerconceptually simpler–– requires fewer assumptionsrequires fewer assumptions
ANCOVAANCOVA–– easier to administereasier to administer–– can use continuous covariatecan use continuous covariate–– removes effect from error term removes effect from error term andand DVDV
•• usefuluseful in two situations:in two situations:–– covariate related to IV covariate related to IV andand DV (confound)DV (confound)–– covariate related to DV covariate related to DV onlyonly
does does require specific assumptionsrequire specific assumptions
4444
assumptions of ANCOVAassumptions of ANCOVAall the regular ANOVA assumptions:all the regular ANOVA assumptions:–– homogeneous variancehomogeneous variance–– normal distributionnormal distribution–– independence of errorsindependence of errors
plus:plus:–– relationship between covariate and DV is relationship between covariate and DV is linearlinear–– relationship between covariate and DV is linear relationship between covariate and DV is linear
within each groupwithin each group–– relationship between DV and covariate is equal relationship between DV and covariate is equal
across treatment groups across treatment groups -- homogeneity of homogeneity of regression slopesregression slopes
see Lecture 2 and 2nd year stats notes
8/04/2011
23
4545
re: assumption 1re: assumption 1
covariate
DV
Linear relationships
☺
Non-linear relationships
Non-linear relationships generally cannot be detected with ANCOVA – degrades power.
covariate
DV
4646
re: assumption 3re: assumption 3
covariate
DV
covariate
DV
homogeneity of regression slopes
☺
heterogeneity of regression slopes
homogeneity of regression slopes is important because adjustments to treatment means are based upon an average within-cell regression coefficient
8/04/2011
24
4747
4848
adjusting treatment effects: the fine print
the process is still considered questionable
some people object to the idea of comparing some people object to the idea of comparing adjusted treatment means at all adjusted treatment means at all –– “real” observed means are not compared“real” observed means are not compared–– comparison means are estimated using regression slope, comparison means are estimated using regression slope,
which may not be reliablewhich may not be reliable–– if treatment group does affect the covariate as well as the if treatment group does affect the covariate as well as the
DV, what does the adjusted DV mean really tell you?DV, what does the adjusted DV mean really tell you?
some people don’t mind adjusted means when the some people don’t mind adjusted means when the adjustment makes the treatment effect largeradjustment makes the treatment effect larger–– but it doesn’t always make the treatment effect larger, so but it doesn’t always make the treatment effect larger, so
it doesn’t always work in your favor!it doesn’t always work in your favor!
8/04/2011
25
4949driving experience
hand
ling
adjustment has no effecton mean differences
example A
5050driving experience
hand
ling
example B
8/04/2011
26
5151driving experience
hand
ling
adjustment increasesmean differences
example B
5252driving experience
hand
ling
example C
8/04/2011
27
5353driving experience
hand
ling
adjustment decreasesmean differences
example C
5454
A final comparisonA final comparisonThe strength of ANCOVA is The strength of ANCOVA is the ability to handle the ability to handle continuous datacontinuous data–– most psychological variables are continuously most psychological variables are continuously
distributed, distributed, –– splitting splitting people into groups is inefficient (lose info) people into groups is inefficient (lose info)
and error prone (and error prone (mismis--categorisation at group categorisation at group boundaries magnifies error)boundaries magnifies error)
–– if your data is continuous, it is best to analyse it using if your data is continuous, it is best to analyse it using a method which can deal with such data (ANCOVA is a method which can deal with such data (ANCOVA is more powerful than Blocking)more powerful than Blocking)
If adjusted and observed means are very If adjusted and observed means are very different, concerns re interpretation arisedifferent, concerns re interpretation arise
8/04/2011
28
5555
readingsreadings
analysis of covariance (this lecture)Field (3rd ed): Chapter 11Field (2nd ed): Chapter 9Howell (all eds): Chapter 16
standard & hierarchical multiple regression(next lecture)
Field (3rd ed): Chapter 7Field (2nd ed): Chapter 5Howell (all eds): Chapter 15