MANOVA Basics

MANOVA BasicsMANOVA Basics

Lecture 10Lecture 10

Psy 524Psy 524

Andrew AinsworthAndrew Ainsworth

What is MANOVAWhat is MANOVA

Multivariate Analysis Multivariate Analysis of Varianceof Variance an extension of ANOVA in which an extension of ANOVA in which

main effects and interactions are main effects and interactions are assessed on a combination of DVsassessed on a combination of DVs

MANOVA tests whether mean MANOVA tests whether mean differences among groups on a differences among groups on a combination of DVs is likely to combination of DVs is likely to occur by chanceoccur by chance

MANOVAMANOVA

A new DV is created that is a linear A new DV is created that is a linear combination of the individual DVs that combination of the individual DVs that maximizes the difference between groups.maximizes the difference between groups.

In factorial designs a different linear In factorial designs a different linear combination of the DVs is created for each combination of the DVs is created for each main effect and interaction that main effect and interaction that maximizes the group difference maximizes the group difference separately.separately.

Also when the IVs have more than one Also when the IVs have more than one level the DVs can be recombined to level the DVs can be recombined to maximize paired comparisonsmaximize paired comparisons

MANCOVAMANCOVA

is the multivariate extension of is the multivariate extension of ANCOVA where the linear combination ANCOVA where the linear combination of DVs is adjusted for by one or more of DVs is adjusted for by one or more continuous covariates.continuous covariates.

A covariate is a variable that is A covariate is a variable that is related to the DV, which you can’t related to the DV, which you can’t manipulate, but you want to removes manipulate, but you want to removes its (their) relationship from the DV its (their) relationship from the DV before assessing differences on the before assessing differences on the IVs.IVs.

Basic requirementsBasic requirements

2 or more DVs (I, R)2 or more DVs (I, R)

1 or more categorical IVs (N, O)1 or more categorical IVs (N, O)

for MANCOVA you also need 1 or for MANCOVA you also need 1 or more continuous CVs (I, R)more continuous CVs (I, R)

MANOVA advantages MANOVA advantages over ANOVAover ANOVA By measuring multiple DVs you increase By measuring multiple DVs you increase

your chances for finding a group your chances for finding a group differencedifference

With a single DV you “put all of your eggs With a single DV you “put all of your eggs in one basket”in one basket”

Multiple measures usually do not “cost” a Multiple measures usually do not “cost” a great deal more and you are more likely great deal more and you are more likely to find a difference on at least one.to find a difference on at least one.

MANOVA advantages MANOVA advantages over ANOVAover ANOVA Using multiple ANOVAs inflates Using multiple ANOVAs inflates

type 1 error rates and MANOVA type 1 error rates and MANOVA helps control for the inflationhelps control for the inflation

Under certain (rare) conditions Under certain (rare) conditions MANOVA may find differences MANOVA may find differences that do not show up under ANOVAthat do not show up under ANOVA

Under most circumstance the more Under most circumstance the more complex an analysis becomes the less complex an analysis becomes the less power there ispower there is

Kinds of Research Kinds of Research Questions asked by Questions asked by MANOVAMANOVA The questions are mostly the The questions are mostly the

same as ANOVA just on the same as ANOVA just on the linearly combined DVs instead linearly combined DVs instead just one DV.just one DV.

Are there any main Are there any main effects?effects? Holding all other effects constant, Holding all other effects constant,

is a difference among groups is a difference among groups greater than expected by chance?greater than expected by chance?

Are there any main Are there any main effects?effects? Holding constant can mean:Holding constant can mean:

– Controlling for other effects by Controlling for other effects by averaging over them in a factorial averaging over them in a factorial designdesign

– Holding extraneous variables constant Holding extraneous variables constant or counterbalancing/randomizing their or counterbalancing/randomizing their effectseffects

– Using covariates to adjust the Using covariates to adjust the composite DV in order to create a composite DV in order to create a state of “pseudo equality”state of “pseudo equality”

Are there any main Are there any main effects?effects? Tests of main effects are Tests of main effects are

orthogonal if they are completely orthogonal if they are completely crossed and equal samples in crossed and equal samples in each cell, they are only linked by each cell, they are only linked by a common error terma common error term

Are there any Are there any interactions among interactions among the IVs?the IVs?

Does change in the linearly combined DV for Does change in the linearly combined DV for one IV depend on the levels of another IV?one IV depend on the levels of another IV?

For example: Given three types of For example: Given three types of treatment, does one treatment work better treatment, does one treatment work better for men and another work better for women?for men and another work better for women?

If equal samples in each cell then one If equal samples in each cell then one interaction is independent of main effects interaction is independent of main effects and other interactions.and other interactions.

Which DVs are most Which DVs are most important?important?

If there are any significant main If there are any significant main effects or interactions, on which effects or interactions, on which individual DV is there the most individual DV is there the most change (difference), if any, “caused” change (difference), if any, “caused” by the levels of the IV?by the levels of the IV?

You can follow a significant MANOVA You can follow a significant MANOVA with individual ANOVAs in order to with individual ANOVAs in order to see on which DV is there large, see on which DV is there large, medium, small or no effect.medium, small or no effect.

Which DVs are most Which DVs are most important?important? Another procedure is the Roy-Another procedure is the Roy-

Bargman step-down procedure Bargman step-down procedure which uses ANCOVA on each which uses ANCOVA on each individual DV, with higher priority individual DV, with higher priority DVs as covariates.DVs as covariates.

What are the What are the parameter estimates?parameter estimates?

Marginal means are the best population Marginal means are the best population estimates for the main effects and cell estimates for the main effects and cell mean are the best estimates of mean are the best estimates of interactionsinteractions

When the Roy-Bargman step procedure is When the Roy-Bargman step procedure is used then the interpretation is the best used then the interpretation is the best estimates of adjusted population valuesestimates of adjusted population values

All parameters are accompanied by All parameters are accompanied by standard error and/or confidence intervals.standard error and/or confidence intervals.

Which levels of the IV Which levels of the IV are significantly are significantly different?different? If there are significant main effects If there are significant main effects

on IVs with more than two levels on IVs with more than two levels than you need to test which levels than you need to test which levels are different from each otherare different from each other

And if there are interactions the And if there are interactions the interactions need to be taken apart interactions need to be taken apart so that the specific causes of the so that the specific causes of the interaction can be uncovered.interaction can be uncovered.

How strong is the How strong is the IV(s)/composite DV IV(s)/composite DV association?association? What is the proportion of the What is the proportion of the

composite DV explained by each composite DV explained by each IV?IV?

You can also then pick out the You can also then pick out the strength of association between strength of association between the IVs and each DV separately.the IVs and each DV separately.

Does use of covariates Does use of covariates significantly adjust the composite significantly adjust the composite DV scores?DV scores?

The test of sphericity in repeated The test of sphericity in repeated measures ANOVA is often violatedmeasures ANOVA is often violated

Corrections include:Corrections include:– adjustments of the degrees of freedom adjustments of the degrees of freedom

(e.g. Huynh-Feldt adjustment)(e.g. Huynh-Feldt adjustment)– decomposing the test into multiple decomposing the test into multiple

paired tests (e.g. trend analysis) or paired tests (e.g. trend analysis) or – treating the repeated levels as treating the repeated levels as

multiple DVs (e.g. profile analysis multiple DVs (e.g. profile analysis which we will talk about next)which we will talk about next)

Can MANOVA be used when Can MANOVA be used when assumptions are violated in assumptions are violated in repeated measure ANOVA?repeated measure ANOVA?

Assumptions of Assumptions of MANOVAMANOVA

Theoretical Theoretical ConsiderationsConsiderations The interpretation of MANOVA results The interpretation of MANOVA results

are always taken in the context of the are always taken in the context of the research design. research design.

Once again, fancy statistics do not Once again, fancy statistics do not make up for poor designmake up for poor design

Use of IVs change the interpretation of Use of IVs change the interpretation of other IVs, so choice of IVs to include other IVs, so choice of IVs to include needs to be thought about carefullyneeds to be thought about carefully

Theoretical Theoretical ConsiderationsConsiderations

Choice of DVs also needs to be carefully Choice of DVs also needs to be carefully considered, highly correlated DVs severely considered, highly correlated DVs severely weaken the power of the analysis.weaken the power of the analysis.

Choice of the order in which DVs are Choice of the order in which DVs are entered in the stepdown analysis has an entered in the stepdown analysis has an impact on interpretation, DVs that are impact on interpretation, DVs that are causally (in theory) more important need causally (in theory) more important need to be given higher priorityto be given higher priority

Generalizability is limited to the Generalizability is limited to the population studiedpopulation studied

Missing data, unequal samples, Missing data, unequal samples, number of subjects and powernumber of subjects and power

Missing data needs to be handled in the Missing data needs to be handled in the usual waysusual ways

Unequal samples cause non-orthogonality Unequal samples cause non-orthogonality and the total sums of squares is less than and the total sums of squares is less than all of the effects and error added up. This all of the effects and error added up. This is handled by using either:is handled by using either:– Type 3 sums of squares assumes the data was Type 3 sums of squares assumes the data was

intended to be equal and the lack of balance intended to be equal and the lack of balance does not reflect anything meaningfuldoes not reflect anything meaningful

– Type 1 sums of square which weights the Type 1 sums of square which weights the samples by size and emphasizes the difference samples by size and emphasizes the difference in samples is meaningfulin samples is meaningful


You need more cases than DVs in every You need more cases than DVs in every cell of the design and this can become cell of the design and this can become difficult when the design becomes difficult when the design becomes complexcomplex

If there are more DVs than cases in any If there are more DVs than cases in any cell the cell will become singular and cell the cell will become singular and cannot be inverted. If there are only a cannot be inverted. If there are only a few cases more than DVs the few cases more than DVs the assumption of equality of covariance assumption of equality of covariance matrices is likely to be rejected.matrices is likely to be rejected.


Plus, with a small cases/DV ratio power Plus, with a small cases/DV ratio power is likely to be very small and the chance is likely to be very small and the chance of finding a significant effect, even of finding a significant effect, even when there is one, is very unlikelywhen there is one, is very unlikely

you can use programs like GANOVA to you can use programs like GANOVA to calculate power in MANOVA designs or calculate power in MANOVA designs or you can estimate it by picking the DV you can estimate it by picking the DV with the smallest effect expected and with the smallest effect expected and calculate power on that variable in a calculate power on that variable in a univariate methodunivariate method


Power in MANOVA also depends on Power in MANOVA also depends on the relationships among the DVs. the relationships among the DVs. – Power is highest when the pooled Power is highest when the pooled

within cell correlation is high and within cell correlation is high and negative. If the pooled within negative. If the pooled within correlation is positive, zero or correlation is positive, zero or moderately negative the power is moderately negative the power is much lessmuch less

Multivariate normalityMultivariate normality

assumes that the means of the assumes that the means of the various DVs in each cell and all various DVs in each cell and all linear combinations of them are linear combinations of them are normally distributed.normally distributed.

Difficult to show explicitlyDifficult to show explicitly

Multivariate normalityMultivariate normality

In univariate tests robustness against In univariate tests robustness against violation of the assumption is assured when violation of the assumption is assured when the degrees of freedom for error is 20 or the degrees of freedom for error is 20 or more and equal samplesmore and equal samples

If there is at least 20 cases in the smallest If there is at least 20 cases in the smallest cell the test is robust to violations of cell the test is robust to violations of multivariate normality even when there is multivariate normality even when there is unequal n.unequal n.

If you have smaller unbalanced designs than If you have smaller unbalanced designs than the assumption is assessed on the basis of the assumption is assessed on the basis of researcher judgment.researcher judgment.

Absence of outliersAbsence of outliers

univariate and multivariate univariate and multivariate outliers need to be assessed in outliers need to be assessed in every cell of the designevery cell of the design

LinearityLinearity

MANOVA and MANCOVA assume MANOVA and MANCOVA assume linear relationships between all linear relationships between all DVs, all CVs and all DV/CV pairsDVs, all CVs and all DV/CV pairs

LinearityLinearity

Deviations from linearity reduce Deviations from linearity reduce the power of the test because:the power of the test because:

– the linear combination of DVs does the linear combination of DVs does not maximize the difference not maximize the difference between the groupsbetween the groups

– the CVs do not maximally adjust the the CVs do not maximally adjust the error.error.

Homogeneity of Homogeneity of regressionregression no IV by CV interactionno IV by CV interaction

Reliability of CVs and Reliability of CVs and DVsDVs reliability of CVs discussed reliability of CVs discussed

previously.previously.

In the stepdown procedure in In the stepdown procedure in

order for proper interpretation of order for proper interpretation of the DVs as CVs the DVs should the DVs as CVs the DVs should also have reliability in excess of .8also have reliability in excess of .8

Absence of Absence of Multicollinearity/SingularityMulticollinearity/Singularity

in each cell of the design. in each cell of the design.

You do not want redundant DVs You do not want redundant DVs or CVsor CVs

Homogeneity of Homogeneity of Covariance MatricesCovariance Matrices this is the multivariate equivalent of this is the multivariate equivalent of

homogeneity of variance. homogeneity of variance.

Assumes that the Assumes that the variance/covariance matrix in each variance/covariance matrix in each cell of the design is sampled from cell of the design is sampled from the same population so they can be the same population so they can be reasonably pooled together to make reasonably pooled together to make an error terman error term

Homogeneity of Homogeneity of Covariance MatricesCovariance Matrices If sample sizes are equal MANOVA If sample sizes are equal MANOVA

has been shown to be robust to has been shown to be robust to violations so Box’s M test can be violations so Box’s M test can be ignored (it is highly sensitive ignored (it is highly sensitive anyway)anyway)

Homogeneity of Homogeneity of Covariance MatricesCovariance Matrices If sample sizes are unequal than If sample sizes are unequal than

evaluate Box’s M test at alpha evaluate Box’s M test at alpha < .001. If this is met than a < .001. If this is met than a violation has occurred and the violation has occurred and the robustness of the test is robustness of the test is questionable. questionable.

Homogeneity of Homogeneity of Covariance MatricesCovariance Matrices

Look than at the data:Look than at the data:– If cells with larger samples have larger If cells with larger samples have larger

variances than the test is most likely variances than the test is most likely robustrobust

– If the cells with fewer cases have larger If the cells with fewer cases have larger variances than only null hypotheses are variances than only null hypotheses are retained with confidence but to reject retained with confidence but to reject them is questionable. Use of a more them is questionable. Use of a more stringent criterion (Pillai’s criteria stringent criterion (Pillai’s criteria instead of Wilk’s; more on this later)instead of Wilk’s; more on this later)

Documents

MANOVA Basics