23
LECTURE 17 MANOVA

LECTURE 17 MANOVA. Other Measures Pillai-Bartlett trace, V Multiple discriminant analysis (MDA) is the part of MANOVA where canonical roots are

  • View
    219

  • Download
    0

Embed Size (px)

Citation preview

LECTURE 17MANOVA

Known population covariance. For a two group experimental design with several outcomes in

which the variances and covariances are known and assumed to be identical for both groups, yy,

the null hypothesis is

H0: E - C = 0

with alternative hypothesis

H1: E - C 0 .

The test statistic

_ _ _ _X2 = [ nE nC / ( nE + nC)] ( yE. – yC.)-1

yy ( yE. – yC.)

is chi-square distributed with df = p.

Unknown population covariance. For most experimental and observationalstudies the population covariance matrix is not known. This matrix, labeled Wfor “within” is simply the covariance matrix “unstandardized” by the samplesize

SSCPyy = Wyy = (n1 –1) Syy1 + (n2 - 1) Syy2 .

An assumption of the model is that the population covariance matrices for allgroups are homogeneous. The following statistic is computed for groups withsample size nE and nC : _ _ _ _

T2 = [ nE nC (nE + nC -2) / (nE + nC ) ] ( yE – yC)W-1yy ( yE. – yC) ,

Hotelling’s T2 , distributed as an F-statistic with df = p, n1 + n2 – p -1 :

F = [ (n-p)/p(n-1] T2 .

Wilk’s lambda:

= W / B + W

The matrix W is the pooled group SSCP matrices as above. This plays the same role as

within-group sum of squares does in ANOVA. The matrix B + W is the equivalent of a

total sum of squares. It is more easily represented as the covariance matrix of all data

ignoring group membership.

The between-groups sum of squares and cross products, is not usually computed directly

but as the difference between T and W. Thus, for a single outcome Wilk’s lambda can

be shown to be

W=1 = SSe / SST = SSe / [SStreat + SSe ]

= 1 – R2

Other Measures

Pillai-Bartlett trace, VMultiple discriminant analysis (MDA) is the part of MANOVA where canonical roots are calculated. Each significant root is a dimension on which the vector of group means is differentiated. The Pillai-Bartlett trace is the sum of explained variances on the discriminant variates, which are the variables which are computed based on the canonical coefficients for a given root. Olson (1976) found V to be the most robust of the four tests and is sometimes preferred for this reason.

Roy's greatest characteristic root (GCRis similar to the Pillai-Bartlett trace but is based only on the first (and hence most important) root.Specifically, let lambda be the largest eigenvalue, then GCR = lambda/(1 + lambda). Note that Roy's largest root is sometimes also equated with the largest eigenvalue, as in SPSS's GLM procedure (however, SPSS reports GCR for MANOVA). GCR is less robust than the other tests in the face of violations of the assumption of multivariate normality.

_ _d1 = yE1 – yC1

_ _d2 = yE2 – yC2

Confidenceinterval foroutcome y2

centered at 0

Confidenceinterval foroutcome y1

centered at 0

1 - confidenceellipsoid

Fig. 15.2: Spatial representation of two group MANOVA difference vector and confidence ellipsoid for two independent outcomes

0,0

t-test for y2

is notsignificant

t-test for y1

is notsignificant

Dependent variable contrasts

Just as one can specify contrasts concerning levels of independentvariables, it is possible to specify contrasts among multivariate outcomes.In many research situations this is both feasible and of interest, because theoutcomes form a theoretical hierarchy or are related to each other intheoretically interesting ways. In MANOVA a matrix of the contrasts isdeveloped in which the rows are the contrast coefficients and columnsrepresent the outcomes.

In a study of gender differences in five outcomes, suppose that the outcomes are separable into twotheoretical orientations, internal and external. Along with a global MANOVA analysis of whethermales differ from females on all five outcomes, we might be interested in whether they differ betweenthe internal and external orientation, and within each orientation. They M matrix might have fivenonorthogonal contrasts for those questions:

OUTCOME: Y1 Y2 Y3 Y4 Y5

3 3 -2 -2 -2

-1 1 0 0 0

M = 0 0 1 -1 0

0 0 1 0 -1

0 0 0 1 -1

Clearly, the set of difference that the last three contrasts represent are an arbitrary set for pairwisedifference. Instead, two orthogonal contrasts might be used if there were theoretical reasons forparticular comparisons. In SAS PROC GLM, the MANOVA command, the matrix is specified asshown except for placing commas after each contrast set and a semi-colon at the end.

_ _d1 = yE1 – yC1

_ _d2 = yE2 – yC2

Confidenceinterval foroutcome y2

centered at 0

Confidenceinterval foroutcome y1

centered at 0

1 - confidenceellipsoid

Fig. 15.3: Spatial representation of two group MANOVA difference vector and confidence ellipsoid for two correlated outcomes

0,0

t-test for y2

is notsignificant

t-test for y1

is significant

0,0

C1

C2

C3

11

13

12

y1

y2

y3

1

1

3

12

13

Fig. 15.4: SEM representation of MANOVA with 4 groups and 3 dependent variables

21

31

11

32

2

C1

C2

C3

11*

13*

12*

y1

y2

y3

1

1

3

12

13

Fig. 15.4: Canonical SEM representation of MANOVA with 4 groups and 3 dependent variables

21*

31*

11*

32

2

2 --- fixed value

From 1st model

A

B

AB

11

13

12

y1

y2

y3

1

1

3

12

13

21

31

11

32

2

Fig. 15.6: MANOVA 2 x 2 factorial design with three outcomes

Assumptions- Homoscedasticity

• Box's M: Box's M tests MANOVA's assumption of homoscedasticity using the F distribution. If p(M)<.05, then the covariances are significantly different. Thus we want M not to be significant, rejecting the null hypothesis that the covariances are not homogeneous. That is, the probability value of this F should be greater than .05 to demonstrate that the assumption of homoscedasticity is upheld. Note, however, that Box's M is extremely sensitive to violations of the assumption of normality, making the Box's M test less useful than might otherwise appear. For this reason, some researchers test at the p=.001 level, especially when sample sizes are unequal.

Assumptions-Normality

Multivariate normal distribution. For purposes of significance testing, variables follow multivariate normal distributions. In practice, it is common to assume multivariate normality if each variable considered separately follows a normal distribution. MANOVA is robust in the face of most violations of this assumption if sample size is not small (ex., <20).

DISCRIMINANT ANALYSIS

The inverse of the MANOVA problem

Discriminant functions. The problem of discriminant analysis is to produce a score D that is a linear

combination of the predictors

D = a1y1 + a2y2 + …apyp

that maximally separates the groups. In effect, this is an ANOVA problem with D as the dependent

variable, and the criterion used to select the regression weights a1 …ap is

C = [ SSgroups / SStotal ] for all possible D.

y1

y2

D = a1y1 + a2y2

Fig. 15.8: First discriminant function in predictor space

*

Group 1 mean

Group 2 mean

* Group 3 mean

*

y1

y2

D = a1y1 + a2y2

Fig. 15.8: Second discriminant function in predictor space

*

Group 1 mean

Group 2 mean

* Group 3 mean

*

Bartlett’s V statistic:

V = - [ N –1 – (I + p)/2 ] ln , a chi-square statistic with (I-1)p degrees of freedom,

and V1 = - [ N –1 – (I + p)/2 ] ln (1 + 1 ).

If V is not significant it is assumed that V1 is not significant. If V is significant, then Vr is tested:

Vr = V – V1 , a chi-square statistic with (I-2)(p-1) degrees of freedom.

If Vr is not significant it is concluded that only the first function is signficant. If Vr is signficant, thesecond function statistic V2 is computed and subtracted from Vr and the remainder is tested, an iteration ofthe first procedure. This continues for all functions

D1

D2

* Group 1

Mean of Group 1 on D1

Mean of Group 1 on D2

Group 2 *

Mean of Group 2 on D2

Mean of Group 2 on D2

Group 3 * Mean of Group 3 on D2

Mean of Group 3 on D1

Fig. 15.10: Centroids for 3 groups in two discriminant function space

Canonical Discriminant Function 2

-6.0 -4.0 -2.0 .0 2.0 4.0 6.0 +---------+---------+---------+---------+---------+---------+ 6.0 + 32 + I 32 I I 32 I I 32 I I 32 I I 32 I 4.0 + + + 32 + + + + I 32 I I 32 I I 32 I I 32 I I 32 I 2.0 + + + 32 + + + + I 32 I I 32 I I 3112 I I 31 12 I

I *31 122* I .0 + + + 31 *14*22 + Canonical Discriminant Function 1 I 31 14 44222 I I 31 14 44422 I I 31 14 4422 I I 31 14 44222 I I 31 14 44422 I -2.0 + + 31 14+ + 44222 + + I 31 14 44422 I I 31 14 4422 I I 31 14 44222 I I 31 14 44422 I I 31 14 44222I -4.0 + + 31 + 14+ + + 444+ I 31 14 I I 31 14 I I 31 14 I I 31 14 I I 31 14 I -6.0 + 31 14 + +---------+---------+---------+---------+---------+---------+ -6.0 -4.0 -2.0 .0 2.0 4.0 6.0 Canonical Discriminant Function 1

_

Symbols used in territorial map

Symbol Group Label------ ----- --------------------

1 2 2 3 3 4 4 5

* Indicates a group centroid

Fig. 15.11: Territorial map for four groups in two-discriminant function space

Sex

y1

y2

a1 (r1 = w1/211a1) e1

e2

Note: Structure coefficients in parenthesis

Fig. 15.13: Discriminant function path diagram showing regression and structure coefficients

a2 (r2 = w1/222a2)

D

y2 y2

1 12 111 1 1 2 222 22 2 2 2 1 11 1 22222222222 22 11111 1 111 1 1 2 1 22212222222 2 1 2 11 1 11 2 2222 22122 11211 11 1 111 111 111 2 2 111 2221 2 21 211 1 21 22 22122 12211111 1 2 1 12 111 11 2 2 121 2222222 22 2 1 1 12 2222 22 1221211 1 1 1 1 111 1 2 1111 2212112222 2222 2 2 1 1 211 1 1 222 2 2 12211 1 1 1 1 111 112111211 1222122 2 2 2222 1 11121 12122 2 2 221 1 2 1 111 11 111111111 22 11 22222222 2 22 y1 1 1 211 22122 11 1112111 111 1 1 11122 111 2 1222 2222222 2 1 1211 222 12 11 1 1211 1 1 111211 1 11112212111222 22 2 2 2 2 1222 1 1 1 2 11222 111211 1 1 1 1 11 11211 121111212212222 2 2 2 1 1 1 11111 1 1211221211111 1 11 111121 2 212222 2 222 1 1 111 11111 11 1112 11 111 111 111111 21111 122222 22 2 11 111 1 11 1 1 1 111 11 1 1111 21122 2 22 2 1 11111 11211 2111 111 112111 1 11121222 12222 2 2 1 1 1 1111111 1111 1 1 111 1 2211 11 2111 222 2 2

Fig. 15.14: Scatterplots for two predictors of two groups with approximate curves separating the groups for classification

3 groups

2

1

Y1

Y2

Y3

1

2

e1

e2

e3

DISCRIMINANT ANALYSIS AS A STRUCTURAL EQUATION DIAGRAM