Least Squares ANOVA/ANCOVcore.ecu.edu/psyc/wuenschk/MV/LSANOVA/LeastSq.pdfSquares Mean Square F Value Pr > F Model 3 45.66667 15.22222 4.46 0.0404 R-Square 0.6256 The result of this

LeastSq.docx

Least Squares Analyses of Variance and Covariance©

One-Way ANOVA

Read Sections 1 and 2 in Chapter 16 of Howell. Run the program “ANOVA1-LS.sas,” which can be found on my SAS programs page. The data here are from Table 16.1 of Howell.

Dummy Variable Coding

DATA Dummy; INPUT Y X1-X3 @@; TITLE1 'Dummy Variable Coded 1-Way ANOVA'; CARDS; 8 1 0 0 9 1 0 0 7 1 0 0 5 0 1 0 7 0 1 0 3 0 1 0 3 0 0 1 4 0 0 1 1 0 0 1 6 0 0 0 4 0 0 0 9 0 0 0 PROC REG simple corr; MODEL Y = X1-X3; run; The dummy variable coding matrix is:

Group X1 X2 X3

1 1 0 0

2 0 1 0

3 0 0 1

4 0 0 0

Look at the values of X1-X3 in the data in the Data Dummy section of the program file. X1 codes whether or not an observation is from Group 1 (0 = no, 1 = yes), X2 whether or not it is from Group 2, and X3 whether or not it is from Group 3. Only k-1 (4-1) dummy variables are needed, since an observation that is not in any of the first k-1 groups must be in the kth group.

For each dummy variable the partial coefficients (b) represent a contrast between its group and the reference group (the one coded with all 0’s), that is, X1’s partials code Group 1 vs. Group 4, X2 codes Group 2 vs. Group 4, and X3 codes Group 3 vs. Group 4.

© Copyright 2019 Karl L. Wuensch - All rights reserved.

http://core.ecu.edu/psyc/wuenschk/SAS/SAS-Programs.htm

Page 2

Analysis of Variance

Source DF Sum of Squares

Mean Square

F Value Pr > F

Model 3 45.66667 15.22222 4.46 0.0404

Error 8 27.33333 3.41667

Corrected Total 11 73.00000

R-Square 0.6256

Variable DF Parameter Estimate t Value Pr > |t|

Intercept 1 6.33333 5.93 0.0003

X1 1 1.66667 1.10 0.3016

X2 1 -1.33333 -0.88 0.4028

X3 1 -3.66667 -2.43 0.0412

• The intercept is the mean of the reference group.

• For each X the b is the difference between its group’s mean and the mean of the reference group. For example, the b for X1 is the mean for Group 1 minus the mean for Group 4, (8 - 6.33) = 1.67.

Look at the correlations among the X’s and note that with equal n’s the off-diagonal correlations are constant. Now look at the output from the regression analysis. Note that the omnibus F of 4.455 is the same that would be obtained from a traditional ANOVA. Also note the following about the partial statistics:

Do note that only Group 3 differs significantly from the reference group.

Effects Coding

Look at the Data Effects section of the program and its output. This is the type of coding that Howell uses. The design matrix is exactly like that in dummy variable coding except that the “reference group” is coded with “-1” on each X. The design matrix is:

Page 3

Group X1 X2 X3

1 1 0 0

2 0 1 0

3 0 0 1

4 -1 -1 -1

DATA Effects; INPUT Y X1-X3 @@; Title1 'Effects Coded 1-Way ANOVA'; CARDS; 8 1 0 0 9 1 0 0 7 1 0 0 5 0 1 0 7 0 1 0 3 0 1 0 3 0 0 1 4 0 0 1 1 0 0 1 6 -1 -1 -1 4 -1 -1 -1 9 -1 -1 -1 PROC REG simple corr; MODEL Y = X1-X3; run;

The values of F, p, and R2 will all be the same with one type of coding as with any other type of coding.


Mean Square F Value Pr > F

Model 3 45.66667 15.22222 4.46 0.0404

R-Square 0.6256

The result of this coding scheme is that each X’s partial coefficients now represents one group versus the grand mean, that is, X1 represents Group 1 versus the grand mean, X2 represents Group 2 versus the grand mean, etc. As before, equal n’s cause constant off-diagonal r’s among the X’s. Note that the omnibus F from the regression analysis is still 4.455.

• The intercept is now equal to the grand mean.

• Each X’s b now equals the difference between its group’s mean and the grand mean, for example, for X1 b = (8 - 5.5) = 2.5.

Page 4


Intercept 1 5.50000 10.31 <.0001

X1 1 2.50000 2.71 0.0269

X2 1 -0.50000 -0.54 0.6032

X3 1 -2.83333 -3.07 0.0154

Contrast Coding

Look at Data Contrast section of the program and its output. The design matrix here codes a complete orthogonal set of comparisons:

Group X1 X2 X3

1 1 1 0

2 1 -1 0

3 -1 0 1

4 -1 0 -1

DATA Contrast; INPUT Y X1-X3 @@; TITLE1 'Contrast Coded 1-Way ANOVA'; CARDS; 8 1 1 0 9 1 1 0 7 1 1 0 5 1 -1 0 7 1 -1 0 3 1 -1 0 3 -1 0 1 4 -1 0 1 1 -1 0 1 6 -1 0 -1 4 -1 0 -1 9 -1 0 -1


Intercept 1 5.50000 10.31 <.0001

X1 1 1.00000 1.87 0.0978

X2 1 1.50000 -1.99 0.0821

X3 1 -1.83333 -2.43 0.0412

• The intercept equals the grand mean.

• Each of the b’s is ½ of the difference between the means of the groups contrasted. For example, the b for X1 is one half of the difference between the

Page 5

mean of Groups 1 and 2 (6.5) and the mean of Groups 3 and 4 (4.5), (6.5-4.5)/2=1.

X1 contrasts Groups 1 & 2 with Groups 3 & 4, X2 contrasts Group 1 with Group 2, and X3 contrasts Group 3 with Group 4. For each contrast varaiable (X1, X2, and X3), the sum of the coefficients must be 0. Furthermore, the products of the coefficients for and one contrast variable and any other contrast variable must sum to 0 – for example, for X1 and X2, (1)(1)+1(-1)+(-1)(0) +(-1)(0) = 1-1+0+0=0.

Again, the off-diagonal r’s among the X’s are constant, but this time, since the design matrix codes orthogonal contrasts, the r’s all are 0’s. The omnibus F is still 4.455.

Standard Contrast Coding

There are some advantages of using a “standard set of weights.” The coefficients for the one set of means must equal +1 divided by the number of conditions in that set while those for the other set must equal -1 divided by the number of conditions in that other set. The sum of the absolute values of the coefficients must be 2. For our design, here are standard weights:

Group X1 X2 X3

1 ½ ½ 0

2 ½ -½ 0

3 -½ 0 ½

4 -½ 0 -½

DATA StandardContrast; INPUT Y X1-X3 @@; TITLE1 'Standard Contrast Coded 1-Way ANOVA'; CARDS; 8 .5 .5 0 9 .5 .5 0 7 .5 .5 0 5 .5 -.5 0 7 .5 -.5 0 3 .5 -.5 0 3 -.5 0 .5 4 -.5 0 .5 1 -.5 0 .5 6 -.5 0 -.5 4 -.5 0 -.5 9 -.5 0 -.5 PROC REG simple corr; MODEL Y = X1-X3; run;

Now the slopes equal the differences between the means of the groups contrasted.

Page 6


Intercept 1 5.50000 0.53359 <.0001

X1 1 2.00000 1.06719 0.0978

X2 1 3.00000 1.50923 0.0821

X3 1 -3.66667 1.50923 0.0412

Let GLM Do It. The code in the Data GLM section of the program does the ANOVA with Proc GLM. The contrast statements reproduce the contrasts earlier produced with contrast coding. Each F from PROC GLM CONTRAST statements is the square of t from PROC REG.

DATA GLM; INPUT Y A @@; TITLE1 'Let GLM Do It'; CARDS; 8 1 9 1 7 1 5 2 7 2 3 2 3 3 4 3 1 3 6 4 4 4 9 4 PROC GLM; CLASS A; MODEL Y = A / SS1; CONTRAST '12 VS 34' A 1 1 -1 -1; CONTRAST '1 VS 2' A 1 -1 0 0; CONTRAST '3 VS 4' A 0 0 1 -1; CONTRAST '12s VS 34s' A .5 .5 -.5 -.5; CONTRAST '1s VS 2s' A .5 -.5 0 0; CONTRAST '3s VS 4s' A 0 0 .5 -.5; run; run; quit;

Contrast DF Contrast SS Mean Square F Value Pr > F

12 VS 34 1 12.00000000 12.00000000 3.51 0.0978

1 VS 2 1 13.50000000 13.50000000 3.95 0.0821

3 VS 4 1 20.16666667 20.16666667 5.90 0.0412

12s VS 34s 1 12.00000000 12.00000000 3.51 0.0978

1s VS 2s 1 13.50000000 13.50000000 3.95 0.0821

3s VS 4s 1 20.16666667 20.16666667 5.90 0.0412

Two-Way ANOVA

Read Sections 16.3 and 16.4 in Howell. I also recommend that you reread my handout Four Types of Sums of Squares for ANOVA Effects. Run the programs ANOVA2-LS-Eq.sas and ANOV2-LS-UnEq.sas.

http://core.ecu.edu/psyc/wuenschk/SAS/SS1234.pdf

Page 7

Orthogonal ANOVA, 2 x 4 = 8 groups

ANOVA2-LS-Eq.sas uses the data from Table 16.2 of Howell and the effects coding matrix presented in Section 16.3.

• 2 x 4 ANOVA = 8 cells

• One dummy variable to code the main effect of A

• Three dummy variables to code the main effect of B

• 1 x 3 = 3 dummy variables to code the interaction as products of the dummy variable for A by those for B

• as many X’s as degrees of freedom -- 1 for factor A, 3 for factor B, and 3 for the interaction between A and B

• If we had three levels of A and four of B we would have 11 X’s: A1, A2, B1, B2, B3, A1B1, A1B2, A1B3, A2B1, A2B2, A2B3.

Cell A1 B1 B2 B3 A1B1 A1B2 A1B3

a1b1 1 1 0 0 1 0 0

a1b2 1 0 1 0 0 1 0

a1b3 1 0 0 1 0 0 1

a1b4 1 -1 -1 -1 -1 -1 -1

a2b1 -1 1 0 0 -1 0 0

a2b2 -1 0 1 0 0 -1 0

a2b3 -1 0 0 1 0 0 -1

a2b4 -1 -1 -1 -1 1 1 1

DATA SOL; DROP I; INPUT A1 B1 B2 B3 A1B1 A1B2 A1B3; DO I=1 TO 4; INPUT Y @@; OUTPUT; END; CARDS; 1 1 0 0 1 0 0 (a1b1) 5 7 9 8 1 0 1 0 0 1 0 (a1b2) 2 5 7 3

Page 8

1 0 0 1 0 0 1 (a1b3) 8 11 12 14 1 -1 -1 -1 -1 -1 -1 11 15 16 10 -1 1 0 0 -1 0 0 7 9 10 9 -1 0 1 0 0 -1 0 3 8 9 11 -1 0 0 1 0 0 -1 9 12 14 8 -1 -1 -1 -1 1 1 1 11 14 10 12 PROC REG; full: MODEL Y = A1 B1 B2 B3 A1B1 A1B2 A1B3; a_x_b: MODEL Y = A1 B1 B2 B3; b: MODEL Y = A1 A1B1 A1B2 A1B3;

a: MODEL Y = B1 B2 B3 A1B1 A1B2 A1B3;



Mean Square

F Value Pr > F

Model 7 231.96875 33.13839 5.71 0.0006

Error 24 139.25000 5.80208


R-Square 0.6249

Variable DF Parameter Estimate

t Value Pr > |t|

Intercept 1 9.34375 21.94 <.0001

A1 1 -0.40625 -0.95 0.3496

B1 1 -1.34375 -1.82 0.0809

B2 1 -3.34375 -4.53 0.0001

Page 9

Variable DF Parameter Estimate

t Value Pr > |t|

B3 1 1.65625 2.25 0.0342

A1B1 1 -0.34375 -0.47 0.6454

A1B2 1 -1.34375 -1.82 0.0809

A1B3 1 0.65625 0.89 0.3824

Look at the output from the full model. The error SS there is the SSE for the factorial ANOVA, 139.25 on 24 df. The model SS (cells SS) there, 231.96875, is the sum of the A, B, and AxB sums-of-squares from a factorial ANOVA. The b’s are as indicated in part (c) of Table 16.2 in Howell, and the intercept equals the grand mean.

The REG Procedure Model: a_x_b



Mean Square

F Value Pr > F

Model 4 204.62500 51.15625 8.29 0.0002

Error 27 166.59375 6.17014


Root MSE 2.48398 R-Square 0.5512

Now look at the output from Model: A_X_B. We have deleted from this model the three terms that code the interaction. The interaction sum-of-squares is simply the difference between the full model and this reduced model, 231.96875 - 204.625 =

27.34375. The 2 for the interaction is simply the difference between the full model R2 and the reduced model R2, (.6249 - .5512) = .0737, which also equals the interaction SS divided by the total SS. We can test the significance of the interaction term by testing the significance of the reduction in the regression SS that accompanied the deletion of the dummy variables that coded the interaction. Using a partial F, we obtain the same value of F we would get using the traditional means (interaction mean square divided by error mean square):

Page 10

571.1)80208.5)(3(

2046250096875.231

))((=

−=

−

−=

full

reducedfull

MSErf

SSSSF

The REG Procedure Model: b



Mean Square

F Value Pr > F

Model 4 32.62500 8.15625 0.65 0.6315

Error 27 338.59375 12.54051



The Model: B output is for a reduced model with the three terms coding the main

effect of B deleted. You find the SS and 2 for B by subtracting the appropriate reduced model statistics from the full model statistics.

The REG Procedure Model: a



Mean Square

F Value Pr > F

Model 6 226.68750 37.78125 6.54 0.0003

Error 25 144.53125 5.78125



The Model: A output is for a reduced model with the one term coding the main effect of A deleted.

Page 11

The results you get from testing each of the three reduced models are the same you would get using PROC ANOVA or PROC GLM, which do the dummy coding for you.

Nonorthogonal Analysis

ANOV2-LS-UnEq.sas uses the unequal n’s data from Table 16.5 of Howell. The coding scheme is the same as in the previous analysis. Obtain sums-of-squares for A, B, and AxB in the same way as you did in the previous analysis and you will have done an Overall and Spiegel Method I analysis. Do note that the sums-of-squares do not sum to the total SS, since we have excluded variance that is ambiguous. Each effect is partialled for every other effect. The results from such an analysis are identical to those provided by the TYPE III SS computed by PROC.

Source DF Sum of Squares Mean Square F Value Pr > F

Model 7 207.7055556 29.6722222 4.55 0.0017

Error 28 182.6000000 6.5214286


Source DF Type III SS Mean Square F Value Pr > F

A 1 3.7555556 3.7555556 0.58 0.4543

B 3 177.9556246 59.3185415 9.10 0.0002

A*B 3 19.2755003 6.4251668 0.99 0.4139

3.7555556 + 177.9556246+ 177.9556246 = 359.667 390.3055556

Analysis of Covariance

Read Sections 16.5 through 16.11 in Howell and Chapter 6 in Tabachnick and Fidell. As explained there, the ANCOV is simply a least-squares ANOVA where the covariate or covariates are entered into the model prior to or simultaneously with the categorical variables. The effect of each categorical variable is adjusted for the covariate(s).

Do note the additional assumptions involved in ANCOV (that each covariate has a linear relationship with the outcome variable and that the slope for that relationship does not change across levels of the categorical predictor variable(s). Carefully read Howell’s cautions about interpreting analyses of covariance when subjects have not been randomly assigned to treatment groups. Run the programs ANCOV1.sas and ANCOV2.sas.

Page 12

One-Way ANCOV

I am not going to burden you with doing ANCOV with PROC REG—I think you already have the basic idea of least-squares analyses mastered. Look at ANCOV1.sas and its output. These data were obtained from Figure 2 in the article, "Relationships among models of salary bias," by M. H. Birnbaum (1985, American Psychologist, pp. 862-866) and are said to be representative of data obtained in various studies of sex bias in faculty salaries. I did double the sample size from that displayed in the plot from which I harvested the data. We can imagine that we have data from three different departments’ faculty members: The professor’s Gender (1 = male, 2 = female), an objective measure of the professor’s QUALIFICations (a composite of things like number of publications, ratings of instruction, etc.), and SALARY (in thousands of 1985 dollars).

Proc Format; Value sx 1='M' 2='F'; run; DATA LOTUS; INPUT GENDER QUALIFIC SALARY @@; format gender sx.; cards; 2 20 20 2 30 20 2 25 25 2 35 25 1 28 27 1 38 27 2 20 30 2 30 30 2 30 30 2 40 30 1 32 32 1 42 32 2 25 35 2 35 35 1 28 37 1 38 47 1 38 47 1 48 47 2 30 40 2 40 40 1 32 42 1 42 42 1 38 47 1 48 47 2 21 21 2 29 19 2 26 26 2 34 24 1 29 28 1 37 26 2 21 31 2 29 29 2 31 31 2 39 29 1 33 33 1 41 31 2 26 36 2 34 34 1 29 38 1 37 46 1 39 48 1 47 46 2 31 41 2 39 39 1 33 43 1 41 41 1 39 48 1 47 46 PROC PLOT;PLOT SALARY*QUALIFIC=GENDER;

The data are plotted, using the symbol for gender as the plotting symbol. The plot suggests three lines, one for each department (salaries being highest in the business department and lowest in the sociology department), but that is not our primary interest. Do note that salaries go up as qualifications go up. Also note that the M’s tend to be plotted higher and more to the right than the F’s.

Page 13

PROC ANOVA; CLASS GENDER; Model QUALIFIC SALARY = Gender; Means Gender; TITLE3 'Gender differences in qualifications and in salary ($thousands)'; run; quit;

Dependent Variable: QUALIFIC


Model 1 705.333333 705.333333 18.13 0.0001

Error 46 1789.333333 38.898551


Dependent Variable: SALARY


Model 1 1083.000000 1083.000000 20.30 <.0001

Error 46 2454.000000 53.347826


Page 14

Level of GENDER

N QUALIFIC SALARY

Mean Std Dev Mean Std Dev

F 24 30.0000000 6.22756844 30.0000000 6.63324958

M 24 37.6666667 6.24615824 39.5000000 7.91805861

The sexes differ significantly on both qualification and salaries.

PROC ANOVA does two simple ANOVAs, one on the qualifications data (later to be used as a covariate) and one on the salary data. Both are significant. This is going to make the interpretation of the ANCOV difficult, since we will be adjusting group means on the salary variable to remove the effect of the qualifications variable (the covariate), but the groups differ on both. The interpretation would be more straightforward if the groups did not differ on the covariate, in which case adjusting for the covariate would simply reduce the error term, providing for a more powerful analysis. The error SS (1789.3) from the analysis on the covariate is that which Howell calls SSe(c) when discussing making comparisons between pairs of adjusted means.

PROC GLM; CLASS Gender; MODEL Salary = Qualific|Gender / SS3; title3 'Test Qualific*Gender for Homogeneity of Regression'; run; quit;


QUALIFIC 1 466.2404624 466.2404624 10.45 0.0023

GENDER 1 1.8971440 1.8971440 0.04 0.8376

QUALIFIC*GENDER 1 24.0109542 24.0109542 0.54 0.4671

Page 15

The first invocation of PROC GLM is used to test the homogeneity of regression assumption. PROC ANOVA does not allow any continuous effects (such as a continuous covariate). The model statement includes (when the bar notation is expanded) the interaction term, Qualific*Gender. Some computing time is saved by asking for only sequential (SS1) sums of squares. Were Qualific*Gender significant, we would have a significant violation of the homogeneity of regression assumption (the slopes of the lines for predicting salary from qualifications would differ significantly between genders), which would, I opine, be a very interesting finding in its own right.

What should you do if your primary interest was to test the effect of groups after holding constant the effect of the covariate, but the damn interaction is significant? The answer is “the same thing you would do if you were interested in the main effect of Factor A but factorial ANOVA showed that there was an interaction between A and B – look at the simple main effects of A at levels of B.” That is, do a moderation analysis. See ANCOV_HeroRegr.

PROC GLM; CLASS Gender; MODEL Salary = Gender Qualific(Gender) / SS3 SOLUTION; title3 'Get Slopes for Salary Predicted From Qualifications for Each Group'; run; quit;

Parameter Estimate Standard Error t Value Pr > |t|

Intercept 15.90936107 B 8.50893111 1.87 0.0682

GENDER F 2.25207391 B 10.92141376 0.21 0.8376

GENDER M 0.00000000 B . . .

QUALIFIC(GENDER) F

0.39461883 0.22364744 1.76 0.0846

QUALIFIC(GENDER) M

0.62630015 0.22298182 2.81 0.0074

Note: The X'X matrix has been found to be singular, and a generalized inverse was used to solve the normal equations. Terms whose estimates are followed by the letter 'B' are not uniquely estimable.

The second invocation of PROC GLM is used to obtain the slopes for predicting salary from qualifications within each level of Gender—QUALIFIC(GENDER). We already know that these two slopes do not differ significantly, but I do find it interesting that the slope for the male faculty is higher than that for the female faculty.

PROC GLM; CLASS GENDER; MODEL SALARY = QUALIFIC GENDER / SS3 EFFECTSIZE alpha =.1; LSMEANS GENDER; TITLE3 'ANCOV: Gender differences in salary holding qualifications constant.'; run; quit;

http://core.ecu.edu/psyc/wuenschk/MV/LSANOVA/ANCOV_HeroRegr.pdf

Page 16


Model 2 1549.875559 774.937779 17.55 <.0001

Error 45 1987.124441 44.158321


R-Square

0.438189

Source DF Type III SS Mean Square

F Value

Pr > F Total Variation Accounted For

Semipartial Eta-Square

Semipartial Omega-Square

Conservative 90%

Confidence Limits

QUALIFIC 1 466.8755589 466.8755589 10.57 0.0022 0.1320 0.1180 0.0174 0.2815

GENDER 1 268.3640677 268.3640677 6.08 0.0176 0.0759 0.0626 0.0000 0.2136

The third invocation of PROC GLM is used to do the Analysis of Covariance. The LSMEANS are estimates of what the group means would be if the groups did not differ on qualifications. If you have more than two groups, you will probably want to use the PDIFF option, for example, LSMEANS GROUP / PDIFF. The matrix of p-values produced with the PDIFF option are for pairwise comparisons between adjusted means (with no adjustment of per-comparison alpha). You can adjust the alpha-criterion downwards (Bonferroni, Sidak) if you are worried about familywise error rates.

We can estimate the magnitude of effect of gender with an eta-squared statistic, the ratio of the gender sum of squares to the total sum of squares, 268.364 / 3537 = .076. This is equivalent to the increase in R2 when we add gender to a model for predicting salary from the covariate(s). The Proc Corr shows that r for predicting salary from qualifications is .60193. Proc GLM shows that the R2 for predicting salary from qualifications and gender is .438189. Accordingly, eta-squared = .438189 - .601932 = .076. If men and women were equally qualified, 7.6% of the differences in salaries would be explained by gender. Look back at the ANOVA comparing the genders on salary. The eta-squared there was .306. If we ignore qualifications, 30.6% of the differences in salaries is explained by gender (which is confounded with qualifications and other unknown variables).

We could also use d̂ to estimate the magnitude of the difference between groups. The raw difference between adjusted means is 37.54 – 31.96 = 5.58 thousand dollars. The standardizer will be the square root of the MSE from the ANOVA or from the ANCOV. Howell (following the advice of Cortina & Nouri) recommends the former.

Page 17

Accordingly, 76.35.53

58.5ˆ ==d . If we were to ignore qualifications (by using the

unadjusted means), 30.135.53

305.39ˆ =−

=d .

Notice that after removing the interaction term the within-gender regression line have identical slopes.

GENDER SALARY LSMEAN

F 31.9580849

M 37.5419151

Our results indicate that even when we statistically adjust for differences in qualifications, men receive a salary significantly higher than that of women. This would seem to be pretty good evidence of bias against women, but will the results look the same if we view them from a different perspective? Look at the last invocation of PROC GLM. Here we compared the genders on qualifications after removing the effect of salary.

PROC GLM; CLASS GENDER; MODEL QUALIFIC = SALARY GENDER / SS3 EFFECTSIZE alpha =.1; LSMEANS GENDER; TITLE3 'ANCOV: Gender differences in qualifications holding salary constant.'; run; quit;

Page 18


Model 2 1045.755501 522.877751 16.24 <.0001

Error 45 1448.911165 32.198026


R-Square

0.419196

GENDER QUALIFIC LSMEAN

F 31.7691524

M 35.8975143

The results indicate that when we equate the groups on salary the mean qualifications of the men is significantly greater than that of the women. That looks like bias too, but in the opposite direction. ANCOV is a slippery thing, especially when dealing with data from a confounded design – where the covariate is correlated not only with the “dependent” variable but with the “independent” variable as well.

Two-Way ANCOV

PROC ANOVA; CLASS task smoke; MODEL errors distract = task|smoke; MEANS task|smoke; TITLE3 'Two-way ANOVAs using errors and distract as DVs'; Title4 'Were the design unbalanced (unequal n''s) you would use'; title5 'PROC GLM here instead of PROC ANOVA.'; run; quit;

Look at ANCOV2.sas and its output. The data are from Table 16.11 in Howell. The program is a straightforward extension of ANCOV1.sas to a two-way design. First PROC ANOVA is used to evaluate effects of type of task (pattern recognition, cognitive, or driving simulation) and smoking condition (active smoking, delayed smoking, nonsmokers) on the covariate (distractability) and on the dependent variable (errors made on the task). Were the design unbalanced (unequal n’s) you would need to use PROC GLM with Type III sums-of-squares here. The model SS, 6051.8, from the ANOVA on the covariate is the SScells(c) from Howell’s discussion of comparing adjusted means. The error SS from the same analysis, 54,285,8, is Howell’s SSe(c) and the SSSmoke, 730, is Howell’s SSg(c).

Page 19

Dependent Variable: errors


Model 8 31744.72593 3968.09074 36.80 <.0001

Error 126 13587.20000 107.83492


R-Square

0.700273

Source DF Anova SS Mean Square F Value Pr > F

task 2 28661.52593 14330.76296 132.90 <.0001

smoke 2 354.54815 177.27407 1.64 0.1973

task*smoke 4 2728.65185 682.16296 6.33 0.0001

Dependent Variable: distract


Model 8 6051.83704 756.47963 1.76 0.0919

Error 126 54285.86667 430.84021


R-Square

0.100299

Source DF Anova SS Mean Square F Value Pr > F

task 2 2124.192593 1062.096296 2.47 0.0891

smoke 2 730.014815 365.007407 0.85 0.4310

task*smoke 4 3197.629630 799.407407 1.86 0.1224

PROC GLM; CLASS task smoke; MODEL errors = task|smoke|distract / SS3; title3 'Test Homogeneity of Regression Within Cells and Treatments'; run; quit;

Page 20


task 2 450.583854 225.291927 4.66 0.0113

smoke 2 252.027090 126.013545 2.60 0.0782

task*smoke 4 172.095239 43.023810 0.89 0.4727

distract 1 2790.512721 2790.512721 57.68 <.0001

distract*task 2 2535.238279 1267.619140 26.20 <.0001

distract*smoke 2 335.100007 167.550004 3.46 0.0346

distract*task*smoke 4 142.923907 35.730977 0.74 0.5675

PROC GLM is first used to test homogeneity of regression within cells and

treatments. The DISTRACTTASK F tests the null hypothesis that the slope for predicting ERRORS from DISTRACT is the same for all three types of task. The

DISTRACTSMOKE tests slopes across smoking groups. The

DISTRACTTASKSMOKE tests the null that the slope is the same in every cell of the

two-way design. Howell did not extract the DISTRACTTASK and

DISTRACTSMOKE terms from the error term and he did not test them, although in the third edition of his text (p. 562) he admitted that a good case could be made for testing those effects (he wanted their 3 df in the error term). Our analysis indicates that we have no problem with heterogeneity of regression across cells, but notice that there is heterogeneity of regression across tasks and across smoking groups.

PROC GLM; CLASS task smoke; MODEL errors = task|smoke distract(task smoke) / SS1 SOLUTION; title2 'Obtain Within-Cell Slopes';

Parameter Estimate t Value Pr > |t|

distract(task*smoke) Cognitive Active Smoker 0.42879834 6.06 <.0001

distract(task*smoke) Cognitive Delayed Smoker 0.70242022 9.57 <.0001

distract(task*smoke) Cognitive NonSmoker 0.46499665 5.25 <.0001

distract(task*smoke) Driving Active Smoker 0.01477336 0.16 0.8723

distract(task*smoke) Driving Delayed Smoker 0.25431768 2.26 0.0254

distract(task*smoke) Driving NonSmoker 0.04721085 0.46 0.6429

distract(task*smoke) Pattern Active Smoker 0.10611299 1.16 0.2483

distract(task*smoke) Pattern Delayed Smoker 0.10582833 1.15 0.2510

distract(task*smoke) Pattern NonSmoker 0.01719385 0.15 0.8811

Page 21

PROC GLM is next used to obtain the slopes for each cell. Ignore the Biased estimates for within treatment slopes. Although these slopes do not differ enough across cells to produce significant heterogeneity of regression, inspection of the slopes shows why the DISTRACT*TASK effect was significant. Look at how high the slopes are for the cognitive task as compared to the other two tasks. Clearly the number of errors increased more rapidly with participants’ level of distractibility with the cognitive task than with the other tasks, especially for those nicotine junkies who had been deprived of their drug. You can also see the (smaller) DISTRACT*SMOKE effect, with the slopes for the “delayed smokers” (smokers who had not had a smoke in three hours) being larger than for the other participants.

PROC GLM; CLASS task smoke; MODEL errors = distract task|smoke / SOLUTION; LSMEANS task|smoke / PDIFF; title2 'The ANCOV With Means and Adjusted Means'; run; quit;

Source DF Type I SS Mean Square F Value Pr > F

distract 1 10450.42887 10450.42887 146.08 <.0001

Parameter Estimate t Value Pr > |t|

distract 0.29251199 8.06 <.0001

The next GLM does the ANCOV. Note that DISTRACT is significantly correlated with ERRORS (p < .001, Type I SS). Remember that the Type I SS reported here does not adjust the first term in the model (the covariate) for the later terms in the model. Howell prefers to adjust the covariate for the other effects in the model, so he uses SPSS unique (same as SAS Type III) SS to test the covariate. The common slope used to adjust scores is 0.2925.


distract 1 4644.87583 4644.87583 64.93 <.0001

task 2 23870.48521 11935.24261 166.84 <.0001

smoke 2 563.25742 281.62871 3.94 0.0220

task*smoke 4 1626.51071 406.62768 5.68 0.0003

I copied the adjusted means into SPSS and used UNIANOVA to produce an interaction plot.

Page 22 UNIANOVA Errors BY Task Smoking /METHOD=SSTYPE(3) /INTERCEPT=INCLUDE /PLOT=PROFILE(Smoking*Task) /CRITERIA=ALPHA(0.05) /DESIGN=Task Smoking Task*Smoking.

As you can see, the smoking condition had a greater effect on performance on the cognitive task than on the other tasks. The very large main effect of type of task is obvious in that plot too, with errors being much more likely with the cognitive task than with the other two tasks.

task errors LSMEAN LSMEAN Number

Cognitive 37.3303851 1

Driving 7.7487793 2

Pattern 9.6986133 3

Least Squares Means for effect task Pr > |t| for H0: LSMean(i)=LSMean(j)


i/j 1 2 3

1 <.0001 <.0001

2 <.0001 0.2783

3 <.0001 0.2783

Page 23

If we ignore the interaction and look at the comparisons between marginal means (using the PDIFF output, and not worrying about familywise error), we see that, for the type of task variable, there were significantly more errors with the cognitive task than with the other two types of tasks.

smoke errors LSMEAN LSMEAN Number

Active Smoker 19.7209913 1

Delayed Smoker 19.6964821 2

NonSmoker 15.3603044 3

Least Squares Means for effect smoke Pr > |t| for H0: LSMean(i)=LSMean(j)


i/j 1 2 3

1 0.9891 0.0159

2 0.9891 0.0171

3 0.0159 0.0171

On the smoking variable, we see that the nonsmokers made significantly fewer errors than did those in the two groups of smokers.

TASK, SMOKE, and TASKSMOKE all have significant effects after we adjust for the covariate (Type III SS). Since the interaction is significant, we need to do some simple main effects analyses.

PROC SORT; BY task; PROC GLM; CLASS smoke; MODEL errors = smoke distract / solution; LSMEANS smoke / PDIFF; BY task; title2 'Simple Main Effects ANCOV by Level of Task; run; quit;

task=Pattern


Model 3 110.8138431 36.9379477 1.93 0.1401

Error 41 785.4972680 19.1584700


Page 24

R-Square

0.123633


distract 1 110.0943469 110.0943469 5.75 0.0212

smoke 2 0.7194962 0.3597481 0.02 0.9814

Parameter Estimate

distract 0.0845153768

The simple main effects analysis done with the data from the pattern recognition task shows that the smoking groups did not differ significantly. The Type I SSsmoke gives us a test of the effect of smoking history ignoring the covariate. The slope used to adjust the scores on the pattern recognition test is 0.085, notably less than the 0.293 used in the factorial ANCOV.

task=Cognitive


Model 3 9789.51850 3263.17283 29.38 <.0001

Error 41 4554.25928 111.07949


R-Square

0.682492


distract 1 7858.981826 7858.981826 70.75 <.0001

smoke 2 1930.536674 965.268337 8.69 0.0007

Parameter Estimate

distract 0.53683291

Page 25







i/j 1 2 3

1 0.9597 0.0010

2 0.9597 0.0008

3 0.0010 0.0008

When we look at the analysis of the data from the cognitive task, we see that the smoking groups differ significantly. The nonsmokers made significantly fewer errors than did the participants in both the smoking groups. The slope used for adjusting scores (0.537) is notably more than it was with the factorial ANCOV or with the other two tasks. This is due to the DISTRACT*TASK interaction which Howell choose to ignore, but we detected.

task=Driving


Model 3 552.772000 184.257333 8.61 0.0001

Error 41 877.539111 21.403393


R-Square

0.386470


distract 1 296.6702306 296.6702306 13.86 0.0006

smoke 2 256.1017692 128.0508846 5.98 0.0053

Page 26

Parameter Estimate

distract 0.089789078







i/j 1 2 3

1 0.0219 0.0017

2 0.0219 0.2505

3 0.0017 0.2505

Finally, with the driving task, we see that the smoking groups differ significantly, with the active smokers making significantly fewer errors than did the delayed smokers and the nonsmokers. I guess the stimulant properties of nicotine are of some value when driving.

Controlling Familywise Error When Using PDIFF

If the comparisons being made involve only three means, I recommend Fisher’s procedure – that is, do not adjust the p values, but require that the main effect be statistically significant – if it is not, none of the pairwise differences are significant. If the comparisons involve more than three means, you can tell SAS to adjust the p values to control familywise error. For example, LSMEANS smoke / PDIFF ADJUST=TUKEY; would apply a Tukey adjustment. Other adjustments available

include BONferroni, SIDAK, DUNNETT, and SCHEFFE.

References and Recommended Readings

• Birnbaum, M. H. (1985). Relationships among models of salary bias. American Psychologist, 40, 862–866.

• Howell, D. C. (2013). Statistical methods for psychology (8th ed.). Belmont, CA: Cengage Wadsworth. ISBN-13: 978-1-111-83548-4.

http://psych.fullerton.edu/mbirnbaum/papers/Birnbaum_AP_1985.pdf

http://www.cengagebrain.com/shop/isbn/9781111835484

Page 27

• Huck, S. W., & McLean, R. A. (1975). Using a repeated measures ANOVA to analyze the data from a pretest-posttest design: A potentially confusing task. Psychological Bulletin, 82. 511-518.

• Maxwell, S. E., Delaney, H. D., & Dill. (1984). Another look at ANCOVA versus blocking. Psychological Bulletin, 95, 136-147.

• Maxwell, S. E., Delaney, H. D., & Manheimer, J. M. (1985). ANOVA of residuals and ANCOVA: Correcting an illusion by using model comparisons and graphs. Journal of Educational and Behavioral Statistics, 10, 197-209. doi: 10.3102/10769986010003197

• Rausch, J. R., Maxwell, S. E., & and Kelley, K. (2003). Analytic methods for questions pertaining to a randomized pretest, posttest, follow-up design. Journal of Clinical Child and Adolescent Psychology, 32, 467-486.

• Tabachnick, B. G., & Fidell, L. S. (2013). Using multivariate statistics (6th ed.). Boston: Pearson. ISBN-10: 0205849571. ISBN-13: 9780205849574.

Example of Presentation of Results from One-Way ANCOV

The Pretest-Posttest x Groups Design: How to Analyze the Data

Matching and ANCOV with Confounded Variables

Effect Size Confidence Intervals in ANCOV

Return to Wuensch’s Stats Lessons Page

Copyright 2019 Karl L. Wuensch - All rights reserved.

http://www.pearsonhighered.com/educator/product/Using-Multivariate-Statistics-6E/0205849571.page

http://core.ecu.edu/psyc/wuenschk/MV/LSANOVA/ANCOV1-Results.docx

http://core.ecu.edu/psyc/wuenschk/MV/LSANOVA/Pretest-Posttest-ANCOV.pdf

http://core.ecu.edu/psyc/wuenschk/SAS/confound.pdf

http://core.ecu.edu/psyc/wuenschk/MV/LSANOVA/ANCOV_CI_Eta-squared.docx

http://core.ecu.edu/psyc/wuenschk/StatsLessons.htm

http://core.ecu.edu/psyc/wuenschk/klw.htm

Documents

Least Squares ANOVA/ANCOVcore.ecu.edu/psyc/wuenschk/MV/LSANOVA/LeastSq.pdfSquares Mean Square F Value Pr > F Model 3 45.66667 15.22222 4.46 0.0404 R-Square 0.6256 The result of this