Factorials with no replication - homepage.stat.uiowa.eduhomepage.stat.uiowa.edu/~rdecook/stat5201/notes/3-10_factorial_n… · Algorithm for Tukey 1 d.f. test 1 Fit the preliminary

Factorials with no replication

STAT:5201

Week 8 - Lecture 2

1 / 36

Two-way ANOVA with no replication

Consider a two-way ANOVA with no replication, a = 3 and b = 4.

B Source df1 2 3 4 A 2

1 B 3A 2 AB 6 If the full model is fit,

3 error 0 ⇐ no d.f. left for errorc. Total 11 (no estimate for σ2),

can’t test for effects.

Common option: Assume there is no 2-way interactionSource df

A 2B 3

error 6 The d.f. for interaction are used for the error.c. Total 11

2 / 36

Larger factorials with no replication

Use highest order interaction as the error term?If we have many factors (e.g. A, B, C, D), we could assume that thehighest-order interaction (e.g. A*B*C*D) is non-existent, and use thisinteraction term as the error. Then we have an estimate for σ2 andwe can formally test the next highest level of interaction (e.g. 3-wayinteraction terms).

If this interaction is not present, then the interaction is a goodestimate for σ2. If the interaction is present, then using the term toestimate σ2 is not a good choice.

In this process, we don’t formally test for the interaction (because wecan’t), but we just assume it to be non-existent. So, we’re vulnerableto making a mistake, but a factorial without replication forces you tomake some extra assumptions and choices.

3 / 36

Larger factorials with no replication

In general, somehow we need to get degrees of freedom for the errorso we can perform hypothesis tests on the terms in the model.

NOTE: Don’t forget the principle of hierarchy if removing terms from the model.

Tukey developed a 1 d.f. of test for nonadditivity in these cases thatdetects a certain kind of interaction (using only 1 d.f.), and wediscuss that test here.

4 / 36

Two-way ANOVA with no replication

Another option: Use 1 d.f. to test for special form of nonadditivity.Source df

A 2B 3

error 6 ⇐ Use 1 d.f. from here to do a Tukey 1 d.f. test.c. Total 11

Source dfA 2B 3

nonadd 1 ⇐ Tukey said we can use one parametererror 5 to test for a certain form of nonadditivity.

c. Total 11

5 / 36

Tukey 1 d.f. test for nonadditivity (9.2.4 OLRT)

If we have a certain kind of interaction present, then the Tukey 1d.f. test may be able to detect it.

If the test suggests that the nonaddivity is present, then we proceedby utilizing a power transformation to remove (or at leastsubstantially diminish) the interaction, and then re-fit themain-effects only model on the transformed scale.

NOTE: Transforming the response can potentially remove interactionOR potentially create interaction. Tukey’s process is meant to removeit, but be aware that transforming Y can affect the nature of themean structure.

6 / 36

Tukey 1 d.f. test for nonadditivity in Two-way ANOVA

If there is no replication in a Two-way ANOVA, then we are not ableto test for the interaction in the classical model...

yij = µ+ αi + βj + (αβ)ij + εij and εijiid∼ N(0, σ2).

But Tukey suggested a certain model that has interaction thatrequires only 1 d.f. for testing...

yij = µ+ αi + βj + λ · αi · βj + εij and εijiid∼ N(0, σ2).

↑one extra parameter beyond main effects parameters

This is a ‘multiplicative’ form of interaction where the (αβ)ij are suchthat (αβ)ij ∝ αi · βj . If λ = 1, then (αβ)ij = αi · βj .

7 / 36


We will use the notation shown in Oehlert as

(αβ)ij =η

µ· αi · βj

and η is the 1 extra parameter needed beyond the main effects model.

If this type of interaction exists (i.e. η 6= 0), then we proceed byapplying a power transformation of “1− η” to the response which willapproximately remove the interaction, and then fit the main effectsmodel on the transformed data. This type of interaction is said to bea transformable nonadditivity.

8 / 36


Simulation Example

Suppose we have a true additive (no interaction) model with a = 3and b = 2 on some original scale...

yij = µ+ αi + βj + εij and εijiid∼ N(0, σ2)

and using sum-to-zero constraints we let µ = 7.5,α1 = −5, α2 = 0, α3 = 5,β1 = −1.5, β2 = 1.5 σ2 = 0.15.

Simulated data on original scale (additive mean structure)

24

68

1012

14

Data on original scale (additive)

A

mea

n of

y

1 2 3

B

21

9 / 36


But suppose we only get to observe the data after it has beenrescaled as a power transformation (not on the original scale).

For this example, we will consider Yscaled = Y 2.5. In reality, we do notknow what the scaling is, but we can try to estimate it.

Here is what the observed data looks like...

0200

400

600

800

Data after power transformation (nonadditive)

A

mea

n of

ysc

aled

1 2 3

B

21

10 / 36


Can we apply a transformation in order to remove the interaction inthe observed data? In this case, yes. And this is what Tukey’smethod attempts to do.

Tukey’s test first checks for this type of interaction, then provides asuggested transformation to remove the interaction (or at leastsubstantially reduce it).

We have a 2-factor study with no replication here, so we are limitedin our options...

11 / 36


Two-factor study with no replication...

Source dfA 2B 1error 2c.Total 5

FullModel

Source dfA 2B 1AB 2error 0c.Total 5

MainEffectsModel(assumedaddi3ve)

Tukey’sModelTotestfornonaddi3vity

Source dfA 2B 1nonadd 1error 1c.Total 5

By using the main effects parameters and 1 more parameter “η”, wecan fit a model with a multiplicative interaction (in order to test forthe interaction) using only 1 more d.f. compared to the main effectsmodel.

12 / 36

Algorithm for Tukey 1 d.f. test

1 Fit the preliminary “additive” model.

2 Use y from the additive model, create new variable nonadd =y2ij

2µ .

3 Fit a new model that includes the main effects and the continuousvariable nonadd .

4 Test for significance of a Tukey-type interaction as H0 : η = 0(i.e. H0 : βnonadd = 0). If p-value ≤ α, then there is Tukey-typeinteraction.

5 If the interaction is present, then transform the response using apower transformation “1-η” (or 1− βnonadd) and fit the main effectsmodel on the transformed response. Report the results from thetransformed data additive model.

6 NOTE: If you do not re-scale y2 by 2µ, then it will not affect thep-value for the test, but βnonadd will not be equal to η, which relatesto the suggested transformation.

13 / 36


Example (R Simulation - fit main effects model on original data andcreate ‘nonadd’ variable)

# Set the dummy variable coding to sum-to-zero constraints

> options(contrasts=c("contr.sum","contr.poly"))

# Fit the additive model to the observed data

> lm.out <- lm(yscaled~A+B)

# Create and include the ‘nonadd’ term

> nonadd <- lm.out$fitted^2/(2*lm.out$coef[1])

> lm.out.tukey.test <- lm(yscaled~A+B+nonadd)

14 / 36


Example (R Simulation - test for significance of ‘nonadd’)

# Test the ‘nonadd’ term

> summary(lm.out.tukey.test)

Coefficients:

Estimate Std. Error t value Pr(>|t|)

(Intercept) 80.11499 23.16047 3.459 0.1792

A1 -65.30272 22.40435 -2.915 0.2104

A2 55.47675 16.50254 3.362 0.1841

B1 -27.04728 13.43146 -2.014 0.2934

nonadd 0.78803 0.08244 9.559 0.0664 <--

The ‘nonadd’ term is significant at the α = 0.10 level.

15 / 36


Apply the suggested power transformation of “1-η” and re-fit themain effects model to the re-scaled data, report results ontransformed scale.

Example (R Simulation - apply suggested transformation)

> eta.hat <- coef(lm.out.tukey.test)[5]

# Power transformation

> 1-eta.hat

0.2119691

> y.new <- yscaled^(1-eta.hat)

> interaction.plot(A,B,y.new)

16 / 36


The lines in this profile plot of the transformed data look fairly parallel.

1.5

2.0

2.5

3.0

3.5

4.0

A

mea

n of

y.n

ew

1 2 3

B

21

Example (R Simulation - fit main effects model on transformed data)

> anova(lm(y.new~A+B))

Analysis of Variance Table

Response: y.new

Df Sum Sq Mean Sq F value Pr(>F)

A 2 3.4397 1.71983 101.365 0.009769 **

B 1 0.9820 0.98199 57.878 0.016843 *

Residuals 2 0.0339 0.01697

17 / 36


Because A was significant and it had 3 levels, do a follow-up pairwisecomparison with multiple-comparison adjustment for factor A.

Example (R Simulation, aov() and TukeyHSD())

# Use aov() object to get Tukey HSD for factor A

> a1 <- aov(y.new~A+B)

> TukeyHSD(a1, "A", conf.level=0.95)

Tukey multiple comparisons of means

95% family-wise confidence level

Fit: aov(formula = y.new ~ A + B)

$A

diff lwr upr p adj

2-1 1.1198517 0.3525447 1.887159 0.0241178

3-1 1.8402334 1.0729265 2.607540 0.0090624

3-2 0.7203817 -0.0469252 1.487689 0.056344818 / 36


Example (R Simulation, lm() and multcomp package)

# Use multcomp package to get Tukey HSD for factor A

> library(multcomp)

> lm.out <- lm(y.new~A+B)

> summary(glht(lm.out,linfct=mcp(A="Tukey")))

Simultaneous Tests for General Linear Hypotheses

Multiple Comparisons of Means: Tukey Contrasts

Fit: lm(formula = y.new ~ A + B)

Linear Hypotheses:

Estimate Std. Error t value Pr(>|t|)

2 - 1 == 0 1.1199 0.1303 8.597 0.02420 *

3 - 1 == 0 1.8402 0.1303 14.128 0.00855 **

3 - 2 == 0 0.7204 0.1303 5.530 0.05647 .

(Adjusted p values reported -- single-step method)

19 / 36


Example (R Simulation, lm() and emmeans package)

# Use emmeans package to get Tukey HSD for factor A

> library(emmeans)

> emmeans(lm.out, list(pairwise ~ A), adjust = "tukey")

Results are averaged over the levels of: B

Confidence level used: 0.95

$‘pairwise differences of A‘

contrast estimate SE df t.ratio p.value

1 - 2 -1.12 0.13 2 -8.597 0.0241

1 - 3 -1.84 0.13 2 -14.128 0.0091

2 - 3 -0.72 0.13 2 -5.530 0.0563

Results are averaged over the levels of: B

P value adjustment: tukey method for comparing a family

of 3 estimates 20 / 36


Example (R Simulation, lm() and emmeans package)

> plot(emmeans(lm.out, "A"), comparisons = TRUE)

1

2

3

1.5 2.0 2.5 3.0 3.5 4.0emmean

A

Estimated marginal means and 95% SCIs shown (Tukey adjusted). Thered arrows are meant to let the user know which are significantly different,just notation.

21 / 36

Tukey 1 d.f. test for nonadditivity in Three-way ANOVA

Professor Lenth provided the following example for using the Tukey 1d.f. test in SAS for a Three-way ANOVA with no replication. The data isin Problem 8.2 Oehlert.

Example (Density of particle board relative to three factors)

“The Effect of Three Levels of Logging Slash on the Properties of AspenPlaner Shavings Particleboard”.

Response: Final density (lb/cf) of particle boardThree levels of slash chips in process 0%, 25%, or 50%Two target densities of 42 or 48 lb/cfThree levels of resin 6%, 9%, or 12%

Obs slash target resin response

1 1 1 1 40.9

2 2 1 1 41.9

3 3 1 1 42.0

4 1 2 1 44.4

. . <continues> . .22 / 36


Professor Lenth’s process:

1 Fit a model with all main effects and 2-way interactions (i.e. use the3-way interaction as the error term).

2 Check diagnostics.

3 Test for 2-way interaction.

4 As all 2-way interactions are not significant, pool those 3 terms intothe error and re-fit the main-effects only model.

5 Use Tukey’s 1 d.f. test for this main effects model with 3 factors totest for a special 3-way multiplicative interaction.

23 / 36


Example (SAS)

Fit model with main effects and 2-way interactions

proc glm data=pboard plot=diagnostics;

class slash target resin;

model response = slash target resin

slash*target slash*resin target*resin;

output out=diagnost r=resids p=fitted;

run;

The GLM Procedure

Dependent Variable: response

Sum of

Source DF Squares Mean Square F Value Pr > F

Model 13 171.6122222 13.2009402 28.61 0.0027

Error 4 1.8455556 0.4613889

Corrected Total 17 173.4577778

NOTE: the 4 d.f. for error are from the 3-way interaction term (4=2*1*2).24 / 36


Example (SAS)

Fit model with main effects and 2-way interactions

Source DF Type III SS Mean Square F Value Pr > F

slash 2 9.7377778 4.8688889 10.55 0.0254

target 1 105.6088889 105.6088889 228.89 0.0001

resin 2 52.1244444 26.0622222 56.49 0.0012

slash*target 2 0.8844444 0.4422222 0.96 0.4570

slash*resin 4 2.6655556 0.6663889 1.44 0.3652

target*resin 2 0.5911111 0.2955556 0.64 0.5737

None of the 2-way interactions are significant.

25 / 36


Example (SAS)

Pool the 2-way interactions into the error (i.e. fit main effects only)



model response = slash target resin;

output out=diagnost r=resids p=fitted;

run;


Sum of


Model 5 167.4711111 33.4942222 67.14 <.0001

Error 12 5.9866667 0.4988889


Get y values to create ‘nonadd’ term, and re-fit model including ‘nonadd’

data pboard; set diagnost;

nonadd=fitted**2;

run;26 / 36


Example (SAS)

Re-fit model including ‘nonadd’ continuous termShort-hand model being fit: Y = A + B + C + Y 2

old + ε

NOTE: Using y2 instead of y2

2µwill not affect the p-value, but βnonadd will not directly give you

the suggested transformation to remove any existing interaction.



model response = slash target resin nonadd/solution;

run;

The GLM Procedure


Sum of


Model 6 167.4905844 27.9150974 51.46 <.0001

Error 11 5.9671934 0.5424721


NOTE: 1 more d.f. in model and 1 less d.f. in error compared to main effects model.27 / 36


Example (SAS)

Source DF Type III SS Mean Square F Value Pr > F

slash 2 0.31699300 0.15849650 0.29 0.7523

target 1 0.32176632 0.32176632 0.59 0.4574

resin 2 0.33116828 0.16558414 0.31 0.7430

nonadd 1 0.01947328 0.01947328 0.04 0.8532

‘nonadd’ term not significant ⇒ additive model OK.

NOTE: DO NOT INCLUDE ‘nonadd’ in the final model! It’s for diagnosticpurposes only - not modeling.

We will consider the the additive model for the factors to be sufficient, andproceed with modeling and conclusions.

28 / 36


Example (SAS)

Our follow-up analysis considers the marginal effects for each factor sincethere are no significant interactions.

Both slash (0, 25, 50) and resin (6, 9, 12) are quantitative variables, sowe can consider linear and quadratic contrasts for these terms. We’ll use

orthogonal polynomial contrasts for this and tell SAS that these arefactors in the ‘class’ statement.

For target, we’ll estimate the two means and the difference between them.We’ll use the estimate statement for the contrasts for which we canprovide meaningful interpretations, and contrasts statements for things Ijust want to test.

29 / 36


Example (SAS)



model response = slash target resin;

estimate "slash lin" slash -1 0 1/divisor=50; /* coded levels (1,2,3) */

contrast "slash quad" slash 1 -2 1;

estimate "resin lin" resin -1 0 1/divisor=6; /* coded levels (1,2,3) */

contrast "resin quad" resin 1 -2 1;

estimate "target 42" intercept 3 target 3 0 slash 1 1 1 resin 1 1 1/divisor=3;

estimate "target 48" intercept 3 target 0 3 slash 1 1 1 resin 1 1 1/divisor=3;

estimate "target diff" target -1 1;

lsmeans target/diff cl;

run;

NOTE 1: divisor=50 and divisor=6 are relevant due to actual levels of factors, at (0,25,50)and (6,9,12), respectively.

NOTE 2: ”Target 42” estimate coincides with Y1.. = µ+ α1 + β. + γ.

= µ+ α1 +∑βj

3+

∑γk

3= (3µ+3α1+(β1+β2+β3)+(γ1+γ2+γ3))

3

30 / 36


Example (SAS)

Contrast DF Contrast SS Mean Square F Value Pr > F

slash quad 1 0.01777778 0.01777778 0.04 0.8534

resin quad 1 0.87111111 0.87111111 1.75 0.2110

⇒ Quadratic terms are not needed.

Standard

Parameter Estimate Error t Value Pr > |t|

slash lin 0.0360000 0.00815589 4.41 0.0008

resin lin 0.6888889 0.06796574 10.14 <.0001

target 42 43.7666667 0.23544022 185.89 <.0001

target 48 48.6111111 0.23544022 206.47 <.0001

target diff 4.8444444 0.33296276 14.55 <.0001

⇒ Linear in resin and slash.

31 / 36


Example (SAS)

Standard


slash lin 0.0360000 0.00815589 4.41 0.0008

resin lin 0.6888889 0.06796574 10.14 <.0001

target 42 43.7666667 0.23544022 185.89 <.0001

target 48 48.6111111 0.23544022 206.47 <.0001

target diff 4.8444444 0.33296276 14.55 <.0001

Least Squares Means

response

target LSMEAN 95% Confidence Limits

1 43.766667 43.253686 44.279647

2 48.611111 48.098131 49.124091

Least Squares Means for Effect target

Difference

Between 95% Confidence Limits for

i j Means LSMean(i)-LSMean(j)

1 2 -4.844444 -5.569908 -4.118981

The output from the LSMEANS matches the custom-estimates we madefor target means and difference between target means.

32 / 36


Example (SAS)

Because we used divisor=50 and divisor=6 in our linear contrasts forslash and resin, those estimates actually provide us with the estimatedslopes when they are considered continuous predictors.

Standard


slash lin 0.0360000 0.00815589 4.41 0.0008

resin lin 0.6888889 0.06796574 10.14 <.0001

target 42 43.7666667 0.23544022 185.89 <.0001

target 48 48.6111111 0.23544022 206.47 <.0001

target diff 4.8444444 0.33296276 14.55 <.0001

The main conclusions are:

For each 1% increase in slash density, the board density increases by 0.036 lb/ft3 onaverage, plus or minus about 0.017 (with 95% confidence – multiplying the SE byt0.025,12 = 2.18).

For each 1% increase in resin, the board density increases by 0.69 lb/ft3 on average, plusor minus about 0.14.

When we were trying to get a density of 42 or 48, respectively we actually achieved meandensities of 43.77 ± 0.47 and 48.61 ± 0.47; the difference is 4.84 ± 0.66.

33 / 36


Example (SAS)

Additional Note: In case you are uncomfortable with, or don’t believethat we estimated the slopes correctly, an ordinary regression model(i.e. not specifying slash or resin as class variable) will yield the sameestimate. The standard errors are slightly different because this modeldoesn’t include the quadratic effects.

data pboard2; set pboard;

slash = 25*(slash-1); /*(1,2,3) --> (0,25,50)*/

resin = 3*(resin+1); /*(1,2,3) --> (6,9,12) */

run;

proc glm data=pboard2 plot=diagnostics;

class target; /* slash and resin are continuous*/

model response = slash target resin/solution;

run;

34 / 36


Example (SAS)

Standard


Intercept 41.51111111 B 0.68104857 60.95 <.0001

slash 0.03600000 0.00809206 4.45 0.0006

target 1 -4.84444444 B 0.33035708 -14.66 <.0001

target 2 0.00000000 B . . .

resin 0.68888889 0.06743386 10.22 <.0001

35 / 36

References

John Tukey (1949)

One degree of freedom for non-additivity

Biometrics 5, 232 – 242.

Basilio A. Rojas (1973)

On Tukey’s test of additivity

Biometrics 29, 45 – 52.

Gary .W. Oehlert (2000)

A First Course in Design and Analysis of Experiments

New York: W.H. Freeman and Company.

36 / 36

Documents

Factorials with no replication - homepage.stat.uiowa.eduhomepage.stat.uiowa.edu/~rdecook/stat5201/notes/3-10_factorial_n… · Algorithm for Tukey 1 d.f. test 1 Fit the preliminary