Experimental Design and

Analysis of Variance

Chapter 11

McGraw-Hill/Irwin Copyright © 2011 by The McGraw-Hill Companies, Inc. All rights reserved.

Experimental Design and

Analysis of Variance

11.1 Basic Concepts of Experimental Design

11.2 One-Way Analysis of Variance

11-2

11.1 Basic Concepts of

Experimental Design

We have considered only one way of collecting and comparing data:

Using independent random samples

Often data is collected as the result of an experiment

To systematically study how one or more factors (variables) influence the variable that is being studied

11-3

Experimental Design

In an experiment, there is strict control over the factors

(independent variables) contributing to the experiment

The values or levels of the factors are called treatments

The objective is to compare and estimate the effects of

different treatments on the response variable

The different treatments are assigned to objects (the

test subjects) called experimental units

When a treatment is applied to more than one

experimental unit, the treatment is being “replicated”

11-4

Experimental Design

A designed experiment is an experiment where

the analyst controls which treatments are used

and how they are applied to the experimental

units

Example: An oil company wishes to study how

three different gasoline types (A, B, and C) affect

the mileage of a midsized car.

Response Variable: Mileage

Treatments: Gasoline Type (A, B, and C)

Experimental Units: Midsized Cars

11-5

Experimental Design

In a completely randomized experimental design,

independent random samples are assigned to each

of the treatments

For example, suppose three experimental units are to be

assigned to five treatments

For completely randomized experimental design, randomly

pick three experimental units for one treatment, randomly

pick three different experimental units from those

remaining for the next treatment, and so on

11-6

Experimental Design

Once the experimental units are assigned

and the experiment is performed, a value of

the response variable is observed for each

experimental unit

Obtain a sample of values for the response

variable for each treatment

11-7

Example: Battery Testing

Suppose you wish to determine which of

three brands of AA battery (Energizer,

Eveready, and Tiger) lasts the longest when

used in a remote controlled car. You have 30

cars, so you assign 10 to each battery brand.

Determine the following:

Response Variable

Treatment

Experimental Unit

Gasoline Mileage Case

North American Oil Company is attempting to

develop a reasonably priced gasoline that will

deliver improved gasoline mileages. As part

of its development process, the company

would like to compare the effects of three

types of gasoline (A, B and C) on gasoline

mileage. To test the three types of gasoline,

the company assigned 5 cars for each type of

gasoline and measured the mileages.

11.2 One-Way Analysis of

Variance

Objective is to estimate and compare the effects of the

different treatments on the response variable.

Given p treatments on a response variable, we try to

estimate the differences between the means i of each

treatment.

11-10

ANOVA

Want to study the effects of all p treatments on a response variable For each treatment, find the mean and standard deviation

of all possible values of the response variable when using that treatment

For treatment i, find treatment mean µi

One-way analysis of variance estimates and compares the effects of the different treatments on the response variable By estimating and comparing the treatment means µ1, µ2,

…, µp

One-way analysis of variance, or one-way ANOVA

11-11

ANOVA Notation

p is the total number of treatments

i is the representation of a treatment (ex: A, B, C)

ni denotes the size of the sample randomly selected for treatment i

xij is the jth value of the response variable using treatment i

i is the average of the sample of ni values for treatment i i is the point estimate of the treatment mean µi

si is the standard deviation of the sample of nivalues for treatment i si is the point estimate for the treatment (population)

standard deviation σi

11-12

Gasoline Mileage Case

p = 3 i = A, B, C

nA = nB = nC = 5

Type A Type B Type C

xA1=34.0 xB1=35.3 xC1=33.3

xA2=35.0 xB2=36.5 xC2=34.0

xA3=34.3 xB3=36.4 xC3=34.7

xA4=35.5 xB4=37.0 xC4=33.0

xA5=35.8 xB5=37.6 xC5=34.9

Gasoline Mileage Case

The mean of a sample is the point

estimate for the corresponding

treatment mean

A = 34.92 mpg estimates A

B = 36.56 mpg estimates B

C = 33.98 mpg estimates C

Gasoline Mileage Case

Gasoline Mileage Case

The standard deviation of a sample is the

point estimate for the corresponding

treatment standard estimates

sA = 0.7662 mpg estimates σA

sB = 0.8503 mpg estimates σB

sC = 0.8349 mpg estimates σC

One-Way ANOVA

Assumptions

1. Completely randomized experimental design

Assume that a sample has been selected

randomly for each of the p treatments on the

response variable using a completely randomized

experimental design

2. Constant variance

The p populations of values of the response

variable (associated with the p treatments) all

have the same variance

11-17

3. Normality

The p populations of values of the response

variable all have normal distributions

4. Independence

The samples of experimental units are randomly

selected, independent samples

11-18

One-Way ANOVA

Assumptions

One-Way ANOVA

Assumptions

To make sure that unequal variances will not

be a problem:

Take the same sample size per treatment

Check the different sample standard deviations

General Rule: The one-way ANOVA results will

be approximately correct if the largest sample

standard deviation is no more than twice the

smallest sample standard deviation.

Gasoline Mileage Case

The standard deviation of a sample is the

point estimate for the corresponding

treatment standard estimates

sA = 0.7662 mpg estimates σA

sB = 0.8503 mpg estimates σB

sC = 0.8349 mpg estimates σC

Testing for Significant Differences

Between Treatment Means

Are there any statistically significant differences

between the sample (treatment) means?

The null hypothesis is that the mean of all p

treatments are the same

H0: µ1 = µ2 = … = µp

The alternative is that some (or all, but at least two)

of the p treatments have different effects on the

mean response

Ha: at least two of µ1, µ2 , …, µp differ

11-21

Testing for Significant Differences

Between Treatment Means

Compare the between-treatment variability

to the within-treatment variability

Between-treatment variability is the variability of

the sample means from sample to sample

Ex: Variability between A, B, C

Within-treatment variability is the variability of the

treatments (that is, the values) within each sample

Ex: Variability between A and xA1, xA2,…, xA5

11-22

Comparing Between-Treatment

Variability and Within-Treatment

Variability

11-23

Partitioning the Total Variability

in the Response

Total

Variability

= Between

Treatment

Variability

+ Within

Treatment

Variability

Total Sum of

Squares

= Treatment Sum of

Squares

+ Error Sum of

Squares

SSTO = SST + SSE

p

i

n

j

iij

p

i

n

j

p

i

iiij

ii

xxxxnxx1 1

2

1 1 1

22

11-24

Mean Squares

The treatment mean-squares is

The error mean-squares is

1

p

SSTMST

pn

SSEMSE

11-25

Gasoline Mileage Case

222

1

2xxnxxnxxnxxnSST CCBBAA

p

i

ii

222153.3598.335153.3556.365153.3592.345

0493.17

CBAi n

j

CCj

n

j

BBj

n

j

AAj

p

i

n

j

iij xxxxxxxxSSE1

2

1

2

1

2

1 1

2

028.8

F Test for Difference Between

Treatment Means

Suppose that we want to compare p

treatment means

The null hypothesis is that all treatment

means are the same:

H0: µ1 = µ2 = … = µp

The alternative hypothesis is that they are not

all the same:

Ha: at least two of µ1, µ2 , …, µp differ

11-27

F Test for Difference Between

Treatment Means

Define the F statistic:

The p-value is the area under the F curve to

the right of F, where the F curve has p – 1

numerator and n – p denominator degrees of

freedom

pnSSE

pSST

MSE

MSTF=

1

11-28

F Test for Difference Between

Treatment MeansReject H0 in favor of Ha at the a level of

significance if

F > Fa , or if

p-value < a

Fa is based on p – 1

numerator and n – p

denominator degrees

of freedom

11-29

Gasoline Mileage Case

Computing for the F statistic

To test H0 at a = 0.05, we use F0.05 with

Numerator: p – 1 = 3 – 1 = 2

Denominator: n – p = 15 – 3 = 12

F0.05 = 3.89

Since F = 12.74 > F0.05 = 3.89, we reject H0

74.12

315028.8

130493.17

1

pnSSE

pSST

MSE

MSTF=

Excel Output: ANOVA Test

Anova: Single Factor

SUMMARY

Groups Count Sum Average Variance

Type A 5 174.6 34.92 0.587

Type B 5 182.8 36.56 0.723

Type C 5 169.9 33.98 0.697

Excel Output: ANOVA Test

ANOVA

Source of Variation SS df MS F P-value F crit

Between Groups 17.0493 2 8.5246 12.7424 0.0011 3.8853

Within Groups 8.028 12 0.669

Total 25.07733 14

F Test for Difference Between

Treatment Means

From the F test, we can conclude that at

least two of the treatment means differ. But

how do we know which ones differ?

We compare two means at a time. (Pairwise

Comparison)

Pairwise Comparisons,

Individual Intervals

• Tukey simultaneous 100(1 - a)% confidence interval for µi – µh:

• qa is the upper a percentage point of the studentized range for p and (n – p) from Table A.9

• m denotes common sample size

m

MSEqxx αhi

Pairwise Comparisons,

Individual Intervals

If the sample sizes of the two treatment

means are unequal:

hi

αhi

nnMSE

qxx

11

2

Confidence Intervals for

Treatment Means

A point estimate of the treatment mean is the

sample mean of a treatment

We can also make a confidence interval for

each treatment with a confidence level of (1-

a)

i

in

MSEtx 2/a

Hypothesis Testing Between

Treatment Means

Ho: i - h = 0

Ha: i -h 0

This test tells us whether the two treatment

means are equal or different.

hi

hi

nnMSE

xxt

11

Hypothesis Testing Between

Treatment Means

Critical Value = r = p , v = n – p

Rejection Rule: If the test statistic is greater

than the critical value, reject Ho.

If we reject Ho, this means that the two

treatment means are not equal.

2

aq

Hypothesis Testing Between

Treatment Means

Tukey simultaneous comparison t-values (d.f. = 12)

Type C Type A Type B

33.98 34.92 36.56

Type C 33.98

Type A 34.92 1.82

Type B 36.56 4.99 3.17

critical values for experimentwise error rate:

0.05 2.67

0.01 3.56

Hypothesis Testing Between

Treatment Means

p-values for pairwise t-tests

Type C Type A Type B

33.98 34.92 36.56

Type C 33.98

Type A 34.92 .0942

Type B 36.56 .0003 .0081