1

Analysis Of Variance Compiled by T.O. Antwi-Asare, U.G

ANOVA

•Analysis of variance compares two or more population means of interval data.

• Specifically, we are interested in determining whether differences exist between the population means.

• The procedure works by analyzing the sample variances.

• The assumptions underlying the analysis of variance technique are

• the same as those used in the t test when comparing two different means.

•We assume that the samples are randomly and independently drawn

• from Normally distributed populations which have equal variances.

•We deal with variable within the interval scale or ratio scale

• To formalise this we break down the total variance of all the observations into

1. the variance due to differences between treatments or factors, and

2. the variance due to differences within treatments (also known as the error variance).

•we have to work with three sums of squares:

• The total sum of squares measures (squared) deviations from the overall or grand average using all the observations. It ignores the existence of the different factors.

• The between sum of squares is based upon the averages for each factor and measures how they deviate from the grand average.

• The within sum of squares is based on squared deviations of observations from their own factor mean.

• Total sum of squares=Between Sum

•of Squares + Within Sum of Squares

• The larger - the between sum of squares relative to the within sum of squares, the more likely it is that the null is false.

One Way Analysis of Variance • Example • An apple juice manufacturer is planning to

develop a new product -a liquid concentrate. • The marketing manager has to decide how to

market the new product. • Three strategies are considered

• Emphasize the convenience of using the product.

• Emphasize the quality of the product. • Emphasize the product’s low price.

One Way Analysis of Variance

• Example: An experiment was conducted as follows: In

three cities an advertisement campaign was launched .

• In each city only one of the three characteristics

(convenience, quality, and price) was emphasized.

• The weekly sales were recorded for twenty weeks

following the beginning of the campaigns.

•Problem assumptions

• The data are interval

• The problem objective is to compare sales in the

three cities.

•We hypothesize that the three population means

are equal

One Way Analysis of Variance

Convenience Quality Price529 804 672658 630 531793 774 443514 717 596663 679 602719 604 502711 620 659606 697 689461 706 675529 615 512498 492 691663 719 733604 787 698495 699 776485 572 561557 523 572353 584 469557 634 581542 580 679614 624 532

Weekly sales

H0: m1 = m2= m3

H1: At least two means differ

To build the statistic needed to test the hypotheses we use the following notation:

• Solution Defining the Hypotheses

Independent samples are drawn from k populations (treatments).

1 2 k

X11

x21

.

.

. Xn1,1

1

1

x

n

X12

x22

.

.

. Xn2,2

2

2

x

n

X1k

x2k

.

.

. Xnk,k

k

k

x

n

Sample size

Sample mean

First observation, first sample

Second observation, second sample

X is the “response variable”. The variables’ value are called “responses”.

Notation

Terminology

• In the context of this problem… Response variable – weekly sales

Responses – actual sale values Experimental unit – weeks in the three cities when we record sales figures. Factor – the criterion by which we classify the populations (the treatments). In this problems the factor is the marketing strategy.

Factor levels – the population (treatment) names. In this problem factor levels are the marketing strategies.

Two types of variability are employed when testing for the equality of the population means

The rationale of the test statistic

20

25

30

1

7

Treatment 1 Treatment 2 Treatment 3

10

12

19

9

Treatment 1 Treatment 2 Treatment 3

20

16 15 14

11 10

9

10x1

15x2

20x3

10x1

15x2

20x3

The sample means are the same as before, but the larger within-sample variability makes it harder to draw a conclusion about the population means.

A small variability within the samples makes it easier to draw a conclusion about the population means.

The rationale behind the test statistic – Part I

• If the null hypothesis is true, we would expect all the sample means to be close to one another (and as a result, close to the grand mean).

• If the alternative hypothesis is true, at least some of the sample means would differ.

• Thus, we measure variability between sample means.

• The variability between the sample means is measured as the sum of squared distances between each treatment mean and the grand mean.

This sum is called the

Sum of Squares for Treatments-SST or Between Sum of Squares BSS

In our example treatments are represented by the different advertising strategies.

Variability between sample means

•NOTE:

•Here SST ≠ Total Sum of Squares TSS = BSS

• It is the Between Sum of Squares

2k

1jjj

)xx(nSSTor BSS

There are k treatments

The size of sample j The mean of sample j or Factor j or treatment j

Sum of squares for treatments (SST) or Between Sum of Squares BSS

Note: When the sample means are close to one another, their distance from the grand mean is small, leading to a small SST. Thus, large SST indicates large variation between sample means, which supports H1.

• Solution – continued Calculate SST or BSS

2k

1jjj

321

)xx(nSST

65.60800.653577.55x

xx

= 20(577.55 - 613.07)2 + 20(653.00 - 613.07)2 + 20(608.65 - 613.07)2

= 57,512.23

The grand mean is calculated by

k

kk

nnn

xnxnxnX

...

...

21

2211

Sum of squares for treatments (SST) or BSS

• Large variability within the samples weakens the “ability” of the sample means to represent their corresponding population means.

• Therefore, even though sample means may markedly differ from one another, SST must be judged relative to the “within samples variability”.

The rationale behind test statistic – Part II

• The variability within samples is measured by adding all the squared distances between observations and their sample means.

• This sum is called the

Sum of Squares for Error – SSE or WSS

In our example this is the sum of all squared differences between sales in city j and the sample mean of city j (over all the three cities).

Within samples variability SSE or WSS (Within Sum of Squares) or ESS

• For example:

• SSE or WSS (n1 - 1)s12 + (n2 -1)s2

2 + (n3 -1)s32 + …+

(nk – 1)sk

= 𝑛𝑗 − 1𝑘𝑗=1 𝑠𝑗

2

k = no. of treatments

k

jjij

n

i

xxSSEj

1

2

1

)(

• 𝑛𝑗 − 1𝑘𝑗=1 𝑠𝑗

2 = 𝑆𝑆𝐸 𝑜𝑟 𝑊𝑆𝑆 where is the column j mean

k

jjij

n

i

xxSSEj

1

2

1

)(

jx

• Solution Continued: Calculate SSE Sum of squares for errors (SSE)

k

jjij

n

i

xxSSE

sss

j

1

2

1

2

3

2

2

2

1

)(

24.670,811,238,700.775,10

Or, SSE (n1 - 1)s12 + (n2 -1)s2

2 + (n3 -1)s32

= (20 -1)10,774.44 + (20 -1)7,238.61+ (20-1)8,670.24 = 506,983.50

To perform the test we need to calculate the mean squares as follows:

The mean sum of squares

Calculation of MST - Mean Square for Treatments

12.756,28

13

23.512,57

1

k

SSTMST

Calculation of MSE Mean Square for Error

45.894,8

360

50.983,509

kn

SSEMSE

23.3

45.894,8

12.756,28

MSE

MSTF

Calculation of the test statistic

with the following degrees of freedom: v1=k -1 and v2=n-k

Required Conditions: 1. The populations tested are normally distributed. 2. The variances of all the populations tested are equal.

And finally the Decision Rule

H0: m1 = m2 = …=mk H1: At least two means differ Test statistic: Reject H0 if: F>Fa,k-1,n-k

MSE

MSTF

The F test rejection region

The F test

Ho: m1 = m2= m3

H1: At least two means differ Test statistic F= MST/ MSE= 3.23 15.3FFF:.R.R 360,13,05.0knk a 1

Since 3.23 > 3.15, there is sufficient evidence to reject Ho in favor of H1, and argue that at least one of the mean sales is different than the others.

23.3

17.894,8

12.756,28

MSE

MSTF

ANOVA

Anova: Single Factor

SUMMARY

Groups Count Sum Average Variance

Convenience 20 11551 577.55 10775.00

Quality 20 13060 653.00 7238.11

Price 20 12173 608.65 8670.24

ANOVA

Source of Variation SS df MS F P-value F crit

Between Groups 57512 2 28756 3.23 0.0468 3.16

Within Groups 506984 57 8894

Total(TSS) 564496 59

• 𝑛𝑗 − 1𝑘𝑗=1 𝑠𝑗

2= SSE

2k

1jjj

)xx(nSSTor BSS

Question

•The reaction times of three groups of sportsmen were measured on a particular task, with the following results (time in milliseconds):

•Racing drivers 31 28 39 42 36 30

•Tennis players 41 35 41 48 44 39 38

•Boxers 44 47 35 38 51

•Test whether there is a difference in reaction times between the three groups.

Introduction

ANOVA is the technique where the total variance present in the data set is spilt up into non- negative components where each component is due to one factor or cause of variation.

Factors of variation

Assignable Non-assignable

Can be many Error or Random

variation

ANOVA is used to test hypotheses about

differences between two or more means.

The t-test can only be used to test differences

between two means.

When there are more than two means, it is

possible to compare each mean with each other

mean using t-tests.

However, conducting multiple t-tests can lead

to severe inflation of the Type I error type.

ANOVA is used to test differences among

several means for significance without

increasing the Type I error rate using an F test

Utility

The ANOVA Procedure:

This is the ten step procedure for analysis of variance:

1.Description of data

2.Assumption: Along with the assumptions, we represent the model for each design we discuss.

3. Hypothesis

4.Test statistic

5.Distribution of test statistic

6.Decision rule

7.Calculation of test statistic: The results of the arithmetic calculations will be summarized in a table called the analysis of variance (ANOVA) table. The entries in the table make it easy to evaluate the results of the analysis.

8.Statistical decision

9.Conclusion

10.Determination of p value

ONE-WAY ANOVA-

Completely Randomized Design (CRD)

One-way ANOVA:

It is the simplest type of ANOVA, in which

only one source of variation, or factor, is

investigated.

It is an extension to three or more samples of

the t test procedure for use with two

independent samples

In another way t test for use with two

independent samples is a special case of one-

way analysis of variance.

Experimental design used for one-way ANOVA is called

Completely randomised design.

This tests the effect of equality of several treatments of

one assignable cause of variation.

Based on two principles- Replication and

randomization.

Advantages:

Very simple:

Reduces the experimental error to a great extent.

We can reduce or increase some treatments.

Suitable for laboratory experiments.

Disadvantages: Design is not suitable if the experimental

units are not homogeneous.

Design is not so much efficient and sensitive as compared

to others.

Local control is completely neglected.

Not suitable for field experiment.

Hypothesis Testing Steps:

1. Description of data: The measurements( or

observation) resulting from a completely randomized

experimental design, along with the means and totals.

Available

Subjects

Random

numbers

02

01

03 05 04 06 08 07 09

10 11 12 13 15 14

16

09

16

06 14 15 11 04 02 10

07 05 13 03 01 12

08

16 09 06 15 14 11 02 04 10 07 05 13 03 12 01 08

Table of Sample Values for the CRD

Treatment

1 2 3 … K

x11 x12 x13 … x1k

x21 x22 x23 …. X2k

. . . .

xn11

xn22

xn33 xnkk

Total T.1 T.2 T.3 T.k T..

Mean x.1 x.2 x.3 x.k x..

Table of Sample Values for the Randomized Complete Block

Design

Treatments

Blocks 1 2 3 … k Total Mean

1 x11 x12 x13 ... x1k T 1. x 1.

2 x21 x22 x23 … x2k T 2. x 2.

.

.

. n xn1 xn2 xn3 …. xnk Tn. X n.

Total T.1 T.2 T.3 … T.k T..

Mean x.1 x.2 x.3 … x.k x..

T.j = xij = total of the jth treatment

x.j = T.j/nj = mean of jth treatment

T .. = T.j = xij = total of all observations

x.. = T../N

, N = nj

xij = the ith observation resulting from the jth treatment

(there are a total of k treatment)

2. Assumption:

The Model

The one-way analysis of variance may be written as

follows:

xij = m j eij; i=1,2…nj, j= 1,2….k

The terms in this model are defined as follows:

1. m represents the mean of all the k population means

and is called the grand mean.

2. j represents the difference between the mean of the

jth population and the grand mean and is called the

treatment effect.

3. eij represents the amount by which an individual

measurement differs from the mean of the population to

which it belongs and is called the error term.

Assumptions of the Model

The k sets of observed data constitute k independent

random samples from the respective populations.

Each of the populations from which the samples come is

normally distributed with mean mj and variance j2.

Each of the populations has the same variance. That is

12= 2

2…= k2= 2, the common variance.

The j are unknown constants and j = 0, since the sum

of all deviations of the mj from their mean, m, is zero.

The (errors) eij have a mean of 0, since the mean of xij is

mj

The eij have a variance equal to the variance of the xij,

since the eij and xij differ only by a constant.

The eij are normally (and independently) distributed.

3. Hypothesis:

We test the null hypothesis that all population or

treatment means are equal against the alternative that the

members of at least one pair are not equal. We may state the

hypothesis as follows

H0: µ1 = µ2 =…..= µk

HA: not all µj are equal

If the population means are equal, each treatment effect is

equal to zero, so that alternatively, the hypothesis may be

stated as

H0: τj = 0, j=1,2,…….,k

HA: not all τj =0

4. Test statistic:

Table: Analysis of Variance Table for the Completely Randomized Design

The Total Sum of squares(TSS): It is the sum of the squares of the deviations of individual observations taken together.

Source of

variation Sum of square d.f Mean square Variance

ratio

Among

sample

k-1 MSA=SSA/(k-1)

MS due to Treatment

V.R=MSA/MSW=F

Within

samples

N-k MSW=SSW/(N-k)

MS due to error

Total N-1

k

jjj xxnSSA

1

2

...)(

k

j

n j

ijij

SSW xx1 1

. )(

2

k

j

n j

iij xxSST

1 1..)(

2

The Within Groups of Sum of Squares:

The first step in the computation call for performing some

calculations within each group. These calculation involve computing within each group the sum of squared deviations of the individual observations from their mean. When these calculations have been performed within each group, we obtain the sum of the individual group results.

The Among Groups Sum of Squares:

To obtain the second component of the total sum of square,

we compute for each group the squared deviation of the group mean from the grand mean and multiply the result by the size of the group. Finally we add these results over all groups. Total sum of square is equal to the sum of the among and the within sum of square.

TSS=SSA+SSW

The First Estimate of σ2:

Within any sample

Provides an unbiased estimate of the true variance of the

population from which the sample came. Under the

assumption that the population variances are all equal, we

may pool the k estimate to obtain

1

1. )(

2

n

xx

j

n j

jjij

k

jjn

xxk

j

n j

ijij

1

2

)1(

1 1. )(

The Second Estimate of σ2:

The second estimate of σ2 may be obtain from the familiar

formula for the variance of sample means, . If we

solve this equation for σ2, the variance of the population

from which the samples were drawn, we have

An unbiased estimate of , computed from sample data, is provided by

If we substitute this quantity into equation we obtain the

desired estimate of σ2

nx

2

2

22

xn

1

1... )(

2

k

k

jj xx

1

1... )(

2

k

k

jjn xx

2

x

When the sample sizes are not all equal, an estimate of σ2 based on

the variability among sample means is provided by

The Variance Ratio:

What we need to do now is to compare these two estimates of σ2, and we do this by computing the following variance ratio,

which is the desired test statistic:

1

1... )(

2

k

k

jjj xxn

V.R =

Among groups mean square

Within groups mean square

6. Distribution of Test statistic:

F distribution we use in a given situation depends on

the number of degrees of freedom associated with the

sample variance in the numerator and the number of

degrees of freedom associated with the sample variance in

the denominator.

we compute V.R. in situations of this type by placing

the among groups mean square in the numerator and the

within groups mean square in the denominator , so that the

numerator degrees of freedom is equal to the number of

groups minus 1, (k-1), and the denominator degrees of

freedom value is equal to

k

j

k

jjj

kNknn1 1

)1(

7. Significance Level:

Once the appropriate F distribution has been

determined, the size of the observed V.R. that will cause rejection of the hypothesis of equal population variances depends on the significance level chosen. The significance level chosen determines the critical value of F, the value that separates the nonrejection region from the rejection region.

8. Statistical decision:

To reach a decision we must compare our computed V.R. with the critical value of F, which we obtain by entering Table G with k-1 numerator degrees of freedom and N-k denominator degrees of freedom .

If the computed V.R. is equal to or greater than the critical value of F, we reject the null hypothesis. If the computed value of V.R. is smaller than the critical value of F, we do not reject the null hypothesis.

9. Conclusion:

When we reject H0 we conclude that not all population

means are equal. When we fail to reject H0, we conclude

that the population means may be equal.

10. Determination of p value