12 Analysis of Variance - NCCU

12 Analysis of VarianceI. Comparing two population variances • F-distribution : F-test • Testing hypotheses
II. Comparing k3 population means : • Testing • Decompose the sample variations • ANOVA F-tests
I. Assumptions II. One-way ANOVA III. Two-way ANOVA
2 2
2 11
2 2
k210 ...:H µ==µ=µ
3
• Smaller variance = less variation – Better performance in quality control (QC)
• Example. Two machines are set to produce steel bars of the same length. The bars should have the same mean length. In addition, we want to ensure they have similar variation.
– Smaller risk in stock market • Example. A sample of 10 internet stocks and 10 utility stocks
shows the same mean rate of return, but there is likely more variation in the internet stocks.
4
– Two normal populations – Equal population variances:
• Ratio of two sample variances ~ F-distribution with degrees of freedom (n1-1, n2-1)
– Q : What is the sampling distribution of ?
)1n,1n(2 2
2 1
21 F~
S SF −−=
2 2
6
F-distribution
• A family of F-distribution : – determined by the degrees of freedom in the numerator and
denominator. • F-distribution is continuous on (0, ∞). • Positively skewed, non-symmetrical,
7
F(n1-1,n2-1,α/2)
F(n1-1,n2-1)
24.021.4/1F/1F )05.0,6,7()05.01,7,6( ===−
– Two-sided :
• Step 2. Select the significance level α: 0.01, 0.05 or 0.10
• Step 3. Determine the test statistic :
2 2
2 11
2 2
2 2
2 1
S SF =
2 2
2 11
2 2
11
– Two-sided :
• Step 5. Collect data, calculate the F-value and draw statistical conclusion.
)2/,1n,1n( )2/1,1n,1n(2
12
Example. P389
Lammers Limos offers limousine service from Toledo to Detroit. Two routes are considered. One is via US 25 and the other via I-75.
The times to the airport by each route are studied. The following sample data is collected. Using the 0.10 significance level, is there a difference in
the variation in the driving times using the two routes?
Route Sample Summary statistics )s,x,n( 2 US Route 25 52, 67, 56, 45, 70, 54, 64 (7, 58.29, 8.88472) Interstate 75 59, 60, 61, 51, 56, 63, 57, 65 (8, 59.00, 4.37532)
13
–
• Step 2. Select the significance level α= 0.10, α/2= 0.05
• Step 3. Determine the test statistic :
2 2
2 11
2 2
2 2
2 1
S SF =
14
• Step 4. Determine the critical value(s)/rejection region : – A two-sided F-test – n1-1=7-1=6, n2-1=8-1=7
24.0 21.4 1
• Step 5. Collect data and calculate F-value :
• Statistical conclusion : at significance level 0.10, the null hypothesis of no difference in variances is rejected.
• Decision : There is a difference in the variation of the travel times along the two routes.
87.3F23.4 )3753.4( )9947.8(
S SF )05.0,7,6(2
52 59
64 57 F 4.2264
3.8660
Thus, the null hypothesis is rejected at α=0.1.
18
• Testing the null hypothesis :
• Assumptions : – There are k populations classified by a factor. – k Populations :
• Normal distributions • equal variances,
22 k
2 2
19
k21 µµµ
Draw k independent samples ...
0
• Treatment: intervention of interest • Other variables
– Factor level: possible levels of the factor – Q: Does a treatment effect exist?
21
• Recall : to compare 2 population means, a t-test is used. • If the equality of the means of 4 groups A, B, C, D is
studied, , 6 t-tests are needed. – A vs. B – A vs. C – A vs. D – B vs. C – B vs. D – C vs. D
DCBA0 :H µ=µ=µ=µ
22
• What happen if the number of tests becomes large? – Inflation of overall Type I error rate!
– Assume each hypothesis is tested at α=0.05, for each hypothesis P(individual type I error) = 0.05
– Overall type I error rate = P(At least one null hypothesis is incorrectly rejected) = 1-P(No null hypothesis is incorrectly rejected) = 1-P(Null hypothesis 1 is not incorrectly rejected)…P(null
hypothesis 6 is not incorrectly rejected) = 1- 0.95×0.95×…×0.95=1-0.956
=0.265 >> 0.05
23
One-way ANOVA : an example
– An example. • Does a fertilizer affect the wheat production? • Treatment : 3 commercial brands of fertilizer(3-level) • Factor: treatment, farms, etc.
24
–Experiment : –divides the field into 12 plots of equal size –The wheat is planted at the same time in the same manner –Each fertilizer was randomly assigns to 4 plots –At the end of the growing season, the number of bushels of wheat produced on each plot is recorded.
–GOAL :
Is there a difference in the mean production of wheat ?
“Different treatment causes different mean production”?
“The production is related to the treatment”?
“Treatment vs. production”
One-way ANOVA :
Example. P393 The manager of a financial center wants to compare the
productivity, as measured by the number of customers served, among three employees. Four days are randomly selected an the number of customers served by each employee is recorded.
Treatment : 3 employees GOAL :
“The productivity is related to the employee”?
“Treatment vs. productivity”
26
Are the group means the same and equal to the overall mean?
Employee production Sample mean difference to over
all mean Wolfe 55 54 59 56 56 -2 White 66 76 67 71 70 12 Korosa 47 51 46 48 48 -10
Overall mean = 58
27
Customers served
em pl
oy ee
=Overall mean Customers served
• Decompose total variation of the observations into... – Due to treatment – explainable variation; – Due to randomness – unexplainable variation.
• For every observation x in the data, the variation is (x – overall mean)
= (x – trt. mean) + (trt. mean – overall mean) = randomness + treatment effect
30
• Total variation, SStotal : – Estimated mean : Overall sample mean
– Total variation = sum of squared differences between observations and the overall mean = Σn obsn’sx - overall mean)2 = total sum of squared = SStotal
– Degree of freedom = n – 1 • One d.f. is lost due to estimate overall mean!
32
Example.
Employee production Sample mean difference to over all mean
Wolfe 55 54 59 56 56 -2 White 66 76 67 71 70 12 Korosa 47 51 46 48 48 -10
Overall mean = 58
= (55-58)2 + …+ (56-58)2
+(66-58)2+…+(71-58)2
33
• Total variation can be decomposed into two parts : – Due to treatments, SST:
Treatment variation = sum of squared differences between the treatment mean and
the overall mean = Σn obsn’streatment mean - overall mean)2 = SST
– Degree of freedom = k – 1 • There are k treatment means and one overall mean.
– Mean square error due to treatment = MST=SST/(k-1)
34
mean difference to overall mean
Wolfe 55 54 59 56 56 -2 White 66 76 67 71 70 12 Korosa 47 51 46 48 48 -10
Example.
= (56-58)2 + …+ (56-58)2
+(70-58)2+…+(70-58)2
+ (48-58)2+…(48-58)2
d. f. = k – 1 = 3-1=2, MST=992/2= 496
4 in Wolfe group
4 in White group
4 in Korosa group
35
– Due to randomness, error, SSE : Random variation = sum of squared differences between observations and its treatment mean = Σn obsn’s ( x – treatment mean )2 = SSE
– Degree of freedom = n-k • There are n observations, k d.f. are lost due to estimating k treatment
means.
36
Brand production Sample mean difference to over
all mean Wolfe 55 54 59 56 56 -2 White 66 76 67 71 70 12 Korosa 47 51 46 48 48 -10
Overall mean = 58
= (55-56)2 + …+ (56-56)2
+(66-70)2+…+(71-70)2
d. f. = n – k = 12 – 3 = 9, MSE=90/9= 10
37
Treatment B:
Treatment C:
X, X,…,X
X, X,….,X
Example. Treatment Obns Within Treatment Between Treatment Diff (Diff)2 Diff (Diff)2
Wolfe 55 -1 1 -2 4 54 -2 4 -2 4 59 3 9 -2 4 56 0 0 -2 4
Trt mean 56 Marginal total 14 16
White 66 -4 16 12 144 76 6 36 12 144 67 -3 9 12 144 71 1 1 12 144
Korosa 47 -1 1 -10 100 51 3 9 -10 100 46 -2 4 -10 100 48 0 0 -10 100
Overall mean 58 Sum of squares SSE =14+62+14=90 SST=16+576+400=992 SS total = SSE + SST = 90+992=1082
56-58=-2
39
• Test statistic
• If H0 is true : treatment means are equal, – all treatment means the overall mean, – MST0 , F0.
• Otherwise, H0 is rejected if F-value is significantly large, i.e.
)kn/(SSE )1k/(SST
MSE MSTF
Source of variation
Sum of Squares
Degrees of Freedom
Mean Square F
Treatment SST k-1 SST/(k-1)=MST MST/MSE Error SSE n-k SSE/(n-k)=MSE Total SStotal n-1
41
Example.
90 9 10
42
Formulae
• SStotal =
• SST =
where = overall mean; = the j-th treatment mean nj = sample size in the j-th treatment group
• SSE = SStotal - SST
Example. P396
• Professor B. had students in his class rate his performance as {Excellent, Good, Fair, Poor}. The rating of each student was matched with the course grade. Is there a difference in the mean score of the student in each of the four rating categories? α=0.01.
• Rating Grade • Treatment = the rating : {E, G, F, P} • Response = course grade
44
Rating Excellent Good Fair Poor Course grades 94
90 85 80
70 73 76 78 80 68 65
68 70 72 65 74 65
gruop mean 349/4 =87.25
45
• Step 1. State hypotheses : – H0 : the mean scores are the same for the four ratings
H1 : the mean scores are not all the same.
– H1 : H0 is not true.
• Step 2. Select the 0.01 significance level
• Step 3. The ANOVA test, F=MST/MSE, is considered.
• Step 4. Since k-1=4-1=3, n-k=22-4=18, and α=0.01, the critical value F(3, 18, 0.01)=5.09
PFGE0 :H µ=µ=µ=µ
46
• Step 5. Collect data, calculate F-value and draw a conclusion.(See table 12-2) – SStotal =
– SST =
– SSE = SStotal – SST = 1485.09-890.68=594.41 – MST=SST/(k-1)=890.68/3=296.89 – MSE=SSE/(n-k)=594.41/18=33.02 – F=MST/MSE=296.89/33.02=8.99>5.09, H0 is rejected.
09.148522/)1664(344,127n/)x(x 2 n
ANOVA
594.4 18 33.0
MST=890/3=296
n-1=22-1=21
Conclusion : since F=8.99>5.09, or p-value < 0.01, H0 is rejected!
48
Inferences about pairs of treatment means
• If is rejected, how to know which two treatment means differ? – Through confidence intervals. 1. The confidence intervals for pair difference , μi-μj :
– If {0} is included in the intervals, no difference. 2. The individual confidence intervals for μ1 , …,μk :
– If “overlapping” occurs, no difference.
• Assumptions : normal, common population variances.
• The population variance is estimated by MSE, df =n-k
k210 ...:H µ==µ=µ ?0ji =µ−µ
49
The pooled variance estimate : Sp 2
D.f. of Sp 2 : n1+n2-2
– The 100(1-α)% confidence interval of μ1 –μ2 :

50
1. The confidence interval for the difference in treatment means μi-μj :
• If “0” is included in the confidence interval, we conclude no significant difference in the treatment means.
– Example.
– Since {0} is included in the interval, there is no significant difference in these two treatment means.
i j (n k, / 2) i j(x x ) t MSE(1/ n 1/ n )α−− ± +
)17,7(125
+±− α−
51
• If “0” is not included in the interval, the endpoints have the same sign, this indicates that the means differ. – Example. “Excellent vs Poor”
– Both endpoints are positive, we conclude these treatment means differ significantly. Students who rated “E” have significantly higher grades than those who rated as “P”.
)04.26,46.10()79.725.18( )6/14/1(33101.2)6925.87(
)n/1n/1(MSEt)xx( 41)2/,kn(41
2. Individual confidence interval for μ1,..,μk Page 404. MINITAB
• (*) : treatment mean • (---) : 95% confidence interval • If no overlap in the intervals, the pair of means differ. • (Excellent, Fair), (Excellent, Poor) have different means.
53
Two-way analysis of variance(Optional)
• More factors can be considered in the experiment. • If there are two treatments, a two-way ANOVA is
used for data analysis. • Example. Factors A and B.
– Experiment: • The whole population are classified by A and B. • For each combination of A and B, take a sample data. • Thus, there are a×b independent samples.
54
... population 11 population 12 population 1a
a11211 µµµ
...
...
....
....
....
.... ....
55
• Hypotheses testing: – A-effect – B-effect – A×B-interaction effect
• “Interaction effect”: – Particular effect from the combination of A and B. – For example, for A at level i, B at level j,
• “No interaction”: effect(A=i, B=j)=effect(A=i)+effect(B=j)
• “Interaction”: effect(A=i, B=j)≠effect(A=i)+effect(B=j)
56
Level 1 of factor B
Level 2 of factor B
1 2 3
Level 1and 2 of factor B
Factor A exists, factor B does not exist. No interaction.Factor A and factor B exist; no
interaction
Factor A does not exist. Factor B exists. No interaction.
Interaction
n se
n se
n se
n se
57
Two-way analysis of variance
• The total variations are decomposed into – Due to A: SSA, MSA – Due to B: SSB, MSB – Due to A*B, interaction: SSAB, MSAB – Due to random error: SSE
58
• Only when there are replications in each combination, the interaction effect can be estimated and tested. Otherwise, the interaction is assumed ignorable.
– Treatment effect is detected by comparing between- group(treatment) and within-group(error) variations.
– Interaction effect: a combination=a group
– When there is no replication in each combination, the within-group variation can’t be obtained. The interaction effect can’t be tested.
59
… … b X(b11)…,X(b1n) … X(ba1)…,X(ban)
ANOVA Table
A SSA a-1 MSA MSA/MSE B SSB b-1 MSB MSB/MSE
A*B SSAB (a-1)(b-1) MSAB MSAB/MSE Error SSE n-ab MSE Total SStotal n-1
60
A randomized block design • In order to reduce the error variation, a blocking variable can
be considered in the design. • Blocking variable :
– Recall a matched-pair design. Observations are matched by another variable, blocking variable, to reduce variation.
– When the variable is included in the ANOVA, SSE will be reduced. F(Treatment)↑ easier to get a significant result.
• Experiment: – Consider the subject as the blocking variable. b subjects in the
experiment. – Give all k treatments to every subject and record the responses. – There are b×k observations from b subjects. k dependent samples.
61
62
Data : Treatment
Block 1 … k Block mean 1 X11 … X1k 1)B(X … … b Xb1 … Xbk
b)B(X
63
Source of variation
Sum of Squares
Degrees of Freedom
Mean Square F
Treatments SST k-1 MST=SST/(k-1) MST/MSE Blocks SSB b-1 MSB=SSB/(b-1) MSB/MSE Error SSE (k-1)(b-1) MSE=SSE/(k-1)(b-1) Total SS total n-1
• Sum of squares blocks, SSB,
where = overall mean; = the b-th block mean nb = sample size in the b-th block group
n/)x(xnxnxn)xx(nSSB B
64
Example. P406
• WARTA is expanding bus service from the suburb of Starbrick into the central business district of Warren.
• Determine whether there is a difference in the mean travel times along four different routes :1) US 6; 2) West End; 3) Hickory St.; 4)Rte. 59.
• There are many different drivers, the test is set up so each driver drove along each route.
• At significance level 0.05, is there a difference in the mean travel time along the four routes if the factor “driver” is not considered ?
• If the effect of the drivers is considered, is there a difference?
65
Tt. Rte. 59 Sum Mean Sum
squares D 18 20 20 22 80 20 1608 S 21 22 24 24 91 22.75 2077 O 20 23 25 23 91 22.75 2083 Z 25 21 28 25 99 24.75 2475 F 26 24 28 25 103 25.75 2661 Sum 110 110 125 119 Total sum =464 Mean 22 22 25 23.8 Grand mean=23.2 Sum square
2466 2430 3169 2839 Total sum square=10904
Block mean
Treatment mean
Case I. One-way ANOVA : no driver effect
• Step 1. State hypotheses : – H0 : the mean times are the same for the four routes
H1 : the mean times are not all the same.
– H1 : H0 is not true.
• Step 2. Select the 0.05 significance level
• Step 3. The ANOVA test, F=MST/MSE, is considered.
• Step 4. Since k-1=4-1=3, n-k=20-4=16, and α=0.05, the critical value F(3, 16, 0.05)=3.24
43210 :H µ=µ=µ=µ
67
• Step 5. Collect data, calculate F-value and draw a conclusion.(See table 12-3) – SStotal =
– SST =
– SSE = SStotal – SST = 139.2-32.4=106.8 – MST=SST/(k-1)=32.4/3=10.8 – MSE=SSE/(n-k)=106.8/16=6.675 – F=MST/MSE=10.8/6.675=1.6178<3.24, H0 is not rejected.
2.13920/)464(10904n/)x(x 2 n
1 5 110 22 11.5
2 5 110 22 2.5
3 5 125 25 11
4 5 119 23.8 1.7
ANOVA
106.8 16 6.675
Case II. Two-way ANOVA : with driver effect
• Step 1. State hypotheses : – H0 : the mean times are the same for the four routes
H1 : the mean times are not all the same. –
H1 : H0 is not true – H0 : the mean times are the same for the five drivers
H1 : the mean times are not all the same. –
H1 : H0 is not true. • Step 2. Each select the 0.05 significance level
• Step 3. The ANOVA test, – F=MST/MSE, is considered for treatment-routes effect – F=MSB/MSE, is considered for blocking-drivers effect
FZOSD0 :H µ=µ=µ=µ=µ
43210 :H µ=µ=µ=µ
70
• Step 4. – Since k-1=4-1=3, B-1=5-1=4, – (k-1)(B-1)=3×4=12, or (n-1)-(k-1)-(B-1)=19-3-4=12 – and α=0.05, – the critical values F(3, 12, 0.05)=3.49, F(4,12, 0.05)=3.26
• Step 5. Collect data, calculate F-value and draw a conclusion.(See table 12-3) – SStotal = 139.2 – SST = 32.4, MST=10.8 – SSB =
– MSB = SSB/(B-1)=78.2/4=19.55 – SSE = SStotal – SST – SSB = 139.2-32.4-78.2=28.6 – MSE = SSE/(k-1)(B-1) = 28.6/12=2.383
2.78 )2.2375.25(4)2.2375.24(4)2.2375.22(4
71
• Step 5. calculate F-value and draw a conclusion. – For treatment effect, since
F= MST/MSE = 10.8/2.383=4.53 > 3.49 the null hypothesis for the four routes is rejected. The mean travel times are concluded different for different routes. There is a significant “route” effect.
– For blocking effect, since F= MSB/MSE = 19.55/2.383=8.20 > 3.26
the null hypothesis for the five drivers is rejected. The mean travel times are concluded different for different drivers. There is a significant “driver” effect.
72
1 4 80 20 2.666667
2 4 91 22.75 2.25
3 4 91 22.75 4.25
4 4 99 24.75 8.25
5 4 103 25.75 2.916667
1 5 110 22 11.5
2 5 110 22 2.5
3 5 125 25 11
4 5 119 23.8 1.7
ANOVA
28.6 12 2.383333
F-
– Two-way ANOVA with replicates:
Anova
• F-test : 19, 21
• One-way ANOVA : – 23, 24, 26, 31, 33 – 29 :When there is only two treatments, ANOVA and t-test result in
same conclusions. Further, t2=F – 42 : Complete an ANOVA table with knowledge of SS total, and
sample means of treatment groups. • Two-way ANOVA : 35, 38
• Bonus(1%) : using EXCEL to solve 36 and 41.
12 Analysis of Variance
F-distribution
F-distribution
One-way ANOVA : an example
Chart 12-2: when treatment effect does not exist
The ANOVA : n observations, k treatment groups
The ANOVA Test :
2. Individual confidence interval for μ1,..,μk Page 404. MINITAB
Two-way analysis of variance(Optional)
A randomized block design

Documents

12 Analysis of Variance - NCCU