Upload
carnig
View
39
Download
3
Embed Size (px)
DESCRIPTION
Experimental Statistics - week 7. Chapter 15: Factorial Models (15.5). Chapter 17: Random Effects Models. Model for 2-factor Design. where. Hypothetical Cell Means. Auditory. Visual. 5 10 15. Auditory. Visual. - PowerPoint PPT Presentation
Citation preview
1
Experimental StatisticsExperimental Statistics - week 7 - week 7Experimental StatisticsExperimental Statistics - week 7 - week 7
Chapter 15:
Factorial Models (15.5)
Chapter 17:
Random Effects Models
2
Model for 2-factor Design
ijk i j ij ijky
1 1 1 1
0a b a b
i j ij iji j i j
where
3
Auditory
Visual
5 10 15
Hypothetical Cell MeansHypothetical Cell Means
Auditory
Visual
5 10 15
4
2 2... .. ...
1 1 1 1
2. . ...
1
2. .. . . ...
1 1
2...
1
( ) ( )
( )
( )
( )
a b n a
ijk ii j k i
b
jj
a b
ij i ji j
ijkk
y y bn y y
an y y
n y y y y
y y
1 1
a b n
i j
Sum-of-Squares Breakdown
(2-factor ANOVA)
SSA
SSB
SSAB
SSE
5
2-Factor ANOVA Table(2-Factor Completely Randomized Design)
Source SS df MS F
Main Effects
A SSA a 1
B SSB b1
Interaction
AB SSAB (a 1)(b1)
Error SSE ab(n 1) Total TSS abn
/( 1)MSB SSB b
/ ( 1)MSE SSE ab n
/MSA MSE
(page 900)
/( 1)( 1)MSAB SSAB a b
/MSB MSE
/( 1)MSA SSA a
/MSAB MSE
6
0
( 1, ( 1))
H
MSBF F b ab n
MSE
Reject at level if
0 1 2: 0
: 0
a
a i
H
H
at least one
Hypotheses:
Main Effects:
0 1 2: 0
: 0
b
a j
H
H
at least one
0
( 1, ( 1))
H
MSAF F a ab n
MSE
Reject at level if
0
(( 1)( 1), ( 1))
H
MSABF F a b ab n
MSE
Reject at level if
Interactions:
0 11 12: 0
: 0
ab
a ij
H
H
at least one
7
STIMULUS EXAMPLE:
Personal computer presents stimulus, and person responds.
Study of how RESPONSE TIME is effected by a WARNING given prior to the stimulus:
2-factors of interest:
Warning Type --- auditory or visual
Time between warning and stimulus -- 5 sec, 10 sec, or 15 sec.
8
.204 .257
.170 .279
.181 .269
.167 .283
.182 .235
.187 .260
.202 .256
.198 .281
.236 .258
Auditory Visual
5 sec
10 sec
15 sec
WarningTime
9
Stimulus Data The GLM Procedure Dependent Variable: response Sum of Source DF Squares Mean Square F Value Pr > F Model 5 0.02554894 0.00510979 17.66 <.0001 Error 12 0.00347200 0.00028933 Corrected Total 17 0.02902094 R-Square Coeff Var Root MSE response Mean 0.880362 7.458622 0.017010 0.228056 Source DF Type I SS Mean Square F Value Pr > F type 1 0.02354450 0.02354450 81.38 <.0001 time 2 0.00115811 0.00057906 2.00 0.1778
type*time 2 0.00084633 0.00042317 1.46 0.2701
GLM Output
10
Testing ProcedureTesting Procedure2 factor CRD Design
Step 1. Test for interaction.
Step 2.(a) IF there IS NOT a significant interaction - test the main effects
(b) IF there IS a significant interaction - compare cell means
11
Stimulus Example
1.46MSAB
FMSE
Test for Interaction:
.2701P
Therefore we DO NOT reject the null hypothesis of no interaction.
12
Stimulus Data
13
Stimulus Example
1.46MSAB
FMSE
Test for Interaction:
.2701P
Therefore we DO NOT reject the null hypothesis of no interaction.
Thus - based on the testing procedure, we next test for main effects.
1 2 22
( )α/MSE
y y tN
| |
Testing Main Effects:Testing Main Effects:
For each main effect (i.e. A and B)
0H- test
0H- if is rejected, compare marginal means
using an appropriate procedure (eg. LSD or BON)
Note: I’ll use LSD from this point on unless otherwise noted.
1 2(y y2 marginal means and ) are declared
to be significantly different (using LSD) ifIn General:
where N denotes the # of observations involved in the computation of a marginal mean.
15
.204 .257
.170 .279
.181 .269
.167 .283
.182 .235
.187 .260
.202 .256
.198 .281
.236 .258
Auditory Visual
5 sec
10 sec
15 sec
WarningTime
1..y .192 2..y .264
.1.y .227
.2.y .219
.3.y .239
16
Stimulus Example
81.38MSA
FMSE
Test for Main Effects:
Thus, there is a significant effect due to type but not time
A (type): .0001P
B (time): 2.00MSB
FMSE
.1778P
- i.e. we can use LSD to compare marginal means for type
- we will do this here for illustration although MC not needed when there are only 2 groups
17
The GLM Procedure
t Tests (LSD) for response NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 12 Error Mean Square 0.000289 Critical Value of t 2.17881 Least Significant Difference 0.0175 Means with the same letter are not significantly different. t Grouping Mean N type A 0.264222 9 V B 0.191889 9 A
GLM Output -- Comparing “Types”
18
The GLM Procedure
t Tests (LSD) for response NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 12 Error Mean Square 0.000289 Critical Value of t 2.17881 Least Significant Difference 0.0214 Means with the same letter are not significantly different. t Grouping Mean N time A 0.238500 6 15 A A 0.226667 6 5 A A 0.219000 6 10
GLM Output -- Comparing “Times”
19
Pilot Plant Data
Variable = Chemical Yield
Factors: A – Temperature (160, 180) B – Catalyst (C1 , C2)
59 74 61 70 50 69 58 67
50 81 54 85 46 79 44 81
o o160 180
C1
C2
Catalyst
Temperature
20
21
22
Pilot Plant -- Probability Plot of Residuals
23
DATA one;INPUT temp catalyst$ yield;datalines;160 C1 59160 C1 61 . . .180 C2 79180 C2 81;PROC GLM; class temp catalyst; MODEL yield=temp catalyst temp*catalyst; Title 'Pilot Plant Example -- 2-way ANOVA'; MEANS temp catalyst/LSD; RUN;PROC SORT;BY temp catalyst;PROC MEANS; BY temp catalyst; OUTPUT OUT=cells MEAN=yield;RUN;
24
Pilot Plant Example -- 2-way ANOVA General Linear Models Procedure Dependent Variable: YIELD Sum of MeanSource DF Squares Square F Value Pr > F Model 3 2525.0000000 841.6666667 58.05 0.0001 Error 12 174.0000000 14.5000000 Corrected Total 15 2699.0000000 R-Square C.V. Root MSE YIELD Mean 0.935532 5.926672 3.8078866 64.250000 Source DF Type I SS Mean Square F Value Pr > F TEMP 1 2116.0000000 2116.0000000 145.93 0.0001CATALYST 1 9.0000000 9.0000000 0.62 0.4461TEMP*CATALYST 1 400.0000000 400.0000000 27.59 0.0002
Pilot Plant -- GLM Output
25
RECALL: RECALL: Testing ProcedureTesting Procedure 2 factor CRD Design
Step 1. Test for interaction.
Step 2.(a) IF there IS NOT a significant interaction - test the main effects
(b) IF there IS a significant interaction - compare cell means
26
Pilot Plant Example
27.59MSAB
FMSE
Test for Interaction:
.0002P
Therefore we reject the null hypothesis of no interaction - and conclude that there is an interaction between temperature and catalyst.
Thus, we DO NOT test main effects
27
28
29
Since there is a significant interaction, we do not test for main effects!
- instead compare “Cell Means”
- NOTE: interaction plot is a plot of the cell means
30
Pilot Plant Data
Variable = Chemical Yield
Factors: A – Temperature (160, 180) B – Catalyst (C1 , C2)
59 74 61 70 50 69 58 67
50 81 54 85 46 79 44 81
o o160 180
C1
C2
Catalyst
Temperature
31
Pilot Plant Data -- cell means
57.0 70.0 48.5 81.5
o o160 180
C1
C2
Catalyst
Temperature
1 2 22
( )α/MSE
y y tN
| |
Comparing Cell Means:Comparing Cell Means:
If there is significant interaction, then we compare the a x b cell means using the criteria below.
1 2(y y2 cell means and ) are declared
to be significantly different (using LSD) if
Procedure similar to that for comparing marginal means:
where N denotes the # of observations involved in the computation of a cell mean.
33
The GLM Procedure
t Tests (LSD) for yield NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 12 Error Mean Square 14.5 Critical Value of t 2.17881 Least Significant Difference 4.1483 Means with the same letter are not significantly different. t Grouping Mean N temp A 75.750 8 180 B 52.750 8 160
GLM Output -- Comparing “Temps”
- disregard
34
The GLM Procedure t Tests (LSD) for yield
NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 12 Error Mean Square 14.5 Critical Value of t 2.17881 Least Significant Difference 4.1483 Means with the same letter are not significantly different. t Grouping Mean N catalyst A 65.000 8 C2 A A 63.500 8 C1
GLM Output -- Comparing “Catalysts”
- disregard
35
Note:
- SAS does not provide a comparison of cell means
36
NOTE:
I will be out of the office tomorrow.
37
Testing Procedure RevistedTesting Procedure Revisted2 factor CRD Design
Step 1. Test for interaction.
Step 2.(a) IF there IS NOT a significant interaction - test the main effects
(b) IF there IS a significant interaction - compare a x b cell means (by hand)
Main Idea:
We are trying to determine whether the factors effect the response either individually or collectively.
38
Statistics 5372: Experimental StatisticsAssignment Report Form
Name:Lecture Assigned: Data Set or Problem Description Key Results of the Analysis
Conclusions in the Language of the Problem Appendices:
A. Tables and Figures Cited in the Report B. SAS Log from the Final SAS Run Notes:
1. All assignments should be typed using a word processor according to the format above. 2. SAS output should consist only of tables and figures cited in the report. The report should refer to these tables and figures using numbers you assign, i.e. Table 1, etc. 3. The data should be listed somewhere in the report. (within SAS code is ok)
39
Homework Assignment
Due March 1, 2005
15.41, page 935
In this problem the authors consider two measures of the stability of a drug: MG/ML and pH. They ran a 2-factor ANOVA for each of these response variables using storage time and laboratory used in the analysis as the classification variables. There are 4 storage times considered and 2 labs. The data are in the table on page 935 and the resulting 2-factor ANOVA tables are shown on 935-936.
Using SAS, reproduce the ANOVA tables given in the book, and complete an assignment report form for the two analyses.
40
.204 .257
.170 .279
.181 .269
.167 .283
.182 .235
.187 .260
.202 .256
.198 .281
.236 .258
Auditory Visual
5 sec
10 sec
15 sec
WarningTime
1..y .192 2..y .264
.1.y .227
.2.y .219
.3.y .239
.. .228y
41
Note: For balanced designs,
..y
average of all data values
average of marginal row means
average of marginal column means
i.e. for STIMULUS data
.228 = (.227+.219+.239)/3
= (.192+.264)/2
42
.204 .257
.170 .279
.181 .269
.167 .283
.182 .235
.187 .260
.202 .256
.198 .281
.236 .258
Auditory Visual
5 sec
10 sec
15 sec
WarningTime
1..y .190 2..y .264
.1.y .231
.2.y .219
.3.y .239
.. ???y
Now Consider:
In this case, average of marginal row means
average of marginal col. means
43
• Every Combination of the Factor Levels has an Equal Number of Repeats
• Sums of Squares– Uniquely Calculated
» Usual Textbook Formulas
• Not Every Combination of the Factor Levels has an Equal Number of Repeats
• Sums of Squares– Not Uniquely Calculated
» Usual Textbook Formulas Are Not Valid
Balanced Experimental DesignsBalanced Experimental Designs
Unbalanced Experimental DesignsUnbalanced Experimental Designs
44
- they typically use “Textbook Formulas”
Many Software Programs Cannot Properly Calculate Sums of Squares for Unbalanced Designs
SAS:
- use Type III sums of squares
-- analysis is closest to that for “Balanced Experiments”
Unbalanced Experimental DesignsUnbalanced Experimental Designs
- must Use Proc GLM, not Proc ANOVA
- Type I and Type III sums-of-squares results will not generally agree
45
The GLM Procedure Dependent Variable: response Sum of Source DF Squares Mean Square F Value Pr > F Model 5 0.02547774 0.00509555 19.13 <.0001 Error 11 0.00293050 0.00026641 Corrected Total 16 0.02840824 R-Square Coeff Var Root MSE response Mean 0.896843 7.112913 0.016322 0.229471 Source DF Type I SS Mean Square F Value Pr > F type 1 0.02309680 0.02309680 86.70 <.0001 time 2 0.00122742 0.00061371 2.30 0.1460 type*time 2 0.00115351 0.00057676 2.16 0.1611 Source DF Type III SS Mean Square F Value Pr > F type 1 0.02367796 0.02367796 88.88 <.0001 time 2 0.00130085 0.00065042 2.44 0.1326 type*time 2 0.00115351 0.00057676 2.16 0.1611
Unbalanced Data -- GLM Output
46
Model for 3-factor Factorial Design
ijkm i j k
ij ik jk
ijk
ijkm
y
1 1 1
0a b c
i j ki j k
where
and also, the sum over any subscript of a 2 or 3 factor interaction is zero
47
2....
1 1 1 1
( )a b c n
ijkmi j k k
y y
= SSA + SSB + SSC
+ SSAB + SSAC + SSBC + SSABC + SSE
Sum-of-Squares Breakdown
(3-factor ANOVA)
48
3-Factor ANOVA Table(3-Factor Completely Randomized Design)
Source SS df MS F
Main Effects
A SSA a 1 B SSB b 1C SSC c1Interactions
AB SSAB (a 1)(b1)AC SSAC (a 1)(c1)BC SSBC (b 1)(c1)ABC SSABC (a 1)(b1)(c1)
Error SSE abc(n 1) Total TSS abcn
/( 1)MSB SSB b
/ ( 1)MSE SSE ab n
/MSA MSE
See page 908
/( 1)( 1)MSAB SSAB a b
/MSB MSE/( 1)MSA SSA a
/MSAB MSE
/( 1)MSC SSC c /MSC MSE
/( 1)( 1)MSBC SSBC b c /( 1)( 1)MSAC SSAC a c
/( 1)( 1)( 1)MSABC SSAB a b c
/MSAC MSE/MSBC MSE/MSABC MSE
49
Popcorn Data
Factors(A) Brand (3 brands)
(B) Power of Microwave (500, 600 watts)
(C) 4, 4.5 minutes
n = 2 replications per cell
Response variable -- % of kernels that popped
50
Popcorn Data
1 500 4.5 70.31 500 4.5 91.01 500 4 72.71 500 4 81.91 600 4.5 78.71 600 4.5 88.71 600 4 74.11 600 4 72.12 500 4.5 93.42 500 4.5 76.32 500 4 45.32 500 4 47.62 600 4.5 92.22 600 4.5 84.72 600 4 66.32 600 4 45.73 500 4.5 50.13 500 4.5 81.53 500 4 51.43 500 4 67.73 600 4.5 71.53 600 4.5 80.03 600 4 64.03 600 4 77.0
51
PROC GLM;class brand power time;MODEL percent=brand power time brand*power brand*time power*time brand*power*time;Title 'Popcorn Example -- 3-Factor ANOVA';MEANS brand power time/LSD; RUN;
SAS GLM Code – 3 Factor Model
MODEL percent=brand power time brand*power brand*time power*time brand*power*time
The Statement
can be written as
MODEL percent=brand | power | time;
52
The GLM ProcedureDependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model 11 3589.988333 326.362576 2.71 0.0503 Error 12 1444.170000 120.347500 Corrected Total 23 5034.158333
R-Square Coeff Var Root MSE percent Mean 0.713126 15.27011 10.97030 71.84167
Source DF Type I SS Mean Square F Value Pr > F brand 2 566.690833 283.345417 2.35 0.1372 power 1 180.401667 180.401667 1.50 0.2443 time 1 1545.615000 1545.615000 12.84 0.0038 brand*power 2 125.125833 62.562917 0.52 0.6074 brand*time 2 1127.672500 563.836250 4.69 0.0314 power*time 1 0.015000 0.015000 0.00 0.9913 brand*power*time 2 44.467500 22.233750 0.18 0.8336
53
Testing ProcedureTesting Procedure 3 factor CRD Design
Step 1. Test for 3rd order interaction.
IF there IS a significant 3rd order interaction - compare cell means
IF there IS NOT a significant 3rd order interaction - test 2nd order interactions
IF there IS NOT a sig. 2nd order interaction - test the main effects
IF there IS a significant 2rd order interaction - compare associated cell means
In general -- test main effects only for variables not involved in a significant 2nd or 3rd order interaction
54
Examine brand x time cell meansExamine Power main effect
The GLM ProcedureDependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model 11 3589.988333 326.362576 2.71 0.0503 Error 12 1444.170000 120.347500 Corrected Total 23 5034.158333
R-Square Coeff Var Root MSE percent Mean 0.713126 15.27011 10.97030 71.84167
Source DF Type I SS Mean Square F Value Pr > F brand 2 566.690833 283.345417 2.35 0.1372 power 1 180.401667 180.401667 1.50 0.2443 time 1 1545.615000 1545.615000 12.84 0.0038 brand*power 2 125.125833 62.562917 0.52 0.6074 brand*time 2 1127.672500 563.836250 4.69 0.0314 power*time 1 0.015000 0.015000 0.00 0.9913 brand*power*time 2 44.467500 22.233750 0.18 0.8336
55
56
To complete the analysis:1. The F-test for Power was not significant (.2443)
2. Compare the 6 cell means plotted in interaction plot using procedure analogous to the one used for pilot plant data.
PROC SORT data=one;BY brand time; PROC MEANS mean std data=one;BY brand time; OUTPUT OUT=cells MEAN=percent; Title 'Brand x Time Cell Means for Popcorn Data'; RUN;
Obs brand time _TYPE_ _FREQ_ percent 1 1 4 0 4 75.200 2 1 4.5 0 4 82.175 3 2 4 0 4 51.225 4 2 4.5 0 4 86.650 5 3 4 0 4 65.025 6 3 4.5 0 4 70.775
Conclusions:
57
Models with Random Effects
Fixed-Effects Models -- the models we’ve studied to this point -- factor levels have been specifically selected - investigator is interested in testing effects of these specific levels on the response variable
Examples: -- CAR data - interested in performance of these 5 gasolines
-- Pilot Plant data - interested in the specific temperatures (160o and 180o) and catalysts (C1 and C2)
58
Random-Effect Factor -- the factor has a large number of possible levels
-- the levels used in the analysis are a random sample from the population of all possible levels
- investigator wants to draw conclusions about the population from which these levels were chosen
(not the specific levels themselves)
59
Fixed Effects vs Random Effects
This determination affects
- the model
- the hypothesis tested
- the conclusions drawn
- the F-tests involved (sometimes)
60
1-Factor Random Effects Model
ij i ijy
Assumptions:
'i s3. are independent.
4. 'ij s 2 normally distributed with mean 0 and variance
ii2. is th random observation on factor A
2-- normally distributed with mean 0 and variance
'ij s5. are independent.
i ij 6. The r.v.'s and are independent.
1. is overall mean
61
Hypotheses:
Ho:
Ha:
Ho says (considering the variability of the yij’s) :
- the component of the variance due to “Factor” has zero variance
-- i.e. no factor-to-factor variation
- all of the variability observed is just unexplained subject-to-subject variation
62
DATA one;INPUT operator output;DATALINES;1 175.41 171.71 173.01 170.52 168.52 162.72 165.02 164.13 170.13 173.43 175.73 170.74 175.24 175.74 180.14 183.7;PROC GLM; CLASS operator; MODEL output=operator; RANDOM operator; TITLE ‘Operator Data: One Factor Random Effects Model';RUN;
These are data from an experiment studying the effect of four operators (chosen randomly) on the output of a particular machine.
63
The GLM Procedure Dependent Variable: output Sum of Source DF Squares Mean Square F Value Pr > F Model 3 371.8718750 123.9572917 14.91 0.0002 Error 12 99.7925000 8.3160417 Corrected Total 15 471.6643750
R-Square Coeff Var Root MSE output Mean 0.788425 1.674472 2.883755 172.2188
Source DF Type I SS Mean Square F Value Pr > F operator 3 371.8718750 123.9572917 14.91 0.0002 The GLM Procedure Source Type III Expected Mean Square operator Var(Error) + 4 Var(operator)
One Factor Random effects Model
64
We reject Ho : (p = .0002)
and we conclude that there isvariability due to operator
Conclusion:
Note:Multiple comparisons are not used in random effects analyses
-- we are interested in whether there is variability due to operator
- not interested in which operators performed better, etc. (they were randomly chosen)
65
Note:
- The same F test is used to test Ho for the 1-factor Fixed Effects and 1-factor Random effects models are the same
- Interpretations differ