The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs

The Essentials of 2-Level Design of The Essentials of 2-Level Design of ExperimentsExperiments

Part I: The Essentials of Full Factorial DesignsPart I: The Essentials of Full Factorial Designs

Developed by Don Edwards, John Grego and James Developed by Don Edwards, John Grego and James LynchLynch

Center for Reliability and Quality SciencesCenter for Reliability and Quality SciencesDepartment of StatisticsDepartment of Statistics

University of South CarolinaUniversity of South Carolina803-777-7800803-777-7800

Part I.3 The Essentials of 2-Cubed DesignsPart I.3 The Essentials of 2-Cubed Designs

MethodologyMethodology– Cube PlotsCube Plots– Estimating Main EffectsEstimating Main Effects– Estimating Interactions Estimating Interactions

(Interaction Tables and Graphs)(Interaction Tables and Graphs)

Part I.3 The Essentials of 2-Cubed DesignsPart I.3 The Essentials of 2-Cubed Designs

Statistical Significance:Statistical Significance:When is an Effect “Real”?When is an Effect “Real”?

An Example With InteractionsAn Example With Interactions A U-Do-It Case StudyA U-Do-It Case Study ReplicationReplication Rope Pull Exercise Rope Pull Exercise

Statistical SignificanceStatistical SignificanceWhen is an Effect “Real”?When is an Effect “Real”?

(As Opposed to Being “Due to Error”)(As Opposed to Being “Due to Error”)IntroductionIntroduction

The Effects (Main and Interactions) The Effects (Main and Interactions) We Compute are Really Estimates We Compute are Really Estimates of the “True Effects” (Remember of the “True Effects” (Remember MAE).MAE).

All the True Effects are Probably All the True Effects are Probably Nonzero, but Some are Very Small - Nonzero, but Some are Very Small - It is More Correct to Ask If an Effect It is More Correct to Ask If an Effect is “Distinguishable from Error” or is “Distinguishable from Error” or “Indistinguishable from Error”.“Indistinguishable from Error”.


(As Opposed to Being “Due to Error”)(As Opposed to Being “Due to Error”)IntroductionIntroduction

We will Discuss Tools to Help in We will Discuss Tools to Help in This DecisionThis Decision– Normal Probability Plots of Normal Probability Plots of

Estimated EffectsEstimated Effects– ReplicationReplication– ANOVAANOVA


Normal Probability Plots of Estimated EffectsNormal Probability Plots of Estimated Effects What if all the true effects were What if all the true effects were

zero, so that estimated effects zero, so that estimated effects represented only random error?represented only random error?

What if all the true effects were What if all the true effects were zero, so that estimated effects zero, so that estimated effects represented only random error?represented only random error?


Normal Probability Plots - BackgroundNormal Probability Plots - Background In 1959, Cuthbert Daniel Found a Way to Plot the In 1959, Cuthbert Daniel Found a Way to Plot the

Estimated Effects so that Effects Due to Random Error Estimated Effects so that Effects Due to Random Error Fall (Roughly) on a Straight Line in the PlotFall (Roughly) on a Straight Line in the Plot

To Construct a Normal Probability Plot of the EffectsTo Construct a Normal Probability Plot of the Effects– 1. Order the Estimated Effects from Smallest to Largest (Minus 1. Order the Estimated Effects from Smallest to Largest (Minus

Signs Count: -1 is Less Than 2, For Example).Signs Count: -1 is Less Than 2, For Example).

– 2. Plot the Points (E2. Plot the Points (Eii,P,Pii), i = 1,..,m on Normal Probability Paper, ), i = 1,..,m on Normal Probability Paper, Where m = Number of Effects, EWhere m = Number of Effects, Eii is the i is the ithth Smallest Effect (Put the Smallest Effect (Put the E’s on the Horizontal Axis), and PE’s on the Horizontal Axis), and Pii = 100(i-0.5)/m. = 100(i-0.5)/m.

– 3. Normal Probability Paper is on the next Slide for m = 7.3. Normal Probability Paper is on the next Slide for m = 7.

To Use This PaperTo Use This Paper– Scale the x-axis (Horizontal Axis) to Cover the Range of the Scale the x-axis (Horizontal Axis) to Cover the Range of the

EffectsEffects– Plot the Smallest Value on Line 1, the Next Smallest on Line 2, Plot the Smallest Value on Line 1, the Next Smallest on Line 2,

etc.etc.


Normal Probability Plots - Seven Effects Normal Probability Plots - Seven Effects PaperPaper

Effects

7 Effects Plot

1

2

3

4

5

6

7


Normal Probability Plots - InterpretationNormal Probability Plots - Interpretation

If There are Enough Effects Plotted, and If There are Enough Effects Plotted, and Some are Due to Random Error, These Some are Due to Random Error, These Will Lie Approximately on a Straight Will Lie Approximately on a Straight Line Centered at 0. Sketch in the Line.Line Centered at 0. Sketch in the Line.

Identify Any Effects That Fall off the Identify Any Effects That Fall off the Line to the Upper Right and Lower Left. Line to the Upper Right and Lower Left. These Effects are Probably These Effects are Probably NotNot Due to Due to Noise; They are “Distinguishable from Noise; They are “Distinguishable from Error”.Error”.


Normal Probability Plots - Example 2Normal Probability Plots - Example 2

Effects

7 Effects Plot

1

2

3

4

5

6

7

543210-1

A

Ordered Effects: -1, -.5, 0, .5, .5, 1.5, 5

Methodology Methodology Example 3 - PC Response TimeExample 3 - PC Response Time

ObjectiveObjectiveReduce Company’s PC Response TimeReduce Company’s PC Response Time

FactorsFactors– A: Cache (Two Levels Lo, Hi)A: Cache (Two Levels Lo, Hi)– B: Machine (Lo - 200MHz, 64 MB RAM,B: Machine (Lo - 200MHz, 64 MB RAM,

Hi - 400MHz, 1GB RAM Hi - 400MHz, 1GB RAM– C: Line (Lo - 56K modem, Hi - LAN)C: Line (Lo - 56K modem, Hi - LAN)


Response: PC Response TimeResponse: PC Response Time FactorsFactors

– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB RAM,B: Machine (Lo - 200MHz, 64 MB RAM,

Hi - 400MHz, 1GB RAM) Hi - 400MHz, 1GB RAM)– C: Line (Lo - 56K modem, Hi - LAN)C: Line (Lo - 56K modem, Hi - LAN)


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB RAM,B: Machine (Lo - 200MHz, 64 MB RAM,

Hi - 400MHz, 1GB RAM) Hi - 400MHz, 1GB RAM)– C: Line (Lo - 56K modem, Hi - LAN)C: Line (Lo - 56K modem, Hi - LAN)

Factors ResponseA B C yLo Lo Lo 51Hi Lo Lo 29.5Lo Hi Lo 39.8Hi Hi Lo 13.5Lo Lo Hi 25.5Hi Lo Hi 25.8Lo Hi Hi 7Hi Hi Hi 6.8

Factors ResponseA B C yLo Lo Lo 51Hi Lo Lo 29.5Lo Hi Lo 39.8Hi Hi Lo 13.5Lo Lo Hi 25.5Hi Lo Hi 25.8Lo Hi Hi 7Hi Hi Hi 6.8

C

B

A

+

+

+

_

_

_

39.8

7

25.5

29.5

25.8

6.8

13.5

51

C

B

A

+

+

+

_

_

_

39.8

7

25.5

29.5

25.8

6.8

13.5

51


Cube PlotCube Plot Response: PC Response TimeResponse: PC Response Time FactorsFactors

– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM)RAM, Hi - 400MHz, 1GB RAM)– C: Line (Lo - 56K modem,C: Line (Lo - 56K modem,

Hi - LAN) Hi - LAN)


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM)RAM, Hi - 400MHz, 1GB RAM)– C: Line (Lo - 56K modem,C: Line (Lo - 56K modem,

Hi - LAN) Hi - LAN)

MethodologyMethodology Example 3 - PC Response TimeExample 3 - PC Response Time

Estimating the Effects - Signs TablesEstimating the Effects - Signs Tables

Main Effects Interaction EffectsActual

RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1

Sum 198.9 -47.7 -64.70 -68.7 -5.3 47.9 -10.3 4.3Divisor 8 4 4 4 4 4 4 4Effect 24.9 -11.9 -16.2 -17.2 -1.3 12 -2.6 1


RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1


Methodology Methodology Example 3 - PC Response TimeExample 3 - PC Response TimeEffects Normal Probability PlotEffects Normal Probability Plot

100-10-20

1.5

0.5

-0.5

-1.5

Effects

C

B

A

ACNormal Scores

Ordered Effects: -17.2, -16.2, -11.9, -2.6, -1.3, 1, 12

100-10-20

1.5

0.5

-0.5

-1.5

Effects

C

B

A

ACNormal Scores

Ordered Effects: -17.2, -16.2, -11.9, -2.6, -1.3, 1, 12

MethodologyMethodologyInteraction Tables and GraphsInteraction Tables and Graphs

Tools for Aiding Interpretation of Tools for Aiding Interpretation of SIGNIFICANT Two-Way InteractionsSIGNIFICANT Two-Way Interactions

At the Right is a Blank AB At the Right is a Blank AB Interaction TableInteraction Table

In the Table, 1 Corresponds to the In the Table, 1 Corresponds to the Lo Level and 2 to the Hi LevelLo Level and 2 to the Hi Level

Tools for Aiding Interpretation of Tools for Aiding Interpretation of SIGNIFICANT Two-Way InteractionsSIGNIFICANT Two-Way Interactions

At the Right is a Blank AB At the Right is a Blank AB Interaction TableInteraction Table

In the Table, 1 Corresponds to the In the Table, 1 Corresponds to the Lo Level and 2 to the Hi LevelLo Level and 2 to the Hi Level

B: -1 1

A:

-1

A-1B-1C-1 A-1B1C-1

A-1B-1C1 A-1B1C1

A-1B-1 A-1B1

1

A1B-1C-1 A1B1C-1

A1B-1C1 A1B1C1

A1B-1 A1B1


AC Interaction TableAC Interaction Table

Timey

CacheA

LineC

51 -1 -129.5 1 -139.8 -1 -113.5 1 -125.5 -1 125.8 1 1

7 -1 16.8 1 1

Timey

CacheA

LineC

51 -1 -129.5 1 -139.8 -1 -113.5 1 -125.5 -1 125.8 1 1

7 -1 16.8 1 1

C: Line

-1 1

A: Cache

-1

51.0 25.5

39.8 7.0

90.8 32.5

A-1C-1 = 45.4 A-1C1 = 16.25

1

29.5 25.8

13.5 6.8

43.0 32.6

A1C-1 = 21.5 A-1C1 = 16.3

Methodology Methodology

Interaction Tables and GraphsInteraction Tables and Graphs Interaction Plots - ConstructionInteraction Plots - Construction

1. For a Given Pair of Factors (Say A 1. For a Given Pair of Factors (Say A and B) Find the Average Response at and B) Find the Average Response at Each of Their Four Level Combinations.Each of Their Four Level Combinations.

2. Plot These with Response on the 2. Plot These with Response on the Vertical Axis, Using One of the Factor’s Vertical Axis, Using One of the Factor’s Levels (Say B) on the Horizontal Axis. Levels (Say B) on the Horizontal Axis. Connect and Label the Averages with Connect and Label the Averages with the Same Level of the Other Factor (A).the Same Level of the Other Factor (A).

MethodologyMethodology

Interaction Tables and GraphsInteraction Tables and Graphs Interaction Plots - InterpretationInteraction Plots - Interpretation

1. If the Lines are 1. If the Lines are RoughlyRoughly Parallel, Parallel, There is No Strong Interaction.There is No Strong Interaction.

2. If There is Interaction, the Plot Shows 2. If There is Interaction, the Plot Shows Clearly the Effect of a Factor at Each of Clearly the Effect of a Factor at Each of the Levels of the Other Factor.the Levels of the Other Factor.

3. Maximizing and Minimizing 3. Maximizing and Minimizing Combinations of the Factors are Easily Combinations of the Factors are Easily Identified on the Plot and in the Table.Identified on the Plot and in the Table.

MethodologyMethodologyExample 3Example 3

AC Interaction Table and AC Interaction Table and GraphGraph


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM)RAM, Hi - 400MHz, 1GB RAM)– C: LineC: Line

(Lo - 56K modem, Hi - (Lo - 56K modem, Hi - LAN)LAN)


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB – RAM, Hi - 400MHz, 1GB RAM)RAM, Hi - 400MHz, 1GB RAM)– C: LineC: Line

(Lo - 56K modem, Hi - (Lo - 56K modem, Hi - LAN)LAN)

-1 1-1 1

1 1-1-1

45

35

25

15

C

AAC Interaction Graph

y=24.9

- -

-1 1-1 1

1 1-1-1

45

35

25

15

C


y=24.9

- -

C: Line

-1 1

A: Cache

-1

51.0 25.5

39.8 7.0

90.8 32.5

A-1C-1 = 45.4 A-1C1 = 16.25

1

29.5 25.8

13.5 6.8

43.0 32.6

A1C-1 = 21.5 A-1C1 = 16.3

MethodologyMethodologyExample 3 - AC Interaction GraphExample 3 - AC Interaction Graph

-1 1-1 1

1 1-1-1

45

35

25

15

C


y=24.9

- -

-1 1-1 1

1 1-1-1

45

35

25

15

C


y=24.9

- -


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB

RAM, Hi - 400MHz, 1GB RAM) RAM, Hi - 400MHz, 1GB RAM)

– C: LineC: Line(Lo - 56K modem, Hi - (Lo - 56K modem, Hi -

LAN)LAN)


– A: Cache (Two Levels Lo,Hi)A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz, 64 MB B: Machine (Lo - 200MHz, 64 MB

RAM, Hi - 400MHz, 1GB RAM) RAM, Hi - 400MHz, 1GB RAM)

– C: LineC: Line(Lo - 56K modem, Hi - (Lo - 56K modem, Hi -

LAN)LAN)

MethodologyMethodologyExample 3 - AC Interaction Example 3 - AC Interaction

InterpretationInterpretationNoise Factors versus Control Noise Factors versus Control

FactorsFactors

To minimize the response, choose B Hi and C To minimize the response, choose B Hi and C Hi. When C is Hi, the effect of A is negligible. Hi. When C is Hi, the effect of A is negligible.

Refer back to the cube plot—the (B Hi, C Hi) Refer back to the cube plot—the (B Hi, C Hi) edge had the two lowest readings. Our edge had the two lowest readings. Our analysis shows that this was not due to a BC analysis shows that this was not due to a BC interaction, but to a significant B main interaction, but to a significant B main effect and the particular form of the effect and the particular form of the significant AC interaction.significant AC interaction.

Response: PC Response Time

Factors– A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz,

64 MB RAM, Hi - 400MHz, 1GB RAM)

– C: Line(Lo - 56K modem,

Hi - LAN)

Response: PC Response Time

Factors– A: Cache (Two Levels Lo,Hi)– B: Machine (Lo - 200MHz,

64 MB RAM, Hi - 400MHz, 1GB RAM)

– C: Line(Lo - 56K modem,

Hi - LAN)

MethodologyMethodology Example 3 - Estimating the Mean Response: A = +1, B Example 3 - Estimating the Mean Response: A = +1, B

= -1, C = +1= -1, C = +1 Estimated Mean ResponseEstimated Mean Response

(EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B)(EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2

For A = +1, B = -1, C = +1, EMR = 24.9 + [(+1)(-11.9)+(-1)(-16.2)+(+1)(-17.2)+(1)For A = +1, B = -1, C = +1, EMR = 24.9 + [(+1)(-11.9)+(-1)(-16.2)+(+1)(-17.2)+(1)(12)]/2 = 24.4(12)]/2 = 24.4

Notice that for A = +1 and C = +1, Notice that for A = +1 and C = +1, [(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2[(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2

= [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = -8.55 = A= [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = -8.55 = A22CC22 – y; so – y; so EMR=24.9-EMR=24.9- 8.55=16.358.55=16.35

Estimated Mean ResponseEstimated Mean Response(EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B)(EMR) = y + [(Sign of A)(Effect of A)+(Sign of B)(Effect of B)

+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2 For A = +1, B = -1, C = +1, EMR = 24.9 + [(+1)(-11.9)+(-1)(-16.2)+(+1)(-17.2)+(1)For A = +1, B = -1, C = +1, EMR = 24.9 + [(+1)(-11.9)+(-1)(-16.2)+(+1)(-17.2)+(1)

(12)]/2 = 24.4(12)]/2 = 24.4 Notice that for A = +1 and C = +1, Notice that for A = +1 and C = +1,

[(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2[(Sign of A)(Effect of A)+(Sign of C)(Effect of C)+(Sign of AC)(Effect of AC)]/2= [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = -8.55 = A= [(+1)(-11.9)+(+1)(-17.2)+(1)(12)]/2 = -17.1/2 = -8.55 = A22CC22 – y; so – y; so

EMR=24.9-EMR=24.9- 8.55=16.358.55=16.35


RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1



RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1


For Calculating EMR Include:

– Significant Main Effects

– Significant Interactions, and All Their Lower Order Interactions and Main Effects

For Calculating EMR Include:

– Significant Main Effects

– Significant Interactions, and All Their Lower Order Interactions and Main Effects

MethodologyMethodologyU-Do-ItU-Do-It: : Example 3 - Estimate the ResponseExample 3 - Estimate the Response

A = +1, B = +1, C = +1 and A = +1, B = +1, C = A = +1, B = +1, C = +1 and A = +1, B = +1, C = -1-1


RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1



RunTime

yCache

AMachine

BLine

C AB AC BC ABC5 51 -1 -1 -1 1 1 1 -12 29.5 1 -1 -1 -1 -1 1 11 39.8 -1 1 -1 -1 1 -1 14 13.5 1 1 -1 1 -1 -1 -13 25.5 -1 -1 1 1 -1 -1 16 25.8 1 -1 1 -1 1 -1 -18 7 -1 1 1 -1 -1 1 -17 6.8 1 1 1 1 1 1 1


Documents

The Essentials of 2-Level Design of Experiments Part I: The Essentials of Full Factorial Designs