39
A Quick-Look A Quick-Look Design-of-Experiments (DOE) Design-of-Experiments (DOE) Orientation Orientation Carol Ventresca Carol Ventresca Carol @SynGenics.com Carol @SynGenics.com John C. Sparks John C. Sparks [email protected] [email protected] © 2008 SynGenics Corporation. All rights reserved.

A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks [email protected] © 2008 SynGenics Corporation

Embed Size (px)

Citation preview

Page 1: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

A Quick-LookA Quick-LookDesign-of-Experiments (DOE)Design-of-Experiments (DOE)

OrientationOrientation

Carol Ventresca Carol Ventresca Carol @SynGenics.comCarol @SynGenics.com

John C. SparksJohn C. [email protected]@SynGenics.com

© 2008 SynGenics Corporation. All rights reserved.

Page 2: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 2

Presentation Topics

Introductory Experiments with a Black Box Using One-Factor-at-a-Time Methodology Using an Orthogonal Array via a “Designed Experiment”

What Exactly is DOE? History When Applicable

The Classic “Dial Problem” Air Force Example: Vane Cleaning Experiment Summary and Resources

Page 3: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 3

Experiments with a Black Box

Controllable inputs:X variables

y1

Outputs:Y Variables

Objective:Determine Y = F(X)In the presence of Z

x1 x2 x3 x4 x5

z1 z2 z3 z4 z5

y2

y3

y4

Uncontrollable inputs:Z variables

Standard DOE nomenclature for black box experimentation

Page 4: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 4

Goal of Testing and AssociatedTest Program Options

Via experimentation, engineer must assess the response of a system as a function of several variables or factors Each factor has at least two different operating levels Any change to any one factor necessitates an additional test

Traditional Options Full Factorial (FF): Solid option, but quickly discarded with the ballooning of

factor/level combinations Example: FF for six two-level factors necessitates 26 = 64 individual tests in order

to capture all factor/level combinations Engineering Judgment: Normally a poor option since this approach by nature

allows random pursuit of rabbit trails Leads to a situation known as the “random test matrix”

One-Factor-at-a-Time (OFT): Poor option, process attempts to optimize in serial fashion with no regard to synergistic or “interactive” combinations

Once an individual factor comes up for optimization and has its level fixed, all other levels of the same factor are disregarded for the remainder of experimentation

Hence, interaction effects between factor levels are never fully assessed

Page 5: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 5

OFT Test Program Applied to a Specific Black Box

Unknown FunctionalMechanism:

Y1 = f(x1,x2,x3,x4)

y1

Inputs

Output

x1

x2

x3

x4

Goals:1) Use OFT test methods to maximize the output y1

2) Discover the operating characteristics of the black box in terms of an algebraic equation relating cause to effect

Page 6: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 6

Actual Function HiddenWithin the Black Box

y1

Inputs

Output

x1

x2

x3

x4

y1=45+12x1+8x2+10x1x2+5x3-2x1x3-6x1x4+x4

Nature of which is TBD

Page 7: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 7

OFT Factors and Factor Levels(Chosen for Illustration Purposes Only)

Factor Lo Hi

x1 -1 1

x2 -1 1

x3 -1 1

x4 -1 1

Note: A FF test program would consists of 24 = 16 individual tests.

x1 x2 x3 x4

-1, 1, -1, 1

1

-1

1

-1

11

-1-1

-1, -1, -1, -1

1, 1, 1, 1

Page 8: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 8

“Straight-to-the-Chase” with aFive-Run OFT Test Program

Run: Comment x1 x2 x3 x4 Run Code y1

1: Baseline -1 -1 -1 -1 (1) 21

2: x1 lockdown 1 -1 -1 -1 x1 51

3: x2 lockdown 1 1 -1 -1 x1x2 77

4: x3 lockdown 1 1 1 -1 x1x2x3 83

5: x4 off the optimum 1 1 1 1 x1x2x3x4 73

6: Final Lockdown 1 1 1 -1 x1x2x3 83

Page 9: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 9

OFT Model Building withFive Data Points

Start with an assumed fully determined linear model

y1 = a0 + a1x1 + a2x2 + a3x3 + a4x4

Where

a0 – a1 – a2 – a3 – a4 = 21a0 + a1 – a2 – a3 – a4 = 51a0 + a1 + a2 – a3 – a4 = 77a0 + a1 + a2 + a3 – a4 = 83a0 + a1 + a2 + a3 + a4 = 73

Solving for the five unknown coefficients

y1 = 47 + 15x1 + 13x2 + 3x3 – 5x4

Optimizing

y1 = 47 + 15(1) + 13(1) + 3(1) – 5(-1) =83

Page 10: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 10

The OFT Model Fails to Predict For Many Combinations

True: y1=45+12x1+8x2+10x1x2+5x3-2x1x3-6x1x4+x4

OFT Model: y1 = 47 + 15x1 + 13x2 + 3x3 – 5x4

Combination OFT True Combination OFT True

-1, -1, -1, -1 21 21 1, -1, -1, -1 51 51

-1, -1, -1, 1 11 35 1, -1, -1, 1 41 31

-1, -1, 1, -1 27 35 1, -1, 1, -1 53 47

-1, -1, 1, 1 17 49 1, -1, 1, 1 43 37

-1, 1, -1, -1 47 17 1, 1, -1, -1 77 77

-1, 1, -1, 1 37 31 1, 1, -1, 1 67 67

-1, 1, 1, -1 53 31 1, 1, 1, -1 83 83

-1, 1, 1, 1 43 45 1, 1, 1, 1 73 73

Page 11: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 11

OFT Advantages and Disadvantages

Advantages Search process locates the maximum value Similar search process locates the minimum value Does so in five runs

Disadvantages Leads to wrong functional model

Factor main effects only; no interactions Poor overall prediction capability

Example OFT predicts six settings out of sixteen Blue is fortuitous Due to unaccounted-for interactions

Page 12: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 12

An OFT Test Program thatFails to Identify the Maximum

y1 = 7 + 2x1 – 3x2 + x3 + 2x1x2 – 4x2x3

Run: Comment x1 x2 x3 y1

1: Baseline -1 -1 -1 5

2: x1 lockdown per economics 1 -1 -1 5

3: x2 lockdown 1 1 -1 11

4: x3 less than maximum 1 1 1 5

Final lockdown 1 1 -1 11

True Maximum 1 -1 1 15

Page 13: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 13

2IV4-1 DOE Test Program in

Comparison to Companion OFT

-1 1 1 -1 21

1 -1 -1

-1 1 -1

1 -1 1

-1 -1 1

1 1 -1

-1 -1 -1

1

1

-1

1

-1

-1

49

31

31

31

47

77

x3 x1x3 x1x4 x4 y1

F I I F

-1 1-1

-1 1

1 -1

1 -1

-1 -1

-1 -1

1 1

-1

-1

-1

1

1

1

x2 x1x2x1

F IF

1 1

1

1

1

1

1

GM

ColumnAssign

(1)

x3x4

x2x4

x2x3

x1x4

x1x3

x1x2

1 1 1 1 731 1 11x1x2x3x4

Run

y1 = c0 + c1x1 + c2x2 +c3x1x2 +c4x3 +c5x1x3+c6x1x4 +c7x4

Page 14: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 14

Calculating Coefficients viaMatrix Orthogonality (1/2)

1 1 -1

-1 -1

1 -1

-1 1

-1 1

1 -1

-1 -1

1

1

-1

1

-1

-1

-1 1-1

-1 1

1 -1

1 -1

-1 -1

-1 -1

1 1

-1

-1

-1

1

1

1

-1

1

-1

1

-1

1

-1

1 1 1 11 1 1

1 1

1

1

1

1

1

1

-1 1 -1 1 -1 1 -1 1

c0

c1

c2

c3

c4

c5

c6

c7

● ● -1 1 -1 1 -1 1 -1 1

21

49

31

31

31

47

77

73

= ●

c4

0 0 0 0 8 0 0 0

c0

c1

c2

c3

c4

c5

c6

c7

● -21 +49 -31 +31 -31 +47 -77 +73=» » 8c4 = 40 » c4=5

Page 15: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 15

Calculating Coefficients viaMatrix Orthogonality (2/2)

1 1 -1

-1 -1

1 -1

-1 1

-1 1

1 -1

-1 -1

1

1

-1

1

-1

-1

-1 1-1

-1 1

1 -1

1 -1

-1 -1

-1 -1

1 1

-1

-1

-1

1

1

1

-1

1

-1

1

-1

1

-1

1 1 1 11 1 1

1 1

1

1

1

1

1

1

1 1 -1 -1 -1 -1 1 1

c0

c1

c2

c3

c4

c5

c6

c7

● ● 1 1 -1 -1 -1 -1 1 1

21

49

31

31

31

47

77

73

= ●

c3

Each coefficient is calculated in like fashion resulting in

y1 = 45+12x1+8x2+10x1x2+5x3-2x1x3-6x1x4+x4

Unveiling the black-box functional relationship

Page 16: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 16

DOE Advantages and Disadvantages

Allows for the inclusion of interactions into mathematical models and higher order terms when needed

Allows efficient evaluation of the coefficients associated with the mathematical model via the use of orthogonal arrays

Allows for multiple use and examination of test data per a variety of statistically sound techniques

Allows needed data to be generated using a minimum number of individual tests—time and cost savings!

Requires more up-front planning than traditional testing in that several pre-test issues must be addressed in asystematic fashion

Requires that the full DOE test program be executed in order to properly interpret data and results

Page 17: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 17

What Exactly is DOE?

DOE is one of the core “Six-Sigma” methodologies Statistically selects “axiomatic points” in the design space Selection enables maximum information return on investment made Used to systematically analyze the nature and cause of variation by

means of controlled testing (as opposed to examining available data) Cause is linked to effect by establishing through experimentation the

coefficients for pre-determined “best-fit” models Linear models: two-level experiments Piece-wise linear models: multi-level “orthogonal type” experiments Non-linear (general second-order quadratic model): response

surface methodsTest programs built upon sound DOE principles are

Significantly compressed and extremely efficient Produce high-quality and reusable data

Page 18: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 18

When Can We Use DOE?

Fact: Any physical phenomenon or process that can be thought of in terms of a stimulus-response model can be analyzed using DOE.

Stimulus A

Stimulus B

Stimulus C

Stimulus D

ResponseOuch!

Page 19: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 19

The Classic Dial Problem

In the early 1990s, the Air Force conducted a Halon Replacement test program that examined the effects of 14 two-level factors upon a single response variable: “pounds of fire suppressant needed to extinguish a fire”. The question was asked, “What are the best settings for our 14 dials in order to minimize the response variable?”

Factors

Response

-1 1

A

B

C

D

E

F

G

H

I

J

K

L

M

N

Levels

Objective: Minimize the quantity of fire suppressant needed to extinguish a fire.

Page 20: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 20

16,384 Rows X 15 Columns

The Previous Question Equates to the Classic Dial Problem

In the previous Air Force example, a full factorial test program consists of 214 or 16,384 production runs This many runs is definitely out of the question! Can you image the size of the associated matrix!

We are limited by time and money and can typically make only fifty production runs or so.

But how do we pick the right fifty? By experience? By guessing? By convenience?

Page 21: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 21

DOE Solves the Dial Problem!

In the Halon Replacement test program, a special-purpose

“orthogonal array” having just 32 rows (one row per run) was used. This array not only solved the dial problem but also produced high-

quality experimental results extremely useful in identifying a

minimum.

Factors

Response

-1 1

A

B

C

D

E

F

G

H

I

J

K

L

M

N

Levels

Page 22: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 22

Two-Level Orthogonal Array of Exact Type Used in Halon-Replacement Test Program

Cols 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32Factors GM a b c d e f g h i j k l m n o error

Rows

abcdefghijklmn

acbdegfhikjlmo

adbcehfgiljkno

aebfcgdhimjnko

afbechdginjmlo

agbhcedfiokmln

ahbgcfdejoknlm

aibjckdlemfngo

ajbicldkenfmho

akblcidjeogmhn

albkcjdifognhm

ambncoeifjgkhl

anbmdoejfiglhk

aocmdnekflgihj

bocndmelfkgjhi

1 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -12 1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 13 1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -14 1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 15 1 -1 -1 -1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 -1 -1 -1 -16 1 -1 -1 -1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 -1 -1 -1 -1 1 1 1 17 1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -18 1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 19 1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -110 1 -1 1 1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 111 1 -1 1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -112 1 -1 1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 113 1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -114 1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 -1 -1 1 1 -1 -1 1 115 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -116 1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 117 1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -118 1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 119 1 1 1 -1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -120 1 1 1 -1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 121 1 1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -122 1 1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 -1 1 1 -1 1 -1 -1 123 1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -124 1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 125 1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -126 1 1 -1 1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 127 1 1 -1 1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -128 1 1 -1 1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 129 1 1 -1 1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -130 1 1 -1 1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 -1 1 -1 1 -1 1 -1 131 1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -132 1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1 -1 1

Response Values

Page 23: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 23

Some Typical DOE Compression Ratios for Two-Level Experiments

# Two-LevelFactors

2 3 4 5 6 7 8 91011121314

FF

4 8 16 32 64 128 256 512 1,024 2,048 4,096 8,19216,384

DOE

4 8 816161616323232323232

Ratio

111/21/21/41/81/161/161/321/641/1281/2561/512

Unique IndividualTests Required Standard DOE

Nomenclature

22-0 or 22

23-0 or 23

24-1

25-1

26-2

27-3

28-4

29-4

210-5

211-6

212-7

213-8

214-9

Page 24: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 24

An Actual Air Force Gas Turbine Engine Vane Cleaning Experiment

A gas-turbine engine vane becomes corroded during service and requires periodic cleaning. Very high pressure water is delivered through a tiny nozzle orifice in order to cleanse the vanes. The response variable (Quality Characteristic) is percent contamination remaining after the cleansing procedure. A designed experiment is conducted in order to find the factor-level combination that minimizes the quality characteristic. (Lower is better.)

V∞

Page 25: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 25

Factors, Levels, and Output Using Standard XYZ Descriptors

Factor Sound-Alike XYZ -1 Level 1 Level

Orifice Size O x1 0.07in 0.1in

Standoff Distance S x2 0.5in 1.0in

Pressure P x3 20KSI 35KSI

Feed Rate F x4 20ipm 30ipm

Pump RPM R x5 1500rpm 2000rpm

Output % y1

Page 26: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 26

Vane Cleaning Dial Model

Factors

Response

-1 1

x1

x2

x3

x4

x5

Levels

y1

A Full Factorial experiment would consists of 25 = 32 individual trials.

Page 27: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 27

2III5-2 Designed Experiment

Showing the “Alias Structure”

x4x3x2x1

-1 -1 -1 -1 10.1

1 1 1

-1 -1 1

1 1 -1

-1 1 -1

1 -1 1

-1 1 1

1 -1 -1

1

1

-1

1

-1

-1

1

11.9

9.2

11.3

8.9

13.5

7.8

13.1

x1x3 x5 y1

x2x4 x2x5

x2x3

x1x5

x1x4Alias

Structure

-1 -1-1

1

-1 -1

1 1

1 1

-1 1

-1 1

1 -1

-1

-1

-1

1

1

1

1 1 -1

1

1

1

1

1

1

1

x1x2

GM

x3x4 x3x5x4x5

Assumed Model Form: y1 = c0+c1x1+c2x2+c3x1x2+c4x3+c5x1x3+c6x4 +c7x5

Page 28: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 28

Coefficient Pareto Chart and“Scree Line” for Half Effects

00.20.40.60.8

11.21.41.61.8

x3 x1x3 x2 x4 x1 x5 x1x2

Scree is the rubble at the bottom of a cliff

Red columns: deemed insignificant and will be rolled into error

Blue columns: significant or part of significant two-factor interaction

Page 29: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 29

Cube Plot for the Three Retained Factors x1, x2, and x3

10.625

x 2

x 3

x1

-1,-1,-1: 10.1

-1, 1, 1: 11.3

1,-1, 1: 13.5

1, 1, -1: 7.8

1,-1,-1: 8.9

-1, 1,-1: 9.2

1, 1, 1: 13.1

-1,-1, 1: 11.9

9.0

12.45

X3 effect = 3.45c4 = 1.725

10.725 10.875

10.35

11.1

X2 effect = -0.75c2 = -0.375

X1 effect = 0.20c1 = 0.1

Page 30: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 30

The x1x3 Interaction Plot

x 3

x1

-1, -1: 9.65

1, 1: 13.3

1, -1: 8.35

-1, 1: 11.6

9.0

12.45

10.625 10.725

10.875

X1X3 effect = -1.5c5 = -0.75

x1

x1 trends upward when x3 = 1

x1 trends downward when x3 = -1

11.475

9.975

Page 31: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 31

Linear Model and Optimal Settings

y1 = 10.725 + 1.725x3 - 0.75x1x3 - 0.375x2 + 0.1x1

Only factors deemed “significant” by themselves or part of a significant “two-factor” interaction are included

The others are part of the error

Methodology for minimizing y1

Set x3 = -1

Set x1x3 = 1 which implies x1 = 1

Set x2 = 1

Minimum: y1 = 10.725 - 1.725 - 0.75 - 0.375 + 0.1

Implies theoretical best y1 = 7.975Must be verified through a series of confirmation

experiments

Page 32: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 32

Vane Cleaning ANOVA Table

Factor Ci SS Comment

GM 10.725 920.205 Shows a process!

x1 0.1 0.08 Part of sig. 2FI

x2 0.0 0.0

x1x2 0.0 0.0 Include with error

x3 1.725 23.805 Big driver

x1x3 -0.75 4.5

x4 -0.125 0.125 Include with error

x5 0.05 0.02 Include with error

Totals 949.86 Also, we have Σ ci2 = 949.86

Source V SS F Ratio Significance Comment

GM 1 920.205 19,051.86 >>99%

x3 1 23.805 492.85 99%

x1x3 1 4.5 93.16 99%

x2 1 1.125 23.29 95%

x1 1 0.08 1.86 Must include Part of sig. 2FI

Error 3 0.043 divisor

Page 33: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 33

Overall 90% Confidence Interval

Calculating the overall 90% CI

where F90% (1, 3) = 5.54

SGE = 8/(1+4) = 1.6

IHL = (5.54x0.043 / 1.6)0.5 = 0.385

CI is (7.975 – 0.385, 7.975 + 0.385) = (7.59, 8.36)

95% CI is (7.453, 8.496)

99% CI is (7.018, 8.932)

Page 34: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 34

General Applicability of the DOE Process as Presented

Even with the introduction of advanced techniques and models, the general DOE procedural protocol as presented

in this orientation is still applicable.

Run123456789

A111222333

B123123123

C123231312

D123312231

R___________________________

L9To the right is an L9, which can be used as a full factorial design for two three-level piecewise linear factors or as a fully-saturated design for four three-level piecewise linear factors. All general DOE process topics still apply even though previously discussed computational methods will need to modified to accommodate the additional levels.

Page 35: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 35

A Short Laundry List of More Advanced DOE Topics

Use of “non-geometric”, fully-saturated screening designs such as L12, L20, and L28

Use of piecewise-linear, multi-level designs such as the L9 just shown Use of center points in a design to check for quadrature Design resolution, aliasing, and use of “fold-over” designs Use of blocking and blocking factors Use of and limitations of response-surface methodologies

e.g. Central-composite and Box-Bhenken When DOE might not work past screening phase

Highly interactive and non-linear phenomena such as turbulence Use of DOE as a preprocessor to major computer codes

Any analysis code can be looked upon as a numerical “test facility” DOE can be used to pre-screen input parameters, cutting down on number

of runs and subsequent total runtime GE and Pratt-Whitney notable examples

Use of DOE to analyze “available data” Requires systemic data mining and elucidation of patterns Can be very tough to perform!

Page 36: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 36

A Small Central Composite DesignFor Three Factors x1, x2, and x3

Run x1 x2 x3

1 -1 -1 -1

2 -1 -1 1

3 -1 1 -1

4 -1 1 1

5 1 -1 -1

6 1 -1 1

7 1 1 -1

8 1 1 1

Run x1 x2 x3

9 0 0 0

10 0 0 0

11 0 0 -0

12 0 0 0

Run x1 x2 x3

13 -1.68 0 0

14 1.68 0 0

15 0 -1.68 0

16 0 1.68 0

17 0 0 -1.68

18 0 0 1.68

+ +

Factorial Points Center Points Axial Points

Center points are used to check curvature. If curvature is significant, then axial points are added to build a quadratic model. Axial points are not

usually added for insignificant main effects. Continuous factors are a must!

Page 37: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 37

Run Diagram Showing Factorial, Center, and Axial Points

x 2

x 3

x1-1,-1,-1

-1, 1, 1

1,-1, 1

1, 1, -1

1,-1,-1

-1, 1,-1

1, 1, 1

-1,-1, 1

1.68, 0, 00, 0, 0

0, 0,1.68

0, 1.68, 0

-1.68, 0, 00, -1.68, 0

0, 0,-1.68

Page 38: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 38

SynGenics Two-Day DOE Course Description and Objectives

Course Description Basic introduction to “two-level” DOE that includes

The importance of experimental design How to plan and design an experiment The role and use of “orthogonal” arrays How to conduct a statistically designed experiment How to analyze results from a statistically designed experiment

Take-away tool box in this course is limited to two-level designs and associated analysis techniques

Course Objectives Be able to plan, execute, and analyze a simple two-level designed

experiment Be able to understand and assess more complex two-level

designed experiments as presented by Air Force contractors Be cognizant of advanced DOE methodologies that go beyond the

basic two-level designs

Page 39: A Quick-Look Design-of-Experiments (DOE) Orientation Carol Ventresca Carol @SynGenics.com John C. Sparks John@SynGenics.com © 2008 SynGenics Corporation

© 2008 SynGenics Corporation. All rights reserved. 39