I1 - Experiment design Tools

Six SigmaExperiment Design ToolsExperiment Design ToolsMost companies collect data to monitor the health of their processes, but most of this data is not acted on in any way because it does not contain meaningful information. Often, when these compa-nies decide to perform an experiment in an effort to produce more meaningful data on a specific CT characteristic, they design a onefactor-at-a-time test, but progress is slow and expensive using this technique. On the other hand, companies who learn to use Design of Experiments (DOE) are able to leverage their efforts by conducting well thought out experiments where several factors are altered simultaneously in one efficient series of tests. DOE provides a wealth of information and in a shorter period of time. The role of a statistically designed experiment is to identify the most influential factors (vital few) associated with a particular CT characteristic and to define their relationships using analytical quantities. Since interactions between various factors are also defined, a single DOE can yield many revealing facts allowing the experimenters to quickly improve their process. It is most effective to conduct DOE's early in the design process, experimenting with the effect of tolerancing and failure modes, so that when the product or service is introduced to the market place, customer satisfaction will be high and warranty claims minimal. The same experimental techniques can then be used throughout the life cycle to continuously improve, taking into account new information as it becomes available. There are two components to a DOE: Experimental Design and Analytical Procedure. It is best to think these out carefully before conducting the DOE in order to produce the best result. Depending on the level of understanding available about a process, there are various types of DOE that can be employed. Screening designs are typically used early in the process where many factors are involved, Characterization designs narrow the number of factors down to only a few, and Optimization designs focus on only one or two factors, but in much more depth. To ensure that a DOE is designed correctly, it is necessary to consider such topics as Yates standard order, factor levels using the 1, 1 notation, column contrast, balance, orthogonality, confounding and interactions between factors. Also, there are design factors that influence the time and cost aspects of the DOE, such as replications and the basic design type. The two-level two-factor design is relatively common in practice and makes a good starting point for most problems. When replications are taken into account, very meaningful results can be achieved with as few as 8 to 16 runs. When you don't know which factor has a substantial effect on a response variable, and you want to verify several factors, then the fractional factorial design is the preferred analytical tool. It is common practice to do a fractional factorial experiment when we have five factors or more. In this case, you only conduct runs which represent a portion of the matrix, because, for example, a 6-factor two-level experiment with no replications would involve 64 runs. The advantage, therefore, of a fractional factorial experiment is that fewer tests are required. When it comes to analysis of the data from a DOE, we must first evaluate the statistical significance. This is done by computing the One-Way ANOVA or the N-Way ANOVA (for more than one factor). The practical significance can be evaluated through the study of sums of squares, pie charts, Pareto diagrams, main effects plots, and normal probability plots. Note that there are situations where factors are statistically significant, but not practically significant. In any analysis, it is important to analyze the residuals prior to any conclusions being drawn. A step-by-step approach to designing and conducting any DOE must be adopted. The use of cross-

functional teams in the selection of factors to study, the design of the experiment, conducting the experiment, analyses of the data, recommendations, and implementation of DOE results all require careful planning and attention to detail to achieve success.

Key Questions What is a "statistically designed experiment" and what roles do they play in Six Sigma? What are the guiding principles surrounding statistically designed experiments? How can the practical and statistical influence of a single CTP be exposed? How can the practical and statistical influences of two or more CTP's be exposed? How can conflicts between competing performance objectives be resolved? What is the process and guidelines for planning statistically designed experiments?

Key Questions A DOE is a systematic method where a number of factors are changed simultaneously, following a predetermined pattern to investigate their effect on the response. The role of a statistically designed experiment is to determine the most influential factors associated with a particular CT characteristic (vital few) and to understand their relationships using analytical quantities. The fundamental guiding principles surrounding statistically designed experiments are: Process Relationships, Replication, Randomization, and Experimental Control. The statistical influence of a single CTP can be exposed through hypothesis testing using tools such as OneWay ANOVA, and the practical influence can be exposed through the study of sums of squares presenting the results in pie charts or Pareto diagrams. The statistical influence of two or more CTP's can be exposed through hypothesis testing using tools such as NWay ANOVA, and the practical significance can be exposed with the use of graphs such as main effects and normal probability plots. Conflicts between competing performance objectives can be resolved by the use of interaction effects between factors in order to find the best compromise settings. The process and guidelines for planning statistically designed experiments are: Define the problem, set the objective, select response variables, select factor levels, conduct experiment(s), analyze results, and ensure results are well understood and communicated.

Why/When to ExperimentTO IDENTIFY THE VITAL FEW TO ENSURE A ROBUST DESIGN TO SAVE MONEYHigh Focus on defect detection and correction

DO AS EARLY AS POSSIBLE IN PRODUCT LIFE CYCLE TO MAXIMIZE DESIGN QUALITY

YCost to deal with poor quality Focus on defect prevention

= f(X1, X2, , Xn) CTPs Factors Improve3

CTQs or CTCs or CTDs

1_05_01_002

Low Product DesignExperiment Design Tools

Process Design

Manufacturing

Field

Why/When to Experiment At a planning session, Six Sigma team members are evaluating whether or not they should conduct an experiment on their process. They ask the Black Belt to outline a criteria for this. The Black Belt explains why and when to use experiments: Experimentation can be used to improve the performance, quality and cost associated with key customer requirements. We use them to confirm and quantify the important inputs (CTPs) to our processes. Determine the "vital few" variables when there are no analytical equations to guide us. Experimentation can significantly reduce new product (or service) development cycle times and it is conceivable that manufacturing tolerances can be relaxed. A robust design can be created if we test it prior to design freeze. A product or service that is robust to variations in input factors and operating environment will cost society less over the life of the product, resulting in its customer loyalty. The need for product inspection and performance testing can be significantly reduced with extensive experimentation. We should always experiment as early in the cycle as possible, so as to improve quality, thus reducing rework, warranty claims and dissatisfied customers.

The Black Belt draws the above chart on the blackboard, summarizing his point that we should begin to test key business opportunities with a series of well planned experiments as early in the development cycle as possible in order to reduce the cost of poor quality, and in addition, throughout the product life cycle to achieve continuous improvement.

Fundamental Principles of Experimentation1. Understand Process Relationships Among process variables Among response variables

2. ReplicationIncreased precision of effect estimates Provide measure of experimental error

3. RandomizationSelect experimental units randomly Randomize run sequence

4. Experimental ControlActive variation of process variables Control other sources of variation

Fundamental Principles of Experimentation During Black Belt training, a new Black Belt asks the Instructor to review the fundamentals of Experimentation. The Instructor outlines the four basic principles, using the example of the classical apple falling on the head of Isaac Newton. "The first, and probably the most important principle", says the Instructor, "is to fully understand the process. In this example, the energy imparted by the falling apple is the process of interest to Isaac Newton. He set out to design a series of experiments to determine the relationships between the mass of the apple and the fall distance as they relate to the energy imparted." "Replication is the second principle of the experimentation process. Usually, two to five replications are conducted for each set of conditions in order to ensure that the "white noise" is captured. Replication allows a measure of experimental error." "The third characteristic of an experiment is to ensure that the order of conducting a series of runs is randomized in order to avoid false conclusions due to factors which may vary over the course of the experiment, such as temperature, wind velocity, etc. These variables will probably have small effects on the results, which will result in experimental error." "The fourth fundamental principle of experimentation is what is known as experimental control. This simply means that you deliberately change some variables, while carefully controlling all other variables as much as possible, noting the effect."

Role of ExperimentationExperimentation - the manipulation of controllable factors ( indepen dent variables ) at different levels to see their effect on some response (dependent variable).1. Factor A affects the mean 2. Factor B affects the standard deviationB1 A2 Y

A1

B2 Y

3. Factor C affects the mean and the standard deviationC11_05_01_004

4. Factor D has no effect

YExperiment Design Tools

Y

Role of Experimentation The Master Black Belt is asked to define the role of experimentation by the Champion. He explains that a statistically designed experiment is a test or a series of tests in which purposeful changes are made to the input variables of a process or system so that we may observe and identify reasons for changes in the output response. Once we have obtained the results of these tests, we can then decide how to change the settings or levels of some factors in order to produce better yield, improved quality or some form of improvement of our CT characteristic. The test results may lead us to select new factors which were dismissed or overlooked initially. We would then design a new experiment, and so on, until we achieve the desired results. The examples above imply that each factor can have independent effects on the result. We strive to find factors which have independent effects like this, but this is not always possible. The objective in many cases may be to develop a robust process, that is, a process affected minimally by the many sources of variability, i.e.: white noise; which we cannot control. The Master Black Belt concludes by saying that unless we experiment with our processes, guided by our judgement, we will never identify ways to improve those processes. Experimentation provides us with a sound, effective and economical method for determining the best way to change.

Improve5

C2

D1

D2

Sources of Experimental Factors

Input Factors (Xs)People Material Equipment Policies Procedures Methods Environment

Output (Ys)Service

PROCESS

CTCA blending of inputs to achieve the desired outputsProduct

CTQ CTD

Task

1_05_01_005

Experiment Design Tools

Sources of Experimental Factors At a Six Sigma team meeting, a team member asks the Black Belt if he could help the team select factors for an experiment on their process. The Black Belt points out that factors can come from a wide range of sources as shown in the list above and that it is impossible to select the most influential factors unless you fully understand the process. He suggests that they review the FMEA's that had been completed recently by the team, since these contain an excellent checklist of what is important to this particular process and act as a summary of the collective knowledge about the process from the brainstorming sessions. The Black Belt also says that they need to be aware that factors can be continuous in nature (like volts, pressure, etc.) or can be discrete (like new/old, machine 1/machine 2, etc.). Also, each factor needs to be independently controllable. The team now felt that they knew how to go about selecting factors for their first experiment.

Improve6

Types of Experimental FactorsCONTROLLABLE INPUTSFactors that we can control, and elect to hold constant for the experiment

OUTPUT (Y) INPUTS (Xs)Factors that we wish to vary in a DOE

Experimental Error Measurement Other Environment

UNCONTROLLABLE INPUTS1_05_01_006

White Noise

PROCESS UNDER STUDY

Response variable has variation due to: Effects: Factors Interactions

Factors that we cannot control Environmental UnknownExperiment Design Tools

Types of Experimental Factors The team realizes that there are different types of factors involved in their process some controllable and others uncontrollable. They ask the Master Black Belt for advice in how to handle these. The Master Black Belt draws the above chart on the board and describes a process he uses to ensure that all factors are considered when designing a DOE. Before beginning, he reminds the team that a review of the FMEA for their process may provide clues as to how to categorize the various factors. "The first step", says the Master Black Belt, "is to divide all known factors into one of the three categories as shown in the chart. Of all the possible inputs that could be used in a DOE, you need to decide which ones you wish to vary in the experiment, and which ones you can move to the controllable (but hold constant) category. It may be time well spent to put as many factors into this category as possible, especially ones that could be varied, but are not likely to be allowed to vary for certification reasons, or just because it doesn't make sense. Also, the factors that cannot be controlled are important to identify because there may be ways that their effect can be reduced by careful design of the experiment, or recording of data that would otherwise have been ignored or overlooked." The Master Black Belt goes on to say that the input factors remaining are likely to be few (i.e. 2 to 5) so that the experiment can focus on only these, resulting in as few runs as possible. He pointed out that, by the time this exercise is finished, the experimenters will already be more knowledgeable about their process and be ready to begin the design of their DOE.

What is "Design of Experiments" (DOE)Definition

Improve7

General The controlled, active variation of a process to identify better settings or procedures to improve the product quality. Specific A DOE is a systematic method where a number of factors are changed simultaneously, following a predetermined pattern, to investigate their effect on the response or output.

What is "Design of Experiments" (DOE) When the Black Belt is requested to define the concept of a DOE to the Champion, he uses the above definition. As can be seen, this definition applies equally to transactional, engineering design, or to manufacturing processes. The Black Belt explains that a DOE must have two basic elements a design where the team decides what to focus on, and a procedure for analyzing the data once it has been collected. He explains that a DOE is a way to generate information about a process which can then be used to improve it. The Champion says: "Tell me more, I am interested to see if I can apply this tool in the Customer Services area". The Black Belt outlines other features of a DOE: It is a way of separating the vital few from the trivial many factors. It is a systematic scientific approach which allows decisions to be based on facts. It is a planned set of trials (runs) with specific measures, carefully recorded. It is a structured way to economically improve any process. It is dynamic, testing many factors (and interactions) in a short time.

In conclusion, the DOE process consists of a series of experiments where competing settings of each factor are combined in order to efficiently reach valid and relevant conclusions.

The Shower Experiment

Flow Rate of Run 1 2 3 4 Cold Water Low High Low High

Flow Rate of Hot Water Low Low High High

1_05_01_008

Y = f (X1, X2) Water Temperature = f (flow of hot water, flow of cold water)Experiment Design Tools

The Shower Experiment During a refresher session on Design of Experiment (DOE), a Champion asks a Master Black Belt: "Do you have a simple example to show us?". "Of course, we will use the shower example. The process is quite simple. By adjusting hot and cold knobs, you will have optimal water temperature which represents the response of the process." "In most industries, we use two levels to do an experiment. In this case, let's suppose that the knobs can be set only at high or low levels. The two knobs represent the two variables or factors of the experiment. "Then we 2 have a two-level, two-factor matrix, i.e. 2 , meaning that four combinations are possible to get results". "We now can prepare the design matrix with factor Flow Rate of Cold Water and Flow Rate of Hot Water." "It seems to be very easy, why don't we use this tool more often?" asks a Champion. "This is a very simple example that we've done on the board and it represents a part of the planning of a DOE. Is it obvious to determine which are the best settings? The experiment itself still needs to be done. Who wants to volunteer?" asks the Master Black Belt. "Remember, conducting an experiment requires human and material resources, parts, raw material, money, time, etc. In order to get worthwhile results, each DOE should be carefully planned with all the required resources" concludes the Master Black Belt.

Improve9

Components of a DOE1. Experimental Design Fixed Effects Random Effects Mixed Effects

Questions:What result do we expect to get? What are we going to measure? How will we collect the data? How will we present the data?

Questions:How will we analyze the data? How will we find relationships? How will we find interactions?1_05_01_009

2. Analytical Procedure ANOVA Regression Etc.Improve10

How sophisticated does the analysis have to be?Experiment Design Tools

Components of a DOE In his presentation describing the various features of a DOE, the Master Black Belt feels he should go into more detail. He summarizes the questions that one needs to ask as shown above and explains the two components as follows: 1. Experimental Design: A means to structure the variables for experimental manipulation. This can be broken down into three types of effects. Fixed Effects The experimental levels are selected by the team. Conclusions apply only to selected factor levels. Most experiment designs are of this type. Random Effects Levels are randomly selected from a population. Conclusions can be generalized to the population. Mixed Effects A combination of fixed and random models.

2. Analytical Procedure: A means to extract relevant information from the experiment. Analysis of variance (ANOVA) Regression Find a relationship between variables.

In conclusion, the Master Black Belt states that, in the majority of cases, we use "Fixed Effects" designs and ANOVA for the analysis procedure.

Why Not One-Factor-at-a-Time? The autoclave operator insists that he should not vary more than one parameter at a time, fearing that the results would be impossible to interpret. The Six Sigma Black Belt, trying to convince him otherwise, explains that since there are so many factors (potential CTPs) involved in even simple processes, it would be very time consuming and expensive to do a series of experiments "one-factor-at-a-time". He also points out how focusing on "onefactor-at-a-time" will probably miss the optimum combination of factors. The Black Belt illustrates this with a two-factor example. He explains that by varying the temperature (factor A), a zone of maximum response is determined (1 above). Then by keeping temperature constant, the pressure (factor B) is varied to determine its best response (2 above). One may conclude that this process defines the optimum settings, but if we overlay this picture onto the real situation (3 above), the so-called optimum is far from defining the maximum potential value (4 above). As one can imagine, this becomes more pronounced when more variables are involved. A much better approach is what is termed a "full factorial design" where a few factors are varied at the same time, in random order, to yield a matrix of effects. This approach includes interaction effects which can be very important in some cases, and would be missed completely in the "one-factor-at-a-time" approach. The Black Belt sits down with the autoclave operator and the Six Sigma team to design such an experiment. They are anxious to try a DOE because they feel that there are significant improvements that can be achieved.

Benefits of Statistical Experiments

Change One Evaluation criteria Time to get results Cost Chance of detecting optimum Confidence in results Ability to identify independent interactions and main Expert Observe Very Long Very High Very Low Very Low Long High Medium Medium Long High Overall High Overall Suboptimum likely High Guess Opinion Everything Factor Stat at once At-a-time Long Short Rel Low Very Good High DOE

Low Low

Moderate Moderate

Moderate Very Low

effects

None

None

None

None

None

Yes

Conclusion: Statistical Experiments are Rich With Information!!Benefits of Statistical Experiments A Black Belt observes over the course of the past year that there is a lot of folklore about the benefits of observation, judgement and "one-factor-at-a-time" as techniques for deciding how to make a change to a process. He decides to map the various techniques and to identify their strengths and weaknesses compared to statistical experiments. The above table is the result of his efforts. Using a subjective scale, one can see that, for all criteria selected, the statistical experiment is far superior than any of the others. After creating the table, the Black Belt makes some notes in the way of conclusions: Statistical experiments provide fast, low cost, optimal results with a high degree of confidence. However, in the short-term, they do cost money and disrupt the process, so they must be properly planned. In addition, statistical experiments are the only technique to provide the data for "main effects" and "interactions" between various factors. We use experiments to supplement our judgement, not replace it. Properly designed experiments will permit simple interpretation of results, which may not be possible otherwise. Proper experimental design is considered to be more important than sophisticated statistical analysis. The cost of a DOE is much lower than the cost of poor quality (long-term).

Types of ExperimentsVery Informal Trial and Error Methods Introduce a Change and See What Happens Produced Under Controlled Conditions

Pilot Runs Set up to Produce a Desired Effect

One-factor-at-a-Time Experiments Vary One Factor and Keep All Other Factors Constant

Planned Comparisons of Two Methods Background Variables Considered in Plan Study Separate Effects and Interactions

Experiment With 5 to 20 Factors Screening Studies1_05_01_012

Modeling, Multiple Factor Levels, Optimization


Types of Experiments After having some exposure to the basic concept of a DOE, a Black Belt in training asks how the traditional approach that he had been familiar with compares to the Six Sigma approach. "Good question," replied the Instructor, "let me write out what I'll call the "hierarchy" of experiment types." He writes down the first four headings and everyone agrees that they know what these involve. He went on to write the headings that describe the Six Sigma approach. At this point in the course, the students are not familiar with these, but they are amazed to think that up to 20 factors can be dealt with within one experiment. The Black Belt in training asks: "Does this mean that we will no longer use the traditional types of experiments?". The Instructor replies that they will still be used on occasion a special lot or a pilot run could still be used on occasion if you were wanting to check the data collection process for calibration purposes, etc. He said that once everyone had some experience with the Six Sigma approach, they would readily see that it gives much better information than the traditional approach and that the new tools would be easier to use.

Improve13

Very Formal

Comprehensive Experimental Plan With Many Phases

Six Sigma Approach

Experiment Planned With Two to Four Factors

Traditional Approach

Running Special Lots or Batches

Classes of DOE

Screening

Characterization Optimization

A Factors C F I

E B K J D F

B K J F K

H G

Time1_05_01_013


Classes of DOE There are several classes of DOE that can be applied to improve a process. When a team member asks the Master Black Belt to give an overview of how the various classes of DOE are applied, he draws the above chart and elaborates on each as follows: Screening designs are generally used to identify the "vital few" process factors, putting the "trivial many" aside. Usually, such designs involve a large number of factors and are limited to only two levels. Characterization designs are generally used to study simple effects involving a small number of factors (3 or 4). Such designs are usually based on full factorial models using two or three levels. Optimization designs (also referred to a modeling designs) are used to study complicated effects and interactions involving one or two factors. In general, such designs are employed to yield a precise mathematical model of the process under investigation and may involve several levels (to capture nonlinearities).

The Black Belt points out that the above class descriptions are very general in nature and should not be used as the sole benchmark when designing an experiment. In practice, more than one experiment is generally required to obtain a meaningful model, the results of one being used to design the next. Of course, each experiment is different from the ones before, being tailor-designed for the particular factor and levels being investigated. He also points out that, most often, we work with screening designs, occasionally with characterization designs, and rarely with optimization designs.

Improve14

Terms Used in DOEProcess Factor Variable A B C Coded Value +1 -1 +1 -1

Setting

Pressure High (40psi) Low (20 psi) Machine Shift New Old Number 1 Number 2

Y = f (A, B, C)Factors Treatment or CombinationProcess Variables B C -1 -1 +1 +1 -1 -1 +1 +1 -1 -1 -1 -1 +1 +1 +1 +1 Response Variable 5.4 6.1 7.2 3.7 9.3 8.3 10.7 5.5

+1 -1 Sequence Order 1 2 3 4 5 6

A -1

Order

+1 -1 +1 -1 +1 -1 +1

Data

1_05_01_014

Levels

7 8


Terms Used in DOE As a means of introducing the topic of DOE to the Six Sigma team, the Black Belt presents the above chart which represents the design anatomy of a typical experiment. He is asked to review the primary terms, which he does as follows: DOE Design of Experiment, or "an invitation for an information event to occur". Order Sequence order or run order for the experiment. Factor Process variable that will be purposely altered during the experiment. Level Factor setting. Treatment Combination Combination of levels for this element of the experiment. High Level High setting (e.g.: 40 psi, new, shift 1, etc.). Low Level Low setting (e.g.: 20 psi, old, shift 2, etc.). Response Variable The subject of the experiment (CTQ, CTD, CTC). Data The result i.e. the magnitude of the response variable.

The Black Belt points out that the matrix shown is in the form convenient for the design of the experiment, and that later on we will see how to arrange this data so the experiment can be conducted. From this brief description, each team member has a clear understanding of the design structure of a DOE and begins to plan an experiment that applies to their process.

Improve15

Main Effects and InteractionsYB- Condition

Run 1 2 3 4

A -1 1 -1 1#1

B -1 -1 1 1#2

A*B 1 -1 -1 1Interaction Effect

Y 3.0 5.0 5.0 12.0Y

5 3

=2

AB+ Condition

A+

12=7

Main Effect1_05_01_015

AExperiment Design Tools

A+

Main Effects and Interactions Before beginning to design their experiment, the team leader asks the Black Belt to explain what is meant by "main effect" and "interaction". The Black Belt uses the above example to describe the two terms. Main effect describes the direct effect of factor A on the response variable (Y). Likewise, B is another main effect. Factors A and B are generally assumed to be totally independent of one another (2 vs. 7). Interaction effect describes the effect of the product A x B on the response variable (Y) when two factors have a combined effect on the response variable. The Black Belt points out that one of the powerful advantages of a well designed DOE is that it has the benefit of being able to identify interactions which would be totally missed in a one-factor-at-a-time experiment. The interaction effects can be used to achieve the desired result. The two charts on the right hand side above show the interaction effect, i.e.: when B is high, the yield (Y) of the process is over three times better than when B is low.

Improve16

5

Contrast - DefinitionContrast of factor F = F+ - FVectored Vectored Column Column Response Response A -1 1 -1 1 Y

DESIGN MATRIXRun Order 1 2 3 4 (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y 1.5 4.5 4.5 13.5

* 1.5 * 4.5 * 4.5

= -1.5 = 4.5

= -4.5

* 13.5 = 13.512.0

Example ComputationRun Order 1 2 3 41_05_01_016

A -1 1 -1 1 12.0

B -1 -1 1 1

AB 1 -1 -1 1

Y1 1.5 4.5 4.5 13.5

(1) a b ab Contrast

Column ContrastSum of Vectored Responses


Contrast Definition During a training session on Design of Experiment (DOE), a Black Belt asks: "You mentioned the word CONTRAST in your explanation, but I don't understand what it means". The Master Black Belt responds: "Actual factor settings for an experiment, such as Low, High, 80 psi, 120 psi, etc, are replaced by coded values (+1, -1) called coefficients in a design matrix. The coefficients of a factor or interaction are used to calculate its contrast. To define a contrast, a set of coefficients have to conform to conditions known as "balanced" and "orthogonality", which we will learn later." "As you can see on the visual, the contrast of one factor or interaction is the result of the sum of the coefficients multiplied by the response of each run. In other words, we can say that a column contrast is the linear sum of vectored responses." "Thus, for a main effect, a positive contrast means that the response variable is greater when the factor is set at its high level (+1), compared to the low level (-1). A negative result means the inverse, i.e. that the response variable is greater when the factor is set at its low level (-1), compared to the high level (+1). Then contrast compares two things or groups of things by comparing their difference. It is just a special way of making a comparison." "Is it then accurate to state that the term contrast is used for more than one thing?" asks a Black Belt. "Yes, actually contrast is an intermediate calculation used in statistical analysis of a DOE, and it also defines a set of coefficients used in a design matrix" concludes the Master Black Belt.

Improve17

Contrast - PropertiesDESIGN MATRIX

Run 1 2 3 4

A

B

AB 1 -1 -1 1

-1 -1 1 -1 -1 1 1 1

Balanced

X i =

0 for each factor sum

1_05_01_017

OrthogonalExperiment Design Tools

Xi X j =

0 for all dot product pairs

Contrast Properties During a training session on Design of Experiment (DOE), a Master Black Belt says: "We know that a column of a design matrix is considered as a contrast under certain conditions, but do you know what these conditions or properties are"? A Black Belt responds: "Balanced and orthogonality, but I don't have a clue as to what these mean". The Master Black Belt says: "Well, take a design matrix for example. We have factors and interactions between factors. To be considered as a contrast, the sum of a set of coefficients must equals zero. This is the basic condition to have contrast. When the sum of each column equals zero, the experiment is said to be balanced. In a case of a twolevel design, this condition means that for one column, there is the same number of High (+1) and the Low (-1) levels." "When you closely look at the picture, we see that the AB interaction is created by multiplying the coefficients of factor A by the coefficients of factor B, and the sum of the coefficients of AB contrast is also equals to zero. For every pair of contrasts, when the sum of the product equals zero, then the design matrix is said to be orthogonal. This feature ensures the effects are independent. Because all effects can be independently estimated, small effects will not be washed out or masked by large effects. Actually, you don't have to build orthogonal contrast. Minitab software will automatically provide a balanced and orthogonal designs" concludes the Master Black Belt.

Improve18

Run Order for a DOERandomized RunOrder1 2 3 4 5 6 7 8 9 10 11 12 13 141_05_01_018

Design Order16 15 10 11 3 5 13 8 12 11 4 6 2 9 7 4

Factor A1 -1 1 -1 -1 -1 -1 1 1 -1 1 1 1 -1 -1 1

Factor B1 1 -1 1 1 -1 -1 1 1 -1 -1 -1 -1 -1 1 1

Factor C1 1 -1 -1 -1 1 1 1 -1 -1 1 1 -1 -1 1 -1

15 16


Run Order for a DOE Once the factors and levels to be investigated are identified, you must then decide how many replications you wish to do. The use of Minitab makes it easy to lay out how to actually run the experiment. For a three-factor, two-level experiment, with two replications, the above table shows how the run sequence would appear. The Black Belt states: "You should note that the original order from the design matrix is now randomized. The original design order is given in the second column. If we wanted to have three replications instead of two, the list of runs would be increased by 8 and the whole list would then be randomized." A team member asks: "Why is it necessary to randomize the run sequence?". The Black Belt responds by saying: "Running the experimental trials in a random order protects against the effects of unknown nonexperimental variables (like environment) that might bias the results. In other words, if some unknown factor is changing continuously over the period of the experiment, its influence will be minimized by this technique."

Improve19

How to Select Factor Levels80

Response Variable (Y)

70 60 50 40 B- high 400 Y True Effect True Effect 500 A-high 300 30 B-low 20 10 0 A-low 200 B difference A difference

100

600

Factor SettingsY

1_05_01_019

Experimental Effect Lo (-) Hi (+) Lo (-) Hi (+)

Experimental Effect

Factor Settings

Factor Settings


How to Select Factor Levels A Black Belt makes the statement that selecting factor levels is a critical step in determining the success of a DOE. "If that's true," said the Champion, "can you help us with a criteria to do this?". "Yes," said the Black Belt, "but remember, it is much better to plan a series of experiments from the beginning than it is to try to do it in only one. You should plan to spend only 25% of your allotted time and budget on your first experiment." The Black Belt outlines the important aspects as follows: Draw on all past experience to provide knowledge of your process. The intent is to capture a large meaningful result, well beyond white noise or measurement resolution. Select step sizes to ensure not missing the optimum. Levels for each factor should be balanced in relation to each other (i.e.: no one dominant). Levels need to be logical, rational, practical, and safe. Initial levels should be bold or wide, reducing to narrower steps in subsequent experiments. A good rule of thumb is 20% beyond input spec limits. If no spec limits exist, use 3 standard deviations to set levels. Use a pilot run to verify sensitivity to level selections.

Improve20

Confounding Effects

MATRIX APrinter 1Paper Type Low Paper Type Low Paper Type Low Paper Type LowST AN DA RD

Printer2Paper Type High Paper Type High Paper Type High Paper Type High

Coded values-1 1

Printer#1 #2

PaperLow Quality High Quality

MATRIX BRun1 21_05_01_020

Printer 1-1 1 -1 1

Paper-1 -1 1 1

3 4


Confounding Effects A Black Belt is working on a project to minimize the number of rejected reports due to bad printing quality that occur with the two main printers located in the computer room. After a rigorous study, the factor "quality of paper" will be analyzed. During a coaching session with a Master Black Belt, the Black Belt proposes to conduct a Design of Experiment (DOE). "What is the objective of the experiment?" asks the Master Black Belt. The Black Belt says "We want to know the effects of the "quality of paper" on printing quality. In this case, we hypothesized that an increase in "quality of paper" increases the quality of printing. To test this theory, we did the design matrix A initially, but in this case it shows all low settings for printer 1 and high settings for printer 2. I fear that the differences in the response variable due to "quality of paper" won't be separated from the unique effects of the printer. In other words, the effect due to "quality of paper" will be confounded or mixed in with the printer in use." The Black Belt asks the Master Black Belt for advice. "If we design a new matrix as shown in B with both types of paper in both printers, we will be able to differentiate the contribution of each paper type in the response variable, assuming that the influence of all other background variables have been adequately controlled or otherwise, neutralized" says the Master Black Belt. "Yes, I see that by doing this, we will be able to draw meaningful conclusion from the collected data" concludes the Black Belt.

Strategy of Experimentation1. Define the Problem 2. Establish the Objective

Improve21

3. Select the Response Variable(s) 4. Select the independent factor(s) 5. Choose the factor levels 6. Select the Experimental Design 7. Collect the Data 8. Analyze the Data 9. Draw Conclusions 10. Achieve the Objective "The purpose of an experiment is to better understand the real world, not to understand the experimental data."William Diamond IBM Retired Statistician Strategy of Experimentation A Champion asks the Master Black Belt to list the general steps involved in conducting a Design of Experiment (DOE). "Well, after the Measure and Analyze phases, we should define and therefore state the problem. For example, the mean of the process under study is not on target or there is too much variation, and as result we are producing a large amount of rework. The objective is to reduce the PPM defective to avoid rework. At this phase, we select the independent variables that we want to test, the X1, X2, , XN. Now, we are ready to select appropriate levels of each factor knowing what would make sense for our process. We then assign these levels -1 and +1 notation." "When do we start collecting data?" asks a Champion. "One more step is required before collecting data" replies the Master Black Belt. "The selection of the experiment design, or the structure of the experiment i.e. Full Factorial, Fractional Factorial, etc." "All of these preceding steps constitute the planning part of an experiment. An experiment design is the deliberate arrangement of competing settings among one or more independent variables, and when properly planned or designed, it will permit relatively simple interpretation of the results, which may not be possible otherwise." "I'm impressed," says the Champion, "you seem to be very comfortable with this subject. If you don't mind, I would like to assist you in conducting one experiment, so I will learn more about this tool and be able to provide more assistance to the Black Belts".

Barriers to Effective Experimentation At the Annual Six Sigma Symposium, a guest speaker presents the powerful tool "Design of Experiment" (DOE). He describes all the advantages and steps to conduct a valuable experiment. During the question period, an Black Belt asks: "I see that there are many advantages to using this tool, but what are the difficulties or obstacles that we should be aware of before using it?" "This is a good question", replies the guest speaker, "let me show you a list of the main barriers to effective experimentation. As we can see, eliminating some of these barriers is within your reach. Why should we conduct a DOE if the problem definition or the objectives are not clear? Some of these barriers may be attributable to a lack of training and information. Ask why there are negative perceptions in your organization? You are deploying efforts but do you adequately communicate your plan and progress to your organization? Think about it".

"As in any organization, because of a lack of time and because of other priorities, management may not always be able to provide you with adequate support and coaching. You have to involve them in the realization of a DOE, so that they will start to learn, understand and promote the use of these powerful tools. But once again, I insist on the importance of carefully planning your DOE. A good plan, which is adequately communicated may prevent negative perceptions and promote management's active involvement. With perseverance, we can initiate change even in an environment of excessive bureaucracy. My final message to you is DON'T GIVE UP", says the guest speaker.

The Car Mileage ExperimentExperiment Design:

Objective: How to improve fuel economy on the highway.

Factor A. Type of Gasoline B. Cruise Speed C. Tire Pressure D. Windows

Level 1 (-) Regular 100 Km/hr 20psi Open Off

Level 2 (+) High Octane 130 Km/hr 30psi Closed On

Measure : Litres/100 Km.

E. Air Conditioner

Results:Factor (in Order Of Importance) B. C. D.1_05_01_023

Best Factor Level 100 Km/hr Windows Closed/A.C. Off 30psi Off High Octane Closed

Cruise Speed Tire Pressure Windows Air Conditioner Type of Gasoline

DE. Windows * A.C.

E. A.Experiment Design Tools

The Car Mileage Experiment A Master Black Belt conducts a DOE on his automobile to try to understand which factors are important contributors to fuel economy. During a training session, he presents the results to a class of Black Belts in training. He makes the following points as a way of illustrating the various aspects that need to be considered in any DOE. This one relatively simple experiment produced a wealth of knowledge. The actual order of importance of the various factors is contrary to his original thoughts (order in the first box above). An interaction between the windows and the air conditioner proves to be the second largest effect (not even imagined). What probably really matters is minimizing cost, not maximizing fuel economy. If, for instance, the type of gasoline had been more significant, one needs to include the extra cost for super grade to determine if it is still an advantage.

Improve24

Before concluding that the type of gasoline, for instance, is significant, one needs to ensure that the ability to measure fuel mileage is accurate enough to differentiate such small effects (i.e.: is measurement error causing a false conclusion). There are generally many other factors involved. These must be kept constant as much as possible in order to obtain valid results.

One Factor AnalysisRe sponse 0.99265 Fa ctor 1 1 1 1 1 2 2 2 2 2 3 3 3 3 3

it0.87233 in 1.17466 M 0.887090.79582 1.12497 1.27382 1.20943

ab

1.06742

1_05_01_024


One Factor Analysis A printed circuit board is related to high field failure rates. Following an analysis of the problem a cross-functional team concludes that the primary failure is due to low adhesion of the circuitry. After careful consideration of the evidence the team hypothesized that the problem is related to the type of solder paste used during production. Three types of paste are available; Wilsons, Acme, and Twecho. An Engineer on the team is tasked with evaluating the pastes to determine which one is the best. To make this evaluation he randomly selects 15 printed wiring boards and randomly assigns them to the three paste types. After curing he subjects each board to a tensile test to quantify the adhesion of the different pastes. At a team meeting the Engineer gave the team some general information about the single factor analysis which he was conducting. "The dependent variable is called the response. It must be measured on an interval or ratio scale; in our case the measurements will be in pounds. The independent variable is commonly known as a factor. In all cases the factor must be treated as a categorical variable. Our factor, of course, is the paste. Continuous factors must be classified into subgroups or levels. If the factor is discrete, as in our case, further classification may not be required as the subgroups naturally exist." "Now that you have defined the factor levels and have measured each response, how will you analyze this data?" one team member asked.

Improve25

Level 1 Wilsons 0.99265 1.06742 0.87233 1.17466 0.88709

Level 2 Acme 0.79582 1.12497 1.27382 1.20943 1.00341

Level 3 Twecho 1.70682 1.63187 1.70582 1.61013 1.27879

Factor Response

1.00341 1.70682 1.63187 1.70582 1.61013 1.27879

"We will look at the influence of the different levels both practically and statistically. To do this we will enter the data in Minitab, stack it, conduct a one-way ANOVA, analyze the sum of squares, create a main effects plot, and do a residual analysis."

Sum of Squares As Related To One-Way ANOVAAnalysis of Variance Source DF SS MS Factor 2 1.0127 0.5064 Error 12 0.3331 0.0278 Total 14 1.3458Level 1 Wilsons 0.99265 1.06742 0.87233 1.17466 0.88709 0.016 0.064 75% 25% Level 2 Acme 0.79582 1.12497 1.27382 1.20943 1.00341 0.036 0.143 Level 3 Twecho 1.70682 1.63187 1.70582 1.61013 1.27879

F 18.24

P 0.000

a St

ti

al ic st

1_05_01_025


Sum of Squares as Related to One-Way ANOVA An Engineer was evaluating three brands of soldering paste used on printed circuit boards. Fifteen tensile strength measurements were taken; five for each of the different types. To estimate the practical significance, the Engineer calculated the sum of squares within (SSW), between (SSB), and total (SST). By adding the SS for each of the levels (0.64 + 0.143 + 0.126) he established the SSW (0.33), this represented the error. By combining the response data for all of the levels he calculated SST. Next he subtracted the SSW from SST which resulted in the SSB. The SSB represented the source of variation or the variation attributable to the factor under study. He calculated a ratio of the SST compared to SSB and the SSW and found that the SSB amounted to 75% of the variation and SSW was attributable to 25% of the total variation. He concluded that the difference in the levels was an appreciable influence of variation. He reviewed his findings with the Black Belt. After reviewing the Engineer's analysis the Black Belt remarked, "Your analysis has revealed the practical influence of the factor under study. To better understand the implications of the differences you have uncovered we need to look at the data statistically." The Black Belt entered the data into Minitab, completed a one-way analysis of variance (one-way ANOVA) and reviewed the results with the Engineer. "The one-way ANOVA also calculates the SS within, between and total. Based on the SS values a mean square (MS) ratio, which is the F value, is derived. This report also generates a P value which also indicates that there is a statistically significant difference (P < 0.05) in at least two of the means differ from the rest. These values expose the statistical influence of the different levels. The practical influence of different levels can also be exposed using Minitab by taking the SS and calculating a ratio as we did in Excel. The practical can then be displayed using charts such as a pie chart."

Improve26

ct ra P

i

l ca 0.1261.01 0.33 1.35

0.032

Variance Column SS Column SS Between SS Within SS Total (Source) (Error) (Total)

One-Way ANOVA Table An Engineer is working on a project to maximize the tensile strength of the bond of circuitry on a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. The Engineer was unsure how to analyze the data and asked the Black Belt for help. After reviewing the data, the Black Belt suggested two possible analysis tools, the t test and the one-way ANOVA. "In Minitab we can run both of these tests. However, ANOVA has the advantage of comparing various levels, while the t test makes paired comparisons. It appears that the Twecho paste is producing a significantly higher tensile strength." "How do you know this?" the Engineer asked. "The one-way ANOVA output in Minitab produces a graphical representation of the mean confidence intervals and an F statistic and a P value. We can see that the confidence interval around the mean of the Twecho paste is completely outside the range of the confidence interval of the other two paste. The use of ANOVA has some underlying assumptions of the data; the population variances of the response are equal across all levels for a given factor, and the residuals are constant and normally distributed with a mean of zero and a constant, but finite variance." "How do I analyze the residuals?" the Engineer asked. "This can also be done in Minitab with the residual analysis tool."

One-Way Main Effects Plot

Level 1 Wilsons 0.99265 1.06742 0.87233 1.17466 0.88709

Level 2 Acme 0.79582 1.12497 1.27382 1.20943 1.00341

Level 3 Twecho 1.70682 1.63187 1.70582 1.61013 1.27879

Main Effects Plot - Means for Response

1.6

1.5

Response

1.4

1.3

1.2

1.1

1.0

1_05_01_027

Factor


Improve28

1

2

3

One-Way Main Effects Plot An Engineer is working on a project to maximize the tensile strength of the bond of the circuitry to a printed circuit board. Three different types of paste were suggested as possible solutions to the low tensile strength problem. Fifteen measurements were taken, 5 on each type of paste. To analyze the data the engineer created a one-way main effects plot. On the chart he depicted the average response for the three different levels, that is, the three different types of paste. Having completed the analysis the Engineer presented the results to the project Black Belt. "As you can see by this main effects plot the paste at level three provides us with the maximum tensile strength of the three types; this is the type that will be specified from now on." "This looks very good and seems to support the selection that you have made. However, do you have statistical evidence to support this selection?" The Black Belt asked. "Statistical evidence?" The Engineer asked. "The main effect plot is a very good graphical tool to help substantiate your conclusions; however, it should be used to compliment other tools. With this tool alone I can't be sure that the variation presented in the plot is significant. I need to know with a certain degree of confidence if the differences between the averages could have happened as a result of chance random sampling variation or not. To do this we need to first use tools such as one-way ANOVA."

Residual Model DiagnosticsNormal Plot of Residuals4 3 2 1 0 -1 -2 -3 -4 -5 -2 -1 0 1 2

I Chart of Residuals103.0SL=7.536

Residual

Residual

0

X=0.000

-3.0SL=-7.536

-10 0 5 10 15

Normal Score

Observation Number

Histogram of Residuals4 3 2 1 0 -5 -4 -3 -2 -1 0 1 2 3 4 4 3 2 1 0 -1 -2 -3 -4 -5 10

Residuals vs. Fits

Frequency

Residual

11

12

13

14

Residual

Fit

Residual Analysis An Engineer was performing a one-way ANOVA comparing three different types of soldering paste. The

Engineer remembered from his training that one of the underlying assumptions of the ANOVA is that the residuals are random and normally distributed. However, he did not know how to verify this. He asked the project Black Belt for help. The Black Belt explained. "When you are doing your one-way ANOVA in Minitab you need to store both the data's fits and residuals. This data is then used to construct a residuals plot." Together they generated a Residual Model Diagnostics plot. The Black Belt reviewed the plot and proclaimed, "Very good, it appears that your assumptions have been meet." "How can you tell?" the Engineer asked. "The two left charts, the Normal Plot of Residuals and the Histogram of Residuals, both give an indication of the normality of the residuals; both look normal. The I Chart for Residuals gives us an indication of change in the residuals through time. This chart implies that the points are randomly distributed through time; that is, the points are evenly distributed above and below the central line, most points fall near the central line, some points approach the control limits, and no points are beyond the control limits. The last chart, Residuals vs. Fits, gives us evidence of changes in residual variation at the different levels. Each group appears to be similar in variance."

The Tw o-Level Tw o-Factor (2 D esign - 4 R epresentationsG eom etricH H IG

2)

StandardLow Interior Space

A High

B

Low BH H IG

(1) b

a ab

LO W

A

High

TreeLow

A lgebraic AHigh Run O rder

A

B

1 2 3 4

1_05_01_029

Low

BHighExperim ent D esign Tools

The Two-Level Two-Factor (22) Design 4 Representations "A two-factor, two-level design can be represented in at least four different ways" says a Master Black Belt to a Champion. In the "Geometric" representation, the corners of the square represent the levels of factors A and B, from "lowlow", to "high-high". The "Standard", or Yates designation (in the upper right image), shows the notation of all treatments taken from the treatment combination. "a" indicates "A high" and "B low", "b" indicates "B high" and "A low". "ab" indicates that both factors (A and B) are high. The notation used in the Yates designation refers to the factor set at the

Improve30

Design

-1 +1 -1 +1

-1 -1 +1 +1

high level, except for the case of (1), which refers to both factors (A and B) being set at the low level. The "Tree", also called "Taxonomy" or "Characterization Chart", illustrates the experiment design for factors A and B, with their respective levels (low and high). The fourth image, called "Algebraic" form is used to represent the settings used for each run (from 1 to 4), with a plus (+1) or minus (-1) sign to denote if the factors are set to their high or low settings. The Champion recognizes that the "Algebraic" and "Yates Standard Notation" are directly related to the runs of the experiment. The Master Black Belt adds: "the "Algebraic" notation respects Yates Standard Order, and is the most common representation of full factorial experiments".

Other Factorial Design Notation Matrix (22)Two-Level Two-factor DesignSign NotationRun No.

Alternative Notation

Japanese Notation

Classical Notation

A+ +

B+ +

A0 1 0 1

B0 0 1 1

A1 3 1 2

B1 1 2 2

A(1) a b ab

B

1 2 3 41_05_01_030


Other Factorial Design Notation Matrix (22) A Black Belt asks her Master Black Belt: "In our DOE, we always identify individual trials with 1 or 1 . I have just read a book where the author is using another designation. Is there any difference and advantages in using one compared to the others?" k The Master Black Belt replies: "Remember, the 2 factorial designs, i.e. a factorial experiment with k factors, k each at two levels, has widespread industrial applicability. The experiment consists of 2 trials, one trial at each combination of levels of the factors. To identify the individual trials, different notations are used. One convention is to label each factor by a letter (or numeral) and then to denote the two levels (versions) of each factor by a plus (+) and a minus (-) sign. Commonly the minus sign refers to lower level, the standard condition, or absence of the factor (e.g. not lubricated vs. lubricated bearing). Thus if there are two factors at two levels, labeled A and

Improve31

2 B, the four trials comprising 2 factorial design can be represented as shown in this table." "You are familiar with the 1, 1 notation used in Minitab, which is similar to the sign notation. Alternative notations are to employ 0 and 1, respectively, or, following the Japanese tradition earlier established by Taguchi, 1 and 2 for the two version of each factor. The classical convention is to denote the two versions of each factor by the presence and absence of its corresponding lowercase letter, here the trial in which all factors are at their "low" level is denoted by (1)." The Master Black Belt adds: "The convention we have adopted is to use the -1, 1 notation, which makes it possible to identify, factor levels and contrast coefficients used to calculate factor and interaction contrasts when performing DOE analysis."

Yates Standard Order (2 2)Quenching Speed = A Low: 1 m/s = High: 2 m/s = m/s = meter per second Aging Time = B Low: High: Bath = C Low: High: 4min = 8min = Water = Glycol = FactorsRun Order

-1 +1

Yates Standard Order Table (2FactorsRun Order Treatment combination

3)

-1 +1 -1 +1

A-1 +1 -1 +1 -1 +1 -1 +1

B-1 -1 +1 +1 -1 -1 +1 +1

C-1 -1 -1 -1 +1 +1 +1 +1

AQuenching Speed

BAging Time

CBath

1_05_01_031

Settings for a two-level two-factor design


Yates Standard Order (22) A Six Sigma team conducts an experiment in the heat treatment process. Three factors are identified: Quenching speed (A), Aging time (B), and Type of bath (C). The levels of each factor are set as follows: Quenching speed (factor A): Aging time(factor B): Bath type (factor C): Low: 1 m/s Low: 4 min Low: water (-1) (-1) (-1) High: 2 m/s High: 8 min High: Glycol (+1) (+1) (+1)

"When designing the experiment, the Yates Standard order is the one most often used", says the Master Black Belt. We used a two-level, three-factor design. For each run, the corresponding factor level is identified as -1 when the factor is set at "low", and as +1 when it is set at "high". The first combination shown in the Yates Standard Order will be a treatment with Quenching speed set at -1 (low, or 1 m/s), Aging time at -1 (low, or 4 min.), and Type of bath set at -1 (low, or water). This run is

Improve32

1 2 3 4 5 6 7 8

1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s 1 m/s 2 m/s

4 min 4 min 8 min 8 min 4 min 4 min 8 min 8 min

Water Water Water Water Glycol Glycol Glycol Glycol

1 2 3 4 5 6 7 8

(1) a b ab c ac bc abc

th represented in the first row by the numbers: -1, -1, -1. All other runs are set in the same fashion, up to the 8 run where the settings are +1, +1, +1. The Master Black Belt explains that it is important to randomize the Yates Standard Order prior any experiment in order to average out the noise caused by nuisance variables if they exist. The Master Black Belt explains that the same table can be used to plan an experiment for a two-level, two-factor design. The team recognizes that using this Yates Standard Order can help them avoid errors in preparing an experiment.

Computing Contrasts and Effects (22)Ohms LawVoltage = Resistance x AmperageSpecifications

Uv Y

= R * I A = A*B

Response Y: V Maximize Factor A: 6.0 50% Factor B: 1.0A 50% Y

Factor Low High -1 +1 A 3.0 9.0 B 0.5 1.5

Main Effect Plot for B

Run Order 1 2 3 4 (1) a b ab Contrast Effect Avg +1_05_01_032

A -1 1 -1 1 12.0 6.0 9.0 3.0 6.0

B -1 -1 1 1 12.0 6.0 9.0 3.0 6.0

AB 1 -1 -1 1 6.0 3.0 7.5 4.5 3.0

Y1 1.5 4.5 4.5 13.5

9.0

3.0

Avg


Computing Contrasts and Effects (22) A Six Sigma team conducts an experiment. Two factors are considered (Resistance and Amperage), and the objective is to maximize output (voltage). Resistance (factor A), is specified at 6.0 ? ? 50%, and Amperage (factor B) at 1.0 A ? 50%. The team defines the factor settings at Alow = 3.0 ?, Ahigh = 9.0 ? and Blow = 0.5 A and Bhigh = 1.5A. The experiment is designed according to Yates Standard Order for one replicate only, and the output (voltage) is recorded for each run (e.g. Run #3: Voltage equals 4.5 V for a Resistance of 3.0 ? and an Amperage of 1.5 A). The Black Belt computes the contrasts for each factor (e.g. 12 for factor A), and the corresponding effects (6.0 for both factor A and B). Similarly, he calculates the effect for the interaction of the two factors (3.0 for the interaction AB). The Black Belt explains that since the effect of B is equal to +6.0, the voltage (output) increases by an average of 6 volts when the amperage (factor B) increases from low to high. Since the "B effect" is positive, the voltage (output) will vary in the same direction as the direction of change of factor B.

Improve33

A and AB effects are neutralized

Factor B -1 +1 B B+ (A- & A +) (A- & A +) (AB - & AB +) (AB - & AB +)

The team concludes that to maximize voltage (output), Amperage (factor B) must be kept at the high level. The Black Belt highlights the fact that with this full factorial design, each contrast (A, B or AB), represents the effect of only one particular factor or interaction since the other two are neutralized by the combination of + and signs. In other words, the contrast of A represents variation due only to factor A since B and AB are neutralized. Finally, the Black Belt reminds the team of the characteristics of the contrasts: a) Balanced, and b) Orthogonal, to ensure independence of effects.

Computing the Sums of Squares and Practical Significance (22) "Why do we calculate the contrasts"? Ask a team member. "Contrasts are part of the foundation of Analysis of Variance", replies the Black Belt. They make it possible to compute the effects and the Sums of Squares. In our two-level, two-factor design, the contrasts for factors A and B, along with their interaction AB allow us to compute the Sums of Squares. In our experiment we see that contrast A is positive, meaning that the output is higher when we set factor A at the high level compared to when it is set at the low level." "What is the purpose of the pie chart?", asks another team member. "This chart, created from the Sums-of Squares in the ANOVA table, shows the practical significance of the sources of variation" replies the Black Belt. "In our experiment, SSfactor A accounts for 52%, SSfactor B for 38% and SSinteraction A*B for 10% of the SStotal (i.e. the total change in the response variable. We can also conclude that 90% of the variation of the response variable (CT Characteristic) is explained when factors A and B vary between low and high settings. The team recognizes the advantage of using a pie chart to graphically represent the practical significance of the effects exerted by the factors, and note that the remaining 10% represent the interaction effect (factors AB). Finally, the Black Belt indicates that a Pareto Diagram can also be used instead of a Pie Chart. The Black Belt recalls that before considering the practical significance we must evaluate the statistical significance of the different effects.

Plotting the Main Effects (2 2)EffectA = y = y A + yA replicates taken atthe treatment Effect A = 1 { contrast A } combination 2n_ _

n=Number of

or

Y 10.5

Main Effect Plot for AAverage Y at High condition of factor A

3.5Analysis of Factor AY15.0

Average Y at Low condition of factor A

Y Main Effect Plot for B10.0 +1 4.0Average Y at High condition of factor B Average Y at Low condition of factor B

B:+1

-1

Factor A

Y

10.56.0

B:+1 B:-1

1_05_01_034

3.5

5.0

2.0

B:-1 A:-1 A:+1

-1

Factor B

+1


Plotting the Main Effects (22) After conducting an experiment and recording the responses, the team asks the Black Belt to explain the next steps. We perform an analytical procedure by calculating and plotting the Main Effects. In our experiments we can see that when factor A is high (+1), the average output equals 10.5, and when it's low (-1), the average equals only 3.5. The change in the output i.e. 7.0 can only be attributed to the difference between the levels of factor A. This is because the settings neutralize both the effects due to factor B, and the interaction AB by the combination of their and + at each level of A. This is the result of contrasts that are balanced and orthogonal in well-designed experiments. The Black Belt then asks "Which settings maximize output, and what will be the output level for such settings?" A team member answers "When factor A is set at high, the output average is 10.5". "You are partially correct", says the Black Belt, "however, we cannot only consider factor A. We must also consider factor B. Another team member adds "Maximum output is 15.0 when factor A is high and factor B is also high. "This is the correct and complete answer" says the Black Belt. The team calculates the effect of factor A and then they conclude that factors A and B have the different effects. Finally, they acknowledge the advantage of plotting the main effects to depict the results of the experiments, and to visualize the difference in the response output between low and high settings for each factor.

Improve35

Plotting the Interaction Effect (2_ _ n=Number of replicates taken at the treatment combination

2)

Effect

or

AB AB

= y = y AB + y AB = 1 { contrast 2nAB

Effect

}

Interaction Plot for ABY15.0

Both lines are not parallel thus indicating an interaction effect6.0 5.0

B:+1

Analysis of Interaction AB Y15.0 B:+1

B:+1

B:-1

Y

8.5 5.5 6.0 5.0 B:+1 B:-1 A:-1 A:+1

1_05_01_035

2.0

B:-1 A:-1 A:+1

2.0


Plotting the Interaction Effect (22) "During the analytical procedure of the experiment, we extract relevant information" explains the Black Belt, "A major part of this is to determine if there is an interaction effect between factors". "Interaction occurs when the effect of a factor is a function of the level of another factor. For example, when factor A changes from its low to high level, and factor B is kept at its low level, output changes by an average of 4.0 units (from 2.0 to 6.0). When B is kept at its high level and factor A varies from its low to high level, the output changes by an average of 10.0 units (from 5.0 to 15.0). This is an indication of interaction between factors A and B, i.e. they have a combine effect on the output." "If the lines in the interaction plot are parallel, then there is no interaction between the factors. Conversely, if the lines are not parallel, an interaction exists between the factors. Note that graphs should be used to visualize the result of the ANOVA table. Graphs are useful to visualize significant interactions between factors but they should not be the only means of interpreting data." "Do interactions apply to transactional processes"? ask a team member. "Yes", replies the Black Belt, "I remember a project where we analyzed invoices that were overdue. The first factor was "Customer", and two levels were considered: "Airlines" and "Manufacturers". The second factor was "Invoice value". We were surprised to discover that there was a important interaction effect between the two factors (Invoice value and Customer). The Black Belt adds: "two factor interactions are common. However, three factor and higher are rarely seen in practice in our type of industry.

Improve36

B:-1

Statistical Significance and Practical Significance (2 2)Output Run 1 2 3 4 Standard Order (1) a b ab A -1 1 -1 1 B -1 -1 1 1 AB 1 -1 -1 1 Y1 40.7 74.2 44.1 72.6 Y2 38.6 68.4 42.1 86.0 Y3 38.4 64.4 41.1 85.6

St Si atis gn ti ifi ca ca lly ntB 5% A 89%

Source A B A*B Residual Error Total

Degree Of Freedom 1 1 1 8 11

Sums-ofSquares 3,543.20 182.52 63.48 172.71 3,961.92

Mean Squares 3,543.20 182.52 63.48 21.59

F-value 164.12 8.45 2.94

p-Value 0.0000 0.0197 0.1247

Significant

*** ***

A*B 2%

Residual Error 4%

Source A B A*B Residual Error Total


Statistical Significance and Practical Significance (22) An engineer consults with a Black Belt after conducting a two-level, two factor experiment (factors A and B). Three replicates per setting were done, and now he evaluates the statistical significance of the main effects for each factor and the effect of interaction between factors A and B. The Black Belt states that the engineer correctly conducted a two way ANOVA using Minitab. "The F test indicates p values for factors A and B are smaller than 5%. This means that changes in factors A and B significantly affect the output of the process. It is a clear indication that the change is not due to chance, and that the condition can be reproduced. It is what we call Statistically Significant". The Black Belt also highlights the importance of verifying the practical significance of the results using the Sums of Squares. "This is called Practical Significance and in this case changes in factor A represent 89.4% of the total variation." The engineer recognizes that although factor B is Statistically Significant it does not account for more than 5 % in practical terms, thus with little interest for him at this stage.

Improve37

1_05_01_036

Sums-ofSquares 3,543.2 182.5 63.5 172.7 3,961.9

Percentage % 89% 5% 2% 4% 100%

Practically Significant

Residual Analysis (2 2)Normal Plot shows no non-normality Shows no residuals out of control beyond the control limits

Residual Model DiagnosticsNormal Plot of Residuals3 2 1 0 -1 -2 -3 -2 -1 0 1 Normal Score 2 5 4 3 2 1 0 -1 -2 -3 -4 -5

I Chart of Residuals3.0SL=3.909

Residual

Residual

X=0.000

-3.0SL=-3.909

0

5 10 Observation Number

Histogram of Residuals4 Frequency 3 2 1 0 -3 -2 -1 0 1 2 Residual1_05_01_037

Residuals vs. Fits3 2 Residual 1 0 -1 -2 -3 40 50 60 Fit 70 80 90

Histogram of Residuals normally distributed around 0Experiment Design Tools

Variable does not change across values of the Output

Residual Analysis (22) While analyzing his first 2 x 2 factorial design, a Black Belt notices that in Minitab he can graph the residuals. To refresh the concept of residuals, the Black Belt consults with a Master Black Belt. "When conducting a DOE, our objective is to test the appropriate hypotheses regarding the treatment of effects, and to estimate them. The models of the errors are assumed to be independent random variables that are 2 normally distributed with mean equal to zero and variance ? , which is assumed constant for all levels of the factors." The Black Belt notices: "First, we must verify that residuals are normally distributed. The normal plot shows dots aligned on a straight line indicating that residuals are indeed normally distributed." "Second, we verify that residuals are normally distributed center around zero. The histogram of the residuals shows this normal distribution and the centering around zero." "Third, we consider the Individual Chart of Residuals. The dots are randomly distributed, no data out of the control limits and no particular pattern, thus we may conclude that the residuals are randomly distributed regarding the experiments run order." "Finally, we check the random distribution of the residuals versus the settings. The fourth graph shows no particular pattern and the dots are randomly distributed, thus we conclude that the residuals are normally distributed." The Black Belt remembers the importance to analyze residuals prior to any conclusions about experiment result.

Improve38

Modeling - 2

2

Estimated Effects and Coefficients for yieldTerm Constant A B A*B Effect 34.367 7.800 4.600 Coef 58.017 17.183 3.900 2.300 StDevCoef 1.341 1.341 1.341 1.341 43.25 12.81 2.91 1.71 T 0.000 0.000 0.020 0.125 P

Analysis of Variance for yieldSource Main Effects 2-Way Interactions Residual Error Pure Error Total DF 2 1 8 8 11 Seq SS 3725.72 63.48 172.71 172.71 3961.92 Adj SS 3725.72 63.48 172.71 172.71 Adj MS 1862.86 63.48 21.59 21.59 F 86.29 2.94 P 0.000 0.125


Modeling 22 "The results of the DOE quantify the effect that the two factors and the interaction have on the process response" says the Champion. He then continues, "are we limited two these factor settings, or is it possible to know more about the process behavior?" "It is correct to say that the DOE analysis gives us a good knowledge about the factor effects and their interaction" answers the Black Belt. "However, there are other things to understand. From the mathematical model we can see that if no change is made, then our process will have an average yield of 58.01% . We can also see that the level of Factor A (Temperature) is key in controlling the yield of the process as compared to Factor B (Pressure)." "The average response output Yield when Temperature (A) and Pressure (B) are set to their High level is computed using the model: Y = 58.017 + 17.183*1 + 3.9*1 = 79.1. With a similar calculation, we can determine that when A is High and B is Low, the average yield will be Y = 58.017 + 17.183*1 - 3.9*(-1) = 71.3." "We usually assume a linear response, in such a case this mathematical model is valid", says the Black Belt. "To confirm this linearity, we use other techniques such as center points or multi-level factorial design" The Champion recognizes the advantage to represent the process behavior using a mathematical model which makes it possible to estimate process output for factor levels other than the pre-established settings.

Improve39

Y =58.017 +17.183 * A +3.9 * B1_05_01_038

Multi-Factor Experiment

On Lo Off In Out Hi Front Back Left Right

Up

Down

2 2

2

2

2 2

2Improve40

1_05_01_039

5 factors : 2 5 = 32 combinations 6 factors : 2 6 = 64 combinations 7 factors : 2 7 = 128 combinationsExperiment Design Tools

Multi-Factor Experiment A Black Belt from a manufacturing area is discussing her project with a Master Black Belt. "During the team's brainstorming session, we have identified several factors that can influence our CTQ characteristic, the angularity of the parts formed on the Fluid Cell Press. We would like to do an experiment with at least 5 factors, but this would be too large an experiment to do a full factorial. We had identified these factors a long time ago but we don't know yet which ones have the biggest impact, "leverage variables". So far, I have calculated that with these 5 factors at two levels, I will need 32 runs. With 6 factors, 64 runs and, 7 factors will require 128 runs and I have not included any replications yet. What do you recommend I do?" The Master Black Belt replies, "if there are many factors in the experiment, a full factorial experiment will involve a large number of tests, even when only two levels of each factor are being investigated. So, when you don't know which factor has a substantial effect on a response variable, and you want to verify several factors, then the fractional factorial design is the preferred analytical tool. In fact, it is common practice to do a fractional factorial experiment when we have 5 factors or more."

Fractional Factorial Designs

Full Factorial

Fractional Factorial

Example with 4 FactorsMain EffectsA B C D1_05_01_040

InteractionsAB, AC, AD, BC, BD, CD ABC, ABD, ACD, BCD ABCD

Main EffectsA B C D

InteractionsAB, AC, AD, BC, BD, CDImprove41


Fractional Factorial Designs A Black Belt is presenting her Design of Experiment plan to a Champion: "For my project on the Fluid Cell Press, I will use a fractional factorial design. We want to study four factors at two levels. Instead of doing a full factorial 4 design, which represents 16 runs (2 ) and is very time consuming, we will do a half fraction of it, which consists of 8 runs." "I don't understand how you can do that. How will you be able to test all the possible factor combinations if you do not use a full factorial experiment?", asks the Champion. "A fractional factorial design is a carefully prescribed subset of all possible combinations of a full factorial design. Statisticians have shown that for 5 factors or more, you can obtain nearly the same information as a full factorial with a well-designed fractional factorial experiment. When compared to a full factorial design, a certain amount of information will be lost, but the impact is minimal. The trade-off is mainly that we lose information regarding the interactions between three factors or more. However, for most situations in industry, it is considered reasonable to assume that third order and higher order interactions (interaction between three factors or more) are statistically negligible." "The benefits of conducting a fractional factorial experiment are that fewer tests are required compared to the full factorial design, reducing the required time and resources, and yet we can still assess many factors at the same time."

Forming a Half-Fraction of a Full Three Factor DesignRun A 1 2 3 4 5 6 7 8 -1 1 -1 1 -1 1 -1 1 B -1 -1 1 1 -1 -1 1 1 C -1 -1 -1 -1 1 1 1 1 AB 1 -1 -1 1 1 -1 -1 1 AC 1 -1 1 -1 -1 1 -1 1 BC ABC 1 1 -1 -1 -1 -1 1 1 -1 1 1 -1 1 -1 -1 1

Generator: ABC=1

Same Contrast

Run AConfounding : A & BC or alias B & AC structure C & AB

B -1 1 -1 1

C -1 -1 1 1

AB -1 -1 1 1

AC -1 1 -1 1

BC ABC 1 -1 -1 1 1 1 1 1


Forming a Half Fraction of a Full Three Factor Design "I know that Minitab creates fractional factorial designs, but for my own understanding I am interested in knowing how to construct one. Could you explain this to me?", asks a Black Belt to a Six Sigma Instructor. "OK, we will take a simple example with a three factor design. For most situations in our industry, it is considered reasonable to assume third order interactions to be statistically negligible, so we will use the ABC interaction to create the fraction. First, we have to sort the entire matrix by the highest order interaction. In this instance, we must sort on the ABC column. The fractional design is formed by selecting all those treatment combinations where the ABC effect is either "1" or "-1"." "The sign of the ABC effect constitutes a "block". Therefore, the design generator can be ABC=-1 or ABC=1. In our case, if we choose the generator ABC=1, we will keep only the runs 2, 3, 5 and 8. Look what happens to the matrix. First, the ABC effect is lost since it was used as the generator to form the half fraction of the full factorial matrix. Second, the unique effect of "A" can not be separated form the unique effect of the BC interaction because both columns have the same sign at each experimental run. The same thing happens for B with AC, and C with AB. This is called confounding or alias structure. Note that I could have chosen ABC=-1 as the design generator and arrived at the same conclusions." "The design becomes these 4 runs and we only need the three columns for A, B, and C. The other columns constitute a repetition of these three. For three factors with this design, the main effects are confounded with the two factor interactions."

Improve42

1_05_01_041

2 3 5 8

1 -1 -1 1

Design Resolution

Resolution III IV

Main Confounding Pattern

Use

Main effects are confounded To screen many factors with second order interactions (Screening Designs) Second order interactions are To study uncomplicated confounded with other second effects and interactions order effects interactions (Characterization Designs) Second order interactions are confounded with third order interactions Same as resolution IV. For resolution V and higher: to study more complicated effects involving 2 or 3 factors and obtain amathema tical model. (Optimization Designs)

V


Design Resolution "In Minitab, if I use the function "Create Factorial Design" (under Stat > DOE), I have noticed that for a specific number of factors that we want to study, different types of designs are available. I am not sure that I understand the terminology associated to the designs, could you explain this to me?", asks a Black Belt to a Master Black Belt. "Yes, sure! Depending on the number of factors and runs, the resolution defines the alias or confounding structure of a design. For example, a resolution III design means that the main effects are confounded with interactions between factors." "That design doesn't seem to be very useful", says the Black Belt. "Well, these designs are called Screening Designs and a small number of runs are required to use them", replies the Master Black Belt. "However, the quantity of information is limited, so these designs are generally used to initially screen a large number of factors, as their name implies. Another experiment is then needed to study, in more detail, the important factors that have been found with the resolution III design and their interactions effects." "With resolution IV and V designs, main effects are not confounded with interactions but interactions are confounded together", continues the Master Black Belt. "We consider them as Characterization Designs and they are generally used to study relatively uncomplicated effects and interactions from a smaller number of factors compared to resolution III designs. Resolution v and higher designs are called Optimization Designs and are most often used to study more complicated effects and relationships involving two or three factors. Such designs are also employed to obtain a mathematical model describing the relationship between the CT characteristic and the factors (Y = f(X1, , XN)).

Improve43

1_05_01_042

Choosing a Factorial DesignNumber of Runs* 2 4 8 16 32 64 128 full 3 III full IV full III V full III IV VI full III IV IV VII full IV IV V VIII Number of factors 4 5 6 7 8

2

6 2IVImprove44

1_05_01_043

*Note: the number of runs is for 1 replication.Experiment Design Tools

Choosing a Factorial Design "For my project on the automatic riveting machine, six factors have been selected as potential Critical to Process characteristics. For each of them, we have identified two levels (high and low) that can be used in an experiment. What design do you suggest I use?" asks a Black Belt to a Master Black Belt. "For six factors, there are four types of factorial designs that are available: a resolution III design which consist of 8 runs, a resolution IV design with 16 runs, a resolution VI with 32 runs and a full factorial with 64 runs. The design you select will depend on what you need to know, your level of knowledge of the process and resource availability." The Black Belt replies, "well, it is my first experiment on this machine. From the Analysis Phase certain factors appeared to have an influence on our CTQ, but we don't really know much about them. Interactions between factors are possible but we don't know if they exist." "In this situation, you don't have a lot of knowledge about the factors and you are definitely not at an optimization stage, I suggest you use the resolution IV design with 16 runs, replies the Master Black Belt. Don't forget that you will have to include a few replications which will significantly increase the amount of runs." 6-2 "Why is this design called a 2 ?" asks the Black Belt. "This refers to the notation that is used to describe k-p factorial design. In general we use 2 where "2" indicates that the factors are studied at two levels, "k" represents the number of factors and "p" what fraction of a full factorial the design is. Then, p=1 it means a fraction of the full factorial, p=2 means a fraction and so on. In your case, p=2 means that your design is fraction of a full factorial for 6 factors at two levels."

Objective, Response Variables and Factors

CTQ target: 90 O

CTQ: Angularity

Machine

Tools

Materials

Angularity of the partsPeopleExperiment Design Tools

Methods

Environment

Objective, Response Variables And Factors A Black Belt presents her Design of Experiment plan to a group of Champions. "The objective of the Design of Experiment is to identify the factors that affect the quality of the parts formed in the Fluid Cell Press. The CTQ characteristic that we want to improve is the angularity of the parts. This CTQ will be measured on two typical parts that we produce: "curved" and "straight" parts. Two separate statistical analyses will be done, one for each type of part." "We regularly rework the parts produced in this process. Can you please clarify the problem with the CTQ?", asks a Champion. "Sure", replies the Black Belt. "The parts produced have an angle that do not meet the target of 90 degrees. An operation of check and straighten has to be done practically on every part to bring the CTQ on target." "Do you have a good measuring system to obtain accurate data for your DOE?", asks the Champion from the Quality department. "Yes, we have organized a location where all the parts will be measured, and a Gage R&R study has been conducted with the team members who will measure the parts. The result of the Gage R&R study detected a problem with the measuring system, but it has been corrected." "The Black Belt continues, "with the information gathered since the beginning of the project and a recent brainstorming session, we have identified more than 60 causes that can affect the angularity of the parts. We used a fishbone diagram to summarize the causes in the following categories: Machine, Tools, Materials, People, Methods, and Environment."

Improve45

1_05_01_044

Select Factors and LevelsFactors A: Loading of tray B: Height of blocks C: Grain direction D: Covering type E: Pressure F: Forming Speed G: Spring-back Levels Low (-1) High (1) 20% 50% 1/4 1 1/4 45 90 Soft Hard 8,000 PSI 15,000 PSI Normal Slow 2 4Y True Effect

In general, the levels should be set wide but not so much that they extend beyond the operational range of the factor.

Response

Experimental Effect Lo (-1) Hi Factor Settings (+1)

Exp

Documents

I1 - Experiment design Tools