Diploma in Statistics Design and Analysis of Experiments Lecture 1.11 Diploma in Statistics Design and Analysis of Experiments Lecturer:Dr. Michael Stuart,

Diploma in StatisticsDesign and Analysis of Experiments

Lecture 1.1 1


Lecturer: Dr. Michael Stuart,

Department of Statistics

Office: LB 101

email: [email protected]

Too short; cover parts 1-3 in first half; need more for the 2nd half.

Try boys shoes, comparing two t-tests


Lecture 1.1 2

Design and Analysis of ExperimentsCourse Outline

• Experimental and observational studies

• Basic design principles for experiments

– Randomisation

– Blocking (pairing)

– Factorial structure

• Standard designs, illustrated


Lecture 1.1 3

Design and Analysis of ExperimentsCourse Outline

• Analysis of experimental data

– Exploratory data analysis

– Parameter estimation and significance testing

– Analysis of variance

– Model validation, diagnostics

• Computer laboratories

• Strategies for Experimenting


Lecture 1.1 4

Design and Analysis of ExperimentsReferences

Mullins, E., Statistics for the Quality Control Chemistry Laboratory, Royal Society of Chemistry, 2003, particularly Chapters 4-5, 7-8.

Detailed coverage of much of the module, in a specific context.

Montgomery, D.C., Design and analysis of experiments, 6th ed., Wiley, 2005.

A comprehensive text, covers much more than this module, including statistical theory. Not always authoritative.


Lecture 1.1 5

Design and Analysis of ExperimentsFurther reading

Box, G.E.P, Hunter, J.S. and Hunter, W.G., Statistics for Experimenters, 2nd. ed., Wiley, 2005.Includes many gems of wisdom from these masters of the genre, though not a course text.

Daniel, C., Applications of Statistics to Industrial Experimentation, Wiley, 1976.Includes many gems of wisdom from this master of the genre, using methodology appropriate for an industrial setting.

Altman, D.G., Practical Statistics for Medical Research, Chapman & Hall / CRC, 1991.Does what it says on the tin!


Lecture 1.1 6

Lecture 1.11. Introduction to Course

2. Case study on process improvement

statistical assessment of a process change

strategy for experimentation

3. Experimental vs observational study

another illustration

4. Multifactor designs

efficiency

interaction


Lecture 1.1 7

2 Case study on process improvement

• Comparison of standard (old) and new processes for manufacture of electronic components

• Key issues

– homogeneity for valid comparison

– systematic allocation

– random allocation


Lecture 1.1 8

Experimental design

• 50 components sampled per day,

• 6 days per week,

• 8 weeks,

• Systematic layout, as follows

Week Number

1 2 3 4 5 6 7 8

Monday Old New Old New Old New Old New

Tuesday New Old New Old New Old New Old

Wednesday Old New Old New Old New Old New

Thursday New Old New Old New Old New Old

Friday Old New Old New Old New Old New

Saturday New Old New Old New Old New Old


Lecture 1.1 9

Results

Day Defectives Day Defectives Day Defectives Day Defectives 1 0 13 1 25 0 37 2 2 0 14 0 26 0 38 0 3 6 15 3 27 0 39 0 4 3 16 1 28 2 40 0 5 3 17 0 29 0 41 0 6 3 18 2 30 0 42 0 7 4 19 0 31 1 43 1 8 1 20 1 32 1 44 0 9 0 21 2 33 0 45 2 10 2 22 0 34 0 46 0 11 0 23 1 35 0 47 0 12 0 24 3 36 2 48 0

Numbers of defectives per daily sample of 50for 48 days (8 weeks)


Lecture 1.1 10

Comparison of two processes over eight weeks:data for first four weeks

Number of Defectives

in Samples of 50 Units

Week Day pair

Old Process

New Process

Difference (New – Old)

1 1 0 0 0 1 2 6 3 –3 1 3 3 3 0 2 4 1 4 +3 2 5 2 0 –2 2 6 0 0 0 3 7 1 0 –1 3 8 3 1 –2 3 9 0 2 +2 4 10 1 0 –1 4 11 0 2 +2 4 12 3 1 –2


Lecture 1.1 11

Comparison of two processes over eight weeks:data for last four weeks, with eight week summary

Number of Defectives

in Samples of 50 Units

Week Day pair

Old Process

New Process

Difference (New – Old)

5 13 0 0 0 5 14 0 2 +2 5 15 0 0 0 6 16 1 1 0 6 17 0 0 0 6 18 2 0 –2 7 19 2 0 –2 7 20 0 0 0 7 21 0 0 0 8 22 0 1 +1 8 23 0 2 +2 8 24 0 0 0 Total 25 22 –3

8 week Average, per 50 units 1.04 0.92 –0.13 8 week Average, per cent 2.08 1.83 –0.25


Lecture 1.1 12

Differences in numbers defective,with control limits

4 8 12 16 20 24

Day Pair

-8

-6

-4

-2

0

2

4

6

8

Difference

No statistical significance!


Lecture 1.1 13

Alternative design(proposed by engineers)

Week Number

1 2 3 4 5 6 7 8

Monday Old Old Old Old New New New New

Tuesday Old Old Old Old New New New New

Wednesday Old Old Old Old New New New New

Thursday Old Old Old Old New New New New

Friday Old Old Old Old New New New New

Saturday Old Old Old Old New New New New

Assume this design was used;

check for no effect


Lecture 1.1 14

Defect rates, per cent, with differences,for the first and second four week periods

First

Period Second Period

Difference

Both Processes 3.0 0.9 2.1

Old Process 3.3 0.8 2.5

New Process 2.7 1.0 1.7


Lecture 1.1 15

75.3

56.0

1.2

1200

1.999.0

1200

973

9.00.3

n

)P̂100(P̂

n

)P̂100(P̂

P̂P̂Z

2

22

1

11

21

Defect rates, per cent, with differences,for the first and second four week periods

highly statistically significant!


Lecture 1.1 16

Exercise

Assess the statistical significance of the difference in defect rates, %, between the first period and second period for the old process.

Homework

Assess the statistical significance of the difference in defect rates, %, between the first period and second period for the new process.


Lecture 1.1 17

Numbers defective in time order

Long term downward trend,

systematic bias

How can this be?

6 12 18 24 30 36 42 48

Day

0

1

2

3

4

5

6

Defectives


Lecture 1.1 18

How to avoid systematic bias

• Make comparisons under

homogeneous experimental conditions

• 1 Systematic arrangement, as implemented:

avoids known biases

• 2 Random allocation:

within each day pair, allocate old and new processes at random

avoids known and unknown biases


Lecture 1.1 19

Two design principles

• Blocking

– identify homogeneous blocks of experimental units

– assess effects of experimental change within homogeneous blocks

– average effects across blocks

• Randomisation

– allocate experimental conditions to units at random


Lecture 1.1 20

Strategy for ExperimentationThe SIPOC Process Model

Cu

stom

ers

Process

Su

pplie

rs Inputs Outputs

S I P O C


Lecture 1.1 21

Strategy for ExperimentationStatistical Thinking

Cu

stom

er

Process

Su

pplie

r Inputs Outputs

Process management and

improvement

Input measures

Process measures

Process changes Output

measures

Supplier performance

Customer Feedback


Lecture 1.1 22

Strategy for ExperimentationShewhart's PDCA Cycle

Check

Act

Plan

Do


Lecture 1.1 23

Strategy for ExperimentationShewhart's PDCA Cycle

• Plan: Plan a change to the process, predict its effect, plan to measure the effect

• Do: Implement the change as an experiment and measure the effect

• Check: Analyse the results to learn what effect the change had, if any

• Act: If successful, make the change permanent, proceed to plan the next improvement

or

if not, proceed to plan an alternative change


Lecture 1.1 24

Strategy for Experimentation:new vs old manufacturing process

Plan:

• Compare defect rates for old process and new (cheaper) process

– predict reduction, or no increase, in number of defectives using new process

• Sample output over an eight week period, six days per week

– select 50 components at random per day

• Count number of defectives per sample


Lecture 1.1 25

Do:

• Implement plan

• Record daily numbers of defectives

Assessing experimental process for manufacturing electronic components


Lecture 1.1 26

Check:

• Analyse data

• test statistical significance of the change



Lecture 1.1 27

Act:

• If no worse, make the change permanent,

– proceed to plan the next improvement

or

• if not, proceed to plan an alternative change



Lecture 1.1 28

3 Observational vs Experimental study

Alternative design:

• sample 1200 components from old process inventory,

• sample 1200 components from new process inventory,

• compare


Lecture 1.1 29

Example 2: walking babies

•How long does it take a baby to walk?

•Can this be affected by special training programs?

4 "training" programs:

1. special exercises

2. normal daily exercise

3. weekly check

4. end of study check

each of 24 babies allocated at random to groups of 6 in each program.


Lecture 1.1 30



Lecture 1.1 31


Alternative design:

each of 4 different consultants prescribes one of the four training programs,

select a sample randomly from babies assigned to each program.

Problems:

assignment of babies to programsequivelent to

assignment of mothers to consultants

lurking variables!


Lecture 1.1 32

Walking babiesvs

Defective components

Level of control:

less control

means

more variation


Lecture 1.1 33

4 Multi-factor experiments

• Traditional versus statistical design

– efficiency

– interaction

• Several levels

• Several factors


Lecture 1.1 34

Illustration of a traditional design,with 12 experimental runs

Pressure

Temperature

High

High

Low

Low4321 YYYY

8765 YYYY

1211109 YYYY

(best)


Lecture 1.1 35

Illustration of a full factorial design,with 12 experimental runs

Pressure

Temperature

High

High

Low

Low321 YYY

121110 YYY987 YYY

654 YYY


Lecture 1.1 36

Interaction between the factors

Pressure

Temperature

High

High

Low

Low65

75

70

60

5

15

5 5

best

best

best


Lecture 1.1 37

Multilevel Interaction:Emotional Arousal

160 subjects,

– 80 male (M),– 80 female (F)

shown one of 4 pictures:

– nude female,– nude male,– infant,– landscape.

Response variable:

– level of emotional arousal


Lecture 1.1 38

Infa

nt

Land

sdca

pe

Nud

e F

emal

e

Nud

e M

ale

10

15

20

25

Male

PicturesIn

fant

Land

sdca

pe

Nud

e F

emal

e

Nud

e M

ale

10

15

20

25

Female

Pictures

Levels of Arousal of Males and Females to Different Visual Stimuli

Interaction between FactorsCase study: Emotional Arousal


Lecture 1.1 39

Non-linear response:Optimisation vs Improvement


Lecture 1.1 40

Optimising performance; hill climbing

50 52 54 56 58 60 62 64

Temperature

65

66

67

68

69

70

Yield


Lecture 1.1 41

50 52 54 56 58 60 62 64

Temperature

65

66

67

68

69

70

Yield



Lecture 1.1 42

50 52 54 56 58 60 62 64

Temperature

65

66

67

68

69

70

Yield



Lecture 1.1 43

50 52 54 56 58 60 62 64

Temperature

65

66

67

68

69

70

Yield



Lecture 1.1 44

50 52 54 56 58 60 62 64

Temperature

65

66

67

68

69

70

Yield



Lecture 1.1 45

Several factors

2 - level factors: 2 factors: 22 = 4 runs

3 factors: 23 = 8 runs





Multi-level: 2 × 3 × 4 × 5 = 120 runs


Lecture 1.1 46

Reading

SA Sections 1.9, 11.4 - 11.6

EM Sections 4.3, 4.5.1, 5.2

DCM Section 2.5, 3.1 - 3.3

Documents

Diploma in Statistics Design and Analysis of Experiments Lecture 1.11 Diploma in Statistics Design and Analysis of Experiments Lecturer:Dr. Michael Stuart,