Analyze Phase Introduction to Hypothesis Testing

Analyze PhaseIntroduction to Hypothesis

Testing

Analyze PhaseIntroduction to Hypothesis

Testing

© Open Source Six Sigma, LLCOSSS LSS Black Belt v10.0 - Analyze Phase 2

Hypothesis Testing (ND)

Tests for Central TendencyTests for Central Tendency

Tests for VarianceTests for Variance

ANOVAANOVA

Hypothesis Testing PurposeHypothesis Testing Purpose

Hypothesis Testing NND P1Hypothesis Testing NND P1

Hypothesis Testing ND P1Hypothesis Testing ND P1

Intro to Hypothesis TestingIntro to Hypothesis Testing

Inferential StatisticsInferential Statistics

“X” Sifting“X” Sifting

Welcome to AnalyzeWelcome to Analyze

Hypothesis Testing ND P2Hypothesis Testing ND P2

Wrap Up & Action ItemsWrap Up & Action Items

Hypothesis Testing NND P2Hypothesis Testing NND P2


Six Sigma Goals and Hypothesis Testing

Our goal is to improve our Process Capability, this translates to the need to move the process Mean (or proportion) and reduce the Standard Deviation.

– Because it is too expensive or too impractical (not to mention theoretically impossible) to collect population data, we will make decisions based on sample data.

– Because we are dealing with sample data, there is some uncertainty about the true population parameters.

Hypothesis Testing helps us make fact-based decisions about whether there are different population parameters or that the differences are just due to expected sample variation.

12011611210810410096

LSL USLProcess Data

Sample N 150StDev(Within) 2.35158StDev(Overall) 5.41996

LSL 100.00000Target *USL 120.00000Sample Mean 108.65832

Potential (Within) Capability

CCpk 1.42

Overall Capability

Pp 0.62PPL 0.53PPU 0.70Ppk

Cp

0.53Cpm *

1.42CPL 1.23CPU 1.61Cpk 1.23

Observed PerformancePPM < LSL 6666.67PPM > USL 0.00PPM Total 6666.67

Exp. Within PerformancePPM < LSL 115.74PPM > USL 0.71PPM Total 116.45

Exp. Overall PerformancePPM < LSL 55078.48PPM > USL 18193.49PPM Total 73271.97

WithinOverall

Process Capability of Process Before

120117114111108105102

LSL USLProcess Data

Sample N 100StDev(Within) 1.55861StDev(Overall) 1.54407

LSL 100.00000Target *USL 120.00000Sample Mean 109.86078

Potential (Within) Capability

CCpk 2.14

Overall Capability

Pp 2.16PPL 2.13PPU 2.19Ppk

Cp

2.13Cpm *

2.14CPL 2.11CPU 2.17Cpk 2.11

Observed PerformancePPM < LSL 0.00PPM > USL 0.00PPM Total 0.00

Exp. Within PerformancePPM < LSL 0.00PPM > USL 0.00PPM Total 0.00

Exp. Overall PerformancePPM < LSL 0.00PPM > USL 0.00PPM Total 0.00

WithinOverall

Process Capability of Process After


Purpose of Hypothesis Testing

The purpose of appropriate Hypothesis Testing is to integrate the Voice of the Process with the Voice of the Business to make data-based decisions to resolve problems.

Hypothesis Testing can help avoid high costs of experimental efforts by using existing data. This can be likened to:

– Local store costs versus mini bar expenses.– There may be a need to eventually use experimentation, but

careful data analysis can indicate a direction for experimentation if necessary.

The probability of occurrence is based on a pre-determined statistical confidence.

Decisions are based on:– Beliefs (past experience)– Preferences (current needs)– Evidence (statistical data)– Risk (acceptable level of failure)


The Basic Concept for Hypothesis Tests

Recall from the discussion on classes and cause of distributions that a data set may seem Normal, yet still be made up of multiple distributions.

Hypothesis Testing can help establish a statistical difference between factors from different distributions.

3210-1-2-3

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

x

freq

3210-1-2-3

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0.0

x

freq

Did my sample come from this population? Or this? Or this?


Significant Difference

Are the two distributions “significantly” different from each other? How sure are we of our decision?

How do the number of observations affect our confidence in detecting population Mean?

Sample 2Sample 1


Detecting Significance

Statistics provide a methodology to detect differences.

– Examples might include differences in suppliers, shifts or equipment.

– Two types of significant differences occur and must be well understood, practical and statistical.

– Failure to tie these two differences together is one of the most common errors in statistics.

HO: The sky is not falling.

HA: The sky is falling.


Practical vs. Statistical

Practical Difference: The difference which results in an improvement of practical or economic value to the company.

– Example, an improvement in yield from 96 to 99 percent.

Statistical Difference: A difference or change to the process that probably (with some defined degree of confidence) did not happen by chance.

– Examples might include differences in suppliers, markets or servers.

We will see that it is possible to realize a statistically significant difference without

realizing a practically significant difference.


Detecting Significance

During the Measure Phase, it is important that the nature of the problem be well understood. In understanding the problem, the practical difference to be achieved must match the statistical difference.

The difference can be either a change in the Mean or in the variance.

Detection of a difference is then accomplished using statistical Hypothesis Testing.

Mean Shift

Variation Reduction


Hypothesis Testing

A Hypothesis Test is an a priori theory relating to differences between variables.

A statistical test or Hypothesis Test is performed to prove or disprove the theory.

A Hypothesis Test converts the practical problem into a statistical problem.

– Since relatively small sample sizes are used to estimate population parameters, there is always a chance of collecting a non-representative sample.

– Inferential statistics allows us to estimate the probability of getting a non-representative sample.


DICE Example

We could throw it a number of times and track how many each face occurred. With a standard die, we would expect each face to occur 1/6 or 16.67% of the time.

If we threw the die 5 times and got 5 one’s, what would you conclude? How sure can you be?

– Pr (1 one) = 0.1667 Pr (5 ones) = (0.1667)5 = 0.00013

There are approximately 1.3 chances out of 1000 that we could have gotten 5 ones with a standard die.

Therefore, we would say we are willing to take a 0.1% chance of being wrong about our hypothesis that the die was “loaded” since the results do not come close to our predicted outcome.


Hypothesis Testing

β n

α

DECISIONS


Statistical Hypotheses

A hypothesis is a predetermined theory about the nature of, or relationships between variables. Statistical tests can prove (with a certain degree of confidence) that a relationship exists.

We have two alternatives for hypothesis.

– The “null hypothesis” Ho assumes that there are no differences or relationships. This is the default assumption of all statistical tests.

– The “alternative hypothesis” Ha states that there is a difference or relationship.

Making a decision does not FIX a problem, taking action does.

P-value > 0.05 Ho = no difference or relationshipP-value < 0.05 Ha = is a difference or relationshipP-value > 0.05 Ho = no difference or relationshipP-value < 0.05 Ha = is a difference or relationship


Steps to Statistical Hypothesis Test

1. State the Practical Problem.

2. State the Statistical Problem.

a) HO: ___ = ___

b) HA: ___ ≠ ,>,< ___

3. Select the appropriate statistical test and risk levels.

a) α = .05

b) β = .10

4. Establish the sample size required to detect the difference.

5. State the Statistical Solution.

6. State the Practical Solution.Noooot THAT

practical solution!


How Likely is Unlikely?

Any differences between observed data and claims made under H0 may be real or due to chance.

Hypothesis Tests determine the probabilities of these differences occurring solely due to chance and call them P-values.

The a level of a test (level of significance) represents the yardstick against which P-values are measured and H0 is rejected if the P-value is less than the alpha level.

The most commonly used levels are 5%, 10% and 1%.


Hypothesis Testing Risk

The alpha risk or Type 1 Error (generally called the “Producer’s Risk”) is the probability that we could be wrong in saying that something is “different.” It is an assessment of the likelihood that the observed difference could have occurred by random chance. Alpha is the primary decision-making tool of most statistical tests.

Type 1Error

Type IIError

CorrectDecision

CorrectDecision

Actual ConditionsNot

DifferentDifferent

Not Not DifferentDifferent

StatisticalConclusion

s

(Ho is True) (Ho is False)

(Fail to Reject Ho)

DifferentDifferent(Reject Ho)


Alpha Risk

Alpha ( ) risks are expressed relative to a reference distribution.

Distributions include:

– t-distribution

– z-distribution

– 2- distribution

– F-distribution

Region of

DOUBT

Region of

DOUBT

Accept as chance differences

The a-level is represented by the clouded areas.

Sample results in this area lead to rejection of H0.

The a-level is represented by the clouded areas.

Sample results in this area lead to rejection of H0.


Hypothesis Testing Risk

The beta risk or Type 2 Error (also called the “Consumer’s Risk”) is the probability that we could be wrong in saying that two or more things are the same when, in fact, they are different.

Type 1Error

Type IIError

CorrectDecision

CorrectDecision

Actual ConditionsNot

DifferentDifferent

Not Not DifferentDifferent

StatisticalConclusion

s

(Ho is True) (Ho is False)

(Fail to Reject Ho)

DifferentDifferent(Reject Ho)


Beta Risk

Beta Risk is the probability of failing to reject the null hypothesis when a difference exists.

Distribution if Ha is true

Critical value of test statistic

Critical value of test statistic

Reject H0

= Pr(Type 1 error)

Accept H0

= Pr(Type II error)

= 0.05

H0 value

Distribution if H0 is true


Distinguishing between Two Samples

Recall from the Central Limit Theorem as the number of individual observations increase the Standard Error decreases.

In this example when n=2 we cannot distinguish the difference between the Means (> 5% overlap, P-value > 0.05).

When n=30, we can distinguish between the Means (< 5% overlap, P-value < 0.05) There is a significant difference.

Theoretical Distribution of MeansWhen n = 2 = 5S = 1

Theoretical Distribution of MeansWhen n = 30= 5S = 1


Delta Sigma—The Ratio between and S

Delta () is the size of the difference between two Means or one Mean and a target value.

Sigma (S) is the sample Standard Deviation of the distribution of individuals of one or both of the samples under question.

When & S is large, we don’t need statistics because the differences are so large.

If the variance of the data is large, it is difficult to establish differences. We need larger sample sizes to reduce uncertainty.

Large Delta

Large S

We want to be 95% confident in all of our estimates!


Typical Questions on Sampling

Question: “How many samples should we take?”

Answer: “Well, that depends on the size of your delta and Standard Deviation”.

Question: “How should we conduct the sampling?”Answer: “Well, that depends on what you want to know”.

Question: “Was the sample we took large enough?”Answer: “Well, that depends on the size of your delta and

Standard Deviation”.

Question: “Should we take some more samples just to be sure?”Answer: “No, not if you took the correct number of samples the first time!”


The Perfect Sample Size

The minimum sample size required to provide exactly 5% overlap (risk). In order to distinguish the Delta.

Note: If you are working with Non-normal Data, multiply your calculated sample size by 1.1

40 50 60 7040 50 60 70

40 60 7050

Population


Hypothesis Testing Roadmap

Normal

Test of Equal Variance 1 Sample t-test1 Sample Variance

Variance Not EqualVariance Equal

2 Sample T One Way ANOVA 2 Sample T One Way ANOVA

Continuous

Data



Non Normal

Test of Equal Variance Median Test

Mann-Whitney Several Median Tests

Continuous

Data



Attribute Data

One Factor Two Factors

One Sample Proportion

Two Sample Proportion

Minitab:Stat - Basic Stats - 2 ProportionsIf P-value < 0.05 the proportions are different

Chi Square Test (Contingency

Table)Minitab:Stat - Tables - Chi-Square TestIf P-value < 0.05 the factors are not independent

Chi Square Test (Contingency

Table)Minitab:Stat - Tables - Chi-Square TestIf P-value < 0.05 at least one proportion is different

Two or More Samples

Two SamplesOne

Sample

Attribute

Data


Common Pitfalls to Avoid

While using Hypothesis Testing the following facts should be borne in mind at the conclusion stage:

– The decision is about Ho and NOT Ha.

– The conclusion statement is whether the contention of Ha was upheld.

– The null hypothesis (Ho) is on trial.

– When a decision has been made:• Nothing has been proved.• It is just a decision.• All decisions can lead to errors (Types I and II).

– If the decision is to “Reject Ho,” then the conclusion should read “There is sufficient evidence at the α level of significance to show that “state the alternative hypothesis Ha.”

– If the decision is to “Fail to Reject Ho,” then the conclusion should read “There isn’t sufficient evidence at the α level of significance to show that “state the alternative hypothesis.”


Summary

At this point, you should be able to:

• Articulate the purpose of Hypothesis Testing

• Explain the concepts of the Central Tendency

• Be familiar with the types of Hypothesis Tests

Documents

Analyze Phase Introduction to Hypothesis Testing