Upload
richard-ferreria
View
1.895
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Citation preview
Chapter 11
Testing a Claim
11.1 SIGNIFICANCE TESTS:
THE BASICS
The Pizza Problem
• Let us suppose that a certain pizza company claims that they deliver their pizza in an average of 20 minutes
• Now, we are told “average time” so it’s possible that they’ve delivered a pizza in 5 minutes, and it’s also possible that they delivered a pizza in 30 minutes
• If we order pizza 10 times, what average time will convince you that they’re claim is wrong?
• Welcome to significance testing!
How significance testing works
1. Assume that a claim about an average or proportion is true
2. Compute the average or prop of a sample
3. Compare the sample with the sampling distribution for the claim and sample size.
4. If the probability of obtaining the sample avg or prop is too low, we conclude that our claim is improbable, and reject it.
How significance testing works
In all cases, we are comparing the sample with the sampling distribution for the claim and sample size
PHANTOMS (a framework)
As with Confidence Intervals, there is an acronym to help you remember the steps of a significance test
• State the Parameter• State the Hypothesis pair• Check the Assumptions• State the Name of the test• Find the value of the Test Statistic• Obtain a p-value• Make a decision• Summarize
State Parameters
• Parameters work the same way they did in Confidence Intervals
• = The true average of the Pizza Company’s delivery times
• x-bar = the average delivery time for a sample of 10 deliveries from the Pizza Company
• p = the proportion of all deliveries from the Pizza Company that are delivered in less than 20 minutes
• p-hat = the proportion of sample of 10 deliveries from the Pizza Company that are delivered in less than 20 minutes
Stating Hypotheses
Hypotheses come in pairs:• “the null hypothesis” – H0 “H naught”
– This is the presumed claim– For our purposes, our null hypothesis
will always be in the forms:“= __”“p = ___”
Stating Hypotheses
Hypotheses come in pairs:• “the alternative hypothesis”– Ha
– This is the suspicion of the researcher– There are 3 alt hyps that we can test
1. “ ≠ ___” (two-sided alternative)
2. “p > ___” (one-sided alternative)3. “ < ___” (one-sided alternative)
Stating Hypotheses
Notice: Hypotheses are always about the parameter ( or p, never xbar or phat)
Written Examples“H0: = 20 minutes
Ha: > 20 minutes”
“H0: p = 0.5
Ha: p < 0.5”
Checking the Assumptions
• Since we are comparing our samples to a sampling distribution (just like the last chapter), the assumptions are the same
• We will review them now:
Checking the Assumptions
Assumptions for mean• SRS• Independence
N > 10n• Normality (a, b, or c must be true)
(a) population is Normal, or(b) n > 30; Central Limit Theorem, or(c) Sample is approximately normal:
(1) histogram single peak and symmetric, (2) Normal probability plot is linear,(3) no Outliers
Checking the Assumptions
Assumptions for proportions• SRS• Independence
N > 10n• Normality
np > 10nq > 10
Name of Test
• “one-sided z test for means”• “two-sided z test for means”• “one-sided t test for means”• “two-sided t test for means”• “one-sided z test for proportions”• “two-sided z test for proportions”• More on these later
Test Statistics
• Test Statistics are always of the form:
• Standard Deviation of the sampling distribution depends on the characteristic tested
estimate - null hypothesistest statistic
std dev of sampling dist (std error)
Test Statistics
• Std Dev for mean ( known):
• Std Dev for mean ( unknown):
• Std Dev for proportions:
n
s
n
p q
n
Notice that we use ‘p’ and not ‘p-hat’
P-values
• The P-value is the probability of obtaining a measurement as extreme as the test statistic
• At its most basic, computing the P-Value is the same as computing area from a Normal curve or Student’s t-distribution
• Computation varies slightly when using 2-sided alternative vs. 1-sided alternative
P-values
Two sided alternatives• For these alt hyps, we calculate a p-
val based on area “from two tails”
P-values
Example:• Let’s assume our sample of 24 has:
x-bar = 22 and s = 1.53• H0: = 20
Ha: 20
• “2-sided t-test for means”
P-values
Example (cont)• Test Statistic:
P-values
Example (cont)• Test Statistic:
22 206.404
/ 1.53 / 24
xts n
P-values
Example (cont)• Test Statistic:
22 206.404
/ 1.53 / 24
xts n
P-values
Example (cont)• Test Statistic:
22 206.404
/ 1.53 / 24
xts n
P-values
Example (cont)• Test Statistic:
• P-value
22 206.404
/ 1.53 / 24
xts n
23
P-val =2 P test stat
2 6.404
0.000000778
t
P t
P-values
Example (cont)• Test Statistic:
• P-value
22 206.404
/ 1.53 / 24
xts n
23
P-val =2 P test stat
2 6.404
0.000000778
t
P t
P-values
Example (cont)• Test Statistic:
• P-value
22 206.404
/ 1.53 / 24
xts n
23
P-val =2 P test stat
2 6.404
0.00000155
t
P t
P-values
One sided alternatives• Calculate the area tail indicated by
the alternative hypothesis for P-value
P-values
One sided alternatives• If H0 p = .53 and Ha: p > 0.53
then P-val = P(z > test stat)• If Ha: < 10, • then P-val = P(t < test stat)
P-values
Example• Let’s assume:• H0 p = .22 and Ha : p < 0.22
p-hat = 0.20 from n = 55• Test statistic
P-values
Example• Let’s assume:• H0 p = .22 and Ha : p < 0.22
p-hat = 0.20 from n = 55• Test statistic
0.20 0.22
/ 0.2 0.8 / 55
0.371
p p
pq n
P-values
Example• Let’s assume:• H0 p = .22 and Ha : p < 0.22
p-hat = 0.20 from n = 55• Test statistic
0.20 0.22
/ 0.2 0.8 / 55
0.371
p p
pq n
P-values
Example• Let’s assume:• H0 p = .22 and Ha : p < 0.22
p-hat = 0.20 from n = 55• Test statistic
0.20 0.22
/ 0.2 0.8 / 55
0.371
p p
pq n
P-values
Example (cont.)• P-value
• Would you say this is “likely” or “unlikely”?
P-val = 0.371P z 0.3553
Making a decision
• The P-value serves as the indicator• If the test statistic is likely under the
presumed sampling distribution (i.e. the p-value is large), then we have no reason to reject the null-hypothesis
• If the test statistic is unlikely (i.e. the p-value is small), then we have reason to reject the null-hypothesis.
• “If the p-value is low, reject the Hoe”
Making a decision
Significance level (‘alpha’ )• This is the probability level at which we will
reject H0
• Typical sig levels = 0.10, 0.05, 0.01• If no significance level is given, we will
generally reject at the = 0.05 level.• When p-val < , then we
“reject H0 at the = __ level”
• When H0 is rejected, we say the data is “statistically significant at the = __ level”
Making a decision
“Reject or Fail to Reject”• When p val > alpha, we “fail to reject H0”– This means that we do not have evidence to
show H0 is incorrect
– This does not mean, H0 is “correct”
• When p val < alpha we “reject H0”– This means that H0 is unlikely
– The new estimate for or p is our sample data (x-bar or p-hat)
WOW
• That was a lot of information!• We will be going over this
information again at a slower pace in the coming weeks.
• We’ll work out the mechanics later• Understanding the basics and the
“whys” right now will help you in the future!
Assignment 11.1
• Page 693 #3, 5, 7-8, 11-14
11.2 CARRYING OUT SIGNIFICANCE TESTS
z-test for a population mean
• This is the appropriate test when is known.
• Test Statistic:
/
xz
n
z-test for a population mean
• P-value:
Example 11.10
The mean systolic blood pressure for males 35 to 44 years is 128, and the standard deviation in this population is 15. The medical records of 72 male executives in this age group finds the mean systolic blood pressure is 129.93. Is this evidence that the mean blood pressure for all the company’s younger male executives is different than the national average?
Example 11.10
• We are going to check to see if our sample comes from a population with the same and sigma as the national population.
• Because of this, our parameter will come from the national averages.
• The null hypothesis will assume that younger male executives have the same mean blood pressure as the national average.
• The null hypothesis will always assume “things are equal”
Example 11.10
Parameter• “Let = average blood pressure of all
younger male executives in the company”
• “Let x-bar = average blood pressure in the sample of 72 younger male executives from the company”
Example 11.10
Hypotheses
• = 128 128
• Notice that we will need the 2-sided P-value
Example 11.10
Assumptions• Simple Random Sample
“We are not told that our sample is from an SRS. We should check how this sample was chosen. We will proceed as though this sample was an SRS”
• Independence“We are not told the size the population of young male executives. We should check that the population is greater than 10(72) = 720.”
• Normality“Because we have a large sample, the Central Limit Theorem guarantees that the sampling distribution is approximately Normal”
Example 11.10
Assumptions (cont.)• The preceding example illustrates ‘what to do’
if you think that an assumption is not met.• If you believe that an assumption is not met:
(1) state the condition that must be qualified, (2) mention that it “needs to checked,” and (3) state you will “proceed as though this assumption was met”
Always try to carry out the significance test.
Example 11.10
Name of the Test• “We will conduct a z-test for a
population mean”
Example 11.10
Test Statistic
/
xz
n
129.93 128
15 / 72
1.092z
Example 11.10
P-value p value 2 1.092P z
p value 0.2748
Example 11.10
Make a Decision• We are not given an in this example
we should use the standard 0.05 significance level.
• The p-value is larger than our , so we should reject the null hypothesis
• Note: nothing needs to be written for this part of PHANTOMS
Example 11.10
Summarize“Approximately 27% of the time, a sample of size n =72 will produce an average at least as extreme as 129.93. Since this p-value is larger than a presumed = 0.05, we cannot reject our null hypothesis.We have no evidence to suggest that the mean systolic blood pressure of young executives is not 128.”
Approximately 27% of the time, a sample of size n =72 will produce an average at least as extreme as 129.93.
Example 11.10
Summarize (cont.)Note that the summary contains 3 parts:(1) Interpret the p-value(2) Compare the p-value with (3) Interpret the conclusion in context
Since this p-value is larger than a presumed = 0.05, we cannot reject our null hypothesis.We have no evidence to suggest that the mean systolic blood pressure of young executives is not 128.
Tests and Confidence Intervals
• A “two-sided alternative” and the “confidence interval” are the same test.
• A test will reject the null hypothesis of a two-sided alternative when the test statistic is outside the confident interval with CL = 1 -
• The link between confidence intervals and a two-sided test is called “duality”
• Refer to example 11.12
Assignment 11.2
• Page 709 #27, 29, 31-33
11.3 USE AND ABUSE OF TESTS
More on Significance Levels
• The significance level for a test is informed by the plausibility of H0.– If H0 is particularly “strong” or has a
many years behind it, then the evidence must also be “strong” (small )
– If we were trying to disprove the gravitational constant, the would have to be very, very small!
More on Significance Levels
• What are the consequences of rejecting H0?– There will always be a cost/benefit to
rejecting H0
– If it is more expensive to reject than it is to fail to reject, then the evidence must be strong (small )
– Consider the Toyota brake recall 2009
More on Significance Levels
• There is no “hard line” between reject and fail to reject– There isn’t a real difference between
= 0.10 and = 0.11– There is no sharp border between
“statistically significant” and “statistically insignificant”
– As P-value decreases, the strength of the evidence increases
– Although = 0.05 is ‘handy rule of thumb,’ it is not a universal rule
Cautions
• Don’t forget to examine the data– The presence of outliers can affect whether
the significance tests are plausible
• “Statistically Significant” is not the same thing as “Important”– Lack of significance may signal an
important conclusion
• A Test of Significance is not appropriate for all data sets
11.4 USING INFERENCE TO MAKE DECISIONS
“What if” we made the wrong decision?
There are two kinds of wrong decisions:• Reject a H0 that was actually true– This is a “TYPE I ERROR”
• Fail to reject H0 that was false– This is a “TYPE II ERROR”
• Some students find it helpful to think: “You can reject one hoe, but who can fail to reject two hoes”– whatever floats your boat, eh?
“What if” we made the wrong decision?
TYPE I ERROR• The null hypothesis was true!• The probability that we made this
error will be same as (since H0 was true)– You will need to know how to recognize
this error in context and – You will need to know the probability of
making a Type I error
“What if” we made the wrong decision?
TYPE II ERROR• In this case, the null hypothesis was
incorrect, but we failed to reject it• The probability of making a Type II
error is a “what if” calculation– “What if is actually 42- what’s the
probability that I fail to reject?”
• The probability of making a Type II error is known as
Type II Errors
This is the alternative samplingdistribution. Remember:H0 is (presumed) false
Type II Errors
This is
Type II Error
• is the area of the tail for the sampling distribution of the “what if” parameter value
• H0: = 5, xbar = 5.8, = 0.7, n = 40
• Calculation of when a = 6
• Since 0 > we need to calculate the left tail area
.
5.8 6
0.7 / 40
1.807
0.0354
P z
P z
Type II Error
• Mercifully, the AP exam will never ask you to compute
• You will be asked to interpret • Remember that is always
dependent on an alternative value of the parameter
.
Power
• The probability that the significance test will reject H0 at an level for an alternative value of the parameter is the power of the test against the alternative.
• Power = 1- • Power is the probability of not
making a TYPE II error• Lots of power is a good thing!
How to increase power
(1)Increase the significance level ()(2)Consider an alternative parameter that
is further away from the null hypothesis(3)Increase the sample size(4)Decrease
All the above have the effect of decreasing .
Less = More power