11.3: Using Inference to make Decisions

AP StatisticsNPHS

Choosing a Level of Significance: Things to think about

(1) How plausible is H0? A study that finds that smoking increases the risk of

Alzheimer's. You read a study that claims to have evidence that

smoking is really good for you.

(2) What are the consequences of rejecting H0? You find evidence that cats sleep more than dogs. You find evidence that a new drug may have

harmful side-effects…but your company has invested millions of dollars in an ad campaign for the drug.

Statistical Significance vs. Practical ImportanceYou decide to run a significance test to see if a particular SAT

prep program increases scores on the Math portion. You know from previous research that the average score on the Math section is 510 with a standard deviation of 50. You take a sample of 200 students and find that they have an average score of 515. Use a 5% level of significance. H0: μ = 510 Ha: μ > 510 P-Value: 0.02167

We can reject the null hypothesis that the prep program does not improve scores…but is a 5 point increase worth anything?

Beware Outliers!!!Pesky little outliers can destroy

the significant of otherwise significant data.

They can also make data appear significant when it actually is not.

Always do a graphical analysis of your data The effect you are searching for

should be evidence in your plotsConfidence intervals can help

you get a better idea

Beware Outliers!!!Be aware of “dropouts”

from statistical analysis.

Make sure that all the data is represented in the analysis.

Lack of SignificanceExample 11.14 In an experiment to compare methods for reducing

transmission of HIV, subjects were randomly assigned to a treatment group and a control group. Result: the treatment group and the control group had the same rate of HIV infection. Researchers described this as an “incident rate ratio” of 1.00. (>1.00 means greater rate of infection among treatment group, <1.00 means greater rate among control).

The 95% confidence interval for the incident rate ratio was reported at 0.63 to 1.58.

Can you really say that the treatment has no effect?

Lack of SignificanceDesign a study so that it has a high

probability of finding a real effect. What could you do to increase the chances of

finding an effect?

Invalid Statistical Inference Hawthorne effect

What is the term for a study where neither the subject nor the administrator knows who is getting what treatment?

Invalid Statistical Inference The importance of an SRS from the population of

INTEREST.

Multiple AnalysesA study using an alpha level of 0.05 is run for

20 different types of soda to see if there is an association between drinking soda and scoring well on a math test.

It is found that one soda, Mountain Dew, did increase scores.

Why is this not good evidence of an effect?