The Practice of StatisticsThird Edition

Chapter 5:Producing Data

Copyright © 2008 by W. H. Freeman & Company

Daniel S. Yates

Case StudyCan eating chocolate be good for you?

Study 1: Does chocolate consumption during pregnancy have an effect on infant temperament at age six months. Pregnant women rated their stress levels and Choc consumption and then their children’s temperament at six months.

Study 2: Each subject consumed a cocoa beverage containing 900 mg of flavornols daily for 5 days. Blood flow was measured on th 1st and 5th day of the study.

Study 3: Volunteers were given either bittersweet or fake dark chocolate. On another day the volunteers were switched. The researchers also did not know who ate what chocolate. An ultrasound was used to see the functioning of the cells in the main artery.

Activity 5A

Child CareObservational StudyA study of childcare enrolled 1364 infants in

child care and followed them through the 6th grade.

They published an article in 2003 stating that

“the more time children spent in child care from birth to age four and a half, the more adults tended to rate them, both at age 4 1/2 and in Kindergarten, as less likely to get along with others, as more assertive, as disobedient, and as aggressive.”

5.1 Designing SamplesObjectives

Define population and sample.Explain how sampling differs from a

census.Explain what is meant by a voluntary

response sample.Give an example of a voluntary

response sample.Explain what is meant by

convenience sampling.Define what it means for a sampling

method to be biased.Define, carefully, a simple random

sample (SRS)

82% of scientists agree that the “media do not understand statistics well enough to explain new findings”

5.1 Objectives cont…List the four steps involved in choosing a SRS.Explain what is meant by systematic random sampling.Use a table of random digits to select a simple random

sample.Define a probability sample.Given a population, determine the strata of interest,

and select a stratified random sample.Define a cluster sample.Define undercoverage and nonresponses as sources of

bias in sample surveys.Give an example of response bias in a survey

question.Write a survey question in which the wording of the

question is likely to influence the response.Identify the major advantage of large random samples.

Sampling method refers to the process used to choose the sample from the population.

Call-in opinion polls/Online PolesVoluntary responsePeople who spend the time and money to

respond to call in poles or who take the trouble to respond to an open invitation are not representative of the entire adult population.

Convenience SamplingInterviewing at the mall

Why is this not a good sample?Mall shoppers are not representative of the

entire population.They are…RicherMore likely to be teenagers or retiredInterviewers tend to select neat safe-

looking individuals.

(Drawing names out of a hat)

Systematic random samplingA random sampling design that gives each

individual, but not each sample, an equal chance of being chosen.

For example, you choose one player from each of 12 NFL teams of the same size. Each player has an equal chance of being chosen, but there is no chance of them all coming from the same team.

Table B

Joan’s accounting firmJoan’s small accounting firm serves 30 business

clients . Joan wants to interview a SRS of 5 clients.Step1: Give each client a numerical label, using as few digits

as possible. Two digits are needed to label 30 clients so the clients are each given a number from 01-30.

Step 2: enter table B anywhere. Suppose we enter at line 130, which is

69051 64817 87174 09517 84534 06489 87201 97245The first 10 2-digit numbers are69 05 16 48 17 87 17 40 95 17 84 53 40 64 89 20 19 72 45The first five clients within our range are 05,16,17,20,19Ignore any that repeat because they have already been

chosen.Step 3: Stop when you have 5Step 4: Match each number back to a client.

Activity 5B:The Simple Random Sample applet

http://www.whfreeman.com/tps3e

The use of chance to select the sample is the essential principle of statistical sampling.

Who wrote that song?Stratified Sampling

In order to pay their members Royalties based on how much their songs are played, the ASCAPTapes about 60,000 hours from The 53 million hours of local radioPrograms across the country each year.

Radios are stratified by type of community, geographic location and the size of the audience. In all there are 432 strata. Tapes are made at random hours for randomly selected members of each strata.

What do AP students think?Cluster SamplingIn order to determine if AP Stats students

think they had enough time on the exam, a number of schools are randomly selected from a list of all the schools.

All the students who took the exam in these schools are then surveyed.

The students interviewed would constitute a cluster sample.

Assignment 5.1P. 273 5.1-5.7

P. 279-2805.9,

Cautions about Sample Surveys

Ring, ring nobody homeNonresponse

Effect of interview method on poll resultsSurvey methods

In response to the statement “I never doubt the existence of God” subjects are asked to choose from the responses…

Completely agree mostly agree mostly disagree completely disagreeIn 1990, the subjects were interviewed in person and were

handed a card with the four responses on it.In 1991, the pole switched to telephone interviews.In 1990, 60% Said completely agree in line with earlier

results.In 1991, 71% completely agreed.The change might be explained by the effect of hearing

“completely agree” stated first by the interviewer.

Did you vote?Response bias

What they said

I voted I did not vote

What they did Voted 358 13

Did not vote 120 172

As you can see 72% of people said they voted while only 54% actually voted.

Should we ban disposable diapers?Wording of questionsIt is estimated that disposable diapers

account for less than 2% of the trash in today’s landfills. In contrast, beverage containers, third class mail and yard wastes are estimated to account for about 21% of the trash in landfills. Given this, in your opinion, would it be fair to ban disposable diapers?

84% of the sample opposed banning disposable diapers.

Doubting the HolocaustWording of questionsDoes it seem possible, or does it seem

impossible to you that the extermination of the Jews never happened?

22% said possible.

Does it seem possible to you that the extermination of the Jews never happened or do you feel certain that it happened?

1 % said possible

Cautions!!!!Insist on knowing the exact question

asked, the rate of nonresponse, and the date and method of the survey before you trust a poll result.

We can improve our results by knowing that larger random samples give more precise results than smaller samples.

5.1 exercises5.19-5.29 oddRead p. 284-285

Activity 5C The class survey revisited

1. Consider the questions in order. As you look at each question, see if it contains bias. Does it contain phrasing or wording that might be confusing? Will any questions evoke response bias?

2. Make any changes that the group thinks are needed. Remember that the survey should be anonymous so that the class as a whole are described.

3. Print the final version of the survey. Make a copy for each person in the class and one to record results on.

4. Each student should complete the survey.5. Place the surveys upside down in a pile. Shuffle the surveys.6. Designate someone to tally the responses as homework and

prepare a cumulative summary. Give a copy to each student in the class.

5.2 Designing Experiments

Effects of Class sizeExperimental units, subjects and treatments

Do smaller class sizes in elementary school really benefit students in areas such as scores on standardized test scores?

The subjects were 6385 students who were beginning kindergarten. Each student was assigned to one of three treatment groups ( 22-25 students) with one teacher, regular class with a full time teacher and a teachers aid, and a small class, (13-17 students).

These treatments are levels of a single factor, (the type of class)

The students stayed in the same type of classes for four years and then returned to regular classes.

In later years, students from the small classes had higher test scores , had better high school grades, etc…

Advertising:Response variablesWhat are the results of repeated exposure to an advertising message?The treatment in the study of repeat exposure

to advertising. Combining the levels of the two factor forms six treatments.

Basic Principles of statistical design1. Control2. Replication3. Randomization

1) ControlA simple experiment in a controlled laboratory

environment often has the formTreatment---› Observed ResponseIn field experiments, this design can be too

simplePlacebo effect: many patients respond

favorably, to any treatment, even a placebo. A placebo is a dummy treatment.

Control group: The group of subjects that receive the dummy or placebo treatment.

Treating UlcersPlacebo effect“Gastric Freezing” is a clever treatment

for ulcers in the upper intestine.Gastric freezing- observe pain reliefThis is a poor design. The patient relief

may have been due to the placebo effect Placebo effect: many subjects respond

favorably to any treatment even a placebo

Control GroupA later experiment divided the ulcer patients

into two groups. One group received the gastric freezing as before, the other received a placebo treatment.

The results; 34% of the treatment group improved, but so did 38% of the placebo group.

This and other experiments showed that gastric freezing was no better than a placebo.

The group of patients that receive the placebo treatment is called the control group.

Caution!!!Don’t confuse control and control

group. Control refers to the overall effort to

minimize variability in the way experimental units are obtained and treated.

2) ReplicationEven with control, there will still be natural

variability or systematic differences between the groups.

If we assign many units to each group, the effects of chance will average out.

Use enough subjects to reduce chance variation.

3) Randomization

The use of chance to divide experimental units into groups.

Cell phones and driving 1Random assignmentDoes talking on a hands-free cell phone distract drivers.

Randomization: the use of chance to divide experimintal units into groups.

The Physicians’ Health StudyRandom assignment to treatmentsDoes regularly taking aspirin help protect

people against heart attacts?The subjects were 21,996 male physicians. There were two factors, each having two levels

aspirin (yes or no) and beta-caratene (yes or no)

On odd-numbered days, the subjects took either a white tablet that contained aspirin or a dummy pill. On even numbered days they took either a beta carotene pill or a placebo.

The study looked at several response variables heart attacks, cancers, etc.

The treatment in the Physicians’ Health Study.

ResultsAfter several years, 239 of the placebo

group but only 139 of the aspirin groups had suffered heart attacts.

This is good evidence that aspirin prevents heart attacks but beta carotene does not.

Completely randomized design: When all experimental units are allocated at random among all treatments.

Outline of a completely randomized design comparing six treatments

Activity 5eSelecting random samples by calculator.1. STAT/4:clrList/2nd/L1/ENTER2. MATH/PRB/5:randInt(1,40,20) ->L1 to

select 20 subjects randomly from a group of 40. You should see the first 5 numbers in a list of 20 numbers. To see the rest of the numbers keep pressing the right arrow key.

AssignmentP. 293-2945,31-5.37 oddP. 298 5.39

Block DesignA block is a group of

experimental units or subjects that are known before the experiment to be similar in some way that is expected to systematically affect the response to the treatments. In a block design, the random assignment of units to treatments is carried out separately within each block.

Outline of a block design for comparing cancer therapies. The blocks consist of male and female subjects. The treatments are three therapies for cancer.

SoybeansBlock DesignThe soil type and fertility of farmland differs

by location. Because of this, a test of the effect of tillage (two types) and pesticide application ( three application schedules) on soybean yields uses small fields as blocks. Each block is divided into six plots, and the six treatments are randomly assigned to plots separately within each block.

Tennessee StarBlock designThe Tennessee STAR class size experiment

used a block design. It was important to compare different class types in the same school. In all 79 schools participated in the study. That is there were 79 blocks.

Remember!!!!A wise experimenter will form blocks on

the most important unavoidable sources of variability among experimental units.

Control what you can,Block what you can’t,And randomize the rest.

Matched Pairs Designs: .matching subjects in different ways can

produce more precise results than simple randomization.

The simplest use of matching is a matched pairs design, which compares just two treatments.

The subjects are matched in pairs with similar subjects and one receives the treat ment and one does not.

Cell phones and driving 2Matched pairsIn the cell phone study, two treatments

were studied. Driving in a simulator and driving a simulator while talking on a cell phone. The braking times were compared.

In a matched pairs design, all students drove with and without using the cell phone.

The order in which they drove with and without the cell phone was random to reduce the effect of it being the first time they drove the simulator.

Cereal leaf beetlesMatched pairs designAre cereal beetles more strongly attracted

by the color yellow or the color green? Farmers want to know in order to color beetle traps so that the beetles are attracted to them.

If we mount boards of both colors on each pole then the effect of location will be accounted for. We can compare the # of beetles trapped on the yellow board to the number of beetles trapped on the green board on the same pole.

Is the placebo effect pseudoscience?

14 healthy men age 20-30 were given a saltwater injection that caused pain in their jaws. They were then injected with a placebo and told it was a painkiller.

Researchers monitored their brain activity during this process.

Each man’s brain released more natural pain killing endorphins after the placebos were administered.

Cautions about ExperimentationLack of Realisms: The subjects or

treatments or setting of an experiment may not realistically duplicate the conditions we really want to study.

Statistical analysis of an experiment cannot tell us how far the results will generalize to other settings.

Placebo cigarettes?Lack of realismA study of the effects of marijuana

recruited young men who used marijuana. Some were given marijuana while others were given placebo cigarettes. The one with the placebos knew they were not real and complained loudly.

Response to advertising Lack of realismA study compares two TV ads by showing

TV programs to student subjects. The students know it is “just an

experiment”.

Genes and behavior Lack of realismTo study genetic influence on behavior,

experimentors “knock out” a gene in one group of mice and compare their behavior with that of a control group of normal mice. The same experiment was repeated in different labs across the country with different results.

Small differences in lab environments can have big effects on the behavior of the mice.

CautionStatistical analysis of

an experiment cannot tell us how far the results will generalize to the general public.

Assignment 5.2Bp,. 303-3045.43-5.47 odd

What You Should have LearnedA. Sampling

1. Identify the population in a sampling situation.2. Recognize bias due to voluntary response sampling

and other inferior sampling methods.3. Select a simple Random Sample (SRS) from a

population.4. Recognize cluster sampling and how it differs from

other sampling methods.5. Recognize the presence of undercoverage and

nonresponse as sources of error in a sample survey.6. Use Random Digits to select a stratified random

sample from a population when the strata are identified.

B. Experiments1. Recognize whether a study is an observational study or an experiment.2. Recognize bias due to confounding of explanatory variables with lurking

variables in either an observational study or an experiment.3. Identify the factors(explanatory variables), treatments, response variables, and

experimental units or subjects in an experiment.

4. Outline the design of a completely randomized experiment using a diagram. The diagram in a specific case should show the sizes of the groups, the specific treatments and the response variables.

5. Carry out the random assignment of subjects

6. Recognize the placebo effect. Recognize when the double blind technique should be used.

7. Recognize a block design and when it would be appropriate. Know when a matched pairs design would be appropriate and how to design one.

8. Explain why a randomized comparative experiment can give good evidence for cause and effect relationships.

Chapter Review Exercises