Designing ExperimentsThe Caffeine Experiment

Designing Samples Population vs Sample

Is a part of the population that

we actually examine in

order to gather information

Entire group of individuals that

we want information

from.

Sampling - involves studying a part in order to gain information about the whole

Census - attempts to contact every individual in the entire population

BiasThe design of a study

is BIASED if it systematically favors

certain outcomes

Bias on Wording

Confusing or leading questions can introduce strong bias, and even minor changes in wording

can chance the survey’s outcome

Never trust the results of a sample survey until you have read the exact

questions posed.

Designing Samples1. SRS Simple Random Sample is choosing a

sample by equal chance.

2. Stratified Random Sample

Grouping sample with similar individuals called strata. Then choosing an SRS in each stratum.

3. Multi-stage samplesGrouping sample into clusters, then employing an SRS, Stratified RS, or another type of sampling design

Placebo Effect

Placebo Is Something That Is Identical (in appearance, taste, smell, etc.) To The treatment received by the treatment group, except that it contains no active ingredients.

When a person doesn’t know who is receiving which treatment, that person is BLIND.

•When every individual in one of these classes is

blinded, the experiment is called SINGLE

BLIND.

•If every individual in both classes is blinded,

then the experiment is DOUBLE BLIND

Four Key Principles of a Good Experiment:

Principle #1: DIRECT CONTROL means holding extraneous variables constant for all treatment groups so that their effects are not confounded with the explanatory variable.

Principle #2: BLOCKING is when subjects are divided into homogeneous groups (blocks) based on some extraneous variable and then separated into different treatment groups.

What if men have different pulse rates than women? • The differences in pulse rate due to gender will be an additional source of variability in our experiment, which will make it harder to see the difference due to the treatment.

How can we eliminate this source of variability?

• Eliminate one gender from the study, but then we could only draw conclusions about one gender

• Make sure there is a representative number of men and women in each treatment. For example, if there are 20 women and 30 men in the experiment, then the experimental group should have10 women and 15 men and the control group should have the same.

• In this example, we have formed 2 blocks: men and women. Then, we assigned treatments to the subjects within each block.

Principle #3: RANDOMIZATION is random assignment of subjects to treatments to ensure that the experiment doesn’t systematically favor one treatment over the other.

• Draw names from a hat. The first half chosen are in one group, the remaining names in the other.

• Number the class from 01-36. Then, generate random numbers without replacement until half are chosen for one group. The remaining names go in the other group.

• For matched pairs we can flip a coin to determine which subjects go into which group. If its heads, the first person in the pair goes to A and the other to B. If its tails, it’s the opposite.

How do we randomize?

Principle #4: REPLICATION means ensuring that there is an adequate number of observations in each treatment group.

Increasing the SAMPLE SIZE makes randomization more

effective. The more subjects we have, the more balanced

our treatment groups will be. For example, if we have 10

subjects and only 2 have a certain unknown characteristic,

it is quite likely that both of those subjects will end up in

the same treatment group simply by chance.

Classwork1. EXPLAIN WHY BLINDING IS REASONABLE STRATEGY IN MANY EXPERIMENTS?

2. GIVE AN EXAMPLE OF AN EXPERIMENT FOR EACH OF THE FOLLOWING:

•SINGLE-BLIND EXPERIMENT WITH THE SUBJECT BLINDED

•SINGLE-BLIND EXPERIMENT WITH THE INDIVIDUALS MEASURING THE RESPONSE BLINDED

•DOUBLE-BLINDED EXPERIMENT

•AN EXPERIMENT THAT IS NOT POSSIBLE TO BLIND