More About Factorial Design

• Suppose experiment tests whether taking a game theory class causes a person to get more papers published

• We cannot simply examine whether people who took a game theory class had more papers published due to possible effect of self-selection into game theory class.

• Therefore, we should randomly assign people to game theory class.

Does Game Theory Lead to More Articles?

• DV: Number of papers published 4 years later• IV: Took game theory class or not

• Two groups: 1. Take game theory class2. Did not take game theory class

Does Class on Experiments Lead to More Articles?

• We also may want to determine whether taking a class on experiments helps a person get more articles published.

• Now we randomize whether a person takes a class on experiments

2 x 2 Factorial Design

Class Taken Game Theory

No Game Theory

Experiments

No Experiments

Now we Have 4 groups in the experiment, randomly assigned.Each cell should have equal numbers of subjects

2 x 2 x 3 Factorial

• Now we randomly assign whether a person studies international relations, public policy, or economics.

• This is now a 2 x 2 x 3 factorial design with 12 groups

Economics Game Theory

No Game Theory

Experiments

No Experiments

Public Policy Game Theory

No Game Theory

Experiments

No Experiments

International Relations

Game Theory

No Game Theory

Experiments

No Experiments

Power Analysis: How Many Subjects Needed?

• Example: G*Power (free!)• Hypothesize the magnitude of the effect (such

as one additional paper published)• Calculate sample size needed for two

independent samples difference in means (average number of articles published for each of two groups)

• Answer will be number of subjects needed in each group

Natural Experiments

Exploiting Randomization When It Occurs

Natural Experiments

• Experimenter does not pre-plan experiment

• Experimenter finds a naturally-occurring random assignment mechanism that serves as a treatment

Why Randomization?

• Randomization allows comparison across groups by controlling for differences in other variables.

• The expected values of other variables are the same between across randomly-assigned treatment groups.

• If other variables have equal expected values across treatments, then differences in outcomes must be due to treatment variable

Dartmouth Roommate “Experiment”

• First-year roommates at Dartmouth College assigned randomly within “like-groups” based on gender and whether student– Smokes– Listen to music while studying– Stay up late– Are neat or messy

• Assignment random within 25=32 different groups

Treatment=Roommate

• In this case, the treatment variable is a student’s roommate

• The experiment does not have a true “control group” but many different treatments in the form of roommates

• Also in this case, the roommates are “treatments” for each other

First Step: Is Assignment Truly Random?

• Are there any variables that determine one’s assigned roommate other than– Gender– Smoking– Late night habits– Cleanliness– Listening to music

DV’s and IV’s

• The DV in this case is a student’s GPA• The most important IV is the roommate’s GPA

• Other IV’s included as control variables are a student’s own academic index (entrance exam scores and high school grades) and roommate’s academic index

Statistical Model

Conclusions

• Roommates influence each other’s performance in school– Top performers reinforce each other– Bottom performers reinforce each other

• Roommates also influence choice of social group such as fraternity or sorority

• Roommates do not influence each other’s choice of major

Other Examples of Natural Experiments