2. Start with Hypothesis Testing
You go through the 6 steps of Hypothesis Testing and get a
significant result
So what?
The probability of obtaining a significant result is highly
correlated with the size of the sample
More numbers = increase probability of significance
Sometimes the impact is meaningless
3. Significance
Statistical Significance
Reached a result that has a probability of less than 5% that it is
due to error
Practical Significance
Looks at whether the difference is useful in the real world
http://www.youtube.com/watch?v=rOyK_K0SOaU
4. Effect Size
d = Cohens Effect Size
1 = Mean of Group 1
2 = Mean of Group 2
= Standard Deviation
5. Effect Size
Results in a measure of standard deviation
Can be interpreted as noted by Cohen on the left
A large effect size would mean there would be a large impact on the
population
6. Effect Size
Used to compare studies
Can have different
Methods
Statistics
Metrics
Only measuring the difference in mean 1 and mean 2 divided by
SD
7. Visible Learning
Meta Analysis of 800 Meta Analyses
Summary of many articles on the topic
Uses Effect Sizes
1 effect size = 1 standard deviation
1 standard deviation = 2 years increase
Cohen said Blatantly obvious
PhD Students vs High School Students on IQ
8. Visible Learning
Distribution of Effect Sizes, Chapter 2, pg 16
Most have a positive effect
Doing almost anything increases achievement
Set the bar at d = 0.40
Average of all effect sizes
This is not just placing a teacher in front of a room
Benchmark to notice real differences