View
222
Download
0
Category
Preview:
Citation preview
© Nancy E. Mayo 2004
Sample Size Estimations
Demystifying Sample Size Calculations
Graphics contributed by
Dr. Gillian Bartlett
© Nancy E. Mayo 2004
Choosing the Study Population
Question
Background
ReasonableQuestion
Population
© Nancy E. Mayo 2004
Study Population
?
Sam
ple
Size
Exposure
Outcom
e
Confo
undi
ng
Analysis
© Nancy E. Mayo 2004
COMMON QUESTIONS1. How many subjects (specimens) do I need?2. How do I analyze my data?3. What do I put in the data analysis section?
COMMON ANSWERS1. What is your question?2. What is your outcome?3. How is it measured?4. How big an effect do you want to see?5. Is the effect meaningful?
© Nancy E. Mayo 2004
Clinically Meaningful Change
Meaningful to whom?• Clinician - usually impairments• Patient – function (disability),
quality of life• Society - health services utilization,
cost• Payer – disability, prescription
medication
© Nancy E. Mayo 2004
Clinically Meaningful Change
• Norm referenced– refers to changes that would put
someone within normal values or within a % of normal
• Criterion referenced– change anchored in future benefit– change is associated with increased
probability of distant outcomes – relevant when impact is on pathology
but benefit not reaped for years
© Nancy E. Mayo 2004
Clinically Meaningful Change
• Content referenced– for outcomes measured by scales– translates change into what would
have had to have changed on the scale– e.g. 5 points on Barthel Index -
changed 1 level on 1 item.
• Minimally detectable change– Subjects can detect improvement
© Nancy E. Mayo 2004
How BIG is BIG?
Effect sizeEffect size: ratio of change to variability0.2 - 0.3 – small0.5 – moderate
0.8 - large
© Nancy E. Mayo 2004
signal is difficult to detect against excessive background
noise
Change greater than “noise”
© Nancy E. Mayo 2004
Raw vs. Cooked Data (order rare)
Mean 0.55SD 0.51Ratio 1.1
Mean 50.6SD 15.8Ratio 3.2
166
153
169
156
178
033
029
048
155
035
151
150
049
034
022
178
156
168
043
039
Cooked Data(<50, >=50)Raw Data
Mean 0.55SD 0.51Ratio 1.1
Mean 50.6SD 15.8Ratio 3.2
166
153
169
156
178
033
029
048
155
035
151
150
049
034
022
178
156
168
043
039
Cooked Data(<50, >=50)Raw Data
© Nancy E. Mayo 2004
Examples of the Pitfalls of Cooking Data
Mean 0.55SD 0.51Ratio 1.1
Mean 50.6SD 15.8Ratio 3.2
178
178
169
168
166
156
156
155
153
151
150
049
048
043
039
035
034
033
029
022
Cooked Data(<50, >=50)Raw Data
Mean 0.55SD 0.51Ratio 1.1
Mean 50.6SD 15.8Ratio 3.2
178
178
169
168
166
156
156
155
153
151
150
049
048
043
039
035
034
033
029
022
Cooked Data(<50, >=50)Raw Data
© Nancy E. Mayo 2004
Sample Size Formula = SD / delta
Effect size = delta / SD
Delta = difference
DEMYSTIFIED
© Nancy E. Mayo 2004
Relationship between Effect Size
and Sample Size
0 0.5 1 1.5 2 2.5
100
80
60
40
20
0
Sam
ple
Siz
e p
er
Gro
up
Effect Size
(Two group design)
© Nancy E. Mayo 2004
Calculation of Sample Size for Comparing Two Independent
Means ( za – zb ) SD
n = 2 ___________ xexp - xcon
Where:Za = z value for the risk of a Type I error (significance level)
1.96 for 0.05Zb = z value for the risk of a Type II error (power)
1.96 for 0.95 (two-tailed) -1.65 for 0.95 (one-tailed)
SD = standard deviation of outcome in the general population
xcon = mean of control group
xexp = mean of experimental group
n = number of subjects per group
2
© Nancy E. Mayo 2004
Calculation of Sample Size for Comparing Two Independent
Proportionsn = za √ 2 pcon (1 - pcon ) – zb √ pexp (1 – pexp ) + pcon (1 - pcon )
2______________________________________________
pexp - pcon
Where:za = z value for the risk of a Type I error (significance level)
1.96 for 0.05zb = z value for the risk of a Type II error (power)
1.96 for 0.95 (two-tailed) -1.65 for 0.96 (one-tailed)
pcon = prevalence of outcome in control group
pcon = prevalence of outcome in experimental group
n = number of subjects per group
Colton (pg 168-169)
© Nancy E. Mayo 2004
Sample Size Required Per Group for Comparing Two Independent
MeansPOWER
.80 .90 .95
0.50 (2.0) 5 7 8
1.0 (1.0) 17 23 27
1.25 (0.8) 26 34 42
1.50 (0.67) 37 49 60
2.0 (0.5) 60 86 105
Rati
o o
f S
D t
o d
iffere
nce ∆
b
etw
een
mean
s (∆
/SD
)
© Nancy E. Mayo 2004
Sample Size Required Per Group for Comparing Two Independent Proportions: 80%
Power
2517.90
4528.80
6636.75
10349.70
18370.65
408107.60
18668.55
4081034525.50
15741766230.45
3769138.40
141715151.35
3137243.30
113411359.25
21988.20
726160.15
475.10
.50.40.30.20.10.05
PREVALENCE OF OUTCOME IN CONTROL GROUP
2517.90
4528.80
6636.75
10349.70
18370.65
408107.60
18668.55
4081034525.50
15741766230.45
3769138.40
141715151.35
3137243.30
113411359.25
21988.20
726160.15
475.10
.50.40.30.20.10.05
PREVALENCE OF OUTCOME IN CONTROL GROUP
Pre
vale
nce o
f ou
tcom
e i
n e
xp
eri
men
tal
gro
up
© Nancy E. Mayo 2004
Pre
vale
nce o
f ou
tcom
e i
n e
xp
eri
men
tal
gro
up
3625.90
7041.80
1035534.75
1637543.70
2931095633.65
6611707541.60
263029910753.55
6611637036.50
25782819745.45
60914458.40
231824178.35
50511165.30
8̀5017790.25
349137.20
1174251.15
758.10
.50.40.30.20.10.05
PREVALENCE OF OUTCOME IN CONTROL GROUP
3625.90
7041.80
1035534.75
1637543.70
2931095633.65
6611707541.60
263029910753.55
6611637036.50
25782819745.45
60914458.40
231824178.35
50511165.30
8̀5017790.25
349137.20
1174251.15
758.10
.50.40.30.20.10.05
PREVALENCE OF OUTCOME IN CONTROL GROUP
Sample Size Required Per Group for Comparing Two Independent Proportions: 95% Power
More complex data situations
• Convert each component to the simple 2-group comparison or correlation
• Estimate (calculate) sample size for the contrast that has the smallest effect and build up
• Remember if using correlation as the base, you are not testing it against 0 you are testing it against a correlation that you do not think is important
© Nancy E. Mayo 2004
…More
• Consider the impact on power to maintain a given effect size if other variables are in the model
© Nancy E. Mayo 2004
Regression
• Green indicates that adequate power (80%) can be achieved for moderate effect sizes with a sample size N > 50 + 8m, where m is the number of covariates to be modeled.
© Nancy E. Mayo 2004
Adjustment only
• no parameters are estimated• no hypotheses tested• to maintain the same degree of
power, only 1 additional subject is required per level (l) or per degree of freedom (df) inherent in the co-variate
© Nancy E. Mayo 2004
Summary
• Variable under study• N > 50 + 8m (moderate effect size, 80%
power)• Adjustment only• Continuous = + 1 per covariate df
Dichotomous = 5-9 events per covariate • Sub-group analysis• Sample size for main effect * 4 for
interaction with group © Nancy E. Mayo 2004
© Nancy E. Mayo 2004
Marking Scheme for Protocol
• Background 10• Question 5• Population 5 • Design 5• Procedures 5• Measures 10• Analysis 5• Sample Size 5• Bonus points – above and beyond the
call of duty
Recommended