© Nancy E. Mayo 2004 Sample Size Estimations Demystifying Sample Size Calculations Graphics contributed by Dr. Gillian Bartlett

© Nancy E. Mayo 2004

Sample Size Estimations

Demystifying Sample Size Calculations

Graphics contributed by

Dr. Gillian Bartlett


Choosing the Study Population

Question

Background

ReasonableQuestion

Population


Study Population

?

Sam

ple

Size

Exposure

Outcom

e

Confo

undi

ng

Analysis


COMMON QUESTIONS1. How many subjects (specimens) do I need?2. How do I analyze my data?3. What do I put in the data analysis section?

COMMON ANSWERS1. What is your question?2. What is your outcome?3. How is it measured?4. How big an effect do you want to see?5. Is the effect meaningful?


Clinically Meaningful Change

Meaningful to whom?• Clinician - usually impairments• Patient – function (disability),

quality of life• Society - health services utilization,

cost• Payer – disability, prescription

medication



• Norm referenced– refers to changes that would put

someone within normal values or within a % of normal

• Criterion referenced– change anchored in future benefit– change is associated with increased

probability of distant outcomes – relevant when impact is on pathology

but benefit not reaped for years



• Content referenced– for outcomes measured by scales– translates change into what would

have had to have changed on the scale– e.g. 5 points on Barthel Index -

changed 1 level on 1 item.

• Minimally detectable change– Subjects can detect improvement


How BIG is BIG?

Effect sizeEffect size: ratio of change to variability0.2 - 0.3 – small0.5 – moderate

0.8 - large


signal is difficult to detect against excessive background

noise

Change greater than “noise”


Raw vs. Cooked Data (order rare)

Mean 0.55SD 0.51Ratio 1.1


166

153

169

156

178

033

029

048

155

035

151

150

049

034

022

178

156

168

043

039

Cooked Data(<50, >=50)Raw Data



166

153

169

156

178

033

029

048

155

035

151

150

049

034

022

178

156

168

043

039



Examples of the Pitfalls of Cooking Data



178

178

169

168

166

156

156

155

153

151

150

049

048

043

039

035

034

033

029

022




178

178

169

168

166

156

156

155

153

151

150

049

048

043

039

035

034

033

029

022



Sample Size Formula = SD / delta

Effect size = delta / SD

Delta = difference

DEMYSTIFIED


Relationship between Effect Size

and Sample Size

0 0.5 1 1.5 2 2.5

100

80

60

40

20

0

Sam

ple

Siz

e p

er

Gro

up

Effect Size

(Two group design)


Calculation of Sample Size for Comparing Two Independent

Means ( za – zb ) SD

n = 2 ___________ xexp - xcon

Where:Za = z value for the risk of a Type I error (significance level)

1.96 for 0.05Zb = z value for the risk of a Type II error (power)

1.96 for 0.95 (two-tailed) -1.65 for 0.95 (one-tailed)

SD = standard deviation of outcome in the general population

xcon = mean of control group

xexp = mean of experimental group

n = number of subjects per group

2


Calculation of Sample Size for Comparing Two Independent

Proportionsn = za √ 2 pcon (1 - pcon ) – zb √ pexp (1 – pexp ) + pcon (1 - pcon )

2______________________________________________

pexp - pcon

Where:za = z value for the risk of a Type I error (significance level)

1.96 for 0.05zb = z value for the risk of a Type II error (power)

1.96 for 0.95 (two-tailed) -1.65 for 0.96 (one-tailed)

pcon = prevalence of outcome in control group

pcon = prevalence of outcome in experimental group

n = number of subjects per group

Colton (pg 168-169)


Sample Size Required Per Group for Comparing Two Independent

MeansPOWER

.80 .90 .95

0.50 (2.0) 5 7 8

1.0 (1.0) 17 23 27

1.25 (0.8) 26 34 42

1.50 (0.67) 37 49 60

2.0 (0.5) 60 86 105

Rati

o o

f S

D t

o d

iffere

nce ∆

b

etw

een

mean

s (∆

/SD

)


Sample Size Required Per Group for Comparing Two Independent Proportions: 80%

Power

2517.90

4528.80

6636.75

10349.70

18370.65

408107.60

18668.55

4081034525.50

15741766230.45

3769138.40

141715151.35

3137243.30

113411359.25

21988.20

726160.15

475.10

.50.40.30.20.10.05

PREVALENCE OF OUTCOME IN CONTROL GROUP

2517.90

4528.80

6636.75

10349.70

18370.65

408107.60

18668.55

4081034525.50

15741766230.45

3769138.40

141715151.35

3137243.30

113411359.25

21988.20

726160.15

475.10

.50.40.30.20.10.05


Pre

vale

nce o

f ou

tcom

e i

n e

xp

eri

men

tal

gro

up


Pre

vale

nce o

f ou

tcom

e i

n e

xp

eri

men

tal

gro

up

3625.90

7041.80

1035534.75

1637543.70

2931095633.65

6611707541.60

263029910753.55

6611637036.50

25782819745.45

60914458.40

231824178.35

50511165.30

8̀5017790.25

349137.20

1174251.15

758.10

.50.40.30.20.10.05


3625.90

7041.80

1035534.75

1637543.70

2931095633.65

6611707541.60

263029910753.55

6611637036.50

25782819745.45

60914458.40

231824178.35

50511165.30

8̀5017790.25

349137.20

1174251.15

758.10

.50.40.30.20.10.05


Sample Size Required Per Group for Comparing Two Independent Proportions: 95% Power

More complex data situations

• Convert each component to the simple 2-group comparison or correlation

• Estimate (calculate) sample size for the contrast that has the smallest effect and build up

• Remember if using correlation as the base, you are not testing it against 0 you are testing it against a correlation that you do not think is important


…More

• Consider the impact on power to maintain a given effect size if other variables are in the model


Regression

• Green indicates that adequate power (80%) can be achieved for moderate effect sizes with a sample size N > 50 + 8m, where m is the number of covariates to be modeled.


Adjustment only

• no parameters are estimated• no hypotheses tested• to maintain the same degree of

power, only 1 additional subject is required per level (l) or per degree of freedom (df) inherent in the co-variate


Summary

• Variable under study• N > 50 + 8m (moderate effect size, 80%

power)• Adjustment only• Continuous = + 1 per covariate df

Dichotomous = 5-9 events per covariate • Sub-group analysis• Sample size for main effect * 4 for

interaction with group © Nancy E. Mayo 2004


Marking Scheme for Protocol

• Background 10• Question 5• Population 5 • Design 5• Procedures 5• Measures 10• Analysis 5• Sample Size 5• Bonus points – above and beyond the

call of duty

Documents

© Nancy E. Mayo 2004 Sample Size Estimations Demystifying Sample Size Calculations Graphics contributed by Dr. Gillian Bartlett