Embed Size (px)
Citation preview
Kaming Lo, M.P.H.Biostatistics Collaboration and Consulting Core
Division of BiostatisticsDepartment of Epidemiology and Public Health
Introduction
High quality research results from a comprehensive plan which involve: Population selection Randomization Methodology Measurement Tools Power and Sample Size
Why calculate sample size?
In statistics, the validity of the analysis depends upon: how much information can be used. how precise the information is.
Sample size calculations allow the investigator to: determine the minimum amount of information
needed for answering the research question.
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
Type I and Type II Errors
Type I Error, α When researcher rejects the Null Hypothesis
given the Null Hypothesis is true.
Type II Error, β When researcher does not reject/accepts
the Null Hypothesis given the Alternative Hypothesis is actually true.
Power “Probability of rejecting the null hypothesis
given that the alternative hypothesis is true.”
Compliment of Type II Error.
Reject Null Do Not Reject Null
Null is TrueType I Error
(α)Confidence Level
(1- α)
Alternativeis True
Power(1- β)
Type II Error(β)
Understanding Power (example) A population, based on previous studies, is known to have
a normal distribution with a mean of 20 and a standard deviation of 4.
An investigator was interested in another population with the same standard deviation.
He wanted to test if the mean is equal to 20.
He took a random sample of 44 from the studied population.
One sample t-test can be used to test the hypothesis:
H0: µ = 20 vs Ha: µ ≠ 20
Understanding Power (example) cont’d Based on an α of 0.05, the investigator
calculated the critical values that would allow him reject the null hypothesis, which is 18.8 and 21.2. In another words, if the mean of his sample is below 18.8 or greater than 21.2, he would get a significant difference.
Understanding Power (example) cont’d
Reject Null Do Not Reject Null
Null is TrueType I Error
(α)Confidence Level
(1- α)
Alternativeis True
Power(1- β)
Type II Error(β)
Figure 1.
What affects power? There are a few factors that would affect power:
Sample size (n)n increases Power increases
Type I error rates/significance level (α)α increases Power increases
Variability (σ)σ increases Power decreases
Effect size (Δ, it is the changes in magnitude of the outcome that is considered scientifically important.)Δ increases Power increases
Besides, power would be greater in a one-tailed test compared to a two-tailed test.
How much power is needed
Achieving 80% power is generally acceptable.
Too much or too little power could be an issue.
Underpower and Overpower
Consider a study to test for the difference between the effects of two drugs for diabetic patients. An investigator hypothesized that drug B would have a higher mean reduction on the Hemoglobin A1c than drug A by 1%.
Underpower
Too little sample.
May results in no difference between drug A and B even there may actually be some significant differences.
Wasted funds in conducting the trial that returns no meaningful results.
Overpower Too much sample.
May always find difference between drug A and B even the difference is not actually of scientific importance, e.g. the difference detect may actually be 0.3% instead of 1%.
Wasted funds in recruiting the extra subjects that are not really necessary.
Outline
Power Concerns in Sample Size Calculations Common Formulae Useful Tools Collaboration with Statistician Take Away Messages
Concerns in Sample Size Calculations
Hypotheses, both primary and secondary
Primary outcomes and variables of interest Continuous/categorical data
Effect size What is considered clinically importance?
Variability of the outcomes if continuous
Study designs, for examples: Randomized controlled trial Non-randomized trial Observational study
Data structure, for examples: Parallel data Paired data Repeated measures
Concerns in Sample Size Calculations cont’d
Outline
Power Concerns in Sample Size Calculations Common Formulas Useful Tools Collaboration with Statistician Take Away Messages
Classical formula for testing difference of two means
H0: µ1 = µ2 vs. Ha: µ1 ≠ µ2
n =Sample size need in each group σ =Common standard deviation Z1-α/2 =Standardized value at desired α Z1-β =Standardized value at desired β Δ =Effect size
Classical formula for testing difference of two proportions
H0: p1 = p2 vs. Ha: p1 ≠ p2
n =Sample size need in each group p1 =Proportion in group 1 p2 =Proportion in group 2 ṗ =(p1 + p2)/2 Z1-α/2 =Standardized value at desired α Z1-β =Standardized value at desired β Δ =Effect size
Sample Size Calculation Classical formulas have many statistical
assumptions, such as normality, independent groups, equal variance, and more.
Often not the case in the reality.
If assumptions are violated or if studies involve complex study designs or statistical analyses. Simulation maybe needed. Simplify study design.
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
Why Statisticians?
A statistician understands: Which study design and statistical method
are more effective/powerful for answering the research question.
What statistical needs should be considered during the planning phase of the study.
How to help reducing the cost while maintaining the power of the study.
What to prepare before meeting with statistician
An investigator should prepare as many of the following as possible: The study objectives The primary hypothesis Outcome variables Effect size
What to prepare before meeting with statistician cont’d
Preliminary information, which can usually be obtained through previous literature: Means/proportions Standard deviation if continuous If not readily available, one might consider a
pilot study or consult with an expert in that research area to ask for an expectation on the values.
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
Other Useful Tools Java Applet developed by Lenth RV (2006)
http://www.cs.uiowa.edu/~rlenth/Power/ Epi Info by CDC
Commercial Software: SAS SPSS nQuery PASS
Outline
Power Concerns in Sample Size Calculations Common Formulae Collaboration with Statistician Other Useful Tools Take Away Messages
Take Away Messages Understanding what affects power is the key to
determine the best sample size. Different factors (α, σ, Δ, etc) Study designs and data structures. Statistical methods.
Formulas don’t always work! (Beware of the assumptions behind them).
Take Away Messages cont’d Seek inputs from a statistician if in doubt
The earlier involvement of a statistician in a study the better.
Can assure a higher statistical power by choosing effective study design and analysis approach.
A researcher who comes to a statistician prepared will get the best results from the consultation Knowing the hypothesis, primary outcomes, effect size Research on any preliminary data (mean, sd, proportions,
etc)
Biostatistics Collaboration and Consulting Core (BCCC)
Mission Statement:To assure that the appropriate statistical methodology is incorporated in research.
The BCCC operates as a cost center, offering support activities to faculty, staff, and students. All fees are based on UM policy B020 for Recharge or Cost Centers.
BCCC Activities:1. Study Design 7. Abstract/Manuscript Preparation2. Randomization Schemes 8. Grant Preparation3. Statistical Analysis Plan (SAP) 9. Survey/Questionnaire Design4. Sample Size Estimation or Power Analysis 10.Protocol Review5. Statistical Analysis 11.Safety Committee6. Consulting Statistician for Staff and Professional Meetings
BCCC Support: BCCC Free Support:Short Term Quick Consulting – 30 minutesGrantsOngoing Collaboration Plan Initial Meeting – 1 hours – Project Initiation and Agreement
BCCC Contact Information:
Clinical Research Building, 10th Floor1120 N.W. 14th Street (R-669),
Miami, FL 33136
Contact person: Maria Jimenez-RodriguezTel: 305-243-4465
E-mail: [email protected]: www.biostat.med.miami.edu/core
