View
28
Download
0
Category
Tags:
Preview:
Citation preview
Pulmonary Board Review:Study Design and Statistical Principles
Terry Shaneyfelt, MD, MPHUAB Division of General Internal Medicine
@EBMTeacher
EBMTeacher.com
UABEBMcourseYouTube logo image credit (CC0): http://commons.wikimedia.org/wiki/File:Youtube.svg
Topics to be covered Diagnostic testing
Choosing a test and interpreting studies
Randomized controlled trialsRandomization, power, types 1 and 2 errors,
outcome measures
Observational studiesStudy design, RR, OR
ScreeningOutcomes and biases
Diagnostic Testing Choosing a test
Sensitivity, specificity, likelihood ratiosSpPin and SnNout
Interpreting the results of a diagnostic test studyPositive and negative predictive values
What is the role of testing?
35yo F with 2% pretest probability
48yo M with 27% pretest probability
75yo M with 99% pretest probability
Rule in with high pretest probabilityRule out with low pretest probability
Which test would you choose to rule in disease?
Sensitivity Specificity LR + LR -
Test A 95% 80% 4.75 0.06
Test B 90% 90% 9 0.11
Test C 70% 95% 14 0.32
Which test would you choose to rule out disease?
Rule in with specific testsRule out with sensitive tests
SpPin ( A specific test, if positive, rules in disease in a high risk person)
SnNout (A sensitive test, if negative, rules out disease in a low risk person)
Alternatively: Choose test with highest positive LR (to rule in) and/or lowest negative LR (to rule out)
Sensitivity Specificity LR + LR -
Test A 95% 80% 4.75 0.06
Test B 90% 90% 9 0.11
Test C 70% 95% 14 0.32
To learn more watch my other videos
Sensitivity: http://bit.ly/1FOlqry
Specificity: http://bit.ly/1IYjv2A
LR: http://bit.ly/1JOofZz
CTA High Pretest Probability
IntermediateLow Pretest Probability
Positive Predictive Value
(%)96 92 58
Negative Predictive Value
(%)60 89 96
Adapted from Table 5 PIOPED II (NEJM 2006;354:2317)
In patients with high pretest probability what is the chance that the patient has a PE with a negative
CTA?
Sens 83%Spec 96%
Predictive value varies with prevalence
To learn more watch my other videos
PPV: http://bit.ly/1HDXWpm
NPV: http://bit.ly/1Faegbq
RCTs Study design
Randomization
Power, type 2 error, and sample size
p-values and type I error
Outcome measuresRRR, ARR, HR, NNT
Image from PrevMedFellow (CC A SA license): http://commons.wikimedia.org/wiki/File:Flowchart_of_Phases_of_Parallel_Randomized_Trial_-_Modified_from_CONSORT_2010.png
Control
What do you think is the greatest risk of bias in a therapy study?
A. Failure to randomize
B. Failure to conceal allocation
C. Failure to blind participants and study personnel
D. Failure to use intention to treat analysis
E. Failure to treat groups equally except for the intervention
2 Reasons:
1. Reduces selection bias
2. Equally distributes prognostic factors (both known and unknown)
Why Is Randomization So Important?
The validity of a clinical trial depends on treated & control patients being prognostically equal, other than
the intervention being tested
TRUTHDifference No difference
Study Conclusion
Difference
No difference
Beta/ Type II error
Alpha/ Type I error
We estimated that with enrollment of 1130 subjects, the study would have 90% power to show a significant difference between the two groups in the time to the first acute exacerbation of COPD, assuming that 50% of the participants in the control group and 40% in the azithromycin group would have an acute exacerbation, that the rate of nonadherence would be 20%, and that 6% of participants would die or be lost to follow-up during the study, with a two-sided type I error of 0.05.
Azithromycin for Prevention of Exacerbations of COPD NEJM 2011;365:689
Power
Power (greater the desired power the greater the sample size)
Estimated difference between groups (smaller the difference the greater the sample size)
Type 1 error rate (usually 0.05 but the smaller the greater the sample size)
Variability in the measurements made within each comparison group (greater the variability the greater the sample size)
Sample size is affected by…
Azithromycin(# of events)
Placebo(# of events)
Hazard Ratio(95% CI)
p-value
Hospitalization related to COPD
156 2000.82
(0.64 - 1.07)0.15
ED or urgent care visit
199 2570.81
(0.63 - 1.04)0.09
Unscheduled office visit
1202 13450.85
(0.74 - 0.98)0.02
Adapted from Table 2 from NEJM 2011; 365:689
What does a HR of 0.82 mean?
What does the confidence interval mean?
Where do p-values come from?
Statistical Approach to Compare 2 Groups
Calculate:1. Main effect2. Variance in main
effect
State a null hypothesis (the main effect is 0)
Calculate the test statistic to
determine p value
Calculate the 95% confidence interval around the main
effect
New Drug
Placebo
Statistical Tests Mathematical formulas that produce test
statistics to assess the likelihood that chance (or sampling error) accounts for the results observed in the study
Many different tests. Choice depends on several factors:
Type of data (continuous, dichotomous, etc) Distribution of data (normally distributed or not) Study design (# of groups, etc)
TRUTHDifference No difference
Study Conclusion
Difference
No difference
Beta/ Type II error
Alpha/ Type I error
2 errors can be made with hypothesis testing
P-value
Probability that the results seen (or one more extreme) could have occurred by chance alone
○ Assuming that there is in fact no difference between groups (null hypothesis)
Cannot tell you if there is bias in a study
Does not indicate clinical significance
To learn more watch my other videos
NNT: http://bit.ly/1F4xONy
RRR: http://bit.ly/1Fyef3F
Used to determineprognosis & harm
Outcome Measure: RR
Population
Exposed
Outcome
No outcome
Unexposed
Outcome
No outcome
Time
Cohort Study
Establish incidence (risk) directly
Multiple outcomes
Study of rare exposures
Strengths Weaknesses
Not good for rare diseases
Not good for diseases that take a long time to develop
Can’t study multiple exposures
Cohort: Strengths & Weaknesses
The incidence of pulmonary embolism in the COPD cohort was 1.37 per 10,000 persons/year and in the non-COPD cohort was 0.35 per 10,000 persons/year.
Multiple ways to express riskIncidenceRisk difference (attributable risk)Relative risk (risk ratio)
Interpreting RRRR = 1 (no association)RR > 1 (increased risk of disease)RR < 1 (decreased risk of disease)
http://bit.ly/1dtFFhV
Outcome Measure: OR
Exposed
Diseased
(Cases)
Non-Diseased
(Controls)
Non- Exposed
Exposed
Non-Exposed
Time
Case-Control Study
Good for diseases with long latency
Good for rare diseases
Can determine multiple exposures
Faster results
Strengths Weaknesses
Can’t establish estimate of risk directly nor determine prevalence
Can only study one disease
More prone to bias
Case-Control: Strengths & Weaknesses
Low-dose glucocorticoid use (prednisolone daily dose equivalent 5 mg) carried a twofold increased risk of PE (OR, 1.8; 95% CI, 1.3-2.4), whereas a 10-fold increased risk was observed for the highest dose of glucocorticoids (prednisolone 30 mg) (OR, 9.6; 95% CI, 4.3-20.5). The authors are incorrect in the statements of risk. Do you know why?
Can only determine relative frequency of exposure among cases and controlsOdds ratio
Interpreting OROR = 1 (no difference of exposure)OR > 1 (frequency of exposure higher among cases)OR < 1 (frequency of exposure lower among cases)
http://bit.ly/1HHm2Nd
Screening
Prevalence vs incidence screens
Outcomes of screening studies
BiasesLead timeLength timeOverdiagnosis
Identification of disease or a risk factor in asymptomatic individuals
Screening
Biologic Onset Outcomes
Clinical diagnosis
Screen detection
Fundamental Principles of Screening-1
3 prerequisites:▪ Disease must have a great enough burden of
suffering
▪ Screening test can identify disease earlier than usual
▪ Earlier therapy leads to better outcomes
Fundamental Principles of Screening-2
Target disorders are relatively rare (low prevalence)
Must screen large numbers of people Most positive tests are false positives
Risks of screening tend to be rare but apply to all
Benefits accrue only to a few
Disease Prevalence is Low LOW low predictive value
Sensitivity=95%Specificity=95%
LR=19
Prevalence Predictive value
10% 67%
1% 16%
0.1% 1.8%Watch Predictive Value Estimates From Studies Can Be Misleading
http://youtu.be/3zq82uiGS3o
When choosing a test for a screening program you want the test to be….?
1. Highly sensitive
2. Highly specific
Do you find more cases of disease on the first round of screening or subsequent rounds?
Dx
Dx
Dx
Dx
Dx
Dx
Dx
Dx
Dx
Dx
Dx
Dx
1 2 3Round of screening
Number of cases newly detected
5 3 2
What is the appropriate outcome of a screening study?
A. Survival?
B. Mortality?
C. Disease detected?
Solving lead time bias problem: compare age-specific mortality between screened and unscreened. Not survival! Count from date of randomization
Adapted from Clinical Epidemiology The Essentials 3rd edition
Dx †
Dx †
Unscreened
Screened but early Rx ineffective
Ons
et
Early
Usu
al
Lead Time Bias
Compare outcomes via RCT with a control group and a group offered screening
Count all outcomes regardless of method of detection
Avoiding Length Time Bias
Recommended