www.metcardio.org

What the clinician needs to know about statistics: when is when is biostatistics not to be trusted?biostatistics not to be trusted?

Giuseppe Biondi ZoccaiGiuseppe Biondi Zoccai

Division of Cardiology, University of Turin, Turin, ItalyDivision of Cardiology, University of Turin, Turin, Italy

Meta-analysis and Evidence-based medicine Training in Meta-analysis and Evidence-based medicine Training in Cardiology (METCARDIO), Ospedaletti, ItalyCardiology (METCARDIO), Ospedaletti, Italy

2nd Fellows’ Meeting, 2-3 October 2009, Bubbio – 3 October 2009, 10:30-10:45

www.metcardio.org

LEARNING GOALS

• What are the goals of biostatistics in clinical research?• Is there a hierarchy in

biostatistics?• When is biostatistics not to be

trusted?

www.metcardio.org

LEARNING GOALS

• What are the goals of biostatistics in clinical research?• Is there a hierarchy in

biostatistics?• When is biostatistics not to be

trusted?

www.metcardio.org

GOALS OF BIOSTATISTICS• Biostatistics is mainly used for causality

appraisal in clinical research• Biostatistics alone cannot however enable

causality inference (i.e. necessary but not sufficient)• As it works within Popper’s approach, it can

only ultimately falsify and never actually prove

www.metcardio.org

KARL POPPER’S EPISTEMOLOGY

You can never prove that something is correct in science, you can only disprove something, i.e. show it is wrong

Thus, only falsifiable hypotheses are scientific*(and that’s why religion is not scientific)

www.metcardio.org

• Strength:* precisely (p<0.05) defined and strong relative risk (≤0.83 or ≥1.20) in the absence of multiplicity issues (strongest)

• Consistency:* finding of association needs to be replicated in other studies

• Temporality: to infer causality, exposure must appropriately precede outcome

• Coherence: makes cause-effect relationship should not conflict with natural history/biologic facts

Mente et al, Arch Intern Med 2009

BRADFORD HILL CRITERIA FOR CAUSATION

*STATISTICS PLAYS A ROLE HERE

www.metcardio.org

• Biological gradient:* dose of exposure and risk of disease are positively (or negatively) related

• Experiment: experimental evidence from laboratory studies (weak) or randomized clinical trials (strongest)

• Specificity: specific exposure is related to 1 disease only (NA for multifactorial diseases)

• Plausibility: makes biological/clinical sense (weak)• Analogy: arguing on analogical reasoning (weakest)

BRADFORD HILL CRITERIA FOR CAUSATION

Mente et al, Arch Intern Med 2009

*STATISTICS PLAYS A ROLE HERE

www.metcardio.org

LEARNING GOALS

• What are the goals of biostatistics in clinical research?• Is there a hierarchy in

biostatistics?• When is biostatistics not to be

trusted?

www.metcardio.org

EBM HIERARCHY OF EVIDENCE1. N of 1 randomized controlled trial

2. Systematic reviews of homogeneous randomized trials

3. Single randomized trial

4. Systematic review of observational studies addressing patient-important outcomes

5. Single observational study addressing patient-important outcomes

6. Physiologic studies (eg blood pressure, cardiac output, exercise capacity, bone density, and so forth)

7. Unsystematic clinical observations

Guyatt and Rennie, Users’ guide to the medical literature, 2002

www.metcardio.org

PARALLEL HIERARCHY OF RESEARCH

Biondi-Zoccai et al, Ital Heart J 2003

Qualitative reviews

Systematic reviews

Meta-analyses from individual studies

Meta-analyses from individual patient data

Case reports and series

Observational studies

Observational controlled studies

Single center randomized controlled trials

Multicenter randomized controlled trials

MORE FLEXIBLE BUT LESS VALIDMORE FLEXIBLE BUT LESS VALID

LESS FLEXIBLE BUT MORE VALIDLESS FLEXIBLE BUT MORE VALID

www.metcardio.org

IS A RANDOMIZED TRIAL ALWAYS NEEDED?

Smith et al, BMJ 2003

WE WERE UNABLE TO IDENTIFY ANY RCT OF PARACHUTE INTERVENTION FOR GRAVITATIONAL CHALLENGE… WE THINK THAT EVERYONE MIGHT BENEFIT IF THE MOST RADICAL PROTAGONISTS OF EBM ORGANIZED AND PARTICIPATED IN A DOUBLE BLIND, RANDOMIZED, PLACEBO CONTROLLED TRIAL OF PARACHUTE INTERVENTION

www.metcardio.org

LEARNING GOALS

• What are the goals of biostatistics in clinical research?• Is there a hierarchy in

biostatistics?• When is biostatistics not to be

trusted?

www.metcardio.org

THE INAPPROPRIATE USE OF A FANCY STATISTICAL TOOL

Cepeda et al, Am J Epidemiol 2003

PROPENSITY SCORES ARE USELESS IN LARGE STUDIES WITH AN ADEQUATE (>7) NUMBER OF EVENTS PER COVARIATE

www.metcardio.org

THE WEAK STATISTICAL EVIDENCE

Hannan et al, Circulation 2006

DO YOU TRUST A CONFIDENCE INTERVAL REACHING 1.01 (OR A P=0.049)?

www.metcardio.org

THE LARGE CONFIDENCE INTERVAL

Patti et al, Circulation 2005

CONFIDENCE INTERVAL SPANNING FROM 0.15 TO 0.97

www.metcardio.org

THE UNREASONABLY COMPLEX METHOD

Huyhn et al, Circulation 2009

?

www.metcardio.org

THE UNREASONABLY COMPLEX METHOD

Huyhn et al, Circulation 2009

…WE SELECTED NONINFORMATIVE PRIOR DISTRIBUTIONS… THESE INCLUDED NORMAL DENSITIES (MEAN, 0; 0.00001 [VARIANCE OF 105]) FOR THE LOGARITHM OF THE ORS AND (UNIFORM ON THE INTERVAL [0,2]). TO ENSURE CONVERGENCE OF THE GIBBS SAMPLER ALGORITHM, 3 MARKOV MONTE CARLO CHAINS WERE RUN, AND CONVERGENCE WAS ASSESSED AFTER 60000 ITERATIONS. THE FINAL STATISTICS WERE BASED ON 120000 ITERATIONS, 100000 OF THEM FOR BURN-IN.

??

www.metcardio.org

TAKE HOME MESSAGES

www.metcardio.org

TAKE HOME MESSAGES• Biostatistics is merely necessary, but not sufficient

alone to infer causality• Biostatistical comparisons can really be trusted only if

stemming from one or more randomized trials (e.g. in a meta-analysis)

• Sophisticated statistics cannot remedy faulty study designs, fabricated or missing data

• Sophisticated statistics can cloud a manuscript weaknesses, ominously separating clinicians from decision-making

www.metcardio.org

Thank you for your attention

For any correspondence: gbiondizoccai@gmail.com

For these and further slides on these topics feel free to visit the metcardio.org website:http://www.metcardio.org/slides.html