1 Don’t Panic Basic Statistics You Can Understand

1

Don’t Panic

Basic Statistics You Can Understand

Case Presentation

Symptom cluster:Sweaty palms Pale Increased heart rateGlassy-eyed stareLoss of affect

Diagnosis: Photonumerophobia The fear that one’s fear of numbers will come to light

(DSM IV)

-- attributed to David Sackett

Rene´ Magritte. The Air and the Song, 1928.

“This is not a pipe”

A Far-fetched Analogy

Arcane point #1. Statistics are an approximation of the Truth

This painting is a representation of a pipe, not the pipe itself

Statistical significance is not the truth, but an approximation of the truth

The “Truth”• What we do for people helps them to live longer or live better

• Research helps us get closer to understanding the Truth Our goal: figure out how close the statistics represent

the truth

Arcane point #2. Users of statistics don’t have to be statisticians

I am a user of statistics, not a statistician (I have friends, however, who are statisticians)

You don’t have to know a lot about statistics to effectively use statistics

Don’t focus on whether the statistics are right• Learn to figure out what the statistics are trying to

tell you

6

P< .05

The Shrine of Statistics:The Sacred P-Value

7

P Value

"Probability" level

The likelihood that the difference observed between two interventions could have arisen by chance

Arbitrarily set at 5% risk (P = 0.05)

8

P value example

P Value

Depends on several factors

• How large the effect was

• How consistent the effect was

• How many patients were studied

As all of these factors increase, the likelihood of finding statistical

significance increases

Once we’ve decide the difference was NOT due to chance, we have to

decide on clinical significance

10

This ain’t physics

“Unfortunately, the publication standards of medical journals are quite low compared to other science fields such as physics . . . Presumably necessary to assure that possible helpful therapies are not kept from needy patients for far too long.

No respectable physics journal would publish a result with a p-value of a few percents. In fact, the publication standard in physics is typically a p-value of 0.0001 . . .”

Victor Stenger, PhDProfessor Emeritus of Physics, U. of Hawaii

Discovered that the neutrino has mass

Wringing out information from P values

"Highly Significant" — P < 0.001

• If the number of patients is small, P value tells us that the effect

was either large or consistent (or both)

• If the number of patients is large, the size of the effect may not be

that large

“Not Significant” P > 0.05 (i.e., 0.15)

• If the number of patients is small, there might not have been enough patients to find a difference that truly does exist

• If the number of patients is large, we can be confident that either there is no difference between treatments, or the treatment effect is not consistent


"Borderline Significance" — P = 0.08 — ????

• May have reached significance had their been more patients in the study

• The effect size may be small or inconsistent

• No conclusions can be drawn, except that more study is needed

• Only counts in horseshoes, hand grenades and dancing


14

Number Needed to Treat

The number of patients that need to be treated for one additional patient to receive benefit

The number of patients that need to be treated to prevent one additional outcome

Takes into account the relative risks as well as the absolute risk of no treatment

NNT = 100% in treatment group - % in control group

15

NNTs for Prevention

Condition Treatment Outcome NNT

Heart failure (NHYA I or II)

Enalapril vs. placebo 1 death at one year 100

Hypertension in patients with type 2 diabetes

HTN treatment 1 diabetes-related death over 10 years 15

Hyperlipidemia – primary prevention

Simvastatin vs. no treatment

1 death over 1 years

163

Hyperlipidemia – secondary prevention

Various vs. placebo 1 MI or CVA over 5 years 16

DVT Warfarin (target INR = 1.5-2.0) vs. placebo for 1 yr

1 VTE over 1 year

22

16

NNTs for Treatment

Condition Treatment Outcome NNT

H. Pylori Triple therapy Eradication 1.1

Peptic Ulcer H. Pylori tx vs. H2 tx for 6-8 wks

Ulcer cure at 1 year 1.8

Migraine 1 dose sumatriptan vs. placebo

Headache relief at 2 hours 2.6

Bacterial conjunctivitis

Topical abx vs. placebo

For early clinical remission (3-5 days) 5

Herpes Zoster Acyclovir vs. placebo

Prevent PHN at 6 months

Not effective

Other statistics you need to understand research

18

Relative Risk

The risk of harm with one treatment as compared with another

The risk of benefit with one treatment as compared with another

If RR = 1, then there is no difference between the two treatments

Depends only on the relative difference between the two treatments

Does not take into account the risk of no treatment — the "absolute risk"

19

Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Scandinavian Simvastatin Survival Study Group

Summary: 4444 patients with high cholesterol and CHD were given either simvastatin or placebo for a

median of 5.4 years.

Results: 256 (12%) in the placebo group died 182 (8%) in the simvastatin group died Relative risk = 0.70 Risk reduction = 30%

But, what is the NNT = ?

Examples of whererelative risk can be misleading

21

Meta-analysis: Statins to prevent stroke and MI

Meta-analysis of 29 studies, 10,000+ patients Statins vs. control (usual care)

• Stroke risk: 0.82 (18% decrease)

• MI risk: 0.74 (26% decrease)

18% from what? 26% from what?

Briel M. Effects of statins on stroke prevention in patients with and without coronary heart disease: A meta-analysis of randomized controlled trials. Am J Med 2004;117

22

Meta-analysis: Statins to prevent stroke and MI

Stroke Risk

• Low risk: 0.2%

• High risk (CHD): 0.9%/year MI risk

• Low risk: 0.9%

• High risk: 3.7%/year

Briel M. Effects of statins on stroke prevention in patients with and without coronary heart disease: A meta-analysis of randomized controlled trials. Am J Med 2004;117:

NNT to prevent 1 stroke/1 yr:• Low risk: 2,778 patients

• High risk: 617 patients

NNT to prevent 1 MI/1 year:• Low risk: 427 patients • High risk: 104 patients

23

Improved GI Tolerance with Biaxin XL

0

1

2

3

Biaxin Biaxin XL

Incidence of GI side effects

Reformulation of Clarithromycin

66% decrease Relative risk = 0.34

Pe

rce

nt

Pe

rce

nt

3%

1%

24

Confidence Interval

"A statistic of a statistic"

Statistics are estimates• Confidence intervals tells us the upper and lower

possibilities of our statistical estimates

25

Example: Results from the UKPDS

26

0.940.80 1.1

Risk could be this low

Risk could be this high

95% C.I.

Since the 95% CI crosses 1.0, the difference is not significant

1.0

“Confidence Interval” vs. “Credible Interval” Likelihood of finding truth in past vs likelihood

of predicting truth in future

A Word on Combined Outcomes: Truth or Fishing Expedition?

Beware of switched outcomes• secondary analysis or outcomes vs a priori primary

outcome

• Pioglitazone trial PROactive 10 (a priori primary composite outcome not significant so secondary one reported) Am Heart J 2008;155:712-7.


If the composite outcome is statistically different

• make sure at least 1 individual clinically relevant outcome is significantly different.

• CV research: 43% of composite outcomes the most significant outcome was not clinically relevant. Lincoff, A. M. et al. JAMA 2007;298:1180-1188

Table 3. Cardiovascular Event Rates for Combined Trials Stratified by Study Type.

Lincoff, A. M. et al. JAMA 2007;298:1180-1188

Copyright restrictions may apply.


Beware of “dominant” DOE outcome

• United Kingdom Prospective Diabetes Study• 21 outcomes

• “Any diabetes-related outcome” decreased 12% (0.79-0.99)

• Only significantly decreased outcome: photocoagulation

• Other 20 outcomes not affected

• Results not confirmed in ACCORD, VDAT, ADVANCE

Lancet. 1998 Sep 12;352(9131):837-53.

33

Don’t Be Afraid of Statistics

Statistical significance is a requirement for determining clinical significance, but is not enough to signify a clinical difference

The P value tells us the risk that the difference between two treatments was due to chance

Relative risk tells part, but not all of the story; NNT does it better

Confidence intervals help us to understand how close our estimate is to the "truth"

Diarrhea was somewhat more common in the amoxicillin-clavulanate group (29.9% vs 19.6%; P = .045, number needed to treat to harm = ?

NNTH = 100 = 10 30 - 20

Using intention-to-treat analysis, 59.2% of the patients assigned to antidepressants survived compared with 36.4% who received placebo (P = .03; number needed to treat = 4).

How many people won’t receive a benefit when treated?

What is the clinical relevance of this research? What’s missing?

Osteoporotic fractures, in general, are important, but the dangerous fractures that must be prevented are hip fractures.

Documents

1 Don’t Panic Basic Statistics You Can Understand