Some help understanding and interpreting medical...

Some help understanding and interpreting

medical statistics …

John Norrie

Centre for Healthcare Randomised Trials (CHaRT)

University of Aberdeen, Scotland

Barcelona, June, 2013

www.eurordis.org

Acknowledgment

Thanks to Ferran Torres, Josep Torrent y Farnell and Julia Saperia for creating the previous versions of this workshop and providing the material that this version is based on.

Understanding and Interpreting medical statistics

Outline

• Why statistics

• Descriptive statistics

• Population and samples

• Hypothesis Tests and P-values

• Statistical “vs” clinical significance

• Statistical errors (a and b)

• Sample size

• Estimation of treatment effect: Confidence Intervals

• A worked example

What does statistics mean to you?

information

percentage

variability data

median

average

graphs

measurement

probability

confidence

evidence

spread

Why Statistics?

• Statistics is the science of uncertainty – it is all about

investigating, understanding, and allowing for VARIABILITY

Variability

Why Statistics?

• Medicine is a quantitative science but not exact

– Not like physics or chemistry

• Variation characterises much of medicine

• Statistics is about handling and quantifying variation and

uncertainty

• Humans differ in response to exposure to adverse effects

Example: not every smoker dies of lung cancer

some non-smokers die of lung cancer

• Humans differ in response to treatment

Example: penicillin does not cure all infections

• Humans differ in disease symptoms Example: Sometimes cough and sometimes wheeze are presenting features for

asthma

Why Statistics Are Necessary

• Statistics can tell us whether events could have happened by

chance and to make decisions

• We need to use Statistics because of variability in our data

•Generalize: can what we know help to predict what will

happen in new and different situations?

Probability

• Probability is the methodological ‘glue’ that holds statistics together

• Given uncertainty is at the heart of statistical thinking, probability is our way of quantifying how likely various things that may happen will take place

• For example, a fair coin is tossed – in the long run, we would expect to get the same number of ‘heads’ as ‘tails’

If we threw the coin 10 times, intuitively we would think it much less likely to get 10 heads in a row (or for that matter ten tails) than say 5 heads and 5 tails (in any order) …

Why? Because there are lots of ways of getting 5 heads + 5 tails, only one way of getting 10 heads and no tails ….

• Probability is challenging – consider conditional probability …

draw a

sample

Population

sample

sample data

Inferential statistics

Draw conclusions for the

whole population based

on information gained

from a sample

Target of study design

representative sample

= ~ all "typical"

representatives are

included

Descriptive

statistics

(a population is a set of all

conceivable observations of a

certain phenomenon)

Population and Samples

Target Population

Population of the Study

Sample

Descriptive statistics (1)

• For continuous variables (e.g. height, blood pressure,

body mass index, blood glucose)

•Histograms

•Measures of location

–mean, median, mode …

•Measures of spread

–variance, standard deviation, range, interquartile

range, minimum, maximum

The histogram

•Taken from http://www.mathsisfun.com/data/histograms.html on 28 May 2012

Johann Carl Friedrich Gauss 1777-1855

~95,5%

mean mean + sd mean + 2sd mean - sd mean - 2sd

Picturing “normal” data

Descriptive statistics (2)

• For categorical variables (e.g. sex, dead/alive,

smoker/non-smoker, blood group)

•Bar charts, (pie charts)…

• Frequencies, percentages, proportions

–when using a percentage, always state the number it

is based on

–e.g. 68% (n=279)

–or perhaps 190/279 (68%)

Memorable quotes

• 50% of what you learn about therapy in the next 5 years is wrong.

(The trouble is we don’t know which 50%) (Anon)

• “…in this world there is nothing certain but death and taxes.” Benjamin Franklin (1706-1790). (also said by Woody Allen)

• 86% of all statistics are invented on the spot (Huff – How to Lie with Statistics)

• “There are lies, damn lies, and statistics” Benjamin Disraeli (1804-1881)

• And now we add ‘… lies, damned lies, statistics, and government

statistics’ !!

Error ? …

Unreliable

or Imprecise

Reliable (precise) but

not valid (accurate)

Reliable

& valid

130 150 170

01 02 03 04 05

True Value

Random vs Systematic error

Random Systematic (Bias)

130 150 170

True Value

Example: Systolic Blood Pressure (mm Hg)

Valid samples?

Population

Likely to occur

Unlikely to occur Invalid Sample and Conclusions

HYPOTHESIS TESTING

• Testing two hypotheses H0: A=B (Null hypothesis – no difference)

H1: A≠B (Alternative hypothesis)

•Calculate test statistic based on the assumption that H0 is true (i.e. there is no real difference)

• Test will give us a p-value: how likely are the collected data if H0 is true

• If this is unlikely (small p-value), we reject H0

23 1 23

P-value

• The p-value is a “tool” to answer the question:

Could the observed results have occurred by chance*?

Remember:

– Decision given the observed results in a SAMPLE

– Extrapolating results to POPULATION

*: accounts exclusively for the random error, not bias

p < .05

“statistically significant”

A intuitive definition

• The p-value is the probability of having observed our data when

the null hypothesis is true

• Steps:

1) Calculate the treatment differences in the sample (A-B)

2) Assume that both treatments are equal (A=B) and then…

3) …calculate the probability of obtaining a magnitude of at least the

observed differences, given the assumption 2

4) We conclude according the probability:

a. p<0.05: the differences are unlikely to be explained by random,

we assume that the treatment explains the differences

b. p>0.05: the differences could be explained by random,

we assume that random explains the differences

P-value

•A “very low” p-value do NOT imply:

Clinical relevance (NO!!!)

Magnitude of the treatment effect (NO!!)

With n or variability

•Please never compare p-values!! (NO!!!)

P-value

•A “statistically significant” result

(p<.05)

tells us NOTHING about clinical or scientific

importance. Only, that the results were

unlikely to be due to chance.

A p-value does NOT account for bias

only for random error

STAT REPORT

Significance level

• p-values are compared to a pre-specified significance level, alpha

• alpha is usually 5% (analogous to 95% confidence intervals)

if p ≤ 0.05 reject the null hypothesis (i.e., the result is statistically significant

at the 5% level) conclude that there is a difference in treatments

if p > 0.05 do not reject the null hypothesis conclude that there is not

sufficient information to reject the null hypothesis

(See Statistics notes: Absence of evidence is not evidence of absence:

BMJ 1995;311:485)

Type I & II Error & Power

Reality

(Population)

A=B A≠B

Conclusion

(sample)

“A=B” p>0.05 OK Type II error

A≠B p<0.05 Type I error

(a) OK

• Type I Error (a)

False positive

Rejecting the null hypothesis when in fact it is true

Standard: a=0.05

In words, chance of finding statistical significance when in fact there truly

was no effect

• Type II Error (b)

False negative

Accepting the null hypothesis when in fact alternative is true

Standard: b=0.20 or 0.10

In words, chance of not finding statistical significance when in fact there

was an effect

• Power

1-Type II Error (b)

Usually in percentage: 80% or 90% (for b =0.1 or 0.2, respectively)

In words, chance of finding statistical significance when in fact there is an

effect

Reality

(Population)

A=B A≠B

Conclusion

(sample)

(a) POWER

Which sorts of error can occur when performing statistical tests?

H0 Correct

False negative

H1 False positive

Correct

Hypoth

True state of nature

Significance level a is usually predetermined in the study protocol, e.g.,

a=0.05 (5%, 1 in 20), a= 0.01 (1%, 1 in 100), a= 0.001 (0.1%, 1 in 1000)

POWER 1- b : The power of a statistical test is the probability that the test will reject

a false null hypothesis.

Statistical mistakes/errors

Compare a statistical test to a court of justice

H0 Innocent

H1 Not Innocent

Correct

False negative (i.e. guilty but not caught)

H1 False positive (i.e. innocent but convicted)

Correct

sion o

True state of nature

False positive: A court finds a person guilty of a crime that they did not actually commit.

False negative: A court finds a person not guilty of a crime that they did commit.

Judged

"innocent"

Judged

“Not in

nocent"

Statistical tests vs court of law

Minimising statistical errors

Remember:

How do I gain adequate data?

Thorough planning of studies

• Defining acceptable levels of statistical error is key to the planning

of studies

• alpha (in clinical trials) is pre-defined by regulatory guidance

(usually)

• beta is not, but deciding on the power (1-beta) of the study is

crucial to enrolling sufficient patients

• the power of a study is usually chosen to be 80% or 90%

• conducting an “underpowered” study is not ethically acceptable

because you know in advance that your results will be inconclusive

Sample Size

• The planned number of participants is calculated on the basis

Expected effect of treatment(s)

Variability of the chosen endpoint

Accepted risks in conclusion

↗ effect ↘ number

↗ variability ↗ number

↗ risk ↘ number

Sample Size

↗ risk ↘ number

ALTURA

Desv. típ. = 25.54

Media = 165.1

N = 2000.00

ALTURA

Desv. típ. = 26.94

Media = 165.0

N = 2000.00

ALTURA

100.090.0

ALTURA

Desv. típ. = 32.27

Media = 165.1

N = 2000.00

Sample Size

↗ risk ↘ number

Reality

(Population)

A=B A≠B

Conclusion

(sample)

(a) POWER

Deciding how many patients to enrol

• the sample size calculation depends on:

– the clinically relevant effect that is expected (1)

– the amount of variability in the data that is expected (2)

– the significance level at which you plan to test (3)

– the power that you hope to achieve in your study (4)

• if you knew (1) and (2), you wouldn’t need to conduct a study

picking your sample size is a gamble

• the smaller the treatment effect, the more patients you need

• the more variable the treatment effect, the more patients you need

• the smaller the risks (or statistical errors) you’re prepared to take,

the more patients you need

95% (or 99%) Confidence Intervals

• Confidence Intervals are ‘better’ than p-values…

This is statistical estimation rather than hypothesis testing

…use the data collected in the trial to give an estimate of the treatment

effect size, together with a measure of how certain we are of our

estimate

• CI is a range of values within which the “true” treatment effect is believed to be found, with a given level of confidence.

95% CI is a range of values within which the ‘true’ treatment effect will lie 95% of the time

A 99% CI is wider than a 95% CI …

• Generally, 95% CI is calculated as

Sample Estimate ± 1.96 x Standard Error

Interval Estimation

Confidence

interval

Sample statistic

(point estimate)

Confidence

limit (lower)

Confidence

limit (upper)

A probability that the population parameter

falls somewhere within the interval.

Superiority study

d > 0 + effect

d = 0 No differences

d < 0 - effect

Test better Control better

Superiority study

d > 0 + effect

d = 0 No differences

d < 0 - effect

Test better Control better

Standard Deviation

• The Standard Deviation is a measure of how spread

out numbers are.

• Its symbol is σ (the greek letter sigma)

• The formula is easy: it is the square root of the

Variance

The average of the squared differences from the Mean.

Calculating a variance

To calculate the variance follow these steps:

• Work out the Mean (the simple average of the numbers)

• Then for each number: subtract the Mean

• and then square the result (the squared difference).

• Then work out the average of those squared differences.

Example

You and your friends have just measured the heights of your dogs (in

millimeters):

The heights (at the shoulders) are: 600mm, 470mm, 170mm, 430mm and 300mm.

First calculate the mean (or average) …

Mean = 600 + 470 + 170 + 430 + 300 =

1970 = 394

Next calculate the variance,

and then the standard deviation

To calculate the Variance, take each difference, square it, and then average the result:

So, the Variance is 21,704.

And the Standard Deviation is just the square root of Variance, so:

Standard Deviation: σ = √21,704 = 147.32... = 147 (to the nearest mm)

First calculate the mean (or average) …

And the good thing about the Standard Deviation is that it is useful.

Now we can show which heights are within one Standard Deviation (147mm) of the

So, using the Standard Deviation we have a "standard" way of knowing what is

normal, and what is extra large or extra small.

Rottweilers are tall dogs. And Dachshunds are a bit short ...

Some help understanding and interpreting medical...

Documents

Kogan - science-education.ru · Левосторонний гидронефроз выявлен у 28 (68,3 %) детей , правосторонний – у 13 (31,7 %) больных

ANÁLISE DA UTILIZAÇÃO DE MEDICAMENTOS EM UMA UNIDADE … › bitstream › 1843 › BUOS-9PFLD4 › 1 › ... · 2019-11-14 · utilizaram medicamentos off label foi 95,5% segundo

Presentación de PowerPoint · el 60,6% consultaba antes de manipular las formas farmacéuticas, el 56,7% conservaba en lugar fresco y seco, el 68,3% revisaba las fechas de caducidad,

CONDENSED CONSOLIDATED FINANCIAL STATEMENTS · The executive team ... is still heavily weighted to Gauteng where 68,3% (2013: 58,7%) of our properties are ... The project to ‘green’

Aktia Bank BOKSLUTSKOMMUNIKÉ 2015file.euroinvestor.com/newsattachments/2016/02... · 1-12/2015: Provisionsnettot fortsatte att öka • Rörelseresultatet uppgick till 64,2 (68,3)

PROGRAMMA 95,5

Профсоюза работников народного · 6000 7000 8000 9000 10000 2017 год 2018 год 2019 год 7478 8092 овек 9226 68,3 % 68,8 % 70,3 % Задача

95,5 TONA MESA ZA 170.000 IZBJEGLICA

L'éducation nationale en chiffres 2011 · 2015-10-31 · à partir de la rentrée 2010 (fonctionnaires stagiaires). ... • dont enseignement scolaire 95,5 Dépense moyenne par habitant

STRATEGI PEMBERITAAN RADIO SUARA BERSATU 95,5 FM …repositori.uin-alauddin.ac.id/14689/1/ALWANI DAUS...Judul Skripsi : Strategi pemberitaan Radio Suara Bersatu 95,5 FM Sinjai (Studi

Inicio | Argentina.gob.arSCHNEIDER FEDERICO NAHUEL ARIAS GABRIELA SOLEDAD INVERNO NESTOR OCAMPO BENITEZ PABLO LUIS RODRIGUEZ CARRA GABRIEL OCAMPO ANA PAULA COIccx PUNTA-JE 96 95,5

COMUNE DI GENOVAstatistica.comune.genova.it/pubblicazioni/download/not...contro 68,3), maggio (81,8 mm. contro 55,4) e giugno (59,0 mm. contro 37,7) sono risultati più piovosi,mentre

Umweltmedizinische „Vergiftungsfälle“ UNG · Mind. Eine Beschwerde 68,3 % 70,4 % 72,6 % Nach J. Mansel, P.Kolip SFB- Projekt „Prävention und Intervention im Kindes - und Jugendalter“,

128 hp (95,5 kW) 10.300 kg - Cotril Máquinas |cotrilmaquinas.com.br/.../uploads/2014/03/pa_carregadeira_12D.pdf · pá carregadeira 12D vem equipada com o consagrado motor MWM, modelo

68,3 (-9 EBITDA 12,7 (-17% 36,9 26 · до 8,9 млрд рублей с 9,4 млрд рублей в четвертом квартале 2019 года и на 1,2 млрд рублей

ETHICAL ASPECTS OF HUMAN MEDICAL RESEARCH …download.eurordis.org.s3.amazonaws.com/training-resources/2015/d3s... · ETHICAL ASPECTS OF HUMAN ... Good Clinical Practice (GCP)

2016.docx · Web viewKATALOG ŠUMAVANKA ZLATÁ STŘÍKANÁ číslo klece pohlaví kroužek body ocenění vystavovatel 1 1.0 3810 95,5 EL Fecková Pohár nejlepší kohout výstavy

HEALTH TECHNOLOGY ASSESSMENT - …download.eurordis.org.s3.amazonaws.com/training-resources/3.Health... · TITRE DE LA PRESENTATION ... •No pain •No depression •Some problem

С Е11Н ТЯБР—ДТ БЬЗТ, АИДЙОФТ Ь—ЙЬЯТЙТ Я 1 БЬЗР98,5 млн. В прошлом году предприятия группы выплавили 95,5

JARDINÓPOLIS - Sebrae SANTA CATARINA SC | Sebrae · 2010-02-01 · Secretaria de Desenvolvimento Regional de SC SDR - Quilombo Área territorial (km²) 68,3 Distância da Capital