Personalized Medicine

• Detection

• Diagnosis

• Treatment

• Survival

Prediction is very difficult, especially about the future.

Niels Bohr

Danish physicist (1885 - 1962)

Biomarkers

Test Information Decision Outcome

1. Discrimination(sensitivity, specificity, predictive value, ROC analysis)

2. Utility

(disease free survival, recurrence rates, survival etc)

Diagnostic tests

Describing test performance

Test Result

Disease No disease

Total

Positive a b a+b

Negative c d c+d

Total a+c c+d a+b+c+d

Properties of a test• Sensitivity:

– a/a+c

• Specificity: – d/c+d

• Positive predictive value:– a/a+b

• Negative predictive value:– d/c+d

The importance of disease prevalence

Test result

Breast cancer

No breast cancer

Total

Positive 360 4,980 5,340

Negative 40 94,620 94,660

Total 400 99,600 100,000

• Screening mammography • Properties of the test

Sensitivity: 90%

a/a+c = 360/400

Specificity: 95%

d/c+d = 94,620/99,600

Positive predictive value:

a/a+b = 360/5340 = 7%

Negative predictive value:

d/c+d =94,620/94,660 = 100%

Desiderata for studies of diagnostic tests.

• “Gold” standard• Test result before outcome known• “Blind” reading• Pre-determined cut-off• Sensitivity and specificity.• Predictive value.• Receiver operating. characteristic

curves (ROC).

Diagnostic tests and the spectrum of disease.

• Spectrum of patients.

• Clinical spectrum• Co-morbid spectrum• Pathologic spectrum

• Potential biases in test evaluation.

• Exclusion of equivocal cases• Work up bias• Test review bias• Incorporation bias

Clinical value of tests

Test

Information

Decision

Outcome

PRINCIPAL AGENT

COMPARATIVE AGENT

INITIAL STATE, RECIPIENTS OF

PRINCIPAL AGENT

INITIAL STATE, RECIPIENTS OF

COMPARATIVE AGENT

SUBSEQUENT EVENTS,

RECIPIENTS OF PRINCIPAL AGENT

SUBSEQUENT EVENTS,

RECIPIENTS OF COMPARATIVE

AGENT

Research Designs-General Structure

• Purpose of research(initial states)

• Prevention.

• Prediction of risk in healthy.

• Treatment response or toxicity

in those with disease.

• Identify factors that influence

outcome (prognosis).

• Types of manoeuver

• Inherited (eg genetic variant).

• Acquired– Self selected (smoking,

alcohol)– Other (treatment).

• Imposed (atomic irradiation).

Principal research designs

Disease

Present Absent

Present a bExposure

Absent c d

Passage of time

Relative risk = a/a+b ÷ c/c+d

Cohort study

Nested case control studiesScreening programs: NBSS, SMPBC, OBSP

case

case

case

6-8 years follow-up

case

control

control

control

control

Baseline mammogramRisk factors

How many subjects (or samples) do you need?

• Number of events (eg deaths).

• Willingness to risk a false positive (Type I) error.

• Willingness to risk a false negative result (Type II) error.

• Magnitude of difference worthwhile to detect.

• Time for accrual and follow-up.

Sample size to detect an improvement in survival (alpha=0.05; 1-beta=0.90)

P2-P1

P1 0.10 0.30 0.50

0.10 395 76 41

0.30 879 118 51

0.50 1020 116 -

Sample size for genetic studies

Odds ratio Allele %

5% 20% 30%1.2 12,217 3730 2896

1.3 5702 1763 1380

1.5 2249 712 566

2.0 687 377 188

SUBSEQUENT EVENTSR }

PRINCIPAL AGENT

COMPARATIVE AGENT

{INITIAL STATE

A trial to change diet

• Vancouver + Surrey• Windsor• London + Sarnia• Hamilton + KW• Toronto

• Funding: Ontario Ministry of Health, Medical Research Council, Canadian Breast Cancer Research Alliance, National Institutes of Health, American Institute for Cancer Research

Screening

Randomization 4,693

Low-fatdiet

Usual diet

>8 years counselingand follow-up

200 2 4 6 8 10 12 14 16 18

Cu

mu

lativ

e h

aza

rdAll invasive breast cancer

HRa = 1.05 (95% CIb : 0.83 - 1.33)

Year

# eventsc I:

C: 16

16

9

25

24

20

18

18

18

23

20

16

20

10

4

6

10

6

1

3

# at riskd I:

C: 2349

2341

2323

2312

2300

2269

2258

2228

2221

2190

2181

2148

1878

1858

1194

1194

742

740

329

324

0.150

0.125

0.100

0.075

0.050

0.025

0.000

Intervention

Comparison

(A)

Association or causation?

• Not all associations are causal

• All causal factors show association

• May be due to bias or confounding

• Genetic associations– Causal– In linkage

disequilibrium with the causal variant

– Population stratification

Population stratification

• Type of confounding• Ethnicity

– associated with disease– associated with genotype– gives spurious association between genotype

and disease

• Can be controlled in analysis (if recognized)• Dispute about importance

Analysis

P<0.05

What does this mean?

The meaning of p-values.

If the TRUE difference between the compared

groups is zero (the null hypothesis), the

PROBABILITY of obtaining a difference as large

or larger than the one observed by CHANCE is p.

Multiple comparisons

• The problem.• If alpha = 0.05• 20 comparisons can be

expected to generate one p<0.05.

• (1-(1-alpha)k, where alpha is the level for significance and k=number of tests.

• What protection?• Few, a priori hypotheses

• Correction for number of tests eg Bonferroni– Alpha/number of tests

• Stringent alpha eg E 10-8

• Replication/validation

Francis Galton’s ox and the “Winner’s curse”.

• Country fair in 1906 - 800 bought tickets and predicted the weight of an ox.

• Actual weight was 1,198 lbs.

• None were close to the actual weight.

• Mean predicted weight (N=787) was 1,197 lbs.

• At auction, most bids cluster around the “true” value of the object.

• The winning bid is always higher than the “true” value.

Replication -validation

• “leave one out”– Applied to “learning set”– Not an independent sample– May help avoid overfitting

• Independent data set– Preferably also an independent investigator

How to get a statistically significant result.

• Count or ignore differences in follow-up time.

• Censor at different time points.• Exclude specific causes of death.• Exploit sub-group analysis.• Use different cut-offs for gene

expression (or other test result).• Note: all of the above increases the

number of statistical tests you can do!

Can you believe the literature?

• Publication bias (author and editor bias).

• Multiple statistical testing.• The “Winner’s curse”.• Bias in the sampling,

measurement or analysis of the data.

• Most published reports are never replicated.

The “Winners Curse”

False positives more likely:Small studiesSmall effects

Early, hypothesis generating studiesFinancial interest

“Hot” field

Ioannidis PLos Medicine 2005

How to stay out of trouble

• Define target population.• Standardize sample collection.• Collect samples at zero time.• Define outcomes at the outset.• Random selection of cases and controls.• Analyze samples without knowledge of

case/control status.• Replicate.