Bayesian Trial Designs: Drug Case Study

Bayesian Trial Designs: Drug Case Study

Donald A. Berrydberry@mdanderson.org

Donald A. Berrydberry@mdanderson.org

BERRY

STATISTICAL INNOVATION

CONSULTANTS

22

OutlineOutline

Some history

Why Bayes?

Adaptive designs

Case study

Some history

Why Bayes?

Adaptive designs

Case study

44

2004 JHU/FDA Workshop:“Can Bayesian Approaches to

Studying New Treatments Improve Regulatory Decision-Making?”

2004 JHU/FDA Workshop:“Can Bayesian Approaches to

Studying New Treatments Improve Regulatory Decision-Making?”

www.prous.com/bayesian2004

www.cfsan.fda.gov/~frf/ bayesdl.html

www.prous.com/bayesian2004

www.cfsan.fda.gov/~frf/ bayesdl.html

55

Upcoming in 2005Upcoming in 2005

Special issue of Clinical Trials

“Bayesian Clinical Trials”Nature Reviews Drug Discovery

Special issue of Clinical Trials

“Bayesian Clinical Trials”Nature Reviews Drug Discovery

66

Selected history of Bayesian trialsSelected history of Bayesian trials

Medical devices (30+)

200+ at M.D. Anderson (Phase I, II, I/II)

Cancer & Leukemia Group B

Pharma ASTIN (Pfizer) Pravigard PAC (BMS) Other

Decision analysis (go to phase III?)

Medical devices (30+)

200+ at M.D. Anderson (Phase I, II, I/II)

Cancer & Leukemia Group B

Pharma ASTIN (Pfizer) Pravigard PAC (BMS) Other

Decision analysis (go to phase III?)

77

Why Bayes?Why Bayes?

On-line learning (ideal for adapting)

Predictive probabilities (including modeling outcome relationships)

Synthesis (via hierarchical modeling, for example)

On-line learning (ideal for adapting)

Predictive probabilities (including modeling outcome relationships)

Synthesis (via hierarchical modeling, for example)

88

PREDICTIVE PROBABILITIES

Critical component of experimental design

In monitoring trials

99

Herceptin in neoadjuvant BCHerceptin in neoadjuvant BC Endpoint: tumor response Balanced randomized, H & C Sample size planned: 164 Interim results after n = 34:

Control: 4/16 = 25% Herceptin: 12/18 = 67%

Not unexpected (prior?) Predictive probab of stat sig: 95% DMC stopped the trial ASCO and JCO—reactions …

Endpoint: tumor response Balanced randomized, H & C Sample size planned: 164 Interim results after n = 34:

Control: 4/16 = 25% Herceptin: 12/18 = 67%

Not unexpected (prior?) Predictive probab of stat sig: 95% DMC stopped the trial ASCO and JCO—reactions …

1010

ADAPTIVE DESIGNS: Approach and Methodology

ADAPTIVE DESIGNS: Approach and MethodologyLook at the accumulating dataUpdate probabilitiesFind predictive probabilitiesUse backward inductionSimulate to find false positive

rate and statistical power

Look at the accumulating dataUpdate probabilitiesFind predictive probabilitiesUse backward inductionSimulate to find false positive

rate and statistical power

1111

Adaptive strategiesAdaptive strategies

Stop early (or late!) Futility Success

Change dosesAdd arms (e.g., combos)Drop armsSeamless phases

Stop early (or late!) Futility Success

Change dosesAdd arms (e.g., combos)Drop armsSeamless phases

1212

GoalsGoals

Learn faster: More efficient trials

More efficient drug/device development

Better treatment of patients in clinical trials

Learn faster: More efficient trials

More efficient drug/device development

Better treatment of patients in clinical trials

1313

Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I)

Adaptive randomization to:IA vs TA vs TI

Max n = 75 End point: Time to CR (< 50 days)

Troxacitabine (T) in acute myeloid leukemia (AML) combined with cytarabine (A) or idarubicin (I)

Adaptive randomization to:IA vs TA vs TI

Max n = 75 End point: Time to CR (< 50 days)

ADAPTIVE RANDOMIZATIONGiles, et al JCO (2003)

ADAPTIVE RANDOMIZATIONGiles, et al JCO (2003)

1414

Adaptive RandomizationAdaptive Randomization

Assign 1/3 to IA (standard) throughout (until only 2 arms)

Adaptive to TA and TI based on current probability > IA

Results

Assign 1/3 to IA (standard) throughout (until only 2 arms)

Adaptive to TA and TI based on current probability > IA

Results

1515

Patient Prob IA Prob TA Prob TI Arm CR<501 0.33 0.33 0.33 TI not2 0.33 0.34 0.32 IA CR3 0.33 0.35 0.32 TI not4 0.33 0.37 0.30 IA not5 0.33 0.38 0.28 IA not6 0.33 0.39 0.28 IA CR7 0.33 0.39 0.27 IA not8 0.33 0.44 0.23 TI not9 0.33 0.47 0.20 TI not

10 0.33 0.43 0.24 TA CR11 0.33 0.50 0.17 TA not12 0.33 0.50 0.17 TA not13 0.33 0.47 0.20 TA not14 0.33 0.57 0.10 TI not15 0.33 0.57 0.10 TA CR16 0.33 0.56 0.11 IA not17 0.33 0.56 0.11 TA CR

1616

Patient Prob IA Prob TA Prob TI Arm CR<5018 0.33 0.55 0.11 TA not19 0.33 0.54 0.13 TA not20 0.33 0.53 0.14 IA CR21 0.33 0.49 0.18 IA CR22 0.33 0.46 0.21 IA CR23 0.33 0.58 0.09 IA CR24 0.33 0.59 0.07 IA CR25 0.87 0.13 0 IA not26 0.87 0.13 0 TA not27 0.96 0.04 0 TA not28 0.96 0.04 0 IA CR29 0.96 0.04 0 IA not30 0.96 0.04 0 IA CR31 0.96 0.04 0 IA not32 0.96 0.04 0 TA not33 0.96 0.04 0 IA not34 0.96 0.04 0 IA CR

Compare n = 75

DropTI

1717

Summary of resultsSummary of results

CR < 50 days: IA: 10/18 = 56% TA: 3/11 =

27% TI: 0/5 = 0%

Criticisms . . .

CR < 50 days: IA: 10/18 = 56% TA: 3/11 =

27% TI: 0/5 = 0%

Criticisms . . .

1818

Consequences of Bayesian Adaptive Approach

Consequences of Bayesian Adaptive Approach

Fundamental change in way we do medical research

More rapid progressWe’ll get the dose right!Better treatment of patients . . . at less cost

Fundamental change in way we do medical research

More rapid progressWe’ll get the dose right!Better treatment of patients . . . at less cost

1919

CASE STUDY: PHASE III TRIALCASE STUDY: PHASE III TRIAL

Dichotomous endpoint Q = P(pE > pS|data)

Min n = 150; Max n = 600 1:1 randomize 1st 50, then assign

to arm E with probability Q Except that 0.2 ≤ P(assign E) ≤ 0.8

Dichotomous endpoint Q = P(pE > pS|data)

Min n = 150; Max n = 600 1:1 randomize 1st 50, then assign

to arm E with probability Q Except that 0.2 ≤ P(assign E) ≤ 0.8

Small company!

2020

Recommendation to DSMB toRecommendation to DSMB to

Stop for superiority if Q ≥ 0.99

Stop accrual for futility if P(pE – pS < 0.10|data) > PF

PF depends on current n . . .

Stop for superiority if Q ≥ 0.99

Stop accrual for futility if P(pE – pS < 0.10|data) > PF

PF depends on current n . . .

2121

0.0

0.2

0.4

0.6

0.8

1.0

0 100 200 300 400 500 600

n

Futility stopping boundary

0.75

0.95

PF

2222

Common prior density for pE & pS

Common prior density for pE & pS

Independent

Reasonably non-informative

Mean = 0.30

SD = 0.20

Independent

Reasonably non-informative

Mean = 0.30

SD = 0.20

2323

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1

p

Beta(1.275, 2.975)

density

2424

UpdatingUpdating

After 20 patients on each arm

8/20 responses on arm S

12/20 responses on arm E

After 20 patients on each arm

8/20 responses on arm S

12/20 responses on arm E

2525

0 .1 .2 .3 .4 .5 .6 .7 .8 .9 1

p

Beta(9.275,

14.975)

Beta(13.275,

10.975)

Q = 0.79

2626

AssumptionsAssumptions

Accrual: 10/month

50-day delay to assess response

Accrual: 10/month

50-day delay to assess response

2727

Need to stratify. But how?Need to stratify. But how?

Suppose probability assign to experimental arm is 30%, with these data . . .

Suppose probability assign to experimental arm is 30%, with these data . . .

2828

Proportions of Patients onExperimental Arm by Strata

Stratum 1Stratum 2

Small Big

Small 6/20 (30%) 10/20 (50%)

Big 6/10 (60%) 2/10 (20%)

Probability of Being Assigned toExperimental Arm for Above Example

Stratum 1Stratum 2

Small Big

Small 37% 24%

Big 19% 44%

2929

One simulation; pS = 0.30, pE = 0.45

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0 6 12 18 24 Months

Proportion Exp

Probability Exp is better 178/243

= 73%

FinalStd 12/38 19/60 20/65Exp 38/83 82/167 87/178

Superiority boundary

3030

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.00 6 12 18 24 Months

Proportion Exp

Probability Exp is better

87/155 = 56%

Probability futility

9 mos. End FinalStd 8/39 15/57 18/68Exp 11/42 32/81 22/87

One simulation; pE = pS = 0.30

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.00 6 12 18 24 Months

Proportion Exp

Probability Exp is better

87/155 = 56%

Futility boundary

3131

Operating characteristicsOperating characteristics

True ORR Mean # of patients (%)Std Exp

Probselect

exp Std Exp Total

Meanlength(mos)

Probmax n

0.3 0.2 <0.001 95 (62.1) 58 (37.9) 153 15 <0.0010.3 0.3 0.05 87 (43.1) 115 (56.9) 202 20 0.0030.3 0.4 0.59 87 (30.4) 199 (69.6) 286 29 0.050.3 0.45 0.88 79 (30.7) 178 (69.3) 257 26 0.020.3 0.5 0.98 59 (29.5) 141 (70.5) 200 20 0.0030.3 0.6 1.0 47 (30.1) 109 (69.9) 156 16 <0.001

3232

FDA: Why do this? What’s the advantage?

FDA: Why do this? What’s the advantage?

Enthusiasm of patients & investigators

Comparison with standard design . . .

Enthusiasm of patients & investigators

Comparison with standard design . . .

3333

Adaptive vs tailored balanced design w/same false-positive rate & power

(Mean number patients by arm)

Adaptive vs tailored balanced design w/same false-positive rate & power

(Mean number patients by arm)

ORR

Arm

pS = 0.20pE = 0.35

pS = 0.30pE = 0.45

pS = 0.40pE = 0.55

Std Exp Std Exp Std Exp

Adaptive 68 168 79 178 74 180

Balanced 171 171 203 203 216 216

Savings 103 3 124 25 142 36

3434

FDA:FDA:

Use flat priors

Error size to 0.025

Other null hypotheses

We fixed all … & willing to modify as necessary

Use flat priors

Error size to 0.025

Other null hypotheses

We fixed all … & willing to modify as necessary

3535

The rest of the story …The rest of the story …

PIs on board

CRO in place

IRBs approved

FDA nixed!

PIs on board

CRO in place

IRBs approved

FDA nixed!

3636

OutlineOutline

Some history

Why Bayes?

Adaptive designs

Case study

Some history

Why Bayes?

Adaptive designs

Case study