A Novel Phase 3 Design Incorporating
Historical Information for the Development of
Antibacterial Agents
Jeff Wetherington, GSK
Kert Viele, Berry Consultants
Tessella Series Webinar
June 18th, 2013
Complicated Urinary Tract
Infections (1)
• Occurs in men and women
– structural or functional abnormalities of the urinary
tract
– hospitalized patients with significant medical or
surgical co-morbidities
• Major cause of hospital admission, extended hospitalizations morbidity, mortality, and excess healthcare costs
• Prescribing physicians have several options for empiric and pathogen-specific treatment
Bayesian Augmented Control for Antibacterial Agents 2
Complicated Urinary Tract
Infections (2)
Yet unmet treatment needs for patients with cUTI
continue to exist given the emergence and
prevalence of multi-drug resistance in uropathogens
Bayesian Augmented Control for Antibacterial Agents 3
Traditional Phase 3
• Randomized, parallel group
• Active controlled, non inferiority
• Narrow non inferiority margins
• Infeasible in the face of increasing unmet need
– >1500 patients enrolled into 2 independent trials
– >5 year clinical development programs
• Urgent need for novel clinical designs for
antibiotic drug development
Bayesian Augmented Control for Antibacterial Agents 4
Historical Studies
Bayesian Augmented Control for Antibacterial Agents 5
0.0
0.2
0.4
0.6
0.8
1.0
Naber 2009 Redman 2010
Doripenem Microbiological Eradication Rate
Fixed Design
• “Standard” 1:1 trial requires 750 subjects total
– 90% power and one sided α=0.025
– power for p=0.83 with noninferiority δ=0.10
– 20% dropout assumed
– note 375 patients on treatment
• Can we do better using the historical
information?
Bayesian Augmented Control for Antibacterial Agents 6
Goals
• Reduce sample size!
• Reduce sample size!
• Maintain
– controlled type I error (most complex…won’t be
able to get “complete” control)
– comparable power around (and slightly below per
expectation) p=0.83
– similar numbers of subjects on treatment (for
secondary analyses)
Bayesian Augmented Control for Antibacterial Agents 7
Preview
• Proposed design incorporates historical borrowing on the control arm using a hierarchical model.
• 20% reduction in sample size
• Similar power near or slight below p=0.83
• Slightly MORE subjects on treatment
• Type I error control
– in a region near 0.83
– based on E[type I error] for a range of perceived likely amounts of “drift” in the true control rate
Bayesian Augmented Control for Antibacterial Agents 8
Experimental parameters
• Dichotomous endpoint, p=Pr(ME)
• Control = Doripenum versus Treatment
• 20% dropout rate
• Goal is 90% for p=0.83 (NI δ=0.10) with
one sided α=0.025
• If possible, would like to leverage two
historical studies from control arm.
– Naber, 230 successes in 280 subjects (82.1%)
– Peninsula, 209 successes in 250 subjects (83.6%)
Bayesian Augmented Control for Antibacterial Agents 9
Notation and Model
• γ0 = logit(true current control rate)
• γ1 = logit(true Naber rate)
• γ2 = logit(true Peninsula rate)
• γ0,γ1,γ2 ~ N(μ,τ)
• π(μ)=N(1,1)
• π(τ2)=IGamma(α=0.001,β=0.001)
• Treatment effect θ ~ N(μθ=0,σθ=100)
• γ0+θ = logit(true treatment rate)
Bayesian Augmented Control for Antibacterial Agents 10
Intuition
• The three Doripenem arms are connected
through the “across studies” distribution
N(µ,τ).
– similar to random effects model on studies
– common model in meta-analysis
• τ is the most important parameter (across
study variance)
– τ≈0 corresponds to γ0≈γ1≈γ2
– τ large corresponds to no borrowing
Bayesian Augmented Control for Antibacterial Agents 11
Intuition • τ has a prior, and is estimated as part of the
model fitting.
• Datasets with high across study variation – produce higher estimates of τ
– the N(µ,τ) across distribution exerts less influence on each group (acts as less informative prior)
– less borrowing
• Datasets with low across study variation – produce lower estimates of τ
– the N(µ,τ) across study distribution can act as quite informative prior
– more borrowing
Bayesian Augmented Control for Antibacterial Agents 12
Intuition
• Only 3 Doripenem arms available
– so only three γ used to estimate τ
• Enough that borrowing is dynamic, but prior
will not fully wash out.
• Important to consider operating
characteristics (as always)
• Big goal
– borrow robustly when current data near p=0.83
– borrow less as current data diverges from p=0.83
Bayesian Augmented Control for Antibacterial Agents 13
Proposed Design A
• N=600 with 2:1 randomization and
borrowing
– 20% savings on N compared to fixed design
– 400 subjects on treatment, more than fixed
design
• Trial declared success if
– Pr(trt rate > ctrl rate – 10%)>0.975
Bayesian Augmented Control for Antibacterial Agents 14
Bayesian Augmented Control for Antibacterial Agents 15
Data
Bayesian Augmented Control for Antibacterial Agents 16
horizontal dashed
line is historical rate
Bayesian Augmented Control for Antibacterial Agents 17
control arm CIs
under none,full
and hierarchical
borrowing
Bayesian Augmented Control for Antibacterial Agents 18
Treatment CI
(same for all borrowing
as prior is noninformative)
Bayesian Augmented Control for Antibacterial Agents 19
Effective
borrowing
N=mean(p)*(1-mean(p)) / var(p)
numborrowed=N-233
Bayesian Augmented Control for Antibacterial Agents 20
probability of trial
success under each
kind of borrowing
Dynamic borrowing
• The effective amount of borrowing for
193/233 on control (almost identical to
history) is 225 of the 530 historical subjects
• Hierarchical modeling produces dynamic
borrowing.
– as the current control varies away from 82.9%
(history), the amount of borrowing decreases
Bayesian Augmented Control for Antibacterial Agents 21
Bayesian Augmented Control for Antibacterial Agents 22
Bayesian Augmented Control for Antibacterial Agents 23
Bayesian Augmented Control for Antibacterial Agents 24
Observed control proportion (x) versus
effective number of borrowed observations (y)
Summary
• The hierarchical model produces dynamic borrowing (through estimation of τ)
– Many historical subjects are borrowed when the current data is consistent with history
– Few historical subjects are borrowing when the current data is inconsistent
• Most effective (gains the most information) if the historical data is “on point”.
• If you happen to get inconsistent data, less borrowing occurs mitigating the costs.
Bayesian Augmented Control for Antibacterial Agents 25
Operating Characteristics
• For all designs, evaluated Pr(trial success) for
– control rates 0.780, 0.805, 0.830, 0.855, 0.880
– treatment rates 10% lower, 5% lower, equal, or
5% greater than control
• Treatment rates 10% lower were used to
compute type I error
• Equal treatment rates used to compute
power.
Bayesian Augmented Control for Antibacterial Agents 26
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990
Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997
Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999
Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000
Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000
Bayesian Augmented Control for Antibacterial Agents 27
Table shows Pr(trial success)
Two entries per cell in the form FIXED / DESIGN A
Design aims to produce greater power AND lower type
I error for consistent control data, with 20% savings on N.
Here type I error reduced from 0.026 to 0.017 AND
power increased from 91% to 94.2%.
Type I error Power
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990
Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997
Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999
Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000
Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000
Bayesian Augmented Control for Antibacterial Agents 28
Table shows Pr(trial success)
Two entries per cell in the form FIXED / DESIGN A
With slight reduction in true control rate, design still
obtains nearly equivalent power.
Type I error Power
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990
Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997
Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999
Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000
Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000
Bayesian Augmented Control for Antibacterial Agents 29
Table shows Pr(trial success)
Two entries per cell in the form FIXED / DESIGN A
If true current control rate is higher than observed
historical rate, power is increased, but we also observe
inflated type I error.
Type I error Power
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.024 / 0.006 0.309 / 0.119 0.842 / 0.702 0.996 / 0.990
Control 0.805 0.026 / 0.007 0.321 / 0.229 0.874 / 0.861 0.996 / 0.997
Control 0.830 0.026 / 0.017 0.342 / 0.376 0.910 / 0.942 0.998 / 0.999
Control 0.855 0.028 / 0.045 0.372 / 0.564 0.929 / 0.966 1.000 / 1.000
Control 0.880 0.030 / 0.100 0.421 / 0.642 0.954 / 0.976 1.000 / 1.000
Bayesian Augmented Control for Antibacterial Agents 30
Table shows Pr(trial success)
Two entries per cell in the form FIXED / DESIGN A
If true current control rate is much lower than the observed
historical rate, then power is reduced (type I error rate
is negligible).
Type I error Power
Summary for Design A • Pros
– 20% reduction in sample size
– more patients on treatment
– increased or equivalent power for true control rates near observed historical data.
– essentially, there is a “sweet spot” where Design A dominates the fixed trial.
• Cons – inflated type I error for true control rates much
above observed historical control rates.
– decreased power for true control rates much below observed historical data
Bayesian Augmented Control for Antibacterial Agents 31
Design B
• As with design A, hierarchical borrowing and
2:1 randomization, with maximum N=600.
• Incorporates futility stopping
– interim analyses N=300, 400, 500, 600.
– trial stopped for futility if
Pr(non-inferiority)<0.15
• Evaluated operating characteristics as before,
including expected sample size.
Bayesian Augmented Control for Antibacterial Agents 32
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.006 / 0.005
456.0
0.119 / 0.133
572.5
0.702 / 0.692
598.5
0.990 / 0.994
599.7
Control 0.805 0.007 / 0.010
499.7
0.229 / 0.237
589.9
0.861 / 0.851
599.7
0.997 / 0.998
599.7
Control 0.830 0.017 / 0.019
537.9
0.376 / 0.376
595.7
0.942 / 0.938
599.7
0.999 / 0.999
599.7
Control 0.855 0.045 / 0.048
569.3
0.564 / 0.567
598.2
0.966 / 0.968
599.7
1.000 / 0.999
599.7
Control 0.880 0.100 / 0.096
580.2
0.642 / 0.638
598.3
0.976 / 0.970
599.7
1.000 / 0.999
599.7
Bayesian Augmented Control for Antibacterial Agents 33
Three entries per cell. Top two are Pr(trial success)
in the form DESIGN A / DESIGN B. Lower number
is expected N (max 600)
Can save another
60-140 subjects
under null hypothesis
depending on true
control rate.
Operating Characteristics
Trt -10% Trt -5% Trt equal Trt +5%
Control 0.780 0.006 / 0.005
456.0
0.119 / 0.133
572.5
0.702 / 0.692
598.5
0.990 / 0.994
599.7
Control 0.805 0.007 / 0.010
499.7
0.229 / 0.237
589.9
0.861 / 0.851
599.7
0.997 / 0.998
599.7
Control 0.830 0.017 / 0.019
537.9
0.376 / 0.376
595.7
0.942 / 0.938
599.7
0.999 / 0.999
599.7
Control 0.855 0.045 / 0.048
569.3
0.564 / 0.567
598.2
0.966 / 0.968
599.7
1.000 / 0.999
599.7
Control 0.880 0.100 / 0.096
580.2
0.642 / 0.638
598.3
0.976 / 0.970
599.7
1.000 / 0.999
599.7
Bayesian Augmented Control for Antibacterial Agents 34
Three entries per cell. Top two are Pr(trial success)
in the form DESIGN A / DESIGN B. Lower number
is expected N (max 600)
Very slight
changes to
power
Summary for Design B
• Similar Pros/Cons relative to fixed design
• Pros relative to design A
– can save 60-140 subjects on average when null hypothesis is true
– when drug is noninferior, design very rarely stops for futility.
– successful trials retain 400 subjects on treatment.
• Cons relative to design A
– very slight power loss (simulations all have 1% or less power loss)
Bayesian Augmented Control for Antibacterial Agents 35
Statistical Summary
• Hierarchical Modeling allows for competitive
alternatives to fixed designs with significant
sample size savings
• Possible risks are associated with drift in the
true control rate from observed historical
rate.
• Futility stopping can result in additional
sample size savings under null hypothesis
without significant cost to power.
Bayesian Augmented Control for Antibacterial Agents 36
Conclusions
• Hierarchical Modeling allows for
competitive alternatives to fixed designs with
significant reduction in cycle time
• Possible risks are associated with drift in the
true control rate from observed historical rate
• Futility stopping can result in minimizing
exposure to ineffective therapy under null
hypothesis without significant cost to power
Bayesian Augmented Control for Antibacterial Agents 37