Some Phase 23 Clinical Trial Designs

Peter F. ThallBiostatistics & Applied Mathematics Dept

M.D. Anderson Cancer Center

Statistical Design Issues in the Development of Type 1 Diabetes TrialNet Clinical Trials

NIDDK, Bethesda, Maryland March 7, 2005

Outline

I. Problems With Conventional Methods

II. Two-Stage Select-and-Test Designs

- Binary Outcomes

- Time-to-Event Outcomes

III. A Bayesian Phase 23 Design

- Comparison to historical data confounds the E-vs-S treatment effect with between-trial effects

- Data from phase 2 are discarded when phase 3 starts- There is a delay between phase 2 and phase 3

CRITICISMS

CONVENTIONAL PHASE 2 CLINICAL TRIAL- Small study of one experimental treatment (E)- Often a single-arm trial of E alone, without randomization- Efficacy and safety are evaluated using an early outcome (Y) - Data on E are compared to historical data on standard rx (S)-If E is “promising” Organize a randomized phase III trial of E-vs-S based on a harder, time-to-event outcome (T)

- Assumes Y is a reasonable surrogate for assessing E-vs-S treatment effects on survival time T

- If one really believes Y predicts T, then ignoring Y data observed in phase 3 wastes information

- While phase 3 is being organized, time is lost

CONVENTIONAL PHASE 3 CLINICAL TRIAL

- Use phase 2 data to decide what to test in phase 3- Randomize between E and S, usually multi-center- Typically based on T= survival time or DFS time- The scientific standard for deciding if E is “effective”

CRITICISMS

The “Select and Compare” Problem

- K new experimental treatments, E1 , . . . , EK ,are available at the same time

Q1: Is any Ej substantively better than S ?Q2: If two or more Ej’s are substantively better than S, which is the best ?- Example: For rx of acute leukemia or MDS

S = idarubicin + ara-C (IA)E1 = IA + MylotargE2 = IA + TopotecanE3 = IA + Fludarabine

A Deeply Religious Approach- Conduct a K+1 arm randomized phase 3 trial including E1 , . . . , EK and S- Perform all pairwise Ej versus S comparisons, controlling the overall false positive rate- Use a sample size that achieves a given power

- “Power” may be defined in many ways, since there are numerous desirable non-null cases.- If no Ej >> S, this is extremely wasteful.- Even if one E i >> S, this is still very inefficient.

Some Problems With This Approach

A Two-Stage Select-and-Test DesignThall, Simon and Ellenberg Biometrika 75:823-831, 1988

Stage 1: Randomize (K+1)n1 patients among E1,...,EK, S . If the best Ej = E* is minimally promising compared to S, then proceed to stage 2. Otherwise, stop and conclude that no Ej is better than S.

Stage 2: Randomize 2n2 additional patients between E* and S. Based on the data on E* and S from both stages, decide whether E* >> S.

Control overall false positive rate and “power”

Stage 1 Stage 2

S

E1

E2

E3

S

E*

If E*>S go to stage 2

If not, stop and accept the null

Using all data on E* and S, decide whether or not E*>>S

(K+1)n1 patients 2n2 patients

T1 and T2 are the test statistics at stages 1 and 2

Stage 1: T1 < y1 Stop and accept Ho: 1 =… = K = 0

T1 > y1 Proceed to stage 2

Stage 2: T2 < y2 Accept Ho: 1 =… = K = 0

T2 > y2 Conclude * > 0

0 0 + 1 0 + 2

Marginal improvement

over S

Clinically significant improvement

over S

Null response rate

with S

Outcome Total # Patients

Error if

T1 < y1

Stop early and accept Ho

(K+1) n1

Some Ej >> S

T1 > y1,T2 < y2 Accept Ho at

stage 2(K+1) n1 + 2 n2

Some Ej >> S

T1 > y1,T2 > y2 Conclude

E*>>S(K+1) n1 + 2 n2

NoEj >> S

Optimization of Design Parameters

The “least favorable configuration” (LFC) is 1 = = K-1 = 0 + 1 and K = 0 + 2

Given = Overall type I error = Generalized power = Pr(Conclude E* >> S | LFC) 1 , 2 and 0 :

Find {n1 , n2 , y1 , y2 } to minimize ½ E(N | Ho) + ½ E(N | LFC)

Numerical IllustrationK = 30 = .40, = .75, = .05, 1 = .05, 2 = .20

n1 = 47, n2 = 63, y1 = .55, y2 = 1.944 188 patients in stage 1

126 patients in stage 2 314 maximum sample size

Pr(T1 < y1 | Ho) = .47 Eo(N) = 255

T2 > 1.944 Choose E* ; if not, retain S

1) Requires binary clinical outcomes that are observed relatively quickly in both phases

2) The null success probability 0 with S must be known

3) Frequentist Underestimates variability

1) Randomize throughout Stage 1 data on E* and S may be combined with the stage 2 data for the final test

2) Avoids the bias of an uncontrolled pre-test selection

3) Controls overall error rates

4) Simple to implement

Advantages of the Phase 23 Design

Drawbacks

Some Additional Practical Issues

1) A binary outcome usually is inadequate in phase 32) Patient prognostic covariate effects are often larger than

any E-vs-S treatment effect 3) If, say, E1,E2,E3 come from three different drug companies,

the company reps are very unlikely to want the drugs compared to each other. Each company typically wants its drug compared only to S (or to nothing at all).

4) If E2 shows promise in phase 2 while a phase III trial of E1-vs-S is ongoing, should one :

a) start a separate trial of E2-vs-S ? b) add E2 to the ongoing phase 3 trial ? c) run the E1-vs-S trial be to completion and then compare

E2 to the winner ?

A Select-and-Test Design With Event TimesSchaid, Wieand and Therneau, Biometrika 77:507-513, 1990

Stage 1: Randomize (K+1)n1 patients among E1,...,EK ,S during initial accrual period [0, t1].

Tj(t) = log rank statistic for Ej-vs-S at time t

1) If T*(t1) = max{T1(t1),.., TK(t1)} < c1 Stop and conclude that no Ej is better than S.2) If T*(t1) = max{T1(t1),.., TK(t1)} > c2 Stop and conclude that E* is better than S.3) Otherwise, move all Ej with c1 < Tj(t1) < c2 to Stage 2

Stage 2: Randomize (K2+1)n2 patients among S and all K2 experimental treatments surviving stage 1.

Based on

Tj(t2) = log rank statistic comparing Ej to S based on all data through time t2, for j=1,…, K2 ,

compare each (surviving) Ej to S using cut-off c3

Control the overall type I error and “power”

Choose n1 , n2 , t1 , ta , t2 , c1 , c2 , c3 to control the overall type I error while achieving a targeted generalized pairwise power

t1 t2

Stage 2 Accrual Terminated

Final Tests

Stage 1Decisions

ta

1) Fairly complicated to run

2) Allowing multiple Ej’s to go to stage 2 may produce a very large trial

3) Frequentist Underestimates variability

1) Randomize throughout Stage 1 data may be combined with the stage 2 data for the final tests

2) Avoids the bias of an uncontrolled pre-test selection

3) Controls overall error rates

4) A computer program is available from Dan Schaid

Advantages of the Schaid-Wieand-Therneau Design

Drawbacks

Patients with unresectable stage II, IIIA, IIIB Non-Small-Cell Lung Cancer (NSCLC)

Standard Rx, S = CHEMO + RADIATIONMedian survival with S = 15.5 months

Experimental Rx, E = S + Ad-p53Ad-p53 is an adenovirus injected into patient’s tumorAd-p53 may restore apoptosis (programmed cell death) while sensitizing cancer cells to chemo-radiation

A “Seamless” Phase 2 3 DesignInoue, Thall, and Berry, Biometrics 58:823-831, 2002

NSCLC trialPatient outcome is bivariate - Survival time, T - Local control, Y = Indicator that aspiration biopsy of the primary tumor at 5 mos. is negative, a “phase II” outcome (in general, Y may be multinomial)

Prognostic variableStage III (85% patients) vs. Stage II (15% patients)

Accrual & Sample Size- Accrual rate during phase 2 = 20 patients/month- Accrual rate during phase 3 = 30 patients/month

(add an additional 10 institutions) - Limitations: 900 patients, follow-up 6 years

Patients with LC are expected to have better survival than those without LC ()

Ad-p53 may improve the probability of

achieving LC ()

Ad-p53 may improve survival without mediation through LC ()

A MIXTURE MODEL Specify p(T | Y=y,Z) and p(Y=y|Z)

Z = covariates, Y = category-valued variable

Overall survival distribution is a mixture:

p(T | Z) = y p(T | Y=y,Z) p(Y=y | Z)

p(T) accounts for Y not observed if T< t0

Z includes treatment and covariate effects

Illustration of the Mixture Model

RX

LC

No LC

5 months

T=Survival Time, LC=Local Control P[T>24] = P[T>24|LC]* P[LC] + P[T>24|no LC]* P[No LC]

If Prob(2-year survival) = .70 with LC and .40 w/o LC

TRIPLING P[LC] from .16 to .48 only increases Prob[T>24] from .45 to .54, that is, by 21%

Summary of the Design’s Main Features

1) The probability model and decision rules account for BOTH local control and survival time

2) Randomize patients between E & S from the start

3) “Phase 2” data are included in “Phase 3” decisions

4) Decide whether to proceed from “Phase 2” to “Phase 3” repeatedly during the interim period [8 - 12] mos, not at only one point in time

5) Continue accrual in “Phase 2” while “Phase 3” is being organized No “down time”

6) Decisions (go to “Phase 3”, stop for futility or utility) are based on predictive probabilities

= [Mean Survival time with Ad-p53] -

[Mean Survival time without Ad-p53]

[ > 0]=[Ad-p53 improves survival]

1(t) = Predictive probability >0 if accrual is closed at time t & patients are followed 12 more months

2(t)= Predictive probability >0 if accrual continues to 900 patients & follow-up continues to 72 months

A BAYESIAN PHASE 23 DESIGN

At t = 8, 10, and 12 months,

Large 2(t) likely Organize phase 3

Large 2(t) unlikely Stop, conclude E<S

Otherwise Continue phase II

Time to organize Phase 3 ~ 9 months

Deciding Whether to Go to Phase 3

At times t = 8, 10, 12, 16, 20, 24,…,72 : Large 1(t) and large 2(t) both likely

Stop and conclude E>S Neither large 1(t) nor large 2(t) likely

Stop & conclude E<S Neither large 1(72) nor large 2(72) likely

Stop & conclude E<S Otherwise Continue

Decision Rules Used in Both Phases

SURVIVAL

H0

HYPOTHESES

Median survival: 15.5 monthsImprovement of 25%

over null median survival

Ad-p53

LC

SURVIVALAd-p53

LC

H1

Simulation Study

THE BAYESIAN PHASE 23 DESIGN :- 9 months to organize phase 3 - Accrual increases from 20 to 30 patients per

month when phase 3 begins

10,000 Simulation Repetitions per scenario

“CONVENTIONAL” DESIGNS

Phase 2 = Simon Optimal 2-stage (1989)

Early stopping rule for futility, size =.05, power =.90 to detect p1=.16 vs. p2 = .36

n1= 21 at stage 1 and n2=30 at stage 2

Phase 3 = Conventional group-sequential design

- O’Brien-Fleming bounds - Early stopping for both efficacy and futility- Overall Size =.05, Power =.80- Maximum # tests = either 4 or 18

A More Realistic Comparison:Duration of the Entire Phase 23 Process

Conventional 2 + 3

Bayesian 23

9.4

30.7

45.4 46.7

20.4 21.6

H1 H0* H0

SURVIVALAd-p53

LC

H1

Ad-p53 SURVIVAL

LC

Advantages of the Bayesian Phase 23 Design

1) All current data are included in all decisions

2) Accrual is not suspended to evaluate early outcomes

3) The trial continues and data accumulate while phase III is being organized

3) The mixture model may yield a much quicker trial

4) Bayesian Accounts honestly for variability

Drawbacks

1) Very complicated (a simpler version is in development)

2) No usable computer code currently available (see above)

Some Additional References

Follman DA, Proschan MA, Geller NL. Monitoring pairwise comparisons in multi-arm clinical trials. Biometrics 50:325-336, 1994.

Simon R, Thall PF, Ellenberg SS. New designs for the selection of treatments to be tested in randomized clinical trials (Disc: p447-451). Statistics in Medicine 13:417-429, 1994.

Strauss N, Simon R. Investigating a sequence of randomized phase II trials to discover promising treatments. Statistics in Medicine 14: 1479-1489, 1995.

Stallard N, Todd S. Sequential designs for phase III clinical trials incorporating treatment selection. Statistics in Medicine 22:689-703, 2003.

Maitournam A, Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-339, 2005.