SERC M&S: Examples(Screening, Enrollment, Randomization, Completion
Modeling & Simulation)
Dennis Sweitzer, Ph.D.April 2016
Application Scopes
A priori Assumptions ⟶ Simulate⟶ Expected Outcomes, Thresholds( e.g., planned timeline, resources, and expected variability)
Ongoing study⟶Model⟶ Simulate⟶ Projections( e.g., projected timeline, resources, and expected variability given real information)
Projections v. A priori Assumptions ⟶Validation (Consistency)( e.g., are projections from incoming data consistent with assumptions)
Projections v. Observations ⟶Validation (Reality)( e.g., do projections from incoming data match planning expectations)
Model + Scenarios⟶ Simulate ⟶ Alterative Projections
✔
✔
✔
✔
✔
Using patient milestone dates (blinded)(SERC ≣ Screening, Enrollment, Randomization, Discontinuation)
And/or Assumptions used in planningSimple Modeling & Simulation can be used:
Modeling: Survival analysis of time between eventsSimulation: Competing Events model using survival results
Examples ⟹
Example: Multi-‐Segment Studies
Study Flowchart
Randomized Treatment Phase
28 to 104 weeks
Screening &
Enrollment
Open-Label Treatment Phase
12 to 36 weeks
Active
Placebo
Inclusion/Exclusion Criteria
Inclusion/Exclusion Criteria
Screen Failure
Drop Outs
Drop Outs
• Long term randomized withdrawal maintenance studies (AstraZeneca)• Open Label Stabilization (3-‐9mo) + Follow to Relapse (1-‐2yr)
• Standard design, but not in Schizophrenia, bipolar, & other mood– ⟶ Uncertain dropout, relapse, & response rates
• Risks of enrolling– Too few (subjects dropout before relapse)⟶ Failed Study– Too many (subjects in Open Label at last relapse)⟶ Costs, Ethics
Competing Events Model
1. Best guess for initial planning2. As study was running, every month:• Update Statistical Model using patient status data• Simulate remainder of study from model
3. Summarize Simulations to:• Predict milestones (timelines, resources)• Test scenarios (of changes in plans)• Validate study assumptions & detect deviations
Enroll OL Pts
OL Dropouts
Relapse
Rand’dPatients
Rand’dDropouts
M&S ProjectionTrial B, Dates of 200th Event Predicted on 29 Oct by Enrollment Cutoff
12-Feb-06
23-May-06
31-Aug-06
9-Dec-06
19-Mar-07
27-Jun-07
5-Oct-07
13-Jan-08
22-Apr-08
31-Jul-08
10-S
ep-05
24-S
ep-05
8-Oct-
05
22-O
ct-05
5-Nov
-05
19-N
ov-05
3-Dec
-05
17-D
ec-05
31-D
ec-05
14-Ja
n-06
28-Ja
n-06
11-F
eb-06
25-F
eb-06
11-M
ar-06
25-M
ar-06
8-Apr-
06
22-A
pr-06
6-May
-06
20-M
ay-06
Enrollment Cutoffs
Region Based Simulation Actual
Projected End of Study, IF…
… Enrollment ends on this date
Reduced costs: stop enrollment on 3 Dec Reduced Risks: stop by 11 March
Maintenance Studies in 2005Trial A, Predicted Dates of 200th Event
22-Feb-06
8-Mar-06
22-Mar-06
5-Apr-06
19-Apr-06
3-May-06
17-May-06
31-May-06
14-Jun-06
28-Jun-06
12-Jul-06
26-Jul-06
9-Aug-06
23-Aug-06
6-Sep-06
20-Sep-06
4-Oct-06
9-Oct-
05
23-O
ct-05
6-Nov
-05
20-N
ov-05
4-Dec
-05
18-D
ec-05
1-Jan
-06
15-Ja
n-06
29-Ja
n-06
12-F
eb-06
26-F
eb-06
12-M
ar-06
26-M
ar-06
9-Apr-
06
23-A
pr-06
Date of Prediction (Oct 1 Enrollment Cutoff)
Pred
icte
d D
ate
of 2
00th
Eve
nt
Region Based Model (Median) Trial Based Actual
Stop enrolling Stop Randomizing Wait as Patients Relapse or Drop out
Another Case Study
Management feedback:“… the simulations are very valuable and the only way we have to plan our timelines. As it has turned out, your simulations seems to be pretty accurate ...”... We would have been guessing and spinning our wheels without them.”
Date # Randomized Relapses / Dropouts Prediction:101st Relapse
3 Aug’06 73 3 / 2 1 Dec … 15 June
6 Sep’06 182 16 / 7 12 Nov … 21 Feb
2 Oct’06 Stopped Enrolling Patients (NB: 3-‐4 month open label) Dec‘06 Stopped Randomizing Patients (All eligible or discontinued)1 Jan’07 101st Relapse Event
Examples
Validation: Protocols A&B assumed: (50% randomized, 30% Relapse) rateModels estimated: Trial A: (33%, 37%) Trial B: (55%, 41%)
Early Issue Identification ⟶ Quick Corrections
Scenario: ¿Add Sites to compensate for low enrollment?• Run simulation with additional sites• Compare between simulations
Scenario: EMEA requested secondary endpoint of Late Relapses (>4wk off Tx), Trial A had stopped enrolling. Should Trial A be reopened? Should Trial B be extended?• Build new endpoint into simulations• Report
More
A presentation I gave at JSM 2006 on the method, with a proceedings paper. https://sites.google.com/site/dennissweitzer/home/modeling-‐multiphase-‐clinical-‐trials-‐time-‐to-‐completion-‐study-‐management
Simple simulation methods using Excel. I’ve long used Excel simulations to aid in planning clinical trials (for quick & transparent models), although methods for doing so are not well publicized. Here’s a presentation of how-‐to:https://sites.google.com/site/dennissweitzer/home/quick-‐simple-‐simulation-‐using-‐ms-‐excel