SERC  M&S:  Examples(Screening,  Enrollment,  Randomization,  Completion  

Modeling  &  Simulation)

Dennis  Sweitzer,  Ph.D.April  2016

Application  Scopes

A  priori  Assumptions  ⟶ Simulate⟶ Expected  Outcomes,  Thresholds(  e.g.,  planned  timeline,  resources,  and  expected  variability)

Ongoing  study⟶Model⟶ Simulate⟶ Projections(  e.g.,  projected  timeline,  resources,  and  expected  variability  given  real  information)

Projections   v. A  priori  Assumptions  ⟶Validation  (Consistency)(  e.g.,  are  projections  from  incoming  data  consistent  with  assumptions)

Projections    v. Observations  ⟶Validation  (Reality)(  e.g.,  do  projections  from  incoming  data  match  planning  expectations)  

Model  +  Scenarios⟶ Simulate  ⟶ Alterative  Projections

✔

✔

✔

✔

✔

Using  patient  milestone  dates  (blinded)(SERC  ≣ Screening,  Enrollment,  Randomization,  Discontinuation)

And/or  Assumptions  used  in  planningSimple  Modeling  &  Simulation  can  be  used:

Modeling:  Survival  analysis  of  time  between  eventsSimulation:  Competing  Events  model  using  survival  results

Examples  ⟹

Example:  Multi-­‐Segment  Studies

Study Flowchart

Randomized Treatment Phase

28 to 104 weeks

Screening &

Enrollment

Open-Label Treatment Phase

12 to 36 weeks

Active

Placebo

Inclusion/Exclusion Criteria

Inclusion/Exclusion Criteria

Screen Failure

Drop Outs

Drop Outs

• Long  term  randomized  withdrawal  maintenance  studies  (AstraZeneca)• Open  Label  Stabilization  (3-­‐9mo)  +  Follow  to  Relapse  (1-­‐2yr)

• Standard  design,  but  not  in  Schizophrenia,  bipolar,  &  other  mood– ⟶ Uncertain  dropout,  relapse,    &  response  rates

• Risks  of  enrolling– Too  few  (subjects  dropout  before  relapse)⟶ Failed  Study– Too  many  (subjects  in  Open  Label  at  last  relapse)⟶ Costs,  Ethics

Competing  Events  Model

1. Best  guess  for  initial  planning2. As  study  was  running,  every  month:• Update  Statistical  Model  using  patient  status  data• Simulate  remainder  of  study  from  model

3. Summarize  Simulations  to:• Predict  milestones  (timelines,  resources)• Test  scenarios  (of  changes  in  plans)• Validate  study  assumptions  &  detect  deviations  

Enroll OL Pts

OL Dropouts

Relapse

Rand’dPatients

Rand’dDropouts

M&S  ProjectionTrial B, Dates of 200th Event Predicted on 29 Oct by Enrollment Cutoff

12-Feb-06

23-May-06

31-Aug-06

9-Dec-06

19-Mar-07

27-Jun-07

5-Oct-07

13-Jan-08

22-Apr-08

31-Jul-08

10-S

ep-05

24-S

ep-05

8-Oct-

05

22-O

ct-05

5-Nov

-05

19-N

ov-05

3-Dec

-05

17-D

ec-05

31-D

ec-05

14-Ja

n-06

28-Ja

n-06

11-F

eb-06

25-F

eb-06

11-M

ar-06

25-M

ar-06

8-Apr-

06

22-A

pr-06

6-May

-06

20-M

ay-06

Enrollment Cutoffs

Region Based Simulation Actual

Projected  End  of  Study,  IF…  

…  Enrollment  ends  on  this  date

Reduced  costs:  stop  enrollment  on  3  Dec                    Reduced  Risks:  stop  by  11  March

Maintenance  Studies  in  2005Trial A, Predicted Dates of 200th Event

22-Feb-06

8-Mar-06

22-Mar-06

5-Apr-06

19-Apr-06

3-May-06

17-May-06

31-May-06

14-Jun-06

28-Jun-06

12-Jul-06

26-Jul-06

9-Aug-06

23-Aug-06

6-Sep-06

20-Sep-06

4-Oct-06

9-Oct-

05

23-O

ct-05

6-Nov

-05

20-N

ov-05

4-Dec

-05

18-D

ec-05

1-Jan

-06

15-Ja

n-06

29-Ja

n-06

12-F

eb-06

26-F

eb-06

12-M

ar-06

26-M

ar-06

9-Apr-

06

23-A

pr-06

Date of Prediction (Oct 1 Enrollment Cutoff)

Pred

icte

d D

ate

of 2

00th

Eve

nt

Region Based Model (Median) Trial Based Actual

Stop  enrolling Stop  Randomizing Wait as  Patients  Relapse  or  Drop  out

Another  Case  Study

Management  feedback:“…  the  simulations  are  very  valuable  and  the  only  way  we  have  to  plan  our  timelines.  As  it  has  turned  out,  your  simulations  seems  to  be  pretty  accurate  ...”...    We would have been guessing and  spinning  our wheels without them.”

Date # Randomized Relapses  /  Dropouts Prediction:101st Relapse

3 Aug’06 73 3  /  2 1  Dec …  15  June

6  Sep’06 182 16 /  7 12  Nov  …  21  Feb

2  Oct’06                                            Stopped  Enrolling  Patients        (NB:  3-­‐4  month  open  label)  Dec‘06                                                  Stopped  Randomizing  Patients  (All  eligible  or  discontinued)1  Jan’07                                                                            101st Relapse  Event

Examples

Validation: Protocols A&B assumed: (50% randomized, 30% Relapse) rateModels estimated: Trial A: (33%, 37%) Trial B: (55%, 41%)

Early  Issue  Identification              ⟶ Quick  Corrections

Scenario:                                       ¿Add Sites to compensate for low enrollment?• Run  simulation  with  additional  sites• Compare  between  simulations

Scenario:   EMEA  requested  secondary  endpoint  of  Late  Relapses  (>4wk  off  Tx),  Trial  A  had  stopped  enrolling.  Should  Trial  A  be  reopened?  Should  Trial  B  be  extended?• Build  new  endpoint  into  simulations• Report

More  

A  presentation  I  gave  at  JSM  2006  on  the  method,  with  a  proceedings  paper.  https://sites.google.com/site/dennissweitzer/home/modeling-­‐multiphase-­‐clinical-­‐trials-­‐time-­‐to-­‐completion-­‐study-­‐management

Simple  simulation  methods  using  Excel.  I’ve  long  used  Excel  simulations  to  aid  in  planning  clinical  trials  (for  quick  &  transparent  models),  although  methods  for  doing  so  are  not  well  publicized.  Here’s  a  presentation  of  how-­‐to:https://sites.google.com/site/dennissweitzer/home/quick-­‐simple-­‐simulation-­‐using-­‐ms-­‐excel