Adaptive Trials and the importance of firewalls …….can ... · – Sponsor audits and UAT Case Study : Firewalls ADWG KOL series ; October 2010 9 ... Borrowing from the banking

www.cytel.com

Adaptive Trials and the importance of firewalls…….can technology play a role?

Judith Quinlan VP; Adaptive Clinical [email protected]

• More complex logistics and the importance of firewalls• Case study

– Background– Challenges

• Design and logistics• Managing Firewalls

– Combining technology and SOPs– Why we have chosen this approach

• Summary

Today

ADWG KOL series ; October 20102

Adaptive “by Design”

Strategic Goal supported by

clearly defined: Objectives &

decision rules

Possible design options selected

Including traditional approach

Simulation to compare design Performance

(within context of CDP)

Simulation adding value by helping to quantifydecision process to

select the design best able to deliver study

objectives

Ensuring appropriateFirewalls in place, more complex logistics

3

Ensuring appropriateFirewalls in place, more complex logistics

n

3ADWG KOL series ; October 2010

• General feeling that guidance presented no surprises

• Positive acceptance coupled with understandable cautions

• In particular concerns relating to documentation regarding “who saw what and when”

• FDA noted the potential to “Develop software for audit trails to document design modifications”

• Also discussion comments made by Bob O’Neil that the FDA wants documentation of who knew what when during the trial conduct prior to unblinding

• John Jenkins’s closing remarks also spoke to the need to build trust and verify

March meeting on:FDA Adaptive Design Guidance


Case Study


• Compound– Safety issues considered unlikely

• Compound already approved in another indication– Now being studied in area of “high unmet medical need”

• Development strategy– Originally phase II followed by two phase III trials

• Traditional phase II originally planned– Already a large trial

• Later redesigned as adaptive trial – But still considered phase II trial

• Agency feedback proposed the possibility, considering this was already a large trial, of converting it to a confirmatory trial “provided”– Able to demonstrate trial is an AWC trial– Demonstrate adequate firewalls were in place

Case Study : Background


• Required further increase in “potential” size of the trial• However financial risk considered worth taking• “Extensive” discussions and simulation work completed in respect

to decision rules for dose selection (s)• DESIGN Challenges :

– Selecting not just the “best” dose, but ensuring we have the minimum effective dose (MED) question covered

• Logistical challenges– Time!!

• Simulations (including custom software & validation)• Documentation!

– Protocol, DMC charter, Simulation report• Establishing firewalls

Case Study : Trial Redesigned again


• Two stage adaptive study design• Part A

– Placebo & 3 active doses• Interim Analysis

– Select either one or two doses (plus placebo) to carry forward into Part B

– Possible sample size increase dependent on results (max cap extra 300 subjects)

• based on Promising Zone concept (ref : April KOL presentation by Cyrus Mehta)

– Guidelines for dose selection predefined.• IDMC expected to combine recommendation with clinical

judgment in making final recommendation for Part B• Part B

– End of study analysis combining data from Part A & B

Case Study : Final Study Design


• Protecting the blind & trial integrity…what were we going to do about firewalls??

– Ensuring restricted access to information during the interim analysis process

• Decision• Do we use our standard approach for managing the interim workflow and

notification process OR• Use newly developed technology for managing the interim workflow

process

• Challenges for software development team : Time!– Focus development on features necessary for this interim – IT Set up requirements for hosted solution– System Validation (including CRF Part 11 compliance)– User Testing– Documentation (user guide)– Sponsor audits and UAT

Case Study : Firewalls


Banks have a responsibility to meet the expectations of their customers.

To put systems &processes in place to provide

Security

Protection

Safe guards


•Restricted access to the bank vault

• Access overseen & administered by limited bank personnel

Further security protection through restricted access for individuals

Implementing Security & Protection


• Secure logins• Dashboard of personal accounts

– Actions:• Status review• Fund transfers

– Audit tracking• All transactions time and date stamped• Know when we accessed our accounts & what changes we

made• Account Transparency

Beyond the vault : web based banking



Borrowing from the banking industry to create Firewalls for

Adaptive Designs

DMCDrug Supply

Randomization coordinator

Independent Statistician

Independent PK

Project team

Sponsor Company

ACES

Blinded summaries for project team

Adaptive Design Implementer (ADI)


ACES

CRO/data Management

1: test secure

Data Transfer

2: Produces validatedSAS program that will be used to generate

response file for primary endpoint as

defined in interim SAP

IDMC secretary & programmer

IDMC Independent Statistician

ACES Design Implementer

(ADI)

3: uploads into system initial

randomization list, design engine &

program that will produce response file. Sets security access &

build study

2: IDMC secretary uploads study documentsIncluding protocol, charter

IDMC

6: test notifications sent to DMC members. DMC members test login to ACES

4: Programs to produce semi-blind safety & secondary analysis reportscompleted and QC’d by Independent statistician

Sponsor contact(blinded

summaries?)

IVRS(new rand list or treat prob)

DrugSupply

NotifiedOf

change

6: test runs completed including notifications

that reports are ready for collection

5: Automated event driven data extraction tested

15Interim Workflow tested pre-interim

Case Study


ACES

CRO/data Management

1: secure Data

TransferIDMC secretary & programmer

IDMC Independent Statistician


(ADI)

3 : ACES is set to run to produce primary

report automatically

5: IDMC secretary uploads semi –blind interim safety & secondary reports into ACES

IDMC

6: IDMC collects reports from ACES’s DMC portal after ACES automatically notifies DMC members that

Reports are ready for collection

4: Produces semi-blind safety & secondary analysis reportsas defined in interim SAP & stored in secure area at ISC. Theses are then

reviewed by IDMC Independent statistician or designee

7: at request of IDMC Chair, independent statistician

verballyunblinds IDMC

8: DMC decision Is converted into

reports for communication to

Sponsor, IVRS & drug supply. Reports are

uploaded into ACES by IDMC

chair or secretary Sponsor contact

(blinded summaries)

IVRSDrugSupply

9: Once loaded & approved by DI, ACES automatically notifies IVRS, drug supply &

Sponsor their reports are ready for collection

8: “if” IDMC recommendation is to stop the trial for serious safety

IDMC chair contacts Sponsor

3: Automated data extraction

16Interim Workflow


(ADI)

3a ; 5a ; 8a DI approves reports prior to distribution & notification

Case Study





This is a view of the execution summary Showing the results of two test runs and one active run

Screen view for ADI


Once ADI accepts & delegates a report a notification is sent to the relevant user that a report is in the system for them to collect

Here a DMC has logged on to see the “system generated “ report is available for review & download

DMC member logged on


The ADI selects and reviews the report .The ADI must accept the report before the system sends out notifications

ADI can allow reports to be visible to others but by default only ADI & DMCHave full access to documents


Two other reports have been uploaded into the system by the ISC & accepted by the ADI. These reports are now visible to the DMC

So here a DMC member has logged on to see these two reports are available for review & downloading by clicking on the link

DMC member logged on


However here the Sponsor has logged on & because no documents have been assigned to the Sponsor, no documents are available


In this snapshot of the audit trail we can see who has been assigned access to the system

Here we can see that a ISC report & DMC min have been uploaded by the ISC and then viewed by a DMC member

• Also a need for SOPs as part of the process• Building on what already exists

– Secure FTP transfer for data and initial randomization list– Internal SOPs governing

• separation of blinded and unblinded staff involved on project– file storage

• Usual confidentiality agreements for DMC members• Audit tracking for access to clinical data base

– Preventing back-door access – However, such audit tracking already exists already for most

companies

Combining technology with SOPs


• Many approaches for managing firewalls exist today

• However for us at least, we believe the combination of technology and SOPs provides us with the secure environment we need for managing the interim analysis workflow process

– A system with audit tracking capabilities & report summary capabilities to address the “who saw what and when” question

• Benefits also for managing the increasing number of adaptive trials we see

– “standardization” for how we present audit trail documentation to agencies• Hopefully to help make agency review easier

– Scalability through workflow automation• Overcoming the high manual workload involved in the interim analysis

process• Allowing a experienced statistician to manage several studies

– Particularly when frequent interims are required

Why we have chosen this approach


• The release of the draft FDA guidance on Adaptive Trials has been a significant milestone– Particularly for those who have worked hard over the past

decade to promote the responsible use of adaptive trials• However the guidance document alone will not guarantee the future

success of adaptive trials• Industry has to continue to build “trust” that we can not only design

adaptive trials that satisfy acceptable statistical criteria, but we can also effectively manage the execution of these trials, without compromising the integrity of the trial

• Acceptable Firewalls are critically important in achieving this goal• Many approaches exist today for managing Firewalls, & I have

shown you just one approach

Summary


Documents

Adaptive Trials and the importance of firewalls …….can ... · – Sponsor audits and UAT Case Study : Firewalls ADWG KOL series ; October 2010 9 ... Borrowing from the banking