Embed Size (px)
Citation preview
Architecture Design of Generic Outcome
Adjudication in CTMS
Wenle Zhao, PhD
Medical University of South Carolina, Charleston, SC, 29425, USA
Society for Clinical Trials 36th Annual MeetingArlington, VA, USA - May 17-20, 2015
Contents
1. The Motivation
2. The Challenges
3. The Generic Database Model
4. The Automated Coordination
5. Summary
1.The Motivation
2. The Challenges
3. The Generic Database Model
4. The Automated Coordination
5. Summary
Patients
Randomization
Treatment A Treatment B
Outcome A Outcome B
Analysis
Accurate Outcome ( Endpoint) Assessment is Important
The validity of the trial results relies on the accuracy of outcome assessment.
Outcome
Safety Outcome Efficacy Outcome
AE is Serious? Quality of Life
Modified Ranking Scale
NIH Stroke Scale
Hamilton Rating Scale for Depression
Hamilton Rating Scale for Anxiety
Most Outcome Assessments are Subjective
AE is Expected?
AE is Related to Study Treatment?
Glasgow Outcome Scale Extended
The Problem: Uneven expertise and potential bias The Solution: Independent central adjudication
1. The Motivation
2.The Challenges
3. The Generic Database Model
4. The Automated Coordination
5. Summary
Example 1: A Simple SAE Adjudication
SAE CRF Submitted
Is serious? Is expected? Is related?
Done
CRF Completed?
May be not.
How to notify the adjudicator?
Auto Email.
When it is expected?
No schedule.
Need more information
?May be
yes.
CRF Updated after submission?Could happen.
Four challenges for coordinating outcome adjudication
Adjudication procedures vary among projectsNo fixed model for every procedure.
Timing of outcome event unpredictableNo schedule.
Narrow time window for completion of adjudication“Time is of the essence.”
Data changing after adjudication procedure started
The target is moving.
1. The Motivation
2. The Challenges
3.The Generic Database Model
4. The Automated Coordination
5. Summary
The Generic DB Structure for Adjudication Procedures
1
∞
∞
1
∞1
1
1∞
1
∞
1
Adjudication ProcedureIDForm IDProcedure Name
Adjudication StepIDProcedure IDAdjudication Step NameEnable-logic conditionDisable-logic conditionQuestion 1Answer 1 Code Group ID……Question 5Answer 5 Code Group IDEdit User Group IDView User Group ID
Adjudication ResultIDAdjudication Step IDRecord IDFreeze Record IDAnswer 1……Answer 5General CommentsLocked
Data (CRF) FormIDForm NameTable Name
Form TableIDSubject IDVisit IDQ01Q02……Updated ByUpdated On
Form Freeze TableFreeze Record ID
Subject IDVisit IDQ01Q02……Updated ByUpdated On
ID
The Model
Allow multiple steps.
Allow multiple updates.
Limit ≤ 5 close-ended
Qs.
Two Tables are included in the Database Template
One table specifies definitions of each adjudication step.
The other table stores adjudication results.
A freeze table is created for each data form table with adjudication
Adjudication Step Enable logic Disable logic Question 1
(Q1)Question 2
(Q2)Question 3
(Q3)
Step 1 PM Review
(S1){New SAE CRF submission}
{S2 submitted after S1} OR
{S3 submitted after S1}
CRF properly completed?
0 = No1 = Yes
Requires MSM review?0 = No1 = Yes
Step 2 MSM
Adjudication(S2)
{S1Q1=1} AND {S1Q2=1} AND
{no S2 submitted after S1} AND
{no S3 submitted after S1}
{S2 submitted after S1} OR
{S3 submitted after S2}
Serious?0 = No1 = Yes
Unexpected?0 = No1 = Yes
Related to study drug?
1 = Unrelated2 = Unlikely3 = Possible4 = Probably5 = Definitely
Step 3 PM Review
(S3)
{S2 submitted} AND
{no S3 submitted after S2}{S3 submitted after S1}
MedWatch Form needed?
0 = No1 = Yes
Example 1: Definition of Three-step SAE Adjudication
Adjudication Step Enable logic Disable logic Question 1(Q1)
Question 2(Q2)
Step 1 Project Manager
(PM) Review{New SAE CRF submission} {S2 submitted after S1}
CRF properly completed?0 = No1 = Yes
Requires CEC review?0 = No1 = Yes
Step 2 Clinical Event
Coordinator (CEC) Review
{S1Q1=1} AND {S1Q2=1} AND{no S2 submitted after S1}
{S1 submitted after S2} OR {S3 submitted after S2} OR{S4 submitted after S2} OR
Even packet complete?0 = No1 = Yes
Event type?1 = Neurological2 = Cardiac3 = Systemic
Step 3 Adjudicator 1
Review
(S2Q1=1) AND {no S5 submitted after S3}
{S1 submitted after S3} OR{S5 submitted after S3}
Outcome category=?1 = ischemic stroke2 = symptomatic
hemorrhagic transformation of an ischemic stroke
13 = transient ischemic attack
Death type =?1 = ischemic2 = hemorrhagic vascular3 = nonvascular
Step 4 Adjudicator 2 (ADJ2) Review
(S2Q1=1) AND {no S5 submitted after S4}
{S1 submitted after S4} OR{S5 submitted after S4}
Step 5 Adjudicator 3 (ADJ3) Review
{S3 submitted after S1} AND{S4 submitted after S1} AND{S3Q1≠S4Q1 or S3Q2≠S4Q2} AND{no S6 submitted after S5} AND
{S1 submitted after S5} OR{S6 submitted after S5}
Step 6Adjudication
Committee Chair (ACC) Review
{S5 submitted after S1} AND{S5Q1≠S3Q1 or S5Q2≠S3Q2} AND{S5Q1≠S4Q1 or S5Q2≠S4Q2} AND{no S1 submitted after S6} AND
{S6 submitted after S1}
Example 2: Definition of Six-step SAE Adjudication
The Generic Table for Adjudication Results
1. The Motivation
2. The Challenges
3. The Generic Database Model
4.The Automated Coordination
5. Summary
Site submits a Case Report Form
Form has adjudication procedure defined?
System evaluates enable logics for each adjudication steps
Enable logic = true & disable logic = false?
System sends email to [Edit] user group members
User views a Case Report Form
User has adjudication step [Edit] permission
& Enable logic = true & disable logic = false?
System displays [Adjudication Edit] button for the user
User adds/editsadjudication result and submits
adjudication record
The last freeze record is the same as the current
CRF record?
System creates new freeze record, links it to the adjudication record.
Y
Y
Y
N
Y
N
N
N
Automated Coordination of Adjudication Activities
Site submits SAE CRF
S1Q1 = Yes?
Step S3 = PM ReviewS3Q1: MedWatch Form required?
System composes MedWatch Form draftN
Y
Y
N
Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires MSM adjudication?
Auto email #1 to PM
Auto email #3 to MSM
Auto email #2 to Site
S1Q2 = Yes?
Step S2 = MSM Adjudication S2Q1: Serious? S2Q2: Unexpected? S2Q3: Related to study treatment?
Site adds/edits SAE CRF
Auto email #4 to PM
S2Q1 = Yes & S2Q2 = Yes &S2Q3 = Possible/ Probably / Definitely?
S3Q1 = Yes?
Auto email #6 to Site
Site edits MedWatch
PM Closes the Adjudication Procedure
Auto email #7 to PM
PM finalizes MedWatch and submits to FDA
Auto email #8 to All Sites
N
Y
Auto email #5 to PM
Y
N
Example 1: Three-step SAE Adjudication Procedure
Site submits SAE CRF
S1Q1 = Yes?
Step S3 = ADJ1 ReviewS3Q1: Outcome category?S3Q2: Death type?
Y
N
Step S1 = PM Reviews S1Q1: CRF properly completed? S1Q2: Requires CEC review?
Auto email #1 to PM
Auto email #3 to CEC
Auto email #2 to Site
S1Q2 = Yes?
Step S2 = CEC Adjudication S2Q1: Event packet complete? S2Q2: Event type?
Site adds/edits SAE CRF and upload the event packet
Auto email #7 to ADJ3
Auto email #9 to PM
PM Closes this
Adjudication Procedure
N
Y
Auto email #5 to ADJ1
Y
Step S5 = ADJ3 ReviewS5Q1: Outcome category?S5Q2: Death type?
Step S4 = ADJ2 ReviewS4Q1: Outcome category?S4Q2: Death type?
S3 = S4?
Auto email #6 to ADJ2
N
System determines adjudicators 1, 2, and 3 based on S2Q2 and the Random number R.
Auto email #4 to Site
YS2Q1 = Yes?
N
Auto email #8 to ACC
Y
Step S6 = Chair ReviewS6Q1: Outcome category?S6Q2: Death type?
S3= S5 or S4 = S5?N
Example 2: Six-step Outcome Adjudication Procedure
In this case, the Clinical Event Coordinator (CEC) completed the adjudication within few hours after the Study Manager (SM) ‘s completeness review passed.
In this case, the procedure has not been completed yet after two months, due to CRF data change.
1. The Motivation
2. The Challenges
3. The Generic Database Model
4. The Automated Coordination
5.Summary
AdjudicationProcedure
From step To stepEvent count
1st quartile(25%)
2nd quartile(50%)
3rd quartile(75 %)
hrs hrs hrs
Three-step SAE
adjudication
CRF submit PM review 1253 21 51 126
PM review MSM review 1226 5 13 35
MSM review PM close 1226 27 56 112
Six-step outcome
event adjudication
CRF submit PM review 609 26 120 477
PM review CEC review 599 96 197 364
CEC review ADJ1/2 review 468 47 204 892
ADJ1/2 review ADJ3 review 76 56 209 383
ADJ3 review ACC review 10 23 23 25
PM: project managerMSM: medical safety monitorCEC: clinical event coordinator
ADJ1/2: adjudicators 1 and 2ADJ3: adjudicator 3ACC: adjudication committee chair
Our Experiences
The Strategy to Success
Allows multiple adjudication procedures per CRF.
Allows multiple steps for each adjudication procedure .
Allows different questions be asked for each adjudication step.
Only close-ended questions are allowed.
No more than 5 questions are allowed.
Using automated emails to help adjudication coordination.
Allows multiple CRF submissions after adjudication started.
Acknowledgement
This research is partly supported by following NIH/NINDS grants:
• U01NS059041 (NETT Palesch, Y. & Durkalski, V.)• U01NS062778 (ProTECT, Palesch Y.)• U01NS062835 (POINT, Palesch, Y.) • U01NS087748 (StrokeNet, Palesch, Y. & Zhao, W.)
The followings are contributed to the design, development, validation and user feedback collection to this work:• Keith Pauls (DCU senior programmer)• Jaemyung Kim (DCU senior programmer)• Catherine Dillon (DCU senior data manager)• The Clinical Coordination Center of the NINDS NETT at
University of Michigan
Thousands investigators in hundreds institutions and thousands patients participating our clinical trials.
