Pawin Numthavaj, M.D.

2008th year Ph.D. student

Section of Clinical Epidemiology

Faculty of Medicine Ramathibodi Hospital

Mahidol University

Journal Club 22/3/2013

Introduction

• Protocol — document that details• Study rationale

• Proposed methods

• Organization

• Ethical considerations

• “Plan” for study conduction at all stages

Introduction

• Diverse stakeholders• Funding agency

• Ethic committees

• Institutional review boards

• Regulatory agencies

• Medical journals• Systematic reviewer, etc.

• Protocols should adequately address key trial elements

• International groups of stakeholders launched SPIRIT

SPIRIT

• Standard Protocol Items: Recommendations for Intervention Trials

• Initiative started in 2007

• Main output is SPIRIT 2013 statement

• 33 items checklist

• SPIRIT 2013 explanation and elaboration documents

SPIRIT

• 115 contributors• Trial investigators• Healthcare professionals• Methodologists• Statisticians• Trial coordinators• Journal editors• Representatives from ethic committees• Representatives from industry and non-industry funders• Regulatory agencies

• Three complimentary methods• Delphi consensus survey• Two systematic reviews to identify existing protocol guidelines• Two face-to-face consensus meetings to finalize the checklist

Experts

Provide opinions or the team select suitable expert to participate in subsequent rounds

Rank their agreement& Summarize by research team ± rate their opinion confidence

Re-rank their agreement with opportunity to change scores in view of group’s response

Intention of SPIRIT

• Intended as a guide for those preparing a full protocol for clinical trial

• Transparent and complete description of what is intended

• If information for a recommended item is not yet available (e.g., funding sources) this should be explicitly stated, and the protocol updated as new information is obtained

Availability

• Published in key medical journals

• Available with additional examples at www.spirit-statement.org

1. Descriptive title identifying the study design, population, intervention, and, if applicable, trial acronym

• Succinct description that conveys• Topic (Study population, interventions)

• Acronym (If any)

• Basic study design Ex. Allocation (Parallel group randomized controlled trials; single group trial)

• Helpful to include• Framework (superiority/inferiority)

• Objective/primary outcome• Study phase (phase III)

Administrative Information

2a.Trial identifier and registry name. If not yet registered, name of intended registry.

• Increased transparency

• Decrease duplication of research effort

• Facilitate identification of ongoing trials for prospective participants

• Identify selective reporting of result

• Registration should occur before recruitment of first participant

Trial registration - registry

2b. All items from the World Health Organization trial registration data set

• Minimum standard list of items to be included in a trial registry in order for a trial to be considered fully registered

• http://www.who.int/ictrp/network/trds/en/index.html

• Serves as brief structured summary of the trial

Trial registration - dataset

3. Date and version identifier

• Helps to mitigate potential confusions over which document is most recent

• Explicitly listing of changes relative to previous version is also important

Protocol version

4. Sources and types of financial, material, and other support

• Relevant information to assess study feasibility and potential competing interests

• Industry funded are more likely to report trial results and conclusions that favor their own interventions• Select effective intervention for evaluation• Less effective control• Selective report outcomes, analysis, or full study

• Minimum• Sources of financial and non-financial support• Specific type (eg, funds, equipment, drugs, services) and time period

• If not yet supported, detail proposed sources

Funding

5a.Names, affiliations, and roles of protocol contributors

• Provide due recognition, accountability, and transparency

• Help to identify competing interests and ghost authorship• Ghost authorship occurs when an individual makes a substantial contribution to

the research but is not listed as an author.

• Also a standard for protocols published in journals such as Trials

Roles and responsibilities - contributorship

5b.Name and contact information for trial sponsor

• Sponsor: individual, company, institution, or organization assuming overall responsibility for the initiation and management of the trial• Not necessarily the main funder

• Provides transparency and accountability

Roles and responsibilities sponsor contact information

5c. Role of study sponsor and funders in study design; collection, management, analysis, and interpretation of data; writing of the report; and the decision to submit for publication

• Explicitly outline roles and responsibilities of sponsor and any funders in• Study design

• Conduct

• Data analysis and interpretation

• Manuscript writing• Dissemination of results

Roles and responsibility sponsor and funder

5d.Composition, roles and responsibilities of the coordinating center, steering committee, endpoint adjudication committee, and data management team

• Outline general membership of various committees or groups involved in trial coordination and conduct

• Roles and responsibilities

Roles and responsibilities committees

6a.Description of research question and justification, summary of relevant studies examining benefits and harms

• Summarize the importance of the research question

• Justify the need for the trial in the context of available evidences

• Strongly recommended that an up-to-date systematic review of relevant studies be summarized and cited in the protocol

• Failure to do so leads to unnecessary research and exposes participants to possible harms

Introduction: Background and rationale

6b.Explanation for choice of comparators

• Clear description of the rationale for the comparator intervention

• Standard treatment in that condition

• Ex: inappropriately low dose of active drug in control will overestimate the relative efficacy of study intervention in real practice

Background and rationale choice of comparators

7. Specific objectives or hypotheses

• Reflect the scientific question to be answered

• Generally phrased using neutral wording rather than particular direction of effect

• For multi-arm trials, clarify the way the way in which all the treatment groups will be compared (eg, A vs B; A vs C), in what outcome

Objectives

8. Description of trial design including type of trial, allocation ratio, and framework

• Types: parallel group, crossover, factorial, single group

• Framework: superiority, equivalence, non-inferiority, exploratory

• Most common design: parallel group, two-arm, superiority trial with 1:1 allocation ratio.

Trial design

9. Description of study settings and list of countries where data will be collected

• At minimum:• Countries

• Types of setting (urban/rural/hospital/community-based)• Likely number of study sites

Methods participant

10.Inclusion and exclusion criteria for participants, eligibility criteria for study centers and individuals who perform intervention (surgeon)

• Define the study population

• Eligibility for care provider• Promote consistency of intervention

• Clear description of eligibility criteria• Enables study personnel to apply these criteria consistently throughout the trial

• Convey key information related to external validity (generalizability)

Eligibility criteria

11a. Interventions for each group with sufficient detail to allow replication (including how and when)

• Drugs/biological agents/placebos• Generic name

• Manufacturer

• Constituent components

• Route of administration

• Dosing schedule (including titration and run-in periods if applicable)

• Non-drug• Details about setting (Item 9)

• Details of individuals administering intervention (pre-trial expertise, specific training)

Interventions

11b. Criteria for discontinuing or modifying allocated interventions for given trial participants

• Define standard criteria for intervention modifications and discontinuations

• Study participants should be retained in the trial whenever possible to enable follow-up data collection and prevent missing data

Interventions modifications

11c. Strategies to improve adherence to intervention protocols and procedures to monitor adherence

• Low adherence: effect on statistical power and interpretation of trial result

• Description of procedures and strategies for monitoring and improving adherence

Interventions adherence

11d. Concomitant care & interventions permitted or prohibited during the trial

• Co-intervention bias: when study groups receive different care/intervention (in addition to assigned trial intervention) that may affect outcomes

• Protocol should list concomitant care and interventions that are allowed, including rescue intervention

Interventions concomitant care

12. Primary, secondary, other outcomes

• Protocols should define four components for each outcome1. Specific measurement variables (e.g. systolic BP)

2. Specific measurement time point of interest for analysis

3. Analysis metric (e.g. change from baseline, final value, actual value changes over time, time to event)

4. Method of aggregation (mean ( median), proportion , rate)

• Explain the rationale

• Number of primary outcomes should be as small as possible• Problems of multiplicity, selective reporting, and interpretation when there are

inconsistent results among outcomes

Outcomes

12. Primary, secondary, other outcomes (continued)

• Development of common set of key outcomes within a specialty can help to deter selective reporting & facilitate comparisons and pooling in meta-analysis

• COMET (Core Outcome Measures in Effectiveness Trials) initiatives facilitate development and application of such standardized sets• www.comet-initiative.org

http://www.comet-initiative.org/about/researchprojects

13. Time schedule of enrolment, interventions, assessments, and visits for participants. Schematic diagram highly recommended

• Clear and concise timeline helps guide trial conduct and enables external review of burden and feasibility

• Key information• Timing of each visit

• Time periods during which trial interventions will be administered• Procedures and assessments performed at each visit

Participant timeline

14. Estimated number of participants needed to achieve study objectives and how it was determined

• If sample size is not derived statistically, then this should be explicitly stated along with rationale (exploratory pilot studies; pragmatic consideration for rare diseases)

• Calculation based on one primary outcome• Also worthwhile calculating power that will be available for other outcomes

Sample size

Report

• Outcomes

• Value assumed for outcome in each group particularly in comparators with citation

• Statistical test

• Alpha (Type I error) level

• Power

• Calculated sample size per group

15. Strategies for achieving adequate participant enrolment to reach target sample size

• Description of where participants will be recruited (clinic, community), by who, and how (advertisement, review of health records)

• Expected recruitment rates

• Duration of recruitment periods

• Plans to monitor recruitment

• Financial/nonfinancial incentives provided to trial investigators/participants

Recruitment

16a. Method of generating the allocation sequence and list of any factors for stratification.

• Randomization: • Decrease selection bias in allocation• Help blinding• Enable the use of probability theory to test the difference in outcome between

groups

• Use of "randomization" term without further elaboration is not sufficient

• If non-random allocation is planned, specific method and rationale should be stated.

Allocation-sequence generation

Key elements of random sequence to specify

• Method of sequence generation (random number table/generator)

• Allocation ratio

• Type of randomization and reason

• Factors used for stratification

Randomization Types

• Simple randomization

• Restricted randomization• Block randomization

• Biased coin and urn randomization

• Stratified randomization

• Minimisation

Simple randomization

• Based on single, constant allocation ratio• 1:1 allocation – analogous to coin toss

• No other allocation approach surpasses the bias prevention and unpredictability of simple randomization

Restricted randomization

• Any randomization approach that is not simple randomization

• Blocked randomization (permuted block randomization)• Assures that study groups of approximately the same size will be generated

when an allocation of 1:1 is used

• Ensures close balance of groups at any time during the trial

• Reducing the unpredictability of the sequence

• Blinding, larger block size, randomly varying block size will reduce this problem

Restricted randomization

• Stratified randomization• Ensured good balance of participant characteristics in each group• Separate randomization within each of two or more strata of participants (e.g.,

categories of age)• Requires some form of restriction (e.g., blocking within strata)• Number of strata should be limited to avoid over-stratification

• Example: Stratified by center in multicenter trial

• Biased coin and urn randomization• Attain similar objective as blocked design without forcing strict equality• Alter the allocation ratio during the course of the trial to rectify imbalances that

might be occurring• Urn randomization: varying allocation ratios based on the magnitude of the

imbalance

Biased Coin

• Altering allocation probability during the course of the trial to rectify balances in group numbers

• Example: • start with 0.5 / 0.5 probability

• If the disparity reaches the limit the probability change to : 0.6 / 0.4

Urn Randomization

• Adaptive biased-coin

• UD (α,β) – UD (2,1)• Urn contained 2 (α) blue balls and 2 green balls

(0.5 / 0.5 probability)

• Drawn ball at random

• Replace ball with two balls (one blue, one green) and one (β) additional ball of opposite color

• Change in probability during each assignment

Restricted randomization

• Minimization

• Assures similar distribution of selected participant factors between study groups

• Randomization lists are not set up in advance

• First participant is truly randomly allocated

• Subsequent participant: treatment allocation that minimizes the imbalance on the selected factors between group at the time

• Advantage: making small groups closely similar in terms of participant characteristics at all stages of trial

• Some methodologists considered superior to randomization• For SPIRIT, minimization is considered a restricted randomization

Need for separate document to describe restricted randomization

• If some type of restricted randomization is to be used (blocked / minimization) the knowledge of specific details could lead to bias• If block size is 6, trial implementers know that if two As and three Bs have

already been done, they will know the next one

• This is a problem in both open label and ineffectively blinded trials

• Do not provide the full details of a restricted randomization scheme (including minimization) in the trial protocol

• Simple randomization procedures should be reported in detail because simple randomization is totally unpredictable

16b. Mechanism of implementing the allocation sequence (e.g., telephone; envelopes), describing any steps to conceal the sequence

• Concealment aims to prevent participants and recruiters from knowing the study group to which the next participant will be assigned

• Common practice: enclose assignments in sequentially numbered, sealed, opaque envelopes

Allocation concealment mechanism

16c. Who will generate sequence, who will enroll participants, and who will assign participants to intervention

• Complete separation of 2 groups of individuals1. Involved in steps before enrolment (sequence generation, allocation

concealment)

2. Involved in implementation of study group assignments

• Specify• Who will implement randomization process

• How and where allocation list will be stored

• Mechanisms to minimize possibility that those enrolling individuals have access to list

Allocation implementation

17a. Who will be blinded after assignment to interventions and how

• Awareness of intervention can introduce:• Ascertainment bias (different outcome measurements)

• Performance bias (decision to discontinue/modify intervention)

• Concomitant intervention

• Exclusion/attrition bias (decision to withdraw/exclude from analysis)

• When blinding is not possible (obvious differences in interventions):• Blind outcome assessors

• Blind hypothesis in terms of which intervention is considered active

Blinding (Masking)

• Description of who is blinded is preferred over ambiguous terminology such as “double blind”• Trial participants

• Care providers

• Data collectors

• Outcome assessors

• Data analysts

• Manuscript writers

• Describe the comparability of blinded intervention• Appearance

• Flavor/taste

• Timing of final unblinding (e.g. after the creation of a locked analysis data set)

• Strategies to reduce potential for unblinding• Pre-trial testing of blinding procedures

• Use unique code for each participants rather than fixed code (A=Group1, B=Group2)• Unblinding of one participant will result in the loss of blinding for all participants

17b. Circumstances under which unblinding is permissible

intervention during the trial

• Emergency unblinding intended to increase the safety of trial participants

• Clear description of emergency unblinding• Prevent unnecessary unblinding

• Facilitates implementation• Enables evaluation of appropriateness of planned procedures

Blinding (Masking) emergency unblinding

18a. Plans for assessment and collection of outcome, baseline, and other trial data

• Related process to promote data quality• Duplicate measurement, training of assessors

• Description of study instruments• Questionnaire, laboratory tests

• Reliability and validity of instruments

• Where data collection form can be found if not in the protocol

Methods data collection, management, and analysis

• Validity and reliability of trial data depends on quality of data collection

• Avoid• Modified versions of validated measurement tools: no longer be considered

validated

• Unpublished measurement scales

• Standard process should be implemented by local study personnel to enhance data quality and reduce bias

• Inclusion of data collection forms in the protocol (i.e., as appendices) is highly recommended

18b. Plans to promote retention and complete follow-up, list of any outcome data to be collected for those who discontinue or deviate from protocols

• Non-retention: participants are prematurely “off study”• Consent withdrawn• Lost to follow-up

• Plan for how to promote retention in order to prevent missing data

• Methods:• Financial reimbursement• Systematic methods and reminders for contacting patients• Scheduling appointments• Monitoring retention• Limiting burden related to follow-up visits and procedures

Data collection method retention

• Non-adherence: deviation from intervention protocol• Does not mean “off study”

• Should not be reason for ceasing to collect data

• All participants be included in an intention-to-treat analysis, regardless of adherence

• Describe• Retention strategies

• What to record from non-adherence participants

• Plan to record reasons for non-adherence and non-retention

19. Plans for data entry, coding, security, and storage, including process to promote quality (double entry, range checks)

• Full description of data entry and coding process

• Measures to promote data quality

• Document data security measures to prevent unauthorized access

• Plans for data storage during and after trial

• Standard coding practice for non-numeric data to reduce errors and observer variations

Data Management

Data entry

• Local data entry• Fast correction of missing/inaccurate data

• Central data entry (in paper forms)• Facilitates blinding

• Standardization

• Training of a core group of data entry personnel

Data entry

• Standard process to improve accuracy of data entry and coding• Verification of proper format (integer)

• Expected range check

• Double data entry (but weigh time and costs against the magnitude of reduction in error rates)

Statistical methods

• Should be fully described in the protocol

• If certain aspects cannot be pre-specified (e.g. method of examining pattern of missing data) then outline planned approach to making final choice

• May have a separate document: statistical analysis plan (SAP)

20a. Statistical methods for analyzing primary and secondary outcomes

• Indicate explicitly each intended analysis comparing study groups

• Pre-specify main (“primary”) analysis of the primary outcome• Analysis methods to be used

• Which trial participants will be included

• How missing data will be handled

• Helpful to indicate effect measure (e.g., relative risk) and significance level that will be used, as well as intended use of confidence intervals

Statistical methods outcomes

• Specify which comparisons (one or more study groups) will be performed, and which will be the main comparison of interest• Reduce risk of false positives (type I) when multiple statistical comparisons are

performed

• Different trial designs dictate most appropriate plans and additional relevant information• Cluster, factorial, crossover, within-person trial requires specific statistical

consideration

20b. Methods for any additional analyses (e.g., subgroup and adjusted analysis)

• Subgroup analysis• Often selectively reported or not specified

• Post hoc analyses: high risk of spurious findings and are discouraged

• Preplanned subgroup analysis should be clearly specified• Definition of categories

• Statistical method

• Hypothesized direction of subgroup effect based on plausibility

Statistical methods additional analyses

• Adjusted analysis• Indicate if there is intention to perform or consider adjusted analyses

• Explicitly specifying any variables for adjustment and how variables will be handled

• If it is not clear which variable will be important for adjustment, objective criteria to be used to select variables should be pre-specified

20c. Definition of analysis population relating to protocol non-adherence and statistical methods to handle missing data

• Intention to treat (ITT) analysis• “as randomized” analysis retains participants in the group which they were

originally allocated

• Included outcome data obtained from all participants regardless of protocol adherence

• Modification of ITT can introduce bias• Particularly if the frequency of and reasons for non-adherence vary between

groups

Statistical methods analysis population and missing data

• Explicitly describe which participants will be included in the main analysis (e.g., all randomized participants, regardless of protocol adherence)

• Avoid ambiguous use of labels such as “intention to treat” or “per protocol” unless they are fully defined in protocol• Most analyses labelled as such do not actually adhere to the definition

Missing Data Strategies

• Address how missing data will be handled and planned methods to impute (estimate) missing data

• Methods of multiple imputation are more complex, but preferred to single imputation (e.g., last observation carried forward)

• Sensitivity analyses are highly recommended to assess the robustness of trial results under different methods of handling missing data

21a. Composition of data monitoring committee (DMC), roles and reporting structure; whether it is independent from sponsor and competing interests

• Primary role of DMC:• Periodically review accumulating data

• Determine if a trial should be modified or discontinued

• Does not usually have executive power, but with trial steering committee or sponsor

• Independent from sponsor and investigators

• Required to declare any competing interests

Data monitoring formal committee

Details of DMC

• Name of the chair and members of DMC• If members are not yet known: intended size and characteristics

• DMC’s roles and responsibilities

• Planned method of functioning

• Degree of independence from those conducting, sponsoring, or funding the trial

21b. Description of any interim analyses and stopping guidelines; who will have access to interim results and make the final decision to terminate the trial

• Result of interim analyses can be part of stopping guideline (stop for benefit, harm, or futility)

• Can also be used for other trial adaptations• Sample size re-estimation

• Alteration to proportion of participants allocated to each study group

• Changes to eligibility criteria

• Complete description of interim analysis plan should be provided, including statistical methods, who will perform, and when

Data monitoring interim analysis

State:

• Who will see the outcome data while the trial is ongoing

• Whether these individuals will remain blinded to study groups

• How the integrity of the trial implementation will be protected

• Who has the ultimate authority to stop or modify the trial• Principal investigator

• Trial steering committee

• Sponsor

22. Plans for collecting, assessing, reporting, and managing reported adverse events and other unintended effects of interventions

• Adverse event: untoward occurrence during the trial, may or may not related to intervention• Symptoms, signs, laboratory values, or health condition

• Adverse effect: type of adverse event that can be attributed to intervention

• Distinction should be made between anticipated/unanticipated, solicited/unsolicited harms

Harms

• Describe procedures for and frequency of harms data collection

• Overall surveillance timeframe

• Instruments to be used and their validity and reliability

• Plans for data analysis of adverse events

• Address the reporting of harms to relevant groups (sponsor, ethics committee, DMC, regulatory agency)

23. Frequency and procedures for auditing trial conduct and whether the process will be independent from investigators and the sponsor

• Auditing: periodic independent review of core trial processes and documents

• Multicenter trials• Exploring trial dataset

• Performing site visits

• Can be initially conducted across all sites and subsequently conducted using risk-based approaches (e.g., on sites with highest enrollment rates, large number of withdrawals, or atypical numbers of reported adverse events)

Auditing

24. Plans for seeking research ethics committee / institutional review board approval

• Document where approval has been obtained, or outline plans to seek such approval

Research ethics approval

25. Plans for communicating important protocol modifications to relevant parties

• Substantive protocol amendments be reviewed by independent parties such as REC/IRB and transparently described in trial reports

• Describe process of making amendments• Who will be responsible for the decision to amend

• How changes will be communicated to relevant stakeholders (e.g., IRB, regulatory agencies)

Protocol amendments

26a. Who will obtain informed consent or assent and how

• Model consent or assent form should be provided as a protocol appendix

• Assent – minor’s affirmative agreement to participate in the trial• Signing a document that provides age appropriate information about the study

• Details of consent process

• Details of status, experience, and training of team members who will conduct it

• Cluster randomized trials – may not be possible to obtain consent before randomization – explanation should be provided in the protocol

Consent or assent

26b. Additional consent provisions for collection and use of participant data and biological specimens in ancillary studies

• Ancillary studies – additional processes and considerations relating to consent

• Should be detailed in the protocol

• Options to consider consent for• Use of data and specimens in specified protocol

• Use in future research unrelated to clinical condition under study

• Submission to an unrelated bio-repository• Contact by investigators for further information and consent-related purposes

Consent or assent ancillary studies

27. How personal information will be collected, shared, and maintained in order to protect confidentiality before, during, and after trial

• Describe means whereby personal information is collected, kept secure, and maintained

1. Creation of coded depersonalized data

2. Secure maintenance of data and linking code in separate locations (encrypted digital files, password protected folders and storage media)

3. Limiting access to minimum number of individuals necessary for quality control, audit, and analysis

Confidentiality

28. Financial and other competing interests for principal investigators

• Financial:• Salary support/grants

• Ownership of stocks or options

• Honorariums (for advice, authorship, or public speaking)

• Paid consultancy or service on advisory boards

• Receipt of patents or patents pending

• Nonfinancial:• Academic commitments

• Personal/professional relationships

• Political, religious, other affiliations with special interests or advocacy positions

Declaration of interests

29. Who will have access to the final trial dataset, and disclosure of contractual agreements that limit such access for investigators

• Identify the individuals involved in the trial who will have access to full dataset

• Restriction in access for trial investigators should also be explicitly described

Access to data

30. Provisions for ancillary and post-trial care and for compensation to those who suffer harm from participation

• Declaration of Helsinki:• “Protocol should describe arrangements for post-study access by study

participants to interventions identified as beneficial in the study or access to other appropriate care or benefits”

• Describe plans to provide/pay for ancillary care during the trial

• Identify any intervention, benefits, or other care that sponsor will continue to provide to participants and host communities after the trial

• Plans to compensate participants from trial-related harms

Ancillary and post-trial care

31a. Plan to communicate trial results to participants, healthcare professionals, public, and other relevant groups, including publication restriction

• Plan to disseminate results to key stakeholders, including process and timeframe for approving and submitting reports

• Explicit statement that results will be disseminated regardless of the magnitude or direction of effect

• Publication restriction should be disclosed in the protocol for review by REC/IRBs, funders, and other stakeholders

Dissemination policy trial results

31b. Authorship eligibility guidelines and any intended use of professional writers

• Individual who fulfil authorship criteria should not be hidden (ghost authorship)

• Those who do not fulfil criteria should not be granted authorship (guest authorship)

• Professional medical writers are sometimes hired to improve clarity and structure in a trial report• Plans for employment of writer and their funding source both in protocol and

trial reports

Dissemination policy authorship

31c. Plans for granting public access to the full protocol, participant level dataset, and statistical codes

• Indicate whether the trial protocol, full study report, anonymizedparticipant level dataset, and statistical code will be made publicly available

Dissemination policy reproducible research

32. Model consent form and other related documentation given to participants and authorized surrogates

• Several different consent documents may also be needed• Pediatric trial:

• Parental permission

• Participant assent documents

• Multicenter trial: revised consent complied with local requirements

Appendix: Informed consent materials

33. Plans for collection, laboratory evaluation, and storage of biological specimens and future use in ancillary studies

• Protocols should describe details about specimen collection, storage, and evaluation, including location of repositories

• State whether collected samples and associated participant related data will be de-identified or coded to protect participant confidentiality

Biological specimens

Thank You!