View
221
Download
3
Category
Preview:
Citation preview
Design and Analysis of Sequential Multiple AssignmentRandomized Trials (SMARTs)
Bibhas ChakrabortyDuke-NUS Medical School, National University of Singapore
bibhas.chakraborty@duke-nus.edu.sg
IMS Workshop on Design of Healthcare StudiesSingapore
July 5, 2017
1 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
2 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
Personalized Medicine and Dynamic Treatment Regimes
Personalized or Precision Medicine
Paradigm shift from “one size fits all” to individualized, patient-centric care
– Can address inherent heterogeneity across patients
– Can also address variability within patient, over time
– Can increase patient compliance, thus increasing the chance of treatment success
– Likely to reduce the overall cost of health care
Overarching Methodological Questions:
– How to decide on the optimal treatment for an individual patient?
– How to make these treatment decisions evidence-based or data-driven?
4 / 66
Personalized Medicine and Dynamic Treatment Regimes
Treatment Regimes or Regimens
One perspective in personalized medicine: given an individual patient’scharacteristics, can we identify a treatment, among the available options, that ismost likely to confer the most benefit?
A treatment regimen is a rule (or rules) that specifies how to allocate treatmentbased on an individual patient’s characteristics
If treatment is administered only once, then there is a single decision point, oftenthe time of diagnosis
In case of chronic diseases, treatments are typically administered at multipledecision points
5 / 66
Personalized Medicine and Dynamic Treatment Regimes
Dynamic Treatment Regime(n)s
A dynamic treatment regimen (DTR) is a sequence of decision rules, one perdecision point (or stage), that specify how to adapt the type, dosage and timingof treatment according to the ongoing information on an individual patient
– Each decision rule takes a patient’s treatment and covariate history as inputs, andoutputs a recommended treatment
Decision points can occur at regular intervals (e.g. yearly follow-up visits) orclinical events (e.g. remission, relapse, complication)
6 / 66
Personalized Medicine and Dynamic Treatment Regimes
Dynamic Treatment Regimens (DTRs)
DTRs offer a data-driven framework for operationalizing the adaptive clinicaldecision-making in a time-varying setting, and thereby potentially improving it
– Clinical decision support systems for treating chronic diseases
A subject’s stage-specific treatment is not known at the start of a dynamicregimen, since treatment depends on time-varying variables
7 / 66
Personalized Medicine and Dynamic Treatment Regimes
ADHD Example: One Simple DTR
“Give Low-intensity BMOD initially; if the subject responds, then continue BMOD,otherwise prescribe BMOD + MEDS”
8 / 66
Personalized Medicine and Dynamic Treatment Regimes
Treatment Regimen vs. Realized Treatment Experience
Subjects following the same DTR can have different realized treatmentexperiences:
– Subject 1 experiences “Low-intensity BMOD, followed by response, followed byLow-intensity BMOD”
– Subject 2 experiences “Low-intensity BMOD, followed by non-response, followedby BMOD + MEDS”
9 / 66
Personalized Medicine and Dynamic Treatment Regimes
Notation and Data Structure
K stages (or decision points) on a single patient:
O1,A1, . . . ,OK ,AK ,OK+1
Oj : Observation (pre-treatment) at the j-th stageAj : Treatment (action) at the j-th stage, Aj ∈ Aj
Hj : History at the j-th stage,Hj = {O1,A1, . . . ,Oj−1,Aj−1,Oj}Y : Primary Outcome (assume larger is better)
A DTR is a sequence of decision rules:
d ≡ (d1, . . . , dK) with dj(hj) ∈ Aj
10 / 66
Personalized Medicine and Dynamic Treatment Regimes
The Big Scientific Questions in DTR Research
What would be the mean outcome if the population were to follow a particularpre-conceived DTR?
How do the mean outcomes compare among two or more DTRs?
What is the optimal DTR in terms of the mean outcome?
– What individual information (tailoring or prescriptive variables) do we use to makethese decisions?
11 / 66
Personalized Medicine and Dynamic Treatment Regimes
The Big Statistical Questions
1 What is the right kind of data for comparing two or more DTRs, or estimatingoptimal DTRs? What is the appropriate study design?
– Sequential Multiple Assignment Randomized Trial (SMART)
2 How can we compare pre-conceived, embedded DTRs?– primary analysis of SMART data
3 How can we estimate the “optimal” DTR for a given patient?
– secondary analysis of SMART data– e.g. Q-learning, a stagewise regression-based approach
12 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
SMART Design
DTR Discovery: Experimental Data
As is well-known, experimental data are generally better than observational data(if you can afford)
Standard RCTs assess the effectiveness of a single dose-level of a singletreatment, as compared to another
Estimating the sequence of treatments that optimizes response in a longitudinalsetting requires studying the elements in the sequence
Can this be accomplished by a series of single-stage RCTs?
14 / 66
SMART Design
Example: Treating MDD
Suppose we wish to compare both front-line and second-line treatment of majordepressive disorder (MDD):
– Front-line options: citalopram (Cit) or cognitive behavioral therapy (CBT)
– Second-line options: treatment switch to Cit, CBT, or Lithium (Li)
– All responders to first-line therapy will continue with maintenance and follow-up
15 / 66
SMART Design
Example: Treating MDD
CBT
Cit
R
Maintenance dose +telephone monitoring
RCBT
Li
RCit
Li
Telephone monitoring
16 / 66
SMART Design
Example: Treating MDD
CBT
Cit
R
Maintenance dose +telephone monitoring
RCBT
Li
RCit
Li
Telephone monitoring
16 / 66
SMART Design
Front-line Treatment of MDD
CBT
Cit
R
Suppose we observe 60% response with Cit, andonly 50% with CBT
Conclude: Cit is the best front-line therapy
Now run another one-stage trial amongst Citnon-responders
17 / 66
SMART Design
Second-line Treatment of MDD
Cit
Maintenance dose +telephone monitoring
RCBT
Li
60% respond
We now observe 40% response to CBT and 20% to Li
Conclude: CBT is the best second-line therapy
Final treatment sequence: “Cit, followed by CBT for non-responders”
– Under this regimen, we expect to see 76% of patients respond
18 / 66
SMART Design
Delayed Effect
What if initial treatment with CBT increases treatment adherence⇒ subsequenttherapies more successful?
CBT
Cit
R
Maintenance dose +telephone monitoring
RCBT
Li
RCit
Li
Telephone monitoring
60% respond
50% respond
40% respond
20% respond
60% respond
30% respond
Optimal DTR: “CBT, followed by Cit for non-responders”; 80% responseexpected
19 / 66
SMART Design
SMART!
Two single-stage trials would not have detected the best overall strategy fortreatment
Instead, we should have used a two-stage trial
Such two-stage trials are known as SMARTs (Lavori and Dawson, 2004;Murphy, 2005), i.e.
Sequential Multiple Assignment Randomized Trials
Susan Murphy won the MacArthur Foundation “Genius Grant” in 2013 forinventing the SMART design:
http://www.macfound.org/fellows/898/
20 / 66
SMART Design
SMARTs, in general
Multi-stage trials with a goal to inform the development of DTRs
Same subjects participate throughout (they are followed through stages oftreatment)
Each stage corresponds to a treatment decision
At each stage the patient is randomized to one of the available treatment options
Treatment options at randomization may be restricted on ethical grounds,depending on intermediate outcome and/or treatment history
21 / 66
SMART Design
SMART: The Gains
Ability to detect
– delayed therapeutic effects (treatment interactions)
– diagnostic effects
More generalizable than standard RCT?
Better recruitment and retention potential than standard RCT?– By virtue of the option to alter a non-functioning treatment while in the trial
22 / 66
SMART Design
SMART: The Costs
Unfortunately there is no free lunch!!
More expensive than a single-stage RCT
Longer follow-up
Need more participants only if the trial is powered to compare whole sequences oftreatments – but there exist less expensive alternatives to power SMARTs!
More complex methods for planning and analysis: requires an experiencedstatistician, or one with time to learn new methods
May require additional work to fund: still relatively new, unfamiliar
23 / 66
SMART Design
SMART vs. Crossover Trial Designs
Operationally, SMARTs look similar to crossover trials
However, conceptually they are very different
– Unlike SMART, Crossover trials aim to evaluate stand-alone treatments, not DTRs
– Unlike a crossover trial, treatment allocation in a SMART is typically adaptive to asubject’s intermediate outcome
– Crossover trials attempt to “wash out” the “carry-over” effect while SMARTsattempt to capture it
24 / 66
SMART Design
SMART vs. Usual Adaptive Trial Designs
SMARTs are different from usual adaptive trials
– Within-subject vs. between-subject adaptation
In an adaptive trial, the randomization probabilities can change during the courseof the trial depending on the relative success of already-recruited patients oneach of the treatments
– Adaptive trials typically aim to reduce the number of trial participants exposed tothe inferior treatment
– Adaptive trials are most useful for examining questions where the outcome followsfairly quickly after treatment and/or the recruitment process is (relatively) slow
– SMARTs are fixed trial designs – while randomization probabilities may depend oncovariates, those probabilities remain constant over the course of the trial
25 / 66
SMART Design
From SMART to SMART-AR
Integration of the two concepts (SMART and adaptive design) is a topic ofcurrent research
– SMARTs can be made more ethically appealing by incorporating adaptiverandomization
– In certain modern contexts (e.g., implementation research and mHealth), SMARTwith Adaptive Randomization (SMART-AR)1 has been developed recently
– In general, how best to do this is not known yet
1Cheung YK, Chakraborty B, and Davidson K (2015). Sequential multiple assignment randomized trial(SMART) with adaptive randomization for quality improvement in depression treatment program.Biometrics, 71(2): 450-459.
26 / 66
SMART Design
Does anyone actually use SMARTs?
ADHD Trial (PI: Pelham; see Lei et al., 2012, for design details)
27 / 66
SMART Design
Other Examples of SMART Studies
Schizophrenia: CATIE (Schneider et al., 2001)
Depression: STAR*D (Rush et al., 2003)
Prostate Cancer: Trials at MD Anderson Cancer Center (e.g., Thall et al., 2000)
Leukemia: CALGB Protocol 8923 (e.g., Stone et al., 1995; Wahed and Tsiatis,2004)
Smoking Cessation: Project Quit (Strecher et al., 2008; Chakraborty et al.,2010)
Alcohol Dependence: Oslin et al. (see, e.g., Lei et al., 2012)
Many recent examples available at the Methodology Center, PennStateUniversity website:
http://methodology.psu.edu/ra/smart/projects
28 / 66
SMART Design Primary Analysis and Sample Size
SMART Design Principles
Primary and Secondary Hypotheses
Choose scientifically important primary hypotheses that also aid in developingDTRs
– Power trial to address these hypotheses
Choose secondary hypotheses that further develop the DTR, and userandomization to eliminate confounding
– Trial is not necessarily powered to address these hypotheses
– Still better than post hoc observational analyses
– Underpowered randomizations can be viewed as pilot studies for future full-blowncomparisons
From a funding perspective, it may be more feasible to design a standardtwo-arm trial as primary goal, and add the secondary randomizations and DTRestimation as a secondary goal
29 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Embedded DTRs
Embedded DTR-1
30 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Embedded DTRs
Embedded DTR-2
31 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Embedded DTRs
Embedded DTR-3
32 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Embedded DTRs
Embedded DTR-4
33 / 66
SMART Design Primary Analysis and Sample Size
Primary Hypothesis and Sample Size: Scenario 1
Hypothesize that averaging over the secondary treatments, the initial treatmentBMOD is as good as the initial treatment MEDS
– Sample size formula is same as that for a two group comparison
34 / 66
SMART Design Primary Analysis and Sample Size
Primary Hypothesis and Sample Size: Scenario 2
Hypothesize that among non-responders an augmentation (BMOD+MEDS) is asgood as an intensification of treatment
– Sample size formula is same as that for a two group comparison ofnon-responders (overall sample size depends on the presumed non-response rate)
35 / 66
SMART Design Primary Analysis and Sample Size
Primary Hypothesis and Sample Size: Scenario 3
Hypothesize that the “red” DTR (embedded DTR-2) is as good as the “green” DTR(embedded DTR-3)
– Sample size formula involves a two group comparison of “weighted” means(overall sample size depends on the presumed non-response rate)
36 / 66
SMART Design Primary Analysis and Sample Size
A Bit of Formalism...
Objective: compare two embedded regimens in the SMART with continuousoutcome
Embedded regimens are:
d1 =(
BMOD,BMODR(BMOD + MEDS)1−R)
d2 =(
BMOD,BMODR(BMOD + +)1−R)
d3 =(
MEDS,MEDSR(BMOD + MEDS)1−R)
d4 =(
MEDS,MEDSR(MEDS + +)1−R)
(where R = 1 denotes responder and R = 0 denotes non-responder)
37 / 66
SMART Design Primary Analysis and Sample Size
Data Structure
(O1i,A1i,O2i,A2i,Yi) for i = 1, · · · ,N, where
– O1 = ∅
– A1 ∈ {BMOD,MEDS}
– O2 = R is the response indicator (1/0)
– A2 ∈ {BMOD,BMOD ++,MEDS,MEDS ++,BMOD + MEDS}
– Y is the continuous end-of-trial primary outcome
– N is the trial size
38 / 66
SMART Design Primary Analysis and Sample Size
Regimen Mean: The Key Estimand
For the embedded regimen d1 = (BMOD,BMODR(BMOD + MEDS)1−R), themean outcome is
µd1 = Ed1(Y)
= E[ I{A1 = BMOD,A2 = BMODR(BMOD + MEDS)1−R}
π1(A1)π2(A2|A1,R)Y]
= E(Wd1 Y), say
– π1 and π2 are the randomization probabilities
– Inverse probability weighting (IPW) approach
– Change of probability distribution (measure) by Radon-Nikodym derivative
Similarly, one can write expressions for other regimens d2, d3, d4
39 / 66
SMART Design Primary Analysis and Sample Size
Estimation of Regimen Mean
The regimen mean can be estimated by the IPW estimator (Robins et al., 2000):
Yd1 =
N∑i=1
Wd1i Yi
/ N∑i=1
Wd1i
For large samples (Murphy, 2005):
Var(Yd1) =1
N2
N∑i=1
(Wd1i )2(Yi − Yd1)
2
40 / 66
SMART Design Primary Analysis and Sample Size
Sample Size Requirements
Assume continuous outcome
Key Design Parameters:
Effect Size = ∆µσ (Cohen’s d)
Type I Error Rate = α = 0.05Desired Power = 1− β = 0.8
Initial Response Rate = γ = 0.5
Trial Size:
Effect Scenario 1 Scenario 2 Scenario 3Size0.3 N1 = 350 N2 = N1
(1−γ) = 700 N3 = N1 × (2− γ) = 5250.5 N1 = 128 N2 = N1
(1−γ) = 256 N3 = N1 × (2− γ) = 1920.8 N1 = 52 N2 = N1
(1−γ) = 104 N3 = N1 × (2− γ) = 78
41 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Other Comparisons
Comparing embedded DTR-1 (d1) with embedded DTR-2 (d2)
d1 and d2 share the initial treatment of BMOD
Patients who respond to BMOD are ‘consistent’ with both d1 and d2, andcontribute information on the overall response to both regimens
Specialized methods are needed to account for “re-using” their information
42 / 66
SMART Design Primary Analysis and Sample Size
SMART Design: Other Comparisons
Alternatively, we may wish to:compare d1 vs. d2 vs. d3 vs. d4 to see which of the four regimens leads to the bestexpected outcome
The above question requires more specialized methods to account for bothmultiple comparisons as well as the re-use of data for patients who are consistentwith more than one regimen
43 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
Analysis: Estimation of Optimal DTRs
Secondary Analysis of SMART Data
Goal: To find optimal treatment sequence for each individual patient by deeplytailoring on their time-varying covariates and intermediate outcomes
– This is to develop the evidence-based decision support system for clinicians (theycan apply these regimens while treating future patients)
Methodologically challenging – custom-made analytic tools necessary
– One popular approach is Q-learning, a stage-wise regression-based method
– We developed an R package called qLearn (Xin et al., 2012) that conductsQ-learning (Freely available at CRAN):
http://cran.r-project.org/web/packages/qLearn/
45 / 66
Analysis: Estimation of Optimal DTRs Q-learning
Q-learning: Introduction
Q-learning (Watkins, 1989)
– A popular method from Reinforcement (Machine) Learning
– A generalization of least squares regression to multistage decision problems(Murphy, 2005)
– Adapted to and implemented in the DTR context with several variations (Zhao et al.,2009; Chakraborty et al., 2010; Schulte et al., 2012; Song et al., 2014)
– Easy to understand and implement, and forms a good basis for more complexapproaches
The intuition comes from dynamic programming (Bellman, 1957) in case themultivariate distribution of the data is known
– Q-learning is an approximate dynamic programming approach
46 / 66
Analysis: Estimation of Optimal DTRs Q-learning
Dynamic Programming (DP): The Background
Two Stages
Move backward in time to take care of the delayed effects
Define the “Quality of treatment”, Q-functions:
Q2(h2, a2) = E[Y∣∣∣H2 = h2,A2 = a2
]Q1(h1, a1) = E
[max
a2Q2(H2, a2)︸ ︷︷ ︸
delayed effect
∣∣∣H1 = h1,A1 = a1
]
Optimal DTR:
dj(hj) = arg maxaj
Qj(hj, aj), j = 1, 2
What if the true Q-functions are unknown?
47 / 66
Analysis: Estimation of Optimal DTRs Q-learning
From DP to Q-learning
DP requires modeling the joint multivariate distribution (likelihood) oftime-varying covariates and outcome
This is hard!– The knowledge needed to model the joint distribution is often unavailable
– Model mis-specification can lead to incorrect conclusions
Turn to semi-parametric methods
In particular, Q-learning estimates the true Q-functions from data usingregression models
48 / 66
Analysis: Estimation of Optimal DTRs Q-learning
Q-learning: Typical Implementation (K = 2)
Linear regression models for Q-functions:
Qj(Hj,Aj;βj, ψj) = βTj Hj + (ψT
j Hj)Aj, j = 1, 2,
At stage 2, regress Y on (H2, H2A2) to obtain (β2, ψ2)
Construct stage-1 Pseudo-outcome:
Y1i = maxa2
Q2(H2i, a2; β2, ψ2), i = 1, . . . , n
At stage 1, regress Y1 on (H1, H1A1) to obtain (β1, ψ1)
Estimated Optimal DTR:
dj(hj) = arg maxaj
Qj(hj, aj; βj, ψj) = sign(ψTj hj)
49 / 66
Analysis: Estimation of Optimal DTRs Q-learning
Q-learning: Remarks
Q-learning is appealing because it is easy to implement in standard software, andeasy to explain to clinical collaborators who are familiar with regression
The approach has several limitations, e.g.:– Not robust to model mis-specification
– Only limited results are available for discrete outcomes
More sophisticated approaches exist, at least for some treatment/outcome types
50 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
Non-regular Inference
Inference for Optimal Regimen Parameters
dj(hj) = sign(ψTj hj)
“Regimen parameters” ψj – parameters that index the decision rules
– Reduce the number of variables on which data must be collected for futureimplementations of the DTR
– Know when there is insufficient evidence in the data to recommend one treatmentover another – choose treatment based on cost, familiarity, preference etc.
Inference for the optimal regimen parameters based on Q-learning has been atopic of active research for last 10+ years (Robins, 2004; Moodie and Richardson,2010; Chakraborty et al., 2010; 2013; Laber et al., 2014; Song et al., 2014)
52 / 66
Non-regular Inference
Non-regularity in Inference for ψ1 (K = 2)
Y1i = maxa2 Q2(H2i, a2; β2, ψ2) = βT2 H2i + |ψT
2 H2i|
Due to the non-differentiability of Y1i, the asymptotic distribution of ψ1 does notconverge uniformly over the parameter space – non-regular (Robins, 2004; Laberet al., 2014)
– It is problematic if p > 0, where p def= P[ψT
2 H2 = 0]
– The problem persists even when |ψT2 H2| is “small” with non-zero probability (“local
asymptotics”; Laber et al., 2011, 2014)
Practical consequence: Both Wald type CIs and standard bootstrap CIs performpoorly (Robins, 2004; Moodie and Richardson, 2010; Chakraborty et al., 2010)
In a K-stage setting, the same issues arise for all ψk, k = K − 1, . . . , 1
53 / 66
Non-regular Inference
m-out-of-n Bootstrap: A Feasible Solution
m-out-of-n bootstrap is a tool for remedying bootstrap inconsistency due tonon-smoothness (Shao, 1994; Bickel et al., 1997)
Efron’s nonparametric bootstrap with a smaller resample size, m = o(n)
Choice of m has always been difficult – resulting in a historical lack of popularityof the approach
We developed a choice of m for the regime parameters in the context ofQ-learning – adaptive to the degree of non-regularity present in the data2
2Chakraborty B, Laber EB, and Zhao Y (2013). Inference for optimal dynamic treatment regimes usingan adaptive m-out-of-n bootstrap scheme. Biometrics, 69: 714 - 723.
54 / 66
Non-regular Inference
Our Approach
Key idea: Since non-regularity arises when p > 0, an adaptive choice of mshould depend on an estimate of p
Consider a class of resample sizes: m = n1+α(1−p)
1+α , where α > 0 is a tuningparameter
Estimate p by “pre-test” of ψT2 H2 = 0 for fixed H2 over the training data set:
p =1n
n∑i=1
I
{n(ψT
2 H2,i)2
HT2,iΣ2H2,i
≤ χ21,1−ν
}
Plug in p for p in the above formula for m to get: m = n1+α(1−p)
1+α
55 / 66
Non-regular Inference
Implementation
α can be chosen in a data-driven way via double-bootstrapping (Davison andHinkley, 1997)
R package qLearn: http://cran.r-project.org/web/packages/qLearn/
Constructing one CI via double bootstrap takes about 3 minutes on a machinewith dual core 2.53 GHz processor and 4GB RAM
56 / 66
Non-regular Inference
Inference for ψ10: Simulation Design
A Simple Class of Generative Models
O1,A1,A2 ∈ {−1, 1} with probability 0.5
O2 ∈ {−1, 1} with P[O2 = 1|O1,A1] =exp(δ1O1 + δ2A1)
1 + exp(δ1O1 + δ2A1)
Y|· ∼ N(γ1 + γ2O1 + γ3A1 + γ4O1A1 + γ5A2 + γ6O2A2 + γ7A1A2, 1)
Analysis Model:
Q2 = β20 + β21O1 + β22A1 + β23O1A1 + (ψ20 + ψ21O2 + ψ22A1)︸ ︷︷ ︸ψT
2 S2
A2
Q1 = β10 + β11O1 + (ψ10 + ψ11O1)A1
The size of the stage-2 treatment effect ψT2 S2 determines the extent of nonregularity, e.g.
p = P[ψT2 S2 = 0]
57 / 66
Non-regular Inference
Inference for ψ10: Simulation Design
Example Generative Models3
Example γT δT Type p1 (0, 0, 0, 0, 0, 0, 0) (0.5, 0.5) NR 12 (0, 0, 0, 0, 0.01, 0, 0) (0.5, 0.5) NNR 03 (0, 0,−0.5, 0, 0.5, 0, 0.5) (0.5, 0.5) NR 0.54 (0, 0,−0.5, 0, 0.5, 0, 0.49) (0.5, 0.5) NNR 05 (0, 0,−0.5, 0, 1.0, 0.5, 0.5) (1.0, 0.0) NR 0.256 (0, 0,−0.5, 0, 0.25, 0.5, 0.5) (0.1, 0.1) R 07 (0, 0,−0.25, 0, 0.75, 0.5, 0.5) (0.1, 0.1) R 08 (0, 0, 0, 0, 0.25, 0, 0.25) (0, 0) NR 0.59 (0, 0, 0, 0, 0.25, 0, 0.24) (0, 0) NNR 0
3Ex. 1 – 6 taken from Chakraborty et al. (2010), and Ex. 7 – 9 taken from Laber et al. (2014)58 / 66
Non-regular Inference
Inference for ψ10: Simulation Design
Focus on the 95% nominal CI for the stage-1 treatment effect parameter ψ10
Compare Monte Carlo estimates of coverage and mean width of– n-out-of-n bootstrap (usual)– m-out-of-n bootstrap
1000 simulated data sets, each of size n = 300
1000 bootstrap replications to construct CIs
59 / 66
Non-regular Inference
Coverage and Mean Width of the 95% nominal CI for ψ10
Table : Coverage Rates (color-coded as under-coverage, nominal coverage)
Ex. 1NR
Ex. 2NNR
Ex. 3NR
Ex. 4NNR
Ex. 5NR
Ex. 6R
Ex. 7R
Ex. 8NR
Ex. 9NNR
n-out-of-n 0.936 0.932 0.928 0.921 0.933 0.931 0.944 0.925 0.922m-out-of-n 0.964 0.964 0.953 0.950 0.939 0.947 0.944 0.955 0.960
Table : Mean Width of CIs
Ex. 1NR
Ex. 2NNR
Ex. 3NR
Ex. 4NNR
Ex. 5NR
Ex. 6R
Ex. 7R
Ex. 8NR
Ex. 9NNR
n-out-of-n 0.269 0.269 0.300 0.300 0.320 0.309 0.314 0.299 0.299m-out-of-n 0.331 0.331 0.321 0.323 0.330 0.336 0.322 0.328 0.328
60 / 66
Non-regular Inference
m-out-of-n Bootstrap: Remarks
At least in case of SMARTs, non-regular settings (i.e., small treatment effects)are more likely to occur than one would think – due to clinical equipoise(Freedman, 1987)
– Hence any method of inference in the DTR context should deal with non-regularityseriously
m-out-of-n bootstrap offers a feasible solution to the inference problem– The procedure is consistent, and successfully adapts to the degree of non-regularity
present in the data
– It is conceptually simple, likely to be palatable to practitioners
– It has already been implemented in the R package qLearn, to facilitate widedissemination
Applying the m-out-of-n bootstrap procedure to settings with more stages andmore treatment choices per stage is conceptually straightforward, but can beoperationally messy
61 / 66
Outline
1 Personalized Medicine and Dynamic Treatment Regimes
2 SMART DesignPrimary Analysis and Sample Size
3 Analysis: Estimation of Optimal DTRsQ-learning
4 Non-regular Inference
5 Discussion
Discussion
Take-home Messages
SMART is a cutting-edge clinical trial design that formalizes the sequentialdecision making in clinical practice
– Very relevant for chronic conditions
– Does not necessarily require a lot of sample size (depends on the primary question)
A variety of methods exist for (secondary) analysis of SMART data– Q-learning is probably the simplest, and already implemented in the R package
qLearn
– More non-parametric methods (e.g. outcome-weighted learning) have beendeveloped more recently
Inference is tricky, but methods exist to handle the associated non-regularity
63 / 66
Shameless Self-Promotion!!
Shoot your questions, comments, criticisms, request for slides to:bibhas.chakraborty@duke-nus.edu.sg
Discussion
Why move through stages? Why not run an “all-at-once”multivariable regression?
Berkson’s Paradox or Collider-stratification Bias: There may be non-causal association(s) evenwith randomized data, leading to biased stage-1 effects (Berkson, 1946; Greenland, 2003;
Murphy, 2005)66 / 66
Recommended