Upload
carlos-hahn
View
214
Download
0
Tags:
Embed Size (px)
Citation preview
Subgroup Analyses: Can Subgroup Analyses: Can We ‘Smooth' out the Rough We ‘Smooth' out the Rough
Edges?Edges?
Daniel Sargent, PhDDaniel Sargent, PhD
Mayo ClinicMayo Clinic
Sept 28, 2006Sept 28, 2006
OutlineOutline
MotivationMotivation Subgroups ARE medicine (especially its Subgroups ARE medicine (especially its
future)future) ExamplesExamples
Good and bad conductGood and bad conduct StrategiesStrategies
Hierarchical modelsHierarchical models Smoothing approachesSmoothing approaches
ConclusionConclusion
Subgroups analysis: My Subgroups analysis: My Definition & My BiasDefinition & My Bias
Definition: An effort to draw inference on Definition: An effort to draw inference on an effect of an intervention in a set of an effect of an intervention in a set of patients smaller than the entire patients smaller than the entire experimental cohortexperimental cohort
Bias: Such inferences will be more robust Bias: Such inferences will be more robust when based on a model using all patients when based on a model using all patients than an analysis restricted to just the than an analysis restricted to just the cohort of interestcohort of interest
Subgroups are medicineSubgroups are medicine
If all patients were the same, wouldn’t need If all patients were the same, wouldn’t need physiciansphysicians
Human Genome Project massively Human Genome Project massively expanding knowledge baseexpanding knowledge base
Technology, biology, chemistry, etc. Technology, biology, chemistry, etc. allowing manufacture of highly specific, allowing manufacture of highly specific, targeted compoundstargeted compounds
Patients seek ‘tailored’ treatment Patients seek ‘tailored’ treatment recommendationsrecommendations
Example: Colon Cancer: Model-Example: Colon Cancer: Model-Derived Estimates of 5 year DFS (%) Derived Estimates of 5 year DFS (%) with with SurgerySurgery plus plus Adjuvant TherapyAdjuvant Therapy
Nodal Status
T stage Low Grade High Grade
S S 0 nodes T3 73 65 T4 60 51 T1-T2 62 53 1-4 nodes T3 49 38 T4 33 23 T1-T2 39 28 > 5 nodes T3 24 15 T4 11 5
+AT +AT 77 70
66 57 75 68 65 56 52 40 57 46 43 32 27 17
+AT +AT 77 70
66 57 75 68 65 56 52 40 57 46 43 32 27 17
Gill, JCO 2004; http://www.mayoclinic.com/calcs
Example: Breast CancerExample: Breast Cancer
Most common cancer in women in the US The HER-2 gene is overexpressed in 25-30% of
breast cancers; associated with worse prognosis.
Trastuzumab, a humanized monoclonal antibody targets the HER-2 receptor; previous trials have demonstrated activity in the treatment of HER-2 overexpressing late stage breast cancer.
Performed a clinical trial testing trastuzumab in subset of HER-2 positive women with early stage breast cancer
0 1 2 3 4 5
50
60
70
80
90
10
0
0 1 2 3 4 5
50
60
70
80
90
10
0
Disease-Free Survival Survival
Years Years
AC→T+H →H134 events
AC→T+H →H62 events
AC→T261 events
HR=0.48, 2P=3x10-12
HR=0.67,2P=0.015
94%91%
92%
87%
87%
75%
85%
67%
AC→T92 events
Romond et al, NEJM 2005
Avoiding subgroup analysis: Avoiding subgroup analysis: Targeted Phase II/III TrialsTargeted Phase II/III Trials
Patient Selection for targeted therapiesPatient Selection for targeted therapies Test the recommended dose on patients Test the recommended dose on patients
who are most likely to respond based on who are most likely to respond based on their molecular expression levelstheir molecular expression levels
May result in a large savings of patients May result in a large savings of patients (Simon & Maitournam, Clinical Cancer Research 2004)(Simon & Maitournam, Clinical Cancer Research 2004)
Trials in targeted populationsTrials in targeted populations
Gains in Gains in efficiency efficiency depend on depend on marker marker prevalence and prevalence and relative efficacy relative efficacy in marker + and in marker + and marker – patientsmarker – patients
Details: Session Details: Session #13 tomorrow#13 tomorrow
PrevalencePrevalence Relative Relative EfficacyEfficacy
Efficiency Efficiency GainGain
25%25% 0%0% 16x16x
25%25% 50%50% 2.5x2.5x
50%50% 0%0% 4x4x
50%50% 50%50% 1.8x1.8x
75%75% 0%0% 1.8x1.8x
75%75% 50%50% 1.3x1.3x(Simon & Maitournam, CCR 2004)
Case Study: Stage II colon cancerCase Study: Stage II colon cancer
Colon cancer: Prognosis defined by stageColon cancer: Prognosis defined by stage Prior trials generally enrolled patients with both Prior trials generally enrolled patients with both
stage II and III diseasestage II and III disease Previous randomized trials uniformly Previous randomized trials uniformly
demonstrate benefit of chemotherapy in demonstrate benefit of chemotherapy in stage III patients (node positive)stage III patients (node positive)
Previous trials & pooled analyses mixed Previous trials & pooled analyses mixed regarding benefit in stage II patientsregarding benefit in stage II patients
No single trial powered for modest effect No single trial powered for modest effect seen in stage II ( ↑ 2-3% in 5 year survival)seen in stage II ( ↑ 2-3% in 5 year survival)
Meta-analysis Stage II Adjuvant TherapyMeta-analysis Stage II Adjuvant Therapy
Benson et al. J Clin Oncol. 2004
N=2,732RR=0.88P=0.08
American Society of Clinical American Society of Clinical Oncology Guidelines 2004Oncology Guidelines 2004
Direct evidence from randomized trials does not Direct evidence from randomized trials does not support routine use of chemotherapy for patients support routine use of chemotherapy for patients with stage II colon cancer. with stage II colon cancer.
Those who accept the relative benefit in stage III Those who accept the relative benefit in stage III disease as adequate indirect evidence of benefit disease as adequate indirect evidence of benefit for stage II disease are justified in considering for stage II disease are justified in considering chemotherapy, particularly for patients with high-chemotherapy, particularly for patients with high-risk stage II disease. risk stage II disease.
Ultimate clinical decision should be based on Ultimate clinical decision should be based on discussions with the patientdiscussions with the patient..
Benson et al. J Clin Oncol. 2004
Primary end-point: disease-free survival (DFS)Primary end-point: disease-free survival (DFS)
R
LV5FU2
FOLFOX4: LV5FU2 + oxaliplatin 85 mg/m²
N=2246
Stage II: 40%
Stage III: 60%
New therapy: FOLFOXNew therapy: FOLFOX
de Gramont et al., ASCO 2005
6.6%
Disease-free Survival (ITT)Disease-free Survival (ITT)1.0
0.9
0.8
0.7
0.6
0.5
0.3
0.4
0.2
0.1
0.0 0 666 12 18 24 30 36 42 48 54 60
Months
Events
FOLFOX4 279/1123 (24.8%)
LV5FU2 345/1123 (30.7%)
HR [95% CI]: 0.77 [0.65 – 0.90]
DF
S p
rob
abil
ity
p<0.001
de Gramont et al., ASCO 2005
Disease-free Survival (ITT) Disease-free Survival (ITT) Stage II and Stage III PatientsStage II and Stage III Patients
1.0
0.9
0.8
0.7
0.6
0.5
0.3
0.4
0.2
0.1
0.0 0
FOLFOX4 – 451 Stage IILV5FU2 – 448 Stage IIFOLFOX4 – 672 Stage IIILV5FU2 – 675 Stage III
HR [95% CI]:0.82 [0.60 – 1.13] Stage II0.75 [0.62 – 0.89] Stage III
Months
DF
S p
rob
abil
ity
666 12 18 24 30 36 42 48 54 60Data cut-off: January 16, 2005
8.6%
3.5%
de Gramont et al., ASCO 2005
DFS (months)
DFS in high-risk* stage II patientsDFS in high-risk* stage II patientsDFS in high-risk* stage II patientsDFS in high-risk* stage II patients1.01.0
0.90.9
0.80.8
0.70.7
0.60.6
Pro
bab
ility
*T4 and/or bowel obstruction and/or tumor perforation and/or poorly differentiated tumor and/or venous invasion and/or <10 examined LNsData cut-off: January 16, 2005
0 6 12 18 24 30 36 42 48
5.4%
HR 0.76FOLFOX4 – 286 HRStage IILV5FU2 – 290 HR Stage II
de Gramont et al., ASCO 2005
FDA ActionFDA Action
Approval of FOLFOX therapy only in stage Approval of FOLFOX therapy only in stage III patients, even though trial designed for III patients, even though trial designed for stage II and III patientsstage II and III patients
Possible rationalePossible rationale Standard chemotherapy vs control not shown Standard chemotherapy vs control not shown
beneficial in stage II patientsbeneficial in stage II patients This trial not significant for experimental vs This trial not significant for experimental vs
standard chemotherapystandard chemotherapy
Stage II trial: QUASARStage II trial: QUASAR
Chemotherapy(n = 1622)*
Observation(n = 1617)
No clear indicationfor chemotherapy
(n = 3239)
RANDOMIZE
Colon or rectal cancer
• Stage I-III• Complete resection
with no evidence of residual disease
Gray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp. Accessed November 2004.
% o
f P
atie
nts
QUASAR: Overall SurvivalQUASAR: Overall Survival
P = .025-year OS, Observation = 77.4% vs Chemotherapy = 80.3%Relative risk = 0.83 (95% CI, 0.71-0.97)
YearsGray et al. ASCO 2004. Abstract 3501. At: http://www.asco.org/ac/1,1003,_12-002511-00_18-0026-00_19-0010698,00.asp.Accessed November 2004.
0 1 2 3 4 5 6 7 8 9 100
20
40
60
80
100Observation (n=1622)
Chemotherapy (n=1617)
Implication: Stage II patientsImplication: Stage II patients
Compared to control, 5-FU provides 2-3% ↑ in OS, Compared to control, 5-FU provides 2-3% ↑ in OS, statistically significant in a single trialstatistically significant in a single trial Debate over clinical relevanceDebate over clinical relevance
In a large trial, FOLFOX provides 3-4% ↑ in DFS In a large trial, FOLFOX provides 3-4% ↑ in DFS compared to 5-FU, not statistically significant for compared to 5-FU, not statistically significant for stage II alonestage II alone No hint of interaction between rx and stage, p = 0.77No hint of interaction between rx and stage, p = 0.77 On its own, debatable benefit compared to 5-FUOn its own, debatable benefit compared to 5-FU
Cross trial comparison: FOLFOX may result in 5-7% Cross trial comparison: FOLFOX may result in 5-7% improvement vs control, but not approvedimprovement vs control, but not approved No debate about clinical relevance No debate about clinical relevance
Grothey & Sargent, JCO 2005
Stage II Colon Cancer: Stage II Colon Cancer: Lessons LearnedLessons Learned
Decisions based on subgroups may seem Decisions based on subgroups may seem rational at the time, but lead to unintended rational at the time, but lead to unintended consequencesconsequences Results may make further trials impossible Results may make further trials impossible
(FOLFOX vs control) (FOLFOX vs control) Need better approaches to analyze Need better approaches to analyze
subgroups using modeling (or meta-subgroups using modeling (or meta-analyses), not individual trial resultsanalyses), not individual trial results
Potential solution for prospectively Potential solution for prospectively defined subgroups: Hierarchical defined subgroups: Hierarchical
modelsmodels Goal: Test a treatment in a number of Goal: Test a treatment in a number of
populationspopulations Hypothesis: Effect may depend vary Hypothesis: Effect may depend vary
between populationsbetween populations Example: Targeted cancer therapyExample: Targeted cancer therapy
Mechanism of action based therapyMechanism of action based therapy Multiple tumor types express ‘target’, to Multiple tumor types express ‘target’, to
varying degreesvarying degrees
Basic statistical formulationBasic statistical formulation
Suppose N subgroups, with mean Suppose N subgroups, with mean response response ii, i=1,...N, i=1,...N
Assume Assume i i ~ N(,2)
If Bayesian, put a prior on 2 Depending on estimate of 2, allows
heterogeneity between subgroups Easily extends to non-normal models
Hierarchical Model: ExampleHierarchical Model: Example
Phase II clinical trial of a new agent Phase II clinical trial of a new agent specifically targeted at patients with a specifically targeted at patients with a methylated MGMT promotermethylated MGMT promoter
Prevalence from 10% to 60% across Prevalence from 10% to 60% across various cancer typesvarious cancer types High prevalence seen in Head and Neck, High prevalence seen in Head and Neck,
Esophageal, Colorectal, and Non Small-Cell Esophageal, Colorectal, and Non Small-Cell Lung CancerLung Cancer
Goal: Determine if overall efficacy > 10%, Goal: Determine if overall efficacy > 10%, but efficacy may depend on tumor typebut efficacy may depend on tumor type
Logistic regression ExampleLogistic regression Example
Hierarchical logistic model for tumor responseHierarchical logistic model for tumor response Stopping rules for each tumor siteStopping rules for each tumor site
P ( Response rateP ( Response rateii > 10%) < 10% > 10%) < 10% OROR
P (Response rateP (Response rateii > 10%) < 25% & > 10%) < 25% &
P (Response rateP (Response rateOverallOverall > 10%) < 10% > 10%) < 10%
Simulation for operating characteristicsSimulation for operating characteristics BenefitsBenefits
Single trial (opposed to 4)Single trial (opposed to 4) Use all data formally but flexibly Use all data formally but flexibly
Survival ExampleSurvival Example
Survival following chemotherapy for colon Survival following chemotherapy for colon cancercancer
Pooled analysis of 5 trials, suggestion of a Pooled analysis of 5 trials, suggestion of a study-specific treatment effect (a different type of study-specific treatment effect (a different type of subgroup)subgroup)
Fit a random effect Cox modelFit a random effect Cox model (t; x) = (t; x) = 0i0i(t) exp (x(t) exp (xii)) i i ~ N(,2) Can either model 0 parametrically, or use
Cox model
Model ResultsModel Results
StudyStudy Single fixed Single fixed Treatment Treatment EffectEffect
Study Specific Study Specific Fixed Treatment Fixed Treatment EffectEffect
Study Specific Study Specific Random Random Treatment EffectTreatment Effect
OverallOverall -0.22 (0.14)-0.22 (0.14) -0.21 (0.20)-0.21 (0.20)
11 -0.25 (0.34)-0.25 (0.34) -0.21 (0.20)-0.21 (0.20)
22 0.24 (0.35)0.24 (0.35) -0.11 (0.22)-0.11 (0.22)
33 -0.28 (0.33)-0.28 (0.33) -0.22 (0.20)-0.22 (0.20)
44 -0.25 (0.20)-0.25 (0.20) -0.22 (0.17)-0.22 (0.17)
55 -1.10 (0.68)-1.10 (0.68) -0.29 (0.28)-0.29 (0.28)
Prior mean for precision (1/2) = 50, posterior mean 106, Little evidence of heterogeneity
Sargent et al, 2000
Another approach: Modeling Another approach: Modeling Interactions using ShrinkageInteractions using Shrinkage
Subgroup analyses are fundamentally looking at Subgroup analyses are fundamentally looking at interactionsinteractions
In multi-factor experiment, the number of In multi-factor experiment, the number of interactions can explodeinteractions can explode
Well known that shrinkage (or model averaging) Well known that shrinkage (or model averaging) provides much better performance than all or provides much better performance than all or nothing approach (stepwise)nothing approach (stepwise)
Idea: Include interactions in model, but shrink Idea: Include interactions in model, but shrink them away if they are not strongly supported by them away if they are not strongly supported by the datathe data
Another approach: Modeling Another approach: Modeling Interactions using shrinkageInteractions using shrinkage
Dental ExperimentDental Experiment Dentures are often made with a soft liner between the gums and Dentures are often made with a soft liner between the gums and
the hard denture basethe hard denture base Polishing the liner can cause a gap between the liner and the Polishing the liner can cause a gap between the liner and the
basebase Such gaps harbor pathogens like Such gaps harbor pathogens like CandidaCandida
The experimentThe experiment Main interest: new vs. standard soft liner materialMain interest: new vs. standard soft liner material Factor M:Factor M: 2 materials2 materials Factor P:Factor P: 4 polishing methods4 polishing methods Factor F:Factor F: 8 finishing methods8 finishing methods
Fully crossed design, no replicationFully crossed design, no replication Outcome measure: gap btwn liner & base, in logOutcome measure: gap btwn liner & base, in log1010 mm mm
Pesun, Hodges & Lai (2002) Pesun, Hodges & Lai (2002) J. Prosthetic DentistryJ. Prosthetic Dentistry
Smoothing interactions: Smoothing interactions: Smoothed ANOVASmoothed ANOVA
Fit full ANOVA model (include all interactions) Fit full ANOVA model (include all interactions) yy = = XX + + yy is 64 x 1, contains log is 64 x 1, contains log1010 gap gap ee is 64 x 1, normal mean 0, precision is 64 x 1, normal mean 0, precision 00II6464
XX is 64 x 64 is 64 x 64 is 64 x 1; we will smooth/shrink its elementsis 64 x 1; we will smooth/shrink its elements
12 main effects, 52 interactions12 main effects, 52 interactions Model interactionsModel interactions
• kk ~ N (0,1/ ~ N (0,1/ kk) , k=13, …, 64) , k=13, …, 64• Large Large k k implies implies kk shrunk toward 0 shrunk toward 0
Smoothed ANOVA: The Smoothed ANOVA: The model/prior for the model/prior for the kk
How to model the interactionsHow to model the interactions Each interaction smoothed by its own Each interaction smoothed by its own kk
Each effect's Each effect's kk are all the same, are all the same, effecteffect All two-way interactions are smoothed by a All two-way interactions are smoothed by a
single single Mix the above optionsMix the above options
Use priors on Use priors on kk to specify desired to specify desired
operating characteristics for interactionsoperating characteristics for interactions
Use Degrees of Freedom to set Use Degrees of Freedom to set priors for the priors for the kk
Hodges & Sargent (2001 Hodges & Sargent (2001 BiometrikaBiometrika) extended ) extended methods for computing DF in standard ANOVA methods for computing DF in standard ANOVA to linear hierarchical modelsto linear hierarchical models
Hodges et al (Hodges et al (TechnometricsTechnometrics, 2006) present , 2006) present methodology to use DF to set priorsmethodology to use DF to set priors Example: I want the 51 2-way interactions to share 5 Example: I want the 51 2-way interactions to share 5
degrees of freedomdegrees of freedom
See references for technical detailsSee references for technical details Ongoing work: extending to non-linear (Cox) Ongoing work: extending to non-linear (Cox)
modelsmodels
Summary: Smoothed ANOVASummary: Smoothed ANOVA
Subgroup analyses are fundamentally Subgroup analyses are fundamentally looking at interactionslooking at interactions
A priori have low probability of a significant A priori have low probability of a significant interaction, but don’t want to exclude the interaction, but don’t want to exclude the possibilitypossibility
Idea: Include interactions in model, but Idea: Include interactions in model, but shrink themshrink them
SummarySummary
Subgroup analysis is essential to clinical Subgroup analysis is essential to clinical researchresearch
People usually perform such analyses with People usually perform such analyses with best of intentionsbest of intentions
Up-front thought can allow us to Up-front thought can allow us to Carefully define population under studyCarefully define population under study Pre-specify sub-populations to be examinedPre-specify sub-populations to be examined
Hierarchical/Shrinkage models offer Hierarchical/Shrinkage models offer attractive possibilities for addressing attractive possibilities for addressing subgroups, if defined prospectivelysubgroups, if defined prospectively
Thank YouThank You
AcknowledgementsAcknowledgements Smoothed ANOVA: Jim HodgesSmoothed ANOVA: Jim Hodges Colon Cancer: Axel Grothey, Aimery Colon Cancer: Axel Grothey, Aimery
deGramont, Sharlene GilldeGramont, Sharlene Gill