Characteristics Associated WithDecreased or Increased MortalityRisk From Glycemic TherapyAmong Patients With Type 2Diabetes and High CardiovascularRisk:Machine LearningAnalysis ofthe ACCORD TrialDiabetes Care 2018;41:604–612 | https://doi.org/10.2337/dc17-2252

OBJECTIVE

Identifying patients whomay experience decreased or increased mortality risk fromintensive glycemic therapy for type 2 diabetes remains an important clinical chal-lenge. We sought to identify characteristics of patients at high cardiovascular riskwith decreased or increasedmortality risk from glycemic therapy for type 2 diabetesusing newmethods to identify complex combinations of treatment effect modifiers.

RESEARCH DESIGN AND METHODS

The machine learning method of gradient forest analysis was applied to understandthe variation in all-causemortality within theAction to Control Cardiovascular Risk inDiabetes (ACCORD) trial (N = 10,251), whose participants were 40–79 years old withtype 2 diabetes, hemoglobin A1c (HbA1c) ‡7.5% (58 mmol/mol), cardiovascular dis-ease (CVD) or multiple CVD risk factors, and randomized to target HbA1c <6.0%(42 mmol/mol; intensive) or 7.0–7.9% (53–63 mmol/mol; standard). Covariates in-cluded demographics, BMI, hemoglobin glycosylation index (HGI; observed minusexpected HbA1c derived from prerandomization fasting plasma glucose), other bio-markers, history, and medications.

RESULTS

The analysis identified four groups defined by age, BMI, and HGI with varied risk formortality under intensive glycemic therapy. The lowest risk group (HGI <0.44,BMI <30 kg/m2, age <61 years) had an absolute mortality risk decrease of 2.3%attributable to intensive therapy (95% CI 0.2 to 4.5, P = 0.038; number needed totreat: 43), whereas the highest risk group (HGI ‡0.44) had an absolute mortality riskincrease of 3.7% attributable to intensive therapy (95% CI 1.5 to 6.0; P < 0.001;number needed to harm: 27).

CONCLUSIONS

Age, BMI, and HGI may help individualize prediction of the benefit and harm fromintensive glycemic therapy.

1Center for Primary Care and Outcomes Research,Center for Population Health Sciences, Depart-ments of Medicine and Health Research and Pol-icy, Stanford University, Palo Alto, CA2Harvard Medical School, Boston, MA3Department of Veterans Affairs Eastern Colo-rado Healthcare System, Denver, CO4Division of General Internal Medicine, Depart-ment of Medicine, University of Colorado Schoolof Medicine, Denver, CO5Diabetes Unit, Massachusetts General Hospital,Boston, MA6Division of General Internal Medicine, Massa-chusetts General Hospital, Boston, MA

Corresponding author: Sanjay Basu, basus@stanford.edu.

Received 26 October 2017 and accepted 5December 2017.

This article contains Supplementary Data onlineat http://care.diabetesjournals.org/lookup/suppl/doi:10.2337/dc17-2252/-/DC1.

© 2017 by the American Diabetes Association.Readers may use this article as long as the workis properly cited, the use is educational and notfor profit, and the work is not altered. More infor-mation is available at http://www.diabetesjournals.org/content/license.

Sanjay Basu,1,2 Sridharan Raghavan,3,4

Deborah J. Wexler,2,5 and

Seth A. Berkowitz2,5,6

604 Diabetes Care Volume 41, March 2018

CARDIOVASC

ULA

RANDMETABOLICRISK

Individualizing the glycemic target forpatients with type 2 diabetes is now theguideline-recommended strategy (1), buthow best to individualize glycemic targetsremains unclear. A major reason for cau-tion regarding intensive glycemic targetsis the Action to Control CardiovascularRisk in Diabetes (ACCORD) trial (N =10,251, conducted 2001–2009) (2), whichwas halted due to increased all-causemortality in the intensive therapy arm.ACCORD targeted nearly normal glycemiclevels in the intensive glycemic therapyarm, achieving a median hemoglobin A1c(HbA1c) of 6.4% (46 mmol/mol), com-pared with an achieved HbA1c of 7.5%(58 mmol/mol) in the standard therapyarm.Meta-analyses of data fromACCORDand other trials find that microvascularevents are reduced with intensive glyce-mic control (3), but the lack of overallmortality benefit in trials as well as theincreased mortality observed in ACCORDrenders uncertain the risk-to-benefit cal-culation in any given patient.Although current guidelines donot rec-

ommend targets as low as those used inACCORD, real-world evidence suggestsmany patients are treated with multidrugregimens to levels achieved in the inten-sive therapy armof ACCORD (4–6). There-fore, understanding the heterogeneoustreatment effects (HTEs) of intensive gly-cemic therapy with regard to mortality isimportant. Specifically, which subgroupof patients in ACCORD was most likelyto experience increased mortality? Con-versely, because some patients do derivecardiovascular benefit from glycemictherapy (7), did any subgroups in ACCORDexperience benefit? Unfortunately, uni-variable subgroup analyses of the trialdata have been unable to explain the ma-jor variations inexcessmortality inACCORDfrom intensive therapy (8,9), despite ex-amining factors including hypoglyce-mia and hypoglycemia unawareness(which was actually less common amongthose who died in the intensive therapyarm) (10,11), age (12), cardiac autonomicdysfunction (13), weight gain (9), and rateof HbA1c reduction (14). Although sev-eral factors in combination are thoughtto account for mortality HTEs, univariablesubgroup analyses are not capable ofidentifying them and are subject tofalse-positive findings due to multipletesting (15,16).Recently, the advancement ofmachine

learning methodsdparticularly the

approach of gradient forest analysis(17)dhas aided the search for HTEs (Fig.1). Gradient forest analysis can partitiona trial population into subgroups char-acterized by multiple simultaneous char-acteristics (multivariable rather thanunivariable analysis), using cross-validationto reduce the likelihood of false-positiveresults (17). The gradient forest approachalso inherently accounts for interactionsamong multiple variables (e.g., be-tween age and HbA1c) and is unbiased inpredicting the difference in treatment

effect between study arms, unlike oldermachine learning methods that can bebiased and focus on the absolute rate ofevents (e.g., risk of mortality) rather thanHTEs (e.g., how individual features affectthe treatment’s ability to reduce the riskof mortality) (17).

The objective of this studywas to applygradient forest analysis to identify sub-groups of ACCORD participants with de-creased or increased risk of all-causemortality attributable to intensive

therapy.

Figure 1—Conceptualization of gradient forest analysis to detect HTEs from trial data. Our imple-mentation of gradient forest analysis involved repeated randomsampling fromboth armsof the trialdata set to compute the treatment effectdthe difference in the probability of the primary outcomebetween the intensive and standard glycemic therapy armsdamong subgroups of trial participants.After selecting subsamples of the trial data, our approach selected combinations of explanatoryvariables (X1, X2) from one subsection of data to divide the study population subsets withlower vs. higher treatment effectsdin this case, all-cause mortalitydwhen comparing intensivevs. standard therapy. We then used another subsection of data to update the preliminary values ofthe explanatory variables used to subdivide the population into final values that maximized be-tween-group differences and minimized within-group differences in treatment effects among eachsubgroup. By using multiple subsections of data for the estimation of subgroups, the methodproduces unbiased estimates of HTE that are robust to outliers (17). The overall process is thenrepeated thousands of times to identify which variables and cut point values define consistentsubgroups across thousands of random samplings from the trial data. The final subgroups chosen atthe end of the decision tree are referred to as “leaves” of the tree.

care.diabetesjournals.org Basu and Associates 605

RESEARCH DESIGN AND METHODS

Source of DataACCORD was a randomized, controlledtrial of intensive versus standard glycemiccontrol (open-label target of HbA1c

,6.0% [42 mmol/mol] vs. 7.0–7.9% [53–63 mmol/mol], respectively), with a mul-tifactorial design in which participantswere additionally randomized to inten-sive versus standard lipid treatment(double-blinded assignment to fibrateplus statin or placebo plus statin, respec-tively), or intensive versus standard bloodpressure treatment (open-label targetof systolic blood pressure ,120 mmHgor,140 mmHg, respectively) (2). The tri-al was conducted at 77 clinical sites inNorth America between January 2001and June 2009. Participants in both armsreceived glucose-lowering medications.The glycemic control component of thetrial was terminated early due to highermortality in the intensive therapy arm,with a median on-protocol follow-uptime of 3.7 years and a median on- plusoff-protocol follow-up time of 4.9 years.The full duration of available data wereused in this project. This analysis wasapproved by the Stanford University In-stitutional Review Board (e-Protocol#39321).

ParticipantsParticipants (Supplementary Table 1)were 40–79 years oldwith type 2 diabetes,HbA1c $7.5% (58 mmol/mol), and priorevidence of cardiovascular disease (CVD)or risk factors for CVD (e.g., dyslipidemia,hypertension, smoking, or obesity; thosewithout a prior cardiovascular event werebetween the ages of 55 and 79) (2,18,19).Exclusion criteria for ACCORD includedBMI .45 kg/m2, serum creatinine .1.5mg/dL, or serious illnesses that mightlimit trial participation or life expectancy.Data from all study arms were included,with variables identifying glycemic, bloodpressure, and lipid study arm to controlfor randomized therapy selection (15).

OutcomeThe primary outcome for the currentstudywas the difference in all-causemor-tality between therapy arms, assessedfrom the point of enrollment to thetime of study termination in June 2009.Mortality assessment in ACCORD wasmasked to therapy arm. The secondaryoutcomewas the difference in compos-ite microvascular events (including

nephropathy, retinopathy, and neuropa-thy) between study arms, defined inACCORD as renal failure, end-stage renaldisease (dialysis), serum creatinine .3.3mg/dL, photocoagulation or vitrectomy, orMichigan Neuropathy Screening Instru-mentscore.2.0.Aswithmortality,assess-ment of microvascular events in ACCORDwas masked to therapy arm. The second-ary outcome was chosen to determinewhether subgroups of participants identi-fiedbased onHTEs formortality exhibitedsimilar HTEs in diabetes-relatedmicrovas-cular complications because the stron-gest support for intensive therapy hascome from studies of reduced microvas-cular events. To help find groups withhigh mortality risk and low microvascularbenefit, and vice versa, the decision treebased on the primary outcome wastested on the secondary outcome to de-termine whether the same features thatpredicted a higher or lower effect of in-tensive treatment on mortality wouldalso predict a higher or lower effect ofintensive treatment on microvascularevents.

PredictorsPotential predictor variables for HTEs(itemized in Supplementary Table 1) in-cluded the subset of characteristics pre-viously hypothesized to be related tocardiovascular or all-cause mortalityamong persons with type 2 diabetes: de-mographics (age, sex, race/ethnicity),study arm, type and number of glucose-lowering medications (including insulinuse and oral glucose-loweringmedicationby class, individually, and in combination),diabetes history (years since diabetesdiagnosis, hypoglycemia in prior 7 days),prior ulcer or amputation, history of eyedisease or surgery, loss of vibratory sen-sation or monofilament sensation, bio-markers (HbA1c, fasting blood glucose,hemoglobin glycosylation index [HGI][20] [defined as observed 2 predictedHbA1c [%], where predicted HbA1c =0.009 3 fasting plasma glucose [mg/dL] +6.8, using the single baseline fastingplasma glucose], lipid profile, serum cre-atinine, estimated glomerular filtrationrate by the Modification of Diet in RenalDisease (MDRD) Study equation, serumpotassium, urine microalbumin, urine cre-atinine,alanineaminotransferase, creatininephosphokinase, systolic and diastolic bloodpressure, heart rate, and BMI), and CVD co-variates (tobacco smoking; atrial fibrillation

or other arrhythmia by electrocardiogram;left ventricular hypertrophy by electrocar-diogram; prior myocardial infarction, stroke,angina, bypass surgery, percutaneous coro-nary intervention, or other vascular proce-dure; and blood pressure medications,cholesterol medications, and anticoagu-lant/antiplatelet medications). HGI was in-cluded among the covariates because itwas previously suggested as a potentiallyuseful indicator of diabetes severity as wellas a predictor of HTEs in mortality amongpersonswith type2diabetes (20–22). Treat-ment arm (intensive vs. standard) is inher-ently part of the gradient forest analysisbecause the outcome is defined as differ-ence in mortality between the two arms.

All predictor variables were taken fromthe baseline (prerandomization) studyvisit because our goal was to identifyfactors clinicians could use before thedecision to set more, or less, intensiveglycemic targets. Therefore, time-varyingcovariates were not incorporated into theanalysis.

Sample SizeA total of 10,251participantswere includedfrom the ACCORD trial, which includes thecomplete sample of participants enrolled.

Missing DataMissing data were not imputed be-cause,1% of data for any predictor vari-able were missing from the trial data set.

Statistical Analysis MethodTo ensure transparency and reproducibil-ity of the analysis, statistical code is linkedat https://sdr.stanford.edu concurrentwith publication. Our implementation ofgradient forest analysis proceeded in foursteps (Fig. 1). First, ACCORD trial datawere divided in half randomly, with anequal number of intensive and standardglycemic control arm participants in eachof the twodata subsets. Second, variableswere chosen by randomly sampling sub-sets of potential predictors to construct adecision tree made of those predictorsthat could split the first of the two sub-samples of data into subgroups withhigher and lower treatment effect (seeFig. 1). Treatment effect was defined asthe absolute difference in the all-causemortality rate between the intensiveand standard therapy arms. Subgroupswere required to be .5% of the overallstudy sample; we tested the consistencyof the approach to ensure the same result

606 Mortality Risk From Glycemic Therapy Diabetes Care Volume 41, March 2018

if we used limits of .1% to .8%. Third,once the initial decision tree was con-structed from the first subsample ofdata, the values of each predictor thatwould define branches in the decisiontree were refined using the second sub-sample of data so that the final subgroupsat the bottom of the tree (“leaves” of thetree) had maximum between-group dif-ferences and minimum within-group dif-ferences in treatment effect. Refinementin the second data subset reduces the in-fluence of outliers and helps produce un-biased HTE estimates (17). The overallapproach was repeated 4,000 timesfrom the first step to produce a “forest”of trees by repeated random resamplingof the data (cross-validation). No changein estimated variable importance was ob-served beyond 4,000 trees. Variable im-portance was defined as the frequencywith which a given variable was incorpo-rated into a tree at the first, second, andfurther split points (i.e., a variable canchange positions between trees, but vari-able selection for each position is trackedto monitor its importance). After the for-est was constructed and cross-validated,the summary (average) decision tree wasselected that separated participants intothe subgroups that were most consistentacross all trees in the forest (23).To assess performance of the summary

decision tree, the absolute risk differencein mortality was calculated between theintensive and standard glycemic controlarms within each subgroup (leaf) of thetrial population and compared across thesubgroups (Q test for heterogeneityamong subgroups and stratified log-ranktest for trend in Kaplan-Meier all-causemortality rates across subgroups). Abso-lute risk difference is the guideline-recommended outcome variable becauseit provides a clinically meaningful abso-lute, as opposed to relative, measure ofeffect (24–26). In addition, we estimatedthe Cox proportional hazards model forthe outcome of mortality by treatmentarm within each leaf, the hazard ratio oftreatment, and the C statistic (area underthe receiver operating characteristiccurve) for discrimination of higher fromlower overall mortality by leaf.HTEmodels should not be confused for

riskmodels (e.g., Coxmodels of the risk ofmortality). An HTE model seeks to deter-mine characteristics that are associatedwith treatment effectiveness. Hence, itmodels the difference in event rates

between treatment arms (the treatmenteffect) and tries tofind the covariates thatare associated with the treatment beingmore effective or less effective. A riskmodel, by contrast, finds correlates asso-ciated with a given outcome, such asidentifying characteristics associatedwith the risk for mortality. Hence, it mod-els the absolute event rate and tries tofind the covariates (e.g., such as sex,blood pressure, etc.) that make overallmortality higher or lower; treatmentmay ormaynot bea covariate. A standardrisk model does not specifically look forthose factors that modify the treatmenteffect (i.e., interaction terms betweenstudy arm and covariates), whereas ourgradient forest approach focuses exclu-sively on finding influential interactionterms, indicating those factors that mod-ify the treatment effect. Furthermore, se-lection of an interaction term betweentreatment and effect modifiers may bereduced in significance by the larger ef-fect on model fit and C statistic by thenoninteracted terms and reveal only mod-ification on a relative scale versus the abso-lute scale of the gradient forest approach.

Sensitivity AnalysesIn sensitivity analyses, the summary de-cision treewas testedwith the alternativeoutcome of difference in CVD mortalitybetween study arms, defined in ACCORDas mortality suspected to be attributableto myocardial infarction, other acute cor-onary event, cardiovascular procedure,congestive heart failure, arrhythmia, orstroke. The effect of intensive therapywas notably larger (more adverse) forCVDmortality than for all-cause mortalityin the ACCORD trial (2).

Analyses were performed in R 3.3.3software (The R Project for Statistical Com-puting, Vienna, Austria).

RESULTS

ParticipantsOf the 10,251 study participants includedin the analysis, 718 died during studyfollow-up from all causes, including 327participants (6.4%) in the standard therapyarm and 391 participants (7.6%)in the in-tensive therapy arm. CVD was attributedas the cause of death for 331 participants(3.2% of participants, 46.1% of deaths), in-cluding 144 (2.8% of participants) in thestandard glycemic therapy arm and 187(3.6% of participants) in the intensive glyce-mic therapy arm. As in the original ACCORD

publication (2), the hazard ratio of treat-ment was 1.17 (95% CI 0.98, 1.40) for all-cause mortality in the intensive versusstandard glycemic group overall, after in-cluding all predictor covariates in a stan-dard Cox regressionmodel, and1.20 (95%CI1.04, 1.39)without predictor covariatesincluded.

Model SpecificationThe summary decision tree (Fig. 2) sepa-rated the ACCORDpopulation by variationin all-cause mortality rate differences be-tween the standard and intensive therapyarms. Thefirst split of the treewas definedby the HGI, which was selected as the keysplitting variable in 2,390 of 4,000 trees(59.8%). For participants with low HGI(,0.44, or 75% of the study sample),the next split was defined by BMI, whichwas selected as a subsequent splittingvariable in 2,322 of 4,000 trees (58.1%).The group with a low BMI (,30 kg/m2, aderived value rounded to the nearestkg/m2) was further split by age (,61years), which was selected in 1,814 of the4,000 trees (45.4%). The three variables de-fining the decision tree were available for9,801 of the 10,251 ACCORD trial partici-pants (95.6%).

Model PerformanceThe summary decision tree split the studysample into groups with significantly dif-ferent risk for all-cause mortality from in-tensive glycemic therapy, as reported inTable 1 (P , 0.001 by the Q test for het-erogeneity in absolute mortality risk dif-ference between intensive vs. standardtherapy among the four groups, andP , 0.001 by the stratified log-rank testfor a trend in absolute mortality differ-ence from subgroup 1 through sub-group 4).

Subgroup (leaf) 1 had 877 participants(8.6% of the 10,251-participant totalsample) and was defined by HGI ,0.44,BMI ,30 kg/m2, and age ,61 years old.Subgroup 1 had an absolute mortalityrate reduction (benefit) of 2.3% from in-tensive glycemic therapy (95% CI 0.2 to4.5 decrease; hazard ratio 0.41; 95% CI0.17, 0.98; P = 0.038 by the log-rank testadjusting for censoring). Participants insubgroup 1 had a number needed to treat(NNT) of 43 over 5 years to observe 1 lessdeathwith intensive rather than standardglycemic therapy.

Subgroup (leaf) 2 had 1,717 partici-pants (16.7% of sample) and was defined

care.diabetesjournals.org Basu and Associates 607

by HGI ,0.44, BMI ,30 kg/m2, andage $61 years old. Subgroup 2 had nosignificant absolute mortality rate reduc-tion or increase, with an absolute risk in-crease of 0.7% from intensive glycemictherapy (95% CI 1.6 decrease to 3.1 in-crease; hazard ratio 1.11, 95% CI 0.77,1.60; P = 0.560).Subgroup (leaf) 3 had 4,678 partici-

pants (45.6% of sample) and was definedby HGI ,0.44 and BMI $30 kg/m2. Sub-group 3 had no significant absolute mor-tality rate reduction or increase, with anabsolute risk increase of 0.9% from inten-sive glycemic therapy (95%CI 0.4 decreaseto 2.1 increase) and a hazard ratio of 1.12(95% CI 0.91, 1.50; P = 0.220).Subgroup (leaf) 4 had 2,529 partici-

pants (24.7% of sample) and was definedby HGI $0.44. Subgroup 4 had an abso-lute mortality rate increase of 3.7% fromintensive glycemic therapy (95% CI 1.5 to6.0 increase) and a hazard ratio of 1.57(95% CI 1.20, 2.04; P , 0.001). Partici-pants in subgroup 4 had a number neededto harm of 27 over 5 years associated with1 additional death in the intensive thanstandard glycemic therapy arm.Figure 3 illustrates the survival curves

among the intensive and standard

glycemic therapy arms of ACCORD, strat-ified by the subgroups. SupplementaryTable 2 lists the other clinical featuresamong the subgroups by arm, revealingthat covariates were balanced across thetherapy arms within each subgroup.Hence, imbalance in important covariatesbetween arms did not result from thestratification into subgroups, meaningthat the gradient forest analysis did notproduce confounding by measured cova-riates. Critically, Supplementary Table 2also reveals that because no single pre-dictor variable could explain the sub-groups, the decision tree did notcapture features that would be otherwiseobvious from a univariable subgroupanalysis; rather, themultivariate machinelearning analysis had the power to revealvariations in mortality that would not bedetectable to univariable subgroup anal-yses along any of the measured variablesin the study. Overall, the out-of-the-bagerror rate of the model, a measure of theprediction error during out-of-samplecross-validation, was low, with a valueof 5.6%.

We evaluated whether the secondaryoutcome of composite microvascularevents varied among the subgroups

(Supplementary Table 3 and Supplemen-tary Fig. 1). The average decrease in mi-crovascular outcomes was nonsignificantfor all four subgroups, consistent withthe overall results of the ACCORD trial(24). However, the average outcomeswere better for subgroup 1 (absoluterisk decrease of 4.2%, 95% CI 10.6 de-crease to 2.1 increase, P = 0.15) than forsubgroup 4 (absolute risk decrease of2.3%, 95% CI 6.1 decrease to 1.5 increase,P = 0.60).

In sensitivity analyses, we evaluatedthe summary decision tree with the out-come of CVD mortality (SupplementaryTable 4 and Supplementary Fig. 2). Aswith absolute risk differences in all-causemortality, absolute risk differences inCVD mortality between intensive andstandard glycemic therapy differed signif-icantly between the four subgroups (P,0.001 for heterogeneity and for trend).Subgroup 1 had an absolute cardiovascu-lar mortality risk decrease of 1.7% in theintensive therapy arm (95% CI 0.2 to 3.2decrease, P = 0.027), and subgroup 4 hadan absolute cardiovascular mortality riskincrease of 2.3% in the intensive ther-apy arm (95% CI 0.6 to 3.9 increase,P = 0.004).

Figure 2—Summary risk stratification decision tree developed to identify the absolute change in risk of all-cause mortality among persons with type 2diabetes subject to intensive therapy, based on baseline characteristics of individual participants in the ACCORD trial (2001–2009, N = 10,251). Negativevalues indicate reduced absolutemortality (benefit from intensive glycemic control), whereas positive values indicate increased absolutemortality (harmfrom intensive glycemic control).

608 Mortality Risk From Glycemic Therapy Diabetes Care Volume 41, March 2018

CONCLUSIONS

We sought to inform clinical decisions re-garding the safety of intensive glycemictherapy among patients with type 2 dia-betes and elevated CVD risk by identify-ing HTE in all-cause mortality within theACCORD trial. We found that by usingthe covariates of HGI, age, and BMI, wecould classify participants in the ACCORDtrial into subgroups with clinically mean-ingful differences in mortality attribut-able to intensive glycemic therapy. Themean all-cause mortality rate among in-dividuals with diabetes in the U.S. is;6%over 5 years, so an absolute risk increaseof 4% or an absolute risk reduction of 2%is clinically meaningful (25). Approxi-mately 25% (n = 2,529) of participantsbelonged to a subgroup experiencingincreasedmortality attributable to intensiveglycemic therapy, whereas 9% (n = 877)belonged to a subgroup that experiencedreduced mortality attributable to inten-sive glycemic therapy. We did not findthat hypoglycemia, medication classes,number of medications, combinations ofmedications, baseline diabetes complica-tions, or cardiovascular risk factors couldexplain the HTEs from intensive glycemictherapy. We also did not find a trade-offbetween microvascular and mortalityrisk, because the patientswith the highestmortality risk from intensive therapy alsohad the least evidence of microvascularbenefit, and vice versa.

Our findings support and extend priorstudies of glycemic control in diabetesmanagement. We found that despitethe average treatment effect of highermortality, there were some groups thatmay have benefited from, along withsome that were likely harmed by, inten-sive glycemic therapy; nearly two-thirds(n = 6,395) experienced neither benefitnor harm. Because the risk of benefitand of harm varies among individualswith type 2 diabetes, our results supportcurrent guidelines that advocate for indi-vidualized treatment decisions and alsohelp such guidelines to be made opera-tional in clinical practice (1). Clinically, thedecision tree we developed through adata-drivenmultivariate subgroup analysisuses readily available clinical data andmayassist clinician-patient discussions aboutglycemic therapy. Although the ACCORDHbA1c target of ,6.0% (42 mmol/mol) isnot guideline recommended, many pa-tients are currently treated to ,6.5%

Tab

le1—Changein

abso

lute

riskofall-cau

semorta

lityfro

mintensive

versusstan

dard

glycem

icco

ntro

l,amongsu

bgroupsidentifi

edbygrad

ientforest

analysis

oftheACCORDtria

l(20

01–2009,

N=10

,251)

Group

Inten

sivetherap

yStan

dard

therap

y

Totald

eaths(N

=9,801

of10,251

with

variablesto

stratifyrisk),n

(%)

Death

sam

ong

inten

sivetherap

y(N

=4,900

of5,128

with

variablesto

stratifyrisk),n

(%)

Death

sam

ong

standard

therap

y(N

=4,901

of5,123

with

variablesto

stratifyrisk),n

(%)

Inten

sivevs.

standard

treatmen

t,hazard

ratio(95%

CI),C

statistic(95%

CI)

Absolute

riskdifferen

ce,%(95%

CI)

Pvalue

(log-ranktest)

ofdifference

inevent

ratesbetw

eenarm

s,within

leaf

Stratified

log-ran

ktest

(differen

cein

treatmen

teffect

across

leaves)

Leaf1(leftm

ostinFig.2)

(HGI,

0.44,BMI,

30kg/m

2,age,61

years)424

45325

(2.9)7(1.7)

18(4.0)

0.41(0.17,0.98),0.64(0.52,0.76)

22.3

(24.5

to20.2)

0.04

,0.001

Leaf2(HGI,

0.44,BMI,

30kg/m

2,age

$61

years)811

906116

(6.8)58

(7.2)58

(6.4)1.11

(0.77,1.60),0.62(0.56,0.67)

0.7(2

1.6to

3.1)0.56

Leaf3(HGI,

0.44,BMI$30

kg/m2)

2,3752,303

250(5.3)

137(5.8)

113(4.9)

1.12(0.91,1.50),0.64(0.60,0.68)

0.9(2

0.4to

2.1)0.22

Leaf4(rightm

ostinFig.2)(H

GI$

0.44)1,290

1,239234

(9.3)143

(11.1)91

(7.3)1.57

(1.20,2.04),0.66(0.62,0.71)

3.7(1.5

to6.0)

,0.001

SeeFig.2

forvisu

alizationof

subgroups.N

otethat

thehazard

ratioof

intensivevs.standard

treatmentand

theCstatistic

(areaund

erthe

receiverop

eratingcharacteristic

curve)fordiscrim

inationof

higherfro

mlow

eroverallm

ortalitybyleaf

was

estimated

theCox

proportionalhazards

modelfor

theou

tcomeof

mortality

bytreatm

entarm

within

eachleaf.

care.diabetesjournals.org Basu and Associates 609

(48 mmol/mol, the achieved mean in theintensive therapy arm) with regimensother than metformin alone (5,19). Be-cause ;25% of ACCORD-eligible patientswere observed to have high risk of harmfrom intensive therapy, deescalation ofglycemic therapy may be warranted forsome patients. Our study also adds to agrowing body of literature, includinga prior study using ACCORD data, that ahighHGImay bean important indicator ofdiabetes severity as well as a predictor of

HTEs in mortality among persons withtype 2 diabetes (20–22). A higher HGImay indicate higher postprandial glucoselevels and increased glycemic variability.Notably, the HGI in this report does notrequire that mean glucose levels be de-termined by continuous glucose monitor-ing, as is common in studies of type 1diabetes. Rather HGI was calculatedusing a single HbA1c and fasting plasmaglucose measurement, offering poten-tial convenience for clinical use.

More broadly, these results point to-ward the application of innovative meth-ods for the detection of HTEs from clinicaltrial data. Our findings highlight the pointthat trial summary statistics, which areaverages, may obscure clinically importantheterogeneities and that the rigorous ap-plication of machine learning methodswith conservative cross-validation ap-proaches may aid in finding consistentsubgroups that experience substan-tial differences in treatment effects.

Figure 3—Survival curves for all-cause mortality among subsets identified by each subgroup in the decision tree (see Fig. 2 for subgroups). P values arefrom the stratified log-rank test adjusted for censorship of Kaplan-Meier all-causemortality rates among the intensive vs. standard glycemic therapy arm.A: Leaf 1 (leftmost in Fig. 2) (HGI ,0.44, BMI ,30 kg/m2, and age ,61 years). B: Leaf 2 (HGI ,0.44, BMI ,30 kg/m2, and age $61 years). C: Leaf3 (HGI,0.44 and BMI$30 kg/m2). D: Leaf 4 (rightmost in Fig. 2) (HGI$0.44).

610 Mortality Risk From Glycemic Therapy Diabetes Care Volume 41, March 2018

Extensive theoretical and empirical re-search suggests that the ability of conven-tional univariable subgroup analysesto detect clinically important heterogene-ity in treatment effects is very limited(26–28). Previous studies of HTEs inACCORD data have considered single var-iables, finding that hypoglycemia and car-diac autonomic dysfunction did notexplain the harm of intensive therapy(10,11,13). The machine learning methodaccounting for multiple simultaneous co-variates and interactions between themwas therefore able to explain the varia-tion inmortality better than previous uni-variable analyses. We in fact found in oursensitivity analyses that no single covari-ate would be able to distinguish the sub-groups, and therefore, our multivariablemachine learning analysis had the powerto explain variations that were not possi-ble to find with traditional univariablesubgroup analyses.A prior analysis of age as a source of

HTEs found that younger age was associ-ated with increased harm (12). Our find-ing that younger age, in combinationwithlower BMI and HGI, is instead associatedwith benefit may represent the interac-tion of factors not considered in univari-able analyses. In general, consideringseveral factors in combination may be re-quired to explain clinically important var-iations in benefit and harm seen in clinicaltrials. Consequently, multivariate HTEmodeling has been increasingly recom-mended (15,16,29). Our data-driven ap-proach also adjusts for type I error dueto multiple hypothesis testing, a majordisadvantage of traditional subgroupanalysis methods. We used rigorouscross-validation to reduce the chance offalse-positive findings.Our analysis nevertheless has im-

portant limitations. As a result of theACCORD trial being stopped early, wecould assess only shorter-term outcomes.Further, the ACCORD trial was conductedbefore the widespread availability ofsodium–glucose cotransporter 2 andglucagon-like peptide 1 agents, whichhave cardiovascular benefits that affectthe risk of mortality with glycemic ther-apy (30–33). In addition, because wewanted a clinical decision tree that wasuseful in practice, we focused on pre-treatment characteristics rather thantime-varying covariates, which may bemore useful in predicting outcomes overtime but are also more complex for

clinicians to use. Next, although we usedmethods to minimize the risk of type Ierror and did not observe imbalance incovariates within subgroups, our study isnevertheless a post hoc analysis of a sin-gle trial. With machine learning methods,as with correlative statistical methods ingeneral, variable selection does not provecausality, and the variables selected mayonly be surrogates for more complexphysiological processes. HGI is a summarymeasure that may not have a definitivephysiological meaning and can be calcu-lated in alternative ways; here, it servesas a useful and readily calculable markerof complex physiological processes andwas found to separate the variation inmortality better than alternative covari-ates. HGI thus likely reflects a complexunderlying heterogeneity in treatment ef-fect. Explaining mechanistically the phys-iological relationships that underlie theHTEs observed is not possible from theavailable data, although they are broadlyconsistent with clinical observation andpoint to areas for further study (34). Inaddition, the number of deaths amongthe standard and intensive therapies inleaf 1 were too small (7 and 18 subjects).Finally, it is important to note that theseresults naturally apply to the populationthat met inclusion criteria for ACCORD,which includes people with type 2 diabe-tes with HbA1c of $7.5%, who were be-tween the ages of 40 and 79 years andhad CVD or were between the ages of55 and 79 years and had anatomical evi-dence of significant atherosclerosis, albu-minuria, left ventricular hypertrophy, orat least two additional risk factors forCVD, such as dyslipidemia, hypertension,tobacco smoking, or obesity.

Our study suggests several directionsfor futurework. Because only internal val-idation was done in this report, pro-spectively validating the decision tree onan independent trial data set and onpopulation-based observational datawould help assess the generalizability ofour findings. Ultimately, it will be impor-tant to evaluate the effect of using thedecision tree on clinical practice and pa-tient outcomes. More generally, HTEs arelikely to be the norm, rather than the ex-ception, in many areas of investigation.Therefore, it may be advantageous to de-sign trials that can identify HTEs up front,rather than relying on post hoc analysisas we have done here. A prior simulationstudy revealed that alternative trial

designs, which randomize persons in astepwise fashion to incrementally higherlevels of therapy intensification, could in-crease statistical power to detect HTEs andprovide more granular estimates of treat-ment benefit or harm (28). Finally, theanalysis suggests that HGImay be a usefulclinical indicator of risk and advanced di-abetes, necessitating future prospectivestudy as a useful clinical biomarker.

Cliniciansmay use HGI, age, and BMI tohelp individualize decisions about glyce-mic control among people with type 2diabetes. This may lead to deescalationof therapy for many patients while alsoidentifying patients who do not face in-creased all-causemortality risk from theircurrent glycemic therapy. Further, themethods used in this study offer a princi-pled way to help inform individualizedcare using data from randomized trials.The application of similar methods mayenable us to learn more from the contri-bution that clinical trial participantsmake, bringing us closer to the goal ofpersonalized medicine.

Acknowledgments. The manuscript was pre-pared using ACCORD research materials ob-tained from the National Heart, Lung, andBlood Institute Biologic Specimen and Data Re-pository Information Coordinating Center anddoes not necessarily reflect the opinion or viewsof the ACCORD trial or the National Heart, Lung,and Blood Institute.Funding. Research reported in this publicationwassupportedby theNational InstituteonMinorityHealth and Health Disparities (DP2-MD-010478and U54-MD-010724 to S.B.) of the NationalInstitutes of Health, the American Heart Asso-ciation (17MCPRP33670728 to S.R.), and theNational InstituteofDiabetes andDigestiveandKidney Diseases (U01-DK-098246 and R18-DK-10273 to D.J.W. and K23-DK-109200 to S.A.B.)of the National Institutes of Health.The content is solely the responsibility of the

authors and does not necessarily represent theofficial views of the National Institutes of Healthor the American Heart Association.Duality of Interest. No potential conflicts of in-terest relevant to this article were reported.AuthorContributions.S.B.wrotethefirstdraftofthemanuscript. S.B., S.R., D.J.W., and S.A.B. revisedthe manuscript, reviewed the results, and contrib-uted to the discussion. S.B. and S.A.B. conductedanalyses. S.A.B. conceived of the research idea. S.B.is the guarantor of this work and, as such, had fullaccess to all the data in the study and takesresponsibility for the integrity of the data and theaccuracy of the data analysis.

References1. AmericanDiabetesAssociation. Glycemic targets.Sec. 6. InStandardsofMedicalCare inDiabetesd2017.Diabetes Care 2017;40(Suppl. 1):S48–S56

care.diabetesjournals.org Basu and Associates 611

2. Action to Control Cardiovascular Risk in Diabe-tes StudyGroup; Gerstein HC,MillerME, ByingtonRP, et al. Effects of intensive glucose lowering intype2diabetes. N Engl JMed 2008;358:2545–25593. Zoungas S, Arima H, Gerstein HC, et al.; Collab-orators on Trials of Lowering Glucose (CONTROL)group. Effects of intensive glucose control on mi-crovascular outcomes in patients with type 2 di-abetes: a meta-analysis of individual participantdata from randomised controlled trials. Lancet Di-abetes Endocrinol 2017;5:431–4374. Sussman JB, Kerr EA, Saini SD, et al. Rates ofdeintensification of blood pressure and glycemicmedication treatment based on levels of controland life expectancy in older patients with di-abetes mellitus. JAMA Intern Med 2015;175:1942–19495. Lipska KJ, Ross JS, Miao Y, Shah ND, Lee SJ,Steinman MA. Potential overtreatment of diabe-tes mellitus in older adults with tight glycemiccontrol. JAMA Intern Med 2015;175:356–3626. McCoy RG, Van Houten HK, Ross JS, MontoriVM, Shah ND. HbA1c overtesting and overtreat-ment among US adults with controlled type 2 di-abetes, 2001-13: observational population basedstudy. BMJ 2015;351:h6138.7. Holman RR, Paul SK, BethelMA,Matthews DR,Neil HA. 10-year follow-up of intensive glucosecontrol in type 2 diabetes. N Engl J Med 2008;359:1577–15898. Skyler JS, Bergenstal R, BonowRO,et al.; Amer-ican Diabetes Association; American College ofCardiology Foundation; American Heart Association.Intensive glycemic control and the preventionof cardiovascular events: implications of theACCORD, ADVANCE, and VA diabetes trials:a position statement of the American Diabe-tes Association and a scientific statement ofthe American College of Cardiology Founda-tion and the American Heart Association. Di-abetes Care 2009;32:187–1929. Riddle MC. Counterpoint: intensive glucosecontrol and mortality in ACCORD–still lookingfor clues. Diabetes Care 2010;33:2722–2724

10. Bonds DE, Miller ME, Bergenstal RM, et al.The association between symptomatic, severe hy-poglycaemia andmortality in type 2 diabetes: ret-rospective epidemiological analysis of theACCORD study. BMJ 2010;340:b490911. Seaquist ER, Miller ME, Bonds DE, et al.;ACCORD Investigators. The impact of frequent andunrecognized hypoglycemia on mortality in theACCORD study. Diabetes Care 2012;35:409–414

12. Miller ME, Williamson JD, Gerstein HC, et al.;ACCORD Investigators. Effects of randomizationto intensive glucose control on adverse events,cardiovascular disease, andmortality in older ver-sus younger adults in the ACCORD trial. DiabetesCare 2014;37:634–64313. Pop-Busui R, Evans GW, Gerstein HC, et al.;Action to Control Cardiovascular Risk in DiabetesStudy Group. Effects of cardiac autonomic dys-function on mortality risk in the Action to ControlCardiovascular Risk in Diabetes (ACCORD) trial.Diabetes Care 2010;33:1578–158414. Riddle MC, Ambrosius WT, Brillon DJ, et al.;Action to Control Cardiovascular Risk in DiabetesInvestigators. Epidemiologic relationshipsbetweenA1C and all-cause mortality during a median 3.4-year follow-up of glycemic treatment in theACCORD trial. Diabetes Care 2010;33:983–99015. Burke JF, Hayward RA, Nelson JP, Kent DM.Using internally developed risk models to assessheterogeneity in treatment effects in clinicaltrials. Circ Cardiovasc Qual Outcomes. 2014;7:163–169.16. Hayward RA, Kent DM, Vijan S, Hofer TP.Mul-tivariable risk prediction can greatly enhance thestatistical power of clinical trial subgroup analysis.BMC Med Res Methodol 2006;6:1817. Athey S, Imbens G. Recursive partitioning forheterogeneous causal effects. Proc Natl Acad SciU S A 2016;113:7353–736018. ACCORD Study Group; Buse JB, Bigger JT,Byington RP et al. Action to Control Cardiovascu-lar Risk in Diabetes (ACCORD) trial: design andmethods. Am J Cardiol 2007;99:21i–33i19. Gerstein HC, Riddle MC, Kendall DM, et al.;ACCORD Study Group. Glycemia treatment strat-egies in the Action to Control Cardiovascular Riskin Diabetes (ACCORD) trial. Am J Cardiol 2007;99:34i–43i20. Hempe JM, Liu S, Myers L, McCarter RJ, BuseJB, Fonseca V. The hemoglobin glycation indexidentifies subpopulations with harms or benefitsfrom intensive treatment in the ACCORD trial. Di-abetes Care 2015;38:1067–107421. van Steen SC, Schrieks IC, Hoekstra JB, et al.;AleCardio study group. The haemoglobin glyca-tion index as predictor of diabetes-related com-plications in the AleCardio trial. Eur J Prev Cardiol2017;24:858–86622. McCarter RJ, Hempe JM, Gomez R, ChalewSA. Biological variation in HbA1c predicts risk ofretinopathy and nephropathy in type 1 diabetes.Diabetes Care 2004;27:1259–1264

23. Banerjee M, Ding Y, Noone AM. Identifyingrepresentative trees from ensembles. Stat Med2012;31:1601–161624. Ismail-Beigi F, Craven T, Banerji M, et al. Ef-fect of intensive treatment of hyperglycaemia onmicrovascular complications of type 2 diabetes inACCORD: a randomised trial. Lancet 2010;376:419–43025. National Center for Health Statistics. Health,United States, 2016: With Chartbook on Long-Term Trends in Health. Washington, D.C., Depart-ment of Health and Human Services, 201726. VanderWeele TJ, Knol MJ. Interpretation ofsubgroup analyses in randomized trials: heteroge-neity versus secondary interventions. Ann InternMed 2011;154:680–68327. Wallach JD, Sullivan PG, Trepanowski JF,Sainani KL, Steyerberg EW, Ioannidis JP. Evalu-ation of evidence of statistical support andcorroboration of subgroup claims in random-ized clinical trials. JAMA Intern Med 2017;177:554–56028. Basu S, Sussman JB, Hayward RA. Detectingheterogeneous treatment effects to guide per-sonalized blood pressure treatment: a modelingstudy of randomized clinical trials. Ann InternMed 2017;166:354–36029. Kent DM, Rothwell PM, Ioannidis JP, AltmanDG, Hayward RA. Assessing and reporting hetero-geneity in treatment effects in clinical trials: a pro-posal. Trials 2010;11:8530. Marso SP, Bain SC, Consoli A, et al.; SUSTAIN-6 Investigators. Semaglutide and cardiovascularoutcomes in patients with type 2 diabetes. NEngl J Med 2016;375:1834–184431. Marso SP, Daniels GH, Brown-Frandsen K,et al.; LEADER Steering Committee; LEADER TrialInvestigators. Liraglutide and cardiovascular out-comes in type 2 diabetes. N Engl J Med 2016;375:311–32232. Zinman B,Wanner C, Lachin JM, et al.; EMPA-REG OUTCOME Investigators. Empagliflozin,cardiovascular outcomes, and mortality intype 2 diabetes. N Engl J Med 2015;373:2117–222833. NealB, PerkovicV,MahaffeyKW, et al.; CANVASProgram Collaborative Group. Canagliflozin andcardiovascular and renal events in type 2 diabe-tes. N Engl J Med 2017;377:644–65734. American Diabetes Association. Pharmaco-logic approaches to glycemic treatment. Sec. 8.In Standards of Medical Care in Diabetesd2017.Diabetes Care 2017;40(Suppl. 1):S64–S74

612 Mortality Risk From Glycemic Therapy Diabetes Care Volume 41, March 2018