Progress in Cardiovascular Diseases 54 (2012) 367–371www.onlinepcd.com

Observational Epidemiology Within Randomized Clinical Trials:Getting a Lot for (Almost) Nothing

George Howarda,⁎, Virginia J. HowardbaDepartment of Biostatistics, School of Public Health, University of Alabama at Birmingham, Birmingham, AL 35294-0022bDepartment of Epidemiology, School of Public Health, University of Alabama at Birmingham, Birmingham, AL 35294-0022

Abstract Randomized clinical trials (RCTs) are considered the gold standard approach to establish

Statement of Conf⁎ Address reprint

and Chair, DepartmenRoom 327D Ryals B35294-0022.

E-mail address: g

0033-0620/$ – see frodoi:10.1016/j.pcad.20

relationships between exposures/treatments and outcomes. Analyses that examine theassociation between the randomized factor and outcomes are “protected by randomization”from potential confounding factors. Despite limitations largely arising from the lack ofgeneralizability of findings, RCTs offer a rich environment to assess associations betweenother factors and outcomes which are by definition observational epidemiological studies.Herein we discuss the limitations of these analyses, but also the opportunities that arise fromthe use of observational epidemiological assessments that can be performed: 1) betweenfactors assessed prior to randomization, 2) analyses of longitudinal outcomes both in thecohort all together, and among subjects randomized to placebo treatment, and 3) analyses of“associated” series of patients (such as non-randomized registries or screenees for the RCT).While these assessments of associations are not protected by randomization, with properplanning these assessments within the RCT framework can be done in a powerful andeffective manner. (Prog Cardiovasc Dis 2012;54:367-371)

© 2012 Published by Elsevier Inc.

Associations between exposures and outcomes arediscovered and documented in studies that span aspectrum of research designs from ecological studiesthrough observational epidemiology to the randomizedclinical trial (RCT), with the RCT considered the “goldstandard” approach for establishing relationships. Thestandard 2-group RCT focuses on the impact of asingle exposure or intervention, whereas a smallernumber of trials uses a factorial design to examine 2(or a very limited number) exposures simultaneously.Because randomization will tend to remove thepotential impact of confounding factors, the essenceof the analysis and interpretation of an RCT requires

lict of Interest: see page 371.requests to George Howard, DrPH, Professort of Biostatistics, UAB School of Public Health,uilding, 1665 University Blvd, Birmingham, AL

howard@uab.edu (G. Howard).

nt matter © 2012 Published by Elsevier Inc.11.08.003

collection of data only on the intervention assignmentand outcomes (including adverse events), with otherfactors being “controlled” by randomization. However,a broader data collection effort is normally performedin RCTs to define subgroups for stratified analyses,provide insights to the mechanism of action, andaddress secondary aims. The product of this extensivedata collection in RCTs can be a rich resource to assessa wide spectrum of additional hypotheses; analyses notfocusing on the effect of the randomization factor areobservational epidemiological assessments.

In addition, RCT eligibility requirements lead toscreening of more potential subjects than will be subse-quently included in the randomized cohort. Because theinfrastructure (staff, clinics, electronic systems, etc)and recruitment strategies and training are in place forthe conduct of the randomized study, some minimal datacollection on these screenees can be performed withgreat efficiency. Finally, randomized trials sometimesdevelop series of subjects collected in associated

367

Abbreviations and Acronyms

RCT = randomized clinicaltrial

368 G. Howard, V.J. Howard / Progress in Cardiovascular Diseases 54 (2012) 367–371

registries or lead-in se-ries. These types ofactivities provide greatopportunity for the as-sessment of associations

of exposures and outcomes that are not formally part ofthe randomization experiment.

The natural conduct of a clinical trial has as a by-product the creation of a database characterizing thepopulation. Analyses within this database focusing on theimpact of the randomization factor on outcome measureswould be considered experimental in nature and representthe “randomized trial” aspect of the study. However, giventhe existence of such a database, there is no restriction thatprevents analyses of associations not involving therandomization factor. If the randomization factor is notinvolved in the analysis, then this analysis is notexperimental in nature, is not “protected” by randomiza-tion, and is by definition observational epidemiology.

Barriers to observational epidemiology studiesin RCTs

For both RCTs and observational epidemiology, a keyconsideration is the ability to generalize study findings to amore broad population. Recruitment for some RCTs isconducted in hospitals or clinician offices, and as such, itcould be argued that the population may be generalizableonly to participants seen in offices of specific types ofphysicians or certain hospitals. Because the hypothesisaddressed by these clinical trials is the clinical manage-ment of such patients, the ability to generalize beyond thispopulation is not an issue for the RCT. However,generalizations of these clinical populations to a morebroad population do raise concern. For observationalepidemiology, the need to generalize the findings of astudy introduces the issue of the representativeness of theRCT study cohort, a concern that, as noted, may generallynot be a high-priority issue for RCTs.

Most epidemiological analyses can be broadly classi-fied to have 1 of 2 goals: (1) descriptions of thedistribution of factors in the population includingthe prevalence or incidence rates of outcomes and (2)the assessment of the relationship between exposures andoutcomes. The importance of the representativeness of thepopulation is a key consideration in reports describing thedistribution of factors (including prevalence or incidencerates) because the distribution of these factors in anonrepresentative group of individuals would not offeraccurate assessment of more general populations.

It is clear that frequently, individuals participating inRCTs can be nonrepresentative of the more generalpopulations. For example, Martinsson et al1 showed thatparticipants in an RCT assessing physical activitymaintenance were younger, more highly educated, and

more likely to report recently improved health than werenonparticipants. Surman et al2 found that persons enrolledin RCTs of attention deficit/hyperactivity disorder hadlower levels of functional impairment and less psychiatriccomorbidity than observed in community samples ofpersons with the same condition. Herrington et al3 showedlower prevalence of cardiovascular risk factors amongparticipants of the Heart and Estrogen/Progestin Replace-ment Study Trial (HERS) than what was observed insimilar women (with coronary disease) in the NationalHealth and Nutrition Examination Survey III. Ahmad etal4 noted that, among 556 RCTs of tobacco use and 184clinical trials of HIV infection and that although 70%smoking mortality and 99% HIV mortality occurred inlow- and middle-income countries, less than 5% and 33%of trials were conducted in these countries. Theseexamples are offered as evidence of the well-acceptedposition that participants in RCTs tend to be bettereducated, healthier, and have higher levels of socioeco-nomic status than the general population. Because in-dividuals with higher socioeconomic status have betterhealth outcomes than the general population, it may beproblematic to generalize the estimates of the distributionof factors (including prevalence and incidence rates) madefrom RCTs to the more general population. Olschewskiand Scheurlen5 formalized the assessment of externalvalidity through a proposed “comprehensive cohort study”approach where outcomes are compared between random-ized patients and nonrandomized eligible patients, wherethe external validity (ie, the generalizability of the findingsdo a more general population) might be questioned incases where outcome differences exist; however, thisapproach is not uniformly accepted.6

Because of these concerns of representativeness, itmay be tempting to quickly discharge the use ofepidemiological analysis of RCT cohorts; however, itmay be worthwhile to consider the many reasons whyRCTs may tend to have nonrepresentative cohorts. In asystematic review of 65 articles assessing the barriers toparticipation in cancer clinical trials, the most frequentlyreported barrier was mistrust of research and themedical system (reported in 20 studies),7 a concernthat is also shared by those asked to participate inobservational epidemiological studies. However, dataare more limited regarding specific to barriers toobservational epidemiology.8 Other commonly reportedbarriers that could also easily be shared by RCT andobservational epidemiology include costs of participating(17 studies), demographics (16 studies), availability oftransportation (15 studies), time commitment (9 studies),and family issues (8 studies).7 In comparison, there wererelatively few barriers that one would speculate wouldapplymore strongly to RCT participation, but these includethe potential lack of education about clinical trials (13studies) and fear (10 studies).7 It would seem thatepidemiological studies and RCTs share many barriers to

369G. Howard, V.J. Howard / Progress in Cardiovascular Diseases 54 (2012) 367–371

participation, perhaps supporting the position that differ-ences in the representativeness of the population is not assubstantial as sometimes perceived. As such, althoughconcerns regarding generalizability of findings persist, thevalidity of the distribution of factors in some RCTsmay notbe substantially weaker than in observational epidemio-logical studies.

The other type of analysis in observational epidemi-ological studies evaluates the relationships betweenexposures and outcomes. Although a representativesample is certainly desirable, it is not clear that therepresentativeness of the population is critical to thistype of analysis. For example, suppose that there isinterest in assessing the relationship between hyperten-sion (an exposure) and incident myocardial infarction(an outcome). Suppose that this is assessed in an RCTthat has a higher socioeconomic status than the generalpopulation. After adjustment for the factor quantifyingthe nonrepresentativeness (socioeconomic status), thatthe RCT population is nonrepresentative would onlyaffect the estimated association between the exposure(hypertension) and the outcome (myocardial eventoutcomes) if this factor is acting as an effect modifierof the relationship of interest. Admittedly, concerns dopersist that the trial population may be nonrepresentativeon some unknown or unmeasured confounding factor;however, this same concern would equally exist for anobservational epidemiological cohort. For this reasonthen, there are fewer barriers to performing observa-tional epidemiological analysis examining associationsbetween factors than for performing observationalepidemiological analysis describing the distribution offactors (including incidence or prevalence).

Opportunities for observational epidemiologicalanalysis in RCT cohorts

Analysis of baseline data

Many RCTs recruit the cohort over some period butthen take several years for completion of longitudinaloutcome assessments. Once the data set describing thebaseline characteristics of the cohort is cleaned (frequentlylong before outcome assessment has been completed),there is a rich opportunity for cross-sectional analysis ofthese baseline data. Because these analyses would notinvolve factors assessed after the assigned interventionwas initiated, concern regarding unmasking the treatmentassignments is not an issue.

Examples of such analyses abound. For example,Wren et al9 examined the association of gait on surgicaldecision making among 178 children with cerebral palsy,Rogers et al10 examined biomarkers of splenic functionin 193 sickle cell infants, and McKelvie et al11 examinedbaseline predictors of N-terminal pro b-type natriureticpeptide (NT proBNP) levels in 3562 patients in the

Irbesartan in Heart Failure with Preserved EjectionFraction Trial. In each of these reports, an analysis ofbaseline data was performed that explicitly did notinvolve the treatment assignment of the trial—that is, across-sectional observational epidemiological analysis.

These epidemiological analyses of cross-sectionalassessment in the baseline data of RCT cohorts providean efficient, cost-effective, and powerful evaluation ofassociations. As importantly, they also provide a pathwayfor publication of results while awaiting the long-termfollow-up of study participants in the RCT cohorts.Publication of such findings from the trial may beimportant in securing funding to complete the RCT ormay contribute ideas relating to the development of futureclinical trials while also providing support for the academiccareer of those involved in the conduct of the RCT.

Analyses of longitudinal (postrandomization) outcomeswithin the randomized cohort

Within RCT cohorts, any analysis not involving therandomization factor is, by definition, observationalepidemiology. Observational epidemiological analyses ofRCT cohorts are not restricted to cross-sectional analysisof baseline data but can also include any analyses ofoutcomes (even primary study outcomes) that do notdirectly involve associations with the randomizationfactor. These analyses that do not involve the randomiza-tion factor can be performed in the entire cohort (ie,involving all intervention treatment groups). For example,Riddle et al12 examined the association of baseline A1Cand all-cause mortality in the Action to Control Cardio-vascular Risk in Diabetes (ACCORD) trial (patients wererandomized to assess intensiveness of glucose control),Desai et al13 examined the association of baseline systolicblood pressure and all-cause mortality and heart failurehospitalization in the Beta-Blocker Stroke Trial (BEST)trial (patients were randomized to assess β-blockers), andKim et al14 assessed the association of menopause statuson the risk of incident diabetes in the Diabetes PreventionProgram (randomization to lifestyle and metformin).These analyses implicitly or explicitly assume that thetreatment itself does not modify the association of theexposure and outcome, although the analysis couldinvolve adjustment for the randomization factor as a“nuisance” covariate, for example, as performed byOvbiagele et al15 in their assessment of nonspecificprescription adherence and ischemic stroke outcomes afteradjustment for treatment assignment in the VitaminIntervention for Stroke Prevention (VISP) trial.

Absent the concern that the randomization factor couldinfluence the observational assessment of other exposuresand outcomes, the analysis of the entire RCT cohort offersthe advantage of a larger sample size and associatedincreased statistical power to detect associations. However,when there is concern that the study treatment modifies the

370 G. Howard, V.J. Howard / Progress in Cardiovascular Diseases 54 (2012) 367–371

association of exposures and outcome, then the analysiscan be restricted to exposure-outcome associationsassessed in the placebo treatment group only. Examplesof analyses restricted to the placebo group of RCTs includethe assessment of predictors of survival and metastasis-freeinterval among patients with prostate cancer randomized toatrasentan/placebo by Smith et al16 and the report by Titanet al17 examining fibroblast growth factor 23 as a predictorof renal outcome among persons with diabetic nephrop-athy randomized to enalapril or placebo. A particularadvantage of epidemiological analyses in placebo-treatedsubjects within a RCT is the opportunity to pool acrossstudies using different active treatments in a similarsubject population. This avoids differential effects of thedifferent active treatments used across the RCTs, such asin the report by Irizarry et al18 examining predictors ofcognitive decline among the placebo-treated patients in 6Alzheimer disease studies and the report by Lui andBlumhardt19 of changes in disability outcomes in theplacebo group of 2 randomized trials of multiplesclerosis. Although the representativeness of RCT cohortsis a concern, this would seem to be a lesser problem inthe assessment of exposure-outcome associations andmay also be an issue in many observational epidemio-logical studies as well.

Analyses in “associated” series of participants

The conduct of an RCT implies considerable infra-structure including the creation and staffing of clinicalcenters, establishment of communications (includingability to monitor and transfer data), creation of standingcommittees (recruitment, quality control, publications, andother committees), creation of documentation and pro-cedures, creation of study instruments, and purchase andstandardization of equipment. In addition, many partici-pants must be screened to identify those eligiblefor randomization, and this screening requires documen-tation factors at a level that is similar to the baseline visitof many observational epidemiological studies. Assuch, the conduct of an RCT either naturally produces ormakes it quite efficient to additionally collect data onassociated series of issues that can be used to addressepidemiological issues.

For example, the Carotid Revascularization Endarter-ectomy versus Stenting Trial (CREST) randomizedpatients to surgical vs endovascular intervention to addresshigh-grade carotid stenosis. At the time the study wasinitiated (2000), carotid stenting was a relatively newprocedure without a community of well-trained interven-tionalists. This was in contrast to endarterectomy, aprocedure with decades of experience to refine techniquesand provide training to surgeons. To avoid the possibilitythat lack of operator experience would place stenting at aninappropriate disadvantage in the trial, the study initiated a“lead-in registry” where operators would perform up to 20

carotid stent procedures to document their proficiencybefore being allowed to randomize patients in the mainstudy. However, with more than 100 centers and 150operators, this lead-in registry produced data on outcomesof more than 1500 carotid stent procedures and allowedanalysis of predictors of outcomes showing the high risk ofstenting at older ages of patients.20 Similar registries ofpatients not treated in clinical trials are relatively commonin the surgical community.

In addition to cohorts established external to therandomized cohort, the evaluation of potential participantsfor eligibility provides the baseline characterization of acohort that is frequently much larger in size than the cohortthat is eventually randomized. Although it does requireadditional resources for the collection and management ofdata, this characterization can be provided as a natural by-product of the conduct of the trial (eg, evaluationsperformed by clinicians and staff already supported bythe trial and data collected and managed to documenteligibility). With this investment in the evaluation of thosenot randomized, it can be very efficient to make anadditional investment to follow the cohort for outcomesand produce powerful epidemiological opportunities.Perhaps the 2 most notable examples of following thosescreened for a trial are the Woman's Health InitiativeObservational Study,21 a longitudinal cohort of 93,676screenees for the Woman's Health Initiative RandomizedTrial, and the 320,870 black and white screenees for theMultiple Risk Factor Intervention Trial (MRFIT).22

Although follow-up of these cohorts were huge additionalefforts, both of these cohorts have been remarkablyproductive. For example, the recent report by Belin etal21 from the Woman's Health Initiative ObservationalStudy examined diet quality as a predictor of incidentcardiovascular events, and the report by Thomas et al22

from the MRFIT screenees examined the dilution ofobserved racial differences in cardiovascular mortality thatwas attributable to traditional risk factors.

Hence, thoughtful planning of the use of resources thatare already supported for the conduct of an RCT make thecreation and evaluation of associated cohorts for epide-miological evaluations quite efficient. However, successin these efforts requires planning and the allocation ofadditional resources to ensure the high-quality data neededfor the epidemiological investigations.

Conclusions

Randomized trials focus on the association of therandomization factor with outcomes. The conduct of theRCT, however, produces many opportunities for describ-ing the distribution of factors and associations betweenoutcomes and exposures where the analyses do not involvethe randomization factor. These evaluations are notexperimental, are not protected by randomization, and,

371G. Howard, V.J. Howard / Progress in Cardiovascular Diseases 54 (2012) 367–371

as such, are observational epidemiological studies. Carefulplanning and forethought can make it possible to providepowerful insights from these analyses in a manner that canbe collected using synergy with the RCT and providingsubstantial efficiency. Finally, the impact of the represen-tativeness of the cohort needs to be considered in thegeneralization of the findings to larger populations.

Statement of Conflict of Interest

All authors declare that there are no conflicts of interest.

References

1. Martinson BC, Crain AL, Sherwood NE, et al: Population reach andrecruitment bias in a maintenance RCT in physically active olderadults. J Phys Act Health 2010;7:127-135.

2. Surman CB, Monuteaux MC, Petty CR, et al: Representativeness ofparticipants in a clinical trial for attention-deficit/hyperactivitydisorder? Comparison with adults from a large observational study.J Clin Psychiatry 2010;71:1612-1616.

3. Herrington DM, Fong J, Sempos CT, et al: Comparison of the Heartand Estrogen/Progestin Replacement Study (HERS) cohort withwomen with coronary disease from the National Health and NutritionExamination Survey III (NHANES III). Am Heart J 1998;136:115-124.

4. Ahmad N, Boutron I, Dechartres A, et al: Geographical represen-tativeness of published and ongoing randomized controlled trials.The example of: tobacco consumption and HIV infection. PLoS One2011;6:e16878.

5. Olschewski M, Scheurlen H: Comprehensive cohort study: analternative to randomized consent design in a breast preservationtrial. Methods Inf in Med 1985;24:131-134.

6. Schmoor C, Olschewski M, Schumacher M: Randomized and non-randomized patients in clinical trials: experiences with comprehen-sive cohort studies. Stat Med 1996;15:263-271.

7. Ford JG, Howerton MW, Lai GY, et al: Barriers to recruitingunderrepresented populations to cancer clinical trials: a systematicreview. Cancer 2008;112:228-242.

8. Williams B, Irvine L, Mcinnis AR, et al: When “no” might not quitemean “no”; the importance of informed and meaningful non-consent:results from a survey of individuals refusing participation in a health-related research project. BMC Health Serv Res 2007;7:59.

9. Wren TA, Otsuka NY, Bowen RE, et al: Influence of gait analysis ondecision-making for lower extremity orthopaedic surgery: Baseline

data from a randomized controlled trial. Gait Posture 2011;34:364-369.

10. Rogers ZR, Wang WC, Luo Z, et al: Biomarkers of splenic functionin infants with sickle cell anemia: baseline data from the BABYHUG Trial. Blood 2011;117:2614-2617.

11. McKelvie RS, Komajda M, McMurray J, et al: Baseline plasma NT-proBNP and clinical characteristics: results from the irbesartan inheart failure with preserved ejection fraction trial. J Card Fail2010;16:128-134.

12. Riddle MC, Ambrosius WT, Brillon DJ, et al: Epidemiologicrelationships between A1C and all-cause mortality during a median3.4-year follow-up of glycemic treatment in the ACCORD. DiabetesCare 2010;33:983-990.

13. Desai RV, Banach M, Ahmed MI, et al: Impact of baseline systolicblood pressure on long-term outcomes in patients with advancedchronic systolic heart failure (insights from the BEST trial). Am JCardiol 2010;106:221-227.

14. Kim C, Edelstein SL, Crandall JP, et al, for the Diabetes PreventionProgram Research Group: Menopause and risk of diabetes in theDiabetes Prevention Program. Menopause 2011;18:857-868.

15. Ovbiagele B, Campbell S, Faiz A, et al: Relationship between non-specific prescription pill adherence and ischemic stroke outcomes.Cerebrovasc Dis 2010;29:146-153.

16. Smith MR, Cook R, Lee KA, et al: Disease and host characteristics aspredictors of time to first bone metastasis and death in men withprogressive castration-resistant nonmetastatic prostate cancer. Can-cer 2011;117:2077-2085.

17. Titan SM, Zatz R, Graciolli FG, et al: FGF-23 as a predictor of renaloutcome in diabetic nephropathy. Clin J Am Soc Nephrol2011;6:241-247.

18. Irizarry MC, Webb DJ, Bains C, et al: Predictors of placebo groupdecline in the Alzheimer's disease Assessment Scale-cognitivesubscale (ADAS-Cog) in 24 week clinical trials of Alzheimer'sdisease. J Alzheimers Dis 2008;14:301-311.

19. Liu C, Blumhardt LD: Disability outcome measures in therapeutictrials of relapsing-remitting multiple sclerosis: effects of heteroge-neity of disease course in placebo cohorts. J Neurol NeurosurgPsychiatry 2000;6:450-457.

20. Hobson RW, Howard VJ, Roubin GS, et al: Howard G for theCREST Investigators. Carotid artery stenting is associated withincreased complications in octogenarians: 30-day stroke and deathrates in the CREST lead-in phase. J Vasc Surg 2004;40:1106-1111.

21. Belin RJ, Greenland P, Allison M, et al: Diet quality and the risk ofcardiovascular disease: the Women's Health Initiative (WHI). Am JClin Nutr 2011;94:49-57.

22. Thomas AJ, Eberly LE, Davey Smith G, et al: Race/ethnicity,income, major risk factors, and cardiovascular disease mortality. AmJ Public Health 2005;95:1417-1423.