Developing and Implementing Randomized Effectiveness Trials in General Medical Settings

© 2006 American Psychological Association. Published by Blackwell Publishing on behalf of the American Psychological Association.All rights reserved. For permissions, please email: [email protected]

53

Blackwell Publishing LtdOxford, UKCPSPClinical Psychology: Science and Practice0969-5893Clinical Psychology: Science and Practice, V13 N1, © American Psychological Association D12 2006; all rights reserved.131Original ArticleDEVELOPING EFFECTIVENESS TRIALS • Zatzick et al.CLINICAL PSYCHOLOGY: SCIENCE and PRACTICE · V13 N1, SPRING 2006

Developing and Implementing Randomized Effectiveness Trials in General Medical Settings

Douglas F. Zatzick, Gregory E. Simon, Amy W. Wagner, University of Washington School of Medicine

This article discusses the development of two mental

health services research programs: one targeting depres-

sion treatment in primary care and the other early inter-

vention for posttraumatic stress disorders in acute care.

Both programs have used the randomized effectiveness

trial to assess the delivery of empirically supported psycho-

therapeutic and psychopharmacologic interventions in

these general medical settings. This article explores the

fellowship training, clinical experiences, and conceptual

frameworks that have informed the progression of the

two research programs. Specific modifications to the

traditional randomized clinical trial design employed in

the effectiveness trials are discussed. This article con-

cludes with reflection on intervention development tra-

jectories and interdisciplinary team compositions that

may facilitate the development of mental health inter-

ventions that both derive from the best scientific evi-

dence and can be feasibly delivered in real-world

treatment settings.

Key words:

acute care, depression, health services

research, primary care, PTSD, randomized effectiveness

trial.

[Clin Psychol Sci Prac 13

:

53–68, 2006]

A substantial percentage of patients with mental dis-orders receive treatment in general medical settings.Over 25% of treatment-seeking depressed patients arecared for exclusively in the primary care medical setting

(Kessler et al., 2003). Similarly, acutely injured traumasurvivors who are at risk for the development of post-traumatic stress disorder (PTSD) initially present to acutecare medical settings (Zatzick, Roy-Byrne et al., 2004).Each year in the United States, approximately 2.5 millionAmericans incur injuries so severe that they requireemergent inpatient surgical admissions (Bonnie, Fulco,& Liverman, 1999).

Controlled trials have established the efficacy of psycho-therapeutic and psychopharmacologic interventions inthe treatment of depression and PTSD (Foa, Keane, &Friedman, 2000; Whooley & Simon, 2000). Guidelinesderived from these studies have been developed (Foaet al., 2000; Karasu, Gelenberg, Merriam, & Wang,2002; Public Health Service & Agency for Health CarePolicy and Research, 1993; Ursano et al., 2004). Despitethese advancements, it appears that many Americanswith these disorders either go untreated or do notachieve guideline-level treatments (National Institute ofMental Health, 1999; Southam-Gerow, Ringeisen, &Sherrill, 2006). A major difficulty is that state-of-the-art,empirically supported mental health treatments devel-oped by researchers are not being transferred to the real-world treatment sites such as general medical settings(Street, Niederehe, & Lebowitz, 2000).

Using the evolution of two mental health servicesresearch programs as examples, this article explores apromising set of methods for increasing the ecologicallyvalid use of evidence-based practices in medical settings.The two exemplar programs include one mature pro-gram focusing on adult patients presenting with depres-sive disorders to primary care medical clinics, and asecond developing program targeting the detection andtreatment of PTSD and related comorbidities in acute

Address correspondence to Douglas F. Zatzick, Universityof Washington, School of Medicine, Box 359911, HMC,325 Ninth Ave., Seattle, WA 98104. E-mail: [email protected].

CLINICAL PSYCHOLOGY: SCIENCE AND PRACTICE • V13 N1, SPRING 2006 54

care medical settings. In both programs, the randomizedclinical trial (RCT) design has been used to test theeffectiveness of multifaceted interventions that combineempirically supported interventions with care manage-ment strategies.

The first portion of this article examines the training,clinical experiences, and conceptual frameworks thathave informed the progression of the two health servicesresearch programs. The second portion of this article sum-marizes specific alterations in RCT design and imple-mentation employed in our effectiveness trials. This articleconcludes with reflection on intervention developmenttrajectories and interdisciplinary team compositions thatmay optimally facilitate mental health interventions that simul-taneously derive from the best scientific evidence and thatcan be robustly delivered in real-world treatment settings.

IMMERSION IN INTERDISCIPLINARY TRAINING AND

COLLABORATION AS A FOUNDATION FOR REAL-WORLD

INTERVENTION DEVELOPMENT

Intervention development in the health services researchprograms has emphasized an integration of academic andreal-world clinical viewpoints through the cultivationof clinician-investigators and interdisciplinary teams thatstraddle the two perspectives. After health servicesfellowship training, Dr. Simon (Health MaintenanceOrganization [HMO] primary care outpatient clinic)and Dr. Zatzick (level I trauma center consultation liaisonpsychiatry service) entered positions that allowed themto become immersed in the real-world treatment settingsin which they were to develop interventions researchprograms. Dr. Simon and Dr. Zatzick have simultane-ously functioned as frontline clinical providers, adminis-trators, and health services researchers in the primary andacute care medical settings.

Clinical immersion as frontline providers has allowedfor regular opportunities for case-based application ofpromising treatment strategies with patients presentingto general medical settings (Weisz, Chu, & Polo, 2004;Weisz, Southam-Gerow, Gordis, & Connor-Smith, 2003).During these routine patient encounters, members of theinterdisciplinary treatment development team (e.g., MDpsychiatrist and PhD psychologist clinician-investigatorsadapting empirically supported interventions for generalmedical settings) work together and collaboratively withother frontline providers in the provision of combined

medical and mental health treatment. The interdisciplinaryteam may have accumulated substantial context-specificexperience with a particular intervention approach beforeever fielding an effectiveness trial.

Another important implication of this immersion isthat, ultimately, the investigators will have to implementas practitioners any intervention they develop. Therefore,considerations such as the acceptability of new interven-tions to patients and the impact the new treatment will haveon practitioner caseloads weigh uniquely into decisionssurrounding intervention development. Implementationof novel interventions also inevitably entails discussionswith HMO and trauma center administrators regardingthe feasibility and costs of the new intervention.

Ongoing clinical and administrative interactionswithin the health plan and trauma center also facilitateinterdisciplinary research collaborations. By necessity, theresearch teams in these effectiveness trials include notonly MD, PhD, or MSW mental health investigators, butalso general medical providers. Decisions about clinicaltrial design and implementation and intervention develop-ment trajectories are often made collaboratively withinput from all team members.

PERSPECTIVES THAT HAVE INFORMED THE DEVELOPMENT OF

THE HEALTH SERVICES INTERVENTION PROGRAMS

Clinical Epidemiology

Clinical epidemiology is the science of making predic-tions about individual patients by describing clinical eventsin populations of patients (Fletcher, Fletcher, & Wagner,1996). The methods used by clinical epidemiologists aimto frame the care of the individual patient in the contextof the larger population of patients that present for carein a specified health service delivery setting (Feinstein,1985). Clinical epidemiologic methods encourage theassessment of a broad base of information derived fromdiverse research designs that include and extend beyondthe RCT (Feinstein, 1985; Fletcher et al., 1996).

Both health services research programs have developedinterventions that strive to attain external validity byaddressing the mental health needs of populations ofpatients presenting for treatment in primary and acutecare settings. One manner in which the primary and acutecare research programs have operationalized a clinicalepidemiologic approach is through the use of popula-tion-based automated data systems (Simon, Unutzer,

DEVELOPING EFFECTIVENESS TRIALS • ZATZICK ET AL. 55

Young, & Pincus, 2000). The population-based auto-mated data systems provide clinical and demographicinformation on all patients treated within the healthservice delivery system so that characteristics of an indi-vidual patient or subgroup of patients included in aninvestigation can be compared to the population ofpatients presenting for care.

Typically, a series of cross-sectional or prospectivecohort studies lead up to randomized effectiveness trials.These preparatory investigations use the data systems inconjunction with structured clinical assessments tocharacterize the demographic and clinical characteristicsand service utilization patterns of the populations to beexamined in the clinical trial (Unutzer et al., 1997;Zatzick, Jurkovich et al., 2004). For instance, prelimi-nary investigations have used automated medical recorddata to gain insight into the processes of care underlyingthe detection of patients with mental health symptoms/diagnoses (Zatzick et al., 2005), used the automated datasystems to identify clinical populations to be targeted ina clinical trial (Unutzer et al., 1997; Zatzick et al., inpress), and used the data systems to characterize popula-tion utilization and cost parameters (Simon et al., 1996;Zatzick et al., 2000).

With regard to the actual conduct of an effectivenesstrial, key aspects of screening, enrollment, and follow-upare captured by the automated data systems. Often, thefirst portion of the results section in a article will com-pare the demographic and clinical characteristics of thepopulation of patients presenting for care during thetime period of the study with the characteristics of studypatients (Zatzick, Roy-Byrne et al., 2004). The auto-mated data systems also provide key data related topolicy-relevant outcome domains, such as primary care andspecialty mental health service utilization (Simon, Lud-man, Tutty, Operskalski, & Von Korff, 2004), pharmacydata on medication refills (Katon et al., 1999; Simonet al., 1996; Simon et al., 2004), and emergency depart-ment and inpatient surgical utilization data documentingrecurrent injury admissions (Gentilello et al., 1999;Rivara, Koepsell, Jurkovich, Gurney, & Soderberg, 1993).

Clinically Applied Social Science

As with other areas of clinical research, health servicetraining emphasizes the relevance of a range of socialscience disciplines to clinical investigation and practice

(Eisenberg & Kleinman, 1981). This perspective is opera-tionalized in many health service fellowships by includ-ing PhD-level social and behavioral science consultants(e.g., anthropologists, health economists, psychologists)as core faculty. This core faculty provides mentorship tofellows by advising research design and analysis, and byattending regular work-in-progress research meetings.Ideally, this training imparts the perspective that multipleexplanatory frameworks may be required in order tounderstand the processes and outcomes of care relevantto both patients and providers in real-world treatmentsettings (Katon & Kleinman, 1981).

Attaining Policy Relevance by Incorporating a Quality of Care

Conceptual Framework

Health services research also focuses on methods forimproving and evaluating the quality of healthcare deliv-ery (Committee on Quality of Health Care in America,2001). The Institute of Medicine’s (IOM) 2001 report“Crossing the Quality Chasm” provides an excellentexample of a conceptual framework that aims to informthe development of high-quality treatments within theAmerican health-care system.

Developing empirically supported treatments (ESTs)that derive from the best available scientific knowledgeis one clear component of high-quality health care(Committee on Quality of Health Care in America,2001). Other domains of quality care include efficiency,timeliness, safety, providing equitable care, and develop-ing treatments that are patient-centered (throughout thisarticle, these quality of care domains will be defined andexamples will be provided).

The health services research questions that haveinformed the design and implementation of randomizedeffectiveness trials by the two research programs derivein part from quality of care conceptual frameworks. Forinstance, in an efficient health-care system, resources areoptimally used to attain the best value for each health-care expenditure (Committee on Quality of Health Carein America, 2001). In answering research questionspertinent to the efficiency criteria, the health servicesresearch programs have broadened clinical trial outcomeassessment beyond symptoms to include measures ofhealth service utilization and cost. The HMO depressionstudies have involved ongoing collaboration with healtheconomists and use of automated data systems to assess


the impact of combined interventions on the costs ofcare delivery (Simon et al., 1996; Simon, Von Korff,Rutter, & Wagner, 2000). The acute care research pro-gram has focused on the development of trauma registryinformation systems to track trauma center health ser-vice utilization and costs secondary to recurrent traumaticinjury (Gentilello et al., 1999; Rivara et al., 1993;Zatzick et al., 2000).

Similarly, the research questions driving a randomizedeffectiveness trial may be primarily focused on serviceutilization and cost outcomes (i.e., addressing questionsof relevance to the efficiency quality of care domain).For instance, by the late 1990s, a series of HMO effec-tiveness trials had established that combined interven-tions could significantly reduce depressive symptoms inintervention patients compared to patients receiving careas usual (Katon et al., 1999; Katon et al., 1995). Thecost-effectiveness of these multifaceted interventionstrategies remained unclear and pressures to reduce costsrelated to mental health expenditures remained a clearpolicy directive (Simon, Von Korff et al., 2000). There-fore, Simon and colleagues designed a dismantling studyof primary care provider pharmacotherapy that aimed toidentify the least costly method of delivering medicationcare for depression (Simon, Von Korff et al., 2000). Thisrandomized effectiveness trial addressed this policy-relevant question: Can a low-cost program using compu-terized records to provide feedback and advice to primarycare physicians substitute for a more expensive care man-agement program that utilized clinician-administeredtelephone assessments and feedback? The investigationfound that providing feedback without care managementyielded no significant benefits to depressed primary carepatients.

Altering the Design of the RCT in General Medical Settings in

Order to Address Questions of Relevance to a Health Services

Research Agenda

1

A body of RCT data now exists documenting the effec-tiveness of combined mental health interventions for pri-mary care patients with depressive spectrum disorders(Dietrich et al., 2004; Miranda et al., 2003; Schulberget al., 1996; Wells et al., 2000).

2

For over a decade, Dr.Simon, in collaboration with Dr. Wayne Katon at theUniversity of Washington, has conducted a series of ran-domized effectiveness trials of multifaceted psychothera-

peutic, psychopharmacologic, and care managementinterventions for patients with depressive disorders whoinitially present to primary care settings (Katon et al.,1995; Simon et al., 2004; Unutzer et al., 2002).

2

A seriesof articles review the specific features of the design andimplementation of these effectiveness trials (Katon &Gonzales, 1994; Simon, Wagner, & Von Korff, 1995).

2

In one of the first such reports, Simon et al. (1995)described the design of an RCT comparing pharma-cologic treatments for major depressive disorder in primarycare. The authors observed that efficacy studies ofdepression treatment might not generalize to everydaypatient care in primary care settings. Key design ele-ments of efficacy trials, such as the strict selection crite-ria, the recruitment of highly motivated patients, andrestrictive/nonadaptive treatment protocols that wereintended to minimize extraneous variations and isolatedifferences in outcome because of treatment assignment,limited the generalizability of efficacy trials to primarycare settings. Empirical investigations across diverse real-world service delivery settings have corroborated theseinitial observations (Simon et al., 2004; Southam-Gerow,Weisz, & Kendall, 2003; Weisz et al., 2003).

A number of characteristics of the acute care settingcreate a challenging environment for the delivery ofearly interventions for PTSD developed in efficacy trials(Litz, Gray, Bryant, & Adler, 2002). The standardizedregular appointments that constitute an implicit founda-tion of treatment delivery in efficacy trials may be dis-rupted as acute care patients move rapidly across surgicalinpatient, primary care outpatient, and communityhealth service delivery sectors in the days and weeksimmediately following an injury (Horowitz, Kassam-Adams, & Bergstein, 2001; Sabin, Zatzick, Jurkovich, &Rivara, in press). Acute care centers dedicate extensiveresources to the evaluation and triage of injured patients,but currently have sparse resources devoted to the promo-tion of care continuity or longitudinal mental health fol-low-up (Committee on Trauma American College ofSurgeons, in press). Medical record and self-report healthservice utilization data derived from prospective studiesof injured trauma survivors suggest that few acute carepatients attain mental health specialty care over thecourse of the year after an injury (Dunn et al., 2003;Jaycox, Marshall, & Schell, 2004; Sabin et al., in press;Zatzick et al., 2005).


Therefore, delivering interventions in primary andacute care required alterations to the traditional RCT;these alterations are discussed in the following text withspecific examples derived from primary care interven-tions targeting depression (Katon et al., 1995; Simonet al., 2004; Simon et al., 1996; Simon, Von Korff et al.,2000)

2

and acute care interventions targeting PTSD andrelated disorders (Gentilello et al., 1999; Zatzick, Roy-Byrne et al., 2004; Zatzick, Roy-Byrne et al., 2001).

Population-Based Screening and Minimal Exclusion Criteria

Efficacy trials frequently recruit from specialty mentalhealth clinics or advertisements and extensively screenpotential participants. For instance, early cognitivebehavioral therapy (CBT) intervention trials targetingPTSD have entailed selective enrollment and complexscreening procedures that require the transfer of treat-ment from acute care facilities to outpatient specialtymental health settings in the weeks and months immedi-ately following the trauma (Bisson, Shepherd, Joy, Prob-ert, & Newcombe, 2004; Bryant, Moulds, Guthrie,Dang, & Nixon, 2003; Ehlers et al., 2003).

2

The selective recruitment procedures and entry criteriaemployed in efficacy trials potentially exclude large sub-populations of patients. A substantial proportion of injuredtrauma survivors are alcohol or drug toxicology positive,meet structured clinical interview criteria for substance-related disorders at the time of the acute care admission,or both (Soderstrom et al., 1997; Zatzick, Jurkovichet al., 2004). Yet most (Bisson et al., 2004; Bryant et al.,2003; Foa, Hearst-Ikeda, & Perry, 1995),

2

but not all(Gillespie, Duffy, Hackmann, & Clark, 2002), early CBTintervention trials have excluded active substance abusers.

Because the clinical samples derived from selectiverecruitment practices employed in efficacy trials do notgeneralize to primary or acute care patients, the primaryand acute care research programs have altered effective-ness trial recruitment procedures to minimize exclusioncriteria and maximize generalizability. Acute care effec-tiveness trials targeting PTSD have had no substanceabuse exclusions (Zatzick, Roy-Byrne et al., 2004; Zatzick,Roy-Byrne et al., 2001). In many primary care depres-sion studies, potential participants are excluded only ifthe baseline assessment reveals clinical conditions thatclearly require different treatments (e.g., psychosis,mania). The primary and acute care effectiveness trials

avoid complicated screening procedures or delays inbeginning active treatment as they may burden providersand patients, resulting in loss of representative samples.

Alternative Service Delivery Models: Multifaceted Interventions

Both health services research programs have developedmultifaceted intervention models that combine empiri-cally supported psychotherapeutic and psychopharmaco-logic treatments and care management strategies. Theimpetus for developing combined interventions derivesfrom both applied and policy considerations.

The scope of impairment that primary and acute carepatients present with extends beyond symptomatic crite-ria traditionally addressed by interventions developed inefficacy studies (Kazdin, 1996). Primary and acute carepatients frequently present with two or more Axis I diag-noses. For example, epidemiologic studies suggest thatthe majority of patients with PTSD meet current or life-time criteria for one or more comorbid Axis I disorder(Kessler, Sonnega, Bromet, Hughs, & Nelson, 1995). Inthe acute care setting, patients with high PTSD symp-tom levels also demonstrate high levels of depressivesymptoms (O’Donnell, Creamer, & Pattison, 2004;Shalev et al., 1998; Zatzick, Jurkovich et al., 2004,Zatzick et al., in press) and comorbid substance abuse(Soderstrom et al., 1997; Zatzick, Jurkovich et al., 2004).

Primary and acute care patients frequently presentwith Axis I diagnoses (e.g., PTSD, major depression) inconjunction with comorbid medical conditions (e.g.,fractures, diabetes). Medical comorbidities, such as dia-betes in depressed patients or complications of traumaticinjury in acute care patients, require the development ofcare management strategies in order to engage patients,and support the delivery of psychotherapeutic and psycho-pharmacologic interventions (Katon et al., 2004). Bothhealth services research programs have incorporated “col-laborative care” disease management strategies as a keycomponent of the combined interventions (Von Korff,Gruman, Schaefer, Curry, & Wagner, 1997). Collabora-tive care models seek to find the optimal roles for gene-ral medical providers and mental health practitioners inthe delivery of care for patients with chronic medicaland psychiatric conditions in real-world treatment settings.

Collaborative care trials employ nurses, social workers,or other allied health professionals as care managers; caremanager interventionists are selected to be representative


of frontline clinical providers who would deliver theintervention under normal practice conditions. A keyfunction of the care managers is to enhance patientengagement in and adherence to longitudinal interven-tion protocols. Because primary and acute care patientsmay be less motivated to enter treatment than patientsenrolled in efficacy trials, the primary and acute careintervention programs have developed specific care man-agement techniques designed to facilitate treatmententry and adherence. Specific engagement techniquesemployed by care managers include intensive in-personand telephone outreach efforts, patient concern elicita-tion, shared patient–provider problem solving, ensuringcontinuity of care by facilitating attendance with sched-uled appointments, and close monitoring of treatmentadherence and outcomes (McKay & Bannon, 2004;Simon et al., 2004; Zatzick, Roy-Byrne et al., 2004). Inthese activities, care managers interface with patients andtheir families, acute and primary care providers, andmental health professionals. These care managementactivities are perhaps best conceptualized as treatmentenhancement techniques that are designed to engagepatients in longitudinal interventions and facilitate thedelivery of the empirically supported treatment compo-nents of the multifaceted interventions (Kazdin, 1996).

The effectiveness trials attempt to broadly address thisquestion: Can representative samples of primary and acutecare patients be engaged in care and receive cost-effectiveESTs that ultimately reduce symptoms and improvefunction? For example, Simon et al. (2004) comparedpatients receiving usual HMO primary care pharmaco-therapy for depression with standard pharmacotherapyaugmented by telephone care management and telephoneCBT targeting depressive symptoms. The care manage-ment program differed substantially from traditionalpsychotherapy not only in mode of delivery (telephoneversus face-to-face) but also in its much greater emphasison outreach and motivational enhancement. During tele-phone outreach, care managers used motivational inter-viewing (MI) interventions (Miller & Rollnick, 2002) toaddress ongoing protocol involvement and adherence tomedication and psychotherapy regimes. Compared tousual care pharmacotherapy, standard pharmacotherapyaugmented with the telephone care management/psychotherapy program was associated with significantreductions in depressive symptoms and significantly

improved satisfaction with depression care (Simon et al.,2004).

The acute care intervention trials have also developedcare management components designed to engage trau-matically injured patients in longitudinal interventions.These interventions address many of the previouslyreported difficulties in engaging and retaining acutelytraumatized patients in ongoing treatment (Roy-Byrneet al., 2004; Schwarz & Kowalski, 1992; Weisaeth, 2001).

2

In an effort to develop treatment engagement tech-niques that could be combined with empirically sup-ported interventions, qualitative and clinical epidemiologicmethods were combined in a population-based clinicalphenomenologic study of injured trauma survivors’ post-traumatic concerns (Zatzick, Kang et al., 2001). As hadbeen observed in other real-world treatment settings(Weisz et al., 2003), the concerns of injured patients andfamilies presenting to acute care extended beyond post-traumatic psychologic complaints to domains such asfuture impact on physical function and consequences forpatients’ work and finances (Zatzick, Kang et al., 2001).

Thus, in order to engage injured patients in earlyposttraumatic intervention, the concern elicitationprocedure was incorporated into postinjury care manage-ment. The care manager began the acute care inter-vention by meeting injured patients by the bedside inthe inpatient surgical ward and eliciting, tracking, andtargeting for improvement in each patient’s uniqueconstellation of posttraumatic concerns. Each patient’smost pressing posttraumatic concerns continued to beaddressed by the care manager over the course of the firstsix months after the injury. The care management proce-dure also contained a care coordination component thatserved to link trauma center services with primary careand community follow-up. The care management proce-dure effectively engaged and retained randomly sampledhospitalized surgical patients in the longitudinal protocol,so that they were later available to receive empiricallysupported CBT and pharmacotherapy targeting PTSD(Zatzick, Roy-Byrne et al., 2004). In our effectivenesstrials, both the care management and the empiricallysupported treatment components are manualized so thatfidelity may be assessed (Simon et al., 2004; Simon, VonKorff et al., 2000; Zatzick, Roy-Byrne et al., 2004).

Beyond applied considerations, such as patient engage-ment, policy considerations have also informed the


combining of interventions in the health services researchprograms. Quality of care conceptual frameworks sug-gest that developing mental health interventions forpopulations of acute care patients requires the incorpora-tion of specific intervention components that address thesafety criteria (Committee on Quality of Health Carein America, 2001). According to the IOM criteria, carethat is safe avoids and anticipates threats to patient wel-fare, such as recurrent traumatic injury. Guidelines forthe treatment of PTSD require that measures to ensurepatient safety, particularly in regard to recurrent trauma,be in place before trauma-focused psychotherapy; medi-cation intervention be initiated; or both (Foa et al.,2000; Resnick, Acierno, Holmes, Dammeyer, & Kil-patrick, 2000). Patients admitted to level I trauma centershave a markedly increased risk for recurrent traumaticlife events when compared to the general population(Ramstad, Russo, & Zatzick, 2004). Automated traumacenter health service utilization data suggest that up to10% of injured trauma survivors may incur recurrentinjuries requiring either an emergency department visitor inpatient hospital admission over the course of theyear after an index acute care injury admission (Gen-tilello et al., 1999; Rivara et al., 1993). Alcohol intoxica-tion at the time of the index trauma center admission isassociated with a 2.5-fold increased risk of recurrenttraumatic injury (Rivara et al., 1993). Therefore, inorder to adequately address safety criteria, acute caremental health interventions have combined empiricallysupported MI interventions targeting reductions inpostinjury alcohol consumption and recurrent traumaticinjury (Dunn, 2003; Gentilello et al., 1999) with empir-ically supported CBT and pharmacotherapy interven-tions targeting PTSD (Zatzick, Roy-Byrne et al., 2004).

Alternative Clinical Trial Designs

3

The health services research programs have altered thedesign of the RCT to include stepped-care proceduresand usual care control conditions. The IOM quality ofcare framework suggests that equitable care should notvary in quality because of individual demographic orclinical characteristics (Committee on Quality of HealthCare in America, 2001). Delivering equitable care at thelevel of the population suggests the introduction ofstepped-care procedures (Engel & Katon, 1999; Otto,Pollack, & Maki, 2000). Currently and for the foresee-

able future, resources devoted to mental health treatmentswill be limited in primary and acute care medical set-tings. Stepped care involves the allocation of increasinglygreater resources to patients who do not respond to ini-tial low-intensity interventions. As symptoms sponta-neously remit in a proportion of patients or as somepatients respond to low-intensity treatments, resourcesare increasingly dedicated in a stepped-wise fashion topatients with persistent symptoms.

In the outpatient medical treatment of depressedpatients, a substantial proportion of patients may respondto initial interventions delivered by primary care physi-cians (Katon et al., 1999). In acute care, prospectivecohort studies demonstrate that a substantial percentageof patients initially symptomatic with PTSD spon-taneously remit over the course of the year postinjury(Shalev et al., 1998). These observations suggest that theadaptation of a stepped-care approach to primary andacute care interventions could facilitate the equitable dis-tribution of limited mental health resources. A key issuein the development of stepped-care procedures is thedetermination of a priori decision rules and care path-way algorithms (Kazdin, 1996).

For example, in the most recent acute care randomizedeffectiveness trial, efficacious mental health interventionstargeting PTSD and alcohol abuse were added to a caremanagement procedure in a systematic, stepped manner(Zatzick, Roy-Byrne et al., 2004; Zatzick, Roy-Byrneet al. 2001). Care management began within days afterthe injury and continued for six months after the injury.Clearly defined decision algorithms dictated when carewas to be intensified. All patients who were alcohol tox-icology positive on admission or who at any point dur-ing the trial demonstrated postinjury alcohol abuse thatcould be considered hazardous and risked injury recur-rence, received stepped-up, evidence-based MI inter-vention targeting postinjury alcohol consumption(Zatzick, Roy-Byrne et al., 2004). An MSW-level clini-cian was trained by an expert PhD-level consultant todeliver the MI intervention. The MI intervention con-sisted of an initial surgical ward session followed bybooster sessions.

Three months after the injury, each interventionpatient was assessed with the Structured Clinical Inter-view for DSM-IV (SCID) PTSD module (Zatzick, Roy-Byrne et al., 2004). All patients who received a PTSD


diagnosis at this assessment were given their preferenceof CBT, pharmacotherapy, or combined CBT and medi-cation intervention. The empirically supported PTSDtreatments were delivered by the PhD psychotherapistand MD psychiatrist of the intervention team. Preference-driven intervention administration was consistent withthe patient-centered approach that aimed to engage andretain intervention patients in longitudinal post injurycare (Committee on Quality of Health Care in America,2001).

The primary research question was whether patientswho received the stepped-care intervention would man-ifest clinically and statistically significant improvementsin PTSD symptoms and alcohol abuse/dependence diag-noses when compared to control patients receiving careas usual. Intervention patients demonstrated clinicallyand statistically significant reductions in alcohol abuse/dependence and modest, statistically significant PTSDprevention when compared to control patients receivingusual care (Zatzick, Roy-Byrne et al., 2004). Significantprevention of the development of PTSD in interventionpatients was temporally associated with the initiation ofCBT and pharmacotherapy for PTSD three months postinjury. However, because the intervention was designedas a stepped-care procedure, the possibility that ongoingcare management activities also contributed to the pre-vention of PTSD cannot be ruled out.

Randomized effectiveness trials of primary and acutecare interventions typically employ usual care controlgroups. There are both policy and applied considerationsthat inform this choice. From a policy perspective, thecomparison of interest is how the intervention does inrelation to a control condition receiving usual care asdelivered in the community (Gold, Siegel, Russell, &Weinstein, 1996). From an applied standpoint, it is quitedifficult to blind patients and providers (i.e., to imple-ment double-blinded placebo-controlled designs) in real-world effectiveness trials. Blinding of patients and providersrequires that patients assigned to different treatmentsreceive indistinguishable care (e.g., equal numbers of visits).In effectiveness trials, attempts to enforce blinding mayobfuscate key outcomes including patterns of healthservice utilization, patient treatment adherence, and theacceptability of the intervention to providers. Althoughother features of the clinical trial frame may be altered ineffectiveness trials, randomized assignment to interven-

tion or usual care control conditions with allocationconcealment remains a core feature of the design.

Intervention Targets: Adapting Treatment Delivery to the Health

Care System

Interventions derived from efficacy trials conducted inmental health specialty settings may require adaptationto fit the characteristics of primary and acute carehealth service delivery systems. The process of adaptationbegins with clinical epidemiologic observation of pointsof patient contact with the healthcare system, descrip-tions of breakdowns in the recognition and treatment process,and characterization of the system’s greatest potentialpoints for care improvement (Sabin et al., in press).Therefore, our multifaceted interventions not only targetsymptomatic outcomes (e.g., depression and PTSD) butalso the processes of care delivery. Intervention compo-nents targeting the process of care delivery have beendirected toward patients, providers, and systems of care.

Intervention components targeting care processesderive in part from the observation that the nature andextent of patient–provider relationships that predate theclinical trial vary across service delivery sectors. In pri-mary care, most patients receive treatment in the contextof an ongoing relationship. In many of the Group HealthCooperatire primary care depression studies, patients arereferred by their primary care provider and the interven-tion is developed around this preexisting patient–providerrelationship. Patients are typically provided educationalmaterials designed to promote engagement in treatmentand improve treatment adherence (Katon et al., 1999).These materials emphasize effective collaboration withthe treating primary care physician. Intervention com-ponents targeting the provider have included practitionereducation surrounding depression care, case-basedteaching, and patient-specific feedback. Although pro-vider education and patient-specific feedback may beimportant elements of multifaceted interventions, expe-rience suggests that provider education alone (Lin et al.,1997) or feedback alone (Simon, Von Korff et al., 2000)is not sufficient to improve patient outcomes. Mentalhealth specialists (e.g., psychiatrists) have been availablein some of the HMO interventions to consult with pri-mary care providers regarding general depression treatmentapproaches or the specifics of psychotropic medicationprescription or both (Katon et al., 1995).


Acute care presents an alternative extreme. Acute carepatients may not have consistent preexisting contactwith a primary healthcare provider (Sabin et al., inpress). Acute care visits are unplanned, so when patientsdo present with preexisting providers, these practi-tioners may be only distantly involved in the injury careepisode.

Therefore, one process of care target in the acutecare interventions has been working to ensure thatinjured patients have adequate primary care and com-munity follow-up after they are discharged from thetrauma center. The acute care interventions have devel-oped trauma center-to-community linkage teams (Wells,Miranda, Bruce, Alegria, & Wallerstein, 2004). Theseteams are comprised of both trauma center and commu-nity providers who are knowledgeable about or involvedin community services that may be of benefit to hetero-geneous injured trauma survivors (e.g., communityresources for homeless patients, programs that specializein healthcare delivery for first-generation Americans,community-based traumatic brain injury support groups,pastoral care referrals for patients that prefer theseservices or wish to augment traditional mental healthservices). In working to improve care coordination andlinkage, care managers regularly consult with/refer tomembers of the trauma center-to-community linkageteam. Early posttraumatic care coordination activitiesfocus on the establishment of linkages. Later interven-tion components then work with community providersto ensure that either empirically supported PTSD inter-ventions are appropriately initiated or, because linkagescan take weeks or even months to solidify (Zatzick,Kang et al., 2001), empirically supported PTSD inter-ventions initiated by the acute care team are appropri-ately monitored and maintained.

Assessing Symptomatic and Policy-Relevant Outcomes

Incorporating a health services research agenda that aimsto develop high-quality treatments for populations ofpatients treated in general medical settings has requiredthe assessment of a broad profile of outcomes thatinclude, yet extend beyond, symptomatic improvement.The primary and acute care RCTs not only addressquestions related to symptomatic improvements but alsoquestions related to engagement in and adherence tocare, cost-effectiveness, and impairment in role, social,

or physical function domains (Katon et al., 1999; Linet al., 2003; Simon et al., 2004).

As with efficacy trials, outcome data collection isconducted independently of the clinical intervention onthe intent-to-treat sample. Automated health serviceutilization data accrues on the intent-to-treat sampleregardless of whether patients remain in contact withthe independent outcome assessment team of theinvestigation.

IMPLICATIONS FOR INTERVENTION DEVELOPMENT

TRAJECTORIES AND INTERDISCIPLINARY COLLABORATION

We acknowledge that our implementation of primaryand acute care RCTs that test the effectiveness of multi-faceted interventions and address questions of relevanceto a health services research agenda can leave importantscientific issues surrounding the combining of treatmentsunderexamined (Kazdin, 1996, 2001). The combining ofempirically supported psychotherapeutic and psycho-pharmacologic treatments with each other, and thenwith novel care management strategies, raises a series ofquestions regarding active treatment components, opti-mal treatment combinations, and intervention parame-ters that can be varied to improve specific outcomes(Kazdin, 1996, 2001).

Given the historical progression of mental healthintervention development, we believe the rationale fordeveloping and implementing combined treatments iswell justified. Combined or complex treatments mayhave greater value in everyday practice than in efficacytrial settings. For example, the Agency for Health CarePolicy (Public Health Service & Agency for Health CarePolicy and Research, 1993) and American PsychiatricAssociation (Karasu et al., 2002) guidelines for manage-ment of depression do not recommend combining anti-depressant medication and psychotherapy as a first-linetreatment except for patients with more severe, chronic,or complicated depression. This recommendation wasbased on efficacy studies finding that combined treat-ment offered little benefit over medication alone in thispopulation (Public Health Service & Agency for HealthCare Policy and Research, 1993). Those trials, however,found that psychotherapy did not add significantly toantidepressant treatment as provided in efficacy trials(e.g., motivated patients, frequent visits, expert clini-cians). Effectiveness trials, however, have found that brief


psychotherapies clearly add to the antidepressant treat-ment typically provided in primary care (Simon et al.,2004), even for patients with moderate depression. A keyconclusion to be drawn from this and other real-worldinvestigations (Weisz et al., 2003; Zatzick, Roy-Byrneet al., 2004) is that, unlike the highly motivated patientsrecruited into efficacy trials, patients presenting to generalmedical settings may require care management strategiesor other psychosocial treatments that focus on outreach,engagement, and treatment adherence.

Reflection on this issue leads to a reconsiderationof assumptions about the trajectory of mental healthintervention development. In the past, mental healthintervention development has been conceptualized as anatural, unidirectional progression from efficacy studiesthrough effectiveness and dissemination akin to the med-ical–pharmaceutical model of intervention development(Weisz et al., 2004). Recent commentary suggests thatdecisions about the most effective progression of mentalhealth intervention development may be empiric (Addis,2002) and that methodologically diverse investigationsthat oscillate across efficacy, effectiveness, and dissemina-tion paradigms may optimally inform intervention devel-opment (Kazdin, 2001; National Institute of MentalHealth, 1999; Rounsaville, Carroll, & Onken, 2001;Schoenwald & Hoagwood, 2001). In this continuum ofintervention development, our health services researchprograms can be seen to have predominantly focused onusing the RCT to address questions of whether effica-cious ESTs can have a beneficial effect when transportedto real-world primary and acute care settings (NationalInstitute of Mental Health, 1999; Schoenwald & Hoag-wood, 2001; Street et al., 2000).

Reconceptualization of intervention developmenttrajectories would suggest at least two potential direc-tions for future investigations. A series of questionsremain regarding variations in patient, provider, organi-zational, and service delivery system characteristics thatmay influence the dissemination of combined interven-tions with proven effectiveness (Schoenwald & Hoag-wood, 2001). Previous commentary has suggested thedevelopment of representative sampling frames to assessprovider and service delivery system characteristics(Norquist, 2001; Wells, 1999). Such studies could beginto address this important investigative question: Howdoes the “individual” site in which the intervention was

developed differ from the “population” of sites or sys-tems that will ultimately be available for interventiondissemination? As an example, there is a limited number(i.e., 1,154) of trauma centers in the United States; inan effort to enhance national disaster preparedness, sys-tematic description of these acute care centers and thetrauma care systems to which they are linked have begun(MacKenzie et al., 2003). Representative samplingprocedures

4

could be employed in order to further char-acterize variability in patient, provider, organizational,and service system parameters pertinent to the deliveryof mental health interventions (Hemmelgarn, Glisson, &Dukes, 2001; Schoenwald & Hoagwood, 2001). Theseinitial descriptive investigations could serve to identifycharacteristics of trauma care systems that might requiremodification in order to facilitate the broader dissemina-tion of the combined interventions developed and testedin previous randomized effectiveness trials.

A second direction for intervention development is areturn to more highly controlled investigations that seekto determine key components of the combined inter-vention packages. We admire elegantly designed efficacytrials that have used theoretically driven rationales tocombine and dismantle interventions with the aim ofidentifying active treatment components (Bryant et al.,2003; Foa et al., 1999; Jacobson et al., 1996). We believethe randomized effectiveness trial design has the poten-tial to address questions with this degree of refinement(Simon, Von Korff et al., 2000). Subsequent hybridefficacy/effectiveness designs may provide importantanswers to questions that arise when psychotherapeutic,psychopharmacologic, and care management interven-tions are combined into multifaceted interventions anddelivered in real-world settings.

We are aware that, as health services researchers, ourpredilection may be to use the RCT to ask researchquestions that directly and expediently address issuesrelated to the quality and costs of real-world interventiondelivery, yet bypass important theoretical considerations.Here a key question is how to optimally construct inter-disciplinary research groups that can work collaborativelyto bridge research paradigms and methods (Gonzales,Ringeisen, & Chambers, 2002; Pellmar & Eisenberg,2000). In the primary and acute care research pro-grams, clinical psychologists form an integral part of inter-disciplinary research teams that also include health


services researchers, psychopharmacologists, social scientists,patient advocates, epidemiologists, and internist, pedi-atric, and trauma surgical physician investigators. Clini-cal psychologists are the team members who oversee themodification and delivery of empirically supported psycho-therapeutic interventions in these randomized effective-ness trials. Psychologist clinician-investigators are wellpositioned to develop hybrid investigations that integratequestions of theoretical importance surrounding com-bined intervention development in real-world settings(King, 2004).

As an example, Amy Wagner, PhD, has been the clin-ical psychologist collaborator overseeing the CBT inter-vention component of the acute care health serviceprogram’s multifaceted intervention (Wagner, 2003).Over the past four years, as a member of the interdisci-plinary acute care research team, she has been a strongadvocate of theory-based intervention development andadherence to empirically supported, manualized treat-ment protocols. At the same time, aspects of the treat-ment setting and patient population have requiredalterations in the delivery of standard CBT for PTSD(e.g., flexibility around scheduling, conducting sessionsin patients’ homes or over the telephone, assessing andproblem-solving obstacles unique to injury survivorssuch as pain and immobility, frontline trauma centerprovider unfamiliarity with manualized psychotherapyprotocols).

Dr. Wagner began the collaboration as an outpatientprovider in a university-based clinic. Motivated in partby a desire to obtain a more immersive clinical under-standing of the acute care health service delivery con-text, she has moved her primary practice site to our levelI trauma center. A product of these collaborative experi-ences is the initiation of a pilot hybrid RCT of earlyCBT intervention for PTSD and depression in acutelyinjured trauma survivors (Wagner, 2003). This hybridefficacy/effectiveness RCT combines characteristics ofthe acute care effectiveness trials (i.e., a population-basedsampling procedure that utilizes automated trauma regis-try information systems to screen potential participantsand initial patient contact/recruitment by the bedsidein the surgical ward) with procedure characteristic ofefficacy trials (selective entry criteria, including a studyentry requirement of PTSD as diagnosed by structuredclinical interview and treatment delivery by Dr. Wagner,

an expert PhD-level clinician). Outcomes are not yetavailable; however, the aim is to continue working intandem with the interdisciplinary team and modifying thecomponents of the combined intervention in accordancewith the outcomes of this, and future, component analyses.

CONCLUSION

The goal of this review has been to make transparent thetraining, methodologic emphasis, and conceptual frame-works that have informed the design and implementa-tion of our randomized effectiveness trials. We believethat ongoing interdisciplinary collaboration may lead tothe development of mental health interventions thatsimultaneously derive from the best scientific evidenceand are acceptable to populations of patients presentingfor treatment in real-world settings, readily implementedby the spectrum of general medical and specialty mentalhealth providers, and sustainable at an institutional andsocietal level.

NOTES

1. The health services research programs use CONSORTstatement flow diagrams to describe patient flow through ran

-

domized effectiveness trials (Begg et al., 1996). In the generalmedical literature and psychiatric literature, the CONSORTflow diagram has become a “gold standard” for reportingdesign and flow through a randomized trial. Recent commen-tary from clinical psychologists conducting efficacy studies hasencouraged broader use of the CONSORT recommendations(Harvey, Bryant, & Tarrier, 2003; McGrath, Stinson, & David-son, 2003). Particularly as hybrid efficacy/effectiveness clinicaltrial designs become more common, we believe that cleardescriptions of trial parameters and modifications systemati-cally represented in such diagrams will be invaluable for con-veying key aspects of trial designs and for making comparisonsacross trials.

2. Throughout the article, we have cited literature illustra-tively rather than exhaustively for the sake of efficiency. Doingso has excluded important and high-quality citations. A moredetailed bibliography is available from the first author of thisarticle.

3. The review does not highlight many RCT design anddata analytic issues that confront real-world effectiveness trials.There are under development novel clinical trial designs thathold promise for enhancing inferences that can be derived fromeffectiveness studies (TenHave, Coyne, Salzer, & Katz, 2003).Also, procedures for the appropriate analyses of longitudinalRCT data have been developed (Gibbons et al., 1993) and applied


in our effectiveness trials (Simon, Von Korff et al., 2000;Zatzick, Roy-Byrne et al., 2004), but will not receive detailedattention in this review.

4. Insights derived from epidemiologic approaches/repre-sentative sampling procedures may have currently underappre-ciated implications for the medical–pharmaceutical interventiondevelopment trajectory as well. For example, a diverse group ofcandidate medications are being considered as early interven-tion agents in the secondary prevention of PTSD (Friedman,2002). The rationale for initial trials of corticosteroids (Schell-ing et al., 2004), beta-adrenergic antagonists (Pitman et al.,2002), and analgesics (Saxe et al., 2001) has been articulated.Initial reviews of population-based data on routine acute caremedication reveal high frequencies of analgesic prescription atthe time of hospital discharge, consistent with high rates ofself-reported physical health and bodily pain concerns immedi-ately following an injury (Zatzick, Kang et al., 2001). Theseobservations would favor initial feasibility tests/efficacy trials ofcompounds targeting pain or even direct basic researchers todevelop compounds that simultaneously target pain and anxi-ety. In this way, population-based clinical data derived fromreal-world practice settings could enhance the efficiency (e.g.,selection of compounds more feasibly applied to real-worldtreatment settings) and trajectory (e.g., basic research on promis-ing compounds) of pharmaceutical intervention developmentin the medical–pharmaceutical model.

ACKNOWLEDGMENT

Supported by National Institute of Mental Health grantsMH01610 and MH51338. The authors appreciate the contri-bution of the special issue editors Michael Southam-Gerow,Heather Ringeisen, and Joel Sherrill in developing this article.The authors thank Joan Russo and Barbara McCann for help-ful review of a version of this article.

REFERENCES

Addis, M. E. (2002). Methods for disseminating research prod-ucts and increasing evidence-based practice: Promises,obstacles, and future directions.

Clinical Psychology: Scienceand Practice

,

9

, 367–378.Begg, C., Cho, M., Eastwood, S., Horton, R., Moher, D.,

Olkin, I., et al. (1996). Improving the quality of reportingof randomized controlled trials: The CONSORT statement.

Journal of the American Medical Association

,

276

, 637–639.Bisson, J. I., Shepherd, J. P., Joy, D., Probert, R., & New-

combe, R. G. (2004). Early cognitive–behavioural therapyfor post-traumatic stress symptoms after physical injury.Randomised controlled trial.

British Journal of Psychiatry

,

184

, 63–69.

Bonnie, R. J., Fulco, C. E., & Liverman, C. T. (Eds.). (1999).

Reducing the burden of injury: Advancing prevention and treat-ment

. Washington, DC: National Academy Press.Bryant, R. A., Moulds, M. L., Guthrie, R. M., Dang, S. T., &

Nixon, R. D. (2003). Imaginal exposure alone and imagi-nal exposure with cognitive restructuring in treatment ofposttraumatic stress disorder.

Journal of Consulting and Clini-cal Psychology

,

71

, 706–712.Committee on Quality of Health Care in America. (2001).

Crossing the quality chasm: A new health system for the 21stcentury.

Washington, DC: National Academy Press.Committee on Trauma American College of Surgeons.

(in press). Performance improvement. In

Resources for theoptimal care of the injured patient

. USA: The American Col-lege of Surgeons.

Dietrich, A. J., Oxman, T. E., Williams, J. W., Schulberg, H. C.,Bruce, M. L., Lee, P. W., et al. (2004). Re-engineeringsystems for the treatment of depression in primary care:Cluster randomised controlled trial.

British Medical Journal

,

329

(7466), 602–608.Dunn, C. (2003). Brief motivational interviewing interven-

tions targeting substance abuse in the acute care medicalsetting.

Seminars in Clinical Neuropsychiatry

,

8

(3), 188–196.Dunn, C., Zatzick, D., Russo, J., Rivara, F. P., Roy-Byrne, P.,

Ries, R., et al. (2003). Hazardous drinking by traumapatients during the year after injury.

Journal of Trauma:Injury,

Infection, and

Critical Care

,

54

(4), 707–712.Ehlers, A., Clark, D., Hackmann, A., McManus, F., Fennell,

M., Herbert, C., et al. (2003). A randomized controlledtrial of cognitive therapy, a self-help booklet, and repeatedassessment as early interventions for posttraumatic stress dis-order.

Archives of General Psychiatry

,

60

, 1024–1032.Eisenberg, L., & Kleinman, A. (1981). Clinical social science.

In L. Eisenberg & A. Kleinman (Eds.),

The relevance of socialscience for medicine

(pp. 1–26). The Netherlands: D. ReidelPublishing Co.

Engel, C. C., & Katon, W. (1999). Population and need-basedprevention of unexplained physical symptoms in the com-munity. In Institute of Medicine Special Emphasis Panel onthe Health of Deployed Forces (Ed.),

Institute of medicine,strategies to protect the health of U.S. forces: Medical surveillance,record taking and risk reduction

(pp. 173–212). Washington,DC: National Academy Press.

Feinstein, A. R. (1985).

Clinical epidemiology: The architecture ofclinical research

. Philadelphia, PA: W. B. Saunders Co.Fletcher, R. H., Fletcher, S. W., & Wagner, E. H. (1996).

Clinical epidemiology: The essentials

(3rd. ed.). Baltimore,MD: Williams & Wilkins.

Foa, E. B., Dancu, C. V., Hembree, E. A., Jaycox, L. H.,Meadows, E. A., & Street, G. P. (1999). A comparison of


exposure therapy, stress inoculation training, and their com-bination for reducing posttraumatic stress disorder in femaleassault victims.

Journal of Consulting and Clinical Psychology

,

67

, 194–200.Foa, E. B., Hearst-Ikeda, D., & Perry, K. J. (1995). Evaluation

of a brief cognitive–behavioral program for the preventionof chronic PTSD in recent assault victims.

Journal of Con-sulting and Clinical Psychology

,

63

(6), 948–955.Foa, E. B., Keane, T. M., & Friedman, M. J. (2000).

Effectivetreatments for PTSD: Practice guidelines from the InternationalSociety for Traumatic Stress studies

. New York: Guilford Press.Friedman, M. J. (2002). Future pharmocotherapy for post-

traumatic stress disorder: Prevention and treatment.

Psychi-atric Clinics of North America

,

25

, 427–441.Gentilello, L. M., Rivara, F. P., Donovan, D. M., Jurkovich,

G. J., Daranciang, E., Dunn, C. W., et al. (1999). Alcoholinterventions in a trauma center as a means of reducing therisk of injury recurrence.

Annals of Surgery

,

230

(4), 473–480.Gibbons, R. D., Hedeker, D., Elkin, I., Waternaux, C.,

Kraemer, H. C., Greenhouse, J. B., et al. (1993). Someconceptual and statistical issues in analysis of longitudinalpsychiatric data.


,

50

, 739–750.Gillespie, K., Duffy, M., Hackmann, A., & Clark, D. M.

(2002). Community based cognitive therapy in the treat-ment of posttraumatic stress disorder following the Omaghbomb.

Behaviour Research and Therapy

,

40

(4), 345–357.Gold, M. R., Siegel, J. E., Russell, L. B., & Weinstein, M. C.

(Eds.). (1996).

Cost-effectiveness in health and medicine.

NewYork: Oxford University Press.

Gonzales, J. J., Ringeisen, H., & Chambers, D. A. (2002). Thetangled and thorny path of science to practice: Tensions ininterpreting and applying “evidence.”

Clinical Psychology:Science and Practice

,

9

, 204–209.Harvey, A. G., Bryant, R. A., & Tarrier, N. (2003). Cognitive

behaviour therapy for posttraumatic stress disorder.

ClinicalPsychology Review

,

23

(3), 501–522.Hemmelgarn, A. L., Glisson, C., & Dukes, D. (2001). Emer-

gency room culture and the emotional support componentof family-centered care.

Children’s Health Care

,

30

(2), 93–110.Horowitz, L., Kassam-Adams, N., & Bergstein, J. (2001).

Mental health aspects of emergency medical services forchildren: Summary of a consensus conference.

AcademicEmergency Medicine

,

8

(12), 1187–1196.Jacobson, N. S., Dobson, K. S., Truax, P. A., Addis, M. E.,

Koerner, K., Gollan, J. K., et al. (1996). A componentanalysis of cognitive–behavioral treatment for depression.

Journal of Consulting and Clinical Psychology

,

64

, 295–304.Jaycox, L. H., Marshall, G. N., & Schell, T. (2004). Use of

mental health services by men injured through communityviolence.

Psychiatric Services

,

55

(4), 415–420.

Karasu, T. B., Gelenberg, A., Merriam, A., & Wang, P. (2002).Practice guideline for the treatment of patients with majordepressive disorder. In

Practice guidelines for the treatment ofpsychiatric disorders

(2nd ed., pp. 463–634). Washington,DC: American Psychiatric Association.

Katon, W., & Gonzales, J. (1994). A review of randomized tri-als of psychiatric consultation-liaison studies in primarycare.

Psychosomatics

,

35

, 268–278.Katon, W., & Kleinman, A. (1981). Doctor–patient negotia-

tion and other social science strategies in patient care. In L.Eisenberg & A. Kleinman (Eds.),

The relevance of social sciencefor medicine

(pp. 253–279). The Netherlands: D. ReidelPublishing Co.

Katon, W., Von Korff, M., Lin, E., Simon, G., Walker, E.,Unutzer, J., et al. (1999). Stepped collaborative care for pri-mary care patients with persistent depression: A randomizedtrial.


,

56

, 1109–1115.Katon, W., Von Korff, M., Lin, E., Walker, E., Simon, G. E.,

Bush, T. B., et al. (1995). Collaborative management toachieve treatment guidelines: Impact on depression in pri-mary care.

Journal of the American Medical Association

,

273

,1026–1031.

Katon, W., Von Korff, M., Lin, E. H., Simon, G., Ludman, E.,Russo, J., et al. (2004). The pathways study: A randomizedtrial of collaborative care in patients with diabetes anddepression.


,

61

, 1042–1049.Kazdin, A. (1996). Combined and multimodal treatments in

child and adolescent psychotherapy: Issues, challenges, andresearch directions.

Clinical Psychology: Science and Practice

,

3

,69–100.

Kazdin, A. (2001). Progression of therapy research and clinicalapplication of treatment require better understanding of thechange process.

Clinical Psychology: Science and Practice

,

8

(2),143–151.

Kessler, R. C., Berglund, P. A., Demler, O., Jin, R., Koretz,D., Merikangas, K. R., et al. (2003). The epidemiology ofmajor depressive disorder: Results from the NationalComorbidity Survey Replication (NCS-R).

Journal of theAmerican Medical Association

,

289

(23), 3095–3105.Kessler, R. C., Sonnega, A., Bromet, E., Hughs, M., &

Nelson, C. B. (1995). Posttraumatic stress disorder in theNational Comorbidity Survey.


,

52

, 1048–1060.King, C. A. (2004). Psychologists in academic health settings:

Key contributors to dynamic interplay among research,clinical practice, and policy domains.

Journal of Clinical Psy-chology in Medical Settings

,

11

(2), 83–90.Lin, E. H., Katon, W., Von Korff, M., Tang, L., Williams, J. W. Jr,

Kroenke, K., et al. (2003). Effect of improving depressioncare on pain and functional outcomes among older adults


with arthritis: A randomized controlled trial.

Journal of theAmerican Medical Association

,

290

(18), 2428–2429.Lin, E. H., Katon, W. J., Simon, G., Von Korff, M., Bush, T. M.,

Rutter, C. M., et al. (1997). Achieving guidelines for thetreatment of depression in primary care: Is physician educa-tion enough?

Medical Care

,

35

, 831–842.Litz, B. T., Gray, M. J., Bryant, R. A., & Adler, A. B. (2002).

Early interventions for trauma: Current status and futuredirections. Clinical Psychology: Science and Practice, 9, 112–134.

MacKenzie, E. J., Hoyt, D. B., Sacra, J. C., Jurkovich, G. J.,Carlini, A. R., Teitelbaum, S. D., et al. (2003). Nationalinventory of hospital trauma centers. Journal of the AmericanMedical Association, 289(12), 1515–1522.

McGrath, P. J., Stinson, J., & Davidson, K. (2003). Commen-tary: The Journal of Pediatric Psychology should adopt theCONSORT statement as a way of improving the evidencebase in pediatric psychology. Journal of Pediatric Psychology,28(3), 169–171.

McKay, M. M., & Bannon, W. M. (2004). Engaging familiesin child mental health services. Child and Adolescent Psychia-tric Clinics of North America, 13(4), 905–921, vii.

Miller, W. R., & Rollnick, S. (2002). Motivational interviewing:Preparing people for change (2nd ed.). New York: GuilfordPress.

Miranda, J., Chung, J. Y., Green, B. L., Krupnick, J.,Siddique, J., Revicki, D. A., et al. (2003). Treating depres-sion in predominantly low-income young minoritywomen: A randomized controlled trial. Journal of the Ameri-can Medical Association, 290(1), 57–65.

National Institute of Mental Health. (1999). Bridging science andservice: A report by the National Advisory Mental HealthCouncil’s Clinical Treatment and Services Research Workgroup(No. 99–4353). National Institutes of Health.

Norquist, G. (2001). Practice research networks: Promisesand pitfalls. Clinical Psychology: Science and Practice, 8, 173–175.

O’Donnell, M. L., Creamer, M., & Pattison, P. (2004). Post-traumatic stress disorder and depression following trauma:Understanding comorbidity. American Journal of Psychiatry,161, 1390–1396.

Otto, M. W., Pollack, M. H., & Maki, K. M. (2000). Empiri-cally supported treatments for panic disorder: Costs, bene-fits, and stepped care. Journal of Consulting and ClinicalPsychology, 68(4), 556–563.

Pellmar, T. C., & Eisenberg, L. (Eds.). (2000). Bridging disci-plines in the brain, behavioral, and clinical sciences. Washington,DC: National Academy Press.

Pitman, R. K., Sanders, K. M., Zusman, R. M., Healy, A. R.,Cheema, F., Lasko, N. B., et al. (2002). Pilot study ofsecondary prevention of posttraumatic stress disorder withpropranolol. Biological Psychiatry, 51(2), 189–192.

Public Health Service, & Agency for Health Care Policy andResearch. (1993). Depression in primary care: Treatment ofmajor depression (Vol. 2). Rockville, MD: U.S. Departmentof Health and Human Services.

Ramstad, S. M., Russo, J., & Zatzick, D. F. (2004). Is it anaccident? Recurrent traumatic life events in level I traumacenter patients compared to the general population. Journalof Traumatic Stress, 17(6), 529–534.

Resnick, H., Acierno, R., Holmes, M., Dammeyer, M., &Kilpatrick, D. (2000). Emergency evaluation and interven-tion with female victims of rape and other violence. Journalof Clinical Psychology, 56(10), 1317–1333.

Rivara, F. P., Koepsell, T. D., Jurkovich, G. J., Gurney, J. G.,& Soderberg, R. (1993). The effects of alcohol abuse onreadmission for trauma. Journal of the American Medical Asso-ciation, 270(16), 1962–1964.

Rounsaville, B., Carroll, K. M., & Onken, L. S. (2001).Methodological diversity and theory in the stage model:Reply to Kazdin. Clinical Psychology: Science and Practice, 8,152–154.

Roy-Byrne, P. P., Russo, J., Michelson, E., Zatzick, D.,Pitman, R. K., & Berliner, L. (2004). Risk factors and outcomein ambulatory assault victims presenting to the acute emergencydepartment setting: Implications for secondary preventionstudies in PTSD. Depression and Anxiety, 19(2), 77–84.

Sabin, J., Zatzick, D., Jurkovich, G., & Rivara, F. (in press).Primary care utilization and detection of emotional distressfollowing adolescent traumatic injury: Identifying an unmetneed. Pediatrics.

Saxe, G., Stoddard, F., Courtney, D., Cunningham, K.,Chawla, N., Sheridan, R., et al. (2001). Relationshipbetween acute morphine and the course of PTSD in chil-dren with burns. Journal of the American Academy of Child &Adolescent Psychiatry, 40(8), 915–921.

Schelling, G., Kilger, E., Roozendaal, B., de Quervain, D. J.,Briegel, J., Dagge, A., et al. (2004). Stress doses of hydro-cortisone, traumatic memories, and symptoms of post-traumatic stress disorder in patients after cardiac surgery: Arandomized study. Biological Psychiatry, 55(6), 627–633.

Schoenwald, S. K., & Hoagwood, K. (2001). Effectiveness,transportability, and dissemination of interventions: Whatmatters when? Psychiatric Services, 52(9), 1190–1197.

Schulberg, H. C., Block, M. R., Madonia, M. J., Scott, C. P.,Rodriguez, E., Imber, S. D., et al. (1996). Treating majordepression in primary care practice: Eight-month clinicaloutcomes. Archives of General Psychiatry, 53(10), 913–919.

Schwarz, E. D., & Kowalski, J. M. (1992). Malignant memo-ries: Reluctance to utilize mental health services after adisaster. The Journal of Nervous and Mental Disease, 180(12),767–772.


Shalev, A. Y., Freedman, S., Peri, T., Brandes, D., Sahar, T.,Orr, S. P., et al. (1998). Prospective study of posttraumaticstress disorder and depression following trauma. AmericanJournal of Psychiatry, 155, 630–637.

Simon, G. E., Ludman, E. J., Tutty, S., Operskalski, B., & VonKorff, M. (2004). Telephone psychotherapy and telephonecare management for primary care patients starting anti-depressant treatment: A randomized controlled trial. Journalof the American Medical Association, 292(8), 935–942.

Simon, G. E., Unutzer, J., Young, B. E., & Pincus, H. A.(2000). Large medical databases, population-based research,and patient confidentiality. American Journal of Psychiatry,157, 1731–1737.

Simon, G. E., Von Korff, M., Heiligenstein, J. H., Revicki, D.A., Grothaus, L., Katon, W., et al. (1996). Initial anti-depressant choice in primary care: Effectiveness and cost offluoxetine vs. tricyclic antidepressants. Journal of the AmericanMedical Association, 275(24), 1897–1902.

Simon, G. E., Von Korff, M., Rutter, C., & Wagner, E. H.(2000). Randomised trial of monitoring, feedback, andmanagement of care by telephone to improve treatment ofdepression in primary care. British Medical Journal,320(7234), 550–554.

Simon, G. E., Wagner, E., & Von Korff, M. (1995). Cost-effectiveness comparisons using “real world” randomizedtrials: The case of new antidepressant drugs. Journal of Clini-cal Epidemiology, 48(3), 363–373.

Soderstrom, C. A., Smith, G. S., Dischinger, P. C., McDuff,D. R., Hebel, J. R., Gorelick, D. A., et al. (1997). Psycho-active substance use disorders among seriously injuredtrauma center patients. Journal of the American Medical Associ-ation, 277(22), 1769–1774.

Southam-Gerow, M. A., Ringeisen, H. L., & Sherrill, J. T.(2006). Integrating interventions and services research:Progress and prospects. Clinical Psychology: Science and Prac-tice, 13, 1–8.

Southam-Gerow, M. A., Weisz, J. R., & Kendall, P. C. (2003).Youth with anxiety disorders in research and service clinics:Examining client differences and similarities. Journal of Clin-ical Child and Adolescent Psychology, 32(3), 375–385.

Street, L. L., Niederehe, G., & Lebowitz, B. D. (2000).Toward greater public health relevance for psycho-therapeutic interventions research: A NIMH workshopreport. Clinical Psychology: Science and Practice, 7, 127–137.

Ten Have, T. R., Coyne, J. C., Salzer, M., & Katz, I. (2003).Research to improve the quality of care for depression:Alternatives to the simple randomized clinical trial. GeneralHospital Psychiatry, 25, 115–123.

Unutzer, J., Katon, W., Callahan, C. M., Williams, J. W.,Hunkeler, E., Harpole, L., et al. (2002). Collaborative care

management of late-life depression in the primary care set-ting: A randomized controlled trial. Journal of the AmericanMedical Association, 288(22), 2836–2845.

Unutzer, J., Patrick, D. L., Simon, G., Grembowski, D.,Walker, E., Rutter, C., et al. (1997). Depressive symptomsand the cost of health services in HMO patients aged 65and older. A 4-year prospective study. Journal of the AmericanMedical Association, 277(20), 1618–1623.

Ursano, R. J., Bell, C., Eth, S., Friedman, M., Norwood, A.,Pfefferbaum, B., et al. (2004). Practice guideline for thetreatment of patients with acute stress disorder and post-traumatic stress disorder. American Journal of Psychiatry,161(Suppl. 11), 3–31.

Von Korff, M., Gruman, J., Schaefer, J., Curry, S. J., &Wagner, E. H. (1997). Collaborative management ofchronic illness. Annals of Internal Medicine, 127, 1097–1102.

Wagner, A. W. (2003). Cognitive–behavioral therapy forPTSD: Applications to injured trauma survivors. Seminars inClinical Neuropsychiatry, 8(3), 175–187.

Weisaeth, L. (2001). Acute posttraumatic stress: Nonacceptanceof early intervention. Journal of Psychiatry, 62(17), 35–40.

Weisz, J. R., Chu, B. C., & Polo, A. J. (2004). Treatment dis-semination and evidence-based practice: Strengtheningintervention through clinician–researcher collaboration.Clinical Psychology: Science and Practice, 11, 300–307.

Weisz, J. R., Southam-Gerow, M. A., Gordis, E. B., & Connor-Smith, J. (2003). Primary and secondary control enhance-ment training for youth depression. In A. E. Kazdin & J. R.Weisz (Eds.), Evidence-based psychotherapies for children andadolescents (pp. 165–183). New York: Guilford Press.

Wells, K. B. (1999). Treatment research at the crossroads:The scientific interface of clinical trials and effectivenessresearch. American Journal of Psychiatry, 156(1), 5–10.

Wells, K. B., Miranda, J., Bruce, M. L., Alegria, M., & Waller-stein, N. (2004). Bridging community intervention andmental health services research. American Journal of Psychia-try, 161(6), 955–963.

Wells, K. B., Sherbourne, C., Schoenbaum, M., Duan, N.,Meredith, L., Unutzer, J., et al. (2000). Impact of dissemi-nating quality improvement programs for depression inmanaged primary care: A randomized controlled trial. Jour-nal of the American Medical Association, 283, 212–220.

Whooley, M. A., & Simon, G. E. (2000). Managing depres-sion in medical outpatients. The New England Journal ofMedicine, 343(26), 1942–1950.

Zatzick, D. F., Jurkovich, G., Russo, J., Roy-Byrne, P.,Katon, W., Wagner, A., et al. (2004). Posttraumatic distress,alcohol disorders, and recurrent trauma across level 1 traumacenters. Journal of Trauma: Injury, Infection, and Critical Care,57(2), 360–366.


Zatzick, D. F., Kang, S., Kim, S., Leigh, P., Kravitz, R., Drake,C., et al. (2000). Patients with recognized psychiatric dis-orders in trauma surgery: Incidence, inpatient length of stay,and cost. Journal of Trauma: Injury, Infection, and Critical Care,49(3), 487–495.

Zatzick, D. F., Kang, S. M., Hinton, W. L., Kelly, R. H., Hilty,D. M., Franz, C. E., et al. (2001). Posttraumatic concerns:A patient-centered approach to outcome assessmentafter traumatic physical injury. Medical Care, 39(4), 327–339.

Zatzick, D. F., Roy-Byrne, P., Russo, J., Rivara, F., Droesch,R., Wagner, A., et al. (2004). A randomized effectivenesstrial of stepped collaborative care for acutely injured traumasurvivors. Archives of General Psychiatry, 61(5), 498–506.

Zatzick, D. F., Roy-Byrne, P., Russo, J., Rivara, F. P.,Koike, A., Jurkovich, G. J., et al. (2001). Collaborative

interventions for physically injured trauma survivors:A pilot randomized effectiveness trial. General Hospital Psy-chiatry, 23(3), 114–123.

Zatzick, D. F., Russo, J., Grossman, D., Jurkovich, G. J.,Sabin, J., Berliner, L., et al. (in press). PTSD and depressivesymptoms, alcohol use, and recurrent traumatic life eventsin a representative sample of injured adolescents and theirparents. Journal of Pediatric Psychology.

Zatzick, D. F., Russo, J., Rivara, F., Roy-Byrne, P., Jurkovich,G., & Katon, W. (2005). The detection and treatment ofposttraumatic distress and substance intoxication in theacute care inpatient setting. General Hospital Psychiatry,27(1), 57–62.

Received December 14, 2004; revised August 5, 2005; accepted September 15, 2005.

Documents

Developing and Implementing Randomized Effectiveness Trials in General Medical Settings