Hiding in plain sight: Lessons from landmark clinical trials

Steve Goodman, MD, PhD

This talk is brought to you by…

Project ImpACT: a little history Started in 2004 PIs. S. Goodman, H. Marks Coordinator: Karen Robinson Research Assistants: >20 Goal: To find (the most) “important” RCTs

conducted in all of medicine and public health, 1948-Present.

Purposes To find and profile in depth the 100 “most

important” RCTs in medicine and public health since 1948 (UK Streptomycin trial).

To create a resource of important trials to provide examples of the critical importance that RCTs have played in medicine and public health.

To learn from the history of RCTs lessons that have been forgotten or restricted to specialty fields of medicine, and to enhance cross-specialty learning.

Help establish CCT as an international center for the study of the history and methodology of clinical trials.

Dimensions of “importance”

Trials that affect....1. Practice of medicine or public health2. Methods of designing, conducting or

analyzing trials3. Course of subsequent research4. Understanding of pathophysiology and

biology5. Regulations, law or policy6. Trial ethics

ImpACT activities Field research

Finding important trials within specific fields of medicine/PH

Soliciting nominations Vetting nominations for profiles

Profiling trials Structured 3-6 page summary of RCT

CTSeminar/14_ECMO/ECMO_14/Profile_14_ECM0_13Sep06.doc

Selecting ImpACT trials Conference with members of advisory board and

international field experts to discuss and select important trials in that specialty

In-depth “chapters” on selected ImpACT trials

ImpACT fields1. Oncology2. Cardiology3. GI4. Neurology5. Endocrine/metab6. ID7. Pulmonary8. Hematology9. Allergy/Immunology10. Dermatology

11. Ophthalmology*12. Ob/Gyn*13. Neo/perinatology*14. Pediatrics15. Psychiatry16. Surgery 17. Anesth./Crit. Care18. Radiology19. Health services20. Public Health

ImpACT Website http://www.projectimpact.info/Home/

default.cfm Ca 180 trials profiled. Ca 20 trials selected as ImpACT Landmark

trials. Last year SCT/ImpACT Trial of the Year award

given for the first time.

Trials for today

Polio (1954) Salk vaccine trial

ECMO (1986) Extra-corporeal membrane oxygenation for

neonatal pulmonary hypterension WHI (2003)

Post-menopausal estrogen therapy for reduction of CAD risk

NSABP-06 (1982) Lumpectomy vs. Mastectomy trial for early stage

breast cancer

Trials I would love to talk about Retrolental fibroplasia collaborative trial (1952) Neonatal hypothermia (1959) Coronary Drug Project (1966) MR FIT (Multiple Risk Factor Intervention Trial)

(1972) UGDP (1974) CRASH Trial (1992) EC/IC (Extra-cranial/Intra-cranial bypass) Macular Photocoagulation Study (1991) Parkinson’s stem cell trial w/sham surgery. NETT (National Emphysema Treatment Trial) (2004)

Background

Poliomyelitis is a highly seasonal viral disease. 95% infections are asymptomatic. Small percentage cause permanent paralysis.

Polio peaked in 1952 with 58,000 cases (~37 per 100,000). Hardest struck were middle and upper class children.

The National Foundation for Infantile Paralysis (NFIP), or March of Dimes, mounted public donation campaign to raise funds to develop an effective vaccine.

Small field trial of the formalin-inactivated Salk vaccine (N=700) showed it to be safe and potent.

Salk vaccine trial timeline January, 1953: NFIP Immunization committee considers

Salk preliminary data and whether trial is needed. Salk wants to do more development and safety studies

May 1953: O’Connor pushes trials and convenes committee w/o virologists to decide on design. Intense discussion of need for randomization, and nature of control. O’Connor and Salk opposed randomization, key scientists (Joseph Bell) supportive. Observed control preferred.

June 1953: Tommy Francis, PhD, U Mich, chosen to direct trial. Insists on randomization with saline injected placebo.

November, 1953: States contacted re participation. Several object to randomization, others object to observed controls. Negotiation starts with health authorities in every state.

January 11, 1954: Final decision on trial design. 11 states agree to randomization, 33 states use observed controls.

Trial logistics Basil O’Connor, head of NFIP Tommy Francis, Trial director 15,000 public schools 60,000 MDs, nurses, public health officials 64,000 teachers/principals >200,000 lay volunteers Trial funding:

Average public donation

$7 Million

$0.27

Debate over randomization Observational design supported by Basil

O’Connor, head of NFIP. Randomization pushed by some scientists, vehemently opposed by Salk:

“. . .if we are aware of the fact that the presence of antibody is effective in preventing the experimental disease in animals and in man, then what moral justification can there be for intentionally injecting children with salt solution or some other placebo for the purpose of determining whether or not a procedure that produces antibody formation is effective. . .I would feel that every child who is injected with a placebo and becomes paralyzed will do so at my hands. . .The use of a placebo control, I am afraid, is a fetish of orthodoxy and would serve to create a ‘beautiful epidemiologic’ experiment but would make the humanitarian shudder and would make Hippocrates turn over in his grave”.

Time urgencyBasil O’Connor, Fall 1953, during trial design debate:“I have just figured out that during this coming summer thirty

or forty thousand children will get polio. About fifteen thousand of them will be paralyzed and more than a thousand will die. If we have the capacity to prevent this, we have a social responsibility here that none of you have been talking about. Let me remind you that we are supported by the people and it is our duty to save lives, no matter how many difficulties may be involved.”

Tom Rivers, MD, Chair of NFIP Immunization committee:“We all believed that the Foundation had to put on a field trial

in the spring of 1954; if it didn’t, the lid would be off and the following year everybody and his aunt would be trying out their own vaccines unless the Public Health Service could put a muzzle on them.”

.

Public participation Thousands of informational meetings held with

health officials, parents and children across the country.

NYC health commissioner, Leona Baumgartner, recalling talking to children at P.S. 61 in Manhattan’s Lower East Side about the placebo controls:

“Those youngsters [gave] as good a description of a controlled experiment as I’ve ever heard. I was very excited, because it seemed to me that if you could teach a generation of kids about what a controlled experiment was, and about what science really was, this was a plus value regardless of whether the vaccine was any good or not.”

Salk vaccine trial timeline March, 1954: Viral particles found in 4/6 vaccine

lots. Winchell broadcasts warning against participation caused withdrawal of Wisconsin and 150K participants.

April 25, 1954: Meeting to decide on whether to go forward on trial, with several committee members unable to sleep.

April 26, 1954: Innoculations start, ending June 15.

Study Design

Two different studies of school children, aged 6-9 years:

Randomized, double-masked, placebo-controlled trial: (455,474 1st-3rd graders from 11 states)

Observational Study (567,210 participants from 33 states) 2nd graders received vaccine 1st and 3rd graders served as controls

On the parental consent form, the phrase ‘I give my permission’ was changed to ‘I hereby request,’ implying that not every child would be fortunate enough to be picked.

Timeline, cont. April 12, 1955: Results announced.

Vaccine licensed by HHS 2 hrs later, shipments started 4 hrs hence.

April 13, 1955: Nationwide immunization commences with stockpiled vaccine.

April 26, 1955: Cutter incident (contaminated vaccine – >200 polio cases, 59 paralytic, 5 deaths)

Results

Placebo Control Trial: Total cases reported differed significantly between vaccinated and controls

(respective attack rates were 41 vs 81, P<0.001). Difference in cases of paralytic polio was marked (33 among vaccinated vs 115

among controls (attack rates of 16 vs 57; P<0.001) Observational Study:

Also measured significant difference in incidence of paralytic polio with 38 cases among vaccinated vs 330 cases among controls (attack rates were 17 vs 46; P<0.001).

Polio incidence in the randomized study was similar to that in the observational study and neither study showed any appreciable difference between the two treatment arms in incidence of nonparalytic polio.

MMWR. 41(55):46, Sept 1993, CDC, Atlanta, Georgia.

Vaccine Cohort

# Participants

All Reported Cases Paralytic Polio Nonparalytic Polio

# Cases Attack Rate # Cases Attack Rate # Cases Attack Rate

Placebo-Controlled Trial

Vaccinated 200,745 82 41 33 16 24 12

Placebo 201,229 162 81 115 57 27 13

Observational Control Trial

Vaccinated 221,998 76 34 38 17 18 8

Controls 725,173 439 61 330 46 61 8

Impact Vastly accelerated the vaccine development process and

undoubtedly saved hundreds of lives and thousands of paralytic cases.

Enhanced public support for medical research at the time and possibly the future (through the involved children.)

Improved adherence to vaccine manufacturing protocols and safety testing guidelines.

New understanding of poliovirus epidemiology and serology helped shape future vaccine studies.

The role in the Cutter incident of the recently formed EIS at the CDC established its reputation.

The process by which each community decided the terms of its participation gives new meaning to the term “consent”.

Lessons from the Salk trial (T. Burke, 2004)

VERY large trials are possible. Committed, effective leadership is essential Precise case definitions are important Clinical endpoints are key; correlates of immunity

don’t suffice Serious potential safety problems may not be

detected in even a VERY large trial A nongovernmental organization can do the job, A successful phase III trial is just a beginning. Sometimes observational trials must do; you cannot

foist randomization on unwilling participants. Meaningful public (aka consumer) involvement led

to rapid public acceptance of both risks and benefits.

The ECMO controversy

When tough cases don’t make good trials

The ECMO Circuit

Background

ECMO had not improved survival in adults with acute respiratory failure, but it was hypothesized that it might be more effective in treating neonatal respiratory failure.

Robert Bartlett began treating critically ill infants with ECMO. Encouraging results (6/16 survived)

By the 1980s, Bartlett had achieved a 75% survival in patients judged to have a 5% survival when managed by conventional therapy.

Bartlett Article

Bartlett: Play-the-Winner Design

ECMOSurvive

d

CMTDied

10 ECMO: survived

1 CMT: died

Trial Motivation Increasing demand for more rigorous

evaluation of ECMO in neonates: Bartlett conducted a trial using a “play-the-

winner” design 1 patient received conventional therapy and died (smallest,

most premature, lightest) 11 patients received ECMO and survived

Article published in same issue (Wung et al. Pediatrics, 1985) reported 100% survival in a cohort of similar neonates treated with new low intensity ventilation.

The ECMO Controversy

“Children’s Hospital made a serious error.”

- Office for Protection from Research Risks, NIH

The Harvard Neonatal ECMO Trial Randomized newborns with PPHN to conventional therapy versus ECMO

Conventional Therapy

NICU: 7th Floor

Neonatologists

No patients had ever been offered ECMO

Anti-ECMO

ECMO

PICU: 5th Floor

Anesthesiologists & Surgeons

Already had experience with ECMO for newborns with CDH

Pro-ECMO

The Harvard Neonatal ECMO Trial: Randomized Consent

Eligible Newborn

RANDOMIZE

Do not seek consent

Seek consent for ECMO

CMT

CMT

ECMOYes

No

Results Phase I: 19 children enrolled and randomized

Conventional Medical Therapy Group: 6/10 survived ECMO Group: 9/9 survived

Phase II: 20 patients assigned to the ECMO group 19/20 survived

Newspaper coverage

A Harvard study on newborns draws fire;Doctors faulted for limiting life-saving

treatmentAugust 7, 1989Richard A. Knox, Boston Globe

A Harvard University study involving mortally ill newborns is being challenged as unethical in a debate that raises important questions about how to do research on promising new therapies.

The Response to the ECMO Trial

The hospital IRB “made decisions that rightfully belonged to the parents. They really blew it.”Charles McCarthy, Director of OPRR

The doctors “were doing exactly what physicians did before we had a doctrine of informed consent - making decisions for parents.” George Annas, Boston University

The NIH Office for Protection from Research Risks (OPRR) reprimanded the hospital:

A Justification

With comment… Donald A. Berry. : Ethics and ECMO Robert E. Kass, Joel B. Greenhouse. A

Bayesian Perspective. Richard Royall. Colin B. Begg. Peter Armitage, D. Stephen Coad. D. Y. Lin, L. J. Wei. Richard G. Cornell. Janis Hardwick. Recent Progress in Clinical Trial

Designs that Adapt for Ethical Purposes. James H. Ware. Rejoinder

The Problem of “Randomized Consent”

"It's clear to me they did not ask consent because it would be hard to get a control group otherwise. Properly informed parents would say 'No thank you.’”- Richard M. Royall, Johns Hopkins School of Public Health

“[Seeking consent from those randomized to the conventional medical treatment] would impose unacceptable psychological burdens on the parents of children not included in the ECMO arm of the study.”

- Dr. Peter H. Wolff , Chair of Children’s Hospital IRB

The Problem of Adaptive Randomization"The clear expectation was that more patients would die on conventional therapy. So the question is whether having an excess of deaths balances the worth of information gained. Since I believe such information is available without randomizing, my answer is a resounding no.”

- Donald A. Berry, University of Minnesota

"The Harvard study was informed by the slightly hysterical view that we must immediately stop a study as soon as we have an idea which treatment might be better. If we're not careful, we will soon have a system in which we can establish nothing.”

- Paul Meier, University of Chicago

Are RCTs the only way to learn? “The brilliant success of the RCT

has now become a form of intellectual tyranny” Freireich

“We should not proceed on the fallacious assumption that where there is no randomization, there is no truth.” Royall

Meinert on ECMO (Pediatrics, 1990)

I have reservations about any trial that is designed to detect a treatment difference of “miracle” proportions, such as the one postulated in this trial. Generally, use of such a difference for sample size rationale means either that the investigators “shopped” for a set of specifications to yield a desired sample size or that the trial is being done to demonstrate or “prove” the value of a treatment.

…The desire to shield patients from upsetting discussions by using nontraditional approaches to consent is a natural one, but its pursuit should be reserved for settings in which there is a legitimate state of equipoise.

Chalmers: “A Belated Randomized Trial” “A plague on all houses except those who reported

and conducted [this] study…” All new treatments should be administered under a

randomized protocol. “IRBs should recognize the ethical and scientific

deficiencies of uncontrolled pilot trials and innovative practice. They should demand… that physicians provide evidence for all that they do.”

“Professional ethicists should get off the back of people performing good clinical research…In the absence of evidence of efficacy, both the withholding and delivery of a new treatment are equally less ethical than [RCTs].”

Payers should only pay for innovate therapies under protocol.

Chalmers, continued Trial registration would have to start. (KD

cited) Data monitoring and peer review boards

would have to decide when to stop. Electronic communication would have to be

devised to keep track of rapid changes. A portion of the money spent for care would

be spent on experimentation.

Lantos and Fader, NEJM, 1991

NIH Consensus ConferenceReport of the Workshop on

Diffusion of ECMO TechnologyExtracorporeal Membrane Oxygenation

May 31,1990—June 1,1990Rockville, Maryland

Sponsored by:National Institute of Child Health and Human Development,

NIHOffice of Medical Applications of Research, NIH

National Institute of Neurological Disorders and Stroke, NIHNational Heart, Lung, and Blood Institute, NIHAgency for Health Care Policy and Research

Federal Drug Administration

NIH Consensus RecommendationsPrevention and Therapeutic Alternatives

Alternative therapies and prevention have not been adequately explored. There is a serious lack of knowledge regarding epidemiologic factors that influence the requirement for ECMO therapy.

The actual incidence of conditions treated with ECMO.. is not well documented. …and the relative effectiveness of alternatives versus ECMO is unclear. …

Education and prevention strategies should be developed …. It is not yet known whether the use of ECMO can be reduced through education …..

Additional research The increasing use of ECMO especially in

new patient populations creates an urgent need for further research. Clinical studies are needed to determine when ECMO is the most appropriate treatment alternative and what technical improvements are safe and feasible. The short-term and long-term effects of ECMO on the nervous system, pulmonary system, cardiac system, and the blood in all age groups must be further defined. Quality of life as well as specific biologic parameters should be studied.

The UK Neonatal ECMO Trial 1993-1995: 185 neonates randomized to

ECMO vs CMT

Trial stopped early by DSMB, ECMO survival 60/93 = 65% CMT survival 38/92 = 41%, p<0.0005

Postscript, 2005Extracorporeal Life Support: History and New Directions

Robert H. Bartlett, MD

“The use of ECLS peaked in 1992 and has been steadily decreasing since that time. The primary reasons are identification and avoidance of ventilator-induced lung injury and the use of inhaled nitric oxide as a pulmonary vaso dilator. In our experience, 40% of the patients who would have required support with ECMO in 1992 are now successfully treated with pressure-controlled (low stretch) ventilation and inhaled nitric oxide using standard or oscillation ventilator techniques.”

Issues raised How to evaluate promising new technology /

interventions whose development produces results that disturb equipoise.

How to create a system – both professional and economic- that rewards evaluation as much as innovation.

Whether RCTs are necessary for breakthrough technologies.

How to evaluate interventions with possible large short term benefit and similar long term harm.

Damage can be done to the RCT enterprise if it is imposed on subjects, or a society, who are unwilling to accept it, or don’t understand it.

Women’s Health Initiative1993 - 2002

Previous systematic review showed increased risk of cardiovascular events with HRT

Controversies

Several top women epidemiologists complained about the size and complexity of the study

Huge budget ($625 million) was criticized as being unnecessary, that we were certain enough of the answer.

Impact

Results were shocking and extremely surprising

Significant reduction in initiation as well as a marked discontinuation of HRT

Reinforced the importance of conducting large scale RCTs

04/11/23

Endpoints in WHI trial

04/11/23

WHI stopping rule

Efficacy Test CHD against an O-F upper 2.5%ile If benefit for CHD, test Global index (any

endpoint) against 5% upper boundary.Safety

Lower 5% O-F Boundary for any major AE. AND >1 SE in harm direction for Global

index.

04/11/23

04/11/23

04/11/23

04/11/23

WHI DSMB recommendation

04/11/23

NIH Letter #1

04/11/23

Issues in stopping the ERT trial

04/11/23

No stopping guidelines were crossed No adverse CHD effect, beneficial (??) breast

ca effect. Global index showed equal risk, benefit.

BUT!!! Strong evidence for high stroke risk. WHIMS trial showed slight (unexpected)

increase in dementia in both trials. No CHD benefit.

DSMB vote: May, 2003 “Each one of us realized our vote was on a knife edge.” 5-4 vote to continue. Degrees of confidence (0-100) were 45-55. NIH convenes ad-hoc panel. 2 members of original

DSMB observe as non-voting members. July, 2003: New panel votes to continue. November, 2003: DSMB meets again, augmented w/2

members of ad hoc panel dementia experts. Decide to continue trial w/letter to subjects explaining stroke risk and breast cancer benefit.

Unable to craft letter. Re-vote: 5-4 to continue – different members in majority. Recommendation to continue sent to NHLBI. NHLBI stops trial on 2/04.

Lessons Importance of board independence.

NIH and CC both had structural conflicts. NIH replaced DSMB members and required permission for

additional analyses, both inimical to optimal board function.

NIH reps sat in DSMB closed sessions. Monitor for both benefit and harm regardless of prior. Need flexibility for weighting harms when monitoring

multiple endpoints in long-term prevention trials. Is obligation to participants different in prevention than treatment trials?

“JW adds special thanks to William Harlan who prevailed on her to join the DSMB when she knew that the trial was going to show that hormone replacement therapy reduced coronary disease and therefore that serving on this committee would be boring.”

Comparison of Total Mastectomy and Lumpectomy With or Without Radiation (NSABP B-06) 1976-1984

Bernard Fisher, MD Chairman and Scientific Director, NSABP

Carol K. Redmond, ScD Director, NSABP Biostatistics Center

National Surgical Adjuvant Breast and Bowel Project

Background Radical mastectomy—removal of the affected breast, underarm lymph nodes and

both underlying chest muscles (en bloc dissection)—was the standard of care for breast cancer.

Two divergent hypotheses existed regarding tumor biology. Most believed that cancer was a local/regional disease spread by the lymph system to distant sites. Others proposed that breast cancer was a systemic disease better treated by systemic therapy.

During the 1970s, social pressure/advocacy groups were starting to raise awareness of breast cancer, advocating research and promoting less disfiguring surgery.

The NSABP—a clinical trials cooperative group supported by the NCI—had previously found that patients treated by total mastectomy without axillary-node dissection and pectoral muscle removal were at no higher risk of distant disease or death than those undergoing a Halstead operation (radical mastectomy).

This prospective, randomized, clinical trial wished to determine: The effectiveness of lumpectomy for breast preservation The effect of post surgery radiation therapy on ipsilateral breast tumor incidence

reduction The clinical importance of tumor multicentricity

Study Design

In 1976, 2163 women with primary operable breast cancer and with tumors ≤ 4cm were randomized into:

MastecomyLumpectomyLumpectomy with radiation

All patients underwent an axillary lymph node dissection

Standard randomized treatment assignments (stratified by participating institution) were changed to a Zelen“prerandomization” design during the course of the trial.

Issues

77

Eight percent of women (174/2163) did not accept randomly assigned treatment but were followed.

Due to low enrollment rates, the standard randomization protocol (stratified by participating institution) was changed to a Zelen design (prerandomization) during the course of the trial. This change was accompanied by a six-fold increase in enrollment rates. This practice provoked considerable ethical debate.

Results At five years, 10% of patients initially treated with lumpectomy were found to

have a tumor at the margins of resected specimens and therefore subsequently underwent total mastectomy. The incidence of positive margins was similar in both lumpectomy groups. Recurrences at the margin were positively related to the presence, number of positive nodes, and centrally located tumors.

Treatment by lumpectomy with or without breast irradiation resulted in disease-free, distant-disease-free and overall survival at five years that was no different than breast removal.

Distant-disease-free (p not significant) and disease-free survival (p=0.04) were higher in patients undergoing lumpectomy plus radiation as compared to the total mastectomy group.

Among women treated by lumpectomy, the addition of radiation therapy significantly decreased tumor recurrence (P<0.001) in the ipsilateral breast . The advantage of irradiation therapy was observed in both subjects with negative and positive nodes.

Coupling breast irradiation with lumpectomy also improved disease-free survival (p=0.02) when compared to lumpectomy alone. However, no significant differences between the two groups were found with respect to distant-disease-free or overall survival.

The fewest treatment failures were observed in the group receiving lumpectomy + radiation therapy. More than half of the first-reported treatment failures in all three treatment groups were due to distant recurrences. Second cancers and deaths from other causes were distributed equally across treatment groups.

Impact

79

Findings along with B-04 trial repudiated the Halstedian principles of breast cancer treatment and provided support for the theory that breast cancer is a systemic disease.

Lumpectomy was shown to be an acceptable alternative to mastectomy for women with Stage I and II breast cancer. Radiation therapy was shown to be beneficial in reducing local recurrence of tumor in lumpectomy patients..

At St. Luc Hospital, a Dr. Poisson admitted fraud after a routine audit revealed that records had been altered. He stated that he altered records to ease the entry requirements because he felt the treatment provided by the NSABP protocol was beneficial to potential participants in the study.

Consequences of “fraud” Fisher: “All Hell Broke Loose” Fisher and Carol Redmond dismissed from NSABP 2 congressional hearings, one under John Dingell New Clinical Trials Monitoring Branch at NCI 1993 Federal Register notice: Roger Poisson guilty

of research misconduct Poisson falsified or fabricated 111 separate

instances of data in 14 NSABP clinical trials (<0.5% of total).

Chicago Tribune Front Page Story March 13, 1994: “Fraud in Breast Cancer Study: Doctor Lied on Data for Decade”

150 articles tagged “scientific misconduct” by NLM

Postscript In 1997, Bernard Fisher received an apology from

the University of Pittsburgh, his rank as Distinguished University Service Professor reinstated, NCI and U. Pitt. restrictions lifted, and $2.75 million for wrongful punishment.

NCI said: "Through his role as the scientific leader of the National Surgical Adjuvant Breast and Bowel Project (NSABP), [Fisher] has not only changed the way breast cancer is treated, but enlightened medical science to view breast cancer as not just a tumor confined to the breast, but as a systemic disease requiring more than surgical intervention,"

Impact

84

This trial raised ethical issues regarding both informed consent, fraud, and the appropriateness of actions against the investigators after the fraud had been detected. Thus, its impact and legacy was broad, affecting the understanding of biology, clinical practice, the ethics of randomization, the impact and handling of research misconduct, and the role of regulatory agencies in clinical trial oversight.

Advisory Board John Bailar III David Brown Sir Iain Chalmers Kay Dickersin Benjamin

Djulbegovic Sir Richard Doll Susan Ellenberg Norman Fost P. J. Devereaux Cynthia Mulrow

Barbara S. Hawkins Brian Haynes Joseph Lau Curtis L. Meinert Steven Piantadosi David L. Sackett Jonathan M. Samet Harold Sox Sean Tunis Jan P. Vandenbroucke Janet Wittes

ImpACT Staff

Emily Evans Gila Neta* Alison Brown

Sara Lowther*

Olaide Odelola

ImpACT staff Current Allison Brown Emily Evans Gila Neta Olaide Odelola

Former Nyasha Bakare (2006) Juliana Cuervo (2007) Lara Devgan (2006-2007) Ishveena Duggal (2004) Julia Gage (2005-2006) Emily Henkle (2006)

Jay Herson (2005) David Majure (2006-2007) Tram Lam (2005-2006) Sara Lowther (2005-2007) Nancy Maldeis (2004-2005) Todd Meyers (2005) Kamal Patel (2006) Sarah Post (2003) Samara Rifkin (2006) Jordana Rothschild (2004) Lauren Singer (2004) Ellen Wasserman (2003-

2005) Lashawn Worsley (2005)

Project leaders

Harry Marks Karen Robinson

Final thoughts on the need for RCTsThe fatal tendency of mankind to leave off

thinking about a thing when it is no longer doubtful is the cause of half their errors.

John Stuart Mill (On Liberty,1859)

In theory there is no difference between theory and practice. In practice there is.

Yogi Berra