PRACTICE GUIDELINES FROM RANDOMIZED CLINICAL TRIALSGroup 1

Nicole Bettes, Kelsey Brown, Greg Forlenza, Chen Gao, Joe Ostrem, Li Ou, Shauna Overgaard, Jincheng Zhou

Topic

• In the absence of data from randomized clinical trials, strong recommendations about the use of an intervention should not be made.

Levels of Evidence for Intervention Studies

From the National Institute for Clinical Excellence (NICE) Guideline Development Methods Updated March 2005

Clinical Practice Guideline Development

• Developed to assist providers with care of given medical conditions.

• Often incorporated into performance measures and quality of care assessments for providers.

• Many are based on scant evidence.

• Historical examples of major harm which has occurred from guidelines not based on RCT’s.

Example Guideline• Ophthalmic Screening for Diabetic Retinopathy

Forlenza GP, Stewart MW. Diabetic Retinopathy in Children. Pediatr Endocrinol Rev. 2012 Dec-2013 Jan; 10(2):2017-26

Strengths and Weaknesses of Approaches

• Strengths of RCT’s:• Well designed RCT’s are Unbiased• Well designed RCT’s are Objective• Well designed RCT’s are Consistent

• Weaknesses of “best clinical judgment:”• Initial judgments on clinical practice are often wrong (46%)

resulting in costly reversals in care.• Reasonable judgment uses subjective criteria which is susceptible

to bias and conflicts of interest.• Lacks consistency across guidelines.

Prasad V. Why Randomized Controlled Trials are Needed to Accept New Practices. May Clin Proc. October 2013:88(10):1046-50

Example 1: Thalidomide• Originally sold in 1950s as an OTC sedative.

• In late ‘50’s and early ‘60’s it was recommended to pregnant women by their OB/GYN’s to control morning sickness.

• Later found to cause serious birth defects and miscarriages.

• Led to serious birth defects of more than 10,000 children.

Kim JH, Scialli AR. Thalidomide: The tragedy of Birth Defects and the Effective Treatment of Disease. Toxcol Sci 122(1), 1-6(2011).

Example 2: Diabetes Care and Complications Trial• Prior to 1993, expert opinion on care of insulin-dependent

diabetes mellitus (type 1 diabetes) was split between intense blood sugar control and “conventional” blood sugar control.

• Between 1983 and 1993 patients with T1DM were randomly assigned to one of the 2 therapies.

• The study showed that intensive therapy reduced microvascular complications by up to 76% compared to “conventional” therapy!

• Study results drove development of new devices and medications and still serves as foundation of current diabetes care.

DCCT. The Effect of Intensive Treatment of Diabetes. NEJM 329;14: 977-986. Sep 30,1993.

Example 3: Hormone Replacement Therapy• Prior to 2002, clinical practice guidelines favored the use

of HRT during and after menopause.

• Millions of women all over the world used HRT.

• > 22 million prescriptions written in the U.S. in 2000.

• Women’s Health Initiative (WHI) put HRT through a placebo-controlled RCT.

• WHI study stopped early because of overwhelming evidence of risks of HRT.

• Increased risk of invasive breast cancer by 26%, of heart attacks and other coronary events by 29%, and of strokes by 41%.

Randal. The End of an Era? J Natl Cancer Inst 2002; 94:1116-1118. August 7, 2000.

Conclusions• RCT’s mark the highest grade of medical evidence.

• Clinical practice guidelines provide diagnostic and treatment advice for thousands of medical providers affecting millions of people.

• Medical practice not based on high grade evidence has a long history of causing significant harm to patients.

• Incorrect treatments from faulty guidelines lead to costly reversals in care.

• Providers and patients deserve the highest level of treatment guidelines.

Group 2

Recommendations on Intervention Without RCT Data

Main Points:

1. Generalizability of RCT

2. Feasibility of RCT

3. US Department Health and Human Services Reports

• Examples of practice guidelines developed in the

absence of RCT

Limited Generalizability

Select groups of participants Question specific

Specific disease

Relatively homogenous sample Inclusion/ Exclusion criteria

Sampling of the locals Site specific

Sample size

Limited Generalizability Findings applicable to population studied

Dose extrapolation Empirical recommendations

Interracial differences Race specific recommendation

Cancer therapy in Japanese vs. Caucasians

Blood pressure management in African Americans vs. Caucasians

Genetics variability in response Poor responders Optimal responders

Feasibility of RCT

Approval process is complex, costly, heterogeneous

and time-consuming

Multicenter trials require many separate approvals, often taking >1

year to complete, costing upwards of $100,000

In China, it takes > 1 year to obtain approvals

Ethical issues

Placebos - unethical to deny standard treatment, or deny a

therapeutic treatment

Unethical to impose a harmful intervention upon individuals

US Department of Health and Human Services

Recommendations Understanding the relevance and weight of evidence is an important

consideration when creating evidence-based guidelines

All levels of evidence are important and have their respective

value and, therefore, should be considered

Evidence-based practice (EBP) requires clinical decision making

1. Sound external research evidence

2. Individual clinical expertise

3. The needs of the individual patient

EBP is the conscientious and judicious use of current best

evidence in conjunction with clinical expertise and patient

values to guide clinical practice guidelines

1. Clinical practice guidelines for the American Academy of Orthopaedic

Surgeons (AAOS, 2010)

2. Clinical practice guidelines for Practicing Chiropractors’ Committee

on Radiology Protocols (PCCRP, 2006)

3. Clinical practice guidelines for the diagnosis and treatment of patients

with encephalitis (Infectious Diseases Society of America; Tunkel et

al. Clinical Infectious Diseases 2008)

Examples

Practice guidelines developed in the absence of RCT data as reported by the US Department of Health and Human Services

Clinical practice guidelines for the American Academy of Orthopaedic Surgeons (AAOS, 2010)

– Developed using “Best Available Evidence”• First considered RCT data

• However, often RCT data not available, therefore used:– Prospective controlled trials

– Prospective comparative studies

– Retrospective comparative studies

– Case-series studies

• Acknowledge must “develop using a systematic review of current scientific and clinical information”

• Noted that, “clinical patients may not necessarily be the same as those found in a clinical trial. Patient care and treatment (guidelines) should always be based on a clinician’s independent medical judgment, given the individual patient’s clinical circumstances.”

18

Additional Information

The Eastern Association for the Surgery of Trauma (EAST) Working Group– Practice Management Guidelines for Trauma Surgeries

– Assessment of scientific evidence – all relevant empirical data should be evaluated for the clinical use of interventions:

• RCT

• Prospective cohort studies

• Retrospective cohort studies

• Observational studies

• Prevalence studies

• Case control Studies

• Case reports

• Case series studies

• Expert opinion

19

Clinical practice guidelines for Practicing Chiropractors’ Committee on Radiology Protocols (PCCRP, 2006)

– When developing clinical practice guidelines, cannot restrict evidence to RCT• RCT for chiropractic medicine is not always relevant

• RCT narrow in methodology, which is not useful in clinical practice

• Effectiveness evidence cannot be collected by RCT as “randomization and controlled

conditions play no part in everyday clinical practice.”

– Most prevalent type of research in chiropractic medicine is:• Non-randomized clinical trials

• Case series

• Case studies

– Reasons for lack of RCT in chiropractic medicine:• Not relevant to actual practice

• Labor intensiveness

• Cost

• Usually restricted to university research groups, which in this field there is lack of research at the

university level

– Studies have shown that well-done case studies most often demonstrate findings

consistent with that of RCT {Venning et al. (BMJ 1982), Benson and Hartz et al (NEJM

2000)., Rosener et al. (JMPT 2003), Concato et al. (NEJM 2000)}.

– Therefore, to develop chiropractic clinical practice guidelines that is truly evidence-based

and still relevant to the chiropractic profession, all levels of evidence need to be

considered.20

Infectious Diseases Society of America– Developed guidelines for the diagnosis and treatment of patients with

encephalitis (Tunkel et al. Clinical Infectious Diseases 2008)

– Guidelines are intended to be used by health care providers who care for patients with encephalitis; provide clinicians with evidence-based recommendations

– Determining guidelines required a systematic review and analysis of evidence that included epidemiologic, clinical and laboratory-based data

– Lack of RCT data as etiology of encephalitis unknown and no definitive treatment strategies available

– Given the lack of RCT data on the diagnosis, treatment and management of encephalitis, many recommendations were developed from:

• Case reports

• Small series reports

• Expert opinion

21

Problems with RCT• The approval process is complex, costly, heterogeneous and time-

consuming.– Multicenter trials require many separate approvals, take more than 1 year to

complete, and cost upwards of $100,000– In China, it takes >1 year to obtain approval– Slows medical community therapeutic progress

• A “one-size fits all” approach is used – Regulation of low-risk trials of well-understood drugs is similar to regulation of trials

of completely new drugs, for which the risks are unknown.

• Trial participants may not represent the population as a whole and, therefore, results may not apply more generally– Participants in RCT tend to be a distinct population that does not represent the

“real-world” population an intervention would eventually target (i.e. people who sign up for RCT usually have tried many other treatment options and are desperate for a treatment. Moreover, they are further winnowed down as researchers eliminate would-be participants with co-morbid conditions, etc.)

– RCT are typically designed with internal validity in mind (i.e. the ability to trace causal inferences to the intervention) while ignoring external validity (i.e. generalization of the results)

– Participants are carefully monitored and more adherent to treatment and follow-up than real-world participants

22

Problems with RCT

• Some medical questions do not permit random assignment of participants as it may be unethical to do so

• Requires clinical equipoise with an acceptable control (standard of care treatment)

• An acceptable standard of care medical therapy or an alternative therapy may not exist

23

• 20% of clinical guidelines are cancer related (The

National Guideline Clearinghouse)

• 80% of all NCCN recommendations were category

I: high level of evidence with uniform consensus

(Poonacha et al. JCO 2013)

• There is a huge gap - need of guidelines to aid in

decision making in patient care and high level

evidence is not available.

24