RCT critical appraisal

Critical appraisal of clinical trials

Specific Types of Study

Randomised Controlled Trial (RCT)• Population is randomly allocated to two groups• One group is given a specific treatment or

intervention • On average the groups are identical because they

are randomised and therefore any difference in the measured outcome is due to the intervention

• Specified follow up period and specified outcomes• e.g. drug better than placebo; surgical procedure

compared with sham

Randomised Controlled Trial (RCT)

Advantages• Allows rigorous evaluation of a single variable in

a previously defined population e.g. a new drug.• Prospective i.e. collect the information after you

decide to do the study.• Tries to disprove the null hypothesis• Tries to eradicate bias because the two groups

are identical.• Allows for meta-analysis later.

RCT’s are

• Unnecessary if:• Clearly successful intervention• Previous RCT’s or meta-analyses

• Impractical when:• Unethical to randomise• Large number needed

• Inappropriate when:• Looking at prognosis• Looking at validity of diagnostic tests• Looking at quality of care

Background (1)

Practicing physicians must rely on the literature to keep current on recent developments on new therapies as well as providing additional evidence on therapies which have been long used in practice

Accurate reporting of a clinical trial is important to aid the practicing physician in deciding to adopt a new therapy or modify therapies currently in use

Background (2)• Proposals for requirements for reporting

of randomised trials• JAMA 1994;272:1926-31• Ann Intern Med 1994;121:894-5

• JAMA Editorial in 1995 suggests two groups produce a unified statement

• Consolidated Standards of Reporting (CONSORT)• JAMA 1996; 276:637-9

Common Elements of Reporting for All Trials

• Population under study• Therapy details• Experimental design• Patient accounting• Quality control procedures• Statistical analysis

Special Reporting Requirements• Non-randomized trials• Randomized trials

The Term Prospective Trial Is a “Pleonasm”

• “Trial” used increasingly in combination with “prospective”

• “A PubMed search ‘prospective trial’ yielded 507 hits for the period 1999-2001. However, ‘prospective’ is a pleonasm (superfluous)”

• Cross-sectional trial!!• “Because all trials are prospective by

definition; the only way to do a retrospective trial is for the investigator to travel back in time with a box of pills.” • Letter in The Lancet, Sept 2002 by Martijn B Katan,

Netherlands

Reporting in Clinical Trials

• Describe the Plan• Report the Results• Confess to Problems • Interpret Objectively (no

spin!)


“The Published Paper”

1. Identify clinical investigators and institutions (experience, reputation)

2. Also, identify• Sponsor (federal, industry)• Data management team• Statistical analysis team


“The Published Paper”3. Introduction & Background

a. Biochemical theoryb. Animal workc. Phase I/II clinical studiesd. Previous large clinical studiese. Other pharmaceutical analogues


Definition of question

What was the primary question?- Clearly defined in advance

Reporting Clinical Trials

4. Methods Section• Outcome variables• Eligibility criteria

• Inclusion• Exclusion

• Randomization Procedures• Sample size justification• Treatment & Control

Randomization• Sequence generation

Method used to generate the allocation sequence, including any details about restriction

• What does restriction mean?

• Allocation concealmentMethod used to implement the random

allocation sequence

• ImplementationWho generated the allocation sequence, who

enrolled participants, and who assigned participants to their groups

Allocation concealment

• A technique used to prevent selection bias by concealing the allocation sequence from those assigning participants to intervention groups, until the moment of assignment.

Allocation concealment prevents researchers from (unconsciously or otherwise) influencing which participants are assigned to a given intervention group.

Importance of allocation concealment

• Unclearly concealed and inadequately concealed trials, compared to adequately concealed trials, exaggerated the estimates of an intervention’s effectiveness by 30% to 40%, on average.

Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408-412.

Reporting Clinical Trials

4. Methods Section (continued)• Outcome assessment & blinding• Measures of patient safety and

adverse events• Predefined Subgroups• Data monitoring plan• Analysis Plan summary


Treatment groupsa. Definitions

- Experimental- Control

b. Dose escalationc. Withdrawal for toxicity


5. Description of resultsa. Full accounting of all patients entered

on trials- Completeness- LTFU- Withdrawal

b. Comparison of treatment groups as assigned (ITT), baseline characteristics

c. Simple comparison of primary outcome variables using means, proportions, graphs, with measures of statistical precision (SE's, P-values)

d. Thorough analysis of side-effect data/adverse effects


5. Discussione. Adequate handling of the possible

impact of missing values, dropouts, non-adherence

f.Discussion, allowance for multiplicity in number of interim analyses, number of endpoints


5. DiscussionConsistency of results

a. among investigators and centers

b.Other independent studies of same drug or analogues

c. Subgroups consistencyd.Primary and secondary outcomes


6. Conclusiona.Brief summary

b.Strengths/weaknesses consistent with data

c.Generalizabilityd.Trade off in side effects -

risk/benefit

THE LANCETDouble-blind trial of aspirin in primary

prevention of myocardial infarction in patients with stable

chronic angina pectoris

Clinical trials have demonstrated a prophylactic role for aspirin in myocardial infarction and in unstable angina pectoris. The Swedish Angina Pectoris Aspirin Trial (SAPAT) is the first prospective study of aspirin in stable angina.

Lancet 1992;340:1421-25

“After showing good tolerance of sotalol for at least three weeks the patients were randomised double blind to aspirin 75 mg daily (n=1009) or placebo (n=1026).”

Obstetrics & Gynecology

Bacterial Vaginosis: Treatment with Topical Intravaginal Clindamycin Phosphate

We tested topical intravaginal clindamycin phosphate at practitioners use systemic metronidazole and acconcentrations of 0.1, 1.0, and 2.0% in the treatment of common gastrointestinal side effects. Alternative62 women with symtomatic bacterial vaginosis in a pro- apy, particularly for pregnant women, is highly pective, randomized, double-blind, placebo-controlled able. This study examined the safety and efficacy

“. . . prospective, randomized, double-blind, placebo-controlled trial.”

Obstet Gynecol 1990;76:118-23.

British Journal of Obstetrics and GynaecologyOctober 1991, Vol. 98, pp 980-987

(Patho)physiological Implications of Chronic Dietary Sodium Restriction During Pregnancy; a longitudinal prospective randomized study

Abstract: Objective--To study the possible pathophysiological implications of long continued dietary sodium restriction in pregnancyDesign--Longitudinal prospective randomized study of the effects of a low sodium diet compared with unrestricted sodium intake in pregnacy.Setting--Academic Department of Obstetrics and Gynaecology at Sint Radboud Hospital, Nijmegen, The Netherlands.

“Design - Longitudinal, prospective randomized . . .”

Br J Obstet Gynaecol 1991;98:980-7

A Controlled Trial of Povidone-Iodine as Prophylaxis Against Ophthalmia Neonatorum

Abstract Background. Neonatal conjunctivitis Ophthalmic neonatorum) continues to cause blindness, because the agents used prophylactically to prevent this condition are not completely effective and are not widely available in many parts of the world. Povidone--iodine ophthalmic solution is an effective antibacterial agent with broad antibacterial and antiviral activity to which no bacteria are known to be resistant, and it is

“Randomization was achieved by rotating the three medications after each was used for a week.”

NEJM 1995;332:562-6

British Journal of Obstetrics and Gynaecology

March 1991, Vol. 98, pp 260-264 Stopping Smoking in Pregnancy: Effect of a Self-help manual in Controlled Trial

Summary. For medical reasons, encouraging women to stop smokingduring pregnancy and post partum has high priority. Many smokerswant to stop smoking but decline clinical treatment when it is offered. The aim of this study was to find a method which was accepted by a large number of smokers, had a high success rate and, at the same time,

“Women were randomized . . . born on days 1-10 of every month formed the control group (n=231), and those born on days 11-31 formed the treatment group (n=492). ”

Br J Obstet Gynaecol 1991; 98:260-4.

Obstetrics and GynecologyVol. 77, No. 3, March 1991

Nifedipine in the Treatment of Severe Preeclampsia

We conducted a randomized clinical trial in which patients with severe preeclampsia between 26-36 weeks of gestation receive either nifedipine (10-30 mg sublingually, then 40-120 mg/day orally; N= 24) or hydralazine (6.25-12.5 mg intravenously, then 80-120 mg/day orally; N= 25)

“We conducted a randomized controlled trial. . . ”

“Subjects were assigned to the nifedipine or hydralazine group according to the week of the month.”

Obstet Gynecol 1991; 77:331-7

Effectiveness of antibiotic prophylaxis in preventing bacteriuria after multichannel urodynamic investigations: A blind, randomized study in 124 female patients

Am J Obstet Gynecol;1991;165:679-81

On completion of the procedures, the patients were randomly assigned to prophylaxis or nonprophylaxis groups according to hospital number. Both the physician and the nurse technician were blind as to which assignment the patient received. Patients in group A received nitrofurantoin 50 mg four times and phenazopyridine hydrochloride 200 mg three times for 1 day. Patients in group B received phenazopyridine hydrochloride only. The code was broken at the completion of the study.

Randomization Process

Proper Approach 4 Major General Medical Journals

Generation of Allocation 49%Sequence

Allocation Concealment 26%

Both 15%

[Lancet 1990; 335: 149-153.]

CONSORT (1)

• “Intent is to make experimental process more clear, flawed or not, so that users of the data can more appropriately evaluate its validity for their purposes”• checklist• figure• available at www.consort-

statement.org

CONSORT (2)

• Widely adopted by medical journals• required by many from Jan 1, 1997

• Available in six languages

The CONSORT Statement:Revised Recommendationsfor Improving the Quality of Reportsfor Parallel-Group Randomized Trials

JAMA, April 18, 2001—Vol 285, No. 15 1987

SPECIAL COMMUNICATION

David Moher, MSc

Kenneth F. Schulz, PhD, MBA

Douglas Altman, DSc

for the CONSORT Group

Dissemination• Major general & internal medicine

journals endorsed CONSORTRequired authors to submit RCT reports using

template

• Editorial groups that have endorsed CONSORTWorld Association of Medical Editors (WAME)Council of Science Editors (CSE) International Committee of Medical Journal Editors

(ICMJE or Vancouver Group)

The Revised CONSORT Statement for Reporting Randomized Trials:Explanation and Elaboration

17 April 2001 Annals of Internal Medicine Volume 134 Number 8 663

ACADEMIA AND CLINIC

Douglas G. Altman, DSc; Kenneth F. Schulz, PhD; David Moher, MSc; Matthias Egger, MD; Frank Davidoff, MD; Diana Elbourne, PhD; Peter C. Gøtzsche, MD; and Thomas Lang, MA, for the CONSORT Group

Participant flow through a randomized trial . . .

important?

Revamped web site (www.consort-statement.org)

• Reprints of statement and E & E document (copyright is in the public domain)

• Data bank of “good” and “not so good” examples of trial reporting

• Reference citations of new evidence

Does the CONSORT checklist work? Methods

• 4 “general & internal medicine” journals 3 endorsed CONSORT (BMJ, JAMA, Lancet) 1 did not (NEJM)

• Hand searched all 4 journals for 1994 and 1998 (January to June), respectively

• Used number of CONSORT items, Jadad score and adequacy of allocation concealment Trained 2 assessors

Does CONSORT work? More results

Total Unclear allocation concealment

Journal 1994

N 1998

N Pre Mean (SD)

change (CI)

Pre Mean

Change (CI)

BMJ 14 20 2.07 (0.92) 0.43 (-0.34, .20) 79 -29 (-62, 4)

JAMA 29 20 3.00 (1.04) 0.35 (-0.29, .99) 59 -14 (-43, 16)

Lancet 28 37 2.75 (0.89) 0.68 (0.14, .22)** 54 -24 (-48, 1)

Total Adopters

71 77 2.72 (1.00) 0.45 (0.08, .82)*** 61 -22 (-38, -6)****

Total Non-

Adopter

26 37 3.12 (1.03) -0.01 (-0.55,0.54) 69 -8 (-33, 17)

Moher D, Jones A, Lepage L, for the CONSORT group. Use of the CONSORT statement and quality of reports of randomized trials: a comparative before and after evaluation? JAMA 2001;285:1992-1995.

Documents

RCT critical appraisal