Upload
harold-spencer
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
1
Randomised Controlled Trials in the Social Sciences:Challenges and Prospects
I . Boutron
Dept of Epidemiology, Biostatistics and Clinical Research
Assessing Nonpharmacological treatments
Challenges and reporting guidelines for Randomised Controlled Trials
2
Overview
• Specific methodological issues when assessing Nonpharmacological treatments
• Quality tools
• Reporting guidelines
3
Overview
• Specific methodological issues when assessing Nonpharmacological treatments
• Quality tools
• Reporting guidelines
Different types of NPT
- surgery
- technical operation (arthroscopy,…)
- rehabilitation
- education
- orthopedic braces or orthosis
- laser treatment
Participative interventions
Non implantable devicesTherapist-dependent interventions
Non-pharmacological treatments (NPT) take in many different treatments
5
Assessing Nonpharmacological treatments
• Randomized controlled trials (RCT)– Gold standard of therapeutic evaluation
• The design, conduct and reporting of RCTs aim at providing valid results
• Standard of the methodology of RCTs has been developed in the context of pharmacological treatments (PT)
• Specific methodological issues when assessing NPTs– Blinding– Placebo– Complexity of treatments– Care providers’ influence
6
Are NPT trials specific?Quality
A. PT (n=60) versus NPT (n=50)
B. Comparison
(1) Oral drug, intra-venous, intra-muscular (n = 46)
(2) Intra-articular injection (n = 14)
(3) Surgery, arthroscopy, joint lavage (n = 23)
(4) Rehabilitation, acupuncture, education(n = 27)
Quality of trials assessing PT and NPT of hip and knee osteoarthritis publishedin high impact factor journals (1992-2002)*
* Boutron, Tubach, Giraudeau, Ravaud Jama, 2003
scale
A B
- 1 0 1 2 3 4 5 6
Jadad - 1 0 1 2 3 4 5 6
Jadad scale
P T NPT
1
2 3 4
7
Are NPT trials specific?The feasibility of blinding
Feasibility of blinding patients
96
2
48
2
10
42
0%
20%
40%
60%
80%
100%
NPT (n = 50) PT (n = 60)
%
Possible
Difficult
Impossible
Feasibility of blinding care providers
9686
2
2
2
12
0%
20%
40%
60%
80%
100%
NPT (n = 50) PT (n = 60)
%
Possible
Difficult
Impossible
Feasibility of blinding outcome assessors
98
502
16
34
0%
20%
40%
60%
80%
100%
NPT (n = 50) PT (n = 60)
%
Possible
Difficult
Impossible
* Boutron I, Tubach F, Giraudeau B, Ravaud P, JCE, 2003
8
Are NPT trials specific?The success of blinding
Risk of unblinding patients
27
14
6586
80%
20%
40%
60%
80%
100%
NPT (n = 26) PT (n = 59)
%
Null
Moderate
Important
Risk of unblinding care providers
5719
43
68
130%
20%
40%
60%
80%
100%
NPT (n = 7) PT (n = 59)
%
Null
Moderate
Important
Risk of unblinding outcome assessors
8 10
36
90
56
0%
20%
40%
60%
80%
100%
NPT (n = 25) PT (n = 60)
%
Null
Moderate
Important
9
Are NPT trials specific? Placebo
PT Surgery Rehabilitation, education
Placebo Matching placebo
Simulated procedure Sham intervention
Example Saline solution in identical syringes
Sham arthroscopy* Sham education program with similar frequency
and duration of sessions but with
different content **
Issue - Ethical issue False negative
* Moseley et al., NEJM 2002
**Edworthy et al. J Rheumatol 1999
10
Are NPT trials specific?Placebo
Paterson, C. et al. BMJ 2005
Characteristic element(e.g., content of the education program)
Incidental element (e.g., healthcare providers – patients’ interaction effect)
Incidental element (placebo effect)
Effect not related to treatment (e.g., natural course of the disease, regression to mean )
Real treatment
effect
Measured treatment
effect
Characteristic element(e.g., content of the education program)
Incidental element (e.g., healthcare providers – patients’ interaction effect)
Incidental element (placebo effect)
Effect not related to treatment (e.g., natural course of the disease, regression to mean )
Real treatment
effect
Measured treatment
effect
11
Are NPT trials specific? Placebo
• Externally clearly different placebo
– Physiotherapy for knee OA* • Placebo treatment consisted of sham
ultrasonography
– Assessment of lifestyle advice**• Placebo treatment consisted of a syrup placebo
*Bennell, Ann Rheum Dis, 2005**Spigt, JCE, 2005
12
Are NPT trials specific? The methods of blinding – Full blinding
• Full blinding attempted • externally identical placebo/control
• Unblinded care providers not involved in the subsequent patient care
• Example– Transplantation of embryonic dopamine neurons**
• four twist-drill holes made through the frontal bone after local anesthesia
• the dura mater was not penetrated
* Boutron, Ravaud, Submitted** Freed, NEJM, 2001
Study cohort: 145 non-pharmacological trials (2004)*
13
Are Non pharmacological trials specific? The methods of blinding – Partial blinding
• Partial blinding attempted (blind trial hypothesis)
– Externally clearly different placebo / control
– Manipulation of information • Patients not informed of the existence of a placebo
• Patients not aware of the nature of the placebo
• Confidence in treatments sometimes tested
14
Are Non pharmacological trials specific? The methods of blinding – Partial blinding
• Zelen design or modified zelen design– Usual care compared to a complex, physical
therapy-based intervention for patello-femoral joint osteoarthritis of the knee
• 1) Researchers invited patients to participate in a cohort.
• 2) Randomization
• 3) Patients randomized to the intervention arm informed that they would receive the experimental treatment and signed a second consent form
Quilty, J Rheumatol. 2003
15
Are Non pharmacological trials specific? The methods of blinding – Outcome assessors• Blinding of outcome assessors
– Centralized assessment (video, audiotape or photography )
– Outcome assessors blinded of study hypothesis• To assess the analgesic effect of breast feeding in term neonates
compared to mothers’ arms, pacifiers, placebo (i.e., sterile water) or glucose
– Videotape of the children during the painful procedure– Two specially trained observers independently assessed the recordings using
a specific scale. – Observers were blinded to the purpose of the study and had been told that
they were assessing agreement of their scores in different situations
– Patients reported outcomes• Blinding is impossible if patients cannot be blinded
Carbajal R, Bmj. 2003
16
Are NPT trials specific? Complex intervention
• Several components– Rehabilitation : exercises, drugs, education etc
• Description of the intervention– Quantitative data
• Number of sessions, timing of each session, duration of each session
– Qualitative data• Content of each session, how it is delivered, supervision, content of
information exchanged etc
• Standardization procedure– Specific training– Quality control procedures
• Potential gap between the intended intervention (as described in the protocol ) and the actual administered intervention
• Barrier for systematic reviews
17
Are NPT trials specific? Influence of care providers
• Systematic review of the surgical and medical volume-outcome literature (Jan 1, 1980 - Dec 31, 2000)*– 71% of hospital volume-outcomes studies positive (88/124)
– 70% of physician volume-outcomes studies positive (31/44)
– No studies showed the opposite relationship
– Relationship strongest for high risk/rare surgeries:• Pancreas/esophagus cancer, pediatric cardiac surgery• NNT at high v. low volume provider: 7 to 11
– Much more modest volume-outcome effect for common procedures (CABG, CEA, PTCA, breast/colon cancer)
• NNT: 62 to 500
*Halm et al, Ann Intern Med 2002
18
Are NPT trials specific?
• Specific issues in assessing NPT – Blinding– Placebo– Complexity of treatments– Healthcare providers’ influence
• Need of specific standards
19
Overview
• Specific methodological issues when assessing Nonpharmacological treatment
• Quality tools
• Reporting guidelines
20
Quality toolsWhy?
• Assessing the quality of reports of trials is particularly important – clinicians’ critical appraisal of healthcare
literature
– systematic reviews
• Accurate estimates of the treatment effect
21
Quality toolsWhy?
• Several quality tools• Few quality tools were developped according to
scientific standards– Jadad scale*– Delphi list**
• Validated quality tools were mainly developped in the context of PT– Importance of blinding– No item related to care providers and the complexity
of the intervention
** Jadad, Control Clin Trials, 1996* Verhagen, JCE, 1998
22
Development of a specific checklistThe example of CLEAR NPT
• Quality: Internal validity• Delphi consensus method• Selection of items
• From existing checklist or scales• From specific items• From the Collaborative Review Groups of the Cochrane
Collaboration recommandations• Interviews of clinicians
• Experts• Members of Collaborative Review Groups of the Cochrane
Collaboration (n=41)• Clinicians involved in RCTs assessing NPTs (n=58)• Methodologists, epidemiologists (n=55)
– 55 experts participated(36%)
Boutron & Ravaud, JCE, 2005
23
Decision of the steering comitteeSelection of items
with a score of 8/9for more than 40% of experts
Decision of the steering comitteeSelection of items
With a score of 9/9for more than 40% of experts
Decision of the steering comitteeSélection of all items
Addition of 2 items for randomisationAddition of 1 item for the intention-to-treat analyses
Final Checklist10 items
5 sub items
Checklist11 items
Checklist21 items
Initial checklist38 items
1st Round54 experts/55
2nd Round46 experts/55
3rd Round49 experts/55
24
CLEAR NPT
• Was the generation of allocation sequences adequate?• Was the treatment allocation concealed?• Were the main outcomes analyzed according to the
intention-to-treat principle?
• Were details of the intervention administered to each group made available?
• Were care providers’ experience or skill in each arm appropriate?
• Was participant (ie, patients) adherence assessed quantitatively?
• Was the follow-up schedule the same in each group?
25
CLEAR NPT
• Were participants adequately blinded?
• Were care providers or persons caring for the participants adequately blinded?
– If care providers and or participants were not adequately blinded• Were all other treatments and care (ie, co-interventions) the same in each
randomized group?
• Were withdrawals and lost to follow-up the same in each randomized group?
• Were outcome assessors adequately blinded to assess the primary outcomes? – If outcome assessors were not adequately blinded,
• Were specific methods used to avoid ascertainment bias (systematic differences in outcome assessment)
26
CLEAR NPT
• Adequate assessment of trials quality
• Improvement of critical appraisal of medical litterature
• Improvement of the quality of systematic reviews
27
Overview
• Specific methodological issues when assessing Nonpharmacological treatment
• Quality tools
• Reporting guidelines
28
Reporting guidelinesWhy?
• Critical appraisal of the quality of clinical trials is possible only if the design, conduct, and analysis of RCTs are thoroughly and accurately described in published articles
• Evidence of incomplete and inadequate reporting
29
Reporting guidelines
30
31
Reporting guidelines
32
Reporting guidelinesDissemination
• Publication– 11 journals– Over than 1000 citations
• Editors endorsements including the ICMJE (International Committee for Medical Journal Editors)
– instructions to authors– submission check-list
• Website– www.consort-statement.org
33
Reporting guidelinesDissemination
• Various extensions of the CONSORT Statement since the original version (parallel arm, superiority, efficacy)
• Cluster RCT
• Harm
• Herbal therapy
• Non inferiority trials
• Several other extensions are in progress• 2x2 factorial design
34
Reporting guidelines for NPT trials Why?
• 119 RCTs published in 1998 and 1999 in three major cardiothoracic journals (Annals of Thoracic Surgery, European Journal of Cardio-thoracic Surgery, The Journal of Thoracic and Cardiovascular Surgery)
• 70 % of RCTs fulfilled less than half of the CONSORT criteria
Anyannu, Eur J of Cardio-thoracic Surgery, 2004
35
Reporting guidelines for NPT trials Why?
NPT PT
Blinding of patients 46% 98%
Blinding of healthcare providers 43% 83%
Blinding of outcome assessors 72% 98%
Study of trials assessing PT and NPT of hip and knee osteoarthritis (1992-2002)
When blinding was judge possible, blinding was less often
reported in NPT trials
* Boutron & Ravaud, Jama, 2003
36
Reporting guidelines for NPT trials Why?
Study of surgical papers (n = 158) published in 2004*
• Reporting of details of the intended intervention– Surgical procedure 87 %– Pre-operative care 15 %– Anesthesia 35 %– Post-operative care 49 %
• Surgeons– Selection criteria for surgeon 40 %– Number of interventions performed
by surgeon 11 %– Number of surgeons involved 33 %
* Jacquier I, Boutron I, Ravaud P, Ann. Surgery
37
Reporting guidelines for NPT trials Why?
Study of rehabilitation papers (n = 171) published between 1997-1998*
• Timing of the intervention 68 %
• Description of the intervention 50 %
• Intervention’s adherence 34%
* Dijkers at al. Am J Phys Med Rehabil, 2002.
38
Reporting guidelinesDevelopment of the extention of the CONSORT to NPT
Steering committee– D. Moher– P. Ravaud– I. Boutron
Selection of experts– Editors– Clinicians– Methodologists
Two stages method– Preliminary survey
– Consensus meeting
39
Preliminary survey
• Objective– To identify items to be discussed based the
experts’ opinion
• Methods– All items of the CONSORT checklist– 7 specific questions on key issues were added– Items were selected for discussion if more than
1/3 of the experts answered that the item should be modified or another item should be added
40
Consensus meeting
• 3 days– February 8-10th 2006, Paris
• 30 experts
• Agenda– Presentations– Discussions– Consensus
• Manuscript in process
41
Conclusions
• Specific issues in assessing NPT– Placebo– Blinding– Complexity of the intervention– Care providers’ influence
• Specific guidelines– Specific quality tools
• CLEAR NPT
– Specific reporting guidelines• Development of an extension of the CONSORT for
NonPharmacological treatment (CONSORT meeting, Paris, 2006)