Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
Cluster Randomised Trials:
Sources of Bias
Ada Keding
Professor David Torgerson
Problems with Cluster
Randomisation• Possible Selection Bias;
• Inadequate uptake of intervention by either
group reduces study power;
• Sample size needs to be increased
(typically between 50% to 100%), which
will often increase the cost and complexity
of a trial.
Selection Bias in Randomised
Trials• This should not occur in an individually
randomised trial unless the randomisation has been subverted.
• However, in cluster trials it is possible for selection bias to occur after successful cluster randomisation.
• This defeats the objective of randomisation.
Selection Bias in Cluster Trials
• Randomisation abolishes selection bias in
a cluster trial ONLY if all eligible cluster
members, or a random sample, within the
cluster are included in the trial.
Recruitment into cluster trials
• A key issue is individual participant
recruitment into cluster trials.
• There are a number of ways where biased
participant recruitment can occur, which
can lead to baseline imbalances in
important prognostic factors.
UK BEAM Trial
• The UKBEAM pilot study used a cluster design. Eligible patients were identified by GPs for trial inclusion.
• GP practices were randomised to usual care or extra training.
• The ‘primary care team’ were trained to deliver ‘active’ management of backpain.
UK BEAM Selection bias
• The pilot showed that practices allocated
to ‘active management’ were more adept
at identifying patients with low back pain
and including them in the trial.
• Patients had different characteristics in
one arm than the other.
UK BEAM participant
recruitment
Roland = 8.9
Aberdeen = 28.6
SF36 = 61.8
165 Recruited Participants
13 Active Management
102,063 registered patients
Roland = 10.3
Aberdeen = 34.2
SF36 = 55.2
66 Recruited Participants
13 Usual Care
106,834 registered patients
26 Practices
Type title here
P = 0.025
P = 0.001
P = 0.03
UKBEAM pilot study.
Recruitment by Practice Status
0
20
40
60
80
100
120
140
160
180
Apr 98 May 98 Jun 98 Jul 98 Aug 98 Sep 98 Oct 98 Nov 98 Dec 98 Jan 99 Feb 99 Mar 99 Apr 99
Nu
mb
er o
f p
arti
cip
an
ts
Another musculoskeletal trial
• SAPPHIRE trial of guidelines educating GPs on identification and management of shoulder pain
• Very similar design to UKBEAM
• At first TMC evidence of recruitment bias occurring – independent members of TSC recommended continuation of trial
Feb
Mar
April
May
June
July
Aug
Sept
OctNov
Dec
JanFeb
Mar
April
May
June
July
Cumulative actual - untrained 4 10 13 19 25 26 27 27 28 35 38 40 41 43 44 44 47
Cumulative actual - trained 10 22 29 37 44 50 57 66 67 73 78 83 88 93 97 104 104
0
20
40
60
80
100
120
Nu
mb
ers
recru
ited
Month of recruitment
Cumulative actual - untrained
Cumulative actual - trained
Review of Cluster Trials
• Because of the ‘BEAM’ problem York
Trials Unit decided to undertake a
methodological review of cluster trials.
• They identified all cluster trials published
in the BMJ, Lancet, NEJM since 1997.
Puffer et al. BMJ 2003;327:785.
Results
• We identified 36 relevant trials. ONLY 13 had
identified participants prior to randomisation.
• Of the 23 not identifying participants a priori 7
showed evidence of differential recruitment or
consent.
• Other biases included differential of inclusion
criteria or attrition.
• In total 14 (39%) showed evidence of bias.
Misleading trial
• One trial (Jorhdoy, Lancet 2000) where there
was no evidence of biased recruitment was later
found to have suffered recruitment bias in
another publication.
• This was an RCT for home care for terminally ill
patients.
• There was, no evidence, in the Lancet paper of
a problem. BUT…
Baseline Characteristics
Intervention Control
Live in Flat 40% 23% P < 0.001
Married 67% 59% P = 0.07
Access to
help
80% 70% P = 0.04
P values adjusted for clustering.
Jordhoy Palliative Medicine 2002 16:43-49.
Study Were patients
identified prior to
randomisation?
Identification prior to
randomisation possible?
Is recruitment bias likely?
Althabew1 No No Not likely –
Baquiw2 No No Not likely –
Bellaryw3 Yes Yes Not possible
Campbellw4 No Yes - Possible –
Cheynew5 No Yes Possible –
Clarkew6 No Yes - Possible –
Cummingw7 No No Not likely –
Daviesw8 No No Possible –
Doigw9 No No Not likely –
Hiscockw10 Yes Yes Not possible
Jewkesw11 No Yes Possible –
Jeyaratnamw12 No No Possible –
Kersew13 Yes Yes Not possible
Kumarw14 Unclear No Unclear
Melesew15 Yes Yes Not possible
Moetw16 Yes Yes Not possible
Pasanenw17 Yes Yes Not possible
Rahmanw18 No No Not likely –
Ruelw19 No Yes – Not likely –
Soligardw20 Yes Yes Not possible
SUMMITw21 Yes Yes Not possible
Tolw22 No Yes. Possible –
Woodw23 No Yes – Possible
Zengw24 Yes Yes Not possible
2008 cluster trials
Summary of 2008 trials
• Of the 24 trials, eight could have used
individual randomisation as an alternative
to cluster allocation. Seven studies could
have recruited participants prior to cluster
randomisation but did not. In 8 studies
where recruitment bias was possible more
than half (5) demonstrated some evidence
of differential recruitment rates.
Brierley et al, 2012 J Eval in Clin Prac 18:878.
Copyright ©2009 BMJ Publishing Group Ltd. Morrell, C J. et al. BMJ 2009;338:a3045
Fig 1 Trial profile of clusters, all women and women with six week EPDS score >=12 in control and intervention groups. Note: 597 women were not sent 12 month questionnaire as their baby was <12 months, and 1879 women were not sent 18 month questionnaire as
their baby was not 18 months when follow-up interval ended
Note: 35 per
cluster for
control group
and 44 per
cluster for
intervention –
26% difference
Continuing evidence of a
problem• In 2018 is there still a problem with cluster
randomised trials?
• Let’s look at the latest research on this
from Dept of Health Sciences at York.
Meta-analysis of age for
individually randomised trials
Difference = 0.005
(95% CI -0.026 to 0.035)
I2 = 0%
Meta-analysis of cluster trials of
age
Difference in age: -
0.05
(95% CI -0.057 to -
0.0426)
I2 = 93.2%
Bolzern et al
2018 – J
Clin Epi – in
press
Hip Protector Trial
650 in hip protector group
8 Clusters
1075 in control group
15 Clusters
1725 eligible participants
Kannus. N Eng J Med 2000;343:1506.
At this point trial is balanced for all co-variates
Hip Protector Trial
446
At baseline
69%
204 refused
(31%)
650 in hip protector group
8 Clusters
981
At baseline
91%
94 refused
(9%)
1075 in control group
15 Clusters
1725 eligible participants
Selection Bias
Figure 2
The Lancet 2016 388, 2264-2271DOI: (10.1016/S0140-6736(16)30384-1)
Copyright © 2016 Elsevier Ltd Terms and Conditions
Figure 1
The Lancet 2016 388, 2264-2271DOI: (10.1016/S0140-6736(16)30384-1)
Copyright © 2016 Elsevier Ltd Terms and Conditions
Summary
• Cluster trials are open to bias and too
many researchers do not design them
sufficiently well to avoid their vulnerability
to selection bias.
• Stepped wedge studies will probably be
more at risk than ‘normal’ cluster trials as it
may be more difficult to recruit before
randomisation.