Journal of Clinical Epidemiology 67 (2014) 667e671

ORIGINAL ARTICLES

A theoretical analysis showed that blinding cannot eliminate potential forbias associated with beliefs about allocation in randomized clinical trials

Erin Mathieua, Robert D. Herbertb,*, Kevin McGeechanc, Jemma J. Herbertd,e,Alexandra L. Barrattc

aSchool of Medicine, University of Western Sydney, Campbelltown, NSW 2560, AustraliabNeuroscience Research Australia, Barker St, Randwick, City Road, Camperdown, NSW 2031, AustraliacSydney School of Public Health, University of Sydney, City Road, Camperdown, NSW 2006, Australia

dFaculty of Science, University of Sydney, City Road, Camperdown, NSW 2006, AustraliaeFaculty of Engineering and Information Technologies, University of Sydney, City Road, Camperdown, NSW 2006, Australia

Accepted 3 February 2014

Abstract

Objectives: To explore the theoretical justification for blinding in randomized trials and make recommendations concerning the imple-mentation and interpretation of blinded randomized trials.

Study Design and Setting: A theoretical analysis was conducted of the potential for bias in randomized trials with successful blinding(ie, trials in which beliefs about allocation to treatment or control groups are independent of actual allocation). The analysis identified con-ditions that must be satisfied to ensure that blinding eliminates the potential for bias associated with beliefs about allocation.

Results: Even when beliefs about allocation are independent of actual allocation, they can still cause bias. The potential for bias iseliminated when the belief is uniformly one of complete ambivalence about allocation.

Conclusion: Even when blinding succeeds in making beliefs about allocation independent of actual allocation, beliefs about allocationmay still cause bias. It is difficult to determine the extent of bias in any particular trial. Bias could be eliminated by establishing a state ofcomplete ambivalence about the allocation of every trial participant, but universal ambivalence may be difficult to achieve and may reducethe generalizability of the trial’s findings. � 2014 Elsevier Inc. All rights reserved.

Keywords: Blinding; Masking; Clinical trials; Randomized controlled trials; Clinical trials; RCTs; Bias

1. Introduction

Blinding is a ubiquitous feature of contemporary clinicaltrials. In this article, we explore the theoretical justificationfor blinding in randomized trials and make recommenda-tions about how blinding should be implemented.

Blinding involves manipulating or concealing informa-tion about which group individual trial participants are allo-cated to. (Typically, in randomized trials, participants areallocated to treated and control groups, although in sometrials, the allocation is to groups receiving different activeinterventions. Here, for convenience, we will refer totreated and control groups.) Any person involved in the trialcould be blinded, but it is usually considered particularlyimportant to blind trial participants, the clinicians who treat

Conflict of interest: None.

* Corresponding author. Tel.: þ61-2-9399-1833.

E-mail address: r.herbert@neura.edu.au (R.D. Herbert).

0895-4356/$ - see front matter � 2014 Elsevier Inc. All rights reserved.

http://dx.doi.org/10.1016/j.jclinepi.2014.02.001

trial participants, and the assessors who ascertain and docu-ment trial outcomes.

The aim of blinding is to prevent bias caused by beliefsabout participants’ allocations [1,2]. Beliefs about alloca-tion could bias estimates of the effects of intervention inseveral ways: trial participants who believe they were allo-cated to the treated group might experience placebo effects;in trials with self-reported outcomes, participants’ beliefsabout allocation could influence reports of outcomes; clini-cians could provide different cointerventions to trial partic-ipants who they believe have been allocated to treated andcontrol groups; and assessors could measure or report trialoutcomes differently in trial participants who they believehave been allocated to treated and control groups. Impor-tantly, these biases are mediated by beliefs about allocationand not necessarily by knowledge of allocationdthe beliefsneed not be correct to produce bias [1].

A range of procedures can be used to blind trial partic-ipants, clinicians, and assessors. The most widely used pro-cedures involve providing a placebo or sham intervention to

668 E. Mathieu et al. / Journal of Clinical Epidemiology 67 (2014) 667e671

What is new?

Key findings� In clinical trials, blinding cannot generally be guar-

anteed to prevent bias caused by beliefs aboutallocation.

� Theoretically, the potential for bias caused by be-liefs about allocation could be eliminated if allpeople associated with a trial (including all partic-ipants, clinicians and assessors) were ambivalentabout the allocations of all trial participants.

What this adds to what was known?� Meta-epidemiologic studies suggest that adequately

blinded trials are less biased than inadequately blindedtrials.

What is the implication and what should changenow?� Bias could be eliminated by establishing a state of

complete ambivalence about the allocation ofevery trial participant, but universal ambivalencemay be difficult to achieve and may reduce thegeneralisability of the trial’s findings.

participants in the control arm of the trial and restricting ac-cess to information about the allocations of individualparticipants.

In practice, it may be difficult to blind participants, cli-nicians, and assessors to all participants’ allocations. Thenature of some interventions, particularly nonpharmaco-logic interventions, can make the provision of a placeboor sham intervention difficult or impossible. If the placeboor sham intervention is poorly designed or poorly imple-mented, it may be possible for participants, clinicians, orassessors to distinguish the placebo or sham from the activeintervention. There may be procedural failures that causeparticipants, clinicians, or assessors to be informed of anindividual trial participant’s allocation. And some interven-tions have such large therapeutic effects or such distinctive

Table 1. Statistical notation

Belief about allocation (i)Believes was allocated to the ‘‘treated’’ group (t)‘‘Uncertain’’ about allocation (u) OBelieves was allocated to the ‘‘control’’ group (c) O

Whole group

Each trial participant falls into one of six cells depending on his or her acallocation (row i, values t, u, or c). True outcomes are denoted by O, and obseabout allocation that is independent of the true outcome, and b$Oij is that poutcome.

side effects that it may become obvious when a participanthas been allocated to the active arm of the trial. Nonethe-less, it may be possible to achieve complete or near-complete blinding in some trials. A trial may be said tohave achieved the goal of blinding if the beliefs of peopleinvolved in the trial (participants, clinicians, or assessors)about participants’ allocations are independent of the par-ticipants’ actual allocations.

The theoretical justification for implementing blinding inclinical trials is rarely explicitly articulated but appears to bethat blinding can prevent bias that might otherwise arise as aresult of placebo effects, differential provision of cointer-ventions, and differential ascertainment of outcomes [3].In the following section, we examine that rationale by iden-tifying the conditions under which blinding ensures unbi-ased estimates of effects of intervention. We show thatbeliefs about allocation may cause bias even when blindingis successful (ie, even when beliefs about participants’ allo-cations are independent of actual allocation).

2. Identification of conditions that ensure there is nobias due to beliefs about allocation

Consider a two-armed trial in which each participant israndomly allocated to group j (either a treated group T or acontrol group C; Table 1). People associated with the trial(these people could be participants, clinicians, or asses-sors) may have varying beliefs, i, about each participant’sallocation. We consider the situation in which there arethree strata of belief, although the approach used here gen-eralizes easily to the situation in which there are manystrata of belief. The three strata of belief are belief thatallocation was to the treated group (t), belief that allocationwas to the control group (c), or ambivalence about alloca-tion (u). Thus, each participant falls into one of the sixcells in Table 1; the particular cell depends on the partici-pant’s actual allocation (column j, with values T or C ) andthe belief about the participant’s allocation (row i, withvalues t, u, or c). The number of participants in each cellis nij, and the statistic summarizing the outcome of eachcell is Oij. Outcomes may be measured on a continuousscale, in which case the summary statistic, Oij, is a mean.Alternatively, the outcome may be binary (coded 0 or 1), inwhich case Oij is a proportion. The true outcome in any

Allocation ( j )

Treated (T ) Control (C )

O 0tT5OtT þ at þ bt$OtT O 0

tC5OtC þ at þ bt$OtC0uT5OuT þ au þ bu$OuT O 0

uC5OuC þ au þ bu$OuC0cT5OcT þ ac þ bc$OcT O 0

cC5OcC þ ac þ bc$OcC

O 0T5Siðni$O

0iT Þ=Sini O 0

C5Siðni$O0iCÞ=Sini

tual allocation (column j, values T or C ) and the belief about his or herrved outcomes are denoted by O0. a is that part of the bias due to beliefart of the bias due to belief about allocation that is proportional to the

669E. Mathieu et al. / Journal of Clinical Epidemiology 67 (2014) 667e671

cell, Oij, is not observed. Instead, we observe O0ij, which is

the true outcome distorted by error due to belief about allo-cation. The error can be considered to consist of two parts:aij, an error that is independent of the true outcome and bij,an error that is proportional to the outcome. aij and bij willusually be unknown. The total error due to belief aboutallocation in any cell is aij þ bij$Oij.

In the following analyses, it is assumed that the goal ofblinding has been achieved. That is, it is assumed that be-liefs about allocation are independent of actual allocation.As a result of randomization and effective blinding, there isan expectation of balance within each stratum of belief.Consequently we expect that within each stratum, therewill be the same number of treated and control participants(niT 5 niC 5 ni) and there will be balance in all covariatesand covariate interactions established before randomiza-tion. In the following analysis, we will assume that exactbalance has been realized, recognizing that in individualtrials, the actual values will be subject to sampling error.As the goal of blinding has been achieved, the degree towhich outcomes are distorted by beliefs about allocationdepends on the stratum of belief but not on allocation.That is, aiT 5 aiC 5 ai and biT 5 biC 5 bi. SoO0

ij5Oij þ ai þ bi$Oij:We now consider the conditions that must be satisfied to

ensure that estimates of the effect of intervention are notbiased by beliefs about allocation. We separately considerconditions required for unbiased estimates of absolute andrelative effects of intervention.

2.1. Absolute estimates of effect of intervention

The absolute effect of intervention is estimated by takingthe difference in the summary statistics of the treated andcontrol groups. When the summary statistic is a mean, thisyields the mean effect of intervention, and when the sum-mary statistic is a proportion, this yields the absolute riskreduction. The mean effect or absolute risk reduction is un-biased by beliefs about allocation if and only ifO0

C � O0T5OC � OT : O

0C is the niC-weighted mean of all

O0iC and O0

T is the niT-weighted mean of all O0iT . So this

equality can be rewritten in terms of ni and Oij:

ð

Sini½OiC þ ai þ biOiC�Þ=Sini

� ðSini½OiT þ ai þ biOiT �Þ=Sini

5 ðSiniOiCÞ=Sini � ðSiniOiTÞ=Sini

which simplifies to:

SinibiOiC �SinibiOiT 50 ð1Þ

Equation 1 implies that the mean effect or absolute risk

reduction is unbiased by beliefs about allocation if, but notonly if, in every stratum, either (1) the degree to which be-liefs about allocation distort the trial outcome is independentof the magnitude of the outcome (ie, bi 5 0) or (2) there isno effect of intervention (ie, OiC 5 OiT). When the first

condition is satisfied, outcomes of the treated and controlarms of the trial are distorted by the same absolute amount,if at all, so estimates of the absolute effect of intervention(the difference between treated and control group outcomes)are unbiased. When the second condition is satisfied, theextent to which outcomes are distorted by beliefs about allo-cation scales with outcome, and the extent of the scaling isthe same in both groups, so estimates of the absolute effectof intervention are unbiased. Either of these two conditionsis sufficient to ensure that estimates of the absolute effectof intervention are unbiased, but they are not the only condi-tions under which the estimates can be unbiased. It ispossible that even when neither condition is satisfied, thebiases in subgroups with particular beliefs about allocationfortuitously cancel out the biases in other subgroups, so thatthe overall estimate of the absolute effect is unbiased.

Unfortunately, it is not usually possible to know, in anyparticular trial, whether these conditions have been satisfied,so it is not usually possible to know whether estimates of theabsolute effect of intervention in any particular trial are un-biased. This is because there is not yet any procedure to es-timate, in any particular trial, whether the distortions in trialoutcomes are caused by beliefs about allocation scale withthe magnitude of the outcomes. Nor is it possible to deter-mine if the intervention truly has no effectdwe only haveaccess to estimates of the effect of intervention that arepotentially biased by beliefs about allocation, not estimatesof the true effect of intervention. So, in general, it is notpossible to know, for any particular trial, if the estimated ab-solute effect of intervention is unbiased by beliefs aboutallocation. That is true even when, as has been assumedhere, the goal of blinding has been achieved and beliefsabout allocation are independent of actual allocation.

2.2. Relative estimates of effect of intervention

The effect of intervention may also be expressed in rela-tive terms. Here, we focus on the relative risk, estimated bytaking the ratio of the summary statistics of the treated andcontrol groups. The observed relative risk is unbiased bybeliefs about allocation if and only if O0

T=O0C5OT=OC:

This equality can be rewritten in terms of ni and Oij:

ðSini½OiTþaiþbiOiT �Þ=SiniðSini½OiCþaiþbiOiC�Þ=Sini

5ðSiniOiTÞ=SiniðSiniOiCÞ=Sini

ðSini½OiT þ ai þ biOiT �ÞðSini½OiC þ ai þ biOiC�Þ5

ðSiniOiTÞðSiniOiCÞ ð2Þ

Equation 2 implies that the relative risk is unbiased bybeliefs about allocation if, but not only if, in every stratum,either (1) the degree to which beliefs about allocationdistort the trial outcome is proportional to the magnitudeof the outcome (ie, ai 5 0) or (2) there is no effect of inter-vention (ie, OiC 5 OiT). As with absolute estimates of ef-fect, the relative risk could also be unbiased if the biasesin subgroups with particular beliefs about allocation

670 E. Mathieu et al. / Journal of Clinical Epidemiology 67 (2014) 667e671

fortuitously cancel out biases in other subgroups. But again,it is not possible to determine if, in any particular trial, theconditions required to obtain an unbiased relative risk havebeen satisfied. There can be no guarantee that the estimatedrelative risk is not biased by beliefs about allocation, evenwhen the goal of blinding has been achieved.

3. Implications for implementation of blinding

A key issue in the design of blinded trials concerns theimmediate objective of blinding. Should the immediateobjective be to make people involved in the trial (partici-pants, clinicians, and assessors) believe that all trial partic-ipants were in the treated group? (After all, the shamintervention provided in clinical trials is almost always de-signed to make the control intervention appear like theexperimental intervention.) Or should, alternatively, theobjective be to generate complete ambivalence in whichthere is no tendency to believe that any individual partici-pant received the experimental or control intervention? Orshould the objective be simply to establish the same patternsof belief regarding the allocations of participants in thetreated and control groups, even if that meant there was arange of beliefs regarding the allocation of each participant?This last objective is the minimum condition that must besatisfied to claim that beliefs about allocation are indepen-dent of actual allocation and therefore that the goal of blind-ing has been achieved. To our knowledge, there has beenlittle or no explicit analysis in the literature on blindingabout what the immediate objective of blinding should be.

Insofar as the role of blinding is to reduce bias caused bybeliefs about allocation, the immediate objective of blindingshould be to establish a state of ambivalence concerning theallocation of all participants. That is because errors in out-comes caused by beliefs about allocation are likely to besmallest when there is complete ambivalence about alloca-tions. Indeed, it may be valid to argue that, by definition,belief about allocation cannot distort trial outcomes whenthe belief is one of complete ambivalence. If we were pre-pared to accept that ambivalence does not distort trial out-comes, and if in a particular trial, we could establishcomplete ambivalence about the allocation of every partici-pant in the trial, then we could be confident that beliefs aboutallocation could not bias estimates of the mean effect or ab-solute risk reduction or relative risk (because both Equations1 and 2 are satisfied when all i 5 u and au 5 bu 5 0).

There are twoproblemswith the objective of establishing astate of ambivalence concerning the allocation of all partici-pants. The first is that this objectivemay be difficult or impos-sible to achieve. Even when attempts are made to disguiseallocation, for examplewith a sham intervention, participants,clinicians, and assessors may develop ‘‘hunches’’ (beliefs)about which group particular participants were allocated to[2]. Hunchesmay be insightful: theymay be based on percep-tible differences between real and sham interventions or, ifthe intervention is effective, hunches could be based on

knowledge of outcomes [4]. Alternatively, hunches could bemisleading: people may develop beliefs about interventionthat are based on irrelevant observations. It is likely that peo-ple develop beliefs based on irrelevant observations becauseoften participants, clinicians, and assessors have incorrect be-liefs about allocation. To the extent that participants, clini-cians, and assessors are inclined to develop either insightfulor misleading beliefs about allocation, the objective ofinducing a state of ambivalence concerning allocation willnot be achieved. A second problem with the objective of es-tablishing a state of ambivalence concerning the allocationof all participants is that the goal of reducing bias (maxi-mizing internal validity) may come at the cost of generaliz-ability (maximizing external validity). That is because theultimate aim of clinical trials must be to determine the effectof intervention (the difference between outcomes with andwithout intervention) in people believed to have been treated.Trials havegreater external validity if it is believed that all par-ticipants in the trial were treated.

Patient preference trials [5e8] and two-stage randomizedtrials [9,10] provide insights into the extent to which the ef-fects of treatment are associated with preexisting preferencesfor one treatment rather than another. These designs aregenerally more useful when the comparison is betweentwo active treatments rather than between an active and aplacebo or sham treatments. Potentially, such trials provideestimates of treatment effects in those who do not have atreatment preference. To the extent that bias in outcomesassociated with beliefs in outcomes is mediated by treatmentpreferences, estimates obtained from preference trials oftreatment effects in those who have no preference may beless biased than estimates derived from a mixed population.However, bias in outcomes associated with beliefs aboutallocation may be mediated by mechanisms that are indepen-dent of treatment preference (eg, bias could occur becausetrial participants report what they think the investigatorswant to hear). So, although the estimates obtained from pref-erence trials of the effects of treatment in people withouttreatment preference may be less biased than estimatesderived from a mixed population, they are not guaranteedto be free of bias associated with beliefs about allocation.

4. Concluding comments

Clinical trialists can control what people involved in atrial know about the allocation of trial participants by im-plementing procedures such as the use of a placebo or shaminterventions. But bias is caused by beliefs about allocation,not by knowledge of allocation, and beliefs need not bebased on knowledge. As a consequence, blinding is animperfect mechanism for controlling bias associated withbeliefs about allocation: blinding can reduce bias, but itusually cannot ensure complete control of bias because ofbeliefs about allocation.

We have shown that even when the goal of blinding isachieved and beliefs about allocation are independent of

671E. Mathieu et al. / Journal of Clinical Epidemiology 67 (2014) 667e671

actual allocation, estimates of effect of treatment may still bebiased by beliefs about allocation. This does not mean that itis not advisable to implement blinding in clinical trials.Blinding controls knowledge of allocation, and knowledgeof allocation influences beliefs about allocation. In theabsence of knowledge of allocation, beliefs about allocationwill be independent of actual allocation. And, although theremay still be bias when beliefs about allocation are indepen-dent of actual allocation, this bias is likely to be less thanwhen belief about allocation is not independent of actualallocation. Consequently, the potential for bias caused by be-liefs about allocation is likely to be minimized when blind-ing ensures beliefs about allocation are independent of actualallocation. This theoretical conclusion is consistent with thelimited empirical data: meta-epidemiological studies providesome empirical evidence that adequately blinded trials areless biased than inadequately blinded trials [11e14].

In conclusion, we have shown that blinding cannot beguaranteed to prevent bias caused by beliefs about alloca-tion. In general, it is not possible to know, for a particulartrial, the extent to which estimates of the effect of interven-tion are biased by beliefs about allocation. Researchers whoimplement blinding in trials with the intention of mini-mizing bias should strive to generate a state of completeambivalence about each participant’s allocation in trial par-ticipants, clinicians, and assessors because this state mini-mizes potential for bias. In practice, such a state may bedifficult to achieve and may compromise the generaliz-ability of the trial’s findings. We emphasize it is not ourintention to suggest that clinical trials should not be blinded.Instead, we have attempted to elucidate how blinding canprevent bias and the conditions that must be satisfied if thatobjective is to be achieved. Readers may find this analysishelpful when designing and interpreting clinical trials.

Appendix

Supplementary data

Supplementary data related to this article can be found athttp://dx.doi.org/10.1016/j.jclinepi.2014.02.001.

References

[1] Rees JR, Wade TJ, Levy DA, Colford JM Jr, Hilton JF. Changes in

beliefs identify unblinding in randomized controlled trials: a

method to meet CONSORT guidelines. Contemp Clin Trials 2005;

26:25e37.

[2] Sackett DL. Measuring the success of blinding in RCTs: don’t, must,

can’t or needn’t? Int J Epidemiol 2007;36:664e5.

[3] Herbert RD, Jamtvedt G, Mead J, Hagen KB. Practical evidence-

based physiotherapy. 2nd ed. Oxford, UK: Elsevier; 2011.

[4] Day SJ, Altman DG. Blinding in clinical trials and other studies. BMJ

2000;321:504.

[5] King M, Nazareth I, Lampe F, Bower P, Chandler M, Morou M, et al.

Impact of participant and physician intervention preferences on ran-

domized trials: a systematic review. JAMA 2005;293:1089e99.

[6] Preference Collaborative Review Group. Patients’ preferences within

randomised trials: systematic review and patient level meta-analysis.

BMJ 2008;337:a1864.

[7] Franco MR, Ferreira ML, Ferreira PH, Maher CG, Pinto RZ,

Cherkin DC. Methodological limitations prevent definitive conclu-

sions on the effects of patients’ preferences in randomized clinical tri-

als evaluating musculoskeletal conditions. J Clin Epidemiol 2013;66:

586e98.

[8] Prady SL, Burch J, Crouch S, MacPherson H. Insufficient evidence to

determine the impact of patient preferences on clinical outcomes in

acupuncture trials: a systematic review. J Clin Epidemiol 2013;66:

308e18.

[9] McCaffery KJ, Turner R, Macaskill P, Walter SD, Chan SF, Irwig L.

Determining the impact of informed choice: separating treatment ef-

fects from the effects of choice and selection in randomized trials.

Med Decis Making 2011;31:229e36.

[10] R€ucker G. A two-stage trial design for testing treatment, self-

selection and treatment preference effects. Stat Med 1989;8:

477e85.

[11] Nuesch E, Reichenbach S, Trelle S, Rutjes AW, Liewald K, Sterchi R,

et al. The importance of allocation concealment and patient blinding

in osteoarthritis trials: a meta-epidemiologic study. Arthritis Rheum

2009;61:1633e41.

[12] Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of

bias. Dimensions of methodological quality associated with estimates

of treatment effects in controlled trials. JAMA 1995;273:408e12.

[13] Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Sta-

tistical methods for assessing the influence of study characteristics on

treatment effects in ‘meta-epidemiological’ research. Stat Med 2002;

21:1513e24.

[14] Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, et al.

Empirical evidence of bias in treatment effect estimates in controlled

trials with different interventions and outcomes: meta-epidemiological

study. BMJ 2008;336:601e5.