The implications of publication bias for meta-analysis' other parameter

STATISTICS IN MEDICINEStatist. Med. 2006; 25:2911–2921Published online 12 December 2005 in Wiley InterScience (www.interscience.wiley.com). DOI: 10.1002/sim.2293

The implications of publication bias for meta-analysis’other parameter

Dan Jackson∗;†

Centre for Operational Research and Applied Statistics; University of Salford; Manchester M5 4WT; U.K.

SUMMARY

Perhaps the greatest threat to the validity of a meta-analysis is the possibility of publication bias, wherestudies that are interesting or statistically signi�cant are more likely to be published than those withless encouraging results. In particular, there is the concern that this bias might be ‘one-sided’, wherestudies indicating that the treatment is bene�cial have a greater probability of publication. The impactthat this type of bias has on the estimate of treatment e�ect has received a great deal of attention butthis also has implications for estimates of between-study variance. Using step functions to model thebias it can be demonstrated that it is impossible to make generalizations concerning how we shouldrevise estimates of between-study variance when presented with the possibility of publication bias. Todetermine this, assumptions must be made concerning the form that the bias takes, which is unknownin practice. Copyright ? 2005 John Wiley & Sons, Ltd.

KEY WORDS: meta-analysis; publication bias; between-study variation

1. INTRODUCTION

The use of meta-analysis to combine the results from separate studies concerned with the sametreatment or issue has become popular in many subject disciplines. Despite this, concernshave been raised regarding the application of these techniques [1]. For example, there is thedi�culty in attempting to combine obviously disparate results to produce a single estimate oftreatment e�ect. In order to model the apparent di�erences and yet still obtain a meaningfulestimate, the random e�ects model, described in detail in Section 2, has been suggested [2, 3].This incorporates a between-study variance, denoted by �2, which re�ects the di�erences inthe studies’ results which cannot be explained by within-study sampling variation alone. Thedi�culty with this approach lies in determining the value of �2, as the estimation of this islikely to be imprecise for meta-analyses with small numbers of studies. Despite this, �2 is

∗Correspondence to: Dan Jackson, Centre for Operational Research and Applied Statistics, University of Salford,Manchester M5 4WT, U.K.

†E-mail: [email protected]

Received October 2004Copyright ? 2005 John Wiley & Sons, Ltd. Accepted April 2005

2912 D. JACKSON

typically estimated and then assumed �xed and known for the purposes of pooling the resultsand making inferences [2].A further di�culty is presented by publication and related biases. This refers to the possi-

bility that studies with interesting or statistically signi�cant results may be more publishable.These published results are then more likely to be found and included in the resulting meta-analysis and if this type of bias has indeed occurred then the inferences made will inevitablybe misleading. There has been an awareness of bias in published results for some time [4];evidence that this is prevalent in medical statistics has also been provided [5, 6].In particular, there is the concern that the bias might be ‘one sided’, where studies which

indicate that the treatment is bene�cial are more likely to be published. For example, twoof the most recent developments in modelling publication bias, Copas’ method [7, 8] andTrim and Fill [9, 10], model the bias in this way. The main concern is the impact that thishas on inference concerning the treatment e�ect. However, since �2 is also estimated in arandom e�ects meta-analysis, publication bias impacts upon the assessment of this. Although�2 might be regarded as a nuisance parameter by some, in the context of meta-analysis itsmagnitude has important implications. For example, a small value relative to the studies’within-study variances means that the inevitable di�erences between the studies have notgreatly in�uenced the results from one study to the next. The most extreme instance of this iswhen �2 is assumed to be zero, which results in a �xed e�ects model. Such a model simpli�esthe interpretations, as this means that all studies have the same underlying treatment e�ect. Incontrast to this, if �2 is large then this indicates that there are potentially important di�erencesin the studies’ results. Measures have been developed in order to quantify the impact of thisheterogeneity [11] and the importance of exploring the reasons for this has been emphasized[12, 13].In this paper, the implications of one-sided publication bias for the estimate of �2 are

explored. The rest of the paper is set out as follows. In Section 2 the random e�ects model ispresented and applied to a familiar data set. In Section 3 a simple weight function is introducedto model one sided publication bias and in Section 4 the implications of this for the estimateof �2 are assessed. The results are illustrated using the example data set in Section 5. Finally,Section 6 provides some �nal comments, as well as ideas for further work.

2. THE RANDOM EFFECTS MODEL

A simple random e�ects meta-analysis assumes that yi|�i ∼N(�i; �2i ), where yi, �i and �2i arethe estimate of treatment e�ect, the true treatment e�ect and the within-study variance of theith study, respectively. It is assumed that the values of �2i are �xed and known [2] (but areestimated in practice) and that �i ∼N(�; �2), where � is the overall treatment e�ect and �2is the between-study variance. Marginally this provides yi ∼N(�; �2i + �2), and it is furtherassumed that the studies are independent. Conventionally �2 is estimated and then assumed�xed and known in the analysis or, at least, that this provides a suitable approximation. Thisresults in the point estimate of treatment e�ect

�̂=∑n

i=1(�2i + �

2)−1yi∑ni=1(�

2i + �2)−1

Copyright ? 2005 John Wiley & Sons, Ltd. Statist. Med. 2006; 25:2911–2921

PUBLICATION BIAS AND ESTIMATES OF BETWEEN-STUDY VARIATION 2913

where n is the number of studies, and hence inferences can be made concerning � as

�̂∼N(�;

1∑ni=1(�

2i + �2)−1

)

In addition to this standard approach, an alternative method for meta-analysis has been pro-posed [14, 15]. Since this also involves estimating �2, this procedure is also subject to theconclusions of this paper.Many approaches for estimating �2 are possible but the most popular is the DerSimonian–

Laird [3] estimate. This makes use of the Q statistic,

Q=n∑i=1wi(yi − �y)2 (1)

where wi=1=�2i and �y=∑n

i=1 wiyi=∑n

i=1 wi, the estimate of � under the assumption that�2 = 0. Under the null hypothesis that �2 = 0, Q follows a �2n−1 distribution [16]. Hencestatistical tests to ascertain if a �xed e�ects model is reasonable can be performed. TheDerSimonian–Laird estimate, however, makes rather di�erent use of Q, and uses this to pro-vide a moments estimate of �2. Under the assumptions of the random e�ects model it can beshown that the expectation of Q is given by

E[Q]= (n− 1) +(

n∑i=1wi −

∑ni=1 w

2i∑n

i=1 wi

)�2

which provides the DerSimonian–Laird estimate

�̂2DL = max(0;

Q − (n− 1)∑ni=1 wi −

∑ni=1 w

2i =∑n

i=1 wi

)(2)

2.1. Applying the random e�ects model to the sclerotherapy data

The sclerotherapy data shown in Table I will be used to illustrate the ideas. It concernsthe number of deaths in assessing the e�ectiveness of endoscopic sclerotherapy for preventingdeath in patients with cirrhosis and oesophagogastric varices, and has been used as an exampleseveral times [17, 18]. The values of yi and �2i denote the studies’ log odds ratios and within-study variance obtained, after adding halves to all cell entries, in the usual way [16]. Notethat as calculated here, a negative log odds ratio indicates that the treatment is bene�cial.Using the random e�ects model as described above provides a DerSimonian–Laird estimateof �̂2DL =0:303. Using this value in a random e�ects meta-analysis results in a point estimatefor � of −0:336 and a 95 per cent con�dence interval for this is (−0:680; 0:009). Although thecon�dence interval includes zero, this suggests that there may indeed be an overall treatmente�ect. At the very least we may infer that the treatment is not very harmful.Although this analysis is encouraging, it may be that the data is subject to publication

bias. The funnel plot (Figure 1) is rather asymmetric as the smaller studies (those withsmaller precision) tend to have more optimistic estimates of treatment e�ect. Note that forthis plot, precision is de�ned as 1=�i. Of course there are various explanations for this apparentasymmetry including a particular type of heterogeneity where the quality of studies dependson their size. If this is the case then the smaller studies may provide more optimistic results inthe absence of publication bias and it is very di�cult to distinguish between this heterogeneity


2914 D. JACKSON

Table I. The sclerotherapy data.

Study (i) yi �2i

1 −2:156 0.5432 −1:455 0.1773 0:174 0.4834 −0:535 0.4885 −0:568 0.1936 0:067 0.1897 −0:712 0.1578 −0:717 0.1399 −0:956 0.22310 −1:760 2.50111 −0:042 0.19112 0:024 0.23813 0:852 0.08314 −0:511 0.15615 0:296 0.15816 −0:504 0.78917 0:011 0.38518 0:303 0.44819 0:108 0.472

Log odds ratio

Pre

cisi

on

-2.0 -1.5 -1.0 -0.5 0.0 0.5

1.0

1.5

2.0

2.5

3.0

3.5

Figure 1. Funnel plot of the sclerotherapy data.

and publication bias in practice. However, this asymmetry is indicative of bias and suggeststhat studies with less promising results may be less likely to be published. Although modelshave been developed to examine the e�ect of study characteristics on estimates of treatmente�ect [19], the intention here is to ascertain how we should consider revising the estimateof �2 in the random e�ects model with publication bias in mind. The usual concern is thatthis type of publication bias leads to bias in the estimate of treatment e�ect. For example,Terrin et al. [20] examine the performance of methods that adjust estimates of treatment e�ectand highlight the di�culties when interpreting funnel plots if �2¿0. Despite this, the direct



implications of publication bias for the estimate of �2 have received little attention in theliterature and this paper is the �rst to address this particular issue in detail.Arguing somewhat informally, perhaps the most natural concern is that the bias, if it is

indeed present, has resulted in an underestimate of the between-study variance. If studies withless promising results have been censored, as suggested by Figure 1, then the range of thesample values of yi has been reduced. The sample might therefore be expected to producea smaller estimate of between-study variance than the value inherent in the population ofstudies. If this is indeed the case we should consider increasing our estimate of �2 in orderto take into account the publication bias.However, it is also possible to argue that this type of bias will lead to an overestimate

of �2. By censoring studies with less promising results, any naive estimate of �, which isessentially a weighted average of the yi, is pulled away from the true overall treatment e�ect.Hence there is the tendency for large studies, centred around �, to provide results that di�ergreatly from estimates of treatment e�ect, relative to their values of �2i . These studies, whichcarry the largest weight in the meta-analysis, therefore become hard to reconcile with theestimate of treatment e�ect unless there is a su�ciently large value of �2 to incorporate them.Note the squared deviations of the values of yi from �y in the de�nition of Q in Equation (1).If the bigger studies provide unusually large squared deviations from �y then the e�ect ofoverestimating �2 becomes apparent. If this is the correct argument then we should considerdecreasing our estimate due to the apparent publication bias.The implications of publication bias are therefore hard to assess informally. Since the two

arguments directly contradict each other, they cannot both be correct. Despite this di�culty,by using parametric models for the bias, the implications of this can be assessed quantitatively.

3. MODELLING PUBLICATION BIAS

Weight functions have been a popular approach for modelling publication bias [21–28]. Thisis a parametric model describing the probability of publishing a study given its outcome. Letyi denote the result from the ith study. We assume a standard model for the distribution of yi,Pi(yi), and a model for the selection process, Pi(published|yi). The function Pi(published|yi)is referred to as the weight function and hence the distribution of yi, given that the study ispublished, is obtained as

Pi(yi|published)= Pi(yi)Pi(published|yi)Pi(published)(3)

Since the studies in the meta-analysis are published, the observed studies’ results are dis-tributed as in Equation (3). We will assume, despite the rather small size of many of thestudies, that the random e�ects model is suitable for modelling the studies’ results beforeselection. Using the random e�ects model in this way means that, for the sclerotherapy data,yi ∼N(�; �2i + �2). Once a weight function has been decided upon, Equation (3) gives thedistribution of the observed studies’ results which can then be used to assess the implicationsof the bias.Although a variety of weight functions have been considered, the same themes recurred

for all and the �ndings are most clearly demonstrated by perhaps the very simplest of weight


2916 D. JACKSON

functions

Pi(published|yi)={1 if yi¡S

(1− �) if yi¿S(4)

where S is a ‘step’ or ‘cut-o�’ point and � describes the severity of the bias. Of course,06�61, although this requirement can be relaxed somewhat, as explained in Section 5. If�=0 then all the available studies are published, there is no publication bias, and the randome�ects model applies. For �¿0, however, studies with less promising results have less chanceof being published and publication bias is present. By not specifying the value of S at thisstage, the weight function describes a family of weight functions with di�erent possible cut-o�s. This step function re�ects the type of bias suspected from the funnel plot, as studies thatproduce a log odds ratio greater than some value S have less chance of being published andhence some such studies are e�ectively censored by this selection process. This provides anexplanation for the asymmetric funnel plot in Figure 1.Of course, many other types of weight function are possible, but attention will be restricted

to the one described in Equation (4). As will be seen in the sections that follow, an ex-amination of the properties of this weight function alone is enough to make some importantconclusions.

4. ASSESSING THE IMPLICATIONS OF THE PUBLICATION BIAS

Having decided upon a model for the publication bias, the implications of it will be determinedfor the DerSimonian–Laird estimate of �2. For mathematical convenience, the analysis in thissection will be carried out in terms of the semi-standardized values zi=(yi − �)=�i. In termsof these values we have that Zi ∼N(0; 1 + �2=�2i ), which is equivalent to Zi ∼N(0; 1 + wi�2)and therefore that

Pi(zi)=1√

1 + wi�2�

(zi√

1 + wi�2

)(5)

where �(·) denotes the standard normal density function. The weight function becomes

Pi(published|zi)={1 if zi¡(S − �)√wi(1− �) if zi¿(S − �)√wi

(6)

and Q is given by

Q=n∑i=1z2i − (

∑ni=1

√wizi)2∑n

i=1 wi(7)

Taking expectations of Equation (7) and a little re-arrangement gives E[Q] for a meta-analysisof published studies as

E[Q]=n∑i=1

(1−wiw·

)E[Z2i |published]+

∑ni=1 wiE[Zi|published]2

w·− (∑n

i=1√wiE[Zi|published])2

w·(8)



where w·=∑n

i=1 wi. Substituting Equations (5) and (6) into Equation (3) gives the distributionof the observed values of zi; the denominator can simply be obtained as the integral of thenumerator with respect to zi. This gives

Pi(zi|published)= (1=√1 + wi�2)�(zi=

√1 + wi�2)(1− �)�i(zi)

�(�((S − �)√wi=√1 + wi�2)− 1) + 1

where �(·) denotes the standard normal cumulative distribution function and �i(zi)=0 ifzi¡(S − �)√wi and �i(zi)=1 otherwise. From this, the values of E[Zi|published] and E[Z2i |published] can be obtained as

E[Zi|published]= −�√1 + wi�2�((S − �)√wi=

√1 + wi�2)

�(�((S − �)√wi=√1 + wi�2)− 1) + 1

(9)

and

E[Z2i |published]=1 + wi�2 − �(S − �)√wi + w2i �2�((S − �)√wi=

√1 + wi�2)

�(�((S − �)√wi=√1 + wi�2)− 1) + 1

(10)

Hence the expectation of Q can be obtained by substituting Equations (9) and (10) into (8).If �=0 then E[Q] is a function of �2 only. However, if � �=0, then E[Q]=f(�2; �; �; S), i.e.the expectation of Q is a function of all four parameters. It is therefore not straightforward toextend the DerSimonian–Laird approach to the case where there is publication bias. However,an estimate of �2 which provides E[Q]=Q, exactly as the DerSimonian–Laird method doesfor the random e�ects model, can be found numerically for particular values of �, � andS. Hence alternative estimates for models with publication bias can be obtained in this way.Negative estimates can be truncated to zero in the same way as in Equation (2).Now that estimates can be produced, we are able to return to the unresolved issue of

whether we should consider increasing or decreasing estimates of �2 when taking into accountthe possibility of publication bias. In all the analysis that follows, S will be regarded as a�xed constant. Of course, di�erent values of S result in di�erent weight functions, so that theimplications of the publication bias will be determined in terms of this rather arbitrary value.For estimation purposes we solve f(�̂2; �; �; S)−Q=0 and, for a particular value of � (and

of course S), it is of interest to see what happens to our estimate of �2 as � increases fromzero and we move away from the random e�ects model (�=0) to one with publication bias(� ¿ 0). Assuming that the estimate of �2 has not been truncated to zero, d�̂2=d� evaluatedat �=0 is of interest, as the sign of this describes whether we need to increase or decreaseour estimate of �2 as � increases from zero. This provides relevant information concerninghow estimates that take into account publication bias compare to their naive counterparts.Since we �x Q to its observed value for estimation purposes, by implicit di�erentiation (for

details, see, e.g. Reference [29, p. 974]) we have that

d�̂2

d�=− @=@�f(�̂

2; �; �; S)@=@�̂2f(�̂2; �; �; S)

(11)

From equations (8)–(10), (11) can be evaluated at �=0 and provides

d�̂2

d�

∣∣∣∣�=0

=(S − �)∑n

i=1(w· − wi)√wi + w2i �̂2DL�((S − �)√wi=

√1 + wi�̂2DL)

w2· −∑ni=1 w

2i

(12)


2918 D. JACKSON

where �̂2DL is the DerSimonian–Laird estimate of �2. Since the summation in the numerator and

the denominator are both clearly positive, whether our estimate of �2 increases or decreasesas � increases from zero depends only on the sign of the term (S −�). If S¿�, our estimateincreases as we move to the model with publication bias. This makes intuitive sense. As thestudies’ results are centred around �, S¿� means that results are e�ectively being removedentirely from the right-hand side of the plot. The removal of these studies means that theDerSimonian–Laird estimate of between-study variance using the published results is likelyto be less than the variance inherent in the population and hence we need to increase ourestimate of �2.If, however, S¡�, studies are being removed from the centre of the plot as well as the

right-hand side. These central studies are closely clustered around � and by removing themin addition those at the right-hand side indicates that the DerSimonian–Laird estimate ofbetween-study variance using the published results is likely to be greater than the true valueof �2.Although this helps us to interpret the analytical result, this does not help us informally

assess the implications of publication bias for our estimate of �2. As data generally providesvery little information about the value of S, assessing the magnitude of this relative to � isa virtual impossibility and hence so is determining the impact of the publication bias. Thiscon�rms the earlier dilemma in attempting this from a visual inspection of the funnel plot.

5. APPLYING THE RESULTS TO THE SCLEROTHERAPY DATA

The analytical results obtained in the previous section can easily be illustrated for the sclero-therapy data. As explained above, a value of � is required. For the two illustrative examplesdescribed below, the naive estimate of −0:336 will be adopted. For a particular value of S,the estimate of �2 can be obtained for a range of values of � by solving f(�̂2; �; �; S)−Q=0,where Q is the observed value of the Q statistic, as suggested in Section 4. The results ofthis process are shown using S=0 and −0:5 in Figures 2 and 3, respectively. Also shown asa dotted line is the �rst-order Taylor series of �̂2(�) about �=0. This requires d�̂2=d�|�=0,which is obtained from Equation (12). The �gures illustrate that (S−�) determines the impactof the publication bias in the way described in Section 4. They also indicate that, for thesevalues of � and S, evaluating the gradient at �=0 gives a good indication of what happensfor larger values of � as the linear approximation works rather well for moderate values of�. However, for really large values of � and therefore severe publication bias, the curvesincrease sharply. In particular note that in Figure 3, although the gradient is negative at �=0as predicted by the theory in the previous section, this is not the case for larger values of� and indeed �̂2(�)¿�̂2DL when � becomes very large. This reminds us that (12) applies to�=0 and should not be interpreted as meaning that, for example, S¡� will provide smallerestimates of �2 for all values of �. The rule only describes what happens to the estimate of�2 as we begin to take publication bias into account and consider small values of �.Note that Figures 2 and 3 include some negative values of �. Although these are somewhat

unsatisfactory, as this means that the weight function is not bounded above by one andtherefore cannot be interpreted directly as a probability, it can still be interpreted as therelative probability of publication [25]. Hence these values are included to illustrate the linearapproximations provided by the dotted lines more clearly.



Beta

Est

imat

e of

Bet

wee

n-S

tudy

Var

ianc

e

-0.2 0.0 0.2 0.4 0.6 0.8 1.0

0.4

0.6

0.8

1.0

Figure 2. ‘DerSimonian–Laird’ estimates of �2 for �= − 0:336 and S =0.

Beta

Est

imat

e of

Bet

wee

n-S

tudy

Var

ianc

e

-0.2 0.0 0.2 0.4 0.6 0.8 1.0

0.4

0.6

0.8

1.0

1.2

Figure 3. ‘DerSimonian–Laird’ estimates of �2 for �= − 0:336 and S = − 0:5.

Perhaps the main conclusion from the �gures is the ease in �nding models that cause theestimate of �2 to both increase and decrease as � increases from zero. The impossibility ofmaking general statements about the implications of publication bias for estimates of between-study variance are therefore easily demonstrated for this and other examples.

6. CONCLUSIONS

This paper has examined the implications of publication bias for the DerSimonian–Lairdestimate of between-study variance in a meta-analysis. By restricting attention to perhaps thesimplest of weight functions it is readily demonstrated that estimates can both increase anddecrease as we move from the random e�ects model to a model with publication bias. It istherefore impossible to state, with any generality, how we should consider revising estimatesof �2 when faced with an apparently asymmetrical funnel plot without making some verystrong assumptions.


2920 D. JACKSON

Other issues are also waiting to be addressed, such as the bias of naive estimates of �2

when publication bias is present. Publication bias also has implications for tests of H0 : �2 = 0and the coverage of con�dence intervals such as those provided by Biggersta� and Tweedie[2]. As the DerSimonian–Laird type estimate described here can only be obtained for a givenvalue of � and �, the results are hard to contextualize in entirely estimation based procedureswhich produce bias corrected estimates.In addition to the �ndings described here, some results have also been obtained for other

estimation methods and weight functions, which will hopefully form the subject of a futurepaper. In particular, the impact of the weight function given in (4) with S replaced by S�i, sothat the statistical signi�cance of studies provides the cut-o� point, has been examined. Verysimilar conclusions were reached for this weight function. Considering the distinct possibilityof other models can only complicate matters. The weight function adopted here is not intendedas a good model for the data, but rather as a simple and natural model chosen to indicatethe di�culty in assessing the implications of publication bias for estimates of �2. It is clearfrom the analysis that careful modelling is needed to say anything at all about this, as theimplications depend on the parametric form of the weight function. Meta-analysts should beaware of this and be wary of making simplistic interpretations when presented with asymmetricfunnel plots. Adages such as ‘one sided publication bias leads to underestimates of �2’ aresimply not supported by mathematical analysis. This �nding is one which practitioners shouldbe aware of, so that they can avoid making such spurious conclusions when interpreting thepotential impact of publication bias.

REFERENCES

1. Egger M, Smith GD. Misleading meta-analysis. British Medical Journal 1997; 310:752–754.2. Biggersta� BJ, Tweedie RL. Incorporating variability of estimates of heterogeneity in the random e�ects modelin meta-analysis. Statistics in Medicine 1997; 16:753–768.

3. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controlled Clinical Trials 1986; 7:177–188.4. Sterling TD. Publication decisions and their possible e�ects on inferences drawn from tests of signi�cance.Journal of the American Statistical Association 1959; 54:30–34.

5. Begg CB, Berlin JA. Publication bias: a problem in interpreting medical data (with discussion). Journal of theRoyal Statistical Society Series A 1988; 151:419–463.

6. Easterbrook PJ, Berlin JA, Goplan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337:867–872.

7. Copas J, Shi JQ. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics 2000; 1:247–262.8. Copas J, Shi JQ. A sensitivity analysis for publication bias in systematic reviews. Statistical Methods in MedicalResearch 2001; 10:251–265.

9. Duval S, Tweedie R. A nonparametric trim and �ll method for accounting for publication bias in meta-analysis.Journal of the American Statistical Association 2000; 95:89–98.

10. Duval S, Tweedie R. Trim and �ll: a simple funnel plot based method of testing and adjusting for publicationbias in meta-analysis. Biometrics 2000; 56:455–463.

11. Higgins JPT, Thompson SG. Quantifying heterogeneity in a meta-analysis. Statistics in Medicine 2002; 21:1539–1558.

12. Glasziou PP, Sanders SL. Investigating causes of heterogeneity in systematic reviews. Statistics in Medicine2002; 21:1503–1511.

13. Thompson SG. Why sources of heterogeneity should be investigated. British Medical Journal 1994; 309:1351–1355.

14. Hartung J. An alternative method for meta-analysis. Biometrical Journal 1999; 8:901–916.15. Hartung J, Knapp G. A re�ned method for the meta-analysis of controlled clinical trials with binary outcome.

Statistics in Medicine 2001; 20:3875–3889.16. Sutton AJ, Abrams KR, Jones DR, Sheldon DR, Song F. Methods for Meta-analysis in Medical Research.

Wiley: New York, 2002.17. Sharp SJ, Thompson SG. Analysing the relationship between treatment e�ect and underlying risk in meta-

analysis: comparison and development of approaches. Statistics in Medicine 2000; 19:3251–3274.



18. Thompson SG, Sharp SJ. Explaining heterogeneity in meta-analysis: a comparison of methods. Statistics inMedicine 1999; 18:2693–2708.

19. Sterne JAC, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the in�uenceof study characteristics on treatment e�ects in ‘meta-epidemiological’ research. Statistics in Medicine 2002;21:1513–1524.

20. Terrin N, Schmid CH, Lau J, Olkin I. Adjusting for publication bias in the presence of heterogeneity. Statisticsin Medicine 2003; 22:2113–2126.

21. Cleary RJ, Cassella G. An application of Gibbs sampling to estimation in meta-analysis: accounting forpublication bias. Journal of Education and Behavioural Statistics 1997; 22:141–154.

22. Copas J, Jackson D. A bound for publication bias based on the fraction of unpublished studies. Biometrics2004; 60:146–153.

23. Dear KBG, Begg CB. An approach for assessing publication bias prior to performing a meta-analysis. StatisticalScience 1992; 7:237–245.

24. Givens GH, Smith DD, Tweedie RL. Publication bias in meta-analysis: a Bayesian data augmentation approachto account for issues exempli�ed in the passive smoking debate. Statistical Science 1997; 12:221–250.

25. Hedges LV. Modelling publication selection e�ects in meta-analysis. Statistical Science 1992; 7:246–255.26. Iyengar S, Greenhouse JB. Selection models and the �le drawer problem. Statistical Science 1988; 3:109–135.27. Larose DT, Dey DK. Modelling publication bias using weighted distributions in a Bayesian framework.

Computational Statistics and Data Analysis 1998; 26:279–302.28. Rust RT, Lehmann DR, Farly JU. Estimating publication bias in meta-analysis. Journal of Marketing Research

1990; 27:220–226.29. Anton H. Calculus with Analytic Geometry (3rd edn). Wiley: New York, 1988.30. Hardy RJ, Thompson SG. A likelihood approach to meta-analysis with random e�ects. Statistics in Medicine

1996; 15:619–629.


Documents

The implications of publication bias for meta-analysis' other parameter