PREVENTIVE MEDICINE 25, 102–104 (1996)ARTICLE NO. 0035

Editors’ Requests of Peer Reviewers: A Study and a Proposal

ERICA FRANK, M.D., M.P.H.

Department of Family and Preventive Medicine, Department of Medicine, Emory University School of Medicine,69 Butler Street, S.E., Atlanta, Georgia 30303-3219

ments for authors; it is implicit in such requests thatBackground. Despite the pivotal role of peer review observations should be substantive but courteous. In

in the scientific process, there has never been a formal addition to these author-specific comments, reviewersstudy of the way in which reviewers’ opinions are elic- are usually asked for editor-specific comments to whichited. This article describes such a study and makes authors are not privy.recommendations for a more standardized approach The secrecy inherent in such encoded and confiden-to the peer review process. We hope, through this pub- tial communications may be appropriate and neces-lication and related efforts, to encourage such a pro-

sary. What is probably inappropriate and unnecessarycess in Preventive Medicine and in other journals.is the shroud surrounding manuscript assessment; re-Method. The study population included the top 100viewers may be unaware of editors’ needs or publishingjournals (as rated by the 1989 Institute for Scientificethics, and authors are often uncertain of the criteriaInformation citation frequency index). Only journalsby which their manuscripts are evaluated. This is thewith primarily U.S.-based editorial offices are in-first published study of judgments routinely requestedcluded (n Å 73). Participants provided their reviewerin peer review; this article examines these parametersforms, cover letters for reviewers, and instructions forand proposes an ideal.reviewers that were used routinely. All data were col-

lected in February 1992.METHODSResults. The response rate was 97.3%. Journals var-

ied substantially in many of their reviewer requests.I studied the review systems used by the top 100While 96% of journals asked reviewers to recommend

journals as rated by the 1989 Institute for Scientificacceptance or rejection, only 72% asked reviewers toInformation (ISI) citation frequency index (1); these areassess manuscripts’ novelty, 69% requested assess-the most frequently cited journals through 1988. Forments of clarity, and only 51% asked for assessments

of the reasonableness of manuscripts’ conclusions. reasons of consistency, access, and budget, I includedSimilarly, only 46% of journals reminded reviewers only journals with primarily U.S.-based editorial of-that manuscripts were confidential documents, 51% fices; this left a population of 73 journals, 71 of whichprovided a separate cover letter, and 25% provided ex- agreed to participate. One listed journal has since dis-tensive (ú1 full page) reviewer instructions. continued publication, and 3 ISI-designated journals

Conclusions. While review uniformity is not requi- are actually books and therefore not representative ofsite, differences between review protocols may not be the journal peer review process; I therefore used thea function of specific journals’ needs, but of some jour- 67 remaining journals as my denominator for all calcu-nals’ excluding important review components from ex- lations. I requested that these journals fax any re-plicit mention. Based in part on the Council of Biology viewer forms, standard reviewer cover letters, and in-Editors’ recommendations, standard components of structions for reviewers that they used. All data werereviews and of instructions for reviewers are sug- collected in February 1992.gested. If editors’ expectations were more explicit, re-

I assessed whether there was a standard reviewerviewers’ comments could better serve editorial needs.form used, a standard cover letter, a sheet for separate

q 1996 Academic Press, Inc.author comments enclosed with the manuscript or a re-quest for the reviewers to provide one if necessary, adiscussion of reviewer anonymity (and if so, if it wasINTRODUCTIONroutinely given or given at the discretion of the editoror the reviewer), and a discussion of manuscript confi-The peer review process is frequently criticized for

its subjectivity and its secrecy. It is true that many dentiality or other ethical issues and whether extensive(ú1 page) reviewer instructions were given. I also exam-messages are encoded in the confidential communica-

tions between editors and peer reviewers. For example, ined whether reviewers’ opinions were specifically solic-ited for recommendations on acceptance or rejection; oneditors often ask reviewers for separate narrative com-

1020091-7435/96 $18.00Copyright q 1996 by Academic Press, Inc.All rights of reproduction in any form reserved.

/ a201$$2114 04-10-96 16:28:49 pma AP: PM

103EDITORS’ REQUESTS OF PEER REVIEWERS

TABLE 1 appears to be no particular pattern to their variability.Furthermore, reviewer guidance seems routinizedPercentage of Journals Using Various Formswithin journals, rather than being a manuscript-,and Common Form Componentstopic-, or reviewer-determined condition.

Forms and instructions provided to reviewers If we wish to know what peer reviewers routinelyReviewer form 92.5% provide in the peer review process, this study has sev-Cover letter 50.7%

eral limitations. First, I examined what editors ask ofSeparate author comments—enclosed 64.2%Separate author comments—as needed 13.4% peer reviewers, not what the peer reviewers actuallyReviewer anonymity—discussed 53.7% give the editors. What reviewers actually provide may

Always given (72.2%) be a more interesting, but a more complex, study ques-Editor’s choice (5.5%)Reviewer’s choice (22.2%)

Confidentiality 46.3%Other ethical issues 35.8% TABLE 2Extensive instructions 25.4%

Proposed Ideal for Editor–Reviewer CommunicationsaAssessments requested

Acceptance/rejection 95.5%Reviewers should be asked toPriority/importance/significance/interest 88.1%

• Maintain manuscript confidentialitySoundness/quality 79.1%• Disclose conflicts of interestOriginality/novelty 71.6%• Be objectiveClarity 68.7%• Be specific in comments to authors and editorsReasonableness of conclusions 50.7%• Respond promptly (and comply with a specified date forAppropriateness for journal 50.7%

review completion)• Designate areas in which they would like to review

Reviewers should address manuscripts’• Qualitythe manuscript’s priority, importance, significance, or• Importance

interest; on its soundness or quality, originality or nov- • Scientific rigorelty, or clarity; on the reasonableness of its conclusions; • Novelty

• Clarity (in text and in tables/figures/illustrations)and on its appropriateness for that journal.• Ethics (including divulged and undivulged authorial biases)• Specific merits and problems, as contained in its title,RESULTS

abstract, key words, introduction, methods (includinginterventions/procedures, statistics, and descriptions of

The response rate was 97.3%. Table 1 shows the per- hazards), results, interpretations and other components of thecentage of journals that use various forms and the com- discussion (including the appropriateness of its claims and

conclusions and statements of study limitations), tables,mon constituents of those forms. While some compo-figures, acknowledgments, and citationsnents of peer review are used fairly regularly (such

• Need for an accompanying editorial (based on controversialismas standard reviewer forms and requests for separateand newsworthiness)

author comments), many potentially important param- • Appropriateness and priority for the journal in questioneters (such as reviewer anonymity, manuscript confi- • Appropriateness in its current form (versus, for example,

reduction to a letter to the editor)dentiality, or other ethical issues) were frequently inex-plicit. Editors may wish to

• Provide thorough instructions for reviewers (especially firstDISCUSSION time reviewers)

• Allow reviewers to share manuscripts with appropriatecolleagues, in a confidential manner, for the purpose ofThese data demonstrate that the peer review processassistance with reviewis not now uniform. This finding may be of concern to

• Permit reviewers to write minor suggestions directly ontoauthors, reviewers, editors, readers, publishers, and manuscriptspatients. • Permit reviewers to shred and recycle the manuscript

• Encourage reviewers to fax reviews to the editorial officeThere are two major counterarguments to concerns• Blind reviewers to authors’ identitiesabout this nonuniformity. The first is that it does not• Allow reviewers to sign reviewsmatter; everyone knows the rules and it is superfluous • Send reviewers copies of others’ reviews

to make them explicit. Yet peer review is a subtle and • Send reviewers copies of the revised manuscriptstill-imperfect art that includes reviews of variable ef- • Inform reviewers regarding manuscripts’ final disposition

• Ask reviewers to specify other areas in which they would likefort and quality (2); providing more explicit guidelinesto reviewmight produce responses that better address editors’

• Provide reviewers a token of gratitudeand authors’ needs. The second counterargument isthat distinct journals have distinct needs and therefore a Some of the listed items were suggested by narrative in the Style

Manual of the Council of Biology Editors (3).provide differing types of reviewer guidance. Yet, there

/ a201$$2114 04-10-96 16:28:49 pma AP: PM

104 ERICA FRANK

tion. Second, I included only the most cited journals. ers can be customized in reviewer cover letters. Themore explicitly editors identify their needs (e.g., byThese are the most prominent (and, arguably, the mostspecifying some of the parameters listed in Table 2),important) journals, and it is likely that these journals,and the more clearly editors communicate this to au-because of their high volume and visibility, may havethors and reviewers, the more appropriate the manu-more stringent and standardized review criteria thanscripts could be that are submitted to journals, and thedo less prominent journals. This would, therefore,fairer, more efficient, and better the peer review pro-likely provide a conservative estimate of the noninclu-cess should become.sion of review criteria. Third, I included only domestic

journals, partly because of a desire for homogeneityACKNOWLEDGMENTSand comparability, but largely for practical constraints.

Fourth, several sets of journals were edited from the I gratefully acknowledge the efforts of Ms. Donna Adelman and thesame offices (and hence had similar editorial pro- thoughtful comments of Dr. Randall White and Ms. Linda Hengstler.

Portions of this article were presented at the Second Internationalcesses); this also would produce a conservative esti-Congress on Peer Review in Biomedical Publication held in Chicago,mate of heterogeneity.Illinois, in September 1993, and at the Workshop for Editors of Re-The Council of Biology Editors has outlined an out- ferred Scientific Journals held at Vanderbilt University in March

standing model for the relationship between editors 1992. No grant support was received for this research, although sal-ary support was received from an institutional National Researchand reviewers (3). In Table 2 I have listed these sugges-Service Award to Dr. John Farquhar, Stanford Center for Researchtions and have added several additional parameters toin Disease Prevention, 5T32-HL-07034, from the National Heart,create a model review system. Lung, and Blood Institute.

The peer review process is not uniform now, and Iwould not suggest that it should be. There are, how- REFERENCESever, several areas that should be routinely explicitly

1. Garfield E. 1989 journal citation reports. Philadelphia: Inst foraddressed which are often neglected. These areas in-Sci Information, 1989.

clude ethical issues (including specific requests for con-2. Lock S. A difficult balance: editorial peer review in medicine.

fidentiality and disclosures of conflicts of interest) and Philadelphia: Inst for Sci Information, 1986.journal-specific, explicit criteria for what is needed of 3. Council of Biology Editors. CBE style manual, 5th ed. Bethesda

(MD): Council of Biology Editors, 1983.reviewers. Some of these criteria can be routinized; oth-

/ a201$$2114 04-10-96 16:28:49 pma AP: PM