Peer review

  • View
    32

  • Download
    0

Embed Size (px)

DESCRIPTION

Peer review. Peer Review: what is it?. The central process of academia: your peers review, judge your work. The source of rigour in science. Hellish and unavoidable process of academia. The source of great pain, misery and crushing hammer blows to your confidence. - PowerPoint PPT Presentation

Text of Peer review

  • Peer review

  • Peer Review: what is it? The central process of academia: your peers review, judge your work.

    The source of rigour in science.

    Hellish and unavoidable process of academia. The source of great pain, misery and crushing hammer blows to your confidence.

  • http://www.scieng.ed.ac.uk/transkil.htmScientific Paper ProductionMy goal: introduce you to the hardship of peer review (with a positive coda)

  • Peer Review: how does it work? used to assess our two major currencies: journal publications and grant-gettingBut process similar: your work sent to a journal editor or funding council (e.g. NERC, MRC) or charity (e.g. Wellcome), who then mediate a (largely) anonymous peer-review process

    Your document sent to peers who pass judgement with a report specifically asked to highlight both positives and negatives

  • Who are your peers? Colleagues in the same fieldThough definition of same can varythey are your competitors (= - bias)Because we move in small circles, they are your friends (= + bias). But dont underestimate your friends

  • *small circles Referees are always out thereThey go to your talks at conferences- take the opportunity to impress Give a good talk, be well read etc. IMPRESS! It could make your life much easierThoroughly understand the work of potential referees. think about who might referee your paperCriticise the work of others very carefully (e.g. in the intro of manuscripts)

  • Who are your peers? You can often chooseJournals and funding bodies require you to nominate referees, though they will not use all of your suggestionsYour friends may or may not treat you better- so choosing refs is not easydeserves careful, careful thoughtBut overall, it has been shown that choosing helps

  • Science, Vol 309, Issue 5743, 1974 , 23 September 2005Can suggest referees. Can ask that certain refs be excludedIt pays to know your field very well

  • Who are your referees?Typically, you will not know- even if they are your suggested ref, they probably wont sign review. Freedom to be cruel?But they will know who you areBiasHelps famous people?But can also help you due to having met, socialised, impressed?

  • Referees are often wrong, often right and frequently nuts

  • Oikos 67: 577-581 (1993)

    .. the explanation is interesting, and worth stating (but) I didn't need to read the paper to get the point

    Annals of the Entomological Society of America 88: 100-103 (1995)

    This is still a most unexciting paper, but it is probably useful to confirm experimentally what everyone knows intuitively. And so I accept it for the Annals.Slight Insanity

  • Reviewer 1.By necessity they [models] are so simplified and artificial that they possess little ability to represent even the most basic dynamics of the real world

    Any manager that took this approach should be fired

    Reviewer 2.a simple and elegant model

    What makes this an exciting paper is that the conclusions can be readily acted upon by land managersSlight Insanity

  • REFEREE 2: [this paper concerns] is an interesting topic, but this paper does not provide any substantial original contribution. Furthermore, the authors cite the literature very selectively, .the failure to cite the literature in some instances, and the blatant misinterpretation of the literature in other cases, borders on unethical behavior.

    More Severe Insanity

  • Table 1 reviews studies of the experiment in XXXX et al. (2002), in which the decoy greatly increased the absolute preference for the target, there is an asterisk stating that "the authors report this result, however no supporting statistics are presented". XXXX et al. (2002) present these data on page 185 . The decoy effect was so remarkably strong, that statistics were almost unnecessary. Standard errors are shown in the figure, showing clear differences between means, and the figure caption states that the differences were statistically significant. In the text, there are two paragraphs full of supporting statistics, with test statistic values and probability values.

  • Some referees just dont get out enough

  • Seems hopeless,Ms had 35 paragraphsNumber of negative comments from referee 70Number of positive comments by referee: 1 (though only allusion)

  • But for the same articleReviewer #2(Remarks for the Author):I think this is a well written paper that details clearly a set of elegant laboratory experiments. The three key results (i.e. that virulence is temperature dependent, that census date matters and that there are strong GxE interactions that alter resistance rankings) are highly significant in the context of coevolution and have clear implications for many previous studies of parasite-mediated selection. As such, I think it makes a timely contribution to a topical research area and I fully support its publication in Evolution.

  • Another nice contrastReviewer 1>This is a very silly paper. It is not necessary to construct a model to see>that if two costs (not mating with the fittest male and not mating with a>compatible male) impinge on mating behaviour, the one with the larger effect>is likely to dominate. This is all that figure 1 shows. Previous verbal>arguments are sufficient. It would also be useful if the authors set out>what incompatibilities they are modelling. Is this an investigation of>interspecific incompatibilities?

    Reviewer 2>>This is an interesting and well-written paper that represents the>>first quantitative attempt to incorporate both "good genes" sexual>>selection and genetic compatibility-based polyandry in a single>>theoretical framework. Although the authors present a very simple>>model that is not explicitly genetic, the paper nonetheless makes a>>very valuable contribution to the important issue of whether females>>should exercise choice at the precopulatory stage or engage in>>polyandry.

  • OverallDont put too much weight on the comments of any one referee.

  • Referees can be the wind beneath your wings

  • And even when they criticise, find silver liningsHelpful: a source of good ideas (your next version of the paper will be improved), a source of collaboration (you will often meet them)

  • What are the potential outcomes?

  • What are the potential outcomes?Reject Your paper is pretty shabby, but well let you spend a couple of months fixing it up before we review again. No guarantees, but reason for hopeAccept provided you fix this and that (only editor re-reviews)Accept without revision (doesnt happen)

  • What can you do in response?First, there is what you have to do: respondBy extremely polite and tactful- this can be incredibly hardsupervisor can provide you with examples of appropriate responsesReferee has spent often considerable time on youBear in mind that the comments you have received, may not be all the comments. Be easy on the editor (esp when they may not be professional)Bear in mind that every referees comment, no matter how harshly put or seemingly insane, will have a helpful kernelAt the very least, their stupidity might reflect an less than perfect explanation on your part (this is anyways a good hook)address their comments even if you plan to submit elsewhere

  • Dear Editor,

    Thank you for the offer of resubmission.

    The referees gave the paper an impressively thorough reading, and I would like to thank them for that. The comments they made were on the whole fair- which meant the revision was a lot of work! But the paper is clearly much improved.

    I believe have addressed the major concern of both referee's and the AE, namely, I have fixed up and justified the tests for selection

    Minor points:The referee was correct to point out that

    We sympathise with the referees confusion over table one, and have taken steps to rectify what was admittedly a muddled presentation of that aspect of the.Responding to referees (always done via the editor)

  • You may have to work on the editor a bit as well..

    3/21/05

    Dear Dr. Wilson, Thank you for offering to examine a revision of our ms Mechanism versus phenomenon in invertebrate immunity. We expected our article to generate discussion, and thus were pleased to see that two of the referees appeared to be stimulated by our ideas, and were generally supportive. It was also reassuring that only one referee (referee 3) launched criticisms that concerned NI. We deal with the comments of referee 3 first, followed by responses to referees 1 and 2. Throughout, any quotes from a referee is in italics, our comments in regular type:

  • Additional things you can do For key criticisms, contact referee through editor- this can lead to something fruitfulYou cannot escape peer review, so learn from the process. It is an opportunity to improve your paper and your science communication generally

  • Then, if refereeing yourself:Always, always bear in mind how you felt when reading harsh reviews, and temper your own reviewsJust as all referees comments have merits, all papers have good bits, so mention them.

    It is entirely possible to be rigorous and helpful

    ************************************