Peer Review Process - HEC

Peer Review Process by Dr. Sadia Nadeem

Peer Review ProcessResearch Journals and Publications Section Division, R&D

(26th November 2020)Capacity Building Program for the Year 20-21

1

Dr. Sadia NadeemDean FAST School of Management, NUCES

[email protected]

mailto:[email protected]


Agenda Peer Review and its Importance in Scholarly Publishing

Best Practices in Peer Review: Ensuring Transparency in Peer Review Process

Online Tools to Expedite the Peer Review Process

Malpractices in Peer Review and Measures to Avoid the Same

Conclusions

(Audience: editors, reviewers and other stakeholders)

2

Peer Review Process by Dr. Sadia Nadeem 3

Section 1Peer Review and its Importance

in Scholarly Publishing


Process of Peer Review for Journal Articles1. The author(s) of an academic article submit an (anonymous) manuscript to the editor.2. The editor has knowledge of the identify of the author e.g. through a separate uploaded cover page, a

cover page generated by the online system, or an email. 3. The editor either ‘scans’ the manuscript himself/herself, or forwards it to the ‘associate editor’ of the

relevant area for a quick ‘scan’. 4. If the manuscript is out of scope, or poor quality, it is rejected by the editor or associate editor (desk

rejection).5. If the manuscript may be suitable, the editor/associate editor identifies reviewers for the manuscript. 6. The editor/associate editor has a list of suitable reviewers, or may approach new academicians to act

as reviewers; usually 2-4 reviewers independently evaluate a manuscript. 7. The reviewers give their consent to review the manuscript, and commit to an assigned deadline.

(this is the start of the review process)

4


Process of Peer Review for Journal Articles(continued)

8. The reviewers read the manuscript in great detail, and evaluate the manuscript on the form provided by the journal (decisions, marks and detailed comments); this feedback is submitted to the editor/associate editor.

9. The editor/associate editor reads all the evaluations, compiles them, adds his/her own comments, and forwards the feedback to the corresponding author.

10. The main decisions after the first round of peer review are (i) Reject, (ii) Major Revisions, (iii) Minor Revisions, or (iv) Accept.

a. Reject means the manuscript cannot re-submitted to the journalb. Major Revisions means significant changes are required after which the reviewers and the

editor/associate editor take any of the four decisions mentioned in Item 10.c. Minor Revisions means changes are not high risk, but even in this case, if not addressed, may lead to any

of the four decisions mentioned in Item 10. d. Accept means the manuscript is accepted as it is, though English, typos etc. may need to be corrected.

5


Process of Peer Review for Journal Articles(continued)

8. Manuscripts usually go through 2-4 rounds or peer review, though they may go through more rounds.

9. It is almost impossible that a paper is accepted in the first round. 10. The comments from the peer reviewers and editor are more detailed in the first few rounds, and

then become narrower and more specific in the subsequent rounds. 11. Ideally, the same reviewers review the manuscript in several rounds.12. The author(s) must provide a detailed ‘Response to the Reviewers’ at the end of each round. 13. Once the paper is accepted (say in round two, three or four), it is forwarded to the type setter and

the publisher. 14. The reviewers do not have the knowledge of the identify of the authors till the end of the peer

review process.

6


The Importance of the Peer Review ProcessCreating knowledge which is pushing the boundaries of knowledge as per experts (peer reviewers) in that domain.

Peers supporting each other to create knowledge.

A levelled playing field (anonymous process).

A responsibility of academician, which is usually a voluntary activity to contribute to the body of knowledge.

Publications (Acceptance/Rejection) leads to promotions, pay increases and monetary rewards.

As custodians of the development of future generations, our integrity and sense of responsibility in this critical process is of utmost importance.

7


Section 2Best Practices in Peer Review:

Ensuring Transparency in Peer

Review Process


Best Practices in Peer Review (Item 1-8)1. Review process (covered under Section 1)

2. Review forms for the peer review process

3. Review timelines

4. Timely correspondence during the review process

5. Communication tone and feedback during the review process

6. Identifying suitable reviewers

7. Auditable review process

8. Systems for the review process

9


Item 2: Review Forms for the Peer Review Process•Desk Rejection: usually an email giving a few key reasons (no forms are needed).

The editor may occasionally provide detailed comments (a page or two), but a few

key reasons are sufficient.

•Rounds of peer review need forms; the same form is used for all rounds.

•Parts of the evaluation form:

1. A main decision: (i) Reject, (ii) Major Revisions, (iii) Minor Revisions, or (iv) Accept.

2. Comments◦ Providing detailed comments on areas specified by the form

◦ Providing comments (open, but we all know which areas to cover)

3. Quantitative evaluation (giving marks against specific items)

10


Desk Rejection: Points for Reflection •Does the article fit in the scope of this journal? Will it help our readers to make better decisions and, if so, how?

•Is the article important? Will the article add enough to existing knowledge?

•Does the article read well and make sense? Does it have a clear message?

(may be used by the reviewers as well for initial reflection)

11


Review Form for Research Articles: Comments

No form at all – freedom to write. Likely to receive 2-3 pages of comments from each reviewer Four reviewers could mean 10 pages of comments

Max comments I have seen is 22 pages of comments (not my review but in a training workshop)

12


Review Form for Research Articles: Sample Questions

Please provide detailed feedback to the following questions.

1. Originality - does the work add enough to what is already in the published literature? If so, what does it add? Please cite relevant references to support your comments on originality.

2. Research questions - Is the research question clearly defined and appropriately answered?

3. Research design - Is the overall design of the study appropriate and adequate to answer the research question?

4. Methods – Does the manuscript adequately describe the methods? Are sufficient details provided about the population, sampling, data collection procedures, data collection instruments, protocols followed etc.?

13


Review Form for Research Articles:Sample Questions (continued)

5. Results – Do the results answer the research question? Are the results credible and well presented?

6. Interpretation and conclusions – In light of the methods and results, are the interpretations and conclusions appropriate?

7. References – are the references up to date and relevant? Please suggest any glaring omissions.

8. Abstract – does the abstract accurately reflect the paper?

9. Practical and policy implications – have the authors clearly extracted appropriate practical and policy implications (or implications for other stakeholders, for various disciplines).

14


The Editor’s Nightmare

Filling the form with Yes/No only, or giving one line answers.

Should editors accept such answers?

Such evaluations are not at all acceptable, not in the first round or two.

Also, very generic answer do not add value.

15


Quantitative Evaluation: An example(only quantitative is never appropriate)Strongly Disagree (1) to Strongly Agree (5)

•The manuscript contains new and significant information to justify publication.

•The Abstract clearly and accurately describes the content of the article.

•The research objective is significant and concisely stated.

•The methodological approaches are appropriate.

•The interpretations and conclusions are justified by the results.

•Adequate reference is made to other work in the field.

•The language and writing style are appropriate.

•The practical and or policy implications are drawn out at the end of the paper.

(‘Provide detailed comments’ after this would mean comments covering all the above areas)

16


Some Additional QuestionsPlease rate the priority for publishing this article (1 is the highest priority, 10 is the lowest priority)

On your reading of this manuscript, what do you consider to be the likelihood of the author(s) being able to successfully resolve the issues raised in your review?

Would you be willing to review a revision of this manuscript?

Confidential comments for editor.

Comments for authors.

17


Item 3: Review Timelines1. Submission to desk review: A week to a 2 weeks

2. Forwarding manuscript to reviewers: A week to 4 weeks

3. Time for Round 1 of review: 1-3 months.

4. Time for resubmission in first round: 1-3 months.

5. Time for Round 2 of review: 1-3 months. (3 months for major review)

6. Time for resubmission in second round: 1-3 months. (3 months for major review)

7. Round 3 or 4 are usually minor revisions only and usually one month is sufficient.

18


Item 4. Timely correspondence during the review processIf you are using an online system, can the authors review the status of their paper? E.g. ◦ With the editor

◦ Under review

◦ With the associate editor

Is the date of each status change visible?

If using email submissions, there should be timely response to the email queries.

19


Item 5. Communication tone and feedback during the review process (Extracted from AOM guidelines)

Mainly for reviewers; editors can also guide reviewers.

Be Constructive: Don't just point out problems, also point out solutions. Be developmental in your comments. Be Concise: Try not to cover the same ground in multiple comments; consolidate your coverage of a given theme in a single point.Be Polite and Conversational: Be professional and "author friendly" in your tone.Identify Some Strengths: Open your review with what you liked, before focusing the bulk of your review on your criticisms and concerns.Give specific additional recommendations for missing literature. Cite page numbers and line numbers when referring to specific sections of the manuscript and providing feedback. Don't Be "Two-Faced": Don't send a different message in your Comments to the Author than you do in your Comments to the Editor.Be On Time: It is important to return your review on time so that the action editor can guarantee the authors a quick turnaround.

20


Item 6. Identifying Suitable ReviewersIdeally, expert in subject area and in the methods.

Often the compromise is (i) at least one clear subject area expert, (ii) at least one expert on methods with some expertise on relevant subject area.

Adding a novice reviewers as an additional reviewer with expert reviewers usually helps in training the reviewers.

Ethics and integrity of the reviewers.

Communication tone of the reviewers.

21


Item 7. Auditable Review Process Editors must maintain a complete record of the review process.

Online systems maintain this record, but there should be adequate backup.

Complete correspondence of the rejected (including desk rejected) manuscripts and accepted manuscripts should be available for audit.

22


Item 8. Systems for the review process1. Once upon a time: Hard copy submissions and correspondence

2. Email submissions and correspondence

3. Online systems

23


Section 3Online Tools to Expedite the Peer

Review Process


Online Tools to Expedite the Peer Review Process Open Journal System (OJS) to be covered in detail by Dr. Akhter Shiren on 9th

December 2020.

ScholarOne

Editorial Manager

Evise

Universities have build their own systems; HEC may evaluate and recommend.

25


Session on OJS

26


Section 4Malpractices in Peer Review and

Measures to Avoid the Same


Malpractices in Peer Review (we all know the solutions)Editors may share the identity of the authors with the reviewer(s).

Editors or associate editors may exchange favors for publishing in each others journals.

Editors may intentionally choose ‘polite’ reviewers or ‘harsh’ reviewers to indirectly increase or decrease the chances of publications of a manuscript.

Accepting money/bribes for accepting a publication or for speeding up the review process (NOT the official/announced publication fee, which is altogether separate from this).

Delaying reviews, or not replying to queries (when system does not provide updates).

The editor, associate editor, or reviewer copying ideas (or using data where applicable) from papers submitted for review.

28


Solution

Ustad ka ahteraam, Ustad ki Issat

“Destroying any nation does not require the use of atomic bombs or the use of long-range missiles. It only requires lowering the quality of education and allowing cheating in the examinations by students”.

29


Ethical Dilemmas (AoM Video and More)

Should I review a paper for which I am well qualified but also fairly certain of the author’s identity?

(What should I do if the editor asks me for a favorable review?)

Should I decline to review a paper if I feel lack of expertise to do a high quality review?

If the paper I am reviewing closely relates to my current work is it possible to cite the work in my paper?

Do I have an ethical responsibility to meet the deadlines of the review assignments I am given?

Do I have an ethical responsibility to review articles for journals and conferences?

30


Section 5

Conclusions


ConclusionsCreating knowledge which is pushing the boundaries of knowledge as per experts (peer reviewers) in that domain.

Peers supporting each other to create knowledge.

A levelled playing field (anonymous process).

A responsibility of academician, which is usually a voluntary activity to contribute to the body of knowledge.

Publications (Acceptance/Rejection) leads to promotions, pay increases and monetary rewards.

As custodians of the development of future generations, our integrity and sense of responsibility in this critical process is of utmost importance.

32


Question Answer Session

Peer Review ProcessResearch Journals and Publications Section Division, R&D

(26th November 2020)Capacity Building Program for the Year 20-21

33

Dr. Sadia NadeemDean FAST School of Management, NUCES

[email protected]

mailto:[email protected]

Documents

Peer Review Process - HEC