THE SCIENTIFIC REVIEW PROGRAMTHE SCIENTIFIC REVIEW PROGRAM TRAINING PROGRAMTRAINING PROGRAM

(SRPTP)(SRPTP)

MAJOR REVIEW PHASES

March 2, 2017

Disclaimer: The National Institute of Allergy and Infectious Diseases (NIAID) Scientific Review Program (SRP) makes its training documents available to NIH staff as a service. In return for their use we request that you do not share or transmit any of the materials to anyone outside of the NIH. We do our best to keep these documents current, but we do not guarantee the content to be correct or up to date. Documents are reviewed periodically or as needed. If you have any questions or comments, please contact us at NIAIDSRPTrainingCommittee@niaid.nih.gov .

Major Review Phases—May, 2015

2

Major Review Phases—May, 2015

Goals for This Module Place peer review in the context of NIAID Grants Workflow Define the major milestones of NIAID SRP Peer Review Provide an overview of the workflow for NIAID SRP Peer Review Identify the related standard documents, SRP SOPs, and NIAID SOPs Define the specific tasks of the NIAID SRP team

Table of AcronymsARA Awaiting Receipt of Application OA Office of AcquisitionsBC Branch Chief OER Office of Extramural ResearchCMO Committee Management Officer PA Program AnalystCO Contracting Officer PO Program Officer (grants) OR Project Officer

(contracts)COI Conflict of Interest RO SRP Referral OfficerCOTR Contracting Officer Technical

Representative (contracts)RSS Reviewer Support Site

CS Contracting Specialist SEP Special Emphasis PanelCSR Center for Scientific Review SOP Standard Operating ProcedureESA Extramural Support Assistant SOW Statement of WorkFACA Federal Advisory Committee Act SREA Sci. Review Eval. ActivityGMB Grants Management Branch SRO Scientific Review OfficerWMC Women, minorities and children TEC Technical Evaluation CriteriaHS Human Subject TEG Technical Evaluation GroupI/C Institutes and Centers TER Technical Evaluation ReportIAR Internet Assisted Review TESS Technical Evaluation Score SheetIRG Integrated Review Group TW Technical WriterLOI Letter of Intent VA Vertebrate Animal

3

Major Review Phases—May, 2015Table of Contents

Introduction.................................................................................................................................................. 4Major Review Phases: Outline..................................................................................................................... 5

Pre-Receipt............................................................................................................................................. 5Receipt................................................................................................................................................... 6

Receive Applications and Proposals...................................................................................................6Process Applications and Proposals...................................................................................................6

Perform Pre-Meeting Activities...............................................................................................................6Perform Administrative Review...........................................................................................................6Identify, Recruit, Assign Reviewers.....................................................................................................7Inform and Report............................................................................................................................... 9Organize Review Meeting................................................................................................................... 9Distribute Review Materials...............................................................................................................10Prepare for Meeting........................................................................................................................... 11

Conduct Peer Review Meeting.............................................................................................................12Establish Working Environment.........................................................................................................12Oversee Peer Review Process..........................................................................................................12Collect Meeting Materials.................................................................................................................. 13

Perform Post-Meeting Activities............................................................................................................13Release Scores and Scientific Codes...............................................................................................13Release Roster, Meeting, Minutes....................................................................................................14Reimburse and Thank Reviewers.....................................................................................................14Release Summary Statement or Technical Evaluation Report..........................................................14Archive Meeting................................................................................................................................. 15

4

Major Review Phases—May, 2015

5

Major Review Phases—May, 2015

IntroductionThis training module covers the major phases of a typical review at NIAID’s Scientific Review Program (SRP). It covers face-to-face and teleconference reviews of grants (paper and electronic) and contracts. Atypical reviews (non-FACA, internet assisted meetings etc.) are covered in a distinct module titled “Special Review Situations.” The module follows the NIH Peer Review Business Process Flow (for grants) shown in Figure 1. Individual steps are presented as distinct bullets. Associated NIAID SRP tools (standard operating procedures and standard documents) are listed for each bullet. The person responsible for each step is identified. Most steps are common to grant and contract review, but steps specific to one or the other are identified. Each review is unique and individual steps may be accomplished in a variety of ways.

Additional information about NIH grant review can be found in the NIH Policy Manual. The 2013 version is here https://policymanual.nih.gov/manage/chapter/view/4204-204b (you need to login with your NIH password/PIV card).

Figure 1

6

Major Review Phases—May, 2015

Table Format KeyTask WHO GCB DOCS

This is a task completed by the SRO for a grant review. There are no associated documents.

S G

This is an optional task completed by the ESA staff or the SRO for a grant or contract review. There are no associated documents.

R, S B

WHO: S = SRO; R = Review Specialist; RO = SRP Referral Officer, BC = Branch Chief; TW = Technical Writer, PA = Program Analyst; OA=Office of Acquisitions; O= Offeror

GCB: G = Grants, C = Contracts, B = BothDOCS: Hyperlinks to related standard documents and SOPs

Major Review Phases: OutlinePre-ReceiptTask WH

OGCB DOCS

Check assignment in RIMS.See RIMS - Scientific Review Program.

S B

Participate in initiative development, as assigned. S B

Attend Reverse Site Visits, as requested. See SOP for guidance

S G Site Visits, Grantee SOP

Read and research initiative prior to receipt date. S B http://grants2.nih.gov/grants/guide/ https://www.fbo.gov/

Answer questions from applicants. Submit ARA form if necessary.

S G ARA form ARA SOP

Ask for and/or share list of LOIs/Letters of response with PO/CO. Communicate with all concerned staff regarding expected response and alignment with goal of FOA/Contract.

S B

Ask ESA for assistance. S, R B

Look at checklists for key documents/functions. S, R B Checklist for Contracts Checklist for Contracts

Teleconferences Checklist Grant and Contracts

Large Meetings Checklist for Grants Checklist for Grant

Teleconferences

Complete Planning worksheet (in WPS). Discuss with all concerned staff at startup meeting. Give Planning worksheet to concerned staff.

S, R, BC

B Work Planning System (WPS)

Pre-recruit reviewers, as necessary. S G

7

Major Review Phases—May, 2015

ReceiptReceive Applications and ProposalsTask WHO GCB DOCS

Import electronic application into IMPAC II. S, RO G

Check SF424 grant queue for electronic applications. S, R G

Submit electronic proposals into the Contract Proposal System (eCPS): https:// ecps . nih .gov .

O C

Set up Reviewer Support Site (RSS): https://eRSS.nih.gov

S C eRSS SRO User Guide eRSS Reviewer Guide

OA accepts electronic proposals in eCPS OA C

Import electronic proposals from eCPS into eRSS S, R C

Use an e901 form to: change an IRG assignment, change the application number, request multiple I/C, withdraw or defer an application, if necessary.

S G e-901 Assignment Change Request

901 SOP

Process Applications and ProposalsTask WHO GCB DOCS

Assign SEP number. RO B

Create shell in IMPAC II. RO B

Assign applications to shell in IMPAC II. RO G

Upload proposals from eCPS into eRSS. S C eRSS SRO User Guide

For multiproject applications, enter subcomponents in IMPAC II.

R G

Compare receipt to LOIs; check on those missing, if any; discuss with CO/PO.

S B

Compare applications with IMPAC II list; resolve discrepancies.

S, R G

Contact PIs who sent in LOIs but whose applications were not received.

S G

Contact CO about the number of proposals received S C

Perform Pre-Meeting ActivitiesPerform Administrative ReviewTask WHO GCB DOCS

Perform administrative review; use SRDMS database if needed

S, R, PA

B SRDMS Information

Construct master COI list for applications R, S G COI Entry Guidelines

Construct COI list for proposals in eRSS or using the R, S C eRSS SRO User Guide

8

Major Review Phases—May, 2015

SRDMS module for contracts.

Identify expertise required. S B Attachment 4: Reviewer Expertise Form

eRSS SRO User Guide

Prepare “COI package” S, R B see below

A. Grant meeting if using eRA Peer Reviewer Recruitment Control Center: Add additional key personnel to the list of involved personnel in eRA Peer Review. Prepare the Expertise List and the Personal Information Form to be emailed to potential reviewers.

S, R G Making Availability Calendars COI Pre-Review Certification for

Staff Attachment 3: Reviewer

Personal Information Form

B. Grant meeting (not using eRA Peer Reviewer Recruitment Phase): Prepare the COI pack to email to reviewers.

S, R G Pre- And Post-Meeting COI Forms Approved By OER

Making Availability Calendars Attachment 1: COI Pre-Review

Certification for Grant Reviewers (doc)

Attachment 1: COI Pre-Review Certification for Grant Reviewers (pdf)

COI Pre-Review Certification for Staff

Attachment 2: Key Personnel and Institutions

Attachment 3: Reviewer Personal Information Form

C. Complete the Proposal Template and upload into eRSS (this contains the list of involved personnel)

S, R C eRSS SRO User Guide

Identify, Recruit, Assign ReviewersTask WHO GCB DOCS

Determine whether the review will be conduct using one SEP or multiple SEPs.

S, BC

G

Recruit reviewers. S B Sample Reviewer Tracking Workbook

Reviewer Availability Calendar and Doodle

Set up IAR: Enable reviewers in eRA commons and let them know about the automatic e-mail invitation they are going to receive to join the review meeting

S, R G https://commons.era.nih.gov/ commons/

Send COI pack S, R B see below

A. Grant meeting if using eRA Peer Review Recruitment Control Center:

Tell reviewers to complete COI and confidentiality certification in IAR.

S, R G Making Availability Calendars COI Pre-Review Certification for

Staff Attachment 2: Key Personnel

and Institutions Attachment 3: Reviewer

9

Major Review Phases—May, 2015

Personal Information Form

B. Grant meeting (not using eRA Peer Review Recruitment Phase): Email COI attachments to reviewers. Collect and tally responses.

S, R G Making Availability Calendars Attachment 1: COI Pre-Review

Certification for Grant Reviewers (doc)

Attachment 1: COI Pre-Review Certification for Grant Reviewers (pdf)

COI Pre-Review Certification for Staff

Attachment 2: Key Personnel and Institutions

Attachment 3: Reviewer Personal Information Form

C. Tell reviewers to complete the pre-review process in eRSS.

S, R C eRSS SRO User Guide SOP: Ban on Federal Lobbyists

Compile ROS Roster. D B

Assign reviewers. Select Chairperson. S B

Upload meeting documents S,R B Documents for Mailout Managing IAR Meeting

Materials eRSS SRO User Guide

Additional Tasks for Face to Face Meetings

Send travel instructions to reviewers. S,R B Non-NIH Affiliated Traveler Addition Form

Travel and Reimbursement Instructions for Federal Reviewers

Travel Information for Non- Federal Reviewers

Checking Reviewer Flight Status Request For Premium Class

Upgrade (Medical Need)

Send roster to World Travel Service. R B

Inform and ReportTask WHO GCB

Meet with concerned staff. Discuss potential review dates, responsiveness, streamlining, potential reviewers, and order of summary statement/TER delivery.

S B

Ensure that staff return COI forms R, S B

Request relaxation of COI from OER, (via SRP Director) if necessary.

S, BC

B Example COI Waiver for Grants and Contracts

COI Waiver Instructions

Request waiver for concurrent service or same institution for peer reviewers via CMO, if needed.

S B Concurrent Service or Same Institution Waiver

10

Major Review Phases—May, 2015

Request waiver for NIAID SRP contractor staff to attend closed sessions of FACA-regulated peer review meetings if necessary.

S, BC

B SOP: Contractor Staff Attendance of Closed Peer Rev Mtgs

Complete Federal Register notice; deliver to CMO (30 days minimum).

R B Federal Register Notice Fillable Form

Federal Register Notice SOP Review meeting nomenclature

chart

If date or place changes, provide corrected Federal Register notice to CMO.

R B

Organize Review MeetingTask WHO GCB

Determine type of meeting venue, i.e., hotel or Fishers Lane conference room if face to face, and secure it.

R, S B BPA Hotel Contract Fishers Lane Meeting and

Conference Services Hotel Contract Process for

Face-to-Face Meetings Scientific Review and Evaluation

Activities (SREA) site

Set up teleconferences as needed. R B Teleconference Matrix Example IMPAC II Coding for

Teleconferences Teleconference Systems

Enter meeting data into IMPAC II once date, time, and location have been determined.

R B

Distribute Review MaterialsTask WH

OGCB DOCS

Prepare and check critique templates. S, TW

G Critique Template Examples

Prepare TESS in RSS and send printouts to CS/CO as courtesy.

S C

Give reviewers access to review material in IAR (grants) or eRSS (contracts)

Managing IAR Meeting Materials

As an alternative, distribute applications or proposals to reviewers using an Adobe Portfolio/Security Envelope uploaded to the SEFT site or sent on a CD.

S, R B SEFT User Guide

As an alternative, assemble a paper mailout package (if non-electronic).

S, R B

Perform subsequent mailouts as needed. R, S B

Note: Teleconference reviewers for application reviews

will also use Voter Sheets, and Pre- and Post-COI forms as back-ups to IAR, which can be

11

Major Review Phases—May, 2015

posted in IAR Meeting Materials or emailed shortly before the teleconference review. Teleconference reviewers for contract proposal reviews will receive similar backup documents to be used if eRSS fails.

Prepare for MeetingTask WHO GCB

Send “concerned staff” memo with roster and agenda. R, S B Pre-Meeting Memo Template for: Pre-Meeting Memo - Grants Pre-Meeting Memo - Contracts

Hold pre-meeting orientation teleconferences as necessary. Distribute orientation dial-out or dial-in information. Discuss discussion points with PO/CO.

R, S B Scheduling Calendar Info Meeting title instruction SOP Review Meeting Nomenclature

Chart Teleconference Systems Folder

Grant and Contact reviewers: Approximately one week before critique due date. Include reminder for posting and review dates, call in information for teleconference reviewers, travel information.

S B

Update/correct roster, agenda, and assignment list as necessary.

S, R B

Contact staff to coordinate remarks at meeting. S B

Prepare opening and closing remarks; prepare notes. S B Pre-Meeting and Meeting Checklist

Prepare preliminary Human Subject (HS) and Vertebrate Animal (VA) codes and biohazard comments.

S B

Arrange pre-meeting session with Chair as necessary. S B Meeting Guidance for Chair

Provide “Order of Review” and streamlining plans to Chair.

S B

Prepare meeting folders and deliver to SRO to be checked two days ahead (face-to-face only).

R B Folder Contents Template: Grants

Folder Contents Template: Contracts

Pre-Meeting and Meeting Checklist

Conference Room Reservation SOP

Folder Label Machine Macro Template (Avery5366)

Post-Review Evaluation Form IAR Final Scoring Instructions Seating Chart Template (MS

Word) Seating Chart Template (MS

PowerPoint)

12

Major Review Phases—May, 2015

Supplementary Comment Sheet Form

Travel Instructions for Non-Fed Reviewers (WTS)

Deliver folders to Program Director and BC one day ahead (face-to-face only).

R B

Check IAR site for critiques and scores. S G

Check eRSS site for TESSs and scores. S C

Additional Tasks for Face to Face meetings

Deliver rooming list (annotated roster) to hotel. R B

Notify ESA of federal reviewers to arrange federal travel orders.

S, R B

Update BPA for hotel as necessary. R B

Ensure that equipment necessary for the meeting is operational.

R B

Arrange for laptop with remote login privileges and personal profile, if needed.

R, S B

Prepare sign in forms, folders, and tent cards. R B Sign-In Form Tentcard Macro Template Tentcard Template

Pack box with supplies, equipment, materials. R B What to Bring to the Meeting

Have readily available final roster, assignment list, reviewer contact information, extra staff COI documents.

R B

Meet with Chair just before review, as necessary. R B Meeting Guidance for Chair

Conduct Peer Review MeetingEstablish Working Environment Task for Face to Face meetings WH

OGCB DOCS

Arrive at least two hours early; set up room; coordinate with hotel or Fishers Lane staff.

S, R B

Welcome and sign in reviewers and NIH staff each day.

S, BC, R

B Sign-In Form

Work with hotel or Fishers Lane staff to maintain room environment; monitor reviewers’ conflicts; monitor room security; have the room locked in the evening.

S, R B

Oversee Peer Review Process

Task WHO GCB

Set up for teleconference reviewers. R G

13

Major Review Phases—May, 2015

Provide opening remarks as necessary. S B

Assist the Chair as necessary. S B

Facilitate the evaluation of the applications/proposals by the panel.

S B

Take notes. S B

Oversee post-review evaluations. S B Post-Review Evaluation Form

Thank reviewers and provide closing remarks. S B

Remind reviewers to revise critiques and criterion scores on IAR.

S G

Collect Meeting MaterialsTask WHO GCB DOCS

Collect the following: (use email or FedEx if teleconference):

Voter Sheets (if used) R G

Supplemental comment sheets (pink sheets) R G

Signed TESS forms (if not on RSS) S C

Signed pre- and post-review COI forms from all reviewers (if not electronic)

R, S B

Collect signed and dated SEP minutes from Chair (if not electronic)

S, R B

Additional Tasks for Face to Face meetings at hotels

Box up materials for return to office. R B

Check out with hotel staff, check hotel contract (do not sign).

S, R B

Return materials to office. S, R B

Perform Post-Meeting ActivitiesRelease Scores and Scientific CodesTask WH

OGCB

Check (recheck) score sheets; resolve any problem entries.

R, S G

Contact reviewers as necessary to obtain missing scores/Voter Sheets (if not electronic), revised critiques.

S G

Enter scores (if not electronic) and codes into IMPAC II but do not release; print reports for SRO review.

R G

Carefully review score matrices; resolve any issues. S G

Determine final official coding for each application S, R G

14

Major Review Phases—May, 2015

(human subjects, WMC inclusion, animal welfare), which is needed in order to release meeting.

Notify PO of expected release date and time for scores.

S G

Release scores in consultation with SRO. R G

Send post-meeting memo with scores to “concerned staff” within five business days.

S, R G PostMtg Memo Template Grants

Pre- and Post-Mtg Distribution List Flyer

Review score sheets from eRSS (or CO/CS). Check; resolve issues.

S C

Release Roster, Meeting, MinutesTask WH

OGCB

Email SEP minutes to OCM. R, S B PostMtg Docs Required by NIAID OCM

Finalize entries in IMPAC II regarding meeting and attendees.

R B

Release meeting in IMPAC II. R G

Reimburse and Thank ReviewersTask WH

OGCB

Prepare the SREA Meeting Report within three business days of the meeting.

R B PostMtg Docs Required by NIAID OCM

PostMtg SOP - SREA Roster Report

Prepare and mail thank you letters. S, R B Thank you letter instructions Thank You Letter Envelope

Macro Thank You Letter Macro

Template

Release Summary Statement or Technical Evaluation ReportTask WH

OGCB

Concatenate summary statement, using either the Summary Statement macros or SRDMS

R, PA

G Concatenation Macro User Manual

SS Concatenation Macro Shell for BOXLESS Critiques

BOXED SS Concatenation Macro Shell: Only use for reviews using the Boxed Critique Templates

SRDMS Summary Statement Guide

15

Major Review Phases—May, 2015

Lightly edit summary statement critiques for spelling, grammar, etc.

TW, S

G

Draft and edit Resume and Summary of Discussion and Resume sections.

S G

Notify PO of expected release dates for summary statements.

S G

Upload, check, and release summary statement. S, R G

Prepare TER shell (this is in the process of being replaced by eRSS).

S, R C TER Shell Template

Email TER Package to OA (includes original SEP minutes, pre- and post-COIs, copies of waivers, TESSs, and TER). Have CO sign document delivery form.

S C Deliverables to CO Form PostMtg Memo Template

Contract

Archive MeetingTask WH

OGCB

Transfer files for archiving. R B Meeting Materials Archiving Checklist

Post-Review Document Retention Guidelines

Provide post-review information for RIMS. S, PA

B RIMS Single-NIAID Initiative Data Template

RIMS Single-NIH Initiative Data Template

RIMS General User Overview RIMS SRP Data Cheat Sheet RIMS SRP Module Guide

Provide post-review evaluation summary to BC to add to RIMS.

S, BC

B

Maintain SRP files; purge according to SOP. R, S B Post-Review Document Retention Guidelines

Discard unneeded materials after council meets (grants) or award made (contracts); shred confidential materials (e.g., assignments).

R, S B

16