UCSF08 - ITN Concept Proposal_Award_2008

University of California Larry L. Sautter Award Submission

Concept Proposal Project University of California, San Francisco

Submitted By:

Mr. Michael Williams, MA University of California

Executive Director, Information Technology UC San Francisco Diabetes Center, Immune Tolerance Network

and Chief Information Officer

UC San Francisco Neurology, Epilepsy Phenome/Genome Project

Telephone: (415) 860-3581 Email: [email protected]

Date Submitted: Friday, May 23, 2008

Page 1

Table of Contents

1. PROJECT TITLE ........................................................................................... 2 2. SUBMITTER’S DETAILS .............................................................................. 2 3. NAMES OF PROJECT LEADER(S) AND TEAM MEMBERS ...................... 3

3.1. TEAM LEADERS ................................................................................................. 3 3.2. TEAM MEMBERS ................................................................................................ 3

4. PROJECT SIGNIFICANCE ........................................................................... 4 5. PROJECT DESCRIPTION ............................................................................ 5

5.1. WHO IS THE IMMUNE TOLERANCE NETWORK? ...................................................... 5 5.2. WHAT IS THE CONCEPT PROPOSAL PROCESS? .................................................... 5 5.3. CONCEPT PROPOSAL SYSTEM ............................................................................ 6 5.4. SITUATION PRIOR TO THE CONCEPT PROPOSAL SYSTEM ...................................... 6 5.5. THE CONCEPT PROPOSAL PLATFORM ................................................................. 7 5.6. PROCESS IMPROVEMENTS ................................................................................ 13

TECHNOLOGY UTILIZED ................................................................................. 14 6. TIMEFRAME OF IMPLEMENTATION ........................................................ 15 APPENDIX – SYSTEM FLOW DIAGRAMS ....................................................... 16

Page 2

1. Project Title

Concept Proposal System

2. Submitter’s Details

Mr. Michael Williams, MA University of California Executive Director, Information Technology UC San Francisco Diabetes Center, Immune Tolerance Network and Chief Information Officer UC San Francisco Neurology, Epilepsy Phenome/Genome Project Telephone: (415) 860‐3581 Email: [email protected]

Page 3

3. Names of project leader(s) and team members

3.1. Team Leaders

Michael Williams, M.A. Executive Director, Information Technology UC San Francisco Diabetes Center, Immune Tolerance Network and Chief Information Officer UC San Francisco Neurology, Epilepsy Phenome/Genome Project Kevin Miller Programmer Analyst III UC San Francisco Neurology, Epilepsy Phenome/Genome Project

3.2. Team Members

Immune Tolerance Network Information Technology: Ryan Haile Project Manager UC San Francisco Diabetes Center, Immune Tolerance Network Project Sponsors:

Michael Williams, M.A. Executive Director, Information Technology, Immune Tolerance Network Jeff Bluestone, Ph.D. Director Diabetes Center and Immune Tolerance Network Philip Bernstein, Ph.D. Executive Director, Strategic Review and Planning Anita Weinblatt Director, Office of Scientific Review (start of project)

Page 4

4. Project Significance

By deploying the Concept Proposal System within the Immune Tolerance Network (ITN) at the University of California, San Francisco (UCSF) we have implemented a system that gives the Immune Tolerance Network a fast, secure, and reliable online system to manage proposal funding submissions. We have reduced the administrative cost by not only managing the reviews online but by also reducing reviews from quarterly to bi‐yearly due to the ability to manage multiple proposals simultaneously. The Immune Tolerance Network strives to be ecologically friendly and with the deployment of the Concept Proposal System we are fully electronic from the original proposal submission to the final emailed decision.

Page 5

5. Project Description

5.1. Who is the Immune Tolerance Network?

The mission of the Immune Tolerance Network (ITN) is to prevent and cure human disease. Based at the University of California, San Francisco (UCSF), the ITN is a collaborative research project that seeks out, develops and performs clinical trials and biological assays of immune tolerance. ITN supported researchers are developing new approaches to induce, maintain, and monitor tolerance with the goal of designing new immune therapies for kidney and islet transplantation, autoimmune diseases and allergy and asthma. Key to our success is the ability to collect, store and analyze the huge amount of data collected on ITN’s 30+ global clinical trials at 90+ medical centers, in a secure and effective manner, so a reliable, scalable and adaptable IT infrastructure is paramount in this endeavor. The ITN is in the 8th year of 14‐year contracts from the NIH, National Institute of Allergy and Infectious Diseases (NIAID), the National Institute of Diabetes and Digestive and Kidney Disorders (NIDDK) and the Juvenile Diabetes Research Foundation.

5.2. What is the Concept Proposal Process?

Concept Proposal is the process the Immune Tolerance Network uses to evaluate submitted proposals from Private Investigators seeking funding for a clinical research trial. The ITN receives the application then assigns internal and external subject matter experts to review the proposal. The subject matter experts then review the supporting documentation and submit a recommendation to the ITN on whether or not to accept or reject the proposal. When the ITN receives the recommendation from the subject matter experts the proposal is then opened up for discussion among the subject matter experts to answer questions as well as to hear the argument as to why the ITN should consider the proposal or why it should be discarded. Upon completion of the discussion period the ITN will make a final recommendation to either accept or reject the proposal. Upon the acceptance of a proposal the ITN provides funding and guidance through a clinical trial.

Page 6

5.3. Concept Proposal System

The goal was to create a system that streamlined ITN’s review process using the latest technologies to improve the workflow. The system is built as a centralized environment to organize, collect, and manage submitted proposal through an online portal that is secure, fast, and reliable. The portal is designed to allow participants to read, review, vote and discuss multiple proposals through a web browser. Concept Proposals are submitted via an on‐line form located on the Network web site. These short proposals are intended to provide reviewers with a general overview of the clinical strategy or tolerance assay, the study rationale, relevance to tolerance and the requirements of the proposed studies.

Applicants submitting Concept Proposals will be notified of the results of the review process within 4‐6 weeks following the application cut‐off. Successful applicants will be invited to submit a more comprehensive proposal to the Network. Assistance with statistical/sampling considerations will be available from the Network's Statistical and Clinical Coordinating Center for the preparation of Full Applications. Authors of Full Applications will be invited to present a summary of their proposals at the subsequent Network Steering Committee meeting for final review. Authors of approved Full Applications will be notified and teamed with an appropriate Clinical Subgroup to begin protocol development.

The Network application process is a two‐stage process designed to fast‐track applications from proposal to protocol development. Applicants submit a short 4‐page Concept Proposal to the ITN that provides ITN reviewers with basic conceptual information and justification. Applicants whose proposals are of interest to the ITN will then be asked to submit a more complete Full Application for review.

The process differs significantly from traditional study section reviews by maintaining a much more interactive flavor designed to assist potential investigators the preparation of highly focused, relevant scientific proposals. Generally, the entire review process for each round of applications will be complete within 4 months following the initial deadlines.

5.4. Situation Prior to the Concept Proposal System

Prior to the Concept Proposal System the ITN faced several obstacles when reviewing submitted proposals in a timely manner. The process to submit a proposal prior to the Concept Proposal system included a principal investigator requesting an application from the ITN which would be sent via standard mail or Email. The investigator would fill out the required documents and return them to the ITN. Upon receiving the proposal, the scientific review coordinator would validate each proposal submitted to

Page 7

the ITN. Once the proposal passed the initial screening the scientific review coordinator would start the process of creating a review cycle. Select members of the ITN would be chosen to participate in the review cycle. The selected reviewers would receive notifications via standard mail. The scientific review coordinator would then create a “Conflict of Interest” form to be signed by each person who is reviewing the proposal. The scientific review coordinator would then make copies of the submitted proposal for each reviewer and send them via standard mail. The reviewers would return the proposal with their review of the proposal as well as with a recommendation on whether or not to continue to move forward. The scientific review coordinator would then gather all the reviewers’ notes/recommendations and place them in a single binder location to discuss at future meetings. The scientific review coordinator would schedule several discussions via conference calls to hear the reviewer’s opinion for each proposal. Upon completion of the discussion period the scientific review coordinator would notify the Principle Investigator of the final decision via standard mail. The major process issues are listed below

1. Timeframe from Proposal Submission to the final decision was lengthy 2. Required an extensive amount of time for one individual to collect the

information and organize it in an efficient manner. 3. Inability to read anonymous reviews from fellow Reviewers. 4. Discussions were limited to scheduled meetings and conference calls. 5. Difficult to review multiple Proposals 6. Environmentally unfriendly process 7. Limited ability to back up current and previous submitted proposal 8. Inability to review previous proposal submission in a timely manner

5.5. The Concept Proposal Platform

The Concept Proposal application was developed by the Immune Tolerance Information Technology team to service the following functional sites.

Concept Proposal Public Website

The Concept Proposal Public Website, https://conceptproposal.immunetolerance.org was published in early 2008 to mark the beginning of our new approach to reviewing submissions electronically.

Page 8

Concept Proposal Submission

The online submission is designed to allow external users to submit the required information and uploading all supporting documents for a proposed Clinical Research Trial. Implementing an online submission site allows for the ITN to reduce time by eliminating the process of sending/receiving proposals via standard mail as well as reducing cost with eliminating time and labor of handling paper requests. The online submission form is available at https://conceptproposal.immunetolerance.org/ITNPublicSubmission/Pages/SubmissionForm.aspx .

Page 9

Upon submission of a proposal an automated confirmation email is sent to the submitter. Additionally an email is sent to the Concept Proposal administrators to initiate a new Concept Proposal cycle.

Concept Proposal – Review Process

The review process starts with an administrator configuring a new Concept Proposal site for the submitted proposal. Once the site is setup the security is configured individually based on the roles assigned to each person. With the site created and the individual roles assigned the administrator sends an automated email to all users within the system announcing the opening of a new review cycle. Following the opening announcement the administrator will send automated emails to each person notifying them of their role during the review cycle with an embedded link to the proposal site.

Page 10

Initial Review Site:

There are five defined steps when reviewing submitted proposals. Below will show each step with a description on how it improved the process and created an efficient, reliable, and secure environment to review proposals. Each step has a significant impact on how our process has changed from a paper process to a paperless online process. Reviewing the criteria ITN uses when evaluating proposals is the first step when a participant begins the review process. The short description is listed along with a link to a PDF file for the detailed description. Traditionally the ITN would send out the criteria either by standard mail or Email which would require resources to insure delivery. With implementing a secure online portal we are now able to post the required documents for the participants to read at their convenience.

Page 11

Reviewing all supporting documentation in an online secure manner relieved the ITN of the costs, time, and resources to send the supporting proposal documentation to individual participants.

With the creation of an online review page, which includes selecting an overall recommendation, the ITN has reduced the amount of time and resources that required an administrator to collect, organize, and sort multiple paper reviews and recommendations.

Page 12

Previously during the read and review process the administrator would be required to conceal the identity of each review response as well as the individual recommendations before sending responses to all participants. This process was time consuming and costly. Today when you submit your recommendation and review responses online the system anatomizes the reviews and recommendations to use during the read reviews phase of the review cycle hence eliminating cost by reducing the administrative effort.

The final step when reviewing a submitted proposal is to have an open discussion with the participants to hear each reviewer’s opinion for each proposal. With the online

Page 13

discussion board the participants are able to create new discussion threads and respond to current threads without the need to schedule multiple conference calls with the participants. The bulletin board has effectively reduced multiple meetings with online discussions.

The review cycle commences with closing notification emails sent to the participants as well as a final decision email sent to the submitter.

5.6. Process Improvements

The Concept Proposal System elevated many of the issues within the ITN Concept Proposal process. The process issues addressed with the Concept Proposal System

1. Significantly reduced timeframe from Proposal Submission to the final decision

2. Eliminated the majority of time that was required by an individual to collect the information and organize it in an efficient manner.

3. Enabled access to anonymous reviews 4. Incorporated online discussions without requiring scheduled meetings and

conference calls. 5. Enabled the ITN to review multiple proposals in a timely manner 6. Reduced global footprint by eliminating paper and human resources 7. Proposals are backed up using ITN’s Tier 1 infrastructure. 8. All previous proposals are available online.

Page 14

Technology Utilized

The Concept Proposal System was built using Microsoft SharePoint 2007 and .NET Platforms. They are used in conjunction with Microsoft SQL Database and Visual Studio 2005 Professional Edition. Security to this system is managed using the Immune Tolerance Network’s Active Directory. SharePoint 2007 provides workflow and document management for the Concept Proposal System. The system is published externally with the use of IIS 6.0 and SSL.

Page 15

6. Timeframe of implementation

April 2006 Requirements Gatherings / Planning / Design April 2007 Development Commences August 2007 Use case testing April 2008 System Published

Page 16

Appendix – System Flow Diagrams

FLOW DIAGRAMConcept Proposal

Submission v. 1.5

Complete the form

Upload the completed template form

Request to perform a trial and need funding

Sto

rage

M

edia

Review Requests

Scientific Review Coordinator

Principle Investigator



Download request formIn PDF format

Clinical Trial Concept Proposal Template Form

Revised Concept Proposal Template Form

JDRFTemplate Form

Revised JDRF Template Form

Complete the form

Logon to the Site

Validate Credentials Incorrect Error Page




Using email ID and proposal number

Successful Submission Confirmation Page

Page 17

Start New Cycle

Stor

age

Med

ia

Scientific Review CoordinatorThe Proposals are now available for

the reviewers to read concept proposals, post reviews and cast vote

Change cycle status to “Read Reviews and Discuss”

Scientific Review CoordinatorReviewer can now read concept

proposals, read other reviewers reviews and join the discussion board

Cycle Closes

Concept Proposalretrieved

Make decision regarding requesting Full Application

Proposal Status Set

Discussions

Reviews and Votes

Decisions

Create Proposal Site automatically for each

Concept Proposal

Cycle Opening Notification

Post their reviews and cast their vote

Discussion Forum Notification

Scientific Review Coordinator would send this notification - All

Network Steering Committee members would be included

v. 1.1FLOW DIAGRAM Concept Proposal Cycle

Select submitted proposals which meet the ITN criteria


Scientific Review CoordinatorAll General Reviewers and Read only reviews would be included

Assign various Reviewers and conflicts of interest to the Concept

Proposal Site

Site Opening Notification

Scientific Review CoordinatorPersonal emails to Discussion Leader, Assigned Reviewers

and Ad-Hoc reviewers

Open Cycle and set status “Review & Vote”

Reviewers shall read the reviews and discuss on specific proposals

Reviewers

Scientific ReviewCoordinator, Executive

Director of Scientific Review


Reviewers


Page 18

Set Cycle Status to“Review & Vote”

Logon to the Site

Validate Credentials

Stor

age

Med

ia

correct

Assigned Reviewer

Incorrect / Primary Conflict Reviewer

Reviewers

System checks Reviewer Type

DiscussionLeader

Ad-hoc Reviewer

Error Page

General Reviewer Read Only

Confirmation Screen

v. 1.0FLOW DIAGRAM Concept ProposalPost Reviews

Scientific ReviewCoordinator

Submit Reviewsand vote

Reviewers

Page 19

Set Cycle Status to “Read Review & Discuss”

Logon to the Site

Validate Credentials

Correct

Incorrect / PrimaryConflict Reviewer Error Page

Enter Discussion Board

System checks Reviewer Type

Stor

age

Med

ia

Discussion details

v. 1.1FLOW DIAGRAM Concept ProposalRead Reviews & Discuss


Reviewers can still view the reviews

Reviewers

Assigned Reviewer

DiscussionLeader

Ad-hoc Reviewer

General Reviewer Read Only

Read Collated Reviews

Reviewers

Reviewers

Page 20

Start New Cycle / Set Cycle Status

Change Cycle Status

Send Cycle Opening Notification to General

Reviewers and Read Only Reviewers

Send “Concept Proposal subject of discussion”

notification to General Reviewers

v. 1.0FLOW DIAGRAM Concept ProposalReviewer Notification



Change Cycle status to “Review & Vote”

Change Cycle status to “Read Reviews & Discuss”


Site 1Sent Open Site Notification to

Discussion Leader, Assigned Reviewer & Ad-Hoc Reviewers

assigned to this site

Site nSent Open Site Notification to

Discussion Leader, Assigned Reviewer & Ad-Hoc Reviewers

assigned to this site




...

Page 21

Start New CycleNaming is automatically

by Cycle ID

v. 1.0FLOW DIAGRAM Concept ProposalAdminister New Cycle


Assign submitted Concept Proposals to Cycle


Each CP in the cycle presents a site

of the cycle

Site 1 Site 2 Site n

Assign supporting documents to site




Assign reviewers to site



Set Cycle Status to:Cycle Open




Validate that no other cycle is open

Open Cycle

No Cycle is open


Another CycleIs open

NotifyScientific Review

Coordinator

Cycle Status is set to:In Preparation

Set email notification text for cycle


SendEmail Notifications


Set email notification text

for site


for site


for site



Documents

UCSF08 - ITN Concept Proposal_Award_2008