When ePRO

PRO Selection Tool:• Which collection method for PRO data will best

support your trial—Paper or ePRO?

Online ePRO Decision Tool:• Which ePRO collection method will best support your protocol?

US 1.877.360.2901 • EUR 41.22.879.91.00

The importance of learning how patients feel and function

when taking a new clinical therapy has been acknowledged

by the FDA, EMA and other global

regulatory authorities. Sponsors

currently engaged in drug development

programs appreciate and leverage

the added value of patient-reported

outcome (PRO) data. They no longer

ask if PROs should be collected, but what phase to begin PRO collection.

While it is well known that collecting

subjective data on electronic devices

(instead of on paper diaries) reduces

trial costs and duration, quantifying

the return on investment ePRO provides by trial phase can

be elusive. Most major pharmaceutical companies have

declared ePRO as the de facto standard for PRO endpoints.

But many mid-sized pharmaceutical companies and CROs are

just beginning to explore ePRO. For them, comparing the costs

of paper diaries to ePRO requires assigning

costs to the associated support functions

for paper methods. Such costs (in time and

money) are distributed across departments

(clinical operations, data management,

submissions, etc.) and it can be nearly

impossible to fully account for the many

distributed costs of paper.

This issue of Insights is intended to identify

the costs (delays and expenses) of collecting

patient-reported outcomes on paper; and

compare these against electronic PRO

capture. The intention of this issue to provide clinical teams

with industry data that can refute the presumption that paper

methods are cheaper than ePRO.

3

U S H E A D Q U A R T E R S :

PHT Corporation500 Rutherford AvenueBoston, MA 02129 USAToll-Free: US 1.877-360-2901

E U R O P E A N H E A D Q U A R T E R S :

PHT Corporation Sàrl2, chemin Louis-Hubert1213 Petit-Lancy, Geneva, SwitzerlandPhone: 41.22.879.91.00

www.phtcorp.comCopyright © 2010 PHT Corporation

Rev 092010

6

PHT Insights — When and How to Justify Electronic Patient-Reported Outcomes (ePROs)

Contents

Trial conditions dictate whether and when ePRO should supersede paper diaries.

p.1

eClinical Trial Solutions Market, Global, Cost Saving Scenario of ePRO Solutions, 2008

p.2

Step 1- Identify and select your company’s trial technol-ogy adoption stage

p.2

Step 2- Identify the hidden cost centers for collecting PRO on paper diaries

p.3

Step 3 - Compare your trial conditions to average or median costs for global trials within Phases 2 & 3

p.3

Table II: How to Quantify Specifi c ePRO Benefi ts p.4

Conclusion p.6

1 Doward et al, “Patient reported outcomes: looking beyond the label claim”, August 2010.2 GBI Research, “The Future of eClinical Trial Solutions - Market Forecasts to 2015, Competitive Benchmarking and Case Studies,” GBI Research, GBIHC021MR (December 2009): p.3. 3 Applied Clinical Trials, “The Upfront Cost Hurdle of EDC”, April 2010: p. 524 Pompa et al, GlaxoSmithKline, July 2008 Powerpoint presentation “ePRO Awareness Session”. 5 Stone, et al., Patient non-compliance with paper diaries, BMJ 324: 1193, 18 May 2002.6 FDA, Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims, p.14, December 2009” 7 Paraxel’s Bio/Pharmaceutical R&D Statistical Sourcebook 2009/2010, with extrapolations.

Call PHT for these FREE ePRO decision tools:

• Provides regulatory agencieswith proof that the protocoltime requirements were met

• Provides on-demand data forevaluation or analysis

• Improves subject compliancewhen private or sensitive datais collected

• Enables smaller sample sizes

ePRO

• Paper is practical for supervisedPRO data collection where thesubject population is abundant,PRO data are not personal, orwhen there are no plans for PROdata submission.

PAPER1. Will PRO data beused to supporta claim?

NO

YES

NO

YES

4. Is subject recruitmentdifficult?

NO

YES

2. Is the PRO dataintended for anFDA/EMEA/otheragency Fast Track,Accelerated Approvalor Priority Review?

YES

3. Are you collectingsensitive, personaldata?

US 1.877.360.2901EUR 41.22.879.91.00www.phtcorp.com

• Regulatory agencies recommend that the PROdata collection method be capable of ensuringthat reports are done on schedule, as defined bythe protocol time requirement. “If a patient diaryor some other form of unsupervised data entry isused, we plan to review the clinical trial protocolto determine what steps are taken to ensure thatpatients make entries according to the clinicaltrial design and not, for example, just before aclinic visit when their reports will be collected.”

• ePRO data has been shown to cluster to the meanvalue (lower standard deviation) compared to paperdata, enabling smaller sample sizes in Phase 2trials and more rapid, conclusive Phase 3 trials.

• ePRO on-demand data enables rapid access forindependent evaluation or analysis, and is appropriatefor any regulatory body that will want to meet withthe sponsor regularly to review the data (possibly foran expedited process).

• Rapid access to data enables adaptive design andother trial efficiencies.

• Subjects welcome the opportunity to record withoutembarrassment, and appreciate that caregivers andfamily lack access to their personally revealinginformation.

• Subjects are more willing to provide morecomplete data.

ePRO ePRO ePRO ePRO

NO

©2010 PHT Corporationv12010Trust Your Patient Data

Insist on ePRO data for rapid analysis.

‘…patient reported outcome

evidence is increasingly

viewed as an essential

complement to traditional

objective clinical evidence

for establishing a product’s

competitive advantage in

the marketplace.’ 1


How to Determine if ePRO Suits Your Trial ConditionsThis issue of Insights assumes

that the reader understands

that ePRO is a good fi t for a

trial, and needs to justify the

costs of automation.

A previous issue of Insights

entitled ‘Collecting Electronic

Patient-Reported Outcomes (ePROs): Comparing the 5 Proven

Ways to acquire Attributed Patient-Reported Data’ details

how to determine if ePRO is appropriate for a particular trial.

This is available for download at http://www.phtcorp.com/resources/insights_newsletters/

Identify the hidden cost centers for collecting PRO on paper diaries:After identifying your adoption stage, the next step is to identify which ePRO benefi t(s) are most relevant to the evaluation of paper vs. ePRO data collection.

For example, clinical trial managers whose organization or therapeutic division is in Stage1 for technology adoption are advised to focus on reduced site burden, increased compliance, confi rmed time-stamped data entry, increased data availability, reduced data loss and decreased enrollment tracking time:

“Using an e-diary is always recommendable,

when assessing principal criteria, whatever the

phase (II or III). In this case, the improvement

in data quality with ePRO far exceeds the

investment in technology.”

Mrs. Silvia Munoz ChimenoClinical Project ManagerServier Spain

Step 3: Step 2:

Compare your trial conditions to average or median costs for global trials within Phases 2 & 3:Refer to Table II–How to Quantify Specifi c ePRO Benefi ts on page 4 to identify which ePRO benefi t to quantify, based on your organization’s technology phase:

Example1: Sponsor in Stage1 technology adoption

Phase 2 trial with 160 subjects and 19 sites.

Average cost $4,160,000.

ePRO will conservatively save 10% ($416,000.) of trial costs for these reasons:

• Reduced site burden $300,000.

• Increased compliance 67,000.

• Confi rmed time-stamped data entry 141,000.

• Increased data availability 162,000.

• Reduced data loss 52,000.

• Decreased enrollment tracking time 15,000.

Using calculations from Table II, the projected savings would be $737,000. (18%)




ePRO will conservatively save 10% ($1,544,000.) of trial costs for these reasons:







• Decrease in data variance 154,000.

• Decreased database lock 1,315,000.

• Fewer queries and faster resolution time 309,000.

Using calculations from Table II, the projected savings would be $3,901,000. (25%)

Conclusion

There are many tangible economic reasons to utilize

ePRO instead of paper diaries in a clinical trial. Data

supports the solid business case to elect ePRO for trials

across phases whenever the patient perspective is

critically important to prove the safety and effi cacy of a

therapy. Even if the patients’ perspective is NOT critical,

Sponsors may elect ePRO to remain competitive and

elicit maximum data regarding patient experiences, side

effects, outcomes and the overall benefi ts a given drug

offers vs. the competition.

More robust online ROI and ePRO Decision Tools are

available from your PHT Account Executive. These Tools

are designed to guide clinical trial managers through

a short series of questions about their trial, in order to

summarize which ePRO modality is best suited for the

trial, and how to quantify and justify collecting trial data

on ePRO vs. paper diaries.

PHT LogPad– Hand Held for patients on-the-go

PHT SitePad Tablet –The mobile

touch-screen tablet for ePRO data

collected at sites

ePRO Benefits Stage1 Stage2 Stage3 Stage4

A. Reduced site burden

B. Timeliness of diary presentation

C. Confirmed time stamped data entry

D. Increased data availability

E. Reduced data loss

F. Decreased enrollment tracking time X X

G. Decreased Data Variance X X

H. Decreased Database Lock time

I. Fewer queries and faster resolution time

J. Reduced trial duration

K. Decreased monitoring visits

L. Increased reuse of systems & components

X X X X

X X X X

X X X X

X X X X

X X X X

X X

X X

X X X

X X X

X X X

X X

X X

Technology Stage

X

X

X

X

Stage4

X

X

X

X

X

X

X

X

When and How to Justify Electronic Patient-Reported Outcomes (PROs)

Step 1:

Identify and select your company’s trial technology adoption stage: To frame the context of your decision, clinical trial managers should acknowledge the current level of ePRO adoption at their organization. PHT has found the stage of ePRO adoption, as defi ned below, helps frame the decision and ROI for using ePRO in a particular trial. The adoption stage helps specify which ePRO attributes will be most important to a given trial ROI evaluation. The stages:

Stage1 Sponsors are starting to use a basic ePRO system for simple uses, such as integrated voice response (IVR) for patient qualifi cation, enrollment and/or randomization. Paper is the most frequently utilized method of data collection.

Stage2 IT groups within Sponsors are adopting EDC technologies to gain trial data effi ciencies. Having recognized value from simple IVR ePRO use, these sponsors are testing other ePRO modalities such as mobile data capture devices for the subject and/or for the site. Since ePRO systems drive cross-functional benefi ts, data management and health outcome professionals collaborate on ePRO evaluations.

Stage3 These Sponsors have shifted from paper to EDC. For those trials where primary and secondary endpoints will be drawn from data collected directly from patients, the clinical teams expect to use ePRO. Clinical and data teams prefer ePRO over paper although in some instances management may require them to use paper for certain trials. For sponsors at this stage, outsourcing professionals are often responsible for selecting CROs and ePRO systems that are proven to be successful for large international projects and who have an unblemished record with regulatory authorities.

Stage4All Sponsor clinical management systems are fully integrated with one or more EDC systems. All note taking, signatures, lab results, imaging and ePROs are electronic. ePRO Systems are used for trials in phases 2 and 3, and in phase 4 (post marketing surveillance) trials. Sponsors seek long-term partnerships with vendors, especially those willing to share trial risks.

Cost Saving Scenario of ePRO Solutions

The paper diary method is a lengthier process than ePRO, resulting in increases in cost and time. If Pharmaceutical Companies are adopting ePRO solutions, they save considerable cost and time.2

This document is intended to assist in calculating the fi nancial advantages of ePRO over Paper in 3 stepsStep 1: Identify and select your company’s trial technology adoption stage; Step 2: Identify the hidden cost centers for collecting PRO on paper diaries; andStep 3: Compare your trial conditions to average or median costs for trials within Phases 2 & 3.

M any clinical trial managers within Sponsors and CROs are familiar with the data quality and integrity benefi ts of ePRO,

and recommend ePRO for phase 3 trials where PRO measures serve as endpoint data. Documented advantages of ePRO include

• Simplifi cation of site tasks (storing, correcting, shipping paper diaries),

• Reductions in data variance,

• Shorter study duration,

• Improved protocol compliance through real-time monitoring (via time-stamped data entry reports), and

• Reduced time to database lock.

Sponsors and CROs recognize that the ‘…effi ciencies linked to new technologies generally increase over time as end users gain familiarity with them through repeated use.’3 As a result of the quantifi able effi ciencies and downstream savings of ePRO, more clinical trial managers are also electing ePRO for their phase 2 trials. Given the critical proceed or stop decisions that are often made based on Phase 2 results, this makes sense even when there are relatively fewer subjects and a shorter timeline. Such decisions benefi t from a full understanding of the experience of each of the subjects, and ePRO consistently delivers more pertinent information per subject.

Unlike their counterparts within the top ten pharmaceutical giants who have come to regard paper diary collection as the exception, clinical trial managers within mid-sized pharmas and biotechs must demonstrate the fi nancial advantages of collecting PRO electronically, since their organizations are still accustomed to paper diaries.

2 4 5

Benefi t of ePRO How to Calculate7 ePRO Savings

A. Reduced site burden When enrollment criteria for studies is calculated by ePRO, a signifi cant burden from staff is removed. ePRO also ensures that only patients who belong in the study are actually enrolled.4

Cost benefi t of Reduced Site Burden:

• It costs $20,000 to activate a site and $2,000/mo to maintain one

• Average number of sites in Phase 2 & 3 global trials is 35

• Average duration of Phase 2 & 3 global trials is 33 months

• Average site activation and maintenance is $3,000,000 with paper PRO

ePRO reduces site burden of paper administration by 20% $600,000

B. Increased compliance The worthlessness (inaccuracy, untimeliness) of [paper diary] data is now well accepted, beginning with anecdotal evidence experienced by researchers decades ago and proven in controlled examination like that published in the British Medical Journal in 2002.5

Cost benefi t of Increased Compliance:

• Data Cleaning consumes 8% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Data Cleaning costs $784,000 with paper PRO

ePRO improves compliance by 20%, saving that in Data Cleaning $157,000

C. Confi rmed time-stamped data entry When evaluating PRO-based claims, we intend to review the clinical trial protocol to determine what steps were taken to ensure that patients understood the instrument recall period.6

Cost benefi t of Timely Diary Presentation:

• Patient treatment after enrollment costs 17% of the trial

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Patient treatment after enrollment costs $1,700,000

with paper PRO

ePRO reduces patient treatment costs by 20%, eliminating the cost of diary data that cannot be submitted due to illegible, untimely or irrelevant paper entries $340,000

D. Increased Data Availability Cost benefi t of Increased Data Availability:

• Site Monitoring consumes 7% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Site Monitoring costs $685,000 with paper PRO

• Increased data availability with ePRO reduces site visits by 20% $137,000

• Subject Monitoring consumes 10% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Subject Monitoring costs $980,000 with paper PRO

• ePRO reduces study coordinator time required to collect, review and assign new paper Daily Diaries at each visit, by 25% $245,000

Total ePRO savings of increased data availability $382,000

Benefi t of ePRO How to Calculate ePRO Savings

E. Decreased Data Loss Cost benefi t of Decreased Data Loss:

• Statistical Analysis consumes 5% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Statistical Analysis costs $490,000 with paper PRO

ePRO reduces statistical analysis by reducing paper data loss by 25% $122,500

F. Decreased Enrollment Tracking Time Cost benefi t of Decreased Enrollment Tracking Time:

• Site Selection/Enrollment/Contract consumes 12% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Site Selection/Enrollment/Contract consumes $1,176,000

with paper PRO

• Enrollment tracking consumes 6% or $588,000 with paper PRO

ePRO decreases study coordinator enrollment tracking time by 50% $294,000

G. Decrease in Data Variance Cost benefi t of Decreased Data Variance:


• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Statistical analysis costs $490,000 with paper PRO

ePRO reduces data variance by 20% $98,000

H. Decreased Database Lock Time Cost benefi t of Decreased Datase Lock Time [valid for Phase 3 trials]:


• Average cost of Phase 3 global trials is $15,440,000 • Data Cleaning consumes 8% , or $1,235,000 with paper PRO

• ePRO reduces data cleaning by 20% $247,000

• Average cost of Phase 3 global trials is $15,440,000

• Database lock consumes 37% of the trial time and cost, or $5,712,800

• ePRO reduces database lock time by 18.7% $1,068,000

Total ePRO savings of decreased database lock time. $1,315,000

I. Fewer Queries & Faster Resolution Time Cost benefi t of Reduced Queries:

• Subject Monitoring & Administration consumes 10% of trial time and cost

• Average cost of Phase 2 & Phase 3 global trials is $9,800,000 • Subject Monitoring & Administration costs $980,000

with paper PRO

• Queries and data cleaning consumes 25% of Subject Monitoring & Administration, or $245,000

ePRO reduces queries by 80%, saving 2% of trial time and cost $196,000

Table II: How to Quantify Specifi c ePRO Benefi tsBenefi t of ePRO How to Calculate Benefi t of ePRO How to Calculate ePRO Savings

Table II: How to Quantify Specifi c ePRO Benefi ts continued

When and How to Justify Electronic Patient-Reported Outcomes (ePROs) When and How to Justify Electronic Patient-Reported Outcomes (ePROs) When and How to Justify Electronic Patient-Reported Outcomes (ePROs)

Source: GBI Research

Saved Cost

Sent to DM

X

Paper Diaries ePRO

Subject

Site Review andColection

Site Monitor

Data Entry

Sponsor DB

Medical Monitor and Statistician




US 1.877.360.2901 • EUR 41.22.879.91.00














the return on investment ePRO provides by trial phase can be

elusive. Most major pharmaceutical companies have declared

ePRO as the de facto standard for PRO endpoints. But many

mid-sized pharmaceutical companies and CROs are just

beginning to explore ePRO. For them, comparing the costs of

paper diaries to ePRO requires assigning















3






Rev 092010

6


Contents


p.1


p.2


p.2


p.3


p.3


Conclusion p.6







ePRO



NO

YES

NO

YES


NO

YES


YES









ePRO ePRO ePRO ePRO

NO































Step 3: Step 2:



























Conclusion





















collected at sites














X X X X

X X X X

X X X X

X X X X

X X X X

X X

X X

X X X

X X X

X X X

X X

X X

Technology Stage

X

X

X

X

Stage4

X

X

X

X

X

X

X

X

When and How to Justify Electronic Patient-Reported Outcomes (ePROs)

Step 1:


















2 4 5


















with paper PRO


















with paper PRO


















with paper PRO







Saved Cost

Sent to DM

X

Paper Diaries ePRO

Subject


Site Monitor

Data Entry

Sponsor DB


Step 1:


















2 4 5


















with paper PRO


















with paper PRO


















with paper PRO







Saved Cost

Sent to DM

X

Paper Diaries ePRO

Subject


Site Monitor

Data Entry

Sponsor DB





US 1.877.360.2901 • EUR 41.22.879.91.00














the return on investment ePRO provides by trial phase can be

elusive. Most major pharmaceutical companies have declared

ePRO as the de facto standard for PRO endpoints. But many

mid-sized pharmaceutical companies and CROs are just

beginning to explore ePRO. For them, comparing the costs of

paper diaries to ePRO requires assigning















3






Rev 092010

6


Contents


p.1


p.2


p.2


p.3


p.3


Conclusion p.6







ePRO



NO

YES

NO

YES


NO

YES


YES









ePRO ePRO ePRO ePRO

NO































Step 3: Step 2:



























Conclusion





















collected at sites














X X X X

X X X X

X X X X

X X X X

X X X X

X X

X X

X X X

X X X

X X X

X X

X X

Technology Stage

X

X

X

X

Stage4

X

X

X

X

X

X

X

X

When and How to Justify Electronic Patient-Reported Outcomes (PROs)

Health & Medicine

When ePRO