9
Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies Brian W. Bresnahan, PhD, Sean D. Rundell, DPT, PhD Rationale: More efficient and better informed healthcare systems are expected to have improved knowledge of the impact of interventions on patient outcomes and resources used by patients and providers in specific health conditions. Objectives: To describe trends related to putting patients at the center of healthcare decision making, regulatory trends and best practice recommendations for developing high-quality patient-reported outcomes (PROs), and strategic issues related to including PROs in studies. Materials and Methods: We summarize PRO concepts, definitions, and broadly-accepted scientific standards for developing, assessing, and interpreting PROs. Three conceptual models are presented as examples for assessing PROs in relation to other outcomes. We discuss different perspectives for stakeholders, including regulatory issues pertaining to formal guidance for PRO development and for use in trials. We provide examples of PROs used in studies for assessing health outcomes in oncology and resource-use outcomes in low back pain patients. Results: Psychometric scientists working closely with multi-disciplinary teams and regulatory authorities have greatly improved the science of collecting, assessing, and understanding patient-reported outcomes in clinical trials. A simplified framework is presented for strategic consid- erations for including PROs in studies, such as the appropriate timing for PRO endpoints. Asking patients about their health status and/or use of resources improves our understanding of how interventions and care processes may impact their lives and their budgets. We provide examples from a back pain trial of patient-reported resource-use questionnaires for medicines taken and other services or products used by patients. Conclusions: Healthcare stakeholders are placing increased emphasis on resource use and the impact of interventions on patients, including effects associated with diagnostic tests. Patient-reported outcomes are being used in clinical practice and in clinical research, supported by formal best-practice guidelines. Radiology has a role as an engaged stakeholder in the design, conduct, and interpretation of patient-based evidence, and in its relevance to health policy implementation. Key Words: Patient-reported outcomes; clinical studies; radiology; endpoints; health outcomes. ªAUR, 2014 P atient-reported outcomes (PROs) provide valuable health information. Patients, providers, and health systems are being encouraged and incentivized to involve patients and their families more directly in health care decision making. Examples of support for additional ev- idence development related to patient outcomes in the United States include the creation of the Patient-Centered Outcomes Research Institute (PCORI) (1), published priorities in comparative effectiveness research (CER) (2–4), and the Agency for Healthcare Research and Quality’s (AHRQ’s) emphasis on patient-centered outcome research (5). ‘‘Pa- tient-centered outcomes research (PCOR) helps people and their caregivers communicate and make informed health care decisions, allowing their voices to be heard in assessing the value of health care options’’ (1). Patient input is being sought in developing research priorities, informing shared decision-making models, selecting meaningful outcomes for clinical research, and disseminating information to patients and providers. Consequently, PROs and other relevant patient-reported data have become more ingrained in clinical research, observational research, clinical care, and quality improvement initiatives. In addition to engaging patients in research and in their health care decision making, PROs are also valuable to more compre- hensively measure treatment benefits and harms in clinical research and CER. Manufacturers may choose to include PROs in clinical trials to assess the impact of interventions on patients, which has potential use in marketing their products. Clinicians, health outcomes researchers, payers, or health policy makers may desire PRO data to identify optimal strategies to improve outcomes, whether related to increasing positive health effects in patients or reducing harms and/or negative effects. In this article, we provide a general overview of PROs and published recommendations for their inclusion in clinical Acad Radiol 2014; 21:1129–1137 From the Department of Radiology, University of Washington, 1959 NE Pacific Street, Seattle, WA 98195 (B.W.B., S.D.R.) and Comparative Effectiveness, Cost and Outcomes Research Center, University of Washington, Seattle, Washington (B.W.B., S.D.R.) Received April 1, 2014; accepted May 27, 2014. Funding: No external support was provided for the development of this manuscript. Conflicts of Interest: No direct conflicts of interest are declared by the authors. Address correspondence to: B.W.B. e-mail: [email protected] ªAUR, 2014 http://dx.doi.org/10.1016/j.acra.2014.05.008 1129

Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

  • Upload
    sean-d

  • View
    212

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Including Patient-Reported Outcomesand Patient-Reported Resource-Use

Questionnaires in Studies

Brian W. Bresnahan, PhD, Sean D. Rundell, DPT, PhD

Ac

FrStCoW20thdebr

ªht

Rationale: More efficient and better informed healthcare systems are expected to have improved knowledge of the impact of interventions

on patient outcomes and resources used by patients and providers in specific health conditions.

Objectives: To describe trends related to putting patients at the center of healthcare decision making, regulatory trends and best practicerecommendations for developing high-quality patient-reported outcomes (PROs), and strategic issues related to including PROs in studies.

Materials and Methods: We summarize PRO concepts, definitions, and broadly-accepted scientific standards for developing, assessing,

and interpreting PROs. Three conceptual models are presented as examples for assessing PROs in relation to other outcomes. We discussdifferent perspectives for stakeholders, including regulatory issuespertaining to formal guidance forPROdevelopment and for use in trials.We

provide examples of PROs used in studies for assessing health outcomes in oncology and resource-use outcomes in low back pain patients.

Results: Psychometric scientists working closely withmulti-disciplinary teams and regulatory authorities have greatly improved the science of

collecting, assessing, and understanding patient-reported outcomes in clinical trials. A simplified framework is presented for strategic consid-erations for including PROs in studies, such as the appropriate timing for PROendpoints. Askingpatients about their health status and/or use of

resources improvesour understanding of how interventions andcareprocessesmay impact their lives and their budgets.Weprovide examples

from a back pain trial of patient-reported resource-use questionnaires for medicines taken and other services or products used by patients.

Conclusions: Healthcare stakeholders are placing increased emphasis on resource use and the impact of interventions on patients,

including effects associated with diagnostic tests. Patient-reported outcomes are being used in clinical practice and in clinical research,

supported by formal best-practice guidelines. Radiology has a role as an engaged stakeholder in the design, conduct, and interpretation of

patient-based evidence, and in its relevance to health policy implementation.

Key Words: Patient-reported outcomes; clinical studies; radiology; endpoints; health outcomes.

ªAUR, 2014

Patient-reported outcomes (PROs) provide valuable

health information. Patients, providers, and health

systems are being encouraged and incentivized to

involve patients and their families more directly in health

care decision making. Examples of support for additional ev-

idence development related to patient outcomes in the United

States include the creation of the Patient-Centered Outcomes

Research Institute (PCORI) (1), published priorities in

comparative effectiveness research (CER) (2–4), and the

Agency for Healthcare Research and Quality’s (AHRQ’s)

emphasis on patient-centered outcome research (5). ‘‘Pa-

tient-centered outcomes research (PCOR) helps people and

ad Radiol 2014; 21:1129–1137

om the Department of Radiology, University of Washington, 1959 NE Pacificreet, Seattle, WA 98195 (B.W.B., S.D.R.) and Comparative Effectiveness,st and Outcomes Research Center, University of Washington, Seattle,ashington (B.W.B., S.D.R.) Received April 1, 2014; accepted May 27,14. Funding: No external support was provided for the development ofis manuscript. Conflicts of Interest: No direct conflicts of interest areclared by the authors. Address correspondence to: B.W.B. e-mail:[email protected]

AUR, 2014tp://dx.doi.org/10.1016/j.acra.2014.05.008

their caregivers communicate and make informed health

care decisions, allowing their voices to be heard in assessing

the value of health care options’’ (1). Patient input is being

sought in developing research priorities, informing shared

decision-making models, selecting meaningful outcomes for

clinical research, and disseminating information to patients

and providers. Consequently, PROs and other relevant

patient-reported data have become more ingrained in clinical

research, observational research, clinical care, and quality

improvement initiatives.

In addition to engaging patients in research and in their health

care decision making, PROs are also valuable to more compre-

hensively measure treatment benefits and harms in clinical

research and CER. Manufacturers may choose to include

PROs in clinical trials to assess the impact of interventions on

patients, which has potential use in marketing their products.

Clinicians, health outcomes researchers, payers, or health policy

makers may desire PRO data to identify optimal strategies to

improve outcomes,whether related to increasingpositive health

effects in patients or reducing harms and/or negative effects.

In this article, we provide a general overview of PROs and

published recommendations for their inclusion in clinical

1129

Page 2: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

BRESNAHAN AND RUNDELL Academic Radiology, Vol 21, No 9, September 2014

trials. We review definitions specific to PROs and identify US-

based and international guidance related to regulatory processes

and standards. Specifically, we outline recommendations associ-

atedwith evidence generation using PROs in studies to support

labeling claims (6–8). We use simplified graphics to depict

strategic issues, provide examples of patient-reported diaries

and PRO questions, and list practical aspects of including

PROs and resource-use questionnaires in research.

SCIENTIFIC STANDARDS FOR PROS

General Guidance on Including Patient-BasedOutcomes in Studies

The development of and appropriate use of high-quality

PROs require scientific expertise, time, and substantial

expense associated with questionnaire (instrument) develop-

ment, testing, validation, and refinement. Multiple disciplines

have collaborated in developing strong guidance on health

outcomes measurement, developing PROs and resource-use

instruments, and incorporating PROs in clinical research.

Extensive texts by experts in PRO development, analysis,

and interpretation provide guidance on designing instru-

ments, selecting instruments, and assessing responses (9,10).

These multidisciplinary works have focused on

� Defining health and health domains,

� Measuring physical, social, and psychological well-being,

� Unique issues in assessing mental status, pain, and general

health status and quality of life (QOL),

� Including QOL or other PRO scales in clinical trials,

� Scales, tests, and measures,

� Selecting and administering instruments,

� Analyzing, interpreting, and presenting PRO data,

� Distinct perspectives, including cultural and language is-

sues, and

� Issues and implications in health policy and health economics.

AUS Food andDrugAdministration (FDA)Web site descrip-

tively and graphically outlines their Clinical Outcomes Assess-

ment (COA) Qualification Program, including discussion of

four types of outcomes: PROs, clinician-reported outcomes,

observer-reported outcomes, and performance outcome mea-

sures (11). The educational information outlines the US FDA’s

COA-focused ‘‘wheel’’representing the components of PRO in-

strument development processes. It contains five spokes outlining

steps to follow in evaluating a ‘‘concept of interest for a claim:’’

� Identify content of use and concept of interest,

� Draft instrument and evaluate content validity,

� Cross-sectional evaluation of other measurement properties,

� Longitudinal evaluation of measurement properties/inter-

pretation methods, and

� Modify instrument.

Other resources are available toprovide scientific guidanceon

PRO measurement (12), recommendations for documenting

1130

PRO-based data via PROEvidence Dossiers, eliciting concepts

for PROs, and assessing respondent understanding (13–15).

Definitions and Desired Properties of PROs

More psychometric research has been conducted in the assess-

ment of patient-reported health outcomes, such as in symptom

severity or symptom impact scales and health-related quality-

of-life (HRQOL) measures, compared to research on the

best approaches to collecting resource-use data or financial

end points, such as patient expenditures or their willingness

to pay for specific interventions. Important data may not be

captured in electronic records or may be difficult to assess via

patient observation or physical examination. In the absence

of more objective tests or mechanisms (eg, blood pressure

monitor), we can ask patients how they feel, their levels of

symptoms, and how their symptoms impact their lives, as

well as their general ability to function in their personal lives

or in social settings. For patient-reported data, scientific princi-

ples have centered on using appropriate language, clarity, and

structure for asking questions, the scoring of and evaluation

of responses in instruments, analyzing PROdata in comparative

studies, determining clinical meaning and meaning to the pa-

tient for changes in outcomes, and assessing how representative

or generalizable findings may be for health systems.

Desired properties of patient-reported instruments are

rooted in methods of survey-based research and cognitive psy-

chology. The following are standard definitions for standard

characteristics of well-tested health outcomes measures (8):

� Measurement (psychometric) properties: All the attributes rele-

vant to the application of a PRO instrument including the

content validity, construct validity, reliability, and ability to

detect change (responsiveness).

� Content validity: Evidence from qualitative research demon-

strating that the instrument measures the concept of inter-

est including evidence that the items and domains of an

instrument are appropriate and comprehensive relative to

its intended measurement concept, population, and use.

Testing other measurement properties will not replace or

rectify problems with content validity.

� Construct validity: Evidence that relationships among items,

domains, and concepts conform to a priori hypotheses con-

cerning logical relationships that should exist with other

measures or characteristics of patients and patient groups.

� Reliability: The ability of a PRO instrument to yield consis-

tent reproducible estimates of true treatment effect.

� Ability to detect change (responsiveness): Evidence that a PRO

instrument can identify differences in scores over time in

individuals or groups who have changed with respect to

the measurement concept.

Conceptual Models Providing Context for PROs

We provide three examples of models for conceptualizing

how patient outcomes and PROs may be framed in relation

to other measures/outcomes.

Page 3: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Figure 1. Modified example of a conceptual framework for head

and neck cancer. Adapted from Wilson and Cleary, Journal of the

American Medical Association (1995).

Academic Radiology, Vol 21, No 9, September 2014 INCLUDING PROs IN STUDIES

1) Wilson and Cleary (1995) reported an oft-cited conceptual

model depicting the association of alternative factors or

concepts in influencing health status. They outlined a rela-

tion among biological and physiological factors, symptoms,

function, general health perceptions, and overall QOL (22).

2) WorldHealthOrganization researchers developed the Inter-

nationalClassification of Functioning,Disability andHealth

(ICF), a classification of health and health-related domains.

The ICF tool allows for the explicit influence of environ-

mental factors on health. The ICF conceptual framework

maps the interaction of a health condition (disease or disor-

der); body functions and structures, activities, and participa-

tion; and environmental and personal factors (23).

3) Calvert et al. (2013) developed a Consolidated Standards of

Reporting Trials PRO extension aimed to improve the

quality of PRO reporting in clinical trials and in informing

clinical practice and health policy. Their model outlines

how PROs and epidemiology data from population studies

and clinical trials can be used to inform guideline develop-

ment. More transparent PRO reporting may influence

clinical trial designs andmay allow stakeholders to assess ev-

idence of intervention effects more consistently (24).

Conceptual frameworks can be helpful in studying the rela-

tion among various outcomes (eg, genetic, biologic, clinical,

or diagnostic) and patient symptoms, patient function, and

HRQOL. Figure 1 presents a conceptual scenario for assessing

PROs and outcomes in head and neck cancer, which represents

a simplified adaptation of the model of Wilson and Cleary. For

example, we can study the impact on daily functioning from

head and neck cancer and treatment interventions in relation

to clinician-reported outcomes, physiologic outcomes, fam-

ily/caregiver outcomes, and the impact on PROs.

Multidisciplinary Study Teams

Designing and conducting clinical and observational studies,

including those using PROs, should incorporate input from

multiple stakeholders. It is recommended that effective study

teams be comprised interdisciplinary researchers, as well as

patients, their advocates, and payers or health technology

assessment experts when appropriate. Study teams, particu-

larly those within medical product companies, often include

statisticians, clinicians, regulatory experts, health services

researchers, clinical research coordinators, international

health systems experts for global trials, data coordinators, pro-

grammers, marketing representatives, and other relevant

personnel. Academic teams would likely include the same sci-

entific members but may not include marketing or regulatory

representatives if not studying a product for review and mar-

keting approval. Public relations and communications experts

are helpful when engaging in promotional activities or media

communication related to health research.

Regulatory and Research Guidance for Including PROsin Trials

For more than 20 years, teams of cognitive psychologists, (bio)

statisticians, psychometricians, health outcome researchers, ep-

idemiologists, health service researchers, clinicians, economists,

regulatory agencies, and other stakeholders have partnered to

recommend the best practices for designing patient-based

instruments and incorporatingPROs in clinical research (eg, in-

strument psychometrics, collection, analysis, and interpreta-

tion) (6–10). In 2009, the US FDA finalized a guidance

document for industry on studying PROs in clinical trials to

provide a framework for developing end point models related

to the treatment of disease and the impact on symptoms, with

recommendations for primary and secondary end points. The

report covers evaluations of PRO instruments (ie, properties),

clinical trial designs and using PROs, and recommended data

analysis approaches. A major investment in PROs by the

US National Institutes of Health led to the innovative

Patient-Reported Outcomes Measurement Information

Systems (PROMIS) tools for assessing health status. Advanced

measurement science approaches andmultidisciplinary collabo-

ration led to extensive research on instruments, measurement,

interpretation, along with tools, recommendations, and

available instruments for assessing physical, mental, and social

well-being in studies (16,17).

Resource-Use questionnaires completed by patients. Patients’resource-use data are a key component of comprehensive

cost-effectiveness assessment (CEA). A 1993 US Panel on

Cost-Effectiveness in Medicine (18) provided recommenda-

tions related to conducting health economic evaluations,

followed by an International Society of Pharmaceutical Out-

comesResearch task force report on ‘‘good research practices’’

for conducting CEA in conjunction with clinical studies (19).

Cost-effectiveness data may or may not use data collected

directly from patients, but specific health conditions may war-

rant taking steps to collect patient-reported resource use.

The prospective use of cost-utility assessment (CUA)

generally requires PRO data collection and/or preference

elicitation. Responses to health outcome measures coordinate

1131

Page 4: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

BRESNAHAN AND RUNDELL Academic Radiology, Vol 21, No 9, September 2014

with ‘‘utility weights’’ corresponding to alternative health

states related to a condition or its treatments (eg, improved

function or health detriment). Retrospective or prospective

CEAs/CUAs may use published utility weights that were pre-

viously collected on a specific target group of patients, derived

using patient-reported (or community-reported) approaches.

Generalizing or applying weights to specific settings or pe-

riods may be a challenge. Several textbooks discuss methods

and standards for economic evaluation and CEA to improve

health care and decision making in medicine (20,21).

Use or cost data may be obtained from IT systems (eg, elec-

tronic medical record and/or claims data) of participating

study sites or health systems. Specific types of resource use,

however, may require patient-reported input. Examples

include time spent on care or attending provider visits, per-

sonal expenditures on condition-related products for the

home, filled prescriptions, purchase of over-the-counter

(OTC) medications, or actual prescription and OTCmedica-

tions taken, including dose. Copayment or coinsurance infor-

mation may be substantial and worth collecting for distinct

medical interventions, such as oncology medications/thera-

pies, arthritis medications, magnetic resonance imaging or

molecular imaging procedures, or surgeries.

STRATEGIC DEVELOPMENT OF PATIENT-REPORTED EVIDENCE

Considering PRO Impact Early in Product Life Cyclesfor Medicines and Procedures

Gondek et al. provide examples of how PROs may be consid-

ered for inclusion in oncology trials during a product’s life cy-

cle, focusing on industry-sponsored trials (25). Many

principles of strategic product development have relevance

to medical products in general. For medicines or procedures,

phase 1 studies may assess estimated ranges of ‘‘target product

profiles’’ and potential PRO opportunities. Target product

profiles include expected positive and negative effects and po-

tential ranges of magnitudes of effects for interventions (eg, ef-

ficacy, safety, administration). Target profiles can be used to

estimate risk–benefit balance or other clinical or economic

metrics (26). These profiles can subsequently be refined and

assessed in phase 2 clinical studies, which are increasingly

testing PRO instruments (eg, symptoms, HRQOL, other)

in association with alternative doses, regimens, or procedural

techniques, and in correlation with other study end points.

Larger phase 3 studies should include the final instruments

and cultural/language translations based on the regulatory or

market strategy. Organizations are using portfolio planning ap-

proaches to improve decisionmaking for multiproduct research

and development pipelines. This process includes developing

strong evidence profiles for products with good differentiation

opportunities and terminating development earlier for products

with lower probabilities of technical success or market success.

PRO collection in clinical development programs may inform

1132

the ‘‘evidence upside’’ for products targeting competitive mar-

ketplaces (ie, supporting a differentiation strategy) and inform

the ‘‘evidence downside’’ for products with poor safety profiles,

dose uncertainty, or a lack of effect on PROs that are a compet-

itive standard for specific conditions. Evidence standards for

assessing quantitative benefit–risk balance for medical products

are likely to increasingly become linked to patient-reported

general health status measures, condition-specific PROs, and

patient satisfaction metrics.

Products in competitive markets can use high-quality PRO

evidence to enhance value propositions. PRO strategies may

differentiate devices or medicines or they may be needed to

meet marketplace evidence standards. Needs for testing the

impact of medical devices on patient outcomes will continue

to increase, particularly for devices impacting patients’ symp-

toms and ability to function. Knee replacement and hip

replacement products, for example, are being advertised

directly to consumers. They will need to demonstrate value

to payers and patients through evidence of impact on PROs,

including productivity, time to return towork, ability to func-

tion, and HRQOL.

Timing of PRO Inclusion in Studies

PROs should be assessed in association with other study end

points and demographic characteristics. With appropriate po-

wer, subgroups, health system characteristics, and/or clinical

care metrics may also be evaluated in relation to PROs. Link-

ing the expected timing of clinical effects of interventions and

how they may impact symptoms, function, or HRQOL will

help to determine a PRO inclusion strategy.

Researchers want to select the best available measure and

determine when to administer PROs during a study. Psychol-

ogists, biostatisticians, and health services researchers are

essential participants in making these decisions. The first

step is to determine an appropriate research question, and

how the research objectives align with a study’s general out-

comes assessment plan and the PRO collection strategy. Hol-

lingworth and Jarvik (27) describe and depict how standard

scientific methods (ask question, design study, obtain data,

assess, modify) are associated with evidence-based radiology

and the health technology assessment decision hierarchy,

including levels assessing cost effectiveness and patient

outcomes.

Aligning the timing and responsiveness of a PRO to the

timing of clinical outcomes based on a treatment, procedure,

test, or device is challenging. Preliminary (eg, phase 2) data

are helpful in developing a comprehensive PRO strategy. In-

terventions may have shorter term effects on patients,

requiring short-term PRO collection, while others may

have a delayed impact subsequent to a treatment or procedure,

thus requiring delayed timing for PRO end points. Figure 2

presents a simplified high-level perspective on connecting ex-

pected clinical effects of interventions in trials with PROmea-

surement qualities.

Page 5: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Figure 2. Simplified example of patient-

reported outcomes (PRO) timing in relationto timing of clinical effect from intervention.

Academic Radiology, Vol 21, No 9, September 2014 INCLUDING PROs IN STUDIES

Additional Practical Issues Related to Including PROsin Trials

Researchers should

� Link records with electronic medical records and other IT

system (ie, evolve from paper-based systems) to evaluate

clinical measures, demographics, PROs, resource-use

questionnaires, claims-based resource use, systems data.

� Use well-tested, psychometrically appropriate instruments,

particularly when desiring to publish the evidence and/or

use evidence for marketing purposes.

� Include an instructions page for respondents, outlining ex-

pectations for them in completing the forms.

� Pilot test instruments and batteries of multiple instruments

to determine the expected completion time needed and to

identify challenges patients may have in completing

questionnaires.

� Include PRO and resource-use questionnaire in the overall

clinical trial or observational study schedule of events (eg,

baseline and other time points).

� Include PRO training for investigators, clinical research

coordinators, and data coordinators as part of clinical trial

investigator meetings. PRO data quality standards are no

different than those for other clinical endpoints, including

a goal to minimize missing data.

B Many PROs ask about symptoms or function within

the past week or month, or in relation to the day of

completion or receiving an intervention. There is usu-

ally a finite window after events in which patients are

more likely to have more accurate recall.

B More intensive patient-reported diaries, consisting of

resource-use questions that might not be available

except for direct patient completion, may ask patients

to complete daily information (eg, medication dose)

or several times a week for a specified short period.

� Use prestudies or smaller studies to test PRO and utiliza-

tion questionnaires in the target population (eg, elderly)

and expect an iterative process until the research team is

comfortable with the forms/platform and the strategy to

prioritize, collect, and assess PROs.

EXAMPLES FROM CLINICAL STUDIES

Numerous cases of using PROs and resource-use diaries are

available in published literature. We provide examples of

PROs used in back pain and oncology studies. Out-of-

pocket expenses, time spent on care, and utilization not

captured by electronic health records are important outcomes

that need to be collected directly from the patient. Low back

pain and leg pain in older adults can produce significant

disability and a decreased quality of life, which places a large

burden on patients, providers, payers, and health systems.

Oncology conditions impact physical, mental, and emotional

well-being and can have substantial financial implications on

patients. Patient-reported resource-use data can complement

other health record information to provide a more compre-

hensive perspective for economic analyses.

PROs are the primary method to assess the concept of in-

terest in back pain studies, that is, pain in patients (levels, types,

and impact on function) (28). Two widely used back pain–

specific instruments are the Roland Morris Disability Ques-

tionnaire (RMDQ) (29,30) and the Oswestry Disability

Index (31). Visual analog scales (VAS) and numeric rating

scales (NRS) are also commonly used in pain studies,

including those studying back pain.

Graves et al. (32) used the RMDQ and other PROs to

examine associations in receiving early magnetic resonance

11

33
Page 6: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Figure 3. Simplified example of patient-reported medication resource-use questionnaires.

BRESNAHAN AND RUNDELL Academic Radiology, Vol 21, No 9, September 2014

imaging and low back pain outcomes inWashington State La-

bor and Industry (WA L&I) claimants. They used a

population-based prospective cohort design to assess health

outcomes and disability status 1 year after an injury. In the

Lumbar Epidural Steroid Injections for Spinal Stenosis

(LESS) double-blind, randomized, controlled trial, patients

with spinal stenosis were randomized to receive an epidural

steroid injection with a local anesthetic or a local anesthetic

alone. Patients received up to two injections during the first

6 weeks of their randomized intervention and subsequently

had the option to crossover to the other intervention arm.

Outcomes included the RMDQ, back and leg pain NRS

measures, and other PROs, collected at baseline, 3 weeks, 6

weeks, 3 months, 6 months, and 12 months after enrollment

(33). In LESS, the PRO collection strategy aligned with the

expected timing of clinical effects from interventions. More

intensive PRO and resource-use data collection was used in

the first 6 weeks, consistent with the primary end point

timing. In addition to ‘‘in-system’’ claims-based resource use

captured in IT systems of participating sites, the LESS study

captured ‘‘out-of-system’’ use and medication use directly

from patient-completed resource-use questionnaires. Exam-

ples of these resource-use questionnaires are displayed below

for medication use and for provider visits (Figs 3 and 4).

1134

PROs are alsowidely used in oncology studies and extensive

research has produced guidance on collecting, analyzing, and

interpreting PROs. Researchers have evaluated PROs used

in oncology trials, provided recommendations for improve-

ments, and highlighted challenges and issues with integrating

PROs in National Cancer Institute–sponsored oncology trials

conducted in clinical trial networks (34–36). Differences in

general approach at sites or in systems for PRO integration

and variance in standards for PRO data quality assurance

were identified as challenges to overcome. Osoba (37) con-

ducted a review of HRQOL and PROs used in cancer clinical

trials from 2000 to 2010, assessing 118 studies in breast, lung,

prostate, colorectal, and other cancers for the use of PROs,

findings, methodological issues, predictive/prognostic poten-

tial, and integration with biomedical end points. An extensive

1999 report by theUnited Kingdom’sNationalHealth Service

Health Technology Assessment (HTA) Research and Devel-

opment Programme reviewed and evaluated methods for the

assessment of HRQOL and survival data from studies (38).

The HTA report highlighted methods and challenges for

assessing survival time and PROs, given incomplete data or

missing data due to death, protocol violations, or study

dropout due to other reasons is a major issue in oncology

studies. In 2012, the Center for Medical Technology Policy

Page 7: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Figure 4. Simplified example of patient-

reported treatment and service resource-use questionnaires.

Figure 5. Oncology patient-reported out-

comes example: Functional Assessment of

Cancer Therapy–General 7 (version 4) ques-tionnaire (40,41); FACT-G7 (version 4) re-

printed with permission from developers at

FACIT.org.

Academic Radiology, Vol 21, No 9, September 2014 INCLUDING PROs IN STUDIES

released an Effectiveness Guidance Document for incorpo-

rating PROs in CER in adult oncology (39). Figures 5 and 6

present simplified examples of cancer-related PRO scales

used in clinical trials and/or clinical care.

CONCLUSIONS

We are in a new era of health care, with greater focus on pa-

tients, the outcomes they experience, and the resource and

financial burden on patients, families, payers, and health sys-

tems. However, patient-related terminology may not be

used in a consistent manner, and alternative stakeholders

may use similar terms quite differently. As discussed, PROs

used in clinical research often refer to specific, psychometri-

cally appropriate patient-reported measures collected at spe-

cific time points in a clinical trial to assess the impact of

experimental interventions on patients. Alternatively, in clin-

ical care settings, patient-centered concepts deemed impor-

tant to patients as part of their care processes, such as

symptom improvement, communication with providers, or

adequate access to interventions may not always be assessed

systematically, comparatively, or in a manner consistent with

trial-based approaches. In addition, the collection of PROs

in trials or in practice does not guarantee that the outcomes

reported are the most informative or useful to patients or

other stakeholders. Research organizations and/or health

1135

Page 8: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Figure 6. Adapted example of a cancer-

related patient-reported outcomes (PRO)

nausea scale used by the University of Ro-chester Cancer Center Research Base (34)

Printed with permission.

BRESNAHAN AND RUNDELL Academic Radiology, Vol 21, No 9, September 2014

policy setters, including AHRQ, PCORI, payers, manufac-

turers, or health systems, may have distinct needs or mandates

for patient-based information, and their objectives for

wanting patient-reported evidence may not be aligned.

Although not without similar challenges, patient-reported

resource-use questionnaires provide an excellent option to

collect additional patient-based outcomes in clinical research,

which can complement claims-based or systems-based

resource-use evaluation. For studies examining pain, other

debilitating conditions, or interventions impacting how pa-

tients feel and function, PROs may be the best source of

data or the only appropriate method of obtaining relevant

data on symptoms, HRQOL, or other outcomes important

to patients.

From a radiology perspective, as pointed out by Carlos et al.

(42), the imaging community should be actively engaged

stakeholders in designing, conducting, and disseminating

patient-centered research. Multidisciplinary collaborations

can be helpful in investigating and appropriately accentuating

those uses of imaging that add substantial value to patients and

health systems, as well as identifying imaging applications that

may be less appropriate in specific patient populations.

In this manuscript, we have given examples of studies on

patients with pain conditions that have complex interactions

among pain, function, medication use, HRQOL, and eco-

nomic outcomes. A single PRO measure likely will not cap-

ture a comprehensive view of all pain-related outcomes.

General health measures and condition-specific instruments

are needed. Likewise, patients with cancer may experience

adverse effects associated with treatments, and/or improve-

ments in energy or functional ability, along with other health

effects. PRO data should be assessed in conjunction with sur-

vival data, tumor response, or other biomarker information

for patient subgroups and alternative treatment strategies.

One limitation may be additional time and/or expense associ-

ated with collecting PRO data. From a practical perspective,

potentially needing patient consent (with documentation)

1136

for PRO collection adds challenges for lower cost pragmatic

trials. Institutional review boards may require that translations

be made available for non–English-speaking patients.

Patient-reported resource-use questionnaires provide more

comprehensive profiles of consequences associated with alter-

native interventions under study, including health impact and

financial implications. Electronic medical records and claims

data are powerful tools to assess clinical and economic

resource use, but they generally do not include patients’

out-of-pocket expenditures or the amount of time spent

engaging health systems or caring for patients or family mem-

bers. Patients, clinicians, researchers, and other stakeholders

can use outcome information from studies to make more

informed health care decisions, as well as to inform guidelines

and/or health policy.

ACKNOWLEDGMENTS

The authors would like to acknowledge Dr. David Cella for

providing permission to use the FACT-G7 (version 4)as an

oncology example of a patient-reported outcome (PRO),

Dr. Jeffrey Sloan for permission to adapt the nausea scale

developed and used by the University of Rochester Cancer

Center Research Base as an example of a PRO used in cancer

care and trials, Dr. Jarvik and Dr. Friedly of the Back pain

Outcomes using Longitudinal Data (BOLD) Project for

permission to use the back pain PRO examples, and Mr.

Daniel Willems and Mr. Marc Schiermeier for assistance

with the manuscript figures.

REFERENCES

1. http://www.pcori.org (accessed March 17, 2014).

2. Institute of Medicine, http://www.nap.edu/openbook.php?record_

id=12648 (accessed March 21, 2014).

3. Inglehart JK. Prioritizing comparative-effectiveness research—IOM rec-

ommendations. NEJM 2009; 361(4):325–328.

Page 9: Including Patient-Reported Outcomes and Patient-Reported Resource-Use Questionnaires in Studies

Academic Radiology, Vol 21, No 9, September 2014 INCLUDING PROs IN STUDIES

4. Pandharipande PV, Gazelle GS. Comparative effectiveness research:

what it means for radiology. Radiology 2009; 253(3):600–605. http:

//dx.doi.org/10.1148/radiol.2533091286.

5. http://www.ahrq.gov/(accessed March 17, 2014).

6. Chassany O, Sagnier P, Marquis P, et al. Patient-reported outcomes: the

example of health-related quality of life—a European guidance document

for the improved integration of health-related quality of life assessment in

the drug regulation process. DIJ 2002; 36:209–238.

7. http://www.ema.europa.eu/docs/en_GB/document_library/Scientific_guideline/

2009/09/WC500003637.pdf (accesses March 12, 2014).

8. United States Health and Human Services, Food and Drug Administration,

guidance for industry, patient-reported outcomemeasures: use in medical

product development to support labeling claims, December 2009. (http://

www.fda.gov/MedicalDevices/DeviceRegulationandGuidance/Guidance

Documents/default.htm, accessed March 12, 2014).

9. Spilker Bert, Cramer Joyce, eds. Quality of life & pharmacoeconomics: an

introduction. Philadelphia: Lippincott-Raven, 1998.

10. McDowell I, Newell C. Measuring health: a guide to rating scales and ques-

tionnaires. 2nd ed. New York: Oxford Press, 1996.

11. http://www.fda.gov/Drugs/DevelopmentApprovalProcess/DrugDevelopment

ToolsQualificationProgram/ucm284077.htm (accessed March 18, 2014).

12. http://phi.uhce.ox.ac.uk/home.php, (accessed March 17, 2014).

13. Revicki DA, Gnanasakthy A, Weinfurt K. Documenting the rationale and

psychometric characteristics of patient reported outcomes for labeling

and promotional claims: the PRO Evidence Dossier. Qual Life Res 2007;

16:717–723.

14. Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity—Establishing

and reporting the evidence in newly-developed patient-reported out-

comes (PRO) instruments for medical product evaluation: ISPOR PRO

good research practices task force report: Part 1-Eliciting concepts for a

new PRO instrument. Value Health 2011; 14:967–977 [Epub October 13,

2011].

15. Patrick DL, Burke LB, Gwaltney CJ, et al. Content validity—Establishing

and reporting the evidence in newly-developed patient-reported out-

comes (PRO) instruments for medical product evaluation: ISPOR PRO

good research practices task force report: Part 2 – assessing respondent

understanding. Value Health 2011; 14:978–988 [Epub October 13, 2011].

16. http://www.nihpromis.org/about/overview (accessed March 13, 2014).

17. http://www.assessmentcenter.net/documents/InstrumentLibrary.pdf, ac-

cessed March 21, 2014).

18. Gold MR, Siegel JE, Russell LB, et al. Cost-effectiveness in health and

medicine. 1996.

19. Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-

effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA Task

Force report. Value in Health 2005; 8(5):521–533.

20. Neumann PJ. Using cost-effectiveness analysis to improve health care:

opportunities and barriers. Oxford: Oxford University Press, 2004.

21. Drummond MF, O’Brien B, Stoddart GL, et al. Methods for the economic

evaluation of health care programmes. 3rd ed. Oxford: Oxford University

Press, 2005.

22. Wilson IB, Cleary BD. Linking clinical variables with health-related quality

of life. A conceptual model of patient outcomes. JAMA 1995; 273(1):

59–65.

23. http://www.who.int/classifications/icf/en/; accessed on March 14, 2014.

24. Calvert M, Brundage M, Jacobsen PB, et al. The CONSORT patient-

reported outcome (PRO) extension: implications for clinical trials and prac-

tice. Health and Quality of Life Outcomes 2013; 11:184–190.

25. Gondek K, Sagnier PP, Gilchrist K, et al. Current status of patient-reported

outcomes in industry-sponsored oncology clinical trials and product la-

bels. J Clin Oncol 2007; 25:5087–5093.

26. Garrison LP, Jr, Bauch CT, Bresnahan BW, et al. Using cost-effectiveness

analysis to support R&D portfolio prioritization for product innovations in

measles vaccination. J Infectious Diseases 2011; 204(Suppl 1):S124–S132.

27. Hollingworth W, Jarvik JG. Technology assessment in radiology: putting

the evidence in evidence-based radiology. Radiology 2007; 244(1):31–38.

28. Smeets R, Koke A, Lin C-W, et al. Measures of function in low back pain/

disorders: Low Back Pain Rating Scale (LBPRS), Oswestry Disability Index

(ODI), Progressive Isoinertial Lifting Evaluation (PILE), Quebec Back Pain

Disability Scale (QBPDS), and Roland-Morris Disability Questionnaire

(RDQ). Arthritis Care & Research 2011 Nov; 63(S11):S158–S173.

29. Roland M, Morris R. A study of the natural history of back pain. Part I:

development of a reliable and sensitive measure of disability in low-back

pain. Spine (Phila Pa 1976) 1983; 8(2):141–144.

30. RolandM,FairbankJ.TheRoland-MorrisDisabilityQuestionnaireand theOs-

westry Disability Questionnaire. Spine (Phila Pa 1976) 2000; 25:3115–3124.

31. Fairbank JC, Couper J, Davies JB, et al. The Oswestry Low Back Pain

Disability Questionnaire. Physiotherapy 1980; 66:271–273.

32. Graves JM, Fulton-Kehoe D, Jarvik JG, et al. Early imaging for acute low

back pain one-year health and disability outcomes among Washington

State workers. Spine 2012; 37(18):1617–1627.

33. Friedly JL, Bresnahan BW, Comstock B, et al. Study protocol–Lumbar

Epidural Steroid Injections for Spinal Stenosis (LESS): a double-blind ran-

domized controlled trial of epidural steroid injections for lumbar spinal ste-

nosis among older adults. BioMed Central Musculoskeletal Disorders

2012; 13:48.

34. Sloan JA, Berk L, Roscoe J, et al. Integrating patient-reported outcomes

into cancer symptommanagement clinical trials supported by the National

Cancer Institute–sponsored clinical trials networks. J Clin Oncol 2007;

25(32):5070–5077.

35. Brunner DW, Bryan CJ, Aaronson N, et al. Issues and challenges with inte-

grating patient-reported outcomes in clinical trials supported by the Na-

tional Cancer Institute–sponsored clinical trials networks. J Clin Oncol

2007; 25(32):5051–5057.

36. Minasian LM,O’Mara AM, Reeve BB, et al. Health-related quality of life and

symptom management research sponsored by the National Cancer Insti-

tute. J Clin Oncol 2007; 25:5128–5132.

37. Osoba D. Health-related quality of life and cancer clinical trials. Therapeu-

tic Advances in Medical Oncology 2011; 3(2):57–71.

38. Billingham LJ, Abrams KR, Jones DR. Methods for the analysis of quality-

of-life and survival data in health technology assessment. Health Technol

Assess 1999; 3(10).

39. Center for Medical Technology Policy, Recommendations for incorporating

patient-reported outcomes (PROs) into clinical comparative effectiveness

research (CER) in adult oncology, effectiveness guidance document, Pub-

lished Version 1.0, May 20, 2012. (http://www.cmtpnet.org/resource-

center/view/egd-on-patient-reported-outcomes-in-cer-and-adult-oncology/

; accessed March 17, 2014).

40. Yanez B, Pearman T, Lis CG, et al. The FACT-G7: a rapid version of the

functional assessment of cancer therapy-general (FACT-G) for monitoring

symptoms and concerns in oncology practice and research. Annals of

Oncology 2013; 24:1073–1078.

41. http://www.facit.org/FACITOrg/Questionnaires (accessedMarch 21, 2014).

42. Carlos R, Buist DS, Wernli KJ, et al. Patient centered outcomes in imaging:

quantifying value. JACR 2012; 9(10):725–728.

1137