Measurement in Clinical Trials - An Examination of the FDA Position on Review of Clinical Outcome Assessments (COAs)

Measurement in Clinical Trials

An Examination of the FDA Position on Review of

Clinical Outcome Assessments (COAs)

12 September 2013

This is a Solution Provider Webinar brought to you by DIA in cooperation with ICON plc.

The views and opinions expressed in the following PowerPoint slides are those of the individual presenter and should not be attributed to Drug Information Association, Inc. (“DIA”), its directors, officers, employees, volunteers, members, chapters, councils, or Special Interest Area Communities or affiliates.

These PowerPoint slides are the intellectual property of the individual presenter and are protected under the copyright laws of the United States of America and other countries. Used by permission. All rights reserved. Drug Information Association, DIA and DIA logo are registered trademarks or trademarks of Drug Information Association Inc. All other trademarks are the property of their respective owners.

Disclaimer

2

• ICON Late Phase and Outcomes Research patient reported outcome (PRO) services provide consultancy and high quality research based on opinion-leading scientific expertise.

• Our PRO team has a network of international opinion leaders, regulatory experts, and methodological and clinical experts, including collaborative relationships with a number of key researchers at: University of Oxford, University of Sheffield, University of Washington, and the University of York.

• Members of our team also make contributions to professional debate on a number of fronts through participation at CPath, ISPOR, ISOQOL, DIA, and the EuroQol Group and have worked to provide guidance to NICE.

ICON Late Phase and Outcomes Research

Drug Information Association www.diahome.org 3

Agenda

Emuella Flood, Director, ICON Late Phase and Outcomes Research• Classifications and definitions of COAs• FDA evidence requirements for COAs• ISPOR ClinROs Good Measurement Practices Task Force

Cicely Kerr, Senior Outcomes Researcher, ICON Late Phase and Outcomes Research• Challenges and experiences of evaluating non-PRO COAs to PRO

standards

April Naegeli, Research Scientist, Global Health Outcomes, Eli Lilly and Company• Industry perspective on the FDA position on review of COAs


• Clinical Outcome Assessments (COAs)– Purpose and classification

• FDA Evidence Requirements for COAs

• ISPOR Task Force on Clinician-reported Outcomes (ClinROs)

Overview


Purpose of Outcome Assessments

• To provide evidence of a specific treatment benefit

• Treatment benefit = favorable effect on a meaningful aspect of how a patient: Feels Functions Survives

• Usually effectiveness benefit, but may be a safety benefit

• Regulatory context• Product approval• Labelling claims

Marc Walton, Clinical Trials Assessment Workshop, October 2011Drug Information Association www.diahome.org 6

Feels and Functions as Effectiveness

• Feels A patient’s physical sensation or perceived mental state related

to health within typical ‘daily’ life Pain Severely low mood (depression)

• Functions Patient’s ability to perform an activity that is meaningful part of a

typical ‘daily’ life Not ability to perform actions not part of usual life

Marc Walton, Clinical Trials Assessment Workshop, October 2011Drug Information Association www.diahome.org 7

Classification of Outcome Assessments in Clinical Studies

• Patient-reported Outcomes (PROs)

• Observer-reported Outcomes (ObsROs)

• Clinician-reported Outcomes (ClinROs)

• Biomarkers

• Survival

Clinical Outcome

Assessments (COAs)

Outcome Assessments

but NOTCOAs


Patient-reported Outcome (PRO)

Any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response

by a clinician or anyone else.

FDA Guidance, 2009

Direct Indirect

Pain severity NRS Pain medication use


FDA Workshop:Observer-reported Outcome (ObsRO)

An assessment that is determined by an observer who does not have professional training that is relevant to the measurement being made, i.e., a non-clinical observer such as a teacher or caregiver. This type of assessment is often used when the patient is unable to self-report (e.g., young children, cognitively impaired). An ObsROshould only be used in the reporting of observable events (e.g. signs or behaviors); ObsROs cannot be validly used to assess symptoms (e.g., pain) or other unobservable concepts.

Direct Indirect

Observed seizures Observed infantbehavior


FDA Workshop:Clinician-reported Outcome (ClinRO)

An assessment that is determined by an observer with some recognized professional training (e.g., physician, nurse, physical therapist, etc.) that is relevant to the measurement being made; training necessary in carrying out assessment or interpreting results

Direct Indirect

PANSS Joint flexion spasticity6‐min walk test


FDAs Regulations for Medical Product Approval

The effectiveness of a treatment must be based on substantial evidence including evidence that all assessments of treatment benefit

are well defined and reliable

21 Code of Federal Regulations 314.125(b)(5) and 314.126 (b) (6)


FDA Requirements for PRO Endpoints

• Guidance on how FDA evaluates PRO endpoints

• Measuring and characterising benefit of medical product treatment as perceived by the patient

• To support labeling claims


PROs to Support Labeling Claims

PROs are well developed and adequately validated to measure what they are intended to measure in the specific context of use represented

by the clinical study design

FDA Guidance, 2009


PRO Development to Support Label Claim

• Define context of use

• Define concept of interest

• Demonstrate content validity– Qualitative research with patients– Literature review– KOL/clinician input

• Documentation of other psychometric properties– Reliability– Validity– Sensitivity to change

• Interpretation of scores– Individual patient PRO score change over a predetermined time period that should be

interpreted as treatment benefit– a priori, determined empirically


Unique Aspects of ClinROs

• Different types

– Ratings (PANSS)– Readings (spleen size, swollen joints)– Performance-based (6-min walk test, FEV1)– Composite (Mayo)

• Challenge of establishing content validity

– Best approach?– Whose perspective?

• Importance of standardized instructions for performing assessment, scoring or interpreting results

• Inter-rater reliability


• Clinician-Reported Outcomes Good Measurement Practices Task force– Chair: John H. Powers, III, MD FACP FIDSA

– Content: • define which outcome assessments should be considered ClinROs• develop recommendations for the evaluation of ClinROs that support claim of treatment

benefit. • identified good measurement principles• provide practical recommendations to develop and evaluate these outcome measures

• Aim to submit report to Value in Health by the end of 2013

ISPOR Task Force on ClinROs


Evaluating non-PRO COAs to PRO Standards: Taking ClinROs as an example

• General issues with applying FDA PRO guidance requirements to ClinROs

• Issues for existing instruments– Review of published evidence for two existing ClinROs

• Additional issues for new instrument development

• Challenges in validation of ClinROs/ObsROs

Type of ClinRO Condition Concept


PROs ClinROs

Conceptual framework and scoring rationale/validity

Published diagram linking of items to concepts/scoring domains

Relevance for certain types of ClinROs e.g PBMs, Readings? What constitutes acceptable rationale?

Content validity Patient input into concept definition and item generation (qualitative concept elicitation interviews), cognitive debriefing with patients

Whose input is important? How should this be achieved? Evidence requirements for appropriateness of ClinRO vs ObsRO vs PRO for measurement concept?

Construct validity Convergent, known groups What are appropriate comparator COAs for convergent validity? Relevance of predictive validity?

Reliability Internal consistency, test‐retest Inter‐rater reliability in place of internal consistency, intra‐rater =test‐retest reliability but specific methodological guidance lacking

Ability to detect change Within person changes over time in those known to have changed in respect to concept

Clinician/clinically assessed vs patient‐reported change?

Responder definition Change in individual score that indicates a treatment benefit. Derived using anchor‐based methods

Clinician/clinical anchors vs patient anchors for change?

Training manual Required Even more important for ClinROs than PROs?

Translation and linguistic validation; Electronic migration and equivalence

Required to specified standards Not necessarily common current practice for ClinROs

General issues with applying FDA PRO evidence requirements to ClinROs


Issues for existing instruments: 6MWT, EDSS

Specific issues from reviewing published evidence for existing ClinROs1. 6 minute walk test (6MWT)

– PBM– Measures the distance that a patient can walk on a flat, hard

surface in a period of 6 minutes– Used in various rehabilitation contexts for assessment of

mobility, gait, aerobic capacity/endurance

2. Expanded Disability Status Scale (EDSS)– ClinRO requiring clinical judgement– Widely used measure for clinical evaluation of multiple

sclerosis (MS) patients and clinical trial endpoint – Quantify and monitor change in disability in MS – Clinicians rate impairment in eight functional systems

according to their observations using specific coding for each to give 8 functional system (FS) and guide overall EDSS score


Example EDSS FS


EDSS overall score


6MWT EDSS

Conceptual framework and scoring rationale

None published – is this even relevant?

None published ‐ Focuses on mobility over other FS in scoring overall EDSS without specific rationale

Content validity No clear instrument developer or published information on development

No published information• Clinician co‐authors on publications

without description of any role in instrument development

• Relevance of patient input?

Publishedevidence of construct validity and reliability

• Sparse: Validity and reliabilityoften assumed rather than tested

• Assessment methods and patient instructions vary widely or are not described

• Some published evidence of convergent/divergent validity unclear if sufficient

• Inter‐rater reliability above threshold for group but not individual comparison studies, lower for FS scores than overall EDSS

• Concern that clinicians may have developed their own short‐cuts in administration

Gaps in published evidence for 6MWT, EDSS


Specific issues for existing ClinROs

• May be little published evidence to draw on– In particular, likely lack of published evidence from original

development work• What evidence there is may indicate poor performance • Challenge of addressing evidence gaps and improving performance

– Potentially evaluating familiar, standard instruments for clinical practice

• To what extent can instrument content or scoring be revised?• Clinician familiarity and experience may have implications for

evaluation


• How to assess the need for development of a new ClinRO/ObsRO• How to build evidence of content validity during instrument

development– Relevance and importance of patient input into this evaluation?

• May depend on concept, condition, ClinRO content or administration (e.g. level/type of clinician interaction with patient for completion)

• Resolving conflicting views – which holds greater weight?• Potential for development of composite or multiple

PRO/ClinRO/ObsRO measurement of same concept– May be relevant for conditions associated with varied/diminishing

potential for patient-report– Rationale for scoring e.g. weighting

Additional issues for new instrument development


• Validation sample requirements– Sample size and variation in

• Clinicians/observers• Patients assessed

• Methods– In-person patient assessment vs rating of example video

consultation or standardised patient scenario– May depend on concept, condition, instrument content or

complexity of administration/rating– Implications

• Measurement• Logistical• Ethical

Challenges in validation of ClinROs/ObsROs


• April N. Naegeli, DrPH, MPH is a fulltime employee and holds stocks and/or stock options with Eli Lilly and Company.

• The views and opinions expressed in the following PowerPoint slides are those of the individual presenter and should not be attributed to Eli Lilly and Company.

Disclosure

27

• Drug development & external environment current state of affairs

• Lessons learned from PRO instrument development

• Anticipated Requisites for advancing review & development of COAs

• Expectations for advancing review & development of COAs

Overview

28

• COAs – Commonly used as efficacy endpoints – Measure important aspects of treatment benefit – Basis of medical product approval & labeling claims

• FDA – Critical Path Initiative

• Adequacy of development & testing of existing instruments– Uncertainty related to what is actually being measured

• Industry– Efforts to cut drug development costs & timelines to increase

productivity & efficiency

Drug Development – Current State of Affairs


• Patient focus• Translation into clinical practice• Comparative Effectiveness Research• Health Technology Assessment

External Environment – Current State of Affairs


• Pre-competitive forums– Example: PRO Consortium

• Industry development for specific molecule

Pathways for COA Review & Development


Advantages• Improve adequate

product evaluation

• Well-defined & reliable outcomes

• Patient focus

Challenges• Comparisons with

existing treatment

• Lack of clear regulatory guidance

• Impact to drug development programs unclear

Advantages/Challenges for Review & Development of COAs


• Inconsistency in interpretation & application of guidance within FDA

• Evolving regulatory expectations• Lengthy & resource intensive process• Consortium best forum for engaging multiple

stakeholders

Lessons Learned from Review & Development of PROs


• Stakeholder involvement• Timing for review & development• Communication

Anticipated Requisites for Advancing Review & Development of COAs


• Regulatory agencies• Industry

– COA/PRO• Outcomes researchers

– Non-PRO COAs• Medical/safety representatives

• Medical community/academia• Professional organizations• Patients• Advocacy groups

Stakeholder Involvement


• Involvement of regulatory bodies?– Identify in Phase I – EOP2 too late

• Timelines for sharing evidence?

Timing for Review & Development


Pre‐IND/Phase 1 Phase 2A Phase 2B Phase 3 NDA/BLA Submission

• Communication needs to be– Frequent– Meeting and/or discussion in addition to written

feedback– Constructive feedback– Maintained throughout development & validation

process• Existing Communication Tools

– Target Product Profile (TPP)– Drug Development Tools (DDT) Qualification Program– ?

Communication with FDA Review Divisions & SEALD


• Prioritize therapeutic areas where COAs are needed to support product labeling claims– List of disease areas to start

• Public engagement forums – Gastroenterology Regulatory Endpoints and the Advancement of

Therapeutics (GREAT)

• Clear regulatory guidance– Methodologies– Evidence required– Comparisons to current treatments

• Consistency across divisions within the FDA and global regulatory agencies

• Communication throughout COA development

Expectations for Advancing Review & Development of COAs


Patient

Patient‐ centered performance measures

(National Quality Forum)

Comparative Effectiveness Research

Health Technology Assessment

Patient‐focused outcome

measurement (PDUFA‐V)

Drug Development & External Environment –Current State of Affairs


Phase 1 Phase 2 Phase 3

FDA: Critical Path Initiative

Industry: efforts to reduce cost & increase R&D productivity/efficiency

• COAs – Measure important aspects of clinical benefit– Basis of medical product approval & labeling claims– Need to be well-defined & reliable

• Anticipated challenges– Right expertise and stakeholders at the table– Timely interactions with FDA– Maintaining commitment and communication

throughout development & validation process

• Opportunity to improve patient outcomes

Summary


Questions?

Emuella Flood301-841-3859

[email protected]

Cicely Kerr Kellee Howard44 (0)1865 324936 [email protected] [email protected]

April Naegeli317-276-0339

[email protected]


Health & Medicine

Measurement in Clinical Trials - An Examination of the FDA Position on Review of Clinical Outcome Assessments (COAs)