Upload
vancong
View
216
Download
0
Embed Size (px)
Citation preview
1
February 15, 2018
MDICx webinar seriesFrom Stories to Evidence: Quantitative patient-preference information to inform product-development and regulatory reviews
Shelby ReedProfessor, Duke School of Medicine
F. Reed JohnsonProfessor, Duke School of Medicine
Juan Marcos GonzalezAssistant Professor, Duke School of Medicine
From Stories to Evidence: Quantitative patient-preference information to inform product-development and regulatory reviewsMDIC Webinar February 15, 2018
Shelby ReedProfessor, Duke School of Medicine
F. Reed JohnsonProfessor, Duke School of Medicine
Juan Marcos GonzalezAssistant Professor, Duke School of Medicine
This presentation reflects the views of the authors and should not be construed to represent the policies of the U.S. FDA.
Preference Evaluation Research Group (PrefER)
Patient-centered healthcare movement
3
Who is qualified to make relative-importance value judgments?
Clinicians
Politicians
Patients Informed consent
Individualistic ethic:Individuals are the best judge of their own welfare
Patient preferences are critical in determining when aproducts benefits outweigh its risks .
-- Robert M. Califf (JAMA 2017) 4
Increasing Support from FDA
4
FDAs guidance on benefit-risk determinations for device approvals describes patient tolerance for risk and perspective on benefit as an explicit factor the agency may consider in approval decisions.
Dr. Rob Califf, FDA Deputy, Former FDA Commissioner
6
You dont know peoples preferences unless you ask them. How do people look at these differences? And I fell in love with the discrete-choice experiments, which I had heard about from the Business School, but had not seen in action and I think that provides major advantages. So this is a great day for me. Its you know been a long time. Who would have thought it would come from the device world? I think its a major triumph for the device world that were here today, not just talking about it, but with the FDA very involved. To the extent that FDA takes preferences seriously, I think its a great day.
Release Event for the MDIC Framework for Integrating Patient Perspective into Medical Device Benefit-Risk Assessments and the FDA Center for Devices and Radiological Health Draft Guidance, May 13, 2015
History of CDRH interest in PPI
7
Based on Levitan, NIH HCS Collaboratory and PCORnet Grand Rounds, 2016
Preference-Sensitive Regulatory Decisions
Risks
WeightLoss
Patient Engagement Advisory Board (2015)Bennett LevitanAvailable at: https://www.noexperiencenecessarybook.com/wwzgN/powerpoint-presentation.html
Diet & Exercise
?
8
Less weight loss
Lower risks
Regulatory Impact of the Study EnteroMedicss Maestro Rechargeable
System for weight loss
Device failed to meet its original trial endpoints
Device was approved in January 2015 First new obesity device approved by FDA since
2007
First approval to result from CDRH's patient preference initiative
http://www.fda.gov/NewsEvents/Newsroom/PressAnnouncements/ucm430223.htm 9
FDA understands that patients and care-partners who live with a disease or condition on a daily basis and utilize devices in their care may have developed their own insights into and perspectives on the benefits and risks of devices reviewed...
--August 2016 Guidance
Center for Devices and Radiological Health
10
What are preferences?
Qualitative or quantitative statements of the relative desirability or acceptability of attributes that differ among alternative interventions.
Medical Device Innovation Consortium (PCBR Framework Report 2015)
Stories from individuals
Evidence representative
of a groupOften obtained from surveys
Defined by what people are willing to
give up
or health states care processes convenience features other
Characteristics or features
Approaches Qualitative methods (focus groups, public meetings)
Identify areas of concern
Provide context for product-development and regulatory decisions
Simple quantitative methods (ranking, threshold) Prioritization
Tradeoffs involving only two outcomes
More advanced quantitative methods (choice experiments, best-worst scaling) Tradeoffs involving more than two outcomes
Statistical preference measures (risk tolerance, minimum acceptable benefit, time equivalents)
Publishable regulatory-quality evidence
Todays focus: discrete-choice experiments12
Discrete-Choice Experiments to Quantify Patient Preferences
Developed, tested, and validated over past 40 years in market research
transportation planning
environmental economics
health
Daniel McFadden received the Nobel Prize in Economics in 2000 for conceptual and statistical foundations
Increased interest and regulatory support because of commitment to patient-centered healthcare
10
Example Choice Task: Cystic Fibrosis
14Mohamed, Johnson, Balp, Calado, The Patient 2016
Choice-Experiment Features
Also known as choice-based conjoint analysis
Alternatives consist of combinations of features
Preferences among alternatives depend on the relative importance of features
Respondents indicate choices among hypothetical alternatives
Statistical analysis of pattern of choices indicates relative importance of features
12
Survey Development
EndpointsChoice Tasks
Pretest Interviews
ExperimentalDesign
Prefer X
Prefer Y
Prefer X
Prefer YPrefer X
Level B1Level B2Level R1Level R2
Prefer Y
Level B1Level B2Level R1Level R2
Data Collection& Analysis
Preference Weights
Definitions
Steps in a Choice-Experiment Study
16
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value frameworks
Personalized medicine
USE
Evidence reviews
GUIDANCE
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
17
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value frameworks
Personalized medicine
USE
Evidence reviews
GUIDANCE
Weighted PRO
endpoints
EXAMPLE
18
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
European Organisation for Research and Treatment of Cancer
Mohamed AF, Hauber AB, Johnson FR, Coon CD. Patient preferences and linear scoring rules for patient reported outcomes. Patient. 2010;3(4):217-27.
Patient-Reported Outcome Quality of Life QuestionnairePreference Weights
16
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value frameworks
Personalized medicine
USE
Evidence reviews
GUIDANCE
Weight-Loss
Devices
EXAMPLE
21
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
Preferences and Regulatory Decisions
Regulators Benefit-
Risk Threshold
Benefit
Risk o Disapprove
+ Approve
oo
o
o
oo
++
+
+
++
Patients Benefit-Risk Threshold
+
++
o
o
oo
o
+
22
Preference-Sensitive Regulatory Decisions
Risks
WeightLoss
Patient Engagement Advisory Board (2015)Bennett LevitanAvailable at: https://www.noexperiencenecessarybook.com/wwzgN/powerpoint-presentation.html
Diet & Exercise
?
23
Less weight loss
Lower risks
FDA Obesity Study
24Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015
FDA Obesity Study
25Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015
FDA Obesity Study
Ho et al. Surgical Endoscopy (2015)
30% weight loss 1.2% mortality risk
26Ho, Gonzalez, Lerner, et al., Surgical Endoscopy. 2015
Weight-Loss Device Decision Tool
27
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value Personalized medicine
USE
Evidence reviews
GUIDANCE
Value Frameworks
EXAMPLE
28
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
Value Frameworks
2014 2015 2016 2017
29
StdCareExpTrx
StdCareExpTrx
QALYsQALYsCC
ICER
=
To compare cost-effectiveness for treatments across conditions, outcomes must be measured using the same units.
Traditional Value Assessment
Other Elements of Value
Other Elements of Value
Option value
Value of knowing
Equity
Value of hope
Impact on
others
Dosing regimen
Clinicalbenefits
Cost
Value
Adapted from Garrison L, 2016 and Neumann PJ, 2016.
31
ASCO Value Framework Scoring Rubric
0 to 100 (OS>PFS>RR)
Clinical benefit-20 to 20
Toxicity0 to 10
Palliation*0 to 20
Survival curve* 0 to 10
Trx-free interval*0 to 20
Quality of life*
Clinical benefit Toxicity Bonus points*Net health Benefits Cost per month
DAC:____Patient payment:___
Drug Abacus from MSKCC
$12,000 to $300,000
10% to 30%
1.0 to 3.0
1.0 to 3.0
1.0 to 3.0
1.0 to 3.0 Monthly drug price
http://www.drugabacus.org/drug-abacus/tool/
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value frameworks
Personalized medicine
USE
Evidence reviews
GUIDANCE
Crohns Disease
EXAMPLE
34
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
More time in remission
Higher risk of cancer
Lower risk of infections
Less use of steroids
TNF- inhibitors vs. Corticosteroids for Crohns disease
Evidence Reviews: Benefits and harms of Crohns disease therapies
More time in remission
Higher risk of cancerLower risk of infections
Less use of steroids
35
Choice Question Example: Crohns Disease
PCORI, NCT02316678; PI: James Lewis 36
Patient-Centered Comparative Effectiveness Research
Patient Preferences
Real-World DataDCE
Simulation Model
Time equivalents Treatment 1 Treatment 25.5 3.7
Attributes
Add-on therapy
Hospital-izations Survival
Adverse event 1
Adverse event 2
Time equivalents Treatment 1 Treatment 2Group 1 Group 2 Group 3
Outcomes
Add-on therapy
Hospital-izations Survival
Adverse event 1
Adverse event 2
Time Equivalents
Event rates: Treatment 1Treatment 2
Time equivalents Treatment 1 Treatment 2Group 1 Group 2 Group 3
Comparative Effectiveness
Anti-TNFs Prolonged CS DifferenceRemission-time equivalents, mean (SD)
5.3 (4.0) 4.5 (3.7) 0.8 (0.5 1.1)
Medicaid and Medicare claims analysis
Markov modelDCE
PCORI, NCT02316678; PI: James Lewis38
Potential Patient-Centered Applications
Effects Treatment 1 Treatment 2Add-on therapy - +
Hospitalizations + -
Survival - +
Adverse event 1 + -
Adverse event 2 - +
Traditional Comparative Effectiveness Research
Effects Treatment 1 Treatment 2Add-on therapy - +
Hospitalizations + -
Survival - +
Adverse event 1 + -
Adverse event 2 - +
Time equivalents
Preference-based Comparative Effectiveness Research
Latent-Class Choice-Model EstimatesCrohns Disease
-10-9-8-7-6-5-4-3-2-10
0 4 8 12 0 4 8 12 0 4 8 12 0 2 8 12 0 5 1530 0 2 5 8 0 2 5 8
SevereDuration
ModerateDuration
MildDuration
SteroidDuration
InfectionRisk
CancerRisk
SurgeryRisk
Efficacy Class Steroid Class Risk Class
PCORI, NCT02316678; PI: James Lewis 40
Comparative Effectiveness
Remission-time equivalents
Anti-TNFs Prolonged CSDifference (95% CI)
Efficacy Class 1.3 (6.7) 0.1 (6.4) 1.3 (0.8, 1.7)
Steroid Class 6.9 (2.9) 6.4 (2.7) 0.6 (0.4, 0.8)
Risk Class 7.8 (2.6) 7.3 (2.5) 0.5 (0.3, 0.7)
PCORI, NCT02316678; PI: James Lewis
Medicaid and Medicare claims analysis
Markov modelDCE
41
Applications over product life cycles
CLINICAL DEVELOPMENT
REGULATORY REVIEW
ACCESS
Weighted endpoints
Benefit-risk
Value frameworks
Personalized medicine
USE
Evidence reviews
GUIDANCE
Shoulder dislocation
EXAMPLE
42
Study
1 2 3
Quality + - ++
N 100 50 300
EndptA - + +
EndptB + + -
RTx A
Tx B
Outcome 1Outcome 2Outcome 3
SAE 1SAE 2SAE 3
Side effects
Cost
Convenience
Alternatives
Health outcomes
Disease severity
Preferences in Practice Guidelines
Clinicians must communicate evidence-based options for treatment, inclusive of their benefits and risks, and patients must be allowed to express their goals and preferences.
Gynecol Oncol. 2016;143(1):3-15.
Potential Patient-Centered Applications
Patient characteristicsdemographic and contextual variables
Choice questionsgenerate patient-level preference weights, or classify patients into preference groups
Decision model predict expected outcomes with alternative treatments
Compute net benefitscombine expected outcomes with preference weights
Decision aids
First-time shoulder dislocationPersonalized Medicine
45
0 5 10 15 20 25 30 35
Limits on Shoulder Motion
Avoid High-Risk Activities
Duration of PT
Chance of Recurrence
Out-of-Pocket Cost
Relative Importance
Importance of Attributes in Shoulder Dislocation
N=374
Personalized Medicine
Streufert BD, Reed SD, Johnson FR, Huber JC, Orlando LA, Taylor DC, Mather III RC. Orthop J Sports Med. PMID 28377932
46
Decision-analytic model
Personalized Medicine
Adaptive choice questions to generate patient-level
preference weights
Demographics and contextual variables
Patient Output:
47
Clinical literature
Benefits of Personalized Medicine
Patients More informed patients Improved patient-provider
communication Greater patient satisfaction Improved adherence and
patient outcomes
Health System Improved efficiency of health
care delivery Documentation of patient-
centered care Provide justification for
changes in practice patterns
Opportunities and Challenges
Documenting patient concerns about new devices is useful for regulatory reviews and at other points in product life cycles.
Stated-preference methods are relatively unfamiliar and there is limited experience with health applications.
Methods are highly flexible and can be adapted for evaluating almost any preference-sensitive decision.
FDA has provided guidance on developing patient-preference evidence for devices.
49
50
DISCUSSIONPlease submit your question
via the chat box
Next session
How do you define study objectives and scope?
What study-team skills are required?
How do you work effectively with the technical team?
What steps are required for conducting a regulatory-quality study?
How long will it take and what will it cost?
51
Planning and implementing a choice-experiment study
Contact Information
Shelby [email protected]
919 668 8991
Reed [email protected]
919 668 1075
Juan Marcos [email protected]
919 668 5157
52
mailto:[email protected]:[email protected]:[email protected]
53
Join us for the next 2 sessions
March 15 - Session 2: Example applications and lessons learnedinstrument development
April 19 - Session 3: Example applications and lessons learnedanalysis and reporting
Recordings will be available on http://mdic.org/mdicx
February 15, 2018From Stories to Evidence: Quantitative patient-preference information to inform product-development and regulatory reviewsMDIC Webinar February 15, 2018Patient-centered healthcare movementWho is qualified to make relative-importance value judgments?Increasing Support from FDADr. Rob Califf, FDA Deputy, Former FDA CommissionerHistory of CDRH interest in PPIPreference-Sensitive Regulatory DecisionsRegulatory Impact of the StudyCenter for Devices and Radiological HealthWhat are preferences? ApproachesDiscrete-Choice Experiments to Quantify Patient PreferencesExample Choice Task: Cystic FibrosisChoice-Experiment FeaturesSteps in a Choice-Experiment StudyApplications over product life cyclesApplications over product life cyclesEuropean Organisation for Research and Treatment of CancerApplications over product life cyclesPreferences and Regulatory DecisionsPreference-Sensitive Regulatory DecisionsFDA Obesity StudyFDA Obesity StudyFDA Obesity StudyWeight-Loss Device Decision ToolApplications over product life cyclesValue FrameworksTraditional Value AssessmentOther Elements of ValueASCO Value Framework Scoring RubricDrug Abacus from MSKCC Applications over product life cyclesSlide Number 35Choice Question Example: Crohns DiseasePatient-Centered Comparative Effectiveness ResearchComparative EffectivenessPotential Patient-Centered ApplicationsLatent-Class Choice-Model EstimatesCrohns DiseaseComparative EffectivenessApplications over product life cyclesPreferences in Practice GuidelinesPotential Patient-Centered ApplicationsPersonalized MedicinePersonalized MedicinePersonalized MedicineBenefits of Personalized MedicineOpportunities and ChallengesSlide Number 50Next sessionContact InformationJoin us for the next 2 sessions