Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
1
When Should Observational Studies Be Used For Decision-
Making, or… When Will the Emperor Have
Clothes? Jean R. Slutsky, PA, MSPH - Director, Center for Outcomes and Evidence, Agency
for Healthcare Research and Quality (AHRQ)
Robert W. Dubois, MD, PhD - Chief Science Officer, National Pharmaceutical
Council
Paul Wallace, MD - Senior Vice President and Director, Center for Comparative
Effectiveness Research, The Lewin Group
Marcus Wilson, PharmD – President, HealthCore
Speakers
Robert W. Dubois, MD, PhD - Chief Science Officer, National Pharmaceutical Council
Jean R. Slutsky, PA, MSPH - Director, Center for Outcomes and Evidence, Agency for Healthcare Research and Quality (AHRQ)
Paul Wallace, MD - Senior Vice President and Director, Center for Comparative Effectiveness Research, The Lewin Group
Marcus Wilson, PharmD – President, HealthCore
2
Discussion Topics
• Overview
• Robert W. Dubois, MD, PhD - Chief Science Officer, National Pharmaceutical Council
• AHRQ
• Jean R. Slutsky, PA, MSPH - Director, Center for Outcomes and Evidence, Agency for Healthcare Research and Quality (AHRQ)
• Professional Societies
• Paul Wallace, MD - Senior Vice President and Director, Center for Comparative Effectiveness Research, The Lewin Group
• Payer
• Marcus Wilson, PharmD – President, HealthCore
3
Issues to Explore
• AHRQ produces evidence based summaries
– how and when do they consider RWE?
– would a large and well-designed OS have more weight than an imperfect RCT?
• Professional Societies develop practice guidelines
– are they using RWE in them?
– should they do it more?
• Payment decisions influence care
– when should payers use RWE?
– are they doing so already?
4
RWE Partnerships Continue to Proliferate
PCORI Seeks to Strengthen Quality of Real-World Data and Research Conducted
8
Funding Awards
Ex: Comparative IV vs. Oral Antibiotic
Therapy for Serious Bacterial Infections
Data Infrastructure
Award
$68 M for Clinical or Patient- Powered
Research Networks
PCORI-IOM Roundtable
“Observational Studies in a Learning Health Care System”
5
“Researchers Mine Data From Clinic, Big Insurer”
By Anna Wilde Mathews, January 15, 2013
“UnitedHealth Group and the Mayo Clinic are set to unveil a new research initiative that will draw on millions of health-insurance claims and in-depth clinical patient records…
The initiative will focus on research into "best outcomes for patients at lower costs,"
Wall Street Journal. Researchers Mine Data from Clinic, Big Insurer. January 15, 2013.
ASCO Linking E-Health Data To Promote Real-Time Personalized Cancer Care
The Pink Sheet. ASCO Linking E-Health Data to Promote Real-Time Personalized Carncer Care April 1, 2013. 10
April 1, 2013
For physicians and patients, the network will guide personalized treatment by allowing EHRs to be queried in an effort learn what the best treatment options will be for a given patient based on real-world experiences and real-time analysis of data capture in electronic records.
6
Questions to Ponder
• Are observational studies influencing decision making too much, too little, or just right?
• What barriers need to be addressed so that optimal use occurs?
• What is the role of ISPOR? PCORI? Others?
12
7
IP9: WHEN SHOULD OBSERVATIONAL STUDIES BE USED FOR DECISION-MAKING, OR WHEN
WILL THE EMPEROR HAVE CLOTHES?
Panelist: Paul Wallace MD [email protected]
Collaborators: Anjali Jain*, Robert DuBois +, Jennifer Graff +, Hillary Kleiner*, Sneha Rangarao*, Adam Lustig + and Mariam Siddiqui*
The Lewin Group* and the National Pharmaceutical Council +
Study of the Use of Observational Studies by Guideline Groups
• Study Aim: Qualitative assessment of the use of Observational Studies (OS) among a sample of Guideline Development Groups (GLG)
• Approach
– Key informant interviews with 12 active producers of Clinical Practice Guidelines (GPGs), sampled to represent a range of sponsoring organizations from Government (3),Primary (3) and specialty care (4) professional associations, and Organizations directly involved in delivery of care (2)
– A semi-structured interview guide was developed, with the Transtheoretical Stages of Change (Prochaska) model as a conceptual framework, to explore the state of readiness, current practices, and future plans for use of OS in development of CPGs
8
CPG Development Characteristics • All GLG used a similar hierarchal approach to initial evidence grading, with
RCTs exclusively at the top of the hierarchy for all professional society GLG
• All GLG described use of defined protocols for literature search, assessment of evidence quality, and evidence synthesis (evidence review)
– About half of the interviewed GLG relied on an AHRQ Evidence Practice Center (EPC) after topic nomination or through a specifically contracted evidence review with an EPC or similar organization
– The remaining GLG used internal resources plus volunteers for evidence review
• Most GLG were aware of the recent IOM reports addressing Trusted CPGs and Systematic Reviews, and approximately 2/3 of the interviewed GLG noted changes in their internal processes as a result of the IOM Reports within the last 2 years
• All GLG were aware of GRADE and had made an active choice to employ or not to employ this methodology
Current Patterns of Use of OS
• Almost all GLG noted frequent use of OS to develop CPG recommendations and OS were used in nearly all or most CPGs developed
• Most common uses of OS
– To fill a gap in RCT based evidence for a key question developed during the CPG scoping process; mainly
– Uncontrolled trials and
– Case series
– To identify Safety signals/Evidence of harms
– Case Series and Case Reports (common)
– Registries (uncommon)
9
GRADE: Rating the Quality of Evidence
Source: GRADE Working Group, 2012. See, e.g.: Balshsem H, et al. GRADE guidelines: 3. Rating the quality of evidence. J Clin Epidemiol 2011(64):401-6.
Patterns of Use of OS (cont’d)
• For GLG using GRADE, only rare examples of OS being judged equal or higher quality than RCTs when RCTs and OS address the same issue
• Several examples of CPG recommendations with a strong strength of recommendation based only on OS in absence of applicable RCTs
– Multiple concordant OS
– Large sample size
• Special cases where OS generally form the evidence base for recommendations
– CPGs addressing use of diagnostic technologies (RCTs linking to health outcomes uncommon) - 2 Interviewees
– CPGs with key questions addressing use of Complementary and Alternative Medications and Therapies - 1 Interviewee
10
19
Rounding out the evidence…
PICO(TS)*
P Patient, Population or Problem
A description of the patient(s) of interest. It includes the condition(s), populations or sub-populations, disease severity or stage, co-morbidities, and other patient characteristics or demographics
I Intervention or Exposure
Refers to the specific treatments or approaches with the patient or population. It includes doses, frequency, methods of administering treatments, etc.
C Comparison Describes what is being compared with the intervention described above. It includes alternatives such as placebo, drugs, surgery, lifestyle changes, etc.
O Outcome Describes the specific results of interest. It refers to short, intermediate, and long-term outcomes, and includes specific areas such as quality of life, complications, mortality, morbidity, etc.
(T) Timing, if applicable
Describes the duration of time that is of interest for the particular patient outcome, benefit, or harm to occur (or not occur)
(S) Setting, of applicable
Describes the setting or context of interest. Setting can be a location (such as primary, specialty, or inpatient care), or health policy that frames or restricts the important questions to be answered
*Source: AHRQ. The Effective Health Care Program Stakeholder Guide
Appendix D: Research Questions & PICO(TS) (July 2011). Available at:
http://www.ahrq.gov/clinic/epcpartner/stakeholderguide/
Current Main Uses of OS
In CPG
Can OS improve the application of CPG to diverse groups of patients?
Patient-centered uses of OS?
• To reflect under-represented sub-populations
– Race and ethnicity
– Socioeconomic status
– Geography
• To estimate impact of co-morbidity on apparent effects reflected by RCTs
• Offer guidance for other rare conditions
• Inform shared decision making - Link guidance to patient preference and values…
11
Use of Registries and “Big Data”
• Little current development of systematic processes by interviewed GLG to incorporate data from:
–Registries based on single or multiple condition prevalence
• Including several GLG where the sponsoring organization has extensive investment in condition and procedure oriented registries
–Current large population databases (mainly claims data, but also EMR)
Future GLG directions • Key Trends self- identified by GLGs
– Desire to ‘harmonize’ recommendations across multiple CPGs addressing a common topic
• Driving adoption of common CPG processes, such as GRADE
– Desire to more directly involve patients and care givers in CPG process
• Substantial experimentation- little agreement on preferred approach
• Differentiation between patients, patient advocates, and public comment
• Impact on evidence review and role of OS undefined; most groups expect patient involvement to lead to re-examination of “evidence rules”
12
Patient centered or personalized?
Empowering Healthcare Decision-makers with Real World Evidence
Marcus D Wilson, Pharm.D. President HealthCore a WellPoint Company
Panel: When Should Observational Studies Be Used for Decision-Making or, When Will The Emperor Have Clothes
13
Payer Perspective on Observational Research
Promise: Payers seeks real world evidence in key areas including:
• Adherence1
• Cost1, 2
• Safety1, 3
• Outcomes4
Limitations: By and large where payers see limitations it is the areas of :
• Transparency/integrity of process1
• Data (quality, environment, paucity of economic data)1, 5, 6
• Motives of sponsors1
In addition, many payers would like to be involved in informing CER evidence development efforts1
1. Wang A, Halbert RJ, Baerwaldt T, Nordyke RJ. Us Payer Perspectives on Evidence for Formulary Decision Making. J Oncol Pract. 2012;8(3S):22S-27S. 2. Meyer CM, Phipps R, Cooper D, Wright A. PharmacyLowry SJ, Loggers ET, Bowles EJA, Wagner EH. Evidence Gaps in Advanced Cancer Care: Community-Based Clinicians’ Perspectives and Priorities
for CER. Am J Manag Care. 2012;18(5 Spec No. 2):SP77-SP83. 3. Atkins D. Creating and Synthesizing Evidence with Decision Makers in Mind: Integrating Evidence from Clinical Trials and Other Study Designs. Med Care. 2007;45:S16-S22. 4. Hochman M, McCormick D. Characteristics of Published Comparative Effectiveness Studies of Medications. JAMA. 2010;303(10):951-8. 5. Bourgeois FD, Murthy S, Mandl KD. Comparative Effectiveness Research: an Empirical Study of Trials Registered in Clinical Trials.gov. PLoS One. 2012;7(1):e28820 6. Benefit Forecast for a New Interferon Beta-1a for the Treatment of Multiple Sclerosis: Development of a First-Line [trunc]. J Manag Care Pharm. 2003;9(2):168-74.
Hurdles
Right Questions
Appropriate Methods of Analysis
14
Driver of Poor Clinical & Cost of Care Outcomes: Root of the Real World Evidence Gap
Utilization
Little to No
Evidence
Little to No
Evidence
Evidence Clinical Trials
Conceptualization Regulatory Approval
Unstudied co-morbid conditions
Differing concomitant medications
Varying levels of compliance – i.e. < 80%
Major Challenge: Variances in population characteristics from what was studied
Differing age groups – elderly, pediatrics
Race, ethnicity, gender variances
Varying severity of disease
Evidence Post-Marketing Studies
Hurdles
Right Questions
Appropriate Methods of Analysis
Robust, Reliable and Relevant Data
15
Reliable Evidence Development Starts with Good Data
Question the source as critically as we do the methods of analysis
*Note: ~1500 member population. Select enrichment measures only
Electronic Healthcare Data Proliferation
Health Plan Claims
Inpatient Chart
Information
Outpatient Chart Information
Patient Reported
Information
Value is unlocked with integration
Each Data Source
Contributes Unique
Information
Health Plan Claims provides
a broad view, insight into cost
of care & information such as
actual fill (Rx) history
Outpatient Chart Information
provides rich clinical
information and such
endpoints as prescribed
medications, lab and test
results, OTC advice and
reported use, family history
Inpatient Chart Information
provides rich clinical
information on procedures,
conditions treated, medication
use, lab and test results
The greater the
overlap the deeper
the insight
16
A key step for the RWE Collaboration success: the Longitudinal Patient Record…
Patient Reported & Device
Data
EHR Data
Other Data
Pharmacy
Claims Data
Medical
Claims Data
Eligibility Data
Lab Results
Data
Delivery System Cost
of Care Data
360° view of patient to improve patient health
… and manage total cost of care
RWE: Real World Evidence Collaboration
Hurdles
Right Questions
Appropriate Methods of Analysis
Robust, Reliable and Relevant Data
Available at the Point of Decision
• Ability to Effectively Disseminate findings to Downstream Decision-Makers
17
Everything we do must be done with the intent
of strengthening the bond and improving
decision-making between these two individuals.
Of All the Relationships in Healthcare, This One is the Most Important
Driver of Poor Clinical & Cost of Care Outcomes: Root of the Evidence Gap
Utilization
Little to No
Evidence
Little to No
Evidence
Evidence Clinical Trials
Conceptualization Regulatory Approval
Unstudied co-morbid conditions
Differing concomitant medications
Varying levels of compliance – i.e. < 80%
Major Challenge: Variances in population characteristics from what was studied
Differing age groups – elderly, pediatrics
Race, ethnicity, gender variances
Varying severity of disease
Evidence Post-Marketing Studies
Patient
Patient
Patient Physician
LPR
18
Barriers to Improved Quality & Affordability
Current Complaints
Patient History
Test Results Coverage
Reimbursement Medical Policy
Socioeconomic Issues
New Study Findings
For Complex Patients the time needed to make the most informed set of decisions
1-2 Hours
Amount of time the physician actually spends per patient
13 minutes
Errors &
waste Errors &
waste
Disconnect
Translation of population-based
evidence to the individual in front of
me
Impact of Health Information Technology Improvements
Decision Support
HIT
Research
HIT will facilitate Real World Research to some extent.
More importantly, it will make putting research into action quickly a
reality
Without effective and trusted tools for decision-support the knowledge gap at the point of
care will only widen
Improved quality, safety and affordability
19
Research
&
Analytics
Payer
Policy
Aligned
Payment
Model
Relevant
Information
at POC
Closing the Loop on Evidence Development & Meaningful Impact
Care
Management
Programs