Embed Size (px)
Citation preview
Comparative Effectiveness Research Methods and Research Infrastructure
1
Sean Tunis MD, MScSeptember 17, 2009
Quality of Evidence for Guideline Recommendations in CV disease
Robert Califf, IOM Meeting on Evidence-based Medicine, December 2007
The EBM Paradox
How to reconcile:18,000 RCTs are published per year“Available evidence is limited or poor quality”
The gaps, as seen by decision makersPatients are highly selected Research settings are not typical of communityMissing or incorrect comparators Physiologic or surrogate outcomes, not functionResults are not available when decisions made
The Paradox Explained
Decision makers (patients, consumers, clinicians, payers, policy makers) have had limited ability to influence the clinical research enterprise
4
5
6
IOM Committee’s Definition of CER
The generation and synthesis of evidence that compares the benefits and harms of alternative methods to prevent, diagnose, treat, and monitor a clinical condition or to improve the delivery of care. The purpose of CER is to assist consumers, clinicians, purchasers, and policy makers to make informed decisions that will improve health care at both the individual and population levels.
7
IOM Recommendation 6
The CER Program should fully involve consumers, patients, and caregivers in key aspects of CER, including strategic planning, priority setting, research proposal development, peer review, and dissemination.
8Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact • June 9, 2009
Engaging stakeholders
• Recommendation 1: Conduct a systematic assessment of best practices for effective engagement of decision makers
• Recommendation 2: As a condition of receiving federal funding for any CER study, the investigators must form a stakeholder advisory committee whose function is based on findings of #1
CER Defining Characteristics
Objective of directly informing clinical or health policy decisionCompares at least 2 alternative, each with potential to be best practiceResults at population and subgroup level
Measures outcomes important to patientsMethods and data sources appropriate for the decision of interestConducted in real world settings
9
Categories of CER MethodsSystematic reviews of existing research
Decision modeling, with or without cost information
Retrospective analysis of existing clinical or administrative data
Prospective non-experimental studies, including registries
Experimental studies, including randomized clinical trials (RCTs)
All methods have a role
Inevitable tradeoff between internal validity and feasibility, generalizability, cost, time
The nature of the research question, and the decision maker will influence best practices
Experimental studies will have a crucial role in CER, and there is need for improving design and implementation
Non-experimental methods hold great promise, particularly as methods are refined and data infrastructure is improved
Explanatory / Pragmatic Clinical Trials
• Explanatory: seek to estimate maximum possible effect of an intervention– And to understand how/why effect occurs
• Pragmatic: seek to inform choices between feasible alternative– By estimating real world outcome probabilities
• Schwartz and Lellouch, 1967– Explanatory and pragmatic attitudes in therapeutic trials. J Chron Dis
1967;20:637-48.
• Tunis, Stryer, Clancy 2003– Increasing the value of clinical research for decision makers in clinical
and health policy. JAMA 2003;290:1624-32
Illustrating the Challenge
Clinical utility of CT AngiographyVertebroplasty for back painRadiation therapy for prostate cancerOff-label uses of cancer drugs Coordinated care for chronic illness
13
Medicare Review of CCTA
EPC report from Duke (April 2006)Limited evidence of clinical utility in any population
MedCAC mtg (May 2006)“Uncertain confidence about existing evidence”
Broad local coverage of CCTAMedicare draft policy in 12/07 proposed CED for CCTA in “adequate” studies
Debate on CCTA Coverage
Payers/researchers want RCT with death/AMI outcome20k patients, 2+ years of follow-upNon-experimental options rejected
Vendors / clinicians existing evidence adequate for coverage
Medicare final decision (March 2008)No adequately designed studies show improved outcomes“We believe large, well-designed prospective trials needed”Broad coverage by local contractors retained
NHLBI currently reviewing 3 large RCTs
16
Recommendation 7
The CER Program should devote sufficient resources to research and innovation in the methods of CER, including the development of methodological guidance for CER study design such as the appropriate use of observational data and more informative, practical, and efficient clinical trials.
Baucus-Conrad PCOR Legislation
• “Establish and maintain methodological standards for comparative clinical effectiveness research on major categories of interventions to prevent, diagnose, or treat a clinical condition or improve the delivery of care”
17
SACGHS recommendation
• “Information on clinical utility is critical for managing patients, developing professional guidelines, and making coverage decisions.”
• “HHS should create a public private entity of stakeholders to….establish evidentiary standards and levels of certainty required for different situations”
18
Cardiac Imaging Think Tank
Discussed optimal methods for evaluating clinical utility of cardiac imaging
Co-sponsors: ACC, ACR, SCCT, ASNCprivate payers, CMS, vendors, clinical researchers, consumers, AHRQ, VA, etc.Goal: balance validity with feasibility
Current statusMeeting summary completedDrafting Effectiveness Guidance Document on clinical utility of non-invasive cardiac imaging
Effectiveness Guidance Documents
Analogous to FDA-guidanceTargeted to product developers, clinical researchers
Recommendations for design of clinical studies to generate evidence that is adequate for decision making“reasonable confidence” of improved health outcomes
Started from insights from systematic reviewsMulti-stakeholder advisory group, iterative draft and comment processOngoing work
Gene expression profiling for breast cancerTreatment for chronic woundsCardiac imagingIntegrative medicine
Other CER Methods Initiatives
IOM CER Innovation CollaborativeCTSA subgoal 4B working groupAHRQ methods symposia
21
22Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact • June 9, 2009
Infrastructure for efficient trials• >10% of funds allocated to CER in the next 10 years
should be used in the development of infrastructure for enhancing the efficiency of CER trials:
– Data standards capable of supporting practice-based clinical research
– Informatics grids and other architecture to create a national PBRN for CER trials
– Incentives for participation of investigators and patients in CER trials
– Standard contract language for CER trials that use network infrastructure
– Guidance to institutional review boards on human subjects protection issues commonly encountered in CER trials
23Implementing Comparative Effectiveness Research: Priorities, Methods, and Impact • June 9, 2009
Infrastructure for non-experimental CER• >10% of funds allocated to CER in the next 10 years
should be directed to development of infrastructure for learning from the delivery of health care:
– Distributed data networks for administrative and clinical databases—including Medicare and Medicaid data—and procedures for adding other databases, investigator access, and safeguarding privacy and security
– Technical data standards and a common vocabulary to link databases.
– Incentives for organizations with relevant data to adopt these standards and participate in research networks
– Policy guidance to HHS, other data owners, study sponsors, and investigators that balances the ethical need for evidence to inform decisions with the need to safeguard personal health information
CMTP Projects on CER Methods
PRIORITIESfor
Evidence Development
PRIORITIESfor
Evidence Development
Trial DESIGN and IMPLEMENTATION Trial DESIGN and
IMPLEMENTATION
EffectivenessGUIDANCE Documents
EffectivenessGUIDANCE Documents
Improve the Relevance, Timeliness and Quality of Clinical Research
Improve the Relevance, Timeliness and Quality of Clinical Research
Applied Policy Projects – e.g. CEDApplied Policy Projects – e.g. CED
Recent PCT Papers
• CONSORT extension - BMJ (Nov 2008)– Zwarenstein, Treweek, Gagnier, Altman,
Tunis et al• PRECIS – Journal Clinical Epidemiology
(April 2009)– Thorpe, Zwarenstein, Oxman, Treweek,
Furberg, Altman, Tunis et al• Rethinking RCTs for CER – Annals of
Internal Medicine (July 2009) – Luce et al.26
The blank pragmatic–explanatory continuum indicator summary (PRECIS) “wheel.”
Domain Pragmatic Trial Explanatory Trial
Participants
Participant Eligibility Criteria All participants who have the condition of interest are enrolled, regardless of their anticipated risk, responsiveness, co- morbidities, or past compliance.
Step-wise selection criteria are applied that: (a) restrict study individuals to just those previously shown to be at highest risk of unfavorable outcomes, (b) further restrict these high risk individuals to just those who are thought likely to be highly responsive to the experimental intervention, and (c) include just those high risk, highly responsive study individuals who demonstrate high compliance with pre-trial appointment-keeping and a mock intervention.
PRECIS Domains Illustrating the Extremes of Explanatory and
Pragmatic Approaches to Each Domain
Domain Pragmatic Trial Explanatory Trial
Follow-up and Outcomes
Primary Trial Outcome The primary outcome is an objectively measured, clinically meaningful outcome to the study participants. The outcome does not rely on central adjudication and is one that can be assessed under usual conditions: for example, special tests or training are not required.
The outcome is known to be a direct and immediate consequence of the intervention. The outcome is often clinically meaningful, but may sometimes (early dose-finding trials for example) be a surrogate marker of another downstream outcome of interest. It may also require specialized training or testing not normally used to determine outcome status or central adjudication.
PRECIS Domains Illustrating the Extremes of Explanatory and
Pragmatic Approaches to Each Domain
Pragmatic Pharmaceutical Trials
• Optimize design of phase III trials to be more informative for post-FDA decision makers
• Clarify patient, clinician, payer evidence needs • Identify critical regulatory, methodological
financial, operational barriers• Develop PPCT guidance document• Industry, FDA, CMS, NICE, PBAC, CDR,
Consumer’s Union, Medco, BSBCA, others
32
