EBM Principles Are Not In Crisis
Terry Shaneyfelt, MD, MPHDepartment of Medicine
University of Alabama at BirminghamEBMTeacher.com
@EBMTeacher
Disclosures• Associate editor for BMJ EBM ($)
• Member of American College of Physicians Performance Measurement Committee
• No speakers bureaus, industry grant money, industry stocks
Poorly practiced EBM is not a valid reason to
criticize EBM
High jacked EBM is not a valid reason to
criticize EBM
ACPJC March/April 2002
EBM: A Movement in Crisis?
EBM: A Movement in Crisis?
Problems with the evidence base• Industry sets the research agenda
• Manipulating studies
• Publication bias
• Out of date
• Conflicts of interest / Affect heuristic
Industry shouldn’t be sponsoring studies
Government can’t affordthe failures
How do we get “Real” EBM?
• Patients must demand better evidence, better presented, better explained, and applied in a more personalized way
• Clinical training must go beyond searching and critical appraisal to hone expert judgment and shared decision making skills
• Producers of evidence summaries, guidelines, and decision support tools must take into account who will be using them, for what, and under what circumstances
• Publishers must demand studies meet usability and methodological standards
How do we get “Real” EBM?
• Independent funders must increasingly shape the production, synthesis, and dissemination of high quality clinical and public health
• Research agenda must become broader and more interdisciplinary, embracing the experience of illness, the psychology of evidence interpretation, the sharing of evidence, and how to prevent harm from overdiagnosis
“Real” EBM = different evidence
developed by different people in a
different format
All systems give the results
they are designed to give
Rewards what we have• Goal of industry drugs/devices that get paid for
• What 3rd party payers pay for is what MEDICARE pays for
• What MEDICARE pays for is what the FDA approves
• FDA requires proof of efficacy and safety o Not cost-effectivenesso Not any specific degree of effectivenesso Doesn’t matter how many other similar drugs are out there
“In
div
idu
ali
zed
evi
den
ce”
“Usa
ble
evi
den
ce”
Real EBM Movement
How do we take back EBM?
• The FDA (& equivalent agencies) should:o Demand all studies be registeredo Demand properly conducted trials with hard clinical outcomes (currently
planned trials only reviewed to ensure safety to patients )o Demand publication of all studies o Consider costs and competing array of treatments
• Intensify EBM skills training
• Quality (guidelines) industry needs to be controlled
EBM principles are notin crisis
• Evidence-based moniker has been high jacked
• Algorithmic medicine is antithetical to EBM
• EBM is designed to deal with multimorbidity
• Need lots of changes in the world for “REAL EBM”o Time o Overhaul of drug/device approval/payment schemeo Need experts 1st for “expert judgment”- clinical training improvements
EBM principles are not in crisis
• Good EBM skills can detect problems in the evidence
• Unintended consequences of this movement
• Critical appraisal skills are still criticalo “If peer review was a drug it would never be allowed onto the market. Peer
review would not get onto the market because we have no convincing evidence of its benefits but a lot of evidence of its flaws.”
o “There seems to be no study too fragmented, no hypothesis too trivial, no literature citation too biased or too egotistical, no design too warped, no methodology too bungled, no presentation of results too inaccurate, too obscure, and too contradictory, no analysis too self-serving, no argument too circular, no conclusions too trifling or too unjustified, and no grammar and syntax too offensive for a paper to end up in print.”—Drummond Rennie