Click to edit Master title style Click to edit Master subtitle style UMASS MEDICAL SCHOOL MEYERS PRIMARY CARE INSTITUTE The following presentation contains

Click to edit Master title style

Click to edit Master subtitle style

UMASS MEDICAL SCHOOL MEYERS PRIMARY CARE INSTITUTE

The following presentation contains some items that are covered by copyright and are used

under Fair Use for education and the federally legislated TEACH Act. Any use for other

purposes must follow U.S. copyright rules.

The contents of this module are accurate at the time of publication.




Evidence Based Medicine (EBM)

Looking beyond the hype




The EBM perspective on “evidence”

Considering scientific evidence and dealing with the available information about pharmaceuticals

Critical appraisal of evidence from randomized trials

Issues that impact published trial results

Objectives




Case

Your patient returns after change to a generic medication

The patient has concerns about taking a generic medication and asks for another

You need to evaluate the ad and the medication




EVERYWHERE!




EBM

“Evidence-based medicine is the integration of best research evidence with clinical expertise and patient values.”

-David Sackett

Sackett D, et al. 2000. Churchill Livingston.




Systematic Reviews / Meta-analyses / Randomized Trials

Cohort Studies

Case-control studies

Cross-sectional studies

Case Reports

Expert opinion

Hierarchy of Evidence




Why are randomized trials considered the “best evidence” for answering a clinical question about prescribed

medications?(Please enter your answer below)




Hierarchy of Evidence:An Alternate Opinion

1. Things I believe2. Things I believe despite the available data3. Randomized, controlled clinical trials that

agree with what I believe4. Other prospectively collected data5. Expert opinion6. Trials that don’t agree with what I believe7. What you believe that I don’t

Bleck TP. Br Med J. 2000;321(7255):239.




Basic Mistakes We Make in Thinking

Kida T. 2006. Prometheus Books.

We rely on stories

rather than statistics

We try to confirm our ideas, rather than question them

We don’t appreciate the role of chance

We misperceive the worldWe oversimplifyOur memories are faulty




“How important are these factors in influencing your prescribing?”

Very Important

Drugads

Drugcompany

reps

Patientpreferences

Scientificpapers

Advicefrom

colleagues

Own training&

experience

4%

20%

2%

62%

48%

88%

Avorn J, et al. Am J Med. 1982;73(1):4-8.




Do you think you are immune to the impact of pharmaceutical marketing?

(Please enter your answer below)




Ezetimibe

Jackevicius CA, et al. N Engl J Med. 2008;358(17):1819-1828.

Statins

Canada US

Canada US




1. Design

Are control and treatment groups equivalent?




Hildesheim A, et al. J Am Med Assoc. 2007;298(7):743-753.




1. Design

Were patients, clinicians and investigators blinded?





Sprung CL, et al. N Engl J Med. 2008;358(2):111-124.




1. Design

Were patients, clinicians and investigators blinded?

Was measurement consistent across groups?





Abitbol V, et al. Clin Gastroenterol Hepatol. 2007;5(10):1184–1189.




2. Results

What were the results?

Are they biologically valid?

Is this result clinically significant?

What is the strength of the effect?




Czeisler CA, et al. N Engl J Med. 2005;353(5):476-486.




Magnitude (absolute risk reduction ARR)– ARR = control event rate – treatment event rate

Precision– 95% confidence interval (note studies that use others like 90%)

Is this outcome valid?




Berger JS, et al. Am J Med. 2008;121(1):43-49.




Is this outcome valid?

Number needed to treat (NNT)• NNT = 1 / ARR

= 1 / (control event rate – treatment event rate)

Number needed to harm (NNH)• NNH = 1 / ARI = 1 / Absolute Risk Increase

= 1 / (intervention harm rate – control harm rate)




Clear representation of data?

1 / ARR = NNT

1 / .011 = 90

90 women must take Fosamax for 4 years to prevent 1 hip fracture




3. Are the results relevant?

Do your patients meet the study’s criteria?

Did the study focus on outcomes that you find relevant?

Do the treatment benefits outweigh the potential

harm/costs?




Issues in Relying on Randomized Trial Results

What are the inclusion and exclusion criteria?

Is your patient population represented?

Are the comparison groups appropriate?




Issues in Relying On RCT Results

Sources of funding & potential conflicts of interest




Moses H, et al. J Am Med Assoc. 2005;294(11):1333-1342.

Med

ical Device/B

iotech

F

irms/P

harm

aceutical




Catovsky D, et al. Lancet. 2007;370(9583):230–239.




Catovsky D, et al. Lancet. 2007;370(9583):230–239.




Association of Funding and Conclusions in Randomized Drug Trial

Als-Neilsen B, et al. J Am Med Assoc. 2003;290(7):921-928.

…trials funded by for-profit organizations were significantly more likely to recommend the experimental drug as

treatment of choice (odds ratio, 5.3) compared with trials funded by nonprofit organizations.




Relationships between authors of clinical practice guidelines and the pharmaceutical industry

Choudhry NK, et al. J Am Med Assoc. 2002;287(5):612-617.






Inappropriate comparison treatments




90% of the studies favored the sponsor’s drug

Contradictory conclusions across studies were found

Potential sources of bias occurred

Heres S, et al. Am J Psychiatry. 2006;163(2):185-194.




This is not an appropriate comparison!

Unequivalent Treatment Conditions

Richter JE, et al. Am J Gastroenterol. 2001;96(3):656–665.




How could this study have been better designed?

(Please enter your answer below)




One option: Compare equivalent doses.

Esomeprazole 20 mg vs. Omeprazole 40 mg.

AA







Inadequate power to identify side effects










Inadequate power to identify side effects

Statistics and publication




Statistical Tricks

• A larger target is easier to hit

• Look at multiple subgroups until one is found that has statistical significance

• Negative primary outcome is often buried, left out of reports or never submitted for publication

Chan A, et al. J Am Med Assoc. 2004;291(20):2457-2565.




Ryan ND. Lancet. 2005;366(9489):933-940.




Publication

Ross JS, et al. J Am Med Assoc. 2008;299(15):1800-1812.




Profit

Scientific Advancement

Patient Care

Competing Interests




Recommendations Realize that data we use to make clinical

decisions may be affected by partisan influences

Develop comfort with applying basic EBM principles to all resources

Identify quality resources for pharmaceutical information

Develop and practice skills for continuous, self-directed, enthusiastic life-long learning




References Abitbol V, Briot K, Roux C, Roy C, Seksik P, Charachon A, Bouhnik Y, Coffin B, Allez M, Lamarque D,

Chaussade S. A double-blind placebo-controlled study of intravenous clodronate for prevention of steroid-induced bone loss in Inflammatory Bowel Disease . Clin Gastroenterol Hepatol. 2007;5(10):1184–1189.

Als-Nielsen B, Chen W, Gluud C, Kjaergard LL. Association of funding and conclusions in randomized drug trials: a reflection of treatment effect or adverse events? J Am Med Assoc. 2003;290(7):921-928.

Avorn J, Chen M, Hartley R. Scientific versus commercial sources of influence on the prescribing behavior of physicians. Am J Med. 1982;73(1):4-8.

Berger JS, Brown DL, Becker RC. Aspirin and stable cardiovascular disease. Am J Med. 2008;121(1):43-49.

Bleck TP. Alternatives to evidence based medicine. Different rating scale could be used. Br Med J. 2000;321(7255):239.

Catovsky D, Richards S, Matutes E, Oscier D, Dyer MJS, Bezares RF, Pettitt AR, Hamblin T, Milligan DW, Child JA, Hamilton MS, Dearden CE, Smith AG, Bosanquet AG, Davis Z, Brito-Babapulle V, Else M, Wade R, Hillmen P. Assessment of fludarabine plus cyclophosphamide for patients with Chronic Lymphocytic Leukaemia (the LRF CLL4 Trial): a randomised controlled trial. Lancet. 2007;370(9583):230–239.

Chan A, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials. J Am Med Assoc. 2004;291(20):2457-2465.

Czeisler CA, Walsh JK, Roth T, Hughes RJ, Wright KP, Kingsbury L, Arora S, Schwartz JRL, Niebler GE, Dinges DF. Modafinil for excessive sleepiness associated with shift-work sleep disorder. N Engl J Med. 2005;353(5):476-486.

Choudhry NK, Stelfox HT, Detsky AS. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. J Am Med Assoc. 2002;287(5):612-617.




References Heres S, Davis J, Maino K, Jetzinger E, Kissling W, Leucht S. Why olanzapine beats risperidone, risperidone

beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry. 2006;163(2):185-194.

Hildesheim A, Herrero R, Wacholder S, Rodriguez A, Solomon D, Bratti MC, Schiller J, Gonzalez P, Dubin G, Porras C, Jimenez S, Lowy D. Effect of Human Papillomavirus 16/18 L1 viruslike particle vaccine among young women with preexisting infection. J Am Med Assoc. 2007;298(7):743-753.

Jackevicius CA, Tu JV, Ross JS, Ko DT, Krumholz HM. Use of ezetimibe in the United States and Canada. N Engl J Med. 2008;358(17):1819-1828.

Kida T. Don’t believe everything you think. 2006; Amherst, NH: Prometheus Books.

Moses H, 3rd, Dorsey ER, Matheson DH, Thier SO. Financial anatomy of biomedical research. J Am Med Assoc. 2005;294(11):1333-1342.

Richter JE, Kahrilas PJ, Johanson J, Maton P, Breiter JR, Hwang C, Marino V, Hamelin B, Levine JG. Efficacy and safety of esomeprazole compared with omeprazole in GERD patients with erosive esophagitis: a randomized controlled trial. Am J Gastroenterol. 2001;96(3):656–665.

Ross JS, Hill KP, Egilman DS, Krumholz HM. Guest authorship and ghostwriting in publications related to rofecoxib: a case study of industry documents from rofecoxib litigation. J Am Med Assoc. 2008;299(15):1800-1812.

Ryan ND. Treatment of depression in children and adolescents. Lancet. 2005;366(9489):933-940.

Sackett D, Straus S, Richardson W, Rosenbert W, Haynes R. Evidence-based medicine: how to practice and teach EBM. 2000; Toronto: Churchill Livingston.

Sprung CL, Annane D, Keh D, Moreno R, Singer M, Freivogel K, Weiss Y, Benbenishty J, Kalenka A, Forst H, Laterre P, Reinhart k, Cuthbertson B, Payen D, Briegel J. Hydrocortisone therapy for patients with septic shock. N Engl J Med. 2008;358(2):111-124.




Resources• Consumer Reports Best Buy Drugs• http://www.consumerreports.org/health/best-buy-drugs/index.htm

• Journal Watch• http://www.jwatch.org/

• Medical Letter• http://www.medicalletter.org/

• Medline (through the U.S. National Library of Medicine)• http://www.ncbi.nlm.nih.gov/sites/entrez?db=pubmed

Attorney General Consumer and Prescriber Education Grant Program Initiated Resources

• Brigham & Women's Hospital http://www.rxfacts.org/

• Georgetown University http://www.pharmedout.org/

• MGH Institute of Health Professionals http://www.perxinfo.org/perx.html

• University of Kentucky http://www.cecentral.com/

• University of North Carolina, Chapel Hill http://harryguess.unc.edu/

• University of Massachusetts Medical School/ http://www.umassmed.edu/meyers/index.aspx Meyers Primary Care Institute




Resources – JAMA User’s Guides

JAMA: User’s Guides to the Medical Literature. This series of 32 articles published in JAMA between 1993 and 2000 dedicated

to specific topics and now collected and expanded in the texts above. Individual articles can be accessed through JAMA.

• Introductions:

• Guyatt GH, Rennie D. Users' guides to the medical literature. JAMA 1993;270(17):2096-97.

• Satya-Murti S. Users' Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice Users' Guides to the Medical Literature: A Manual for Evidence-Based Clinical Practice. JAMA 2002;287(11):1464-6.




• Series:• I. How to get started.

• II. How to use an article about therapy or prevention. A. Are the results of the study valid?

• II. How to use an article about therapy or prevention. B. What were the results and will they help me in caring for my patients?

• III. How to use an article about a diagnostic test. A. Are the results of the study valid?

• III. How to use an article about a diagnostic test. B. What are the results and will they help me in caring for my patients?

• IV. How to use an article about harm.

• V. How to use an article about prognosis.

• VI. How to use an overview.





• VII. How to use a clinical decision analysis. A. Are the results of the study valid?

• VII. How to use a clinical decision analysis. B. What are the results and will they help me in caring for my patients?

• VIII. How to use clinical practice guidelines. A. Are the recommendations valid?

• VIII. How to use clinical practice guidelines. B. What are the recommendations and will they help you in caring for your patients?

• IX. A method for grading health care recommendations.

• X. How to use an article reporting variations in the outcomes of health services.

• XI. How to use an article about a clinical utilization review.

• XII. How to use articles about health-related quality of life.

• XIII. How to use an article on economic analysis of clinical practice. A. Are the results of the study valid?





• XIII. How to use an article on economic analysis of clinical practice. B. What are the results and will they help me in caring for my patients?

• XIV. How to Decide on the Applicability of Clinical Trial Results to Your Patient.

• XV. How to Use an Article About Disease Probability for Differential Diagnosis.

• XVI. How to Use a Treatment Recommendation.

• XVII. How to Use Guidelines and Recommendations About Screening.

• XVIII. How to Use an Article Evaluating the Clinical Impact of a Computer-Based Clinical Decision Support System.

• XIX. Applying Clinical Trial Results; A. How to Use an Article Measuring the Effect of an Intervention on Surrogate End Points.





• XIX. Applying Clinical Trial Results; B. Guidelines for Determining Whether a Drug Is Exerting (More Than) a Class Effect.

• XX. Integrating Research Evidence With the Care of the Individual Patient.

• XXI. Using Electronic Health Information Resources in Evidence-Based Practice.

• XXII: How to Use Articles About Clinical Decision Rules.

• XXIII. Qualitative Research in Health Care A. Are the Results of the Study Valid?

• XXIII. Qualitative Research in Health Care B. What Are the Results and How Do They Help Me Care for My Patients?

• XXIV. How to Use an Article on the Clinical Manifestations of Disease.

• XXV. Evidence-Based Medicine: Principles for Applying the Users' Guides to Patient Care.





Resources

Communicating with Patients

Organizational Influences on Prescribing

Pharmaceutical Development and Regulation

Pharmaceutical Marketing

Provider-Pharmaceutical Representative (PR) Communication Links to Web-Access and Downloadable Versions Available at:

http://www.umassmed.edu/meyers/index.aspx

Additional Resource on RCT Also Available

Additional Learning Modules in This Series




Questions

1. Evidence based medicine relies exclusively on the use of published data to treat patients.

A. True B. False




Answer

1. B: False.

In fact David Sacket, one of the leaders of the evidence-based medicine

movement, defines EBM as "the integration of best research evidence with

clinical expertise and patient values." Note the three components: science,

clinical expertise and patient values.




Questions

2. Trials funded by for-profit organizations were significantly more likely to recommend the experimental drug as treatment of choice.

A. True B. False




Answer

2. A: True.

A study published in JAMA in 2003 found that, compared to those funded by

non-profit organizations, trials funded by for-profit organizations were

significantly more likely to recommend the experimental drug as the

treatment of choice (odds ratio was 5.3).




Questions

3. Clinical trials will often compare a new drug to a little used drug in order to falsely inflate the efficacy of the new drug.

A. True B. False




Answer

3. A: True.

This is one way in which even a randomized clinical trial may not provide the

best evidence for a new treatment. Comparison drugs should be gold

standards or standards of care.




Questions

4. Number needed to treat (NNT) is calculated by 1/Absolute risk reduction.

A. True B. False




Answer

4. A: True.

Number needed to harm (NNH) is another useful calculation when

considering clinical benefit and is represented by 1/absolute risk increase.




5. The dose of medication in the study cited to support fexofenadine’s non-sedating properties is 1/3 that of the tablet dose.

A. True B. False

Questions




Answer

5. A: True.

The study cited in the advertisement text uses a dose of 60 mg for the drug

in question, but the tablets themselves are 180 mg, or 3 times that amount.




6. Interpret the following industry promotion for Ultracet:

A. Ultracet is stronger than Ultram.B. Ultracet is stronger than Acetaminophen.C. Ultracet is stronger than Ibuprofen.D. Relative strength of these drugs cannot be determined

without confidence intervals.

Questions




Answer

6. D: Relative strength of these drugs cannot be determined without

confidence intervals.

The graph is designed to encourage the reader to believe answers A

through C, however without information regarding statistical significance or

confidence intervals we do not know if there is any real difference between

the results reported.




Questions

7. Publication bias may withhold important information from prescribers.

A. True B. False




Answer

7. A: True.

Some funders may choose not to release results because of unfavorable or

insignificant outcomes. Negative data can be helpful to prescribers, but

many companies may not want that information to be public. The example

we used from Lancet was a review of randomized controlled trials of

antidepressants in adolescents that showed many industry-funded trials that

did not show efficacy were not published.




Questions

8. In the strength of evidence pyramid, case control studies have an increased ability to detect causality over cohort studies.

A. True B. False




Answer

8. B: False.

Our hierarchy of evidence has systematic reviews, meta analysis and

randomized trials at the top. Under that, in descending order of quality of

evidence are cohort studies, case-control studies, cross-sectional studies,

case reports, and expert opinion.




Questions

9. A 95% confidence interval:

A. Is significant if it crosses 0.B. Estimates the range of the true effect.C. Has no relationship to the number of patients studied.D. Is stronger with a wider range.




Answer

9. B: Estimates the range of the true effect.

The confidence interval is an estimate of the range within which the true

effect lies. There are two pieces to evaluate with a confidence interval. The

first is the width of the confidence interval. This is determined by the number

of patients studied and the variability within the data. Narrower ranges are

better.

The next point to consider is whether the interval includes the null value.

The null value is the measure that indicates no association between

intervention and outcome. Any confidence interval that includes the null

value allows "chance" as an explanation for the results found.




Questions

10. What factors should you consider in determining whether a study’s results are relevant to your patient?

A. The study was conducted with a similar population.B. The study outcomes are relative to your patient.C. Data suggests the benefits are likely to outweigh the risks.D. All of the above.




Answer

10. D: All of the above.

All of the answers help you to determine whether and how to apply study

results to your patient. Taking the answers one at a time, A) if a study

population is very different from your patient (in age, race, gender, co-

morbidities, etc) your patient may have a different outcome on the same

medication; B) recall that the application of evidence-based medicine

requires that the provider integrate patient values into treatment decisions;

C) as we know, all medications carry some risk, balancing those to

determine whether the benefits outweigh the risks for your patient is an

important step in decision-making.




Questions

11. Did completing this module help you to identify issues that impact published trial results?

A. Yes, definitely B. Yes, probably C. Probably notD. Definitely notE. Not sure

12. Would you recommend this training module to a colleague?A. Yes, definitely B. Yes, probably C. Probably notD. Definitely notE. Not sure




Questions

13. Will you do anything differently in your practice as a result of this training module?

A. Yes, definitely B. Yes, probably C. Probably notD. Definitely notE. Not sure

14. Please tell us about any changes you are considering or planning:

Documents

Click to edit Master title style Click to edit Master subtitle style UMASS MEDICAL SCHOOL MEYERS PRIMARY CARE INSTITUTE The following presentation contains