EBM: Critical Appraisal of an Article on Therapy

EBM: Critical Appraisal of an Article on TherapySudan Evidence-Based Medicine Association TemplateDeveloped by: Dr. Imad Salah Ahmed Hassan MD (UK) FACP FRCPI MAcadMEd MSc (UK) MBBS (U of K)

2

Clinical Scenario

Formulate a Well-Built Clinical QuestionPatientInterventionComparisonOutcome

Outline your Search Strategy:Search Terms/Keywords/Mesh termsSearch site/Search EnginePubmed via PICOPubmed Search Box & LimitsPubmed Clinical QueriesPubmed Advanced SearchCochrane LibraryOther

5

Outline Current PracticeTip: From the Introduction Section or from a Guideline

6

Select a Reasonable Piece of Research

Tip:Paste the title, authors, journal details hereType of Study: RCT Retrospective Cohort Case-Control Other:Tip: RCT>cohort>case control>case series Date published:Number of Pages:Quality of journal i.e. impact factor (get from Google!)Linked-In: Author Details/Picture

Reason(s) for Selecting this Piece of ResearchTip: Common ReasonsAnswers my QUESTIONFree articleRecent ArticleReputable Journal Other

Summarize the Results of your ArticleTip: Paste the Conclusion from the Abstract here

9

Apply Your Appraisal SchemeTip: Use the RVRA Scheme: Based on the National & Gulf Center for Evidence-based Health PracticeRelevanceValidityResultsApplicability

Apply Your Appraisal SchemeTip: Use the RVRA SchemeRelevancePersonal KnowledgeValidityMethods SectionResults (Impact)Results SectionApplicabilityPersonal Interpretation/Experience

Apply Your Appraisal SchemeRelevanceDoes the study address a common problem in your practice? Does the study address an important outcome to you or to your patient? (DOE vs. POEM) Assuming that the study conclusion is true; would it lead to a change in your practice?

12

Apply Your Appraisal SchemeValidityDescribe the Research DesignTip: Name the Study Design

Apply Your Appraisal SchemeValidityDescribe the Selection Criteria, Interventions, Outcome Measures & Statistical Tests usedTip: From Method Section or Copy the Method Summary from the Abstract.Copy the Inclusion & Exclusion CriteriaCopy the Study Flow Chart to your PresentationCopy the Inclusion & Exclusion CriteriaInterventions

Outcome Measures: Primary & SecondaryStatistical Tests (Appropriateness)

Apply Your Appraisal SchemeValidityAnswer the following Questions:Was the assignment of patients to treatment randomized?Was the assignment concealed? Method of Randomization: Describe Selection bias: before intervention starts Were the groups similar at the start of the trial? Baseline prognostic factors (demographics, co-morbidity, disease severity, other known confounders) balanced? If different, were these adjusted for? Tip: Copy Table 1 (usually describes the different groups in the study).

15

Apply Your Appraisal SchemeValidityWere patients, their clinicians, and study personnel 'blind' to treatment? If NO Was it possible Can this induce bias Levels of blinding: Open-label, Single, Double-blind, Triple-blind etc. Aside from the experimental intervention, were the groups treated equally?Performance bias: Co-intervention Contamination Compliance

16

ValidityWere all patients who entered the trial properly accounted for and attributed at its conclusion? (ITT & Follow-up)Intention-to-Treat ITT: Were patients analyzed in the groups to which they were randomized (intention to treat analysis)? Attrition Bias: Was follow-up complete & long enough? Sensitivity Analysis (WCS) * Were all clinically important outcomes considered?* Baseline characteristics (Prognostic Factors) influence on Outcomes?Are substitute endpoints/surrogate tests valid? Was the outcome assessor blind (Ascertainment/Detection bias or Measurement bias) Clinical OutcomesQuality of LifeHarm/Side-effects etc. Overall, is the method of the study valid? [ ] Yes (100%) [ ] No (0%) [ ] In-between ( ___%) [ ] Can't Tell

Apply Your Appraisal SchemeResults clinical significance (in MP : Magnitude & Precision) Magnitude of the effect: How large was the treatment effect & Specified Time Factor (Duration of Study)? Risk difference (RD or ARR) Risk Ratio (RR) Relative risk reduction (RRR) NNT NNT for different outcomes NNH Precision of the effect: How precise was the estimate of the treatment effect? Confidence intervals? Narrow or Wide

18

Summarize the Study BiasesSelection, Attrition, Performance, Detection Biases, Conflict of Interest etc.

Summarize the Study StrengthTip: From the Discussion or Letter to the Editor/Editorial Comment etc.E.G.: Number of Patients, Randomization Method, Least Biases, Statistical Tests used, Outcome Measures, Ascertainment, Drop-outs, Length of Follow-up etc.

Summarize the Study WeaknessesTip: From the Discussion or Letter to the Editor/Editorial Comment etc.E.G.: Number of Patients, Randomization Method, Least Biases, Statistical Tests used, Outcome Measures, Ascertainment, Drop-outs, Length of Follow-up etc.

21

Summarize the StudyTip: From the Discussion or Letter to the Editor/Editorial Comment etc.

22

Unanswered Questions from the StudyTip: From the Discussion or Letter to the Editor/Editorial Comment etc.

Apply Your Appraisal SchemeApplicability I (in IPPP: Intervention Population - Preferences-Potential Benefits - Harm Can the results be applied to my patient care? Can you do the Intervention exactly as it is described in the paper Is your Patient similar to the population of the study? Did you examine your patients Preferences? Are the potential benefits outweigh the potential harms & cost, to my patient? Availability-feasibility-cost-side effects Patients similar for demographics, severity, co-morbidity and other prognostic factors? (as per Inclusion & Exclusion Criteria)Compelling reason why the results should not be applied?

25

Resolution of your PICO Question

Narrate/Write your PICO Question & Why is it important.Summarize the Study FindingsSummarize the weaknesses, biases, NNT and NNH etc.Indicate the Level of Evidence & whether you are going to apply the findings to your patient.

26

Useful LinksEBM Glossary of Terms: http://www.cebm.net/glossary/EBM Calculators: http://ktclearinghouse.ca/cebm/practise/ca/calculatorsEBM NNT, LR etc. Websites:http://ktclearinghouse.ca/cebm/toolboxhttp://www.thennt.com/EBM Critical Appraisal Worksheets: http://ktclearinghouse.ca/cebm/practise/ca/worksheetsDiagnosis worksheetHarm worksheetPrognosis worksheetSystematic review (of therapy) worksheetTherapy worksheet

27

28