30
8 TH February 2019 SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

8TH February 2019

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Page 2: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 2

WELCOME, OVERVIEW, AND PARTICIPANTS’ INTRODUCTION

Dr Yvonne ZurynskiAssociate Professor of Health System SustainabilityHonorary A/Prof, Sydney Medical School, The University of SydneyAdjunct A/Prof, Menzies Institute for Medical Research, University of Tasmania

Dr Catalin TufanaruResearch FellowNHMRC Centre of Research Excellence in Digital Health

Dr K-lynn SmithResearch OfficerNHMRC Partnership Centre in Health System Sustainability

Page 3: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• 9.00 am: Welcome, Overview, and Participants’ Introduction• 9.20 am: Session 1: Brief Introduction to systematic reviews (review protocol writing; searching; study

selection; data extraction; critical appraisal; data synthesis; report writing)• 9.55 am: Session 2: The importance of expert opinion and policy documents in healthcare policy and

practice (Session presented by Associate Professor Yvonne Zurynski)• 10.15 am: Morning break• 10.35 am: Session 3: Writing a review protocol for a systematic review of expert opinion and policy

documents • 10.55 am: Session 4: Developing a search strategy and searching for a systematic review of expert

opinion and policy documents [The literature searching process]• 11.15 am: Session 5: Texts/Documents selection for a systematic review of expert opinion and policy

documents (including hands-on practical exercise)• 12.00 pm: Lunch break

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 3

DAY PROGRAM

Page 4: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• 12.30 pm: Session 6: Data extraction for a systematic review of expert opinion and policy documents (including hands-on practical exercise)

• 1.30 pm: Session 7: Critical appraisal for a systematic review of expert opinion and policy documents (including hands-on practical exercise)

• 2.50 pm: Afternoon break• 3.10 pm: Session 8: Data synthesis for a systematic review of expert opinion and policy documents

(including hands-on practical exercise)• 4.10 pm: Session 9: Report writing for a systematic review of expert opinion and policy documents• 4.25 pm: Summative Discussion Session• 5.00 pm: Program Close

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 4

DAY PROGRAM

Page 5: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 5

SESSION 1: BRIEF INTRODUCTION TO SYSTEMATIC REVIEWS

Page 6: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• “A systematic review is a research article that identifies relevant studies, appraises their quality and summarizes their results using a scientific methodology.” [Ref: Khalid Khan, Regina Kunz, Jos Kleijnen and Gerd Antes. Systematic reviews to support evidence based medicine. How to review and apply findings of healthcare research. Second edition. Hodder Arnold, 2011. Page 1]

• “Research Synthesis: A review of a clearly formulated question that uses systematic and explicit methods to identify, select and critically appraise relevant research, and to collect and analyze data from the studies that are included in the synthesis. Statistical methods (meta-analysis) may or may not be used to analyze and summarize the results of the included studies. Systematic review is a synonym of research synthesis.” [Ref: Harris Cooper, Larry V. Hedges, Jeffrey C. Valentine (Editors) The handbook of research synthesis and meta-analysis. Second edition. Russell Sage Foundation, 2009. Page 581]

Systematic Reviews

Australian Institute of Health Innovation

6

SYSTEMATIC REVIEWS – DEFINED

Page 7: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• “Research syntheses focus on empirical research findings and have the goal of integrating past research by drawing overall conclusions (generalizations) from many separate investigations that address identical or related hypotheses. The research synthesist’s goal is to present the state of knowledge concerning the relation(s) of interest and to highlight important issues that research has left unresolved.” [Ref: Harris Cooper. Research Synthesis and Meta-Analysis. A Step-by-Step Approach. Fifth Edition. SAGE, 2017. Page 23]

• “A systematic review attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question. It uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made (Antman 1992, Oxman 1993).” [Ref: Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. Page 6]

Systematic Reviews

Australian Institute of Health Innovation

7

SYSTEMATIC REVIEWS – DEFINED

Page 8: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• “The key characteristics of a systematic review are: a clearly stated set of objectives with pre-defined eligibility criteria for studies; an explicit, reproducible methodology; a systematic search that attempts to identify all studies that would meet the eligibility criteria; an assessment of the validity of the findings of the included studies, for example through the assessment of risk of bias; and a systematic presentation, and synthesis, of the characteristics and findings of the included studies.” [Ref: Green S, Higgins JPT, Alderson P, Clarke M, Mulrow CD, Oxman AD. Chapter 1: Introduction. In: Higgins JPT, Green S (editors), Cochrane Handbook for Systematic Reviews of Interventions. Chichester (UK): John Wiley & Sons, 2008. Page 6]

Systematic Reviews

Australian Institute of Health Innovation

8

THE ESSENTIAL CHARACTERISTICS OF REAL SYSTEMATIC REVIEWS

Page 9: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation

9

SYSTEMATIC REVIEWS VERSUS (TRADITIONAL) LITERATURE REVIEWS

Systematic Reviews (Traditional) Literature Reviews• Informed by an a priori protocol• Clearly articulated objectives and questions• Inclusion and exclusion criteria, stipulated a priori (in the

protocol)• Systematic, exhaustive, comprehensive searching to

identify all relevant studies (published and unpublished)• Critical appraisal of the included studies• Steps taken to reduce bias• Transparent and reproducible methods

(Ref: Aromataris E & Pearson A. The Systematic Review: An Overview. AJN The American Journal of Nursing, 2014;114(3): 47-55.)

• “imprecise in both process and outcome”• “rarely involve systematic procedures to ensure that all

the relevant research was located and included in the synthesis”

• “prone to use post hoc criteria to decide whether individual studies met an acceptable threshold for methodological quality”

• Unreproducible and not transparent

(Ref: Harris Cooper. Research Synthesis and Meta-Analysis. A Step-by-Step Approach. Fifth Edition. SAGE, 2017. Page 24)

Page 10: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

• Review Protocol Development• Review Protocol Registration• Searching [The literature searching process with eight key stages]• Selection or Screening of studies (publications)• Data Extraction• Critical Appraisal (Risk of Bias Assessment)• Data Synthesis• Writing the Systematic Review Report• Publish the report• Disseminate the report

Systematic Reviews

Australian Institute of Health Innovation 10

STAGES/STEPS IN THE SYSTEMATIC REVIEW PROCESS

Page 11: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 11

REPORT WRITING: PRISMA FLOW DIAGRAM

Page 12: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 12

DIFFERENT TYPES OF SYSTEMATIC REVIEWS

• Systematic Reviews (SRs) of primary research studies• SRs of systematic reviews (Umbrella Reviews; Overview of Reviews)• SRs of non-research publications (expert opinion; policy documents; position statements etc.)• Scoping Reviews• ‘Comprehensive’ SRs• Mixed-methods SRs• ‘Living’ SRs• Rapid Reviews

Page 13: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 13

DIFFERENT TYPES OF SYSTEMATIC REVIEWS

Page 14: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 14

DIFFERENT TYPES OF SYSTEMATIC REVIEWS

Page 15: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS

Australian Institute of Health Innovation 15

DIFFERENT TYPES OF SYSTEMATIC REVIEWS

Page 16: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 16

DIFFERENT TYPES OF SYSTEMATIC REVIEWS

Page 17: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 17

REVIEW APPROACHES: UMBRELLA REVIEWS

Page 18: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 18

REVIEW APPROACHES: SCOPING REVIEWS

• Scoping Reviews

Page 19: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 19

REVIEW APPROACHES: RAPID REVIEWS

Page 20: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Review Protocol Development

PRISMA-P: PRISMA for systematic review protocolshttp://www.prisma-statement.org/Extensions/Protocols.aspx

“PRISMA-P was published in 2015 aiming to facilitate the development and reporting of systematic review protocols.” (Source: PRISMA-P Official Website)

Systematic Reviews

Australian Institute of Health Innovation 20

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 21: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Review Protocol Registration

PROSPERO: International prospective register of systematic reviewshttps://www.crd.york.ac.uk/prospero/

“PROSPERO is an international database of prospectively registered systematic reviews in health and social care, welfare, public health, education, crime, justice, and international development, where there is a health related outcome. […] PROSPERO aims to provide a comprehensive listing of systematic reviews registered at inception to help avoid duplication and reduce opportunity for reporting bias by enabling comparison of the completed review with what was planned in the protocol.” (Source: PROSPERO Official Website)

“PROSPERO includes details of any ongoing systematic review of studies in humans or animals that has a health related outcome relevant to human health in the broadest sense. Reviews of clinical studies may be of interventions, service delivery, prognostic factors, risk factors, genetic associations, and epidemiological reviews relevant to health and social care, welfare, public health, education, crime, justice, and international development, as long as there is a health related outcome.” (Source: PROSPERO Official Website)

“Scoping reviews and literature reviews are not eligible for inclusion in PROSPERO.” (Source: PROSPERO Official Website)

Systematic Reviews

Australian Institute of Health Innovation 21

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 22: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Searching

Systematic Reviews

Australian Institute of Health Innovation 22

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 23: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Critical Appraisal (Risk of Bias Assessment)

Systematic Reviews

Australian Institute of Health Innovation 23

STAGES/STEPS AND STANDARDS IN SYSTEMATIC REVIEW PROCESS

Page 24: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Data Synthesis: Narrative Synthesis, Meta-Analysis, Meta-Synthesis

Systematic Reviews

Australian Institute of Health Innovation 24

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 25: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Data Synthesis: Meta-Analysis, Meta-Synthesis

Systematic Reviews

Australian Institute of Health Innovation 25

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 26: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Review Stage: Writing the Systematic Review Report

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses (2009)http://www.prisma-statement.org/

Extensions of the PRISMA Statement http://www.prisma-statement.org/Extensions/Default.aspx

PPRISMA for reviews including harms outcomes (2016)http://www.prisma-statement.org/Extensions/Harms.aspx

“PRISMA is an evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. PRISMA focuses on the reporting of reviews evaluating randomized trials, but can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.” (Source: PRISMA Official Website)

PRISMA extension for Scoping Reviews (PRISMA-ScR): published September 2018

Systematic Reviews

Australian Institute of Health Innovation 26

GUIDANCE AND STANDARDS FOR SYSTEMATIC REVIEW PROCESS

Page 27: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 27

REPORTING STANDARDS

Page 28: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

Systematic Reviews

Australian Institute of Health Innovation 28

POTENTIAL SOLUTIONS: REPORTING STANDARDS

Page 29: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

AMSTAR: measurement tool to assess systematic reviews

Systematic Reviews

Australian Institute of Health Innovation 29

AMSTAR TOOL

Page 30: SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS · 2019-02-07 · SYSTEMATIC REVIEWS OF EXPERT OPINION & POLICY DOCUMENTS Australian Institute of Health Innovation 2 WELCOME,

AMSTAR: measurement tool to assess systematic reviews

Systematic Reviews

Australian Institute of Health Innovation 30

AMSTAR TOOL