R. Heshmat MD; PhD candidate Systematic Review An Introduction

R. Heshmat MD; PhD R. Heshmat MD; PhD

candidatecandidate

Systematic ReviewSystematic Review

An IntroductionAn Introduction


candidatecandidate

ReviewReview

The general term for all attempts to The general term for all attempts to synthesize the results and synthesize the results and conclusions of two or more conclusions of two or more publications on a given topic. A publications on a given topic. A review may or may not be review may or may not be systematic.systematic.


candidatecandidate

Overview=Systematic ReviewOverview=Systematic Review

A review of the evidence on a A review of the evidence on a clearly clearly formulated questionformulated question that uses that uses systematic and explicit methodssystematic and explicit methods to to identify, select and critically appraise identify, select and critically appraise relevant primary research, and to relevant primary research, and to extract and analyze data from the extract and analyze data from the studies that are included in the review. studies that are included in the review.


candidatecandidate

Meta-analysisMeta-analysis

When an systematic review When an systematic review incorporates a specific statistical incorporates a specific statistical strategy for assembling the results of strategy for assembling the results of several studies into a several studies into a single single estimateestimate..


candidatecandidate

A systematic review may, or may A systematic review may, or may not, include meta-analysis.not, include meta-analysis.

Systematic review is always Systematic review is always appropriate and desirable, but it may appropriate and desirable, but it may sometime inappropriate to sometime inappropriate to statistically pool results from statistically pool results from separate studies.separate studies.


candidatecandidate

A single study may produce a false A single study may produce a false negative results, due to inadequate negative results, due to inadequate sample size.sample size.

In the meta-analysis the necessary In the meta-analysis the necessary number of participants can be number of participants can be reached, and small effects can be reached, and small effects can be detected or excluded with detected or excluded with confidence.confidence.


candidatecandidate

Systematic Review:Systematic Review:Epidemiology of resultsEpidemiology of results

Where the findings of an original Where the findings of an original study replace the individual as the study replace the individual as the

unit of analysis.unit of analysis.


candidatecandidate

Narrative reviewsNarrative reviews

The classical review is subjective and The classical review is subjective and therefore prone to bias and error.therefore prone to bias and error.

Selective inclusion of studies that Selective inclusion of studies that support the author’s view is common. support the author’s view is common.

It ignores sample size, effect size, and It ignores sample size, effect size, and research design.research design.

Systematic reviews allow a more Systematic reviews allow a more objectives appraisal. objectives appraisal.


candidatecandidate

Historical notesHistorical notes

The statistical basis of meta-analysis The statistical basis of meta-analysis reaches back to the 17reaches back to the 17thth century in century in astronomy and geodesy.astronomy and geodesy.

In 1976 the psychologist Glass In 1976 the psychologist Glass coined the term “meta-analysis”.coined the term “meta-analysis”.

In the 1980s meta-analysis became In the 1980s meta-analysis became increasingly popular in medicine.increasingly popular in medicine.


candidatecandidate

Finally:Finally:

Cochrane Collaboration was held in Cochrane Collaboration was held in Oxford Oxford

in in October 1993October 1993..

The Collaboration aims to help people The Collaboration aims to help people make well-informed decision about make well-informed decision about health care by preparing, maintaining health care by preparing, maintaining and promoting the accessibility of and promoting the accessibility of systematic reviewsystematic review. .


candidatecandidate

Principles of and procedures for Principles of and procedures for systematic reviewsystematic review

Systematic reviews should be as Systematic reviews should be as carefully planned as any other carefully planned as any other

research project, with a detailed research project, with a detailed written protocol in advance. written protocol in advance.


candidatecandidate

Stage 1

Planning the review

Stage 2

Conducting a review

Stage 3

Reporting & dissemination


candidatecandidate

Stage 1

Planning the review

Phase 0

Identification of the need for a review

Phase 1

Preparation of a proposal for a review

Phase 2

Development of a review protocol


candidatecandidate

Stage 1

Planning the review

Phase 0


Phase 1


Phase 2



candidatecandidate

Stage 1

Planning the review

Phase 0


Phase 1


Phase 2



candidatecandidate

Stage 1

Planning the review

Phase 0


Phase 1


Phase 2



candidatecandidate

Stage 2

Conducting a review

Phase 3

Identification of research

Phase 4

Selection of studies

Phase 5

Study quality assessment

Phase 6

Data extraction and monitoring progress

Phase 7

Data synthesis


candidatecandidate

Stage 2

Conducting a review

Phase 3


Phase 4


Phase 5


Phase 6


Phase 7

Data synthesis


candidatecandidate

Stage 2

Conducting a review

Phase 3


Phase 4


Phase 5


Phase 6


Phase 7

Data synthesis


candidatecandidate

Stage 2

Conducting a review

Phase 3


Phase 4


Phase 5


Phase 6


Phase 7

Data synthesis


candidatecandidate

Stage 2

Conducting a review

Phase 3


Phase 4


Phase 5


Phase 6


Phase 7

Data synthesis


candidatecandidate

Stage 2

Conducting a review

Phase 3


Phase 4


Phase 5


Phase 6


Phase 7

Data synthesis


candidatecandidate

Stage 3


Phase 8

The report and recommendations

Phase 9

Getting evidence into practice


candidatecandidate

Stage 3


Phase 8


Phase 9



candidatecandidate

Stage 3


Phase 8


Phase 9



candidatecandidate

Developing a Protocol for Developing a Protocol for a Systematic Reviewa Systematic Review


candidatecandidate

Role of the protocolRole of the protocol

A written document that forms the ’plan’ for the review

A protocol helps to avoid or minimise bias

Bias may occur in the retrieval, selection, extraction of data and evaluation of results

A protocol can (and should be) sent for external peer review


candidatecandidate

Components of a Components of a protocolprotocol

BackgroundBackground Review questions/ ObjectivesReview questions/ Objectives Search strategySearch strategy Study selection criteria and Study selection criteria and

proceduresprocedures Study quality assessment checklists Study quality assessment checklists

and proceduresand procedures


candidatecandidate

Components of a Components of a protocolprotocol

Data extraction strategyData extraction strategy Data synthesis strategyData synthesis strategy Project timetableProject timetable


candidatecandidate

Background sectionBackground section

Patients / disease characteristicsPatients / disease characteristics Course of diseaseCourse of disease PathophysiologyPathophysiology InterventionsInterventions


candidatecandidate

Research questions/ Research questions/ objectivesobjectives

Population/ participantsPopulation/ participants InterventionsInterventions OutcomesOutcomes Study designsStudy designs


candidatecandidate

Literature searchLiterature search

Search strategy should specify which Search strategy should specify which databases and other sources will be databases and other sources will be searchedsearched

Based on components of review Based on components of review questionsquestions

Do not need to present detailed Do not need to present detailed search strategies in protocolsearch strategies in protocol


candidatecandidate

Possible sources of Possible sources of literatureliterature

Electronic databasesElectronic databases Medline, Embase, PsycLIT, CINAHLMedline, Embase, PsycLIT, CINAHL specialist trial registers e.g. CCTRspecialist trial registers e.g. CCTR

HandsearchingHandsearching Checking reference listsChecking reference lists Personal communicationPersonal communication Pharmaceutical companiesPharmaceutical companies Grey literatureGrey literature


candidatecandidate

Study selection criteriaStudy selection criteria

Should follow from research questionsShould follow from research questions i.e. population, interventions, i.e. population, interventions,

outcomes, study designoutcomes, study design Inclusion and exclusion criteriaInclusion and exclusion criteria Details of selection process should be Details of selection process should be

set out here (how many reviewers, set out here (how many reviewers, how are disagreements resolved?)how are disagreements resolved?)


candidatecandidate

Study quality assessmentStudy quality assessment

Purpose of quality assessment?Purpose of quality assessment? For selection? For data synthesis? For For selection? For data synthesis? For

implications of results?implications of results? Choose appropriate checklist (related Choose appropriate checklist (related

to study design)to study design) Details of assessment process (how Details of assessment process (how

many reviewers, disagreements etc)many reviewers, disagreements etc)


candidatecandidate

Data extraction strategyData extraction strategy

Think about what data you need to Think about what data you need to extract from included studies to extract from included studies to answer the questionsanswer the questions

Pilot a draft data extraction formPilot a draft data extraction form Agree process (how many reviewers, Agree process (how many reviewers,

disagreements etc)disagreements etc) Any manipulation of study data to be Any manipulation of study data to be

reported herereported here


candidatecandidate

Data synthesisData synthesis

Will results be pooled? How?Will results be pooled? How? How will differences between studies How will differences between studies

be taken into account?be taken into account? Proposed sensitivity analyses? Proposed sensitivity analyses?

Subgroups?Subgroups? How will results be displayed?How will results be displayed? May not be possible to be specific at May not be possible to be specific at

protocol stageprotocol stage


candidatecandidate

DisseminationDissemination

How will you publish it? In what How will you publish it? In what format?format?


candidatecandidate

TimescaleTimescale

Set out key milestones (e.g. Set out key milestones (e.g. searching, study selection, data searching, study selection, data extraction, draft report to peer extraction, draft report to peer review, final report)review, final report)

Some stages may overlapSome stages may overlap An end date for the review is the An end date for the review is the

most important!most important!


candidatecandidate

Limitations of Systematic Limitations of Systematic ReviewReview

Reporting bias and the inadequate quality Reporting bias and the inadequate quality of primary research are potentially serious of primary research are potentially serious problems for systematic reviews.problems for systematic reviews.

The quality of component studies is of The quality of component studies is of crucial importance.crucial importance.

The dissemination of research findings is The dissemination of research findings is not a random process; rather it is strongly not a random process; rather it is strongly influenced by the nature and direction of influenced by the nature and direction of results.results.


candidatecandidate

Type of reporting biasType of reporting bias

Publication biasPublication bias The publication or The publication or non-publication of non-publication of research findings, research findings, depending on the depending on the nature and nature and direction of the direction of the resultsresults


candidatecandidate

Time lag biasTime lag bias The rapid or The rapid or delayed publication delayed publication of research finding, of research finding, depending on the depending on the nature and nature and direction of the direction of the resultsresults


candidatecandidate

Multiple (duplicate) Multiple (duplicate) publication biaspublication bias

The multiple or The multiple or singular publication singular publication of research finding, of research finding, depending on the depending on the nature and nature and direction of the direction of the resultsresults


candidatecandidate

Citation bias Citation bias The citation or non-The citation or non-citation of researchcitation of research

finding, depending finding, depending on the nature and on the nature and direction of the direction of the resultsresults


candidatecandidate

Language biasLanguage bias The publication of The publication of research finding in research finding in a particular a particular language, language, depending on the depending on the nature and nature and direction of the direction of the resultsresults


candidatecandidate

Outcome reporting Outcome reporting biasbias

The selective The selective reporting outcomes reporting outcomes but not of others , but not of others , depending on the depending on the nature and nature and direction of the direction of the resultsresults


candidatecandidate

The inclusion of data from The inclusion of data from unpublished studies can itself unpublished studies can itself introduce bias.introduce bias.

Unpublished studies may be of lower Unpublished studies may be of lower methodological quality than methodological quality than published studies.published studies.

Documents

R. Heshmat MD; PhD candidate Systematic Review An Introduction