R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Systematic ReviewSystematic Review
An IntroductionAn Introduction
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
ReviewReview
The general term for all attempts to The general term for all attempts to synthesize the results and synthesize the results and conclusions of two or more conclusions of two or more publications on a given topic. A publications on a given topic. A review may or may not be review may or may not be systematic.systematic.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Overview=Systematic ReviewOverview=Systematic Review
A review of the evidence on a A review of the evidence on a clearly clearly formulated questionformulated question that uses that uses systematic and explicit methodssystematic and explicit methods to to identify, select and critically appraise identify, select and critically appraise relevant primary research, and to relevant primary research, and to extract and analyze data from the extract and analyze data from the studies that are included in the review. studies that are included in the review.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Meta-analysisMeta-analysis
When an systematic review When an systematic review incorporates a specific statistical incorporates a specific statistical strategy for assembling the results of strategy for assembling the results of several studies into a several studies into a single single estimateestimate..
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
A systematic review may, or may A systematic review may, or may not, include meta-analysis.not, include meta-analysis.
Systematic review is always Systematic review is always appropriate and desirable, but it may appropriate and desirable, but it may sometime inappropriate to sometime inappropriate to statistically pool results from statistically pool results from separate studies.separate studies.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
A single study may produce a false A single study may produce a false negative results, due to inadequate negative results, due to inadequate sample size.sample size.
In the meta-analysis the necessary In the meta-analysis the necessary number of participants can be number of participants can be reached, and small effects can be reached, and small effects can be detected or excluded with detected or excluded with confidence.confidence.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Systematic Review:Systematic Review:Epidemiology of resultsEpidemiology of results
Where the findings of an original Where the findings of an original study replace the individual as the study replace the individual as the
unit of analysis.unit of analysis.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Narrative reviewsNarrative reviews
The classical review is subjective and The classical review is subjective and therefore prone to bias and error.therefore prone to bias and error.
Selective inclusion of studies that Selective inclusion of studies that support the author’s view is common. support the author’s view is common.
It ignores sample size, effect size, and It ignores sample size, effect size, and research design.research design.
Systematic reviews allow a more Systematic reviews allow a more objectives appraisal. objectives appraisal.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Historical notesHistorical notes
The statistical basis of meta-analysis The statistical basis of meta-analysis reaches back to the 17reaches back to the 17thth century in century in astronomy and geodesy.astronomy and geodesy.
In 1976 the psychologist Glass In 1976 the psychologist Glass coined the term “meta-analysis”.coined the term “meta-analysis”.
In the 1980s meta-analysis became In the 1980s meta-analysis became increasingly popular in medicine.increasingly popular in medicine.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Finally:Finally:
Cochrane Collaboration was held in Cochrane Collaboration was held in Oxford Oxford
in in October 1993October 1993..
The Collaboration aims to help people The Collaboration aims to help people make well-informed decision about make well-informed decision about health care by preparing, maintaining health care by preparing, maintaining and promoting the accessibility of and promoting the accessibility of systematic reviewsystematic review. .
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Principles of and procedures for Principles of and procedures for systematic reviewsystematic review
Systematic reviews should be as Systematic reviews should be as carefully planned as any other carefully planned as any other
research project, with a detailed research project, with a detailed written protocol in advance. written protocol in advance.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 1
Planning the review
Stage 2
Conducting a review
Stage 3
Reporting & dissemination
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 1
Planning the review
Phase 0
Identification of the need for a review
Phase 1
Preparation of a proposal for a review
Phase 2
Development of a review protocol
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 1
Planning the review
Phase 0
Identification of the need for a review
Phase 1
Preparation of a proposal for a review
Phase 2
Development of a review protocol
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 1
Planning the review
Phase 0
Identification of the need for a review
Phase 1
Preparation of a proposal for a review
Phase 2
Development of a review protocol
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 1
Planning the review
Phase 0
Identification of the need for a review
Phase 1
Preparation of a proposal for a review
Phase 2
Development of a review protocol
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 2
Conducting a review
Phase 3
Identification of research
Phase 4
Selection of studies
Phase 5
Study quality assessment
Phase 6
Data extraction and monitoring progress
Phase 7
Data synthesis
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 3
Reporting & dissemination
Phase 8
The report and recommendations
Phase 9
Getting evidence into practice
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 3
Reporting & dissemination
Phase 8
The report and recommendations
Phase 9
Getting evidence into practice
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Stage 3
Reporting & dissemination
Phase 8
The report and recommendations
Phase 9
Getting evidence into practice
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Developing a Protocol for Developing a Protocol for a Systematic Reviewa Systematic Review
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Role of the protocolRole of the protocol
A written document that forms the ’plan’ for the review
A protocol helps to avoid or minimise bias
Bias may occur in the retrieval, selection, extraction of data and evaluation of results
A protocol can (and should be) sent for external peer review
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Components of a Components of a protocolprotocol
BackgroundBackground Review questions/ ObjectivesReview questions/ Objectives Search strategySearch strategy Study selection criteria and Study selection criteria and
proceduresprocedures Study quality assessment checklists Study quality assessment checklists
and proceduresand procedures
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Components of a Components of a protocolprotocol
Data extraction strategyData extraction strategy Data synthesis strategyData synthesis strategy Project timetableProject timetable
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Background sectionBackground section
Patients / disease characteristicsPatients / disease characteristics Course of diseaseCourse of disease PathophysiologyPathophysiology InterventionsInterventions
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Research questions/ Research questions/ objectivesobjectives
Population/ participantsPopulation/ participants InterventionsInterventions OutcomesOutcomes Study designsStudy designs
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Literature searchLiterature search
Search strategy should specify which Search strategy should specify which databases and other sources will be databases and other sources will be searchedsearched
Based on components of review Based on components of review questionsquestions
Do not need to present detailed Do not need to present detailed search strategies in protocolsearch strategies in protocol
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Possible sources of Possible sources of literatureliterature
Electronic databasesElectronic databases Medline, Embase, PsycLIT, CINAHLMedline, Embase, PsycLIT, CINAHL specialist trial registers e.g. CCTRspecialist trial registers e.g. CCTR
HandsearchingHandsearching Checking reference listsChecking reference lists Personal communicationPersonal communication Pharmaceutical companiesPharmaceutical companies Grey literatureGrey literature
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Study selection criteriaStudy selection criteria
Should follow from research questionsShould follow from research questions i.e. population, interventions, i.e. population, interventions,
outcomes, study designoutcomes, study design Inclusion and exclusion criteriaInclusion and exclusion criteria Details of selection process should be Details of selection process should be
set out here (how many reviewers, set out here (how many reviewers, how are disagreements resolved?)how are disagreements resolved?)
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Study quality assessmentStudy quality assessment
Purpose of quality assessment?Purpose of quality assessment? For selection? For data synthesis? For For selection? For data synthesis? For
implications of results?implications of results? Choose appropriate checklist (related Choose appropriate checklist (related
to study design)to study design) Details of assessment process (how Details of assessment process (how
many reviewers, disagreements etc)many reviewers, disagreements etc)
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Data extraction strategyData extraction strategy
Think about what data you need to Think about what data you need to extract from included studies to extract from included studies to answer the questionsanswer the questions
Pilot a draft data extraction formPilot a draft data extraction form Agree process (how many reviewers, Agree process (how many reviewers,
disagreements etc)disagreements etc) Any manipulation of study data to be Any manipulation of study data to be
reported herereported here
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Data synthesisData synthesis
Will results be pooled? How?Will results be pooled? How? How will differences between studies How will differences between studies
be taken into account?be taken into account? Proposed sensitivity analyses? Proposed sensitivity analyses?
Subgroups?Subgroups? How will results be displayed?How will results be displayed? May not be possible to be specific at May not be possible to be specific at
protocol stageprotocol stage
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
DisseminationDissemination
How will you publish it? In what How will you publish it? In what format?format?
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
TimescaleTimescale
Set out key milestones (e.g. Set out key milestones (e.g. searching, study selection, data searching, study selection, data extraction, draft report to peer extraction, draft report to peer review, final report)review, final report)
Some stages may overlapSome stages may overlap An end date for the review is the An end date for the review is the
most important!most important!
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Limitations of Systematic Limitations of Systematic ReviewReview
Reporting bias and the inadequate quality Reporting bias and the inadequate quality of primary research are potentially serious of primary research are potentially serious problems for systematic reviews.problems for systematic reviews.
The quality of component studies is of The quality of component studies is of crucial importance.crucial importance.
The dissemination of research findings is The dissemination of research findings is not a random process; rather it is strongly not a random process; rather it is strongly influenced by the nature and direction of influenced by the nature and direction of results.results.
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Type of reporting biasType of reporting bias
Publication biasPublication bias The publication or The publication or non-publication of non-publication of research findings, research findings, depending on the depending on the nature and nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Time lag biasTime lag bias The rapid or The rapid or delayed publication delayed publication of research finding, of research finding, depending on the depending on the nature and nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Multiple (duplicate) Multiple (duplicate) publication biaspublication bias
The multiple or The multiple or singular publication singular publication of research finding, of research finding, depending on the depending on the nature and nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Citation bias Citation bias The citation or non-The citation or non-citation of researchcitation of research
finding, depending finding, depending on the nature and on the nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Language biasLanguage bias The publication of The publication of research finding in research finding in a particular a particular language, language, depending on the depending on the nature and nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
Outcome reporting Outcome reporting biasbias
The selective The selective reporting outcomes reporting outcomes but not of others , but not of others , depending on the depending on the nature and nature and direction of the direction of the resultsresults
R. Heshmat MD; PhD R. Heshmat MD; PhD
candidatecandidate
The inclusion of data from The inclusion of data from unpublished studies can itself unpublished studies can itself introduce bias.introduce bias.
Unpublished studies may be of lower Unpublished studies may be of lower methodological quality than methodological quality than published studies.published studies.