1 An Introduction to Systematic Reviews. 2 Information explosion

1

An Introduction to Systematic Reviews

2

Information explosion

3

Daily

•46 RCTs

•1000 Medline New articles

•6,000 biomedical articles

4

3 million articles30,000 journals

Annually

5

Increasing Knowledge

0

2000

4000

6000

8000

1966 1976 1986 1996

Number of articles on Hypertension cited in Medline by Year

Articles

6

Most research published in medical journals is

too poorly done

or

insufficiently relevant

to be clinically useful

7

Too much information, too little time

• There is simply too much information around for people to keep up to date.

• On top of this, high quality information is often not easy to find.

8

Review articles

9

• A ‘review’ is the generic term for any attempt to synthesis the results and conclusions of two or more publications on a given topic.

10

Some reviews are usually based on narrative or commentary and are produced by a

‘content expert’

11

What’s the problem with

“Expert Opinion”?

12

Professor Paul Knipschild has described how Nobel prize winning biochemist Linus Pauling

used selective quotes from the medical literature to "prove" his theory that

vitamin C helps you live longer and feel better.

13

When Knipschild and his colleagues searched the literature systematically for evidence

They found that

14

• One or two trials did strongly suggest that vitamin C could prevent the onset of the common cold.

• There were far more studies which did not show any beneficial effect.

15

Unfortunately, expert reviewers often :

• Make conflicting recommendations

• Their advice frequently lags behind

• Inconsistent with the best available evidence.

16

The use of unsystematic approaches to collecting and summarizing the evidence.

17

In one study, self-rated expertise was inversely related to the methodologic rigor of the review

Oxman AD, Guyatt GH. The science of reviewing research. Ann N Y Acad Sci. 1993;703:125-133; discussion 133-134.

18

The problems are that …

• Too much information, too little time

• Many too poorly done or insufficiently relevant to be clinically useful

• Many have conflicting results

• On top of these, high quality information is often not easy to find

19

Timely response

• A classic example: streptokinase in myocardial infarction – First trial published in 1959

– 15 trials published up to 1977

– Introduced as normal practice in late 80s

– Meta-analysis of treatments in 1992: fall in mortality by 1977, after inclusion of 15 trials

20

21

5

101

1 2

8

7

8

12

4

3

1

1

2

8

7

2

1

1

1

2

8

1

5

15

6

No

t M

en

tio

ne

d

Ro

uti

ne

Ex

pe

rim

en

tal

Ra

re/N

ev

er

Sp

ec

ific

M

M

M

M

M

M

Textbook/ReviewRecommendations

Odds Ratio (Log Scale)

0.5 1.0 2.0

Favours Treatment Favours Control

RCTs Pts

1 23

2 65

3 149

4 316

7 1793

10 254411 265115 331117 392922 5452

P<.01

23 5767

27 612530 634633 657143 21 05954 22 051

67 47 53165 47 185

70 48 154

P<.001

P<.00001

CumulativeYear

1960

1965

1970

1980

1985

1990

Thrombolytic Therapy

21

Bias

Bias means something that will cause a consistent deviation from the truth.

This is different from the play of chance.

22

There are three possible sources of bias in reviews:

• bias arising from the studies included in the review

• bias arising from the studies not included in the review

• Bias arising from the way the review is done.

23

Minimizing bias

We need to do as much as possible to minimize the effects of anything that will cause the results to deviate from the truth.

24

What is a Systematic Review?

25

Systematic review Or Overview

Comprehensively

• locates

• evaluates

• synthesizes

all the available literature on a given topic

using a strict scientific design which

must itself be reported in the review

26

A ‘systematic review’, therefore, aims to be:

• Systematic (e.g. in its identification of literature)

• Explicit (e.g. in its statement of objectives, materials and methods)

• Reproducible (e.g. in its methodology and conclusions)

27

The ‘systematic’ part of systematic reviews is all about

minimizing bias in the way

the review is carried out

28

Systematic reviews are the same as ordinary reviews, only bigger!

• not simply "comprehensive"

• but to answer a specific question

• to reduce bias in the selection and inclusion of studies (language, database, publication, reporting, citation, multiple publication)

• to appraise the quality of the included studies

– Internal validity: minimised systematic error (bias)

External validity: generalisability of findings

• to summarise them objectively

29

They are different!!

30

The UK Medical Research Council now requires a systematic review has been prepared before it will commission a new trial.

• The question has not already been answered

• The results of previous research are used in designing the new trial.

31

The Cochrane Collaboration

www.cochrane.org

32

“It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty,

adapted periodically, of all relevant randomised controlled trials.”

Archie Cochrane

33

Section 2

34

Conducting a Systematic Review

35

Stages of a systematic review

• Planning the review – i.e. identifying the need for a review, and documenting the methodology

• Conducting the review – i.e. finding, selecting, appraising, extracting and synthesising primary research studies

• Reporting and dissemination – i.e. writing up and disseminating the results of the review

36

Development of a review protocol

37

Formulating review questions

Searching & selecting studies

Study quality assessment

Data synthesis

Extracting data from studies

38

Step 0

انجام اين مطالعات به پول، زمان و نيروی •متخصص نياز دارند.

بندي مداخالت و به منظور اولويت• الزم بود شيوع HIVهاي مربوط به پژوهش

ي و قدرت اثرهريك ازعوامل تعيين كنندهاين بيماري محاسبه گردد.

39


• The first and most important decision in preparing a review is to determine its focus

• This is best done by asking clearly framed questions.

40


• define a four part clinical question

breaking the question down into its component parts

41

Question components : PICO

• What types of Participants?

• What types of Interventions?

• What types of Comparison?

• What types of Outcomes?

42

A PICO questionA PICO question

What is the best strategy to prevent smoking in young people?

Time-consuming question:

43

An answerable questionAn answerable question

Q. Are mass media (or school-based or community-based) interventions effective in preventing smoking in young people?

44

How are these questions different?

• Does aspirin improve survival after acute myocardial infarction?

• In patients with acute myocardial infarction, does daily, low-dose, oral aspirin lead to higher survival rates as compared to placebo?

45

Formulation of a therapy question

Is Zinc effective in treating cold?

In children with common cold, is oral Zinc effective in reducing the duration of symptoms, as compared to placebo?

Intervention Outcome

InterventionPatient/problem

Outcome Comparison+ RCTs

46

سواالت مرور سيستماتيك

شيوع عدم آگاهي در مورد راه هاي انتقال •HIV رفتار پر خطر جنسي و اعتياد تزريقي ،

14در گروه هاي مختلف مردم ايراني باالي سال اخير چه ميزان مي باشد؟10سال در

بزرگي اثر هر يك از عوامل فوق در آلودگي • در گروه هاي مختلف مردم ايراني HIVبا

سال چگونه است؟14باالي

47




Data synthesis


48

Selecting studies

• performing a comprehensive, objective, and reproducible search of the literature

• selecting studies which meet the original inclusion and exclusion criteria

can be the most time-consuming and challenging task in preparing a systematic review

49

Data sources for a systematic review• Electronic databases

– MEDLINE and EMBASE

– Specialized or local databases (CINAHL, CENTRAL, AMED,…)

• Hand searching• “Grey literature” ( thesis, Internal reports, pharmaceutical industry files)

• Checking reference lists• Unpublished sources known to experts in the specialty (seek by

personal communication)

• Raw data from published trials

50

Developing a search strategy

• It is always necessary to strike a balance between comprehensiveness and precision when developing a search strategy.

51

Selecting studies

• assessing titles and abstracts

• full text

52

Searching the literature

• Published and unpublished literature

• unbiased search (not just Medline which contains < half of trials)

• conference papers, company reports, et.

• personal communication with experts

53

شرايط انتخاب نوع مطالعه• توصيفي

توصيفي-تحليلي تحليلي

سال ايراني14مردان يا زنان باالي تحت مطالعه جمعيت• _ جهان مديترانه شرقي_ايران مكان مطالعه•

مطالعه • زماناخير 10 سال

HIVآگاهي و نگرش نسبت به انتقال يا پيشگيري از محتوي مطالعه• اثر آموزش بر آگاهي يا نگرش نسبت به راه هاي انتقال

HIV اعتياد تزريقي

رفتار پر خطر جنسي كيفيت قابل قبول مطالعات•

54

جستجوي مقاالت و اسناد

•NGOهانظر مصاحبه با افراد صاحب•

وكليدي(UN-WHOسازمان هاي جهاني)•

منابع جستجو:•IRAN MEDEX•IRAN DOCچكيده مقاالت كنگره ها•پايان نامه ها•مجالت داخلي و خارجي•مراكز دولتي )وزارت بهداشت، •

مطبوعات، بهزيستي، وزارت ارشاد، مركزمديريت بيماريها،

، مراكز تحقيقاتي(زندان•Cochrane Library•MEDLINE•EMBASEبررسي مراجع مقاالت يافت شده•

55

لغات كليدي مورد استفاده

آموزش• نگرش• آگاهي• ايدز•سندرم نقص ايمني اكتسابي•شيوع•/ عوامل تعيين كنندهعوامل خطر /• ايران•اعتياد تزريقي، سوء مصرف •

تزريقي مواد، اعتيادرفتار پر خطر جنسي، روسپي •

گري، بيماري هاي منتقله از راه جنسي، زنان ويژه

قدرت اثر، نسبت شانس، خطر •نسبي، نسبت خطر، بزرگي اثر

• AIDS• HIV

• Knowledge• Attitude• Knowledge/ Attitude/practice• Risk factors• Intra venous drug use (IDU)• High risk sex• Sex workers, heterosexual,

homosexual, MSM• STI• IRAN• Prevalence• relative risk,odds ratio, effect

56

مرور خالصه مقاالت

مرور خالصه مقاالت توسط دو نفر به •صورت مستقل و انتخاب با توجه به

صورت گرفت. معيارهاي انتخاب 93 خالصه مقاله 1000در نهايت از بيش از •

مطالعه انتخاب شدند. مطالعه از نظر 93سپس متن كامل اين •

كيفيت و معيار هاي انتخاب مورد بررسي HIV assessment form.docقرار گرفت

57

Experts in a particular area frequently have pre-formed opinions that can bias their assessments of both the relevance and validity of articles

58




Data synthesis


59

Assessment of study quality

• Assess each study for:

– eligibility for inclusion

– study quality

– reported findings

• Ideally will involve two independent reviewers

60

Assessment of study quality

• Validity: the degree to which the trial design, conduct, analysis, and presentation have minimized or avoided systematic biases.

61

'Quality' scales and checklists

62

های در این مطالعه صرف نظر از گزارش • مطالعه 93نظران، از شفاهی صاحب

مطالعه دارای معیارهای انتخاب 53موجود، بودند

63




Data synthesis


64

Collecting data

• data collection forms• Methods

• Participants

• Interventions

• Outcome measures and results

65

داتنجدول مست

گروه هدف •جمعيت تحت مطالعه )سن، جنس، شغل(• عامل يا عوامل خطر مورد بررسي درمقاله•مكان مطالعه)كشور / شهر(•زمان مطالعه• (population based، مطالعه )توصيفي / تحليلينوع• و نحوه ي كنترل آن ها مخدوش كننده هاي مورد بررسي•

مطالعهدرگيري متغيرهاي مورد مطالعه )پرسشنامه / مدارك ابزار اندازه•

موجود(روايي و پايايي ابزار مورد استفاده•

و قدرت اثر هر يك از عوامل تعيين كننده مذكورميزان شيوع •Extraction Form Final.xls

66




Data synthesis


67

Graphical Display

• The graphical display of results from individual studies on a common scale is a “Forest plot”.

In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis.

Forest plot is an important step, which allows a visual examination of heterogeneity between studies.

68

Odds RatioOdds Ratio

more than 1less than 1 1

Line of no significance

69

Odds Ratio – with Odds Ratio – with pooled effect sizepooled effect size

more than 1less than 1 1

Best/point estimate

Confidence Interval

70

Forest Plot

var1.012601 .071399

Combined

71

Forest Plot

var1-.004408 .60356

Combined

72

Meta-Analysis

• when an overview incorporates a specific statistical strategy for assembling the results of several studies into a single estimate

73

• Systematic reviews do not have to have a meta-analysis

• There are times when it is not appropriate or possible.

75

تجزيه و تحليل نهايي

امكان انجام متاآناليز با توجه به وجود •هتروژنيتي ) نمودار و آماري( بررسي گرديد.

شيوع عدم آگاهي در زير گروه هاي مختلف • محاسبه گرديد.6/32 تا 4/24ما بين

76

77

A Range of Function of Systematic Review

• 1- Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure.

• 2- Increase statistical power and precision to detect an effect

• 3- Develop, refine and test hypothesis• 4- Reduce the subjectivity of study

comparisons by using systematic and explicit comparison procedure

78

Remain:

• 5- Identify data gap in the knowledge base and suggest direction for future research

• 6- Calculate sample size for future studies

79

Section 3

80

Steps of a systematic review

• Step 1: Framing question for a review

• Step 2: Identifying relevant literature

• Step 3: Assessing the quality of the literature

• Step 4: Summarizing the evidence

• Step 5: Interpreting the finding

81

• The principal findings should be related to the main question formulated in step1.

• Other finding should be considered secondary.

82

Validity of the main finding

• Are the searches adequate?

• Is there a risk of publication and related biases?

• Is the quality of the included studies high enough?

83

Limitations of Systematic Review

• Reporting bias and the inadequate quality of primary research are potentially serious problems for systematic reviews.

• The dissemination of research findings is not a random process; rather it is strongly influenced by the nature and direction of results.

84

Type of reporting bias

• Publication bias • The publication or non-publication of research findings, depending on the nature and direction of the results

85

Publication bias

• 1-Arising from the researchers deciding whether or not to submit result

• 2- Arising from the tendency of journals to reject negative studies

• 3-sponsorship

• ….

86

Methods of preventing publication bias

• 1-Registeries

• 2-Editorial policy

87

• Time lag bias • The rapid or delayed publication of research finding, depending on the nature and direction of the results

88

• Multiple (duplicate) publication bias

• The multiple or singular publication of research finding, depending on the nature and direction of the results

89

• Citation bias • The citation or non-citation of research

finding, depending on the nature and direction of the results

90

• Language bias • The publication of research finding in a particular language, depending on the nature and direction of the results

91

• Outcome reporting bias

• The selective reporting outcomes but not of others , depending on the nature and direction of the results

92

Funnel Plot• Plots of the trials’ effect estimates against sample

size, may be useful to assess the validity of meta-analyses

• A symmetrical shape is expected, since greater scatter in estimate is expected for smaller study.

• The cardinal sign of publication bias is a hole in the middle or one side of the plot, that is an area where we would expect to see study result but where there are apparently none.

93

94

Thanks

Documents

1 An Introduction to Systematic Reviews. 2 Information explosion