Observational Data

7/31/2019 Observational Data

1/47

PVL_

COUN

TRY_

DATE00/1

Systematic reviewsSystematic reviews

of observationalof observational

datadata

Dr An a Pi lar Bet r n

Depar t m en t o f Reprod uc t i ve Hea lt h an d Resear ch

W or ld Hea lt h Organ i zat i on


2/47

PVL_

COUN

TRY_

DATE00/2

Epidemiologist know a lot about theEpidemiologist know a lot about the

correct way to conduct a researchcorrect way to conduct a research

study but less about how to reviewstudy but less about how to reviewand synthesize data from multipleand synthesize data from multiple

studies and this, I suggest, is astudies and this, I suggest, is a

principal source of the publicprincipal source of the publicssconfusion when faced with a newconfusion when faced with a new

result from an epidemiological studyresult from an epidemiological study

Bracken MB. IJE 2001:954


3/47

PVL_

COUN

TRY_

DATE00/3

What is a systematic review?What is a systematic review?

A review:

99 clearly formulated questionclearly formulated question

99 uses systematic and explicit methods touses systematic and explicit methods toidentifyidentify andand collectcollect relevant researchrelevant research

99 uses systematic and explicit methods touses systematic and explicit methods toselectselect,, critically appraisecritically appraise andand analyseanalyse

relevant research included.relevant research included.


4/47

PVL_

COUN

TRY_

DATE00/4

What is a systematic review?What is a systematic review?

Statistical methods (meta-analysis)may or may not be used to

summarise the results

of the included

studies


5/47

PVL_

COUN

TRY_

DATE00/5

How much work is a systematicHow much work is a systematic

review?review?

~ 1 1 39 hou r s

~ 3 0 p er so n -w eek s o f f u l l - t i m e w o r k

99 588 for protocol, searching and588 for protocol, searching andretrievalretrieval

99144 for statistical analysis144 for statistical analysis

99 206 for report writing206 for report writing

99 201 for administration201 for administration

Sour ce : A l len I E. JAMA, 19 99 ;28 2 :6 34


6/47

PVL_

COUN

TRY_

DATE00/6

What are observational studies?What are observational studies?

99 Data from existing databaseData from existing database

99 CrossCross--sectional studysectional study

99 Case seriesCase series

99 CaseCase--control studycontrol study

99 Cohort studyCohort study


7/47

PVL_

COUN

TRY_

DATE00/7

RCTRCT

Observational studiesObservational studies


8/47

PVL_

COUN

TRY_

DATE00/8

Why do we need systematic reviewsWhy do we need systematic reviews

of observational studies?of observational studies?

99 Test aetiological hypothesisTest aetiological hypothesis

99 Evaluation of interventions designedEvaluation of interventions designed

to prevent rare outcomesto prevent rare outcomes99 Evaluation if outcomes of interest areEvaluation if outcomes of interest are

far in the futurefar in the future

99 Evaluation of effectiveness in aEvaluation of effectiveness in a

communitycommunity


9/47

PVL_

COUN

TRY_

DATE00/9

MAOS are commonMAOS are common

Type of article Articles (n)

Meta-analysis of:Controlled trials

Observational studies

34

25

Methodological article 15

Tradicional review 15

Other 11

Source: Egger M. Systematic reviews in Health Care.Meta-analysis in context. BMJ Books. 2001


10/47

PVL_

COUN

TRY_

DATE00/10

RCTRCT

(Lack of precision)(Lack of precision)

MetaMeta--analysisanalysis

More reliable estimatesMore reliable estimates


11/47

PVL_

COUN

TRY_

DATE00/11

Observational studiesObservational studies

(Confounding, bias)(Confounding, bias)

MetaMeta--analysisanalysis

More reliable estimates????More reliable estimates????


12/47

PVL_

COUN

TRY_

DATE00/12

Confounding factorsConfounding factors

SmokingSmoking SuicideSuicide

Social/mental states


13/47

PVL_

COUN

TRY_

DATE00/13

Confounding factorsConfounding factors

Coffee consumptionCoffee consumption

Risk of myocardial infarctionRisk of myocardial infarction

Smoking


14/47

PVL_

COUN

TRY_

DATE00/14

The protective effect of betaThe protective effect of beta--carotenecarotene

that wasnthat wasnttCohor tsMale health workers

Social insurance, menSocial insurance, womenMale chemical workersHyperlipidaemic menNursing home residents

Tr ia lsMale smokersSkin cancer patients(Ex)-smokers, asbestosworkersMale physicians


15/47

PVL_

COUN

TRY_

DATE00/15

There are examples ofThere are examples ofobservational studies producingobservational studies producing

similar results of those from RCTsimilar results of those from RCT

But observational studies will always have toBut observational studies will always have todeal withdeal with biasbias andand confoundingconfounding becausebecause

the intervention was deliberately chosen andthe intervention was deliberately chosen and

not randomly allocatednot randomly allocated


16/47

PVL_

COUN

TRY_

DATE00/16

Benson and Hartz,NEJM, 2000;342:1878-86


17/47

PVL_

COUNTRY_

DATE00/17

ConcatoConcato et al.,et al.,

NEJM, 2000;342:1887NEJM, 2000;342:1887--9292


18/47

PVL_

COUNTRY_

DATE00/18This does not mean to return toThis does not mean to return to

narrative reviewsnarrative reviews


19/47

PVL_

COUNTRY_

DATE00/19

Benefits of MAOS:Benefits of MAOS:

99Systematic and explicit rulesSystematic and explicit rules

99 Statistical powerStatistical power

99Insight into variable interactionInsight into variable interaction

99 Detection of discrepanciesDetection of discrepancies

99Deepness into heterogeneityDeepness into heterogeneity

99 Identification of gaps in knowledgeIdentification of gaps in knowledge


20/47

PVL_

COUNTRY_

DATE00/20

Reporting of background shouldReporting of background should

include:include:

1 Problem definition, hypothesisstatement

2 Description of study outcome(s)3 Type of exposure or intervention used

4 Type of study designs used

5 Study population


21/47

PVL_

COUNTRY_

DATE00/21

Reporting of search should include:Reporting of search should include:

66 Qualifications of researchersQualifications of researchers 77 Search strategy including time periodSearch strategy including time period 88 Effort to include all available studiesEffort to include all available studies 99 Databases and registries searchedDatabases and registries searched

1 01 0 Searching software usedSearching software used

1 11 1 Use of hand searchingUse of hand searching 1 21 2 List of citations located and those excluded, includingList of citations located and those excluded, including

justificationjustification

1 31 3 Methods of addressing articles not published in EnglishMethods of addressing articles not published in English 1 41 4 Methods of handling abstracts and unpublished studiesMethods of handling abstracts and unpublished studies

1 51 5 Descriptions of any contact with authorsDescriptions of any contact with authors


22/47

PVL_

COUNTRY_

DATE00/22

Reporting of methods should include:Reporting of methods should include:

1 61 6 Description of relevance/appropriateness of papersDescription of relevance/appropriateness of papersassembled for assessing the hypothesis to be testedassembled for assessing the hypothesis to be tested

1 71 7 Rational for the selection and coding of dataRational for the selection and coding of data 1 81 8 Documentation about how data were classified andDocumentation about how data were classified andcodedcoded

1 91 9 Assessment of confoundingAssessment of confounding 2 02 0 Assessment of study quality, including blinding ofAssessment of study quality, including blinding ofquality assessors; stratification or regression onquality assessors; stratification or regression on

possible predictors of study resultspossible predictors of study results 2 12 1 Assessment of heterogeneityAssessment of heterogeneity 2 22 2 Description of statistical methods in sufficient detail toDescription of statistical methods in sufficient detail to

be replicatedbe replicated

2 32 3 Provision of appropriate tables and graphicsProvision of appropriate tables and graphics


23/47

PVL_

COUNTRY_

DATE00/23

Reporting of results should include:Reporting of results should include:

2 42 4

Graphic summarizing individual studyGraphic summarizing individual study

estimates and overall estimateestimates and overall estimate

2 52 5 Table giving descriptive information forTable giving descriptive information foreach study includedeach study included

2 62 6 Results of sensitivity testing (e.g.Results of sensitivity testing (e.g.

subgroup analysis)subgroup analysis)

2 72 7 Indication of statistical uncertainty ofIndication of statistical uncertainty offindingsfindings


24/47

PVL_

COUNTRY_

DATE00/24

Reporting of discussion shouldReporting of discussion should

include:include:

2 82 8 Quantitative assessment of biasQuantitative assessment of bias

2 92 9 Justification for exclusionJustification for exclusion

3 03 0 Assessment of quality of includedAssessment of quality of included

studiesstudies


25/47

PVL_

COU

NTRY_

DATE00/25

Reporting of conclusions shouldReporting of conclusions should

include:include:

3 13 1 Consideration of alternative explanationsConsideration of alternative explanationsfor observed resultsfor observed results

3 23 2 Generalization of the conclusionsGeneralization of the conclusions

3 33 3 Guidelines for future researchGuidelines for future research

3 43 4 Disclosure of funding sourceDisclosure of funding source


26/47

PVL_

COU

NTRY_

DATE00/26

Quality of reviews in EpidemiologyQuality of reviews in Epidemiology

BreslowBreslow R. AJPH, 1998;88:475R. AJPH, 1998;88:475--77

All 1995 issues of 7 widely readepidemiology journals were

searched for reviews

29 reviews were found29 reviews were found


27/47

PVL_

COU

NTRY_

DATE00/27

Reviews following quality guidelinesReviews following quality guidelines

Gu idel in e Yes Un ab le t o

d e t e r m i n e

No

Search methods stated 6 (21) 1(3) 22(76)

Inclusion criteria

reported

5(17) 4(14) 20(69)

Bias in selecting studiesavoided

3(10) 26(90) 0(0)

Criteria for assessing

validity reported

2(7) 15(52) 12(41)

Methods for combining

findings reported

10(34) 6(21) 13(45)

Conclusions supported

by data

24(83) 4(14) 1(3)

Search restriction:


28/47

PVL_

COU

NTRY_

DATE00/28

Search restriction:General medical journal, 2001

1 (8%)4 (21%)All Methods Employed

4 (31%)8 (42%)Cochrane Databases Searched

2 (15%)7 (37%)Contacted Experts to FindUnpublished Data

4 (31%)5 (26%)Gray Literature Searched(e.g., manual search of conferenceor dissertation abstracts)

10 (77%)17 (89%)Additional Searches Conducted(e.g., manual search of referencelists or textbooks)

6 (46%)13 (68%)

Numerous Databases Searched

(versus just MEDLINE)

13 systematicreviews

19 meta-analyses

Search Procedure

Source: Becker B, Morton S (see http://www.msri.org/calendar/talks/TalkInfo/1268/show_talk)


29/47

PVL_

COU

NTRY_

DATE00/29

Search restriction:

General medical journal, 2001

5 (38%)7 (37%)Attempted to includeunpublished studies

5 (38%)4 (21%)Unclear

7 (54%)7 (37%)English only

0 (0%)2 (11%)English plus other lang.

1 (8%)6 (32%)None

13 systematic

reviews

19 meta-

analysesLanguage Restriction

Source: Becker B, Morton S (see http://www.msri.org/calendar/talks/TalkInfo/1268/show_talk)


30/47

PVL_

COU

NTRY_

DATE00/30

Other citations:Other citations:

99 MulrowMulrow CD. The medical review article: stateCD. The medical review article: state

of the science.of the science. Ann Intern MedAnn Intern Med1987, 6:2331987, 6:233--240.240.

99 McAlister FA, Clark HD, vanMcAlister FA, Clark HD, van WalravenWalraven C et al.C et al.

The medical review article revisited: has theThe medical review article revisited: has thescience improved?science improved? Ann Intern MedAnn Intern Med1999,1999,

131:947131:947--951951

99 Bracken MB. Commentary: towardsBracken MB. Commentary: towardssystematic reviews in epidemiology.systematic reviews in epidemiology. I JEI JE

2001, 30:9542001, 30:954--957.957.


31/47

PVL_

COU

NTRY_

DATE00/31

SummarySummary99 SR and MA of observational studies areSR and MA of observational studies are

as common as reviews of RCTas common as reviews of RCT99 Confounding and selection bias oftenConfounding and selection bias often

distort the findingsdistort the findings

99 Danger in producing very precise butDanger in producing very precise but

spurious resultsspurious results

99 More is gained by examiningMore is gained by examining

heterogeneityheterogeneity


32/47

PVL_

COU

NTRY_

DATE00/32

WHO Systematic review ofWHO Systematic review of

incidence/prevalence ofincidence/prevalence of

maternal mortality andmaternal mortality andmorbidity 1997morbidity 1997--20022002


33/47

PVL_

COU

NTRY_

DATE00/33

ObjectivesObjectives To provide a comprehensive,

standardised and reliable tabulation of

available data on maternal morbidity

To provide up-to-date data for futurematernal mortality estimates

To provide case-fatality rates


34/47

PVL_

COU

NTRY_

DATE00/34

Search strategySearch strategy

Electronic searchElectronic search

Electronic databases (Medline,Electronic databases (Medline, EmbaseEmbase,,

PoplinePopline,, CinahlCinahl,, SocioFileSocioFile, LILACS, CAB,, LILACS, CAB,EconlitEconlit,, BiosisBiosis, PAIS), PAIS)

WHO onWHO on--line regional databasesline regional databases

Internet searches (Google, web of scienceInternet searches (Google, web of science))


35/47

PVL_

COU

NTRY_

DATE00/35

Search strategySearch strategy

Other searchesOther searches

Experts active in the fieldExperts active in the field

WHO regional officesWHO regional offices Hand searchingHand searching

References listsReferences lists Circulating documentsCirculating documents


36/47

PVL_

COU

NTRY_

DATE00/36

WHO systematicWHO systematic

reviewreview


37/47

PVL_

COU

NTRY_

DATE00/37

WHO systematic reviewWHO systematic review


38/47

PVL_

COU

NTRY_

DATE00/38



39/47

PVL_

COU

NTRY_

DATE00/39



40/47

PVL_

COU

NTRY_

DATE00/40

WHOWHO

systematicsystematic

reviewreview

Ci ta t ions iden t i f i ed


41/47

PVL_

COU

NTRY_

DATE00/41

Fu l l - t ex t eva lua t ion

( A r t i c l es and r epo r t s )

4 6 2 6

Ci ta t ions iden t i f i ed

( Ti t l es a n d / o r a b st r a c t s)

6 4 5 8 6

Exc luded

5 9 9 6 0

I n c l u d e d

2 4 4 3Exc luded

1 9 8 8I n p r o c es s

1 9 5

Reasons fo r

exc lus ion

9 2 % n o

r e levan t da t a

6 % s a m p l e

si ze < 2 0 0

2 % o t h e rreasons

Data p rocess ing

comp le te

2 2 0 4

Reasons fo rexc lus ion

5 7 % n or e levan t da t a

1 5 % samp les iz e< 2 0 0

1 1 % n oda tes r epo r t ed

1 7 % o t h erreasons

Regional distribution (n=2204)Regional distribution (n=2204)


42/47

PVL_

COUNTRY_

DATE00/42

g ( )g ( )

Midd le A f r i ca

Sou th e r n A f r i c a

Nor the r n A f r i c a

Eas te rn A f r i ca

Wes te r n A f r i c a

Sou th - eas te r n A sia

Easter n As ia

Wes te r n A sia

Sout h - cen t r a l As iaEaster n Eur ope

Sou th e r n Eu r ope

Wes te r n Eu r ope

Nor the r n Eu r ope

Car ibbean

Cent ra l Am er ica

Sou th A m er i c a

Nor th A m er i c a

A u st r a li a/ N Z

3 2 %

2 6 %

2 2 %

1 7 %3 %

Development status (n=2204)Development status (n=2204)


43/47

PVL_

COUNTRY_

DATE00/43

Development status (n=2204)Development status (n=2204)

1 0 8 5

8 5 4

2 2 8

3 7

0

2 0 0

4 0 0

6 0 0

8 0 0

1 0 0 0

1 2 0 0

Industrialisedcountries

Less developedcountries

Least developedcountries

Multicountry


44/47

PVL_

COUNTRY_

DATE00/44

Resu l t s: m e th odo log ica l qu a l i t y o fResu l t s: m et hod o log ica l qu a l i t y o f

r e p or t ed d a t ar e p or t ed d a t a

M o r b i d i t y

( n = 3 2 1 5 )

M o r t a l i t y

( n = 3 3 5 )

To ta l

( n = 3 5 5 0 )

H ig h 1 0 3 8 1 1 1

Med iu m 1 6 7 0 2 5 0 1 9 2 0

Lo w 1 4 4 2 7 7 1 5 1 9

Rep o r t ed m o r b id i t ies ( n = 3 2 1 5 )Rep o r t ed m o r b id i t ies ( n = 3 2 1 5 )


45/47

PVL_

COUNTRY_

DATE00/45

Rep o r t ed m o r b id i t ies ( n 3 2 1 5 )ep o t ed o b d t es ( 3 5 )

99 Hypertensive disorders ofHypertensive disorders of

pregnancy (16.3%)pregnancy (16.3%)

99Haemorrhage (11.1%)Haemorrhage (11.1%)99 postpartumpostpartum -- 2.7%2.7%

99 antepartumantepartum//intrapartumintrapartum --

2.2%2.2%99 placentaplacenta praeviapraevia -- 1.8%1.8%

99 abruptioabruptio placentaplacenta -- 2.6%2.6%

99 other haemorrhage /other haemorrhage /unspecifiedunspecified -- 1.8%1.8%

99 Abortion (10.7%)Abortion (10.7%)

99Preterm delivery (8.3%)

99Stillbirth (6.3%)Stillbirth (6.3%)

99Diabetes in pregnancy (4.4%)Diabetes in pregnancy (4.4%)

99Anaemia in pregnancy (4.3%)Anaemia in pregnancy (4.3%)

99EctopicEctopic pregnancy (3.0%)pregnancy (3.0%)

99PerinealPerineal tears (2.6%)tears (2.6%)

99PROM (2.6%)PROM (2.6%)

99Uterine rupture (2.1%)Uterine rupture (2.1%)

99Postpartum sepsis (1.6%)Postpartum sepsis (1.6%)

99Depression (1.9%)Depression (1.9%)

99Obstructed labour (1.8%)Obstructed labour (1.8%)

Preterm delivery (8.3%)


46/47

PVL_

COUNTRY_

DATE00/46

William Hamilton, New Yorker, 2001


47/47

PVL_

COUNTRY_

DATE00/47

Documents

Observational Data