53
22/04/57 1 EBM: EBM: Systematic Review & Systematic Review & Meta Meta-analysis analysis RAID433: Critical Thinking in Medicine Section for Clinical Epidemiology & Biostatistics Faculty of Medicine Ramathibodi Hospital v. 2 Scope of lecture Scope of lecture Systematic review and traditional review Methods of conducting systematic review Meta-analysisPublication bias Critical appraisal of systematic review and meta- analysis Validity Result from systematic review Applicability

EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

  • Upload
    others

  • View
    16

  • Download
    0

Embed Size (px)

Citation preview

Page 1: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

1

EBM:EBM:

Systematic Review & Systematic Review &

MetaMeta--analysisanalysis

RAID433: Critical Thinking in Medicine

Section for Clinical Epidemiology & Biostatistics

Faculty of Medicine Ramathibodi Hospital

v. 2

Scope of lectureScope of lecture

• Systematic review and traditional review

• Methods of conducting systematic review

• Meta-analysisPublication bias

• Critical appraisal of systematic review and meta-analysis

• Validity

• Result from systematic review

• Applicability

Page 2: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

2

Trends towards Systematic ReviewTrends towards Systematic Review

There have been more than 400 systematic reviews and meta-analyses published in

the discipline of nephrology since 1990, with the annual number increasing with time.

Garg A X et al. CJASN 2008;3:253-260

Hierarchy of EvidenceHierarchy of Evidence

Page 3: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

3

• Primary analysis (Primary research): Original studies• Original analysis of research data

• Secondary analysis (Secondary research)• Re-analysis of the original data either using another

statistical technique or answering new questions with previously obtained data

Traditional “narrative review” articlesTraditional “narrative review” articles

• Qualitative

• Summary of all primary research on a given topic

• May provide good background information

• Major disadvantages:

• Subjective, unsystematic approach

• May be biased from author who may be very selective of the articles chosen for review

• Conflicting of evidences

Page 4: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

4

What is “Systematic Review”What is “Systematic Review”

• A review of a particular subject undertaken in such a systematic way that risk of bias is reduced.

• Systemic reviews have explicit, scientific, and comprehensive descriptions of their objectives and methods.

Hunink, Glasziou et al, 2001.

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

Page 5: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

5

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

------

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAA

AAAAAAAAAAAAAAAAAAAAAA

Systematic review

Page 6: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

6

Systematic Review vs Narrative ReviewSystematic Review vs Narrative Review

Characteristic Narrative Review Systematic Review

Clinical question Seldom reported, or

address several questions

Focused question: PICO

Search for primary articles

Seldom reported, not comprehensive

Comprehensive search of several sources

Selection of primary articles

Seldom reported,often biased

Explicit inclusion and exclusion criteria

Evaluation of quality of primary studies

Seldom reported, not usually systematic

Methodological quality

of primary articles is assessed

Summary of results of primary studies

Usually qualitative, nonsystematic

Synthesis is systematic, meta-analysis

Conducting Systematic ReviewConducting Systematic Review

1. Define question

2. Conduct literature search

3. Apply inclusion and exclusion criteria

4. Create data abstraction

5. Conduct analysis

Page 7: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

7

11. Define question. Define question

• Specify inclusion and exclusion criteria• Population

• Intervention/Exposure

• Outcome

• Methodology (time, language, publication restriction)

22. Conduct literature search. Conduct literature search

• Decide on information source• Databases

• Experts

• Funding agencies, pharmaceutical companies

• Hand searching

• Trial registries

• Cochrane database of randomized controlled trials

• Citation lists of retrieved articles (Cross-references)

• Identify titles and abstract

Page 8: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

8

Literature searchLiterature search

Page 9: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

9

33. Apply inclusion and exclusion. Apply inclusion and exclusion

• Apply inclusion and exclusion criteria to titles and abstracts

• Obtain full articles to eligible titles and abstracts

• Apply inclusion and exclusion criteria to full articles

• Select final eligible articles

• Assess agreement on study selection

Inclusion and ExclusionInclusion and Exclusion

Page 10: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

10

44. Create data abstraction. Create data abstraction

• Data abstraction• Participants, Interventions, Comparison interventions,

study design

• Result

• Methodologic quality

• Assess agreement on validity assessment

Data Collecting FormsData Collecting Forms

Page 11: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

11

55. Conduct analysis. Conduct analysis

• Determine method of generating pooled estimates across studies

• Generate pooled estimates (if appropriate)

• Explore heterogeneity

• Conduct subgroup analysis if appropriate

• Explore possibility of publication bias

MetaMeta--analysisanalysis

• “The analysis of analyses”

• Statistical techniques for merging and contrasting results across studies

• Synthesis of pooled effect

• Giving greater weight to studies with• Less variance (V)

• More precision (W)

Page 12: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

12

Systematic review and MetaSystematic review and Meta--analysisanalysis

• Systematic review does not necessarily contain meta-analysis

• Identified no or too little studies

• Identified studies with different methods or different outcomes that is not reasonable to combine the effect

• Meta-analysis also does not need to be done from systematic review

• You can do meta-analysis with specific studies selection

• But it is best to be done together.

Process of MetaProcess of Meta--analysisanalysis

• Pooling data• Proportion: Incidence rate, Prevalence

• Mean: Mean, Mean difference, Standardized Mean Difference

• Ratios: Relative risks, Odds ratio, Risk difference

• Diagnostic performances: Sensitivity/Specificity

• Etc.

• Subgroup analysis

Page 13: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

13

Prevalence, RR, OR revisitedPrevalence, RR, OR revisited

Disease (Outcome)

Develop disease (+)

No disease (-)

Exp

osu

re

Have

exposure (+)

A B

Do not

have

exposure (-)

C D

Forest PlotForest Plot

• Plot of effect size and their precision

• Most common way to report result in meta-analysis

• Help identify patterns of effect

• Can spot outliers

Page 14: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

14

Forest plot of the effects of alpha-blockers on treatment

responsiveness; random effect model

Overall (I-squared = 64.2%, p = 0.016)

Cheah P.Y. (2003)

Tugcu V. (2007)

Mehik A. (2003)

Nickel J.C. (2004)

Nickel J.C. (2008)

Alexander R.B. (2004)

Author (year)

1.57 (1.07, 2.30)

1.63 (1.03, 2.57)

2.00 (1.14, 3.52)

3.82 (1.53, 9.55)

2.00 (0.76, 5.23)

1.00 (0.79, 1.27)

1.09 (0.54, 2.21)

RR (95% CI)

100.00

20.53

17.74

10.87

10.20

26.10

14.56

26

20

13

9

68

12

R

17

10

4

18

70

33

NR

16

10

4

5

66

11

R

27

20

16

25

68

34

NR

1.57 (1.07, 2.30)

1.63 (1.03, 2.57)

2.00 (1.14, 3.52)

3.82 (1.53, 9.55)

2.00 (0.76, 5.23)

1.00 (0.79, 1.27)

1.09 (0.54, 2.21)

RR (95% CI)

100.00

20.53

17.74

10.87

10.20

26.10

14.56

%weight

Alpha Placebo

1.2 .5 1 1.57 5 10

Risk Ratio

Fixed Effect Model vs. Random Effects ModelFixed Effect Model vs. Random Effects Model

• These are two mathematical methods for deriving a pool estimation

• Fixed effect model• Assumption: single common (“fixed”) result

• If every study were infinitely large, they would yield an identical result

• No statistical heterogeneity among studies

• Random effect model• Assumption: individual studies are estimating different

treatment effects (distribution with some central value and some degree of variability)

• We interested in the central value and distribution of effect

Page 15: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

15

Page 16: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

16

Fixed Effect Model Random Effect Model

Underlying assumption All trials estimate the same effect

Varying underlying effect

Computational method

Error term comes from

“within trial” (ignore

between-study variability)

Error term comes from

“within trial” and “between trial”

Practicalconsequences

Narrower confidence interval

Wider confidence interval

HeterogeneityHeterogeneity

• Cochrane’s Q Test

• Higgin’s I2

Page 17: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

17

Cochrane’s Q testCochrane’s Q test

Cochrane’s Q testCochrane’s Q test

• Null hypothesis: underlying effect is the same

• Generate p value to test the null hypothesis

• p value = probability to accept null hypothesis

• Low p value: random error is an unlikely explanation of the difference between studies (therefore, there may be some systematic error or bias)

• High p value: underlying assumption of pooling holds true

kH θθθθ ==== ...: 3210

Page 18: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

18

kH θθθθ ==== ...: 3210

Cochrane’s Q testCochrane’s Q test

• This test is less sensitive when the number of studies is low (<25 studies).

• The level of significance is recommended to be at least 0.10.

Higgin’sHiggin’s II22

Higgins JP, Thompson SG, et.al. Measuring inconsistency in meta-analyses. BMJ 2003; 327 (7414): 557-60.

Page 19: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

19

One thing to consider…One thing to consider…

• Because systematic review searched databases of published studies

• Do you think we will get all information available?

• What will happen if we only choose studies that get published?

Publication Bias (Reporting Bias)Publication Bias (Reporting Bias)

• Publication bias and other related biases can be summarised as statistically significant, 'positive' results being:

• more likely to be published (publication bias)

• more likely to be published rapidly (time lag bias)

• more likely to be published in English (language bias)

• more likely to be published more than once (multiple publication bias)

• more likely to be cited by others (citation bias)

Page 20: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

20

Funnel Plot for assessing publication biasFunnel Plot for assessing publication bias

• Plot between log(effect size) and precision (1/variance)

• Assumption: all studies are estimating the same effect, so the estimated effects should be normally distributed about the unknown true effect

• The spread of estimated effect should therefore be proportional to their variances, just like a funnel

• Small study (lower n) – higher variance, more spread

• Large study (higher n) – lower variance, more narrow

1/variance

Ln(effect size)

Page 21: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

21

0

.1

.2

.3

.4

.5

Sta

ndard

err

or

-1 0 1 2ln(RR)

Studies

p < 1%

1% < p < 5%

p > 5%

Figure 5. Contour enhanced-funnel plot

Page 22: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

22

Sources of plot asymmetrySources of plot asymmetry

• Biased selection of study (Selection Bias)

• Heterogeneity of treatment effects

• Poor quality of research methods

• Publication bias

Critical Appraisal of Systematic ReviewCritical Appraisal of Systematic Review

1. Are the results valid?

2. What are the results?

3. How can I apply the results to patient care?

Page 23: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

23

11. Are the results valid?. Are the results valid?

• Did the review include explicit and appropriateeligibility criteria?

• Was biased selection and reporting of studies unlikely?

• Were the primary studies of high methodologicquality?

• Were assessments of studies reproducible?

11..1 1 Did the review include explicit and appropriate eligibility Did the review include explicit and appropriate eligibility

criteria?criteria?

• Too broad or too narrow review?

Consider these systematic review questions:

- All treatment modalities for all types of cancer to look for mortality

- All antiplatelet effect on all CV events

- Aspirin dose to prevent thrombotic stroke in patients who had TIA

Page 24: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

24

- All treatment modalities for all types of cancer to look for mortality (Obviously broad)

- All antiplatelet effect on all CV events (Still broad)

- Aspirin dose to prevent thrombotic stroke in patients who had TIA

Appropriate eligibility criteriaAppropriate eligibility criteria

• Results likely to be similar across range of patients? (older/younger, sicker/less sick)

• Results likely to be similar across range of intervention? (higher/lower dose, expert/non expert use)

• Results likely to be similar across range of outcome measured? (shorter/longer follow-up)

• Did it turn out that results were indeed similar across the range of patient, intervention, and outcome?

Page 25: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

25

Selecting articles that are most likely to provide valid resultsSelecting articles that are most likely to provide valid results

Therapy - Were patients randomized?- Was follow-up complete?

Diagnosis - Was the patient sample representative of those with

the disorder?

- Was the diagnosis verified using credible criteria that

were independent of the items of the medical history,

physical examination, lab tests, or imaging procedures under study?

Harm - Did the investigators demonstrate similarity in all

known determinants of outcome or adjust for

differences in the analysis?- Was follow-up sufficiently complete?

Prognosis - Was there a representative sample of patients?

- Was follow-up sufficiently complete?

11..2 2 Was the Search for Relevant Studies detailed and Was the Search for Relevant Studies detailed and

exhaustive?exhaustive?

• Complete, or at least a representative sample of the available eligible studies

• Bibliographic databases• MEDLINE

• EMBASE

• CINAHL

• Cochrane Central Register of Controlled Trials

• Personal contact

• Scientific meeting, doctoral thesis, ongoing trials by pharmaceutical companies

Page 26: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

26

• Reporting bias: mostly due to the failure to publish studies with negative results

• For unpublished studies, researchers should try to use:

• Full written reports

• Same eligibility criteria as other published reports

Individual patientIndividual patient--data metadata meta--analysisanalysis

• Use individual patient data in primary study, rather than reported data

• Strengthen the influence from a systematic review

• Can facilitate• True intention-to-treat analysis

• Subgroup analysis

Page 27: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

27

11..3 3 Were the primary studies of high Were the primary studies of high methodologicmethodologic quality?quality?

• Methodology• Study design

• Appraisal of quality• Quality scale: ex. JADAD scale

• Quality table (Cochrane’s risk of bias assessment)

• Quality of primary study is assessed using similar tool as you appraised primary study

JADAD ScaleJADAD Scale

Jadad AR, Moore RA, Carroll D,et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996; 17:1–12.

Page 28: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

28

Cochrane’s Risk of Bias assessmentCochrane’s Risk of Bias assessment

Type of bias Description Relevant domains in the Collaboration’s

‘Risk of bias’ tool

Selection bias. Systematic differences between

baseline characteristics of the

groups that are compared.

•Sequence generation.

•Allocation concealment.

Performance bias. Systematic differences between

groups in the care that is

provided, or in exposure to factors

other than the interventions of

interest.

•Blinding of participants and personnel.

•Other potential threats to validity.

Detection bias. Systematic differences between

groups in how outcomes are

determined.

•Blinding of outcome assessment.

•Other potential threats to validity.

Attrition bias. Systematic differences between

groups in withdrawals from a

study.

•Incomplete outcome data

Reporting bias. Systematic differences between

reported and unreported findings.

•Selective outcome reporting

Page 29: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

29

11..4 4 Were Were assessments of studies reproducibleassessments of studies reproducible??

• Systematic reviews are subject to both mistakes (random error) and bias (systematic error)

• Having more than one reviewer guards against these errors

22. What are the results. What are the results

• Were the results similar from study to study?

• What are the overall results of the review?

• How precise were the results

Page 30: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

30

22..1 1 Were the results similar from study to study?Were the results similar from study to study?

• Visual evaluation of variability• How similar are the point estimates

• To what extent do confidence intervals overlap

• Statistical tests evaluating variability• Yes-or-no test for heterogeneity (p value)

• I2 test that quantifies the variability explained by between-study differences in results

Visual evaluation of variabilityVisual evaluation of variability

Page 31: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

31

Statistical testsStatistical tests

• Cochrane Q

• I2 test

22..2 2 What are the overall results of the reviewWhat are the overall results of the review

• Dichotomous outcome• Relative risk

• Odds ratio

• Continuous outcome• Mean different

Page 32: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

32

22..3 3 How precise were the resultsHow precise were the results

• Confidence interval around that estimate

33. How can I apply the results to patient care?. How can I apply the results to patient care?

• Were all patient-important outcomes considered?

• Are any postulated subgroup effects credible?

• What is the overall quality of the evidence?

• Are the benefits worth the costs and potential risks?

Page 33: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

33

33..1 1 Were all patientWere all patient--important outcomes considered?important outcomes considered?

• Adverse effects of therapy

• Costs

33..2 2 Are any postulated subgroup effects credible?Are any postulated subgroup effects credible?

• Subgroup analysis – playing with chance

Page 34: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

34

Guidelines for deciding whether apparent differences in Guidelines for deciding whether apparent differences in

subgroup response are realsubgroup response are real

• Did the hypothesis precede rather than follow the analysis?

• Was the subgroup difference one of a small number of hypothesized effects tested?

• Is the subgroup difference suggested by comparisons within rather than between studies?

• Is the magnitude of the subgroup difference large?

• Is the subgroup difference consistent across studies?

• Was the subgroup difference statistically significant?

• Does external evidence support the hypothesized subgroup difference?

33..3 3 What is the overall quality of the evidence?What is the overall quality of the evidence?

• What do you think about the overall quality?

• Do you believe in the results?

Page 35: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

35

33..4 4 Are the benefits worth the costs and potential risks?Are the benefits worth the costs and potential risks?

• Weight between costs/risks and benefits

• Do not forget patient preference and clinical expertise

Quality of therapeutic studiesQuality of therapeutic studies

• Clinical trials:A systematic study on pharmaceutical products in human subjects in order to discover or verify the effects of and/or identify any adverse reaction to investigational products, and/or to study the absorption, distribution, metabolism and excretion of the products with the objective of ascertaining their efficacy and safety.

Page 36: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

36

Phases of Clinical TrialPhases of Clinical Trial

Phase IClinical pharmacology / toxicology, dose

fining, safety, perfection of technique

Phase IIInitial clinical investigation for

treatment effect, pilot study

Phase IIIFull-scale evaluation of treatment effect

(efficacy)*

Phase IV Post-market surveillance (effectiveness)

Efficacy vs Effectiveness studyEfficacy vs Effectiveness study

• Efficacy: Does receiving treatment work under ideal conditions?

• Limit only to patients that will cooperate fully with medical advice

• Effectiveness: Does offering treatment help under ordinary circumstances?

• Offering treatment to patients and allowing them to accept or reject it as they might ordinarily do

• If a treatment is found to be ineffective:• Lack of efficacy

• Lack of patient acceptance

Page 37: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

37

sample

Population

RandomizationRandomization

Treatment Control

OutcomeNo

outcome OutcomeNo

outcome

Blind assessmentBlind assessment

Randomized Control Trial

EndBeginning

Study subjects

Intervention

Death

Alive

Control

Death

Alive

• Randomization• Concealment• Intention to treat

analysis• Similar prognosis

• Blinding• Patients• Clinicians• Outcome

assessors• Adequate follow -up

Page 38: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

38

Randomized AllocationRandomized Allocation

RandomisationRandomisation

• Allocation of individuals to groups by chance.

• Within the limits of chance variation, randomisationshould make the control and the experimental groups similar at the start of the investigation and ensure that personal judgment and prejudices of the investigator do not influence allocation

Randomized AllocationRandomized Allocation

RandomisationRandomisation

• All subjects in the sample have same probability of being assigned to the experimental or control group.

• Any underlying factors the may affect the outcome are equivalent for each group.

Page 39: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

39

Randomized controlled trials (RCT)Randomized controlled trials (RCT)

Populationof patientsWith the Condition Sample

Improved

Not improved

Improved

Not improved

Outcomes

Experimental

Comparison

Random sampling

R

Type of randomizationType of randomization

• Simple

• Block

• Stratified

Page 40: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

40

Simple (unrestricted) randomizationSimple (unrestricted) randomization

• Coin, dice (not preferred)

• May be biased

• Can not be audit by others

• Table of random numbers

• Computer generated

Block RandomizationBlock Randomization

Subject 1

Subject 2

Subject 3

Subject 4

Block 1

Subject 1

Subject 2

Subject 3

Subject 4

Subject 1

Subject 2

Subject 3

Subject 4

Block 2 Block J

R R R

A B A B A B

Page 41: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

41

Stratified allocationStratified allocation

• Example• Gender: Male or female

Male

Female

R

R

A

A

B

B

Allocation concealmentAllocation concealment

• Generation of an unpredictable randomized allocation sequence .

• It represents the first crucial element of randomization in a RCT.

• Allocation concealment refers to the technique used to implement the sequence, not to generate it.

Page 42: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

42

How to conceal?How to conceal?

• Local: Sequentially numbered, opaque envelopes

• Central: computer-based

• Emergency situation?

Treatment100

Control100

8/80 = 10%

20/100 = 20%

Intention to treat vs. Intention to treat vs. Per protocol Per protocol analysisanalysis

Page 43: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

43

Per protocolPer protocol

or Explanatory trialsor Explanatory trials

• “ Which experimental treatment itself is better?”

• Analysis according to the treatment each patient actually received, regardless of the treatment to which they were randomized

• Dissimilarities between groups

Treatment100

Control100

8/80 = 10%

20/100 = 20%

12/20 = 60%

Intention to treat vs. Intention to treat vs. Per protocol Per protocol analysisanalysis

Page 44: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

44

Intention to treat or management trialsIntention to treat or management trials

• “ Which treatment policy is best at the time the decision must be made?”

• Analysis according to the assigned group

• Increasing the chance of a negative study

Per protocol analysisPer protocol analysis

• Excluding noncompliant patients from the analysis leaves behind those who may be destined to have a better outcome.

• Destroys the unbiased comparison provided by randomization

Page 45: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

45

3 3 CC

• Compliance

• Contamination

• Co-intervention

Why blinding is important?Why blinding is important?

Page 46: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

46

Potential benefits of blinding Potential benefits of blinding

Individuals Potential benefits

Patients • Less likely to have biased psychological or physical

responses to intervention

• More likely to comply with trial regimens

• Less likely to seek additional adjunct interventions

Clinicians • Less likely to differentially administer co-interventions

• Less likely to differentially adjust dose

• Less likely to differentially withdraw participants

• Less likely to differentially encourage or discourage

participants to continue trial

Assessors • Less likely to have biases affect their outcome

assessments, especially with subjective outcomes of

interest

Placebo Placebo

• An inert medication or procedure.

• The placebo effect (usually but not necessarily beneficial) is attributable to the expectation that the regimen will have an effect.

Page 47: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

47

Comparison groupComparison group

• No intervention

• Placebo treatment

• Standard treatment

Outcome measurementOutcome measurement

Surrogate outcome

Low-density lipoprotein

(LDL)

Proteinuria

Clinical outcome

Death

ESRD

Page 48: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

48

Surrogate outcomeSurrogate outcome

• Laboratory or physiologic measurement used as a substitute for an outcome that measure directly

• A surrogate outcome will be consistently reliable only if there is a causal connection between change in surrogate and change in the clinically important outcome.

Composite outcomeComposite outcome

• Combination of multiple endpoints e.g. combined death, MI or stroke

• Frequently used as primary outcome measures in RCT

• Increased statistical efficiency

Page 49: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

49

What should you do when blinding is impossible?What should you do when blinding is impossible?

• Strict to treatment protocol: to prevent unequal co-intervention

• Equally intense follow-up of experimental and control patients.

• External outcome adjudicators: to reduce biased outcome measurement

MethodsMethods

What should they do?

Page 50: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

50

When does loss to followWhen does loss to follow--up seriously threaten validity?up seriously threaten validity?

Trial A Trial B

Treatment Control Treatment Control

No. of patients

randomized

1000 1000 1000 1000

No. of lost to

F/U

30 (3%) 30 (3%) 30 (3%) 30 (3%)

No. of death 200 (20%) 400 (40%) 30 (3%) 60 (6%)

RRR not

counting

patient lost to

F/U

0.2/0.4

= 0.50

0.03/0.06

= 0.50

RRR for worst-

case scenario

0.17/0.4

= 0.43

0.00/0.06

= 0

Treatment effectTreatment effect

Control Experimental ARR RRR RR

80 KG 60 KG 20 KG 25% 75%

My past

weight

My present

weight

The absolute

Weight I lost

The percent

weight I lost

The percent

weight that

remained

Rc Rt Rc-Rt (Rc-Rt)/Rc Rt/Rc

My past risk My present risk The absolute

risk I lost

The percent

risk I lost

The percent

risk

remaining

Page 51: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

51

How large was the treatment effectHow large was the treatment effect??

Incidence of contrast

nephropathy

Absolute Risk

Reduction

ARR

Relative Risk

Reduction

RRR

Number

Needed to

Treat

NNT

Hydration

Control

Event

Rate

CER

NAC

Experimental

Event

Rate

EER

CER - EERCER - EER

CER 1/ARR

9.6% 2.8%

0.10-0.05

= 0.05

= 5%

(0.10-0.05)/0.10

= 0.5

= 50%

Confidence IntervalsConfidence Intervals

-50 -25 0 25 50 75

Relative risk reduction(%)

Page 52: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

52

Number needed to treat (NNT)Number needed to treat (NNT)

Number needed to harm (NNH)Number needed to harm (NNH)

Number needed to treat (NNT)

• Number needed to be treated to prevent one more event

NNT = 1/ (Riskcontrol-Risktreatment)

= 1/ARR

Number needed to harm (NNH)

• Number needed to be treated to harm one more of them

NNH = 1/ (Rtreatment-Rcontrol)

Ask

Acquire

Appraise

Apply

Act & Assess

Patient

dilemma

Principles of

evidence-based

practice

Evidence alone does not

decide – combine with other

knowledge and values

Hierarchy

of evidence

Process of EBP

Page 53: EBM: Systematic Review & MetaMeta--analysis analysis · EBM: Systematic Review & MetaMeta--analysis analysis RAID433: Critical Thinking in Medicine ... -All antiplatelet effect on

22/04/57

53

EBM Sample ScenarioEBM Sample Scenario

• A 30 year old women came into your practice with a history of sore throat for 3 days. Upon physical examination you find slightly enlarged and injected tonsil and injected pharynx. She reported no significant past medical history. She asked you whether any medication can lessen the pain.

• You have seen a senior prescribing corticosteroid injection for such condition, and you wonder whether prescribing steroid in patient with acute sore throat works.

• You then search PubMed with keyword "Adrenal Cortex Hormones"[Mesh] AND "Pharyngitis"[Mesh] AND "Pain"[Mesh] and found a systematic review:

Systematic ReviewSystematic Review

• Corticosteroids for pain relief in sore throat: systematic review and meta-analysis