1

Cell-Free Hemoglobin-Based Blood Substitutes and Risk of Myocardial Infarction and DeathNatason et al., JAMA, Prepublished online April 28, 2008 at http://jama.ama-assn.org/cgi/content/full/299.19.jrv80007

Journal ClubYulia Lin, Transfusion Medicine ResidentUniversity of TorontoMay 16, 2008

2

How to critically appraise a meta-analysis?

Guyatt G, Rennie D (ed). User’s guide to the medical literature: a manual for evidence-based clinical practice. Chicago, IL: AMA Press. 2002. Chapter 1E: Summarizing the evidence.

3

Definitions

Overview Summary of the medical literature that

attempts to address a focused clinical question

Systematic review Using methods designed to reduce the

likelihood of bias Meta-analysis

Review that uses quantitative methods to summarize the results

4

Steps to a Meta-analysis

Question Population, Intervention, Control, Outcome

Literature search Identify all relevant studies

Inclusion and exclusion criteria Select and critically appraise

Data abstraction Collect relevant information

Analysis

Guyatt G, Rennie D. User’s Guide to the Medical Literature (2002) ch 1E

5

How to critically appraise a meta-analysis?

1. Are the results of the study valid?

2. What are the results?

3. Will the results help me in caring for my patients?

6

Are the results of the study valid?

Did the overview address a focused clinical question?

Was the search for relevant studies detailed and exhaustive?

Were the primary studies of high methodologic quality?

Were assessments of studies reproducible?

Question

Literature search

Selection & appraisal

Data abstraction

7

What are the results?

Were the results similar from study to study?

What are the overall results of the review?

How precise were the results?

Analysis

8

Will the results help me in caring for my patients?

Are the results generalizable and can they be applied to my patients?

Were all clinically important outcomes considered?

Are the benefits worth the harms and cost?

Externalvalidity

9

Cell-Free Hemoglobin-Based Blood Substitutes and Risk of Myocardial Infarction and DeathNatason et al., JAMA, Prepublished online April 28, 2008 at http://jama.ama-assn.org/cgi/content/full/299.19.jrv80007

10

Hemoglobin-based blood substitutes (HBBS)

Ideally Universally available Eliminate need for refrigeration Long shelf-life Decreased risk of iatrogenic infection

But the challenge = free hemoglobin Scavenges nitric oxide --> vascular thrombosis

11

Possible solutions?

Chemically alter Hb to create larger, more stable HBBS molecule Cross-link Polymerize Pegylate

12Copyright restrictions may apply.

13

Question

The association between HBBSs and the risk of myocardial infarction and death

Additional purpose Examine regulatory process that permitted

repeated trials with theses agents despite safety concerns

14

Literature Search

Search Strategy PubMed, EMBASE, Cochrane Human RCT in English 1980 to March 25, 2008 “blood substitutes” and “hemoglobin”

Internet FDA advisory committee meeting Press releases from companies

15

Study Criteria

Inclusion RCT Outcome variable: Death or MI

Exclusion Did not involve a HBBS All patients were healthy volunteers or

younger than 19 years Results included in a subsequent report

16Copyright restrictions may apply.

17

Methods

Standard data collection form Outcomes: mortality and MI Descriptive data Intention-to-treat analysis used when

reported Missing data (1 treatment, 4 control)

analyzed both as survivors and nonsurvivors Independent review by 2 authors with a

3rd author resolving discrepancies

18

Statistics

“We assessed the homogeneity using Breslow-Day test and an associated I2 statistic”

To pool or not to pool? Tests of heterogeneity determine whether

the degree of variability in treatment effects between studies is greater than that expected by chance

I2 measures the extent of variation• 0% indicates no observed heterogeneity• Heterogeneity increases as I2 increases

19

To pool or not to pool? Forest plots

Pool if all point estimates are on the same side of the line or if confidence intervals overlap

20

Statistics

“Risks estimated using Cochran-Mantel-Haenszel test with 95% CI”

How to pool the results? Results typically weighted based on study

size (occasionally can be weighted based on quality of study)

21

Statistics

“A fixed effects model was required because of the null values for the estimates of between-study variance”

Models for pooling data in meta-analysis Fixed effects model

Assumes there is a single true value underlying all study results such that if all studies were infinitely large, they would yield identical estimates of the effect

Error comes only from within study variation

22

Models for pooling in meta-analysis

Random effects model Assumes that studies are a random sample

of a population of studies where each study estimates a different underlying true effect and the distribution of these effects is assumed normal around a mean value

Error comes from both within study variation and between study heterogeneity

Random effects model usually has a wider confidence interval

If little heterogeneity, compared to within-study variance, approximates fixed effects model

23

Results

16 trials of 5 HBBSs met inclusion criteria Type: 4 double-blind, 7 single-blind, 4 open-

label, 1 uninformative Patients: 5 trauma, 10 surgical, 1 stroke Outcomes: 12 used death, 10 used MI Median time from completion of trial to

publication 4 years (1 to 6 years)

Trials described in Table 2

24Copyright restrictions may apply.

NNH 62

25Copyright restrictions may apply.

NNH 50

26Copyright restrictions may apply.

27

Subgroup analyses

Results consistent regardless of Patient population Control: non-blood vs. blood product Product removed Type of product

• Low vs. High tetramer content • Low vs. High P50 content

Unpublished vs. published

28Copyright restrictions may apply.

Cumulative Mortality and MI

By 2000, 11 studies completed Mortality RR 1.27; 95% CI 0.99-1.63 MI RR 2.77; 95% CI 1.49-5.15

29

The regulatory process

Sponsors are required by law to report their results to the FDA after studies are completed (whether or not published)

Data are not made public unless product is approved or an advisory committee is convened

Article outlines pitfalls of this system including that 5 trials are ongoing and 1 is being planned

30

Are the results of the study valid?

Did the overview address a focused clinical question? PIO Intervention and outcome Population was varied and one might expect that this might

lead to variations in the absolute risk of mortality/MI but not necessarily the relative risk

Control varied but no difference in sub-group analysis

Was the search for relevant studies detailed and exhaustive? Yes, included unpublished studies and attempts to contact the

companies for more complete data Assumed? Hand searching the reference lists of the articles or

consulting experts in the area No change when unpublished studies excluded

31

Are the results of the study valid?

Were the primary studies of high methodologic quality? Restricted to RCT but all RCTs included Did not comment on validity of primary studies

Were assessments of studies reproducible? Yes 2 investigators reviewed the studies with a 3rd author

resolving any discrepancies

32

What are the results?

Were the results similar from study to study? Yes, both by I2 statistic and forest plot

What are the overall results of the review and how precise were the results? HBBS associated with increased death/MI Death: RR 1.30; 95% CI, 1.06-1.61 MI: RR 2.71; 95% CI, 1.67-4.40

33

Will the results help me in caring for my patients?

Are the results generalizable and can they be applied to my patients? Yes – covers a variety of conditions and settings

Were all clinically important outcomes considered? The study considered the adverse effects of therapy It did not report on the treatment effect though the

endpoints in Table 2 listed avoidance of allogeneic transfusion and in one case, disability score

Are the benefits worth the harms and cost? Weighing the risk of increased mortality and MI

against the risks of avoidance of allogeneic transfusion

No

34

Conclusion

Agree with authors’ conclusions “In the analysis of available data from clinical

trials, HBBSs was associated with a significantly increased risk of death and MI”

Questions?