View
25
Download
2
Category
Tags:
Preview:
DESCRIPTION
THE INCORPORATION OF META-ANALYSIS RESULTS INTO EVIDENCE-BASED DECISION MODELLING. Nicola Cooper, Alex Sutton, Keith Abrams, Paul Lambert Department of Epidemiology & Public Health, University of Leicester. PSI Meeting “Statistical Advances in Health Technology Assessment ” 10 th June 2003. - PowerPoint PPT Presentation
Citation preview
THE INCORPORATION OF META-ANALYSIS RESULTS
INTO EVIDENCE-BASED DECISION
MODELLINGNicola Cooper, Alex Sutton, Keith Abrams,
Paul LambertDepartment of Epidemiology & Public Health, University
of Leicester.PSI Meeting “Statistical Advances in Health
Technology Assessment ”10th June 2003
• Increasingly decision models are being developed to inform complex clinical/economic decisions
• Parameters can include: –clinical effectiveness, –costs, –disease progression rates, and –utilities
• Evidence based - use systematic methods for evidence synthesis to estimate model parameters with appropriate levels of uncertainty
BACKGROUND
• Statistical error
• Systematic error
• Evidence relating to parameters indirectly
• Data quality, publication bias, etc.
SOURCES OF UNCERTAINTY IN
DECISION MODELS
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
81
0
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
81
0
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
81
0
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
81
0
RCT1 RCT2 RCT3 OBS1 OBS2 ROUTINE EXPERTDATA SOURCES
Gen. synthesisMeta-analysisEVIDENCESYNTHESIS
COMPREHENSIVE DECISION MODEL FRAMEWORK
DECISIONMODEL Stroke
No strokeTreating patients with atrial fibrillation?
Warfarin
No warfarin
Stroke
No stroke
Bleed
No bleed
Bleed
No bleed
Bleed
No bleed
Bleed
No bleed
….. …..….. …..
….. …..….. …..….. …..….. …..
….. …..….. …..
Clinical Effect
MODEL INPUTS
Adverse Events
Utility Cost
Opinion pooling
Bayes theorem In combination
EXAMPLES
1) Net Clinical Benefit Approach
• Warfarin use for atrial fibrillation
2) Simple Economic Decision Model
• Prophylactic antibiotic use in caesarean section
3) Markov Economic Decision Model
• Taxane use in advanced breast cancer
1) Net Clinical Benefit Approach
• Warfarin use for atrial fibrillation
2) Simple Economic Decision Model
• Prophylactic antibiotic use in caesarean section
3) Markov Economic Decision Model
• Taxane use in advanced breast cancer
1. Meta-analyse available evidence to obtain a distribution for each model parameter using random effect models
2. Transformation of the pooled results, if necessary, and input into the model directly as a distribution and evaluate the model
3. All analyses (decision model and subsidiary analyses) implemented in one cohesive statistical model/program
4. Implemented in a fully Bayesian way using Markov chain Monte Carlo simulation within WinBUGS software
5. All prior distributions intended to be ‘vague’. Where uncertainty exists in the value of parameters (i.e. most of them!) they are treated as random variables
GENERAL APPROACH
Warfarin for Non-Rheumatic Atrial Fibrillation
• Evidence that post MI, the risk of a stroke is reduced in patients with atrial fibrillation by taking warfarin
• However, there is a risk of a fatal hemorrhage as a result of taking warfarin
• Do the benefits outweigh the risks?
EXAMPLE 1: NET CLINICAL
BENEFIT
EVALUATION OF NET BENEFIT
BENEFITS minus HARMS = NET CLINICAL BENEFIT
(if NCB >0 benefits outweigh harms)
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
EVALUATION OF NET BENEFIT
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
Multivariate riskequations
Meta analysisof RCTs
Metaanalysis of RCTs obs studies QoL study
EVALUATION OF NET BENEFIT
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
0.002 0.004 0.006 0.008 0.010 0.012 0.014
050
100
150
200
250
300
risk of bleed per year
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
810
-1.5 -1.0 -0.5 0.0 0.5 1.0
02
46
reduction in relative risk
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.4
Outcome ratio
Multivariate riskequations
Meta analysisof RCTs
Metaanalysis of RCTs obs studies QoL study
EVALUATION OF NET BENEFIT
Multivariate Risk Equation Data Net Benefit (measured in stroke equivalents)
% of
cohort
T hrombo -
embolism
rate (%
per year
(95% CI))
Mean
(s.e.)
Median
(95%
CrI)
Probability of
Benefit > 0
Simulated PDF
12
17.6 (10.5
to 29.9)
- 0.0004
(0.15)
0.06
( - 0.29 to
0.20)
54.2 %
-0.8 -0.6 -0.4 -0.2 0.0 0.2 0.4
01
23
45
6
2 or 3 Clinical factors
EVALUATION OF NET BENEFIT
(Risk of stroke Relative reduction in risk of stroke)
- (Risk of fatal bleed Outcome ratio)
=
Net Benefit
0.002 0.004 0.006 0.008 0.010 0.012 0.014
050
100
150
200
250
300
risk of bleed per year
-2.95 -2.90 -2.85 -2.80 -2.75 -2.70 -2.65
02
46
810
-1.5 -1.0 -0.5 0.0 0.5 1.0
02
46
reduction in relative risk
0 20 40 60 80 100
0.0
0.1
0.2
0.3
0.4
Outcome ratio
Multivariate riskequations
Meta analysisof RCTs
Metaanalysis of RCTs obs studies QoL study
EVALUATION OF NET BENEFIT
“TAKE-HOME”
POINTS 1
Net-benefit provides a transparent quantitative framework to weigh up benefits and harms of an intervention
Utilises results from two meta-analyses and allows for correlation induced where studies included in both benefit and harm meta-analyses
Credible interval for net benefit can be constructed allowing for uncertainty in all model parameters
Use of Prophylactic Antibiotics to Prevent Wound Infection following Caesarean Section
EXAMPLE 2: SIMPLE DECISION TREE
No infection (1-p2) Cost with antibiotics
Yes
Infection (p2) Cost with antibiotics + Cost of treatment
Prophylactic antibiotics?
No infection (1-p1) Cost with no antibiotics
No
Infection (p1) Cost of treatment
1) Cochrane review of 61 RCTs evaluating prophylactic antibiotics use for caesarean section
2) Event data rare: use “Exact” model for RR 3) Meta-regression: Does treatment effect vary with patients’
underlying risk (pc)?
ln(RRadjusted ) = ln(RRaverage)+ [ln(pc) - mean(ln(pc))]4) Risk of infection without treatment from ‘local’ hospital
data (p1)5) Derive relative risk of treatment effect for ‘local’ hospital
(using regression equation with pc=p1)6) Derive risk of infection if antibiotics introduced to ‘local’
hospital (p2)
p2 = p1 * RRadjusted
METHOD OUTLINE
UNDERLYING BASELINE RISK
ln(R
ela
tive
Ris
k)
ln(control group risk) centred on mean)
ln(relative risk) fit
-2.5 -2 -1.5 -1 -.5 0 .5 1 1.5
-3
-2.5
-2
-1.5
-1
-.5
0
.5
1
1.5
2=0.24
(-0.28 to 0.81)
No treatment effect
Local hospital event rate
RESULTSrr[1] sample: 20000
0.1 0.2 0.3 0.4 0.5
0.0 2.5 5.0 7.5 10.0
RR
diff[1] sample: 20000
-150.0 -100.0 -50.0
0.0 0.01 0.02 0.03 0.04
cost using antibiotics
-£49.53 (-£77.09 to -£26.79)
p1 sample: 20000
0.025 0.075 0.1 0.125
0.0
20.0
40.0
60.0p1
No infection (1-p2) Cost with antibiotics
Yes
Infection (p2) Cost with antibiotics + Cost of treatment
Prophylactic antibiotics?
No infection (1-p1) Cost with no antibiotics
No
Infection (p1) Cost of treatment
p2[1] sample: 20000
0.0 0.02 0.04
0.0 25.0 50.0 75.0 100.0
p20.02
(0.02 to 0.03)
p1 sample: 20000
0.025 0.075 0.1 0.125
0.0
20.0
40.0
60.0 0.08 (0.06 to 0.10)
p1
SENSITIVTY OF PRIOR DISTRIBUTIONS
[1]
[2]
[3]
caterpillar plot
Cost difference -80.0 -60.0 -40.0 -20.0
[1] Gamma(0.001,0.001) on 1/variance
[2] Normal(0,1.0-6) truncated at zero on 1/sd
[3] Uniform(0,20) on 1/sd
[1]
[3]
[2]
Caterpillar plot
-80 -60 -40 -20Cost
“TAKE-HOME” POINTS 2
Incorporates M-A into a decision model adjusting for a differential treatment effect with changes in baseline risk
Meta-regression model takes into account the fact that covariate is part of the definition of outcome
Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propogated
Sensitivity of overall results to prior distribution placed on the random effect term in a M-A
Stable
Progressive
Death
EXAMPLE 3: MARKOV MODEL
Response
Cycle length 3 weeks
QR , CR QS , CS
QP , CP
QD = 0
Quality of Life (Q)Cost (C)
PSP
PPD
PRP
PR PS
PP
Probability (P)
PSR
Taxanes - 2nd line treatment of advanced breast cancer
MODEL PARAMETER
ESTIMATION PSR, TAX – The probability of moving from
stable to response in a 3 week period
3) Transformation of ln(odds)distrn to transition probability
)3/52/(1
/1
]42.01[1
)],(1[1
jjo ttP
mu.rsprtD sample: 12001
-5.0 0.0 5.0
0.0 0.5 1.0 1.5 2.0
2) Pooled ln(odds) distribution1) M-A of RCTs: Annual ln(odds) of responding
Odds - log scale.1 .25 1 5
Combined
Bonneterre
Sjostrom
Nabholtz
Chan
-0.3 (-0.9 to 0.3)
4) Apply to model
Respond
Stable
Progressive
Death
PSR
THE REMAINING PARAMETERS
• The Transition Probabilities need estimating for each intervention being compared
• Costs & Utilities can be extracted from the literature and synthesised using a similar approach within the same framework
META-ANALYSES OF
LITERATURE(where required) No. of
studies Time in weeks
(95% Credible Interval) Progression-free time 3 25 (15 to 24)
Time to response from stable 1 12 (6 to 18) Time to progressive from response 1 35 (29 to 41)
Overall survival time 3 53 (35 to 74) Probabilities
Response rate 4 0.43 (0.29 to 0.58) % moving directly to progressive at stage 2. 1 0.13 (0.08 to 0.18)
% with infections / febrile neutropenia 3 0.18 (0.04 to 0.56) % hospitalised with infection / febrile neutropenia 1 0.08 (0.05 to 0.11)
% dying from infections / febrile neutropenia 1 0.01 (0.00 to 0.02) % discontinue treatment due to adverse event 3 0.16 (0.03 to 0.49)
% with Neutropenia grades 3 & 4 2 0.94 (0.82 to 0.98) % with Anaemia grades 3 & 4 2 0.03 (0.00 to 0.28) % with Diarrhoea grades 3 & 4 3 0.09 (0.06 to 0.14) % with Stomatis grades 3 & 4 3 0.08 (0.04 to 0.14) % with vomiting grades 3 & 4 2 0.03 (0.00 to 0.12)
% with fluid retention grades 3 & 4 3 0.05 (0.02 to 0.12) % with cardiac toxicity grades 3 & 4 1 0.00 (0.00 to 0.02)
TRANSITION PROBABILITIES
FOR MODEL (Derived from M-As)
Transition Probabilities
(95% Credible Interval)
Infection/FN 0.09 (0.02 to 0.32)
Hospitalised due to infection/FN 0.04 (0.03 to 0.05)
Dying from infection/FN after hospitalisation 0.00 (0.00 to 0.01)
Discontinuation due to major adverse events 0.04 (0.04 to 0.16)
Adverse events – Neutropenia 0.50 (0.34 to 0.63)
Adverse events – Anaemia 0.01 (0.00 to 0.07)
Adverse events – Diarrhoea 0.02 (0.01 to 0.37)
Adverse events – Stomatis 0.02 (0.01 to 0.04)
Adverse events – Vomiting 0.01 (0.00 to 0.03)
Adverse events – Fluid retention 0.01 (0.00 to 0.03)
Adverse events – Cardiac toxicity 0.00 (0.00 to 0.01)
Transition directly to ‘progressive’ state 0.12 (0.08 to 0.18)
Transition ‘stable’ to ‘stable’ 0.65 (0.44 to 0.75)
Transition ‘stable’ to ‘response’ 0.16 (0.11 to 0.28)
Transition ‘stable’ to ‘progressive’ 0.18 (0.11 to 0.37)
Transition ‘response’ to ‘response’ 0.94 (0.93 to 0.95)
Transition ‘response’ to ‘progressive’ 0.06 (0.05 to 0.07)
Transition ‘progressive’ to ‘progressive’ 0.93 (0.79 to 0.96)
Transition ‘progressive’ to ‘death’ 0.07 (0.04 to 0.21)
EVALUATION OF THE MODEL
• A cohort of 1,000 persons is run through the model over 35 3-weekly cycles (until the majority of people are dead) for each treatment option
• Costs and utilities are calculated at the end of each cycle and the average cost and utilities for an individual across all 35 cycles for each treatment option are calculated
• This process is repeated 4,000 times (each time different values from each parameter distribution are sampled)
Bayesian (MCMC) Simulations
-£4,000
-£2,000
£0
£2,000
£4,000
£6,000
£8,000
£10,000
-0.50 -0.40 -0.30 -0.20 -0.10 0.00 0.10 0.20 0.30 0.40 0.50
Incremental utility
Inc
rem
en
tal
co
st
Standard dominates
Taxane more effective but more costly
Taxane less costly but less
effective
Taxane
dominates
COST-EFFECTIVENESS PLANE
NW NE
SW SE
C-E ACCEPTABILITY CURVE
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
£0 £50,000 £100,000 £150,000 £200,000 £250,000
Value of ceiling ratio, Rc (£)
Pro
babi
lity
cost
-effe
ctiv
e
'vague' priors
ELICITATION OF PRIORS
e.g. Response RateTaxane
x
x
x x x
x x x x x
x x x x x
x x
x x
x
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
Standard
x
x
x x x x
x x x x x
x x x x x
x x
x x
0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% 55% 60% 65% 70% 75% 80% 85% 90% 95% 100%
SENSITIVITY ANALYSIS
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
£0 £50,000 £100,000 £150,000 £200,000 £250,000
Value of ceiling ratio, Rc (£)
Pro
babi
lity
cost
-effe
ctiv
e
Expert opinion
'vague' priors
“TAKE-HOME” POINTS 3
Synthesis of evidence, transformation of variables & evaluation of a complex Markov model carried out in one unified framework (facilitating sensitivity analysis)
Provides a framework to incorporate prior beliefs of experts
ADVANTAGES OF APPROACH
Synthesis of evidence, transformation of variables & evaluation of a complex decision model carried out in a unified framework
Facilitates sensitivity analysis
Provides a framework to incorporate prior beliefs of experts
Allows for correlation induced where studies included in the estimation of more than one parameter
Uncertainty in all model parameters automatically taken into account
Rare event data modelled ‘exactly’ (i.e. removes the need for continuity corrections) & asymmetry in posterior distribution propagated
FURTHER ISSUES
1. Handling indirect comparisons correctly• E.g. Want to compare A vs. C but evidence
only available on A vs. B & B vs. C etc.• Avoid breaking randomisation
2. Necessary complexity of model?• When to use the different approaches outlined
above?
3. Incorporation of Expected Value of (Perfect/Sample) Information
4. Incorporation of all uncertainties
1. Handling indirect comparisons correctly• E.g. Want to compare A vs. C but evidence
only available on A vs. B & B vs. C etc.• Avoid breaking randomisation
2. Necessary complexity of model?• When to use the different approaches outlined
above?
3. Incorporation of Expected Value of (Perfect/Sample) Information
4. Incorporation of all uncertainties
REFERENCES
1. Cooper NJ, Abrams KR, Sutton AJ, Turner D, Lambert P. Use of Bayesian methods for Markov modelling in cost-effectiveness analysis: An application to taxane use in advanced breast cancer. Journal of the Royal Statistical Society Series A 2003; 166(3).
2. Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A. Comprehensive decision analytical modelling in economic evaluation: A Bayesian approach. Health Economics 2003 (In press)
3. Cooper NJ, Sutton AJ, Abrams KR. Decision analytical economic modeling within a Bayesian framework: Application to prophylactic antibiotics use for caesarean section. Statistical Methods in Medical Research 2002;11: 491-512.
4. Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR. Synthesising both benefit and harm: A Bayesian approach to evaluating clinical net benefit. (Submitted to Journal of Clinical Epidemiology).
1. Cooper NJ, Abrams KR, Sutton AJ, Turner D, Lambert P. Use of Bayesian methods for Markov modelling in cost-effectiveness analysis: An application to taxane use in advanced breast cancer. Journal of the Royal Statistical Society Series A 2003; 166(3).
2. Cooper NJ, Sutton AJ, Abrams KR, Turner D, Wailoo A. Comprehensive decision analytical modelling in economic evaluation: A Bayesian approach. Health Economics 2003 (In press)
3. Cooper NJ, Sutton AJ, Abrams KR. Decision analytical economic modeling within a Bayesian framework: Application to prophylactic antibiotics use for caesarean section. Statistical Methods in Medical Research 2002;11: 491-512.
4. Sutton AJ, Cooper NJ, Abrams KR, Lambert PC, Jones DR. Synthesising both benefit and harm: A Bayesian approach to evaluating clinical net benefit. (Submitted to Journal of Clinical Epidemiology).
Recommended