View
5
Download
0
Category
Preview:
Citation preview
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Escher 3.2: towards effective, transparent andaccountable assessment of benefit-risk
using information technology and evidence synthesis
Gert van Valkenhoef
Department of Epidemiology, University Medical Center Groningen (NL),Faculty of Economics and Business, University of Groningen (NL)
Escher 3.x Cluster Meeting, 8 Dec 2010Utrecht, The Netherlands
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Outline
1 Introduction2 Meta-analysis3 Network meta-analysis4 Benefit-risk analysis5 Discussion
After every part, there will bean opportunity to askquestions.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Outline
1 Introduction2 Meta-analysis3 Network meta-analysis4 Benefit-risk analysis5 Discussion
After every part, there will bean opportunity to askquestions.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Escher 3.2 Goals
Develop a drug information system:
Effective knowledge access and management
Answer drug efficacy and safety questions
in an efficient, transparent and accountable waywithin and across compoundsfor a broad audience (including regulators)
Improve consistency in regulatory decision making
Based on systematic review and meta-analysis
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Effective knowledge access: problems
Review of existing systems:
Evidence-based decision making time-consuming/error-prone
No comprehensive source of trial information existsTrial information is insufficiently structured
Missed opportunities to introduce more structure
Trial registration, regulatory submission and systematic review
It is unclear how the information should be structured
Prototypes should be developed now, to discover thisRelated manuscripts:1) G. van Valkenhoef, T. Tervonen, B. de Brock, H. Hillege, Deficiencies in the transfer and availability of clinicalevidence in drug development and regulation. Manuscript under review.2) T. Tervonen, E.O. de Brock, P.A. de Graeff and H.L. Hillege (2010). Current status and future perspectives onDrug Information Systems.Proceedings of the 18th European Conference on Information Systems (ECIS2010),June 6-9, 2010, Pretoria, South Africa.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Prototype: global requirements
Interviews with major stakeholders
To develop the overall vision for the prototype
Database of clinical trialsAnswer efficacy/safety questionsStreamline benefit-risk decision makingFor regulatory authoritiesUsing aggregated data
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Aggregate data
Regulatory submissions in Europe contain aggregate data
EMA, SmPC, Galvus (EMEA/H/C/000771 -II/0007), updated 2010-04-27.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Aggregate data
Journal articles report aggregate data
Chouinard G, Saxena B, Belanger MC, Ravindran A, Bakish D, Beauclair L, et al. A Canadian multicenter,double-blind study of paroxetine and fluoxetine in major depressive disorder. J Affect Disord. 1999;54:39-48.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Aggregate data
Trials registered with ClinicalTrials.gov have aggregate results
GlaxoSmithKline, “Controlled-release Paroxetine in Major Depressive Disorder (Double-blind, Placebo-controlledStudy)”, ClinicalTrials.gov NCT00866294, updated October 14, 2010.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
ADDIS: Aggregate Data Drug Information System
Assisted evidence synthesis and benefit-risk assessment
Based on a database of clinical trials
Focussed on aggregated dataRelated manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems Knowledge, methods Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems
Knowledge, methods Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems Knowledge, methods
Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems Knowledge, methods Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems Knowledge, methods Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: concurrent engineering
Software Development
Methodology Research
Open problems Knowledge, methods Case studies
Feedback, use cases
Open problems
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: agile
ADDIS requirements highly uncertain
Only vague goals can be set
Much is expected to be discovered ‘on the way’
Agile software development
No full up-front specification of requirements
But: short-term plans and periodic re-evaluation
Supported by 2-3 part-time programmers (since Oct 2009)Related manuscripts:4) G. van Valkenhoef, T. Tervonen, B. de Brock, D. Postmus, Product and Release planning practices for ExtremeProgramming. Proceedings of the 11th International Conference on Agile Software Development (XP2010).5) G. van Valkenhoef, T. Tervonen, B. de Brock, D. Postmus, Quantitative release planning in ExtremeProgramming. Manuscript under review.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: agile
ADDIS requirements highly uncertain
Only vague goals can be set
Much is expected to be discovered ‘on the way’
Agile software development
No full up-front specification of requirements
But: short-term plans and periodic re-evaluation
Supported by 2-3 part-time programmers (since Oct 2009)Related manuscripts:4) G. van Valkenhoef, T. Tervonen, B. de Brock, D. Postmus, Product and Release planning practices for ExtremeProgramming. Proceedings of the 11th International Conference on Agile Software Development (XP2010).5) G. van Valkenhoef, T. Tervonen, B. de Brock, D. Postmus, Quantitative release planning in ExtremeProgramming. Manuscript under review.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: agile
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: open
Open development (http://drugis.org/)
Nightly builds (daily), development builds (bi-weekly)
Release: ca. every 3 months
Mailing list
Subscribe if you’re interested!
Public issue tracker
Anyone can report bugs and track progressRoadmap: short-term plans
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: open
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: open
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: open source
Open source
Aiming for scientific impact
Ensures others will be able to continue the project
Anyone worried about bugs can review the source code
Allows us to re-use many existing OSS components
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Development of ADDIS: open source
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Questions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: ‘dynamic Cochrane’ (automated meta-analysis)
Store trials in sufficient detail to do meta-analysis
Discover required data-model
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: ‘dynamic Cochrane’ (automated meta-analysis)
Store trials in sufficient detail to do meta-analysis
Discover required data-model
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Meta-analysis
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Meta-analysis
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Meta-analysis
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Meta-analysis
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Meta-analysis in ADDIS
Supported since ADDIS v0.4 (December 2009)
Database of trials + characteristics + outcomes
Development of data model
Related manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (1)
Hansen et al. (2005) systematic review:
46 studies comparing n = 10 second-generation AD
On efficacy (HAM-D responders) and adverse events
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (1)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (1)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (1)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
Hansen et al. Ann Intern Med 2005;143:415-426
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
Hansen et al. (2005) systematic review:
46 studies comparing n = 10 second-generation AD
Only 3 meta-analyses, all against fluoxetine
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venlaIs this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venlaIs this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venla
Is this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venlaIs this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venlaIs this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Limits of meta-analysis (2)
Fluoxetine
Paroxetine
Sertraline
Venlafaxine
6
5
6
3 1
2
How to compare paroxetine, sertraline and venlafaxine?
Can we compare sertraline/venlafaxine?
Only one direct trialIgnoring the 11 trials sertr-fluox-venlaIs this justified?
When comparing fluox/parox or fluox/sertr?
Can we ignore the 3 parox-sertr trials?
Parox as comparator → same conclusions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Conclusion
Meta-analysis is good if we compare two drugs
It is problematic for more
Selection bias: choice of common comparator?Are results of different comparisons consistent?
We need a way to include all trials/drugs
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Questions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: automated network meta-analysis
Meta-analysis of > 2 drugs
No existing software does this
Immediate value to scientific community
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: automated network meta-analysis
Meta-analysis of > 2 drugs
No existing software does this
Immediate value to scientific community
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Network meta-analysis
46 studies comparing n = 10 second-generation AD
Paroxetine
Bupropion
(1)
Duloxetine
(1)
Mirtazapine
(2)
Venlafaxine
(2)
Sertraline
(3)
(1)
Escitalopram
(2)
Fluoxetine
(8)
(2)
(1)
(1)
(7)
Fluvoxamine
(2)
(6)
Citalopram
(1)
(3) (1) (2)
(1) (2)
Network meta-analysis: include all evidence in one analysis
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Network Meta-Analysis models
Network meta-analysis models are difficult to specify
Automated in ADDISRelated manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.6) G. van Valkenhoef, T. Tervonen, B. de Brock, H. Hillege, Algorithmic Parameterization of Mixed TreatmentComparisons. Manuscript under review.7) G. van Valkenhoef, B. de Brock, H. Hillege, Automating network meta-analysis. Initiated (conference paper).
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Network Meta-Analysis models
Network meta-analysis models are difficult to specify
Automated in ADDIS
Related manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.6) G. van Valkenhoef, T. Tervonen, B. de Brock, H. Hillege, Algorithmic Parameterization of Mixed TreatmentComparisons. Manuscript under review.7) G. van Valkenhoef, B. de Brock, H. Hillege, Automating network meta-analysis. Initiated (conference paper).
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Network Meta-Analysis models
Network meta-analysis models are difficult to specify
Automated in ADDISRelated manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.6) G. van Valkenhoef, T. Tervonen, B. de Brock, H. Hillege, Algorithmic Parameterization of Mixed TreatmentComparisons. Manuscript under review.7) G. van Valkenhoef, B. de Brock, H. Hillege, Automating network meta-analysis. Initiated (conference paper).
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: data
Study Fluox Parox VenlaChouinard et al, 1999 67/101 67/102De Wilde et al, 1993 25/41 24/37Fava et al, 1998 31/54 32/55Fava et al, 2002 57/92 64/96Gagiano, 1993 27/45 30/45Schone and Ludwig, 1993 9/52 20/54Alves et al, 1999 30/47 25/40De Nayer et al, 2002 27/73 37/73Dierick et al, 1996 95/161 107/153Rudolph and Feiger, 1999 52/103 57/100Silverstone and Ravindran, 1999 77/121 84/128Tylee et al, 1997 58/170 67/171Ballus et al, 2000 23/43 25/41McPartlin et al, 1998 128/178 137/183
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: consistency
Fluox
Parox Venla
df ,vdf ,p
dp,v
pair-wise OR network OR
df ,p 1.24 (0.92, 1.67)
1.22 (0.92, 1.61)
df ,v 1.30 (1.03, 1.65)
1.34 (1.08, 1.67)
dp,v 1.20 (0.80, 1.82)
1.11 (0.82, 1.50)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: consistency
Fluox
Parox Venla
df ,vdf ,p
dp,v
pair-wise OR network OR
df ,p 1.24 (0.92, 1.67)
1.22 (0.92, 1.61)
df ,v 1.30 (1.03, 1.65)
1.34 (1.08, 1.67)
dp,v 1.20 (0.80, 1.82)
1.11 (0.82, 1.50)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: consistency
Fluox
Parox Venla
df ,vdf ,p
dp,v
pair-wise OR network OR
df ,p 1.24 (0.92, 1.67)
1.22 (0.92, 1.61)
df ,v 1.30 (1.03, 1.65)
1.34 (1.08, 1.67)
dp,v 1.20 (0.80, 1.82)
1.11 (0.82, 1.50)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: consistency
Fluox
Parox Venla
df ,v = df ,p + dp,vdf ,p
dp,vassume consistency: direct andindirect estimates lead to thesame conclusions.
pair-wise OR network OR
df ,p 1.24 (0.92, 1.67)
1.22 (0.92, 1.61)
df ,v 1.30 (1.03, 1.65)
1.34 (1.08, 1.67)
dp,v 1.20 (0.80, 1.82)
1.11 (0.82, 1.50)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: consistency
Fluox
Parox Venla
df ,v = df ,p + dp,vdf ,p
dp,vassume consistency: direct andindirect estimates lead to thesame conclusions.
pair-wise OR network OR
df ,p 1.24 (0.92, 1.67) 1.22 (0.92, 1.61)df ,v 1.30 (1.03, 1.65) 1.34 (1.08, 1.67)dp,v 1.20 (0.80, 1.82) 1.11 (0.82, 1.50)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Consistency
We assume consistency: direct and indirect estimates lead to thesame conclusions.
Estimate all relative effects simultaneously
Including all studies
Leading to consistent conclusions
Also estimate missing comparisons
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example
Fluox
Parox Venla
Sertr
Escit
Complexity → consistency greaterconcern
Pair-wise against Fluox, Escitexcluded
Yet, evidence suggests Escit>Fluox
How do the other drugs compare?
Escit 0.59 (0.37, 0.94) 0.69 (0.41, 1.15) 0.74 (0.44, 1.25) 0.81 (0.53, 1.24)Fluox 1.18 (0.91, 1.52) 1.27 (0.99, 1.63) 1.38 (1.10, 1.72)
Parox 1.08 (0.77, 1.51) 1.17 (0.86, 1.59)Sertr 1.09 (0.80, 1.48)
Venla
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Venla > Fluox, Escit > Fluox significant
Not much can be said about the rest
Which should we consider the best?
Which is the worst?
From a Bayesian perspective,
these questions are perfectly reasonable!
We can estimate the probability of Fluoxetine being
‘best’ (rank 1)‘second best’ (rank 2). . .‘worst’ (rank 5)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Venla > Fluox, Escit > Fluox significant
Not much can be said about the rest
Which should we consider the best?
Which is the worst?
From a Bayesian perspective,
these questions are perfectly reasonable!
We can estimate the probability of Fluoxetine being
‘best’ (rank 1)‘second best’ (rank 2). . .‘worst’ (rank 5)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Venla > Fluox, Escit > Fluox significant
Not much can be said about the rest
Which should we consider the best?
Which is the worst?
From a Bayesian perspective,
these questions are perfectly reasonable!
We can estimate the probability of Fluoxetine being
‘best’ (rank 1)‘second best’ (rank 2). . .‘worst’ (rank 5)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Venla > Fluox, Escit > Fluox significant
Not much can be said about the rest
Which should we consider the best?
Which is the worst?
From a Bayesian perspective,
these questions are perfectly reasonable!
We can estimate the probability of Fluoxetine being
‘best’ (rank 1)‘second best’ (rank 2). . .‘worst’ (rank 5)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Venla > Fluox, Escit > Fluox significant
Not much can be said about the rest
Which should we consider the best?
Which is the worst?
From a Bayesian perspective,
these questions are perfectly reasonable!
We can estimate the probability of Fluoxetine being
‘best’ (rank 1)‘second best’ (rank 2). . .‘worst’ (rank 5)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Extended example: rank probability
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Obstacles
Before the conclusions under consistency can be accepted:
Possible inconsistency should be evaluated:
First, by assessing the studies for exchangeabilitySecond, by statistical means (inconsistency/node-split model)
Assess convergence & run-length of the MCMC simulation
Reasonable priors have to be specified
All of these are research topics
And have (preliminary) implementations in ADDISRelated manuscripts:6) G. van Valkenhoef, T. Tervonen, B. de Brock, H. Hillege, Algorithmic Parameterization of Mixed TreatmentComparisons. Manuscript under review.7) G. van Valkenhoef, B. de Brock, H. Hillege, Automating network meta-analysis. Initiated (conference paper).
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Questions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Break!
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: quantitative benefit-risk model
Based on clinical trials or meta-analysis
Making trade-offs explicit
Making uncertainty explicit
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Introduction
ADDIS global requirements:
Database of clinical trials
Answer efficacy/safety questions
Streamline benefit-risk decision making
For regulatory authorities
Using aggregated data
Intermediate goal: quantitative benefit-risk model
Based on clinical trials or meta-analysis
Making trade-offs explicit
Making uncertainty explicit
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
A simple stochastic model
The ‘Lynd & O’Brien’ model:
Based on cost-effectiveness analysis techniques
Compares 2 alternatives
On 2 criteria (benefit vs. risk)
Sample (∆B,∆R) values from a joint distribution
Plot them on a plane
Count how many points are below the threshold µLynd, LD and O’Brien, BJ (2004), “Advances in risk-benefit evaluation using probabilistic simulation methods: anapplication to the prophylaxis of deep vein thrombosis.” Journal of Clinical Epidemiology 57(8):795–803.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Lynd & O’Brien example: set up (ADDIS)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Lynd & O’Brien example: set up (ADDIS)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Lynd & O’Brien example: set up (ADDIS)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Lynd & O’Brien example: data
0.0 0.2 0.4 0.6 0.8 1.0
02
46
8
probability
dens
ity
0.0 0.2 0.4 0.6 0.8 1.0
02
46
8
probability
dens
ity
Fluoxetine
0.0 0.2 0.4 0.6 0.8 1.0
02
46
8
probability
dens
ity
0.0 0.2 0.4 0.6 0.8 1.0
02
46
8
probability
dens
ity
Sertraline
−0.1 0.0 0.1 0.2 0.3
01
23
45
6
−0.2 −0.1 0.0 0.1 0.2
01
23
45
Difference
HAM-D
Dropouts
57/92 70/96
24/92 26/96
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Lynd & O’Brien example: results (ADDIS)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µB better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µ
B better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µ
B better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
Trade-off
Trade-off
µ
B better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µThe acceptability threshold.
We are willing to ‘pay’ µ
units risk to get 1 unit of
benefit.
B better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µThe acceptability threshold.
We are willing to ‘pay’ µ
units risk to get 1 unit of
benefit.
B better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Benefit-risk plane
+Benefit A+Benefit B
+R
isk
A+
Ris
kB
µB better
A better
p = aa+b
count b
count a
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Example: acceptability curve (ADDIS)
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA BR analysis
The Lynd & O’Brien model is limited to 2x2 problems.
Stochastic Multi-criteria Acceptability Analysis (SMAA)allows m × n problems:
m alternativesevaluated on n criteriaperformance of alternative i on criterion j : Ci,j ∼ f (ci,j )
Related manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.8) T. Tervonen, G. van Valkenhoef, E. Buskens, H. Hillege, D. Postmus, A stochastic multi-criteria model forevidence-based decision making in drug benefit-risk analysis. Manuscript under review.9) G. van Valkenhoef, T. Tervonen, J. Zhao, B. de Brock, H. Hillege, D. Postmus, Multi-criteria benefit-riskassessment using network meta-analysis. Partial manuscript.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA BR analysis
SMAA models for benefit-risk:
Can be based on a single trial (8)
Or (network) meta-analysis (9)
And is implemented in ADDIS (3)Related manuscripts:3) G. van Valkenhoef, T. Tervonen, T. Zwinkels, B. de Brock, H. Hillege, ADDIS: a decision support system forevidence-based medicine. Manuscript under review.8) T. Tervonen, G. van Valkenhoef, E. Buskens, H. Hillege, D. Postmus, A stochastic multi-criteria model forevidence-based decision making in drug benefit-risk analysis. Manuscript under review.9) G. van Valkenhoef, T. Tervonen, J. Zhao, B. de Brock, H. Hillege, D. Postmus, Multi-criteria benefit-riskassessment using network meta-analysis. Partial manuscript.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
SMAA modelbased on networkmeta-analysis.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Measurements (input distributions).
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Model without preference information.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Model without preference information.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Preferences for severe depression.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Severe depression results.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Preferences for mild depression.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
SMAA example (ADDIS)
Mild depression results.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Relevance: EMA BR methodology project
Approach/method Relevance to regulators UsefulnessProbabilistic simulation Can illuminate the risk/benefit trade-off when uncertainty is a major
feature of a regulatory decision.Medium
Bayesian statistics Can integrate evidence and its uncertainty, both pre- and post-approval, with multiple criteria in decision models.
High
MCDA Multi-criteria decision analysis extends decision theory to accommo-date multiple, conflicting objectives. Provides common units of valuefor both benefits and risks.
High
Table: MTC/SMAA integrates 2 of 3 quantitative approaches rated’High’ on usefulness, and 1 rated ’Medium’.
EMA (2010). Benefit-risk methodology project work package 2 report: Applicability of current tools and processesfor regulatory benefit-risk assessment. EMA/549682/2010.
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Questions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Escher 3.2 progress
ADDIS software:
database of trialsautomated (network) meta-analysisstochastic benefit-risk models
Research:
Survey of exisiting information systemsAutomating network meta-analysisDevelopment of benefit-risk method
Publications:
Presented at a number of conferencesSeveral papers under review (5)Journal and conference paper in preparationCase study being initiated
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Escher 3.2 progress
ADDIS software:
database of trialsautomated (network) meta-analysisstochastic benefit-risk models
Research:
Survey of exisiting information systemsAutomating network meta-analysisDevelopment of benefit-risk method
Publications:
Presented at a number of conferencesSeveral papers under review (5)Journal and conference paper in preparationCase study being initiated
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Escher 3.2 progress
ADDIS software:
database of trialsautomated (network) meta-analysisstochastic benefit-risk models
Research:
Survey of exisiting information systemsAutomating network meta-analysisDevelopment of benefit-risk method
Publications:
Presented at a number of conferencesSeveral papers under review (5)Journal and conference paper in preparationCase study being initiated
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Future plans
Case studies
(Network) meta-regression
Handle covariates (dose, baseline severity, . . . )Refine the data model
Extend benefit-risk model
Hierarchical model/value treeQualitative attributes
More links with data sources, data sharing
A collaborative web portal?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Questions?
Introduction Meta-analysis Network meta-analysis Benefit-risk analysis Discussion
Thank you!
Recommended