Economic Evaluation using Decision Analytic Modelling I Mira Johri Université de Montréal 2008-10-27

Economic Evaluation using Decision

Analytic Modelling I

Mira Johri

Université de Montréal

2008-10-27

2008-10-27 2

Outline (Part 1)

1. Overview Basic features of a decision model Rationale for modelling

2. Decision trees Conventions Steps to perform a decision analysis Blindness prevention example Strengths & weaknesses

I. Overview

2008-10-27 4

What is a model?

A health care evaluation model is a logical mathematical framework that permits the integration of facts and values and that links these data to outcomes that are of interest to health care decision makers.

Its purpose is to structure evidence on clinical and economic outcomes to help inform decisions about clinical practice and health-care resource allocation under conditions of uncertainty.

2008-10-27 5

What is a model? Cont’d In the context of economic evaluation, a decision

analytic model uses mathematical relationships to define a series of possible consequences (health and economic outcomes of patients or populations) that would flow from the alternative options being evaluated.

It enables us to structure and examine a complex problem.

2008-10-27 6

Example

Should we vaccinate children against meningitis? To vaccinate or not to vaccinate?

It depends: Risk of exposure to bacillus Risk of contracting meningitis Efficacy of the vaccine Risks associated with the vaccine

2008-10-27 7

Meningitis

Vaccinate

Do not vaccinateChoice

2008-10-27 8

Meningitis

Exposed

Not exposedVaccinate

Exposed

Not exposedDo not vaccinate

Choice

2008-10-27 9

Meningitis

Meningitis

No meningitisExposed

Not exposed

Vaccinate

Meningitis

No meningitisExposed

Not exposed

Do not vaccinate

Choice

2008-10-27 10

Quantifying probabilities & outcomes

Based on the inputs of the model, the likelihood of each consequence (health and economic outcomes of patients or populations) is expressed in terms of probabilities, and each consequence has a cost and an outcome.

Risk of exposure to bacillus (0,2)

Risk of contracting meningitis if exposed (0,1)

Efficacy of the vaccine (risk of meningitis if vaccinated 0,01)

Risks associated with the vaccine (0,003)

2008-10-27 11

Dead,5

0

Dead

,41

Alive

,60

Meningitis

,01

No meningitis,99

0

Exposed

,2

Not exposed

,80

No complications

,5

Vaccinate

Dead

,41

Alive,6

0

Meningitis

,1

No meningitis

,90

Exposed,2

Not exposed

,80

Do not vaccinate

Choice

2008-10-27 12

Method of analysis

Calculate the expected cost and expected outcome of each option under evaluation.

For a given option, the expected cost (outcome) is the sum of the costs (outcomes) of each consequence weighted by the probability of that consequence.

Objective: to maximise (minimise) the expected value of positive (negative) outcomes

o In CEA, the analyses are done separately for costs and health outcomes

2008-10-27 13

Meningitis VaccinationDead,5

0

Dead

,41

Alive

,60

Meningitis

,01

No meningitis,99

0

Exposed

,2

Not exposed

,80

No complications

,5

Vaccinate

Dead

,41

Alive,6

0

Meningitis

,1

No meningitis

,90

Exposed,2

Not exposed

,80

Do not vaccinate

Choice

#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4) = 3,8 per 1 000 children

#deaths without vaccination = 0,2 x 0,1 x 0,4 = 8,0 per 1 000 children

1

2008-10-27 14

Meningitis VaccinationDead,5

0

Dead

,41

Alive

,60

Meningitis

,01

No meningitis,99

0

Exposed

,2

Not exposed

,80

No complications

,5

Vaccinate

Dead

,41

Alive,6

0

Meningitis

,1

No meningitis

,90

Exposed,2

Not exposed

,80

Do not vaccinate

Choice

#deaths with vaccination = 0,003 + (0,997 x 0,2 x 0,01 x 0,4) = 3,8 per 1 000 children

#deaths without vaccination = 0,2 x 0,1 x 0,4 = 8,0 per 1 000 children

1

2008-10-27 15

Rationale: Synthesis Economic evaluation studies must use all relevant evidence

(principle of EBM).

All evidence rarely comes from a single source.

A decision-model provides a framework in which a range of evidence can by synthesised and brought to bear on the decision problem. Characterise natural history of a given condition Impact of alternative interventions Costs and health effects contingent on clinical events Relationship between intermediate clinical measure of

effect and ultimate measure of health gain required for CEA

(Adapted from Briggs et al., 2006)

2008-10-27 16

Rationale: Consideration of all relevant comparators Cost-effectiveness must be established in

comparison with all alternative options that could feasibly be used in practice.

A single study or RCT would rarely consider the full range of options.

Data must be brought together from clinical studies using appropriate statistical synthesis methods.

The decision model provides the framework to bring the synthesis to bear on the decision problem.


2008-10-27 17

Rationale: Appropriate time horizon

CEA requires that studies adopt a time horizon that is sufficiently long to reflect all key differences between options in terms of costs and effects.

For many interventions, this will effectively require a lifetime horizon.

Decision models become the framework within which to structure the extrapolation of costs and effects over time.


2008-10-27 18

Rationale: Uncertainty

CEA must indicate how uncertainty in the available evidence relating to a given policy problem translates into decision uncertainty; that is, the probability that a given decision is the correct one.

Through various forms of sensitivity analysis, decision models can help to characterise this uncertainty and present it to decision makers.


II. Decision Trees

2008-10-27 20

Decision Trees - Conventions Events are ordered from left to right.

Temporal order is often followed for sake of clarity.

Different kinds of events are distinguished using shapes called “nodes” Square – a decision node indicating a choice facing the

decision maker, typically at start of tree Circle – a chance node represents an event which has

multiple possible outcomes and is not under the decision maker’s control. For an individual patient, which event they experience is uncertain.

Triangle – a terminal node denotes the endpoint of a scenario.

2008-10-27 21

Decision Trees - Conventions Branches “sprouting” from a decision node represent the set of

actions being considered (strategies) They need not be mutually exclusive (e.g. A, B, A+B)

Branches from a chance node represent the set of possible outcomes of the event Must be mutually exclusive and exhaustive Probabilities must sum to 1.0

Terminal nodes are assigned a value – referred to generically as a payoff

Cost Utility QALY

Expected values are based on the summation of the pathway values weighted by the pathway probabilities.

2008-10-27 22

Steps in Decision Analysis

1. Define the problem

2. Structure the decision and make a tree

3. Fill in the data (probabilities and outcomes)

4. Choose the consequence with the maximum expected value (“roll back the tree”)

5. Perform sensitivity analysis

6. Interpret results

2008-10-27 23

1. Blindness prevention Hypothetical population presenting clinical signs of a

possible, early-stage autoimmune disorder.

If the condition is present, and if it progresses, blindness will result.

An imperfect test (biopsy, with the possibility of false negatives) can help determine whether an individual has the disorder

There exists an effective and inexpensive therapy which lowers probability of progression to blindness.

Side effects of treatment are factored into the costs and life expectancy at the end of each path.

2008-10-27 24

2. The Data

Disease prevalence = 0,5

pBlind (without Tx)=0,12

pBlindTreat = 0,013 TestSens = 0,8 TestSpec = 1

Biopsy cost = $150 Drug therapy cost =

$680 Costs associated with

Blindness = $40 000 Quality-adjusted life

expectancies as given in tree

2008-10-27 25

2 & 3. Blindness prevention - effectiveness

Blindness0,12

6.936

No blindness

#11.56

Disorder present

0,5

No disorder

#13.6

Treat None

Blindness0,013

7.536

No blindness

#12.555

True positive test,treat w/ drug

0,8

Blindness

0,126.933

No blindness#

11.556

False negative test,don't treat

#

Disorder present

0,5

False positive test,treat w/ drug

#12.555

True negative test,don't treat

113.59

No disorder#

Biopsy

Blindness

0,0137.539

No blindness

#12.565

Disorderpresent

0,5

No disorder

#12.565

Treat All

Treatment Options

2008-10-27 26

3. Blindness prevention - effectiveness

Blindness0,12

6.936

No blindness

#11.56

Disorder present

0,5

No disorder

#13.6

Treat None

Blindness0,013

7.536

No blindness

#12.555


0,8

Blindness

0,126.933

No blindness#

11.556


#

Disorder present

0,5


#12.555


113.59

No disorder#

Biopsy

Blindness

0,0137.539

No blindness

#12.565

Disorderpresent

0,5

No disorder

#12.565

Treat All

Treatment Options

What is the value of the decrement in QALYs associated with

Having the disorder? Biopsy? Treatment?

2008-10-27 27

3. Blindness prevention - effectiveness

Blindness0,12

6.936

No blindness

#11.56

Disorder present

0,5

No disorder

#13.6

Treat None

Blindness0,013

7.536

No blindness

#12.555


0,8

Blindness

0,126.933

No blindness#

11.556


#

Disorder present

0,5


#12.555


113.59

No disorder#

Biopsy

Blindness

0,0137.539

No blindness

#12.565

Disorderpresent

0,5

No disorder

#12.565

Treat All

Treatment Options

What is the value of the decrement in QALYs associated with

Having the disorder? 13,6-11,56 = 2,04

Biopsy? 13,6-13,59 = 0,1

Treatment? 13,6-12,565 = 0,035

2008-10-27 28

Blindness0,12

40,000 / (6.936)

No blindness

#0 / (11.56)

Disorder present

0,5

No disorder

#0 / (13.6)

Treat None

Blindness0,013

40,830 / (7.536)

No blindness

#830 / (12.555)


0,8

Blindness

0,1240,150 / (6.933)

No blindness#

150 / (11.556)


#

Disorder present

0,5


#830 / (12.555)


1150 / (13.59)

No disorder#

Biopsy

Blindness

0,01340,680 / (7.539)

No blindness

#680 / (12.565)

Disorderpresent

0,5

No disorder

#680 / (12.565)

Treat All

Treatment Options

3. Blindness prevention – costs & effectiveness

2008-10-27 29

4. Blindness prevention – expected QALYs

Blindness0,120

6,000 QALYs

No blindness

0,88011,000 QALYs

Disorder present

0,50010,400 QALYs

No disorder

0,50013,000 QALYs

Treat None11,700 QALYs

Blindness0,013

7,000 QALYs; P = 0,005

No blindness

0,98712,000 QALYs; P = 0,395


0,80011,935 QALYs

Blindness

0,1206,000 QALYs; P = 0,012

No blindness0,880

11,000 QALYs; P = 0,088


0,20010,400 QALYs

Disorder present

0,50011,628 QALYs


0,00012,000 QALYs


1,00013,000 QALYs; P = 0,500

No disorder0,500

13,000 QALYs

Biopsy12,314 QALYs

Blindness

0,0137,000 QALYs

No blindness

0,98712,000 QALYs

Disorderpresent

0,50011,935 QALYs

No disorder

0,50012,000 QALYs

Treat All11,967 QALYs

Treatment Options Biopsy : 12,314 QALYs

2008-10-27 30

4. Blindness prevention – expected costs

Blindness0,120

$40 000

No blindness

0,880$0

Disorder present

0,500$4 800

No disorder

0,500$0

Treat None$2 400

Blindness0,013

$40 830

No blindness

0,987$830


0,800$1 350

Blindness

0,120$40 150

No blindness0,880

$150


0,200$4 950

Disorder present

0,500$2 070


0,000$830


1,000$150

No disorder0,500

$150

Biopsy$1 110

Blindness

0,013$40 680; P = 0,006

No blindness

0,987$680; P = 0,493

Disorderpresent

0,500$1 200

No disorder

0,500$680; P = 0,500

Treat All$940

Treatment OptionsdiseasePrev=0,5pBlind=0,12pBlindTreat=0,013testSens=0,8testSpec=1

Treat All : $940

2008-10-27 31

4. Blindness prevention – expected costs & QALYs

Blindness0,120

$40 000 / 6,936 QALYs

No blindness

0,880$0 / 11,560 QALYs

Disorder present

0,500$4 800 / 11,005 QALYs

No disorder

0,500$0 / 13,000 QALYs

Treat None$2 400 / 12,003 QALYs

Blindness0,013

$40 830 / 7,536 QALYs

No blindness

0,987$830 / 12,555 QALYs


0,800$1 350 / 12,490 QALYs

Blindness

0,120$40 150 / 6,933 QALYs

No blindness0,880

$150 / 11,556 QALYs


0,200$4 950 / 11,001 QALYs

Disorder present

0,500$2 070 / 12,192 QALYs


0,000$830 / 12,555 QALYs


1,000$150 / 13,590 QALYs

No disorder0,500

$150 / 13,590 QALYs

Biopsy$1 110 / 12,891 QALYs

Blindness

0,013$40 680 / 7,539 QALYs

No blindness

0,987$680 / 12,565 QALYs

Disorderpresent

0,500$1 200 / 12,500 QALYs

No disorder

0,500$680 / 12,565 QALYs

Treat All$940 / 12,532 QALYs


Treat All : $940 / 12,532 QALYs

2008-10-27 32

4. Cost-effectiveness of Strategies for Blindness Prevention

Strategy Cost ΔC Effectiveness

ΔE AVG C/E ΔC/ ΔEΔC/ ΔE

(ICER)(ICER)

Treat All $940 12,53 QALYs

75 $/QALY

Biopsy $1 110 $170 12,89 QALYs

0,36 QALYs 86 $/QALY 474/ QALY474/ QALY

Treat None $2 400 $1 290 12,00 QALYs

-0,89 QALYs

200 $/QALY (Dominated(Dominated))

2008-10-27 33

5. Sensitivity analyses

BlindnesspBlind

40000 / 6,936

No blindness

#0 / 11,56

Disorder present

diseasePrev

No disorder

#0 / (13.6)

Treat None

BlindnesspBlindTreat

40830 / 7,536

No blindness

#830 / 12,555


testSens

Blindness

pBlind40150 / 6,933

No blindness#

150 / 11,556


#

Disorder present

diseasePrev


#830 / 12,555


testSpec150 / 13,59

No disorder#

Biopsy

Blindness

pBlindTreat40680 / 7,539

No blindness

#680 / 12,565

Disorderpresent

diseasePrev

No disorder

#680 / 12,565

Treat All


2008-10-27 34

Sensitivity Analysis on prior disease prevalence

prior disease prevalence

Exp

ecte

d Va

lue

0,0 0,2 0,4 0,6 0,8 1,0

14,0 QALYs

13,5 QALYs

13,0 QALYs

12,5 QALYs

12,0 QALYs

11,5 QALYs

11,0 QALYs

Treat None

Biopsy

Treat All

Threshold Values:

prior disease prevalence = 0,8EV = 12,5 QALYs

2008-10-27 35

Sensitivity Analysis on prob. blind, w/ treatment

prob. blind, w/ treatment

Exp

ecte

d Va

lue

0,0 0,2 0,4 0,6 0,8 1,0

13,0 QALYs

12,5 QALYs

12,0 QALYs

11,5 QALYs

11,0 QALYs

10,5 QALYs

10,0 QALYs

Treat None

Biopsy

Treat All

Threshold Values:

prob. blind, w/ treatment = 0,5EV = 12,0 QALYs

2008-10-27 36

Sensitivity Analysis on prior disease prevalence and prob. blind, w/ treatment


prob

. blin

d, w

/ tre

atm

ent

0,0 0,2 0,4 0,6 0,8 1,0

1,0

0,9

0,8

0,7

0,6

0,5

0,4

0,3

0,2

0,1

0,0

Treat None

Biopsy

Treat All

2008-10-27 37

Sensitivity Analysis on prior disease prevalence and prob. blind, w/ treatment


prob

. blin

d, w

/ tre

atm

ent

0,0 0,2 0,4 0,6 0,8 1,0

1,0

0,9

0,8

0,7

0,6

0,5

0,4

0,3

0,2

0,1

0,0

Treat None

Biopsy

Treat All

Disease probability is 0,9

Which strategy is best?

2008-10-27 38

Cost-Effectiveness AnalysisAt Treatment Options

Effectiveness

Co

st

0,00 QALYs 7,00 QALYs 14,00 QALYs

$2 500,0$2 300,0$2 100,0$1 900,0$1 700,0$1 500,0$1 300,0$1 100,0

$900,0$700,0$500,0$300,0$100,0

Treat None

Biopsy

Treat All

2008-10-27 39

Cost-Effectiveness AnalysisAt Treatment Options

Effectiveness

Co

st


$2 500,0$2 300,0$2 100,0$1 900,0$1 700,0$1 500,0$1 300,0$1 100,0

$900,0$700,0$500,0$300,0$100,0

Treat None

Biopsy

Treat All

2008-10-27 40

Sensitivity Analysis onprior disease prevalence

Effectiveness

Co

st


$5 000,0

$4 500,0

$4 000,0

$3 500,0

$3 000,0

$2 500,0

$2 000,0

$1 500,0

$1 000,0

$500,0

$0,0

Treat None

Biopsy

Treat All

diseasePrev = 0,0

2008-10-27 41


Effectiveness

Co

st


$5 000,0

$4 500,0

$4 000,0

$3 500,0

$3 000,0

$2 500,0

$2 000,0

$1 500,0

$1 000,0

$500,0

$0,0

Treat None

Biopsy

Treat All

diseasePrev = 0,6

2008-10-27 42


Effectiveness

Co

st


$5 000,0

$4 500,0

$4 000,0

$3 500,0

$3 000,0

$2 500,0

$2 000,0

$1 500,0

$1 000,0

$500,0

$0,0

Treat None

Biopsy

Treat All

diseasePrev = 1,0

2008-10-27 43

Decision Trees: Strengths & Weaknesses

Strengths Intuitive, visual form of

the model Can generate rapid

response using available data

Permits long-term projections

Elapse of time not explicit in decision trees

Tree format can become unwieldy when events repeat

2008-10-27 44

Acknowledgements

Slides & examples greatly indebted to: Jeffrey S. Hoch Jean Lachaine Drummond et al. (2005) Briggs, Sculpher, Claxton (2006) Weinstein et al., (2003) Kuntz & Weinstein (200x) Tree Age

Documents

Economic Evaluation using Decision Analytic Modelling I Mira Johri Université de Montréal 2008-10-27