Is it time to ban PSA? In support of OFSA…

Stirling Bryan, PhD

2

Disclosures and Acknowledgements

• I am not aware of any actual or potential conflicts of interest in relation to this presentation.

• Some of my relevant current activities:– Chair, CADTH’s Health Technology Expert Review Panel– Member, CADTH’s Economic Evaluation Guidelines Working Group– Member, BC’s Health Technology Assessment Committee– Director, VCH’s Centre for Clinical Epidemiology & Evaluation

• Collaborators on this topic– Drs. Pelham Barton and Lazaros Andronis (University of

Birmingham, UK)

3

4

-£400

-£200

£0

£200

£400

£600

£800

£1,000

-0.10 -0.05 0.00 0.05 0.10 0.15 0.20

Incremental QALYs

Incr

emen

tal

Co

st

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

£0 £10,000 £20,000 £30,000 £40,000 £50,000 £60,000

Willingness to pay per additional QALY

Pro

bab

ilit

y o

f b

ein

g c

ost

eff

ecti

ve

5

6

7

Definitions

• Sensitivity analysis (SA):– Analysis that aims to assess and determine the influence of

input parameters on the outcomes of an economic evaluation

• Probabilistic sensitivity analysis (PSA):– A form of SA in which probability distributions are applied to

input parameters and parameter values drawn at random from these distributions to generate an empirical distribution of the relevant measure of cost-effectiveness

• Old-fashioned sensitivity analysis (OFSA):– A form of sensitivity analysis in which the input parameters are

varied using alternative plausible values

8Bernard et al (2011)

9

The argument…

• The growing influence of economic analyses (at CADTH, NICE…) emphasises the importance of methodological rigour

• The appropriate characterisation of uncertainty is an essential component in economic analysis

• And so…– Probabilistic analysis (PA) should be the base case

analysis, and so let’s ban PSA!– OFSA has great value and should be based on PA

10

Review of NICE policy documents

• Many instances where a range of results from sensitivity analysis was reported – SA results cited in policy documents come mainly

from OFSAs

• Explicit reference is made in some documents to the usefulness of the sensitivity analysis, particularly OFSA

11

Interviews: NICE committee members

• “The most useful thing about the economic analysis is it focuses the discussion on what actually matters… What is it that can actually switch a decision…?” [3]

• “The other area… people do struggle with is uncertainty. And I don’t think anybody quite knows how to handle cost-effectiveness acceptability curves... For some people it’s a technical problem of understanding what they are. For those who understand what they are, they still don’t understand what they should do with it.” [4]

12

Conclusions

• Both PA and OFSA have their place

• PSA has no place…

• Effective communication between analysts and policy makers remains a challenge