The Duke Stroke Policy Model (SPM)

MIMI

ISIS

TIATIA

ASYASY DTHDTH

HSHS

BleedBleed

Developers

David Matchar, MD -- principal investigator Greg Samsa, PhD -- project director,

statistician Giovanni Parmigiani, PhD -- statistician,

software developer Joe Lipscomb, PhD -- health economist Greg Hagerty, MS -- software developer

Outline

Rationale for modeling (*) SPM described Applying the SPM to a

randomized trial Extensions

Rationale for modeling

Why model? Arguments for modeling Arguments against modeling Discussion Conclusions Application to stroke

Why model? (cont’d)

“To me, decision analysis is just the systematic articulation of common sense: Any decent doctor reflects on alternatives, is aware of uncertainties, modifies judgements on the basis of accumulated evidence, balances risks of various kinds, . . .”

Why model? (cont’d)

“ considers the potential consequences of his or her diagnoses and treatments, and synthesizes all of this in making a reasoned decision that he or she decrees right for the patient…”

(cont’d)

Why model?

“… All that decision analysis is asking the doctor to do is to do this a lot more systematically and in such a way that others can see what is going on and can contribute to the decision process.” -- Howard Raiffa, 1980

Advantages of modeling

Clarifies decision-making Simplifies decision-making Provides comprehensive framework Allows best data to be applied Extrapolates short-term observations into

long-term Encourages “what if” analyses

Disadvantages of modeling

Ignores subjective nuances of patient-level decision-making

Problem may be incorrectly specified Inputs may be incorrect / imprecise Usual outputs are difficult to interpret or

irrelevant to decision-makers

Individual decision-making is subjective

For individual decision-making, primary benefit of modeling is clarification.

As normative process, decision-making works better for groups.– Most applications involve group-, rather than

individual-level, decisions (e.g., CEA, purchasing decisions, guidelines).

An aside

Interactive software (possibly including models) shows great potential to help decision-makers (e.g., patients, physicians, pharmacy benefits managers) clarify and make better decisions. – We are developing prototype for a “user-

friendly version” of the SPM.

Some models are mis-specified

A good model will simplify without over-simplifying.

Poor models exist, but this need not imply that modeling itself is bad.

We need more explicit standards under which models are developed, presented, and assessed.

An observation

The fundamental problem with many of the poor models in circulation is that they assume the answer they are purporting to prove (often, that a treatment which is trivially

effective or even ineffective is nevertheless cost-effective).

Users are understandably wary.

Model inputs may be incorrect/ imprecise

This problem is often most acute for utilities and costs, and least acute for natural history and efficacy.– We need more and better data on cost and

quality of life. The less certain the parameter, the greater

the need for sensitivity analysis.

An aside

In practice, the conclusions of a model / CEA are never stronger than the strength of the evidence regarding efficacy.

If the evidence about efficacy is weak, then modeling / CEA should not be performed.

Usual outputs are difficult to interpret

In academic circles, results are presented as ICERs using the societal perspective.– Present this as a base case for purposes of

publication / benchmarking.– Also present multiple outcomes from multiple

perspectives (vary cost categories, vary time periods, present survival, event-free survival, QALY, …).

General conclusions

Modeling is of great potential benefit and indeed is sometimes the only reasonable way to proceed. However, models must be held to a high standard of proof.

Although the standard reference model cannot be ignored, modeling should be done flexibly, with the needs of the end user in mind.

Application to acute stroke treatment

RCTs follow patients in the short-term, but the large majority of benefits accrue in the long-term.

Simple heuristics will not suffice to adequately trade off complex risks, benefits, and costs.

Modeling allows a large body of evidence to be efficiently synthesized.

Outline

Rationale for modeling Stroke model described (*) Applying the SPM to a

randomized trial Extensions

SPM described

History / background Types of analysis Structure Validation / citations

SPM history / background

SPM development (cont’d)

First version developed in 1993 by Stroke PORT Goals of stroke PORT:

– Summarize epidemiology of stroke – Describe best stroke prevention practices– Describe current practices, and test methods

for improving practice

SPM development

SPM was used:To summarize epidemiology

of stroke To support CEAAs a basic organizing

structure for the PORT

SPM versions Original C++ code (uses waiting time

distributions, research tool, difficult to extend) New S+ code (uses waiting time distributions,

highly structured code used as development tool, inefficient)

New C++/Decision-Maker code (uses Markov-based cycles, intervention language, better interface, extendable)

New C++ version

Decision-Maker used to specify natural history and effect of interventions in a decision tree format

Efficient C++ code used as simulation engine

Expandable into a web-based tool