IIntegration of predictive and ntegration of predictive and retrospective risk analysis retrospective risk analysis in health carein health care

Tjerk van der Schaaf Tjerk van der Schaaf Leiden University Medical Center Leiden University Medical Center

Eindhoven University of TechnologyEindhoven University of Technology

overview

• retrospective method: PRISMA-medical• predictive method : HFMEA • 3 examples of possible integration

– direct comparison of predicted vs “actual”causes (radiotherapy)

– components combined in a Healthcare Safety Management System (convergent approach)

– evaluating major interventions (impact of IT on medication safety)

retrospective risk analysis : PRISMA- medical

• (voluntary) incident reporting and analysis

• learning from actual / reported process deviations

PRISMA-Medical

• Prevention and Recovery Information System for Monitoring and Analysis

• Three subsequent steps:– Description by means of causal trees– Classification according to the Eindhoven

Classification Model (medical version)– Determination of countermeasures by

means of the Classification/Action Matrix

Causal tree example

Wrong route

Lines at same place

Nurses not informed

Similar linesConnection

possibleInadequate

check

No protocol

Catheters not removed

No coding

O

O

T

T H

Eindhoven Classification Model -(medicalversion)

Database

• Root causes for failure failure profile

• Root causes for recovery recovery profile

• Context variables black-spot analysis

PRISMA failure profile: hospital medication errors

0

5

10

15

20

25

Pe

rce

nta

ge

(%

)

T-E

X

TD

TC

TM

O-E

X

OK

OP

OM

OC

H-E

X

HK

K

HR

Q

HR

C

HR

V

HR

I

HR

M

HS

S

HS

T

PR

F X

PRISMA category

Classification/Action Matrix

ECM code

Design: Technology/work-

place

Procedures Information and Commu

nication

Training Motiva

tion

Escala

tion

Reflection

T-EX ×

TD ×

TC ×

TM ×

O-EX ×

OK ×

OP ×

OM ×

OC ×

H-EX ×

HK_ × NO

HR_ ×

HS_ × NO

predictive risk analysis HFMEA / SAFER

• series of group meetings to build a set of failure scenario’s for a (small) process of care : what may go wrong; why; what to do about it

• pro-active appeal

Healthcare Failure Mode and Effect Analysis (HFMEA)

• A systematic approach to identify and prevent product and process problems before they occur

• Developed by the "VA National Center for Patient Safety"

(http://www.patientsafety.gov/)

Relevance of predictive risk analysis

• Retrospective (incident) analysis takes place after incidents did occur hindsight bias

• Because of underreporting, biases can arise in incident databases identification of "missing risks"

Definitions

• Failure Mode: Different ways that a process or subprocess can fail to provide the anticipated result (i.e. think of it as what could go wrong)Prescribing the wrong dose

• Failure Mode Cause: Different reasons as to why a process or subprocess would fail to provide the anticipated result (i.e. think of it as why it would go wrong)Miscalculation

HFMEA process

• Step 1: Define the topic

• Step 2: Assemble the team

• Step 3: Graphically describe the process

• Step 4: Conduct a hazard analysis

• Step 5: Identify actions and outcome measures

examples of integration (1)

• direct comparison of predicted (HFMEA) vs reported causes

• user problems with a new radiation therapy technology

• both types of failure causes expressed in the same PRISMA-medical classification (sub-)categories

PRISMA vs HFMEA : main categories

0%

10%

20%

30%

40%

50%

60%

Per

cen

tag

e

Tech Org Humanother

PRISMA main category

PRISMA

HFMEA : predicted causes

PRISMA vs HFMEA : subcategories

Frequency category HFMEAless than yearly yearly monthly weekly

Weight-factor (= translation to 9 months) 0,1 0,89 9 36

0%

17%

4%

2%

0%

3%

17%

8%

4%

0%

4% 5%

1%

13%

16%

1%1%

0%

4%

1%0%

21%

1%

4%

0%

5%

13%

33%

0% 0%

3%

0%

2%

4%

8%

2%1%

0%

3%

0%

0%

5%

10%

15%

20%

25%

30%

35%T-

EX TD TC TMO-

EX OK OP OM OCH-

EX HKK

HRQ

HRC

HRV

HRI

HRM

HSS

HST

PRF X

PRISMA category

Perc

enta

ge

PRISMA

HFMEA

examples of integration (2)

• combining retrospective and predictive components in an overall Healthcare Safety Management System

• convergent approach of two imperfect risk identification methodologies

• mutual checks, comparisons, and inputs

possible

examples of integration (2)continued

• are repeatedly predicted problems (failure modes) ever being reported?

• can frequently reported problems help to select suitable processes for HFMEA and generate realistic failure modes?

• can frequently predicted causes steer the information gathering after an initial report?

• are proposed interventions for predicted vs “reported” causes similar?

• etc…

examples of integration (3)

• developing a process-based evaluation methodology for major (patient safety) interventions

• predicting and monitoring the impact of IT on medication safety

Medication safety: definitions

Adverse drug reactions Medication error Harm

Medication error No harm

Medical error (not drug related)

Drug

Harm

Error

[Van den Bemt et al., 2000]

Medication errors: causes (1)

• Handwritten prescriptions and drug orders• Look-alike drug names• Sound-alike drugs and verbal orders• Use of abbreviations• Similar packaging and labelling• Inadequate training and supervision• Staff shortages• Overwork and fatigue

[Habraken, 2004]

Medication errors: causes (2)

IT: possibilities and problems

IT: possibilities and problems

[Bates et al., 1995; Bates, 2000]

PrescribingPhysician order entry/ Computerised decision support

TranscribingElectronicorder transcription

DispensingRobots / Bar coding/ Automated dispensingdevices

AdministeringBar coding/ Automateddispensingdevices

MonitoringComputerisedmonitoringof adversedrug events

MedicationadministrationrecordComputerisedmedication administration record

56%

6%

4%

34%

IT: possibilities and problemsIT application PROS CONS

CPOE Legible prescriptions; no handwriting required

Possibility of substitution errors

Data entry only necessary once Failure to warn

Exchange of data is easy

Computerised decision support

Drug information Risk of low vigilance and overtrust

Patient-specific information and advice

Bar coding Ensure five "rights": right drug, right patient, right dose, right route, right time

Degraded coordination and communication

Computerised medical record

Legible prescriptions; no handwriting required

Possibility of substitution errors

Data entry only necessary once

Exchange of data is easy

[Habraken and Van der Schaaf, 2006]

Barriers to the implementation of IT

• Significant costs: technical, process redesign, and implementation and support

• Cultural obstacles: resistance to change

• Privacy and protection of (patient) data

• Lack of data standards

• Lack of (clinical) evaluation

[Habraken, 2004]

Evaluation of effects and impact of IT: PRISMA and HFMEA

• Not only outcomes of care but also the mechanisms underlying those outcomes

• Impact of IT on "error recovery " :– Detection– Diagnosis– Correction

of earlier errors / deviations

Evaluation of effects and impact of IT: PRISMA

• PRISMA can be used to obtain an insight into the behavioural mechanisms underlying medication errors

• Classification/Action Matrix enables us to predict which types of human behaviour will be influenced by IT

Evaluation of effects and impact of IT: PRISMA

ECM code

Design/ Technol

Procedures Information and Communication

Training Motivation Escalation Reflection

T-EX ×

TD ×

TC ×

TM ×

O-EX ×

OK ×

OP ×

OM ×

OC ×

H-EX ×

HK_ × NO

HR_ ×

HS_ × NO

Evaluation of effects and impact of IT: PRISMA

• IT applications would fall in two categories: "technology" and "information and communication"

• In case of improved technology reduction of skill based human errors

• In case of information and communication support reduction of knowledge based errors

• BUT: rule based human errors would not be influenced by IT

Evaluation of effects and impact of IT: PRISMA and HFMEA

• Theoretical predictions could be reinforced by predictive risk analysis, such as HFMEA

• Empirical evaluation of actual impact of IT by means of intensified incident reporting

• Comparison of causal patterns of incidents that occur before, during, and after the IT intervention

Conclusion (1)

• IT often mentioned as prerequisite for reduction of medication errors

• Results regarding effects of IT vary greatly

• Effects of IT on behavioural mechanisms are not/hardly taken into account

• PRISMA and HFMEA offer a framework for in-depth analysis of impact of IT

Conclusion (2)• Two types of predictions can be made of

expected effects of IT on error and error recovery:– Theoretical predictions by means of PRISMA– HFMEA scenario-based predictions

• Intensified incident reporting and analysis would enable a fast comparison between predicted and actual effects

• On-line corrections of implementation process could prevent actual adverse events

Thank you for your attention