Pilcher, David — Prediction in Critical Care: Science, Art or Foolishness?

Prediction in Critical CareScience, Art or Foolishness

David Pilcher@Intensive DaveThe Alfred Hospital & ANZICS CORE

Michael

65 yrs

83kg 174cm

Sepsis

23rd October 1998634 BC

1st January 200021st December 2012

26th April 2026

Predictions are common.

Predictions are common.

“I do a bit of modelling….”

Predictions are common.

“I do a bit of modelling….”

Predictions are common.

Discrimination Calibration Validation

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

Binary Outcomes Only

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

Binary Outcomes Only

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

Binary Outcomes Only

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

Binary Outcomes Only

Burns

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

BurnsArea under Receiver Operator

Characteristic = 0.92

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

50yrs

+ = 50%40%

Mortality

Sen

siti

vity

1- specificity

BurnsArea under Receiver Operator

Characteristic = 0.92

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

50yrs

+ = 50%60%

Mortality

Sen

siti

vity

1- specificity

Area under Receiver Operator

Characteristic = 0.92 Burns

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

0%

5%

10%

15%

20%

Mo

rtal

ity

Year

Predicted Mortality from APACHE III

Observed Mortality

APACHE

Acute Physiology and Chronic Health

Area under Receiver Operator

Characteristic = 0.88

0%

2%

4%

6%

8%

10%

12%

14%

Year

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

APACHE III

Observed Mortality

ANZ Risk of Death

Area under Receiver Operator

Characteristic = 0.90

0%

2%

4%

6%

8%

10%

12%

14%

Year

Discrimination Calibration Validation

0%

20%

40%

60%

80%

100%

0% 20% 40% 60% 80% 100%

Sen

siti

vity

1- specificity

Observed Mortality

ANZ Risk of Death

Area under Receiver Operator

Characteristic = 0.90

ANZROD

Michael

65 yrs

83kg 174cm

Sepsis Is your ICU any good?

Is your ICU any good?

Observed DeathsPredicted Deaths=Standardised

Mortality Ratio

Michael

65 yrs

83kg 174cm

Sepsis

Is your ICU any good?0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

0 200 400 600 800 1000 1200 1400

Number of admissions

More deaths than predicted

Less deaths than predicted

Stan

dar

dis

ed M

ort

alit

y R

atioMichael

65 yrs

83kg 174cm

Sepsis

Is your ICU any good?0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

0 200 400 600 800 1000 1200 1400

Number of admissions

More deaths than predicted

Less deaths than predicted

Stan

dar

dis

ed M

ort

alit

y R

atioMichael

65 yrs

83kg 174cm

Sepsis

23rd October 1998634 BC

1st January 200021st December 2012

26th April 2026

Predictions are common.

23rd October 1998634 BC

1st January 200021st December 2012

26th April 2026

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs26th April 2026

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

0%

5%

10%

15%

20%

25%

30%

35%

40%

Mo

rtal

ity

Year

Declining Mortality for Sepsis & Septic Shock in ANZ ICUs

Predictions are important. They tell us where are now and where we could be in the future.

Thank you@IntensiveDave

Calibration ValidationDiscrimination