Explainable Prediction of Medical Codes from Clinical Text · Future work Exploit the loose structure of discharge summaries Some have already done this [Shi et al., 2017] Sparsify

Explainable Prediction of Medical Codes from Clinical Text

James Mullenbach

Sarah Wiegreffe

Jon Duke

Jimeng Sun

Jacob Eisenstein

The Clinical Coding Problem● Electronic Health Records have seen increasing

adoption in the last 5 years

2


adoption in the last 5 years● They abound with written physician notes

3


adoption in the last 5 years● They abound with written physician notes● ICD: taxonomy of diagnoses and procedures

4


adoption in the last 5 years● They abound with written physician notes● ICD: taxonomy of diagnoses and procedures● Human coding laborious, error-prone [Birman-Deych et

al., 2005]

5

The Clinical Coding Problem, as an NLPer● Highly multi-label classification

○ Of nearly 9,000 labels, predict a subset

6


○ Of nearly 9,000 labels, predict a subset● Testbed for document representations

7


○ Of nearly 9,000 labels, predict a subset● Testbed for document representations● Documents are long and loosely structured

8

The dataset

● Open-access, de-identified

9

The dataset

● Open-access, de-identified● 47k admissions -> 47k documents for training

10

The dataset

● Open-access, de-identified● 47k admissions -> 47k documents for training● Loosely structured:

11

Admission Date: [**2118-6-2**] Discharge Date: [**2118-6-14**]

Date of Birth: Sex: F

Service: MICU and then to [**Doctor Last Name **] Medicine

HISTORY OF PRESENT ILLNESS: This is an 81-year-old femalewith a history of emphysema (not on home O2), who presents...

The dataset


12





Long: Median post-processed document length: 1,341

The dataset


13





Long: Median post-processed document length: 1,341

Many labels: 519.1: ‘Other disease…’491.21: ‘Obstructive …’518.81: ‘Acute respir…’486: ‘Pneumonia, orga…’276.1: ‘Hyposmolality…’244.9: ‘Unspecified h…’31.99: ‘Other operati…’...

Median # labels: 14

Prior approaches● Predict from a subset of labels

14

This work● Predict from all labels

Prior approaches● Predict from a subset of labels● Sub-domain focus, e.g.

radiology

15

This work● Predict from all labels● General ICU setting

Prior approaches● Predict from a subset of labels● Sub-domain focus, e.g.

radiology● Private dataset

16

This work● Predict from all labels● General ICU setting● Open-access data

Modeling desiderata

...who sustained a fall at home she was found to have a large acute on chronic subdural hematoma with extensive midline shift...

17

● Focus on the parts that matter

Modeling desiderata

...who sustained a fall at home she was found to have a large acute on chronic subdural hematoma with extensive midline shift...

18

E849.0: Home accidents

801.26: ...subdural, and extradural hemorrhage...

● Focus on the parts that matter● Treat labels individually

Convolution

19

h1

h2

h3

patient

was

initially

placed

on

embed19

convolve

Convolution + Attention

20

U

α11

α21

α31

vh1

h2

h3

patient

was

initially

placed

on

parameter

variable

embed convolve attend (separately for each label)20

Convolution + Attention = the CAML model

21

U

α11

α21

α31

v

lossh1

h2

h3

patient

was

initially

placed

on

parameter

variable

classifyembed convolve attend21

β

Dealing with the long tail● Huge label space (nearly 9,000 total)

2222

Dealing with the long tail● Huge label space (nearly 9,000 total)● Many labels are similar250.00: “Diabetes mellitus without mention of complication, type II or unspecified type, not stated as uncontrolled”

250.02: “Diabetes mellitus without mention of complication, type II or unspecified type, uncontrolled”

2323

Dealing with the long tail: DR-CAML

24

U

α11

α21

α31

lossh1

h2

h3

patient

was

initially

placed

on

parameter

variable

“Atrial fibrillation” Max-pool CNN Label embedding

L2 similarity

24

v

β

MIMIC results● Few works use the whole

label space

2525

*[Scheurwegs et al., 2017]

MIMIC results● Few works use the whole

label space● 20%, 8% relative

improvement over prior SotA

26

*[Scheurwegs et al., 2017]

26

More MIMIC metrics

27

● Enable future comparison● Precision @ k: decision support use-case

27

But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain

2828

But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code

2929

But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity

3030

But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity● 100 random label-document samples

3131

But, is the model focusing on reasonable areas?● We want interpretability, especially in the clinical domain● Physician evaluation: select all text snippet(s) that explain the code● Baselines: LogReg, CNN, cosine similarity● 100 random label-document samples● “Informative” and “Highly Informative”

3232

Physician evaluation example*Code: 575.4

Full descriptions: Perforation of gallbladder

“. . in the setting of gallbladder perforation secondary to acute acalculous cholecystitis after . . . . . . . . inhalation hospital1 times a day metronidazole mg tablet sig one tablet po tid times . . . . . . . to have an infection in your gallbladder requiring iv antibiotics and tube placement for . . .”

33

*not exact format used33




34

*not exact format used

attention

34




35


attentionCNN




36


attention cosine sim.CNN




37


attention cosine sim.CNN

LogReg

Physician evaluation results● Improves upon CNN,

LogReg

3838

Physician evaluation results● Improves upon CNN,

LogReg● More experts needed!

3939

Future work● Exploit the loose structure of discharge summaries

○ Some have already done this [Shi et al., 2017]

4040


○ Some have already done this [Shi et al., 2017]● Sparsify or structure attention to improve interpretability, accuracy

○ “fusedmax” [Niculae & Blondel, 2017]

4141



○ “fusedmax” [Niculae & Blondel, 2017]● Develop methods for few-shot or zero-shot classification

4242



○ “fusedmax” [Niculae & Blondel, 2017]● Develop methods for few-shot or zero-shot classification● Exploit hierarchy of codes

○ GRAM [Choi et al., 2017]

43

250.0 / \250.00 250.01

DM type 2 DM type 1

43

Summary● ICD coding is valuable and challenging

4444

Summary● ICD coding is valuable and challenging● Convolution + attention works well

4545

Summary● ICD coding is valuable and challenging● Convolution + attention works well● Attention can explain the predictions

4646

Summary● ICD coding is valuable and challenging● Convolution + attention works well● Attention can explain the predictions

4747

Thank you!Questions?

Code, data splits, pretrained models: github.com/jamesmullenbach/caml-mimic