Upload
jeffry-merritt
View
217
Download
0
Tags:
Embed Size (px)
Citation preview
Classification of Emergency Department CT Imaging Reports
using Natural Language Processing and Machine Learning
Efsun Sarioglu, Kabir Yadav, Meaghan Smith, Hyeong-Ah Choi
This project supported by the NIH National Center for Research Resources(UL1RR031988 and KL2RR031987)
Background, Objective & Methods
Use of electronic medical record data for clinical research and quality improvement requires free-text data interpretation for outcomes of interest. Natural language processing has shown
promise for this purpose
To demonstrate real-world performance of a hybrid NLP-machine learning system for automated classification of radiology reports
Approach Overview Multicenter review of consecutive CT
reports obtained for facial trauma using a trained reference standard Medical Language Extraction and Encoding (MedLEE) WEKA 3.7.5 Salford Systems CART 6.6
Results
Total reports: 3710
Positive cases: 460 (12.4%)
Manual coding had high reliability Kappa=0.97 [95% CI 0.94-0.99]
CART Decision Trees (50:50)Raw Text (8-node) NLP (9-node)
Classification Performance
Raw Text
NLP
Precision
0.949 0.968
Recall 0.932 0.964
F-score 0.940 0.966
Unexpectedly high performance of machine learning without NLP
Comparable to inter-rater performance and prior studies of inter-physician agreement
Comparable to prior real-world and simulation studies
Concluding Remarks How’s it novel?
One of only a handful of real-world NLP studies using validated reference standard
Translating existing NLP and machine learning technologies to support CER
Next step: validation Test approach using other clinical cases Evaluate different features or
classification algorithms