CAD Performance Analysis for CAD Performance Analysis for Pulmonary Nodule Detection: Pulmonary Nodule Detection:

Comparison of Thick- and Thin-Slice Comparison of Thick- and Thin-Slice Multi-detector CT ScansMulti-detector CT Scans

Randy D ErnstRandy D Ernst11, Russell C Hardie, Russell C Hardie22, , Metin N GurcanMetin N Gurcan33, Aytekin Oto, Aytekin Oto11,,

Steve K RogersSteve K Rogers33, Jeffrey W Hoffmeister, Jeffrey W Hoffmeister33

1. Department of Radiology, The University of Texas Medical 1. Department of Radiology, The University of Texas Medical Branch, Galveston TX Branch, Galveston TX

2. iCAD Inc. and University of Dayton, Dayton OH 2. iCAD Inc. and University of Dayton, Dayton OH 3. iCAD Inc., Beavercreek OH3. iCAD Inc., Beavercreek OH

PurposePurpose To compare the performance of To compare the performance of

a CAD (a CAD (QuickCue™QuickCue™, Beavercreek, , Beavercreek, OH) system in detecting lung OH) system in detecting lung nodules from nodules from thick- and thin-slicethick- and thin-slice multi-detector row CT.multi-detector row CT.

To evaluate the potential benefit To evaluate the potential benefit of CAD on radiologist sensitivity.of CAD on radiologist sensitivity.

Methods and MaterialsMethods and Materials 57 studies reviewed retrospectively 57 studies reviewed retrospectively Case selection:Case selection:

Obtained during a 5-month periodObtained during a 5-month period Referred from multiple departmentsReferred from multiple departments Contain at least 1 pulmonary nodule Contain at least 1 pulmonary nodule

but fewer than 10 nodules to localizebut fewer than 10 nodules to localize No significant miss-registration, No significant miss-registration,

breathing, surgical changes, pleural breathing, surgical changes, pleural effusions & atelectasiseffusions & atelectasis

Too many nodules to localizeToo many nodules to localize

Methods and MaterialsMethods and Materials

4-slice 4-slice multi-detector row CT multi-detector row CT (Lightspeed; GE Medical Systems)(Lightspeed; GE Medical Systems)

HQ setting with 7.5 mm/rotationHQ setting with 7.5 mm/rotation Standard-dose (160 - 270 mA, 120 Standard-dose (160 - 270 mA, 120

kVp)kVp) Images reconstructed at 5-mm Images reconstructed at 5-mm

(thick) and 2.5 mm (thin) slice (thick) and 2.5 mm (thin) slice thicknessesthicknesses

Methods and MaterialsMethods and Materials 140 nodules (3 mm - 25 mm) were 140 nodules (3 mm - 25 mm) were

identifiedidentified pre-CAD by radiologists pre-CAD by radiologists from thick-slice cases onlyfrom thick-slice cases only mean nodule size 7.3 mean nodule size 7.3 ±± 4.2 4.2 mm mm

Truth marks were mapped to the thick-Truth marks were mapped to the thick-slice 5mm data.slice 5mm data.

Gold standards for nodule truth created Gold standards for nodule truth created from post-CAD radiologist review from post-CAD radiologist review One gold standard for thick-sliceOne gold standard for thick-slice Separate gold standard for thin-sliceSeparate gold standard for thin-slice

CAD SystemCAD System (QuickCue™(QuickCue™, iCAD Inc., iCAD Inc.))

3D LungSegmentation

3D CandidateSegmentation

CalculateFeatures

DICOMImages

Classifier

DetectionMask

CAD SystemCAD System Candidates segmented by Candidates segmented by

thresholding and morphological thresholding and morphological processingprocessing

2D and 3D features computed for 2D and 3D features computed for each candidateeach candidate

Anatomical information (hilus, Anatomical information (hilus, airways, aorta, etc.) compared to airways, aorta, etc.) compared to reduce false positivesreduce false positives

A classifier applied for final decisionA classifier applied for final decision

CAD detected 72.1% (101/140) of the CAD detected 72.1% (101/140) of the thick gold standard truth nodulesthick gold standard truth nodules

CAD detected 35 additional CAD detected 35 additional radiologist-confirmed nodules, an radiologist-confirmed nodules, an increase of 25% (35/140) in sensitivityincrease of 25% (35/140) in sensitivity

5.6 (317/57) false-positives per case5.6 (317/57) false-positives per case 55 due to atelectasis55 due to atelectasis 18 due to scarring18 due to scarring

Review of Thick-Slice Review of Thick-Slice CAD ResultsCAD Results

Venn Diagram for ThickVenn Diagram for Thick

3

39 35

317

CADPre-CAD Review

Post-CAD ReviewGold Standard

101

0 0

CAD detected 80.7% (113/140) of CAD detected 80.7% (113/140) of the pre-CAD truth nodules.the pre-CAD truth nodules.

CAD detected 94 additional CAD detected 94 additional radiologist-confirmed nodules, an radiologist-confirmed nodules, an increase of 67.1% (94/140).increase of 67.1% (94/140).

4.6 (262/57) false-positives reported 4.6 (262/57) false-positives reported per case.per case. 70 due to atelectasis70 due to atelectasis 39 due to scarring39 due to scarring

Review of Thin-Slice Review of Thin-Slice CAD ResultsCAD Results

Venn Diagram for ThinVenn Diagram for Thin

0

26 94

262

CAD using thin-slicePre-CAD Review using thick-slice with detections mapped to thin-slice

Post-CAD Review of thin-sliceGold Standard

113

0 0

ComparisonComparisonThick-slice Thick-slice casescases

Thin-slice Thin-slice casescases

CAD CAD sensitivitysensitivity

72.1%72.1% 80.7%80.7%

Radiologist Radiologist sensitivity sensitivity increase increase after CADafter CAD

+25%+25% +67.1%+67.1%

FPsFPs 5.6 5.6 4.64.6

FROC Curve for CAD FROC Curve for CAD

0 2 4 6 8 10 12 140

0.2

0.4

0.6

0.8

1

Average False Positives Per Case

Prob

abili

ty o

f Det

ectio

n

Thin SliceThick Slice

CAD detections - Thick-SliceCAD detections - Thick-Slice

0 5 10 15 20 250

5

10

15

20

25

30

35

Size (mm)

Num

ber o

f Nod

ules

Radiologist DetectedRadiologist Missed

CAD detections -Thin-SliceCAD detections -Thin-Slice

0 5 10 15 20 250

10

20

30

40

50

Size (mm)

Num

ber o

f Nod

ules

Radiologist DetectedRadiologist Missed

5 primary lung cancers5 primary lung cancers 24 cases of metastatic cancer including 24 cases of metastatic cancer including

7 lymphomas, 4 breast, 4 head and neck, 2 7 lymphomas, 4 breast, 4 head and neck, 2 colon, 2 pancreas, 1 carcinoid, 1 seminoma,  1 colon, 2 pancreas, 1 carcinoid, 1 seminoma,  1 ovarian, 1 melanoma and 1 tracheal ovarian, 1 melanoma and 1 tracheal papillomatosispapillomatosis

23 cases of infection, including23 cases of infection, including 19 granulomatous disease either calcified, 19 granulomatous disease either calcified,

stable on follow-up or biopsy proven. 4 were stable on follow-up or biopsy proven. 4 were presumed infection that resolved with follow-up presumed infection that resolved with follow-up

1 case proved to be a thrombosed AVM1 case proved to be a thrombosed AVM 4 cases lost to follow up4 cases lost to follow up

Case Follow-upCase Follow-up

Example Example TPsTPs

Examples of Examples of nodules that are nodules that are detected by both detected by both radiologist and radiologist and CADCAD

Example TPsExample TPs Examples of nodules that are initially Examples of nodules that are initially

missed by radiologists then detected missed by radiologists then detected after reviewing CADafter reviewing CAD

Review of CAD ResultsReview of CAD Results Sources of false positivesSources of false positives

Vessel intersectionsVessel intersections Inaccurate lung segmentationInaccurate lung segmentation Partial volume effectsPartial volume effects Other lung abnormalities (scarring, Other lung abnormalities (scarring,

atelectasis)atelectasis)

Example FPsExample FPs

Review of CAD ResultsReview of CAD Results Sources of false negatives (missed Sources of false negatives (missed

nodules)nodules) Low density, irregularLow density, irregular Strong connectivity with vesselsStrong connectivity with vessels Imperfect candidate segmentationImperfect candidate segmentation Inaccurate lung segmentationInaccurate lung segmentation

5mm Thick slice

Example FNsExample FNs

5mm Thick slice

2.5mm Thin slice

5mm Thick slice

2.5mm Thin slice

ConclusionConclusionss Sensitivity and specificity of the CAD Sensitivity and specificity of the CAD

system increased when used with system increased when used with thin-slice scans versus thick-slice thin-slice scans versus thick-slice scans.scans.

CAD improved radiologist sensitivity CAD improved radiologist sensitivity on both thick- and thin-slice scans.on both thick- and thin-slice scans.

CAD improvement was greater for CAD improvement was greater for thin-slice scans.thin-slice scans.