2/3/04 Sacks 1

Clinical Description William Sacks, PhD, MD—ODE/CDRH

Clinical Description William Sacks, PhD, MD—ODE/CDRH

R2 Technology, Inc.R2 Technology, Inc.

ImageChecker CT CAD ImageChecker CT CAD Software SystemSoftware System

P030012P030012

Radiological Devices Advisory PanelRadiological Devices Advisory Panel

February 3, 2004February 3, 2004

R2 Technology, Inc.R2 Technology, Inc.

ImageChecker CT CAD ImageChecker CT CAD Software SystemSoftware System

P030012P030012

Radiological Devices Advisory PanelRadiological Devices Advisory Panel

February 3, 2004February 3, 2004

2/3/04 Sacks 2

OutlineOutline• Character of deviceCharacter of device

• Clinical utilityClinical utility

• Instructions for useInstructions for use

• Issues new to this PMAIssues new to this PMA

OutlineOutline• Character of deviceCharacter of device

• Clinical utilityClinical utility

• Instructions for useInstructions for use

• Issues new to this PMAIssues new to this PMA

2/3/04 Sacks 3

Character of deviceCharacter of device• For chest For chest CTCT scans scans

• For CTs done For CTs done for any indicationfor any indication

• Detects Detects solidsolid lung nodules lung nodules

E.g., not ground glass opacitiesE.g., not ground glass opacities ( (GGOs)GGOs)

• Nodules between 4-30 mmNodules between 4-30 mm

• Computer-Aided Computer-Aided DetectionDetection (CAD) (CAD)

Character of deviceCharacter of device• For chest For chest CTCT scans scans

• For CTs done For CTs done for any indicationfor any indication

• Detects Detects solidsolid lung nodules lung nodules

E.g., not ground glass opacitiesE.g., not ground glass opacities ( (GGOs)GGOs)

• Nodules between 4-30 mmNodules between 4-30 mm

• Computer-Aided Computer-Aided DetectionDetection (CAD) (CAD)

2/3/04 Sacks 4

Computer-Aided Computer-Aided Detection(CAD)/Diagnosis(CADx)Detection(CAD)/Diagnosis(CADx)

CAD CADxCAD CADxDetectionDetection DiscriminationDiscrimination

Scans entire imageScans entire image Scans only portion Scans only portion selected by userselected by user

Marks imagesMarks images Gives level of Gives level of suspicionsuspicion

2/3/04 Sacks 5

Clinical UtilityClinical Utility• Many nodules missed in clinical Many nodules missed in clinical practice: practice:

other pathology distractsother pathology distracts

hundreds of images per examhundreds of images per exam

• CAD intended to reduce missedCAD intended to reduce missed nodules; i.e., intended to nodules; i.e., intended to

increaseincrease user’s user’s SENSITIVITYSENSITIVITY to detecting to detecting lung nodules lung nodules

Clinical UtilityClinical Utility• Many nodules missed in clinical Many nodules missed in clinical practice: practice:

other pathology distractsother pathology distracts

hundreds of images per examhundreds of images per exam

• CAD intended to reduce missedCAD intended to reduce missed nodules; i.e., intended to nodules; i.e., intended to

increaseincrease user’s user’s SENSITIVITYSENSITIVITY to detecting to detecting lung nodules lung nodules

2/3/04 Sacks 6

Instructions for UseInstructions for Use• Reader reviews films unaided first Reader reviews films unaided first

• CAD marks candidate nodulesCAD marks candidate nodules

• Reader looks again in vicinity of marks Reader looks again in vicinity of marks

• If CAD fails to mark a nodule judged actionableIf CAD fails to mark a nodule judged actionableon unaided review, reader should on unaided review, reader should retainretain initial judgmentinitial judgment

Instructions for UseInstructions for Use• Reader reviews films unaided first Reader reviews films unaided first

• CAD marks candidate nodulesCAD marks candidate nodules

• Reader looks again in vicinity of marks Reader looks again in vicinity of marks

• If CAD fails to mark a nodule judged actionableIf CAD fails to mark a nodule judged actionableon unaided review, reader should on unaided review, reader should retainretain initial judgmentinitial judgment

2/3/04 Sacks 7

Issues new to this PMAIssues new to this PMA• CAD targetCAD target

• Definition of “truth”Definition of “truth”

• Unit of analysis Unit of analysis

• EndpointsEndpoints

Issues new to this PMAIssues new to this PMA• CAD targetCAD target

• Definition of “truth”Definition of “truth”

• Unit of analysis Unit of analysis

• EndpointsEndpoints

2/3/04 Sacks 8

CAD TARGETCAD TARGET

Not malignant noduleNot malignant nodule

butbut

Actionable noduleActionable nodule

CAD TARGETCAD TARGET

Not malignant noduleNot malignant nodule

butbut

Actionable noduleActionable nodule

2/3/04 Sacks 9

DEFINITION OF “TRUTH”DEFINITION OF “TRUTH”

Not biopsy (tissue Not biopsy (tissue histology)histology)

butbut

Expert panelExpert panel

DEFINITION OF “TRUTH”DEFINITION OF “TRUTH”

Not biopsy (tissue Not biopsy (tissue histology)histology)

butbut

Expert panelExpert panel

2/3/04 Sacks 10

UNIT OF ANALYSISUNIT OF ANALYSIS

Statistical unit = personStatistical unit = person

further broken down intofurther broken down into

Lung quadrantLung quadrant

UNIT OF ANALYSISUNIT OF ANALYSIS

Statistical unit = personStatistical unit = person

further broken down intofurther broken down into

Lung quadrantLung quadrant

2/3/04 Sacks 11

ENDPOINTS ENDPOINTS SensitivitySensitivity and and specificityspecificity (or (or

FPFFPF) of ) of action recommendationaction recommendation and/orand/or

ENDPOINTS ENDPOINTS SensitivitySensitivity and and specificityspecificity (or (or

FPFFPF) of ) of action recommendationaction recommendation and/orand/or

ROCROC

2/3/04 Sacks 12

CLINICAL STUDYCLINICAL STUDY• 3 3 expert radiologists on panel to expert radiologists on panel to

determine “truth” for each nodule determine “truth” for each nodule

• 1515 other radiologists with range of experience, other radiologists with range of experience, called “the readers” called “the readers”

• 9090 subjects ( subjects (360360 lung quadrants) lung quadrants)

• 15 readers used 15 readers used 100-point scale100-point scale for for confidence-in-actionability of each confidence-in-actionability of each

case case

CLINICAL STUDYCLINICAL STUDY• 3 3 expert radiologists on panel to expert radiologists on panel to

determine “truth” for each nodule determine “truth” for each nodule

• 1515 other radiologists with range of experience, other radiologists with range of experience, called “the readers” called “the readers”

• 9090 subjects ( subjects (360360 lung quadrants) lung quadrants)

• 15 readers used 15 readers used 100-point scale100-point scale for for confidence-in-actionability of each confidence-in-actionability of each

case case

2/3/04 Sacks 13

CLINICAL STUDYCLINICAL STUDY

Presentation by Presentation by

Nicholas Petrick, Ph.D.Nicholas Petrick, Ph.D.

OST/CDRHOST/CDRH

CLINICAL STUDYCLINICAL STUDY

Presentation by Presentation by

Nicholas Petrick, Ph.D.Nicholas Petrick, Ph.D.

OST/CDRHOST/CDRH