Open Source Imaging ToolkitsOpen Source Imaging Toolkits

Rick AvilaRick Avila

Director of Medical ApplicationsDirector of Medical ApplicationsKitware, Inc.Kitware, Inc.

rick.avila@kitware.comrick.avila@kitware.com

April 27, 2006 KitwareKitware

KitwareKitware

The Software Gap The Software Gap

Digital healthcare networks Digital healthcare networks

are rolling out across the are rolling out across the

planet…planet…

Digital acquisition devices are Digital acquisition devices are

exceeding observer ability…exceeding observer ability…

Comprehensive disease Comprehensive disease

management tools are needed management tools are needed

within a PACS workflow…within a PACS workflow…

BrainBrainLungLung

ColonColonCardiacCardiac

CT Lung Scan Slice ThicknessCT Lung Scan Slice Thickness

0.00

2.00

4.00

6.00

8.00

10.00

12.00

1996 1998 2001 2002 2004

KitwareKitware

Improving Global Development EfficiencyImproving Global Development Efficiency

ClinicalStudy Data

Publication

Closed systems yield slow advancementClosed systems yield slow advancement

BaselineData

Need rapid acquire / develop / validate / review cyclesNeed rapid acquire / develop / validate / review cycles

Validation/Analysis

Env/Algs/Apps

Open ScienceOpen Science

1.1. Open DataOpen Data

2.2. Open SourceOpen Source

3.3. Open PublicationOpen Publication

YearsYears

KitwareKitware

Leveraging Global Resources Leveraging Global Resources

ExpertsExperts• ClinicalClinical

• ComputingComputing

Imaging ToolkitsImaging Toolkits• Software ArchitectureSoftware Architecture

• AlgorithmsAlgorithms

• DocumentationDocumentation

• Validation TestsValidation Tests

Development ToolsDevelopment Tools• Build UtilitiesBuild Utilities

• Quality DashboardQuality Dashboard

Computing ResourcesComputing Resources• Hardware & SoftwareHardware & Software

DatasetsDatasets• All Diseases/ModalitiesAll Diseases/Modalities

KitwareKitware

Open Source Imaging ToolkitsOpen Source Imaging Toolkits

Visualization Toolkit (VTK)Visualization Toolkit (VTK) Insight Toolkit (ITK) Insight Toolkit (ITK)

Rendering and DisplayRendering and Display Image Analysis Image Analysis

High Quality, State-of-the-art Toolkits for Research and Product DevelopmentHigh Quality, State-of-the-art Toolkits for Research and Product Development

KitwareKitware

The Visualization Toolkit The Visualization Toolkit (www.vtk.org)(www.vtk.org)

– C++ class library for visualizationC++ class library for visualization

– Designed using object-oriented principlesDesigned using object-oriented principles

– Portable across Unix, Linux, Windows, MacOSXPortable across Unix, Linux, Windows, MacOSX

– Automated Java, TCL, Python bindingsAutomated Java, TCL, Python bindings

– Automated nightly testingAutomated nightly testing

– Supports 3D/2D graphics, visualization, image Supports 3D/2D graphics, visualization, image

processing, and volume renderingprocessing, and volume rendering

– BSD-style licenseBSD-style license

– C++ class library for visualizationC++ class library for visualization

– Designed using object-oriented principlesDesigned using object-oriented principles

– Portable across Unix, Linux, Windows, MacOSXPortable across Unix, Linux, Windows, MacOSX

– Automated Java, TCL, Python bindingsAutomated Java, TCL, Python bindings

– Automated nightly testingAutomated nightly testing

– Supports 3D/2D graphics, visualization, image Supports 3D/2D graphics, visualization, image

processing, and volume renderingprocessing, and volume rendering

– BSD-style licenseBSD-style license

KitwareKitware

VTKVTK

KitwareKitware

VTK by the NumbersVTK by the Numbers

CommunityCommunity6565 core developerscore developers26552655 person mailing listperson mailing list

Source CodeSource Code1,2981,298 C++ classesC++ classes737,359737,359 lines of C++ codelines of C++ code

Daily Testing SnapshotDaily Testing Snapshot7272 machine configurations (hw machine configurations (hw

+ sw)+ sw)7474 build and test sessions/daybuild and test sessions/day897897 regression tests/buildregression tests/build

VTK C++ FilesVTK C++ Files

16751822

2013

2443

0

500

1000

1500

2000

2500

3000

2003 2004 2005 2006

KitwareKitware

t0

t 8

t

I(t0,) = Q(t) e dt (1)

t = t0

t 8 - a(t’) + sc(t’)dt’t’ = t0

t

’ I(t, ’) d ’ (2)Q(t) = q(t) + sc(

4

’ (In Scattering)

(Emission)

(Absorption)

(Out Scattering)

t’

Volume RenderingVolume Rendering

KitwareKitware

Volume Rendering ArchitectureVolume Rendering Architecture

RendererProp

Collection

Volume

VolumeMapper

VolumeProperty

ImageData

......

Property

Input Mapper

VolumeRayCastMapper

VolumeRayCastFunction

GradientEstimator

GradientEncoder

GradientShader

KitwareKitware

Volume Rendering MethodsVolume Rendering Methods

1) Ray Casting1) Ray Casting

• Floating PointFloating Point

• Composite w/shadeComposite w/shade

• Gradient Magnitude OpacityGradient Magnitude Opacity

• MIPMIP

• IsoSurfaceIsoSurface

• Fixed PointFixed Point

2) Shear Warp2) Shear Warp

3) Texture Mapping3) Texture Mapping

• 2D2D

• 3D3D

4) VolumePro Hardware (past)4) VolumePro Hardware (past)

……with support for LOD control, multi-threading, …with support for LOD control, multi-threading, …

KitwareKitware

VTK Dashboard using DARTVTK Dashboard using DART

KitwareKitware

CMake CMake (www.cmake.org)(www.cmake.org)

Recently AdoptedRecently Adopted

By Linux KDEBy Linux KDE

KitwareKitware

ITK: The Insight Toolkit ITK: The Insight Toolkit (www.itk.org)(www.itk.org)

• Toolkit started in 2000Toolkit started in 2000• Open source segmentation & registration algorithmsOpen source segmentation & registration algorithms

$13M directly invested to date$13M directly invested to date

KitwareKitware

ITK DevelopersITK Developers

KitwareKitware

• Image ProcessingImage Processing

• SegmentationSegmentation

• RegistrationRegistration

• No Graphical User Interface (GUI)No Graphical User Interface (GUI)

• No VisualizationNo Visualization

ITKITK

KitwareKitware

ITK by the NumbersITK by the Numbers

CommunityCommunity5858 core developerscore developers11381138 person mailing listperson mailing list

Source CodeSource Code11751175 C++ classesC++ classes458,770458,770 lines of C++ codelines of C++ code

Daily Testing SnapshotDaily Testing Snapshot8484 machine configurations (hw + sw)machine configurations (hw + sw)107107 build and test sessions/daybuild and test sessions/day11951195 regression tests/buildregression tests/build

KitwareKitware

ImageRegistrationFramework

MultiResolution

RegistrationFramework

PDEBased

Registration

FEMBased

Registration

Components

Registration in ITKRegistration in ITK

FixedImage

MovingImage

Metric

Transform

Interpolator

Optimizer

Registration Method

Fixed Image Moving Image Registered Moving Image

KitwareKitware

Code

InputData

JournalRepository

WebSite

ResultsData

Author

BuildMachines

PDF doc

Insight JournalInsight Journal

KitwareKitware

Use of VTK / ITKUse of VTK / ITK

• ResearchResearch– Algorithm Development & ComparisonAlgorithm Development & Comparison– PrototypesPrototypes– Higher-Level Application ToolkitsHigher-Level Application Toolkits

• ProductsProducts– ScannersScanners– Analysis WorkstationsAnalysis Workstations– PACSPACS– Interventional DevicesInterventional Devices– Research InterfacesResearch Interfaces

KitwareKitware

Open Source ImpactOpen Source Impact

Hans Johnson Lab, University of IowaHans Johnson Lab, University of Iowa

– Lab uses ITK and VTKLab uses ITK and VTK

– Student working on atlas based segmentation methodStudent working on atlas based segmentation method

– Danielsson signed distance filter is a bottleneckDanielsson signed distance filter is a bottleneck

Insight Journal 2/20/2006Insight Journal 2/20/2006

– 9:26am:9:26am: IJ mail arrives about EDT filter (PAMI IJ mail arrives about EDT filter (PAMI

‘03)‘03)

– 12:30am:12:30am: Print and read documentationPrint and read documentation

– 1:00pm:1:00pm: Embed and run EDT codeEmbed and run EDT code

– 1:03pm:1:03pm: !#%?! It runs in only 3 minutes? !#%?! It runs in only 3 minutes?

(10x)(10x)

– 1:10pm:1:10pm: Compare results & do dance of joyCompare results & do dance of joy

– 4:00pm:4:00pm: Submit Insight Journal reviewSubmit Insight Journal review

KitwareKitware

NAMIC: SLICERNAMIC: SLICER

KitwareKitware

Mac: OSIRIXMac: OSIRIX

Over 8000 Users ~250 Downloads/DayOver 8000 Users ~250 Downloads/Day

KitwareKitware

BrainLab Vector Vision LinkBrainLab Vector Vision Link

Provides a direct Provides a direct interface with the interface with the VectorVision IGS VectorVision IGS navigation system to navigation system to design and implement design and implement new features and new features and functionalityfunctionality

KitwareKitware

Product Use: MeVisLabProduct Use: MeVisLab

KitwareKitware

VolView Clinical (Work in Progress)VolView Clinical (Work in Progress)

Not FDA Approved for Clinical UseNot FDA Approved for Clinical Use

KitwareKitware

Open Source Toolkit GrowthOpen Source Toolkit Growth

7,359,0007,359,000 hits/month on public.kitware.com hits/month on public.kitware.com

KitwareKitware

SummarySummary

Open Source Imaging ToolkitsOpen Source Imaging Toolkits

– Efficiently leverage global skills and resourcesEfficiently leverage global skills and resources

– Represents a large and growing resourceRepresents a large and growing resource

– Accelerates healthcare research and product Accelerates healthcare research and product developmentdevelopment

– Will drive down the cost of healthcareWill drive down the cost of healthcare

KitwareKitware

Thank YouThank You

KitwareKitware

Doxygen Manual PagesDoxygen Manual Pages

KitwareKitware

Insight Software Consortium (ISC)Insight Software Consortium (ISC)

A non-profit that provides open-science environments A non-profit that provides open-science environments for developers and users involved in medical image for developers and users involved in medical image

analysis researchanalysis research

Users’ and Developers’ Point of ViewUsers’ and Developers’ Point of View ISC ProvidesISC ProvidesDoes the software work?Does the software work? DashboardsDashboards

Time and interaction to download and install?Time and interaction to download and install? PackagingPackaging

Documentation online and available for download?Documentation online and available for download? Documentation StandardDocumentation Standard

Consistent code quality?Consistent code quality? Coding StandardCoding Standard

Active support for individual packages, e.g., email lists?Active support for individual packages, e.g., email lists? Support StandardSupport Standard

How can I learn more about the software?How can I learn more about the software? ISC hosts tutorials: ISC hosts tutorials: SPIE…SPIE…

How can I contribute my research and dataHow can I contribute my research and data The Insight JournalThe Insight Journal

Open-source changes too quickly! Open-source changes too quickly! Backward compatible policyBackward compatible policy

What open-science sites are available?What open-science sites are available? The ISC website, The IJThe ISC website, The IJ

KitwareKitware

• 37 Submissions37 Submissions

• 90 Reviews 90 Reviews

• by 31 Reviewersby 31 Reviewers

• 261 Registered Users261 Registered Users

• Anonymous reviews allowed, but none were Anonymous reviews allowed, but none were

submittedsubmitted

• Reviewers scored Reviewers scored

– SensitivitySensitivity

– HelpfulnessHelpfulness

Workshop ResponseWorkshop Response

KitwareKitware

Open Data: NIH RIDER DatabaseOpen Data: NIH RIDER Database

KitwareKitware

vtkVolumeMappervtkVolumeMapper

Code is ContinuouslyCode is Continuously

Explored and Tested Explored and Tested

by a Large Number ofby a Large Number of

Users and Experts Users and Experts

KitwareKitware

Image Guided Surgery ToolkitImage Guided Surgery Toolkit

KitwareKitware