NA-MICNational Alliance for Medical Image Computing
Georgia Tech Contributions to NAMIC
Allen Tannenbaum
Georgia Tech Researchers
• Delphine Nain (GRA): Shape analysis, rule-based and stochastic methods in segmentation, validation methods (Received Ph.D. 2006)
• John Melonakos (GRA): Knowledge-based segmentation; rule-based segmentation, DTI, fMRI
• Eric Pichon: DTI Tractography, Statistically-based segmentation, validation, visualization (Received Ph.D. 2005)
• Shawn Lawton (GRA): Conformal and optimal transport methods for image registration
• Marc Niethammer: Active contour methods (Received Ph.D. 2004); posdoc 2005; now working with Martha Shenton
• Ramsey Al-Hakim (Undergraduate researcher): Rule-based segmentation methods
• Vandana Mohan (GRA): DTI, directional based segmentation• Yi Gao (GRA): Shape based analysis and representations• Xavier LeFaucheur (GRA): KPCA, GPCA
NAMIC Summary
• Our Contributions:– Finding white matter tracts and blood vessels– Shape representation– Geometric and shape driven segmentation– Conformal and optimal transport methods for visualization and
registration– Rule-based segmentation– Statistical/PDE methods– Stochastic methods for curvature based flows in medical imaging
• Benefits to us:– Imaging for neuroengineering and neuroscience research– Work directly with clinicians– Opportunity to have our methods directly impact medical imaging
technology
• Cost functional:
• Resulting maximizing flow:
– unidirectional flow, efficient implementation – part of open-source software 3D Slicer
favors largeregions
favors homogeneous
regions
normalizedhistogram
Region-based segmentation
Pichon, Tannenbaum & Kikinis, MICCAI and MedIA
Region-based Segmentation Results:
White and gray matter (MRI)
comparison to groundtruth from [kaus01]
2-d sagittal slice of image and proposed segmentation
3-d rendering
proposed
groundtruth
Region-based Segmentation Results:
Brain ventricle (MRI)
comparison to groundtruth from [kaus01]
2-d axial slice of image and proposed segmentation
3-d rendering
proposed
groundtruth
• Define a direction-dependent local cost:
• Examples– Tractography: using high angular resolution dataset,
define (p,d) to be small if there is a fiber at position p in direction d, large otherwise
– Segmentation: define (p,d) from a direction-dependent pattern detector
localcost
Direction-dependent segmentation(Active Contours in a Finsler Metric)
positiontangentdirection
localcost
globalcost
curve
localcost
Direction-dependent segmentation
Define the global cost of a curve by integrating the local cost. Minimal cost curves can be obtained using dynamic programming or calculus of variations.
Applications
proposedtechnique
streamline technique(based on tensor field)
2-d axial slide of tensor field (based on S/S0)
Diffusion MRI tractography Vessel Segmentation
We represent shapes with spherical wavelet basis functionslocalized in space and scale
Localized Shape Analysis
Spherical wavelet functions
Resolution 2 Resolution 4
To describe a shape in a population, each wavelet coefficient encodes variation from the mean shape at a particular scale & location
Original Caudate
Mean Caudate
Low Resolution Wavelet coeffs
Low and High Resolution coeffs
= + …+ +…+
Characterization of local variations could be important for shape analysis since a disease, such as cancer, could affect only a portion’s of an organ’s surface
• Our technique learns a shape prior from the distribution of the wavelet coefficients
• In an estimation task, our prior incorporates local details that a previous technique (PCA) does not encode and significantly improves the approximation of shapes.
Localized Shape Analysis
Used multiscale shape prior in a segmentation framework and apply it to the task of shape classification
Ground Truth PCA estimationWavelet prior estimation
Comparison of techniques for estimation of a test shape (not used to learn shape prior)
Evolve Shapeevolve ,p
Data PriorShape Prior
Shape Representation Segmentation
• Region-Based Active Contour
Pose: Rotations,Translations, Scaling
shape
Prior: • represent shape in eigenvector subspace ( coordinates)• constrain value (+/- 4 std)
Shape Driven Segmentation
Automatic Brain Registration
• Given a segmented surface
• Use deep sulci as landmarks
• Use conformal map to flatten surface to an annulus
• Use mass preserving map to register the two annuli
• This technique could help perform 3D atlas registration extremely quickly
Area Preserving Surface Warping of Minimal Distortion
Optimal transport allows one to find area correcting flattening. After conformally flattening surface, apply area correcting map to find area-preserving flattening of minimal distortion.
Bayesian Classifier Image Filter:This filter performs Bayesian image segmentation.
John Melonakos, Georgia Institute of Technology
Luis Ibanez, Kitware (software)Karthik Krishnan, Kitware (software)
Algorithm: The basic idea is to incorporate prior knowledgeinto the segmentation through Bayes’ rule. Image noise is removed via an affine invariant anisotropic smoothing of the posteriors
AlgorithmTeam
Publication Illustration of Algorithm
Clinical Applications: test by segmenting brain volumes into white, gray and background
Open Source SoftwareITK code developed during the NAMIC Programming Week in 2006 has been ported to an ITK filter and is in the NAMIC Sandbox and ITK CVS repository.
J. Melonakos, K. Krishnan, and A. Tannenbaum. “An ITK Filter for Bayesian Segmentation: itkBayesianClassifierImageFilter”.Insight Journal, 2006
Raw Image Manual Segmentation ITK Filter Output
Conformal Flattening:ITK Conformal Flattening Filter. This is useful for the visualization of irregular surfaces.
Yi Gao, Georgia Institute of TechnologyJohn Melonakos, Georgia Institute of Technology
Jim Miller, GE (software)Luis Ibanez, Kitware (software)
Algorithm: Use conformal mapping to map an irregular surface onto a sphere while preserving the angle.
class: itkConformalFlatteningFilterAPIs: filter->setPointP(cellId); filter->mapToPlane( ); filter->setScale( scaleFactor );
Y. Gao, J. Melonakos, and A. Tannenbaum. “Conformal Flattening ITK Filter.” ISC/NA-MIC Workshop on Open Science at MICCAI 2006
ITK code developed during the NAMIC Programming Week in 2006 has been ported to an ITK filter and is in the NAMIC Sandbox
Clinical Applications: This is useful for the visualization of irregular surfaces.In the special case of FMRI visualization, the flattened view facilitates the understanding of the mapping of function to spatial location.
AlgorithmTeam
Publication
Open Source SoftwareBrain Surface
Conformal Mapping ofBrain Surface
Illustration of Algorithm
3D shape analysis using Spherical Wavelets:Develop algorithm and ITK software module for Spherical Wavelet computation
Xavier Le Faucheur, Georgia Institute of TechnologyDelphine Nain, Georgia Institute of TechnologyYi Gao, Georgia Institute of Technology
Martin Styner, University of North CarolinaJohn Melonakos, GT /GELuis Ibanez, Kitware (software)
Algorithm: decomposition of a scalar signal defined on a spherical mesh into spherical wavelet coefficients and vice-versa. This algorithm can be used to represent shape via a conformal mapping to the sphere, and then an encoding of the (x,y,z) signal into wavelet coefficients.
Clinical Applications: The multiscale shape representation can be used for segmentation and shape analysis [1-2]
AlgorithmTeam
[3] X. LeFaucheur, Y. Gao, D. Nain, A. Tannenbaum “Spherical Wavelet ITK Filter”. To be submitted to Insight journal.
ITK code developed during the NAMIC Programming Week in 2006 is in the NAMIC Sandbox.
Publications
Open Source Software
Illustration of Algorithm
LP HP
LP HP
LP HP
SphericalWaveletObject
SphericalWaveletObject
Signal-to-Coeffs Filter
Coeffs-to-Signal Filter
[1] D. Nain, et al. “Multiscale 3D Shape Analysis using Spherical Wavelets”. In MICCAI, 2005 [2] D. Nain, et al. “3D Segmentation using Spherical Wavelets”. In MICCAI, 2006
Rule Based Segmentation:Semi-Automatic Segmentation of Brain Structures using rules created by neuroanatomists
Ramsey Al Hakim, Georgia Institute of TechnologyShawn Lankton, Georgia Institute of TechnologyTauseef Rehman, Georgia Institute of TechnologyJohn Melonakos, Georgia Institute of TechnologyDelphine Nain, Georgia Institute of Technology
Jim Levitt, Brigham and Women’s HospitalJim Fallon, UCI
Striatum Segmentation: Manually input most superior/dorsal point on putamen and anterior commissure; striatum is then delineated automatically based on rules of Dr. James Levitt.Dorso-Lateral Prefrontal Cortex Segmentation: Manually input temporal lobe tip and frontal pole points based on Fallon-Kindermann Rules; DLPFC is then segmented automatically using the bayesian classification tool
Clinical Applications: Study the effects of schizophrenia on shape.
AlgorithmsTeam
Rule-Based Segmentation Module is in Slicer 2
Publications
Open Source Software
Illustration of Algorithm
[1] R. Al-Hakim, J.Fallon, D. Nain, J. Melonakos, A. Tannenbaum. A DLPFC semi-automatic segmenter. In SPIE Medical Imaging, 2006.
[2] J. Melonakos, R. Al-Hakim, J. Fallon, and A. Tannenbaum. Knowledge-Based Segmentation of Brain MRI Scans Using the Insight Toolkit. Insight Journal, 2005.
Post Putamen
Pre CaudatePost Caudate
Nucleus Accumbens
Pre Putamen
superior/dorsal point
on putamen
Anterior commissure