Upload
nguyenque
View
236
Download
0
Embed Size (px)
Citation preview
Computational Anatomy Modeling of Abdominal Organs and Musculoskeletal Structures
Yoshinobu Sato
Graduate School of Information ScienceNara Institute of Science and Technology (NAIST)
Japan
Symposium on Statistical Shape Models & ApplicationsDelémont, Switzerland
June 11‐13, 2014
Imaging-based Computational Biomedicine Lab
Osaka University
Osaka
Kyoto
Nara
NAISTTokyo
Information ScienceMaterial ScienceBiological Science
NAISTNara Institute of Science
and Technology
Statistical Shape Models (SSMs) & Applications in this talk
Abdominal Organs
Musculoskeletal Structures
Implants & Host Bones
Hierarchical SSM
Conditional SSMMuscles
PLSR prediction‐based conditional SSMs & probabilistic atlas
Non‐conditional Conditional
SSM & statistical distance maps
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
http://www.comp‐anatomy.org/Google search by “computational anatomy”.
Locations of eight core groups
MEXT Grant‐in‐aid for Scientific Research, JapanComputational Anatomy for Computer‐Aided Diagnosis and TherapySep 2009 ‐Mar 2014Fund: $10 millionPrincipal Investigator: Prof. Hidefumi Kobatake(TUAT: Tokyo University of Agriculture & Technology)Eight core groups
Basic theories and technologiesApplication systemsClinical evaluations
The aim was to develop computational anatomy models of the human body (especially in torso), which represent inter‐subject variability of anatomy across a population, and their applications.
One of our goals: Complete understanding of whole‐body CT images
http://www.comp‐anatomy.org/Google search by “computational anatomy”.
Osaka Univ.(My former affiliation)
MEXT Grant‐in‐aid for Scientific Research, JapanComputational Anatomy for Computer‐Aided Diagnosis and TherapySep 2009 ‐Mar 2014Fund: $10 millionPrincipal Investigator: Prof. Hidefumi Kobatake(TUAT: Tokyo University of Agriculture & Technology)Eight core groups
Basic theories and technologies (Tokyo, Osaka, Gifu)Application systemsClinical evaluations
Conventional Representation of Human Anatomy
• Book Atlas– Detailed illustrations of
typical anatomy
• 3D Digital Atlas– Detailed segmented 3D data of
a specific subject
Frank H. Netter, Atlas of Human Anatomyhttp://www.voxel‐man.de/
Visible Human data (NIH)
Semi‐automated segmentation
VOXEL‐MAN (Univ. Hamburg)
They are constructed by Manual Drawing or Semi‐automated Segmentation.They only show One Typical Example or One Particular Example.
3D Digital Atlas
Visible Human DataSemi‐automated segmentation
One Particular Anatomy
Reconstructed from Special data with Labor‐intensive efforts
VOXEL‐MAN (Univ. Hamburg)
Visible Human DataSemi‐automated segmentation
One Particular Anatomy
Reconstructed from Special data with Labor‐intensive efforts
VOXEL‐MAN (Univ. Hamburg)
GoalPatient 3D Data
Fully‐automated segmentation
Patient‐Specific Anatomy(equivalent to Visible Human & VOXEL MAN)
From Clinical data as Routine work
GoalPatient 3D Data
Fully‐automated segmentation
From Clinical data as Routine work
Patient‐Specific Anatomy(equivalent to Visible Human & VOXEL MAN)
ApproachPatient 3D Data
Fully‐automated segmentation
Reconstructed from Clinical data as Routine work
Computational Anatomy Models Representing Inter‐Patient Variability of Multiple Organs
….….
Atlas (training) datasets
Shape & Location Priors in Bayesian Estimation
Patient‐Specific Anatomy(equivalent to Visible Human & VOXEL MAN)
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
Target Abdominal Organs
• Liver (brown) • Spleen (blue violet)• Kidneys (pink)• Pancreas (yellow)• Gallbladder (green) • Aorta and artery
branches (red) • Inferior vena cava (IVC)
and vein branches (cyan)
• Upper GI tract (cream yellow)
Segmented OrgansToshi Okada, PhD
(Currently, University of Tsukuba)
Masatoshi Hori, MD
Organ segmentation via computational anatomy
Conventional framework
Computational Anatomy (CA) Model
Automated Construction
AutomatedSegmentation
Abdominal CT
Target 3D data Patient
anatomy
[Okada MICCAI 2007][Okada Acad Radiol 2008]
Manually‐traced organ shape data
Training dataLabeled DICOM data
Intensity priorsShape and location priors*
*Inter‐Patient Anatomical Variability of Organ Shape and Location
Inter‐Patient Anatomical Variability of Organ Shape: Conventional Representation
Segmentation ≒ Voxel‐wise MAP (Maximum a Posterior) estimation (Initialization is unnecessary after spatial normalization.)
Probabilistic Atlas (PA)Voxel‐wise probability map of organ existence
in the normalized abdominal space
[Okada et al. MICCAI 2007] [Park et al. TMI 2003]
Inter‐Patient Anatomical Variability of Organ Shape: Conventional Representation
Segmentation ≒ Statistically constrained deformable model fitting ≒ Global MAP estimation (Initialization is needed.)
Statistical Shape Model (SSM) (PCA of 3D shape)Statistical constraints (inter‐patient variability) on shape and location
in the normalized abdominal space
[Okada et al. MICCAI 2007] [Lamecker et al. 2004]
Roles of SSM from the mathematical viewpoint:
• Effective (Dimensionality) Reduction of Solution Space (fewer parameters for representing target shapes)
• Prior Probability Distributions in Bayesian Estimation
e1
eN
e2
)|()()|( MDPMPDMP
n‐dimensional solution space representing all shapes
v2
v1
Reduced mL‐d solution space for possible liver shapes (mL<<n)
Reduced mF‐d solution space for possible femur shapes (mF<<n)
P(M)
v1
e3
Prior LikelihoodPosterior
Conventional Method [Okada MICCAI 2007] +
A Single Organ Segmentation Method
CT image
Spatial standardization
Initial segmentation by PA
SSM refinement
Graph‐cut refinement
Segmentation result
Probabilistic Atlas (PA)
Statistical Shape Model (SSM)
Intensity Model
Liver
[Okada MICCAI 2007]
+ Likelihood
Prior
Conventional Method [Okada MICCAI 2007] +
A Single Organ Segmentation Method
CT image
Spatial normalization
Initial segmentation by PA
SSM refinement
Graph‐cut refinement
Segmentation result
Probabilistic Atlas (PA)
Statistical Shape Model (ML‐SSM)
Intensity Model
Right kidney
[Okada MICCAI 2007]
+ Likelihood
Prior
Extension to multi‐organ modeling and segmentation
Organ segmentation via computational anatomyConventional framework
Computational Anatomy (CA) Model
Automated Construction
AutomatedSegmentation
Abdominal CT
Target 3D data Patient
anatomy
[Okada MICCAI 2007][Okada Acad Radiol 2008]
Manually‐traced organ shape data
Training dataLabeled DICOM data
Intensity priorsShape and location priors*
LimitationsInter‐relations among organs are not utilized.
P(Liver) P(Spleen|Liver)
P(L‐Kidney|Liver,Spleen)
P(Pancreas|Liver,Spleen)P(Gallbladder|Liver)
P(R‐Kidney|Liver)
Organ correlation graph (OCG)Conditional shape & location prior (SSM & PA) network
[Okada, MICCAI 2013]
• Given predictor organs P, PLSR predicts the target organ shape. The prediction error E(P) is given by
E(P) = S ‐ S’(P) (S is true shape and S’(P) predicted shape.)
PLSR (Partial Least Squares Regression)Prediction‐based Conditional Priors
…
Training data Predictor organs P
PLSR predictor S’(P)
Predicted target shape S’
Training Phase Execution Phase
Predictor
Target
[Okada, MICCAI 2013]
• Given predictor organs P, PLSR predicts the target organ shape. The prediction error E(P) is given by
E(P) = S ‐ S’(P) (S is true shape and S’(P) predicted shape.)• Among all possible combinations of predictor organs, predictor
organs Pminimizing prediction error E(P) are selected for each target organ, which define arcs of OCG (organ correlation graph).
PLSR (Partial Least Squares Regression)Prediction‐based Conditional Priors
…
Training data Predictor organs P
PLSR predictor S’(P)
Predicted target shape S’
Training Phase Execution Phase
Predictor
Target
[Okada, MICCAI 2013]
P(Liver) P(Spleen|Liver)
P(L‐Kidney|Liver,Spleen)
P(Pancreas|Liver,Spleen)
[Okada, MICCAI 2013]
P(Gallbladder|Liver)
P(R‐Kidney|Liver)
Organ correlation graphConditional shape & location prior (SSM & PA) network
Anchor organ
Predictor organ
Predictor organ
Predictor organ
Prediction‐based Statistical AtlasProbabilistic Atlas (PA)
• Prediction error E is modeled as probabilistic atlas (PA) to generate less ambiguous PA. E = S ‐ S’ (S: True shape, S’: Predicted shape, E: Prediction error)
P(Pancreas|Liver,Spleen)
P(R‐Kidney|Liver)
P(Pancreas)
P(R‐Kidney)
P(Gallbladder|Liver )P(Gallbladder)
Prediction‐based (Conditional)Conventional
P(Liver)P(Spleen|Liver)
P(L‐Kidney|Liver)
P(Pancreas|Liver)
P(Gallbladder|Liver)
P(R‐Kidney|Liver)
Organ correlation graph (OCG)Conditional shape & location prior (SSM & PA) network
Anchor organ
Predictor
Predictor
Predictor
Predictor
Predictor
[Okada, MICCAI 2013]
Probabilistic Atlas using Known Liver Shape
Prediction‐based Statistical AtlasProbabilistic Atlas (PA)
• Prediction error E is modeled as probabilistic atlas (PA) to generate less ambiguous PA. E = S ‐ S’ (S: True shape, S’: Predicted shape, E: Prediction error)
Prediction‐based (Conditional)ConventionalPredictor: Liver Predictor: Liver, Spleen, Kidneys
Prediction‐based Statistical AtlasStatistical Shape Model (SSM)
• The prediction error E is also modeled using PCA in prediction‐based SSM to obtain more constrained variability.E = S ‐ S’ (S: True shape, S’: Predicted shape, E: Prediction error)
P(Pancreas|Liver,Spleen)
P(R‐Kidney|Liver)P(R‐Kidney)
P(Gallbladder|Liver )P(Gallbladder)
Prediction‐based (Conditional)ConventionalP(Pancreas)
Prediction‐based Segmentation Method
CT image
Spatial standardization
Initial segmentation by PA
ML‐SSM refinement
Graph‐cut refinement
Segmentation result
Prediction‐based PA
Prediction‐based SSM
Intensity Model
Segmentation results of predictor organs
Organ segmentation via computational anatomy
Multi‐organ interrelation modeling
Target‐data specific model
Customized Computational Anatomy (CA) Model
Automated Construction
AutomatedSegmentation
Abdominal CT
AutomatedCustomization
Target 3D data Patient
anatomy
Manually‐traced organ shape data
Training dataLabeled DICOM data
Intensity priors
Generic Computational Anatomy (CA) Models
Shape and location priors
Multi‐organ modeling inherent in anatomy
[Okada Abd‐ImgWS 2011]
[Okada EMBC 2012]
Intensity prior modeling (IM)• In abdominal CT segmentation, we have to deal with a variety of contrast enhancement (CE) patterns.
• A new intensity prior model (IM) has to be constructed to deal with a new CE pattern.
Contrast‐enhancedLate arterial phase
Non (blood) contrast but oral contrast
Contrast‐enhancedVenous phase
Contrast‐enhancedEarly arterial phase
Intensity prior modeling (IM)• Supervised intensity modeling (IM) : Conventional
– Intensity prior modeling from “labeled” DICOM dataset• A set of CT images and manual traces on them for each CE
• Unsupervised intensity modeling (IM): Proposed– Intensity prior modeling from “unlabeled” DICOM dataset
• A set of CT images but no traces for each CE pattern
– Target data specific (no training dataset for IM)
Contrast‐enhancedLate arterial phase
Non (blood) contrast but oral contrast
Contrast‐enhancedVenous phase
Contrast‐enhancedEarly arterial phase
Organ segmentation via computational anatomyTowards easily customizable and extendable systems
Target‐data specific model
Customized Computational Anatomy (CA) Model
Automated Construction
AutomatedSegmentation
Abdominal CT
AutomatedCustomization
Target 3D data
Patient anatomy
[Okada Abd‐ImgWS 2011]
[Okada EMBC 2012]
Manually‐traced organ shape data
Training dataLabeled DICOM data
Intensity priors
Generic Computational Anatomy (CA) Models
Shape and location priors
Multi‐organ modeling inherent in anatomy
no
Organ segmentation via computational anatomyTowards easily customizable and extendable systems
Multi‐organ modeling inherent in anatomy
Generic Computational Anatomy (CA) Models
Imaging‐condition/Target‐data specific model
Customized Computational Anatomy (CA) Model
Shape and location priors
Automated Construction
AutomatedSegmentation
Abdominal CT
AutomatedCustomization
Target 3D data
Patient anatomy
[Okada MICCAI 2013]
Manually‐traced organ shape data
Training data Unlabeled DICOM of specific imaging method/protocol
AutomatedCustomization
Intensity priors
Joint segmentation and intensity modeling
Organ segmentation via computational anatomyTowards easily customizable and extendable systems
Multi‐organ modeling inherent in anatomy
Generic Computational Anatomy (CA) Models
Imaging‐condition/Target‐data specific model
Customized Computational Anatomy (CA) Model
Automated Construction
AutomatedSegmentation
Abdominal CT
AutomatedCustomization
Target 3D data
Patient anatomy
[Okada MICCAI 2013]
Manually‐traced organ shape data
Training data Joint segmentation and intensity modeling
Shape and location priors
Intensity priors
Cope with Unknown Imaging Condition
Results
Experiments• Upper abdominal CT data at two different hospitals were used.
– Non‐contrast (but artifact due to oral contrast) at NIH: 12 cases– Venous phase at NIH: 25 cases– Early and late arterial phases at Osaka Univ. Hospital
• Old protocol: Slice thickness 2.5 mm: 10 cases for each phase• New protocol: Slice thickness 0.625 mm: 39 cases for each phase
– Totally, CT data of 134 cases (86 patients) with 4 different CE patterns were used.
• 2‐fold cross validation was performed. CT data with the same CE pattern as test data were not involved in any parameter tuning.
• The segmentation methods were fully automated.Contrast‐enhancedLate arterial phase
Non (blood) contrast but oral contrast
Contrast‐enhancedVenous phase
Contrast‐enhancedEarly arterial phase
Case 1 (Osaka, Late arterial phase)
Ground truth Prediction‐based CA(Unsupervised IM)
Conventional CA(Unsupervised IM)
Conventional CA(Supervised IM)
Jaccard Index Liver Spleen R-Kidney L-Kidney Pancreas Gallbladder Aorta IVC
Prediction 0.941 0.980 0.963 0.747 0.543 0.935 0.681
Basic(IC-IM)
0.936 0.985 0.964 0.430 0.591 0.833 0.467
Basic(Supervised IC-IM) 0.940 0.984 0.963 0.578 0.933 0.817 0.438
0.916
• Pancreas, aorta, and IVC were better segmented in the proposed prediction‐based method than our conventional method.
Prediction(Unsupervised IM)
Conventional(Unsupervised IM)
Conventional(Supervised IM)
GI‐tract
Conventional Prediction‐based
Conventional
Prediction‐based
Ground truth
Stomach
Esophagus
Duodenum
[Hirayama, 2013]
Summary of abdominal multi‐organ segmentation
• Multi‐organ modeling and segmentation methods were proposed which effectively utilize the organ interrelations.
• Unsupervised intensity prior modeling combined with prediction‐based CA models can make the method adaptive to different CE patterns.
• Once key organs are segmented, other structures including GI‐tract, vessel branches, and tumors are effectively segmented and anatomically identified.
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
Musculoskeletal anatomy
Muscle tissues
17 Muscles
FutoshiYokota, MS
Pelvis & Femur
Nobuhiko Sugano, MD
Masaki Takao, MD
Diseased hip joint
Unaffected hipPrimary
osteoarthritisSecondary
osteoarthritis( Crowe 1)
Secondary osteoarthritis( Crowe 2)
Collapsed hip
[Yokota, MICCAI 2013]
100 CT data of Total Hip Arthroplasty (THA) patients:All patients had healthy hip on one side and diseased the other
1. Globally consistent initial segmentation using hierarchical hip SSM2. Accurate segmentation of joint part using conditional SSMs3. Final refinement by graph cut
Approach of bone segmentation
Hierarchical hip SSM
Conditional femoral head SSM
[Yokota, MICCAI 2013]
More AccurateMore RobustSpecificity
>Generality
<
[de Bruijne MICCAI 2006]
[Okada, MICCAI 2007]
Conditional SSM
Given partPelvis and distal femur
Conditionalfemoral head SSM
Standardfemoral head SSM
[Yokota, MICCAI 2013]
[de Bruijne MICCAI 2006]
Segmentation by Hierarchical SSM fitting
• Initial rough segmentation of bone regions using simple thresholding where joints part is not separeted.
[Yokota et al. MICCAI 2009]
Segmentation by Hierarchical SSM fitting
• Coarse fine fitting of hierarchical SSM is performed.
[Yokota et al. MICCAI 2009]
Segmentation by Hierarchical SSM fitting
• Coarse fine fitting of hierarchical SSM is performed.– Initial fitting of combined pelvis and femur SSM
– Subsequent fitting of pelvis & femur SSMs with consistency constraint
– Fitting and edge updating are repeated.
[Yokota et al. MICCAI 2009]
CT image Ground truth Independent SSMs Conditional SSM
Primary osteoarthritis
Secondary osteoarthritis( Crowe 1)
Secondary osteoarthritis( Crowe 2)
Collapsed hip
ResultsRed: pelvis Green: femur
Musculoskeletal anatomyPelvis & Femur Muscle tissues 17 Muscles
Different patients
Initialbone & skin segmentation
Hierarchical multi‐atlas label fusionAutomatically segmented patient label images
Atlas datasets
Finalsegmentation
Skin, pelvis & femur
Final stage: 17 muscle segmentation
Intensity images
Label images for label fusion
Target CT image First stage:
Muscle tissue segmentation
Muscle tissue
….
2 datasets
….
38 datasets
5 selected muscles
Second stage: 5 selected muscle segmentation
….
38 datasets
….….
Automatically segmented patient label image
Label images for spatial normalization (cancelation of variability)
[Yokota, CAOS 2012]Best Technical Paper Award
Musculoskeletal segmentationResults
Three‐stage(1.9 mm error)
Two‐stage(3.0 mm error)
Single‐stage(4.1 mm error)
Front views
Back views
[Yokota, CAOS 2012]
Musculoskeletal segmentationResults
Three‐stage(1.9 mm error)
Two‐stage(3.0 mm error)
Single‐stage(4.1 mm error)
Ground truthOriginal CT images
[Yokota, CAOS 2012]
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
One of our main goals: Complete understanding of whole‐body CT images
http://www.comp‐anatomy.org/Google search by “computational anatomy”.
MEXT Grant‐in‐aid for Scientific Research, JapanComputational Anatomy for Computer‐Aided Diagnosis and TherapySep 2009 ‐Mar 2014Fund: $10 millionPrincipal Investigator: Prof. Hidefumi Kobatake(TUAT: Tokyo University of Agriculture & Technology)Eight core groups
Basic theories and technologiesApplication systemsClinical evaluations
Locations of eight core groups
Tokyo
TUATNagoya
Gifu
Kyushu
Yamaguchi
Tokushima
Osaka
Example of collaborationAbdominal module (Tokyo & Osaka)
Landmark Localization
Abdominal Bounding‐box Localization
Abdominal Multi‐organ Segmentation
, , … ,
Training data
, , … ,
Random forest regression
OsakaTokyo & OsakaTokyo
Prof. Masutani(Univ. of TokyoCurrently, Hiroshima City Univ.)
Musculoskeletal modules
(Gifu, OsakaTokushima)
Lung module (Tokushima)
Vessel modules (Nagoya, Osaka)
Prof. Mori(Nagoya Univ.)
Prof. Fujita(Gifu Univ.)
Prof. Niki(Univ. Tokushima)
Non‐contrast CTFully‐automated Segmentation
• Our computational anatomy project: Overview
• Anatomy modeling– Abdominal anatomy– Musculoskeletal anatomy– Whole‐body anatomy
• Therapeutic modeling– Surgeon’s expertise modeling
• Artificial joint surgery (Total Hip Arthroplasty: THA)
Outline
Cup planning of mildly and severely diseased pelvises: Our problem
Mildly diseased case Severely diseased case• The position and size of the acetabular cup should be basically determined
so as to recover the original anatomy of the acetabulum.
Cup planning of mildly and severely diseased pelvises: Our problem
Mildly diseased case Severely diseased case• The position and size of the acetabular cup should be basically determined
so as to recover the original anatomy of the acetabulum. • Although it is not so difficult to predict the original anatomy for mildly
diseased case, it is somewhat difficult for severely diseased acetabulum due to its severe deformation and shift.
Bone‐Implant Statistical Model (1)Prior probability of likely spatial relations
between patient bone and implant
Patient Pelvis Shape Data: D
Cup Plan
Pelvis‐Cup Statistical Model P(Xpelvis, Xcup)
Surgical Plan Database
Automated Planning
Statistical Analysis
Maximize P(Xpelvis, Xcup)P(D|Xpelvis)
Statistical Shape Model (SSM)
Otomaru et al.CAOS 2009
Maximum a Posterior (MAP) Estimation
Bone‐Implant Statistical Model (2)Prior probability of likely spatial relations
between patient bone and implant
Patient Femoral Cavity Shape Data: D
Stem Plan
Femoral Cavity ‐ Stem Statistical Model P(Xfemur, Xstem)
Surgical Plan Database
Automated Planning
Maximize P(Xfemur, Xstem)P(D|Xfemur)
Statistical Distance Map (SDM)penetration 0 gap
Otomaru et al.Med Image Anal2012
Maximum a Posterior (MAP) Estimation
Summary of this talk
• Statistical shape models (SSMs) and other statistical atlas representation incorporating interrelations among multiple organs (structures) are presented.
• Their applications were demonstrated to– Abdominal organs– Musculoskeletal structures– Bone implant surgical planning
• These problems are formulated as MAP estimation based on Bayes theorem, where SSMs are regarded as prior probability distributions.
Sunrise at Yakushi Temple, Nara, Japan
Thank you!