View
219
Download
1
Tags:
Embed Size (px)
Citation preview
Medical ImageRegistration
Dept. of Biomedical Engineering
Biomedical Image Analysis
www.bmia.bmt.tue.nl
Image registration definition
‘‘ Image registration is about determining a spatial transformation –
or mapping – that relates positions in one image, to corresponding
positions in one or more other images’’
• 3D - 3D• 3D - 2D• 3D/2D - patient
Source image Target image
Example from our group
Medtronic Polestar N20Intra-operative MRI
Pre-Operative Intra-Operative
Student Project Wenxin Wang: REGISTRATION
Many more examples of imaging modalities
X-rays CTAngiographyMRI
Ultrasound SPECT PET
Application of image registration
Same modality, same patient
- monitoring and quantifying disease progression over time,
- evaluation of intra-operative brain deformation, etc…
Different modalities, same patient
- correction for different patient position between scans,
- linking between structural and functional images, etc…
Same modality, different patients
- atlas construction
- studies of variability between subjects, etc…
Temporalregistration
PET
Fusion of images
MRI CT
Colored overlay
PET - CT
Region of interest (ROI) selection & color display
Fusion of images
CT scan of a thyroid gland Fusion of SPECT and CT
Fusion of images
Protein localization
Different spectral bandsfor optical biomarkers
Fusion of images
Mapping of calculatedprobability maps
Fusion of images
Functional MRI maps onAnatomical MRI
fMRI
Weighted intensitycombination
Fusion of images
CT MRI Also possiblewith intermittendpresentation(flicker)
Fusion of images
Checkerboard fusion
Fusion of images
Linkedcursor
Fusion of images
Radiotherapy planning
Iso-dosis contours on CT
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
• Matching with pointbased methods• Matching with surface based methods• Matching with intensity based methods
CTimages
Dynamicseries
Workstation Perfusion images
o
o
o
o
CT Perfusion: matching over time
Marcel QuistPhilips Medical Systems
Medical IT – Advanced Development
• infarct• tumor properties• blood perfusion
o
o
o
o
CT PerfusionMarcel Quist
Philips Medical SystemsMedical IT – Advanced Development
CTimages
Dynamicseries
Workstation Perfusion images
• infarct• tumor properties• blood perfusion
Blood volume Blood current Time to maximumAver. passage time
Courtesy: Charité, Berlin
Functionalperfusionimages
Registration
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
• Matching with pointbased methods• Matching with surface based methods• Matching with intensity based methods
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Image markers
Point-Based Registration
Coordinates for the fiducials can be found on multiple images
One set of fiducials can be lined up with another.
Fiducials
Devicepositiontracking
2 cameras
Finding the Fiducials
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
2D Affine Transforms
Translations by tx and ty
x1 = a x0 + b y0 + tx
y1 = c x0 + d y0 + ty
Rotation around the origin by radians
x1 = cos() x0 + sin() y0
y1 = -sin() x0 + cos() y0
Zooms by sx and sy
x1 = sx x0
y1 = sy y0
Shearx1 = x0 + h y0
y1 = y0
http://www.dt.org/html/meshwarp.html
3D Rigid-body Transformations
A 3D rigid body transform is defined by:
3 translations - in X, Y & Z directions
3 rotations - about X, Y & Z axes
The order of the operations matters
1000
0100
00cossin
00sincos
1000
0cos0sin
0010
0sin0cos
1000
0cossin0
0sincos0
0001
1000
Zt100
Y010
X001
rans
trans
trans
ΩΩ
ΩΩ
ΘΘ
ΘΘ
ΦΦ
ΦΦ
Translations Pitchabout x axis
Rollabout y axis
Yawabout z axis
Geometrical transformations
• Rigid• preserves straightness of lines• intra-patient, rigid anatomy• rotation, translation, zoom, skew
• Curved• inter-patient• atlas• tissue deformation
Image Metrics
FixedImage
MovingImage
Metric
Transform
Interpolator
Value
Parameters
Distance measures
link to pdf
Image Metrics – similarity measures
1. Subtraction:
2. Mean squared differences:
3. Correlation coefficient:
if the intensities are linearly related.
Demo
Entropy
A measure of dispersion or disorder.
High entropy high disorder.
Mutual information
A measure of how well one random variable
(image intensities) “explains” another.
High mutual information high similarity
Similarity Based on Information Theory
Mutual Information
Correct registration Large mis-registration
Wachowiak et al., Proc. SPIE Medical Imaging, 2003
Entropy
Mutual information
Normalized mutual information
H X i 1
np iln p i H X , Y
i 1
nj 1
mp i jln p i j
IX , Y H X H Y H X , Y IX , Y
H X H Y H X , Y
MR – MR (identical images)Translation 2 and 5 mm.
Mutual Information
Mutual Information
MR – CTTranslation 2 and 5 mm.
Demo
Two images are similar if changes of intensity occur at the same locations.
Gradient Field
Normalized Gradient Field:
Regularized Normalized Gradient Field:
Registration Distance Measure (1): Normalized Gradient
Field
I I n
2 2
I
I
n
I
Distance measure of NGF:22
NGF 2 2D [ , ] ( ) ( ) ( ) ( ) sin( )R T R T R T n n n n
Normalized Gradient Field
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Optimization
Optimization involves finding some “best”
parameters according to an “objective function”,
which is either minimised or maximised
The “objective function” is often related to a
probability based on some model
Value of parameter
Objective function
Most probable solution (global
optimum)Local optimumLocal optimum
Plotting the Metric
Mean Squared Differences
Transform Parametric Space
Sensitivity analysis
The Best Transform Parameters
Evaluation of thefull parameter space
is equivalent to performingoptimization by exhaustive searchVery Safe
but
Very SlowBetter Optimization Methods: for example: Gradient Descent
Optimization in Image Registration
Main goal: To determine the transformation
parameters that result in the minimum value of a
‘distance measure’.
Transformation parameters:
Translations
Rotations
Scaling
Find the “best”, or optimum value
of an objective (cost) function.
Very large research area.
Multitude of applications.
Image Registration Framework
FixedImage
MovingImage
Metric
Transform
InterpolatorOptimize
r
Parameters
Applications of Optimization
Engineering designBusiness and
industry
Radiotherapyplanning
Biology and medicine
Economics
Systems biologyManagement
Design ofmaterials
Manufacturing design
BioinformaticsProteomics
Image registration
Finance
Simulation and modeling
Global and local optimization
Local Optimization
Start
End
Local Optimization
Start
Global Optimization
End
Global Optimization
Gradient Descent Optimizer f( x , y )
S = L ∙ G( x , y )f( x , y )
∆
G( x , y ) =
S = Step
L = LearningRate
Gradient Descent Optimizer f( x , y )
S = L ∙ G( x , y )f( x , y )
∆
G( x , y ) =
Registration Framework
ReferenceImage
TemplateImage
CalculateDistanceMeasure
ConditionMet?
TransformedTemplateImage
OptimizeTransformationParameters
TransformTemplateImage
Yes
NO
Multi-Resolution Registration Framework
Registration
Registration
Registration
Fixed Image Moving Image
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Multi-Modality Registration
Fixed Image Moving Image
Registered Moving Image
Classification of registration algorithms:
• Image dimensionality 2D, 3D, time, ...• Registration basis point sets, markers, surfaces, ...• Geometrical transformations affine, perspective, ...• Degree of interaction user initialization, automatic• Optimization procedure max distance, gradient descent• Modalities multi-modal, intra-modal, ...• Subject inter-patient, atlas, ...• Object head, vertebra, liver, ...
Visual Integration Platform for Enhanced Reality (VIPER)
Collaboration withDr. Wieslaw Nowinski,Cerefy Atlas,A*Star, Singapore
Substantia Nigra
NucleusSubthalami
Motor Tract
Atlas
Substantia Nigra
NucleusSubthalami
Motor Tract
Atlas
Cerefy Anat.Brain Atlas
Wieslaw Nowinski, Singapore
Anatomy atlas vs. function atlas (fMRI)
Manual marking of recognizable landmarks in both atlas and high resolution data.
D30L D32L
D28L
D31L
D29L
D30L D32L
D28L
D31L
D29L
TT88s / L+5mm
D30LD30L D32LD32L
D28LD28L
D31LD31L
D29LD29L
D30LD30L D32LD32L
D28LD28L
D31LD31L
D29LD29L
TT88s / L+5mm
Example of slice TT88s / L+5mm
Registration ofreference databy landmarks
Select points on conditions:
Clearly visible in both atlas and reference
data;
Distribution in whole brain volume;
Number of landmarks is unlimited.E. BenninkJ. Korbeeck