Image Registration Using Image Registration Using Mutual InformationMutual Information

Gen-Jia Jaguar LiGen-Jia Jaguar LiAdvisor: Shu-Yen WanAdvisor: Shu-Yen Wan

Problem descriptionProblem description

Flowchart of image Flowchart of image registrationregistration

A problem of optimization

Objective function1. Transformation2. Interpolation3. Similarity measure

(e.g. mutual information)

Optimization1. Degree of freedom2. optimization strategies

Give an example in AvizoGive an example in Avizo

Papers ReviewPapers Review

Mutual-Information-Based Registration of Medical Images: A Survey [Pluim JP, Maintz JB, Viergever MA. IEEE Trans Med Imaging. 2003 Aug;22(8):986-1004]

Nonrigid Image Registration Using Conditional Mutual Information. [Loeckx D, Slagmolen P, Maes F, Vandermeulen D, Suetens P. IEEE Trans Med Imaging. 2009 May 12.]

Mutual-Information-Based Registration of Medical Images: A Survey

Pluim JP, Maintz JB, Viergever MA

IEEE Trans Med Imaging. 2003 Aug;22(8):986-1004

IntroductionIntroduction

A new idea in approximately 1994 (ColligA new idea in approximately 1994 (Collignon and colleagues and Viola and Wells)non and colleagues and Viola and Wells)

OutlineOutline Definition of entropy and its interpretationDefinition of entropy and its interpretation Mutual informationMutual information Survey of literatureSurvey of literature

EntropyEntropy

Measure of information (Measure of information (entropyentropy))

Hartley 1928Hartley 1928

Shannon entropy 1948Shannon entropy 1948

EntropyEntropyInterpretationsInterpretations

Amount of information an event Amount of information an event gives when it takes placegives when it takes place

Uncertainty about the outcome of an Uncertainty about the outcome of an eventevent

Dispersion of the probabilities with Dispersion of the probabilities with which the events take placewhich the events take place

Image RegisrationImage Regisration Woods et al. 1990 (first introduced)Woods et al. 1990 (first introduced)

regions of similar tissue in one image regions of similar tissue in one image would correspond to regions in the would correspond to regions in the other imageother image

Hill et al. 1993 (an adaption of Hill et al. 1993 (an adaption of Woods)Woods)

regions are defined in feature spaceregions are defined in feature space

Feature space (or joint Feature space (or joint histogram)histogram)

Fig. 1. Example of a feature space for (a) a CT image and (b) an MR image.(c) Along the axes of the feature space, the gray values of the two images areplotted: from left to right for CT and from top to bottom for MR. The featurespace is constructed by counting the number of times a combination of grayvalues occurs. For each pair of corresponding points (x; y), with x a point inthe CT image and y a point in the MR image, the entry (I (x); I (y))in the feature space on the right is increased. A distinguishable cluster in thefeature space is the stretched vertical cluster, which is the rather homogeneousarea of brain in the CT image corresponding to a range of gray values for theMR image.

Feature space (or joint Feature space (or joint histogram)histogram)

Fig. 2. Joint gray value histograms of anMRimage with itself. (a) Histogram shows the situation when the images are registered. Because the images are identical, all gray value correspondences lie on the diagonal. (b), (c), and (d) show the resulting histograms when one MR image is rotated with respect to the other by angles of 2, 5, and 10, respectively. The corresponding joint entropy values are (a) 3.82; (b) 6.79; (c) 6.98; and (d) 7.15..

Image RegistrationImage Registrationmeasures of dispersionmeasures of dispersion

Hill et al. 1994 (skewness of distribution)Hill et al. 1994 (skewness of distribution)Third-order moment of the joint histogrThird-order moment of the joint histogramam

Collignon et al and Studholme et al. 1995 (SCollignon et al and Studholme et al. 1995 (Shannon entropy for joint distribution)hannon entropy for joint distribution)Minimizes their joint entropyMinimizes their joint entropy

Mutual InformationMutual Information

Collignon et al 1995-1998 & Viola and Wells Collignon et al 1995-1998 & Viola and Wells 19951995applied to rigid registration of multi-moapplied to rigid registration of multi-modality image and with a few years it becodality image and with a few years it become the most investigated measure for mme the most investigated measure for medical image registrationedical image registration

Mutual InformationMutual InformationDefinitionDefinition

The best explains the term “mutual information”

The most closely related to joint entropy

Related to Kullback-Leibler distance summation of p(i)log(p(i)/q(i))

Mutual InformationMutual InformationPropertiesProperties

I(A,B) = I(B,A)I(A,B) = I(B,A)

I(A,A) = H(A)I(A,A) = H(A)

I(A,B) <= H(A), I(A,B) <= H(B)I(A,B) <= H(A), I(A,B) <= H(B)

I(A,B) >= 0I(A,B) >= 0

I(A,B) = 0 I(A,B) = 0

MethodMethodPreprocessingPreprocessing

Region or structures defineRegion or structures define

Low-pass filtering, e.g. ultrasound imageLow-pass filtering, e.g. ultrasound image

Thresholding or filtering to remove noiseThresholding or filtering to remove noise

ResampleResample

MethodMethodMeasure (Mutual Information)Measure (Mutual Information)

EntropyEntropy Shannon entropyShannon entropy Jumarie entropy or Renyi entropyJumarie entropy or Renyi entropy

NormalizationNormalization NMI, NMI, Studholme et al. 1997, 1999Studholme et al. 1997, 1999 ECC, ECC, Collignon 1998 and Maes 1997Collignon 1998 and Maes 1997

Spatial informationSpatial information MI ignored the neighboring voxelsMI ignored the neighboring voxels

MethodMethodTransformationTransformation

Degree of freedomDegree of freedom Rigid (translation and rotation)Rigid (translation and rotation) Affine (scaling and shear)Affine (scaling and shear) Curved (mapping of straight lines to Curved (mapping of straight lines to

curves)curves)

MethodMethodImplementationImplementation

InterpolationInterpolation

Probability distribution estimationProbability distribution estimation

OptimizationOptimization

AccelerationAcceleration

MethodMethodImage DimensionalityImage Dimensionality

Majority of researches treat registraion of 3D iMajority of researches treat registraion of 3D imagemage

2-D / 2-D, less reliable estimation of the proba2-D / 2-D, less reliable estimation of the probability distributionsbility distributions

2-D / 3-D, find the correspondence between th2-D / 3-D, find the correspondence between the operative scene and a preoperative imagee operative scene and a preoperative image

MethodMethodNumber of ImagesNumber of Images

Fig5. Different definitions of the mutual information (shaded areas) of three images (a)–(c). The dark gray color in (c) signifies that the area is counted twice.The circles denote the entropy of an image; joint entropy is the union of circles.

DiscussionDiscussion

The underlying process is difficult to envisageThe underlying process is difficult to envisage

It can be used without need for It can be used without need for preprocessing, user initialization or preprocessing, user initialization or parameter tuningparameter tuning

May not be a universal cure: thin structures May not be a universal cure: thin structures (e.g. retinal images) or MR and ultrasound (e.g. retinal images) or MR and ultrasound imagesimages

What We KnowWhat We Know

NMI with respect to image overlap is NMI with respect to image overlap is a useful adaptationa useful adaptation

Curved registration based on MI is Curved registration based on MI is viable, but still challenge.viable, but still challenge.

Interpolation method influences both Interpolation method influences both accuracy and smoothness of measureaccuracy and smoothness of measure

In the FutureIn the Future

Curved registration is still challengeCurved registration is still challenge

Ultrasound, the most challengingUltrasound, the most challenging

Gray values of neighboring voxels are unGray values of neighboring voxels are uncorrelatedcorrelated

Nonrigid Image Registration Using Conditional Mutual Information.

Loeckx D, Slagmolen P, Maes F, Vandermeulen D, Suetens P

IEEE Trans Med Imaging. 2009 May 12.[Epub ahead of print]