Phase-Adaptive Image Signal Fusion using Complex-valued Wavelets

Alexander Wong, David Clausi, and Paul FieguthSystems Design Engineering, University of Waterloo, Canada

{a28wong,dclausi,pfieguth}@engmail.uwaterloo.ca

Abstract

This paper presents a novel method for medical im-age signal fusion using complex-valued wavelets to en-hance the information content in the fused signal froma perceptual manner. The proposed method introducesan adaptively weighted aggregation of signal charac-teristics based on the phase characteristics of medicalimage signals. The proposed method exploits the phasecharacteristics of the image signals to adaptively ac-centuate important anatomical and functional charac-teristics captured by each image signal during the sig-nal fusion process. Experimental results show that theproposed method can improve the visualization of im-portant anatomical and functional characteristics fromdifferent medical image signals in the fused image sig-nal when compared with non-adaptive image signal fu-sion methods.

1 Introduction

Signal fusion is the process of combining multiplesignals from various sources into a single compositesignal. Recent research activity in multimodal imagesignal fusion has been focused on medical image sig-nal fusion. Image signals acquired using devices suchas x-rays, CT (Computed Tomography), MR (MagneticResonance), and PET (Positron Emission Tomography)provide unique information about the anatomical andfunctional characteristics of the human body that areimportant for disease diagnosis. For example, CT pro-vides detailed bone structure information, MR providesdetailed tissue information, and PET provides detailedinformation about bodily functions such as metabolism.However, medical image signals acquired using a sin-gle imaging modality may be inadequate for the diag-nosis of disease. Given the complementary nature ofthese signals, a more complete visualization of a pa-tient’s health can be obtained by integrating importantinformation content from each of the imaging modali-

ties into a single fused image signal.

Various techniques have been proposed for the pur-pose of image signal fusion. Conventional image sig-nal fusion methods include color fusion [1], and im-age signal fusion using mathematical and morpholog-ical operators (i.e., add/average, subtract, and, or, max,min) [2]. Some recent work have focused on a sta-tistical approach to the problem of image signal fu-sion [3, 4]. While considerably more complex, this ap-proach can provide improved visualizations when com-pared to color fusion and simple operator-based meth-ods since the contribution of information from the indi-vidual image signals are weighted based on their statis-tical significance. Other recent work have focused onwavelet-based fusion methods [5, 6], where the waveletcoefficients are considered during the fusion process.These methods utilize the statistical properties of thewavelet coefficients to provide improved visualizationsof details in the images.

Much of the existing literature in medical image sig-nal fusion have focused on a statistical perspective to theproblem, with the enhancement of information contentin the fused image signal from a perceptual perspec-tive largely unexplored. This is very important sincethe goal of medical image signal fusion is to aid clini-cians in the diagnosis of disease. By accentuating im-portant anatomical and functional characteristics fromthe individual medical image signals, the fused imagesignal should provide better insight on the condition ofa patient. In this paper, we propose a novel approach tomedical image signal fusion that adapts the constructionof the fused image signal based on anatomical and func-tional characteristics captured by a medical image sig-nal using phase characteristics obtained from complex-valued wavelets.

This paper is organized as follows. The proposedmedical image signal fusion method is described in Sec-tion 2. Experimental results are presented and discussedin Section 3. Finally, conclusions are drawn in Sec-tion 4.

978-1-4244-2175-6/08/$25.00 ©2008 IEEE

2 Novel Fusion Method

Suppose that we are given n 2-D signalsf1(x, y), f2(x, y), ..., fn(x, y) that have been ac-quired using different imaging modalities and alignedtogether using the multi-modal registration techniqueproposed in [7]. The goal of multimodal image signalfusion is to integrate the information content of theindividual image signals into a single fused imagesignal. This can be expressed as:

g(x, y) = H(f1(x, y), f2(x, y), ..., fn(x, y)) (1)

where g is the fused image signal andH is the signal fu-sion process. A simple but effective method for imagesignal fusion is to perform a simple normalized aggre-gation of the image signals. This can be expressed as:

g(x, y) =1n

n∑i=1

fi(x, y) (2)

The main problem with this simple normalized ag-gregation of image signals is that all information con-tent within the image signals are treated the same.Therefore, important anatomical and functional char-acteristics that are prominent within a particular imag-ing modality are treated no differently than unimportantcharacteristics within other imaging modalities. Sinceclinicians utilize important anatomical and functionalcharacteristics for the purpose of disease diagnosis, itis intuitive that such characteristics are accentuated andemphasized within the individual image signals. Toachieve this goal, a normalized weighted aggregationapproach to image signal fusion can be used to accentu-ate such characteristics during the image signal fusionprocess. This can be expressed as:

g(x, y) =

n∑i=1

wi(x, y)fi(x, y)

n∑i=1

wi(x, y)(3)

where wi(x, y) is the weight assigned to informationcontent at (x, y) in signal i. Since the underlying goalof the proposed method is to enhance the visualizationof information content from a perceptual perspective, itis intuitive that the assigned weights are associated withthe importance of anatomical and functional character-istics within a signal based on human perception.

To adaptively adjust weights based on the im-portance of anatomical and functional characteristicswithin an image signal, a weighting function that re-flects the perceptual sensitivity of the human vision sys-tem to such characteristics is necessary. An important

characteristic in context of the human vision system isphase coherence [8, 9, 10]. This is based on the theorythat perceptually important signal characteristics suchas important anatomical and functional characteristicsoccur at points in a signal where there is maximal phaseorder, and has subsequently been reinforced based onpsychophysical evidence [8]. A major advantage ofthis approach is that, since only phase information isutilized, it is insensitive to the underlying amplitudinalcharacteristics of the image signal. This is particularlyimportant in medical image signals, where amplitudinalvariations due to intrinsic and extrinsic imaging condi-tions can occur and post-processing techniques such ascontrast enhancement are often performed. Given theseadvantages, the proposed method utilizes phase co-herence characteristics obtained using complex-valuedwavelets to adapt weights during the signal aggregationprocess.

The proposed method uses the robust iterative ap-proach to complex wavelet phase coherence estima-tion [11] to extract the maximum moment of phasecoherence γ(x, y). An iterative phase-adaptive bilat-eral estimation scheme is used to improve the phasecoherence estimate. The value of γ increases as per-ceptual importance for signal characteristics at a pointincreases. As such, high phase coherence coincidewith important anatomical and functional characteris-tics within an image signal.

The weight assigned to information at point (x, y)from image signal fi can then be determined based ona scaled exponential of the maximum moment of phasecoherence:

w(x, y) = αeγ(x,y)β (4)

where α is the scaling coefficient, and β is a decay co-efficient. Based on this formulation, higher weights aregiven to perceptually important signal characteristicssuch as important anatomical and functional character-istics within a medical image signal to accentuate suchfeatures in the fused image signal. Finally, the fused im-age signal can be computed using the following phase-adaptive weighted signal aggregation formulation:

g(x, y) =

n∑i=1

αeγ(x,y)β fi(x, y)

n∑i=1

αeγ(x,y)β

(5)

3 Experimental Results

The proposed fusion method was tested using threesets of medical image signals to evaluate its effective-ness at improving visualization of medical image sig-nals. Each set consists of two image signals acquired

using different imaging modalities. A summary of thetest sets is given as follows:

1. TEST 1: T1-weighted MR and CT, Axial CranialSlice, from Visible Human project.

2. TEST 2: T1-weighted MR and PET, Axial CranialSlice, from Whole Brain Atlas [12].

3. TEST 3: T1-weighted MR and T2-weighted MR,Coronal Torso Slice, from Visible Human project.

The non-adaptive signal aggregation approach [2]is also performed for comparison purposes. Objectivemeasures such as PSNR and MI are not suitable forevaluating multimodal image signal fusion since theydo not reflect the human perception system. Therefore,a qualitative approach to evaluating image signal fusionshould be used. The fused image signals produced us-ing the proposed method are shown in Figures 1, 2, and3. It can be seen that the important anatomical andfunctional characteristics from each of the image sig-nals are noticeably accentuated in the proposed methodcompared to the non-adaptive signal aggregation ap-proach [2], hence providing a clearer visualization ofa patient’s condition. For example, the bone structurecharacteristics captured by the CT signal in TEST 1is noticeably clearer and prominent in the fused signalproduced by the proposed method than that producedusing non-adaptive signal aggregation. These experi-mental results demonstrate the effectiveness of the pro-posed method in improving visualization of medical im-age signals.

4 Conclusions

In this paper, we introduced a novel perceptual basedapproach to image signal fusion using complex-valuedwavelets. By adapting the weighted aggregation pro-cess of the image signal fusion problem based on lo-cal phase characteristics, the fused image signal canbe tuned to accentuate important anatomical and func-tional characteristics captured in the individual imagesignals. Experimental results show the effectiveness ofthe proposed method in improving the visualization ofinformation content in images. Future work involvesvalidating our results with clinicians and investigatingalternative signal aggregation processes.

Acknowledgments

This research has been sponsored by the Natural Sci-ences and Engineering Research Council of Canada.

The authors would also like to thank the National Li-brary of Medicine for the Visual Human Project dataand Dr. Keith Johnson from Massachusetts GeneralHospital.

References

[1] Z. Xue and R. Blum, “Concealed Weapon Detec-tion using Color Image Fusion,” Proc. Interna-tional Conference on Information Fusion, vol. 1,pp. 622-627, 2003.

[2] Able Software Corporation. 3-D Doctor, Internet:http://www.ablesw.com/3d-doctor/index.html.

[3] R. Blum, “Robust image fusion using a statisticalsignal processing approach,” Information Fusion,vol. 6, no. 2, pp. 119-128, 2005.

[4] G. Liu, Z. Jing, S. Sum, “Image fusion based onan expectation maximization algorithm,” OpticalEngineering, vol. 44, no. 7, 2005.

[5] Q. Guihong, Z. Dali, and Y. Pingfan, “Medicalimage fusion by wavelet transform modulus max-ima,” Opt. Express, vol. 9, pp. 184-190, 2001.

[6] S. Garg, K. Kiran, R. Mohan, and U. Tiwary,“Multilevel Medical Image Fusion using Seg-mented Image by Level Set Evolution with RegionCompetition,” IEEE EMBS, pp. 7680-7683, 2001.

[7] A. Wong and W. Bishop, “Efficient Least SquaresFusion of MRI and CT Images Using a Phase Con-gruency Model,” Pattern Recognition Letters, vol.29, no. 3, pp. 173-180, 2008.

[8] M. Morrone and D. Burr, “Feature detection in hu-man vision: A phase-dependent energy model,”Proc. the Royal Society of London B, vol. 235, pp.221-245, 1988.

[9] P. Kovesi, “Phase congruency detects corners andedges,” Proc. The Australian Pattern RecognitionSociety Conference, pp. 309-318, 2003.

[10] Z. Wang and E. Simoncelli, “Local Phase Co-herence and the Perception of Blur,” Advances inNeural Information Processing Systems, vol. 16,2004.

[11] A. Wong, “An iterative approach to improved localphase coherence estimation,” Proc. CRV, 2008.

[12] Johnson, K., Becker, J. TheWhole Brain Atlas. Internet:http://www.med.harvard.edu/AANLIB/home.html.

Figure 1. TEST 1: a) T1 MRI, b) CT, c) non-adaptive aggregation [2], d) proposed method

Figure 2. TEST 2: a) T1 MRI, b) PET, c) non-adaptive aggregation [2], d) proposed method

Figure 3. TEST 3: a) T1 MRI, b) T2 MRI, c) non-adaptive aggregation [2], d) proposed method