9
Efficient intensity-hue-saturation-based image fusion with saturation compensation Te-Ming Tu Shun-Chi Su Hsuen-Chyun Shyu Ping Sheng Huang Chung Cheng Institute of Technology Department of Electrical Engineering Taoyuan, Taiwan 33509 Republic of China Abstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread image fusion methods in the remote sensing community. However, both are sensitive to the characteristics of the analyzed area. Restated, they often face color distortion problems with fused images. This work investigates models representing the color space in images. Schemes governing saturation change during fusion are also evaluated to derive the correct saturation values for each pixel in a fused image. In addition, algorithms that compensate distorted saturation values for dis- tinct image fusion methods are derived and discussed. Methods to ex- tend current image fusion methods to any arbitrary order are also out- lined. Finally, an illustrative example of a fused SPOT image demonstrates the color distortion problem while an IKONOS image is adopted to evaluate various contemporary image fusion methods. © 2001 Society of Photo-Optical Instrumentation Engineers. [DOI: 10.1117/1.1355956] Subject terms: image fusion; intensity-hue-saturation (IHS); Brovey transform; IHS with saturation compensation. Paper 200088 received Mar. 6, 2000; revised manuscript received Nov. 16, 2000; accepted for publication Nov. 16, 2000. 1 Introduction With the development of new imaging sensors, a meaning- ful fusion method for all employed imaging sources be- comes necessary. Image fusion is a novel means of com- bining the spectral information of a coarse-resolution image with the spatial resolution of a finer image. The resulting merged image is a product that synergistically combines the best features of each of its components. Previous literature describes various methods for merging the high-resolution panchromatic image with low-resolution multispectral images. 1–6 Owing to their efficiency and implementation, the intensity-hue-saturation ~IHS! image fusion 1–3 and Brovey transform 2,7 are the two most popular fusion meth- ods for remote sensing images. To reach image fusion goals, the IHS method uses three low-resolution multispec- tral images in distinct bands and transforms them into an IHS space. The intensity component of the IHS space is then replaced by the high-resolution panchromatic image and transformed back into the original RGB space with previous H and S components. Meanwhile, the Brovey transform is a simple fusion method that preserves the rela- tive spectral contributions of each pixel, but replaces its overall brightness with the high-resolution panchromatic image. Although these two methods have been employed as standard procedures in many commercial packages, both are sensitive to the characteristics of the analyzed area, i.e., they suffer from spectral degradation or color distortion problems. This investigation presents a relatively detailed study in- dicating that the color distortion problem arises from the change of the saturation during the fusion process. To ef- fectively resolve this shortcoming, the influence of satura- tion on the IHS, Brovey, and other image fusion methods is derived and discussed. Meanwhile, an efficient saturation compensation scheme is presented that obtains a high- resolution fused image without spectral degradation. How to extend order three image fusion methods to any arbitrary order is also discussed. Finally, experimental results for distinct image fusion methods are demonstrated. The satu- ration compensation scheme obviously performs signifi- cantly better than the existing method for the SPOT and IKONOS images. 2 New Look to IHS Image Fusion Several different mathematical representations of the trans- formation can convert RGB tristimulus values into the pa- rameters of human color perception and vice versa. 5 Be- yond their computational speeds, these algorithms differ mainly in the choice of coordinate systems ~cylindrical or spherical coordinates!, the primary color used as the hue reference point, and the method used in calculating the in- tensity component of the transformations. A typical RGB- IHS conversion system is the following F I n 1 n 2 G 5 F 1/3 1/3 1/3 2&/6 2&/6 2 &/6 1/& 21/& 0 G F R G B G ~1! and F R G B G 5 F 1 21/& 1/& 1 21/& 21/& 1 & 0 G F I n 1 n 2 G . ~2! 720 Opt. Eng. 40(5) 720728 (May 2001) 0091-3286/2001/$15.00 © 2001 Society of Photo-Optical Instrumentation Engineers

Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

  • Upload
    hathien

  • View
    218

  • Download
    1

Embed Size (px)

Citation preview

Page 1: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

Efficient intensity-hue-saturation-based imagefusion with saturation compensation

Te-Ming TuShun-Chi SuHsuen-Chyun ShyuPing Sheng HuangChung Cheng Institute of TechnologyDepartment of Electrical EngineeringTaoyuan, Taiwan 33509Republic of China

Abstract. The intensity-hue-saturation (IHS) and Brovey transforms aretwo widespread image fusion methods in the remote sensing community.However, both are sensitive to the characteristics of the analyzed area.Restated, they often face color distortion problems with fused images.This work investigates models representing the color space in images.Schemes governing saturation change during fusion are also evaluatedto derive the correct saturation values for each pixel in a fused image. Inaddition, algorithms that compensate distorted saturation values for dis-tinct image fusion methods are derived and discussed. Methods to ex-tend current image fusion methods to any arbitrary order are also out-lined. Finally, an illustrative example of a fused SPOT imagedemonstrates the color distortion problem while an IKONOS image isadopted to evaluate various contemporary image fusion methods.© 2001 Society of Photo-Optical Instrumentation Engineers. [DOI: 10.1117/1.1355956]

Subject terms: image fusion; intensity-hue-saturation (IHS); Brovey transform;IHS with saturation compensation.

Paper 200088 received Mar. 6, 2000; revised manuscript received Nov. 16, 2000;accepted for publication Nov. 16, 2000.

ing-e-omge

ngtheturetional,

-ionec-anis

ageitheyelaitsticd aboti.e

on

in-heef-

ra-s istion

igh-owaryforatu-ifi-nd

ns-a-

ffer

ein-

B-

1 Introduction

With the development of new imaging sensors, a meanful fusion method for all employed imaging sources bcomes necessary. Image fusion is a novel means of cbining the spectral information of a coarse-resolution imawith the spatial resolution of a finer image. The resultimerged image is a product that synergistically combinesbest features of each of its components. Previous literadescribes various methods for merging the high-resolupanchromatic image with low-resolution multispectrimages.1–6 Owing to their efficiency and implementationthe intensity-hue-saturation~IHS! image fusion1–3 andBrovey transform2,7 are the two most popular fusion methods for remote sensing images. To reach image fusgoals, the IHS method uses three low-resolution multisptral images in distinct bands and transforms them intoIHS space. The intensity component of the IHS spacethen replaced by the high-resolution panchromatic imand transformed back into the original RGB space wprevious H and S components. Meanwhile, the Brovtransform is a simple fusion method that preserves the rtive spectral contributions of each pixel, but replacesoverall brightness with the high-resolution panchromaimage. Although these two methods have been employestandard procedures in many commercial packages,are sensitive to the characteristics of the analyzed area,they suffer from spectral degradation or color distortiproblems.

This investigation presents a relatively detailed studydicating that the color distortion problem arises from tchange of the saturation during the fusion process. Tofectively resolve this shortcoming, the influence of satution on the IHS, Brovey, and other image fusion methodderived and discussed. Meanwhile, an efficient satura

720 Opt. Eng. 40(5) 720–728 (May 2001) 0091-3286/2001/$15.0

-

-

sh.,

compensation scheme is presented that obtains a hresolution fused image without spectral degradation. Hto extend order three image fusion methods to any arbitrorder is also discussed. Finally, experimental resultsdistinct image fusion methods are demonstrated. The sration compensation scheme obviously performs signcantly better than the existing method for the SPOT aIKONOS images.

2 New Look to IHS Image Fusion

Several different mathematical representations of the traformation can convert RGB tristimulus values into the prameters of human color perception and vice versa.5 Be-yond their computational speeds, these algorithms dimainly in the choice of coordinate systems~cylindrical orspherical coordinates!, the primary color used as the hureference point, and the method used in calculating thetensity component of the transformations. A typical RGIHS conversion system is the following

F In1n2

G5F 1/3 1/3 1/3

2&/6 2&/6 2&/6

1/& 21/& 0G F R

GBG ~1!

and

F RGBG5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F I

n1n2

G . ~2!

0 © 2001 Society of Photo-Optical Instrumentation Engineers

Page 2: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

ile

ggee is

the

ns,u-r-

eed

eetheri-

the

e.

heatu-oldgorS

thex-heby

er

sity. In

onsi-ring

n

Tu et al.: Efficient intensity-hue-saturation-based image . . .

Variablesn1 andn2 in Eqs.~1! and ~2! can be consideredas x and y axes in the Cartesian coordinate system, whintensity I indicates thez axis. In this way, the hue~H! andsaturation~S! can be represented by

H5tan21S n2

n1D and S5~n121n22!1/2. ~3!

2.1 IHS Image Fusion With Efficient Computation

Let R0, G0, B0, n10 , andn20 represent the correspondinvalues for the resized original image. In the IHS imafusion method, the intensity component of the IHS spacreplaced by the high-resolution panchromatic image~Pan!and transformed back into the original RGB space withprevious H and S components. That is,

F Rnew

Gnew

Bnew

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F Pan

n10

n20

G . ~4!

Directly implementing the IHS method in Eq.~4! re-quires numerous multiplication and addition operatiomaking it computationally inefficient. To develop a comptationally efficient method without the coordinate transfomation, Eq.~4! can be rewritten as

F Rnew

Gnew

Bnew

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F I1~Pan-I!

n10

n20

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F I1d

n10

n20

G5F R01d

G01dB01d

G ~5!

whered5Pan2I. Equation~5! states that the fused imag@Rnew,Gnew,Bnew#T can be easily obtained from the resizoriginal image@R0,G0,B0#

T, simply by addition. Hence theIHS methods are implemented efficiently.

2.2 Effect of Saturation in IHS Image Fusion

A feature of the IHS method~and also many other imagfusion methods! is that the color of the image may bchanged following fusion. This phenomenon is termedcolor distortion problem herein. Numerous previous expe

ments also encountered this problem, indicating thatsaturation of images fused by using Eq.~4! is higher orlower than the original value in the multispectral imagHowever, simply partitioning Eq.~4! as

F Rnew

Gnew

Bnew

G5FPanPanPan

G1F 21/& 1/&

21/& 21/&

& 0G Fn10

n20G ~6!

reveals that both H and S defined by Eq.~3! are preservedduring fusion. However, since all the R, G, and B of tfused image may be changed during fusion, the new sration value cannot be obtained directly through thevalues ofn1 and n2. Therefore the saturation preservinproperty in Eq.~4! produces a distorted saturation value fthe new~fused! image, a confused imperfection of the IHmethod.

An alternative RGB-IHS conversion system rotatesRGB cube until the horizontal plane is parallel to the Mawell triangle and the vertical axis lies on the gray line of tRGB cube. This conversion system can be represented8

I5~R1G1B!/3 ~7!

H5H cos21~a!, if G>R

2p2cos21~a!, if G<R,

a5~2B2G2R!/2

@~B2G!21~B2R!~G2R!#1/2 ~8!

and

S5123 min~R,G,B!

R1G1B. ~9!

The relation of Eqs.~1!–~3! and~7!–~9! was derived byLedley.9 These two RGB-IHS conversion systems diffmainly in their representations of the saturation. In Eqs.~1!through ~3!, pixels with identical (n121n22)1/2 have thesame saturation value, which is independent of the intenI. These points form a saturation barrel in the IHS spacethe conversion system in Eqs.~7!–~9!, pixels with identical(n121n22)1/2 are located on the surface of a saturaticone with saturation values proportional to distinct intenties. To discover how the hue and saturation change duthe image fusion process,d in Eq. ~5! can be substitutedinto Eqs.~8! and~9! and the notations defined before it cabe adopted. Following image fusion,a in Eq. ~8! can berecalculated as

a85~2~B01d!2~G01d!2~R01d!!/2

$@~B01d!2~G01d!#21@~B01d!2~R01d!#@~G01d!2~R01d!#%1/2

5~2B02G02R0!/2

@~B02G0!21~B02R0!~G02R0!#

1/25a0 . ~10!

721Optical Engineering, Vol. 40 No. 5, May 2001

Page 3: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

gefored

q.

,tionob-

is

bm-

ed-

tion.

-nt

ns-eyvesre-n-

Tu et al.: Efficient intensity-hue-saturation-based image . . .

Equation~10! indicates that the hue H for the fused imausing Eqs.~7!–~9! is unchanged, i.e., the same as thatthe original multispectral color image, when I is replacby Pan. Therefore the hue H in images fused by Eq.~4! isalways correct. In contrast, by Eq.~9!

S05123X0

R01G01B05

I2X0

I, ~11!

where X0 denotes the minimal among R0, G0, and B0.When substitutingd and corresponding notations into E~11!, the new value of the saturation becomes

S15123 min~R01d,G01d,B01d!

R01G01B013d

5123~X01d!

R01G01B013d5

I2X0

Pan. ~12!

Obviously, the saturation changes when PanÞI. The differ-ence between these two values is

DS5S82S05S I2X0

Pan D2S I2X0

I D52~ I2X0!d

Pan3I.

~13!

SinceX0 is the smallest value among R0, G0, and B0,I-X0 in Eq. ~13! must be positive. HenceDS is negative ifd is positive and vice versa. Consequently,d is crucial inthe color distortion problem. Since Pan rarely equals Iddoes not generally equal zero. At this point, the saturaof the fused image changes and the color distortion prlem appears.

2.3 An Efficient Saturation Compensate Method

In the RGB-IHS conversion system presented in Eqs.~1!–~3!, multiplying bothn1 andn2 by factork will scale thesaturation byk while the hue remains unchanged. That

H5tan21S n2

n1D5tan21S k•n2

k•n1D5H, ~14!

S5~ n121 n22!1/25@~k•n1!21~k•n2!2#1/2

5k•~n121n22!1/25k•S. ~15!

The idea demonstrated in Eqs.~14! and~15! can be used tocompensate for the distorted saturation value producedEqs.~4! or ~5!. This saturation compensation can be acco

722 Optical Engineering, Vol. 40 No. 5, May 2001

y

plished simply by shifting S0 to S8 or multiplying a factor(11DS/S0) into Eq. ~6!. That is

F Rs

Gs

Bs

G5FPanPanPan

G1S 11DS

S0D F 21/& 1/&

21/& 21/&

& 0G Fn10

n20G , ~16!

where@Rs,Gs,Bs#T is the resulting saturation-compensat

fused image. In Eq.~16!, although the hue remains unchanged, the saturation S0 is compensated to S8. This pro-cess obtains an image fusion with compensated saturaMoreover, since 11DS/S05I/Pan, Eq.~16! can be rewrit-ten as follows,

F Rs

Gs

Bs

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F Pan

g8•n10

g8•n20

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F g8•I1~Pan2g8•I!

g8•n10

g8•n20

G5g8•F R

GBG1F g9

g9g9

G , ~17!

whereg85I/Pan511DS/S0 and g95Pan2g8•I, respec-tively.

Equation~17! requires only one scalar matrix multiplication and one matrix addition. Hence it is an efficiesaturation-compensated image fusion process.

3 Brovey Transform and the Other Color-SpaceImage Fusion

3.1 Effect of Saturation in Brovey Transform

This section presents a brief overview of the Brovey traform and its link with saturation compensation. The Brovtransform is a simple image fusion method that preserthe relative spectral contributions of each pixel, butplaces its overall brightness with the high-resolution pachromatic image. Brovey image fusion is defined as2,7

F Rnew

Gnew

Bnew

G5g•F R0

G0

B0

G5Pan

I F R0

G0

B0

G , ~18!

Page 4: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

n.

hethe

esS.

can

HSn

od

ion

ri-Som-

foreolor

vi-d

Tu et al.: Efficient intensity-hue-saturation-based image . . .

whereg5Pan/I51/g8. To evaluate the effect of saturatiofor this transform, Eqs.~1!–~3! can be substituted into Eq~18! to obtain an IHS-like form as follows:

F Rnew

Gnew

Bnew

G5g•F 1 21/& 1/&

1 21/& 21/&

1 & 0G F I

n10

n20

G5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F Pan

g•n10

g•n20

G . ~19!

Equation~19! demonstrates that although the hue for tfused image using Brovey transform is unchanged,saturation is changed to

S95g•~n1021n20

2!1/25g•S0. ~20!

However, substituting Eq.~18! into ~8! and ~9!, respec-tively, reveals the interesting fact that both the new valuof the hue and saturation equal the original values in IHThat is

a95~2Bnew2Gnew2Rnew!/2

@~Bnew2Gnew!21~Bnew2Rnew!~Gnew2Rnew!#1/2

5g•~2B02G02R0!/2

g•@~B02G0!21~B02R0!1~G02R0!#

1/2

5~2B02G02R0!/2

@~B02G0!21~B02R0!~G02R0!#

1/25a0 ~21!

S95123 min~Rnew,Gnew,Bnew!

Rnew1Gnew1Bnew

5123 min~g•R0,g•G0,g•B0!

g•~R01G01B0!

5123 min~R0,G0,B0!

R01G01B05S0. ~22!

Therefore, the saturation compensation in this systembe directly shifted from S9 to S8 by multiplying a factor(g82g) into Eq. ~18!. That is

F Rs

Gs

Bs

G5F Rnew

Gnew

Bnew

G1~g82g!F 21/& 1/&

21/& 21/&

& 0G Fn10

n20G

5g•F 1 21/& 1/&

1 21/& 21/&

1 & 0G F I

n10

n20

G1~g82g!F 21/& 1/&

21/& 21/&

& 0G Fn10

n20G

5FPanPanPan

G1g8•F 21/& 1/&

21/& 21/&

& 0G Fn10

n20G

5F 1 21/& 1/&

1 21/& 21/&

1 & 0G F Pan

g8•n10

g8•n20

G@ identical to Eq. ~17!#. ~23!

In summary, the saturation relationship among the I@Eqs. ~4! or ~5!#, Brovey transform, and the compensatiomethod developed herein@Eq. ~17!# are

S05g•S8 ~24!

for the original IHS and saturation compensation methand

S95g2•S8 ~25!

for the Brovery transform and saturation compensatmethod.

Consequently, the color distortion problem is most seous in the Brovey transform and lightly better in the IHmethod. The saturation compensated method can cpletely solve the color distortion problem.

3.2 Saturation Compensation in the Other Color-Space Image Fusion

The saturation compensation formulation described becan be extended to other color spaces, such as a YIQ ctelevision system10 and a pseudo Karhunen-Loeve~KL !color coordinate system.11

3.2.1 YIQ color coordinate system

YIQ is the color coordinate system of the National Telesion System Committee~NTSC! television standard defineas10

723Optical Engineering, Vol. 40 No. 5, May 2001

Page 5: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

the

e

o

u-is

atethe

ana-s-

-agea

as

m-

IHSare

beberrm

ar-m-nta-

oteim-od

ionnd. Iftherma-

Tu et al.: Efficient intensity-hue-saturation-based image . . .

FYIQG5F 0.299 0.587 0.114

0.596 20.274 20.322

0.211 20.523 0.312G F R

GBG ~26!

H5tan21S Q

I D and S5~ I21Q2!1/2. ~27!

The hue and saturation here are defined differently toIHS system in Eqs.~1! and~2!. The inverse transformationfrom YIQ to RGB can be calculated by multiplying thYIQ values by the inverse of the above matrix, i.e.,

F RGBG5F 1 0.956 0.621

1 20.273 20.647

1 21.104 1.701G FY

IQG . ~28!

Architecturally, Eq.~28! also can be partitioned into twdistinct spaces as

F RGBG5FY

YYG1F 0.956 0.621

20.273 20.647

21.104 1.701G F I

QG . ~29!

Similarly, a YIQ image fusion with compensated satration can be constructed by replacing Y by Pan. That

F Rs

Gs

Bs

G5F 1 0.956 0.621

1 20.273 20.647

1 21.104 1.701G F Pan

g8•Ig8•Q

G5g8•F R

GBG1F g9

g9g9

G . ~30!

The compensation factors used here areg85Y/Pan andg95Pan2g8•Y, respectively.

3.3 Pseudo Karhunen-Loeve Color CoordinateSystem

Generally, the color components in the RGB coordinsystem are strongly correlated with each other. Ifsecond-order statistics of the image are known, derivingorthogonal color coordinate system by the KL transformtion is possible. However, considering the difficulty of e

724 Optical Engineering, Vol. 40 No. 5, May 2001

timating the color covariance for each individual, a compromise is to estimate the covariance of a general imand find the KL transformation for that image. Suchpseudo KL transform, given in Ref. 11, is illustratedfollows:

FPC1PC2PC3

G5F 1/3 1/3 1/3

1/2 0 21/2

21/4 1/2 21/4G F R

GBG . ~31!

Its inverse transformation is then given by

F RGBG5F 1 1 22/3

1 0 4/3

1 21 22/3G FPC1

PC2PC3

G5FPC1

PC1PC1

G1F 1 22/3

0 4/3

21 22/3G FPC2

PC3G . ~32!

Proceeding as before, a pseudo KL image fusion with copensated saturation can be derived as

F Rs

Gs

Bs

G5F 1 1 22/3

1 0 4/3

1 21 22/3G F Pan

g8•PC2g8•PC3

G5g8•F RGBG1F g9

g9g9

G ,

~33!

where the compensation factors areg85PC1/Pan andg95Pan2g8•PC1, respectively.

3.4 Order Consideration

Most contemporary image fusion techniques, such asand YIQ, focus on three-band multispectral images, andconsidered order three because they employ a 333 matrixas their transform kernel. The pseudo KL transform canapplied to remote sensing images with an arbitrary numof bands, so its order is arbitrary. The Brovey transfodoes not require a transform kernel and can handle anbitrary number of bands, so its order is also arbitrary. Iage fusion techniques with an arbitrary order are advageous because they can easily handle higher order remsensing images like the hyperspectral or multispectralages with more than three bands. A merit of the methproposed herein lies not only on its color compensatcapability for fused images but also its ability to extetraditional three order transformations to arbitrary orderwe consider the fusion of remote sensing images only,proposed method does not need to calculate the transfotion kernel. Some examples are listed as follows.For arbitrary order IHS:

Page 6: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

tra

ein

ofag--e ted-mtedestd

e-

S,

thattails.sed

inalis-

ef-the

ree-n-

inuson

o-erycor-on,the

gedalld 2.S,ofbut

rst

ralThethe

keim-his

m-

es

Ctiona-age

-mo-3

Tu et al.: Efficient intensity-hue-saturation-based image . . .

Fi ,IHS5Mi1b1 , ~34!

whereMi denotes the resized hyperspectral or multispecimage of bandi, b15Pan2k, k5((1/l )( i 51

l M i), and lrepresents the order or the number of bands.For Brover transform:

Fi ,Brover5a1•Mi ~35!

a15Pan

k. ~36!

And for saturation compensation method developed her

Fi ,SC5a2•Mi1b2 ~37!

a25k/Pan andb25Pan2a2•k. ~38!

These parametersa1 , a2 , b1 , andb2 , can be viewed asspatially variable gain factors.

4 Experimental Results

4.1 Experiment 1

The most widely applied fusion procedure is the merginga panchromatic SPOT image with three-color SPOT imery or multispectral LANDSAT™ imagery. The first experiment presents an example of a fused SPOT imaghighlight the color distortion problem. The test images usherein include 10-m resolution panchromatic and 20color images of a rural area of Taichung, Taiwan, collecon 4 March 1994 by SPOT satellite. The size of the timage is 5123512 pixels for the panchromatic image anis displayed in Fig. 1. Figure 2~a! presents the resized threcolor imagery of the dotted area in Fig. 1 while Figs. 2~b!–

Fig. 1 The panchromatic SPOT image.

l

,

o

2~d! illustrate the fusion results by Brovey transform, IHand our IHS with saturation compensation~IHS-SC!method, respectively. Comparing spatial effect revealsthe results of the three methods share the same deRiver, houses, streets, and so on are evident in the fuimages using all three methods. Compared to the origcolor image, the color of the fused image is clearly dtorted for the two former methods~with reds darkening!,and only the IHS-SC image provides the correct colorfect. The red areas in the IHS-SC image have almostsame color and brightness as those in the original thcolor SPOT imagery. Meanwhile, the color of the residetial areas is also almost identical in Figs. 2~b! and 2~d!.

At present, no quantitative methods have been foundcurrent literature to evaluate the performance of varioimage fusion techniques. Most of the evaluation is basedvisual examination. In this work we use the correlation cefficients between the original resized three-color imagand the merged three-color image as an index for therectness of spectral quality. For spatial quality evaluatiwe have calculated the correlation coefficients betweenpanchromatic image and the intensity image of the merimage as another index. Correlation coefficients forthree image fusion methods are depicted in Tables 1 anThese metrics verify our previous discussion that the IHBrovey transform, and IHS-SC can preserve almost allthe spatial information that Pan has in the fused image,must face a color distortion problem, which is the woresulted by the Brovey transform.

4.2 Experiment 2

For local environmental applications, both the spectresolution and good spatial resolution are necessary.second experiment conducted in this section is based onnew IKONOS image and designed specifically to mavarious comparisons among several variations of IHSage fusion. Eight image fusion methods compared in texperiment include Brovey transform~BT!, principal com-ponents analysis~PCA! image fusion, IHS, YIQ, pseudoKL ~PKL!, IHS with saturation compensation~IHS-SC!,YIQ with saturation compensation~YIQ-SC! and pseudoKL with saturation compensation~PKL-SC!. Reasons forselecting these methods for comparison are as follows:

1. comparing IHS-SC against BT, PCA, and IHS deonstrates the color distortion problem,

2. comparing IHS against YIQ and PKL demonstratthe equality among these three methods, and

3. comparing IHS-SC against YIQ-SC and PKL-Sdemonstrates that the saturation compensamethod developed herein, when giving distinct prameters, can be applied to any contemporary imfusion methods.

The test images, a 1-m resolution panchromatic and 4color image of Beijing, China, were collected on 22 Octber 1999 by Space Imaging’s IKONOS satellite. Figuredisplays the test panchromatic image while Fig. 4~a! pre-sents its resized multispectral image of the dotted area~us-ing R, G, B bands only!. Meanwhile, Figs. 4~b!–4~i!

725Optical Engineering, Vol. 40 No. 5, May 2001

Page 7: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

Tu et al.: Efficient intensity-hue-saturation-based image . . .

726 Optical Engi

Fig. 2 Color composites of: (a) the resized SPOT XS image; (b) the fused image using the Broveytransform, (c) the fused image using the IHS method, and (d) the fused image using the IHS-SCmethod.

Table 1 The correlation coefficients between three-color (R, G, andB) resized SPOT images and images fused by distinct methods andtheir average values.

R G B Average

IHS 0.7692 0.8824 0.8657 0.8391

BT 0.5095 0.8751 0.8781 0.7542

IHS-SC 0.9587 0.9651 0.9654 0.9631

Table 2 The correlation coefficients between the panchromatic im-age and the intensity image of the merged image.

IHS BT IHS-SC

0.9868 0.9875 0.9852

neering, Vol. 40 No. 5, May 2001

Fig. 3 The panchromatic IKONOS image.

Page 8: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

Tu et al.: Efficient intensity-hue-saturation-based image . . .

Fig. 4 Color composites of: (a) the resized multispectral IKONOS image, and of the fused image using(b) BT, (c) PCA, (d) IHS, (e) YIQ, (f) PKL, (g) IHS-SC, (h) YIQ-SC, and (i) PKL-SC.

yen-e

or-ex-

ther

illustrate the fusion results by image fusion [email protected]~a!–4~h!# listed previously. The color distortion mainlinvolves green and red. Also, only the saturation compsation method~s! can provide the correct color effects in thfused image~s!.

5 Conclusion

This work presents a novel image fusion process that incporates saturation compensation. The mathematical andperimental results for the test images agree closely. Fur

727Optical Engineering, Vol. 40 No. 5, May 2001

Page 9: Efficient intensity-hue-saturation-based image fusion …cnspci/references/dip/fusion/tu2001.pdfAbstract. The intensity-hue-saturation (IHS) and Brovey transforms are two widespread

ilarex-re

ks.lemion

ralnkta.n-is

y-and

n oftral

-

on

an,m-

ote.

lu-

lor

Tu et al.: Efficient intensity-hue-saturation-based image . . .

experiments using SPOT and IKONOS data reach simconclusions. Consequently, the effect of saturation istremely sensitive to the characteristics of the analyzed aand generally presents difficulties in image fusion worThe method proposed herein not only solves this probbut also gives the ability to extend present image fusmethods to arbitrary order.

Acknowledgments

The authors would like to thank Dr. P. F. Hsieh of CentUniversity for providing the SPOT data. We also thaSpace Imaging company for providing the IKONOS daFinally, we would like to thank the National Science Coucil of the Republic of China for financially supporting thwork under Contract No. NSC 89-2213-E-014-013.

References

1. W. J. Carper, T. M. Lillesand, and R. W. Kiefer, ‘‘The use of intesithue-saturation transformations for merging SPOT panchromaticmultispectral image data,’’Photogramm. Eng. Remote Sens.56~4!,459–467~1990!.

2. P. S. Chavez, Jr., S. C. Sides, and J. A. Anderson, ‘‘Comparisothree difference methods to merge multiresolution and multispecdata: Landsat TM and SPOT panchromatic,’’Photogramm. Eng. Remote Sens.57~3!, 295–303~1991!.

3. E. Kathleen and A. D. Philip, ‘‘The use of intesity-hue-saturatitransformation for producing color shaded relief images,’’Photo-gramm. Eng. Remote Sens.60~11!, 1369–1374~1994!.

4. N. Jorge, O. Xavier, F. Octavi, P. Albert, P. Vicenc, and A. Rom‘‘Multiresolution-based imaged fusion with additive wavelet decoposition,’’ IEEE Trans. Geosci. Remote Sens.37~3!, 1204–1211~1999!.

5. C. Pohl and J. L. Van Genderen, ‘‘Multisensor image fusion in remsensing: concepts, methods and applications,’’Int. J. Remote Sens19~5!, 823–854~1998!.

6. T. Ranchin and L. Wald, ‘‘Fusion of high spatial and spectral resotion images: the ARSIS concept and its implementation,’’Photo-gramm. Eng. Remote Sens.66~1!, 49–61~2000!.

7. ER Mapper 5.0 Reference, Earth Resource Mapping Pty Ltd.~1995!.8. R. C. Gonzalez and R. E. Woods,Digital Image Processing, Addison

Wesley Publishing Company, Reading, MA~1992!.9. R. S. Ledley, M. Buas, and T. J. Golab, ‘‘Fundamentals of true-co

image processing,’’Proc. Intl. Conf. Pattern Recognition1, 791–795~1990!.

10. W. K. Pratt,Digital Image Processing, 2nd ed. Wiley, New York~1991!.

11. R. M. Haralick and L. G. Shapiro,Computer and Robot Vision,Addison-Wesley, Reading, MA~1992!.

728 Optical Engineering, Vol. 40 No. 5, May 2001

a

Te-Ming Tu received the BSEE degreefrom Chung Cheng Institute of Technologyin 1986, the MSEE degree from NationalSun Yat-Sen University in 1991, and thePhD degree in electrical engineering fromthe National Cheng Kung University, in1996. Since 1981, he has served in theR.O.C. Army. He was a teaching assistantfrom 1986 to 1989, an instructor from 1991to 1993, and is currently an associate pro-fessor in the Department of Electrical Engi-

neering, Chung Cheng Institute of Technology. His current researchinterests include multispectral/hyperspectral remote sensing, medi-cal imaging, independent component analysis, and statistical pat-tern recognition.

Shun-Chi Su obtained his BS and MS inelectrical engineering from Chung ChengInstitute of Technology in 1993 and 1998,respectively. He is currently pursuing hisPhD degree at the Chung Cheng Instituteof Technology. His research interests in-clude multispectral/hyperspectral remotesensing, parallel processing, and statisticalpattern recognition.

Hsuen-Chyun Shyu received his BS, MS,and PhD degrees from Chung Cheng Insti-tute of Technology, Ta-Hsi, Tao-Yuan, Na-tional Taiwan University, Taipei, Taiwan,and University of Southern California, LosAngeles, in 1974, 1978, and 1986, respec-tively, all in electrical engineering. Cur-rently he is an associate professor in theDepartment of Electrical Engineering,Chung Cheng Institute of Technology. Hisresearch interests include digital signal

processing, radar system, wavelet transforms, and neural networks.

Ping Sheng Huang received his BSEE de-gree from Chung Cheng Institute of Tech-nology in 1985, the MS degree in computerscience from University of Southern Cali-fornia in 1990, and the PhD degree in elec-tronics and computer science from Univer-sity of Southampton, United Kingdom, in1999. Currently he is an associate profes-sor in the Department of Electrical Engi-neering, Chung Cheng Institute of Technol-ogy. His research interests include digital

image processing, biometric identification, and statistical patternrecognition.