Comparative Study of Semi-Implicit Schmes

IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 5, MAY 2007 1303

Comparative Study of Semi-Implicit Schemes forNonlinear Diffusion in Hyperspectral Imagery

Julio M. Duarte-Carvajalino, Student Member, IEEE, Paul E. Castillo, and Miguel Velez-Reyes, Senior Member, IEEE

AbstractNonlinear diffusion has been successfully employedover the past two decades to enhance images by reducing undesir-able intensity variability within the objects in the image, while en-hancing the contrast of the boundaries (edges) in scalar and, morerecently, in vector-valued images, such as color, multispectral, andhyperspectral imagery. In this paper, we show that nonlinear dif-fusion can improve the classification accuracy of hyperspectral im-agery by reducing the spatial and spectral variability of the image,while preserving the boundaries of the objects. We also show thatsemi-implicit schemes can speedup significantly the evolution ofthe nonlinear diffusion equation with respect to traditional explicitschemes.

Index TermsHyperspectral imaging, nonlinear diffusion, par-tial differential equations (PDEs), preconditioning, remote sensing,scale space, semi-implicit schemes, vector image processing.

I. INTRODUCTION

REMOTE-SENSING imaging can provide synoptic,repetitive, consistent, and comprehensive environmentalmonitoring of the earth ecosystem, revealing patterns andrelationships unavailable when using traditional data-gatheringtechniques. Hyperspectral imaging technology has the potentialof extracting more plentiful and accurate environmental infor-mation, given the enhanced discrimination capabilities of highspectral resolution imagery. However, the natural variabilityof the material spectra, noise, and degradation added by thetransmission media and sensor system reduce the separabilityof the different structures in hyperspectral imagery, reducingthe accuracy of segmentation and classification algorithms.

For two decades, techniques based on partial differentialequations (PDEs) have been used in image processing forimage segmentation, deblurring, restoration, smoothing, andmultiscale image representation. Among these techniques, par-abolic PDEs have found a lot of attention for image smoothingand image restoration purposes.

This paper introduces image smoothing of hyperspectral im-ages, using a regularized nonlinear diffusion PDE. We show that

Manuscript received December 20, 2005; revised January 24, 2007. Thiswork was supported in part by CenSSIS, the Center for Subsurface Sensingand Imaging Systems, under the Engineering Research Center Program ofthe National Science Foundation (NSF) Award Number EEC-9986821 andthe NSF-EPSCOR fellowship, from the PR program. The associate editorcoordinating the review of this manuscript and approving it for publication wasDr. Jacques Blanc-Talon.

J. M. Duarte-Carvajalino and M. Velez-Reyes are with the Laboratory of Ap-plied Remote Sensing and Image Processing (LARSIP), University of PuertoRico, Mayagez, PR 00681-9048 USA (e-mail: [email protected]; [email protected]).

P. E. Castillo is with the Department of Mathematics, University of PuertoRico, Mayagez, PR 00681-9018 USA (e-mail: [email protected]).

Digital Object Identifier 10.1109/TIP.2007.894266

semi-implicit discretization schemes have better performance(in terms of accuracy and CPU time) than traditional explicitschemes to solve the nonlinear diffusion PDE on hyperspectralimagery. We also show that nonlinear diffusion can be usedto reduce the spatial and spectral variability in hyperspectralimagery, improving classification accuracy (Sections IV andV). We extend approximated semi-implicit schemes such as:additive operator splitting (AOS) and alternating directionimplicit (ADI) schemes to vector-valued images. Additionally,we also evaluate the use of the preconditioned conjugatedgradient (PCG) linear solvers as an alternative to AOS and ADIschemes.

The performance of the vector-valued nonlinear diffusionPDE is studied using four hyperspectral images. The firstis a synthetic image that allows a controlled environment tomeasure CPU speed and accuracy in the solution of the PDE.The second and third images are real hyperspectral imagesacquired using the NASA AVIRIS hyperspectral sensor1 on theNorthwest Indian Pines test site (1992) and the Cuprite miningdistrict (1997) in Nevada. The last hyperspectral image is anindoor image taken with the SOC700 Hyperspectral Imager bythe Surface Optics Corporation.2 The real hyperspectral imagesare used here for the evaluation of the effect of nonlineardiffusion on image classification.

To our knowledge, this is the first extension of semi-implicitschemes to discretize and solve the nonlinear diffusion on hy-perspectral imagery, detailing the effect of nonlinear diffusionon the spatial and spectral variability of the image, as well as itseffect on classification accuracy, using real and synthetic hyper-spectral images.

Nonlinear diffusion generates a scale-space representation ofthe image (see Section II), which constitutes a powerful tool forimage processing, in computer vision. The scale-space repre-sentation of an image facilitates the detection of different struc-tures in the image at their appropriate image scale. Hyperspec-tral remote sensing can benefit enormously from the scale-spaceframework, which has been largely limited in their applicationsto grayscale and color images to improve object recognitionfrom images taken on remote-sensing platforms.

II. BACKGROUND

A. Classification of Hyperspectral ImageryRemote-sensing sensors are producing high spectral and spa-

tial resolution imagery, increasing their potential for environ-mental monitoring, precision farming, insurance and car nav-igation at global and local scales. More recently, hyperspec-

1http://aviris.jpl.nasa.gov2http://surfaceoptics.com

1057-7149/$25.00 2007 IEEE

1304 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 16, NO. 5, MAY 2007

tral imaging technology has found applications beyond earthremote sensing in agriculture, medicine, biology, pharmaceu-ticals, forensics, color vision, target detection, archaeology, andmany others near field applications. However, classification ofHSI imagery is primarily made on a pixel by pixel basis withclassification accuracy figures in the range 80%85%, and theyhave not changed significantly in recent decades [1]. The nat-ural variability of the material spectra and the noise added bythe transmission media and sensor system make necessary theuse of statistical methods for information extraction and patternrecognition on hyperspectral imagery.

Statistical parametric and nonparametric classificationmethods that derive directly from the Bayes rule suffer fromthe Hughes phenomena [2], that is, in order to estimate accu-rately the density distribution of each class in high-dimensionalfeature spaces, a prohibitively large amount of training samplesis required. Hyperspectral imagery with hundreds of channelspresents such a challenge.

Instead of processing remotely sensed imagery at the pixellevel, it has been proposed [3] to segment the images as a dis-joint set of regions that have homogenous distinctive spectralresponse and spatial uniformity. State of the art, object-based,and object-oriented segmentation algorithms have been recentlyused for remotely sensed multispectral imagery [4][7], butlittle has been done on hyperspectral imagery due to the largedimensionality of the data.

The underlying assumption in nonlinear diffusion is that thetrue image is piecewise smooth and the original image is cor-rupted by noise. The scale-space framework introduced by thediffusion equation has been also used for image compression[8], in conjunction with level sets to detect movement (opticflux) in image sequences [9], information extraction [10] andimage restoration [11], registration [12], and segmentation inte-grating level sets in a common framework [13].

B. Image Smoothing Using PDEs

Image smoothing by parabolic PDEs can be seen as a contin-uous transformation of the original image into a space of pro-gressively smoother images identified by the scale or levelof image smoothing, in terms of pixel resolution [14]. How-ever, the structures on an image can be of any size, that is, theycan be located at different image scales, in the continuum scalescale generated by the PDE. The adequate selection of an imagescale smoothes out undesirable variability at lower scales thatconstitute a source of error in segmentation and classificationalgorithms.

Perona and Malik [15] proposed a nonlinear diffusion PDEdefined in such a way that forward diffusion (smoothing)occurs more likely within the image structures; meanwhile,backward diffusion (sharpening) may occur on their boundaries(edges). Later, the pioneering work of Alvarez et al. [14] provedthat every scale scale that satisfies some natural architectural(recursivity, causality, regularity, locality, and consistency),information-reducing, and invariance properties (stability andshape-preserving) is governed by a second order PDE withthe original image as its initial condition. They also showedthat the PeronaMalik nonlinear diffusion equation was ill

posed, though it can be made well posed if the flux is alwaysnon-negative.

In 1996, Weickert [16] established the well posedness andaxiomatic requirements that a discretized diffusion equationmust satisfy in order to share similar scale-space propertiesas the continuous PDE. The usual discretization scheme ofthe nonlinear diffusion PDE is explicit, because it is simple toimplement. However, explicit schemes are limited by numer-ical stability conditions to small scale steps, which make themcomputationally expensive. Otherwise, semi-implicit schemesmuch better stability properties, so they can evolve at largerscale steps [16].

C. Nonlinear Diffusion PDEThe nonlinear diffusion equation proposed by Perona and

Malik for scalar images is given by the following PDE, withreflecting boundaries [16]

(1)

where is the smoothed image, at the spatial positiongiven by the coordinates vector , at scale , defined onthe domain , with boundary . The remaining terms in(1) are the diffusion coefficient , which is a nonlinearfunction of the image gradient is the original, noisyimage and is the derivative along the normal to that de-fines the boundary conditions of the PDE.

By a convenient selection of the diffusion coefficient in (1),the intensity of the image is allowed to diffuse within the imagestructures, eliminating, thus, the intraobject variability, whilepreventing diffusion on the edges, characterized by a highintensity gradient. However, as Alvarez et al. showed [14],[17], [18], the diffusion coefficients proposed by PeronaMaliklead to ill posedness of (1). Another shortcoming of thePeronaMalik equation is that noise on the edges may beamplified by backward diffusion. Alvarez et al. [17] showedthat the PeronaMalik equation can be made well posed,by smoothing isotropically the image, before computing theimage gradient used by the diffusion coefficient. Equation (2)corresponds to the regularized version of the PeronaMalikPDE, where, for simplicity, we have dropped the dependenceon the spatial coordinates, , and time , andis a smoothed version of obtained by convolving the imagewith a zero-mean Gaussian kernel of variance . With thesame boundary conditions as (1), the regularized PeronaMalikequation is given by

(2)

In our computations, we use the nonlinear diffusion coefficientproposed by Weickert [19]

(3)

DUARTE-CARVAJALINO et al.: COMPARATIVE STUDY OF SEMI-IMPLICIT SCHEMES 1305

where segmentation-like results are obtained using , andis the value that makes the flux

increasing for and decreasing for, but always non-negative.

D. Explicit Scheme

For a 2-D scalar image with , (2) can be decom-posed as

(4)

Let us call , the number of pixels in the image along theaxis and the number of pixels along the axis. Numberingthe pixels of the image in major column format, the explicitdiscretization of (4), in matrix-vector notation, is given by [19]

(5)

where , being the discretization of timeand the discretization of the spatial coordinates, isa vector of length , corresponding to the image (taken inmajor column format) at scale and are both matricesof size , being the identity matrix andthe matrix of diffusion coefficients at scale given by

(6)

In (6), and are, respectively, the image intensity and dif-fusion coefficient at coordinates and scale

with being the indices of the image, inmajor column format. Here, we make the usual assumption that

so that is the scale step.

E. Semi-Implicit Schemes

From consistency and stability considerations [19], [20], theexplicit scheme indicated in (5) requires that , whichconstitutes a severe limitation on the step size. Alternatively, wecan use semi-implicit discretization schemes, given by [19]

(7)

where and are defined as before. Semi-implicit schemesare unconditionally stable for all values of [19], [20]. How-ever, (7) requires us to solve a linear system with equa-tions and unknowns, at each iteration step. The extra compu-tational cost required to update the solution is compensated bythe numerical stability of semi-implicit schemes that allow us tochoose much larger scale steps, limited only by the accuracy ofthe computed solution.

We consider here the AOS and ADI semi-implicit methodsthat decompose (7) as a sum (AOS) or a product (ADI) of twotridiagonal systems, which can be solved, in linear time, using

the Thomas algorithm [19]. Additionally, we use several precon-ditioning techniques to accelerate the convergence of the conju-gated gradient method, which is the optimum iterative methodto solve large sparse linear systems, , provided thatis symmetric positive definite, which is regularly the case of dis-cretized PDEs.

1) Additive Operator Splitting (AOS) Method: AOS approx-imates the solution of (7) as [19]

(8)

where . Since, andare both tridiagonal matrices, and can be obtained in lineartime using the Thomas algorithm.

2) Alternating Direction Implicit (ADI) Methods: Here, wewill consider, the three most widely used ADI methods [20],[21]: Locally one-dimensional (LOD), DouglasRachford, andPeacemanRachford. The simplest approximation to (7) isgiven by ADI-LOD

(9)

The DouglasRachford method solves (7) as

(10)

AOS and the ADI schemes considered until now are onlyfirst order accurate in scale. A scheme that is second orderaccurate in scale, for the isotropic diffusion equation, is thePeacemanRachford scheme given in (11) [20]. However, thisscheme does not achieve second order accuracy in scale whenit is used on the nonlinear anisotropic diffusion equation, sincethe diffusion coefficients are computed at the previous step[21], tough, better accuracies can be expected using this schemeif is close to , i.e., at small scale steps

(11)

3) Preconditioned Conjugated Gradient (PCG): The con-jugated gradient (CG) can be considered as an acceleration ofsteepest descent to solve the linear system , whenis symmetric positive definite [20]. The basic idea of precondi-tioning is to replace the system by [22]

(12)

where matrix is called the preconditioner of andis a matrix with better condition number than , such that theconjugated gradient method converges faster, and the operation

must be performed fast, for any vector .


The simplest preconditioner for is based on theSymmetric successive over-relaxation method (SSOR), whichhas an explicit formula for the preconditioner [20]

(13)

where being a lower triangular matrix and. For any vector , the product is equivalent

to solve the system as

(14)

where can be computed in linear time using forwardand backward substitution, since and are lowerand upper tridiagonal triangular matrices, respectively.

Another preconditioner commonly used in practice is the in-complete Cholesky factorization that approximates as theproduct , where and is the error in theapproximation. In our work, we use the incomplete Choleskyfactorization with 0 drop tolerance as indicated in [23], whichmeans that has the same sparcity pattern as the lower trian-gular part of .

AOS and ADI methods provide also an approximation to ma-trix , as given by (9)(11); hence, they can be used also aspreconditioners. In particular, ADI is usually run with a fixednumber of times as a preconditioning step as in [24].

III. EXTENSION TO HYPERSPECTRAL IMAGERYA hyperspectral image is an especial case of multispectral im-

ages in the sense that we have now hundreds of bands, insteadof tents of bands as it is usual in multispectral images, providingmuch more information about the physical nature of the under-lying substrate.

The first problem one face trying to extend the methods usedin computer vision for grayscale image processing to vector-valued images is to extend the concept of gradient. The firstformal treatment of gradient in vector-valued images is due toDi Zenzo in 1986 [25].

Let be a vector-valued image, withcomponents . Hence, thefirst fundamental form in differential geometry is given by [26]

(15)

For a unit vector is a measure of therate of change in the image on the direction. The extremaof are obtained by the eigenvalues of the matrixin the directions given by the eigenvectors. Let be, re-spectively, the maximum and minimum values of the rate ofchange in and the respective directions of maximal

and minimal rate of change. Hence, the strength of an edge ona vector-valued image is a function that measuresthe dissimilarity between and . A possible choice for

is , which reduces to

(16)

Of course, other possible choices are available for asin the Beltrami flow frame-

work [27]. However, we use (16) for our extension of the non-linear diffusion PDE to hyperspectral images, because it can beinterpreted as the Euclidean distance between two close vectors(recall that we assumed ). One can think that, inhyperspectral images, other similarity metrics, commonly usedin remote sensing, could also be used here instead of the imagegradient, but we will not explore those possibilities here anylonger.

We can represent a hyperspectral image as a matrix, where is the number of pixels

in the image, and each vector corresponds to the spectralsignature of the th pixel in the image, taken in major columnformat. Hence, the semi-implicit scheme given in (5) becomesnow for a hyperspectral image

(17)

where and are matrices of size is the identity ma-trix and is defined by the diffusion coefficients as

(18)

Notice that we introduce in (18) the factor (number of spec-tral bands) to normalize the measure of dissimilarity betweentwo vector valued pixels.

The AOS scheme can be extended to hyperspectral imageryin a straightforward manner, by changing to in (8) as

(19)


We can find the unknown matrices and in (19), using theThomas algorithm with the spectral vectors instead of scalars,updating simultaneously all image bands. ADI methods can beextended to vector valued images, similarly as we did here withthe AOS scheme.

Let us consider now the extension of PCG methods to vectorvalued images. Since the CG method works with the image,taken in major column format, as a single vector, we will usea different, but equivalent, matrix representation of the hyper-spectral image, namely , where eachcolumn corresponds to each image band, taken in majorcolumn format. Since matrix is the same for all image bands,the preconditioner is also the same and (12) is now

(20)

Solving (20) independently for each image band requiressolving times a system of equations and unknownsat each iteration step. In order to speed up the process, wepropose here to update all image bands, simultaneously, basedon the mean value of the image, along the spectral direction,i.e., update all image bands, based on the scalar image

(21)

Additionally, ADI and AOS schemes, used as preconditioners,require the inclusion of a reduction factor , in order to avoid in-stability on the conjugated gradient method at high scale steps. Ifwe want to find , where is the preconditioner in (22),shown at the bottom of the page, and the AOS preconditioner in(23), shown at the bottom of the page, is the ADI-LOD precon-ditioner. The PeacemanRachford and DouglasRachford ADIschemes are more expensive computationally and more sensi-tive to the scale step than ADI-LOD, and, hence, they are notused here as preconditioners.

IV. EXPERIMENTSThe numerical methods indicated in Section II were imple-

mented in Matlab, using the extensions to vector-valued imagesexplained in Section III. The classification was performed withMultispec3 freeware software developed by Landgrebe. All thehyperspectral images were normalized in the [0 1] range.

The choice of an optimum threshold value, , has beenaddressed by several authors [27][29], and it is still an openproblem. However, in this work, we are concerned with therelative running time and accuracy of each of the schemes

3http://dynamo.ecn.purdue.edu/~biehl/MultiSpec/

considered, versus the explicit scheme, and, hence, finding thebest value of is not relevant. The stopping scale is chosenhere as convenient integer value that facilitates the comparisonbetween the different schemes.

It seems reasonable to select in (2), since 99% of theGaussian area is within , that is, pixel from the center,which corresponds to the same stencil used by PeronaMalikin the discretization of the nonlinear diffusion equation, i.e., a3 3 grid. As Weickert [19] suggested, the regularization of theimage can be done efficiently using isotropic diffusion, on eachstep with .

Otherwise, since , then and the regu-larization of the image can be done with a single step of theexplicit scheme, which is stable and computationally cheaperthan the semi-implicit schemes for . In our experi-ments, , which is a value that preserves the edgeson all images considered. Even though this value seems verysmall to produce appreciable smoothing, when it is repeatedon each iteration step, it is enough to avoid enhancing ofimpulsive noise (characterized by a very high peak or valleysurrounded by a smooth neighborhood) without destroying theimage edges.

The four hyperspectral images used in our experiments are asfollows.

1) The Indian Pines image Fig. 1(a) taken with the AVIRIS(Airborne Visible/Infrared Imaging Spectrometer) sensor,flown by NASA/Ames on June 12, over an area 6 mi westof West Lafayette, IN. This image contains 145 145pixels and 220 spectral bands in the 4002500-nm range,for which ground truth exists. We disregard bands 13, 58,77, 103110, 148166, and 218220, from the originalimage either because they do not contain information,they were too noisy or present strong illumination dif-ferences due to the sensor; hence, our Indian Pinesimage has 145 145 pixels and 185 spectral bands in the4102430-nm range.

2) A synthetic hyperspectral test image Fig. 1(b) made fromreal pixels extracted from the Indian Pines image that fillssimple geometric figures: triangle, ellipse, donut, and acommon background. This image has 150 150 pixels andthe same number of bands that the Indian Pines image.The pixels belonging to each geometric figure and back-ground were selected at random and with uniform proba-bility, from the pixels belonging to four different crops inthe Indiana Pines image: the Corn-min field (triangle), theSoybeans-notill field (donut), the Soybeans-min field (el-lipse), and the Hay-windrowed field (the background).

3) The Cuprite image Fig. 1(c) taken over the mining dis-trict, 2 km north of Cuprite, Nevada, with the AVIRIS

(22)

(23)


Fig. 1. Grayscale (spectral mean) representation of: (a) Indian Pines, (b) Syn-thetic, (c) Cuprite, and (d) Noisy False Leaves hyperspectral images.

(Airborne Visible/Infrared Imaging Spectrometer) sensor,flown by NASA/Ames on June 19, 1997. This image con-tains five scenes for a total of 640 2378 pixels and 224bands in the 3702500-nm range. We selected a portion ofthe fourth scene, in the Cuprite image, of size 500 500pixels, that corresponds to part of the mineral mapping inthe Cuprite mining district, reported by the U.S. GeologicalSurvey (USGS) spectroscopy laboratory in 1995, using theexpert system algorithm Tetracorder [30] and signatures of60 sampled fields in the region. We use the USGS imagesas ground truth.4 We selected from this image 50 bands:172221 that correspond to the 20002480-nm vibrationalabsorption region used by the USGS to mapping mineralsin the Cuprite image.

4) The False Leaves indoor image Fig. 1(d) of size 640 640pixels and 120 bands in the 402908-nm range, obtainedby the Surface Optics Company using the SOC-700 hy-perspectral imager. We selected a portion of this image ofsize 540 575 pixels that contains all the objects present inthe original image. Additionally, and given that this spec-trometer has a high spectral resolution, we selected only 30bands of the original image, by taking one of each four con-secutive bands. Since, this image has a high signal-to-noiseratio (SNR) and none of the atmospheric effects that af-fect remote-sensed images, such as those taken with theAVIRIS sensor; we add white Gaussian noise with zeromean and , of amplitude 10% relative to the max-imum amplitude in the image, on each image band, andthen renormalization to the [0 1] range.

4http://speclab.cr.usgs.gov/PAPERS/tetracorder

Fig. 2. Superimposed spectra showing the spectral variability within each ob-ject and background on the synthetic image.

A. Performance in Terms of the Accuracy of the ComputedSolution

The synthetic image is used to quantify the numerical perfor-mance in terms of the accuracy achieved by the different semi-implicit methods implemented, as the scale step increases, rel-ative to a reference image generated using a very small scale step

and the semi-implicit CrankNicholson scheme[20], which is a second order accurate scheme, both in scale andspace.

The highest value of that preserves the edges in thesynthetic image, while reduce most of the internal variabilitywithin the image objects is 0.015. Otherwise, the accuracyof the explicit scheme at its maximum possible step size,

and the accuracy of AOS, ADI and PCG semi-im-plicit schemes for and were allcompared to the reference image. We perform 1000 iterationsof the CrankNicholson scheme so that the real evolution inscale of the PDE is ; hence, the explicitscheme using should be run times and thesemi-implicit schemes should be run 100, 20, 10, 5,and 2 times.

The best values for in the PCG-SSOR scheme were found,simply by sweeping in the 0.01 to 2.0 range at intervals of0.1. The values of found by this mean were 0.5, 0.4, 0.3, 0.15,and 0.05 for and , respectively,and they also correspond to the best values for the syntheticand real hyperspectral images used. Finally, AOS and ADI-LODschemes used as preconditioners were implemented as indicatedin (27) and (28), where best results were found using 1, 0.5,0.25, 0.125, and 0.025 for and ,respectively.

Fig. 2 shows the synthetic image and the spectral variabilitywithin each image object and background, obtained by superpo-sition of the spectrums of each pixel within each image region.

Fig. 3 shows the strong reduction on the spectral variabilitywithin each image region, after nonlinear diffusion, while pre-serving the edges. Table I indicates the reduction on the vari-ance within each image region. Fig. 4 shows the classificationmap using the spectral angle mapper (SAM) in Multispec andall image bands available.


Fig. 3. Superimposed spectra showing the spectral variability within each ob-ject and background on the smoothed synthetic image.

TABLE IREDUCTION IN THE SPATIAL/SPECTRAL VARIABILITY

Fig. 4. Classification of the synthetic image using SAM on (a) original and(b) smoothed images.

Fig. 5 shows the square error of each one of the numericalmethods implemented here, relative to the CrankNicholsonscheme.

From Fig. 5, it can be noticed that all schemes have a largererror than the explicit scheme at . In practice, wefound that a square error above produces visible artifactsin the smoothed image. Hence, one could conclude that usingthe PeacemanRachford and DouglasRachford schemes wecannot achieve scale steps larger than 15 times the explicitscheme without producing visible artifacts in the image. Simi-larly, AOS can only achieve a scale step 25 times larger than theexplicit scheme, result this that agrees with the ones reportedby [19], while the PCG method initialized with ADI-LOD canreach higher step values than the ones used here. The remainingmethods can achieve scale steps up to in this image.

Fig. 5 gives us an idea of the accuracy of the computed solu-tion, as we increase the scale step, in the semi-implicit scheme.However, in practice, the quality of the computed solution notnecessarily translates into higher classification accuracies. We

Fig. 5. Square error on the computed solution of each algorithm, relative to theCrankNicholson scheme.

explore in the next set of experiments the performance of ouralgorithms in terms of speed up and classification accuracies, asthe scale step increases, using real hyperspectral images.

Finally, if we call the disk storage of an image of size(see Section III), then AOS and ADI-LOD re-

quires disk space, the other ADI methods requiredisk space, and the PCG methods require disk space.These values must be kept in mind when selecting between thesemethods to solve the semi-implicit PDE (20), since typical hy-perspectral images requires GBytes of disk storage.

B. Performance in Terms of the Classification AccuracyIn order to test classification accuracy on the original and

smoothed hyperspectral images, we need training and testingsamples. The Indian Pines and Cuprite images are two of thefew hyperspectral remote-sensed images with reported groundtruth. Ground truth is very scarce in remote sensing, given thecosts involved in its acquisition. The False Leaves is an indoorimage with objects that can be easily identified. This image owesits name to the fact that there are some plastic leaves that cannotbe distinguished from the real ones in the visible range; hence,we must use a suitable combination of bands that include thenear infrared wavelengths to detect visually the false leaves andselect the corresponding training and testing samples.

Fig. 6(a) shows the ground truth available for the Indian Pinesimage consisting of 16 classes, of which ten correspond to dif-ferent kind of crops, five correspond to vegetation, and one cor-responds to a building. Fig. 6(b) shows the training and testingsamples selected for 14 of the 16 classes identified on the IndianPines image. The other two classes (Oats and Alfalfa) were notsampled since there are not enough training and testing samplesto perform the classification using classical statistical classifica-tion methods.

Fig. 7 shows the ground truth available for the Cuprite image,which consists of 25 classes of minerals, grouped in five cate-gories: sulfates, carbonates, Kaolinites, Clays, and other min-erals. Fig. 8(a) shows the training and testing samples selectedon 11 classes of the Cuprite image. They are Calcite, Kaoli-nite and Semectite or Muscovite, K-Alumnite, Kaolinite, Alu-nite and Kaolinite or Muscovite, Calcite and Kaolinite, Chal-


Fig. 6. Indian Pines image: (a) Ground truth and (b) training and testing sam-ples (RGB shown corresponds to bands 29, 15, and 12).

Fig. 7. Ground truth Cuprite image.

Fig. 8. Training and testing samples on (a) Cuprite image (RGB correspondsto bands 183, 193, and 207) and (b) False Leaves image (RGB corresponds tobands 90, 68, and 29).

cedony, Na-Montmorillonite, Chlorite and Muscovite or Mont-morillonite, High-Al Muscovite, and Med-Al Muscovite. Weconsider the different kinds of Alunites as a single class, giventhat it is extremely difficult to obtain pure training and testing

samples in this image. The remaining classes were not sampledgiven that they do not provide enough training and testing sam-ples or because they were too difficult of localize within theCuprite image, even with the help of the wavelengths recom-mended by the USGS to identify some of the minerals in thisimage [see Fig. 8(a)].

Fig. 8(b) shows the training and testing samples on each oneof the different objects that can be identified in the Fake Leavesimage using the channels indicated. The classes in this image arethe wall, the jar, the flowerpot, the true leaves [seen as red leaveson Fig. 8(b)], the false leaves [seen as dark leaves on Fig. 8(b)],the metallic case, plastic label, paper label, and lens cover (darkred) of the featured SOC-700 hyperspectral imager that appearsin the image.

We use all the classical classifiers available in Multispec [31]:maximum likelihood (ML), Fisher linear likelihood (FLL), Eu-clidean distance (ED), extraction and classification of homoge-neous objects (ECHO), SAM, and matched filter (MF) to eval-uate how each classifier is affected by the nonlinear diffusionprocess.

Since the smoothed Indian Pines image has 185 spectralbands and the statistical classifiers employed here require moretraining pixels than spectral bands in the image [31], we se-lected 20 bands using the SVD subset band selection algorithmimplemented at the UPRM Matlab toolbox [32] on each one ofthe smoothed images.

The best classification results for the Indian Pines image wereobtained using and runs of the explicit schemeat . Hence, the semi-implicit methods were also runfor and that correspond to

10, 5, 2, and 1 step, respectively. For the Cupriteimage, we selected a value of 0.015 and 50 steps of theexplicit scheme, and, hence, we use the same values of as inthe Indian Pines image for the semi-implicit methods. On theother hand, we obtained good classification results in the noisyFake Leaves image using 0.015 and 100 runs of the explicitscheme; hence, the semi-implicit methods were run for 20, 10,5, and 2 steps, respectively.

The classification results are shown in Tables IIIV for eachimage and numerical method implemented. In these tables,stands for the speedup relative to the explicit method, i.e., theratio between the running time of the explicit method and therunning time of the semi-implicit methods. The running time ofall the algorithms is given in minutes using a PC with 1.5-Gb ofRAM, CPU of 2.8 GHz, and Matlab for windows.

The highest classification accuracies and speedups are in-dicated in the tables, for each method, in bold and cursive.The highest speedups were chosen as the maximum speedupthat keeps the classification accuracy very close or above theclassification accuracy achieved with the explicit scheme. Ofcourse, the best performance is for those methods that achieveclassification accuracies above the explicit method and highspeedups.

From Tables IIIV, one can see that all the smoothed imagesachieve higher classification accuracies than using the originalimage, on all classifiers, except for the ML classifier on theCuprite and Fake Leaves images. The bad performance of MLon these images can be explained by the fact that ML becomes


TABLE IICLASSIFICATION ACCURACIES, INDIAN PINES IMAGE

very unstable when the region of the training samples is too uni-form. This effect affects more the Cuprite and False Leaves im-ages given that the smoothing is higher ( 0.015) than in thecase of the Indian Pines image ( 0.012) and also becausethese images have more bands.

On the other hand, the FLL classifier benefits from the reduc-tion in the variability within the image classes [31], and, hence,it has the highest classification accuracies on all the images.

ECHO classifier is based in Multispec on either a quadraticor Fisher linear spectral-spatial algorithm. The results indicatedon Tables IIIV for ECHO correspond to the maximum valuebetween the two possible classifiers, which was almost alwaysFLL for the smoothed images and quadratic for the original im-ages. In general, ECHO is just a little superior than FLL in clas-sification accuracy. The difference between ECHO and FLL re-duces as the smoothing increases, as can be appreciated on Ta-bles III and IV, where 0.015, meanwhile the differenceis higher in the Indian Pines image, where 0.012. This isdue to the fact that ECHO tries to homogenize the image be-fore classifying it, by choosing a small window (2 2 pixels inour simulations). Hence, if the region within the objects is al-ready smooth, due to diffusion, the difference between ECHOand FLL is reduced.

The remaining classifiers, ED, SAM, and MF are, in general,very insensitive to the scale step, but, in general, they do notachieve good classification accuracies, except for the SAM clas-sifier on the Cuprite image. The relative good performance of

TABLE IIICLASSIFICATION ACCURACIES, CUPRITE IMAGE

SAM on this image agrees with the reported studies on min-eral classification using the spectral angle and a high number ofbands [33].

In terms of the implemented numerical methods, AOS andADI are very insensitive to achieving high speedups andclassification accuracies up to on all the imagesanalyzed here. On the other hand, the DouglasRachford andPeacemanRachford methods are sensitive to the scale step,achieving high classification accuracy only up to ,which limits their speedup. Despite of this limitation, theseschemes are of higher accuracy than AOS and ADI and theyachieved the highest classification accuracies on all the images.

PCG methods are very insensitive to the scale step and allbehave similarly in terms of classification accuracy. The bestclassification accuracies and speed-ups are achieved by PCG-Cholesky initialized by ADI-LOD. These results are similar tothe ones obtained in terms of the accuracy of the computed so-lution (Fig. 4). This means that the accuracy of the computedsolution affects the classification accuracy in the case of ML,FLL, and ECHO classifiers.

It is noteworthy, though, that AOS has a better performancethan the expected from Fig. 4, for the Indian Pines image. Webelieve that this occurs because K is small here; hence, the mag-nitude of the error is lower. AOS is also symmetric, and, hence,the error introduced can be reduced by a classifier as ECHO,which tends to average out random variations in a small window.It is also fortunate that the Indian Pines image consists of large


TABLE IVCLASSIFICATION ACCURACIES, FALSE LEAVES IMAGE

patches of uniform regions allowing that ECHO obtains a greaterreduction on the artifacts introduced by large scale steps. SinceADI-LOD is not symmetric, the artifacts introduced are not re-duced by ECHO. Otherwise, the fortunate situation for AOS inthe Indian Pines image does not applies for the other real im-ages, and, hence, its performance is not as good there.

Otherwise, from Tables IIIV, we can see that the maximumspeedup reduces as the complexity of the image increases, interms of higher object intravariability. The Cuprite and FalseLeaves images are more complex than the Indian Pines image,because the Cuprite image is not a patchy image as it happenswith the Indiana Pines image, and we add a nondepreciableamount of noise to the False Leaves image. This higher com-plexity was dealt in our case by increasing the value of , whichmeans that there is more diffusion in the image, and, hence, fora given , the change is higher with respect to the original imageand the errors in the computed solution (artifacts) are of highermagnitude, affecting more the classification accuracy and re-ducing the speedup that we can achieve on those images.

In order to see the effect of nonlinear smoothing on the clas-sification of the full real hyperspectral images used here, inFigs. 911, we present the classification maps of the original andsmoothed images that achieved the highest classification accu-racies on Tables IIIV. It can be noticed from Figs. 10 and 11that not only the testing samples improve their classification ac-curacy, but that the smoothed images also produce classificationmaps that look more accurate.

Fig. 9. Indian Pines classification map: (a) original; (b) smoothed.

Fig. 10. Cuprite classification map: (a) original; (b) smoothed.

Fig. 11. Fake Leaves classification map: (a) noisy; (b) smoothed.

V. CONCLUSION

PDE-based methods for image enhancement, segmenta-tion, and restoration have a large history of success for scalarand color images in computer vision, but it has been disre-garded in segmentation and classification of hyperspectralimagery. Recently, Lennon et al. [34], [35] implemented thePeronaMalik nonlinear diffusion equation to smooth a hyper-spectral image and classify it using support vector machines.However, they used the original, unregularized explicit schemeof PeronaMalik, given in (1) and used only 17 bands.

This work shows that PDE-based image processing methodscan improve significantly image enhancing, segmentation, andclassification in hyperspectral imagery at a low computational


cost, using semi-implicit schemes. Traditional statistical classi-fication methods are very robust at low-dimensional spaces, butthey require an enormous amount of data at higher dimensionalspaces, as is the case of hyperspectral imagery. Otherwise, par-abolic PDEs offer a well-sounded, common framework to per-form image smoothing, restoration and object-based segmen-tation and classification, with accuracy and highly paralleliz-able discretizations that can speedup PDE image processing inhigh-dimensional spaces.

In particular, this work shows that nonlinear diffusion can en-hance significantly image classification accuracies by reducingboth, the spatial and spectral variability in hyperspectral im-agery. AOS and ADI semi-implicit schemes offer high perfor-mance in terms of accuracy and speedup of the computed so-lution of the nonlinear PDE, when the complexity of the imageis not high in terms of highly variability within the image ob-jects. When the complexity of the image increases, more accu-rate methods such as the Douglas and Peaceman schemes andPCG methods can achieve accuracies and speedups superior tothe less accurate AOS and ADI-LOD methods, justifying theirhigher computational cost.

PCG linear solvers are less sensitive to the scale step as theapproximated ADI and AOS schemes, which mean that highervalues of can be used. However, PCG methods also requiremore space, and finding a good preconditioner is still an art aswe could corroborate here. In fact, PCG-methods also dependon the size of the image, making it difficult to generalize themto all image sizes. Even though image complexity can reducesensibly the speedup that can be achieved with the numericalmethods presented here; we achieved significant speedups of10 and higher on all the images used, over the explicit scheme,which justifies their use in hyperspectral imagery.

REFERENCES[1] G. Wilkinson, Are remotely sensed image classification techniques

improving? Results of a long term trend analysis, presented at theIEEE Workshop on Advances in Techniques for Analysis of RemotelySensed Data, Oct. 2728, 2003.

[2] D. A. Landgrebe, Hyperspectral image data analysis, IEEE SignalProcess. Mag., vol. 19, no. 1, pp. 1728, Jan. 2002.

[3] R. L. Ketting and D. A. Landgrebe, Classification of multispectralimage data by extraction and classification of homogeneous objects,IEEE Trans. Geosci. Electron., vol. GE-14, no. 1, pp. 1926, Jan.1976.

[4] A. Marangoz, M. Oruc, and G. Buyuksalih, Object-oriented imageanalysis and semantic network for extracting the roads and buildingsfrom IKONOS pan-sharpened images, Int. Arch. Photogramm. Re-mote Sens., vol. 35, no. B3, 2004.

[5] W. Tadesse, T. L. Coleman, and T. D. Tsegaye, Improvement of landuse and land cover classification of an urban area using image segmen-tation from Landsat ETM+ data, presented at the Proc. 30th Int. Symp.Remote Sensing of the Environment, Honolulu, HI, 2003, PS I-35.

[6] J. Schiewe, L. Tufte, and M. Ehlers, Potential and problems of multi-scale segmentation methods in remote sensing, GeoBIT/GIS J. SpatialInf. Decision Making, vol. 6, pp. 3439, 2001.

[7] U. C. Benz, P. Hofmann, G. Willhauck, I. Lingenfelder, and M.Heynen, Multi-resolution, object-oriented fuzzy analysis of remotesensing data for GIS-ready information, J. Photogramm. RemoteSens., vol. 58, pp. 239258, Jan. 2004.

[8] T. Sziranyi, I. Kopilovic, and B. P. Toth, Anisotropic diffusion as apreprocessing step for efficient image compression, in Proc. 14th Int.Conf. Pattern Recognition, 1998, vol. 2, pp. 15651567.

[9] L. Alvarez, J. Weickert, and J. Sanchez, Scale-space approach to non-local optical flow calculations, in Proc. 2nd Int. Conf. Scale-SpaceTheories in Computer Vision, 1999, vol. 1682, pp. 235246.

[10] I. Pollak, A. S. Willsky, and Y. Huang, Nonlinear evolution equationsas fast and exact solvers of estimation problems, IEEE Trans. SignalProcess., vol. 53, no. 2, pp. 484498, Feb. 2005.

[11] D. Tschumperle and R. Deriche, Vector-valued image regularizationwith PDEs: A common framework for different applications, IEEETrans. Pattern Anal. Mach. Intell., vol. 27, no. 4, pp. 506517, Apr.2005.

[12] B. Fischer and J. Modersitzki, Fast diffusion registration, in Contem-porary Mathematics: Inverse problems, Image Analysis and MedicalImaging. Philadelphia, PA: AMS, vol. 313, pp. 117129.

[13] R. Malladi and J. A. Sethian, A unified approach to noise removal,edge enhancement, and shape recovery, IEEE Trans. Image Process.,vol. 5, no. 11, pp. 15541568, Nov. 1996.

[14] L. Alvarez, F. Guichard, P. L. Lions, and J. M. Morel, Axioms and fun-damental equations of image processing, Arch. Rational Mech., vol.123, pp. 199257, Sep. 1993.

[15] P. Perona and J. Malik, Scale-space and edge detection usinganisotropic diffusion, IEEE Trans. Pattern Anal. Mach. Intell., vol.12, no. 7, pp. 629639, Jul. 1990.

[16] J. Weickert, Anisotropic diffusion in image processing, Ph.D. disser-tation, Univ. Kaiserslautern, Kaiserslautern, Germany, 1996.

[17] L. Alvarez, P. L. Lions, and J. M. Morel, Image selective smoothingand edge detection by non-linear diffusion, SIAM J. Numer. Anal., vol.29, no. 1, pp. 182193, Jun. 1992.

[18] L. Alvarez, P. L. Lions, and J. M. Morel, Image selective smoothingand edge detection by non-linear diffusion II, SIAM J. Numer. Anal.,vol. 29, no. 3, pp. 845866, June 1992.

[19] J. Weickert, B. M. ter Haar Romeny, and M. A. Viergever, Efficientand reliable schemes for nonlinear diffusion filtering, IEEE Trans.Image Process., vol. 7, no. 3, pp. 398410, Mar. 1998.

[20] J. C. Strikwerda, Finite Difference Schemes and Partial DifferentialEquations. Philadelphia, PA: SIAM, 2004.

[21] D. Barash, T. Schlik, M. Israeli, and R. Kimmel, Multiplicative oper-ator splittings in nonlinear diffusion: From spatial to multiple steps, J.Math. Imag. Vis., vol. 19, pp. 3348, Jul. 2003.

[22] R. Barret, Templates for the Solution of Linear Systems: BuildingBlocks for Iterative Methods. Philadelphia, PA: SIAM, 1994.

[23] Y. Saad, Iterative Methods for Sparse Linear Matrices, 2nded. Philadelphia, PA: SIAM, 2003.

[24] P. Castillo and Y. Saad, Preconditioning the matrix exponentialoperator with applications, J. Sci. Comput., vol. 13, no. 3, Sep.1998.

[25] S. Di Zenzo, A note on the Gradient of a multi-image, Comput. Vis.,Graph., Image Process., vol. 33, no. 1, pp. 116125, Jan. 1986.

[26] G. Sapiro, Geometric Partial Differential Equations and Image Anal-ysis. Cambridge, U.K.: Cambridge Univ. Press, 2001.

[27] M. J. Black, G. Sapiro, D. H. Marimont, and D. Heeger, Robustanisotropic diffusion, IEEE Trans. Image Process., vol. 7, no. 3, pp.421431, Mar. 1998.

[28] F. Voci, S. Eiho, N. Sugimoto, and H. Sekiguchi, Estimating the gra-dient threshold in the PeronaMalik equation, IEEE Signal Process.Mag., vol. 21, no. 3, pp. 3965, May 2004.

[29] P. Mrzek and M. Navara, Selection of optimal stopping time for non-linear diffusion filtering, Int. J. Comput. Vis., vol. 52, no. 23, pp.189203, MayJun. 2003.

[30] R. N. Clark, Imaging spectroscopy: Earth and planetary remotesensing with the USGS Tetracorder and expert systems, J. Geophys.Res., vol. 108, no. E12, pp. 5-15-44, Dec. 2003.

[31] D. A. Landgrebe, Signal Theory Methods in Multispectral RemoteSensing. Hoboken, NJ: Wiley, 2003.

[32] E. Arzuaga-Cruz, A MATLAB toolbox for hyperspectral image anal-ysis, in Proc. IEEE Int. Geosci. Remote Sens. Symp., 2004, vol. 7, pp.48394842.

[33] G. Girouard, A. Bannari, A. El Harti, and A. Desrochers, Validatedspectral angle mapper algorithm for geological mapping: Comparativestudy between quickbird and landsat-TM, in Proc. 20th ISPRS Con-gress: Geo-Imagery Bridging Continents, Istanbul, Turkey, 2004, pp.599604.

[34] M. Lennon, G. Mercier, and L. Hubert-Moy, Nonlinear filtering ofhyperspectral images with anisotropic diffusion, in IEEE Int. Geosci.Remote Sens. Symp., 2002, vol. 4, pp. 24772479.

[35] M. Lennon, G. Mercier, and L. Hubert-Moy, Classification ofhyperspectral images with nonlinear filtering and support vectormachines, in IEEE Int. Geosci. Remote Sens. Symp., 2002, vol. 3,pp. 16701672.


Julio M. Duarte-Carvajalino (S07) received theB.S.E.E. degree (cum laude) from the UniversidadIndustrial de Santander (UIS), Colombia, in 1995,and the M.Sc. degree in electric engineering fromthe University of Puerto Rico, Mayagez (UPRM),in 2003. He is currently pursuing the Ph.D. degreein the Computing and Information Sciences andEngineering doctorate program, UPRM.

From 1995 to 1998, he worked in electric con-struction. From 1998 to 2000, he was an AssistantProfessor at the Universidad Tecnolgica de Bolivar,

Colombia. His research interests are in computer vision, especially imageprocessing using PDEs.

Prof. Duarte-Carvajalino was awarded with the National Science Foundation-EPSCOR fellowship for three consecutive years, from 2003 to 2007. He wasalso included in the National Deans List of 2003/2004 and 2004/2005 and theChancellor List of 2004/2005 and 2005/2006.

Paul E. Castillo received the licence degree inmathematics from UNAH, Tegucigalpa, Honduras,in 1988, and from USTL Montpellier II, France,in 1989; the M.Sc. degree in computational mathe-matics from the University of Puerto Rico, Mayagez(UPRM), in 1995; and the M.Sc. degree in computerscience and the Ph.D. degree in scientific computa-tion from the University of Minnesota, Minneapolis,in 2001.

He was a Postdoctorate at Lawrence LivermoreNational Laboratory, Livermore, CA, from 2001

to 2003, where he worked in the development of a high-order finite-elementcode (FEMSTER). In 2003, he joined the Department of Mathematical Sci-ences, UPRM. His research interests include numerical analysis, in particular,discontinuous Galerkin methods, adaptive finite-element techniques, and thedevelopment of mathematical software for solving physical problems.

Miguel Velez-Reyes (S81M92SM00) receivedthe B.S.E.E. from the University of Puerto Rico,Mayagez (UPRM), in 1985, and the M.Sc. andPh.D. degrees from the Massachusetts Instituteof Technology, Cambridge, in 1988 and 1992,respectively.

In 1992, he joined the faculty of the UPRM wherehe is currently a Professor. He has held faculty in-ternship positions with AT&T Bell Laboratories, AirForce Research Laboratories, and the NASA God-dard Space Flight Center. His teaching and research

interests are in the areas of model-based signal processing, system identifica-tion, parameter estimation, and remote sensing using hyperspectral imaging.He has authored over 60 publications in journals and conference proceedings.He is the Director of the UPRM Tropical Center for Earth and Space Studies,a NASA University Research Center, and the Associate Director of the Centerfor Subsurface Sensing and Imaging Systems, a National Science FoundationEngineering Research Center lead by Northeastern University.

Dr. Velez-Reyes was one of 60 recipients from across the United States andits territories of the Presidential Early Career Award for Scientists and Engineers(PECASE) from the White House in 1997. He is a member of the Academy ofArts and Sciences of Puerto Rico and a member of the Tau Beta Pi, Sigma Xi,and Phi Kappa Phi honor societies.

Documents

Comparative Study of Semi-Implicit Schmes