COLOR DISTRIBUTION EVENNESS AND ITS APPLICATION TOCOLOR-TEXTURE SEGMENTATION

Xin Zhang, Hui Wang, Chao Gao, Yunli WangMultimedia Research & Development Center, National University ofDefense Technology,

Changsha, Hunan Province, P.R. China 410073

ABSTRACT

This paper proposes a new texture description metric, theColor Distribution Evenness (CDE) measure, and discussesits usage of performing multi-scale texture analysis. Further,CDE measure is applied to color-texture segmentation ofnatural images, upon which we propose the ISBEC (ImageSegmentation Based on the distribution Evenness of Colors)algorithm. Experiments verify the effectiveness of CDEmeasure for texture analysis and that of ISBEC forcolor-texture segmentation.

1. INTRODUCTION

Color-texture segmentation is the process ofpartitioning animage into a set of regions, which are visually distinct anduniform with respect to color and texture features. It comesto be a research focus since 2002. However, many existingcolor-texture methods have two common defects: The first isthat they always extract texture feature only from theintensity channel (i.e. the gray-scale component) of the inputimage, e.g. [1-3], or from three color bands independentlyand combine them later, e.g. [4]. This ruins the innateintegrity of color information. The second defect is that thesemethods always employ over-complete texture description,such as LBP [1] or spectral decomposition [2], which isusually very complex while unnecessary for segmentationbecause there are only tens, at most, of different texturescontained in an ordinary natural image.

This paper mainly deals with color-texture segmentationof natural images, and aims to overcome aboveshortcomings. To protect the integrity of color information,the paper exploits color quantization as pre-processing stepand uses the resulted color index map as input for furthersegmentation. To the author's knowledge, this idea is

originated with Deng et al. [5]. Furthermore, to simplifytexture description, the paper proposes the Color DistributionEvenness (CDE) measure, and employs it to represent andanalyze textures. Upon these, a color-texture segmentationmethod, ISBEC (Image Segmentation Based on distributionEvenness of Colors), is proposed. Note that the primarycontribution of this paper is the CDE measure and itsapplication to color-texture segmentation.

The rest part is organized as follows. Section 2 gives theformal definition of CDE in a region or a local window.Section 3 details the flow of ISBEC algorithm. Experimentalresults and concludina remarks are aiven in section 4.

(a) color texture samples (b) the quantized image (K=4)

(c) CDE map on scale 11 (d) CDE map on scale 23Fig.1. Three color texture samples from the Vistex datasets.

2. TEXTURE ANALYSIS BASED ON CDE

As everyone knows, texture is too intractable to attain acommonly accepted definition up to now. A fairly influentialone is given by Jain et al. [6], who define texture as repeatingpatterns of local variations in image intensity that are too fineto be distinguished as separate objects at the observationresolution. Fig. 1 (a) shows three color texture samplesselected from the Vistex dataset. Observing these samples,we can easily find out that colors in textured regions aredistributed (almost) evenly. For instance, black dots in thethird sample scatters evenly on the image plane. The sameobservation does also apply to most of the other samples inthe Vistex database. Moreover, color textures, according toJain's definition, are the repeating patterns of local variation

1-4244-1017-7/07/$25.00 ©2007 IEEE 1655 ICME 2007

in image colors. The repetition of (similar) local patterns alsoimplies that colors are distributed evenly. All these enlightenus that the distribution evenness of colors can be exploited todescribe and analyze texture features.

2.1 Definition of Color Distribution Evenness (CDE)

We first define the CDE in an image region R. Assume thatthere be K different colors in R, represented as{C1, C2,..., CK }I. Suppose the total pixels in color Ck be nk(k =1,...,K ), that is, if we denote the set of pixels incolor Ck also as Ck, then Ck = k .X is the cardinality ofset X. The distance between the centroid ofCk, denotedas Ck, and the geometrical center of R, denoted as CR, can beused to measure the distribution evenness ofCk on R. Thisis motivated by the fact that the distance between Ck and CRwill be equal or very close to zero when color Ck is evenlydistributed in the region. For the simplicity of computation,we use the Squared Euclidean Distance (SED) in theproposed CDE measure. Denoted as Ek , the CDE ofcolor Ck over region R can be formulated as

Ek = ICk -CR 12. (1)

The coordinates of centroid Ck = (xk, Yk ) can be computed by

- 1 _ 1Xk 1Xi, Yk= ZEY (2)

k (xI,yI)eCK nk (x,y )eCK

As a result, CDE in region R can be defined as the weightedsum of each color's CDE

K

ER= ELfkEk (3)k=l

where the weight factor of color Ck is its occurrencefrequency in region R

/Kfk==Znfnk (4)

k=l

From above definition, it is clear that ER .0Particularly, ER =0 when R is a unicolor region. If Rcontains more than one color, the more evenly these colorsdistribute, the smaller ER is. Thus, regarding relativelyeven distribution of colors as a common property of textures,we can analyze the texture features in region R accordingto ER. Moreover, because ER = 0 for unicolor regions, wetreat them also as textured ones. Summing up, we can

conclude that a region is textured when its ER iscomparatively small (close to zero).

2.2 Multi-scale texture analysis based on CDE

It is known that image segmentation is just the process ofclassifying pixels into different groups. Thus, texture featuresin the local neighborhood of each pixel is more significantwhen segmenting, which provides the grouping criterion.Viewing this, we are to define the CDE in the neighborhoodwindow W of a pixel (x, y), denoted as EW (X, Y) .

Also suppose there be K different colors in W. Anddenote its geometrical center as c,. Similar to ER, we candefine EW (x, y) to be

K

EW (x,y) = EfkEWk=l

where fk is the same with (4) and

k 2EW = Ck -CW

(5)

(6)

Ck is the centroid of Ck in W, which can be worked out in

an analogous way as (2).Similar to ER, Ew (x, y) only measures the texture

feature in window W by definition. To make it capable ofdescribing the factual local texture around pixel (x, y), wemust select the local window W appropriately. The first to bedecided is its shape. Though formula (5) has no specialrequirement on the window's shape, for the simplicity ofcomputation, centrosymmetric shapes, such as circular orsquare, are more preferable. Comparative experimentsexpose that circular or square window has little influence onthe final segmentation results. Thus, we choose squarewindow centered on (x, y) for its further simplicity.Particularly, if relative coordinates taking the window'scenter as origin are adopted, EW defined in (5) can beremarkably simplified to

K K

EW(X,Y) = IfkCkl =2fk(xk+2 k) (7)k=l k=l

Accordingly, the coordinates of centroid Ck = (xk Yk)should also be relative.

The second to be decided is the size of window W.Obviously, W should be large enough for textures in it todisplay their property of even distribution of colors. Thus,different textures need windows in different sizes. Sincethere are usually several types of textures in a single image, amore practical way is to choose a series of window sizes

1656

{JS,S,,S3,...} for each image, and detect texture featuresunder each size. This is just the so-called multi-scale textureanalysis. Size Si is also referred to as a scale. According tohuman perceptual habits, a region will be regarded as atextured one only if it contains at least 4-9 similar localpatterns. Therefore, assuming Si > S2 > S3 > ... , we candetermine the largest one as

S1 min{image width, image height} (8)a

where a > 0, and always assigned to be 4-9. Upon this, otherscales can be determined as

Si = Si-, /2 (i = 2,3,...) (9)Each scale should be odd-valued to make the correspondinglocal window has a unique center pixel. Three scales issufficient for usual images, while, for some large images(larger than 640x480), four or even more scales are needed.As an instance, the scale series selected to analyze an imageof 352x288 is{33,17,9}.

After determination of the scale series, we can use

Ew (x, y) to analyze local textures on each scale. If

Ew (x, y) is fairly small (smaller than a pre-definedthreshold) on a scale Si, we can claim pixel (x, y) to be in atextured region; otherwise, EW(x,y) on S5-,,...,Sl shouldbe further considered. If Ew(x,y) is large on all scales,then we can assert that (x, y) does not locates in anytextured region.

3. THE ISBEC ALGORITHM

The working process of ISBEC algorithm is shown in Fig.2,which is very similar to that of JSEG proposed by Deng et al[5]. The fundamental difference between these methods liesin the texture description used: JSEG uses a J-metric whileISBEC uses the CDE measure proposed above.

Firstly, the input image is quantized in the CIE LUVspace to a K-color indexed one with a two-stage method weproposed in [7]. Secondly, resulted color index map is usedto perform color segmentation and texture analysissimultaneously. For color segmentation, region growing isdirectly applied to the index map. This process produces an

over-segmentation mask. Meanwhile, the color index map istaken as the input of a multi-scale texture analysis procedure,whose steps are given detailedly in Fig.2 (on the right). Thefirst step is to determine the scale series according to formula(8) and (9). Then, on each scale, calculate Ew (x, y) foreach pixel (x, y), which produces a CDE map of the input

image. Region growing is adopted to detect textured regionsin the CDE map. The last step is to combine results ondifferent scales: only pixels rejected to be textured on allscales will be treated as non-textured ones in the final results.Thirdly, after separate color and texture segmentation, theirresults are combined by merging together color regionsoverlapping with an identical textured one. Finally, regionsin the combined results are further merged. This processcontains two steps: (L) Non-textured regions are mergedaccording to color similarity (measured by SED of colorvectors). (2)Regions (textured or non-textured) are mergedaccording to their histogram similarity. x2 histogramdifference operator [8] is adopted when comparing twohistograms, which is defined as

x2 (g, h) =IIE (gi -h, )2(10)

Color Index Map

Post processing.combinbe N resulted textureld region maps

Segmentation Results

Fig.2. Flow-chart of the ISBEC algorithm: the left is the overviewflow of ISBEC, and the right is the detailed process of multi-scaletexture analysis based on the proposed CDE measure.

When performing textured region growing, twothresholds are needed. The first one, denoted as T,, is used toselect region seeds. IfEw (x, y) < T , then we take (x, y) asa seed pixel. The second one, denoted as tg, is used in thegrowing process to decide whether the neighbors of anin-region pixel should also be absorbed by the same region.Denoting the maximum and minimum of evenness values ona particular scale asEmax and Emi , the seed determinationthreshold on that scale can be specified as

T Emin + (Emax Emin) (11)

where 0 < a < 1 . With respect to tg, it is evident thatneighboring pixels always have different CDE values. Buttheir absolute difference, denoted as |lE , should berelatively small if they are in the same textured region.

1657

Moreover, the relative CDE difference between twoneighboring pixels, defined asIAE/E,, depends on the sizeofW, where Ew is the CDE of either pixel. The larger Wis, the smaller proportion of points in the window will bereplaced when shifting it at one pixel step, thus the smallerthe relative difference will be. Discovered this relation, wecan choose the textured region growing threshold to be

Tg==,(E, x-Emin )/S (12)

S is the scale, and ,6 is a positive constant. In ourimplementation, a is chosen to be 0.008 and,8 is set to be0.2 empirically.

Fig.3. Detailed segmentation results of "hand" image obtained withthe proposed ISBEC algorithm.

4. EXPERIMENTAL RESULTS AND CONCLUSIONS

We stitch together the samples given in Fig. 1 (a) to form a384 X 128 image, and perform color quantization on theresulted image with K=4. Then we compute its CDE mapson scale 11 and 23. In Fig. 1, (b) is the quantized image; (c)and (d) are the two CDE maps. These results show that alltextured regions will display the property of evendistribution of colors when the scale is large enough,while boundaries do not have this property. Thus, CDEcan be used to detect textured regions.

Fig.3 gives the detailed ISBEC segmentation results ofthe "hand" image (in size 243 X 303). From left to right andtop to bottom are the original image, color quantized one(K=10), three CDE maps on different scales (27, 13, 7) andthe final segmentation. Segmentation results of another threeimages are presented in the left column of Fig.4. Theseimages are randomly selected from the BerkeleySegmentation Datasets (used by [8], is abbreviated to BSD).We compare ISBEC with human segmentation (also given in

BSD) and JSEG [5], whose results are separately displayedin the middle and right column of Fig.4. The reason whyJSEG is chosen as a reference method for comparison is thatit is also a color-texture one, and the working processes, asstated before, of these two methods are very analogous,which is helpful for evaluating the power of the proposedCDE measure. Fig.4 shows that, in most cases, ISBECobtains segmentation closer to the human results than JSEG.

5. REFERENCES

[1] P. Nammalwar, 0. Ghita, P.F. Whelan, "Integration of featuredistributions for colour texture segmentation", IEEE Proc. ofICPR-2004, Vol. 1, Aug. 2004[2] J.Q. Chen, T.N. Pappas, A. Mojsilovic, B. Rogowitz,"Adaptive image segmentation based on color and texture", Proc. ofICIP-2002, Vol. 3, April 2002., pp777-780[3] X. Munoz, J. Freixenet, X. Cufi, et al. "Active Regions forColour Texture Segmentation Integrating Region and BoundaryInformation" , Proc. ofICIP-2003, Vol. 2, 2003, pp.453-456[4] I. Vanhamel, A. Katartzis, H. Sahli, "Hierarchical Seg-mentation via a Diffusion Scheme in Color Texture Feature Space",Proc. ofICIP-2003, Vol. 1, 2003, pp.969-972[5] Y.N. Deng, B.S. Manjunath, "Unsupervised Segmentation ofColor-Texture Regions in Images and Video", IEEE T PAMI,Vol.23, No.8, Aug. 2001, pp. 800-810[6] R. Jain, R. Kasturi, B.G. Schunck. Machine Vision. TheMcGraw-Hill Companies, Inc. 1995, 234-246[7] X. Zhang, Z.M. Song, et al. "Color Quantization of Digitalimages". in Proc. ofPCM-2005, LNCS 3768, 2005, 653-664[8] D.R. Martin, C.C. Fowlkes, J. Malik, "Learning to DetectNatural Image Boundaries Using Local Brightness, Color, andTexture Cues", IEEE T PAMI, Vol. 26, No.5, 2004.5, pp. 530-549

Fig.4. The left column is results of ISBEC, the middle column ishuman segmentation results, and the right column is JSEG results.

1658