C o l o r image segmentation – an innovative approach

May 2003SUT

CCoolloorr image segmentation image segmentation – – an innovative approachan innovative approach

Amin FazelAmin Fazel

May 2003May 2003

Sharif University of Sharif University of TechnologyTechnology

Course Presentationbase on a paper by Tie Qi Chen, Yi Lu

Page 2 of 36

May 2003SUTMachine Vision Course Presentation

Image segmentationImage segmentation

DefinitionDefinition– Process of partitioning image pixel based on

selected image features– Pixel of same region spatially connected and

have similar image feature– Level of subdivision depends on the problem– Segmentation should stop when the objects

of interest have been isolated

Page 3 of 36


Image segmentationImage segmentation

ApplicationsApplications– Image analysis– Machine vision– Target acquisition– Object recognition– …

Page 4 of 36


CCoolloorr image image segmentationsegmentation

DefinitionDefinition– Here selected segmentation feature is color

– This process group pixels into same region that

Spatially connected Have similar color feature

Page 5 of 36


Unsupervised cUnsupervised coolloorr image image segmentationsegmentation

DefinitionDefinition– If below knowledge is not available

Number of regions present in the image Type of region present in the image

Page 6 of 36


Choosing a suitable ccoolloorr space

Common color spaces Common color spaces – RGB, HSI, YIQ, CMY

Benefits of L*u*v* color spaceBenefits of L*u*v* color space1. covers the whole of the visible gamut of colors 2. the difference between two colors can be measured

by their Euclidean distance3. additive mixture of two colors

lies on the line joining them.4. decreases the chance that

any given step in color value will be noticeable on a display

Page 7 of 36


Choosing a suitable ccoolloorr space

L*u*v* parametersL*u*v* parameters– L* is intensity (lightness) :

0 to 100

– u* is redness-greenness : -127 to 128

– v* is yellowness-blueness : -127 to 128

Converting formulaConverting formula– by CIE standard formula

RGB and L*u*v* color space

Page 8 of 36


CCoolloor r histogram histogram

DefinitionDefinition– Three dimensional (3D) discrete feature

space– Provide the color distribution of the image– Is obtained by discretizing the colors in the

color space counting the number of times each discrete color occurs in the image

Page 9 of 36


CCoolloor r histogramhistogram

Example :Example :

Page 10 of 36


Overview of the systemOverview of the system

The system consists of two stagesThe system consists of two stages1. Fuzzy clustering algorithm to generate

clusters of similar colors Using a color histogram of an image The output of the fuzzy clustering algorithm

– Set of non-overlapping color clusters, CL1

Each cluster in CL1 contain similar color All colors in the same cluster are assigned

with the same color label

Page 11 of 36


Overview of the systemOverview of the system

2. Region segmentation algorithm agglomerates the initial clusters based on Spatial connection & Color distance between the adjacent regions

The second set of clusters, CL2, is obtained by labeling image pixels with the corresponding color clusters in CL1

Therefore , |CL2| >> |CL1|

In this stage merges the selected adjacent regions

Page 12 of 36


A A ccoolloorr image image segmentation segmentation systemsystem

Stage 1:ccoolloorr segmentation

Fuzzy clusteringFuzzy clusteringin in ccoolloorr

histogram domainhistogram domain

ccoolloorr Image

Compute Compute HistogramHistogram

In a In a ccoolloorr space space

Map initialMap initialclusters to clusters to

image domainimage domain

CL3:a set ofccoolloorr region

MergingMergingneighboringneighboring

clustersclusters

CL1a color histogram CL2

Stage 2:Region segmentation

Page 13 of 36


A fuzzy clustering A fuzzy clustering algorithmalgorithm

Applying fuzzy logic to color clusteringApplying fuzzy logic to color clustering– Consider a cluster of similar colors as a

fuzzy set

– Represent the likeliness of a color pixel belonging to a fuzzy set by a fuzzy membership function

Page 14 of 36



Two critical issues involved in a fuzzy Two critical issues involved in a fuzzy clustering algorithmclustering algorithm– Generating fuzzy membership function

– Defining a color distance function between two color clusters and a distance function between a color and a color cluster

Page 15 of 36



Fuzzy membership functionFuzzy membership function– Let be the set of possible colors in the image

– Use Gaussian function to define the probability of a color C belonging to a color cluster

– P is the center of the cluster

– R is the radius of the cluster

– ||-|| denote the Euclidean distance between a color and a cluster

22/)( Rpc

R ePCG

nC

Page 16 of 36



……Fuzzy membership functionFuzzy membership function– The probability of a color belonging to the

k-th cluster and not belonging to any other cluster

– M is the number of clusters

– is used as a fuzzy membership function for color k in the color space

,)](1[)(),,;( 1

ki

iRkRMk PCGPCGPPCH

kH

Page 17 of 36



Fuzzy membership functionFuzzy membership function

Page 18 of 36


A fuzzy clustering A fuzzy clustering algorithmalgorithm Important characteristics of membership functionImportant characteristics of membership function

– Belief value decrease when distance between a color C and a color cluster P increase

– Suppresses the belief value of a color to a cluster when it is close to the other clusters

Prevent two clusters moving towards each other during the optimization process

– The belief value of a color belonging to a cluster is always greater than zero

Page 19 of 36



Measure of goodness of fitMeasure of goodness of fit– Express how well a given n-cluster

description matches a given set of data

– Objective function (mean square error)

The colors near the border of each cluster give large contribution to the mean square error

M

k CkiMikiM

i

PCPPCHCfPP1

2

11 ).,,;().(),,(

Page 20 of 36



AlgorithmAlgorithm First cluster is generated by finding such

that

– becomes the initial center of cluster 1, i.e.

The center of the first cluster is optimized through the following iteration until

i

i

C

tMMiMi

C

tMMiMii

tM PPPCHCf

PPPCHCfC

P),,,;().(

),,,;().(.

11

111

1t

Mt

M PP

NjCfCf ik ,,1),()( )( kCf

)(0kM CfP

Page 21 of 36



… … AlgorithmAlgorithm For M > 1, the initial center of cluster , is set

to such that– Function V is the probability of a color not belonging

to any existing cluster

This new cluster is optimized using the previous iterative procedure

The algorithm stops generating a new cluster M when

),,;().(),,;().( 11 MjjMkk PPCVCfPPCVCf

0MP

kC

M

KkRM PCGPPCV

11 )(1),,;(

kC

kMkk CfPPCVCf )(),,;().( 1

Page 22 of 36



– Effect of objective function

used in the cluster generation

Page 23 of 36


A fuzzy clustering A fuzzy clustering algorithmalgorithmThe only parameters that need to be The only parameters that need to be

evaluated areevaluated are– R, the cluster radius– , the distance between the cluster generated in

the previous iteration and the current iteration– , the stop threshold of cluster generation

Parameters and have less effect on the clustering result

The most critical parameter is R

Page 24 of 36


A fuzzy clustering A fuzzy clustering algorithmalgorithm More on Cluster Radius More on Cluster Radius

– Determines how much the clusters can overlap with each other in the histogram domain

– This parameter is provided by the user

– For image that have coarse feature a large R is recommended

– A smaller R is a good choice for image with fine detailed color feature

Page 25 of 36



Effects of the parameter Effects of the parameter RR=64

R=32

R=16

R=8

Page 26 of 36


Image segmentation in image Image segmentation in image domaindomain

At this stage system map the clusters in CL1 to the image domain to obtain CL2

Each cluster in CL2 contains pixels that are– Spatially connected

– Belong to the same color cluster in CL1

– Region segmentation uses following parameters The color distance among neighboring clusters in the

spatial domain Cluster size The maximum number of clusters in CL3

Page 27 of 36



Investigation of clustering merging Investigation of clustering merging methodsmethods

These methods use a common parameter, max_cls, to control the max number of clusters

1. Attempts to merge the adjacent clusters that are similar in colors

– This algorithm use a control parameter to denote color difference threshold

Page 28 of 36



2. Considers the size of clusters as the only selection criterion

– It selects the smallest clusters and merges the clusters with one of its neighbors to witch It has the smallest color distance

3. Considers the color distance as the more important criterion in cluster merging

– This algorithm consists of three passes of merging

Page 29 of 36



3. …continueI. The algorithm repeatedly merges the

smallest clusters with their neighbors that have the closet color distance

II. The algorithm selects a pair of two adjacent clusters that has the smallest color distance within the entire image to merge

III. The algorithm repeatedly merges the smallest cluster with its closest neighbors in color distance

Page 30 of 36



Comparison of clustering result generated by tree Comparison of clustering result generated by tree different spatial merging methoddifferent spatial merging method

a) An egg nebula image

b) Clusters generated by the fuzzy clustering algorithm

c) Clustering result by method 1

d) Clustering result by method 2

e) Clustering result by method 3

Page 31 of 36



computing the color distance between two computing the color distance between two neighboring clusters neighboring clusters A and and B

1. The first function is based on the color difference of the border pixels of clusters A and B.

– – where

,)(_)(_),(_ BBAveABAveBADistB

),(Border

),(,

),(Border

),(

,),(Border

),()(_

),(Border),(),(Border),(

),(Border),(

BA

yxf

BA

yxf

BA

yxfABAve

BAyx BBAyx G

BAyx R

Page 32 of 36



– and

– Where and are the minimum and maximum coordinates of all pixels in A that have direct neighbors in B

– Similarly,

Ayyy

xxxyxBABorder

y)(x, and ;max_min_

max,_min_|),(),(

max_xmin_xx

),(Border

),(,

),(Border

),(

,),(Border

),()(_

),(Border),(),(Border),(

),(Border),(

AB

yxf

AB

yxf

AB

yxfBBAve

AByx BAByx G

AByx R

Page 33 of 36



Illustration of border points between region Illustration of border points between region A and and B

– Border (A, B) contain the yellow

points within the red bounding box

– Border (B, A) contain the blue

points within the red bounding box

Page 34 of 36



2. The second color distance function is based on the central color of a cluster defined as:

–

– where p is a pixel € A– |A| is the size of A– C(p) is the 3-D color vector of pixel p in L*u*v space

the color distance of two clusters is measured using the Euclidean distance between their central color vector

A

pCC Ap

A

)(~

Page 35 of 36


Results of Results of implementationimplementation Clustering resultClustering result

a) The original imageb) 4 clusters generated by the fuzzy clustering algorithmc) 4 clusters generated by the segmentation algorithm in

image domain

Page 36 of 36


Results of Results of implementationimplementation An example of applying the image segmentation An example of applying the image segmentation

system to a car imagesystem to a car image

a) The original imageb) The image 12 color clusters generated by the fuzzy clustering

algorithm and 598 spatial clusters in the image domainc) The segmentation result

Page 37 of 36


Results of Results of implementationimplementation Image segmentation result on two face imagesImage segmentation result on two face images

Setting the cluster radius parameter to R=8, 16, 32 and 64

May 2003SUT

CCoolloorr image segmentation image segmentation – – an innovative approachan innovative approach

Course PresentationBased on a paper by Tie Qi Chen, Yi Lu

Thanks For Your AttentionThanks For Your AttentionThanks For Your AttentionThanks For Your Attention

The End

Documents

C o l o r image segmentation – an innovative approach