Improved Skin Detection Based on Dynamic Threshold Using Multi-Colour Space - IsBAST 2014

7/17/2019 Improved Skin Detection Based on Dynamic Threshold Using Multi-Colour Space - IsBAST 2014

http://slidepdf.com/reader/full/improved-skin-detection-based-on-dynamic-threshold-using-multi-colour-space 1/5

2014 International Symposium on Biometrics and Security Technologies (ISBAST)

978-1-4799-6444-4/14/$31.00 ©2014 IEEE

Improved Skin Detection based on DynamicThreshold using Multi-Colour Space

Mohd. Zamri Osman, Mohd Aizaini Maarof, Mohd Foad Rohani

Information Assurance & Security Research Group (IASRG)Faculty of Computing, Universiti Teknologi Malaysia

81310 Skudai, Johor [email protected],{aizaini, foad}@utm.my

Abstract — Skin colour detection is widely used in

applications such as adult image filtering, steganography,

content-based image retrieval (CBIR), face tracking, face

recognition, and facial surgery. Recently, researchers are more

interested in developing high level skin detection strategy forstill images based on online sample learning approach which

requires no offline training dataset. Previous dynamic skin

color detection works has shown high true positive result than

the static skin detection in term of skin-like colour and

ethnicity factors. However, dynamic skin colour detection also

produced high false positives result which lowers the accuracy

of skin detection. This is due to the current approach of

elliptical mask model that is not flexible for face rotation and is

based on single colour space. Therefore, we propose dynamic

skin colour detection based on multi-colour space. The result

shows the effectiveness of the proposed method by reducing the

false positive rate from 19.6069% to 6.9887% and increased

the precision rate from 81.27% to 91.49%.

Keywords- multi-colour space, dynamic skin classification,skin detection, skin colour model

I. I NTRODUCTION

Skin detection is one or two-class classification [1] problems taken as the fundamental to improve the accuracyof targeted application which deals with skin’s existence. Inone class classification problem, skin detection separate theobject of an interest from the image, meanwhile in two-class problem the whole image is to be discriminated whether itcontains skin pixels or not. However, several factors faced by the researchers that affects the skin detection rate are skintone colour variations, skin-like objects, illumination andcamera characteristic [2]. Other applications that rely on the

skin detection accuracy are adult image filtering [3],steganography, [4] content-based image retrieval (CBIR) [5],face recognition, face tracking and facial surgery.

Human skin pixels in an image can be classified in twoways; either pixel-based or region-based. It has beenreported that pixel-based approach is most widely used dueto several advantages such less computational, robustinformation against rotation and partial occlusion [2].Meanwhile, the region-based approach takes spatial

arrangement aspect of pixels in consideration. In addition,skin detection can be categorized into two strategies whichare low level and high level strategies [6]. In low levelstrategy, training phase with large off-line skin samples is

required in order to model the skin colour. On the other handhigh level strategy requires no off-line training which theskin sample is directly obtained from the face. This strategyrefers to online skin sampling method.

Skin color modeling is used to separate the skin and non-skin pixels by building a decision rules. For instance, explicitthreshold, statistical-based method and neural network aregeneral categories of skin colour modeling [2, 6, 7]. Theexplicit threshold technique uses single or multiple fixed boundaries for each colour space. Skin pixels are the pixelsthat fall within these boundaries and others are classified asnon-skin. In non-parametric technique, a histogram for thegiven colour space is built and then converted to a

probability density function (PDF). If the PDF of a given pixels exceeds a predefined threshold it is considered as skin.On the other hand, parametric techniques used a modeledcolour space with a prescribed shape. For instance Gaussianand elliptical boundary model are parametric skin colourmodeling. In addition, neural network or semi-parametrictechnique is trained by two sets, skin and non-skin set togenerate the decision rule. Multi-layer Perceptron (MLP) andSelf Organized Map (SOP) networks are commonarchitecture used. Zaidan et al. [8] reviewed that, neuralnetwork technique unable to separate skin-like object andmade an attempts but failed to resolve the illuminationconditions. More details of the skin colour modeling can befound in [2].

Cheddad et al. [4, 9] proposed a new colour space in 1Ddistribution. The fixed threshold values are obtained from theoff-line training samples which requires huge of skinsamples to model the skin. However, this method fails tohandle various conditions such as skin-like objects andcomplex background [10]. Subban and Mishra [1] studiedtwenty-one combination of explicit skin cluster in differentcolour spaces. From the analysis done, three combinations oftwo colour spaces can perform better detection accuracy



which are YUV-YIQ, YUV-KL, and YCgCr-YES,respectively.

Several researchers combine multiple colour space to build a skin color model. Rahim et al. [11], Wang and Yuan[12] , Samart et al. [13], Zhu et al. [14] and Xiang andSuandi [15] introduced a multi-skin colour model using

different colour spaces. For instance, Xiang and Suandi [15] proposed a fusion of multi colour space for skinsegmentation using YCbCr-YUV and RGB-YUV. It is foundthat skin segmentation using RGB-YUV multi-colour spaceis better in handling complex image background. Zhu et al.[14] on the other hand introduced multi-skin colour model based on rgb-YCbCr for face detection. Meanwhile, Samartet al. [13] improved the conventional RGB-H-CbCr by proposed novel rule for face detection based on RGB-HSV-YCbCr skin colour model. However, their skin colourmodels are static and require off-line training.

For the past years, researchers are more interested on building dynamic or adaptive skin detector [6, 10, 16-18].This approach does not require offline training skin samples

and less complexity. The motivation of this approach is anonline skin samples that obtained directly from the individualhuman face. Yogarajah et al. [16] introduced a dynamic skincolour detection based on face detected from the images. Thedynamic threshold generated from the face after removed thenon-skin pixels using edge detector and dilation process.Then, 95% of confident interval margin will be accepted tomodel the dynamic threshold. However, from the experimentdone it is shows that this method produced high false positive (FP) rate. There are a lot of ‘black spot’ areasreported from the result. It is reported that in Yogarajah et al.[16] method there are many ‘black spot’ in the segmentedarea which are false positive. This method classified manyskin pixels. Even though this method improved in term of

true positives rather than the explicit static cluster, the false positive is also increased.

Current works done by researchers solely focus ondynamic skin detection in single colour space. Therefore, this paper proposes dynamic threshold based on multi-colourspace. The motivation of using multi-colour spaces is that,combining more than one colour spaces it would increase theskin detection rate compared to single colour space. Our proposed method does not require any off-line learning. Inskin colour extraction, we used online learning for a particular image using the human face skin colourinformation. We employed Viola-Jones [19] face detector,then a Support Vector Machine (SVM) to locate the eyes

coordinate. In addition, we propose elastic elliptical maskmodel according to the eyes angle. This is due to possibilityof skin pixels is collected according to the face rotation toreduce too much non-skin selected. The detected face skinregion contains possible non-skin pixels. A Sobel edgedetector and dilation process is used to remove any possibility of non-skin pixels such as hair, eyebrow tosmoothing the face skin region. Then, we convert the RGBcolour space into YCbCr and HSV colour space.

This paper is organized as follows. Section II discussesthe proposed work. Section III explains the proposeddynamic skin colour detection method. The experimental andevaluation results are discussed in Section IV. Finally, theconclusion is given in Section V.

II. THE PROPOSED WORK

As previously discussed in the Section I the proposedwork is based on multi-colour space of RGB, YCbCr andHSV. In this method, RGB, YCbCr and HSV colour spacewill be used to model the skin pixels. These colour spaceconversions can be found in the [7]. YCbCr defined inEquation 1 is an orthogonal space that commonly used by

Fig. 1. Flowchart of the proposed method

{ (1)

) ) )

√ ) ) ) )

)

)

(2)

European television studios for image compression work.It represents the luma (which is luminance, computed fromnonlinear RGB). YCbCr capable to separate luminance andchrominance that makes this colour space attractive for skincolour modeling. Smooth face skin region is converted into



these colour spaces. A summarized flowchart of the proposework is illustrated in the Fig. 1.

Fig. 1 shows the flowchart of the proposed method whichincludes several phases. The phases can be illustrated in thefollowing sub-section – online skin sampling, singledynamic threshold and dynamic skin classification. If more

than one face detected in the image, the process looped untilthere is no faces detected. Then individual results usingsingle dynamic threshold will be. This is because, each faceswill generate a unique threshold values according to skintone variations obtained from the detected faces.

d, eyes

coordinate

c

a, major

axis

b, minor

axis

center point

(x0,y0)

(a) (b) (c) (d)

(e) (f)

Fig. 2. Elastic elliptical mask model according to eye angle. C is the

rotation angle, d is the distance between two eye points, (a) and (b)respectively 1.2d major axis and 1.1d minor axis

Fig. 3. Online skin colour extraction obtained from the face. (a) input, (b)elastic elliptical mask region based on eye rotation, (c) smooth skin region

using edge detector and dilation process, (d)-(f) RGB, YCbCr and HSVsmoothed skin region

III. DYNAMIC SKIN COLOUR DETECTION

A.

Online Skin Sampling Method

Online skin sampling is the process of extracting theskin pixel directly from the detected face. Here, we proposed an elastic elliptical mask model that createdaccording to the eyes coordinate. This is because the high possibility of skin area if we rotate the elliple according tothe face angle. The elliptical shape is rotated based on the

eyes angle as shown in the Fig. 2. Parameter ‘d ’ is the distance between two eyes,

while ‘a’ and ‘b’ is the major and minor axis of the ellipsesize.

Fig. 3 shows the process of online skin sampling. Thegoal of this process is to extract possibility of skin pixelsfrom the face mask region. However, the face may containnon-skin pixels such as eyebrow, lips, or hair that need to beremoved. Sobel edge detector is used to detect the edge fromthe elastic elliptical mask region (b) and dilation process togrow the detected edges. The white pixels in the (c) representthe possibility of non-skin pixels recognized and need to beremoved. Then, a smoothed skin region is produced. Finally,we convert the existing RGB smoothed skin region toYCbCr and HSV colour space. However, the smooth skinregion may still exist non-skin pixels. We used histogramanalysis of confident interval with 95% employed for each ofthe distribution colour component to determine the acceptedregion and classified as skin pixels.

B. Single Dynamic Threshold

The purpose of this phase is to combine multi-thresholdfrom the multi-colour space of RGB, YCbCr or HSV. Here,we analyzed several combinations of colour spaces using

three colour spaces. Initially, we proposed a single dynamicthreshold values for skin detection based on RGB, YCbCrand HSV colour space. We present several combinationsYCbCr-RGB, CbCr-RGB, YCbCr-SV, and CbCr-SVrespectively. Equation 3 shows the example combination oftwo single dynamic thresholds for the CbCr-SV skin colourmodel. The lower bound will be the minimum value whileupper bound is the maximum threshold value.

(3) ) )

Multiple thresholds calculated during the online skinsampling process. This dynamic threshold values then will be used to classify the skin and non-skin pixels for stillcolour images.

C. Dynamic Skin Classification

Finally, the dynamic threshold values that obtained from

the online skin sampling method will classify the skin pixelaccording to the minimum and maximum threshold valuedefined by Equation 3. Value ‘1’ represent skin and ‘0’ non-skin pixel. If the input image contains more than one faces,the proposed method will be looped and applied forindividual faces until there is no face identified. Lastly, theindividual result of each faces will be merged to produce thefinal detection output as shown in the Fig. 4. White and



black colour respectively represents skin and non-skin pixelarea.

Face 3

Face 1

Face 2

Face 4

Merging

operation

(a)

(b)

(c)

Fig. 4. Merged result for more than one faces. (a) Input image, (b) individual threshold result, (c) final result

Fig. 5. Qualitative comparison using Pratheepan dataset [16] of single face. From left to right represents the input image, ground truth, Yogarajah et al .method [16], Tan et al . method [17] and our proposed method

IV.

EXPERIMENT AND EVALUATION In this section, the performance of the proposed method

applied on the different image conditions, skin tones, colourspaces are compared with the state-of-the-art works. As forour study, only frontal face is considered for evaluation. We perform quantitative analysis under different combination ofcolour space and qualitative analysis on the Pratheepandataset [16]. Since the ground truth is not available, thereforewe provide the ground truth for Pratheepan dataset and it can be access at [20].

From the analysis done, it shows that combination of

YCbCr-SV provides better performance compared to theother three combined colour space and even single colourspace. YCbCr-SV generate less false positive. In addition,high precision and accuracy demonstrated. On the otherhand, qualitative evaluation is presented in Figure 5. Forqualitative evaluation purpose, we compare the result of our proposed method with Yogarajah et al . [16] and Tan et al .[17]. As shown in the Figure 5, Yogarajah et al . [16] resultgenerates a lot of non-skin pixels. Tan et al . [17] generate better result rather than Yogarajah, but still have much non-



skin pixels. Last column is the result from our proposedmethod. We managed to generate less false positive result byclassified less non-skin pixels. Table 1 shows thequantitative comparison of the proposed method based onseveral combination of colour space.

TABLE I. COMPARISON RESULT USING DIFFERENT COLOUR SPACES

Skin Colour Model FP (%)Accuracy

(%)Precision

(%)

YCbCr-SV 6.9887 84.05 91.49

CbCr-SV 6.9984 83.48 91.48

YCbCr-RGB 8.3235 83.48 90.04

CbCr-RGB 8.8040 83.81 89.67

CbCr 14.8803 85.86 85.45

RGB 17.6375 83.32 85.33

HSV 19.6069 81.57 81.27

We analyzed three combinations of colour spaces withsingle colour space. Based on the results, it clearly showsthat the false positive rate can be reduced significantly from

19.6069% to 6.9887 by combining multi-colour space into asingle skin colour model. Result in Table 1 also shows thatYCbCr-SV, CbCr-SV and YCbCr-RGB perform well thansingle colour space. Among the combined colour space,YCbCr-SV generates less false positive result, highaccuracy and high precision. However, our proposedmethod sometime fails in handling image that contains toomany faces. This is because of the proposed method is onlydepending on the performance of face detector and this problem also faced by other researchers that studies in thisdomain. False face detection will result in wrong thresholdvalue and finally will produce poor detected skin regions.

V.

CONCLUSION

This paper proposes an improvement on the dynamicskin detection based on multi-colour space which reducedthe false positive rate in the detection of skin region. We also proposed an elastic elliptical mask model based on the eyeangle. From the experimental results, we managed to reducethe false positive compared to the previous dynamic skindetection method. The improved method also increased the precision rate compared to the single skin colour modelusing the multi-colour space.

ACKNOWLEDGMENT

The authors would like to acknowledge Ministry ofHigher Education (MOHE) and Universiti Teknologi

Malaysia (UTM) for supporting this research under ResearchUniversity Grant (RUG) vote 10J28 and Science Fund Grant(MOSTI) vote 01-01-06-SF1167. We would also like tothanks to Dr. See Seng Chan from Universiti Malaya,Malaysia for giving an access to their dataset.

R EFERENCES

[1] R. Subban and R. Mishra, "Combining Color Spaces for Human SkinDetection in Color Images using Skin Cluster Classifier," in Int. Conf.

on Advances in Recent Technologies in Electrical and Electronics ,2013, pp. 68-73.

[2] P. Kakumanu, S. Makrogiannis, and N. Bourbakis, "A survey of skin-color modeling and detection methods," Pattern Recognition, vol. 40,

pp. 1106-1122, 2007.[3] L. Jiann-Shu, K. Yung-Ming, and C. Pau-choo, "The Adult Image

Identification Based on Online Sampling," in Neural Networks, 2006.

IJCNN '06. International Joint Conference on, 2006, pp. 2566-2571.

[4] A. Cheddad, J. Condell, K. Curran, and P. Mc Kevitt, "A skin tonedetection algorithm for an adaptive approach to steganography," J.

Signal Processing, vol. 89, pp. 2465-2478, 2009.[5] M. A. Mofaddel and S. Sadek, "Adult image content filtering: A

statistical method based on Multi-Color Skin Modeling," in Computer

Technology and Development (ICCTD), 2010 2nd InternationalConference on, 2010, pp. 682-686.

[6] S. Bianco, F. Gasparini, and R. Schettini, "Computational Strategiesfor Skin Detection," in Computational Color Imaging . vol. 7786, S.Tominaga, R. Schettini, and A. Trémeau, Eds., ed: Springer BerlinHeidelberg, 2013, pp. 199-211.

[7] V. Vezhnevets, V. Sazonov, and A. Andreeva, "A Survey on Pixel-Based Skin Color Detection Techniques," in Proceedings of the

GraphiCon 2003, 2003, pp. 85-92.[8] A. A. Zaidan, B. Zafarifar, H. A. Karim, N. N. Ahmad, Z. B.B., and

A. Sali, "An Automated Anti-Pornography System Using A SkinDetector Based on Artificial Intelligence: A Review," International

Journal of Pattern Recognition and Artificial Intell igence, vol. 27, p.1350012, 2013.

[9] A. Cheddad, J. Condell, K. Curran, and P. Mc Kevitt, "A new colorspace for skin tone detection," presented at the IEEE InternationalConference on Image Processing (ICIP), 2009.

[10] N. B. Ibrahim, M. M. Selim, and H. H. Zayed, "A dynamic skindetector based on face skin tone color," in 8th International

Conference on Informatics and Systems (INFOS), 2012, pp. 1-5.[11] N. A. Abdul Rahim, C. W. Kit, and J. See, "RGB-H-CbCr Skin

Colour Model for Human Face Detection," in MMU International

Symposium on Information & Communications Technologies

(M2USIC), Petaling Jaya, Malaysia, 2006.[12] Y. Wang and B. Yuan, "A novel approach for human face detection

from color images under complex background," Pattern Recognition,

vol. 34, pp. 1983-1992, 2001.[13] N. Samart, S. Chiechanwattana, and K. Sunat, "A Novel Rule for

Face Region Detection Based on RGB-HSV-YCbCr Skin Model," inThe 3 rd International Conference on Science and Technology forSustainable Development of the Greater Mekong Sub-region

(STGMS), Lao PDR, 2011, pp. 330-337.[14] Y. Zhu, C. Huang, and J. Chen, "Face detection method based on

multi-feature fusion in YCbCr color space," in Image and Signal

Processing (CISP), 2012 5th International Congress on, 2012, pp.1249-1252.

[15] F. H. Xiang and S. A. Suandi, "Fusion of Multi Color Space forHuman Skin Region Segmentation," International Journal of

Information and Electronics Engineering, vol. 3, pp. 172-174, 2013.[16] P. Yogarajah, J. Condell, K. Curran, A. Cheddad, and P. McKevitt,

"A dynamic threshold approach for skin segmentation in colorimages," in Proc. IEEE ICIP , Hong Kong, 2010, pp. 2225-2228.

[17] W. R. Tan, C. S. Chan, P. Yogarajah, and J. Condell, "A FusionApproach for Efficient Human Skin Detection," Industrial

Informatics, IEEE Transactions on, vol. 8, pp. 138-147, 2012.

[18] I. Hwang, S. H. Lee, B. Min, and N. I. Cho, "Luminance AdaptedSkin Color Modeling for the robust detection skin areas," presented atthe International Conferenc on Image Processing (ICIP), 2013.

[19] P. Viola and M. J. Jones, "Robust Real-Time Face Detection," Int. J.

Comput. Vision, vol. 57, pp. 137-154, 2004.[20] C. Chee Seng. (2014, 12 February). Human Skin Detection Dataset .

Available: http://web.fsktm.um.edu.my/~cschan/datasets.html

Documents

Improved Skin Detection Based on Dynamic Threshold Using Multi-Colour Space - IsBAST 2014