Research Article Efficient Detection of Occlusion …downloads.hindawi.com/journals/tswj/2014/519158.pdfface database [ ] and obtained the best results. Our exper-iments also suggested

Research ArticleEfficient Detection of Occlusion prior toRobust Face Recognition

Rui Min1 Abdenour Hadid2 and Jean-Luc Dugelay1

1 Department of Multimedia Communications EURECOM 450 Route des Chappes 06410 Biot France2 Center for Machine Vision Research Department of Computer Science and EngineeringUniversity of Oulu PO Box 4500 90014 Oulu Finland

Correspondence should be addressed to Rui Min mineurecomfr

Received 26 August 2013 Accepted 7 October 2013 Published 16 January 2014

Academic Editors S Berretti S Hong and T Yamasaki

Copyright copy 2014 Rui Min et alThis is an open access article distributed under the Creative Commons Attribution License whichpermits unrestricted use distribution and reproduction in any medium provided the original work is properly cited

While there has been an enormous amount of research on face recognition under poseilluminationexpression changes and imagedegradations problems caused by occlusions attracted relatively less attention Facial occlusions due for example to sunglasseshatcap scarf and beard can significantly deteriorate performances of face recognition systems in uncontrolled environments suchas video surveillance The goal of this paper is to explore face recognition in the presence of partial occlusions with emphasis onreal-world scenarios (eg sunglasses and scarf) In this paper we propose an efficient approach which consists of first analysingthe presence of potential occlusion on a face and then conducting face recognition on the nonoccluded facial regions based onselective local Gabor binary patterns Experiments demonstrate that the proposed method outperforms the state-of-the-art worksincluding KLD-LGBPHS S-LNMF OA-LBP and RSC Furthermore performances of the proposed approach are evaluated underillumination and extreme facial expression changes provide also significant results

1 Introduction

Face recognition [1] the least intrusive biometric techniquein terms of acquisition has been applied to a wide rangeof commercial and law enforcement applications State-of-the-art face recognition systems perform with high accuracyunder controlled environments but performances drasticallydecrease in practical conditions such as video surveillanceof crowded environments or large camera networks Themain problems are due to changes in facial expressionsillumination conditions face pose variations and presenceof occlusions With emphasis on real-world scenarios in thelast decade problems related to poseilluminationexpressionchanges and image degradations have been widely inves-tigated in the literature In contrast problems caused byocclusions received relatively less investigations althoughfacial occlusion is quite common in real-world applicationsespecially when individuals are not cooperative with thesystem such as in video surveillance applications

Facial occlusions may occur for several intentional orundeliberate reasons (see Figure 1) For example facial acces-sories like sunglasses scarf facial make-up and hatcapare quite common in daily life Medical mask hard hatand helmet are required in many restricted environments(eg hospital and construction areas) Some other peopledo wear veils for religious convictions or cultural habits Inaddition facial occlusions are often related to several severesecurity issues Football hooligans and ATM criminals tendto wear scarves andor sunglasses to prevent their faces frombeing recognized Bank robbers and shop thieves usuallywear a cap when entering places where they commit illegalactions

Because partial occlusions can greatly change the originalappearance of a face image it can significantly deteriorateperformances of classical face recognition systems (suchas [2ndash4] since the face representations are thus largelydistorted) To control partial occlusion is a critical issueto achieve robust face recognition Most of the literature

Hindawi Publishing Corporatione Scientific World JournalVolume 2014 Article ID 519158 10 pageshttpdxdoiorg1011552014519158

2 The Scientific World Journal

(a) (b)

Figure 1 Illustration of different types of facial occlusions (a) ordinary facial occlusions in daily life (b) facial occlusions related to severesecurity issues (ATM crimes football hooligans etc)

works [5ndash17] focus on finding corruption-tolerant featuresor classifiers to reduce the effect of partial occlusions inface representation However information from the occludedparts can still hinder the recognition performance Recentlyresearchers [18ndash21] demonstrated that prior knowledge aboutthe occlusion (eg type location and size) can be used toexclude the information from occluded parts so as to greatlyimprove the recognition rate Hence explicit occlusion anal-ysis is an important step in occlusion-robust face recognitionIn this paper we propose an occlusion analysis methodto improve local Gabor binary pattern based face recog-nition [11] which outperforms literature works including[17ndash21]

The proposed approach consists of first detecting andsegmenting occluded parts (eg sunglassesscarves) and thenapplying face recognition on the nonoccluded facial regionsTo do so the presence of occlusion is first analysed in thepatch-level using Gabor wavelets PCA and SVM Then wesegment the occluded part more precisely from the otherfacial regions by a generalized Potts model Markov randomfield (GPM-MRF) [22]This allows us to identify the presenceof occlusion at the pixel-level so as to preserve as muchas possible face information for the recognition After thecomputation of an occlusion mask indicating which pixelin a face image is occluded we propose a variant of localGabor binary pattern histogram sequences (LGBPHS) [11]to efficiently represent occluded faces by excluding featuresextracted from the occluded pixels Finally we compared ourapproach with traditional approaches [2 4 11] our previousresults [21] and state-of-the-art methods [13 19 20] on ARface database [23] and obtained the best results Our exper-iments also suggested that in comparison with weightingbased method [20] occlusion exclusion (ie weighting as 0or 1) is more appropriate to handle the occlusion problem inface recognition

The rest of this paper is structured as follows First wereview the related works in Section 2 Then the proposedapproach is described in Section 3 Section 4 presents theexperimental results and analysis Finally we draw the con-clusion and discuss future directions in Section 5

2 Related Works

The traditional methodology to address face recognitionunder occlusion is to find corruption-tolerant features or clas-sifiers Toward this goal numerous previousworks confirmedthat locally emphasized algorithms are less sensitive to partialocclusions Penev and Atick [5] proposed the local featureanalysis (LFA) to extract local features by second order statis-tics Martınez [6] proposed a probabilistic approach (AMM)which can compensate for partially occluded faces Tan etal [7] extended Martinezrsquos work by using the self-organizingmap (SOM) to learn the subspace instead of using themixtureof Gaussians In [8] Kim et al proposed a method namedlocally salient ICA (LS-ICA) which only employs locallysalient information in constructing ICA basis In [9] Fidleret al presented a method which combines the reconstructiveand discriminativemodels by constructing a basis containingthe complete discriminative information Park et al [10]proposed to use a line feature based face attributed relationalgraph (ARG) model which encodes the whole geometricstructure information and local features of a face Zhanget al [11] proposed a nonstatistical face representationmdashlocal gabor binary pattern histogram sequence (LGBPHS)to exploit the multiresolution and multiorientation Gabordecomposition In [12] Jia and Martinez proposed the use ofpartial support vector machines (PSVM) in scenarios whereocclusions may occur in both the training and testing sets

More recently facial occlusion handling under the sparserepresentation based classification (SRC) [13] framework hasdemonstrated impressive performances in face recognitionwith occlusions The idea of using SRC for occluded facerecognition is first introduced by Wright et al [13] where anoccluded face is represented as a linear combination of thewhole face gallery added by a vector of errors (occlusion)in the pixel-level and the classification is achieved by L1minimization Zhou et al [14] extend [13] by including aMarkov Random Fields (MRF) model to enforce spatialcontinuity for the additive error vector to address contiguousocclusions In [15] Yang and Zhang applied compressibleimage Gabor features instead of original image pixels as thefeature vector used in SRC to reduce computations in the

The Scientific World Journal 3

Table 1 Summary of literature works in occluded face recognition

Category Abbreviation Full namebrief description

Locality emphasizedfeaturesclassifiers

LFA [5] Local feature analysisAMM [6] Gaussian mixture modelling of part-based Eigenface

SOM-AMM [7] Self-organizing map modelling of part-based EigenfaceLS-ICA [8] Local salient-independent component analysis

RD-Subspace [9] Combining reconstructive and discriminative subspaceARG [10] Attributed relational graph

LGBPHS [11] Local Gabor binary patterns histogram sequencePSVM [12] Partial support vector machines

SRC based methods

SRC [13] Sparse representation based classificationMRF-SRC [14] Markov random field to enforce spatial continuity in SRCGabor-SRC [15] Compressible Gabor feature used in SRCSIFT-SRC [16] SIFT feature used in SRC

RSC [17] Robust sparse coding

Explicit occlusion analysisfacilitated localfeaturelocal componentbased methods

Part-PCA [18] Occlusion analysis + part-based EigenfaceS-LNMF [19] Selective local nonnegative matrix factorization

KLD-LGBPHS [20] Local Gabor binary patterns based on Kullback-Leibler divergenceOA-LBP [21] Occlusion analysis + LBP (our preliminary work)

presence of occlusions Liao and Jain [16] incorporated theSIFTdescriptor into the SRC framework to achieve alignmentfree identification Yang et al [17] proposed a robust sparsecoding (RSC) method which seeks the maximum likelihoodestimation (MLE) solution of the sparse coding problem fornon-GaussianLaplacian occlusions in an iterative mannerEven though the SRC based methods achieve significantidentification results on occluded faces from standard facedatabases (ie AR face database [23]) the prerequisite ofthose methods relies on the large number of training samplesof each identity with sufficient variations But in manypractical face recognition scenarios the training samplesof each subject are often insufficient (the ldquocurse of thedimensionalityrdquo [24] problem in the extreme case only onetemplate face per subject is available)

Lately a few works have revealed that prior knowledgeof occlusions can significantly improve the accuracy of localfeaturelocal component based face recognition Rama et al[18] empirically showed that prior knowledge about occlusion(manually annotated) can improve Eigenface in local patchesIn [19] Oh et al have proposed an algorithm based onlocal nonnegative matrix factorization (LNMF) [25] namedselective LNMF (S-LNMF) that automatically detects thepresence of occlusion in local patches face matching isthen performed by selecting LNMF representation in thenonoccluded patches Zhang et al [20] proposed to useKullback-Leibler divergence (KLD) to estimate the proba-bility distribution of occlusions in the feature space so asto improve the standard LGBPHS based method [11] forpartially occluded face In our preliminary study [21] wealso demonstrated that explicit occlusion analysis can greatlyimprove LBP based face recognition In these studies [18 1921] discard all information from the components which are

occluded whereas [20] assigns a weight (between 0 and 1) toeach component In this paper we consider the first case asocclusion exclusion and the later one as occlusion weighting(note that occlusion exclusion can be regarded as a specialcase of occlusion weighting where the weights are either 0 or1) Because many of the algorithms we have discussed so farwill be extensively analysed and compared in the experimentssection we summarize and categorize the literature works inTable 1 (forwhich abbreviationswill be used in later sections)

Based on our preliminary work [21] in this paper we pro-pose a complete and fully automatic framework to improveface recognition in the presence of partial occlusions Besidesthe occlusion detection module (which was introduced in[21]) which can detect the presence of occlusion in patch-level we adoptedGPM-MRF to detect occlusion in pixel-levelto facilitate later recognition We then propose a customizedcorruption-tolerant local descriptor selective LGBPHSwhichsummarizes features from nonoccluded pixels for efficientface representation and recognition Unlike [11 20] ourapproach applies occlusion exclusion (by assigning weightsas 0 or 1) based on our explicit occlusion analysis Ourresults demonstrate that occlusion exclusion is more efficientthan occlusion weighting since weighting based methodsstill preserve some information from the occluded regionIn addition because the proposed occlusion analysis is anindependent module from the face matching part and nomodel learning step (such as Eigenface [2] Fisherface [3]or SRC [13]) is required in our approach the proposedmethod is not limited by the number of training samplesAs a consequence unlike SRC based methods [13ndash17] theproposed approach can be applied to face recognition withvery limited training samples (one sample per person in theextreme case)


Feature selection

Selective LGBPHS

LBP LGBPHS

Feature extraction

Gaborfiltering

PCA SVM

Occlusion detection and segmentation

Correspondingmatch

Galleryfaces

Occlusion mask

V2

V1

middot middot middot middot middot middot

middot middot middot middot middot middotmiddot middot middot middot middot middot middot middot middot

Figure 2 System flowchart

3 Approach

A comprehensive overview of the proposed system is givenin Figure 2 Given a target (ie probe) face image (which canbe occluded or not) to be recognized the possible presenceof occlusion is first analysed The probe image is divided intoa number of facial components for occlusion detection Eachcomponent is individually analysed by an occlusion detectionmodule As a result potential occluded facial componentsare identified Then an occlusion mask is generated by amore precise segmentation approach to supervise the featureextraction and matching process Based on the resultingocclusion mask its LGBPHS representation is computedusing the features extracted from the nonoccluded regiononly namely selective LGBPHS The recognition is per-formed by comparing the selective LGBPHS from the probeimage against selective LGBPHS from the template imagesusing the same occlusion mask The nearest neighbour (NN)classifier and Chi-square (1205942) distance are adopted for therecognition

31 Occlusion Detection in Local Patches As depicted inFigure 3 our occlusion detection starts by dividing the faceimage into different facial components The number andthe shape of the components are determined by the natureof the occlusion Since our focus in this work is scarf andsunglasses we accordingly divide the face images into twoequal components as shown in Figure 3 The upper part isused for analysing the presence of sunglasses while the lowerpart is used for detecting scarf

311 Gabor Wavelet Based Feature Extraction Gabor wa-velets are used for extracting features from the potentiallyoccluded regions The choice of using Gabor wavelets ismotivated by their biological relevance discriminative powerand computational properties A Gabor wavelet consists ofa complex sinusoidal carrier and a Gaussian envelope whichcan be written as

120595120583120574

(119911) =

10038171003817100381710038171003817119896120583120574

10038171003817100381710038171003817

2

1205752119890(minus119896120583120574

2119911221205752)[119890119894119896120583120574119911 minus 119890

minus12057522]

(1)

where 120583 and 120574 are the orientation and scale of the Gaborkernels 119911 = (119875 119876) is the size of the kernel window sdot

denotes the norm operator 119896120583120574

= 119896120574119890119894120601120583 is a wave vector

where 119896120574= 119896max119891

120574 and 120601120583= 1205871205838 119896max is the maximum

frequency and 119891 is the spacing factor between kernels in thefrequency domain

In our system we set 119911 = (20 20) 120575 = 2120587 119896max = 1205872and119891 = radic2 as also suggested in [20] Five scales 120574 isin [0 4]

and eight orientations120583 isin [0 7] are selected to extract theGabor features In total 40 Gabor wavelets are generated

Once theGaborwavelets are generated feature extractionis performed by convolving the wavelets with the face image119868

119862120583120574

(119909 119910) = 119868 (119909 119910) lowast 120595120583120574

(119911) (2)

Because the phase information of this transform is timevarying we only explore the magnitude information Thecomputed Gabor magnitude pictures (GMPs) thus form a setΩ = 119862

120583120574 120583 isin [0 7] 120574 isin [0 4] in which an augmented

feature vector is constructed by concatenating all the GMPsThe obtained feature vector is downsampled by a factor 120582

(here 120582 = 5) for further processing Note that GMPs are notonly used in occlusion detection but also used to computethe face representation selective LGBPHS as described inSection 33

312 Dimensionality Reduction Using PCA Because the sizeof extracted Gabor feature is rather big in order to reducethe dimension of the feature vectors while preserving itsdiscriminative power we apply principal component analysis(PCA) to maximize the variance in the projected subspacefor the Gabor features To compute the PCA subspace weconsider a training dataset consisting of feature vectors fromboth occluded and nonoccluded image patches Let us denotethe feature vectors from the nonoccluded patches by 119883

119888

and let us denote the feature vectors from the occludedpatches by 119883

119904 The training dataset 119878 can be formed as 119878 =

119883119888

1 119883119888

2 119883

119888

1198722 119883119904

1198722+1 119883

2

119872 where 119872 is the size of

the training dataset The eigenvectors associated with the 119896

largest eigenvalues of (119878minus119878)(119878minus119878)119879 (the covariance matrix of


Featureextraction

Dimensionalityreduction

SVM-basedclassification

Featureextraction



Figure 3 Our occlusion detection scheme

119878) are thus computed to describe the eigenspace The Gaborwavelet based features are then projected onto the computedeigenspace for dimensionality reduction

313 SVM Based Occlusion Detection Occlusion detectioncan be cast as a two-class classification problem Sincenonlinear support vector machines (SVM) are proven to be apowerful tool for discriminating 2 classes of high dimensionaldata we adopted then a nonlinear SVM classifier for occlu-sion detection Let us consider a training set consisting of 119873pairs 119909

119894 119910119894119873

119894=1 where 119909

119894refers to a reduced feature vector

of a facial component 119894 and 119910119894isin minus1 1 is the label which

indicates if the sample 119909119894is occluded or not SVM finds the

optimal separating hyper-plane 120572119894 119894 isin [1119873] by solving a

quadratic programming problem [26] and predicts the labelof an unknown face 119909 by

119891 (119909) = sign(119873

sum

119895=1

120572119895119910119895119870(119909119895 119909) + 119887) (3)

where 119909119895 119895 isin [1119873] are the support vectors Nonlinear

SVM applies kernels 119870(119909119894 119909119895) to fit the maximum-margin

hyper-plane in a transformed feature space In our system theRadial Basis Function (RBF) kernel is usedThe implementa-tion of the nonlinear SVM is provided by LIBSVM [27]

32 Occlusion Segmentation In order to efficiently exploitthe information of facial occlusion for face recognitionwe generate a binary mask 120573 (1 for occluded pixels and 0for nonoccluded pixels) indicating the location of occludedpixels to facilitate later feature extraction and matching inthe recognition phase This mask generation process is calledocclusion segmentation To generate an accurate occlusionmask (which can remove the occluded part meanwhilepreserving as much as information from the nonoccludedpart) we adopt a generalized Potts model Markov randomfield (GPM-MRF) [22] to enforce structural information(shape) of occlusion so as to identify if a given pixel isoccluded or not

Our occlusion segmentation can be formulated as atypical energy-minimization problem in computer visionLet us consider the face image (consists of multiple facialpatches) as an undirected adjacency graph 119866 = (119881 119864) where119881 = V

119894 119894 isin [1119873] denotes the set of 119873 pixels (vertex)

and 119864 denotes the edges between neighbouring pixels Givena set of observations 119874 = 119900

1 119900

119873 corresponding to

the set of vertex 119881 we want to assign a label (occluded 1nonoccluded minus1) to each vertex We model the set of labels

119871 = 119897119894 119894 isin [1119873] (discrete random variables taking values

in Λ = minus1 1) as a first-order Markov random field Thestructural prior is incorporated into theMRFby a generalizedPotts model Then our goal is to find the label set thatmaximizes the posterior probability 119875(119871|119874) which can beachieved by the maximum a posteriori (MAP) estimation[28] that maximizes the joint probability 119875(119874 119871) where

119875 (119874 119871) = 119875 (119874 | 119871) 119875 (119871)

=1

119885exp(minus119880(119871)119879)

(4)

where 119885 is the partition function and 119879 is the temperature119880(119871) is the sum of potentials from all cliques 119862 = 119888

119894 119894 isin

[1119873] which can be written as

119880 (119871) = sum

119888119894isin119862

119881119888(119871)

= sum

119897119894isin119871

Ψ (119900119894| 119897119894) + 120596 sum

(119897119894 119897119895)isin119864

Φ(119897119894 119897119895)

(5)

where 120596 is a weighting parameter controlling the importanceof MRF prior (the choice of 120596 is based on experiments ona validation set) The unary potential Ψ is defined by thelikelihood function

Ψ (119900119894| 119897119894) = minus ln119901 (119900

119894| 119897119894) (6)

We approximate the occlusion likelihood (119897 = 1) as follows

119901 (119900 | 119897 = 1) = eminus1 if 119900 gt 120591

1 minus eminus1 else(7)

where 0 lt 120591 lt 1 and the face likelihood (119897 = minus1) as a constant119888 isin [0 1]

119901 (119900 | 119897 = minus1) = 119888 (8)

Because we have already identified the type of occlusion(obtained by our occlusion detector) we can give an initialguess of observations 119874 (the seed of occlusion mask seeFigure 4(b)) to each type of occlusions The structural infor-mation is enforced into this initial guess via the isotropicMRFprior 119875(119871) where the pairwise potentialΦ(119897

119894 119897119895) has the form

of generalized Potts model as defined in [22]

Φ(119897119894 119897119895) = 119906 (119894 119895) sdot (1 minus 120585 (119897

119894minus 119897119895)) (9)

where 120585(sdot) represents the unit impulse function thenΦ(119897119894 119897119895) = 2119906(119894 119895) if 119894 and 119895 have different labels (119897

119894= 119897119895) and


(a) (b) (c) (d) (e)

Figure 4 Illustration of our occlusion segmentation (a) examples of faces occluded by scarf and sunglasses (b) initial guess of the observationset according to the results from our occlusion detector ((c)(d)) the visualization of 119906(119894 119895) in horizontal and vertical directions respectively(e) the generated occlusion masks (120596 = 150)

zero otherwise The structural information 119906(119894 119895) is obtainedas the first-order derivative after Gaussian filtering (with ker-nel size (5 5)) from the original image Note that maximizingthe joint probability 119875(119874 119871) is equivalent to minimizing thecliques potential119880(119871) and this energyminimization problemcan be solved exactly using graph cuts [29ndash31] in polynomialtimeThe obtained label set (see Figure 4(e)) is converted tothe segmentation mask 120573 isin [0 1] for later recognition task

33 Selective LGBPHS Based Face Representation and Recog-nition To perform the recognition step we propose a variantof LGBPHS [11] based face representation (namely selectiveLGBPHS) which selects features from nonoccluded pixelsonly The choice of using LGBPHS based representation isbased on the following facts (1) it takes the advantage of bothGabor decomposition (multiresolution and multiorientationfor enhanced discriminative power) [6] and LBP description(robustness to monotonic gray scale changes caused byeg illumination variations) [4] (2) block-based histogramrepresentation makes it robust to face misalignment andpose variations to some extent (3) it provides state-of-the-artresults in representing and recognizing face patterns underoccluded conditions [11 20] (4) Gabor features in LGBPHSshare the same computation as in our occlusion detectionmodule

331 LGBPHS Given a face image and its Gabormagnitudespictures (GMPs) Ω = 119862

120583120574 120583 isin [0 7] 120574 isin [0 4]

computed by the method described in Section 312 theGMPs are further encoded by an LBP operator resultingin a new feature descriptionmdashlocal Gabor binary patterns(LGBP) The LBP operator forms labels for the image pixelsby thresholding the 3 times 3 neighbourhood of each pixelwith the center value and considering the result as a binarynumber The histogram of these 2

8= 256 different labels

can then be used as a texture descriptor Each bin (LBP code)can be regarded as a microtexton Local primitives which are

codified by these bins include different types of curved edgesspots and flat areas

The calculation of LGBP codes is computed in a singlescan through each GMP using the LBP operatorThe value ofthe LGBP code of a pixel at position (119909

119888 119910119888) of each scale 120583

and orientation 120574 of GMPs is given by

LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574

119888corresponds to the intensity of the center pixel

(119909119888 119910119888) in the GMP 119862

120583120574 119892120583120574119901 refers to the intensities of 119875

equally spaced pixels on a circle of radius 119877 and 119904 defines athresholding function as follows

119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574

120583 isin [0 7] 120574 isin [0 4] are thus gen-erated via the above procedure In order to exploit the spatialinformation each LGBP map 119866

120583120574is first divided into 119903 local

regions from which histograms are extracted and concate-nated into an enhanced histogram ℎ

120583120574= (ℎ1205831205741

ℎ120583120574119903

)Then the LGBPHS is obtained by concatenating all enhancedhistograms119867 = (ℎ

00 ℎ

47)

332 Selective LGBPHS The original LGBPHS summarizesthe information from all pixels of a face image Given anocclusionmask120573 (generated by our occlusion segmentation)our interest is to extract features from the nonoccludedpixels only Hence we compute each bin ℎ

119894of the histogram

representation using a masking strategy as follows

ℎ119894= sum119909119910

(1 minus 120573 (119909 119910)) sdot 119868 119891 (119909 119910) = 119894 forall119894 isin [0 2119875minus 1]

(12)


where 119894 is the 119894th LGBP code ℎ119894is the number of nonoccluded

pixels with code 119894 and

119868 119860 = 1 119860 is true0 119860 is false

(13)

Then the histograms extracted from all local regions of allGMPs are concatenated into the final representation whichis named selective LGBPHS During matching selectiveLGBPHS is computed for both probe face and template facesbased on the occlusion mask generated from the probe

In the selective LGBPHS description a face is representedin four different levels of locality the LBP labels for thehistogram contain information about the patterns on a pixel-level the labels are summed over a small region to produceinformation on a regional-level the regional histograms areconcatenated to build a description of each GMP finallyhistogram from all GMPs are concatenated to build a globaldescription of the face This locality property in addition tothe information selective capability is behind the robustness(to facial occlusions) of the proposed descriptor

4 Experimental Results and Analysis

To evaluate the proposed approach we performed a set ofexperiments on AR face database [23] and compared ourresult against those of seven different methods includingEigenface [2] LBP [4] OA-LBP [21] LGBPHS [11] KLD-LGBPHS [20] S-LNMF [19] and RSC [17] Among theselected methods KLD-LGBPHS S-LNMF and OA-LBP(our previous work) are the state-of-the-art works whichexplicitly exploit automatic occlusion analysis (whereas Part-PCA [18] is based on manual annotation) to improve facerecognition according to our survey in Section 2 LBP andLGBPHS are selected to represent the locally emphasizedmethods without explicit occlusion analysis Because RSCreports the most recent and very competitive result amongall SRC based methods [17] we select it as the representativealgorithm of SRC based methods for comparison

41 Experimental Data and Setup For our experimentalanalysis we considered the AR face database [23] whichcontains a large number of well-organized real-world occlu-sions The AR database is the standard testing set for theresearch of occluded face recognition and it is used inalmost all literature works [6ndash21] It contains more than4000 face images of 126 subjects (70 men and 56 women)with different facial expressions illumination conditions andocclusions (sunglasses and scarf) Images were taken undercontrolled conditions but no restrictions on wearing (clothesglasses etc) make-up hair style and so forth were imposedto participants Each subject participated in two sessionsseparated by two weeks (14 days) of time The originalimage resolution is 768 times 576 pixels Some examples of faceimages from the AR face database are shown in Figure 5Using eye and nose coordinates we cropped normalized anddownsampled the original images into 128 times 128 pixels

For occlusion detection we randomly selected 150 nonoc-cluded faces 150 faces occluded with scarf and 150 faces

Figure 5 Example of images from the AR face database

Figure 6 The face images are divided into 64 blocks for selectiveLGBPHS representation

wearing sunglasses for training the PCA space and SVMTheupper parts of the faces with sunglasses are used to trainthe SVM-based sunglass detector while the lower parts ofthe faces with scarf are used to train the SVM-based scarfdetector The 150 nonoccluded faces are used in the trainingof both classifiers

For face recognition the face images are then dividedinto 64 blocks as shown in Figure 6 The size of each blockis 16 times 16 pixels The selective LGBPHS is extracted usingthe operator LBP1199062

82(using only uniform patterns 8 equally

spaced pixels on a circle of radius 2) on the 40GMPs yieldingfeature histograms of 151040 bins

To test the proposed algorithm we first selected 240nonoccluded faces from session 1 of the AR database asthe templates images These nonoccluded faces correspondto 80 subjects (40 males and 40 females) with 3 imagesper subject under neutral expression smile and anger Tobuild the evaluation set we considered the corresponding240 nonoccluded faces from session 2 the 240 faces withsunglasses of session 1 and the 240 faces with scarf of session1 under three different illuminations conditions

42 Results of Occlusion Detection The proposed occlusionsegmentation feature extraction and subsequent recognitionall rely on the correct occlusion detection To justify theproposed occlusion detectionmethod we show the detectionrates on all 720 testing images Table 2 illustrates the resultsas a confusion matrix Note that only 2 images (faces withvery bushy beard) from the nonoccluded faces are wronglyclassified as faces with scarfThe correctness of our occlusiondetection ensures the correct feature selection in the laterrecognition steps


Table 2 Results of occlusion detection

No-occlusion Scarf Sunglass Detection rateNon occlusion 238 2 0 9917Scarf 0 240 0 100Sunglass 0 0 240 100

43 Results of Occluded Face Recognition Figure 7 shows theface recognition performance of our approach on three differ-ent test sets clean (nonoccluded) faces faces occluded withscarf and faces occludedwith sunglasses For comparison wealso report results of the state-of-the-art algorithms (for thename abbreviations please refer to Table 1) for both standardface recognition and occluded face recognition Eigenfaces[2] (ie PCA) and LBP [4] are among the most popularalgorithms for standard face recognition We also tested theapproaches which incorporate our occlusion analysis (OA)with the standard Eigenface and LBP namely OA-PCA andOA-LBP [21] Similarly we denote the proposed approach byocclusion analysis assisted LGBPHS (OA-LGBPHS) In orderto justify that the proposed method is more appropriated foroccluded faces we also tested the standard LGBPHS [11] andits variant KLD-LGBPHS [20] on the same data set whereLGBPHS KLD-LGBPHS and OA-LGBPHS apply differentpreprocessing methods to the same face representation Themethod RSC [17] is selected to represent the family ofalgorithms based on sparse representation [13ndash17] in whichRSC is one of the most robust algorithms according tothe reported results It should be noticed that in the poolof selected algorithms KLD-LGBPHS OA-LBP and RSCstand for the state-of-the-art algorithms for occluded facerecognition in each of the 3 categories as we reviewed inSection 2 (see Table 1)

In Figure 7 it is clear that the proposed approach (OA-LGBPHS) obtains the highest identification rates in all 3cases (9917 9583 and 8708 for clean scarf andsunglass faces resp) Without explicit occlusion analysisfacial occlusions such as scarf and sunglasses can greatlydeteriorate the recognition results of PCA and LBP incontrast OA-PCA and OA-LBP surpass their original algo-rithms significantlyWith a long length feature vector (151040bins) LGBPHS demonstrates satisfactory robustness to facialocclusions Without occlusion analysis LGBPHS can alreadyyield close results to OA-LBP under the occlusion conditionsKLD-LGBPHS improves LGBPHS by associating a weightwith each block (which indicates the level of occlusion) toameliorate the impact from occluded regions and the weightis measured as a deviation of the target block from the pre-defined mean model based on Kullback-Leibler divergenceAlthough KLD-LGBPHS greatly increases the results incomparison to LGBPHS (especially for faces occluded bysunglasses) its performance is still inferior to OA-LGBPHSThis result reveals that occlusion exclusion is more efficientthan occlusion weighting since distortions due to facialocclusions do not affect the process of recognition when theoccluded regions are completely discarded

Sparse representation based classification (SRC) is wellknown for its robustness to partial distortions (eg noise

Clean Scarf Sunglasses0

010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP

LGBPHSKLD-LGBPHSOA-LGBPHSRSC

Figure 7 Results of PCA OA-PCA LBP OA-LBP LGBPHS KLD-LGBPHS OA-LGBPHS and RSC on three different testing sets(faces are clean and faces are occluded by scarf and sunglasses)

occlusion etc) as well as its discriminative power Howeverit also suffers from the ldquocurse of dimensionalityrdquo problemwhere in many practical cases the number of templates(of each identity) is insufficient to support the recoveryof correct sparse coefficients On the given data set (240training faces with 3 templates for each identity) robustsparse coding (RSC) yields relatively low identification rates(8625 5667 and 30)

Comparing the results on the test sets of faces withsunglasses and scarves we notice that most methods (exceptfor PCA) are more sensitive to sunglasses than to scarf Thisis an interesting phenomenon which is in agreement withthe psychophysical findings indicating that the eyeseyebrowsregion plays the most important role in face recognition [32]

44 Robustness to Other Facial Variations We comparedour proposed approach against OA-LBP and S-LNMF [19]using similar protocol under the more challenging scenarioin which the gallery face images are taken from session 1 ofAR database while the test sets are taken from session 2 Notethat the two sessions were taken at time interval of 14 daysThe comparative results of our approach against OA-LBP andS-LNMF are illustrated in Table 3

The results in Table 3 clearly show that our proposedapproach outperforms OA-LBP and S-LNMF in all con-figurations showing robustness against sunglasses scarvesscreaming and illumination changes The robustness of our


Table 3 Robustness to different facial variations

Sunglasses Scarf Scream Right lightS-LNMF 49 55 27 51OA-LBP 5417 8125 5250 8625OA-LGBPHS 75 9208 5750 9625

approach to illumination changes and drastic facial expres-sion is brought by the use of local Gabor binary patternswhile the occlusion detection module significantly enhancesthe recognition of faces occluded by sunglasses and scarveseven with time elapsing

Please note that we did not provide the comparativeresults of our approach to all the literature works (accordingto our survey in Section 2) Insteadwe compare our approachto a number of carefully selected methods Because ourmethod exploits explicit occlusion analysis KLD-LGBPHSS-LNMF and OA-LBP which belong to the same cate-gory (see Table 1) are selected for the comparisons in ourexperiment RSC is selected to represent the family of SRCbased face recognition Even though LGBPHS is chosen tostand for the locally emphasized algorithms without explicitocclusion analysis our approach could be directly extendedto other local featureclassifier based methods for potentialimprovements

5 Conclusions

We addressed the problem of face recognition under occlu-sions caused by scarves and sunglasses Our proposedapproach consisted of first conducting explicit occlusionanalysis and then performing face recognition from thenonoccluded regionsThe salient contributions of our presentwork are as follows (i) a novel framework for improving therecognition of occluded faces is proposed (ii) state-of-the-art in face recognition under occlusion is reviewed (iii) anew approach to detect and segment occlusion is thoroughlydescribed (iv) extensive experimental analysis is conducteddemonstrating significant performance enhancement usingthe proposed approach compared to the state-of-the-artmethods under various configurations including robustnessagainst sunglasses scarves nonoccluded faces screamingand illumination changes Although we focused on occlu-sions caused by sunglasses and scarves our methodology canbe directly extended to other sources of occlusion such ashats beards and long hairs As a futurework it is of interest toextend our approach to address face recognition under gen-eral occlusions including not only the most common oneslike sunglasses and scarves but also beards long hairs capsand extreme facialmake-ups Automatic face detection undersevere occlusion such as in video surveillance applicationsis also far from being a solved problem and thus deservesthorough investigations

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper

Acknowledgment

This work is partially funded by the French National ProjectFR OSEO BIORAFALE

References

[1] W Zhao R Chellappa P J Phillips and A Rosenfeld ldquoFacerecognition a literature surveyrdquo ACM Computing Surveys vol35 no 4 pp 399ndash458 2003

[2] M A Turk and A P Pentland ldquoFace recognition using eigen-facesrdquo in Proceedings of the IEEE Computer Society Conferenceon Computer Vision and Pattern Recognition (CVPR rsquo91) pp586ndash591 June 1991

[3] P N Belhumeur J P Hespanha and D J Kriegman ldquoEigen-faces vs fisherfaces recognition using class specific linearprojectionrdquo IEEE Transactions on Pattern Analysis andMachineIntelligence vol 19 no 7 pp 711ndash720 1997

[4] T Ahonen A Hadid and M Pietikainen ldquoFace descriptionwith local binary patterns application to face recognitionrdquo IEEETransactions on Pattern Analysis and Machine Intelligence vol28 no 12 pp 2037ndash2041 2006

[5] P S Penev and J J Atick ldquoLocal feature analysis a generalstatistical theory for object representationrdquo Network vol 7 no3 pp 477ndash500 1996

[6] A M Martınez ldquoRecognizing imprecisely localized partiallyoccluded and expression variant faces from a single sampleper classrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 24 no 6 pp 748ndash763 2002

[7] X Tan S Chen Z-H Zhou and F Zhang ldquoRecognizingpartially occluded expression variant faces from single trainingimage per person with SOM and soft 120581-NN ensemblerdquo IEEETransactions on Neural Networks vol 16 no 4 pp 875ndash8862005

[8] J Kim J Choi J Yi andMTurk ldquoEffective representation usingICA for face recognition robust to local distortion and partialocclusionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 27 no 12 pp 1977ndash1981 2005

[9] S Fidler D Skocaj and A Leonardis ldquoCombining reconstruc-tive and discriminative subspace methods for robust classifi-cation and regression by subsamplingrdquo IEEE Transactions onPattern Analysis andMachine Intelligence vol 28 no 3 pp 337ndash350 2006

[10] B-G Park K-M Lee and S-U Lee ldquoFace recognition usingface-ARGmatchingrdquo IEEETransactions on PatternAnalysis andMachine Intelligence vol 27 no 12 pp 1982ndash1988 2005

[11] W Zhang S Shan W Gao X Chen and H Zhang ldquoLocalGabor Binary Pattern Histogram Sequence (LGBPHS) a novelnon-statistical model for face representation and recognitionrdquoin Proceedings of the 10th IEEE International Conference onComputer Vision (ICCV rsquo05) pp 786ndash791 IEEE ComputerSociety Washington DC USA October 2005

[12] H Jia and A M Martinez ldquoSupport vector machines inface recognition with occlusionsrdquo in Proceedings of the IEEE


Computer Society Conference on Computer Vision and PatternRecognition Workshops (CVPR rsquo09) pp 136ndash141 June 2009

[13] JWright A Y Yang A Ganesh S S Sastry and YMa ldquoRobustface recognition via sparse representationrdquo IEEE Transactionson Pattern Analysis and Machine Intelligence vol 31 no 2 pp210ndash227 2009

[14] Z Zhou A Wagner H Mobahi J Wright and Y Ma ldquoFacerecognition with contiguous occlusion using Markov RandomFieldsrdquo in Proceedings of the 12th International Conference onComputer Vision (ICCV rsquo09) pp 1050ndash1057 October 2009

[15] M Yang and L Zhang ldquoGabor feature based sparse represen-tation for face recognition with gabor occlusion dictionaryrdquo inProceedings of the 11th European conference on Computer vision(ECCV rsquo10) pp 448ndash461 Springer Berlin Germany 2010

[16] S Liao and A K Jain ldquoPartial face recognition an alignmentfree approachrdquo in Proceedings of the International Joint Confer-ence on Biometrics (IJCB rsquo11) October 2011

[17] M Yang L Zhang J Yang and D Zhang ldquoRobust sparse cod-ing for face recognitionrdquo in Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition (CVPR rsquo11) pp625ndash632 June 2011

[18] A Rama F Tarres L Goldmann and T Sikora ldquoMore robustface recognition by considering occlusion informationrdquo in Pro-ceedings of the 8th IEEE International Conference on AutomaticFace and Gesture Recognition (FG rsquo08) September 2008

[19] H J Oh K M Lee and S U Lee ldquoOcclusion invariant facerecognition using selective local non-negative matrix factoriza-tion basis imagesrdquo Image and Vision Computing vol 26 no 11pp 1515ndash1523 2008

[20] W Zhang S Shan X Chen and W Gao ldquoLocal Gabor binarypatterns based on Kullback-Leibler divergence for partiallyoccluded face recognitionrdquo IEEE Signal Processing Letters vol14 no 11 pp 875ndash878 2007

[21] RMin AHadid and J-L Dugelay ldquoImproving the recognitionof faces occluded by facial accessoriesrdquo in Proceedings of theIEEE International Conference on Automatic Face and GestureRecognition and Workshops (FG rsquo11) pp 442ndash447 March 2011

[22] Y BoykovOVeksler andR Zabih ldquoMarkov randomfieldswithefficient approximationsrdquo in Proceedings of the IEEE ComputerSociety Conference on Computer Vision and Pattern Recognitionpp 648ndash655 June 1998

[23] A MMartinez ldquoThe AR face databaserdquo CVC Technical Report24 1998

[24] D L Donoho ldquoHigh-dimensional data analysis the curses andblessings of dimensionalityrdquo in Proceedings of the AmericanMathematical Society Conference Math Challenges of the 21stCentury 2000

[25] S Z Li X W Hou H J Zhang and Q S Cheng ldquoLearningspatially localized parts-based representationrdquo in Proceedingsof the IEEE Computer Society Conference on Computer Visionand Pattern Recognition vol 1 pp I207ndashI212 December 2001

[26] C Cortes and V Vapnik ldquoSupport-vector networksrdquo MachineLearning vol 20 no 3 pp 273ndash297 1995

[27] C C Chang and C J Lin ldquoLIBSVM a library for supportvector machinesrdquo ACM Transactions on Intelligent Systems andTechnology vol 2 no 3 article 27 2011

[28] S Geman and D Geman ldquoStochastic relaxation gibbs distri-butions and the bayesian restoration of imagesrdquo IEEE Transac-tions on Pattern Analysis and Machine Intelligence vol 6 no 6pp 721ndash741 1984

[29] Y Boykov O Veksler and R Zabih ldquoFast approximate energyminimization via graph cutsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 23 no 11 pp 1222ndash12392001

[30] Y Boykov and V Kolmogorov ldquoAn experimental comparisonof min-cutmax-flow algorithms for energy minimization invisionrdquo IEEE Transactions on Pattern Analysis and MachineIntelligence vol 26 no 9 pp 1124ndash1137 2004

[31] V Kolmogorov and R Zabih ldquoWhat energy functions canbe minimized via graph cutsrdquo IEEE Transactions on PatternAnalysis and Machine Intelligence vol 26 no 2 pp 147ndash1592004

[32] P Sinha B Balas YOstrovsky andR Russell ldquoFace recognitionby humans nineteen results all computer vision researchersshould know aboutrdquo Proceedings of the IEEE vol 94 no 11 pp1948ndash1961 2006

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttpwwwhindawicom Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttpwwwhindawicom Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation httpwwwhindawicom

Journal ofEngineeringVolume 2014

Submit your manuscripts athttpwwwhindawicom

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics


Volume 2014

The Scientific World JournalHindawi Publishing Corporation httpwwwhindawicom Volume 2014

SensorsJournal of


Modelling amp Simulation in EngineeringHindawi Publishing Corporation httpwwwhindawicom Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks



(a) (b)

Figure 1 Illustration of different types of facial occlusions (a) ordinary facial occlusions in daily life (b) facial occlusions related to severesecurity issues (ATM crimes football hooligans etc)

works [5ndash17] focus on finding corruption-tolerant featuresor classifiers to reduce the effect of partial occlusions inface representation However information from the occludedparts can still hinder the recognition performance Recentlyresearchers [18ndash21] demonstrated that prior knowledge aboutthe occlusion (eg type location and size) can be used toexclude the information from occluded parts so as to greatlyimprove the recognition rate Hence explicit occlusion anal-ysis is an important step in occlusion-robust face recognitionIn this paper we propose an occlusion analysis methodto improve local Gabor binary pattern based face recog-nition [11] which outperforms literature works including[17ndash21]

The proposed approach consists of first detecting andsegmenting occluded parts (eg sunglassesscarves) and thenapplying face recognition on the nonoccluded facial regionsTo do so the presence of occlusion is first analysed in thepatch-level using Gabor wavelets PCA and SVM Then wesegment the occluded part more precisely from the otherfacial regions by a generalized Potts model Markov randomfield (GPM-MRF) [22]This allows us to identify the presenceof occlusion at the pixel-level so as to preserve as muchas possible face information for the recognition After thecomputation of an occlusion mask indicating which pixelin a face image is occluded we propose a variant of localGabor binary pattern histogram sequences (LGBPHS) [11]to efficiently represent occluded faces by excluding featuresextracted from the occluded pixels Finally we compared ourapproach with traditional approaches [2 4 11] our previousresults [21] and state-of-the-art methods [13 19 20] on ARface database [23] and obtained the best results Our exper-iments also suggested that in comparison with weightingbased method [20] occlusion exclusion (ie weighting as 0or 1) is more appropriate to handle the occlusion problem inface recognition

The rest of this paper is structured as follows First wereview the related works in Section 2 Then the proposedapproach is described in Section 3 Section 4 presents theexperimental results and analysis Finally we draw the con-clusion and discuss future directions in Section 5

2 Related Works

The traditional methodology to address face recognitionunder occlusion is to find corruption-tolerant features or clas-sifiers Toward this goal numerous previousworks confirmedthat locally emphasized algorithms are less sensitive to partialocclusions Penev and Atick [5] proposed the local featureanalysis (LFA) to extract local features by second order statis-tics Martınez [6] proposed a probabilistic approach (AMM)which can compensate for partially occluded faces Tan etal [7] extended Martinezrsquos work by using the self-organizingmap (SOM) to learn the subspace instead of using themixtureof Gaussians In [8] Kim et al proposed a method namedlocally salient ICA (LS-ICA) which only employs locallysalient information in constructing ICA basis In [9] Fidleret al presented a method which combines the reconstructiveand discriminativemodels by constructing a basis containingthe complete discriminative information Park et al [10]proposed to use a line feature based face attributed relationalgraph (ARG) model which encodes the whole geometricstructure information and local features of a face Zhanget al [11] proposed a nonstatistical face representationmdashlocal gabor binary pattern histogram sequence (LGBPHS)to exploit the multiresolution and multiorientation Gabordecomposition In [12] Jia and Martinez proposed the use ofpartial support vector machines (PSVM) in scenarios whereocclusions may occur in both the training and testing sets

More recently facial occlusion handling under the sparserepresentation based classification (SRC) [13] framework hasdemonstrated impressive performances in face recognitionwith occlusions The idea of using SRC for occluded facerecognition is first introduced by Wright et al [13] where anoccluded face is represented as a linear combination of thewhole face gallery added by a vector of errors (occlusion)in the pixel-level and the classification is achieved by L1minimization Zhou et al [14] extend [13] by including aMarkov Random Fields (MRF) model to enforce spatialcontinuity for the additive error vector to address contiguousocclusions In [15] Yang and Zhang applied compressibleimage Gabor features instead of original image pixels as thefeature vector used in SRC to reduce computations in the









SRC based methods











Feature selection

Selective LGBPHS

LBP LGBPHS

Feature extraction

Gaborfiltering

PCA SVM


Correspondingmatch

Galleryfaces

Occlusion mask

V2

V1




3 Approach




120595120583120574

(119911) =

10038171003817100381710038171003817119896120583120574

10038171003817100381710038171003817

2

1205752119890(minus119896120583120574

2119911221205752)[119890119894119896120583120574119911 minus 119890

minus12057522]

(1)



= 119896120574119890119894120601120583 is a wave vector

where 119896120574= 119896max119891






119862120583120574

(119909 119910) = 119868 (119909 119910) lowast 120595120583120574

(119911) (2)






119888



119883119888

1 119883119888

2 119883

119888

1198722 119883119904

1198722+1 119883

2





Featureextraction



Featureextraction






119894 119910119894119873

119894=1 where 119909






119891 (119909) = sign(119873

sum

119895=1

120572119895119910119895119870(119909119895 119909) + 119887) (3)








1 119900





119875 (119874 119871) = 119875 (119874 | 119871) 119875 (119871)

=1

119885exp(minus119880(119871)119879)

(4)


119894 119894 isin


119880 (119871) = sum

119888119894isin119862

119881119888(119871)

= sum

119897119894isin119871

Ψ (119900119894| 119897119894) + 120596 sum

(119897119894 119897119895)isin119864

Φ(119897119894 119897119895)

(5)


Ψ (119900119894| 119897119894) = minus ln119901 (119900

119894| 119897119894) (6)


119901 (119900 | 119897 = 1) = eminus1 if 119900 gt 120591



119901 (119900 | 119897 = minus1) = 119888 (8)


119894 119897119895) has the form


Φ(119897119894 119897119895) = 119906 (119894 119895) sdot (1 minus 120585 (119897

119894minus 119897119895)) (9)


119894= 119897119895) and


(a) (b) (c) (d) (e)





120583120574 120583 isin [0 7] 120574 isin [0 4]






119888 119910119888) of each scale 120583


LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574


(119909119888 119910119888) in the GMP 119862



119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574




120583120574= (ℎ1205831205741

ℎ120583120574119903


00 ℎ

47)




ℎ119894= sum119909119910


(12)




119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation

















SRC based methods











Feature selection

Selective LGBPHS

LBP LGBPHS

Feature extraction

Gaborfiltering

PCA SVM


Correspondingmatch

Galleryfaces

Occlusion mask

V2

V1




3 Approach




120595120583120574

(119911) =

10038171003817100381710038171003817119896120583120574

10038171003817100381710038171003817

2

1205752119890(minus119896120583120574

2119911221205752)[119890119894119896120583120574119911 minus 119890

minus12057522]

(1)



= 119896120574119890119894120601120583 is a wave vector

where 119896120574= 119896max119891






119862120583120574

(119909 119910) = 119868 (119909 119910) lowast 120595120583120574

(119911) (2)






119888



119883119888

1 119883119888

2 119883

119888

1198722 119883119904

1198722+1 119883

2





Featureextraction



Featureextraction






119894 119910119894119873

119894=1 where 119909






119891 (119909) = sign(119873

sum

119895=1

120572119895119910119895119870(119909119895 119909) + 119887) (3)








1 119900





119875 (119874 119871) = 119875 (119874 | 119871) 119875 (119871)

=1

119885exp(minus119880(119871)119879)

(4)


119894 119894 isin


119880 (119871) = sum

119888119894isin119862

119881119888(119871)

= sum

119897119894isin119871

Ψ (119900119894| 119897119894) + 120596 sum

(119897119894 119897119895)isin119864

Φ(119897119894 119897119895)

(5)


Ψ (119900119894| 119897119894) = minus ln119901 (119900

119894| 119897119894) (6)


119901 (119900 | 119897 = 1) = eminus1 if 119900 gt 120591



119901 (119900 | 119897 = minus1) = 119888 (8)


119894 119897119895) has the form


Φ(119897119894 119897119895) = 119906 (119894 119895) sdot (1 minus 120585 (119897

119894minus 119897119895)) (9)


119894= 119897119895) and


(a) (b) (c) (d) (e)





120583120574 120583 isin [0 7] 120574 isin [0 4]






119888 119910119888) of each scale 120583


LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574


(119909119888 119910119888) in the GMP 119862



119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574




120583120574= (ℎ1205831205741

ℎ120583120574119903


00 ℎ

47)




ℎ119894= sum119909119910


(12)




119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Feature selection

Selective LGBPHS

LBP LGBPHS

Feature extraction

Gaborfiltering

PCA SVM


Correspondingmatch

Galleryfaces

Occlusion mask

V2

V1




3 Approach




120595120583120574

(119911) =

10038171003817100381710038171003817119896120583120574

10038171003817100381710038171003817

2

1205752119890(minus119896120583120574

2119911221205752)[119890119894119896120583120574119911 minus 119890

minus12057522]

(1)



= 119896120574119890119894120601120583 is a wave vector

where 119896120574= 119896max119891






119862120583120574

(119909 119910) = 119868 (119909 119910) lowast 120595120583120574

(119911) (2)






119888



119883119888

1 119883119888

2 119883

119888

1198722 119883119904

1198722+1 119883

2





Featureextraction



Featureextraction






119894 119910119894119873

119894=1 where 119909






119891 (119909) = sign(119873

sum

119895=1

120572119895119910119895119870(119909119895 119909) + 119887) (3)








1 119900





119875 (119874 119871) = 119875 (119874 | 119871) 119875 (119871)

=1

119885exp(minus119880(119871)119879)

(4)


119894 119894 isin


119880 (119871) = sum

119888119894isin119862

119881119888(119871)

= sum

119897119894isin119871

Ψ (119900119894| 119897119894) + 120596 sum

(119897119894 119897119895)isin119864

Φ(119897119894 119897119895)

(5)


Ψ (119900119894| 119897119894) = minus ln119901 (119900

119894| 119897119894) (6)


119901 (119900 | 119897 = 1) = eminus1 if 119900 gt 120591



119901 (119900 | 119897 = minus1) = 119888 (8)


119894 119897119895) has the form


Φ(119897119894 119897119895) = 119906 (119894 119895) sdot (1 minus 120585 (119897

119894minus 119897119895)) (9)


119894= 119897119895) and


(a) (b) (c) (d) (e)





120583120574 120583 isin [0 7] 120574 isin [0 4]






119888 119910119888) of each scale 120583


LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574


(119909119888 119910119888) in the GMP 119862



119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574




120583120574= (ℎ1205831205741

ℎ120583120574119903


00 ℎ

47)




ℎ119894= sum119909119910


(12)




119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










Featureextraction



Featureextraction






119894 119910119894119873

119894=1 where 119909






119891 (119909) = sign(119873

sum

119895=1

120572119895119910119895119870(119909119895 119909) + 119887) (3)








1 119900





119875 (119874 119871) = 119875 (119874 | 119871) 119875 (119871)

=1

119885exp(minus119880(119871)119879)

(4)


119894 119894 isin


119880 (119871) = sum

119888119894isin119862

119881119888(119871)

= sum

119897119894isin119871

Ψ (119900119894| 119897119894) + 120596 sum

(119897119894 119897119895)isin119864

Φ(119897119894 119897119895)

(5)


Ψ (119900119894| 119897119894) = minus ln119901 (119900

119894| 119897119894) (6)


119901 (119900 | 119897 = 1) = eminus1 if 119900 gt 120591



119901 (119900 | 119897 = minus1) = 119888 (8)


119894 119897119895) has the form


Φ(119897119894 119897119895) = 119906 (119894 119895) sdot (1 minus 120585 (119897

119894minus 119897119895)) (9)


119894= 119897119895) and


(a) (b) (c) (d) (e)





120583120574 120583 isin [0 7] 120574 isin [0 4]






119888 119910119888) of each scale 120583


LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574


(119909119888 119910119888) in the GMP 119862



119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574




120583120574= (ℎ1205831205741

ℎ120583120574119903


00 ℎ

47)




ℎ119894= sum119909119910


(12)




119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation










(a) (b) (c) (d) (e)





120583120574 120583 isin [0 7] 120574 isin [0 4]






119888 119910119888) of each scale 120583


LGBP120583120574119875119877

=

119875minus1

sum

119901=0

119904 (119892120583120574

119901minus 119892120583120574

119888) 2119875 (10)

where 119892120583120574


(119909119888 119910119888) in the GMP 119862



119904 (119909) = 1 if 119909 ge 0

0 otherwise(11)

120583 times 120574 LGBP maps 119866120583120574




120583120574= (ℎ1205831205741

ℎ120583120574119903


00 ℎ

47)




ℎ119894= sum119909119910


(12)




119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation












119868 119860 = 1 119860 is true0 119860 is false

(13)






















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation
















010203040506070809

1111213

Type of occlusions

Iden

tifica

tion

rate

758

375

83

945

894

58

991

799

17

991

786

25

542

34

17

608

392

08

920

893

75

958

356

67

125

017

00

508

375

83

725

0 841

787

08

300

0

PCAOA-PCALBPOA-LBP












5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation














5 Conclusions




Acknowledgment


References





































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation

































RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation











RoboticsJournal of





Journal of



RotatingMachinery





VLSI Design



Shock and Vibration







Journal of



Volume 2014


SensorsJournal of





Propagation









Documents

Research Article Efficient Detection of Occlusion …downloads.hindawi.com/journals/tswj/2014/519158.pdfface database [ ] and obtained the best results. Our exper-iments also suggested