Embed Size (px)
Citation preview
Pattern Recognition 76 (2018) 50–68
Contents lists available at ScienceDirect
Pattern Recognition
journal homepage: www.elsevier.com/locate/patcog
Color texture description with novel local binary patterns for effective
image retrieval
Chandan Singh
a , ∗, Ekta Walia
b , Kanwal Preet Kaur a
a Department of Computer Science, Punjabi University, Patiala 147002, India b Department of Computer Science, University of Saskatchewan, Canada
a r t i c l e i n f o
Article history:
Received 22 June 2016
Revised 31 July 2017
Accepted 16 October 2017
Available online 17 October 2017
Keywords:
Local binary pattern (LBP)
Local binary pattern for color images (LBPC)
Local binary pattern of hue component
(LBPH)
Local color texture
Image retrieval
a b s t r a c t
We propose a novel local color texture descriptor called local binary pattern for color images (LBPC). The
proposed descriptor uses a plane to threshold color pixels in the neighborhood of a local window into
two categories. To boost the discriminative power of the proposed LBPC operator, local binary patterns of
the hue component in the HSI color space, called the local binary pattern of the hue (LBPH) is derived.
Further, LBPC, LBPH are fused to derive LBPC + LBPH which when combined with the color histogram
(CH) of the hue component results in an effective image retrieval method LBPC + LBPH + CH. The uniform
patterns of the two proposed descriptors ULBPC and ULBPH are combined to yield another low dimension
local color descriptor ULBPC + ULBPH + CH which provides a good tradeoff between retrieval accuracy and
speed. Detailed experiments conducted on Wang, Holidays, Corel- 5K and Corel- 10K datasets demonstrate
that the proposed low dimension descriptors LBPC + LBPH + CH, and ULBPC + ULBPH + CH outperform the
state-of-the-art color texture descriptors in terms of retrieval accuracy and speed.
© 2017 Elsevier Ltd. All rights reserved.
T
c
v
g
p
c
c
m
s
a
d
t
s
o
d
o
s
l
i
m
t
1. Introduction
Content-based image retrieval (CBIR) is one of the active re-
search areas in the field of pattern recognition and artificial intel-
ligence. Even after more than three decades of extensive research,
the quest for more-and-more effective CBIR systems has attracted
many researchers to develop systems which provide high retrieval
rates within low retrieval time. The early CBIR systems were fo-
cused on the grayscale images, but with the widespread use of
color images over Internet and the advantage of a color attribute
for discrimination purpose, color information is being incorporated
to enhance the performance of retrieval systems. A retrieval sys-
tem provides a user with a way to access, browse and fetch im-
ages efficiently from databases. These databases are used in a vari-
ety of fields including information security, biometric system (iris,
fingerprint, and face matching), biodiversity, digital library, crime
prevention, medical imaging, historical archives, video surveillance,
human-computer interaction, etc.
The CBIR systems depend heavily on two steps: feature extrac-
tion and feature matching. Feature extraction is the most impor-
tant step as it requires an image to be represented by highly dis-
criminative features with small variations among the features of
intra-class images and high variations among inter-class images.
∗ Corresponding author.
E-mail addresses: [email protected] (C. Singh), [email protected] (E.
Walia), [email protected] (K.P. Kaur).
t
f
r
fi
https://doi.org/10.1016/j.patcog.2017.10.021
0031-3203/© 2017 Elsevier Ltd. All rights reserved.
he features are desired to be robust to geometric and photometric
hanges such as translation, rotation, scale, occlusion, illumination,
iewpoint, etc.
Broadly, there are two categories of feature representation: (1)
lobal or holistic approach and (2) local approach. A global ap-
roach extracts features from the whole image ignoring the local
haracteristics and spatial relationship between pixels. They are
omputationally efficient and robust to image noise. The global
ethods suffer from their inadequacies to handle some of the is-
ues related to occlusion, view points and illumination changes,
nd local characteristics of image shape. These issues are well ad-
ressed by the local feature extraction methods which extract fea-
ures from local regions of an image. These local regions can be as
imple as partitions of an image or are selected through key points.
The most common global feature extraction techniques based
n color include the MPEG-7 feature sets such as color histograms,
ominant color descriptor, scalable color descriptor, and color lay-
ut descriptor [1] . The color histogram features have been ob-
erved to yield very good performance; they are invariant to trans-
ation and rotation and can be made scale invariant after normal-
zation by the image size. Texture based global feature extraction
ethods include gray level co-occurrence matrices [2] , Tamura tex-
ure features [3] , Markov random field model [4] and Gabor fil-
ering [5] . A comparative performance analysis of these texture
eatures on the retrieval of texture images establishes the supe-
iority of Gabor filters over others [6] . Effect of different Gabor
lter parameters on texture image retrieval has been studied in
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 51
[
f
T
m
e
t
o
t
v
w
i
b
s
r
p
t
m
c
r
t
o
[
R
e
t
g
g
c
t
t
f
t
e
c
[
p
p
a
v
c
f
c
A
i
v
h
i
i
s
p
a
c
e
I
c
t
m
f
p
o
r
n
Y
o
b
g
[
r
c
p
d
a
c
t
t
O
i
c
r
m
Q
o
i
t
l
t
t
t
d
b
r
p
s
l
t
a
g
t
i
t
p
t
p
l
t
o
a
f
f
i
w
r
u
F
c
a
f
p
t
d
t
T
[
h
w
o
u
t
7] . Rotation and scale invariant optimum Gabor filter parameters
or texture image retrieval have been derived by Han and Ma [8] .
hey have observed that the optimum filter parameters provide
uch superior performance than the conventional filter param-
ters. Bianconi and Fernandez [9] have also improved Gabor fil-
er parameters to provide better texture classification results. One
f the problems associated with Gabor filtering is that its fea-
ure extraction process is very slow [7] . Shape features also pro-
ide powerful information for image retrieval. Shape matching is a
ell-researched area with many shape representation and match-
ng techniques [10] . Shape features are generally used in two ways:
oundary-based and region-based. In the boundary-based repre-
entation of an image, the boundary or contour of an object is
equired to be extracted, while in the region-based techniques all
ixels of an image take part in the computation of the shape fea-
ures [10,11] .
The global feature-based methods have been the mainstay of
any CBIR systems. However, their inability to deal with many
omplex issues related to geometric deformations and photomet-
ic changes have necessitated the extensive use of local feature ex-
raction methods, which can resolve these issues effectively. Some
f the effective local feature extraction methods are LBP [12] , SIFT
13] , PCA-SIFT [14] , GLOH [15] , SURF [16] , HOG [17] , DAISY [18] ,
ank-SIFT [19] , BRIEF [20] , ORB [21] , WLD [22] , and many more.
Color and texture provide important information for deriving
ffective f eatures there by ensuring high performance of the re-
rieval systems. The classical texture features were derived for the
rayscale images. Among the various local texture descriptors for
rayscale images, SIFT has proven to be the most effective and suc-
essful descriptor in the state-of-the-art recognition and classifica-
ion system [23] . To capture the texture of color images, it is ex-
ended to several variants e.g. color SIFT [24] . The color SIFT has
urther been evaluated against several color descriptors and found
o outperform them. However, color SIFT is computation intensive
specially when the size of the image or size of the database in-
reases. The local binary pattern (LBP), developed by Ojala et al.
12] is also an effective texture descriptor which is found to be
owerful and successful in many pattern recognition and com-
uter vision applications. It captures local texture features, which
re invariant to illumination changes. It is simple, fast and pro-
ides strong discriminative power as compared to many other lo-
al texture descriptors. The LBP operator has been used success-
ully in texture classification [25-28] , face recognition [29-31] , fa-
ial expression recognition [32-35] , and image retrieval [36,37] , etc.
survey paper [38] provides a comprehensive discussion on facial
mage analysis using the classical LBP operator and several of its
ariants and cites 159 papers in this area.
Most of the works on classical LBP operator and its variants
ave been developed for gray scale image processing. The increas-
ng demand for color images over Internet and their ever increas-
ng use for many practical applications have motivated the re-
earchers to develop descriptors which can represent color texture
attern as effectively as the LBP operator does for gray scale im-
ges. A natural extension of the gray scale LBP operator is to pro-
ess each channel of a color image as a gray scale image. This strat-
gy was used by Mäenpää et al. [39] for color texture description.
n their multispectral LBP (MSLBP), they use three channels of a
olor image and six sets of LBPs opponent color to capture spa-
ial correlation between the two channels of the color spectra. The
ethod is effective. However, it results in a very high dimensional
eature vector. Later, Mäenpää and Pietikäinen [40] conducted ex-
eriments on color texture and observed that instead of taking six
pponent color components only three pairs are sufficient for rep-
esenting cross correlation features between the three color chan-
els. Choi et al. [41] derived LBP histograms for each channel in
C b C r color space and applied PCA to reduce the dimensionality
f the feature vector. They observed that their method performed
etter in face recognition application than the LBP operator on
ray face images, which were derived from color images. Lee et al.
42] derived local color vector binary patterns (LCVBPs) for face
ecognition problem. The LCVBP operator consists of two parts:
olor norm patterns and color angular patterns. The color angular
attern captures discriminative features of two color spectra and
erives spatial correlation of local color texture. The LCVBP oper-
tor is very effective as compared to the LBP of individual color
hannels. To reduce the dimension of LBP operator and apply it
o color images, Zhu et al. [43] proposed an orthogonal combina-
ion of local binary patterns (RGB-OC-LBP). Their proposed RGB-
C-LBP operator performs better than the color SIFT operator in
mage matching, object recognition, and scene classification appli-
ations. Recently, Lan et al. [44] have introduced quaternion local
anking binary pattern (QLRBP), which combines the color infor-
ation provided by multispectral channels in color images. The
LRBP operator is derived using quaternionic representation (QR)
f color images. The QLRBP can handle all color channels directly
n the quaternionic domain and represents color texture without
reating the color channels separately. Li et al. [45] have developed
ocal similarity pattern (CLSP) for representing the color image as
he co-occurrence of its image pixel color quantization informa-
ion and the local color image textural information. In an attempt
o utilize the cross channel information, Dubey et al. [46] have
eveloped two sets of patterns, multichannel adder and decoder-
ased LBPs (MDLBPs) and observed that the latter provides better
etrieval performance than the former. The performance of their
roposed approach is better than the other similar LBP-based de-
criptors, but the size of the feature vectors for both methods is
arge.
In this paper, we propose an operator called local binary pat-
ern for color images (LBPC), which derives texture patterns for
color image similar to the way LBP operator derives texture for
ray scale images. For this purpose, we treat a color pixel as a vec-
or having m -components and form a hyperplane. The hyperplane
s used as a boundary to threshold and partition color pixels into
wo classes. A color pixel in a 3 × 3 neighborhood of the current
ixel is assigned a value 1 if it lies on or above the plane, and
he value 0, if it is below the plane. Thus, the proposed operator
rovides spatial relationship among color pixels, which represents
ocal texture features. We compute histograms of the binary pat-
erns obtained from color images in a way akin to the histograms
f LBP operator for gray scale images. If we use 8-neighborhood of
color pixel, then we have 256 histogram bins, which are used as
eatures representing local texture patterns of color images. We re-
er this operator as the LBPC operator. The dimensionality of LBPC
s reduced by deriving uniform patterns [12] with 59 bins. Since
e are investigating effective descriptors for color images, a natu-
al question arises as what is the retrieval performance when we
se local binary patterns of the color component of color images.
or the purpose of representing local binary patterns of the color
omponents, the H component in the HSI color model [47] is an
ppropriate choice as compared to all other color models. There-
ore, in our second proposed method, we derive the local binary
atterns of H component of HSI color model. We refer this descrip-
or as local binary pattern of the hue (H) component (LBPH). The
iscriminative power of LBPC is enhanced by fusing the LBPC fea-
ures with LBPH features. This approach is referred as LBPC + LBPH.
he color histograms are among the best color image descriptors
4 8,4 9] and hence, in our proposed third approach, we derive color
istogram (CH) of H channel in the HSI color model and fuse it
ith LBPC + LBPH. This approach is referred as LBPC + LBPH + CH. In
rder to reduce the dimension of LBPC + LBPH + CH, we derive the
niform patterns of LBPC and LBPH and fuse them to the CH fea-
ures, which yields an effective low dimension local color texture
52 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
w
n
g
s
λ
θ
s
a
g
w
u
t
f
t
2
o
a
b
h
o
4
t
r
O
a
O
w
S
a
(
t
r
t
c
t
o
t
t
f
f
f
2
t
p
s
c
descriptor called ULBPC + ULBPH + CH whose performance is compa-
rable to LBPC + LBPH + CH. All existing state-of-the-art methods and
proposed methods have been implemented to analyze and com-
pare their image retrieval performance. The feature vector dimen-
sion plays an important role in analyzing retrieval speed of a fea-
ture extraction method. Therefore, the dimension of each method
is taken into account while performing their comparative analysis.
The rest of the paper is organized as follows. In Section 2 , we
provide an overview of some of the existing state-of-the-art local
binary pattern-based descriptors for color images. The proposed
operator LBPC has been derived in Section 3 . Discussion on the
parameters used by LBPC operator is presented in Section 4 . In
Section 5 , we discuss the LBPH operator and the color histogram
(CH) features and their fusion with the LBPC features. Section 6 ex-
plains the various similarity measures and performance parameters
used in image retrieval applications. Detailed experimental analy-
sis of retrieval performance and computation time on various color
image databases is presented in Section 7 . Section 8 concludes the
paper.
2. Related work
In this section, we present an overview of the closely related
works which pertain to the LBP-like methods developed for the
color images to utilize the benefits of cross-correlation among the
color channels. Each method has its own merits and demerits as
explained below.
2.1. Multispectral local binary pattern (MSLBP)
Mäenpää et al. [39] use three channels of a color image in the
RGB color space and six pairs of the opponent colors. The oppo-
nent colors are used to obtain color texture features of two col-
ors, which represent cross-correlation between color values as well
as spatial relationships between them. The three LBP feature vec-
tors are obtained in a way similar to the gray scale LBP features
by treating each of the channels of an RGB image as a gray scale
image [12] . The six opponent LBP feature vectors are derived as
follows:
MSLB P ( i, j ) ( x c , y c ) =
P −1 ∑
p=0
S( v i ( x p , y p ) − v j ( x c , y c ) ) × 2
p , (1)
where ( i, j ) ∈ {(1, 2), (2, 3), (3, 1), (2, 1), (3, 2), (1, 3)}, and
S (v i ( x p , y p ) −v j ( x c , y c )
)=
{1 , i f v i ( x p , y p ) − v j ( x c , y c ) ≥ 0 ,
0 , otherwise.
(2)
Here, v j ( x c , y c ) is the intensity value of the center pixel of a
3 × 3 window from the j th color component image, v i ( x p , y p ) is the
intensity value of the p th neighborhood pixel from the i th color
component image. The total number of MSLBP features is 2304,
which is the size of the feature vector obtained after the concate-
nation of nine LBP operators. The approach provides high recogni-
tion rate. However, the size of the feature vector is too large which
slows down the retrieval speed.
2.2. Local color vector binary pattern (LCVBP)
Local color vector binary pattern [42] consists of color norm bi-
nary patterns and color angular binary patterns. For color norm
binary patterns, we take the norm of all color pixel values as fol-
lows:
I ( x, y ) = ‖
I ( x, y ) ‖
=
√
r 2 ( x, y ) + g 2 ( x, y ) + b 2 ( x, y ) , (3)
here r ( x, y ), g ( x, y ), b ( x, y ) are the pixel values of R, G, B compo-
ents at ( x, y ) coordinates. Thus, a color image is converted into a
ray scale image I ( x, y ) from which LBP feature vector is obtained.
The color angular features value θ ( i, j ) between the i th and j th
pectral bands is computed by
( i, j ) ( x, y ) =
v i ( x, y )
v j ( x, y ) + ε, ( i, j ) ∈ { ( 1 , 2 ) , ( 2 , 3 ) , ( 1 , 3 ) } (4)
( i, j ) ( x, y ) = ta n
−1 (λ( i, j ) ( x, y )
). (5)
Here, ε is a small parameter to avoid the denominator to as-
ume zero value.
The images λ( i, j ) ( x, y ) are calculated between R and G, G and B,
nd B and R components. The three images λ( i, j ) ( x, y ) are treated as
ray images and their three LBP feature vectors are computed in a
ay similar to the LBP operator for gray images. The full LCVBP
ses 4 × 256 = 1024 features. To reduce the number of features,
he authors have proposed uniform LCVBP for which the size of
eature vector is 4 × 59 = 236. This method provides a good
radeoff between the retrieval performance and retrieval time.
.3. Orthogonal combination of local binary pattern (OC-LBP)
The OC-LBP operator [43] reduces the dimensionality of the LBP
perator. It consists of two operators OC-LBP 1 and OC-LBP 2, which
re derived by splitting 8-neighborhood pixels into 2 sets of neigh-
orhoods each comprising 4 pixels. The first set contains the four
orizontal and vertical neighbors of the center pixel and the sec-
nd set contains the 4 diagonal neighbors. Since each set contains
pixels, the total number of patterns in each set is 16. Thus, the
wo sets contain 32 patterns of the OC-LBP operator, which are de-
ived as follows.
C − LB P 1 ( x c , y c ) =
3 ∑
p=0
S( I 2 p − I c ) × 2
p , (6)
nd
C − LB P 2 ( x c , y c ) =
3 ∑
p=0
S( I 2 p+1 − I c ) × 2
p , (7)
here
( z ) =
{1 , i f z ≥ 0 ,
0 otherwise. (8)
The values I c and I p represent intensity values of a gray im-
ge or of a component image in a color space at the center pixel
x c , y c ) and at a neighborhood pixel ( x p , y p ), respectively. In [43] ,
he OC-LBP operator has been derived on six color models, which
epresent various illumination changes. It has been observed that
he RGB-OC-LBP operator provides overall best results in the object
lassification problem. The RGB-OC-LBP operator is applied on the
hree component images in the RGB color space by treating each
f the component image as a gray image. Thus, there are 96 fea-
ures derived by the RGB-OC-LBP operator. The main purpose of
his method is to reduce the dimensionality of the feature vector
rom 768 LBP features of the component images to 96 features, 32
eatures each for the three component images. The retrieval per-
ormance is, however, low.
.4. Quaternionic local ranking binary pattern (QLRBP)
Lan et al. [44] have used quaternionic representation (QR) of
he color images to obtain color texture features. Similar to the
rocedure followed for gray scale images to obtain LBP, they con-
ider a 3 × 3 neighborhood of a color pixel. By using a reference
olor pixel ( r ′ , g ′ , b ′ ) and a color pixel ( r, g, b ) in the window, they
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 53
d
q
o
d
b
θ
w
p
p
a
o
a
o
t
f
i
θ
a
7
l
m
2
p
i
W
t
d
c
u
w
t
t
i
L
w
b
w
t
m
d
C
w
t
t
t
2
b
r
[
c
o
L
t
1
i
m
m
m
{
e
l
l
t
m
m
m
l
l
f
d
m
erive an operator called QLRBP. They use Clifford translation of
uaternionic (CTQ) ranking function, which is complex. The phase
f the function is used for the ranking purpose in a 3 × 3 win-
ow. The CTQ phase between two color pixels ( r, g, b ) and ( r ′ , g ′ ,
′ ) is given by
( x, y ) = ta n
−1
√
( gb ′ − bg ′ ) 2 + ( br ′ − rb ′ ) 2 + ( r g ′ − gr ′ ) 2
−( r r ′ + g g ′ + bb ′ ) , (9)
here r = r( x, y ) , g = g( x, y ) , and b = b( x, y ) . The angles are com-
uted for all 9 pixels in a 3 × 3 window. One may note that the
hase angle θ ( x, y ) is the angle between two vectors I = ( r, g, b )
nd I ′ = ( r ′ , g ′ , b ′ ) . Therefore, the resulting QLRBP operator is an
perator which is based on the angle between a reference pixel I ′ nd a pixel I of the color image. Therefore, the QLRBP features are
btained by considering the angle image θ ( x, y ) in a way similar to
he LBP features of a gray scale image. In order to provide the dif-
erent weights to different color channels, the weighted L 1 -phase
s used by Lan et al. [44] as follows.
( x, y ) = ta n
−1 α1 | gb ′ − bg ′ | + α2 | br ′ − rb ′ | + α3 | r g ′ − gr ′ | −( r r ′ + g g ′ + bb ′ )
· (10)
By setting I ′ to a given value, 256 features are obtained. The
uthors have used three sets of I ′ to drive a feature vector of size
68. Since the reference vector I ′ is set for the whole image, no
ocal information is exploited fully to derive high retrieval perfor-
ance.
.5. Completed local similarity pattern (CLSP)
The CLSP method [45] consists of two phases. In the first
hase, a color image is quantized into S number of clusters us-
ng the K -means algorithm, creating a dictionary of color words
= { w 1 , w 2 , . . . w S } . A cluster word represents its center w i =( r i , g i , b i ) , i = 1 , . . . , S. Next, for each pixel p of the color image,
he Euclidean distance from each cluster center is obtained, i.e.
i 2 (p) = ‖ I p − w i ‖ 2 2
, i = 1 , 2 , . . . , S.
The nearest K 1 distances are selected and the weights u i ( p ) are
omputed as follows.
i ( p ) = u i ( x, y ) =
exp
(−βd i
2 ( p )
)∑ K 1
i =1 exp
(−βd i
2 ( p )
) , i ∈ K 1 − N N ( p ) , (11)
here K 1 − N N (p) denotes a set of K 1 -nearest neighbors of p ob-
ained by using Euclidean distance, and β is a smoothing parame-
er which is taken as 2.
The second phase of CLSP method determines the local similar-
ty pattern (LSP) in a 3 × 3 window as follows.
SP ( x c , y c ) =
P−1 ∑
p=0
b P × 2
p , (12)
here,
P =
{1 , p ∈ K 2 − N N ( x c , y c )
0 otherwise, (13)
here K 2 − N N ( x c , y c ) denotes the K 2 -nearest neighbors of the cen-
er pixel ( x c , y c ) in a 3 × 3 window using the following distance
etric
P =
‖
I p − I c ‖
‖
I p + I c ‖
, p = 0 , 1 , . . . , P − 1 . (14)
Finally, the CLSP histograms are obtained by using the equation:
LSP ( s, t ) =
M−1 ∑
x =0
N−1 ∑
y =0
u s ( x, y ) δ( LSP ( x, y ) = t ) , (15)
here M × N represents the size of the image, and s = 1 , 2 , . . . , S,
= 1 , 2 , . . . , 2 P . In particular, in [45] S = 10 , and P = 8 . The size of
he feature vector is S × 2 P = 2560 . Despite the large size of its fea-
ure vector, the performance of the CLSP operator is not very high.
.6. Multichannel decoded linear binary pattern
Recently, in a new approach, two multichannel decoded local
inary patterns (MDLBP) have been derived for color images to
epresent the cross-channel correlation of the color components
46] . The mentioned approach works as follows. Let the color
hannels be represented by R, G, and B. In the first stage, LBPs
f each channel using 8 neighborhood configuration are derived.
et LBP i t ( x, y ) , i = 0 , 1 , . . . , 7 , represent the i th bit of a string of
he LBP of the t th component at a pixel location ( x , y ). Here, t = , 2 , 3 , and these values correspond to the R , G , and B component
mages. In the first stage, the multichannel adder maLBP i (x, y ) , and
ulti-channel decoder mdLB P i ( x, y ) , i = 0 , 1 , . . . , 7 , are derived as:
aLB P i ( x, y ) =
3 ∑
t=1
LBP i t ( x, y ) , i = 0 , 1 , . . . , 7 , (16)
dLB P i ( x, y ) =
3 ∑
t=1
2
t−1 × LBP i t ( x, y ) , i = 0 , 1 , . . . , 7 . (17)
It is noted that maLBP i (x, y ) ∈ { 0 , 1 , 2 , 3 } , and mdLB P i ( x, y ) ∈ 0 , 1 , . . . , 7 } . Next, four multichannel adder bins maAM
k (x, y ) and
ight multichannel decoder bins mdDM
k (x, y ) are obtained as fol-
ows:
maA M
k ( x, y ) ← 0 , k = 0 , 1 , 2 , 3 , 4 ; mdD M
k ( x, y ) ← 0 ,
k = 0 , 1 , . . . , 7 , (18)
k ← maLB P i ( x, y ) ; maA M
k ( x, y ) ← maA M
k ( x, y ) + 2
i ,
i = 0 , 1 , . . . 7 , (19)
k ← md LB P i ( x, y ) ; md D M
k ( x, y ) ← md D M
k ( x, y ) + 2
i ,
i = 0 , 1 , . . . 7 . (20)
The histogram bins contain values between 0 and 255 at a pixel
ocation ( x, y ), represented by l . For the whole image, the 4 mul-
ichannel adder maAH
k (l) and 8 multichannel decoder histograms
dDM
k (l) are obtained using:
aA H
k ( l ) ← 0 , k = 0 , 1 , 2 , 3 , l = 0 , 1 , . . . , 255 , (21)
dD H
k ( l ) ← 0 , k = 0 , 1 , . . . , 7 , l = 0 , 1 , . . . , 255 , (22)
← maA M
k ( x, y ) ; maA H
k ( l ) ← maA H
k ( l ) + 1 , k = 0 , 1 , 2 , 3 ,
(23)
← md D M
k ( x, y ) ; md D H
k ( l ) ← md D H
k ( l ) + 1 , k = 0 , 1 , . . . 7 ,
(24)
or x = 0 , 1 , . . . , M − 1 , and y = 0 , 1 , . . . , N − 1 .
There are 4 × 256 multichannel adder histograms and 8 × 256
ecoder histograms. Although it provides good recognition perfor-
ance, the size of feature vectors is very high.
54 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
8 neighbors(a)
(P=8, R=1)(b)
(P=12, R=1.5)(c)
(P=8, R=2)(d)
(P=16, R=3)(e)
Fig. 1. 8 neighbors and circularly symmetric neighbor sets for different ( P, R ): (a) 8 neighbors, (b) P = 8 , R = 1 , (C) P = 12 , R = 1 . 5 , (d) P = 8 , R = 2 , and (e) P = 16 , R = 3 .
a
a
d
t
o
u
t
b
4
a
e
u
t
w
t
t
a
i
T
a
s
b
n
∑
w
n
u
s
o
t
q
3. The proposed local binary patterns for color images (LBPC)
3.1. The classical LBP operator
The classical LBP operator is defined for the gray scale images.
The general form of the operator for a circularly symmetric neigh-
bor set of P members on a circle of radius R and center ( x c , y c ),
denoted by LBP P, R ( x c , y c ) is defined as:
LB P P,R ( x c , y c ) =
P−1 ∑
p=0
S ( I ( x p , y p ) − I ( x c , y c ) ) × 2
p , (25)
where
S ( I ( x p , y p ) − I ( x c , y c ) ) =
{1 i f I ( x p , y p ) − I ( x c , y c ) ≥ 0
0 otherwise, (26)
and I ( x, y ) represents the intensity at pixel location ( x, y ). In its
simplest form, 8 neighborhood pixels of a pixel at ( x, y ) are consid-
ered, i.e., P = 8 , and R = 1 . This form provides a fast approach for
the computation of the LBP without involving a circular symmetry
of the members which needs interpolation of intensity. The gen-
eral case of the LBP P, R ( x c , y c ) specifies P equally-spaced locations
on a circle of radius R and center ( x c , y c ) whose coordinates are
determined by
x p = x c + Rcos ( 2 π p/P ) ,
y p = y c + Rsin ( 2 π p/P ) , (27)
p = 0 , 1 , . . . , P − 1 . The intensity values at locations which do not
fall at the original positions are interpolated using bilinear inter-
polation. Fig. 1 (a) depicts the 8 neighborhood configuration and
Fig. 1 (b)–(e) depict the circularly symmetric neighborhoods with
(P, R ) ∈ { (8 , 1) , (12 , 1 . 5) , (8 , 2) , (16 , 3) } in the increasing order of
the radius. The parameter P determines the angular space between
members and the parameters R determines the size of the win-
dow. The original locations of the pixels are shown by open bullets
nd the P neighborhoods by the solid bullets. An original pixel lies
t the cross-section of the horizontal and vertical lines drawn by
ashed lines. The purpose of using such types of grid is to show
he locations of the neighbors which fall on the original positions
f the pixels of the image. For example, for 8 neighborhood config-
ration shown in Fig. 1 (a), all 8 neighbors fall on the original loca-
ions whereas for P = 8 , R = 1 , and P = 8 , R = 2 , of the 8 neigh-
ors, 4 neighbors fall on the original locations and the remaining
neighbors are interpolated. For P = 12 , R = 1 . 5 , all 12 neighbors
re interpolated. The two types of grid used here show the pix-
ls which take part to interpolate a neighbor shown by gray shade
sing the bilinear interpolation which requires 4 pixels for the in-
erpolation process.
The LBP operator derives binary patterns, called LBP patterns,
hose values lie between 0 and 2 P−1 . The LBP patterns are ob-
ained for each pixel of an image and the features are obtained in
he form of the histograms of the LBP patterns. These histograms
re used to represent the texture of a gray scale image. The higher
s the value of P , the larger is the size of the histogram patterns.
o reduce the size of the feature vector “uniform” binary patterns
re used. Let LBP P, R ( x, y ) denote the LBP value at pixel ( x, y ). Let
denote the string of the binary values. Clearly, | s | = P . Out of 2 P
inary patterns only those patterns are termed as “uniform” (de-
oted by ULBP ) which satisfy the following condition:
P
i =1
| s i − s i −1 | + | s 0 − s P | ≤ 2 (28)
here s i is the i th bit of the string. All other patterns are termed
on-uniform and placed into a single group. The total number of
niform patterns is P ( P − 1 ) + 3 which is much smaller than 2 P ,
pecially when P is large. For example, for P = 8 , the total number
f the LBP and ULBP patterns are 256, and 59, respectively.
The classical LBP operator for gray scale images cannot be ex-
ended to color images because a color pixel represents a vector
uantity with ( R , G , B ) components whereas a gray scale pixel is
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 55
a
E
3
t
m
c
t
a
W
m
c
a
c
fi
i
s
a
p
n
o
n
w
a
w
s
g
a
o
p
p
v
t
m
R
n
w
H
t
E
a
t
w
e
a
g
e
s
w
s
r
L
S
I
h
t
n
t
t
s
p
p
4
L
R
t
c
d
c
c
s
4
4
t
a
t
t
e
r
i
r
t
o
4
l
R
B
(
4
c
scalar quantity. Therefore, a comparison of the type given by
q. (26) cannot be made for color pixels to get a binary string.
.2. The proposed LBPC operator
The proposed LBPC operator is based on the concept of parti-
ioning or thresholding color pixels using a hyper-plane in m di-
ensions. Since m = 3 in the present case, the hyper-plane be-
omes an Euclidean plane in 3-D color space. There are other
hresholding alternatives, such as a hyper-sphere, a hyper-cube or
hyper-ellipsoid, but they are not as effective as the hyper-plane.
e have come to this conclusion after conducting detailed experi-
ents using these thresholding alternatives.
The task ahead is to derive a thresholding plane in 3-D
olor space which is performed as follows. Let a vector I( x, y ) =( r( x, y ) , g( x, y ) , b( x, y ) ) or simply, I = ( r, g, b ) be used to represent
color pixel in the R, G, B color space. Thus, the number of color
omponents m is 3. We define a local window of size ( 2 R + 1 ) ×( 2 R + 1 ) , R ≥ 1 , centered at a pixel c with the color vector I c =( r c , g c , b c ) . Let I p = ( r p , g p , b p ) be a neighborhood pixel p . We de-
ne a color plane L in the color space. A plane is derived by defin-
ng the normal to the plane and a reference point in the color
pace. Let the normal to the plane be denoted by n = ( n 1 , n 2 , n 3 )
nd the reference point R o = ( r o , g o , b o ) , then the equation of the
lane with normal vector n and reference point R o is given by:
· ( I p − R o ) = 0 , (29)
r,
1 ( r − r o ) + n 2 ( g − g o ) + n 3 ( b − b o ) = 0 . (30)
hich is the result of the dot product between the vector n
nd the vector formed by joining R o = ( r o , g o , b o ) with I p = ( r, g, b )
hich yields the vector I p − R o . The color plane divides the color
pace into two classes. All pixels that lie on or above the plane are
rouped in one class and all other pixels that lie below the plane
re grouped in another class.
We can select the normal vector n in many ways but an obvi-
us choice is a line joining the pure black pixel (0, 0, 0) and the
ure white pixel (1, 1, 1). This represents gray intensity line and all
rimary colors R, G, and B are present in equal proportion. An ob-
ious choice for the reference point would be the center pixel I c =( r c , g c , b c ) of the square neighborhood. With these values of the
wo parameters, we define a plane called color plane, which is nor-
al to the line joining (0, 0, 0) and (1, 1, 1) and passes through
o = I c = ( r c , g c , b c ) . This facilitates the process of thresholding the
eighborhood pixels I p , p = 0 , 1 , . . . .P − 1 into two classes: those
hich lie on or above-plane and those which lie below-plane.
ere, P is the total number of neighborhood pixels. A decision to
his effect can be taken by evaluating the following expression.
( I p ) = E ( r, g, b ) = n 1 ( r p − r c ) + n 2 ( g p − g c ) + n 3 ( b p − b c ) , (31)
A color pixel I p = ( r p , g p , b p ) is above or on the plane, if E ( I p ) ≥ 0
nd it is below the plane if E ( I p ) < 0 . Therefore, the color pixels in
he neighborhood of the center pixel are divided into two classes
ith a well-defined mechanism. This can be thought of a natural
xtension of the local binary patterns (LBP) of the gray scale im-
ges which are obtained by thresholding the gray values about the
ray value of the center pixel ( x c , y c ) of a local window. In fact, the
xpression in the right hand side in Eq. (31) reduces to an expres-
ion which is used for the classical LBP operator for a gray image if
e set g = b = 0 , and the R -component image is treated as repre-
enting a gray image where r o = r c and n 1 = 0. Mathematically, we
epresent the LBPC as follows.
BP C ( x c , y c ) =
P−1 ∑
p=0
S ( I p ) × 2
p , (32)
( I p ) =
{1 ,
0 ,
E ( I p ) ≥ 0 ,
E ( I p ) < 0 . (33)
The expression E ( I p ) is evaluated using Eq. (31) , where I p = p ( x, y ) , p = 0 , . . . , P − 1 , represents a color pixel in the neighbor-
ood of the pixel I c = I c ( x, y ). For the 8-neighborhood P = 8.
Fig. 2 displays color images taken from Corel-5k dataset [50] in
he first column, columns 2 to 4 display LBP images of the compo-
ent images of R, G, and B, column 5 displays the LBP images of
he intensity value I = ( R + G + B ) / 3 and the last column displays
he LBPC image. It is observed that the LBPC image not only pre-
erves the shape of an object, but it also provides a dense texture
attern as compared to the texture patterns obtained by the com-
onent images and the intensity image.
. Discussion on the parameters used by the LBPC operator
There are three parameters that determine the performance of
BPC: the normal to the plane n = ( n 1 , n 2 , n 3 ) , the reference point
o = ( r o , g o , b o ) through which the plane passes, and the size of
he circularly symmetric neighborhood window( P, R ). A common
hoice for the size of the neighborhood is a 3 × 3 window with
( P, R ) = ( 8 , 1 ) . There are other values of ( P, R ) such as ( P, R ) =( 8 , 2 ) , ( 12 , 1 . 5 ) , ( 16 , 3 ) , and so on, with the neighborhood win-
ow size ( 2 R + 1 ) × ( 2 R + 1 ) . We have shown in Section 3.1 the
onstruction and configuration of some of the neighborhoods. One
an also refer [25,31] for the construction of high order circularly
ymmetric neighborhoods.
.1. Normal to the plane
.1.1. Global natural normal
A most appropriate choice for the direction of the normal to
he plane is parallel to the gray line joining pure black pixel (0,0,0)
nd pure white pixel (1,1,1). This choice of the normal vector leads
o n 1 = n 2 = n 3 . This provides us an isometric plane whose orien-
ation with respect to R, G, B axis is the same. In this case, the
quation of the plane given by Eq. (30) reduces to
+ g + b − ( r o , g o , b o ) = 0 . (34)
The thresholding operation given by Eq. (30) compares the two
ntensity values I = (r + g + b)/3 and I o = ( r o + g o + b o ) / 3 , at I p and I o ,
espectively, which does not contain color information. Therefore,
he contribution of the color texture for the derivation of the LBPC
perators becomes ineffective.
.1.2. Global average normal
We find the global centroid of the color components as fol-
ows.
a =
1
MN
M−1 ∑
x =0
N−1 ∑
y =0
r(x, y ) , G a =
1
MN
M−1 ∑
x =0
N−1 ∑
y =0
g(x, y ) ,
a =
1
MN
M−1 ∑
x =0
N−1 ∑
y =0
b(x, y ) . (35)
The unit normal vector is set to n =R a √
R a 2 + G a 2 + B a 2
, G a √
R a 2 + G a 2 + B a 2
, B a √
R a 2 + G a 2 + B a 2
).
.1.3. Local average normal
The neutral normal n 1 = n 2 = n 3 and the global normal n =( R a , G a , B a ) do not provide the local characteristics of the spatial
orrelation of the color pixels. In order to extract the local features,
56 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
Fig. 2. Original color images and their LBP and LBPC images: (a) original color image, (b) to (e) LBP of R,G,B, and I components, and (f) LBPC image. (For interpretation of
the references to color in this figure legend, the reader is referred to the web version of this article.)
4
c
m
L
l
r
r
R
n
4
we derive local normal to the plane. A local normal is obtained af-
ter averaging the color components in the local window of size
( 2 R + 1) × ( 2 R + 1 ) :
R l =
1
( 2 R + 1 ) 2
2 R ∑
x =0
2 R ∑
y =0
r ( x, y ) G l =
1
( 2 R + 1 ) 2
2 R ∑
x =0
2 R ∑
y =0
g ( x, y ) ,
B l =
1
( 2 R + 1 ) 2
2 R ∑
x =0
2 R ∑
y =0
b ( x, y ) . (36)
The unit normal vector to the plane is set to:
n =
(
R l √
R l 2 + G l
2 + B l 2 ,
G l √
R l 2 + G l
2 + B l 2 ,
B l √
R l 2 + G l
2 + B l 2
)
.
4.1.4. Center normal
Another important method to represent local characteristics is
to use the normal to the plane whose direction ratios are in
proportion to the color components of the center pixel. Let I c =( r c , g c , b c ) represent the center pixel. If n 1 = r c , n 2 = g c , and n 3 =b c , then the unit normal to the plane is defined as center normal.
The unit normal vector is given by
n =
(
r c √
2 2 2 ,
g c √
2 2 2 ,
b c √
2 2 2
)
.
r c + g c + b c r c + g c + b c r c + g c + b c f
.1.5. Mean normal
The minimum and the maximum of the components of the
olor vectors in the neighborhood window are obtained, and their
ean values are assigned to the components of the normal vector.
ike the local normal and the center normal, it also represents the
ocal characteristics of the color image.
min = min ( r p ) , g min = min ( g p ) , b min = min ( b p ) , 0 ≤ p ≤ P − 1 ,
(37)
max = max ( r p ) , g max = max ( g p ) , b max = max ( b p ) , 0 ≤ p ≤ P − 1 ,
(38)
m
= ( r min + r max ) / 2 , G m
= ( g min + g max ) / 2 , B m
= ( b min + b max ) / 2 ,
(39)
The unit mean normal vector is given by
=
(
R m √
R m
2 + G m
2 + B m
2 ,
G m √
R m
2 + G m
2 + B m
2 ,
B m √
R m
2 + G m
2 + B m
2
)
.
.2. Reference point
The color vector at the center of the window is the most pre-
erred choice for the reference point, i.e. ( r o , g o , b o ) = ( r c , g c , b c ) .
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 57
O
o
g
a
w
u
p
5
h
o
h
i
c
t
h
c
R
H
w
θ
b
t
t
l
L
L
w
S
T
a
c
v
h
i
n
f
n
v
6
6
f
m
f
a
t
e
a
O
s
n
d
t
p
t
m
f
f
(
o
C
D
E
D
w
D
S
D
6
q
s
a
P
w
p
a
P
P
ther choices for the reference point are taken as the average
f pixel values in the local window ( r o , g o , b o ) = ( R l , G l , B l ) or the
lobal average ( r o , g o , b o ) = ( R a , G a , B a ) or a neutral color, such
s ( r o , g o , b o ) = ( 0 . 5 , 0 . 5 , 0 . 5 ) and the mean value ( r o , g o , b o ) =( R m
, G m
, B m
) . Among the various choices for the reference point,
e find that the center pixel ( r c , g c , b c ) and the average color val-
es of local window ( R l , G l , B l ) are the most effective reference
oints.
. Local binary pattern of the hue component (LBPH) and color
istogram (CH) and their fusion with LBPC features
The color is a powerful image descriptor. Many objects are rec-
gnized solely by their colors. The HSI color model provides the
ue (H) component, which represents the color texture of a color
mage. Therefore, to segment an image based on color, the hue
omponent is a natural choice. In addition to using RGB color space
o derive LBPC of a color image, we use color histogram (CH) of
ue component and derive the LBP of the hue image, which is
alled LBPH. The hue image is derived from the color image in the
GB color space using the following color conversion formula [47] .
( x, y ) =
{θ ( x, y ) i f b ( x, y ) ≤ g ( x, y ) ,
2 π − θ i f b ( x, y ) > g ( x, y ) , (40)
here
( x, y )
= co s −1
⎧ ⎨
⎩
1 2 [ ( r ( x, y ) − g ( x, y ) ) + ( r ( x, y ) − b ( x, y ) ) ] [
( r ( x, y ) −g ( x, y ) ) 2 + ( r ( x, y ) −b ( x, y ) ) ( g ( x, y ) −b ( x, y ) )
] 1 2
⎫ ⎬
⎭
.
(41)
The function H ( x, y ) is treated like a gray image whose local
inary patterns are obtained in a way akin to the local binary pat-
erns of gray image. These features are referred to as LBPH fea-
ures. The procedure for deriving LBPH features is explained as fol-
ows.
Let ( x c , y c ) be the center of a ( 2 R + 1 ) × ( 2 R + 1 ) window. The
BPH features are obtained as
BP H( x c , y c ) =
P−1 ∑
p=0
S( Q p ) × 2
p , (42)
here
( Q p ) =
{1
0
i f H( x p , y p ) ≥ H( x c , y c ) , otherwise.
(43)
Here ( x p , y p ) is the coordinate of a neighborhood pixel.
Color histograms (CH) are very effective image descriptors [48] .
herefore, in our proposed methods, we include CH features, which
re derived as follows.
Let n be the total number of color bins and h i denote the i th
olor bin for i = 0 , 1 , . . . .n − 1 , which is initialized to zero. The
alue of the i th bin is updated as:
i ← 0 , i = 0 , 1 , 2 , . . . , 2
P − 1 . (44)
= int ( nθ ( x, y ) / 2 π) , h i ← h i + 1 . (45)
f or x = 0 , 1 , . . . , M − 1 , y = 0 , 1 , . . . , N − 1 .
Here, it is assumed that θ ( x, y ) ∈ [0, 2 π ]. All histogram bins are
ormalized by dividing them by the size of the image before their
usion. The three feature vectors LBPC, LBPH, and CH, of size n c ,
h , and n , respectively, are fused together to form a single feature
ector of size n c + n h + n whose components are represented by
{ LBP C [ 0 ] , LBP C [ 1 ] , . . . , LBP C [ n c − 1 ] , LBP H [ 0 ] , LBP H [ 1 ] ,
. . . , LBP H [ n h − 1 ] , CH [ 0 ] , CH [ 1 ] , . . . , CH [ n − 1 ] } .
. Similarity measures and performance parameters
.1. Similarity measures
The performance of a retrieval system depends not only on ef-
ective features but also on strong similarity measures or distance
etrics. There are several similarity measures, which have been
ound to be very successful in image retrieval systems. Since we
re dealing with histogram-based feature vectors, our choice for
he similarity measures will be based on this aspect. Some of the
ffective similarity measures for histogram-based feature vectors
re Chi-square, Canberra, extended-Canberra, and square-chord.
ther commonly used distance measures such as histogram inter-
ection, L 1 -norm, L 2 -norm, cos-correlation, Jeffrey distance, etc. are
ot as effective as the former ones. In [51] , the extended-Canberra
istance has been proposed and compared with several other dis-
ance measures and it has been found to outperform them. In this
aper, we carry out performance comparison analysis with respect
o these similarity measures and find out an effective distance
easure which provides the overall best retrieval performance. The
our distance measures considered in this paper are explained as
ollows.
Let F q
i and F t
i represent the i th feature components of the query
probe) ‘ q ’ and database (gallery) ‘t’ images, respectively. The size
f the feature vector is L . The four distance measures are given by:
anberra distance:
CD ( q, t ) =
L −1 ∑
i =0
∣∣F q i − F t
i
∣∣F q
i + F t
i
. (46)
xtended-Canberra distance :
ECD ( q, t ) =
L −1 ∑
i =0
∣∣F q i − F t
i
∣∣(F q
i + μq
)+ (F t
i + μt )
, (47)
here μq =
1 L
∑ L −1 i =0 F
q i
and μt =
1 L
∑ L −1 i =0 F
t i
. χ2 - distance :
Chi ( q, t ) =
L −1 ∑
i =0
(F q
i − F t
i
)2
F q i
+ F t i
. (48)
quare-Chord distance :
SC ( q, t ) =
L −1 ∑
i =0
(√
F q i
−√
F t i
)2
. (49)
.2. Performance parameters
In our experiments, each image in the database is used as a
uery image. The performance of an image retrieval system is mea-
ured using precision P ( N ) and recall R ( N ) for retrieving top N im-
ges, which are defined in [51] .
( N ) =
I N N
; R ( N ) =
I N M
, (50)
here I N is the number of relevant images retrieved from top N
ositions and M is the total number of images in the dataset that
re similar to query image. The average precision of a single query
(q ) is the mean of all precision values P (n ) , n = 1 , 2 , . . . , N, i.e.
( q ) =
1
N
N ∑
n =1
P ( n ) . (51)
58 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
r
c
c
M
o
a
v
1
a
p
t
o
2
7
w
c
o
o
d
r
d
i
a
t
r
e
i
c
1
c
m
1
s
a
e
7
w
p
m
W
C
o
c
t
a
p
m
n
s
U
T
t
t
b
l
The mean average precision ( mAP ) is the mean of the average
scores over all queries Q:
mAP =
1
Q
Q ∑
q =1
P ( q ) . (52)
The mAP measure contains both the precision and recall infor-
mation and represents the entire ranking [52] .
The P − R graph is not an appropriate measure when the num-
ber of relevant images in each class is variable. The bull’s eye per-
formance ( BEP ) is a better performance measure [53] which is de-
fined for a query image q as:
BEP ( q ) =
I q
M
, (53)
where M is the total number of relevant images in the database
corresponding to the query image q and I q represents the num-
ber of relevant images among the top 2 M retrievals. Clearly, I q ≤ M ,
even if the top 2 M images are searched in the database for the im-
ages relevant to the query image. The average BEP(mBEP) value for
all query images Q is defined as:
mBE P =
1
Q
Q ∑
q =1
BE P ( q ) . (54)
7. Experimental results
This section provides several experimental results to demon-
strate the effectiveness of the proposed methods and compare
their results with the closely related existing seven color texture
operators- LBP of component images, MSLBP, LCVBP, RGB-OC-LBP,
QLRBP, CLSP, and MDLBP. Out of the two MDLBP operators, the
decoder operator performs better than the adder operator [46] .
Hence, we consider the decoder operator for the performance com-
parison. The LBP of the component images is simply an exten-
sion of the LBP of images for component images R, G, and B of
a color image. We also choose Gabor filters because Gabor fil-
ters are among the most effective texture descriptors [5-9] which
have been applied for texture image retrieval for gray scale im-
ages. A design strategy adopted by Manjunath and Ma [5] and
Han and Ma [8] to derive Gabor filters provides better results over
its other implementations. The strategy developed by them en-
sures that the contours of half-peak magnitude support of the fil-
ter responses in the frequency domain touch each other while the
other strategies allow them to overlap. Thus, the Gaussian stan-
dard deviations in 2-dimensions are dependent on the other filter
parameters. These Gabor filter parameters are: maximum center
frequency U h = 0 . 38 , minimum center frequency U l = 0 . 05 , num-
ber of scales S = 4 , and number of orientations O = 4 . The size of
the filter is set to W = 11 . We conducted detailed experiments for
image retrieval and found that their design strategy and filter pa-
rameters yielding better results than the other implementations.
We apply Gabor filters using these parameters on the individual
components of color images and concatenate these features. For
deriving Gabor features, MATLAB functions have been used after
setting the Gabor parameters and modifying the code for the com-
putation of the Gaussian standard deviations in accordance with
works of Manjunath and Ma [5] and Han and Ma [8] . There are
16 Gabor filtered images for each component providing a total of
48 images. The mean and the standard deviation of the magnitude
of the filtered Gabor images have been derived to yield a feature
set of size 96 (16 images per component × 3 components × 2 fea-
tures per image). We have implemented our proposed and exist-
ing six methods in Visual C ++ 6.0 under Microsoft Windows en-
vironment on a PC with 2.50 GHZ CPU and 8GB main memory.
The methods LBP, MSLBP, LCVBP, RGB-OC-LBP, and MDLBP do not
equire any other parameter. The LBP operator is applied on each
omponent image, therefore, the size of LBP feature vector for a
olor image is 768. The size of MSLBP, LCVBP, RGB-OC-LBP, and
DLBP feature vector is 2304, 236, 96, and 2048, respectively. The
perator QLRBP requires three weight parameters α1, α2, and α3,
nd the value of the reference vector I ′ = ( r ′ , g ′ , b ′ ) . We set these
alues same as given by Lan et al. [44] . These values are: α1 = , α2 = 1 , and α3 = 1 . There are three reference vectors I ′ 1 =
( 0 . 9922 , 0 . 0857 , 0 . 090 ) , I ′ 2 = ( 0 . 0912 , 0 . 9908 , 0 . 0999 ) , and I ′ 3
=( 0 . 0852 , 0 . 0855 , 0 . 9927 ) . The three reference quaternions create
feature vector of size 768. The CLSP operator [45] requires three
arameters: size of visual words dictionary S which is set to 10 and
he nearest neighborhood parameters K 1 = 6 , and K 2 = 3 . The CLSP
perator creates a feature vector whose size is S × 256, which is
560.
.1. Datasets
Wang or SIMPLIcity [54] : It is a subset of Corel image database,
hich contains 10 0 0 color images, which are divided into 10
lasses of 100 images each. Each class contains images with res-
lution either 256 × 384 pixels or 384 × 256 pixels. The 10 classes
f Wang image database are: African people, beach, building, bus,
inosaur, elephant, flower, horse, glacier, and food.
Holidays [55] : It contains 500 image groups, and each of which
epresents a distinct scene in all 1491 personal holiday photos un-
ergoing various transformations such as rotation, viewpoint and
llumination changes, blurring, etc. The number of photos in an im-
ge group is variable. The dataset contains a large variety of scene
ypes such as natural, man-made, water and fire effects, etc. The
esolution of the images is very high (2448 × 3204), and for our
xperiments, we scale them to the size of 128 × 128 using bicubic
nterpolation of MATLAB library.
Corel- 5K [50] : This dataset contains 50 0 0 images and covers 50
ategories of images. Every category contains 100 images of size
92 × 128 or 128 × 192 pixels in JPEG format. The dataset Corel- 5K
ontains images including diverse contents such as tiger, mountain,
ushroom, fort, ocean, car, ticket, etc.
Corel- 10K [50] : This dataset contains 10,0 0 0 images and covers
00 categories of images. Every category contains 100 images of
ize 192 × 128 or 128 × 192 pixels in JPEG format. It consists of im-
ges such as cat, rose, sunset, duck, train, musical instrument, fish,
agle, judo-karate, etc.
.2. Selection of parameters for LBPC
There are three parameters to be fixed for the LBPC operator:
indow size, normal to the plane and reference point. To find the
roper window size for LBPC, we present the results of experi-
ents for retrieval performance using different window sizes on
ang and Holidays datasets. Wang dataset is a representative of
orel- 5K and Corel- 10K datasets. Holidays contains variable number
f images in the classes. Thus, the presentation of the results will
over the complete spectrum of our experimental setup. To reduce
he size of the presented data, we take the normal to the plane n
s the “local average normal” and the intensity values of the center
ixels as the reference point R O of the plane. The retrieval perfor-
ance for these two datasets are shown in Tables 1 , and 2 for a
umber of combinations of ( P, R ). For P = 8 and 12, we have pre-
ented results for LBPC and ULBPC. For P = 16 , only the values of
LBPC are shown to avoid large feature size of LBPC which is 2 16 .
he criteria for the selection of the best ( P, R ) values include re-
rieval performance and retrieval speed. After comparing the re-
rieval performance, the combination (8, 2) provides the overall
est results. The performance of the combination (8, 1) is much
ess than (8, 2). Both these combinations have the same retrieval
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 59
Table 1
Mean average precision (mAP) in percent for top 100 images, N = 100 , retrieved by LBPC and ULBPC on Wang dataset
for various neighborhood pixels (P), and radius (R).
(P, R) Method No. of features Distance measures Average
Chi-square Canberra Extended-Canberra Square-chord
(8,1) LBPC 256 54.31 53.85 57.45 54.33 54.98
ULBPC 59 52.61 55.77 55.36 52.62 54.09
(8,2) LBPC 256 55.49 56.27 58.05 55.50 56.32
ULBPC 59 53.28 55.68 55.43 53.29 54.42
(8,3) LBPC 256 54.22 54.60 56.19 54.21 54.66
ULBPC 59 52.20 53.85 54.10 52.18 53.08
(12,1.5) LBPC 4096 55.36 56.99 57.43 55.32 56.27
ULBPC 135 55.21 56.93 57.37 55.24 56.19
(12, 2) LBPC 4096 55.02 54.37 57.97 55.07 55.60
ULBPC 135 54.08 55.81 56.19 54.13 55.05
(16,2) ULBPC 243 55.15 56.56 57.21 55.22 56.03
(16,3) ULBPC 243 55.18 56.58 57.24 55.25 56.06
Table 2
Average bull’s eye performance (mBEP) in percent obtained by LBPC and ULBPC on Holidays dataset for various neigh-
borhood pixels (P), and radius (R).
(P, R) Method No. of features Distance measures
Chi-square Canberra Extended-Canberra Square-chord Average
(8,1) LBPC 256 58.60 58.459 60.48 58.52 59.01
ULBPC 59 57.07 58.33 57.87 57.00 57.57
(8,2) LBPC 256 58.29 59.46 60.60 58.11 59.11
ULBPC 59 56.30 57.15 57.35 56.34 56.79
(8,3) LBPC 256 56.54 57.84 58.52 56.25 57.29
ULBPC 59 54.72 55.75 55.92 54.53 55.23
(12,1.5) LBPC 4096 58.20 59.75 59.96 58.34 59.06
ULBPC 135 58.11 59.71 59.57 58.10 58.87
(12,2) LBPC 4096 57.43 59.32 58.11 58.23 58.27
ULBPC 135 57.37 58.66 58.67 57.26 57.99
(16,2) ULBPC 243 58.02 59.34 59.81 57.73 58.72
(16,3) ULBPC 243 57.70 57.02 57.98 57.38 57.52
t
i
p
t
l
u
t
s
a
p
m
i
n
c
t
t
t
i
p
7
t
R
c
m
t
c
t
a
a
f
a
L
a
t
t
i
i
f
L
L
v
o
w
t
s
M
L
a
h
T
s
r
t
ime because out of the 8 neighbors, 4 neighbors are interpolated
n both the cases. Also, the feature size is the same. Although, the
erformance of some of the ULBPCs of P = 12 , and 16 is higher
han the ULBPC of P = 8 , the size of their feature vectors is much
arger than P = 8 . Therefore, in our all foregoing experiments we
se P = 8 , and R = 2 .
The LBPC operator derives local texture for color images after
hresholding the color pixels using a plane in the local window of
ize 3 × 3. A plane requires two parameters: normal to the plane
nd a reference point. The performance of the LBPC operator de-
ends on the optimal choices of these parameters.
As discussed in Section 4 , we have several choices for the nor-
al to the plane and the reference point. The normal to the plane
ncludes the vector representing gray line using normal vector n =( 1 , 1 , 1 ) , the global average normal n = ( R a , G a , B a ) , local average
ormal n = ( R l , G l , B l ), mean normal n = ( R m
, G m
, B m
), and the
enter normal n = ( R c , G c , B c ) .
Among the various choices for the reference point R o , the cen-
er pixel R o = ( r c , g c , b c ) , is intuitively the best choice, because
he plane should pass through the center pixel about which the
hresholding operation is being performed. However, during exper-
mental analysis we observed that the local average normal, R o =( R l , G l , B l ) , taken as a reference point, also provides competitive
erformance.
.3. Comparison with existing methods
The performance of the proposed methods is compared with
he nine approaches: Gabor filtering, LBP, ULBP, MSLBP, LCVBP,
GB-OC-LBP, QLRBP, CLSP, and MDLBP using the mean average pre-
ision ( mAP ). The number of features used by all methods is also
entioned in the results. In order to analyze the effectiveness of
he “uniform” local binary patterns, it is also applied to the three
omponent images yielding the method ULBP. The uniform pat-
erns are also extended to LBPC and LBPH, resulting in the oper-
tors ULBPC and ULBPH, respectively. The proposed methods are
pplied separately as well in four combinations by fusing their
eatures. The individual methods are LBPC, LBPH, ULBPC, ULBPH,
nd CH and four combinations are LBPC + LBPH, ULBPC + ULBPH,
BPC + LBPH + CH, and ULBPC + ULBPH + CH. These nine methods are
nalyzed to compare their relative retrieval performance in order
o explore the best methods providing high accuracy and low re-
rieval time. It is important to note here that the LCVBP operator
s based on the “uniform” patterns, which provides very compet-
tive results in comparison to the full LCVBP operator using 256
eatures for each of the four combinations. The dimension of the
CVBP operator is 236 as compared to the dimension of the full
CVBP operator, whose size is 1024. Therefore, there is an ad-
antage of speed by using low dimensional feature vector with-
ut much loss of retrieval accuracy. A similar trend is observed
hile using LBPC and LPBH operators, which will be discussed in
he following experimental analysis. Therefore, we compare the re-
ults of 18 approaches. These approaches are: Gabor, LBP, ULBP,
SLBP, LCVBP, RGB-OC-LBP, QLRBP, CLSP, MDLBP, LBPC, ULBPC,
BPH, ULBPH, CH, LBPC + LBPH, ULBPC + ULBPH, LBPC + LBPH + CH,
nd ULBPC + ULBPH + CH.
The mAP values obtained by the various methods for top one
undred images ( N = 100 ) on the Wang dataset are presented in
able 3 . For convenience, we mark the top six methods with bold
tyle and also number them from 1 to 6 in the order of their
etrieval performance. We chose six methods because in most of
he cases our 4 proposed methods turn out to be on the top
60 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
Table 3
Mean average precision ( mAP ) in percent for N = 100 obtained by various approaches in RGB color space on Wang dataset using
various distance measures.
Method No. of features Chi-square Canberra Extended-Canberra Square chord
Existing LBP [12] 3 × 256 = 768 55.28 53.42 56.93 55.33
ULBP [12] 3 × 59 = 177 53.34 56.63 54.19 53.37
MSLBP [39] 9 × 256 = 2304 59.86 56.14 60.62 (5) 59.86
LCVBP [42] 4 × 59 = 236 53.42 57.19 56.83 53.44
RGB-OC-LBP [43] 3 × 32 = 96 47.90 51.18 49.39 47.93
QLRBP [44] 3 × 256 = 768 53.47 52.79 56.03 53.50
CLSP [45] 10 × 256 = 2560 48.37 43.25 45.84 48.05
GABOR [8] 96 58.86 59.54 59.53 58.91
MDLBP [46] 8 × 256 = 2048 59.58 58.15 60.82 (4) 59.58
Proposed LBPC 256 55.49 56.27 58.05 55.05
ULBPC 59 53.28 55.68 55.43 53.29
LBPH 256 49.24 47.57 50.72 49.31
ULBPH 59 47.30 47.77 48.75 47.33
CH 30 47.49 41.84 48.37 47.73
LBPC + LBPH 2 × 256 = 512 58.80 59.09 61.92 (3) 58.82
ULBPC + ULBPH 2 × 59 = 118 56.90 59.16 59.61 (6) 56.89
LBPC + LBPH + CH 2 × 256 + 30 = 542 57.06 61.12 65.16 (1) 56.12
ULBPC + ULBPH + CH 2 × 59 + 30 = 148 56.33 60.97 63.59 (2) 55.46
Average mAP – 53.99 54.32 56.21 53.88
a
C
(
i
i
v
U
t
A
m
a
t
0
L
v
t
and two more methods have been included from the existing
methods. All database images are used as query images. It is ob-
served from the table that the proposed approach LBPC + LBPH + CH
achieves the highest mAP value of 65.16% using the extended-
Canberra distance, followed by the same approach with uniform
patterns, ULBPC + ULBPH + CH, with mAP value of 63.59%. There is
not much difference between the two proposed approaches while
the latter uses only 148 features as compared to 542 features used
by the former approach. The third best mAP value is achieved by
LBPC + LBPH which is 61.92%. The next best mAP value is achieved
by MDLBP which is 60.82% using extended-Canberra. The fifth
best value of mAP is achieved by MSLBP method yielding mAP
value 60.62%. Among the nine state-of-the-art-methods LBP, ULBP,
MSLBP, LCVBP, RGB-OC-LBP, QLRBP, CLSP, GABOR, MDLBP, the per-
formance of MDLBP is the best which provides mAP of 60.82%, fol-
lowed by MSLBP, GABOR, LCVBP, LBP, ULBP, QLRBP, RGB-OC-LBP,
30
40
50
60
70
80
10 20 30 40 50 60 70Recall
Pre
cisi
on
(P) (%
)
Fig. 3. P-R curves for the seven existing methods MSLBP, GABOR, LCVBP, LBP, ULBP, QLRBP
for N = 1 to 100 on Wang dataset.
nd lastly CLSP, yielding mAP values of 60.62% (using extended-
anberra), 59.54% (using Canberra), 57.19% (using Canberra), 56.93%
using extended-Canberra), 56.63% (using Canberra), 56.03% (us-
ng extended-Canberra), 51.18% (using Canberra), and 48.37% (us-
ng chi-square), respectively. The proposed operator LBPC pro-
ides mAP value of 58.05%, which is much more than the LBP,
LBP, LCVBP, RGB-OC-LBP,QLRBP, and CLSP but slightly less than
he other three approaches such as MDLBP, MSLBP, and Gabor.
lthough MDLBP achieves highest mAP among the nine existing
ethods, it has slower retrieval speed (refer Section 7.4 for time
nalysis). When the feature dimension of LBPC is reduced from 256
o 59 by using ULBPC method, the reduction in mAP value is only
.67% while the feature dimension is reduced to 59 from 256. The
BPH and CH provide complementary features, leading to increased
alues of mAP when fused with LBPC or ULBPC. The performance of
he method LBPC + LBPH which yields mAP of 61.92% is much bet-
80 90 100
LBPC+LBPH+CH (Proposed)ULBPC+ULBPH+CH (Proposed)MSLBPGABORLCVBPLBPULBPQLRBPMDLBP
, and MDLBP, and two proposed methods, LBPC + LBPH + CH, and ULBPC + ULBPH + CH
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 61
Table 4
Class-wise mean average precision ( mAP ) in percent for N = 100 obtained by
LBPC + LBPH + CH and MDLBP [46] in RGB color space on Wang dataset using
extended-Canberra distance measure.
Class LBPC + LBPH + CH (Proposed) MDLBP [46] Difference
African people 61.52 55.75 5.77
Beach 43.36 45.21 −1.85
Building 54.57 56.81 −2.24
Bus 87.90 80.79 7.11
Dinosaur 98.88 97.10 1.78
Elephant 44.54 41.53 3.01
Flower 83.08 80.08 3.00
Horse 81.59 61.61 19.98
Glacier 39.89 33.8 6.09
Food 56.31 55.54 0.77
Average 65.16 60.82 4.34
t
m
f
f
m
u
b
b
U
p
t
a
m
C
t
e
t
C
5
v
t
s
n
a
a
G
t
r
G
s
a
a
o
p
c
m
n
i
i
n
o
n
a
m
A
v
d
p
o
i
t
p
t
t
M
r
t
M
t
M
a
f
c
t
o
w
p
a
i
p
U
v
C
w
m
t
er than the performance of their individual operators which yield
AP of 58.05% and 50.72%, respectively. When CH features are
used with LBPC + LBPH to obtain LBPC + LBPH + CH method, the per-
ormance of the resultant method is significantly increased to yield
AP of 65.16% from 61.92%. The method ULBPC + ULBPH + CH which
ses uniform patterns for LBPC and LBPH, yields mAP of 63.59%
y using only 148 features. Therefore, when a tradeoff is required
etween retrieval accuracy and retrieval speed, then the method
LBPC + ULBPH + CH is the first choice among all existing and pro-
osed approaches. Thus, the proposed method LBPC + LBPH + CH is
he best local descriptor for color texture images, followed by
nother proposed method ULBPC + ULBPH + CH that uses low di-
ension feature vector. Lastly, the performance of the extended-
anberra distance measure is overall the best which is reflected in
he average mAP values shown in the last row of Table 3 . The av-
rage mAP values are computed over all methods for a given dis-
ance measure. The average mAP obtained by extended-Canberra,
anberra, chi-square, and square-chord, are 56.21%, 54.32%, 53.99%,
3.88%, respectively. It is seen in the table that out of 18 best mAP
alues, the extended-Canberra provides the best results for thir-
een methods, while the Canberra and chi-square distance mea-
ures provide best results for four and one method, respectively.
The precision versus recall values is plotted in Fig. 3 for
ine methods (2 proposed, 7 existing) which provide the over-
ll best results. The two proposed methods are LBPC + LBPH + CH,
nd ULBPC + ULBPH + CH and the seven existing methods are MSLBP,
ABOR, LCVBP, LBP, ULBP, QLRBP, and MDLBP. It is observed from
he graph that the proposed methods consistently provide the best
etrieval rates for all values of recall followed by MDLBP, MSLBP,
ABOR, LCVBP, LBP, ULBP, and QLRBP. The trend of precision is the
ame for all values of recall N = 1 to 100 .
Table 5
Number of relevant images retrieved by MDLBP (Method A) and proposed method L
N Class
African people Beach Building Bus Dinosaur
A B A B A B A B A B
1 100 100 100 100 100 100 100 100 100 100
2 85 89 62 61 83 81 95 100 98 99
3 82 77 56 57 81 81 97 98 99 100
4 75 75 58 53 72 80 96 99 99 99
5 71 79 59 54 77 71 98 99 98 99
6 67 75 57 50 76 71 94 97 99 100
7 70 75 47 47 72 73 95 98 98 100
8 71 74 45 43 67 66 94 97 98 100
9 71 68 56 39 71 67 96 95 98 100
10 68 71 49 49 69 65 93 96 98 100
11 64 75 54 49 64 66 90 98 100 99
12 65 67 47 51 64 66 89 97 99 100
To analyze the comparative performance of the best methods
mong the proposed (LBPC + LBPH + CH) and the existing (MDLBP)
pproaches, we present the mAP values for each of the ten classes
f the Wang dataset in Table 4 for N = 100 . The performance of the
roposed method LBPC + LBPH + CH is significantly higher for the
lass Horse which is 81.59% as compared to MDLBP which yields an
AP of 61.61%, a value lower by 19.98%. The difference is also sig-
ificant for four other classes Bus, Glacier , and African people , which
s 7.11%, 6.09%, and 5.77%, respectively. We analyzed the objects
n this class and observed that many variations exist such as the
umber of horses which varies from one to six, view angle, texture
f the grass (background) and color of the enclosure (barrier). The
umbers of images having one, two, three, four, five and six horses
re 10, 73, 9, 3, 4, and 1. The other classes for which our proposed
ethod provides significantly better results are Bus, Glacier , and
frican people . Like the class Horse , these classes have a number of
ariations. Other classes do not have many variations in images. A
eeper analysis of the number of the retrieved relevant images was
erformed to compare the performance for each value of N , and
ut of 100 values of N , the results for N = 1 to 12 are presented
n Table 5 . The table shows the number of the relevant images re-
rieved by MDLBP (Method A) and LBPC + LBPH + CH (Method B) at
ositions N = 1 , 2 , . . . , 12 , for each of the classes. It is shown in the
able that for the class Horse the number of relevant images ob-
ained by Method B is significantly higher than those obtained by
ethod A, except for N = 7 , for which Method A provides 3 more
elevant images than Method B. We can observe similar trends for
he other three classes. Bus, Glacier , and African people for which
ethod B provides significantly better retrieval performance. On
he other hand, the class Building shows slightly better results for
ethod A. For other classes there is not very significant difference,
lthough Method B performs slightly better than Method A. Thus,
or out of ten classes, Method B provides better results for eight
lasses, while Method A provides slightly better results for only
wo classes. Our proposed method shows significantly better result
n one of the eight classes, Horse , which contains lots of variations
ithin the objects in the images of this class. The proposed ap-
roach is thus more robust to intra-class variations in the images
s compared to the existing methods.
The retrieval results on the Corel- 5K dataset are presented
n Table 6 . The maximum value of mAP is achieved by the
roposed method LBPC + LBPH + CH which is 43.81%, followed by
LBPC + ULBPH + CH yielding mAP of 41.68%. The next largest mAP
alue is achieved by MDLBP which is 39.99% using extended-
anberra. The fourth largest value of mAP is obtained by MSLBP
hich is 39.95%. Another proposed method LBPC + LBPH achieves
AP of 38.34%. The performance of other existing methods in
he decreasing mAP values is: LCVBP (38.14%), GABOR (37.18%),
BPC + LBPH + CH (Method B) for each value of N( N = 1 to 12 ) on Wang dataset.
Elephant Flower Horse Glacier Food
A B A B A B A B A B
100 100 100 100 100 100 100 100 100 100
78 85 98 94 94 99 53 62 80 81
80 80 95 95 93 97 36 53 80 72
62 75 92 95 92 95 37 50 70 75
60 67 92 93 90 94 35 49 75 78
51 50 92 91 89 91 33 54 68 68
53 57 92 92 95 92 35 44 61 73
49 49 92 91 87 94 33 42 68 70
47 51 91 92 83 94 34 48 65 60
36 47 86 92 82 87 32 44 67 65
36 42 88 94 82 90 35 43 63 64
44 47 93 88 74 88 33 46 60 64
62 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
Table 6
Mean average precision ( mAP ) in percent for N = 100 obtained by various approaches in RGB color space on Corel-5k dataset
using various distance measures.
Method No. of features Chi-square Canberra Extended-Canberra Square chord
Existing LBP [12] 3 × 256 = 768 35.38 32.57 35.75 35.41
ULBP [12] 3 × 59 = 177 32.87 34.27 34.26 32.88
MSLBP [39] 9 × 256 = 2304 35.53 33.26 39.95 (4) 35.33
LCVBP [42] 4 × 59 = 236 36.39 38.14 (6) 37.95 36.40
RGB-OC-LBP [43] 3 × 32 = 96 27.74 28.69 28.12 27.74
QLRBP [44] 3 × 256 = 768 35.21 31.72 36.54 35.22
CLSP [45] 10 × 256 = 2560 29.45 28.96 29.67 30.77
GABOR [8] 96 37.18 36.40 36.95 37.12
MDLBP 8 × 256 = 2048 37.89 34.93 39.99 (3) 38.08
Proposed LBPC 256 31.87 31.41 34.08 32.81
ULBPC 59 27.79 28.08 28.28 27.78
LBPH 256 26.95 24.07 28.23 27.02
ULBPH 59 24.43 24.95 26.16 24.43
CH 30 25.20 23.24 25.91 25.47
LBPC + LBPH 2 × 256 = 512 35.88 35.15 38.34 (5) 35.93
ULBPC + ULBPH 2 × 59 = 118 33.50 34.57 37.60 32.51
LBPC + LBPH + CH 2 × 256 + 30 = 542 34.76 40.36 43.81 (1) 34.30
ULBPC + ULBPH + CH 2 × 59 + 30 = 148 33.49 39.21 41.68 (2) 33.08
Average mAP – 32.31 32.22 34.63 32.35
Fig. 4. P-R curves for the seven existing methods MSLBP, LCVBP, GABOR, QLRBP, LBP, ULBP, and MDLBP and two proposed methods, LBPC + LBPH + CH, and ULBPC + ULBPH + CH
for N = 1 to 100 on Corel- 5K dataset.
s
f
3
r
F
i
t
d
e
s
i
e
t
t
o
t
i
QLRBP (36.54%), LBP (35.75%), ULBP (34.27%), CLSP (30.77%),
and RGB-OC-LBP (28.69%). Interestingly, the difference between
mAP values obtained by the proposed methods LBPC + LBPH + CH
and ULBPC + ULBPH + CH is only 2.13% which is insignificant,
whereas the difference in the size of feature vectors used by
these two proposed approaches is very large. The former method
uses 542 features as compared to less than one-third i.e. 148
features used by the latter. When we look for a good trade-
off between the retrieval accuracy and speed among the exist-
ing methods, then LCVBP is a good choice among the existing
methods, which has 236 features and it achieves mAP value of
38.14%. Among the proposed methods, ULBPC + ULBPH + CH achieves
mAP values of 41.68% with only 148 features. Thus, the proposed
method ULBPC + ULBPH + CH provides mAP values with only 148
features which is larger by 3.54% as compared to LCVBP which
uses 236 features. Out of the 18 best mAP values across the four
distance measures, extended-Canberra provides 13 best values, fol-
lowed by Canberra with 3 best values, chi-square with 1 value and
quare-chord with 1 value. The average mAP values of all methods
or extended-Canberra, square-chord, chi-square, and Canberra, are
4.63%, 32.35%, 32.31%, and 32.22%, respectively. The precision and
ecall values for the nine prominent methods have been plotted in
ig. 4 . It is seen that the proposed methods outperform the exist-
ng methods for all values of recall.
The retrieval results for Corel- 10K are presented in Table 7 . The
rend in the values of mAP is similar to that for Wang and Corel- 5K
atasets. The proposed method LBPC + LBPH + CH obtains the high-
st value of mAP, which is 36.99%, and its “uniform pattern” ver-
ion ULBPC + ULBPH + CH achieves the next highest value 34.32%. It
s important to note here that while there is a marginal differ-
nce of 2.67% in the mAP values of these two proposed method,
he feature size of the latter (148) is very small as compared
o the feature size of the former (542). The third largest value
f mAP is achieved by MDLBP, which is 33.97% with 2048 fea-
ures. The fourth largest value is achieved by LBPC + LBPH, which
s 33.50% by using 512 features. Among the existing methods,
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 63
Table 7
Mean average precision ( mA P) in percent for N = 100 obtained by various approaches in RGB color space on Corel- 10K dataset
using various distance measures.
Method No. of features Chi-square Canberra Extended-Canberra Square-chord
Existing LBP [12] 3 × 256 = 768 28.97 26.71 29.33 28.98
ULBP [12] 3 × 59 = 177 26.96 28.03 28.08 26.97
MSLBP [39] 9 × 256 = 2304 28.99 27.08 31.69 (5) 28.86
LCVBP [42] 4 × 59 = 236 29.25 29.62 29.72 29.26
RGB-OC-LBP [43] 3 × 32 = 96 23.10 23.74 23.33 23.11
QLRBP [44] 3 × 256 = 768 26.40 23.99 27.64 26.39
CLSP [45] 10 × 256 = 2560 24.64 23.71 22.53 24.64
GABOR [8] 96 28.06 27.39 29.68 28.065
MDLBP [46] 8 × 256 = 2048 31.83 29.43 33.97 (3) 33.23
Proposed LBPC 256 24.45 24.37 27.25 25.89
ULBPC 59 21.27 22.40 22.50 21.38
LBPH 256 20.94 19.46 21.98 21.27
ULBPH 59 19.37 19.97 20.13 19.09
CH 30 18.63 17.79 19.18 18.89
LBPC + LBPH 2 × 256 = 512 31.30 32.20 33.50 (4) 31.37
ULBPC + ULBPH 2 × 59 = 118 28.98 29.40 31.02 (6) 27.88
LBPC + LBPH + CH 2 × 256 + 30 = 542 31.79 32.37 36.99 (1) 34.08
ULBPC + ULBPH + CH 2 × 59 + 30 = 148 28.09 32.04 34.32 (2) 26.17
Average BEP – 26.28 26.09 27.94 26.42
10
20
30
40
50
10 20 30 40 50 60 70 80 90 100
LBPC+LBPH+CH (Proposed)
ULBPC+ULBPH+CH(Proposed)
MSLBP
LCVBP
GABOR
LBP
ULBP
QLRBP
MDLBP
Recall
Pre
cisi
on
(P)
Fig. 5. P-R curves for the seven existing methods MSLBP, LCVBP, GABOR, LBP, ULBP, QLRBP, and MDLBP, and two proposed methods, LBPC + LBPH + CH, and ULBPC + ULBPH + CH
for N = 1 to 100 on Corel- 10K dataset.
M
L
R
p
g
w
L
C
d
m
p
b
r
N
s
t
s
m
p
T
i
m
i
p
b
L
U
(
C
t
M
U
l
o
M
t
i
a
a
6
SLBP achieves the second highest mAP value 31.69%, followed by
CVBP (29.72%), GABOR (29.68%), LBP(29.33%), ULBP(28.09%), QL-
BP (27.6 4%), CLSP(24.6 4%), and RGB-OC-LBP (23.74%). The pro-
osed method LBPC + LBPH yields mAP value 33.50% which is
reater than the value of MSLBP. The other proposed methods
ith their decreasing mAP values are: ULBPC + ULBPH (31.02%),
BPC (27.25%), ULBPC (22.50%), LBPH(21.98%), ULBPH (20.13%) and
H(19.18%). Once again the performance of the extended-Canberra
istance measure is much better than the other three distance
easures, which provides an average mAP of 27.94% as com-
ared to the average mAP of 26.42%, 26.28%, and 26.09% provided
y the square-chord, Chi-square, and Canberra distance measures,
espectively. Fig. 5 shows the precision versus recall values for
= 1 to 100 . The proposed methods yield higher values of preci-
ion for all values of recall.
The trend of retrieval performance of all methods with respect
o the size of the three ( Wang, Corel- 5K, Corel- 10K) datasets is con-
istent. As the size of the dataset increases, there is a drop in the
AP values for all methods. The relative drop is almost the same.
The results of the retrieval performance of the existing and
roposed methods on Holidays dataset are presented in Table 8 .
his dataset presents a variety of objects. The number of relevant
mages in a class is variable. Therefore, the retrieval accuracy is
easured in terms of average value of BEP ( mBEP ). It is shown
n the table that the highest value of mBEP is obtained by the
roposed method LBPC + LBPH + CH which is 63.25%, followed
y ULBPC + ULBPC + CH(62.97%), MDLBP (62.46%), MSLBP(61.92%),
BPC + LBPH (61.75%), LBP(61.63%), LBPC(60.48%), LCVBP (60.17%),
LBP(59.19%), ULBPC + ULBPH(58.66%), CLSP(57.86%), ULBPC
57.35%), GABOR (55.34%), QLRBP (55.21%), RGB-OC-LBP(54.15%),
H(51.90%), LBPH(44.38%), and lastly by ULBPH(42.67%). Among
he existing approaches the best value of mBEP is achieved by
DLBP (62.46%), MSLBP(61.92%), LBP(61.63%), LCVBP(60.17%),
LBP(59.19%), CLSP(57.86%), GABOR (55.34%), QLRBP(55.21%), and
astly by RGB-OC-LBP(54.15%). When we compare the performance
f the proposed LBPC method with the existing ten methods, then
DLBP outperforms LBPC marginally. However, it has 2048 fea-
ures as compared to only 256 features used by LBPC. Thus, when
t comes to having a good tradeoff between retrieval accuracy
nd speed, then only three methods ULBPC + ULBPH + CH, LBPC,
nd LCVBP, stand out as the winners with mBEP values of 62.97%,
0.48%, and 60.17%, respectively. The numbers of features used
64 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
Table 8
Average bull’s eye performance ( mBEP ) in percent for obtained by various approaches in RGB color space on Holidays dataset
using various distance measures.
Method No. of features Chi-square Canberra Extended-Canberra Square chord
Existing LBP [12] 3 × 256 = 768 59.63 58.95 61.63 (6) 59.52
ULBP [12] 3 × 59 = 177 57.85 59.12 59.19 57.80
MSLBP [39] 9 × 256 = 2304 59.80 59.62 61.92 (4) 59.79
LCVBP [42] 4 × 59 = 236 59.13 59.67 60.17 54.04
RGB-OC-LBP [43] 3 × 32 = 96 53.87 54.15 54.09 53.87
QLRBP [44] 3 × 256 = 768 55.21 53.27 54.80 55.11
CLSP [45] 10 × 256 = 2560 57.38 57.10 57.86 57.42
GABOR [8] 96 55.28 54.78 54.74 55.34
MDLBP [46] 8 × 256 = 2048 60.81 57.99 62.46 (3) 60.86
Proposed LBPC 256 58.29 59.46 60.48 58.11
ULBPC 59 56.30 57.15 57.35 56.34
LBPH 256 44.32 42.64 44.23 44.38
ULBPH 59 42.43 41.30 42.67 42.39
CH 30 51.14 51.11 51.90 51.17
LBPC + LBPH 2 × 256 = 512 59.48 59.51 61.75 (5) 59.46
ULBPC + ULBPH 2 × 59 = 118 57.25 57.15 58.66 57.16
LBPC + LBPH + CH 2 × 256 + 30 = 542 61.32 60.66 63.25 (1) 61.24
ULBPC + ULBPH + CH 2 × 59 + 30 = 148 60.36 60.06 62.97 (2) 60.17
Average mBEP – 56.10 55.76 57.23 55.79
Table 9
Time taken (seconds) by various methods for feature extraction and image retrieval for top 100 images ( N = 100) on Corel- 5K and Corel- 10K
datasets.
Sr. no. Methods Number of features Feature extraction time (s) Retrieval time(s) Total time(s)
Corel-5K Corel-10K Corel-5K Corel-10K
a) LBP [12] 768 0.099 0.668 1.885 0.767 1.984
b) ULBP [12] 177 0.046 0.293 0.796 0.339 0.842
c) MSLBP [39] 2304 0.731 1.757 4.074 2.488 4.805
d) LCVBP [42] 236 0.051 0.312 0.967 0.363 1.018
e) RGB-OC-LBP [43] 96 0.027 0.203 0.639 0.23 0.666
f) QLRBP [44] 768 0.092 0.668 1.885 0.76 1.977
g) CLSP [45] 2560 0.529 2.273 5.092 2.802 5.621
h) GABOR [8] 96 0.765 0.203 0.639 0.968 1.404
i) MDLBP [46] 2048 0.212 1.556 2.126 1.768 2.338
j) LBPC 256 0.038 0.330 0.977 0.368 1.015
k) ULBPC 59 0.015 0.170 0.593 0.185 0.608
l) LBPH 256 0.040 0.330 0.977 0.37 1.017
m) ULBPH 59 0.016 0.170 0.593 0.186 0.609
n) CH 30 0.014 0.154 0.550 0.168 0.564
o) LBPC + LBPH 512 0.078 0.540 1.427 0.618 1.505
p) ULBPC + ULBPH 118 0.031 0.254 0.702 0.285 0.733
q) LBPC + LBPH + CH 542 0.092 0.596 1.663 0.688 1.755
r) ULBPC + ULBPH + CH 148 0.045 0.283 0.843 0.328 0.888
p
t
v
f
0
T
i
t
i
m
o
r
d
M
C
s
p
o
U
i
1
t
by these methods are 148, 256 and 236, respectively. Among the
various distance measures, the performance of extended-Canberra
is the best with an average mBEP of 57.23%, followed by chi-square
(56.10%), square-chord (55.79%), and Canberra (55.76%). When we
derive the best mBEP value for a method with respect to different
measures, then extended-Canberra distance provides the best
results for all eighteen methods.
7.4. Time analysis
The total time taken by a method for the complete retrieval
process consists of three parts: feature extraction, distance com-
putation D ( q, t ) for a query image q and a database image t , and
sorting of the top N images from a database of M images. The
computation of D ( q, t ) depends on the size L of the feature vec-
tor. The order of time complexity of this step is O ( L ). We use a
simple sorting algorithm (bubble sort) with time complexity of
O ( NM ) to retrieve N top images from M database images. This
remains the same for all the methods but varies amongst the
two datasets as they have different number of database images.
We club the times for distance computation and sorting and call
it retrieval time. For the purpose of the qualitative analysis, we
resent the time for feature extraction, retrieval time and the to-
al time for all 18 methods in Table 9 . The size of the feature
ector is also given. It is observed that CH takes the least time
or feature extraction and image retrieval, which is 0.014 (s) and
.154 (s) for Corel- 5K, and 0.014 (s) and 0.550 (s) for Corel- 10K.
he time taken for feature extraction is the same for all images
n these databases because their size is the same. For the quan-
itative analysis, we present the time analysis results in graph-
cal form in Fig. 6 showing the number of features for all 18
ethods and their feature extraction and the total time (in sec-
nds) for Corel- 5K and Corel- 10K datasets. The methods are ar-
anged in the increasing order of the total time taken for Corel- 10K
ataset. The longest time is taken by CLSP followed by MSLBP, and
DLBP which is 5.621(s), 4.805(s), and 2.338(s) respectively, for
orel- 10K. All these three methods have very high feature dimen-
ions which are 2560, 2304, and 2048, respectively. All our pro-
osed methods are much lower in dimensions (i.e., approximately
ne-fourth for LBPC + LBPH + CH/LBPC + LBPH, and one-thirteenth for
LBPC + ULBPH + CH/ULBPC + ULBPH) and thus take much less time
n comparison to them. LBPC + LBPH, and LBPC + LBPH + CH take
.505 (s), and 1.755 (s) for Corel- 10K and are faster by more than
wice over the average speed of these existing methods . Also, for
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 65
Fig. 6. Time taken for feature extraction and the total retrieval time in (seconds) taken for Corel- 5K and Corel- 10K datasets.
C
a
a
1
l
A
1
U
d
b
f
t
t
s
U
a
s
H
a
C
t
e
orel- 10K, ULBPC + ULBPH, and ULBPC + ULBPH + CH take 0.733 (s)
nd 0.888(s) and are approximately five times faster than the aver-
ge performance of these existing methods .
Amongst the methods with feature dimension between 512 and
024, our proposed methods (LBPC + LBPH, LBPC + LBPH + CH) take
ess time as compared to LBP [12] , QLRBP[44] and MDLBP[46].
lso, amongst the methods with feature dimension between
18 and 256, the proposed methods i.e. ULBPC + ULBPH and
LBPC + ULBPH + CH, take less time than LCVBP [42] for both
atasets. However, their retrieval performance is comparable or
etter than the LCVBP method which takes 0.363 (s), and 1.018 (s)
or Corel -5K and Corel- 10K datasets which is the median of the to-
al time for all the methods presented here. They also take less
ime as compared to Gabor [8] which has only 96 features.
It may be recalled that MDLBP provides the next best re-
ults after our two proposed approaches (LBPC + LBPH + CH and
LBPC + ULBPH + CH). It may also be noted that among the existing
pproaches, MDLBP provides the best retrieval results. Its ranking
tands at third for Wang, Corel- 5K , Corel- 10K, and Holidays datasets.
owever, regarding time efficiency, it is the slowest after CSLSP
nd MSLBP with 2.338(s) for feature extraction and retrieval on
orel- 10K. Thus, the proposed approaches are not only best in
he retrieval performance, but also they are computationally very
fficient.
66 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
8. Conclusion
The proposed local binary patterns for color images (LBPC) for
deriving the color texture patterns are very effective in color im-
age retrieval problem. We use a plane in 3D color space using RGB
color model and threshold the color pixels in a circularly sym-
metric neighborhood of a pixel with equally-spaced members P
and radius R . The values P = 8 , and R = 2 are observed to pro-
vide very good retrieval accuracy and speed in comparison to
any other combination of P and R . The color pixels, which lie
on or above the plane, are assigned a value 1 and those which
lie below the plane are assigned the value 0. The retrieval accu-
racy is further enhanced by deriving the local binary patterns of
the hue (H) component of the HSI color space, which is called
LBPH. The color histogram (CH) is an effective descriptor of color
images and has been used in the proposed methods to enhance
the retrieval performance. The performance of these color descrip-
tors has been analyzed individually as well as in their combined
forms after fusing their features. Since all approaches are based
on normalized histograms, the fusion of these features is a sim-
ple concatenation process. Therefore, equal weightage is given to
all modalities. The uniform patterns of LBPC and LBPH are also
derived. Thus, we have analyzed five different modalities: LBPC,
ULBPC, LBPH, ULBPH, CH, and their four combinations: LBPC + LBPH,
ULBPC + ULBPH, LBPC + LBPH + CH, and ULBPC + ULBPH + CH. Detailed
experimental analysis for retrieval performance and computation
time reveals that the proposed method LBPC + LBPH + CH achieves
the highest value of mAP across all datasets in comparison to the
existing best local color texture descriptor-multichannel decoded
local binary pattern (MDLBP). A deep analysis reveals that the pro-
posed descriptor is very effective when intra-class variation among
images is very high. The proposed method LBPC + LBPH + CH not
only provides the best mAP values but it also provides fast retrieval
speed because it uses very small number of features (542) as com-
pared to the large number of features used by the MDLBP (2048).
Another proposed method ULBPC + ULBPH + CH with only 148 fea-
tures provides retrieval accuracy comparable to LBPC + LBPH + CH,
but with very low retrieval time. In fact, the retrieval accuracy of
ULBPC + ULBPH + CH is slightly lower than LBPC + LBPH + CH. It pro-
vides much better retrieval results at low computational cost as
compared to the best existing method MDLBP that has very high
feature dimension (2048) which results in high computation cost.
The experimental results also reveal that the extended-Canberra
distance measure outperforms all prominent distance measures to
provide the highest mAP values for most of the methods across all
datasets.
Acknowledgements
We appreciate the suggestions and comments given by the
anonymous reviewers to raise the standard of the paper. The au-
thors are grateful to the University Grants Commission (UGC), New
Delhi, India, for providing financial grants for the major research
project entitled, “Development of efficient techniques for feature
extraction and classification for invariant pattern matching and
computer vision applications”, vide its File No.: 43-275/2014(SR).
One of the authors (Kanwal Preet Kaur) is thankful to the Univer-
sity Grant Commission (UGC) and the Ministry of Minority Affairs
(MOMA), Govt. of India, for providing Maulana Azad National Fel-
lowship (F1-17.1/2013-14/MANF-2013-14-SIK-PUN-22020) for car-
rying out the research work leading to the Ph.D. degree .
References
[1] B.S. Manjunath , J.-R. Ohm , V.V. Vasudevan , A. Yamada , Color and texture de-
scriptor, IEEE Trans. Circ. Syst. Video Technol. 11 (6) (2001) 703–715 .
[2] R.M. Haralick , K. Shangmugam , Textural feature for image classification, IEEETrans. Syst. Man Cybern. (6) (1973) 610–621 SMC-3 .
[3] H. Tamura , S. Mori , T. Yamawaki , Texture features corresponding to visual per-ception, IEEE Trans. Syst. Man Cybern. 8 (6) (1978) 460–473 .
[4] G. Cross , A. Jain , Markov random field texture models, IEEE Trans. Pattern Anal.Mach. Intell. 5 (1) (1983) 25–39 .
[5] B.S. Manjunath , W.Y. Ma. , Texture features for browsing and retrieval of imagedata, IEEE Trans. Pattern Anal. Mach. Intell. 18 (8) (1996) 837–842 .
[6] P. Howarth , S. Rüger , Evaluation of texture features for content-based image
retrieval, in: Proceedings of International conference Image and Video Re-trieval, 2004, pp. 326–334 .
[7] L. Chen , G. Lu , D. Zhang , Effects of different Gabor filters parameters on im-age retrieval by texture, in: Proceedings of IEEE 10th InternationalMultimedia
Modelling Conference, 2004, pp. 273–278 . [8] J. Han , K-K. Ma , Rotation-invariant and scale-invariant Gabor features for tex-
ture image retrieval, Image Vis. Comput. 25 (9) (2007) 1474–1481 .
[9] F. Bianconi , A. Fernández , Evaluation of the effects of Gabor filter parameterson texture classification, Pattern Recognit. 40 (12) (2007) 3325–3335 .
[10] D.S. Zhang , G. Lu , Review of shape representation and description techniques,Pattern Recognit. 37 (2004) 1–19 .
[11] J. Vogel , B. Schiele , Performance evaluation and optimization for content-basedimage retrieval, Pattern Recognit. 39 (5) (2006) 897–909 .
[12] T. Ojala , M. Pietikäinen , T. Mäenpää, Multiresolution gray-scale and rotation
invariant texture classification with local binary patterns, IEEE Trans. PatternAnal. Mach. Intell. 24 (7) (2002) 971–987 .
[13] D.G. Lowe , Distinctive image features from scale-invariant keypoints, Int. J.Comput. Vis. 60 (2) (2004) 91–110 .
[14] Y. Ke , R. Sukthankar , PCA-SIFT: a more distinctive representation for local im-age descriptors, in: IEEE Conference on Computer Vision and Pattern Recogni-
tion (CVPR), 2004, pp. 506–513 .
[15] K. Mikolajczyk , C. Schmid , A performance evaluation of local descriptors, IEEETrans. Pattern Anal. Mach. Intell. 27 (10) (2005) 1615–1630 .
[16] H. Bay , A. Ess , T. Tuytelaars , L.J.V. Gool , SURF: speeded-up robust features,Comput. Vis. Image Understand. 110 (3) (2008) 346–359 .
[17] N. Dalal , B. Triggs , Histograms of oriented gradients for human detection, in:Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni-
tion (CVPR), 2005, pp. 886–893 .
[18] E. Tola , V. Lepetit , P. Fua , DAISY: an efficient dense descriptor applied to wide-baseline stereo, IEEE Trans. Pattern Anal. Mach. Intell. 32 (5) (2010) 815–830 .
[19] B. Li , R. Xiao , Z. Li , Z. Cai , B-L. Lu , L. Zhang , Rank-SIFT: learning to rank repeat-able local interest points, in: IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), 2011, pp. 1737–1744 . [20] M. Calonder , V. Lepetit , M.O. Zuysal , T. Trzcinski , C. Strecha , P. Fua , Brief: com-
puting a local binary descriptor very fast, IEEE Trans. Pattern Anal. Mach. Intell.
34 (7) (2012) 1281–1298 . [21] E. Rublee , V. Rabaud , K. Konolige , GR. Bradski , ORB: an efficient alternative to
SIFT or SURF, in: Proceedings of the IEEE International Conference on Com-puter Vision (ICCV), 2011, pp. 2564–2571 .
[22] J. Chen , S. Shan , C. He , G. Zhao , M. Pietikäinen , X. Chen , W. Gao , WLD: a ro-bust local image descriptor, IEEE Trans. Pattern Anal. Mach. Intell. 32 (9) (2010)
1705–1720 . [23] L.J.V.Gool Everingham , C.K.I. Williams , J.M. Winn , A. Zisserman , The Pascal vi-
sual object classes (VOC) challenge, Int. J. Comput. Vis. 88 (2) (2010) 303–338 .
[24] G.J. Burghouts , J.-M. Geusebroek , Performance evaluation of local color invari-ants, Comput. Vis. Image Understand. 113 (1) (2009) 48–62 .
[25] T. Mäenpää, M. Pietikäinen , T. Ojala , Texture classification by multi-predicatelocal binary pattern operators, in: Proceedings of the International Conference
on Pattern Recognition (ICPR), 20 0 0, pp. 3951–3954 . [26] T. Mäenpää, T. Ojala , M. Pietikäinen , M. Soriano , Robust texture classification
by subsets of local binary patterns, in: Proceedings of the International Con-
ference on Pattern Recognition (ICPR), 20 0 0, pp. 3947–3950 . [27] M. Pietikäinen , T. Mäenpää, V. Jaakko , Color texture classification with color
histograms and local binary patterns, in: Workshop on Texture Analysis in Ma-chine Vision, 2002, pp. 109–112 .
[28] Y. Guo , Z. Guoying , M. PietikäInen , Discriminative features for texture descrip-tion, Pattern Recognit. 45 (10) (2012) 3834–3843 .
[29] T. Ahonen , A. Hadid , M. Pietikäinen , Face recognition with local binary pat-
terns, in: Proceedings of the European Conference on Computer Vision (ECCV),2004, pp. 469–481 .
[30] T. Ahonen , A. Hadid , M. Pietikäinen , in: Face Recognition With Local BinaryPatterns Lecture Notes in Computer Science, 3021, Springer-Verlag, Berlin Hei-
delberg New York, 2004, pp. 469–481 . [31] T. Ahonen , A. Hadid , M. Pietikäinen , Face description with local binary pat-
terns: application to face recognition, IEEE Trans. Pattern Anal. Mach. Intell. 28
(12) (2006) 2037–2041 . [32] C. Shan , S. Gong , P.W. McOwan , Robust facial expression recognition using lo-
cal binary patterns, in: Proceedings of IEEE International Conference of ImageProcessing, 2, 2005, pp. 370–373 .
[33] Z. Guoying , M. Pietikainen , Dynamic texture recognition using local binary pat-terns with an application to facial expressions, IEEE Trans. Pattern Anal. Mach.
Intell. 29 (6) (2007) 915–928 .
[34] C. Shan , S. Gong , P.W. McOwan , Facial expression recognition based on lo-cal binary patterns: a comprehensive study, Image Vis. Comput. 27 (6) (2009)
803–816 . [35] S. Zhang , X. Zhao , B. Lei , Facial expression recognition based on local binary
C. Singh et al. / Pattern Recognition 76 (2018) 50–68 67
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
[
patterns and local fisher discriminant analysis, WSEAS Trans. Sig. Process. 8(1) (2012) 21–31 .
36] V. Takala , T. Ahonen , M. Pietikäinen , Block-based methods for image retrievalusing local binary patterns, in: Proceedings of Scandinavian Conference on Im-
age Analysis, 2005, pp. 882–891 . [37] O.A.B. Penatti , E. Valle , R.S. Torres , Comparative study of global color and tex-
ture descriptors for web image retrieval, J. Vis. Commun. Image Represent. 23(2) (2012) 359–380 .
38] D. Huang , S. Caifeng , A. Mohsen , W. Yunhong , C. Liming , Local binary patterns
and its application to facial image analysis: a survey, IEEE Trans. Syst. ManCybern. Part C 41 (6) (2011) 765–781 .
39] T. Mäenpää, M. Pietikäinen , J. Viertola , Separating color and pattern informa-tion for color texture discrimination, in: Proceedings of 16th International Con-
ference on Pattern Recognition, vol. 1, 2002, pp. 668–671 . 40] T. Mäenpää, M. Pietikäinen , Classification with color and texture: jointly or
separately? Pattern Recognit. 37 (8) (2004) 1629–1640 .
[41] J.Y. Choi , K.N. Plataniotis , Y.M. Ro , Using local binary pattern features for facerecognition, in: Proceedings of 17th IEEE International Conference on Image
Processing (ICIF 2010) Sept. 26–29, Hong Kong, 2010, pp. 4541–4544 . 42] S.H. Lee , J.Y. Choi , Y.M. Ro , K.N. Plataniotis , Local color vector binary patterns
from multichannel face images for face recognition, IEEE Trans. Image Process.21 (4) (2012) 2347–2353 .
43] C. Zhu , C.-H. Bichot , L. Chen , Image region description using orthogonal com-
bination of local binary patterns enhanced with color information, PatternRecognit. 46 (2013) 1949–1963 .
44] R. Lan , Z. Yicong , Y.T. Yuan , Quaternionic local ranking binary pattern: a localdescriptor of color images, IEEE Trans. Image Process. 25 (2) (2015) 566–579 .
45] J. Li , N. Sang , C. Gao , Completed local similarity pattern for color image recog-nition, Neurocomputing 182 (2016) 111–117 .
46] S.R. Dubey , S.K. Singh , R.K. Singh , Multichannel decoded local binary patternsfor content-based image retrieval, IEEE Trans. Image Process. 25 (9) (2016)
4018–4032 . [47] R.C. Gonzalez , R.E. Woods , Digital Image Processing, third ed., Prentice- Hall,
2007 . 48] M. Swain , D. Ballard , Color indexing, Int. J. Comput. Vis. 7 (1) (1991) 11–32 .
49] M. Flickner , H. Sawhney , W. Niblack , J. Ashley , Q. Huang , B. Dom , P. Yanker ,
Query by image and video content: the QBIC system, Computer 28 (9) (1995)23–32 .
50] G-H Liu, Image Rank Machine Based on Visual Attention Mechanism < http://www.ci.gxnu.edu.cn/cbir/Dataset.aspx >.
[51] G-H Liu , J-Y Yang , Content-based image retrieval using color difference his-togram, Pattern Recognit. 46 (2013) 188–198 .
52] T. Deselaers , D. Keysers , H. Ney , Features for image retrieval: an experimental
comparison, Inf. Retrieval 11 (2) (2008) 77–107 . 53] S. Li , M-C Lee , C.-M. Pun , Complex Zernike moments feature for shape-based
image retrieval, IEEE Trans. Syst. Man Cybern. Part A 39 (1) (2009) 227–237 . 54] J.Z. Wang , J. Li , G. Wiederhold , Simplicity: semantics-sensitive integrated
matching for picture libraries, IEEE Trans. Pattern Anal. Mach. Intell. 23 (9)(2001) 947–963 .
55] H. Jegou , M. Douze , C. Schmid , Hamming embedding and weak geometric con-
sistency for large scale image search, in: Proceedings of European Conferenceon Computer Vision (ECCV), 2008, pp. 304–317 .
68 C. Singh et al. / Pattern Recognition 76 (2018) 50–68
.Sc. degree in mathematics from Kumaon University, Nainital, India, in 1975 and 1977,
m Indian Institute of Technology, Kanpur, India, in 1982. He joined M/S Jyoti Ltd., Baroda, tiala, India, in 1987. In the year 1994, he joined the Department of Computer Science at
year 1995. Dr. Singh has also served as Dean, Faculty of Engineering and Technology,
e has worked in many diverse areas such as Fluid Dynamics, Finite Element Analysis, years of teaching/research experience. For the last 25 years, he has been working in
ise Removal, Image Superresolution and Medical Image Segmentation. He has published more than 44 papers in various national and international conferences.
ersity, and Masters as well as Ph.D. from Punjabi University, Patiala, India. She served
elhi in 1998, and as lecturer and senior lecturer in the NITTTR, Chandigarh, India for unjabi University, M.M. University, and South Asian University, in India. At present, she is
ty of Saskatchewan, Canada, as a Professional research associate. Currently, she is working applications . Her research interests include Digital Image Watermarking, Content-Based
lysis. She has a number of papers in international journals and conference publications
any reputed image processing journals and conferences. She has also chaired sessions in
computer applications in 2007 from Panjab University, Chandigarh, India, and a post-
stitute of Management, Jalandhar, India, in 2010. She started her career as an assistant 0. She is currently pursuing Ph.D. degree in computer science from Punjabi University,
sing and Content-Based Image Retrieval.
Chandan Singh received B.Sc. degree in Science and M
respectively and a Ph.D. degree in applied mathematics froIndia, in 1982, and later Thapar Corporate R&D Centre, Pa
Punjabi University, Patiala. He became professor in the
Dean, Faculty of Physical Sciences, and Dean, Research. HOptimization and Numerical Analysis. He has almost 40
Pattern Recognition, Image Retrieval, Face Recognition, Nomore than 84 papers in various International journals and
Ekta Walia received B.Sc. degree from Kurukshetra Univ
as a software consultant with DCM DataSystems, New Dseven years. From 2007 to 2014, she served as faculty in P
associated with Department of Computer Science, Universion medical image analysis and computer-aided diagnosis
Image Retrieval, Face Recognition and Medical Image Ana
in these areas. She has been on the reviewing board of mInternational Conferences of repute.
Kanwal Preet Kaur received an undergraduate degree in
graduate degree in computer applications from Lovely Inprofessor in Guru Nanak College, Budhlada, India, in 201
Patiala, India. Her research interests include Image Proces
![Grid-based Visual Terrain Classification for Outdoor Robots ...II. TEXTURE DESCRIPTORS A. Local Binary Patterns Local Binary Patterns (LBP) [20] are very simple, yet powerful texture](https://img.pdfslide.net/doc/110x75/60844d02214aef5add43999c/grid-based-visual-terrain-classiication-for-outdoor-robots-ii-texture-descriptors.jpg)
![ISSN: 1992-8645 LOCAL QUANTIZED EDGE BINARY PATTERNS … · retrieval.Muralaet.al[37] proposed a new texture feature descriptor called local extrema patterns, where it collects the](https://img.pdfslide.net/doc/110x75/5f24a11c4ca8f168f84d0e8c/issn-1992-8645-local-quantized-edge-binary-patterns-retrievalmuralaetal37-proposed.jpg)
