School of Information and Control Engineering, China ... · China University of Mining and Technology, Xuzhou 221116, China 2: School of Mathematics and Statistics, Jiangsu Normal

Sample-Relaxed Two-Dimensional Color PrincipalComponent Analysis for Face Recognition and Image

Reconstruction∗

Mei-Xiang Zhao1, Zhi-Gang Jia2, Dunwei Gong3

1, 3. School of Information and Control Engineering,China University of Mining and Technology, Xuzhou 221116, China

2. School of Mathematics and Statistics,Jiangsu Normal University, Xuzhou 221116, China

Abstract

A sample-relaxed two-dimensional color principal component analysis (SR-2DCPCA)approach is presented for face recognition and image reconstruction based on quaternionmodels. A relaxation vector is automatically generated according to the variances of train-ing color face images with the same label. A sample-relaxed, low-dimensional covariancematrix is constructed based on all the training samples relaxed by a relaxation vector,and its eigenvectors corresponding to the r largest eigenvalues are defined as the optimalprojection. The SR-2DCPCA aims to enlarge the global variance rather than to maximizethe variance of the projected training samples. The numerical results based on real facedata sets validate that SR-2DCPCA has a higher recognition rate than state-of-the-artmethods and is efficient in image reconstruction.

Key words. Face recognition; Image reconstruction; Eigenface; Quaternion matrix.

1 Introduction

In this paper, we present a sample-relaxed color two-dimensional principal component analysis(SR-2DCPCA) approach for face recognition and image reconstruction based on quaternionmodels. Different from 2DPCA [23] and 2DCPCA [8], SR-2DCPCA utilizes the variances oftraining samples with the same label, aims to maximize the variance of the whole (training andtesting) projected images, and has comparable feasibility and effectiveness with state-of-the-artmethods.

Color information is one of the most important characteristics in reflecting structural infor-mation of a face image. It can help human being to accurately identify, segment or watermarkcolor images (see [1], [3]-[7], [13, 17, 20, 21, 24, 25] for example). Various traditional methodsof (grey) face recognition has been improved by fully exploiting color cues. Torres et al. [18]

∗This paper was jointly supported by National Natural Science Foundation of China, Grant Nos.61403155, 61473299, 61375067, 61773384 and 11771188. E-mails: [email protected] (M.-X. Zhao),[email protected] (Z.-G. Jia), [email protected] (D. Gong)

1

arX

iv:1

803.

0383

7v1

[cs

.CV

] 1

0 M

ar 2

018

applied the traditional PCA into R, G and B color channels, respectively, and got a fusionof the recognition results from three color channels. Xiang, Yang and Chen [22] proposed aCPCA approach for color face recognition. They utilized a color image matrix-representationmodel based on the framework of PCA and applied 2DPCA to compute the optimal projectionfor feature extraction. Recently, the quaternion PCA (QPCA) [2], the two-dimensional QPCA(2DQPCA) and bidirectional 2DQPCA [16], and kernel QPCA (KQPCA) and two-dimensionalKQPCA [3], have been proposed to generalize the conventional PCA and 2DPCA to color im-ages, using the quaternion representation. These approaches have achieved a significant successin promoting the robustness and the ratio of face recognition by utilizing color information.

The PCA-like methods for face recognition are based on linear dimension-reducing projec-tion. The projection directions are exactly orthonormal eigenvectors of the covariance matrixof training samples, with the aim to maximize the total scatter of the projected training im-ages. Sirovich and Kirby [14, 10] applied the PCA approach to the representation of (grey)face images, and asserted that any face image can be approximately reconstructed by a facialbasis and a mean face image. Based on this assertion, Turk and Pentland [19] proposed awell-known eigenface method for face recognition. Following that, many properties of PCAhave been studied and PCA has gradually become one of the most powerful approaches forface recognition [12]. As a breakthrough development, Yang et al. [23] proposed the 2DPCAapproach, which constructs the covariance matrix by directly using 2D face image matrices.Generally, the covariance matrix in 2DPCA is of a smaller dimension than that in PCA, whichreduces storage and the computational operations. Moreover, the 2DPCA approach is conve-nient to apply spacial information of face images, and achieves a higher face recognition ratethan PCA in most cases.

With retaining the advantages of 2DPCA, 2DCPCA recently proposed in [8] is based onquaternion matrices rather than quaternion vectors, and hence can fully utilize color informa-tion and the spacial structure of face images. In this method, the generated covariance matrixis Hermitian and low-dimensional. The projection directions are eigenvectors of the covariancematrix corresponding to the r largest eigenvalues. We find that 2DCPCA ignores label infor-mation (if provided) and scatter of training samples with the same label. If these informationis considered when constructing a covariance matrix, the recognition rate of 2DCPCA will beimproved. As far as our knowledge, there has been no approaches of face recognition based onthe framework of 2DCPCA using these information.

The paper is organized as follows. In section 2, we recall two-dimensional color princi-ple component analysis approach proposed in [8]. In section 3, we present a sample-relaxedtwo-dimensional color principal component analysis approach based on quaternion models. Insection 4, we provide the theory of the new approach. In section 5, numerical experimentsare conduct by applying the Georgia Tech face database and the color FERET face database.Finally, the conclusion is draw in section 6.

2 2DCPCA

We firstly recall the two-dimensional color principle component analysis (2DCPCA) from [8].Suppose that an m × n quaternion matrix is in the form of Q = Q0 + Q1i + Q2j + Q3k,

where i, j, k are three imaginary values satisfying

i2 = j2 = k2 = ijk = −1, (2.1)

2

and Qs ∈ Rm×n, s = 0, 1, 2, 3. A pure quaternion matrix is a matrix whose elements arepure quaternions or zero. In the RGB color space, a pixel can be represented with a purequaternion, Ri + Gj + Bk, where R, G, B stand for the values of Red, Green and Bluecomponents, respectively. An m × n color image can be saved as an m × n pure quaternionmatrix, Q = [qab]m×n, in which an element, qab = Rabi+Gabj +Babk, denotes one color pixel,and Rab, Gab and Bab are nonnegative integers [11].

Suppose that there are ` training color image samples in total, denoted as m × n purequaternion matrices, F1, F2, ..., F`, and the mean image of all the training color images can bedefined as follows

Ψ =1

`

∑̀s=1

Fs ∈ Hm×n. (2.2)

Model 1 (2DCPCA). (1) Compute the following color image covariance matrix oftraining samples

Gt =1

`

∑̀s=1

(Fs −Ψ)∗(Fs −Ψ) ∈ Hn×n. (2.3)

(2) Compute the r (1 ≤ r ≤ n) largest eigenvalues of Gt and their correspond-ing eigenvectors (called eigenfaces), denoted as (λ1, v1), . . . , (λr, vr). Let theeigenface subspace be V = span{v1, . . . , vr}.

(3) Compute the projections of ` training color face images in the subspace, V ,

Ps = (Fs −Ψ)V ∈ Hm×r, s = 1, · · · , `. (2.4)

(4) For a given testing sample, F , compute its feature matrix, P = (Fs − Ψ)V .Seek the nearest face image, Fs (1 ≤ s ≤ `), whose feature matrix satisfies that‖Ps − P‖ = min. Fs is output as the person to be recognized.

2DCPCA based on quaternion models can preserve color and spatial information of face im-ages, and its computational complexity of quaternion operations is similar to the computationalcomplexity of real operations of 2DPCA (proposed in [23]).

3 Sample-relaxed 2DCPCA

In this section, we present a sample-relaxed 2DCPCA approach based on quaternion modelsfor face recognition.

Suppose that all the ` color image samples of the training set can be partitioned into xclasses with each containing à (a = 1, . . . , x) samples:

F 11 , · · · , F 1

`1 | F21 , · · · , F 2

`2 | · · · | Fx1 , · · · , F x`x ,

3

where F ab represents the b-th sample of the a-th class. Now, we define a relaxation vector usinglabel and variance within a class, and generate a sample-relaxed covariance matrix of trainingset in the following.

3.1 The relaxation vector

Define the mean image of training samples from the a-th class by

Ψa =1

à

à∑s=1

F as ∈ Hn×n,

and the a-th within-class covariance matrix by

Na =1

à

à∑s=1

(F as −Ψa)∗(F as −Ψa) ∈ Hn×n, (3.1)

where a = 1, . . . , x and∑xa=1 à = `.

The within-class covariance matrix, Na, is a Hermitian quaternion matrix, and is semi-definite positive, with its eigenvalues being nonnegative. The maximal eigenvalue of Na, de-noted as λmax(Na), represents the variance of training samples, F a1 , . . . , F

aà

, in the principalcomponent. Generally, the larger λmax(Na) is, the better scattered the training samples of thea-th class are. If λmax(Na) = 0, all the training samples in the a-th class are same, and thecontribution of the a-th class to the covariance matrix of the training set should be controlledby a small relaxation factor.

Now, we define a relaxation vector for the training classes.

DEFINITION 3.1. Suppose that the training set has x classes and the covariance matrix of thea-th class is Na (1 ≤ a ≤ x). Then the relaxation vector can be defined as

W = [w1, · · · , wx] ∈ Rn, (3.2)

where

wa =eλmax(Na)∑xb=1 e

λmax(Nb)

is the relaxation factor of the a-th class.

If the a-th class contains à (≥ 1) training samples, the relaxation factor of each sample canbe calculated as wa/à. If each training class has only one sample, i.e., `1 = . . . = `x = 1, allthe within-class covariance matrices will be zero matrices, and

λmax(N1) = · · · = λmax(Nx) = 0.

In this case, the relaxation factor of each class will be wa = 1/x, and so will its unique sample.

4

3.2 The relaxed covariance matrix and eigenface subspace

Now, we define the relaxed covariance matrix of training set.

DEFINITION 3.2. With the relaxation vector, W = [w1, · · · , wn]T ∈ Rn, defined by (3.2), therelaxed covariance matrix of training set can be defined as follows

Gw =1

`

x∑a=1

(waà

à∑s=1

(F as −Ψ)∗(F as −Ψ)

)∈ Hn×n, (3.3)

where à means the number of training samples in the a-th class,∑xa=1 à = `, and Ψ is defined

by (2.2).

If there is no label information, or people are unwilling to use such information, the trainingset can be regarded as containing ` classes, with each having one sample, i.e., x = ` and à = 1.In this case, the relaxed covariance matrix, Gw, defined by (3.3) is exactly the covariancematrix, Gt, defined by (2.3).

DEFINITION 3.3. Suppose that Gw is the relaxed covariance matrix of training set. The gen-eralized total scatter criterion can be defined as follows

J(V ) = trace(V ∗GwV ) =

r∑s=1

v∗sGwvs, (3.4)

where V = [v1, · · · , vr] ∈ Hn×r is a quaternion matrix with unitary column vectors.

Note that J(V ) is real and nonnegative since the quaternion matrix, Gw, is Hermitianand positive semi-definite. Our aim is to select the orthogonal projection axes, v1, . . . , vr, tomaximize the criterion, J(V ), i.e.,

{vopt1 , . . . , voptr } = arg max∑rs=1 v

∗sGwvs

s.t. v∗svt =

{1, s = t,0, s 6= t,

s, t = 1, · · · , r. (3.5)

These optimal projection axes are in fact the orthonormal eigenvectors of Gw corresponding tothe r largest eigenvalues.

Once the relaxed covariance matrix, Gw, is built, we can compute its r (1 ≤ r ≤ n)largest (positive) eigenvalues and their corresponding eigenvectors (called eigenfaces), denotedas (λ1, v

opt1 ), . . . , (λr, v

optr ). Let the optimal projection be

V = [vopt1 , · · · , voptr ] (3.6)

and the diagonal matrix beD = diag(λ1, . . . , λr). (3.7)

Then V ∗V = Ir and D > 0. Following that, for any quaternion matrix norm ‖ · ‖, we definethe D-norm ‖ · ‖D as follows

‖A‖D = ‖AD‖, for any quaternion matrix, A ∈ Hm×r.

5

3.3 Color face recognition

In this section, we apply the optimal projection, V , and the positive definite matrix, D, to colorface recognition.

The projections of ` training color face images in the subspace, V , are

Ps = (Fs −Ψ)V ∈ Hm×r, s = 1, · · · , `, (3.8)

where Ψ is defined by (2.2). The columns of Ps, yt = (Fs − Ψ)vt, t = 1, . . . , r, are calledthe principal component (vectors), and Ps is called the feature matrix or feature image of thesample image, Fs. Each principal component of the sample-relaxed 2DCPCA is a quaternionvector. With the feature matrices in hand, we use a nearest neighbour classifier for color facerecognition.

Now, we present a sample-relaxed two-dimensional color principal component analysis (SR-2DCPCA) approach.

Model 2 (SR-2DCPCA). (1) Compute the relaxed color image covariance matrix,Gw, of training samples by (3.3)

(2) Compute the r (1 ≤ r ≤ n) largest eigenvalues of Gw and their correspond-ing eigenvectors (called eigenfaces), denoted as (λ1, v1), . . . , (λr, vr). Letthe eigenface subspace be V = span{v1, . . . , vr} and the weighted matrix beD = diag(λ1, · · · , λr).

(3) Compute the projections of ` training color face images in the subspace, V ,

Ps = (Fs −Ψ)V ∈ Hm×r, s = 1, · · · , `.

(4) For a testing sample, F , compute its feature matrix, P = (Fs − Ψ)V . Seek thenearest face image, Fs (1 ≤ s ≤ `) whose feature matrix satisfies that ‖Ps −P‖D = min. Fs is output as the person to be recognized.

SR-2DCPCA can preserve color and spatial information of color face images as 2DCPCA(proposed in [8]). Compared to 2DCPCA, SR-2DCPCA has an additional computation amounton calculating the relaxation vector, and generally provides a better discriminant for classifi-cation. The aim of the projection of SR-2DCPCA is to maximize the variance of the wholesamples, while that of 2DCPCA is to maximize the variance of training samples. Note thatthe eigenvalue problems of Hermitian quaternion matrices in Model 2 can be solved by the faststructure-preserving algorithm proposed in [9].

Now we provide a toy example.

Example 3.1. As shown in Figure 3.1, 400 randomly generated points are equally separatedinto two classes (denoted as × and ◦, respectively). We choose 100 points from each class astraining samples (denoted as magenta × and ◦ ) and the rest as testing samples (denoted as blue× and ◦). The principle component of 200 training points is computed by 2DCPCA and SR-2DCPCA. In three random cases, the relaxation vectors are [0.4771, 0.5228], [0.6558, 0.3442]and [0.5097, 0.4903]; the computed principle components are plotted with the blue lines. The

6

variances of the training set and the whole 400 points, under the projection of 2DCPCA andSR-2DCPCA, are shown in the following table.

Case Variance of training points Variance of the whole points2DCPCA SR-2DCPCA 2DCPCA SR-2DCPCA

1 4.5638 4.5562 4.2452 4.26172 3.6007 3.5803 4.1267 4.14593 3.6381 3.5582 3.7255 3.7452

3.4 Color image reconstruction

Now, we consider color image reconstruction based on SR-2DCPCA. Without loss of generality,we suppose that Ψ = 0. In this case, the projected color image is Ps = FsV .

THEOREM 3.1. Suppose that V ⊥ ∈ Hn×(n−r) is the unitary complement of V such that [V, V ⊥]is a unitary quaternion matrix. Then the reconstruction of a training sample, Fs, is PsV

∗, and

‖PsV ∗ − Fs‖ = ‖FsV ⊥‖, s = 1, · · · , `. (3.9)

Proof. We only need to prove (3.9). Since quaternion matrix [V, V ⊥] is unitary,

V V ∗ + V ⊥(V ⊥)∗ = In, V∗V = Ir, (V ⊥)∗V ⊥ = In−r, V

∗V ⊥ = 0,

where Im is an m × m identity matrix. The projected image, Ps, of the sample image, Fs,satisfies that

PsV∗ = FsV V

∗ = Fs(In − V ⊥(V ⊥)∗).

Therefore, we have‖PsV ∗ − Fs‖ = ‖FsV ⊥(V ⊥)∗‖ = ‖FsV ⊥‖.

According to [8], the image reconstruction rate of PsV∗ can be defined as follows

Ratios = 1− ‖PsV∗ − Fs‖2‖F‖2

. (3.10)

Since V is generated based on eigenvectors corresponding to the r largest eigenvalues of Gw,PsV

∗ is always a good approximation of the color image, Fs. If the number of chosen principalcomponents r = n, V is a unitary matrix and Ratios = 1, which means Fs = PsV

∗.From the above analysis, SR-2DCPCA is convenient to reconstruct color face images from

the projections, as 2DCPCA proposed in [8].

7

-10 -5 0 5 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.52DCPCA

-10 -5 0 5 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5SR-2DCPCA

-10 -5 0 5 10-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.52DCPCA

-10 -5 0 5 10-2.5

-2

-1.5

-1

-0.5

0

0.5

1

1.5

2

2.5SR-2DCPCA

-10 -5 0 5 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

22DCPCA

-10 -5 0 5 10-2

-1.5

-1

-0.5

0

0.5

1

1.5

2SR-2DCPCA

Figure 3.1: Points in three random cases and the principle components (blue lines).

4 Variance updating

In this section we analysis the contribution of training samples from each class to the varianceof the projected samples.

8

For the b-th class with `b training samples, let

X =[(F b1 −Ψ)∗ | (F b2 −Ψ)∗ | · · · | (F b`b −Ψ)∗

],

A =1

`

∑a6=b

(waà

à∑s=1

(F as −Ψ)∗(F as −Ψ)

).

According to the definition of the relaxed covariance matrix (3.3), we can rewrite Gw as

Gw = A+wb`

(1

`bXX∗). (4.1)

Denote all the eigenvalues of an n-by-nHermitian quaternion matrix, A, as λ1(A), λ2(A), . . . , λn(A)in descending order.

LEMMA 4.1. Suppose that B = A + ρXX∗, where A ∈ Hn×n is Hermitian, X ∈ Hn×m andρ ≥ 0. Then

|λa(B)− λa(A)| ≤ ρ‖X‖22, a = 1, · · · , n.

Proof. The properties for the eigenvalues of Hermitian (complex) matrices [15, Theorem 3.8,page 42] are still right for Hermitian quaternion matrices. Suppose that A,∆A ∈ Hn×n areHermitian matrices, then

λa(A) + λn(∆A) ≤ λa(A+ ∆A) ≤ λa(A) + λ1(∆A) (4.2)

for a = 1, . . . , n. Consequently,

|λa(A+ ∆A)− λa(A)| ≤ ‖∆A‖2, a = 1, · · · , n. (4.3)

Since XX∗ is Hermitian and semi-definite positive, its maximal singular value, ‖XX∗‖2, isexactly its maximal eigenvalue, which is the multiplication of the maximal singular values ofX and X∗. That is ‖XX∗‖2 = ‖X‖22. From equation (4.3), we can obtain that

|λa(B)− λa(A)| ≤ ‖ρXX∗‖2 = ρ‖X‖22, a = 1, · · · , n.

Since Hermitian quaternion matrices, Gw, A and XX∗ are semi-definite positive, we obtainthat

λa(A) ≤ λa(Gw) ≤ λa(A) + λ1(wb``b

XX∗).

Let the variance of projections of all the training samples on V be

ε =

r∑s=1

λs(Gw).

The following theorem characterizes the contribution of training samples from one class to thevariance of all the projected training samples.

9

THEOREM 4.2. With the above notations and a fixed relaxation vector W , the variance ofprojections of all the training samples ε satisfies that

r∑s=1

λs(A) ≤ ε ≤r∑s=1

λs(A) +rwb`‖X‖22.

Proof. From equation (4.1), we obtain that

Gw = A+ ρXX∗, ρ =wb``b

> 0.

Since Hermitian quaternion matrices, Gw and A, are semi-definite positive, we can see that

λs(Gw) ≥ λs(A),

and Lemma 4.1 implies that

λs(Gw) ≤ λs(A) +wb`‖X‖22.

Thus∑rs=1 λs(Gw) ≤

∑rs=1 λs(A) + rwb

` ‖X‖22.

5 Experiments

In this section we compare the efficiencies of five approaches on color face recognition:

• PCA: the Principle Component Analysis (with converting Color-image into Grayscale),

• CPCA: the Color Principle Component Analysis proposed in [22],

• 2DPCA: the Two-Dimensional Principle Component Analysis proposed in [23] (with con-verting Color-image into Grayscale),

• 2DCPCA: the Two-Dimensional Color Principle Component Analysis proposed in [8],

• SR-2DCPCA: the Sample-Relaxed Two-Dimensional Color Principle Component Analysisproposed in Section 3.

All the numerical experiments are performed with MATLAB-R2016 on a personal computerwith Intel(R) Xeon(R) CPU E5-2630 v3 @ 2.4GHz (dual processor) and RAM 32GB.

Example 5.1. In this experiment, we compare SR-2DCPCA with PCA, CPCA, 2DPCA and2DCPCA using the famous Georgia Tech face database [26]. The database contains variouspose faces with various expressions on cluttered backgrounds. All the images are manuallycropped, and then resized to 33 × 44 pixels. Some cropped images are shown in Fig. 5.1. Thefirst x (= 10 or 13) images of each individual are chosen as the training set and the remainingas the testing set. The numbers of the chosen eigenfaces are from 1 to 20. The face recognitionrates of five PCA-like methods are shown in Fig. 5.2. The top and the below figures show theresults for cases that the first x = 10 and 13 face images per individual are chosen for training,respectively.

The maximal recognition rate (MR) and the CPU time for recognizing one face image inaverage are listed in Table 1. We can see that SR-2DCPCA reaches the highest face recognitionrate, and costs less CPU time than CPCA.

10

Figure 5.1: Sample images for one individual of the Georgia Tech face database.

Table 1: Maximal face recognition rate and average CPU time (milliseconds)x CPCA 2DPCA 2DCPCA SR-2DCPCA

MR CPU MR CPU MR CPU MR CPU10 77.6% 7.58 83.2% 0.02 84.8% 0.09 85.2% 0.8513 87.0% 23.80 88.0% 0.06 92.0% 0.30 95.0% 2.16

Example 5.2. In this experiment, we compare three two-dimensional approaches: 2DPCA,2DCPCA and SR-2DCPCA on face recognition and image reconstruction, by using the colorFace Recognition Technology (FERET) database [27]. The database (version 2, DVD2, thumb-nails) contains 269 persons, 3528 color face images, and each person has various numbers offace images with various backgrounds. The minimal number of face images for one person is6, and the maximal one is 44. The size of each cropped color face image is 192 × 128 pixels,and some samples are shown in Fig. 5.3. Taking the first 6 images of each individual as anexample, we randomly choose x(= 1 : 5) image/images as the training set and the remainingas the testing set. This process is repeated 5 times, and the average value is output. For a fixedx, we consider ten cases that the number of the chosen eigenfaces increases from 1 to 10. Theaverage face recognition rate of these ten cases is shown in the top of Fig. 5.4. When x = 5,the face recognition rate with the number of eigenfaces increasing from 1 to 10 is shown in thebelow of Fig. 5.4.

With x = 5, we also employ 2DCPCA and SR-2DCPCA to reconstruct the color face images

11

0 5 10 15 200.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

Acc

ura

cy o

f fa

ce r

eco

gn

itio

n

Number of eigenfaces

PCA

CPCA

2DPCA

C2DPCA

SR−C2DPCA

0 5 10 15 200.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Acc

ura

cy o

f fa

ce r

eco

gn

itio

n


PCA

CPCA

2DPCA

C2DPCA

SR−C2DPCA

Figure 5.2: Face recognition rate of PCA-like methods for the Georgia Tech face database.

(Fs) in training set from their projections (Ps). In Fig. 5.5, the reconstruction of the first fourpersons with 2, 10, 20, 38 and 128 eigenfaces: the top and the below pictures are reconstructed by2DCPCA and SR-2DCPCA, respectively. In Fig. 5.6, we plot the ratios of image reconstructionby 2DCPCA (top) and SR-2DCPCA (below) with the number of eigenfaces changing from 1 to128. The difference in the ratio between the two method is shown in Fig. 5.7 (the ratio ofSR-2DCPCA subtracts that of 2DCPCA). These results indicate that both 2DCPCA and SR-2DCPCA are convenient to reconstruct color face images from projections, and can reconstruct

12

Person1 #1

Person2 #1

Person3 #1

Person4 #1

Person1 #2

Person2 #2

Person3 #2

Person4 #2

Person1 #3

Person2 #3

Person3 #3

Person4 #3

Person1 #4

Person2 #4

Person3 #4

Person4 #4

Person1 #5

Person2 #5

Person3 #5

Person4 #5

Person1 #6

Person2 #6

Person3 #6

Person4 #6

Figure 5.3: Sample images of the color FERET face database.

original color face images with choosing all the eigenvectors to span the eigenface subspace.

6 Conclusion

In this paper, SR-2DCPCA is presented for color face recognition and image reconstructionbased on quaternion models. This novel approach firstly applies label information of trainingsamples, and emphasizes the role of training color face images with the same label which havea large variance. The numerical experiments indicate that SR-2DCPCA has a higher facerecognition rate than state-of-the-art methods and is effective in image reconstruction.

13

1 1.5 2 2.5 3 3.5 4 4.5 50.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

Accu

racy o

f fa

ce r

eco

gn

itio

n

Number of training samples from each person

2DPCA

C2DPCA

SR−C2DPCA

1 2 3 4 5 6 7 8 9 100.66

0.68

0.7

0.72

0.74

0.76

0.78

Accu

racy o

f fa

ce r

eco

gn

itio

n


2DPCA

C2DPCA

SR−C2DPCA

Figure 5.4: Face recognition rate of PCA-like methods for the color FERET face database.

References

References

[1] J. Bai, Y. Wu, J.M. Zhang, F.Q. Chen, Subset based deep learning for RGB-D objectrecognition, Neurocomputing 165 (2015) 280-292.

[2] N.L. Bihan, S.J. Sangwine, Quaternion principal component analysis of color images, in:

14

2 Eigenvectors 10 Eigenvectors 20 Eigenvectors 38 Eigenvectors 128 Eigenvectors








Figure 5.5: Reconstructed images by 2DCPCA (top) and SR-2DCPCA (below).

Proc. 2003 Tenth IEEE Int. Conf. Image Processing (ICIP 2003), vol. 1, 2003, pp. 809-812.

[3] B.J. Chen, J.H. Yang, B. Jeon, X.P. Zhang, Kernel quaternion principal component anal-ysis and its application in RGB-D object recognition, Neurocomputing 266(29) (2017)293-303.

[4] B.J. Chen, H.Z. Shu, G. Coatrieux, G. Chen, X.M. Sun, J.L. Coatrieux, Color imageanalysis by quaternion-type moments, J. Math. Imaging Vision 51 (1) (2015) 124-144.

15

20 40 60 80 100 1200.5

0.6

0.7

0.8

0.9

1


Rati

o o

f im

ag

e r

eco

nstr

ucti

on

C2DPCA

Person1

Person2

Person3

Person4

20 40 60 80 100 1200.5

0.6

0.7

0.8

0.9

1


Rati

o o

f im

ag

e r

eco

nstr

ucti

on

SR−C2DPCA

Person1

Person2

Person3

Person4

Figure 5.6: Ratio of the reconstructed images over the original face images.

[5] Y. Cheng, X. Zhao, K. Huang, T.N. Tan, Semi-supervised learning and feature evaluationfor RGB-D object recognition, Comput. Vision Image Understanding 139 (2015) 149-160.

[6] T.A. Ell, L.B. Nicolas, S.J. Sangwine, Quaternion Fourier Transforms for Signal and ImageProcessing, John Wiley & Sons, 2014.

[7] K.M. Hosny, M.M. Darwish, Highly accurate and numerically stable higher order QPCETmoments for color image representation, Pattern Recogn. Lett. 97(2017) 29-36.

16

20 40 60 80 100 120−4

−3

−2

−1

0

1

2

3x 10

−5


Person1

Person2

Person3

Person4

Figure 5.7: The image reconstruction ratio of SR-2DCPCA subtracted by that of 2DCPCA.

[8] Z.-G. Jia, S.-T. Ling, M.-X. Zhao, Color two-dimensional principal component analysis forface recognition based on quaternion model, LNCS, vol. 10361, 2017, pp. 177-189.

[9] Z.-G. Jia, M. Wei, S.-T. Ling, A new structure-preserving method for quaternion Hermitianeigenvalue problems, J. Comput. Appl. Math. 239(2013) 12-24.

[10] M. Kirby, L. Sirovich, Application of the karhunenloeve procedure for the characterizationof human faces, IEEE Trans. Pattern Anal. Mach. Intell. 12 (1)(1990) 103-108.

[11] S.C. Pei, C.M. Cheng, Quaternion matrix singular value decomposition and its applicationsfor color image processing, in: Proc. 2003 Int. Conf. Image Processing (ICIP 2003), vol. 1,2003, pp. 805-808.

[12] A. Pentland, Looking at people: sensing for ubiquitous and wearable computing, IEEETrans. Pattern Anal. Mach. Intell. 22 (1) (2000) 107-119.

[13] L. Shi, B. Funt, Quaternion color texture segmentation, Comput. Visual Image Under-standing 107 (1) (2007) 88-96.

[14] L. Sirovich, M. Kirby, Low-dimensional procedure for characterization of human faces, J.Optical Soc. Am. 4(1987) 519-524.

[15] G.W. Stewart, Matrix Algorithms Volume II: Eigensystems, SIAM, 2001.

[16] Y.F. Sun, S.Y. Chen, B.C. Yin, Color face recognition based on quaternion matrix repre-sentation, Pattern Recognit. Lett. 32(4)(2011) 597-605.

[17] J.H. Tang, L. Jin, Z.C. Li, S.H. Gao, RGB-D object recognition via incorporating latentdata structure and prior knowledge, IEEE Trans. Multimedia 17(11)(2015) 1899-1908 .

17

[18] L. Torres, J.Y. Reutter, L. Lorente, The importance of the color information in face recog-nition, IEEE Int. Conf. Image Process. 3(1999) 627-631.

[19] M. Turk, A. Pentland, Eigenfaces for recognition. J. Cognitive Neurosci. 3(1)(1991) 71-76.

[20] A.R. Wang, J.W. Lu, J.F. Cai, T.J. Cham, G. Wang, Large-margin multi-modal deeplearning for rgb-d object recognition, IEEE Trans. Multimedia 17 (11) (2015) 1887-1898.

[21] H. Xue, Y. Liu, D. Cai, X. He, Tracking people in RGBD videos using deep learning andmotion clues, Neurocomputing 204 (2016) 70-76.

[22] X. Xiang, J. Yang, Q. Chen, Color face recognition by PCA-like approach, Neurocomputing152(2015) 231-235.

[23] J. Yang, D. Zhang, A. F. Frangi, J. Y. Yang, Two-dimensional PCA: A new approach toappearance-based face representation and recognition, IEEE Trans. Pattern Anal. Mach.Intell. 26(1) (2004) 131-137.

[24] R. Zeng, J.S. Wu, Z.H. Shao, Y. Chen, B.J. Chen, L. Senhadji, H.Z. Shu, Color image clas-sification via quaternion principal component analysis network, Neurocomputing 216(2016)416-428.

[25] Z. Zhang, M.B. Zhao, B. Li, P. Tang, F.Z. Li, Simple yet effective color principal anddiscriminant feature extraction for representing and recognizing color images, Neurocom-puting 149 (2015) 1058-1073 .

[26] The Georgia Tech face database. http://www.anefian.com/research/ facereco.htm.

[27] The color FERET database: https://www.nist.gov/itl/iad/image-group/color-feret-database.

18

http://www.anefian.com/research/

Documents

School of Information and Control Engineering, China ... · China University of Mining and Technology, Xuzhou 221116, China 2: School of Mathematics and Statistics, Jiangsu Normal