Mining Discriminative Components With Low-Rank and

Sparsity Constraints for Face RecognitionQiang Zhang, Baoxin Li

Computer Science and EngineeringArizona State University

Tempe, AZ, 85281qzhang53, baoxin.li@asu.edu

Problem Description

• In many applications, we may acquire multiple copies of signals from the same source (an ensemble of signals);

• Signals in ensemble may be very similar (sharing a common source), but may also have very distinctive differences (e.g., very different acquisition conditions) plus other unique but small variations (e.g., sensor noise).

ExamplesCommon sources

ExamplesDifferent acquisition conditions

ExamplesSensor noises

ExamplesSignals in ensemble

Signals in Ensemble

• The decomposition of the signal has several benefits:– Obtaining better compression rate. E.g.,

distributed compressed sensing [Duarte], joint sparsity model [Duarte 2005];

– Extract more relevant features. E.g., A compressive sensing approach for expression-invariant face recognition [Nagesh 2009];

Example: Face Images

• Given face images of same subjects

Example: Face Images

• Can we identify a “clean” image for them?

Example: Face Images

• And their illumination conditions?

Proposed Model

• We represent each image of an ensemble as

– the image of subject;– the image ensemble;– the common part for Subject ;– a low rank matrix;– a sparse matrix;

Solving the Decomposition

• The proposed model can be formulated as:

Subject to

Comparison with Other Models

• Distributed Compressive Sensing;– , s.t., ;– The proposed method is related to DCS. However,

in DCS, the innovation components are typically assumed to be sparse, which limits its application;

– Instead, the proposed model allows the innovation components have more complex structure, e.g., low rank.

• Robust Principle Component Analysis;

Comparison with Other Models

• Distributed Compressive Sensing;• Robust Principle Component Analysis;– , subject to ;– RPCA decompose a set of images into a low rank

matrix and sparse matrix;– The core differences of RPCA from the proposed

method is that, RPCA represents each image as a vector of a big matrix. Those vectors are linear dependent, in addition to some sparse noise;

Decomposition Algorithms

• We apply augmented Lagrange Multiplier to the proposed formulations:

• and are parameters.

Decomposition Algorithms Cont’d

• We use augmented Lagrange Multiplier and block coordinate descent to solve the proposed problem:– Solve for each , while and are fixed;– Solve for each , while and are fixed;– Solve for each , while and are fixed;– Update and as ;– Check the convergence. If not converged, repeat

the previous steps.

Experiment

• We use three experiments to evaluate the proposed model and algorithm:– Decomposing the synthetic images;– Decomposing the images from extended YaleB

dataset;– Applying decomposed component to classification

tasks;

Decomposing the synthetic images

We create training images by mixing the images with low-rank background images. In addition, we add some sparse-supported noise.

Decomposing the synthetic images

• The decomposition result. From top to bottom: common components, low-rank components and sparse components.

Decomposition: Robustness over Missing Training Instances

• We randomly remove 20% training images and test the robustness of decomposition algorithm.

• From top to bottom: training images and common component, low-rank component.

Decomposing Extended YaleB Dataset

• We use all 2432 images of extended YaleB dataset, which includes 38 subjects and 64 illumination conditions;– Common components capture information unique

to certain subjects;– Low rank components capture the illumination

conditions of the images;– Sparse components capture the sparse-supported

noises and shadows;

Decomposing Extended YaleB Dataset

Left: common components; Right: the low-rank components.

Applying Decomposed Component to Face Recognition

• According to previous discussions, common components captures the information essential to the subject and low rank components captures the variation of the images sets;

• Ideally a image from Subject should lie in a subspace spanned by its common component and the low-rank components .

Face Recognition: SubspacesReconstruct image with common component of Subject 1 and low-rank components: (a) coefficient of the reconstruction, (b) the input image, and (c) the reconstructed image.

Subject 1 Subject 2

Reconstruction with incorrect

common components

Face Recognition: Measuring the Distance of Subspaces

• We use principle angles to measure the distance between two subspaces:– Subspace of Subject – Subspace of test image

• Principle angle provides information of relative position of subspaces in Euclidean space.

Face Recognition: Experiment

• We test the algorithm on extended YaleB dataset and Multi-PIE dataset;– Randomly split the set into training sets and

testing sets;– To test its robustness over missing training

instances, we randomly remove some of the training instances and keep “# train per subject” training instances for each subject;

– The performance is compared with SRC, Volterraface and SUN.

Face Recognition: Robustness over Poses Variations

• we use all the images from 5 near frontal poses (C05, C07, C09, C27, C29), which includes153 conditions for each subject. We randomly pick M=40 illumination conditions for training and the remaining for testing.

#train per subject 20 15 10 5Proposed 99.98±0.03% 99.92±0.06% 99.24±0.06% 90.95±0.70%

SRC 99.98±0.03% 99.45±0.03% 96.79±0.28% 86.98±0.16%Volterrafaces 99.60±0.22% 98.37±0.47% 97.63±0.28% 89.72±1.45%

SUN 99.93±0.05% 99.38±0.14% 97.89±0.30% 88.29±0.02%

Variation Recognition

• The proposed method can also be used to identify the variations (e.g., illumination conditions) of the images;

• For this purpose, we construct two subspaces and measure the distances with principle angle:– Subspace of variation – Subspace of test image

Variation Recognition

We test the proposed algorithm on AR dataset, which contains 100 subjects and 2 sessions, where each session 13 variations. We use the first session for training and second session for testing.We show the confusion matrix, which presents the result in percentages.

variations

Conclusions

• We proposed a novel decomposition of a set of face images of multiple subjects, each with multiple images;

• It facilitates explicit modeling of typical challenges in face recognition, such as illumination conditions and large occlusion;

• For future work, we plan to expand the current algorithm by incorporating another step that attempts to estimate a mapping matrix for assigning a condition label to each image, during the optimization iteration.