• Preprocessing to enhance recognition performance in the presence of: - Illumination variations- Pose/expression/scale variations- Resolution enhancement (deblurring)

• Stand-alone recognition system• Preprocessing/recognition results

- Face Recognition Grand Challenge (NIST)

Preprocessing Overview

Current state-of-the-art face recognition systems degrade significantly in performance due to variations in pose, illumination, and blurring.

Problem:

IMAGECAPTURE

PREPROCESSINGRESTORATION/ENHANCEMENT

FACERECOGNITION

SYSTEM

Solution:

POSE CORRECTION due to mismatch in facial position, facial expression and scaleILLUMINATION CORRECTION due to mismatch in lighting conditions in both indoor and outdoor environmentsDEBLURRING due to mismatch in camera focus, camera lenses, camera resolution and motion blur

Face Recognition

• No a priori information with regards to pose orientation, camera parameters, etc

• No laser scanned images for 3D reconstruction

• No manual detection of feature points• Preprocessing & Stand-Alone Recognition

Highlights of the approach

Principle

• Find a function which maps a given test (probe) image into the correct train (gallery) image

• Approach

where M is the number of training images

• Select that is maximally bijective

)( testopttrain YfX

Mi ,...,2,1

if

)( testii YfX

Recognition Principle

• A function ’f ‘ is found which maps points in the test (probe) to equivalent points in the train (gallery)

)(XfY where

X = Test image (domain)

Y = Train image (co-domain)

= Bijective function mapping X YfOne to One and Onto

(bijection)

X Y

domain (X) f range(X)

Test Train

Inverse Estimation

)(YgX

g

where

Y = Train image (domain)

X = Test image (co-domain)

= Bijective function mapping Y X

X Y

domain (Y) g range(Y)

• A function ’g ‘ is found which maps points in the train (gallery) to equivalent points in the test (probe)

Test Train

Measure of Bijectivity

X Yf

Partition X

][

)(....)()()(....)()( 2121

X

XfXfXfXfXfXfM nn

g

where n is the total number of distinct blocks in X

Blue,Green,Cyan

)(....)()( 21 pXfXfXf Red

)(....)()( 21 pXfXfXf

Measure of Bijectivity

YX g

Partition Y

][

)(....)()()(....)()( 2121

Y

YfYfYfYfYfYfM pp

g

where p is the total number of distinct blocks in Y

Blue,Green,Cyan

)(....)()( 21 pYfYfYf Red

)(....)()( 21 pYfYfYf

Measure of Bijectivity

The Bijectivity score is given by:

'' 4321 gfgf MMMMS

fMgM

'gM

'fM

= Forward (test train)

= Adaptive Forward (test train)

= Backward (train test)

= Adaptive Backward (train test)

4321 ,,, = constants and 10

Mapping Properties

Train Image No.1

Preprocessing Example

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Accepts

Fal

se R

ejec

ts

Yale Subset-I, Viisage SystemPreprocessing Performance

Original SetPreprocessed Set:Exhaustive SearchPreprocessed Set:Exhaustive Search with ConstraintsPreprocessed Set:Fast Search

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Accepts

Fal

se R

ejec

ts

Yale Subset-II, Viisage SystemPreprocessing Performance

Original SetPreprocessed Set:Exhaustive SearchPreprocessed Set:Exhaustive Search with ConstraintsPreprocessed Set:Fast Search

Preprocessing Performance

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Accepts

Fal

se R

ejec

ts

Yale Subset-II, Block Size-8x8, Search Region-88x88One-to-None Mapping Metric

Exhaustive SearchExhaustive Search with ConstraintsFast Search

Metric usedOne-to-None Mapping

Yale Subset-IBlock Size-8x8;Search-56x56

Yale Subset-IIBlock Size-8x8;Search-88x88

Exhaustive Search 100 % 80 %

Exhaustive Searchwith Constraints

100 % 83.33 %

Fast Search 98 % 83.33 %Table 4.2 Yale Results-Stand Alone Recognition-II

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Accepts

Fal

se R

ejec

ts

Yale Subset-I, Block Size-8x8, Search Region-56x56One-to-None Mapping Metric

Exhaustive SearchExhaustive Search with ConstraintsFast Search

Face recognition performance

PIE Subset- Exhaustive SearchBlock Size-8x8; Search-72x72

One-to-None Mapping One-to-One Mapping

Stand Alone Recognition Performance

91.18 % 95.59 %

Table 4.4 PIE Database-Stand Alone Recognition

PCA-based Approach recognition accuracy: 5.88 %

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

False Accepts

Fal

se R

ejec

ts

PIE Subset, Exhaustive Search Block Size-8x8, Search Region-72x72

One-to-None MappingOne-to-One MappingPCA Approach

Face recognition performance

Training Test Preprocessed

Preprocessing for Illumination Correction

• Algorithm based on image adaptive least squares illumination correction

Training image A Testing image B Adaptive segmentation

Image A illuminated as B Image B illuminated as A Least squares estimate of illumination

Preprocessing for Illumination Correction

Set tested (Yale)Enrollment Rate

(Commercial face recognition system)

Original 56 %

Preprocessed 90 %

Enrollment : Process of accepting the image and creating a feature set for recognition.

Preprocessing ResultsIllumination Correction

Comparison with Existing Methods

Test subset3D Morphable Model

Our algorithm

PIE frontal 99.8 % 99.6 %

•3D morphable models : •Good results (FRVT 2002). •Very complex, computationally expensive, •manual labeling of features

1. T. Vetter and V. Blanz, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063--1075, Sept. 2003.

Preprocessing Example

train test

Vector Field Representation of f

Preprocessed Test Image

Train Test (Poor Resolution) Reconstruted Train (Poor Resolution Test)

Test (Good Resolution) Reconstruted Train (Good Resolution Test)

Notre Dame Database

Preprocessing Example

Train Test (Poor Resolution)

Test-cap (Test ----> Train) Reconstructed Train (Test-cap ----> Train)

Preprocessing Example

Notre Dame Database

Recognition Example

Gallery Probe Bijective Mapping

With the correct gallery

White Region measure of bijectivity (52.91%)

Recognition Example

With the incorrect gallery

Gallery Probe Bijective Mapping

White Region measure of bijectivity (33.94%)

Notre Dame Database

Face Recognition Performance

Conclusion and Future Work

• New algorithm for registration and illumination correction to enhance the performance of face recognition systems

• Algorithm is based on properties of the mapping between test and train data

• Mapping produces similarity scores which can be used for a stand-alone face recognition algorithm

• Extend algorithm for high resolution data• Reduce algorithm complexity