View
265
Download
0
Category
Tags:
Preview:
Citation preview
22
Face RecognitionFace Recognition
IntroductionIntroductionFace recognition algorithmsFace recognition algorithmsComparisonComparisonShort summaryShort summary
33
Face Recognition AlgorithmsFace Recognition Algorithms
We will introduceWe will introduceEigenfacesEigenfacesFisherfacesFisherfacesElastic Bunch-Graph Matching Elastic Bunch-Graph Matching
44
EigenfacesEigenfaces
Developed in 1991 by M.TurkDeveloped in 1991 by M.TurkBased on Principal Component Analysis Based on Principal Component Analysis
(PCA)(PCA)Relatively simpleRelatively simpleFastFastRobustRobust
55
EigenfacesEigenfaces
PCA seeks directions that are efficient for PCA seeks directions that are efficient for representing the datarepresenting the data
efficientnot efficient
Class A
Class B
Class A
Class B
77
EigenfacesEigenfaces
PCA reduces the dimension of the dataPCA reduces the dimension of the dataSpeeds up the computational timeSpeeds up the computational time
99
Eigenfaces, the algorithmEigenfaces, the algorithm
AssumptionsAssumptionsSquare images with Width = Height = N Square images with Width = Height = N M is the number of images in the databaseM is the number of images in the databaseP is the number of persons in the databaseP is the number of persons in the database
1010
Eigenfaces, the algorithmEigenfaces, the algorithm
The databaseThe database
2
1
2
N
b
b
b
2
1
2
N
c
c
c
2
1
2
N
d
d
d
2
1
2
N
e
e
e
2
1
2
N
a
a
a
2
1
2
N
f
f
f
2
1
2
N
g
g
g
2
1
2
N
h
h
h
1111
Eigenfaces, the algorithmEigenfaces, the algorithm
We compute the average faceWe compute the average face
2 2 2
1 1 1
2 2 21, 8
N N N
a b h
a b hm where M
M
a b h
1212
Eigenfaces, the algorithmEigenfaces, the algorithm
Then subtract it from the training facesThen subtract it from the training faces
2 2 2 2 2 2 2 2
2 2
1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2
1 1 1 1
2 2
, , , ,
,
m m m m
N N N N N N N N
m m
N N
a m b m c m d m
a m b m c m d ma b c d
a m b m c m d m
e m f m
e m fe f
e m
2 2 2 2 2 2
1 1 1 1
2 2 2 2 2 2, ,m m
N N N N N N
g m h m
m g m h mg h
f m g m h m
1313
Eigenfaces, the algorithmEigenfaces, the algorithm
Now we build the matrix which is Now we build the matrix which is NN22 by by MM
The covariance matrix which is The covariance matrix which is NN22 by by NN22
m m m m m m m mA a b c d e f g h
Cov AA
1414
Eigenfaces, the algorithmEigenfaces, the algorithm
Find eigenvalues of the covariance matrixFind eigenvalues of the covariance matrixThe matrix is very largeThe matrix is very largeThe computational effort is very bigThe computational effort is very big
We are interested in at most We are interested in at most MM eigenvalues eigenvaluesWe can reduce the dimension of the matrixWe can reduce the dimension of the matrix
1515
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute another matrix which is Compute another matrix which is MM by by MM
Find the Find the MM eigenvalues and eigenvectors eigenvalues and eigenvectorsEigenvectors of Eigenvectors of Cov Cov andand L L are are equivalentequivalent
Build matrix Build matrix V V from the eigenvectors offrom the eigenvectors of L L
L A A
1616
Eigenfaces, the algorithmEigenfaces, the algorithm
Eigenvectors of Eigenvectors of CovCov are linear combination are linear combination of image space with the eigenvectors of of image space with the eigenvectors of LL
Eigenvectors represent the variation in the Eigenvectors represent the variation in the facesfaces
U AV V is Matrix of eigenvectors
m m m m m m m mA a b c d e f g h
1717
Eigenfaces, the algorithmEigenfaces, the algorithm
A: collection of the training faces
U: Face Space / Eigen Space
1919
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute for each face its Compute for each face its projection onto projection onto the face spacethe face space
Compute the Compute the thresholdthreshold
1 2 3 4
5 6 7 8
, , , ,
, , ,
m m m m
m m m m
U a U b U c U d
U e U f U g U h
1max , 1..
2 i j for i j M
2020
Eigenfaces: Eigenfaces: Recognition ProcedureRecognition Procedure
To recognize a faceTo recognize a face
Subtract the average face from itSubtract the average face from it
2
1
2
N
r
r
r
2 2
1 1
2 2
m
N N
r m
r mr
r m
2121
Eigenfaces, the algorithmEigenfaces, the algorithm
Compute its Compute its projectionprojection onto the onto the face space Uface space U
mU r
22 1..i i for i M
Compute the distance in the face space Compute the distance in the face space between between the facethe face and and all known facesall known faces
2222
Eigenfaces, the algorithmEigenfaces, the algorithm
Reconstruct the face from eigenfacesReconstruct the face from eigenfaces
Compute the distance between Compute the distance between the facethe face and and its reconstructionits reconstruction
s U
22mr s
2323
Eigenfaces, the algorithmEigenfaces, the algorithm
Distinguish betweenDistinguish between If then it’s not a face; the distance If then it’s not a face; the distance
between between the facethe face and and its reconstructionits reconstruction is is larger than thresholdlarger than threshold
If then it’s a new faceIf then it’s a new face If then it’s a known If then it’s a known
face because the distance in the face space face because the distance in the face space between between the facethe face and and all known facesall known faces is larger is larger than thresholdthan threshold
, ( 1.. )iand i M min iand
2424
Eigenfaces, the algorithmEigenfaces, the algorithm
Problems with eigenfacesProblems with eigenfacesDifferent illuminationDifferent illumination
2525
Eigenfaces, the algorithmEigenfaces, the algorithm
Problems with eigenfacesProblems with eigenfacesDifferent head poseDifferent head poseDifferent alignmentDifferent alignmentDifferent facial expressionDifferent facial expression
2626
FisherfacesFisherfaces
Developed in 1997 by P.Belhumeur et al.Developed in 1997 by P.Belhumeur et al.Based on Fisher’s Linear Discriminant Based on Fisher’s Linear Discriminant
Analysis (LDA)Analysis (LDA)Faster than eigenfaces, in some casesFaster than eigenfaces, in some casesHas lower error ratesHas lower error ratesWorks well even if different illuminationWorks well even if different illuminationWorks well even if different facial express.Works well even if different facial express.
2727
FisherfacesFisherfaces
LDA seeks directions that are efficient for LDA seeks directions that are efficient for discrimination between the datadiscrimination between the data
Class A
Class B
2828
FisherfacesFisherfaces
LDA maximizes the between-class scatterLDA maximizes the between-class scatterLDA minimizes the within-class scatterLDA minimizes the within-class scatter
Class A
Class B
2929
Fisherfaces, the algorithmFisherfaces, the algorithm
AssumptionsAssumptionsSquare images with Width=Height=N Square images with Width=Height=N M is the number of images in the databaseM is the number of images in the databaseP is the number of persons in the databaseP is the number of persons in the database
3030
Fisherfaces, the algorithmFisherfaces, the algorithm
The databaseThe database
2
1
2
N
b
b
b
2
1
2
N
c
c
c
2
1
2
N
d
d
d
2
1
2
N
e
e
e
2
1
2
N
a
a
a
2
1
2
N
f
f
f
2
1
2
N
g
g
g
2
1
2
N
h
h
h
3131
Fisherfaces, the algorithm Fisherfaces, the algorithm
We compute the average We compute the average of all facesof all faces
2 2 2
1 1 1
2 2 21, 8
N N N
a b h
a b hm where M
M
a b h
3232
Fisherfaces, the algorithmFisherfaces, the algorithm
Compute the average face Compute the average face of each personof each person
2 2 2 2
2 2 2 2
1 1 1 1
2 2 2 2
1 1 1 1
2 2 2 2
1 1, ,
2 2
1 1,
2 2
N N N N
N N N N
a b c d
a b c dx y
a b c d
e f g h
e f g hz w
e f g h
3333
Fisherfaces, the algorithmFisherfaces, the algorithm
And subtract them from the training facesAnd subtract them from the training faces
2 2 2 2 2 2 2 2
2 2
1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 2
1 1 1 1
2 2
, , , ,
,
m m m m
N N N N N N N N
m m
N N
a x b x c y d y
a x b x c y d ya b c d
a x b x c y d y
e z f z
e z fe f
e z
2 2 2 2 2 2
1 1 1 1
2 2 2 2 2 2, ,m m
N N N N N N
g w h w
z g w h wg h
f z g w h w
3434
Fisherfaces, the algorithmFisherfaces, the algorithm
We build scatter matrices We build scatter matrices SS11, , SS22, , SS33, , SS44
And the And the within-class within-class scatter matrix scatter matrix SSWW
1 2
3 4
, ,
,
m m m m m m m m
m m m m m m m m
S a a b b S c c d d
S e e f f S g g h h
1 2 3 4WS S S S S
3535
Fisherfaces, the algorithmFisherfaces, the algorithm
The between-class scatter matrixThe between-class scatter matrix
We are seeking the matrix We are seeking the matrix WW maximizing maximizing
2 2 2 2BS x m x m y m y m z m z m w m w m
B
W
W S WJ W
W S W
3636
Fisherfaces, the algorithmFisherfaces, the algorithm
If If SSW W is nonsingular ( ):is nonsingular ( ):Columns of Columns of W W are eigenvectors ofare eigenvectors of
We have to compute the inverse of We have to compute the inverse of SSWW
We have to multiply the matricesWe have to multiply the matricesWe have to compute the eigenvectorsWe have to compute the eigenvectors
1W BS S
2M N
3737
Fisherfaces, the algorithmFisherfaces, the algorithm
If If SSW W is nonsingular ( ):is nonsingular ( ):Simpler:Simpler:
Columns of Columns of W W are eigenvectors satisfyingare eigenvectors satisfying
The eigenvalues are roots ofThe eigenvalues are roots of
Get eigenvectors by solvingGet eigenvectors by solving
B i i W iS w S w
0B i WS S
0B i W iS S w
2M N
3838
Fisherfaces, the algorithmFisherfaces, the algorithm
If If SSW W is singular ( ):is singular ( ):Apply PCA firstApply PCA first
Will reduce the dimension of faces from Will reduce the dimension of faces from NN22 to to MMThere are There are MM MM-dimensional vectors-dimensional vectors
Apply LDA as describedApply LDA as described
2M N
3939
Fisherfaces, the algorithmFisherfaces, the algorithm
Project faces onto the LDA-spaceProject faces onto the LDA-space
To classify the faceTo classify the faceProject it onto the LDA-spaceProject it onto the LDA-spaceRun a nearest-neighbor classifierRun a nearest-neighbor classifier
, ,
,
LDA LDA
LDA LDA
x W x y W y
z W z w W w
4040
Fisherfaces, the algorithmFisherfaces, the algorithm
ProblemsProblemsSmall databases Small databases The face to classify must be in the DBThe face to classify must be in the DB
4343
ComparisonComparison
FERET databaseFERET database
best ID rate: eigenfaces 80.0%, fisherfaces 93.2%best ID rate: eigenfaces 80.0%, fisherfaces 93.2%
4444
ComparisonComparison
EigenfacesEigenfacesproject faces onto a lower dimensional sub-project faces onto a lower dimensional sub-
spacespaceno distinction between inter- and intra-class no distinction between inter- and intra-class
variabilitiesvariabilities optimal for representation but not for optimal for representation but not for
discriminationdiscrimination
4545
ComparisonComparison
FisherfacesFisherfaces find a sub-space which maximizes the ratio of find a sub-space which maximizes the ratio of
inter-class and intra-class variabilityinter-class and intra-class variabilitysame intra-class variability for all classessame intra-class variability for all classes
4646
Local Feature AnalysisLocal Feature Analysis-- Elastic Bunch-Graph Matching-- Elastic Bunch-Graph Matching
4747
Face FeaturesFace Features
Facial recognition utilizes distinctive features of Facial recognition utilizes distinctive features of the face – including: distinct the face – including: distinct micro elementsmicro elements like: like: Mouth, Nose, Eye, Cheekbones, Chin, Lips, Forehead, Mouth, Nose, Eye, Cheekbones, Chin, Lips, Forehead,
EarsEars
Upper outlines of the eye sockets, the areas Upper outlines of the eye sockets, the areas surrounding the cheekbones, the sides of the surrounding the cheekbones, the sides of the mouth, and the location of the nose and eyes. mouth, and the location of the nose and eyes.
The distance between the eyes, the length of the The distance between the eyes, the length of the nose, and the angle of the jaw.nose, and the angle of the jaw.
4848
Face FeaturesFace Features Some technologies do Some technologies do notnot utilize utilize areas of the face areas of the face
located near the hairlinelocated near the hairline, so they are somewhat resistant , so they are somewhat resistant to moderate changes in hairstyle. to moderate changes in hairstyle.
When used in identification mode, facial recognition When used in identification mode, facial recognition technology generally returns technology generally returns candidate lists of close candidate lists of close matchesmatches as opposed to returning a single definitive as opposed to returning a single definitive match as does fingerprint and iris-scan. match as does fingerprint and iris-scan.
The file containing The file containing facialfacial micro featuresmicro features is called a is called a "template." "template."
Using templates, the software then compares that image Using templates, the software then compares that image with another image and produces with another image and produces a scorea score that measures that measures how similar the images are to each other. how similar the images are to each other.
4949
Typical sources of images for use in facial Typical sources of images for use in facial recognition include video camera signals recognition include video camera signals and pre-existing photos such as those in and pre-existing photos such as those in driver's license databases. driver's license databases. including:including:Distance between the micro elementsDistance between the micro elementsA reference featureA reference featureSize of the micro elementSize of the micro elementAmount of head radiated from the face Amount of head radiated from the face
(unseen by human eye). Heat can be (unseen by human eye). Heat can be measured using an infrared camera. measured using an infrared camera.
Face FeaturesFace Features
5050
A face recognition based on local feature analysis
A face is represented as a graph, whose nodes, positioned in correspondence to the facial fiducial points. A fiducial point is a point or line on a scale used for
reference or comparison purposes.
A face recognition system uses an automatic approach to localize the facial fiducial points.
It then determines the head pose and compares the face with the gallery images.
This approach is invariant to rotation, light and scale.
5252
EBGM
Elastic Bunch-Graph Matching (EBGM) algorithm locates landmarks on an image, such as the eyes, nose, and mouth.
Gabor jets are extracted from each landmark and are used to form a face graph for each image. A face graph serves the same function as the projected vectors in the PCA or LDA algorithm; they represent the image in a low dimensional space.
After a face graph has been created for each test image, the algorithm measures the similarity of the face graphs.
Paper:Paper:http://www.snl.salk.edu/~fellous/posters/Bu97poster/BUPoster.pdf
5353
SummarySummary
Three algorithms have been introducedThree algorithms have been introducedEigenfacesEigenfaces
Reduce the dimension of the data from Reduce the dimension of the data from NN22 to to MMVerify if the image is a face at allVerify if the image is a face at allAllow online trainingAllow online trainingFast recognition of facesFast recognition of facesProblems with illumination, head pose etc Problems with illumination, head pose etc
5454
SummarySummaryFisherfaces
Reduce dimension of the data from Reduce dimension of the data from NN22 toto P-1 P-1Can outperform eigenfaces on a representative Can outperform eigenfaces on a representative
DBDBWorks also with various illuminationsWorks also with various illuminationsCan only classify a face which is “known” to DBCan only classify a face which is “known” to DB
Elastic Bunch-Graph MatchingReduce the dimension of the data from Reduce the dimension of the data from NN22 to to MMRecognize face with different posesRecognize face with different posesRecognize face with different expressionsRecognize face with different expressions
5555
ReferencesReferences
[1] M. Turk, A. Pentland, “Face Recognition Using Eigenfaces”[1] M. Turk, A. Pentland, “Face Recognition Using Eigenfaces”
[2] J. Ashbourn, Avanti, V. Bruce, A. Young, ”Face Recognition Based [2] J. Ashbourn, Avanti, V. Bruce, A. Young, ”Face Recognition Based on Symmetrization and Eigenfaces”on Symmetrization and Eigenfaces”
[3] http://www.markus-hofmann.de/eigen.html [3] http://www.markus-hofmann.de/eigen.html
[4] [4] P. Belhumeur, J. Hespanha, D. Kriegman, “Eigenfaces vs Eigenfaces vs Fisherfaces: Recognition using Class Specific Linear Projection”Fisherfaces: Recognition using Class Specific Linear Projection”
[5] R. Duda, P. Hart, D. Stork, “Pattern Classification”, ISBN 0-471-[5] R. Duda, P. Hart, D. Stork, “Pattern Classification”, ISBN 0-471-05669-3, pp. 121-12405669-3, pp. 121-124
[6] F.[6] F. Perronin, J.-L. Dugelay, “Deformable Face Mapping For Person Deformable Face Mapping For Person Identification”, ICIP 2003, BarcelonaIdentification”, ICIP 2003, Barcelona
[7] B. Moghaddam, C. Nastar, and A. Pentland. A bayesian similarity [7] B. Moghaddam, C. Nastar, and A. Pentland. A bayesian similarity measure for direct image matching. ICPR, B:350–358, 1996.measure for direct image matching. ICPR, B:350–358, 1996.
http://www.face-rec.org/interesting-papers/
5656
Hands-on Lab of Face BiometricsHands-on Lab of Face Biometrics
Present one of the following algorithmsElastic Bunch-Graph Matching (EBGM)
algorithmBayesian Intrapersonal/Extrapersonal
Classifier, orOne from
http://www.face-rec.org/interesting-papers/ Hands-on Lab of Face BiometricsHands-on Lab of Face Biometrics
http://www.cs.colostate.edu/evalfacerec/User Guide User Guide
Recommended