Upload
kyle-wilson
View
222
Download
3
Tags:
Embed Size (px)
Citation preview
Automatic Face Feature Localization for Face Recognition
Christopher I. Fallinhonors thesis defense: May 1, 2009
advisor: Dr. Patrick J. Flynn
May 1, 2009 1Chris Fallin - thesis defense
Outline
• Face recognition– Methods of evaluation
• Elastic Bunch Graph Matching– Gabor Jets– Bunch Graphs and Feature Localization
• My Contributions: Automatic Fiducial Points– Information Content model– Fiducial point placement– results
Face Recognition
May 1, 2009 3Chris Fallin - thesis defense
• Subfield of biometrics:– life (bio)– measure (metric)– Extract identifying
information from measures of human traits
• Face recognition: digital images of face
• 2D, 3D, infrared, multimodal, …
http://www-users.cs.york.ac.uk/~nep/research/3Dface/tomh/3DFaces.jpg
May 1, 2009 Chris Fallin - thesis defense 4
Image Set
Face Recognition Evaluation
May 1, 2009 Chris Fallin - thesis defense 5http://en.wikipedia.org/wiki/File:Roc-general.png – used under terms of GNU FDL
System Decision
ROC curvesY N
Actual
Y
N
May 1, 2009 Chris Fallin - thesis defense 6
EER = 11.1%
Rank-one Score
May 1, 2009 Chris Fallin - thesis defense 7
Gallery A B C D E F GA 0.89 0.70 0.10 0.52 0.34 0.48 0.37B 0.70 0.73 0.45 0.82 0.12 0.43 0.44
Probe C 0.10 0.45 0.92 0.89 0.23 0.82 0.13D 0.52 0.82 0.89 0.56 0.20 0.38 0.14E 0.34 0.12 0.23 0.20 0.82 0.52 0.23F 0.48 0.43 0.82 0.38 0.52 0.84 0.11G 0.37 0.44 0.13 0.14 0.23 0.11 0.99
Rank-one: 5/7 = 71.4%
Elastic Bunch Graph Matching (EBGM)
• Wiskott et al., USC/Bochum, mid-90s• Basis of ZN-Face, successful commercial system• We use Face Identity Evaluation System, from
Colorado State• Face features represented by Gabor filter
responses• Features are localized
– Fit an elastic graph onto the features by localization: local optimization problems
May 1, 2009 8Chris Fallin - thesis defense
A Face Graph
May 1, 2009 Chris Fallin - thesis defense 9
[Wiskott99]
Gabor Jets
• Vector of filter responses to 40 Gabor kernels– 5 wavelengths– 8 orientations– Each is complex-valued
• Gabor jets capture information well: Gokberk et al. get 91% rank-one with fixed grid– On FERET: 78.5% max, with
12 grid points
May 1, 2009 Chris Fallin - thesis defense 10
Bunch Graphs
• Each feature has a bunch of canonical jets
• Represents typical features
• Best-match at each feature point for novel images
May 1, 2009Chris Fallin - thesis defense
11
[Wiskott99]
Feature Localization
• Initial alignment: eye locations known a-priori• Overlay bunch graph with average edge
lengths• Take Gabor jets; pick best match in each
bunch• Localize based on displacement estimation
(local optimization problem)
May 1, 2009 12Chris Fallin - thesis defense
The Idea: Automatic Fiducial Point Placement
• Bunch graph training requires manual fiducial point placement– 70 images, 25 points
• Why not statistically determine optimal features to match on?
• We can align/normalize all faces and take some statistical measure at each point in “face space” to determine goodness
• Replaces training step; back-end algorithm is identical
May 1, 2009 13Chris Fallin - thesis defense
Related Work
• Gokberk et al.: choosing fiducial points with genetic algorithms– But their chosen points are global– Same goal as our system, excluding prelocalization
• Salient Points– Wavelet-based approach to image retrieval– Choras et al., 2006: similar approach with
goodness function, but no EBGM
May 1, 2009 14Chris Fallin - thesis defense
Information Content: Variance Model
• Compute goodness function over face-space
• Inter-subject variance over intra-subject variance
• Self-normalizing unitless measure
• Requires multiple images per subject
May 1, 2009 Chris Fallin - thesis defense 15
Computing the goodness function
• FRGC: 5404 images – 700 MB, 128x128 grayscale (7 GB before normalization)
• Each pixel: 12 seconds, on fast Athlon 64• Split into 128 Condor jobs
– Each pixel is independent: easy• Pre-normalize image set, dump to fast-loading
binary format (single file)• Run Condor jobs: three hours• Post-processing to reassemble results
May 1, 2009 Chris Fallin - thesis defense 16
Fiducial Point Placement• Random placement with
probability density• Compute gradient of
goodness function• Probability is product of
gradient and goodness• Place points sequentially,
decay probability around points exponentially
• Mirror-point constraint: mirror placements across centerline, or snap to center
May 1, 2009 Chris Fallin - thesis defense 17
Prelocalization: Pseudo-Bunches
• Displacement estimation requires canonical feature jet from bunch
• We can’t provide this if we have no knowledge of feature
• Solution: fake a jet bunch– Make educated guess with K-means clustering on
jets from all images at given point• Then, run displacement estimation to
prelocalize points on each image
May 1, 2009 18Chris Fallin - thesis defense
Results
• Competitive with original, manual points– In both cases, automatic
training points yield only ~1% performance drop
– With no human training!
• Prelocalization did not work as intended
• Success without this suggested by Gokberk’s results
May 1, 2009 Chris Fallin - thesis defense 19
FERET
FRGC
ROC curves
FERET FRGC
May 1, 2009 Chris Fallin - thesis defense 20
(EER = 11.4%)(orig: 11.1%)
(EER = 31.4%)(orig: 34.8%)
Prelocalization: causes for failure
• Poor pseudo-bunch clustering: K-means often found optimal clustering at self-imposed cap of N/10 clusters– Likely because initial jets are too far off
• Naïve localization: single-step– Bolme thesis compares several optimization
algorithms• Average displacement of 2.628 pixels: larger
than 2.021 pixels in manual points
May 1, 2009 21Chris Fallin - thesis defense
Future Work
• More sophisticated prelocalization• Look at pseudo-bunch statistics to determine
failure mode in more detail• Look at per-fiducial point statistics to
determine where performance is weak• Investigate: are manual pts a theoretical limit,
or can we exceed them?• Try new image classes – test claim of
genericism
May 1, 2009 22Chris Fallin - thesis defense
Questions?
• Email [email protected]• Full thesis and source code will be posted
online: http://c1f.net/research/mark5/
May 1, 2009 23Chris Fallin - thesis defense
Distance Metrics on Jets
• Phase-insensitive: magnitude only– Selects best jet in bunch
• Phase-sensitive– Can solve for
displacement vector: basis of localization
• Displacement estimation
May 1, 2009 Chris Fallin - thesis defense 24