Upload
muhammad-yasir
View
234
Download
0
Embed Size (px)
Citation preview
8/2/2019 Face Detection&Rec
1/46
FACE DETECTION
1
8/2/2019 Face Detection&Rec
2/46
OUTLINE
What is face detection.
The History.
Challenges.
2D-Image Scan.
Biometrics : Skin Texture Analysis. Evaluation of face detection
The Viola and Jones method
Applications.
2
8/2/2019 Face Detection&Rec
3/46
INTRODUCTION
Steps:
Face Detection:differentiate a human face
from the background of the image or a realtime video.
Feature Detection: record its features.
Face Recognition: Compare it to a database.
3
8/2/2019 Face Detection&Rec
4/46
INTRODUCTION
Face interface
Face detection
Face recognition
4
Face detection Face recognitionMr.Chan
Prof..Cheng
Face database
Output:
8/2/2019 Face Detection&Rec
5/46
WHAT IS FACE DETECTION
Technique
employed to
distinguish aHuman face fromthe rest of the
background ofthe image.
5
8/2/2019 Face Detection&Rec
6/46
FACE DETECTION [1] To detect faces in an image (Not recognize it yet)
Challenges
A picture has 0,1 or many faces
Faces are not the same: with spectacles, mustache etc Sizes of faces vary
Available in most digital cameras nowadays
The simple method
Slide a window across the window and detect faces
Too slow, pictures have too many pixels(1280x1024=1.3M pixels)
6
8/2/2019 Face Detection&Rec
7/46
THE HISTORY
During 1964 and 1965, Bledsoe, alongwith Helen Chan and Charles Bisson ,worked on using the computer to
recognize human faces.
He was proud of this work, but because
the funding was provided by anunnamed intelligence agency that didnot allow much publicity, little of thework was published.
7
8/2/2019 Face Detection&Rec
8/46
IMPORTANCE OF FACE DETECTION The first step for any automatic face recognition
system system.
First step in many Human Computer Interactionsystems.
Expression Recognition
Cognitive State/Emotional State Recognition
First step in many surveillance and securitysystems.
Video coding
Automatic Target Recognition(ATR)
8
8/2/2019 Face Detection&Rec
9/46
CHALLENGES
In Plane Rotation
Out Plane
Rotation Lighting
Aging Effects
Facial Expressions Face Covered by
long Hairs or Hand.9
8/2/2019 Face Detection&Rec
10/46
CHALLENGES
10
8/2/2019 Face Detection&Rec
11/46
2D IMAGE SCAN
Different
Approaches:
Knowledge Based
Approach
Feature Invariant
MethodTemplate
Matching Method
11
8/2/2019 Face Detection&Rec
12/46
KNOWLEDGE-BASED APPROACH
It uses human-
coded rules to model
facial features, such
as two symmetric
eyes, a nose in the
middle and a mouth
underneath thenose.
12
8/2/2019 Face Detection&Rec
13/46
KNOWLEDGE-BASED APPROACH-
SUMMARY
pros: Easy to come up with simple rules
Based on the coded rules, facial features in an input image areextracted first, and face candidates are identified
Work well for face localization in uncluttered background
con:
Difficult to translate human knowledge into rules precisely: detailedrules fail to detect faces and general rules may find many falsepositives
Difficult to extend this approach to detect faces in different poses:implausible to enumerate all the possible cases
13
8/2/2019 Face Detection&Rec
14/46
FEATURE INVARIANT METHOD
Feature invariant methods
try to find facial features
which are invariant to
pose, lighting condition orrotation.
Skin colors, edges and
shapes fall into thiscategory.
14
8/2/2019 Face Detection&Rec
15/46
FEATURE INVARIANT METHOD-
NODAL POINT ANALYSIS
Every face has numerous,distinguishable landmarks,the different peaks and
valleys that make up theface
o Distance between theeyes
o Width of the nose
o Depth of the eye sockets
o The shape of the
cheekbones 15
8/2/2019 Face Detection&Rec
16/46
FEATURE INVARIANT METHOD-
SUMMARY
Pros:
Features are invariant to pose and change inorientation.
Cons:
Difficult to locate facial features due toseveral corruption (illumination, noise,occlusion)
Difficult to detect features in complex 16
8/2/2019 Face Detection&Rec
17/46
TEMPLATE MATCHING METHOD
Template matchingmethods calculatethe correlation
between a testimage and a pre-selected facial
templates.
17
8/2/2019 Face Detection&Rec
18/46
TEMPLATE MATCHING METHOD-
SUMMARY
Pros:
Simple
Cons:
Templates needs to be initialized near the faceimages
Difficult to enumerate templates for different
poses (similar to knowledge-based methods)
18
8/2/2019 Face Detection&Rec
19/46
BIOMETRICS
SKIN TEXTURE ANALYSIS
Using skin color to find facesegments is a vulnerabletechnique.
Non-animate objects with
the same color as skin canbe picked up since thetechnique uses colorsegmentation.
Then the face can be pickedup using any of theapproaches.
19
8/2/2019 Face Detection&Rec
20/46
SKIN TEXTURE ANALYSIS:
ADVANTAGES
Lack of restriction to orientation or size of faces.
A good algorithm can handle complexbackgrounds.
It is relatively insensitive to changes inexpression, including blinking, frowning orsmiling
Has the ability to compensate for mustache orbeard growth and the appearance ofeyeglasses.
20
8/2/2019 Face Detection&Rec
21/46
EVALUATION OF FACE DETECTION Detection rate
Give positive results in locations where faces exist
Should be high > 95%
False positive rate
The detector output is positive but it is false (there is actually noface).Definition of False positive: A result that is erroneously positive when a situation is normal. Anexample of a false positive: a particular test designed to detect cancer of the toenail is positive but theperson does not have toenail cancer. (http://www.medterms.com/script/main/art.asp?articlekey=3377)
Should be low
8/2/2019 Face Detection&Rec
22/46
EXERCISEWhat are the detection rate and false
detection rate here?
Answer
detection rate=(8/9)*100%
false detection rate=(1/9)*100%
22
False positive result
9 faces in the picture, 8
are correctly detected.
1 window reportedto have face is infact not a face
8/2/2019 Face Detection&Rec
23/46
THE VIOLA AND JONES METHOD [1] The most famous method
Training may need weeks
Recognition is very fast, e.g. real-time for
digital cameras. Techniques
Integral image for feature extraction
Ada-Boost for feature selectionAttentional cascade for fast rejection of non-face
sub-windows
23
8/2/2019 Face Detection&Rec
24/46
24
IMAGE FEATURES REF[3]Rectangle filters
Rectangle_Feature_value f=
(pixels in white area)
(pixels in shaded area)
8/2/2019 Face Detection&Rec
25/46
EXERCISE Find the
Rectangle_Feature_val
ue(f) of the boxenclosed by the dottedline
Rectangle_Feature_value f=
(pixels in white area)
(pixels in shaded area)
f=(8+7)-(0+1) =15-1= 14
1 2 3 3
3 0 1 3
5 8 7 1
0 2 3 6
25
8/2/2019 Face Detection&Rec
26/46
26
EXAMPLE: A SIMPLE FACE DETECTIONMETHOD USING ONE FEATURE
Result
This is a face:T he eye-part is dark,the nose-part is brightSo f is large, hence it is face
This is not a face.Because f is small
Rectangle_Feature_value ff= (pixels in white area) (pixels in shaded area)
If (f) is large it is face ,i.e.if (f)>threshold, then faceElse non-face
8/2/2019 Face Detection&Rec
27/46
27
HOW TO FIND FEATURES FASTERINTEGRAL IMAGES FAST CALCULATION METHOD[LAZEBNIK09 ]
The integral image =
sum of allpixel valuesabove and to the left of(x,y)
Can be found veryquickly
(x,y)
8/2/2019 Face Detection&Rec
28/46
28
COMPUTING THE INTEGRAL IMAGE[LAZEBNIK09 ]
8/2/2019 Face Detection&Rec
29/46
29
COMPUTING THE INTEGRAL IMAGE[LAZEBNIK09 ]
Cumulative row sum: s(x, y) = s(x1, y) + i(x, y)
Integral image: ii(x, y) = ii(x, y1) + s(x, y)
ii(x, y-1)
s(x-1, y)i(x, y)
8/2/2019 Face Detection&Rec
30/46
30
CALCULATE SUM WITHIN A RECTANGLE A,B,C,D are the values of the
integral images at the corners ofthe rectangle R.
The sum of image values inside Ris:
Area_R = A B C + D
If A,B,C,D are found , only 3additions are needed to findArea_R
D
C A
B
RectangleR
Area= Area_R
8/2/2019 Face Detection&Rec
31/46
WHY DO WE NEED TO FIND PIXEL SUM OF RECTANGLES?ANSWER: WE WANT TO GET FACE FEATURES You may consider these
features as face features
Two eyes= (Area_A-Area_B)
Nose =(Area_C+Area_E-Area_D) Mouth =(Area_F+Area_H-Area_G)
They can be different sizes,
polarity and aspect ratios
31
AB
CDE
FGH
8/2/2019 Face Detection&Rec
32/46
32
FACE FEATURE AND EXAMPLE-1
+2IntegralImage
A face
Shaded area
White areaF=Feat_val =pixel sum in shared area - pixel sum in whiteareaExample
Pixel sum in white area=216+102+78+129+210+111=846
Pixel sum in shared area=10+20+4+7+45+7=93
Feat_val=F=846-93If F>threshold,
feature=+1Else
feature=-1 End if;
We can choose threshold =768 , so feature is+1.
10 20 4
7 45 7
216 102 78
129 210 111
Pixel values insideThe areas
8/2/2019 Face Detection&Rec
33/46
THE DETECTION CHALLENGE Use 24x24 base window
For y=1;y
8/2/2019 Face Detection&Rec
34/46
SOLUTION TO MAKE IT EFFICIENT The whole 162,336 feature set is too large
Solution: select good features to make it moreefficient
Use: Boosting
Boosting
Combine many small weak classifiers to become
a strong classifier.Training is needed
34
8/2/2019 Face Detection&Rec
35/46
35
BOOSTING FOR FACE DETECTION
Define weak learners based onrectangle features
otherwise0
)(if1)(
tttt
t
pxfpxh
window
value of rectangle feature
Pt=polarity{+1,-1}
threshold
8/2/2019 Face Detection&Rec
36/46
36
AdaBoost training E.g. Collect 5000 faces, and 9400 non-faces.
Different scales.
Use AdaBoost for training to build a strongclassifier.
Pick suitable features of different scales andpositions, pick the best few. (Take months to do ,details is in [Viola 2004] paper)
Testing Scan through the image, pick a window and
rescale it to 24x24,
Pass it to the strong classifier for detection.
Report face, if the output is positive
FACE DETECTION USING ADABOOST
8/2/2019 Face Detection&Rec
37/46
37
BOOSTING FOR FACE DETECTION [VIOLA2004] In the paper it shows that the following two features (obtained
after training) in cascaded picked by AdaBoost have 100%
detection rate and 50% false positive rate
But 50% false positive rate is not good enough
Approach [viola2004] :Attentional cascade
type2type3
Pick a window in theimage and rescale it to24x24 as image
I.e.H(face)=Sign{1h1(image)+2h2(image)}
H(face)=+1 faceH(face)=-1non-face
h1(image)h2(image)
8/2/2019 Face Detection&Rec
38/46
38
BOOSTING FOR FACE DETECTION A 200-feature classifier can yield 95% detection rate and a false positive rate
of 1 in 14084 (Still not god enough) Recall: False positive rate
The detector output is positive but it is false (there is actually no face).Definition of False positive: A result that is erroneously positive when a
situation is normal. An example of a false positive: a particular test
designed to detect cancer of the toenail is positive but the person does nothave toenail cancer.(http://www.medterms.com/script/main/art.asp?articlekey=3377)
Still notgoodenough!
False positive rate
CorrectDetection
rate
X10-3
8/2/2019 Face Detection&Rec
39/46
39
TO IMPROVE FALSE POSITIVE RATE:ATTENTIONAL CASCADE
Cascade of many AdaBoost strong classifiers Begin with with simple classifiers to reject many
negative sub-windows
Many non-faces are rejected at the first few
stages. Hence the system is efficient enough for real time
processing.
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False False
Non-face
8/2/2019 Face Detection&Rec
40/46
AN EXAMPLEMore features for later stages in the
cascade [viola2004]
40
2 features 10 features 25 features 50 features
type3type2
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False False
Non-face
8/2/2019 Face Detection&Rec
41/46
41
ATTENTIONAL CASCADEChain classifiers that are
progressively morecomplex and have lowerfalse positive rates: vs false neg determined by
% False Pos
%D
etection
0 50
0
100
Receiver operating
characteristic
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False False
Non-face
False positive rate
8/2/2019 Face Detection&Rec
42/46
42
ATTENTIONAL CASCADE [VIOLA2004] Detection rate for each stage is 0.99 , for 10
stages, overall detection rate is 0.9910 0.9
False positive rate at each stage is 0.3, for 10stages
false positive rate =0.310 610-6)
AdaboostClassifier1
AdaboostClassifier2
AdaboostClassifier3
True True TrueFacefound
Non-face Non-face
Input image
False False False
Non-face
F R iti
8/2/2019 Face Detection&Rec
43/46
DETECTION PROCESS IN PRACTICE[SMYTH2007]Use 24x24 sub-window Scaling
scale the detection (not the input image)
Features evaluated at scales by factors of 1.25at each level Location : move detector around the image (1
pixel increments)
Final detectionsA real face may result in multiple nearbydetections (merge them to become the finalresult)
43
Face RecognitionFace Recognition
FACE RECOGNITION
8/2/2019 Face Detection&Rec
44/46
FACE RECOGNITION
Distinguishing a specific face from other faces
2009 TotallyLooksLike.com
FACE RECOGNITION:
8/2/2019 Face Detection&Rec
45/46
FACE RECOGNITION:
APPLICATIONS Biometrics / access control
""Minority Report" 2002 20th Century Fox
Superbad" 2007 Columbia Pictures
Searching mugshotdatabases
Tagging photo albums Detecting fake ID cards
o no action requiredo scan many people at onceo places: airports, banks, safeso data: laptops, medical info
8/2/2019 Face Detection&Rec
46/46
APPLICATIONS Security measure at
ATMs Digital Cameras
Public Surveillance
(CCTVs) atAirports, Hospitals, etc.
Televisions andcomputers can
save energy by reducingthe
brightness.