Semantic Texton Forests for Image Classification and Segmentation

Lecture 5-6 Object Detection Boosting Tae-Kyun Kim

EE462 MLCVEE462 MLCV1Face Detection Demo2Robust real-time object detector, Viola and Jones, CVPR 01Implemented by Intel OpenCVEE462 MLCVEE462 MLCV3

Multiclass object detection [Torralba et al PAMI 07]A boosting algorithm, originally for binary class problems, has been extended to multi-class problems. EE462 MLCV3Object Detection4

Input is a single image, given without any prior knowledge.EE462 MLCV4Object Detection5

Output is a set of tight bounding boxes (positions and scales) of instances of a target object class (e.g. pedestrian). EE462 MLCV5

6Object DetectionWe scan every scale and pixel location of an image.EE462 MLCV6

Number of windows7Number of Windows: 747,666x# of scales# of pixelsIt ends up with a huge number of candidate sub-windows .EE462 MLCV7

Time per window8

or raw pixels

dimension D

Num of feature vectors: 747,666

Classification:What amount of time are we given to process a single scanning window?SIFTEE462 MLCV8

Time per window9

or raw pixels

dimension D

Num of feature vectors: 747,666Time per window (or vector): 0.00000134 secIn order to finish the task in 1 secNeural Network?Nonlinear SVM?EE462 MLCV9

Examples of face detection10

From Viola, Jones, 2001EE462 MLCV10By Integrating Visual Cues [Darrell et al IJCV 00].

Face pattern detection output (left).Connected components recovered from stereo range data (mid).Flesh hue regions from skin hue classification (right).

11

More traditionallyThe search space is narrowed down EE462 MLCV11

Since about 2001 (Viola &Jones 01) Boosting Simple Features has been a dominating art.Adaboost classification

Weak classifiers: Haar-basis like functions (45,396 (>>T) in total feature pool)12

Weak classifierStrong classifier

EE462 MLCV12Introduction to Boosting Classifiers- AdaBoost (Adaptive Boosting)

EE462 MLCVEE462 MLCV13Sorry for inconsistent notations 14EE462 MLCVEE462 MLCVBoosting15

EE462 MLCVBoosting16

EE462 MLCV

Principally, Boosting doesIteratively reweighting training samples, Assigning higher weights to previously misclassified samples.17

1 round2 rounds3 rounds4 rounds5 rounds50 roundsEE462 MLCV1718In the previous example, We consider all horizontal or vertical lines.

For each line, we define two weaklearners. +1-1-1+1EE462 MLCVAdaBoost19

M : # of weak classifiers to choose, among all weak classifiers, EE462 MLCV20

EE462 MLCVBoosting21

EE462 MLCVBoosting as an optimisation framework22EE462 MLCVEE462 MLCVMinimising Exponential Error23

EE462 MLCV24

EE462 MLCV25

EE462 MLCV26

EE462 MLCV27

EE462 MLCV

Existence of weak learnersDefinition of a baseline learnerData weights:SetBaseline classifier: for all xError is at most .

Each weak learner in Boosting is demanded s.t.

Error of the composite hypothesis goes to zero as boosting rounds increase [Duffy et al 00].28

EE462 MLCV28Robust real-time object detector

EE462 MLCVEE462 MLCV29

Boosting Simple Features [Viola and Jones CVPR 01]Adaboost classification

Weak classifiers: Haar-basis like functions (45,396 in total feature pool)30

Weak classifierStrong classifier

2020EE462 MLCV3031

Learning (concept illustration)Face imagesNon-face imagesResize to 20x20

D=400

weaklearners

Output: EE462 MLCV

Evaluation (testing)32

From Viola, Jones, 2001

For given Non-local maxima suppressionwe apply the boosting classifier to every scan-window. Non-local maxima supression is performed. EE462 MLCV32How to accelerate boosting training and evaluation

EE462 MLCVEE462 MLCV33

Integral ImageA value at (x,y) is the sum of the pixel values above and to the left of (x,y).

The integral image can be computed in one pass over the original image.

34

EE462 MLCV34

Boosting Simple Features [Viola and Jones CVPR 01]Integral imageThe sum of original image values within the rectangle can be computed: Sum = A-B-C+D

This provides the fast evaluation of Haar-basis like features35

EE462 MLCV35

Evaluation (testing)36

From Viola, Jones, 2001

xyii(x,y)EE462 MLCV36Boosting as a Tree-structured Classifier

EE462 MLCVEE462 MLCV37

Boosting (very shallow network) The strong classifier H as boosted decision stumps has a flat structure

Cf. Decision ferns has been shown to outperform trees [Zisserman et al, 07] [Fua et al, 07]38

c0 c1c0 c1c0 c1c0 c1c0 c1c0 c1

xEE462 MLCV38

Boosting -continued

Good generalisation is achieved by a flat structure.It provides fast evaluation.It does sequential optimisation.39A strong boosting classifierBoosting Cascade [viola & Jones 04], Boosting chain [Xiao et al]It is very imbalanced tree structured.It speeds up evaluation by rejecting easy negative samples at early stages. It is hard to design

A strong boosting classifierT = 25102050100EE462 MLCV39

A cascade of classifiersThe detection system requires good detection rate and extremely low false positive rates.False positive rate and detection rate are

f_i is the false positive rate of i-th classifier on the examples that get through to it.The expected number of features evaluated is

p_j is the proportion of windows input to i-th classifier.40

EE462 MLCV40

Demo video: Fast evaluation41

EE462 MLCV41

Object Detection by a Cascade of Classifiers42

Pictures from Romdhani et al. ICCV01 It speeds up object detection by coarse-to-fine search.EE462 MLCV42