Helmut Grabner 1,3, Jan Sochman 2, Horst Bischof 1, Jiri Matas 2

Training Sequential On-line Boosting Classifiers for Visual Tracking

1Institute for Computer Graphics and Vision, Graz University of Technology, Austria2Center for Machine Perception, Czech Technical University, Prague3Computer Vision Laboratory, ETH Zurich, Switzerland

M. Grabner, H. Grabner and H Bischof. Real-time tracking with on-line feature selection. CVPR 2006.

2008/12/09 - ICPR, Tampa 2Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Tracking Requirements

� Adaptivity

� Robustness� Robustness

� Generality

2008/12/09 - ICPR, Tampa 3Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Agenda

� Introduction

� Tracking as Binary Classification� On-line Boosting for Feature Selection� On-line Boosting for Feature Selection

� Sequential On-line Boosting

� Experiments� Conclusion

4Helmut Grabner - Computer Vision Laboratory – ETH Zurich2008/12/09 - ICPR, Tampa

Tracking as ClassificationH. Grabner, M. Grabner and H. Bischof. Real-time tracking

via On-line Boosting, BMVC 2006.

object

backgroundvs.

2008/12/09 - ICPR, Tampa 5Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Tracking as ClassificationH. Grabner, M. Grabner and H. Bischof. Real-time tracking

via On-line Boosting, BMVC 2006.

object

backgroundvs.

2008/12/09 - ICPR, Tampa 6Helmut Grabner - Computer Vision Laboratory – ETH Zurich

search Region

actual object position

from time t to t+1 evaluate classifier on sub-patches

-

+

- -

-

create confidence map

analyze map and set newobject position

update classifier (tracker)

2008/12/09 - ICPR, Tampa 7Helmut Grabner - Computer Vision Laboratory – ETH Zurich

The Classifier

Combination of simple image featuresusing Boosting as Feature Selection

P. Viola and M. Jones. Rapid object detection using a boosted cascade of simple features. CVPR 2001.

2008/12/09 - ICPR, Tampa 8Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Boosting

Strong classifier Weak classifier

Y. Freund and R. Schapire. A decision-theoretic generalization of on-line lear ning and an application to boosting. Journal of Computer and System Sciences, 1997.

Reweighting the training Examples

2008/12/09 - ICPR, Tampa 9Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Boosting for Feature Selection

� Each feature corresponds to a weak classifier

K. Tieu and P. Viola. Booting Image Retrival, CVPR 2000Boosting is performed on the Selectors and not on

� On-line� Introducing “Selector”

� selects one feature from its local feature pool

2008/12/09 - ICPR, Tampa 10Helmut Grabner - Computer Vision Laboratory – ETH Zurich

H. Grabner and H., Bischof. On-line Boosting and Vision, CVPR 2006

the Selectors and not on the weak classifiers

directly.

H. Grabner and H., Bischof. On-line Boosting and Vision, CVPR 2006

2008/12/09 - ICPR, Tampa 11Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Unsolved Problems…

2008/12/09 - ICPR, Tampa 12Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Problems…

h1,1

one

traning

sample

h1,2

h2,1

h2,2

hN,1

hN,2

hN,mestimate

importance

estimate

importance

.

.

.

inital

importance .

.

.

.

hSelector1 hSelector2 hSelectorNN

h1,M h2,M

h2,m

hN,M

update update update

current strong classifier hStrong

repeat for each

trainingsample

.

. .

.

.

(i) TRAINING:Determine the Classifier

Complexity

2008/12/09 - ICPR, Tampa 13Helmut Grabner - Computer Vision Laboratory – ETH Zurich

h1,1

one

traning

sample

h1,2

h2,1

h2,2

hN,1

hN,2

hN,mestimate

importance

estimate

importance

.

.

.

inital

importance .

.

.

.

hSelector1 hSelector2 hSelectorNN

h1,M h2,M

h2,m

hN,M

update update update

current strong classifier hStrong

repeat for each

trainingsample

.

. .

.

.

(ii) TESTING:Optimization of the

classifier evaluation speed

2008/12/09 - ICPR, Tampa 14Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Wald’sSequential

2008/12/09 - ICPR, Tampa 15Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Sequential Probability Rate

Test (SPRT)

Abraham Wald (1902-1950)

Sequential Decision Making

� Sequential Decision Strategy S

Ordered sequence of measurements

decision

� Goal: Train (look for)

2008/12/09 - ICPR, Tampa 16Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Average time-to-decision (-1/+1) Allowed false positive and false negative rate

Ordered sequence of measurements

Sequential Probability Ratio Test

2008/12/09 - ICPR, Tampa 17Helmut Grabner - Computer Vision Laboratory – ETH Zurich

(near) optimal setting

Likelihood ratio

� Likelihood ratio

� Replace with weak classifier

� 1D projection

WaldBoost (1)J. Sochman, J. Matas, WaldBoost – Learning for Time

Constrained Sequential Detection, CVPR, 2005.

� 1D projection

2008/12/09 - ICPR, Tampa 18Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Boosting

� Threshed on the boosted classifier response

WaldBoost (2)J. Sochman, J. Matas, WaldBoost – Learning for Time

Constrained Sequential Detection, CVPR, 2005.

� Used in…� … training for bootstrapping important samples� … testing for early stopping (fast decision making)

2008/12/09 - ICPR, Tampa 19Helmut Grabner - Computer Vision Laboratory – ETH Zurich

h1,1

one

traning

sample

h1,2

h1,M

h2,1

h2,2

h2,M

h2,m

hN,1

hN,2

hN,M

hN,m

estimate

importance

estimate

importance

.

.

.

inital

importance .

.

.

.

.

.

.

.

.

hSelector1 hSelector2 hSelectorN

On-line Boosting WaldBoost

2008/12/09 - ICPR, Tampa 20Helmut Grabner - Computer Vision Laboratory – ETH Zurich

update update update

current strong classifier hStrong

repeat for each

trainingsample

On-line Wald Boost

On-line WaldBoost

2008/12/09 - ICPR, Tampa 21Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Back to Visual Tracking...

(i) TRAINING:Determine the Classifier

Complexity

object

backgroundvs.

2008/12/09 - ICPR, Tampa 22Helmut Grabner - Computer Vision Laboratory – ETH Zurich

search Region

actual object position

from time t to t+1 evaluate classifier on sub-patches

(ii) TESTING:Optimization of the

classifier evaluation speed

-

+

- -

-

create confidence map

analyze map and set newobject position

update classifier (tracker)

2008/12/09 - ICPR, Tampa 23Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Experiments - Tracking

Tracking Result Confidence Map

2008/12/09 - ICPR, Tampa 24Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Experiment: Speedup

On average a factor of

5-10!

2008/12/09 - ICPR, Tampa 25Helmut Grabner - Computer Vision Laboratory – ETH Zurich

5-10!

Conclusion and Outlook

� Wald-decision theory combined with on-line boosting for feature selection

� (i) optimized classifier evaluation

On-line Boosting

� (i) optimized classifier evaluation speed

� (ii) automatic determination of the classifier complexity

2008/12/09 - ICPR, Tampa 26Helmut Grabner - Computer Vision Laboratory – ETH Zurich

On-line WALD-Boosting

Conclusion and Outlook

On-line WALD-Boosting

DetectionBackground ModelingTracking Background ModelingTracking

This presentation

2008/12/09 - ICPR, Tampa 27Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Thank you for your attention!

2008/12/09 - ICPR, Tampa 28Helmut Grabner - Computer Vision Laboratory – ETH Zurich

Thank you for your attention!