Learning to Track: Online Multi-Object Tracking by Decision Making

Yu Xiang1,2, Alexandre Alahi1, and Silvio Savarese1

1Stanford University, 2University of Michigan

ICCV 2015

1

Multi-Object Tracking

2

Autonomous driving

Visual surveillance

Sport Analysis

Robot navigation

Batch Mode vs. Online Mode

• Batch Mode

• Online Mode

t-2 ttime axis

t-1 t+1 t+2

t-2 ttime axis

t-1 t+1 t+2

3

Tracking by Detection

4

Data Association

Tracks at time t-1 Detections at time t

time axis

?

5

Challenges

Noisy detection: false alarms and missing detections6

Challenges

Occlusion

7

Similarity Function for Data Association

Tracks at time t-1 Detections at time ttime axis

0.2

0.8

0.3

0.1

8

• Zhang et al., CVPR’08• Berclaz et al., TPAMI’11• Breitenstein et al., TPAMI’11• Pirsiavash et al., CVPR’11• Butt & Collins, CVPR’13• Milan et al., TPAMI’14Etc.

Simple Powerfulsimilarity measure optimization+Ours

Learning to Track

𝜙1( ),Similarity 𝑤1= + ⋯ 𝜙𝑛( ),𝑤𝑛+

Different features/cues between targets and detections

Weights to combine different cues(to be learned)

9

• Appearance• Location• MotionEtc.

Offline-learning vs. Online-learning

10

Offline-learning vs. Online-learning

Offline-learning

Online-learning

Training time Before Tracking

DuringTracking

With supervision

Use history of the target

11

…

…

• Li et al., CVPR’09• Kim et al., ACCV’12Etc.

Offline-learning vs. Online-learning

12

• Song et al., ECCV’08• Kuo et al., CVPR’10• Bae et al., CVPR’14Etc.

Offline-learning

Online-learning

Training time Before Tracking

DuringTracking

With supervision

Use history of the target

The target is tracked

The target is occluded

The target is tracked again

Our Solution: Tracking by Decision Making

13

Inverse Reinforcement Learning

14

tracked lost tracked

Ground truth trajectory

Tracked Lost Tracked

MarkovDecisionProcess(MDP)

Supervision

Comparison between Different Learning Strategies

15

Offline-learning

Online-learning

Ours

Training time Before Tracking

DuringTracking

Before Tracking

With supervision

Use history of the target

Comparison between Different Learning Strategies

16

Offline-learning

Online-learning

Ours

Training time Before Tracking

DuringTracking

Before Tracking

With supervision

Use history of the target

Outline

•Markov Decision Process (MDP) for a Single Target

•Online Multi-Object Tracking with MDPs

• Experiments

•Conclusion

17

Outline

•Markov Decision Process (MDP) for a Single Target

•Online Multi-Object Tracking with MDPs

• Experiments

•Conclusion

18

Active

Tracked

Inactive

Lost

objectdetection

19

Markov Decision Process for a Single Target

Active

Tracked

Inactive

Lost

objectdetection

20

Markov Decision Process for a Single Target

Active

Tracked

Inactive

Lost

objectdetection

21

Markov Decision Process for a Single Target

Active

Tracked

Inactive

objectdetection

Markov Decision Process for a Single Target

22

Markov Decision Process for a Single Target

TLD Tracker. Z. Kalal, K. Mikolajczyk, and J. Matas. Tracking-learning-detection. TPAMI, 34(7):1409–1422, 2012.23

Active

Tracked

Inactive

Lost

objectdetection Single object tracking

Template Tracking in Tracked StatesFrame 50 Frame 51

24

Template Tracking in Tracked StatesFrame 50 Frame 51

25

Template Tracking in Tracked StatesFrame 50 Frame 51

26

Template Tracking in Tracked StatesFrame 50 Frame 51

Tracked

27

Template Tracking in Tracked StatesFrame 50 Frame 57

28

Template Tracking in Tracked StatesFrame 50 Frame 57

29

Template Tracking in Tracked StatesFrame 50 Frame 57

30

Template Tracking in Tracked StatesFrame 50 Frame 57

Tracked

Lost

31

Active

Tracked

Inactive

Lost

objectdetection

Markov Decision Process for a Single Target

If lost for more than T frames

32

Data Association in Lost States

t-2 t

time axis

t-1

tracked lost

?

33

Learning the Similarity Function

𝜙1( ),Similarity 𝑤1= + ⋯ 𝜙𝑛( ),𝑤𝑛+ + 𝑏

34

( ), 1

( ), 2

…( ), M

Hard positive examples

( ), 1

( ), 2

( ), N

…

Hard negative examples

Inverse reinforcement learning: tracking objects in training videos!

Inverse Reinforcement Learning

t-2

1

2

3

4

t

time axis

t-1

tracked lost

35

Ground truth trajectory

Supervision

Inverse Reinforcement Learning

t-2

1

2

3

4

t

time axis

t-1

tracked lost

36

Ground truth trajectory

Supervision

Inverse Reinforcement Learning

t-2

1

2

3

4

t

time axis

t-1

tracked lost

Wrong decision!Update your weights!

37

Ground truth trajectory

Supervision

( ), 1

Negative example

Inverse Reinforcement Learning

t-2

1

2

3

4

t

time axis

t-1

tracked lostWrong decision!Association to this one!Update your weights!

No association

Try it again

38

Ground truth trajectory

Supervision

( ),Positive example

2

Inverse Reinforcement Learning

t-2

1

2

3

4

t

time axis

t-1

tracked lostGood job!Keep going!No update of the weights

Try it again

39

Ground truth trajectory

Supervision

Active

Tracked

Inactive

Lost

objectdetection

Markov Decision Process for a Single Target

40

Outline

•Markov Decision Process (MDP) for a Single Target

•Online Multi-Object Tracking with MDPs

• Experiments

•Conclusion

41

Ensemble MDPs for Online Multi-Object Tracking

t-2 t

time axis

MDP1

MDP2

MDP3

t-1

42

Step 1: Process tracked targets

t

time axis

MDP1

MDP2

MDP3

t-2 t-1

43

Step 2: Process lost targets

t

time axis

MDP1

MDP2

MDP3

Hungarian algorithm for lost targets

t-2 t-1

44

Step 3: Initialize new targets

t

time axis

MDP1

MDP2

MDP3

Initialize new targets

t-2 t-1

45

Terminate detection

Tracked Lost Tracked

Tracked Lost Tracked

Tracked Tracked Tracked

MDP1

MDP2

MDP3

Online Multi-Object Tracking with MDPs

46

Outline

•Markov Decision Process (MDP) for a Single Target

•Online Multi-Object Tracking with MDPs

• Experiments

•Conclusion

47

Experiments: Dataset

•Multiple Object Tracking Benchmark [1]• 11 training sequences• 11 test sequences• Object detections from the ACF detector [2]

[1] L. Leal-Taixé, A. Milan, I. Reid, S. Roth, and K. Schindler. MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking. arXiv:1504.01942 [cs], 2015.[2] P. Dollár, R. Appel, S. Belongie, and P. Perona. Fast feature pyramids for object detection. TPAMI, 36(8):1532–1545, 2014. 48

Experiments: Analysis on Validation Set

• Contribution of different components

49

Experiments: Analysis on Validation Set

• Contribution of different components

Active

Tracked

Inactive

Lost

objectdetection

50

MOTA: multiple object tracking accuracy

Experiments: Analysis on Validation Set

• Contribution of different components

Active

Tracked

Inactive

Lost

objectdetection

51

MOTA: multiple object tracking accuracy

Experiments: Analysis on Validation Set

• Contribution of different components

Active

Tracked

Inactive

Lost

objectdetection

52

MOTA: multiple object tracking accuracy

Experiments: Analysis on Validation Set

• Contribution of different components

53

𝜙1( ),Similarity 𝑤1=+⋯

𝜙𝑛( ),𝑤𝑛

+

+

𝑏MOTA: multiple object tracking accuracy

Experiments: Analysis on Validation Set

• Contribution of different components

54

𝜙1( ),Similarity 𝑤1=+⋯

𝜙𝑛( ),𝑤𝑛

+

+

𝑏MOTA: multiple object tracking accuracy

Experiments: Analysis on Validation Set

• Cross-domain tracking

55

MOTA: multiple object tracking accuracy

TUD-Stadtmitte

ETH-Bahnhof

ADL-Rundle-6

KITTI-13

PETS09-S2L1

Trai

nin

g Se

qu

ence

s

Testing sequences

Experiments: Analysis on Validation Set

• Cross-domain tracking

56

TUD-Stadtmitte

ETH-Bahnhof

ADL-Rundle-6

KITTI-13

PETS09-S2L1

MOTA: multiple object tracking accuracy

Trai

nin

g Se

qu

ence

s

Testing sequences

Experiments: Analysis on Validation Set

• Cross-domain tracking

57

TUD-Stadtmitte

ETH-Bahnhof

ADL-Rundle-6

KITTI-13

PETS09-S2L1

MOTA: multiple object tracking accuracy

Trai

nin

g Se

qu

ence

s

Testing sequences

Experiments: Evaluation on Test SetTracker Tracking Learning MOTA

DP_NMS [1] Batch N/A 14.5

TC_ODAL [2] Online Online 15.1

TBD [3] Batch Offline 15.9

SMOT [4] Batch N/A 18.2

RMOT [5] Online N/A 18.6

CEM [6] Online N/A 19.3

SegTrack [7] Batch Offline 22.5

MotiCon [8] Batch Offline 23.1

MDP (Ours) Online Online 30.3

[1] Pirsiavash et al., CVPR’ 11[2] Bae et al., CVPR’14[3] Geiger et al., TPAMI’14

58

MOTA: multiple object tracking accuracy

[4] Dicle et al., ICCV’13[5] Yoon et al., WACV’15[6] Milan et al., TPAMI’14

[7] Milan et al., CVPR’15[8] Leal-Taixé et al., CVPR’14

Tracking Results

59

60

MDP [Ours] MotiCon [Leal-Taixé et al., CVPR’14]

61

MDP [Ours] MotiCon [Leal-Taixé et al., CVPR’14]

62

MDP [Ours] MotiCon [Leal-Taixé et al., CVPR’14]

Outline

•Markov Decision Process (MDP) for a Single Target

•Online Multi-Object Tracking with MDPs

• Experiments

•Conclusion

63

Active

Tracked

Inactive

Lost

Conclusion

Object Detection

Single Object Tracking

Data AssociationTarget Re-identification

64

Code

65

Active

Tracked

Inactive

Lost

Object Detection

Single Object Tracking

Data AssociationTarget Re-identification

66

Thank you!