Upload
yu-sheng-yosen-chen
View
509
Download
0
Tags:
Embed Size (px)
Citation preview
Multi-Camera Multi-Human
Tracking System
Copyright © Yu-Sheng Chen [Yosen]
Outline
Top view of the entire tracking system
What local trackers do
Identify people depth order
Handle body occlusion
What central server does
Compare observations across cameras
Classify to different people
Recognize people when re-entering
Yosen Chen, Copyright Reserved 2
Images
Local camera
#A tracker
,
1
A
tz 2
A
tz
2
B
tz1
B
tzLocal camera
#B trackerImages
Central Server
multi-camera correspondence
Person#1 Person#2
Identify how many people
in the observed space
Shared data on Inter-process platform
, 2
A
tz 1
B
tz1
A
tz 2
B
tz
Listen/
Broadcast
What local trackers do?
Multi-human
tracking &
detection
Central
Server
Operation flow of local trackers
Identify
people depth
order
If detect new,
broadcastGet people labels
& people heights.
Handle body
occlusionStart
Keep updating body info. to server databases
Extract
people body
info.
Local trackerCentral server
What local trackers do: Identify depth order
A. By appearances
B. By standing locations
Image source
1:Front
2:Rear
Depth order estimation by head shape completeness is a bad idea!
Yosen Chen, Copyright Reserved 6
What local trackers do: Identify depth order
A. By appearances
B. By standing locations
Image source
1:Front
2:Rear
Yosen Chen, Copyright Reserved 8
Tell the depth order by people’s
standing locations
How to know the standing locations?
To be discussed in the central server part!!
Bd Ground plane
Ad
Camera image
Cd
BdAd Cd depth order: A=1, B=2, C=3
A
BC
Depth order estimation by 3D standing locations is accurate.
Yosen Chen, Copyright Reserved 9
What local trackers do: Handle body occlusion Occluded parts can be estimated by
3D geometry with body modeling
Occlusion Estimation by 3D Body
Modeling
Yosen Chen, Copyright Reserved 11
Ground plane
Camera image
B
AAB
Camera visible parts
Camera invisible parts
Body Occlusion Handling
Yosen Chen, Copyright Reserved 12
Depth order
check
Body occlusion
handling
Target info
storage
Body appearance is available for
color extraction
Body appearance is not available
What central server does?
Compare
observations
across cameras
Local
trackers
Operation flow of central server
Get observation
correspondence
between camerasIf detect new,
broadcast
Return people labels & people heights.Start
Keep updating body info. to server database
People
database
Compare with
database to recognize
/create people
Local trackerCentral server
What central server does: Compare observations across
cameras Geometric-based color comparison
2D position on image plane 3D line in real space
3D standing point in real space 2D feet position on image plane
observation r
Optic center Image plane of
camera j
Ground plane
Optic center
observation q
Image plane of
camerak
3D Geometric Correspondence across
Cameras
Calibrated camera system
Head Point
height
Estimated Standing Location
Yosen Chen, Copyright Reserved 15
What central server does: Classify to different people Use decisions in higher confidence to
determine lower ones.
Observation Correspondence
across Cameras
Object Conf = 3 Object Conf = 2
Matching body color (Block-based body color comparison)
Yosen Chen, Copyright Reserved 17
Different colors mean different labels
Multi-People Correspondence
Yosen Chen, Copyright Reserved 18
1. Multi-people depth order √ 2. body occlusion handling √ 3. Multi-people correspondence √
What central server does: Identify people if re-enter Construct people database once
they got tracked.
Yosen Chen, Copyright Reserved 20
Check height
difference
How to recognize people when they re-enterAre they all from the same person?
Body Color
comparison
by block
Label history
consistency
No, if diff > threshold No, if color unmatched
Yes, they are from
the same person!!
Camera
observations
People model
in database
Future Works…
Extend to more cameras
Challenge of
Communication load & algorithm complexity
Tracking/labeling stability
More challenges:
Outdoors Surveillance (human body detection)
Uncalibrated multi-camera system (Training)
Motive cameras (rotation 3D-2D mapping)
Yosen Chen, Copyright Reserved 21
Thanks for Pf. Li-Chen Fu
Advanced Control Lab, NTU
Intelligent Robot Lab, NTU
Copyright © Yu-Sheng Chen [Yosen]