Upload
emory-porter
View
219
Download
0
Tags:
Embed Size (px)
Citation preview
Real-time Human Tracking by Detection based on HOG and Particle Filter
1
REAL-TIME HUMAN TRACKING BY
DETECTION BASED ON HOG AND
PARTICLE FILTER
Jiu XU, Axel BEAUGENDRE and Satoshi GOTOComputer Sciences and Convergence Information
Technology (ICCIT), 2011 6th International Conference on
Real-time Human Tracking by Detection based on HOG and Particle Filter
2
Outline Introduction Proposed Method
Human DetectionMoving object feature, Hierarchical search algorithm
Human TrackingColor-EdgeTexture histogram, Occlusion Handling
Strategy
Experimental Results Conclusion
Real-time Human Tracking by Detection based on HOG and Particle Filter
3
Introduction The real-time foreground objects
tracking and detection is the most critical and fundamental step in video surveillance systems.
Different from vehicles, it is much more difficult to locate and track the human body out of the background.
Real-time Human Tracking by Detection based on HOG and Particle Filter
4
Introduction
Pedestrian detection by Histogram of Oriented Gradient (HOG) put forward by Dalal [1] [2].
The main idea of this method is to use gradient direction histogram of small pieces to describe the image.
[1] N. Dalal, "Finding People in Images and Videos," PhD thesis, Institut National Polytechnique de Grenoble, 2006.[2] N. Dalal and B. Triggs, "Histogram of oriented gradient for human detection," in CVPR, 2005.
Detection
Real-time Human Tracking by Detection based on HOG and Particle Filter
5
Introduction Feature extraction by adjusting the
parameters, the HOG feature can effectively describe the body’s shape information.
Moreover, it also has the ability of invariance when the small local area occurs dithering and rotation.
Real-time Human Tracking by Detection based on HOG and Particle Filter
6
Introduction However, the computational complexity
of feature extraction makes the method very slow, and difficult to meet the needs of the practical application of the system.
Real-time Human Tracking by Detection based on HOG and Particle Filter
7
Introduction
Some common methods perform tracking by pre-initialized trackers based on Kalman filter [6] or particle filter [7].
Tracking
[6] Gutman, P., Velger, M. “Tracking Targets Using Adaptive Kalman Filtering”, IEEE Transactions on Aerospace and Electronic Systems Vol. 26, No. 5: pp. 691-699 1990.[7] B.Ristic, “Beyond the Kalman Filter: Particle Filters for Tracking Applications”. Arthech House, 2004.
Real-time Human Tracking by Detection based on HOG and Particle Filter
8
Introduction Particle filter is based on the Bayes
principle and is a sequential Monte-Carlo simulation method indicated by probability density of particles.
Real-time Human Tracking by Detection based on HOG and Particle Filter
9
Introduction one of the intractable problems is that
the target is usually occluded by other objects frequently and instantaneously.
To select powerful features to avoid hijacking problems when tracking similar objects.
If several humans walk together as a group, we cannot separate them individually.
Real-time Human Tracking by Detection based on HOG and Particle Filter
10
Introduction The target of this paper is to build a
powerful tracking system for real-time surveillance system.
Combining human detection and tracking and do tracking by detection in order to achieve high accuracy and low time consumption together with occlusion solutions and group segmentations.
Real-time Human Tracking by Detection based on HOG and Particle Filter
11
Outline Introduction Proposed Method
Human DetectionMoving object feature, Hierarchical search algorithm
Human TrackingColor-EdgeTexture histogram, Occlusion Handling
Strategy
Experimental Results Conclusion
Real-time Human Tracking by Detection based on HOG and Particle Filter
12
Proposed Method
a Tracking by Detection System using human detection together with human tracking
Real-time Human Tracking by Detection based on HOG and Particle Filter
13
Human Detection Histogram of Oriented Gradient (HOG)
is quite a popular method of detecting people in static image.
Real-time Human Tracking by Detection based on HOG and Particle Filter
14
Human Detection The image is first divided into blocks
while these blocks overlap with each other. Each block contains four cells.
Real-time Human Tracking by Detection based on HOG and Particle Filter
15
Human Detection For each pixel I(x,y), the
orientation Θ(x,y) and the magnitude m (x, y) of the gradient
Real-time Human Tracking by Detection based on HOG and Particle Filter
16
Human Detection A histogram is calculated for each cell,
and the volume of each bin is the sum of magnitude of the pixels whose orientations are in the corresponding angle interval.
Real-time Human Tracking by Detection based on HOG and Particle Filter
17
Human Detection However, HOG can’t be used into real-
time system due to its high time consumption.
Improvements Moving object feature Hierarchical search algorithm
Real-time Human Tracking by Detection based on HOG and Particle Filter
18
Human Detection Moving object feature Kim [10] presented a new kind of non-
parametric algorithm for background subtraction.
For each pixel, it builds a codebook consisting of one or more codewords.
[10] K. Kim, T. H. Chalidabhonse, D. Harwood, and L. Davis. “Real-time foreground-background segmentation using codebook model”. Elsevier Real-Time Imaging, vol. 11, no.3, 167–256, June 2005.
Real-time Human Tracking by Detection based on HOG and Particle Filter
19
Human Detection In our previous work [13], a block-based
codebook method has been proposed to make it more adoptable for human tracking.
The block feature heighten the pixel-based method to block level, thus we could take advantage of the relationship between neighboring pixels.
[13] Jiu Xu, Ning Jiang, Satoshi Goto, “Block-based Codebook Model with Oriented-Gradient Feature for Real-time Foreground Detection”, IEEE 13th International Workshop on Multimedia Signal Processing(MMSP), 2011.
Real-time Human Tracking by Detection based on HOG and Particle Filter
20
Human Detection
morphological operation(opening, closing)
Real-time Human Tracking by Detection based on HOG and Particle Filter
21
Human Detection Hierarchical search algorithm The expected target might appear at
any position and the size would also keep changing
Real-time Human Tracking by Detection based on HOG and Particle Filter
22
Human Detection From our experiments, we find that the
curve of the number of detected pedestrians under various scaling satisfied the curve of normal distribution probability density function.
We could estimate the minimum interval [a, b].
Image scale levels only from a to b should be detected, reduce the searching levels and the amount of computation as well.
Real-time Human Tracking by Detection based on HOG and Particle Filter
23
Human Tracking
Real-time Human Tracking by Detection based on HOG and Particle Filter
24
Human Tracking Particle tracker initialization
Real-time Human Tracking by Detection based on HOG and Particle Filter
25
Transition model p()- how objects move between frames. Observation model p()- specifies the likelihood of an object being in a specific state. Initial state p()- describes initial distribution
Human Tracking
Real-time Human Tracking by Detection based on HOG and Particle Filter
26
Human Tracking Color-EdgeTexture histogram We proposed a Color-EdgeTexture
Histogram to generate the weight for the observation models.
We choose the HSV color. The brightness variations while in HSV color space we could better separate the brightness with others.HSV = Hue, Saturation, Value
Real-time Human Tracking by Detection based on HOG and Particle Filter
27
Human Tracking Moreover, we add the edge local binary
pattern to describe the shape texture of the moving objects.
Local binary pattern (LBP) [14] is an effective texture description operator, which could be used to measure and extract texture information from the local neighborhood in a gray image.
[14] T. Ojala, M. Pietikainen, and D. Harwood, “A Comparative Study of Texture Measures with Classification Based on Feature Distributions”, Pattern Recognition, vol. 29, pp. 51-59. 1996
Real-time Human Tracking by Detection based on HOG and Particle Filter
28
Human Tracking Local binary pattern (LBP)Consider a pixel (, )
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking LBP value together with H-S color
information improve the performance Two problems :1. time consumption. Since LBP is a pixel-
wise coding, if we calculate all the LBP value inside the whole regions of predicted position of the particles, the calculation is really quite huge, thus it will extremely decrease the real-time ability.
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking2. the LBP value for background part of
the regions within the trackers are also calculated, it will greatly reduce the tracking rate when the size of the object is changing, and the portion of the background becomes larger and larger since the weight of the background is increased.
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking To solve these problems, we use a
concept of edge LBP and the edge LBP only focuses on the edge points of the foreground objects.
canny edge detector
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking we use a kind of H-S-
ForegroundEdgeLBP histogram in which the size is 8x8x8 for each component.
H-S-LBP histogram
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking Particles are weighted according to the
similarity between the target histogram distribution q(u) and the histogram distributions p(u) given by particles.
Real-time Human Tracking by Detection based on HOG and Particle Filter
Human Tracking the weight of the i-th particle is defined
as
color histogram of the target at time k histogram of the i-th particle
Real-time Human Tracking by Detection based on HOG and Particle Filter
35
Human Tracking Occlusion Handling Strategy In traditional particle filter tracking, if
the object meets some partial or total occlusions, the observation model will turn to the occluder and will not track the previous objects any longer.
Real-time Human Tracking by Detection based on HOG and Particle Filter
36
Human Tracking After color-edgetextue histogram, we
define a threshold to this weight.
① If the tracker moves out of the margin of the frame
② If the tracker is still inside the frame and the max weight is great than the threshold
③ If the tracker is still inside the frame and the max weight is less than the threshold
Delete
Update
Keep increase the number of the particles, searching range
Real-time Human Tracking by Detection based on HOG and Particle Filter
37
Outline Introduction Proposed Method
Human DetectionMoving object feature, Hierarchical search algorithm
Human TrackingColor-EdgeTexture histogram, Occlusion Handling
Strategy
Experimental Results Conclusion
Real-time Human Tracking by Detection based on HOG and Particle Filter
38
Experimental Results
Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter
39
Experimental Results
Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter
40
Experimental Results
Other method
Real-time Human Tracking by Detection based on HOG and Particle Filter
41
Experimental Results
Separate group
Real-time Human Tracking by Detection based on HOG and Particle Filter
42
Experimental Results
The time consumption of our method is much less than [8][9][11].
(400 frames long)
Real-time Human Tracking by Detection based on HOG and Particle Filter
43
[8] L.M.Fuentes and S.A.Velastin, “People tracking in surveillance applications”, Image and Vision Computing, pp.1165-1171, 2006
[9] Tao Yang, Quan Pan, Jing Li and Li, S.Z. “Real-time Multiple Objects Tracking with Occlusion Handling in Dynamic Scene” In CVPR, Vol.1. pp. 970-975, 2005
[11] R. Hess and A. Fern, “Discriminatively Trained Particle Filters for Complex Multi-Object Tracking”. In CVPR, 2009.
Experimental Results
Real-time Human Tracking by Detection based on HOG and Particle Filter
44
Uses a combination strategy of HOG human detection method and particle filter human tracking algorithm in surveillance system.
Original HOG feature is not be so efficient in real-time system
In order to enhance the performance in color-based particle tracking
The proposed method has a good robustness in all kinds of situations together with low time consumption.
Conclusion
Real-time Human Tracking by Detection based on HOG and Particle Filter
45
Thanks for listening !!