View
214
Download
0
Category
Tags:
Preview:
Citation preview
1
Visual Object Tracking
Alireza Asvadi
University of Coimbra
May 2014
Video-Vigilance and Biometrics
A review on:
2
Definition and application
“Object tracking” from different points of view
Yilmaz et al. 06
Yang et al. 11
David Forsyth & Jean Ponce 12
Outline:
3
Definition and application
4
What is object tracking?
Estimating the trajectory of an object over time by locating its position in every frame.
5
motion-based recognition
automated surveillance
video indexing
human-computer interaction
traffic monitoring
vehicle navigation
…
Applications:
6
Object tracking from different points of view
Yilmaz et al. 06
Yang et al. 11
David Forsyth & Jean Ponce 12
7
How to categorize? There are different points of view
Object tracking
Yilmaz et al. 06
Yang et al. 11
David Forsyth & Jean Ponce 12
8
Object Tracking: A Survey
Object detection
Object tracking
Point detectors
Background subtraction
Segmentation
Supervised learning
Point tracking
Kernel tracking
Silhouette tracking
Object representation
Feature selection for tracking
Yilmaz et al. 06
9
Object representation:
Points, primitive geometric shapes, object silhouette and contour, articulated shape models and skeletal models.
Object representations are chosen according to the application domain.
10
Feature selection for tracking:
Desirable property of a visual feature is its uniqueness so that the objects can be easily distinguished in the feature space.
many tracking algorithms use a combination of these features.
Color: RGB, L*u*v, L*a*b, HSV
Edges: less sensitive to illumination changes
Optical flow: displacement vectorsBrightness constraint [Horn & Schunk 81]
Texture: measure of the intensity variation of a surface
11
Object detection:
Point detectors: are used to find interest points in images which have an expressive texture in their respective localities (ex. Harris & SIFT).A desirable interest point: invariance to illumination and camera viewpoint
Background subtraction: Object detection can be achieved by building a representation of the scene called the background model and then finding deviations from the model for each incoming frame.
Segmentation: The aim of image segmentation algorithms is to partition the image into perceptually similar regions and can be used for object detection(ex. Mean shift clustering & graph cut).
Supervised learning: Object detection can be performed by learning different object views automatically from a set of examples by means of a supervised learning mechanism (ex. SVM).
Every tracking method requires an object detection mechanism either in every frame or when the object first appears in the video.
12
Taxonomy of tracking methods:
13
Point tracking: Objects detected in consecutive frames are represented by points.
Deterministic Methods: Proximity, maximum velocity (r denotes radius), small velocity-change, common motion, rigidity constraints
Statistical Methods: Kalman filter, particle filter
14
Kernel tracking: Kernel refers to the object shape and appearance. For example, the kernel can be a rectangular template or an elliptical shape with an associated histogram.
Template and density based appearance models: Ex. Template matching & mean shift tracking. Advantage of the mean shift is the elimination of a brute force search.
Multiview appearance models: The objects may appear different from different views. Different views of the object can be learned offline and used for tracking.
15
Silhouette tracking: Silhouette-based object tracker try to find the object region in each frame by means of an object model generated using the previous frames.
Shape matching: Object model is in the form of an edge map. Shape matching performed similar to tracking based on template matching.
Contour tracking: Iteratively evolve an initial contour in the previous frame to its new position in the current frame. This contour evolution requires that some part of the object in the current frame overlap with the object region in the previous frame.
16
Summary:
Point tracking: Good for finding Geometrical and 3D structure of object.Point correspondence is a complicated problem-specially in the presence of occlusions, misdetections, entries, and exits of objects.
Kernel tracking: Real time applicability.One of the limitations of primitive geometric shapes for object representation is thatparts of the objects may be left outside of the defined shape while parts of the background may reside inside it.
Silhouette tracking: Good for modeling object with complex shape.Sensitive to noise. Not capable to deal with object split and merge.
17
Recent advances and trends in visual tracking: A review
Feature descriptors for visual tracking
Online learning based tracking methods
Yang et al. 11
Generative methods
Discriminative methods
18
Feature descriptors for visual tracking:
Gradient features: statistical summarization of the gradients
Color features:
Texture features:
Spatio-temporal features:
Multiple features fusion: HOG-LBP, …
HOGmulti-resolution HOGSIFTSURF…
CSIFT (concatenation of the hue histogram with the SIFT descriptor. Invariant to light intensity)
LBP (Local Binary Patterns) a grayscale invariant textureMB-LBP (multi-scale block LBP)…
HOG/HOF (Histograms of oriented gradients & optic flow)HOG3D (histograms of 3D gradient orientations)ESURF (extends the image SURF descriptor to video)DLBP (dynamical Local Binary Patterns)…
19
Online learning based tracking methods:
The generative method builds a model to describe the appearance of an object and then finds the object by searching for the region most similar to the reference model in each frame.The object model is often updated online to adapt to appearance changes.generative methods would easily fail within cluttered background.
Generative online learning methods:
Discriminative methods pose object tracking as a binary classification problem in which the task is to determine a decision boundary that distinguishes the object from the background without the need to a complex model characterizing the object.To handle appearance changes, the classifier is updated incrementally over time.A major shortcoming of discriminative methods is their noise sensitivity.
Discriminative online learning methods:
There are methods which combine these two methods.
20
Object tracking
Computer Vision A Modern Approach 2nd Edition,Ch 11, TrackingForsyth et al. 12
Tracking by detection
Tracking with dynamics
Applying Data association
21
Time t Time t+1 Time t+n
Tracking by detection:
we have a strong model of the object, we detect the object independently in each frame and can record its position over time.
Occlusions
Time
Similar Objects
Problems:
22
Tracking with dynamics:
Key idea: Given a model of expected motion, predict where objects will occur in next frame.Filtering Problem: Estimate of c based on prediction a and measurement b
Observation (Detected object) + Dynamics
The information from the predictions and measurements are combined to provide the best possible estimate of the location of the train.
the product of two Gaussian functions is another Gaussian function.
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
The Kalman filter:
a bc
23
Problems: Yet it is not good enough
So far, we’ve assumed the entire measurement to be relevant to determining the state. In reality, there may be uninformative measurements may belong to different tracked objects.
Data association: Task of determining which measurements go with which tracks.
Tracking Matching: Match should be close to predicted position
Gating: Omit MeasurementsOutside the gate
24
Tracking: Detection(observation)+ dynamics +Data association
Applying Data association (Gating) Predicted Values By Kalman Filter (Green)
NCC
NCC+Kalman
NCC+Kalman+DA
25
Tracking by detectiondetect the object independently in each frame
tracking=Detection Detection Methods:
Tracking with dynamicsincorporate object dynamics to tracking Methods:tracking=Detection(observation)+dynamics
Applying Data associationEliminate highly unlikely measurementstracking=Detection(observation)+ dynamics +Data association
Summary:
Point detectorsTemplate matchingdensity Based appearance modelsBackground Subtraction…
Filtering MethodsKalman filter…
Methods:Tracking MatchingGating…
26
Reference:
A. Yilmaz, O. Javed, and M. Shah, “Object tracking: A survey,” ACM Computing Surveys, Vol. 38, No. 4, pp. 1–45, December 2006.
H. Yang, L. Shao, F. Zheng, L. Wang, and Z. Song, “Recent advances and trends in visual tracking: A review,” Neurocomputing, Vol. 74, No. 18, pp. 3823-3831.
D. A. Forsyth, J. Ponce, “Computer Vision: A Modern Approach,” Prentice Hall,2nd Edition, 2012.
R. Faragher , “Understanding the Basis of the Kalman Filter Via a Simple and Intuitive Derivation,” IEEE Signal Processing Magazine, September 2012.
27
Thank you for your attention
Recommended