Nalin Pradeep Senthamil Masters Student, ECE Dept.

Advisor,Dr Stan Birchfield

Committee Members,Dr Adam Hoover, Dr Brian Dean

Accurate Tracking of Non-Rigid Objects using Level Sets

Clemson University, Clemson, SC USA Accepted in ICCV, 2009

Outline

Tracking Overview Literature Proposed Approach

Object Fragmentation Region Growing Mechanism GMM modeling (feature-spatial)

Level Set Framework Fragment Motion using Joint-KLT

Results Conclusion

Tracking Overview

Idea: Obtain Trajectories over time to locate object

Three Main Categories Point Tracking – Kalman, Particle filters Kernel Tracking – Collins et al (linear RGB), Comaniciu

(Mean-Shift) Contour Tracking – Shah et al, Cremers et al

Applied to Surveillance – Vessel, human, vehicle etc

Why not internet videos ? – 65,000 videos get uploaded in YouTube everyday (rich market)

Literature

Linear RGB [Collins et al. 2003] Ada-boost classifier [Avidan 2005] Fragments based fixed size [Adam et al. 2006] Key-point Feature learning [Grabner et al. 2007] Shape priors [Cremers et al. 2006] Contour tracking using texture [Shah et al 2005]

Limitations

Ignore secondary cues such as multimodality Lack in determining accurate object shape Usually non-contour based techniques drift during occlusion Often ignore spatial arrangement of pixels

Algorithm Block Diagram

Object Fragmentation

Object Modeling

Strength Map Computation

Level-Set Formulation

Estimate Fragment Motion

Tracker InitializationUser clicked ROI around object

Each object as set of fragments

Update made at each frame

Object Fragmentation

Region Growing Mechanism

Random pixel selected from mask – fragment (f) Neighboring pixels added to (f) within Γ (std deviation) Gaussian Model of (f) updated Each (f) represents a Gaussian ellipsoid Both Object and background are fragmented

Object Modeling (GMM)

Joint feature-spatial space,

Strength Map

( )( ) log

( )fg

bg

p xS x

p x

+ve for FGND-ve for BKGND

Level Set Framework

Level Set is numerical technique for fitting contour Level Set on 2D image is viewed as 3D function

Contour in level set identified at zero level

0 is FGND, 0 is BKGND

Level Set for strength map

In general, Level set evolution defined by

Gradient Descent Iteration

Strength Image

Contour (zero level set)

Strength Image Divergence operator

speedcontour

Level-Set Evolution

Iterations using “Elmo” strength map

Curve can grow inward and outward

Figure shows for first frame as example

Curve evolves from previous contours in subsequent tracking

Joint-KLT: Combines algorithms of KLT and HS

Hence,

Used to align coordinate system of object and model fragments

Increases accuracy of strength map

Fragment Motion

data term smoothness term

Fragment Motion (contd.)

‘N’ features tracked in each fragment are averaged

Motion of each fragment gives ‘prior’ information before computing strength map

Drastic motion can be addressed

KLT Joint-KLT

Results - Videos

Shape Matching Hausdorff metric is mathematical measure to

compare two sets of points

Application in Occlusion Handling and Shape recognition

‘a’ and ‘b’ are two point sets

Occlusion Handling

Rate of decrease in object size determines occlusion

Contour shapes learnt online is used to hallucinate during occlusion

Best shape is identified using Hausdorff distance metric

Previously learnt subsequent shapes are hallucinated during occlusion

Results – Occlusion Videos

Results – More Comparison Videos

Quantitative Comparison

Average Normalized error obtained against ground-truth of sequences at every 5 frames.

Girl Circle

Walk BehindElmo Doll

Conclusion

Tracking algorithm based on modeling object and background with mixture of Gaussians

Simple and efficient region growing mechanism to achieve fast computation

Embedding “strength map” into Level-Set Framework

Joint KLT introduced in the framework to improve accuracy

Future Work: Robust shape prior learning and matching Self-occlusion handling for unknown fragments

Alternative Tracking Framework (outline) Overview Proposed Approach

Vessel Detection Saliency Map Thresholding

Vessel Tracking Strength Map using Linear RGB ML Framework for Search

Results

Object Detection Using Saliency Map Saliency: Property of objects

standing out relative to their neighbors.

There is a statistical relationship between backgrounds of all natural images similar to pre-attentive search done by human visual system.

Zhang et al (CVPR 2007) observed redundancies in log Fourier spectra of natural images. Hence, any statistical singularities in the spectrum can be treated as anomalies.

Saliency Map Computation

Algorithm

Let be the image.

Real part of Fourier Spectrum

Phase

Log Spectrum

Spectral Residual

Saliency Map , j=sqrt(-1)

I x

A f F I

Smoothing in spatial domain

Smoothing in frequency domain

P f F I

L f log(A(f ))

R f L f – h f L fn

21S(x) g(x) F exp R f j * P f

Sample Saliency Map detections

Object Tracking

Objects detected through saliency used as FGND

Immediate surrounding used as BKGND

Strength Model Computed similar to Collins Linear RGB

49 features selected from linear combination used to identify strength map

Maximum Likelihood Framework based search used to localize objects in each frame

Region search was identified based on object velocity

Object Tracking – Strength Model 49 features of RGB are normalized into 0-255 and

discretized into 0-32 histogram bins For each feature,

Variance Ratio of Log-likelihood is identified that best discriminates object from background

probability Small value – 0.01hist-index

Variance of L(i) with respect to a distribution a(i)

Strength Model - Outputs

Object Tracking – ML Framework Objective was to recover tight bound around object ML Framework is like EM algorithm Search objective is to maximize the function (Mean,

Covariance)

mean CovarianceStrength Map

Prevent pixel locations farther from object

Object Tracking – ML Framework To maximize the function, Mean and Covariance

are computed iteratively E-Step

M-Step

Iterated for 2-3 times to get optimal values

Mean and covariance of current estimate

Conclusion Algorithm was real time and supported around 25-30 fps in

speed Saliency map based detection was introduced Concept of “strength map” from adaptive-fragmentation is

applied here Depends only on color (linearRGB), and combination with KLT

features would add robustness to the system. Good way to combine is explored.

Thank you !