View
213
Download
1
Category
Preview:
Citation preview
Avalanche
Ski-Resort
Snow-Clad Mountain
Moving Vistas: Exploiting Motion for Describing ScenesNitesh Shroff, Pavan Turaga, Rama Chellappa
University of Maryland, College Park
Problem Definition and Motivation
Contributions
Dynamic Attributes
Dynamic Attributesmotion information from a global perspective.
Characterize the unconstrained dynamics of scenes using Chaotic Invariants.
Does not require localization or tracking of scene elements.Unconstrained real world Dynamic Scene dataset.
Dynamic Scene Recognition
Dynamics of scene reveals further information !!
Motion of scene elements improve or deteriorate
classification?
How to expand the scope of scene classification to videos?
What makes it difficult?
Scenes are unconstrained and ‘in-the-wild’ -- Large variation in scale, view, illumination, background
Underlying physics of motion -- too complicated or very little is understood of them.
Ray of hope !!!
Underlying process not entirely random but has deterministic component
Can we characterize motion at a global level ??
Yes using dynamic attributes and chaotic invariants
Modeling Dynamics
Requires No assumptions
Purely from the sequence of observations.
Fundamental notion -- all variables in a influence one another.
Constructs state variables from given time series
Estimate embedding dimension and delay
Chaotic Invariants[2,4]
ClassLDS[3]
(GIST)
Bag of
Words
Mean
(GIST)Dynamics
(Chaos)
Statics+
Dynamics
Toranado 70 10 70 60 90
Waves 40 50 70 80 90
Chaotic Traffic 10 20 30 50 70
Whirlpool 20 30 40 30 40
Total 25 24 40 36 52
[1] A.Oliva and A. Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. IJCV, 2001
[2] M. Perc. The dynamics of human gait. European journal of physics, 26(3):525–534, 2005
[3] G. Doretto, A. Chiuso, Y. Wu, and S. Soatto. Dynamic textures, IJCV, 2003
[4]S. Ali, A. Basharat, and M. Shah. Chaotic Invariants for Human Action Recognition. ICCV, 2007.
References
Degree of Busyness: Amount of activity in the video. Highly busy: Sea-waves or Traffic scene --high degree of detailed motion patterns. Low busyness: Waterfall -- largely unchanging and motion typically in a small portion
Degree of Flow Granularity of the structural elements that undergo motion.
Degree of RegularityDegree of Busyness
Reconstruct the phase space.
Characterize it using invariants
Lyapunov Exponent: Rate of separation of nearby trajectories.
Correlation Integral: Density of phase space.
Correlation Dimension: Change in the density of phase space
Coarse: falling rocks in a landslide . Fine: waves in an ocean
Degree of Regularity of motion of structural elements.Irregular or random motion: chaotic traffic Regular motion: smooth traffic
Algorithmic Layout
GIST [1] for
each frame
Each dimension as time series
Chaotic Invariants
Classification&
Learn Attributes
Unconstrained YouTube videos Large Intra-class variation Available at
http://www.umiacs.umd.edu/users/nshroff/DynamicScene.html
Dynamic Scene Dataset
Recognition Accuracy
Linear Separation using Attributes
18 out of 20 correctly classified
Whirlpool Waves
Busyness
Recommended