Upload
nikhilus85
View
1.227
Download
8
Embed Size (px)
DESCRIPTION
Citation preview
Human Action Recognition Based on Spatio-temporal Features
Nikhil Sawant and Dr. K.K. BiswasDept. of CSE, Indian Institute of Technology, Delhi
Third International Conference on Pattern Recognition and Machine Intelligence (PReMi’09)
Human activity recognitionH
ighe
r res
oluti
on
Longer Time Scale
Courtesy : Y. Ke, Fathi and Mori, Bobick and Davis, Schuldt et al, Leibe et al, Vaswani et al.
Pose Estimation
Event Detection
Action Classification
Tracking
Activity Recognition
Use Action recognition?
• Video surveillance
• Interactive environment
• Video classification & indexing
• Movie search
• Assisted Care
• Sports annotation
Broad outline of our techniqueVideo with
human actions
Motions features
Shape features
Spatio-temporal features
Learning features though AdaBoos
Action Class 1 Action Class 2 Action Class 3 Action Class n
Motion analysis using Lucas –kanade
technique
Shape analysis using Viola-Jones feaures
Combining motion and shape features over finite time interval
……………......
Target Localization
• Possible search space is xyt cube• Action needs to be localized in space and time• Target localization helps reducing search space• Background subtraction• ROI marked
Original Video Silhouette Original Video with ROI marked
Motion estimation
• Make use of optical flows for motion estimation
• Optical flow is the pattern of relative motion between the object/object feature points and the viewer/camera
• We make use of Lucas – Kanade, two frame differential method, it comparatively yields robust and dense optical flows
Noise Reduction
• Noise removal by averaging
• Optical flows with magnitude > C * Omean are ignored,
where C – constant [1.5 - 2], Omean - mean of optical flow within ROI
Organizing optical flows
• Optical flows are aggregated near the motion
• Need for representing optical flow in meaningful way
• Fixed sized grid laid over the ROI
• Magnitude and direction of Optical flows within each box bij is averaged and assigned to its centre cij
• All optical flows have same weight
Organizing optical flows (simple averaging)
• Each optical flow given a weight
• More the distance from the centre cij less is the weight and vice-versa
Organizing optical flows (weighted averaging)
• Optical flows are arranged in structured mannered
• Arranged optical flows are easier to analyze
Organizing optical flows
Shape discriptor
• Shape gives information about the action
• Viola-Jones box features used to get shape features
• Shape information combined with motion information
Spatio-temporal descriptor
TLEN
TSPAN
Spatio-temporal descriptor
• Shape and motion features combined over the span of time to form spatio-temporal features
Learning with Adaboost
• Adaboost is state of art learning algorithm
• Linear decision stumps are used as weak hypothesis
• Weak hypothesis combine to form a strong hypothesis
• Strong hypothesis is weighted sum of weak hypothesis
• Training and testing data is kept mutually exclusive
Results
Results (Weizman dataset)
Thanks You