Upload
fala
View
46
Download
3
Tags:
Embed Size (px)
DESCRIPTION
Rule-based Action Recognition using Object Trajectories. UCF VIRAT Efforts. Recognition Process. Input: Time ordered series of 2 dimensional locations of object centroid in image coordinates (trajectory) Output: Action(s) pertaining to the given trajectory Method: - PowerPoint PPT Presentation
Citation preview
Rule-based Action Recognition using Object Trajectories
UCF VIRAT Efforts
Recognition Process Input: • Time ordered series of 2 dimensional locations
of object centroid in image coordinates (trajectory)
Output: • Action(s) pertaining to the given trajectory
Method:• Compute multiple discriminative features for
each trajectory• Classify action(s) using a comprehensive set of
rules
Trajectory Features Dynamics based
• For a curve,r (t) = {(x0,y0),…,(xt,yt)}
• Instantaneous speed, v = || dr / dt ||
• Average speed,vav = ∑n vi
/ n• Acceleration,
a = || dv / dt ||• Arc length,
s = ∫ v dt
Trajectory Features Shape based• Capture geometrical information
• Tangent vector, v = dr / dt
• Unit Tangent vector, T = v / ||v||• Curvature, k(t) = || dT / ds || = ||T’ / v||• Four point cross ratio,
Cr (p1,p2,p3,p4) = (p3-p1)(p4-p2) / (p4-p1)(p3-p2)
Rules Rules are based on quantization of feature values
A decrease followed by an increase in unit tangent vector = Right turn, or vice versa
Two consecutive increase or decrease = U-turn
Rules
Going Straight Turn Right U-turn
Trajectory
Trajectory
Trajectory
dT / dt dT / dt dT / dt
Rules Accelerate / Decelerate events are detected directly from features
Deceleration to zero speed = Stopping
Maintaining speed and direction = Maintain distance
….
Results Trajectories extracted from the UCF Aerial Actions dataset
Handle walking, running, turning left and right, and taking U-turn
Results
Walking Forward
Results
Turn Right
Results
U-turn
Results
U-turn
Results
Turn Right
Results
Turn Left
Results
Running Forward
VIRAT Events
Standing Walking Running Digging Gesturing Carry Object
Person Actions Accelerate
Decelerate Turning Stopping U-turn Maintain distance
Vehicle Actions
Ideas & Future Work Object Classification
• Discriminate between similar trajectories of different objects• Separate person, vehicle and facility detectors• Person-Vehicle and Person-Facility events
Multiple Trajectories per object
• Track multiple points on each object• Recognize stationary trajectories using object
kinematics• Gesturing, Digging• Can also help or eliminate object detector /
classifier
Ideas & Future Work Geo-Registration
• Distance, speed, and acceleration in image plane may not be useful
• Allow use of computed distances on ground• Do not have to be absolute longitude and latitude• Accelerate, Decelerate, Maintain Distance
Invariant features
• Trajectory features should be invariant to: Changes in view Changes in scale
• Projective invariant features can eliminate need for geo-registration
Ideas & Future Work Learning based framework
• Learning and clustering of discriminative features• More robust compared to rule based methods• Representative of training trajectory samples• Associated confidence for each recognition• Recognition of unseen and composite events
Quality of Input Tracks
• Availability of object trajectories is assumed• Features are highly dependant on tracking accuracy• Broken, merged and split tracks severely affect
performance
VIRAT Events
Standing Walking Running Digging Gesturing Carry Object
Person Actions Accelerate
Decelerate Turning Stopping U-turn Maintain distance
Vehicle Actions Loading Unload Open trunk Close trunk Getting into car Getting out of car Enter/exit building
Multi-agent Actions
• With provision of object recognition / classification, multiple trajectories and geo-registration
Thank You!