View
216
Download
0
Tags:
Embed Size (px)
Citation preview
Optical Flow
10-24-2005
Problem
• Problems in motion estimation– Noise, – color (intensity) smoothness, – lighting (shadowing effects), – occlusion, – abrupt movements, etc
• Approaches:– Block matching,– Generalized block matching,– Optical flow (block-based, Horn-Schunck etc)– Bayesian, etc.
• Applications– Video coding and compression,– Segmentation– Object reconstruction (structure-from-motion)– Detection and tracking, etc.
Motion description• 2D motion:
p = [x(t),y(t)] p’= [x(t+ t0), y(t+t0)]
d(t) = [x(t+ t0)-x(t),y(t+t0)-y(t)]
• 3D motion:
Α = [ Χ1, Υ1, Ζ1 ]Τ Β = [ Χ2, Υ2, Ζ2 ]
Τ
= R + T
• Basic projection models: Orthographic
Perspective
x x(t) d1(t)
y y(t) d2(t)
X
Y
Z
2
2
2
X
Y
Z
1
1
1
YyXx
Z
Yy
Z
Xx
l
ll
l
Optical Flow
• Basic assumptions: – Image is smooth locally– Pixel intensity does not change over time (no lighting changes)
• Normal flow:
• Second order differential equation:
Block-based Optical Flow Estimation
• Optical flow estimation within a block (smoothness assumption): all pixels of the block have the same motion
• Error:
• Motion equation:
and
Horn-Schunck
• We want an optical flow field that satisfies the Optical Flow Equation with the minimum variance between the vectors (smoothness)
Gauss-Seidel
Derivative Estimation with Finite differences
Example 1
Example 2
Example 3: frame reconstruction
Reconstructed I2 (second) frame
Reconstructed I2 (second) frame
Application Examples