Upload
elliot
View
74
Download
0
Tags:
Embed Size (px)
DESCRIPTION
Motion Estimation using Markov Random Fields. Hrvoje Bogunović Image Processing Group Faculty of Electrical Engineering and Computing University of Zagreb Summer School on Image Processing, Graz 2004. Overview. Introduction Optical flow M arkov Random Fields OF+MRF combined - PowerPoint PPT Presentation
Citation preview
Motion Estimation using Markov Random Fields
Hrvoje Bogunović
Image Processing Group
Faculty of Electrical Engineering and Computing
University of Zagreb
Summer School on Image Processing, Graz 2004
Overview
• Introduction
• Optical flow
• Markov Random Fields
• OF+MRF combined
• Energy minimization techniques
• Results
Introduction
• Input:– Sequence of images (Video)
• Problem– Extract information about motion
• Applications– Detection, Segmentation, Tracking, Coding
Spatio-temporal spectrum
φ
f
Motion – aliasing
φ
f
1/x
1/t
Large area flicker
Loss of spatialresolution
Large motions - temporal aliasing
φ
fTemporal aliasing
Great loss of spatial resolution
Temporal anti-aliasing
φ
f
• No more overlaping on the f axis. • filtering (anit-aliasing) is performed after sampling, hence the blurring
Motion – eye tracking
φ
f
Motion estimation
• Images are 2-D projections of the 3-D world.
• Problem is represented as a labeling one.– Assign vector to pixel
• Vector field field of movement• Low level vision
– No interpretation
Example Ideal
Problems
• Problem is inherently ill-posed– Solution is not unique
• Aperture problem– Specific to local methods
Optical flow
• Main assumption: Intensity of the object does not change as it moves– Often violated
• First solved by Horn & Schunk– Gradient approach
• Other approaches include– Frequency based– Using corresponding features
Image differencing
Gradient approach
• Local by nature. Aperture problem is significant.
• Image understanding is not required– Very low level
Horn & Schunk
• Intensity stays the same in the direction of movement. I(x,y,t)
• After derivation
Horn & Schunk
• Spatial gradients Ix,Iy
– e.g. Sobel operator
• Temporal gradient It
– Image subtraction
( , ) ( , ) 0x y t
t
I I u v I
I I v
Regularization
• Tikhonov regularization for ill-posed problems
• Add the smoothness term
• Energy function
Result
Problems of the H-S method
• Assumption: There are no discontinuities in the image– Optical flow is over-smoothed.
• Gradient method. Only the edges which are perpendicular to motion vector contribute
• Image regions which are uniform do not contribute.
• Difficulty with large motions (spatial filtering)
Optical flow enhancement
• Optical flow can be piecewise smooth
• Discontinuities can be incorporated
• Solution: use the spatial context
• Problem is posed as a solution of the Bayes classifier. Solution in optimization sense. Search for optimum
Bayes classifier
• Main equation
• Solution using MAP estimation
( , ) ( )( | )
( )
P hypothesis observation P hypothesisP hypothesis observation
P observation
Markov Random Fields
• Suitable: Problems posed as a visual labeling problemn with contextual constraints
• Useful to encode a priori knowledge– required for bayes classifier (smoothness prior)– equvalence to Gibbs random fields (gibbs
distribution, exponential like)• Neighbourhoods• Cliques
– pairs,triples of neighbourhood points)– build the energy function
MRF
• Define sites: rectangular lattice
• Define labels
• define neighbourhood: 4,8 point
• Field is MRF:– P(f)>0
– P(fi|f{S-i})=P(fi|Ni)
Coupled MRF
• Field F is an optical flow field• Field L is a field of discontinuities
– line process
• Position of the two fields.
Context
• neighbourhoods and cliques
Motion estimation equations
Energy for MAP estimation
Parameters are estimated ad hoc
Energy minimization
• Global minimum– Simulated annealing– Genetic Algorithms
• Local minimum– Iterated Conditional Modes (ICM) (steepest
decent)– Highest Confidence First (HCF)
• specific site visiting
Simulated annealing(1) Find the initial temperature of the system T.
(2) Assign initial values of the field to random
(3) For every pixel:
Assign random value to f(i,j)
Calculate the difference in energy before and after If the change is better (diff>0) keep it.
Else keep it with the probability exp(diff/T)
(4) Repeat (3) N1 times
(5) T = f(T) where f decreases monotono
(6) Repeat (3-5) N2 times
Results (Square)
Horn-Schunk OF OF+MRF
Taxi
Results (Taxi)
Line process result (Taxi)
Cube
Results (cube)
Line process result (cube)
Q & A