Upload
lewis-johnston
View
221
Download
0
Tags:
Embed Size (px)
Citation preview
Ch.8Efficient Coding of Visual Scenes byGrouping and Segmentation
Bayesian Brain
Tai Sing Lee and Alan L. Yuille
2008-12-22
Heo, Min-Oh
Contents
Introduction Computational Theories for Scene Segmentation
¨ Weak-membrane model A Computational Algorithm for the Weak-Membrane
Model Generalization of the Weak-Membrane Model
¨ Region competition model¨ Affinity-based model¨ Integration segmentation with shape properties
Biological Evidence¨ Go on to the next speaker…
Introduction
Conjecture ¨ Areas V1 and V2 compute a segmentation
for more compact and parsimonious encoding of images¨ The neural processes are representative of neural mech-
anisms that operate in other areas of the brain for per-forming other higher-level tasks.
Computational Theories for Scene Segmentation Choosing the representation W of the regions
which best fits the image data D based on Mini-mum Description Length (MDL) principle.
Taking logarithm
Encoding cost
Computational Theories for Scene Segmentation Weak-Membrane Model
Data term: Gaussian white noise
Smoothness term: variation on the estimated image intensity is smooth within each region
Penalty term:On the length of the boundaries
d(x,y) : intensity values of images (input image)u(x,y) : unobserved smoothed version of d(x,y)B : set of the boundaries between regionsE(u,B) : encoding cost
Mumford D. and Shah J, Optimal approximations by piecewise smooth functions and associated variational problems. Communications on Pure and Applied Mathematics, 42:577-685, 1989
Computational Theories for Scene Segmentation Reformulation
¨ to deal with boundaries easilyd(x,y) : intensity values of images (input image)u(x,y) : unobserved smoothed version of d(x,y)l(x,y) : line process variables which take on values in [0,1]E(u,l) : encoding cost
Ambrosio L, Tortorelli VM, On the approximations of free discontinuity problems. Preprints di Matermatica, 86, Pisa, Italy: Scuola Normale Superiore, 1990
A Computational Algorithm forthe Weak-Membrane Model
A Computational Algorithm forthe Weak-Membrane Model
Continuation methods¨ At large p, the energy function is con-
vex.¨ As p approaches zero, the energy func-
tion transform back to the original func-tion which can have many local minima.
¨ Strategy (successive gradual relaxation) Initialize p0 with large value, perform steep-
est descent . And decrease p to p1, do it again. Repeat the process. Empirically it yields good results
A Computational Algorithm forthe Weak-Membrane Model The steepest descent equations
¨ The system relaxes to an equilibrium as p decrease from 1 to 0
¨ As p decrease, the boundary responses contract spa-tially to the exact location
Ru: positive rate constant w.r.t uRl : positive rate constant w.r.t l
A Computational Algorithm forthe Weak-Membrane Model
A Computational Algorithm forthe Weak-Membrane Model Segmentation of an image by the weak-membrane model
Generalization of the Weak-Membrane Model Natural Images have …
¨ Texture¨ Shade¨ Color ¨ Shape¨ Material Properties¨ Lighting conditions
How can we segment images with these properties?¨ Region competition model¨ Affinity-based model¨ Integrate segmentation with the estimation of 3D shape properties
Generalization of the Weak-Membrane Model Region competition models
Rr: Region sets for each modelar : model type index variable θr : the parameters of the model
Tu Z, Zhu SC, Image segmentation by data-driven Markov chain Monte Carlo. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 24(5), 2002
Generalization of the Weak-Membrane Model Regions can be encoded as one of these types
¨ Gaussian model of the intensity in the region¨ Shading model
the image intensity follows a simple parameterized form
¨ Simple texture/clutter model
Generalization of the Weak-Membrane Model
Affinity-based Model¨ Affinity weights wij between different image pixels vi and vj
¨ Define a graph with image pixels and the weights. Assigning a label to each image pixel so that pixels with the same
labels define a region
n: the number of pixels k : the number of labels :
Yu SX, Shi J, Multiclass Spectral Clustering. Proc. of the 9th International Conference on Computer Vision, 313-319, 2003
Generalization of the Weak-Membrane Model Integration segmentation with shape properties
¨ Additional constraint on the surface normal Occlusion border
Ω : a subregion of the imaged(x, y) : the intensity of the image at location (x, y) : reflectance function based on standard Lambertian model is surface gradient at position (x, y) is the light source
Generalization of the Weak-Membrane Model Example: Surface interpolation process
¨ (a) input image¨ (b) initial estimate of
surface by needle map¨ (c) rendering with (b)¨ (d) final estimate of
surface orientations¨ (e) shaded rendering
Biological Evidence
Let me introduce the next speaker!