Image Modeling &

SegmentationAly Farag and Asem Ali

Lecture #4-6

2

ImageShape

Spatial

Interaction

Intensity

Image Segmentation

Image Modeling

Others

INTRODUCTION Random Field

Graphical Models Representation

Bayesian Network

Markov Random Field

Markov System

Markov process

Markov Chain

Hidden Markov Models Tutorials by Andrew W. Moore professor of Robotics and Computer Science at CMU http://www.autonlab.org/tutorials/

“Markov Random Field: Theory and Application” Class notes by Kyomin Jung, Ph.D. from MIT Mathematics department. http://web.kaist.ac.kr/~kyomin/Fall09MRF/

“Markov-Gibbs Random Fields in Image Analysis and Synthesis: A Review” Technical Report by Asem Ali

“A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition”, by LAWRENCE R. RABINER

Bayes, Thomas (1763) An essay towards solving a problem in the doctrine of chances. Philosophical Transactions of the Royal Society of London, 53:370-418

Computing a Joint Probability ( Bayes Rule)

Problem Formulation & Spatial Interaction

a zb p q tx

c y rs

1st order neighborhood system

Neighborhood system in is the set of all neighboring pairsN P

Image has a natural structure in which pixels are arranged in 2D array.

if p2 Nq then q2 Npp62 Np

fp;qgp;q2 P

The neighborhood system satisfies

a) b)

5 4 3 4 54 2 1 2 43 1 p 1 34 2 1 2 45 4 3 4 5

Nr = fs;tg

Np = fa;b;c;qg

Example up to the 5th order

3 x 5 image

Nq = fz;p;y;xg

P =f1;2;:::;ng set of image pixel

Observed Image Labeled ImagefI

labeled image.f = f f1; f 2; : : :; fng

set of gray levels (e.g., =256 in 8-bit gray image) set of labels. # classes (e.g., )

L = f1;2;: :: ;K gG = f0;1;:: :;Q ¡ 1g Q

K K = 2 observed image. I = f I 1; I 2; : : : ; I ng I : P ! Gf : P ! L

set of all labelings (e.g., in this case different labeling sets)LnF 236 set of random variables defined on , and is a

configuration of the field F = fF1;F2; : :: ;Fng P fF

Problem Formulation & Spatial Interaction

Fis a Markov Random Field (MRF) w.r.t if its probability mass function abbreviated by satisfiesP (F = f) P (f)

PositivityMarkov Property

Homogeneity

N

1. P (F = f) > 0 for all f 2 F ,2. P (Fp = f pjFP ¡ f pg = f P ¡ f pg) = P (Fp = f pjFN p = fN p ),3. P (Fp = f pjFN p = f N p ) is thesame for all sites p,

ab ? q

c

Markov property establishes the local model

Problem Formulation & Spatial Interaction

A clique is a set of pixels in which all pairs of pixels are mutual neighbors. “single-site” potentials α“two-site” potentials β1“two-site” potentials β2“two-site” potentials β3“two-site” potentials β4

“three-site” potentials η1“three-site” potentials η2“three-site” potentials η3“three-site” potentials η4“four-site” potentials ξ

GRF provides a global model for an image by specifying the (joint) probability distribution

normalizing constant called the partition function, is a control parameter called temperaturepotential function, clique function, summation over cliques “Gibbs energy”set of all cliques.

ZVcC

P (f) = Z¡ 1 exp(¡ Xc2C

Vc(f )=T);

T

Problem Formulation & Spatial Interaction

To identify the Gibbs distribution: Neighborhood system and clique potential.

Cliq

ue ty

pes

of

2nd o

rder

ne

ighb

orho

od

syst

em

Problem Formulation & Spatial Interaction

p

Pairwise Interaction Model EstimationV(f p;f q)In most of the image processing and computer vision literature, the Gibbs energy has been defined in terms of the “single-site” potentials and the “two-site” potentials. This is called the pairwise interaction models.Auto-Models

Besag’74 formulated the energy function of these models as follows:

the potential function for single-pixel cliques

the potential function for all cliques of size 2

The Derin-Elliott model Auto-Normal Model

Problem Formulation & Spatial Interaction

p

Homogenous isotropic pairwise interaction model

Independent of the location of the pixel p

Anisotropic Potts Models

Different Types of these potential functions

11

MGRF-based Image Synthesis

Implement the algorithms in the lecture notes.

Do problems from the appropriate book.

Familiarity and understanding of mathematics only comes with use.

MGRF-based Image Synthesis The synthesis process consists in finding the configuration in “the set of all configurations” which maximizes the probability P(f), and minimizes the Gibbs energy.

The synthesis process is also called sampling. Iterative sampling algorithm is called Gibbs sampler.

Sampling is the process of generating a realization of a random field, given a model whose parameters have been specified.

2nd order neighborhood system

Pairwise interaction model

The Derin-Elliott model p

14

MGRF-based Image Synthesis

q

15

MGRF-based Image Synthesis

16

MGRF-based Image SynthesisRealizations of Derin-Elliott model generated by algorithm (2) with Niter = 100. All images are binary of size 64 x 64.

The random Input

17

MGRF-based Image SynthesisComments:

The number of raster scans Niter of the image is critical. To generate a fine texture, the algorithm should be stopped at 50 < Niter < 100

The values of the model parameters are critical. Hence, MRF exhibits a phase transition phenomenon

To avoid this phenomenon, Metropolis algorithm preserves the number of pixels at each label constant using the exchange step as explained in algorithm (3).

However, the exchange process in this algorithm violates the positivity condition of MRF

200 400 1000

MGRF-based Image SynthesisComments:

The region of the parameter space leading to phase transition was investigated by Picard. In that work, Picard examined the role of the temperature parameter T in the Gibbs distribution

The pattern is not in “equilibrium” unless its energy has decreased to some level where it has stopped changing.

“Markov-Gibbs Random Fields in Image Analysis and Synthesis: A Review” Technical Report by Asem Ali

“Gibbs Random Fields: Temperature and Parameter Analysis”, Technical Report by Rosalind W. Picard, MIT Media Lab.

“Markov/Gibbs modeling: Texture and Temperature”, Technical Report by Rosalind W. Picard, MIT Media Lab.

“Random field models in image analysis” Journal of Applied statistics, by Dube and Jain

“Markov Random Fields and Images”, by Patrick Perez CWI Quarterly

References

(NOTE: you don’t need to read the whole paper in each case, pick and choose the related sections)