View
236
Download
1
Category
Preview:
Citation preview
Spin models for image processing
Alexey Abramovabramov _at_ physik3.gwdg.de
Georg-August University, Bernstein Center for Computational Neuroscience,
III Physikalisches Institut, Göttingen, Germany
Lecture
Alexey Abramov (BCCN, Göttingen) 10-03-11 2/33
It’s a data structure representing a generally rectangular grid of pixels, or points
of color, viewable via a monitor, paper, or other display medium.
Pixel is the smallest piece of
information in an image.
(R,G,B) = (120,70,90)
width
height
In zoomed view every square represents a
pixel.
What is raster graphics image (bitmap)?
Alexey Abramov (BCCN, Göttingen)
Partitioning of an input image into disjoint parts based on some image characteristics.
Image segmentation
input image segments
- homogeneity criteria based methods
- clustering
- region-based growing
- graph cuts
- mean shift segmentation
10-03-11 3/33
Alexey Abramov (BCCN, Göttingen)
The methods of superparamagnetic clustering are nonparametric methods
which solve the segmentation problem by finding the equilibrium states of the
energy function of a ferromagnetic Potts model (without data term).
Superparamagnitec clustering of data
Before update After update
Neighborhood
ε2Di
Superparamagnetic phase regions of aligned spins correspond to a natural
partition of the image data (Blatt et al., 1996).
10-03-11 4/33
Alexey Abramov (BCCN, Göttingen)
A spin variable σk , which can take on q discrete values v1, v2, …, vq , called
spin states, is assigned to each pixel of the image. The energy of the system is
described by
Potts model
where the function Jij is a coupling constant, determining the interaction
strength, where gi and gj are the respective color vectors of the pixels:
where Δ is a normalization constant.
10-03-11 5/33
One iteration of Metropolis algorithm
Current configuration S = 1 S = 2 S = 3 S = 4
Assumption for this example: Jij = 1
1. Ecur = -1
2. 4 → 1 Enew = -5
3. 4 → 2 Enew = 0
4. 4 → 3 Enew = -2
min Emin = -5
5. Enew < Ecur therefore 4 → 1 (move is accepted)Proposed configuration
Alexey Abramov (BCCN, Göttingen)
Final configuration
10-03-11 6/33
Metropolis algorithm (suggested by Metropolis, 1953)
Alexey Abramov (BCCN, Göttingen)
Ising model is a statistical model. The probability of each configuration of spins is
the Boltzmann distribution with inverse temperature β:
To generate configurations using the probability distribution is easiest using
the Metropolis algorithm:
1. pick a spin at random and compute its contribution to the energy Ecur
2. flip the value of the spin and calculate the new contribution Enew
3. if Enew < Ecur then the new state is accepted – a good move
4. if Enew > Ecur then the new state is accepted using the probability – a worse move
5. Repeat.
Simulated Annealing
A certain number of iterations are carried out at each temperature and then
the temperature is decreased. This is repeated until the system freezes into a steady
state.10-03-11 7/33
Alexey Abramov (BCCN, Göttingen)
One iteration of parallel Metropolis algorithm
10-03-11 8/33
Implementation of Metropolis on GPU
Alexey Abramov (BCCN, Göttingen)
1. Generate image tiles taking overlapping into account;
2. Construct CUDA data according to memory structure of GPU;
3. Launch kernel function on the GPU;
4. Decompose CUDA data.
1 2 3 4 5 j j+2j+1 j+3 j+4
1
2
3
4
5
i
i+1
i+2
BLOCK (0.0) BLOCK (1.0)
BLOCK (0.0)
BLOCK (0.1)
10-03-11 9/33
Metropolis algorithm (the cooling schedule)
Alexey Abramov (BCCN, Göttingen)
A certain number of iterations are carried out at each temperature and then
the temperature is decreased. This is repeated until the system freezes into a
steady state.
10-03-11 10/33
Alexey Abramov (BCCN, Göttingen)
Parallel Metropolis algorithm (fast cooling)
Domain fragmentation – large uniform areas are being split into sub-segments
despite the high attractive forces within them.
10-03-11 11/33
Configurations after short annealing
Alexey Abramov (BCCN, Göttingen)
“Cluttered scene” “Filling a cup” “Tsukuba”
10-03-11 12/33
Alexey Abramov (BCCN, Göttingen)
Simulated annealing with short-cut
Resolving domain fragmentation
10-03-11 13/33
Resolving Domain Fragmentation
Alexey Abramov (BCCN, Göttingen)
For the spin configuration with q = 6 after 10 Metropolis iterations:
10-03-11 14/33
Detected object boundaries
Alexey Abramov (BCCN, Göttingen)
“Cluttered scene” “Filling a cup” “Tsukuba”
10-03-11 15/33
Labeling of connected components
Alexey Abramov (BCCN, Göttingen)
“Cluttered scene” “Filling a cup” “Tsukuba”
10-03-11 16/33
Fast connected-component labeling
Alexey Abramov (BCCN, Göttingen)
Labeling of connected components in a binary image is one of the most fundamental
operations in pattern recognition: labeling is required whenever a computer needs to
recognize objects (connected components) in a binary image.
By use of the labeling operation, a binary image is transformed into a symbolic image
in which all pixels belonging to a connected component are assigned a unique label.
- Multi-scan algorithms scan an image in the forward and backward raster directions alternately to propagate label equivalences until no label changes.
- Two-scan algorithms complete labeling in two scans: during the first scan, they assign provisional labels to object pixels, and record label equivalences. Label equivalences are resolved during or after the first scan. During the second scan, all equivalent labels are replaced by their representative label.
- Hybrid algorithms. Algorithms that are between muti-scan algorithms and two-scan algorithms.
10-03-11 17/33
Conventional two-raster-scan algorithms
Alexey Abramov (BCCN, Göttingen)
For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image
where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background
pixels. We assume that V0 and Vb are larger than the value of any provisional label,
and V0 < Vb.
Mask for eight-connected connectivity
10-03-11 18/33
Conventional two-raster-scan algorithms
Alexey Abramov (BCCN, Göttingen)
For an N x N binary image, we use b(x,y) to denote the pixel value at (x,y) in the image
where 1 ≤ x,y ≤ N, and V0 for the value of object pixels and Vb for that of background
pixels. We assume that V0 and Vb are larger than the value of any provisional label,
and V0 < Vb.
Provisional labeling by the first scan
10-03-11 19/33
Conventional two-raster-scan algorithms
Alexey Abramov (BCCN, Göttingen)
With the mask for eight-connected connectivity, for an object pixel b(x,y) there are 16
possible cases in the mask.
(2) - (9) and (13) – (16) b(x,y) can take on any existing provisional label in the mask
without the need for consideration of label equivalences.
10-03-11 20/33
Conventional two-raster-scan algorithms
Alexey Abramov (BCCN, Göttingen)
The order of processing:
b(x-1,y-1) → b(x,y-1) → b(x+1,y-1) → b(x-1,y)
b(x,y)
After b(x,y-1) is processed, the provisional labels for b(x-1,y-1) and b(x,y-1) should
have been combined in the same equivalent label set and have the same
representative label.
Because all provisional labels in the mask have the same representative label, b(x,y)
can take on any of the labels without consideration of any label equivalence.
10-03-11 21/33
The Karnaugh map
Alexey Abramov (BCCN, Göttingen)
The Karnaugh map (K-map) is a method to simplify boolean algebra expressions. In
a Karnaugh map only one boolean variable changes in between adjacent squares.
Once the table is generated and the output possibilities are transcribed, the data is
arranged into the largest possible groups containing 2n cells and the minterm is
generated through the axiom laws of boolean algebra.
A
B
D
C
10-03-11 23/33
The Karnaugh map for the operation resolve
Alexey Abramov (BCCN, Göttingen)
C1
C2
C3
C4
The operation resolve takes much
time compared to other operations,
it should be avoided as much as
possible.
10-03-11 24/33
The Karnaugh map for the operation resolve
Alexey Abramov (BCCN, Göttingen)
if (C3 ≠ VB)
b(x,y) = C3;
else if (C1 ≠ VB)
b(x,y) = C1;
if (C4 ≠ VB)
resolve(C4,C1);
else if (C2 ≠ VB)
b(x,y) = C2;
if (C4 ≠ VB) resolve(C2,C4);
else if (C4)
b(x,y) = C4;
else
b(x,y) = m, m = m + 1.
PseudocodeAfter the first scan, the provisional labels that
are assigned to a connected component in the
given image will be combined in an equivalent
label set and hold the same representative
label. Then, by the second scan, each
provisional label is replaced by its
representative label. Thus, after the second
scan, all pixels in a connected component will
be assigned a unique label.
10-03-11 25/33
Possible accelerations on a GPU
Alexey Abramov (BCCN, Göttingen)
Representative labels can be assigned to all object pixels simultaneously. Therefore
the second scan can be implemented on the GPU in a very efficient manner.
10-03-11 26/33
Execution time on CPU
Alexey Abramov (BCCN, Göttingen)
CCL – the contour-tracing connected-component labeling;
SAUF – the scan plus array-based union-find algorithm;
RTS – the run-based two-scan algorithm.
10-03-11 27/33
Execution time on CPU
Alexey Abramov (BCCN, Göttingen)
CCL – the contour-tracing connected-component labeling;
SAUF – the scan plus array-based union-find algorithm;
RTS – the run-based two-scan algorithm.
10-03-11 28/33
Alexey Abramov (BCCN, Göttingen)
Metropolis procedure with proposed short-cut
10-03-11 29/33
Alexey Abramov (BCCN, Göttingen)
Comparison with graph-based method
Original image Proposed method Graph-based 7
7 Felzenszwalb, et al.: Efficient graph-
based image segmentation. (2004)
10-03-11 30/33
Alexey Abramov (BCCN, Göttingen)
Timing performance
Processing time Total computation time of all steps
512 x 640 px 153.1 ms | 24.4 x 103 ms
500.0 ms
Image size GPU CPU
graph-based (CPU)
Proposed method
10-03-11 31/33
Alexey Abramov (BCCN, Göttingen)
Bibliography
Abramov et al., “Real-time image segmentation on a GPU” (2010)
von Ferber et al., “Cluster update algorithm and recognition” (2000)
Potts, “Some generalized order-disorder transformations” (1952)
Metropolis et al., “Equation of state calculations by fast computing machines” (1953)
Blatt et al., “Superparamagnetic clustering of data” (1996)
Liefeng He et al., “Fast connected-component labeling” (2009)
Felzenszwalb et al., “Efficient graph-based image segmentation” (2004)
10-03-11 32/33
Thank you for your attention !
QUESTIONS ?
Göttingen, 10.03.2011
Recommended