Chapter 7 Image Segmentation Chuan-Yu Chang ( )Ph.D. Dept. of
Electronic Engineering National Yunlin University of Science &
Technology [email protected] Office: ES709 Tel: 05-5342601
Ext. 4337
Slide 2
2 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Introduction Image segmentation refers to the process of
partitioning an image into distinct regions by grouping together
neighborhood pixels based on some pre-defined similarity criterion.
Segmentation is a pixel classification technique that allows the
formation of regions of similarities in the image.
Slide 3
3 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Introduction (cont.) Image segmentation methods can be broadly
classified into three categories: Edge-based methods The edge
information is used to determine boundaries of objects. The
boundaries are then analyzed and modified to form closed regions
belonging to the objects in the image. Pixel-based direct
classification Heuristics or estimation methods derived from the
histogram statistics of the image are used to form closed regions
belonging to the objects in the image. Region-based methods Pixels
are analyzed directly for a region growing process based on a
pre-defined similarity criterion to form closed regions belonging
to the objects in the image.
5 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Edge-Based Segmentation Edge-Based approaches use a spatial
filtering method to compute the first-order or second-order
gradient information of the image. Sobel, directional derivative
masks can be used to compute gradient information of the image.
Laplacian mask can be used to compute second-order gradient
information of the image. For segmentation purpose Edges need to be
linked to form closed regions Gradient information of the image is
used to track and link relevant edges.
Slide 6
6 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Introduction Image segmentation algorithm generally are based on
one of two basic properties of intensity values: Discontinuity
Partition an image based on abrupt changes in intensity. Similarity
Partitioning an image into regions that are similar according to a
set of predefined criteria. There are three basic types of
gray-level discontinuities Points, lines, and edges.
Slide 7
7 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities discontinuity mask (sum of
product)
Slide 8
8 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Point detection T Threshold X Mask (c)
90% threshold
Slide 9
9 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Line detection R 1, R 2, R 3, R 4 mask
| R i |>| R j | i mask threshold
Slide 10
10 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities 45 line line We are interested in
finding all the lines that are one pixel thick and are oriented at
-45. Use the last mask shown in Fig. 10.3.
Slide 11
11 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Edge detection An edge is a set of
connected pixels that lie on the boundary between two regions. The
thickness of the edge is determined by the length of the ramp.
Blurred edges tend to be thick and sharp edges tend to be
thin.
Slide 12
12 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities The first derivative is positive at
the points of transition into and out of the ramp as we move from
left to right along the profile. It is constant for points in the
ramp, It is zero in areas of constant gray level. The second
derivative is positive at the transition associated with the dark
side of the edge, negative at the transition associated wit the
light side of the edge, and zero along the ramp and in areas of
constant gray level.
Slide 13
13 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
detection (cont.) Two additional properties of the second
derivative around an edge: It produces two values for every edge in
an image. An imaginary straight line joining the extreme positive
and negative values of the second derivative would cross zero near
the midpoint of the edge. The zero-crossing property of the second
derivative is quit useful for locating the centers of thick
edges.
Slide 14
14 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities The entire transition from black to
white is a single edge. Image and gray-level profiles of a ramp
edge First derivative image and the gray-level profile Second
derivative image and the gray- level profile =0.1 =1 =10 =0
Slide 15
15 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
detection (cont.) The second derivative is even more sensitive to
noise. Image smoothing should be a serious consideration prior to
the use of derivatives in applications. Summaries of edge detection
To be classified as a meaningful edge point, the transition in gray
level associated with that point has to be significant stronger
than the background at that point. Use a threshold to determine
whether a value is significant or not. We define a point in an
image as being an edge point if its two dimensional first-order
derivative is greater than a specified threshold. A set of such
points that are connected according to a predefined criterion of
connectedness is by definition an edge. Edge segmentation is used
if the edge is short in relation to the dimensions of the image. A
key problem in segmentation is to assemble edge segmentations into
linger edges. If we elect to use the second-derivative to define
the edge points in an image as the zero crossing of its second
derivative.
Slide 16
16 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Gradient operator The gradient of an
image f(x,y) at location (x,y) is defined as the vector the
gradient vector points is the direction of maximum rate of change
of f at coordinates (x,y). The magnitude of the vector denoted f,
where The direction of the gradient vector denoted by The angle is
measured with respect to the x -axis.
Slide 17
17 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Roberts operator G x =(z 9 -z 5 ) G y
=(z 8 -z 6 ) Prewitt operator G x =(z 7 +z 8 +z 9 )-(z 1 +z 2 +z 3
) G y =(z 3 +z 6 +z 9 )-(z 1 +z 4 +z 7 ) Sobel operator G x =(z 7
+2z 8 +z 9 )-(z 1 +2z 2 +z 3 ) G y =(z 3 +2z 6 +z 9 )-(z 1 +2z 4 +z
7 )
19 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Example 10-4 smooth Fig. 10.10 shows
the response of the two components of the gradient, | G x | and | G
y |. The gradient image formed the sum of these two
components.
Slide 20
20 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Example
10-4 5*5 averaging filter Detection of discontinuities
Slide 21
21 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Example
10-4 Sobel mask 45 Detection of discontinuities The horizontal and
vertical Sobel masks respond about equally well to edges oriented
in the minus and plus 45 direction. If we emphasize edges along the
diagonal directions, the one of the mask pairs in Fig. 10.9 should
be used. The absolute responses of the diagonal Sobel masks are
shown in Fig. 10.12. The stronger diagonal response of these masks
is evident in these figures.
Slide 22
22 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities The Laplacian of a 2-D function f(x,y)
is a second-order derivative defined as For a 3x3 region, one of
the two forms encountered most frequently in practice is ( ) A
digital approximation including the diagonal neighbors is given by
( )
24 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Laplacian of a Gaussian (LoG) The
Laplacian is combined with smoothing as a precursor to finding
edges via zero-crossing, consider the function where r 2 =x 2 +y 2
and is the standard deviation. Convolving this function with an
image blurs the image, with the degree of blurring being determined
by the value of . The Laplacian of h is The function is commonly
referred to as the Laplacian of a Gaussian (LoG), sometimes is
called the Mexican hat function
26 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Detection of discontinuities Comparing Figs/ 10.15(b) and (g) The
edges in the zero-crossing image are thinner than the gradient
edges. The edges determined by zero-crossings form numbers closed
loops. spaghetti effect is one of the most serious drawbacks of
this method. The major drawback is the computation of zero
crossing.
28 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Edge detection (edge linking) Edge
linking Based on pixel-by-pixel search to find connectivity among
the edge segmentations. The connectivity can be defined using a
similarity criterion among edge pixels. Local Processing pixel
(3x3, 5x5) Eq.(10.1-4) Eq. (10.1-5)
Slide 29
29 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection ( x,y ) ( x 0,y 0 ) ( x,y ) ( x 0,y
0 ) ( x 0,y 0 ) ( x,y )
Slide 30
30 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Example 10-6: the objective is to
find rectangles whose sizes makes them suitable candidates for
license plates. The formation of these rectangles can be
accomplished by detecting strong horizontal and vertical edges.
Linking all points, that had a gradient value greater than 25 and
whose gradient directions did not differ by more than 15. Sobel
operator Sobel operator (b) (c) edge linking 25 15
Slide 31
31 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Boundary Tracking A* search algorithm Select an edge pixel as the
start node of the boundary and put all of the successor boundary
pixels in a list, OPEN If there is no node in the OPEN list, stop;
otherwise continue. For all nodes in the OPEN list, compute the
cost function t(z) and select the node z with the smallest cost
t(z). Remove the node z from the OPEN list and label it as CLOSED.
The cost function t(z) may be computed as
Slide 32
32 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Boundary Tracking (cont.) If z is the end node, exit with the
solution path by tracking the pointers; otherwise continue. Expand
the node z by finding all successors of z. If there is no
successor, goto Step 2; otherwise continue. If a successor z i is
not labeled yet in any list, put it in the list OPEN with updated
cost as c(z i )=c(z)+s(z, z i )+d(z i ) and a pointer to its
predecessor z. If a successor z i is already labeled as CLOSED or
OPEN, update its value by c(z i )=min[c(z i ),c(z)+s(z,z i )] Put
those CLOSED successors whose cost function x(z i ) s were lowered,
in the OPEN list and redirect to z the pointers from all nodes
whose cost were lowered. Goto Step 2.
Slide 33
33 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Boundary Tracking (cont.) Figure 7.1. Top: An edge map with
magnitude and direction information; Bottom: A graph derived from
the edge map with a minimum cost path (darker arrows) between the
start and end nodes. End Node Start Node
Slide 34
34 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Global Processing via the Hough
Transform xy a (x i, y i ) (x j, y j ) b
Slide 35
35 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Hough Transform accumulator cell
accumulator cell 0 ( x k,y k ) a a b=-x k a+y k b b b a p b q
A(p,q)=A(p,q)+1
40 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Global Processing via
Graph-Theoretic Techniques Graph G =( N,U ) N : node U : N U ( n
i,n j ) arc n i, parent n j successor successor expansion Level 0
start root level goal Cost ( n i,n j ) arc (path) n 1 n k Path
(cost) p q
Slide 41
41 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection edge element H f ( x ) x
Slide 42
42 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection By convention, the point p is on the
right-hand side of the direction of travel along edge elements. To
simplify, we assume that edges start in the top row and terminate
in the last row. p and q are 4-neighbors. An arc exists between two
nodes if the two corresponding edge elements taken in succession
can be part of an edge. The minimum cost path is shown dashed. Let
r(n) be an estimate of the cost of a minimum-cost path from s to n
plus an estimate of the cost of that path from n to a goal node;
Here, g(n) can be chosen as the lowest-cost path from s to n found
so far, and h(n) is obtained by using any variable heuristic
information. (10.2-7)
44 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Graph search algorithm Step1: Mark
the start node OPEN and set g(s)= 0. Step 2: If no node is OPEN
exit with failure; otherwise, continue. Step 3: Mark CLOSE the OPEN
node n whose estimate r(n) computed from Eq.(10.2-7) is smallest.
Step 4: If n is a goal node, exit with the solution path obtained
by tracing back through the pointets; otherwise, continue. Step 5:
Expand node n, generating all of its successors (If there are no
successors go to step 2) Step 6: If a successor n i is not marked,
set Step 7: if a successor n i is marked CLOSED or OPEN, update its
value by letting Mark OPEN those CLOSED successors whose g value
were thus lowered and redirect to n the pointers from all nodes
whose g values were lowered. Go to Step 2.
Slide 45
45 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Edge
Linking and Boundary Detection Example 10-9: noisy chromosome
silhouette( ) The edge is shown in white, superimposed on the
original image.
Slide 46
46 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Thresholding To select a threshold T, that separates the objects
form the background. Then any point (x,y) for which f(x,y)>T is
called an object point; otherwise, the point is called a background
point. Multilevel thresholding Classifies a point ( x,y ) as
belonging to one object class if T 1 T 2 And to the background if
f(x,y) T 2
Slide 47
47 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Thresholding In general, segmentation problems requiring multiple
thresholds are best solved using region growing methods. The
thresholding may be viewed as an operation that involves tests
against a function T of the form where f(x,y) is the gray-level of
point (x,y) and p(x,y) denotes some local property of this point. A
threshold image g(x,y) is defined as Thus, pixels labeled 1
correspond to objects, whereas pixels labeled 0 correspond to the
background. When T depends only on f(x,y) the threshold is called
global. If T depends on both f(x,y) and p(x,y), the threshold is
called local. If T depends on the spatial coordinates x and y, the
threshold is called dynamic or adaptive.
Slide 48
48 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. The
role of illumination An image f(x,y) is formed as the product of a
reflectance component r(x,y) and an illumination component i(x,y).
In ideal illumination, the reflective nature of objects and
background could be easily separable. However, the image resulting
from poor illumination could be quit difficult to segment. Taking
the natural logarithm of Eq.(10.3-3) (10.3-4) (10.3-5)
Slide 49
49 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. The
role of illumination From probability theory, If i(x,y) and r(x,y)
are independent random variables, the histogram of z(x,y) is given
by the convolution of the histograms of i(x,y) and r(x,y).
Slide 50
50 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Fig.
(a)*Fig(c) f ( x,y ) r ( x,y ) i ( x,y ) Pixel-based Direct
Classification Methods
Slide 51
51 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Thresholding Basic global thresholding Select an initial estimate
for T Segment the image using T G 1 : consisting of all pixels with
gray level values > T G 2 : consisting of all pixels with gray
level values 0.9.
Slide 105
105 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Image
Segmentation Using Neural Networks Neural networks do not require
underlying class probability distribution for accurate
classification. The decision boundaries for pixel classification
are adapted through an iterative training process. Neural networks
learns from examples in the training set in which the pixel
classification task has already been performed using manual
methods. A non-linear mapping function between the input features
and the desired output for labeled examples is learned by neural
networks without using any parameterization. After the learning
process, a pixel in a new image can be classified for segmentation
by the neural network
Slide 106
106 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Image
Segmentation Using Neural Networks It is important to select a
useful set of features to be provided to the neural network as
input data for classification. The selection of training examples
is also very important, as they should represent a reasonably
complete statistical distribution of the input data. The structure
of the network and the distribution of training examples play a
major role in determining its performance for accuracy,
generalization and robustness.
Slide 107
107 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Backpropagation Neural Network for Classification The neural
element may receive its input from an input vector or other neural
elements. A weighted sum of these inputs constitutes the argument
of a non-linear activation function such as a sigmoidal function.
The resulting thresholded value of the activation function is the
output of the neural elements. The output is distributed along
weighted connections to other neural elements.
110 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Backpropagation Neural Network for Classification To learn a
specific pattern of input vectors for classification, an iterative
learning algorithm is used with a set of pre-classified training
examples that are labeled with the input vectors and their
respective class outputs. LMS algorithm, (Widrow-Hoff Delta Rule)
The learning algorithm repeatedly presents input vectors of the
training set to the network and forces the network output to
produce the respective classification output. Once the network
converges on all training examples to produce the respective
desired classification outputs, the network is used to classify new
input vectors into the learned classes.
Slide 111
111 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Backpropagation Neural Network for Classification The Least Mean
Squared (LMS) error algorithm can be implemented to train a
feedforward neural network using the following steps: Step 1:
Assign random weights in the range of [-1, +1] to all weights. Step
2: For each classified pattern pair { y (0), y L } in the training
set, do the following steps: Compute the output values of each
neural element using the current weight matrix. Find the error e
(k) between the computed output vector and the desired output
vector for the classified pattern pair. Adjust the weight matrix
using the change w (k) = e (k) [y (k-1) ] Step 3: Repeat Step 2 for
all classified pattern pairs in the training set until the error
vector for each training examples is sufficiently low or zero.
Slide 112
112 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Backpropagation Neural Network for Classification Two problems with
the BPNN It is sensitive to the selection of initial weights and
noise in the training set that can cause the algorithm to get stuck
in local minima in the solution pace. To find optimal network
architecture with the consideration of an optimal number of hidden
layers and neural elements in each of the hidden layers. Fahlman
proposed a cascade-correlation neural network architecture to find
the best architecture by correlating the error patterns with the
inclusion or deletion of a neural element in the hidden layers
based on the learning vectors in the training set.
Slide 113
113 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Radial
Basis Function (RBF) Neural Network for Classification The RBF is
based on the principle of regularization theory for function
approximation. It does not suffer from problems like the BPNN. The
RBF provides more reliable and reproducible results. The basic RBF
network contains an input layer for input signal distribution, a
hidden layer to provide the radial basis function processing, and
an output layer. The RBF representation can be obtained by
clustering the input training data to obtain the centroid vector, c
i for all clusters. A radial basis function is applied to the
Euclidean distance between the input vector x j and its own
centroid. The output of each radial basis function unit is then
weighted and summed through a linear combiner to provide the final
output of the network.
Slide 114
114 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Radial
Basis Function (RBF) Neural Network for Classification The final
output of the network f ( x ) can be given as The matrix of
activation function
Slide 115
115 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. Radial
Basis Function (RBF) Neural Network for Classification The weight
The major issues for implementation of the RBF network are the
location of the centroids and the structure of the radial basis
function. The locations of the radial basis functions depend upon
the data being presented to the network. The optimal number of the
clusters can be obtained from the statistical evaluation of the
population density and variance parameters of initial clusters
through an adaptive K -means or fuzzy clustering method.
Slide 116
116 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. RBF
Network RBF Unit 1 RBF Unit 2 RBF Unit n Input Image Sliding Image
Window Output Linear Combiner RBF Layer
Slide 117
117 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D.
Segmentation of Arterial Structure in Digital Subtraction
Angiograms Specific features with information about gray values,
edges and local contrast of the pixels are needed for effective
segmentation using a neural network based classifier. The feature
vector consists of the gray values of the center pixel and its
neighbors, combined with the contrast value at the center
pixel.
Slide 118
118 (Medical Image Processing Lab.) Chuan-Yu Chang Ph.D. RBF NN
Based Segmentation
