CSE 415 -- (c) S. Tanimoto, 2007 Segmentation and Labeling 1 Segmentation and Labeling Outline: Edge detection Chain coding of curves Segmentation into

CSE 415 -- (c) S. Tanimoto, 2007 Segmentation and Labeling

1

Segmentation and LabelingOutline:

• Edge detection• Chain coding of curves• Segmentation into Regions• Connected components labeling using the

UNION-FIND abstract data type• Blocks world scene analysis using Guzman’s

labeling method


2

Edge DetectionRationale: In order to find out where the objects are and what their properties are, locate their boundaries by finding “edges”.

An edge is a transition from a region of one predominant color or texture to another predominant color or texture.

Approaches: 1. local difference operators. E.g., Sobel, Roberts. * 2. model fitting. E.g., Hueckel, Canny. 3. vector space transformation. E.g., Frei & Chen. 3. contour finding via search. 4. snakes.


3

Edge Detection (cont.)Local Difference Operators

(Oriented)Horizontal differences:Eh[i, j] = F[i, j] – F[i, j-1]

Vertical differences:Ev[i, j] = F[i, j] – F[i-1, j]


4

Edge Detection (cont.)Combining directions

Strength[i,j] = sqrt( Eh2 + Ev

2)

Direction[i,j] = arctan(Ev / Eh )

(note: the latter has a bug when Eh = 0. This can be fixed using alternative cases.)


5

Edge Detection (cont.)Local Difference Operators

(Omnidirectional)

Roberts Cross operator:E[i, j] = sqrt( (F[i, j] – F[i+1, j+1])2 +

(F[i, j+1] – F[i+1, j])2 )

Sobel operator:E[i, j] = sqrt( [(c+2f+i)-(a+2d+g)]2 + [(g+2h+i)-(a+2b+c)]2)

a b c

b e f

c h i


6

Chain Coding of ContoursA chain code is a system for representing elementary vectors by small integers.

= 0 = 1 = 2 = 3To represent a curve, first approximate it as a sequence of elementary vectors, and then write down the code number for each vector in a sequence.


7

Segmentation into Regions

Technique 1: Thresholding

Technique 2: Region growing Example: Connected-components labeling with the UNION-FIND abstract data type.


8

Thresholding

Assumption: We have a monochrome image each of whose pixels belong either to the figure(s) or to the background.

Perform a thresholding transformation:

T[i,j] = 0 if F[i,j] < 1 otherwise

The value is called a threshold.


9

Threshold Selection

Choosing a suitable can be challenging.If is too low, many pixels may be misclassified as figure, whereas, if it is too high, many pixels may be misclassified as background.

The valley method: If the histogram of the input image is bimodal, choose as the threshold, the value between the modes where the histogram is lowest.


10

Valley Method Example2 3 5 11 7 6 25 5 6 11 5 1 2

XX XX X X X X X XX X X X X X1 2 3 4 5 6 7

Modes at 1 and 5. Valley at = 4.


11

Valley Method ExampleResult of thresholding with = 4

0 0 1 00 1 1 01 1 1 00 1 0 0


12

Segmentation into Regions:

A segmentation into regions means dividing up all the pixels of an image into groups.Each group of pixels must be connected to each other rather than separated in different parts of the image.Each group must have the same brightness, color, or other local property.It should not be possible to merge two groups and have the new group be both connected and uniform in property.


13

Segmentation into Regions:Formal Definitions

“Adjacency”

a b cd e fg h i

Pixel e is 4-adjacent to each of b, d, f, and h.a b cd e fg h i

Pixel e is 8-adjacent to each of a, b, c, d, f, g, h, i.


14

Segmentation into Regions:Connectivity

Let I = {p0, p1, …, pN-1} be an image.Let R be a nonempty subset of I. Suppose R has at least two pixels. Let ps1 and psn be two pixels in R.Pixels ps1 and psn are 4-connected in R provided there is a sequence of pixels (ps1, ps2, … psn) such that each successive pair of pixels is 4-adjacent, and all these pixels are in R.


15

Segmentation into Regions:Definition of Region

A region R of I is a 4-connected nonempty subset of I, such that every pair of pixels in R is 4-connected in R.


16

Region Adjacency

Let Ri and Rj be disjoint regions. If pi is a pixel in Ri and pj is a pixel in Rj, and pi is 4-adjacent to pj, then regions Ri and Rj are adjacent to one another.

Then we write Ri adj Rj .


17

Segmentation DefinitionLet I = {p0, p1, …, pN-1} be an image.Let S = {R1, R2, …, Rk} be a set of regions of I.Let P be a uniformity predicate where P(R) is true provided that all the pixels in R have a uniform appearance. For example, P(R) could be defined to be true whenever all pixels in R have the same value.

S is a segmentation of I provided that:(1) The regions cover I. That is, R1 R2 … Rk = I.(2) The regions are disjoint. i j Ri Rj = .(3) Each region is uniform. j, P(Rj).(4) Segments are maximal. If Ri adj Rj then P(Ri Rj). This means no regions can be merged and still maintain uniformity within the resulting region.


18

Computing a Segmentation with the UNION-FIND ADT

Initialize a set of regions by putting each pixel in its own region.

Scan the image in raster-scan order.

At each pixel pi, check its neighbor pj to the west (if it has one) as follows: If the two pixels have the same pixel value, do a FIND on each one getting ci and cj. If ci cj, then perform UNION(ci, cj).

Then check its neighbor pq to the north (if it has one) in the same way.

Perform a second scan of the image and at each pixel p, replace the pixel value by FIND(p)


19

Vision: Scene Labeling

Structure in the world allows us to interpret complex situations by propagating constraints.

This is conveniently demonstrated in computer vision, using a technique known as Guzman labeling.


20

Blocks-World Vision Problem

Given:A 2-D line drawing representing a collection of polyhedral blocks on a table.

Determine:Which faces (bounded regions in the drawing) go together as parts of the same objects.

Which edges (line segments) are internal to objects and which are “occluding” edges.


21

Example Scene Analysis Problem


22

Guzman’s Vertex Labels

Ell Arrow Fork Tee

Arrow: There is an angle of more than 180 deg.Fork: Each of the 3 angles is of less than 180 deg.Tee: There is one angle of exactly 180 deg., and 2 smaller ones.


23

Guzman’s Labelling Technique

1. Classify each vertex into one of the categories: ell, arrow, fork, tee, or other.2. At each vertex, mark each incoming edge according to the role it plays at the vertex (e.g., shank of an arrow, stem of a tee).3. Create single links between neighboring regions each time: a. they are divided by a line segment at a fork, b. they are divided by the shank of an arrow, c. they are in corresponding positions of a configuration of two opposing tees with colinear stems.4. Create a “node” for each region. Whenever two regions are doubly linked, connect their nodes with a “same object” link.


24

Example with Vertex Labels

A

A A

A

A

F

FA

LLL

L

T

T

T

LL

T

T


25

Example with Region Links


26

Example with Object Links


27

Example with Internal and Occluding Edges Identified

occluding

internal


28

Discussion

The work of Guzman was refined by D. Huffman, M. Clowes, and D. Waltz to take into account additional kinds of edges, including crack edges, oriented occluding edges, and concave and convex internal edges.

A key point of this research was to show that constraints imposed by the real world could make seemingly intractable combinatorial interpretation problems actually solvable.

Documents

CSE 415 -- (c) S. Tanimoto, 2007 Segmentation and Labeling 1 Segmentation and Labeling Outline: Edge detection Chain coding of curves Segmentation into