Image Segmentation Zhiqiang wang zwang22@kent.edu some examples

Image Segmentation

Zhiqiang wang zwang22@kent.edu

some examples

image segmentations

Cell segmentation

Active contour method

Interactive method (graph cut)

Other examples

Cell Segmentation

1st Step: Image resize

Since original image’s resolution is 3978*3054, its size is very big and may let extracting algorithm be time consuming.

2nd Step: Image smooth

To simplify image’s content, noise and detail texture should be removed.

Gaussian filter or Nonlinear diffusion method

3rd Step: interactive segmentation

Using interacting method to select which cell we want to extract.

Level set : initial contour

Water shed : seed point

Graph cut: label foreground and background

3rd Step: Find centroids of subregion

After segmentation, we can get 59 subregions. For each region, we find centroids for each subregion as a seed point.

3rd Step: Find centroids of subregion

How to find center point

In some cases, centroid is outside of the subregion. As a seed point, it would impede further segmentation.

Possible solution: erode the subregion until it become a point. computing the distance between inside pixels and the contour of subregion, take the

point which have max distance value as the seed point.

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

180

200

-60

-40

-20

0

20

40

Distance fieldSkeleton of the subregion

Active Contour Model for Image Segmentation

What’s active contour?

This method can also be understood as a special case of a more general technique of matching a deformable model to an image by energy minimization.

AC = Curve fitted iteratively to an image evolve based on its shape and the image value until it stabile (ideally on an object’s boundary).

Advantages of active contour

Threshold Edge detectionAn image of blood vessel

Nice representation of object boundary: Smooth and closed, good for shape analysis and recognition and other applications.

parametric geometric

Curve:

polygon = parametric AC continuous = geometric AC

Parametric Model: Gradient vector flow (GVF)• GVF field is a non-irrotational external force field that points toward the

boundaries when in their proximity and varies smoothly over homogeneous image regions all the way to image borders.

||/ ff Gradient vector flow

Example: Gradient vector flow

• GVF field is a non-irrotational external force field that points toward the boundaries when in their proximity and varies smoothly over homogeneous image regions all the way to image borders.

General Curve evolution• Let a curve moving in time t be denoted by X[x(s,t), y(s,t) ） , where s is curve

parameterization. Let N be the moving curve’s inward normal, and c curvature. And let the curve develop along its normal direction according to the partial differential equation:

Basic deformation equation• Constant Speed Motion (Area decreasing flow)

• Mean curvature motion (Length shortening flow)

• During the evolution process for image segmentation, curvature deformation and/or constant deformation are used and the speed of curve evolution is locally dependent on the image data.

Its main idea of CV model is to minimize the inter class variances

)(

220)(

210

21,,

,,inf121

CoutsideCinside

CCcc

dxdycIdxdycI

dsCccE

CV model

A basic version of the speed function that combine curvature and constant deformation is CV model(Active contour model without edge) Its main idea is to consider the information inside the regions.

Let be the original image to be segmented and C denote the evolving curve. and are positive weights to control C’s smoothness. is the mean value of inside the C and is the mean outside C.

)(

220)(

210

21,,

,,inf121

CoutsideCinside

CCcc

dxdyuIdxdyuI

dsCuuE

0I 1u

0I 2u

← Smooth term

← data term

Evolution speed control (CV model)

220

210 uIuIN

t

C

To minimize the cost function, Euler-lagrange equation is used:

Evolution speed control (CV model)

Its main idea of CV model is to minimize the inter class variances

220

210 cIcIN

t

C

• Mean curvature motion is the steepest descent

flow (or gradient flow) that minimizes arc length of the contour:

Parametric Deformable Model • The curves can be represented as level sets of higher dimensional functions

yielding seamless treatment of topological changes.

Research Problem-- weakness of region based model

failure

success

Evolution speed control--GAC model

• A basic version of the speed function that combine curvature and constant deformation is GAC model:

Smooth term data term

NN

ggt

C

g is an edge-stopping function defined as follow: 1

g 21 G 0I

0G I The term denotes the gradient of a Gaussian smoothed image, where is a smooth parameter.

• During the evolution process for image segmentation, curvature deformation and/or constant deformation are used and the speed of curve evolution is locally dependent on the image data.

GAC model

Features of edge based model

failure

success

3D Case

Interactive segmentation(graph cut and alpha matting)

Reference: Anat Levin, etc. A Closed Form Solution to Natural Image Matting. 2006

Remove complicate background

Over segmentation with meanshift method

Construct graph and perform graph cut agorithm

Source (Label 0)

Sink (Label 1)

Cost to assign to 0

Cost to assign to 1

Cost to split nodes

Construct graph and perform graph cut agorithm

Gaussian Mixture Model and Graph Cut

Gaussian Mixture Model (typically 5-8 components)

Foreground &Background

Background

Foreground

BackgroundG

R

G

RIterated graph cut

More examples

The problem of hard segmentation

Alpha matting

+

Alpha matting

iiiii BFI )1(

+ xx=

Matting is ill posed problem

iiiii BFI )1(

0

1

Scribbles approach

Color Line:

Color lines

213 )1( CCCRC iiii

R

B

G

Color lines

Color Line: 213 )1( CCCRC iiii

R

B

G

Matting results

+

Combine hard segmentation

More examples

Thanks