A Fast Algorithm for Variational Level Set Segmentation

Tony F. Chan

Department of Mathematics, Univ. Calif. Los Angeles

SONAD 2003

McMaster University, Ontario, Canada

May 2, 2003

Reprints: www.math.ucla.edu/~imagers

Supported by ONR and NSF

(Joint work with Bing Song)

OUTLINE

• Review of Variational Level Set Segmentation

• Fast Algorithm

What is an “active contour” ?

Initial Curve Evolutions Detected Objects

objects theof boundaries on the stop tohas curve the

in objectsdetect to curve a evolve

: imagean giving

0

0

uC

u

A fitting term “without edges”

)(

220

2

)(

10 ||||CoutsideCinside

dxdycudxdycu

where Cuaveragec

Cuaveragec

outside )(

inside )(

02

01

Fit > 0 Fit > 0 Fit > 0 Fit ~ 0

Minimize: (Fitting +Regularization)

Fitting not depending on gradient detects “contours without gradient”

)( )(

220

210

21,,

||||

))((||),,(inf21

Cinside Coutside

Ccc

dxdycudxdycu

CinsideAreaCCccF

An active contour model “without edges”

Fitting + Regularization terms (length, area)

C = boundary of an open and bounded domain

|C| = the length of the boundary-curve C

(C. + Vese 98)

Advantages - detects objects without sharp edges

- detects cognitive contours

- robust to noise; no need for pre-smoothing

Examples

Level Set Representation (S. Osher - J. Sethian ‘87)

Inside C

Outside C

Outside C0

0

0

0C

nn

n

||

,||

Normal

divKCurvaturen

Example: mean curvature motion

* Allows automatic topology changes, cusps, merging and breaking.

•Originally developed for tracking fluid interfaces.

0),(|),( yxyxC

C= boundary of an open domain

Variational Formulations and Level Sets(Following Zhao, Chan, Merriman and Osher ’96)

The Heaviside function

The level set formulation of the active contour model

0),( :),( yxyxC

0 if ,0

0 if ,1)(

H

dxdyHCinsideArea

HC

)())((

|)(|||Length

dxdyHcyxudxdyHcyxu

dxdyHHccF

ccFcc

))(1(|),(|)(|),(|

)(|)(|),,(

),,(inf

220

210

21

21,, 21

))),((1()),((),( 21 yxHcyxHcyxu

The Euler-Lagrange equations

),,(inf 21, ,21

ccFcc

dxdyH

dxdyHu

cdxdyH

dxdyHu

c))(1(

))(1(

)( ,)(

)(

)(0

2

0

1

),,(for Equation yxt

),(),,0(

)()(||

)(

0

220

210

yxyx

cucudivt

2 1 andfor Equations cc

Using smooth approximations for the Heaviside and Delta functions

Advantages

Automatically detects interior contours!

Works very well for concave objects

Robust w.r.t. noise

Detects blurred contours

The initial curve can be placed anywhere!

Allows for automatical change of topolgy

Experimental ResultsC of Evolution ),( Averages 21 cc

A plane in a noisy environment

A galaxy

From Active Contours (2D) to Active Surfaces (3D)

MRI DATA FROM LONI-UCLA

Extension to Vector-Valued ImagesTony F. CHAN & Luminita VESE & B. Yezrielev SANDBERG

Applications

* Color images (RGB)

* Multi-spectral (PET, MRI, CT)

channels ),,...,,( ,02,01,00 Nuuuu N

Vector-Valued image:

ectorsconstant v are , 21 cc

The model

)( )(

2,2,0

2,1,0

21

||||

))(()(),,(

Cinside Coutside

iiiii

dxdycudxdycu

CinsideAreaCLengthCccF

(averages of each channel inside and outside the curve)

* The model can be solved using level sets, similar with the scalar case

),,(inf 21,, 21

CccFCcc

Channel 1 with occlusion Channel 2 with noise

Recovered object and averages

Extension to Vector-Valued Images

Color (RGB) picture Intensity (gray-level) picture

R G B

Recovered objects and contours in RGB mode

Color Images

45 transforms of image with Gabor function

F F

F

120

150

180

120

150

180

120

150

180

User Picked Gabor transforms yield final contour

Active Contour for Texture

Logic Operators on Multi-Channel Active Contour

(Chan, Sandberg 2001)• An Example for two channel logical segmentation:

1A 2A

21 (a) AA 21 (b) AA 21 ~ (c) AA

1A 2A

Original image with contour overlapping

Contour only

Time evolution for finding the “tumor” in the first image that is not in the second.

Example of an Application

0

0

2

1

0

0

2

1

0

0

2

1

0

0

2

1

0

0

4-phase segmentation2 level set functions

2-phase segmentation1 level set function

}0{

:Curves

}0{}0{

:Curves

21

Multiphase level set representations and partitions allows for triple junctions, with no vacuum and no overlap of phases phases 2 ),...,( 1

nn

Example: two level set functions and four phases

|)(||)(|

))(1))((1(||)())(1(||

))(1)((||)()(||),(

Energy

))(1))((1()())(1(

))(1)(()()(

vectorConstant ),,,(

functionsset level The ),(

21

212

000212

010

212

100212

110),(

21002101

21102111

00011011

21

HH

dxdyHHcudxdyHHcu

dxdyHHcudxdyHHcucFInf

HHcHHc

HHcHHcu

ccccc

c

0,0 ,0,0 ,0,0 ,0,0

:segmentsor phases 4 ),( functionsset level 2

21212121

21

Phase 11 Phase 10 Phase 01 Phase 00

mean(11)=45 mean(10)=159 mean(01)=9 mean(00)=103

An MRI brain image

Three level set functions representing up to eight phases Six phases are detected, together with the triple junctions.

T

Evolution of the 3 level sets Segmented image

Evolution of the 3 individual level sets.

OUTLINE

• Review of Variational Level Set Segmentation

• Fast Algorithm

A Fast Algorithm for Level Set Based Optimization

(Bing Song and Tony Chan, 2003)

Available at www.math.ucla.edu/applied/cam/index.html

CAM 02-68

(Related work by Gibou & Fedkiw)

Motivation

• Minimize level set based functional

• Examples:– Image segmentation– Inverse problem– Shape analysis and optimization– Data Clustering– …

))((min

HF

How to solve it?

• Drawback– Slow (CFL)

– F need to be differentiable

))(( HFt

• Usually use gradient descent flow

An Insight

• Segmentation only needs sign of but not its value

• Direct search making use of objective function does not require derivatives of functional

New Algorithm

1. Initialization. Partition the domain into and

2. Advance. For each point x in the domain, if the energy F lower when we change to , then update this point.

3. Repeat step 2 until the energy F remains unchanged

0

0

)(x

)(x

An example: Chan-Vese model

))(1(||

)(|||)(|(),),((

2202

210121

Hcu

HcuHccHF

Core Step of Algorithm

0

0

0

1)(

~1

)(~

1~

1~

2222

2111

222

111

n

ncuFF

m

mcuFF

n

uccc

m

uccc

lengthinchangem

mcu

n

ncuF

1)(

1)( 2

12

212

Change in F if is changed to at a pixel:0 0

How to calculate length term?

2,11,

2,,1 ))()(())()((|)(| jijijijiij HHHHH

Change in length term can be updated locally.

Only possible value for length term locally is 0, 1, sqrt(2) depending the 3 points in the length term belong to the same region or not.

A 2-phase example

(a), (b), (c), (d) are four different initial condition.All of them converge in one sweep!

Example with Noise

Converged in 4 steps.

(Gradient Descent on Euler-Lagrange took > 400 steps.)

Convergence of the algorithm

Theorem: For a two phase image, the algorithm will converge in one sweep, independent of sweeping order.

00

0

0

Proof considers the following 2 cases:

Green part of inside of contour will flip sign. Only one of the 2 black parts can change sign

Why is 1-step convergence possible?

• Problem is global: usually cannot have finite step convergence based on local updates only

• But, in our case, we can exactly calculate the global energy change via local update (can update global average locally)

Application to piecewise linear CV model(Vese 2002)

))(1(||

)(|||)(|(),),((

221002

22100121

Hybxbbu

HyaxaauHccHF

Original P.W.Constant

Converged in 4 steps

P.W. Linear

Converged in 6 steps

Application to multiphase CV model(C.- Vese 2000)

• Using n level set to represent 2n phases.

mIi

nIIIn HdxdycucF

m 121

20 |)(|||),(

Multiphase CV model (cont’d)

Converged in 1 step.

Gibou and Fedkiw’s method (02)• First, discard the stiff length term in E-L:

• Take large time step enough to change sign

• If , then

• Followed by anisotropic diffusion to handle noise

• Difference (Ours wrt GF):– A unified framework to handle length term

– F does not need to be differentiable

)(:)()( 222

211 xVcucu

dt

d

0)()( xVx ))(()( xsignx

Conclusion• A fast algorithm to solve the Chan-Vese

segmentation models.• 1-step convergence for 2 phase images (no reg.).• Robust wrt noise: 3-4 step convergence.• The gradient of functional is not needed.• We think it can be applied to more general level

set based optimization problems.• In general, cannot expect finite step convergence

(need global change from local update). Can get stuck in local minima.

That’s all.

Thank you for your patience!

That’s all.

Thank you for your patience!