Segmentation of Tree like Structures as Minimisation Problem applied to Lung Vasculature Pieter...

Segmentation of Tree like Structures as Minimisation Problem applied to Lung

Vasculature

Pieter Bruyninckx

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Overview

• Introduction– What is vessel segmentation– Lung anatomy and pathology

• State of the Art• Minimisation Approach• Discussion

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Introduction

• Vessel segmentation– Extracting vessels from an image (3D)– Multiple representations possible

• Hard: Centerline, border

• Soft: Probability

• Use general properties– Intensity range– Tubular shape– Tree like structure

• Connectedness

• Bifurcations

Pathological cases?

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Lung anatomy and pathology

• General Structure– Vessels

– Bronchi

• Common pathology– Mosaic Perfusion

• Pulmonary embolism

• Small airways disease

• Emphysema

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Overview

• Introduction• State of the Art

– Top-down (initialisation)• Single Vessel

• Tracking

– Bottom-up (no initialisation)• Vessel Enhancement Filter

• Tschirren

• Minimisation Approach• Discussion

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Top-down: Single Vessel

• Initialisation– Start and end-point

• Iteration– Optimal path

– Minimal energy

– Intensity, smoothness

• Challenges– Whole vessel tree?

[Wink2002]

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Top-down: Tracking

• Initialisation– Start point (tree root)

• Iteration– Centerline tracking

– More advanced• Region growing• Wave front propagation• Level sets

• Challenges– Handling bifurcations

– Mathematical framework

[Wink2000]

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Bottom-up: Vessel Enhancement Filters

• Soft segmentation =vessel enhancement:

– Improve hard segmentation– Improve visualization

• No initialisation• Single iteration

– Eigenvalues/vectors Hessian• Vesselness and orientation

– Multiresolution approach– Anisotropic filtering

• Challenges– Physical units?– Sensitive to noise (second

derivatives)[Frangi1998]

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Bottom-up: Tschirren

• Initialisation– Detection of probably-

vessel voxels (intensity)

– Compute orientation

– Classify: • vessel, junction, nodule

• Iteration– Join points into a tree

• Challenge– Rather ad hoc method

[Tschirren2005]

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Overview

• Introduction• State of the Art• Minimisation Approach

– Pre-processing– Initialisation– Iteration– Results

• Discussion

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Minimisation Approach

Initial vesselness

(intensity only)

Patient scan

Initial vessel orientation

(with uncertainty)

Final vesselness and orientation through energy optimization

Energy: function ofneighbourhood and

local vessel orientation and vesselness

Lung Segmentation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Lung Segmentation

• Goal– Separate lung from other tissue

– Allow for intensity-based vessel segmentation

– Increased efficiency

• Implementation– Simple ad hoc algorithm

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Minimisation Approach

Initial vesselness

(intensity only)

Patient scan

Initial vessel orientation

(with uncertainty)

Final vesselness and orientation through energy optimization

Energy: function ofneighbourhood and

local vessel orientation and vesselness

Lung Segmentation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Vesselness Initialisation

• Based on intensities only• First step

– Determine average and standard deviationof local background(10x10x10 mm³)

• Second step– Express vesselness as a ‘probability’ [0,1]

(function of background , and intensity)

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Vesselness Initalisation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Vesselness Initialisation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Minimisation Approach

Initial vesselness

(intensity only)

Patient scan

Initial vessel orientation

(with uncertainty)

Final vesselness and orientation through energy optimization

Energy: function ofneighbourhood and

local vessel orientation and vesselness

Lung Segmentation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Initial Vessel Orientation

• Estimate– Orientation

– Uncertainty

• Method– Compute gradients

perpendicular orientation

– Hessian approach also possible

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Minimisation Approach

Initial vesselness

(intensity only)

Patient scan

Initial vessel orientation

(with uncertainty)

Final vesselness and orientation through energy optimization

Energy: function ofneighbourhood and

local vessel orientation and vesselness

Lung Segmentation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Iteration: Energy Function

• Energy Function f– Energy ~ Vessel-likeness

internal energy EE( ) < E( )

– Energy ~ ‘distance’ to original distance DD( , ) < D( , )

1,0,1 DEf

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Iteration: Energy Function

Vv i

ii uvdwE 2,

2

vn

vnw

i

Ti

i

1,0,1 DEf

ni v

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Iteration: Energy Function

Vv i

ii uvdwE 2,

2

vn

vnw

i

Ti

i

1,0,1 DEf

2

2

2, m Tm T vvuuvud

d( , )

d( , )

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Iteration: Energy Function

• Distance function D– Extension of d

– gn: main directions (orthogonal) ( )

1,0,1 DEf

TTTv ggggggG 332211

Vv

vTjj GvvD

2

2

2

2

2, m Tm T vvuuvud

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Iteration: Energy Function

Vv

vTjj GvvD

2

2

1,0,1 DEf

No uncertainty

No certainty

Some uncertainty

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Optimisation: Method

• Possible approaches– Simulated annealing (SA)

• Slow

– Local alignment• Fast

• Global optimum?

• Proposed solution– Local everywhere– SA at ‘difficult’ locations

Best of both worldsIf difficult locations can be found

(high local energy)

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Results: 2D

original α = 0.1α = 0.4α = 0.6α = 0.8α = 0.9α = 0.95α = 0.99

1,0,1 DEf

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Results: 2D (detail)

original α = 0.99

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Visualisation: 3D

• Result: soft segmentation• Visualisation

– Volume rendering (Voxar3D)– Intensity = vesselness– (no orientation information)– Window/level

(can't see the wood for the trees)

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Results: 3D (Overview)

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Results: 3D (Detail)

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Overview

• Introduction• State of the Art• Minimisation Approach• Discussion

– Current challenges– Future Work

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Current Problems

• Bronchi– Tubular structures– Walls are likely

to be enhanced– Multiresolution needed?

• Speed– Processing time > 24 h

• Rewrite code partially in C• More efficient optimisation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Current Problem: Discrete weights

• Current weights

– Influence of all neighbours Vessels get wider

• Discrete weights– Look at two neighbors– Smaller vessels possible– Better delineation– Efficient local alignment?

2

vn

vnw

i

Ti

i

1,0,1 DEf

Vv i

ii uvdwE 2,

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Future Work: Liver and Bronchi

• Liver vasculature– Complex tree like structure

• Arterial and venous

– More ‘noisy’ background

• Bronchi– Complex tree like structure– Intensity based classification difficult

• Low intense centre surrounded by high intense wall (partial volume artefacts)

– Multiresolution?

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Future work

• Multiresolution– Separation:

• Vessel wall Vessel

– Improve speed?

• Model extensions– Synchronous segmentation:

• Vessels and Bronchi

– Bifurcation detection

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Future Work

• Post processing– Multitude of information

• Vesselness

• Orientation

• (3 degrees of freedom / voxel)

– Advanced visualisation– Improved hard segmentation algorithms

• Validation

Pieter.Bruyninckx@uz.kuleuven.ac.be

•

Minimisation Approach

State of the Art

Introduction

Discussion

Vessel segmentation Thank you for your attention

Questions ?