Upload
miles-barton
View
219
Download
0
Embed Size (px)
Citation preview
SegmentationSegmentation
Lucia Ballerini
Digital Image Processing
Lecture 8
Course book reading: GW 10Course book reading: GW 10
ImageImage
"What are the objects to be analyzed?"
Pre-processing, image enhancement
Segmentation
Binary operations
Morphological operations and feature extraction
Classification and matching
Image analysisDataData
SegmentationSegmentation
► Full segmentation: Individual objects are separated from the background and given individual ID numbers (labels).
► Partial segmentation: The amount of data is reduced (usually by separating objects from background) to speed up the further processing.
► Segmentation is often the most difficult problem to solve in the process; there is no universal solution!
► The problem can be made much easier if solved in cooperation with the constructor of the imaging system (choice of sensors, illumination, background etc) .
Three types of segmentationThree types of segmentation
► Classification – Based on some similarity measure between pixel values. The simplest form is thresholding.
► Edge-based – Search for edges in the image. They are then used as borders between regions
► Region-based – Region growing, merge & split
Common idea: search for discontinuities or/and similitudes in the image
ThresholdingThresholdingglobalglobal or or locallocal
► global: based on some kind of histogram: grey-level, edge, feature etc. Lighting conditions are extremely important, and it will only
work under very controlled circumstances.
► Fixed thresholds: the same value is used in the whole image
► local (or dynamic thresholding): depends on the position in the image. The image is divided into overlapping sections which are thresholded one by one.
Classical automatic Classical automatic thresholding algorithmthresholding algorithm
1.1. Select an initial estimate for TSelect an initial estimate for T
2.2. Segment the image using T. This produces 2 groups: Segment the image using T. This produces 2 groups: GG11 , pixels with value >T and G , pixels with value >T and G2 2 , with value <T, with value <T
3.3. Compute µCompute µ11 and µ and µ22, average pixel value of G, average pixel value of G11 and G and G22
4.4. New threshold: T=1/2(µNew threshold: T=1/2(µ11+µ+µ22))
5.5. Repeat steps 2 to 4 until T stabilizes.Repeat steps 2 to 4 until T stabilizes.
► Very easy + very fastVery easy + very fast► Assumptions: normal dist. + low noiseAssumptions: normal dist. + low noise
Optimal ThresholdingOptimal Thresholding
► based on the shape of the current image histogram. Search for valleys, Gaussian distributions etc.
Background
Real histogram
Optimalthreshold ?
Both
Foreground
HistogramsHistograms
To love… …and to hate
Thresholding and illuminationThresholding and illumination
► Solutions:Solutions: Calibration of the Calibration of the
imaging systemimaging system percentile filter with percentile filter with
very large maskvery large mask Morphological Morphological
operatorsoperators
MR non-uniformityMR non-uniformity
median filtering thresholding
-
More thresholdingMore thresholding► Can also be used on other kinds of histogram: grey-
level, edge, feature etc. Multivariate data ( see next lectures)
► Problems: Only considers the graylevel pixel value, so it can leave “holes”
in segmented objects.► Solution: post-processing with morphological operators
Requires strong assumptions to be efficient Local thresholding is better see region growing techniques
Edge-based SegmentationEdge-based Segmentation
Based on finding discontinuities (local variations of image intensity)
1. Apply an edge detector ex gradient operator (Sobel)
second derivative (Laplace)
2. Threshold the edge image to get a binary image
3. Depending on the type of edge detector: Link edges together to close shapes (using edge direction for ex) Remove spurrious edges
Gradient based procedureGradient based procedure
Sobel
Sobel
Zero-crossing based procedureZero-crossing based procedure
LoG
Laplacian of GaussianLaplacian of Gaussian
Edge-based Segmentation: examplesEdge-based Segmentation: examples
Prewitt: needs edge linking Canny: needs “cleaning”
Region based segmentationRegion based segmentation
►Work by extending some region based on local similarities between pixels region growing (bottom-up method) region splitting and merging (top-down
method)
Bottom-up: from data to representation Top-down: from model to data
Region growing Region growing (bottom-up method)(bottom-up method)
1. Find starting points
2. Include neighbouring pixels with similar features (grey-level, texture, color).
3. Continue untill all pixels have been included with one of the starting points.
► Problems: Not trivial to find good strating points, difficult to
automate Need good criteria for similarity.
WatershedWatershed(a kind of region growing)(a kind of region growing)
► Think of the grey-level image as a landscape. Let water rise from the bottom of each valley (the water from each valley is given its own label). As soon as the water from two valleys meet, build a dam, or watershed. These watersheds will then define the borders between different regions.
Example of watershed directly on a gray-level image
Example of Watershed on a binary image
Watershed: problems and solutionsWatershed: problems and solutions
► Oversegmentation Oversegmentation watershed from markerswatershed from markers
► ComputationComputation new algorithm for fast watershednew algorithm for fast watershed
► Graylevel might not be the optimal choice as the Graylevel might not be the optimal choice as the local similarity measurelocal similarity measure bigger neighborhood when growingbigger neighborhood when growing other local features (statistical, other local features (statistical, edge enhanced
image, distance transformed image…)
Split & MergeSplit & Merge(top-down metod)(top-down metod)
1. Set up som criteria for what is a uniform area (ex mean, variance, bimodality of histogram, texture, etc…)
2. Start with the full image and split it in to 4 sub-images.
3. Check each sub-image. If not uniform, divide into 4 new sub-images.
4. After each iteration, compare touching regions with neighboring regions end merge if uniform.
The method is also called "quadtree" division(and is also used for compression)
Split & MergeSplit & Merge
The Hough transformThe Hough transform
► A method for finding global relationships between pixels.Example: We want to find straight lines in an image 1. Apply edge enhancing filter (ex Laplace) 2. Set a threshold for what filter response is considered a true
”edge pixel” 3. Extract the pixels that are on a straight line using the Hough
transform
original image edge enhanced image
thresholded edge image
The Hough transformThe Hough transformFinding straight lines:
► 1. consider a pixel in position (xi,yi)► 2. equation of a straight line yi=axi+b ► 3. set b=-axi+ yi and draw this (single) line in ”ab-space”► 4. consider the next pixel with position (xj,yj) and draw
the line b=-axj+ yj ”ab-space” (also called parameter space). The poins (a’,b’) where the two lines intersect represent the line y=a’x+b’ in ”xy-space” which will go through both (xi,yi) and (xj,yj).
► 5. draw the line in ab-space corresponding to each pixel in xy-space.
► 6. divide ab-space into accumulator cells and find most common (a’, b’) which will give the line connecting the largest number of pixels
y
The Hough transformThe Hough transform
xy-space
x
ab- or parameter space
b
a
The Hough transformThe Hough transform► In reality we have a problem with y=ax+b because a
reaches infinity for vertical lines.Use instead.
► It is common to use ”filters” for finding the intersection: ”butterfly filters”
► Different variations of the Hough transform can also be used for finding other shapes of the form g(v,c)=0, v is a vector of coordinates, c is a vector of coefficients.
► Possible to find any kind of simple shapeex. circle: (3D parameter space)
sincos yx
( ) ( )x c y c c 12
22
32
The Hough transformThe Hough transform
ConclusionsConclusions
► The segmentation procedureThe segmentation procedure1.1. Pre-processingPre-processing
2.2. SegmentationSegmentation
3.3. Post-processingPost-processing
Like any IP procedureLike any IP procedure
► There exists NO universal segmentation There exists NO universal segmentation methodmethod
► Evaluation of segmentation performance is Evaluation of segmentation performance is importantimportant
SnakesSnakes►Example: segmentation of the brain in MRIExample: segmentation of the brain in MRI
Snake after initialization Snake at equilibrium
User interaction
Snakes (active contours)Snakes (active contours)
► A A snake snake is an active contour parametrically is an active contour parametrically represented by its position v(s)=(x(s), y(s))represented by its position v(s)=(x(s), y(s))
► Each position is associated to an energy:Each position is associated to an energy:
► The final position corresponds to the minimum The final position corresponds to the minimum of the energyof the energy
E E s ds E s dssnake int ext 0
1
0
1
[ ( )] [ ( )]v v
Internal EnergyInternal Energy
The internal energy of the snake is due to The internal energy of the snake is due to bending and it is associated with bending and it is associated with a prioria priori constraints:constraints:
► (s) controls the tension of the contour(s) controls the tension of the contour► (s) controls its rigidity(s) controls its rigidity
E s ss
ss
s
sint[ ( )] ( )
( )( )
( )v
v v
1
2
2 2
2
2
External EnergyExternal Energy
► The external energy depends on the image and The external energy depends on the image and accounts for accounts for a posterioria posteriori information information
► Several energy forms have been proposed Several energy forms have been proposed based on features of interest in the image based on features of interest in the image
► An energy commonly used to attract snakes An energy commonly used to attract snakes towards edges is:towards edges is:
E s G I x yext[ ( )] ( , )v 2
ApplicationsApplications
ApplicationsApplications
(by Terzopoulos)
ConsiderationsConsiderations
► The number of nodes is an important factor for the The number of nodes is an important factor for the behavior of the snake. Ability to resample the contour may behavior of the snake. Ability to resample the contour may be necessary.be necessary.
► If we want a closed contour, we set the first and the last If we want a closed contour, we set the first and the last point equal.point equal.
► Anchor points are necessary to keep the snake in position if Anchor points are necessary to keep the snake in position if the image forces are not enough.the image forces are not enough.
► It may be necessary to allow a snake contour to divide into It may be necessary to allow a snake contour to divide into two contours, or two contours to merge into one contour.two contours, or two contours to merge into one contour.
► Different applications may need different potential Different applications may need different potential functions and different settings of the control parameters functions and different settings of the control parameters (damping, tension and rigidity).(damping, tension and rigidity).
ApplicationsApplications► Tracking of a moving objectTracking of a moving object
An initial estimate for the contour (e.g. interactively defined) is An initial estimate for the contour (e.g. interactively defined) is used in the first frame. used in the first frame.
The contour at equilibrium is used as the starting contour for the The contour at equilibrium is used as the starting contour for the next frame. The snake locks on to the object.next frame. The snake locks on to the object.
► Reconstruction from serial sectionsReconstruction from serial sections The user draws an approximate contour in the first slice. The user draws an approximate contour in the first slice. The contour at equilibrium is used as the starting contour in the The contour at equilibrium is used as the starting contour in the
next slice.next slice. The 3D object is reconstructed from the contours using The 3D object is reconstructed from the contours using
triangulation.triangulation.
► ……....
More segmentationMore segmentation
Important in Image Processing in general:Important in Image Processing in general:
““If you can use expert knowledge (user If you can use expert knowledge (user interaction, modelling,…) at relatively low interaction, modelling,…) at relatively low cost (development, computational,…)”cost (development, computational,…)”
JUST DO IT!!JUST DO IT!!