Upload
noah
View
34
Download
1
Tags:
Embed Size (px)
DESCRIPTION
Implementing the. segmentation technique as a plugin for the GIMP. By: Matthew Marsh Supervisors: Prof Shaun Bangay Mrs Adele Lobb. Problem statement:. The open source community require a quick, easy to use segmentation tool with matting capabilities. Aim:. - PowerPoint PPT Presentation
Citation preview
ImplementingImplementing thethe
By: Matthew MarshBy: Matthew Marsh
Supervisors: Prof Shaun Supervisors: Prof Shaun BangayBangay
Mrs Adele LobbMrs Adele Lobb
segmentation technique segmentation technique as a plugin for the GIMPas a plugin for the GIMP
Problem statement:Problem statement: The open source community require a quick, The open source community require a quick,
easy to use segmentation tool with matting easy to use segmentation tool with matting capabilities. capabilities.
Aim:Aim: The GIMP is an open source image editing The GIMP is an open source image editing
package that requires such a tool.package that requires such a tool. The GIMP allows extra tools to be added as The GIMP allows extra tools to be added as
pluginsplugins Implement a quick, easy to use Implement a quick, easy to use
segmentation tool with matting capabilities segmentation tool with matting capabilities as a plugin for the GIMPas a plugin for the GIMP
What is image What is image segmentation?segmentation?
The process of separating an image The process of separating an image into foreground and background into foreground and background parts.parts.
Foreground = object of interestForeground = object of interest Background = the rest of the imageBackground = the rest of the image
Image Segmentation Image Segmentation ContCont
Every pixel has an alpha value, which Every pixel has an alpha value, which sets its opacitysets its opacity
Image segmentation labels all foreground Image segmentation labels all foreground pixels with a full alpha value of 1,pixels with a full alpha value of 1,
Background pixels labelled with a value Background pixels labelled with a value of 0of 0
Can cause jagged edges in finely Can cause jagged edges in finely textured objectstextured objects
Matting smoothes these edgesMatting smoothes these edges
MattingMatting
Labels pixels with alpha values between the Labels pixels with alpha values between the maximum and minimum valuesmaximum and minimum values
Creates edges which vary smoothly from Creates edges which vary smoothly from background to foreground.background to foreground.
The image on the right is known as the alpha The image on the right is known as the alpha mattematte
How computers perform How computers perform segmentationsegmentation
Information encapsulated in the digital image Information encapsulated in the digital image is used to calculate the best segmentation. is used to calculate the best segmentation.
Edge detection and region detection methods Edge detection and region detection methods are most commonly used to segment images. are most commonly used to segment images.
Boundary information is the edge information Boundary information is the edge information in an image which can be gained by noting the in an image which can be gained by noting the difference between the colours of nearby difference between the colours of nearby pixels. A large difference in colour will pixels. A large difference in colour will indicate an edge. indicate an edge.
A region is the area enclosed by an edge, and A region is the area enclosed by an edge, and the colours of neighbouring pixels in a region the colours of neighbouring pixels in a region are similar. are similar.
““GrabCut”GrabCut”
Need to decide on a segmentation Need to decide on a segmentation techniquetechnique
““GrabCut” has been chosenGrabCut” has been chosen Uses both region and boundary Uses both region and boundary
informationinformation Uses new graph cut technique Uses new graph cut technique Includes a method of border mattingIncludes a method of border matting
How “GrabCut” WorksHow “GrabCut” Works
Clue MarkingClue Marking Clues are used to build up Clues are used to build up
region modelsregion models One model for foreground One model for foreground
and one for backgroundand one for background Region models determine Region models determine
weights between pixel and weights between pixel and Source and Sink nodesSource and Sink nodes
Types of region modelsTypes of region models
3 types of region models have been 3 types of region models have been developeddeveloped Greyscale histogramsGreyscale histograms Gaussian Mixture ModelsGaussian Mixture Models Colour histogramsColour histograms
Greyscale HistogramsGreyscale Histograms
Gaussian Mixture ModelsGaussian Mixture ModelsThe 1D Gaussian FunctionThe 1D Gaussian Function
The 2D Gaussian FunctionThe 2D Gaussian Function
3D Gaussians3D Gaussians A Gaussian Mixture A Gaussian Mixture
Model is the weighted Model is the weighted sum of several sum of several Gaussians.Gaussians.
Encapsulate the concept Encapsulate the concept of a colour of a colour
Each Gaussian is Each Gaussian is centred around a centred around a common colour in the common colour in the imageimage
A GMM has several A GMM has several common colourscommon colours
In this plug-in 5 In this plug-in 5 Gaussians in foreground, Gaussians in foreground, 5 in background5 in background
Colour HistogramsColour Histograms
The sum of three histogramsThe sum of three histograms One histogram for each colour channelOne histogram for each colour channel Thus a Red, Green and Blue histogram Thus a Red, Green and Blue histogram
in RGB colour spacein RGB colour space The probability of a pixel lying in the The probability of a pixel lying in the
foreground region model is the sum of foreground region model is the sum of the probabilities of its red value lying the probabilities of its red value lying in the red histogram, its blue in the in the red histogram, its blue in the blue etc.blue etc.
Matting ImplementationMatting Implementation
““GrabCut” adopts an approach of GrabCut” adopts an approach of border mattingborder matting
Assume a smooth alpha profileAssume a smooth alpha profile Defined by the sigmoid functionDefined by the sigmoid function
Border Matting WindowsBorder Matting Windows
Segmentation ResultsSegmentation Results Greyscale histograms proved to Greyscale histograms proved to
work wellwork well Implementation is fairly fastImplementation is fairly fast Results are goodResults are good
GMM’s proved to be slow due to the GMM’s proved to be slow due to the calculation of the initial 5 Gaussians calculation of the initial 5 Gaussians – also the initialization affected the – also the initialization affected the results substantially results substantially
Colour histograms also work well, Colour histograms also work well, and are basically just an extension and are basically just an extension onto greyscale histogramsonto greyscale histograms
Segmentation results Segmentation results contcont
Matting resultsMatting results
Input imageInput image
Matting significantly improved the Matting significantly improved the segmentation results in most cases.segmentation results in most cases.
Image with Image with mattingmatting
Image without Image without mattingmatting
ConclusionConclusion
Have completed my project goal – Have completed my project goal – provide open source community with provide open source community with a segmentation toola segmentation tool
Have implemented the “GrabCut” Have implemented the “GrabCut” paper and tested their claimspaper and tested their claims
The “GrabCut” segmentation The “GrabCut” segmentation technique is definitely usable, and technique is definitely usable, and produces good results.produces good results.