Basic Image Processing for Robotics Manual

8/10/2019 Basic Image Processing for Robotics Manual

1/103

Robot Practical

Basic Image Processing for Robotics

Maja Rudinac and Xin Wang


2/103

Table of contents

Table of contents................................................................................................................. ii

Foreword: ............................................................................................................................ v

Chapter 1: Introduction ................................................................................................... 21.1 Matlab ....................................................................................................................... 2

1.1.1 Matlab Introduction ........................................................................................... 2

1.1.2 Using Matlab Editor to Create M-file................................................................ 3

1.1.3 Getting Help....................................................................................................... 31.1.4 Starting with Image Processing Toolbox........................................................... 4

1.2 DIPimage .................................................................................................................. 5

1.2.1 DIPimage Introduction....................................................................................... 51.2.2 Getting Started ................................................................................................... 6

Chapter 2: Image representation and manipulation ..................................................... 92.1 Digital Image Representation ................................................................................... 9

2.1.1 Image Sampling and Quantization..................................................................... 9

2.1.2 Data Types and Image Types........................................................................... 122.2 Graphic Geometric Transformation........................................................................ 13

2.3 Basic Operations on Image Array........................................................................... 14

2.3.2 Array Operations.............................................................................................. 15

Chapter 3: Intensity Transformation............................................................................ 16

3.1 Contrast Manipulation ............................................................................................ 16

3.2 Histogram Processing ............................................................................................. 183.2.1 Histogram calculation ...................................................................................... 18

3.2.2 Histogram equalization .................................................................................... 193.3 Threshold ................................................................................................................ 19

Chapter 4: Spatial Filtering ........................................................................................... 224.1 Basic Ideas .............................................................................................................. 22

4.2 Smoothing (Blurring) Filters................................................................................... 24

4.3 Sharpening Filter..................................................................................................... 254.3 Edge Filter............................................................................................................... 25

4.4 Nonlinear filter........................................................................................................ 26

Chapter 5: Frequency based Processing....................................................................... 285.1 The Basics of Fourier Transform............................................................................ 28

5.2 Computing 2-D DFT............................................................................................... 29

5.3 Filtering in the Frequency Domain ......................................................................... 305.3.1 Fundamentals of Filtering in Frequency Domain ............................................ 30

5.3.2 Lowpass Frequency Domain Filters ................................................................ 31

5.3.3 Highpass Frequency Domain Filters................................................................ 32

ii


3/103

Chapter 6: Binary image processing ............................................................................. 336.1 Neighborhood relations........................................................................................... 33

6.2 Binary morphology ................................................................................................. 33

6.2.1 Structuring element.......................................................................................... 34

6.2.2 Erosion ............................................................................................................. 356.2.3 Dilation ............................................................................................................ 35

6.2.4 Opening and closing ........................................................................................ 36

6.2.5 Finding boundary pixels .................................................................................. 366.3 Morphological reconstruction................................................................................. 37

6.4 Gray scale morphology ........................................................................................... 40

6.4.1 Morphological smoothing................................................................................ 406.4.2 Morphological sharpening ............................................................................... 41

6.4.3 Compensating for non uniform illumination (Top-hat transformation) .......... 41

Chapter 7: Measurements .............................................................................................. 43

7.1 Selecting objects ..................................................................................................... 437.2 Measuring in binary images.................................................................................... 44

7.2.1 Area.................................................................................................................. 457.2.2 Perimeter .......................................................................................................... 46

7.2.3 Centroid (Centre of mass)................................................................................ 47

7.2.4 Euler number.................................................................................................... 477.3 Errors Introduced by Binarization .......................................................................... 48

7.4 Measuring in Gray-Value images ........................................................................... 48

7.4.1 Thresholding .................................................................................................... 487.4.2 Calculating Area, Perimeter and Euler number (in DipImage) ....................... 49

7.5 Linking measurements to real world....................................................................... 507.5.1 Pinhole camera model...................................................................................... 50

7.5.2 Thin Lens law .................................................................................................. 51

7.5.3 Calculating real size of the object .................................................................... 52

Chapter 8: Color Image Processing .............................................................................. 53

8.1 RGB color model .................................................................................................... 53

8.2 HSV (HSI) color model .......................................................................................... 558.3 Conversion between different color models ........................................................... 56

8.4 Spatial Filtering of color images............................................................................. 58

8.4.1 Color smoothing............................................................................................... 588.4.2 Color sharpening.............................................................................................. 58

8.4.3 Motion blur in color images............................................................................. 59

8.4.4 Color edge detection ........................................................................................ 608.4.5 Color morphology............................................................................................ 60

8.5 Segmentation of color images................................................................................. 61

Chapter 9: Advanced topics ........................................................................................... 63

9.1 Scale-Spaces ........................................................................................................... 63

9.2 Hough Transform.................................................................................................... 66

iii


4/103

9.3 Mean Shift............................................................................................................... 69

Chapter 10: Image Segmentation .................................................................................. 73

10.1 Point line and edge detection ................................................................................ 74

10.1.1 Detection of isolated points ........................................................................... 74

10.1.2 Line detection................................................................................................. 7510.1.3 Edge detection................................................................................................ 78

10.2 Extracting corners and blobs................................................................................. 80

10.2.1 Corners........................................................................................................... 8110.2.2 Blobs .............................................................................................................. 83

10.3 Clustering methods in segmentation..................................................................... 84

Chapter 11: Image description (Advanced) ................................................................. 88

11.1 Global descriptors ................................................................................................. 89

11.1.1 Color descriptors............................................................................................ 8911.1.2 Texture descriptors......................................................................................... 90

11.1.3 Shape descriptors ........................................................................................... 9211.2 Local descriptors................................................................................................... 95

11.2.1 SIFT Detector................................................................................................. 9511.2.2 SIFT Descriptor ............................................................................................. 96

11.2.3 SIFT Matching............................................................................................... 97

iv


5/103

Foreword:

This manual is part of the Robot Practicals lab course. In the Robotics Practicals, the aim

is to familiarize yourself with the software tools that are necessary to program the robots

at the Delft Biorobotics Lab (DBL). The lab courses are very much hands-on; you will beable to practice everything you learn immediately by doing the exercises that are

contained in the text. Because the aim is to understand the tooling instead of a particular

application, we specially encourage you to explore and try to combine the things you'velearned in new and hopefully exciting ways!

In this practicum we aim to explain basics of Image processing required for solvingsimple problems in robotic vision and to give you basic knowledge for reading state of

the art vision books and papers. For more advanced explanations please refer to the

given literature. All exercises are given in Matlab so basic understanding of Matlabenvironment is prerequisite for this practical.

There is no exam at the end of the course, but student is required to solve all the

assignments within practical and to come to the instructors to discuss the given solutions.Pay attention that some parts of the Practicum are referred as advanced. We advise you

to read these parts and to solve assignments only if you would like deeper understanding

of the topic.

If you have any problems to understand parts of the Practicum after checking the givenliterature, please contact the authors.

We wish you a nice work!

Authors:Maja Rudinac [email protected]

Xin Wang [email protected]

Delft, 2010

v
mailto:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]:[email protected]


6/103

Chapter 1Introduction

Chapter 1: Introduction

Image processing is an area that needs a lot of experimental work to establish the

proposed solution to a given problem. The main goal of this Practicum is to get hands-on

experience with image processing. To do so, you will have to learn the image processing

environment: Matlab and the DIPimage toolbox for scientific image processing. Tofacilitate a quick start, there will not be an in-depth explanation of all the features in this

software. We will briefly present the things as you need them.

In this Practicum, the details of image processing theory are left out. For these please

refer to:

Rafael C. Gonzalez, Richard E. Woods, Digital Image Processing, the secondversion, Prentice Hall, 2002

Rafael C. Gonzalez, Richard E. Woods and Steven L. Eddins, Digital ImageProcessing using MATLAB, the second version, Prentice Hall, 2002

Young, I.T., Gerbrands, J.J. and van Vliet, L.J., The Fundamentals of ImageProcessing, Department of Applied Physics, TUD. It is available online athttp://www.tnw.tudelft.nl/live/pagina.jsp?id=223d8f77-7ef1-4d41-b336

28ae2f9a83ef&lang=en

1.1 Matlab

This subsection serves to get yourself acquainted yourself with Matlab so if you are

already familiar with Matlab, skip this section. If you have never worked with Matlabbefore, it is necessary to check this section and related materials.

Matlab is an abbreviation of Matrix Laboratory. Itis a numerical computingenvironmentand fourth-generation programming languagethat allows matrixmanipulations, plotting

of functions and data, implementation of algorithms, creation of user interfaces, andinterfacing with programs written in other languages, including C, C++, and Fortran.

1.1.1 Matlab Introduction

First, let us have a look at Matlab desktop shown in Fig. 1.1. The desktop is the main

Matlab application window and it contains five sub windows: the Command Window, theWorkspace Browser, the Current Directory Window, the Command History Window, and

one or more Figure Windows which are used to display graphics. Their functions are

following:

The Command Window is the place that user can type in Matlab commands at theprompt (>>) and where the outputs of those commands are displayed.

The workspace stores the set of variables that user creates during a work session. The Command History Window contains a record of the commands a user had

entered in the Command Window. We can re-execute history commands by right-

clicking on commands from Command History Window.

2
http://www.tnw.tudelft.nl/live/pagina.jsp?id=223d8f77-7ef1-4d41-b336http://en.wikipedia.org/wiki/Numerical_analysishttp://en.wikipedia.org/wiki/Fourth-generation_programming_languagehttp://en.wikipedia.org/wiki/Matrix_%28mathematics%29http://en.wikipedia.org/wiki/Function_%28mathematics%29http://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/User_interfacehttp://en.wikipedia.org/wiki/C_%28programming_language%29http://en.wikipedia.org/wiki/C%2B%2Bhttp://en.wikipedia.org/wiki/Fortranhttp://en.wikipedia.org/wiki/Fortranhttp://en.wikipedia.org/wiki/C%2B%2Bhttp://en.wikipedia.org/wiki/C_%28programming_language%29http://en.wikipedia.org/wiki/User_interfacehttp://en.wikipedia.org/wiki/Algorithmhttp://en.wikipedia.org/wiki/Function_%28mathematics%29http://en.wikipedia.org/wiki/Matrix_%28mathematics%29http://en.wikipedia.org/wiki/Fourth-generation_programming_languagehttp://en.wikipedia.org/wiki/Numerical_analysishttp://www.tnw.tudelft.nl/live/pagina.jsp?id=223d8f77-7ef1-4d41-b336


7/103


The Current Directory Window shows the contents of the current directory.

Fig 1.1 Desktop of Matlab

Command

history

WorkspaceCommandwindow

Currentfolder

Any file that runs in Matlab must reside in the current directory or in a directory that is on

the search path. The way to check which directories are on search path, or to add or

modify a search path, is to select Set Pathfrom Filemenu on the desktop, and then use

Set Pathdialog box.

1.1.2 Using Matlab Editor to Create M-file

An m-file is a simple text file in which you can write Matlab commands and save for later

use. When the file is running, Matlab reads the commands and executes them exactly as it

would if you had typed each command sequentially at the Matlab prompt. All m-filenames must end with the extension '.m' (e.g. imageprocessing.m).

To create an m-file, chooseNewfromFilemenu and select m-file. This procedure brings

up a text editor window in which you can enter Matlab commands. Or simply type editat

the prompt in Command Window to create a new m-file. To save the m-file, go to the

File menu and choose Save (remember to save it with the '.m' extension). To open anexisting m-file, go to theFilemenu and choose Openand select .m file that you want.

1.1.3 Getting Help

The principle way to get help online is to use Matlab Help Browser by clicking on

Question marksymbol on desktop toolbar, or by typing helpbrowserat the prompt in theCommand Window. For example, help on a specific function is obtained by selecting the

3


8/103


Search tab, selectingFunction Nameas search type, and then typing in the function name

in the search field. Another way to obtain help for a specific function is by typing docfollowed by the function name at the command prompt.

M-functions have two types of information that can be displayed by user. One is H1line

which contains the function name and one-line description and anther one is Help textblock. Typing in helpat the prompt followed by a function name displays both the H1

line and the help text for that function. If type in look forfollowed by a keyword at the

prompt, we can have all the H1 lines that contain that keyword. It is especially usefulwhen looking for a particular topic without knowing the name of the function.

1.1.4 Starting with Image Processing Toolbox

The Image Processing Toolbox software is a collection of functions that extend the

capability of the MATLAB numeric computing environment for the purpose of imageprocessing.

Before we start, it is better to clear variables and figures before commands are executed.

This avoids undetected errors.

>>clear all>>clc

Read and Display an image (Matlab)

Here, we give an example that how to use the functions in Image Processing Toolbox to

read and display images. Matlab uses function imreadto read images, whose syntax is:

I=imread(filename)

Filename is a string containing the complete name of the image file (including anyapplicable extension). And Table 1.1 gives some of images/graphics formats supported

by imreadin Matlab.

Table 1.1 image formats in Matlab

Format name Description Recognized extension

TIFF Tagged image file format .tif,.tiffJPEG Joint photographic experts group .jpg, .jpeg

GIF Graphics interchange format .gif

BMP Windows bitmap .bmp

PNG Portable network graphics .pngXWD X window dump .xwd

To display images, Matlab uses function imshow:

imshow(I, p)

4


9/103


I is an image array, and p is the number of intensity levels used to display it. If p is

omitted, it defaults to 256 levels.

>>I=imread('football.png');>>imshow(I);

Fig 1.2 Football

In order to get additional information from an image we use following functions:size(I) ; whos(I)

Example 1.2 Obtain additional information from an image (Matlab)

>>size(I)ans = 280 280 3

>>whos IName Size Bytes ClassAttributesI 280x280x3 235200 uint8

As shown in Command Window, sizefunction gives the row and column dimensions as

well as depth of an image( 1 for grayscale images, 3 for color images), and whosdisplays

more information about the image array such as bytes and data type.

1.2 DIPimage

1.2.1 DIPimage Introduction

DIPimage is the additional toolbox which is used under MATLAB to do image

processing. At this time, we will review only the most relevant features. You will get toknow this environment better throughout the course.

The toolbox can be downloaded from: http://www.diplib.org/download

And the DIPimage user guide from: http://www.diplib.org/documentationBy typing in command dipimage in Command Window in Matlab, we can enter the

DIPimage environment.

5


10/103


On the top-left corner of Matlab the DIPimage GUI (Graphical User Interface) will

appear, same as one shown in Fig 1.3. The other 6 windows are used to display images.The GUI contains a menu bar. Spend some time exploring the menus. When you choose

one of the options, the area beneath the menu bar changes into a dialog box that allows

you to enter the parameters for the function you have chosen. Most of these functions

correspond to image processing tasks, as we will see later.

Fig 1.3 GUI of DIPimage

There are two ways of using the functions in this toolbox. The first one is through theGUI, which makes it easy to select parameters and functions. We will be using it very

often at first, but gradually less during the course. The other method is through the

command line.

When solving the problems in this laboratory course, we will be making text files that

contain all commands we used to get to the result. This makes the results reproducible. It

will also avoid lots of repetitive and tedious work. We recommend you make a new filefor each exercise, and give them names that are easy to recognize. The file names should

end in .m, and should not conflict with existing function or variable names.

Similar to Matlab, if you start each file with the commands bellow the variables and the

figure windows will be cleared before your commands are executed. This avoids

undetected errors.

>>clear>>dipclf

1.2.2 Getting Started

Read and display an image (DIPimage)

We need to load an image (from file) into a variable before we can do any imageprocessing. The left-most menu in the GUI is called File I/O, and its first item Read

Image (readim). Select it. Press the Browsebutton, and choose the file twocells.tif .

Now press the Execute button.

6


11/103


Fig 1.4 Two cells

Two things should happen:

1. The image twocells.tif is loaded into the variable image_out, and displayed to a

figure window (see Figure 1.4).

2. The following lines (or something similar) appear in the command window:

image_out = readim('twocells.tif')Displayed in figure 15

This is to show you exactly the same would have happened if you had typed that

command directly in the command window.

Note that if we omitted the extension to the filename, readimcan still find the file without

specifying the file type. We also didnt specify the second argument to the readimfunction. Finally, by not specifying a full path to the file, we asked the function to lookfor it either in the current directory or in the default image directory.

>>image_out = readim(body1);

Note that the contents of variable image_outchanges, but the display is not updated.To update the display, simply type:

>>image_out

You will have noticed that the image in variable image_outis always displayed in thetop-left window. This window is linked to that variable. Images in all the othervariables are displayed to the sixth window. This can be changed, see the DIPimage

manual for instructions.

A grey-value image is displayed by mapping each pixels value in some way to one of the

256 grey-levels supported by the display(ranging from black (0) to white (255)). By

default, pixel values are rounded, negative values being mapped to 0, and values largerthan 255 to 255. This behavior can be changed through the Mappingsmenu on each

7


12/103


figure window. Note that these mappings only change the way an image is displayed, not

the image data itself.

Another menu on the figure window, Actions, contains some tools for interactive

exploration of an image. The two tools we will be using are Pixel testing and Zoom.

Pixel testing allows you to click on a pixel in the image (hold down the button) toexamine the pixel value and coordinates. Note that you can drag the mouse over the

image to examine other pixels as well. The right mouse button does the same thing as the

left, but changes the origin of the coordinate system and also shows the distance to theselected origin (right-click and drag to see this).

The Zoom tool is to enlarge a portion of the image to see it more clearly.

Assignment 1.1:

Create an M file with the function called ImBasic, which has read, display, save and writeoperations. Read the image boy.bmp into Matlab environment, display it on the screen,

check its additional parameters and finally save it in your assignment directory by nameboy_assignment and jpg extension.

8


13/103

Chapter 2Image representation and manipulation

Chapter 2: Image representation and manipulation

2.1 Digital Image Representation

An image can be defined as a two-dimensional function, f(x, y), where x and y are spatial(plane) coordinates, and the amplitude of f at any pair of coordinates (x, y) is called

intensity.

2.1.1 Image Sampling and Quantization

An image is continuous with respect to the x and y coordinates as well as the amplitude.For computer application, we need to convert such an image to a digital form. Digitizing

the coordinate values is called sampling, and digitizing the amplitude is called

quantization. Since x, y and intensity are all finite values we call this image a digitalimage. The whole process is shown in Fig 2.1.

Fig 2.1 Sampling and quantization process

Fig 2.2 Image sampling

As shown in Fig 2.2, we take a monochrome image as an example. Image sampling is a

sampling set of points from 2-D grid which transforms infinite points on an image into

9


14/103


finite points on a digital image. And image quantization uses a set of grey levels to

sample the infinite amplitude into finite grey values. Then we can get image arrayF (i, j):

0,0 0,1 0, 1

1,0 1,1 1, 1

1,0 1,1 1, 1

(0,0) (0,1) (0, 1)

(1,0) (1,1) (1, 1)( , )

( 1,0) ( 1,1) ( 1, 1)

N

N

M M M N

a a a f f f N

a a a f f f NA F i j

a a a f M f M f M N

= =

Each element is this array is called image element, picture element or pixel. A digital

image can be represented as Matlab matrix

(1,1) (1,2) (1, )

(2,1) (2,2) (2, )( , )

( ,1) ( ,2) ( , )

f f f N

f f f Nf i j

f M f M f M N

=

Note that Matlab is different from C and C++ in the way that its first index is from (1, 1)instead of (0, 0). Notice this when programming.

Example 2.1 Downsample an image by factor 2 then upsample by factor 2 (Matlab)

>>img=imread('football.png');>>figure>>imshow(uint8(img));>>title('orginal image');

>>subimg = imresize(img, 0.5);>>figure>>imshow(uint8(subimg))>>title('first subsampling');

>>subsubimg = imresize(subimg, 0.5);>>figure>>imshow(uint8(subsubimg))>>title('second sampling');

>>upimg=imresize(subsubimg,2);>>figure>>imshow(uint8(upimg));>>title('first upsampling');

>>upupimg=imresize(upimg,2);>>figure

>>imshow(uint8(upupimg))>>title('second upsampling');

The resulting images are shown in Figure 2.3 and Figure 2.4. We can see if we first

downsample an image and than upsample the same image to its original size the image

will lose some information. It shows the importance of spatial resolution.

10


15/103


Figure 2.3 Dowsample an image by factor 2

Figure 2.4 Upsample an image by factor 2

Example 2.2 Downsample the amplitude resolution in an image by factor 2 (DIPimage)

>>image_out = readim('twocells.tif','')>>sub1 = subsample(image_out,2)

>>sub2 = subsample(sub1,2)

And the results of reducing grey level can be seen in Figure 2.5.

256 levels 32 levels 16 levels

Figure 2.5 Reducing amplitude resolution

Assignment 2.1:Write your own m file with downsamplefunction without using any functions provided

by Matlab. Read the image kickoff.jpg into Matlab environment, useforor whileloopsto downsample this image and then use imshowto display the final result.

11


16/103


2.1.2 Data Types and Image Types

Data types and conversion

The values of pixels are not restricted to be integer in Matlab. Table 2.1 lists the data

types supported by Matlab.All numeric computations in Matlab are done using double data type. unit8 is also

encountered frequently, especially when reading data from storage devices, as the 8-bit

images are the most common representations in practice.

There are two different ways to convert between data types. One is very straightforward

and has the general syntax

B = data_type_name(A)

And another one uses the functions in Table 2.2.

Table 2.1 Data types in Matlab

Name Description

double Double-precision, floating number in the range 308 30810 10 unit8 Unsigned 8-bit integer in the range [0, 255] (1 byte per element)

unit16 Unsigned 16-bit integer in the range [0, 65536] (2 bytes per element)

unit32 Unsigned 32-bit integer in the range [0, 4294967295] (4 bytes per element)int8 Signed 8-bit integer in the range [-128, 127] (1 byte per element)

int16 Signed 16-bit integer in the range [-32768, 32767] (2 bytes per element)

int32Signed 32-bit integer in the range [-2147483648, 2147483647] (4 bytes perelement)

single Single-precision floating-point number rang (4 bytes perelement)

38 38

10 10

char Characters (2 bytes per element)logical Values are 0 or 1 (1 byte per element)

Table 2.2 data types converting function

Name Converts input to Valid input data types

im2unit8 uint8 logical, unit8, unit16 and double

im2unit16 unit16 logical, unit8, unit16 and double

mat2grey double( range [0,1] ) doubleim2double double logical, unit8, unit16 and double

im2bw logical unit8, unit16 and double

Note that the rounding behavior of im2unit8is different from data_type_nameconversion

function unit8, which simply discards fractional parts. im2unit8sets all the values in input

array less than 0 to 0, and greater than 1 to 1, and multiplies all the other values by 255.Rounding the results of the multiplication to the nearest integer, then the conversion is

completed.

12


17/103


Image types

The toolbox supports four types of images: Intensity images, binary images, indexed

images and RGB images.

It is worth mentioning, a binary image is a logical array of 0s or 1s. A numeric array isconverted to binary using function logical.

B = logical(A)

2.2 Graphic Geometric Transformation

Basic graphic geometric transformations contain scalar, rotation, mirroring, shiftingwhich are useful for image preprocessing.

There are two main methods in Matlab for geometric transformations:

1. Use figure window menu toolswhich has limit applications like zoom in and zoomout.

2. Use Matlab transformations commands and they are more flexible for incorporatinginto existing code.

Example 2.3 Image rotation (Matlab)

>>I = imread('football.png');>>J = imrotate(I,-50,'bilinear','crop');>>figure, imshow(I)>>figure, imshow(J)

The result is shown in Figure 2.6.

Figure 2.6 Image rotation

In DIPimage environment, it is very easy to implement graphic geometrictransformations. Go toManipulationmenu in GUI, and choose the exact transformation

that you want, execute, and then you can see the results. Here we give an example about

image mirror based on X axis. The result is shown in Figure 2.7.

Example 2.4 Image Mirror based on X axis (DIPimage)>>image_out = readim('bear.tif','')

13


18/103


>> out = mirror(image_out,'x-axis',1)

Fig 2.7 Image mirror (DIPimage)

2.3 Basic Operations on Image Array

2.3.1 Image Operation

The basic image array operations are listed in Table 2.3.

The basic image array operations are very useful, i.e. imaddfunction can be used to add

noise to an image, and imsubstractcan be used to detect the motion in a video sequence.

We now give an example about how to use basic image array operations:

Table 2.3 Basic image array operations

function description

imadd Add two images or add a constant to an image

imsubtractSubtract from one image from another image or subtract a constant froman image

immultiply Multiply two images by pixel value or multiply an image by a constant

imdividedivide one image from another image by pixel value or divide an imageby a constant

imabsdiff Calculate the absolute difference about two images

imcomplement Calculate the complement of an imageimlincomb Calculate the line combination of two image by pixel value

Example 2.5: Use imsubtractto detect motion (Matlab).

Here we assume that the background is unchangeable.

>>figure

>>I1 = imread('moving face1.bmp');>>imshow(I1);>>title('video image 1');>>I2 = imread('moving face2.bmp');>>figure>>imshow(I2);>>title('video image 2');>>I3 = imsubtract(double(I2),double(I1));>>figure>>imagesc(I3);>>title('subtraction');

14


19/103


video image 1

video image 2

substraction

50 100 150

200

250

300 350

50

100

150

200

250

Figure 2.8 Image subtraction for motion detection

Assignment 2.2:In Graphic Geometric Transformation, we explained image rotation and image mirror.

Try to work out other GGT applications like image cropping and image scaling as well as

image shifting, using the image lena.tif.

2.3.2 Array Operations

Array operations mainly consist of Matlab indexing operation which is very useful in

array manipulation and program optimization. Please check related literature to check

how to use array index. Also, it is useful to generate some useful and simple image arrays

to try out ideas.

zeros (M, N)generates an N matrix of 0s of double type data. ones (M, N)generates an N matrix of 1s of double type data. true (M, N)generates an N logical matrix of 1s.

false (M, N)generates an N logical matrix of 0s. magic (M, M)generates an M magic square in which the sum along any row,

column or main diagonal is the same.

rand (M, N)generates an N matrix whose entries are uniformly distributedrandom numbers are integers.

randn (M, N)generates an N matrix whose numbers are normally distributedrandom number with mean 0 and variance 1

Assignment 2.3:

Use basic array operations to add image noise to an image region. Read the imagedrop.tif, and use array indexing, imadd and randn function to add noise only to the

drop region.

15


20/103

Chapter 3Intensity Transformation

Chapter 3: Intensity Transformation

In this chapter, most of the examples are related to image enhancement which is usually

the fist step for image processing. And intensity transformations are conducted in spatial

domain which means the operations are directly on the pixels of images.

The spatial domain processes are denoted by expression:

[ ]( , ) ( , )g x y T f x y=

Where f(x, y) is the input image, g(x, y) is the output image, and T is an operator on f,

defined over a special neighbored about point (x, y)as shown in Figure 3.1.

Figure 3.1 Spatial domain based image operation

The principle approach for defining spatial neighborhoods around a point (x, y)is to use asquare or rectangular region centered at (x, y)which is usually called window or mask.

Only the pixels in the neighborhood are used in computing the value ofgat (x, y).

There are two ways to perform intensity transformation: contrast manipulation and

histogram modification.

3.1 Contrast Manipulation

There are lots of ways to change contrast of an image. Some common operations are:

linear scaling and clipping power-law logarithmic transformation reverse function inverse function

In Matlab, imadjustfunction is used for intensity transformation of grey-scale images:

g = imadjust (f, [low_in high_in], [low_out high_out], gamma)

16


21/103


The values between low_inand high_inmap into values between low_outand high_out.

Values below low_inand above high_inare clipped. The input image can be unit8, unit16or double, and the output image has the same data type as the input. The parameter

gamma determines the shape of the curve that map intensity values in f to create g. If

gammais less than 1, the mapping is weighted toward higher (brighter) output values. If

gammais greater than 1, the mapping is weighted toward lower (darker) output values.

Example 3.1 Intensity transformation (Matlab)

>> f = imread('mary.bmp');>> figure>> imshow(f)>> g1 = imadjust(f,[],[],0.5);>> figure>> imshow(g1)>> g2 = imadjust(f,[],[],2);>> figure>> imshow(g2)>> g3 = imadjust(f,[0 1],[1 0]);

>> figure>> imshow(g3)

The results are shown in Figure 3.2.

Figure 3.2 Intensity transformation

We also give an example in DIPimage. In DIPimage GUI environment, thetransformation is given in menu partPoint.

Example 3.2 Intensity stretching (DIPimage)

>> image_out = readim(blood.tif,'')>> a = stretch(image_out,0,100,0,255)

Assignment 4.1:

Use logarithmic transformation to enhance the dark image qdna.tif. Thetransformation function is:

log(1 )g c f= + Assignment 4.2:

Use power-law transformation to enhance the light image sca.tif. The transformation

function is:

g cf=

17


22/103


3.2 Histogram Processing

Histograms are widely used in image processing for operations such as: enhancement,

compression, segmentation and etc.

3.2.1 Histogram calculation

For an image with gray levels in [0, L-1] and N pixels, the histogram is a discretefunction given by

( )k kh r n=

Parameter is the intensity value, and is the number of pixels in the image with

intensity .

krth

k kn

kr

It is a common practice to normalize the histogram function by the number of pixels inthe image:

( ) kk np rN=

The normalized histogram can be used as an estimate of the probability density function

of the image. For enhancement, histograms can be used to infer the type of image quality:dark, bright, low or high contrast.

Example 3.2 Histogram calculation

>>h1 = imhist(f);>>h1_plot = h1(1:5:256);>>horz = 1:5 :256;

>>bar(horz, h1_plot)>>axis([0 255 0 500])>>set(gca, 'xtick',0:10:255)>>set(gca,'ytick',0:20:1000)>>h1 = imhist(f);>>h1_plot = h1(1:5:256);>>horz = 1:5 :256;>>bar(horz, h1_plot)>>axis([0 255 0 3800])>>set(gca, 'xtick',0:10:255)>>set(gca,'ytick',0:200:3800)

Resulting histograms of images from Figure 3.2 are shown in Figure 3.3

Figure 3.3 Histogram calculation

18


23/103


As we can see from Figure 3.3, if the histogram distribution is centered in low part, the

image will be darker, on the contrast, if the histogram distribution is centered in high part,the image will be brighter.

In DIPimage also to calculate histogram following function should be used. In menu

Point, chooseHistogram (diphist) to get the histogram.>>a = stretch(image_out,0,100,0,255)>>diphist(a,[0 255],256,'bar')

3.2.2 Histogram equalization

It is quite acceptable that high contrast images have flat histograms (uniform distribution)

Histogram equalization attempts to transform the original histogram into a flat one for thegoal of a better contrast.

Histogram equalization is implemented in the Image processing toolbox by function

histeq, which has the syntax: g = histeq(f, nlev)

Parameterfis the input image and nlevis the number of intensity levels that is specified

for output image. If the nlevis equal toL(the total number of possible levels in the input

image), histeq directly implements the transformation function. If nlev is less than L,

histeqattempts to distribute the levels so that they will approximate a flat histogram.

Histogram equalization produces a transformation function that is adaptive, in the sense

that it is based on the histogram of a given image. However, once the transformationfunction of an image has been computed, it does not change unless the histogram of the

image changes. But image equalization which spread the levels of the input image over a

wider range of intensity scale can not always enhance image. In particular, it is useful insome applications to be able to specify the shape of the histogram that we wish the

processed image to have. This method is used to generate an image that has a specifiedhistogram which is called histogram matching or histogram specification. In Matlab, we

also use histeqto implement histogram matching, which has the syntax:

g = histeq(f, hspec)

Parameterfis the input image, hspecis the specified histogram, andgis the output image

whose histogram approximates the specified histogram.

3.3 Threshold

Threshold technique is widely used in image segmentation to segment images based on

intensity levels.

Let us now have a look at the histogram in Figure 3.4 which is corresponding to theimage named coins, composed of light objects on a dark background, in such a way

that object and background pixels have intensity levels grouped into two dominant

19


24/103


modes. One obvious way to extract the objects from the background is to select a

threshold T that separates these modes. Threshold function can be described as:

1 ( , )( , )

0 ( , )

if x y T g x y

if x y T

=

f = imread('cameraman.tif'); subplot(1,3,1);>>imshow(f); title('original image');>>subplot(1,3,2);>>imhist(f);>>t1 = 90; % read the value of valley from histogram>>[m n]=size(f);>>f1=zeros(m,n);>>for i=1:m

for j=1:nif f(i,j)>t1f1(i,j)=1;

elsef1(i,j)=0;

endend

end>>subplot(1,3,3);>>imshow(f1);>>title(thresholded image)

20


25/103


Figure 3.6 Result of Image thresholding

DIPimage toolbox has implemented several functions that can do thresholding. At first

we should choose an image in which the objects are clearly defined and are easy tosegment. The Segmentation menu contains the function threshold, which assigns

object or background to each pixel depending on its grey-value. We dont need themfor this image. Select fixed for the Type parameter; this requires you to provide a

threshold. Another solution is to do the thresholding in an alternative way: using

relational operators. Recall that thresholding returns an image with true, where theimage is larger than some threshold. This can be accomplished with the larger than (>)

operator. The result of image thresholding is in Figure 3.7.

Figure 3.7 Result of Image thresholding displayed in DIPimage

Assignment 4.3Combine histogram operation with thresholding method to segment an object from its

background. Read the image cookies.tif into Matlab, use imhistto get the histogram ofthis image, read from the histogram best thresholding value and use this to threshold the

image. Output the final result using function imshow. Try to use subplot function todisplay the original image and processed image in the same figure.

21


26/103

Chapter 4Spatial Filtering

Chapter 4: Spatial Filtering

There are two main types of filtering applied to images: spatial domain filtering and

frequency domain filtering. The frequency domain filtering will be discussed in Chapter

5. This chapter is mainly about spatial filtering.

4.1 Basic Ideas

Spatial Filtering is sometimes also known as neighborhood processing. Neighborhood

processing is a method that defines a center point and performs an operation (or apply a

filter) to only those pixels in predetermined neighborhood of that center point. The resultof the operation is one value, which becomes the value at the center point's location in the

modified image. Each point in the image is processed with its neighbors. The general idea

is shown below as a "sliding filter" that moves throughout the image to calculate thevalue at the center location. The main process follows steps bellow:

1. Defining a center point, (x, y);2. Performing an operation that involves only pixels in a predefined neighborhood around

the center point;

3. Letting the result of that operation be the response of the process at that point;4. Repeating the process for every point in the image. The process is shown in Fig 4.1.

Fig 4.1 Neighborhood processing

There are two types of filters: linear and non-linear filters. Linear filters can be

implemented through a convolution, whereas non-linear filters can not. Therefore, linear

filters are easier to implement, and are important for Fourier analysis. All filters discussedin this section (both linear and non-linear) can be implemented as a combination of the

values in a neighborhood; that is, each pixel in the output image is computed out of a

fixed set of pixels in the neighborhood of the corresponding pixel in the input image.

22


27/103


Typical neighborhood shapes are round or square. We call this neighborhood operation

kernel or mask and it is very important in image filtering. In Table 4.1 we can finddifferent kernels.

Table 4.1 Filter kernelsFilter type Filter Sample 3 3 kernels Description

Averaging

a a a

a b a

a a a

0 0

0 0

a

a b a

a

Smoothing, noise reduction

and blurring filter(focal mean)

Low-pass

Gaussian

c a c

a b a

c a c

Smoothing, noise reduction

and blurring filter(focal weightmean)

High-pass Sharpening

a a a

a b a

a a a

0

0 0

a

a b a

a

0

Mean effect

removal/sharpening filter(focal sum). Provides limited

edge detection. Typically

entries sum to 1 but may be

greater. 3 laplacian kernelstypical add to 1. Large

laplacian kernels(e.g. 7

3

7 )may be more complex and sum

to >1

Edgedetection

0 0 0a b a

a b a

00

0

a ab b

a a

Applied singly or as two-pass

process. These kernels

highlight vertical andhorizontal edges. When used in

combination they are known as

Gradient or Order 1 derivativefilters. Typically a = 1 and b =

1 or 2 and entries sum to 0.

Embossing

0

0

a a

a a a

a a

Edge detecting filters that

enhance edges in a selectedcompass direction to provide

an embossed effect. Theexample here shows a sample

north-east kernel

Gradient

Directional1 1 1

1 2 1

1 1 1

1 1 1

1 2 1

1 1 1

Simply computation ofgradient in one of 8 compass

direction, east and north

directional derivative areillustrated in the first two

examples here.

23


28/103


4.2 Smoothing (Blurring) Filters

MATLAB:

Smoothing filters are used to smooth (blur) edges and noise in the images. In Matlab, it isbest to use imfilter to generate smoothing images. Firstly the smoothing mask should be

defined and afterwards applied on the image to smooth it.

Example 4.1 Apply average filter to an image (Matlab)

>>w =[1/9 1/9 1/91/9 1/9 1/91/9 1/9 1/9];

>>car =imread('car.bmp');>>figure>>imshow(car)>>g =imfilter(car, w);>>figure>>imshow(g)

Figure 4.1 Average filteringDIPimage

Go toFiltersmenu of the GUI and there you will find a lot of filters. Let us take onesmoothing filter as example.

Example 4.2 Apply Gaussian filter to image (DIPimage)SelectGaussian Filter (gaussf)in the menu of the GUI. We need to choose the size of theGaussian filter: the standard deviation in pixels. Try out different values for it, and see

what happens. The filter size (scale) is very important, as will be shown in Figure 4.2.

>>gaussian1 = gaussf(image_out,1,'best')>>gaussian1 = gaussf(image_out,2,'best')>> gaussian1 = gaussf(image_out,3,'best')>> gaussian1 = gaussf(image_out,4,'best')

Figure 4.2 Gaussian filter

24


29/103


4.3 Sharpening Filter

Sharpening filters are used to enhance the edges of objects and adjust the contrast and the

shade characteristics. In combination with threshold they can be used as edge detectors.Sharpening or high-pass filters let high frequencies pass and reduce the lower frequencies

and are extremely sensitive to shut noise. To construct a high-pass filter the kernelcoefficients should be set positive near the center of the kernel and in the outer peripherynegative.

MATLAB:

We will now try to use another Matlab functionfspecialto generate sharpening filter. We

take Laplace filter as example which can enhance discontinuities. First we use negative

Laplace operation to apply on the image, and afterwards we subtract negative Laplaceimage from original to get sharpening image. The result is shown in Figure 4.3.

Example 4.3 Sharpening image (Matlab)

>>f = imread('kickoff_grayscale.jpg');>>w = fspecial('laplacian',0);>>g =imfilter(f, w, 'replicate');>>g = f - g;>>imshow(g)

a) Original image b) Laplace image c) Sharpening imageFigure 4.3 Sharpening Image

DIPimage

The menu item Differential Filters in filters menu contains a general derivative filter, acomplete set of first and second order derivatives, and some combinations like gradient

magnitude and Laplace filters. Let us try to use Laplace filter in DIPimage.

Example 4.4 Apply Laplace filter to an image (DIPimage)

>>image_out = readim('schema.tif','')>>g = laplace(image_out, 1)

4.3 Edge Filter

Edges characterize boundaries and are therefore a problem of fundamental importance in

image processing. Edges in images are areas with strong intensity contrasts. Edge

25


30/103


detecting significantly reduces the amount of useless information, while preserving theimportant structural properties in an image. The majority of different methods are

grouped into two categories, gradient and Laplacian. The gradient method detects the

edges by looking for the maximum and minimum in the first derivative of the image. TheLaplacian method searches for zero crossings in the second derivative of the image to

find edges. More will be explained in Chapter 10, in edge segmentation section.

4.4 Nonlinear filter

A nonlinear filter is the filter whose output is a nonlinear function of the input. Bydefinition, any filter that is not a linear filter is a nonlinear filter. One practical reason to

use nonlinear filters instead of linear filters is that linear filters may be too sensitive to asmall fraction of anomalously large observations at the input. One of the most commonly

used nonlinear filters is the median filter which is very effective to reduce salt-and-pepper

noise in an image.

MATLAB:

There are two different ways to generate median filter, one usingordfilt2function whileanother using medianfunction directly. The result is shown in Figure 4.4.

Example 4.5 Apply median filter to an image (Matlab)

>>f = imread('kickoff_grayscale.jpg');>>fnoise = imnoise(f, 'salt & pepper', 0.2);>>figure>>imshow(fnoise)>>g = medfilt2(fnoise);>>figure

>>imshow(g);

Figure 4.4 Median filtering

DIPimage

Choose median filter(medif)infiltermenu to apply median filter to an image.

Example 4.6 Apply median filter to image (DIPimage)

>>image_out = readim('blood.tif','')>>out = noise(image_out,'saltpepper',0.2,0.2)>>result = medif(out,5,'elliptic')

26
http://www.statistics.com/resources/glossary/f/filter.phphttp://www.statistics.com/resources/glossary/l/linfilt.phphttp://www.statistics.com/resources/glossary/l/linfilt.phphttp://www.statistics.com/resources/glossary/f/filter.php


31/103


Assignment 4.1

1. Load the image shading which contains some text on a shaded background. To

remove this shading, we need to estimate the background. Once this is done, we cancorrect the original image. This is a common procedure, often required to correct for

uneven illumination or dirt on a lens. There are several background shading estimationmethods:

The most used one is the low-pass filter (gaussf). Try finding a relevant parameterfor this filter to obtain an estimate of the background, and then correct the originalimage.

Another method uses maximum and minimum filtering. Since the text is black, amaximum filter with a suitable size will remove all text from the image, bringingthose pixels to the level of the background. The problem is that each background

pixel now also was assigned the value of the maximum in its neighborhood. To

correct this, we need to apply a minimum filter with the same size parameter. Thiswill bring each background pixel to its former value, but the foreground pixels

wont come back!

Use this estimate of the background to correct the original image. Use only Matlab.

27


32/103

Chapter 5Frequency based processing

Chapter 5: Frequency based Processing

Before you start this section, look up Fourier Transform in your book. Note that the

Discrete Fourier Transform (DFT) is not the same as the continuous version: since theimage is sampled, the Fourier Transform is periodic, but the Fourier Transform itself

must also be sampled (hence discrete), thus the image must be assumed periodic too.Fourier transformation offers considerable flexibility in the design and implementation offiltering solutions in area such as image enforcement, image restoration and image data

compression. However, in robotic application, we usually process images in spatial

domain.

5.1 The Basics of Fourier Transform

First introduce some basic ideas about Fourier Transform.

Let f(x, y), for x = 0, 1,, M-1 and y = 0, 1,, N-1, denote an N image. The 2-D,discrete Fourier Transform (DFT) of f, denoted by F(u, v), is given by the equation:

1 12 ( )

0 0

( , ) ( , )M N

j ux M vy N

x y

F u v f x y e

+

= =

= for u = 0,1,2,,M-1and v = 0,1,2,,N-1.

And inversely, discrete Fourier transform is given by:

1 12 ( )

0 0

1( , ) ( , )

M Nj ux M vy N

x y

f x y F u v eMN

+

= =

= for x = 0,1,2,,M-1 and y = 0,1,2,N-1.

And we should know that even if f(x, y)is real, its transform is complex. The principlemethod of visually analyzing a transform is to compute its spectrum. The Fourierspectrum is defined as:

12 2 2( , ) ( , ) ( , )F u v R u v I u v = +

The power spectrum is defined as:

2 2 2( , ) ( , ) ( , ) ( , )p u v F u v R u v I u v= = +

The phase angle is defined as:1 ( , )( , ) tan

( , )

I u vu v

R u v

=

For purpose of visualization it typically is immaterial whether we view

( , )F u v or .( , )P u v

28


33/103


5.2 Computing 2-D DFT

MATLAB

The FFT of an N image array f is obtained in the toolbox with functionfft2, which

has the syntax:2( )F fft f=

This function returns a Fourier transform that is also of size N . Here we give anexample to compute 2-D DFT using Matlab.

Example 5.1 Compute 2-D DFT (Matlab)

>>img=imread('girl.bmp');>>subplot(2,2,1);>>imshow(img);>>title('original image');>>F=fft2(img);>>S=abs(F);

>>subplot(2,2,2);>>imshow(S,[ ]);>>title('Fourier spectrum');>>Fc=fftshift(F);>>subplot(2,2,3);>>imshow(abs(Fc),[ ]);>>title('Centered spectrum');>>S2=log(1+abs(Fc))>>subplot(2,2,4);>>imshow(S2,[ ]);>>title('log transformation enhanced spectrum');

Figure 5.1 Fourier computation (Matlab)

The result is shown in Figure 5.1. Note that due to the periodicity property, the four

bright spots are in the corners of the image. Thus we need fftshiftto move the origin ofthe transform to the center of the frequency rectangle.

29


34/103


DIPimage

In DIPimage, choose menu transformsand menu itemfourier transform(ft).

Example 5.2 Compute 2-D DFT (DIPimage)

>>image_out = readim('acros.tif','')>>result = ft(image_out)


Figure 5.2 Fourier computation (DIPimage)

5.3 Filtering in the Frequency Domain

5.3.1 Fundamentals of Filtering in Frequency Domain

The foundation for linear filtering in both the spatial and frequency domains is theconvolution theorem, which may be written as:

( , )* ( , ) ( , ) ( , )f x y h x y H u v F u v And conversely,( , ) ( , ) ( , )* ( , )f x y h x y H u v F u v

Here the symbol * indicates convolution of the two functions. Thus we can know, from

the two equations, that filtering in the spatial domain consist of convolving an image

( , )f x y with a filter mask, . Now, we give the basic steps in DFT filtering:( , )h x y

1. Obtain the padding parameters using functionpaddedsize:PQ = paddedsize(size(f));

2. Obtain the Fourier transform with padding:F = fft2( f, PQ(1), PQ(2) );

3. Generate a filter function,H, of sizePQ(1) * PQ(2);

4. Multiply the transform by the filter: G = H.* F;

5. Obtain the real part of the inverse FFT of G: G = real (ifft2(G)) ;

6 Crop the top, left rectangle to the original size: G = g(1:size(f,1),1:size(f,2) ).

30


35/103


Why padding?Image and their transforms are automatically considered periodic if we select to work

with DFTs to implement filtering. It is not difficult to visualize that convolving periodic

functions can cause interference between adjacent periods if the periods are close withrespect to the duration of the nonzero parts of the functions. This interference, called

wraparound error, can be avoided by padding the functions with zeros.

5.3.2 Lowpass Frequency Domain Filters

An ideal lowpass filter (ILPF) has the transfer function

0

0

1 ( , )( , )

0 ( , )

if D u v DH u v

if D u v D

=

>where is a specified nonnegative number and is the distance from point

to the center of the filter.

0D ( , )D u v ( , )u v

MATLAB

In Matlab, there are two ways to generate frequency domain filter, one is obtaining filtersfrom spatial filter and the other is generating filters directly in the frequency domain.

With regard to the former one, we use function freqz2. Here we focus on using latter

ways for generating filters.

Example 5.3 Lowpass filtering (Matlab)

>>f=imread('girl.bmp');>>PQ=paddedsize(size(f));>>[U,V]=dftuv(PQ(1),PQ(2));>>D0=0.05*PQ(2);>>F=fft2(f,PQ(1),PQ(2));>>H=exp(-(U.^2+V.^2)/(2*(D0^2)));>>g=dftfilt(f,H);>>subplot(2,2,1);>>imshow(f,[])>>title('original image');>>subplot(2,2,2);>>imshow(fftshift(H),[])>>title('the low pass filter in image form');>>subplot(2,2,3);>>imshow(log(1+abs(fftshift(F))),[])>>title('spectrum');>>subplot(2,2,4);>>imshow(g, [])>>title('the image after low pass filter');


DIPimage

Choose Menufiltersand pick up different filters, and in method used for computation

combo box chooseftas shown in Figure 5.4.

31


36/103


Figure 5.3 Lowpass filtering (Matlab)

Figure 5.4 DIPimage lowpass filter setting

Example 5.4 Lowpass filtering (DIPimage)

>> image_out = readim('trui.tif','')>> result = gaussf(image_out,5,'best')

5.3.3 Highpass Frequency Domain Filters

Given the transfer function ( , )lpH u v of a lowpass filter, we obtain the transfer function

of the corresponding highpass filter by using the simple relation ( , ) 1 ( , )lp lpH u v H u v=

Assignment 5.1:Design a Butterworth high pass filter and apply it to the image kickoff_grayscale.jpg.Look for the website to find related equation of Butterworth low pass filter, and then use

it to get high pass filter and apply it to this image. (Use Matlab)

32


37/103

Chapter 6Binary Image Processing

Chapter 6: Binary image processing

A binary image is a digital image that has only two possible intensity values for each

pixel. Two most commonly used values for image representation are 0for black and 1(or255) for white. Binary image is produced by thresholding a gray scale or color image and

is often output of operations such as image segmentation, masking, silhouette estimation,

etc.. Advantages of binary image processing are simplicity of its processing and lowmemory requirements. Observe example of binary image on Figure 6.1. Binary image

can be created usinggraythresh function in Matlab that performs optimal thresholding.

level = graythresh(Grayscale_image)Binary_image=Grayscale_image


38/103


Table 6.1: Set operations

Set

Operation

MATLAB

Expressions

Name of function

A A Image itselfcA

~A NOT (image complement)

A B |A B OR (union of two images)A B &A B AND (intersection of two images)A B & ~A B DIFFRENCE (difference of two images; the pixels

in A that arent in B)

All morphological operations can be described using this notation. Main morphological

operations that we will cover in this practicum are dilation, erosion, opening and closing.

6.2.1 Structuring element

A structuring element is used in morphological operations to examine the image. It is a

matrix consisting of only 0's and 1's that can have any arbitrary shape and size. The pixelswith values of 1define the neighborhood. It should be carefully chosen depending on the

image that is processed.

MATLAB:

In order to determine the structuring element in Matlab functionstrelis used:

SE = strel (shape, parameters)

This function creates a structuring element, SE, whereshapeis a string specifying desired

shape Depending onshape, strelcan take additional parameters of the type specified by

shape. The table below lists all the supported shapes.

Table 6.2: Structuring elements

Flat structuring element

arbitrary pair

diamond periodiclinedisk rectangle

line square

octagon arbitrary

Examples of structuring elements:

The structuring element consists of a pattern specified as the coordinates of a number of

discrete points relative to some origin. Normally Cartesian coordinates are used and so aconvenient way of representing the element is as a small image on a rectangular grid.

1 1 11 1 1

1 1 1

SE=strel (rectangle, [ 3 3])

origin11 1 1

1

SE=strel (diamond, 1)

1 0 01 0 01 0 1

SE=strel (arbitrary, [1 0 0; 1 0 0; 1 0 1])

34


39/103


Figure 1 shows a number of different structuring elements of various sizes. In each casethe origin is marked with red color. The origin does not have to be in the center of the

structuring element, but often it is. As suggested by the figure, structuring elements that

fit into a 33 grid with its origin at the center are the most commonly seen type.

DipImage:

In DipImage structuring element is defined using parameters filterShape ('rectangular','elliptic', 'diamond', 'parabolic') andfilterSizewithin morphological functions.

6.2.2 Erosion

The basic effect of the erosion on a binary image is to gradually enlarge the boundaries of

regions of foreground pixels (i.e. white pixels, typically). Thus areas of foreground pixelsgrow in size while holes within those regions become smaller. The dilation operator

takes two pieces of data as inputs. The first is the image which is to be dilated; the second

is a structuring element. It is the structuring element that determines the precise effect ofthe dilation on the input image.

Following functions are used:

MATLAB: image_out = imdilate (binary_image, structuring_element)

Dip_image: image_out = dilation (image_in, filterSize, filterShape)

6.2.3 Dilation

Dilation is dual operation to erosion. The basic effect of the dilatation on a binary image

is to gradually enlarge the boundaries of regions of foreground pixels (i.e. white pixels,typically). Thus areas of foreground pixels grow in size while holes within those regions

become smaller. The dilation operator takes two pieces of data as inputs. The first is the

image which is to be dilated; the second is a structuring element. It is the structuringelement that determines the precise effect of the dilation on the input image.


MATLAB: image_out = imdilate (binary_image, structuring_element)

Dip_image: image_out = dilation (image_in, filterSize, filterShape)

Assignment 6.1

Load image sky.jpg and create binary image from it. Choose structuring element and

perform erosion. Try several different structuring elements, vary their size and generateimages similar to Figure 6.4. What you can observe? Now perform dilation. Can you

observe the difference in images? Results are shown in the Figure 6.3.

35


40/103


Original image Binary image Erosion Dilation

Figure 6.3: Effects of erosion and dilation

Binary image Erosion with SE

size 3

Erosion with SE

size 5

Erosion with SE

size 7Figure 6.4: Effects of the size of structuring element on erosion

6.2.4 Opening and closing

The sequence of erosion followed by dilation from previous example is called anopening. It is used to remove small objects from an image while preserving the shape and

size of larger objects in the image. The inverse sequence, dilation followed by erosion, is

a closing and it serves to remove small holes in the objects.


MATLAB: image_out = imopen (binary_image, structuring_element)image_out = imclose (binary_image, structuring_element)

Dip_image: image_out = bopening (image_in, filterSize, filterShape)image_out = bclosing (image_in, filterSize, filterShape)

Assignment 6.2

Load image sky and perform opening operation. Compare result with the result from

Figure 6.3. Is there a difference with the result from Figure 6.3?

6.2.5 Finding boundary pixels

The boundary pixels can be found by first dilating the object and subtracting the originalfrom it. Or by first eroding the image and subtracting it from the original.

1. Boundary_image=imdilate(Original_image) Original_image

2. Boundary_image= Original_image - imerode(Original_image)

36


41/103


Assignment 6.3

Load image sky and extract boundary pixels. Choose structuring element carefully and

try to get the result similar to Figure 6.5. Can you improve the result? Is the quality of

result dependent on the size of the structuring element? Explain why.

Original binary image Extracted pixel boundaries

Figure 6.5: Extracting boundary pixels

6.3 Morphological reconstruction

It is convenient to be able to reconstruct an image that has survived several erosions or

to fill an object that is defined, for example, by a boundary. The formal mechanism for

this has several names including region-filling, reconstruction, and propagation.Reconstruction is a morphological transformation involving two images and a structuring

element. First image, a seed image is the result of the morphological operation (eg

erosion). The other image is original binary image, often called the mask image. Thestructuring element defines connectivity.

One of used method is opening by reconstruction. It restores original shapes of the

object that remained after erosion. For that following commands can be used:

MATLAB:

For reconstruction:reconstructed_image = imreconstruct(seed_image,mask_image,connectivity)

For filling holes:image_out=imfill(binary_image, connectivity, holes)

For clearing border objects:image_out =imclearborder(binary_image, connectivity)

For defining image skeleton:image_skeleton=bwmorph( binary_image, skel, Inf);

For defining branch pixels in image skeleton:image_skeleton=bwmorph( binary_image,branchpoints);

NOTE: default connectivity is 8 or 4 for a 2D image

37


42/103


DipImage:

For reconstruction:image_out=bpropagation(image_seed,image_mask,iterations,connectivity,edgeCondition)

Iterations:the number of steps taken defines the size of the structuring element (0is thesame asInf, meaning repeat until no further changes occur).

EdgeCondition:Make sure to set the edge condition to 0. This is the value of the pixels

just outside the boundary of the image. If you set it to 1, all objects touching the borderwill also be reconstructed.

For clearing border objects:image_out = brmedgeobjs(image_in,connectivity)

For defining image skeleton:image_out = bskeleton(image_in, edgeCondition, endPixelCondition);

For defining branch pixels in image skeleton:image_out = getbranchpixel(image_in)

Assignment 6.4:

Load the image text.jpg and using differentmorphological operations obtain the results

shown at Figure 6.7. Explain the difference between the different images.

Original binary image Eroded image with a vertical line

Image opened with a vertical line Image reconstructed with a vertical line

Holes filled in image Border characters removed

Figure 6.7: Examples of different morphological operations

Assignment 6.5: Quality control of incandescent lamps

Load the image lamps. It contains an image of six bulbs, two of which are to be

discarded. The bulbs must have a contact at the bottom, and it must not touch the outeredge, which is the other contact. Threshold at a low value, such that the bulb is merged

with the background (we are only interested in the fitting, which is characterized by the

two black rings). Now remove the background using imclearborder or brmedgeobjs.

38


43/103


Now devise some operations using not (or ~), dilation, erosion, imreconstruct (orbpropagation) and/or imclearborder(brmedgeobjs) to detect either the good or bad bulbs

(either make a program that rejects bad bulbs or accepts good bulbs).

The colored images were generated with the command overlay. It overlays a grey- valueor color image with a binary image. It is possible to apply this function several times,

each with a different binary image, which can thus be used to mark the image usingseveral colors. Third (optional) parameter determines the color for the binary objects. It ispossible to apply this function several times, each with a different binary image, which

can thus be used to mark the image using several colors.

Image lamps Exercise goal Alternate goal

Figure 6.8: Selecting different objects

Assignment 6.6: Distinguishing nuts from bolts (only DipImage)

Now load the image nuts_bolts and threshold it to get the binary image. Note that the

threshold operation chooses the background as the object (because it is lighter). You willneed to inverse the image before or after the thresolding. Use the bskeleton function to

create a skeleton of the objects. What is the influence of the Edge Condition? What

does End-Pixel Conditioncontrol?With looseendsaway we can transform the nuts seen from the top (with the hole in them)

into a form that is distinguishable from the other objects in the image. Now use thefunctiongetsinglepixelto extract the objects without a hole in them. This new image can

be used as a seed image in bpropagation. The mask image is the original binary image.

The objects with holes are retrieved with b& c (b and not c) if the output image forbpropagation was c. Try extracting the last nut using the bskeleton and the

getbranchpixel functions. Now solve the assignment in Matlab without using DipImage

toolbox. Find the equivalent functions in Matlab to the ones used in DipImage.

Image nuts_bolts Exercise goal

Figure 6.9: Selecting different objects

39


44/103


6.4 Gray scale morphology

All morphological operations that are already explained can be applied on the grayscale

images. Morphological filtering is very useful and often applied on the gray scale imagessince it can severely reduce noise while preserving the edges in the image in the contrast

to linear filters. It is able to distinguish structures based on size, shape or contrast(whether the object is lighter or darker than the background). They can be employed toremove some structures, leaving the rest of the image unchanged. In this sense,

morphology is one step ahead of other image processing tools towards image

interpretation.

6.4.1 Morphological smoothing

Because opening suppresses bright details smaller than the structuring element andclosing suppresses dark details smaller than the structuring element they are often used in

combination for image smoothing and noise removal. Note that the size of the structuringelement is an important parameter. A morphological smoothing with a small structuring element

is an ideal tool to reduce noise in an image.

MATLAB:Image_opening=imopen(imclose(Image, SE), SE);

Image_closing=imclose(imopen(Image, SE), SE);

DipImage:Image_opening = bopening (bclosing (image_in, filterSize, filterShape),filterSize, filterShape)

Image_closing = bclosing (bopening (image_in, filterSize, filterShape),filterSize, filterShape)

The results of the morphological smoothing are shown in Figure 6. 10.

Original image Open-close filtering Close-open filteringFigure 6.10: Morphological smoothing

Assignment 6.7

Load the image erika and construct a smoothing filter that removes most of the hair,but leaves the girls face recognizably human. In process use both the Open-close and the

Close-open filtering and explain the difference between them.

40


45/103


6.4.2 Morphological sharpening

As we already saw in Chapter 3, for sharpening we should use edge detectors. They aremorphological gradient magnitudes:

Edge1 = dilation(A)AEdge2 = Aerosion(A)

In a similar way, we can construct a morphological second derivative:

1 2 ( ) 2* (

2 2

)Edge Edge dilatation A A erosion AMSD

+= =

Note that it has a similar form as Laplace operator, ( 1, -2, 1) across the edge.

A sharper version of the morphological Laplacian can be computed by taking theminimum value of the two edge detectors. Note that the sign of the morphological

Laplacian is used for this purpose (use the function sign).

MSDsharp =sign(Edge1 Edge2) min(Edge1,Edge2)

Assignment 6.8

Observe the results obtained by morphological sharpening in Figure 6.11. Compare the

results with the result obtained by using the regular Laplace operator. What is the

difference?

Original image Linear Laplace Morphological

Laplace

Sharp morph.

LaplaceFigure 6.10: Morphological sharpening

6.4.3 Compensating for non uniform illumination (Top-hat transformation)

Openings can be used to compensate for non uniform background illumination, which

occurs very often in real world scenes. Figure 6.11 (a) shows an image f in which the

background is darker towards the bottom than in upper portion of the image. The unevenillumination makes image thresholding difficult. Figure 6.11 (b) is a thresholded version

in which the grains at the top of the image are well separated from the background but thegrains in the middle are improperly extracted from the background. Opening of the

image can produce the estimate of the background across the image, if the structuring

element is larger than rise grains. Estimated image is shown on the image 6.11 (c).

41


46/103


>> f =imread('rice.tif');Original image>> se = strel('disk', 10);>> fo = imopen(f,se); Estimate of the background

By subtracting background estimate from the original image, we can produce the imagewith reasonably even background. Figure 6.11 (d) shows the result, while the Figure 6.11

(e) shows the new thresholded image. The improvement is apparent.

Subtracting an opened image from the original is called the top-hat transformation.

>> f2 = imtophat(f,se); Image with even background(Equal with f2 = imsubtract(f,fo);)

A related function imbothat(f, se) perfoms the bottom-hat transformation, defines theclosing of the image minus the image. These two functions can be used for contrast

enhancement using commands such as:

>> se = strel('disk',3);

>> g =imsubtract(imadd(f, imtophat(f,se)), imbothat(f,se));

a) Original image b) Thresholded image c) Background estimation

d) Top-hat transformation e) Thresholded top-hat image

Figure 6.11: Top-hat transformation

Assignment 6.10

Load the image plugs and create binary image from it. Improve the quality of the

thresholded image by compensating for uneven illumination. How many connectedcomponents can you count? Using morphological operations, try to separate all the plugs

on the image. How many connected components can you count now? Extract theboundary of each connected component. Can you suggest any method to measure the size

of each plug? Explain it in your own words.

42


47/103

Chapter 7Measurements

Chapter 7: Measurements

In this chapter we describe various ways to measure objects in the image. In previouschapters we saw different ways for image processing but in order to progress to image

analysis we first need to learn how to select objects in images and measure their basicproperties.

7.1 Selecting objects

In a binary image, an object is considered a connected set of pixels. As discussed inSubsection 6.1, there are different connectivity modes that define which pixels are

considered connected (and thus belonging to the same objects). Before we can measure a

property of such an object (say, the number of pixels that define it), we need to extractthe object from the image. The common way of doing this is to label all objects. Labeling

involves finding any foreground pixel in the image, give it a value (the label ID), and

recursively give the same value to all pixels that are connected to it. This process isrepeated until all foreground pixels have been assigned to an object. To extract one

object, all we now need to do is get all pixels with its ID.

MATLAB:

Function bwlabelcomputes all the connected components in a binary image.

[L, num] = bwlabel(binary_image, n)

It returns a matrix L, of the same size as binary_image, containing labels for the

connected objects in binary_image. The variable n can have a value of either 4 or 8,

where 4 specifies 4-connected objects and 8 specifies 8-connected objects. If the

argument is omitted, it defaults to 8.The number of connected objects found in binary_imageis returned in num.

You can use the MATLAB findfunction in conjunction with bwlabelto return vectors

of indices for the pixels that make up a specific object. For example, to return the

coordinates for the pixels in object 2, enter the following:

>>[r, c] = find(bwlabel(binary_image)==2)

In order to show labeled components in different color, as on Figure 7.1 we can use

following functions:

>>RGB = label2rgb(L);Convert label matrix into RGB image

And in order to display just selected object:

>> mask = zeros(size(L));>> mask(r,c)=1;>>imshow(label2rgb(L.*mask))

Labeled objects and the selected one are shown in Figure 7.1.

43


48/103

Chapter 7Measurements

DipImage:Equivalent function is label, where is the connectivity same as in bwlabel function and

minSize, maxSizerepresent minimum and maximum size of objects to be labeled.

image_out = label(image_in,connectivity,minSize,maxSize)

Specific object can be easily extracted using:

>> image_out==2;

a) Labeledcomponents

b) Selected object c) Perimeter of theselected object

d) Centroids of allcomponents

Figure 7.1 Selecting and measuring object

Assignment 7.1:

Loa

Documents

Basic Image Processing for Robotics Manual