1

Machine VisionSummary # 2

Fig. 1. Euclidean and city block distance

A. Distance

Many operations on images such as linear filtering areneighborhood operations, For this reason, the distance betweenpixels plays an important role in image processing. It ispossible to use different definitions of the distance betweenpixels (i, j) and (h, k)(metrics):• Euclidean distance:

De =√(i− h)2 + (j − k)2 (1)

• City block distance

Dc = |i− h|+ |j − k| (2)

A comparison is shown in figure 1.

I. PIXEL ADJACENCY

• Any pixel (i, j) has two vertical and two horizontalneighbors. Each one of them is at 1 unit distance awayfrom (i, j).

• Any pixel (i, j) has four diagonal neighbors. Each oneof them is at Euclidean distance of

√2 away from (i, j).

• The direct horizontal, vertical and the diagonal neighborsof (i, j) together form 8 neighbors called 8–adjacency.

A. Noise in images

Noise can occur during image capture, transmission, or pro-cessing. Usually because we do not have the exact knowledgeof the characteristics of the noise, we use probabilistic models,i.e., probability density functions. Noise can be dependent orindependent of the image signal. Clearly, it is much easier todeal with independent noise.

Fig. 2. Gaussian and impulse noise

Original

Fig. 3. Original image without noise

• Additive noise: f̂(i, j) = f(i, j) + noise• Multiplicative noise: f̂(i, j) = f(i, j)× noise

B. Gaussian and impulse noise

Gaussian and impulse models are among the most widelyused models in image processing.• Gaussian noise: The noise has a Gaussian distribution.

This distribution is widely used for modeling naturalprocesses. The PDF of the Gaussian noise is given by

PDF =2

σ√2πe

−(z−µ)2

2σ2 (3)

where

Machine vision FB

Salt and pepper

Fig. 4. Image with salt and pepper noise

Gaussian

Fig. 5. Image with Gaussian noise

– z : gray-level– µ : mean value of z– σ : standard deviation

• Impulse noise: some pixels are very different from theirneighbors. Impulse noise is usually caused by trans-mission errors, faulty CCD sites etc. Example: salt andpepper noise. The PDF of the impulse noise given by

δ =

Pa for z = aPb for z = b0 otherwise

(4)

• An illustration of the Gaussian and salt and pepper noiseis shown in figures 4 and 5. Figure 3 shows the originalimage and figure 6 shows the histograms.

C. Linear filtering

The image may be corrupted by noise. Image enhancementaims at eliminating undesirable characteristics such as noiseand poor contrast. One way to enhance images is by usingfilters. Filters can be used for many operations:• sharpen details• remove noise

• image smoothing• detect edges in an image, etc.

There are two different ways to perform linear filtering:• in the time domain (actually space domain): the filtering

operation is a convolution. Spatial techniques operate onthe pixels of an image directly.

• in the frequency domain. A convolution in the space do-main becomes a multiplication in the frequency domain.

Linear flittering is simply a convolution that involves oper-ations on neighbors. It is a neighborhood operation.

Consider figure 7. When the input to the system is animpulse δ, the output h is the impulse response. A linearspace invariant (LSI) system can be described by its impulseresponse h. In LSI systems, the input and output are relatedas follows

g(i, j) =∑k,l

f(i− k, j − l)h(k, l) (5)

org(i, j) =

∑k,l

f(k, l)h(i− k, j − l) (6)

This operation is called convolution. The notation for convo-lution is:

g = f ∗ h (7)

where h is the impulse response function of the filterconvolution mask, that is

h ∗ δ = h (8)

• Mathematically, a filter is characterized by its impulseresponse, h, which can be seen as the transfer functionof the filter.

• Numerically, a filter is characterized by its mask, whichis a simple matrix. The general form for a 3 by 3convolution kernel looks like

h11 h12 h13h21 h22 h23h31 h32 h33

(9)

or1∑h(i, j)

h11 h12 h13h21 h22 h23h31 h32 h33

(10)

where the sum of the weights is normalized to one.• Common masks are

– 3× 3– 1× 3– 3× 1– 1× 5– 5× 1– 2× 2– 5× 5

The size of the filter determines the amount of filtering

Large masks =⇒ greater degree of filtering=⇒ loss of image details (11)

• Examples of kernels

2

Machine vision FB

0

500

1000

1500

2000

2500

3000

Original image

0 100 200 300

0

500

1000

1500

2000

2500

3000

Image with salt and pepper noise

0 100 200 300

Fig. 6. Histogram of the original image (left) and the image with salt and pepper noise (right)

Fig. 7. LSI system and filtering mask

– Consider the filter given by

Id =

0 0 00 1 00 0 0

(12)

What is the effect of this Kernel on the image? Theanswer is that it leaves the image unchanged. Youcan look at it as an identity filter.

– Consider the filter given by

0 0 00 0 10 0 0

(13)

This filter shifts the image horizontally.

D. Numerical convolution

• Convolution is a weighted sum of the image pixels. Foreach pixel (i, j) in the image, the output value g(i, j)is calculated by applying the convolution mask, whichresults in a weighted sum of the neighbors of (i, j).The convolution window is moved throughout the entireimage. Figure 8 illustrate the convolution operation.

• Convolution with large filters is expensive in the timedomain.

• Convolution is the treatment of a matrix by another onecalled “kernel”.

E. ExampleWe want to perform a convolution on the following 3 by 3

image

f =

1 2 34 5 67 8 9

(14)

with mask

h =0 1 01 3 10 1 0

(15)

If we write the result of the convolution operation as

g = f ∗ h =

b1 b2 b3b4 b5 b6b7 b8 b9

(16)

then, b1, b2, b5 are given by:

b1 = 3× 1 + 1× 2 + 1× 4 = 9b5 = 3× 5 + 1× 2 + 1× 4 + 1× 6 + 1× 8 = 35

b2 = 1× 1 + 2× 3 + 3× 1 + 3× 5 = 15(17)

and the full matrix is given by:

g = f ∗ h =

0 1 2 3 01 9 15 17 34 25 35 35 67 33 45 41 90 7 8 9 0

(18)

• Linear filtering is a convolution operation where theoutput is a weighted sum of the neighboring pixels,

• Any filter that is not a weighted sum is a nonlinear filter

F. How do frequencies appear in a image?

• Low frequencies in the image appear as slow variationsof the intensities. Low frequencies correspond to longwavelengths.

• High frequencies appear as abrupt changes in imageintensity such as edges and corners. High frequenciescorrespond to short wavelengths

• Low pass filter: This filter passes low frequencies wherethe image intensity changes slowly, and blocks highfrequencies where intensity changes quickly.

3

Machine vision FB

Fig. 8. The convolution operation

• High pass filter: This filter keeps high frequency compo-nents where intensity changes abruptly, and removes lowfrequency components where intensity changes slowly.

• What is the effect of a low pass filter? blurring orsmoothing of the image and the sharp details in the imageare lost.

• What is the effect of a high pass filter? sharpen the image,and therefore, edges are more visible.

G. Mean filterA mean filter is a smoothing filter where each pixel is

replaced by the average value of its neighbors, that is

g(i, j) =1

M

∑k,l∈N

f(k, l) (19)

where N is the neighborhood and M is the number of pixelsin the neighborhood.• Example 1

For a 3× 3 neighborhood we have

g(i, j) =1

9

k=i+1∑k=i−1

l=j+1∑l=j−1

f(k, l) (20)

• Example 2Find the mask of a 3 by 3 mean filter.

g(i, j) = 19 [f(i− 1, j − 1) + f(i− 1, j) + f(i− 1, j + 1)+f(i, j − 1) + f(i, j) + f(i, j + 1)+

f(i+ 1, j − 1) + f(i+ 1, j) + f(i+ 1, j + 1)](21)

and therefore

mask =1

9

1 1 11 1 11 1 1

(22)

This means that a mean filter can be implemented as aconvolution operation with equal weights in the convolu-tion masks.

H. Smoothing filter

A smoothing filter is a low pass filter, i.e., removes highfrequency components. Sharp details in the image are lost. Atypical 3× 3 smoothing filter is given by

mask =

116

18

116

18

14

18

116

18

116

(23)

Three important remarks about this filter

• The sum of all entries is 1.• Symmetry in the horizontal and vertical directions.• The filter has a single peak in the middle called “main

lobe”. This means more importance is given to the centerpixel.

I. Gaussian filter

Gaussian filters are a class of low pass linear filters. Theycan be used for removing Gaussian noise and smoothing ofdigital images. A two dimensional discrete Gaussian filter isgive by

h(i, j) = e−i2+j2

2σ2 (24)

where σ is the standard deviation.

J. Some properties of the Gaussian filter

• In 2D, Gaussian filters are rotationally symmetric. Thisimplies that the filtering operation is independent of thedirection. Converting the Gaussian filter from rectangularto polar coordinates yields

h(r, θ) = e−r2

2σ2 (25)

withr2 = i2 + j2 (26)

Equation (25) does not depend on the angle, and thusfiltering is independent of the direction. The disadvantagewith this is that it is not possible to target a specificdirection for filtering.

• Gaussian filter has a single main lobe at the center withweights decreasing with the distance from the centralpixel. This means that more significance is given to thecenter pixel and its closest neighbors.

• The degree of smoothing (filtering) is determined by σ. Alarger σ implies a wider filter and thus greater smoothing.

• The horizontal and vertical directions in the Gaussian fil-ter are separable. Thus it is possible to perform smoothingin one direction and then in the other one.

• ExampleProve the separability property of the Gaussian filter.

K. Designing Gaussian filters

• Pascal’s triangle is a good way to approximate the coef-ficients. For example, from the 5th row of the Pascal’s

4

Machine vision FB

Fig. 9. Pascal Triangle coefficients

triangle shown in figure 9, it is possible to approximatea five point 1D Gaussian filter using

1 4 6 4 1 (27)

• It is also possible to compute the weights of the maskdirectly from the discrete Gaussian distribution. Theobtained matrix is then normalized so that the sum is1. The normalized filter can be written as

h(i, j) = Ce−i2+j2

2σ2 (28)

where C is the normalization coefficient.• Example: With σ = 1, it is possible to obtain the

following 3 by 3 Gaussian filter:

h =

0.3679 0.6065 0.36790.6065 1.0000 0.60650.3679 0.6065 0.3679

(29)

which becomes after normalization:

h =

0.0751 0.1238 0.07510.1238 0.2042 0.12380.0751 0.1238 0.0751

(30)

L. High pass filter

• There is a relationship between high pass and low passfilters

High pass filtered image =original image− low pass filtered image (31)

This can be expressed as follows

Hp ∗ f = Id ∗ f − Lp ∗ f (32)= (Id − Lp) ∗ f (33)

with– Hp: high pass filter mask– Lp: low pass filter mask– f : original image

Note that Id is not an identity matrix.• Example

Find the mask of the high pass filter corresponding to thelow pass filter given by (30).We have

Hp = Id − low pass (34)

and thus

Hp =

−0.0751 −0.1238 −0.0751−0.1238 0.7958 −0.1238−0.0751 −0.1238 −0.0751

(35)

II. MEDIAN FILTER

The median filter replaces each pixel in the image by themedian value of its neighbors. Median filters are very effectivein removing impulse noise including salt and pepper noise.There are two steps to perform median filtering:• Sort pixels in ascending order.• Select the value of the middle pixel as the new value.‘

A. Example

The original image is given by:

Im =

156 131 125225 96 897 199 202

(36)

After sorting the pixels, we get[7 89 96 125 131 156 199 202 225

](37)

and the new image is given by

ImNew =

156 131 125225 131 897 199 202

(38)

B. Median filter and salt and pepper noise

Median filters are very effective in removing salt and peppernoise. Consider the following example where one pixel has avery different value:

Im =

1 2 36 255 77 9 4

(39)

The result of the median filter is

ImNew =

1 2 36 6 77 9 4

(40)

and the salt noise pixel is removed.Since the median filter uses a sorting algorithm, it is

relatively expensive and complex to compute.

RANK FILTERS

If we consider a neighborhood of N pixels, a filter of rankk will arrange the pixels in the neighborhood in ascendingorder from smallest (M0) to largest (MN ) gray level value andassign the kth value to the center point of the window. Rankfilters k = 1 and k = N are called minimum and maximumfilters, respectively. The median filter is a particular case withk = floor(N/2) + 1 where floor(X) rounds X to the nearestinteger towards minus infinity.

III. SOME USEFUL FUNCTIONS

• imfilter can be used to perform filtering in the spacedomain. The syntax for this function is

B = imfilter(A,H) (41)

where A is the image and H is the filter.

5

Machine vision FB

• fspecial can be used to create a two-dimensional filter(mask). The general syntax for fspecial is

H = fspecial(type) (42)

where type stands for the type of the filter, which includes– ‘average’: averaging filter– ‘disk’: circular averaging filter– ‘gaussian’: Gaussian lowpass filter– ‘unsharp’: unsharp contrast enhancement filter

fspecial may take additional parameters depending onthe type of the filter. For example, for a Gaussian filter,the parameters are the size and the standard deviation.

• imnoise can be used to add noise to an image. Itssyntax is

J = imnoise(I, type, ...) (43)

where type is a string that can have one of these values:– ‘gaussian’: Gaussian white noise with constant mean

and variance.– ‘localvar’: zero-mean Gaussian white noise with an

intensity-dependent variance– ‘salt & pepper’: on and off pixels– ‘speckle’: multiplicative noise

imnoise may take additional parameters related to thetype of the noise.

6