Saarland University, Saarland University Hospital, and Saarland University of Applied Sciences, Germany

Introduction to Image Processing

Izadora Mustaffa MSc.

• A digital representation of a real-world scene. Graphics is the digital representation of an imaginary scene.

• Composed of discrete elements generally called picture element (or pixels for short)

• Pixels are parameterized by– position– intensity– time

• In all combinations, these parameters define still images, video, volume data and moving volumes

Image Perception

• To better process images, we need to have an idea of the human perception of images.

• There are a number of things to bear in mind :

A) OBSERVED INTENSITIES vary as to the background.

• Both gray blocks in the center while appearing be of different intensities, they are actually have the same intensity.

• Perception depends on their surrounding.

• This image varies continuously from light to dark as we travel from left to right. However,

• It is impossible for our eyes not to see a few horizontal edges in this image.

Our visual system tends to undershoot or overshoot around the boundary of regions of different intensities.

For example, suppose we had a light grey blob on a dark grey background. As our eye travels from the dark background to the light region, the boundary of the regionappears lighter than the rest of it.

Conversely, going in the other direction, the boundary of the background appears darker than the rest of it.

Creating Images

• The creation of images involves two main tasks– spatial sampling, – quantization

• Spatial sampling determines the resolution of an image– finer sampling allows for smaller detail– finer sampling requires more pixels and “larger” images

• Quantization determines how many intensity levels are allowed– finer quantization will prevent “false contouring” (artificial edges)– courser quantization allows for compressing images

Spatial Resolution

• Spatial resolution changes affect the amount of visible detail available to humans from an image.

• When a source image's spatial resolution is made larger than the original image and no further information is provided to interpolate pixel value, detail is lost.

Note : Images are from Digital Image Processing, Gonzalez and Woods

Effect of Spatial Sampling

Note : Images are from Digital Image Processing, Gonzalez and Woods

Quantization

• a digital image, by its very nature, is quantized to a discrete number of intensity values.

• an image can be quantized to a lower number of bits (b) or intensity values (2b).

• Example 1 : An RGB image which has 28 = 256 for each plane, which means this image has possible intensity values, in the range of [0, 224] or [0, 16777216]

• Example 2 : A grayscale 28 = 256 possible intensity values, in the range of [0, 255]

Image Types

• The internal representation of a digital image is simply a matrix of numbers representing binary, grayscale, RGB (colour images), and indexed images.

• The internal matrix is naturally not made visible to the us when we look at a computer monitor for example.

• The mapping of the pixel value, to display contrast, gamma, brightness etc. can be manipulated resulting in interesting visual effects.

• Bear in my that, the underlying image is undisturbed when the mapping is manipulated.

Binary Image

Greyscale

RGB

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

50

100

150

200

250

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

50

100

150

200

250

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

0

50

100

150

200

250

Indexed Image

Brightness

• The functional form of brightness is a unit-slope line with varying intercepts :

(darkens) brightness decreased ,0brightness increased , 0

1,0 valuesintensity image with the)(

bb

ubuug

Contrast

• The functional form is given as :

1 and 1 when is white, tomapped isblack andblack tomapped is (white image inverted

and2

10

as long as : decreasescontrast

and

as long as : increasescontrast

intercept theand slope ebetween th iprelationsh ),1(21

)(

bm

bm

g

bm

g

mb

bmuug

Gamma Correction

• High contrast images can be visually appealing• The drawback is that linear contrast enhancement, leads to saturation at both low and high end

of the intensity range.

• A standard nonlinear contrast adjustment scheme is the gamma correction :

contrast reduces 1, contrast increases ,1 where

)(

uug

Gamma Correction

• A sigmoidal non-linearity of the standard scheme is given by

• This non-linearity possesses two variable and is more versatile and produces a more balanced contrast enhancement.

121

2

1

)1(1 and

11 where

)( as scaled when ]1,0[)(

11)(

cec

ec

ccugug

eug u

Histogram Equalization

• Intensity values of a typical image are often distributed unevenly across the full range of 0 to 255 ( for an 8-bit image)

• An image can be transformed so that the distribution of intensity values is flat, that is, each intensity value is equally represented in the image.

• Define N(u) as the number of pixels with intensity value u. This is the image histogram and a discrete approximation to the probability distribution function.

• The cumulative distribution function is defined as:

– C(u) is the number of pixels with intensity values less than or equal to u

• Histogram equalization has a similar effect to contrast enhancement, which intuitively should make sense since the number of black and white pixels are increased.

u

iiNyC

0)()(

Histogram Equalization

0

100

200

300

400

500

600

700

800

900

0 50 100 150 200 250

0

100

200

300

400

500

600

700

800

900

1000

0 50 100 150 200 250

Image Processing

• Image processing usually attempts to mimic the human visual system in extracting meaning from an image

– IMAGE ENHANCEMENT - images are processed so that the result is more suitable for a particular application,

• Sharpening or de-blurring an out of focus image• Highlighting edges• Improving image contrast, or brightening an image• Removing noise

– IMAGE RESTORATION - reversing the damage done to an image , • Removing of blur caused by linear motion• Removal of optical distortion • Removing periodic interference

– IMAGE SEGMENTATION – involves subdividing an image into constituent parts or isolating certain aspects of an image,

• Finding lines, circles, or particular shapes in an image• Identifying objects/region of interest

These classes are not disjoint; a given algorithm may be used for both image enhancement or for image restoration

Restoration Example

Image Enhancement

• Often used to increase the contrast in images that are overly dark or light

• Enhancement algorithms often play to humans’ sensitivity to contrast

• More sophisticated algorithms enhance images in a small neighborhood, allowing overall better enhancement.

Highlighting Edges

• Edge detection extract edges of objects from an image.

• There are a number of algorithms for this, but these may be classified as derivative based or gradient based.

• In derivative based edge detection the algorithm takes first or second derivative on each pixel in the image.

• In case of first derivative at the edge of the image there is a rapid change of intensity.

• While in case of second derivative there is a zero pixel value, termed zero crossing.

• In gradient based edge detection a gradient of consecutive pixels is taken in x and y direction.

• Taking derivative on each and every pixel of the image consumes a lot of computer resources and hence is not practical.

• So usually an operation called kernel operation is carried out.

• A kernel is a small matrix sliding over the image matrix containing coefficients which are multiplied to corresponding image matrix elements and their sum is put at the target pixel.

Edge Detection

• How can you tell that a pixel is on an edge?

Methods of Edge Detection

First Order Derivative /Gradient Methods

Sobel OperatorPrewitt Operator

f

x

xf

x

2

2

xf

x

zero-crossing

Second Order Derivative

LaplacianLaplacian of GaussianDifference of GaussianCanny Edge Detection

Gradient

• Gradient equation:

• Represents direction of most rapid change in intensity

• Gradient direction:

• The edge strength is given by the gradient magnitude

• How can we differentiate a discrete image?

Finite difference approximations:

jijijiji ffffxf

,,11,1,121

jijijiji ffffyf

,1,,11,121

1, jif 1,1 jif

jif , jif ,1

1 1

1 121

xf 1 1

1 121

yf

Convolution masks :

Discrete Edge Operator

1, jif 1,1 jif

jif , jif ,1

1,1 jif

jif ,1

1,1 jif 1, jif 1,1 jif

• Second order partial derivatives:

jijiji fffx

f,1,,122

2

21

1,,1,22

2

21

jijiji fffy

f

• Laplacian :

2

2

2

22

yf

xff

22 1

f

Convolution masks :

1 0

4 1

0

1

0 1 0

or 261

4 1

20 4

1

4

1 4 1

(more accurate)

Discrete Edge Operator

Sobel’sedges in xdirection

101202101

121000121

Sobel’sedges in ydirection

xf yf

Sobel Edge Detection

Prewitt’sedges in xdirection

101101101

111000111

Prewitt’sedges in ydirection

xf yf

Prewitt Edge Detection

• Consider a single row or column of the image– Plotting intensity as a function of position gives a signal

Where is the edge??

Effects of Noise

Where is the edge? Look for peaks in

Solution : Smooth First

…saves one operation.

Derivative Theorem of Convolution

Laplacian of Gaussian operator

Where is the edge? Zero-crossings of bottom graph !

fhx

fhx

2

2

2

2

Laplacian of Gaussian

Laplacian of Gaussian (LoG)

2D Gaussian Edge Operators

Laplacian of GaussianGaussian Derivative of Gaussian (DoG) (LoG)

• is the Laplacian operator:

• is used for gradient (derivative)• is used for Laplacian

Gaussian

• The idea of Gaussian smoothing is to use this 2-D distribution as a `point-spread' function, and this is achieved by convolution.

• Since the image is stored as a collection of discrete pixels, a discrete approximation to the Gaussian function should be produced before performing the convolution

• In theory, the Gaussian distribution is non-zero everywhere, which would require an infinitely large convolution kernel, but in practice it is effectively zero more than about three standard deviations from the mean, so the kernel can be truncated at this point.

• 2-D convolution can be performed by first convolving with a 1-D Gaussian in the xdirection, and then convolving with another 1-D Gaussian in the y direction.

Discrete approximation to Gaussian function with standard deviation, =1.0

Gaussian

kernel size = 8x8, = 1 kernel size = 8x8, = 5 kernel size = 4x4, = 5

Noise added Mean = 0, Variance = 5

Original Image

Laplacian of Gaussian (LoG)

• The Laplacian is a 2-D isotropic measure of the 2nd spatial derivative of an image.

• The Laplacian of an image highlights regions of rapid intensity change and is therefore often used for edge detection (see zero crossing edge detectors).

• The Laplacian is often applied to an image that has first been smoothed with something approximating a Gaussian smoothing filter in order to reduce its sensitivity to noise, and hence the two variants will be described together here.

• The operator normally takes a single graylevel image as input and produces another graylevel image as output.

• The Laplacian operator can be calculated using a convolution filter.

• Since the input image is represented as a set of discrete pixels, we have to find a discrete convolution kernel that can approximate the second derivatives in the definition of the Laplacian.

• Two commonly used used like kernels (discrete approximations to the Laplacian filter)

• Using one of these kernels, the Laplacian can be calculated using standard convolution methods.

• Gaussian smoothing is applied to the image before applying the Laplacian filter to make it less sensitive to noise.

• This pre-processing step reduces the high frequency noise components prior to the differentiation step.

Laplacian of Gaussian (LoG)

• The LoG operator calculates the second spatial derivative of an image.

• This means that in areas where the image has a constant intensity (i.e. where the intensity gradient is zero), the LoG response will be zero. In the vicinity of a change in intensity, however, the LoG response will be positive on the darker side, and negative on the lighter side.

• This means that at a reasonably sharp edge between two regions of uniform but different intensities, the LoG response will be:

– zero at a long distance from the edge, – positive just to one side of the edge, – negative just to the other side of the edge, – zero at some point in between, on the edge itself.

Discrete approximation to LoG with Gaussian, =1.4

Laplacian of Gaussian (LoG)

Laplacian of Gaussian (LoG)

Original Image

kernel size = 5x5, = 0.2

kernel size = 5x5, = 0.05

kernel size = 7x7, = 0.05

kernel size = 7x7, = 0.2

• By itself the effect of the filter is to highlight edges in an image.

Image,

h(x) hxx(x)

h(y) hyy(y)

+ fh

Image, h(x) h(y) + hf

Gaussian Filtering

Laplacian of Gaussian Filtering

f

f

Canny Edge Operator

• Smooth image I with 2D Gaussian:

• Find local edge normal directions for each pixel

• Compute edge magnitudes

• Locate edges by finding zero-crossings along the edge normal directions (non-maximum suppression)

fG

fG

n

fG

02

2

nfG

fG

Non-maximum Suppression

• Check if pixel is local maximum along gradient direction– requires checking interpolated pixels p and r

Canny Edge Operator

Canny with Canny with original

• The choice of depends on desired behavior– large detects large scale edges– small detects fine features

Morphological Operations

• The value of each pixel in the output is based on the corresponding input pixel and its neighbors.

• By choosing appropriately shaped neighbors one can construct an operation that is sensitive to a certain shape in the input image.

• For example : • Bridge – bridges unconnected pixels.• Clean – removes isolated pixels.• Fill – fills isolated interior pixels.• Majority – sets a pixel 1 if five or more pixels in 3x3 neighborhood are 1’s.• Remove – removes interior pixels by setting pixels to zero if its 4

neighbors are 1, thus leaving only boundary pixels on.• Shrink – shrinks object to a point. • Diagonal Fill – fills diagonal elements to eliminate 8 connectivity with

background.

Dilation and Erosion

• Dilation and erosion are two fundamental morphological operations

• Dilation adds pixels to the boundaries of objects in an image, while erosion removes pixels on object boundaries.

• The number of pixels added or removed from the objects in an image depends on the size and shape of the structuring element used to process the image.

Operation RuleDilation The value of the output pixel is the maximum value

of all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to the value 1, the output pixel is set to 1.

Erosion The value of the output pixel is the minimum valueof all the pixels in the input pixel's neighborhood. In a binary image, if any of the pixels is set to 0, the output pixel is setto 0.

Structuring Element

• Sometimes referred as the kernel, but the term is reserved for the use in convolutions.

• Consists of a pattern specified as the coordinates of a number of discrete points relative to some origin.

• Cartesian coordinates are (normally) used and so a convenient way of representing the element is as a small image on a rectangular grid.

• Vary in sizes and shapes. In each case the origin is marked by a ring around that point. The origin does not have to be in the center of the structuring element, but often it is.

Dilation

The dilation operator takes two pieces of data as inputs.

The first is the image which is to be dilated. The second is a (usually small) set of coordinate points known as a structuring element (also known as a kernel).

It is this structuring element that determines the precise effect of the dilation on the input image.

Erosion

The erosion operator takes two pieces of data as inputs.

The first is the image which is to be eroded. The second is a structuring element.

The structuring element that determines the precise effect of the erosion on the input image.

Erosion

• One of the more common is to separate touching objects in a binary image so that they can be counted

Original Image Thresholding Erosion using a disk shape structuring element

Segmentation

• edge detection and basic morphology tools are applied to detect one prostate cancer cell.

original image

Image courtesy of Alan PartinJohns Hopkins University

binary gradient mask dilated gradient mask binary image with filled holes

cleared border imagesegmented imageoutlined original image

Binary mask is dilated

Obtaining binary mask , by determining the threshold value and

Sobel operator

Gaps in the interior of the cell are filled

Object near the boundary is removed

The outer region is smoothed using erosion

Cell of interest is segmented by creating and outline.