Digital Image Processing - uliege.be · 2017-05-04 · 04-05-17 3 Image Formation sensors register average color in each picture elt (pixel) = sampling . sampled image 04/05/2017

04-05-17

1

Digital Image Processing:the basics

F. Farnir – E. Moyse

Biostatistics and Bioinformatics

Faculty of Vet Medicine – University of Liege

04/05/2017 1999-2015 by Richard Alan Peters II 1

Acknowledgement:

Most of the material from these introductory notes has been borrowed from the:

« Lectures on Image Processing »

by Alan Peters (Vanderbilt University School of Engineering)

(Much) more material can be found at:

https://archive.org/details/Lectures_on_Image_Processing


Why Digital Image Processing ?

• Much information is acquired through images

• Satellite (land management), (electronic) microscopy (cellsstructures, tissues, …), echography, scanner, …

• Most images are today digital images, that can be storedon computers and displayed on computers screens

• Image manipulation using computers has thus becomepart of the practicioners’work

• This introductory course aims at giving some basic ideasabout the techniques to store and manipulate images


Wallace and Gromit

04/05/2017

1999-2015 by Richard Alan Peters II

4

Wallace

Gromit likes cheeselikes cheese

reads Electronics for Dogsreads Electronics for Dogs

http://www.aardman.com/wallaceandgromit/index.shtml

Wallace and Gromit will be subjects of some of the imagery in this introduction.

An example of digital image:

04-05-17

2

Image FormationImage Formation

04/05/2017


5

From an object to a of digital image:


04/05/2017


6



projection through lensprojection through lens

image of objectimage of object

04/05/2017


7



projection onto discrete sensor array.

projection onto discrete sensor array.

digital cameradigital camera

04/05/2017


8


04-05-17

3


sensors register average color in each picture elt(pixel) = sampling.

sensors register average color in each picture elt(pixel) = sampling.

sampled imagesampled image

04/05/2017


9

From an object to a of digital image: sampling


continuous colors, discrete locations.continuous colors, discrete locations.

discrete real-valued imagediscrete real-valued image

04/05/2017


10

From an object to a of digital image: sampling

04/05/2017 1999-2015 by Richard Alan Peters II

11

continuous color input

continuouscolors mappedto a finite, discrete setof colors.

continuouscolors mappedto a finite, discrete setof colors.

From an object to a of digital image: quantization

Sampling and Quantization

04/05/2017


12

pixel grid

sampledreal image quantized sampled & quantized

From an object to a of digital image: summary

04-05-17

4

Digital ImageDigital Image

04/05/2017


13

a grid of squares, each of which contains a single color

a grid of squares, each of which contains a single color

each square is called a pixel (for picture element)

each square is called a pixel (for picture element)

Color images have 3 values per pixel (R,G,B); monochrome images have 1 value per pixel (level of grey).

Color and B&W images

Color Images

04/05/2017


14

• Are constructed from three intensity maps.

• Each intensity map is projected through a color filter (e.g., red, green, or blue, or cyan, magenta, or yellow) to create a monochrome image.

• The intensity maps are overlaid to create a color image.

• Each pixel in a color image is a three element vector.

Color Images On a CRTColor Images On a CRT


Color Processing

04/05/20171999-2015 by Richard Alan

Peters II16

requires some knowledge of how we see colors

requires some knowledge of how we see colors

04-05-17

5

Eye’s Light Sensors


17

#(blue) << #(red) < #(green)

cone density near fovea

Color Sensing / Color Perception


These are approximations of the responses to the visible spectrum of the “red”, “green”, and “blue” receptors of a typical human eye.




The simultaneous red + blue response causes us to perceive a continuous range of hues on a circle. No hue is greater than or less than any other hue.

The simultaneous red + blue response causes us to perceive a continuous range of hues on a circle. No hue is greater than or less than any other hue.



luminance

huesaturation

photo receptorsbrain

The eye has 3 types of photoreceptors: sensitive to red, green, or blue light.

The brain transforms RGB into separate brightness and color channels (e.g., LHS).



04-05-17

6

Point ProcessingPoint Processing

04/05/20171999-2015 by Richard Alan

Peters II21

original + gamma- gamma + brightness- brightness

original + contrast- contrast histogram EQhistogram mod


DIP – A first example

Edge detection:

“The ability to measure color or gray level transitions in a meaningful way”

Gonzales & Woods, DIP, 2001


DIP – A first example

Edge detection in vet sciences:• Cellular components automatic

detection • Echographic abnormalities• Bacterial lyses detection• …

Gray-Level Transition Ideal Ramp


04-05-17

7

First derivative

Ramp First derivative

Th

Edge detected !


First derivative

Ramp First derivative

Th

Edge not detected !


2-D derivative: the gradient operator

• The gradient of the image I(x,y) at location (x,y), is the vector:

• The magnitude of the gradient:

• The direction of the gradient vector:

( )

( )

∂∂

∂∂

=

=∇

y

yxIx

yxI

G

GI

y

x

,

,

[ ]22yx GGII +=∇=∇

( )

= −

y

x

G

Gyx 1tan,θ

The Meaning of the Gradient

• It represents the direction of the strongest variation in intensity

( ) 0=

=∇

yx

GI x

,θ

The direction of the edge at location (x,y) is perpendicular to the gradient vector at that point

( )2

πθ −=

=∇

yx

GI y

,

Vertical Horizontal Generic

Edge Strength:

Edge Direction:

[ ]( )

=

+=∇

−

x

y

yx

G

Gyx

GGI

1

22

tan,θ

Y

X

04-05-17

8

Calculating the GradientFor each pixel the gradient is calculated, based on a 3x3 neighborhood around this pixel.

Several “edge detectors” exist (see next slides for 3 examples)

z1 z2 z3

z4 z5 z6

z7 z8 z9


The Sobel Edge Detector

-1 -2 -1

0 0 0

1 2 1

-1 0 1

-2 0 2

-1 0 1

( ) ( )321987 22 zzzzzzGy ++−++≈( ) ( )741963 22 zzzzzzGx ++−++≈

The Sobel Edge Detector

• Example

The Prewitt Edge Detector

-1 -1 -1

0 0 0

1 1 1

-1 0 1

-1 0 1

-1 0 1

( ) ( )321987 zzzzzzGy ++−++≈( ) ( )741963 zzzzzzGx ++−++≈

04-05-17

9

The Roberts Edge Detector

0 0 0

0 -1 0

0 0 1

0 0 0

0 0 -1

0 1 0

59 zzGx −≈ 68 zzGy −≈

The Roberts Edge Detector is actually a 2x2 operator !

The Edge Detection Algorithm

• The gradient is calculated (using any of the four methods described in the previous slides), for each pixel in the picture.

• If the absolute value exceeds a threshold, the pixel belongs to an edge.


A simple example, using R

• Installing the « rtiff » package

• Available in CRAN

• Allows to work on TIFF (Tagged Image File Format) images

• Converters from other formats (JPEG, BMP, …) to TIFF are freely available on internetexample: http://www.online-utility.org/image_converter.jsp?outputType=TIFF



• Creating a simple image: code


04-05-17

10


• Creating a simple image: result



• Creating a simple gradient function: code



• Creating a simple gradient function: code



• Obtaining the edges: result


04-05-17

11

Another example, using R

• Here is an example of histological picture

Tissu.TIFF



• Reading the image file

• Obtaining the edges



• Here are the « edges »… More work needed on thresholds !



• With lower thresholds: better ?


04-05-17

12

Point Processing of Images

• In a digital image, a point = a pixel.

• Point processing (PP) transforms a pixel’s value as function of its value alone: pnew = f(pold)

• It does not depend on the values of the pixel’s neighbors.

• So, “gradients” are not PP procedures

• In a digital image, a point = a pixel.

• Point processing (PP) transforms a pixel’s value as function of its value alone: pnew = f(pold)

• It does not depend on the values of the pixel’s neighbors.

• So, “gradients” are not PP procedures


Examples include:

• Brightness and contrast adjustment

• Gamma correction

• Histogram equalization

• Histogram matching

• Color correction.

Examples include:

• Brightness and contrast adjustment

• Gamma correction

• Histogram equalization

• Histogram matching

• Color correction.

Point Processing of Images






Point Processing: Pixel Values


A point process transforms one intensity level (or color) into another as a function of that one alone. So a point process is

( )out in .f=p p

( ) ( )( )out in, , .r c f r c=p p

That is, the pixel value output is dependent on only the pixel value input. That implies

In words, the output at one location is dependent only the value of the input image at that same location. Other locations don’t matter.

04-05-17

13

( )( )( )

If , ,

and

then , , .

r c b g

f g k

r c b k

==

=

I

J

Point Ops via Functional Mappings

Input Output

II Φ, point operatorΦ, point operator JJImage:

I(r,c)I(r,c) function, ffunction, f J(r,c)J(r,c)Pixel:

[ ]=ΦJ I

The transformation of image I into image J is accomplished by replacing each input intensity, g, with a specific output intensity, k, at every location (r,c,b) where I(r,c,b) = g.

The rule that associates k with g is usually specified with a function, f, so that f (g) = k.




One-band ImageOne-band Image

Three-band ImageThree-band Image

J(r,c) = f ( I(r,c) ),

for all pixel locations, (r,c).

J(r,c,b) = f ( I(r,c,b) ), or

J(r,c,b) = fb ( I(r,c,b) )

for b = 1, 2, 3, and all (r,c).

J(r,c) = f ( I(r,c) ),

for all pixel locations, (r,c).

J(r,c,b) = f ( I(r,c,b) ), or

J(r,c,b) = fb ( I(r,c,b) )

for b = 1, 2, 3, and all (r,c).

One-band ImageOne-band Image

Three-band ImageThree-band Image


Either all 3 bands are mapped through the same function, f, or …

Either all 3 bands are mapped through the same function, f, or …

… each band is mapped through a separate func-tion, fb.

… each band is mapped through a separate func-tion, fb.


Lookup Tables


A lookup table is an indexed list of numbers – a vector – that can be used to implement a discrete function – a mapping from a set of integers, {gin,1, gin,2, ... ,gin,n}, to a set of numbers (integers or not), {gout,1, gout,2, ... ,gout,n}. A lookup table can implement a function such as:

( ) { } { }( ) ( )

out in in out out, 1

in, out,

if , where 0, , 1 and

then define LUT 1 LUT , for 1, n .

n

k k

k k

g f g g n g g

g k g k

== ∈ − ∈

+ = = =

…

…

04-05-17

14

J = LUT(I+1)

Point Operations using Lookup Tables

A lookup table (LUT)can implement a functional mapping.

A lookup table (LUT)can implement a functional mapping.

( ),255,,0

,

…==g

gfk

for

If

{ }…… in values

on takes if and

,255,,0

k

… then the LUTthat implements fis a 256x1 arraywhose (g +1)th

value is k = f (g).

To remap an image, I, to J :


LUT is 256x1. But I may be RxC or RxCx3.

LUT is 256x1. But I may be RxC or RxCx3.

Point Operations = Lookup Table Ops

0 127 255input value

index value

...101102103104105106...

...646869707071...

E.g.:

input output


How to Generate a Lookup Table

{ }( )

32/)127(1255

;

255,,0

.2

−−+=

∈=

xaeax

x

a

σ

… Let

Let

For example, a sigmoid:

a <- 2;x <- 0:255;LUT = 255 ./ (1+exp(-a*(x-127)/32));

Or in R:

This is just one example.This is just one example.


If I is 3-band, then

a) each band is mapped separately using the same

LUT for each band or

b) each band is mapped using different LUTs – one for

each band.

If I is 3-band, then

a) each band is mapped separately using the same

LUT for each band or

b) each band is mapped using different LUTs – one for

each band.

Point Ops on RGB Images using Lookup Tables


a) J = LUT(I+1),

b) J(:,:,b) = LUTb(I(:,:,b) +1), for b = 1, 2, 3.

a) J = LUT(I+1),

b) J(:,:,b) = LUTb(I(:,:,b) +1), for b = 1, 2, 3.

04-05-17

15

0 0

64 32

128 128

192 224

255 255

...

...

...

...

...

...

...

...

1-Band Lookup Table for 3-Band Image

input output

a pixel with this value

a pixel with this value

is mapped to this value

is mapped to this value


Example 3-Band Image …


Lightning at Ramasse, Rhone-Alpes, France by Flickr user, Regarde là-bas,

https://www.flickr.com/photos/marcel_s_s/8624344496/in/pool-tbasab/

RR

GG

BB

R, G, B Bands




RR

GG

BB

… with 3-Band LUT.




3-Band Image with 3-Band LUT.

04-05-17

16


Example of 3-Band Image with 3-Band LUT. Example of 3-Band Image with 3-Band LUT.






Point Processes: Original Image

Kinkaku-ji (金閣寺, Temple of the Golden Pavilion), also known as Rokuon-ji (鹿苑寺, Deer Garden Temple), is a Zen Buddhist temple in Kyoto, Japan. Photo by Alan Peters, August 1993.

Luminance Histogram


For more information on this fascinating, unique

place read the historical novel by Mishima,Yukio,

The Temple of the Golden Pavilion, translated by

Ivan Morris, Shinchosha Publishing Co, Ltd., 1956.

04-05-17

17

( ) ( ) ( )( )

if , , 256, , , , ,

if , , 255255,

r c b gr c b gr c b

r c b g

+ < += + >

IIJ

I

Point Processes: Increase Brightness

0 127 255

g

LUT Mapping{ }0 and 1 2 3 is the band index.g b , ,≥ ∈

saturation point


Point Processes: Decrease Brightness

0 127 255

LUT Mapping

25

5-g

{ }0 and 1 2 3 is the band index.g b , ,≥ ∈

( ) ( )( )( )

0, if , , 0, ,

, , , if , , 0

r c b gr c b

r c b g r c b g

− <= − − >

IJ

I I

zero point


Point Processes: Decrease Contrast

( )( , , ) , , , r c b a r c b s s= − + T I

0 127 255

LUT Mapping

{ }{ }

where 0 1.0,

s 0,1, 2, ,255 , and

1,2,3 .

a

b

≤ <∈

∈

…

Here, s = 127Here, s = 127

s is the center of the contrast function.

s is the center of the contrast function.


Point Processes: Increase Contrast

( ) ( )( )

( )( )

0, if , , 0,

, , , , , if 0 , , 255,

255, if , , 255.

r c b

r c b r c b r c b

r c b

<= ≤ ≤ >

T

J T T

T

( )( , , ) , ,r c b a r c b s s= − + T I

0 127 255

LUT Mapping{ } { }1, s 0, ,255 , 1 2 3a b , ,> ∈ ∈…

zero point

sat. point

Here, s = 127Here, s = 127


04-05-17

18

Point Processes: Contrast Stretch

( ) ( )( ) ( )

( ) ( ) ( )

Let min , , max , ,

min , , max , .

Then,,

, .

m r c M r c

m r c M r c

r c mr c M m m

M m

= =

= =

−= − +

−

I I

J J

IJ J J

I I

I I

J J

IJ

0 127 255

LUT MappingmI M I

mJ

MJ zero point

sat. point

center pointcenter point


Information Loss from Contrast Adjustment

origorig

lo-clo-c

hi-chi-c

histogramshistograms


Information Loss from Contrast Adjustment

origorig

origorig

origorig

lo-clo-c hi-chi-c

lo-clo-c

hi-chi-c

restrest

restrest

lo-clo-c

hi-chi-c

diff diff

diffdiff

abbreviations:originallow-contrasthigh-contrastrestoreddifference

abbreviations:originallow-contrasthigh-contrastrestoreddifference

difference between original and restored low-contrast

difference between original and restored low-contrast

difference between original and restored high-contrast

difference between original and restored high-contrast


Point Processes: Increased Gamma

0 127 255

LUT mapping

( ) ( )1

,, 255 for 1.0

255

r cr c γ γ

= ⋅ >

IJ


04-05-17

19

Point Processes: Decreased Gamma

0 127 255

LUT mappingm M( ) ( )

1

,, 255 for 0 < 1.0

255

r cr c

= ⋅ <

IJ γ γ


Point Processes: Decreased Gamma

0 127 255

LUT mappingm M( ) ( )

1

,, 255 for 0 < 1.0

255

r cr c

= ⋅ <

IJ γ γ


Gamma correction can also be

defined as 255[I/255]γ

rather than 255[I/255]1/γ. The effects are opposite.

Gamma correction can also be

defined as 255[I/255]γ

rather than 255[I/255]1/γ. The effects are opposite.

Gamma Correction: Effect on Histogram


The Probability Density Function of an Image

( )255

0

Let 1 .k

g

A h g=

= +∑ I

( )Note that since 1 is the number of pixels in (the th color band of image ) with value , is the number of pixels in . That is if is rows by columns then .

k

k

h gk g

AR C A R C

+

= ×

I

I II I

( ) ( )

Then,

1 1 1

is the graylevel probability density function of .

k k

k

p g h gA

+ = +I I

I

This is the probability that an arbitrary pixel from Ik has value g.

This is the probability that an arbitrary pixel from Ik has value g.

pdf[lower case]

pdf[lower case]


04-05-17

20

• pband(g +1) is the fraction of pixels in (a specific band of) an image that have intensity value g.

• pband(g +1) is the probability that a pixel randomly selected from the given band has intensity value g.

• Whereas the sum of the histogram hband(g +1) over all g from1 to 256 is equal to the number of pixels in the image, the sum of pband(g+1) over all g is 1.

• pbandis the normalized histogram of the band.

• pband(g +1) is the fraction of pixels in (a specific band of) an image that have intensity value g.

• pband(g +1) is the probability that a pixel randomly selected from the given band has intensity value g.

• Whereas the sum of the histogram hband(g +1) over all g from1 to 256 is equal to the number of pixels in the image, the sum of pband(g+1) over all g is 1.

• pbandis the normalized histogram of the band.

The Probability Density Function of an Image


The Probability Distribution Function of an Image

( ) ( ) ( )( )

( )0

2550 0

0

1 1

1 1 1 ,1

k

k k k

k

g

g g

I

h

P g p hA

h

γ

γ γ

γ

γγ γ

γ

=

= =

=

++ = + = + =

+

∑∑ ∑

∑

I

I I

I

This is the probability that any given pixel from Ik has value less than or equal to g.


PDF[upper case]

PDF[upper case]

Let q = [q1 q2 q3] = I(r,c) be the value of a

randomly selected pixel from I. Let g be a

specific graylevel. The probability that qk ≤ g

is given by

where hIk(γ +1) is

the histogram of

the kth band of I.


The Probability Distribution Function of an Image

( ) ( ) ( )( )

( )0

2550 0

0

1 1

1 1 1 ,1

k

k k k

k

g

g g

I

h

P g p hA

h

γ

γ γ

γ

γγ γ

γ

=

= =

=

++ = + = + =

+

∑∑ ∑

∑

I

I I

I

Let q = [q1 q2 q3] = I(r,c) be the value of a

randomly selected pixel from I. Let g be a

specific graylevel. The probability that qk ≤ g

is given by

where hIk(γ +1) is

the histogram of

the kth band of I.

Also called CDF for “Cumulative Distribution Function”.

Also called CDF for “Cumulative Distribution Function”.




The Cumulative Distribution Function of an Image

• Pband(g+1) is the fraction of pixels in (a specific band of) an image that have intensity values less than or equal to g.

• Pband(g+1) is the probability that a pixel randomly selected from the given band has an intensity value less than or equal to g.

• Pband(g+1) is the cumulative (or running) sum of pband(g+1) from 0 through g inclusive.

• Pband(1) = pband(1) and Pband(256) = 1; Pband(g+1) is nondecreasing.

• Pband(g+1) is the fraction of pixels in (a specific band of) an image that have intensity values less than or equal to g.

• Pband(g+1) is the probability that a pixel randomly selected from the given band has an intensity value less than or equal to g.

• Pband(g+1) is the cumulative (or running) sum of pband(g+1) from 0 through g inclusive.

• Pband(1) = pband(1) and Pband(256) = 1; Pband(g+1) is nondecreasing.

Note: the Probability Distribution Function (PDF, capital letters) and the Cumulative Distribution Function (CDF) are exactly the same things. Both PDF and CDF will refer to it. However, pdf (small letters) is the density function.


04-05-17

21

The pdf vs. the CDF


pdf = normalized value histogram

pdf = normalized value histogram

CDF = running sum of pdf

CDF = running sum of pdf

Point Processes: Histogram Equalization

be the cumulative (probability) distributionfunction of I. Then J has, as closely as possible, a flat (constant) histogram if:

( )Let 1 P γ +I

( ) ( ), , 255 , , 1 .r c b P r c b= ⋅ + IJ I

Task: remap a 1-band image I so that its histogram is as close to constant as possible. This maximizes the contrast evenly across the entire intensity range.

one-band imageone-band imageThe scaled CDF itself is used as the LUT.


That is, to equalize a one-band image, map it through its own CDFmultiplied by the maximum desired output value.

Point Processes: Histogram Equalization

( ) ( ) ( ) ( )( ) ( )

, 1 1, .

1 1

P r c P mr c M m m

P M P m

+ − + = − ++ − +

I I IJ J J

I I I I

IJ

Task: remap image I with min = mI and max = MI so that its histogram is as close to constant as possible and has min = mJand max = MJ .

Then J has, as closely as possible, the correct histogram if

Using intensity extrema

Using intensity extrema

be the cumulative (probability) distribution function of I.

( )Let 1 P γ +I


pdf

Histogram EQ

The CDF (cumulative distribution) × 255 is the LUT for remapping.


CDF


Value Image

Histogram EQ’d Value Image

04-05-17

22

pdf

Histogram EQ

LUT


Value Image

Histogram EQ’d Value Image



Histogram EQ: a R example




tif<-readTiff("lc2.tiff")# Obtain histogramhist(255*tif@red,breaks=0:255,main="Gray levels

histogram",xlab="Gray level")->hhc<-h$countsfor (i in 2:255) { hc[i]<-hc[i-1]+hc[i] }# Obtain CDFhc<-hc/hc[255]# Equalizingtif2<-tifd<-dim(tif@red)for (i in 1:d[1]) {

for (j in 1:d[2]) {tif2@red[i,j]<-hc[floor(255*tif@red[i,j])]tif2@green[i,j]<-hc[floor(255*tif@green[i,j])]tif2@blue[i,j]<-hc[floor(255*tif@blue[i,j])]

}}



Image CDF

04-05-17

23





# Let's generate a random 1-band imageimage<-rnorm(10000,127,50)image[image<0]<-0image[image>255]<-255img<-matrix(image,nr=100)# Histogramhist(image,breaks=0:255,plot=FALSE)->h# Cumulative density functioncdf<-h$countsfor (i in 2:255) {

cdf[i]<-cdf[i]+cdf[i-1]}cdf<-cdf/cdf[255]# Histogram EQimage2<-round(255*cdf[image])# Compare histogramssplit.screen(c(2,1))screen(1)hist(image,breaks=0:255,col="blue",main="Original h istogram")screen(2)hist(image2,breaks=0:255,col="red",main="Transforme d histogram")

Histogram EQ of the Individual Bands


Original Color Image Color Histograms

One histogram for each band.One histogram for each band.



Original Color Image Color CDFs

04-05-17

24



Original Color Image Equalization LUTs

Each band is mapped through its own LUT.Each band is mapped through its own LUT.



Equalized Color Image Histogram of Eq’d Image



Original Color Image Equalized Color Image

Note the unnatural color shifts.Note the unnatural color shifts.

DIP: what’s next ?


Many things…� Color corrections� Fourier transforms

� Low-Pass and High-Pass Filters� Convolutions� Compression� … and much more

Interested students can contact us for further information

Thanks for listening…

Documents

Digital Image Processing - uliege.be · 2017-05-04 · 04-05-17 3 Image Formation sensors register average color in each picture elt (pixel) = sampling . sampled image 04/05/2017