Antal Nagy Department of Image Processing and Computer Graphics University of Szeged 17th SSIP 2009,...

Antal NagyDepartment of Image Processing

and Computer GraphicsUniversity of Szeged

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 1

Human perception Image degradation Convolution, Furier Transform Noise Image operations

◦ Frequency filters◦ Spatial filtering

Inverse filtering Wiener filtering

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 2

Aim◦ to improve the perception

of information images for human viewers

◦ to provide ‘better’ input for other automated image

processing techniques

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 3

No general theory for determining what is good image enhancement◦ If it looks good, it is good!?

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 4

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 5

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 6

http://www.youtube.com/watch?v=_d_l5nsnIvM

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 7

Focus◦ Noise reduction techniques

Quantitative measures can determine which techniques are most appropriate◦ How does it improve e.g.

the result of the next automated image processing step? E.g. image segmentation

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 8

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 9

Non-linear mapping◦ E.g., non-linear sensitivity, image of the straight

line is not straight e.t.c. Blurring

◦ Image of a point is blob Moving during the image acquisition Probabilistic noise

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 10

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 11

),(),(),(),(

),(),(),(),(

vuNvuFvuHvuG

yxyxfyxhyxg

Frequency domain

Spatial domain

◦ Multiplication point by point

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 12

)()()*( hFfFhfF

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 13

* =

=·

Multiplication

Convolution

Fourier transf.Inverse

Fourier transf.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 14

dvduvyuxgvufyxgf

duuxgufxgf

,),(),)((

)( ))((

Definition

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 15

ationdifferenti '*'*)'*(

vitydistributi )*()*()(*

ityassociativ *)*()*(*

multipl.scalar ity withassociativ )(**)()*(

itycommutativ **

gfgfgf

hfgfhgf

hgfhgf

gafgfagfa

fggf

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 16

otherwise,0

,2/1if,1)(

xxg

2/1

2/1

)()(x

x

dfxgf

x

x

x

)(xf

Even functions that are not periodic can be expressed as the integrals of sines and/or cosines multiplied by a weighting function.

The formulation in this case is the Fourier transform.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 17

∑∑ ==

Taught mathematics in Paris

Eventually traveled to Egypt with Napoleon to become the secretary of the Institute of Egypt

After fall of Napoleon worked at Bureau of Statistics

Elected to National Academy of Sciences in 1817

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 18

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 19

dueuFxf

dxexfuF

ixu

ixu

2

2

)()(

)()(

),(1 Lf

(invers transform)(invers transform)

(continous)(continous)

base-functionsbase-functions

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 20

dudvevuFyxf

dydxeyxfvuF

yvxui

yvxui

)(2

)(2

),(),(

),(),(

base-functionsbase-functions

(invers transform)(invers transform)

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 21

))(2cos(

Re )(2

vyuxi

e vyuxi

u=0, v=0 u=1, v=0 u=2, v=0u=-2, v=0 u=-1, v=0

u=0, v=1 u=1, v=1 u=2, v=1u=-2, v=1 u=-1, v=1

u=0, v=2 u=1, v=2 u=2, v=2u=-2, v=2 u=-1, v=2

u=0, v=-1 u=1, v=-1 u=2, v=-1u=-2, v=-1 u=-1, v=-1

u=0, v=-2 u=1, v=-2 u=2, v=-2u=-2, v=-2 u=-1, v=-2

u

v

wavelength:

22/1 vu

F(0,0) - value is by far the largest component of the image,

Other frequency components are usually much smaller,

The magnitude of F(X,Y) decreases quickly◦ Instead of displaying the |F(u,v)| we display

log( 1 + |F(u,v)| ) real function usually

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 22

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 23

x

y

v

u

),(1log vuF),( yxf

The 2D Fourier transform can be separated

The edges on the image appears as point series in perpendicular direction in Fourier transform of the image and vice versa.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 24

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 25

Image-spaceImage-space

Frequency spaceFrequency spaceooriginal rotation linearity riginal rotation linearity shiftshift scale scale

Noise unknown subtraction not possible Periodic noise

◦ N(u,v) can be estimated from G(u,v)

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 26

),(),(),(

and

),(),(),(

vuNvuFvuG

yxyxfyxg

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 27

Gaussian◦ In an image due to

factors Electronic circuit

noise Sensor noise due to

poor illumination High temperature

Rayleigh◦ Range imaging

Exponential and gamma◦ Laser imaging

Impulse◦ Faulty switching

Uniform density◦ Practical situations

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 28

22 2/)(

2

1)(

noiseGaussian

zzezp

azfor 0

for )(2

)(

noiseRayleigh

/)( 2

azeazbzp

baz

4

)4(

and

4/

2

b

baz

0zfor 0

0for )!1(

ap(z)

noise (gamma) Erlang1b

zeb

z azb

22

and

a

b

a

bz

0a where

0zfor 0

0for )(

noise lExponentia

zae

zpaz

22 1

and

1

a

az

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 29

otherwise 0

if 1

)(

Noise Uniform

bzaabzp

12

and2

22 ab

baz

otherwise 0

for

for

)(

noise peper)-and-(salt Impulse

bzP

azP

zp b

a

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 30

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 31

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 32

Electrical and electromechanical interference

Spatial dependent noise Can be reduced via

frequency domain filtering ◦ Pair of impulses

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 33

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 34

T: A=[a(i,j)] → B=[b(i,j)]

b(i,j)=T{a(i,j), S(i,j), i, j}

intensity enviroment position

Global: b(i,j)=T{A} (S(i,j)=A) (e.g. Fourier-transformation)

Local: T{a(i,j), S} given size of S and independent from the position (e.g. convolution with a mask)

Local, adaptive: T{a(i,j), S(i,j), i, j} the size of S(i,j) is independent from the size of image (e.g. adaptive thresholding)

Point operation: T{a(i,j)} (e.g. gamma-correction, histogram-equalization)

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 35

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 36

D0: cutoff frequency

All frequencies less than D0 will be passed,

Other frequencies will be filtered out.Bluring and ringing propertiesScope: noise filtering

otherwise,0

if,1),(

20

22 DYXYXH ILPF

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 37

F

F-1.

Input image

Frequency-mask

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 38

Original 5

15 30

80 230

Cutoff frequencies

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 39

n: order of the filter

Properties: Smooth transition in blurring No ring effect (continouos filter) Smoothed edges

nBLPF

DYX

YXH 2

0

22

1

1),(

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 40

Original 5

15 30

80 230

Cutoff frequenciesCutoff frequencies

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 41

022 2/)(),( Dvu

GLPF evuH

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 42

Original 5

15 30

80 230

Cutoff frequenciesCutoff frequencies

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 43

),(1),( vuLPFvuHPF

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 44

Bandreject and Bandpass Filters Notch Filters

◦ Rejects or passes frequencies in a predefined neighborhood about the frequency rectangle

◦ Zero-phase-shift filters Symmetric about the origin

(u0,v0) (-u0,-v0)

◦ Product of highpass filters whose centers have been translated to the centers of the notches.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 45

Q

kkkNR vuHvuHvuH

1

),(),(),(

),(1),( vuHvuH NRNP

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 46

noisyimage

frequencymask

frequencyimage

filteredimage

11

00

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 47

Mean Filter

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 48

where g input image, S(x,y) neighborhood of (s,t) point,

mn number of pixels in neighborhood.

3x3 neighborhood

1

1

1

1

),(9

1),(ˆ

u v

vyuxgyxf

xySts

tsgmn

yxf),(

),(1

),(ˆ

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 49

111

111

111

9

1

:where

),,)((

),(),(

),(9

1),(ˆ

1

1

1

1

1

1

1

1

h

jihf

vuhvyuxg

vyuxgyxf

u v

u v

AvAveerraagingging ◦ same weight for every pixels in neighborhood,same weight for every pixels in neighborhood,

Weighted averageWeighted average ◦ weights for pixels in the neighborhood (generally decreasing with weights for pixels in the neighborhood (generally decreasing with

the distance).the distance).

The sum of the Noise Filtering/smoothing The sum of the Noise Filtering/smoothing masks elements is 1! masks elements is 1!

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 50

121

242

121

16

1

111

111

111

9

1

Smooth when the difference between the intensity value of the given pixel and the mean of the neighborhood is larger than threshold value defined in advance.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 51

otherwise),,(

,),(),( if),,(),('

jif

Tjifjigjigjig

Arithmetic mean filter

Geometric mean filter◦ Lose less image details

Harmonic mean filter◦ Works well for

salt noise, Gaussian◦ Fails for pepper noise

Contraharmonic mean filter◦ Q>0 pepper, Q<0 salt

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 52

xySts

tsgmn

yxf),(

),(1

),(ˆ

mn

Sts xy

tsgyxf

1

),(

),(),(ˆ

xySts tsg

mnyxf

),( ),(1

),(ˆ

xy

xy

Sts

Q

Sts

Q

tsg

tsg

yxf

),(

),(

1

),(

),(

),(ˆ

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 53

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 54

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 55

Median filter

Max and min filters

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 56

),(median),(ˆ),(

tsgyxfxySts

),(min),(ˆ

),(max),(ˆ

),(

),(

tsgyxf

tsgyxf

xy

xy

Sts

Sts

50%

0%

100%

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 57

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 58

Behavior changes based on statistical characteristic in the filter region◦ Improved filtering power◦ Increase in filter complexity◦ Noise only!

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 59

Adaptive, Local noise reduction filter◦ Mean◦ Variance

Local region Sxy

◦ g(x,y) intensity value◦ the variance of the noise corrupting f(x,y) to

form g(x,y) ◦ mL local mean◦ local variance

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 60

2

2L

Adaptive, Local noise reduction filter1. If is zero, the filter should return the value of

g(x,y) Zero noise case

2. If the local variance is high relative to the filter should return a value close to g(x,y)

Edges should be preserved3. If the variances are equal the filter should

return the arithmetic mean value Local noise averaging

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 61

2

2

Simplest approach to restoration is direct inverse filtering

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 62

experience:H: quickly decreasing function,N: not (so quickly) decreasinglet us cut the high frequencies

),(

),(),(ˆ

vuH

vuGvuF

),(

),(),(

),(

),(),(

),(

),(),(ˆ

vuH

vuNvuF

vuH

vuNvuF

vuH

vuGvuF

No additive noise

If we only know the degradation function◦ Can not recover the undegraded image

H(u,v ) is not known Get around the zero or small value problem

◦ To limit the filter frequencies to values near the origin H(0,0) is usually the highest value of H(u,v) in the

frequency domain

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 63

Inverse filtering makes no explicit provision for handling noise

Approach incorporates◦ Degradation function◦ Statistical characteristics of noise

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 64

Method◦ Considering images and noise as random variable◦ Objective is to find an estimate of the

uncorrupted image f such that the mean square error between them is minimized.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 65

f̂

argument theof value

expected theis where

ˆ 22

E

ffEe

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary

66

spectrum

Wiener filter

input

result

original

spectrum of the result

Edge preserving filters◦ E.g. anisotropic diffusion

Wavelet denoising Point operations

◦ E.g. gamma correction, histogram operations Iterative filtering E.t.c.

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 67

We always should consider some kind of noise model◦ Even when working on phantom data

Should do in automatic way◦ Have to chose carefully the method

Depends on the given task◦ Determining the parameters

What we gain?◦ Less problem afterwards◦ Better final result

Even no other technique will be applied

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 68

Digital Image Processing◦ Gonzalez and Woods◦ www.ImageProcessingPlace.com

Course on Image Processing at University of Szeged◦ Attila Kuba, Kálmán Palágyi

Image Restoration presentation◦ Attila Kuba◦ SSIP 2006

17th SSIP 2009, 2 - 11 July, Debrecen, Hungary 69