Discrete Fourier Transform and Filteringvca.ele.tue.nl/sites/default/files/5XSF0 Mod 03... · (5XSF0), Module 03 Discrete Fourier Transform and Filtering Peter H.N. de With ([email protected]

1

1

PdW-SZ / 2016

Fac. EE SPS-VCA

Enabling Technologies for Sports /

5XSF0 / Module 03 DFT & Filtering

Enabling Technologies for Sports

(5XSF0), Module 03

Discrete Fourier Transform and Filtering

Peter H.N. de With([email protected] )

slides version 1.0

2

PdW-SZ / 2016

Fac. EE SPS-VCA



Overview Module 03

∗ Image Frequency Domain

– Fourier transform

– Properties

∗ Image filtering based on frequency domain

– Various filter types and their response (Gaussian, Laplace..)

– Noise reduction by frequency domain filtering

2

3

PdW-SZ / 2016

Fac. EE SPS-VCA



Module 03 – Part 1

Frequency domain representation

Sampling and Fourier transform, 2-D

extensions, convolution, aliasing,

filtering concept

4

PdW-SZ / 2016

Fac. EE SPS-VCA



Image waveform concept of Fourier

Image: essentially a 2-D

signal waveform

∗ Signal: decomposed in

a sum of components

– Fourier found in 1807

that any periodic or

‘limited energy’

waveform: can be a

weighted sum of sines

and cosines

3

5

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier series and impulses

Characterization of a video signal f(t)∗ Assume function f(t) with period T, which can be expressed as sum

of sines and cosines, with coefficients (n=0, +/- 1, +/- 2, …)

∗ 2. Unit impulses

∗ 3. Sifting process

∗ Exercise: derive discrete versions!

+∞

−∞=

=n

Tntj

nectf

/2)(

πdtetf

Tc

T

T

Tntj

n +

−

−=2/

2/

/2)(

1 π

;)0();0(0)( ∞=≠= δδ tt 1)( =+∞

∞−

dttδ

)()()(00

tfdttttf =−+∞

∞−

δ

6

PdW-SZ / 2016

Fac. EE SPS-VCA



Signal is series of (weighted) impulses

Characterization of a video signal

∗ Assume signal s(t) with sampling period T, which can be expressed as sum impulses, with signal coefficients(n=0, +/- 1, +/- 2, …)

∗ If weights are sn=1, then

equal to sampling function!

+∞

−∞=∆

∆−=n

nTTntsts )()( δ

4

7

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier transform (one cont. variable)

Fourier transform of a signal f(t)∗ Assume function f(t) with of time t, which

∗ 2. Inverse Fourier transform

∗ 3. The equations form a transform pair. Note that with Euler, it also holds that

+∞

∞−

−= dtetfFftj πω 2

)()(

+∞

∞−

+= ωω πdeFtf

ftj 2)()(

)2sin()2cos(2 ftjfte ftj πππ +=

Note that ω is

replaced here to

frequency f in

many books!

If f(t) is real,

then the

transform is

complex!

8

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier transform (cont.) - Example

[ ])(

)sin(

2)(

2/

2/

2

2/

2/

2

fW

fWAWe

fj

AdtAewF

W

W

ftj

W

W

ftj

π

π

πππ =

−==

+

−−

+

−

−

Frequency spectrum is magnitude of transform!

µ = f in the

drawing!

5

9

PdW-SZ / 2016

Fac. EE SPS-VCA



Sampling of a signal – (1)

Sampling of

• continous

signal f(t)

• time period

∆T (or Ts)

• results in

weighted

impulse train

+∞

−∞=

∆−=∆=n

nTnttfTnff )()()( δ

10

PdW-SZ / 2016

Fac. EE SPS-VCA



Sampling of a signal – (2)

Sampling

speed

such that

spectra fit!

• Signal

should be

bandwidth-

limited

µ=f in the

drawing!

6

11

PdW-SZ / 2016

Fac. EE SPS-VCA



Sampling of a signal – (3) Theorem

Reconstruction is possible if one

spectrum can be extracted, thus

max22

12 f

T>

∆

Sampling at 2x

fmax is called the

Nyquist rate

µ=f in the

drawing!

12

PdW-SZ / 2016

Fac. EE SPS-VCA



Sampling Theorem (& Nyquist rate)

Note that µ is

here frequency f

in the drawing!

Assume here

maxmax

)(

fff

TfH

+≤≤−

∆=

7

13

PdW-SZ / 2016

Fac. EE SPS-VCA



Under-Sampling

Under-sampling

gives overlapping

frequency areas

• HF becomes LF!

• Overlap is called

aliasing

• Aliasing always

occurs but can be

suppressed with

low-pass pre-

filters

µ=f in the

drawing!

14

PdW-SZ / 2016

Fac. EE SPS-VCA



Discrete Fourier Transform (one variable)

Discrete Fourier Transform of a signal f(t)∗ Assume function f(t) sampled at time ∆T

∗ 2. Inverse Discrete Fourier transform

∗ 3. This can be derived with taking M samples in interval of frequency

1,...,2,1,01

0

/2 −== −

=

−MmefF

M

n

Mmnj

nm

π

1,...,2,1,01

0

/2 −== −

=

+MneFf

M

m

Mmnj

mn

π

TM

mfMm

Tf

∆=−=

∆≤≤ ;1,...,2,1,0;

10

8

15

PdW-SZ / 2016

Fac. EE SPS-VCA



1-D Discrete Fourier Transform (mod. Not.)

Discrete Fourier Transform of an image signal I(x, y)∗ Assume function I(t) sampled at discr. times x∆T and discr. frequency u

∗ 2. Inverse Discrete Fourier transform

∗ 3. This is the common notation in many books and remainder of the course!

1,...,2,1,0)()(1

0

/2 −== −

=

−MuexfuF

M

x

Muxj π

1,...,2,1,0)(1

)(1

0

/2 −== −

=

+MxeuF

Mxf

M

u

Muxj π

16

PdW-SZ / 2016

Fac. EE SPS-VCA



Properties of 1-D DFT

∗ 1. Both DFT and IDFT are infinitely periodic

∗ 2. Assuming that f(x) is real and m samples are taken, then the DFT gives a set of m complex numbers

∗ 3. It can be shown that the discrete equivalent of convolution is

∗ 4. If f(x) consists of M samples of f(t) taken ∆T units apart, then period duration T=M∆T and ∆u=1/(M∆T)=1/T

)()()()( kMxfxfkMuFuF +=+=

−

=

−=⊗1

0

)()()()(M

m

mxhmfxhxf

9

17

PdW-SZ / 2016

Fac. EE SPS-VCA



1-D DFT computation example∗ 1. Compute F(0), F(1) etc. of the DFT

– F(0)=1+2+4+4=11, F(1)=-3+2j,

– F(2)=-(1+0j), F(3)=-(3+2j)

∗ 2. The inverse DFT gives the computation of f(0)– f(0)=1/4[ 11-3+2j-1-3-2j ]=1

18

PdW-SZ / 2016

Fac. EE SPS-VCA



Extensions to 2-D DFT – (1)

∗ 1. Sampling over 2-D area (image) instead of signal and impulse train becomes 2-D signal

∗ 2. DFT: summing over pixels (x, y), and 2 param’s m, n

∗ DFT periodic in 2-D frequency domain, param’s (u, v)

∗ 2-D sampling theorem: 2 constraints for band-limitations

maxmax 21

;21

vh fZ

fT

>∆

>∆

10

19

PdW-SZ / 2016

Fac. EE SPS-VCA



Extensions to 2-D DFT – (2)

∗ 1. Ideal footprint of 2-D LPF becomes rectangular

∗ 2. Aliasing becomes 2-D pattern (e.g. Moire) µ=f h and

ν= fv in the

drawings!

20

PdW-SZ / 2016

Fac. EE SPS-VCA



Extensions to 2-D DFT – (3) / Moire

11

21

PdW-SZ / 2016

Fac. EE SPS-VCA



Extensions to 2-D DFT – (4) / Moire

22

PdW-SZ / 2016

Fac. EE SPS-VCA



2-D Discrete Fourier Transform

∗ 2-D Discrete Fourier Transform of image signal f(x, y) for 0<=u<=M-1 and 0<=v<=N-1 is

∗ 2. 2-D Inverse Discrete Fourier Transform

∗ 3. The above equations hold for an image of size M x Npixels and together they form a transform pair

−

=

+−−

=

=1

0

)//(21

0

),(),(N

y

NvyMuxjM

x

eyxfvuFπ

−

=

++−

=

=1

0

)//(21

0

),(1

),(M

u

NvyMuxjN

v

evuFMN

yxfπ

12

23

PdW-SZ / 2016

Fac. EE SPS-VCA



Properties of 2-D DFT – (1)

∗ 2-D Spatial and frequency intervals using image signal f(x, y) of size M x N

∗ Translation and rotation

∗ Likewise, a rotation over an angle in f(x,y) gives an equal rotation of F(u, v) (this can be proven with polar coordinates with x=r cos(θ) etc., and u=w cos (ϕ), etc.)

ZNv

TMu

∆=∆

∆=∆

1;

1

0 0

0 0

2 ( / / )

0 0

2 ( / / )

0 0

( , ) ( , )

( , ) ( , )

j ux M vy N

j u x M v y N

f x y e F u u v v

f x x y y F u v e

π

π

+ +

− +

⇔ − −

− − ⇔

24

PdW-SZ / 2016

Fac. EE SPS-VCA



Properties of 2-D DFT – (2)

∗ 2-D Periodicity of the DFT spectra

∗ As a special case (use Euler rule for this with u0=M/2)

∗ As a result, the spectrum shifts such that the F(0,0) is now

at the origin!

etcNkyxfyMkxfyxf

etcNkvuFvMkuFvuF

=+=+=

=+=+=

),(),(),(

.),(),(),(

21

21

)2/,2/()1)(,( NvMuFyxfyx −−⇔− +

13

25

PdW-SZ / 2016

Fac. EE SPS-VCA



Properties

of 2-D DFT

– (3)

Centering

Multiplication

with (-1)x+y

relocates

F(0,0) to the

origin

µ=f in the

drawings!

26

PdW-SZ / 2016

Fac. EE SPS-VCA



Properties

of 2-D DFT

– (4)

Symmetry

14

27

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier spectrum and phase 2-D DFT

∗ 2-D DFT is complex, it can be expressed in polar form

∗ Where the magnitude is the Fourier spectrum and φ the

phase angle

∗ Finally, the power spectrum is

),(),(),(

vujevuFvuF

φ=

=

+=

),(

),(arctan),(

),(),(),(22

vuR

vuIvu

vuIvuRvuF

φ

),(),(),(),(222

vuIvuRvuFvuP +==

28

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier spectrum and phase – Examples

∗ Typically, f(x,y) is real, which implies that 2-D DFT

spectrum is even symmetric about the origin

∗ and φ the phase angle is odd symmetric about the origin

∗ Finally, note that

),(),( vuFvuF −−=

),(),( vuvu −−= φφ

),(),(1

)0,0(1

0

1

0

yxfMNyxfMN

MNFM

x

N

y

== −

=

−

=

15

29

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier spectrum – Example 1

30

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier spectrum – Example 2a

16

31

PdW-SZ / 2016

Fac. EE SPS-VCA



Fourier spectrum – Example 2b Phase

32

PdW-SZ / 2016

Fac. EE SPS-VCA



Summary

2-D DFT

Proper’s 1

17

33

PdW-SZ / 2016

Fac. EE SPS-VCA



Summary

2-D DFT

Proper’s 2

34

PdW-SZ / 2016

Fac. EE SPS-VCA



Summary 2-D DFT Properties – (3)

18

35

PdW-SZ / 2016

Fac. EE SPS-VCA




∗ And remember that due to Euler’s formula and the

cosine/sine rules:

∗ And

∗ Finally, note that

[ ]

[ ]),(),(2

1)22cos(

),(),(2

)22sin(

000000

000000

NvvMuuNvvMuuyvxu

NvvMuuNvvMuuj

yvxu

−−+++⇔+

−−−++⇔+

δδππ

δδππ

1),( ⇔yxδ

36

PdW-SZ / 2016

Fac. EE SPS-VCA




∗ Exercise: derive one of those properties in a step by step

fashion

19

37

PdW-SZ / 2016

Fac. EE SPS-VCA



Module 03 – Part 2

Image Filtering based on DFT

Filtering concept using DFT, filter

types, special cases

38

PdW-SZ / 2016

Fac. EE SPS-VCA



Frequency-domain filtering fundament

∗ Filtering techniques in the frequency domain are based on modifying the Fourier transform

∗ Note that each value of F(u,v) contains information of all image samples!

∗ If a filter has a filter transfer function, we exploit that

Where F, H and g are all matrices of M x N. Assume that Hand F are centered, requiring pre-procesing!

[ ]),(),(),( vuFvuHIDFTyxg =

20

39

PdW-SZ / 2016

Fac. EE SPS-VCA



Necessity of padding with zeros – (1)

∗ Without any measures, the use of the DFT creates a wraparound error when periodicity is created

∗ The solution is to enlarge the the window with extra samples, such as padding with zeros (creates a border)

∗ Padding can be used to decrease ringing in the frequency domain, employing the extended filter impulse response in the time domain.

40

PdW-SZ / 2016

Fac. EE SPS-VCA



Necessity of padding zeros – (2)

Example

21

41

PdW-SZ / 2016

Fac. EE SPS-VCA



Summary of steps for DFT-based filters

1. Determine padding parameters, typically P=2M and Q=2N

2. Form padded image fp(x,y) of size PxQ appending zeros

3. Multiply padded image with (-1)x+y for centering spectrum

4. Compute the DFT, F(u,v), of the centered padded image

5. Generate a real, symmetric filter, H(u,v), of size PxQ and center in the middle, and compute G(u,v)=H(u,v)F(u,v)

6. Obtain the processed image with the IDFT of G(u,v) and take the real part, and re-center it

7. Extracting the g(x, y) result by selecting the MxN region from the top-left quandrant

[ ][ ]{ } yx

pvuGIDFTrealyxg

+−= )1(),(),(

42

PdW-SZ / 2016

Fac. EE SPS-VCA



Example of DFT-based filters / Gaussian2

2/2222

2)(2)(

σσπσπ uxAeuHAexh

−− =⇔=

Both real

functions!LPF HPF

22

43

PdW-SZ / 2016

Fac. EE SPS-VCA



Example of DFT-based filters / Gaussian

44

PdW-SZ / 2016

Fac. EE SPS-VCA



Sobel filter as DFT-based filter example

23

45

PdW-SZ / 2016

Fac. EE SPS-VCA



Ideal 2-D LPF filter – (1) Concept• Passes all frequencies within circle of radius D0 : H(u,v)=1

• Rejects all frequencies outside circle with radius D0 : H(u,v)=0

[ ] 2/122)2/()2/(),( QvPuvuD −+−=

46

PdW-SZ / 2016

Fac. EE SPS-VCA



Ideal 2-D LPF

filter –(2)Performance

comparison shows

• this is not very

practical, as

• most detail is

removed in only

13% of the power

• and ringing occurs

at already 2%

removal

24

47

PdW-SZ / 2016

Fac. EE SPS-VCA



Ideal 2-D LPF filter – (3) Explanation

1. Blur comes from the main lobe in impulse response

2. Ringing comes from the side lobes in impulse response

3. Spread of sinc function is inversely proportional to pass-band of H(u,v). (Discuss the trade-off)

48

PdW-SZ / 2016

Fac. EE SPS-VCA



Smooth 2-D LPF filter – (1) Gaussian1. Establish smooth roll-off in transfer function H(u,v)

2. Ringing comes from the side lobes in impulse response

):(),(0

202/),(

2

DnoteevuHDvuD == − σ

25

49

PdW-SZ / 2016

Fac. EE SPS-VCA



1. Blur can be regulated by variance

2. Ringing is absent

3. Spread of sinc function is smooth

4. Good visual results

Gaussian 2-D

LPF filter – (2)

50

PdW-SZ / 2016

Fac. EE SPS-VCA



2-D HPF filters –

various forms

),(1),( vuHvuHLPHP

−=

0

0

),(:0),(

),(:1),(

DvuDvuH

DvuDvuH

HP

HP

≤=

>=

HPF is opposite of

LPF filter!

26

51

PdW-SZ / 2016

Fac. EE SPS-VCA



Smooth 2-D HPF filter – cf. Gaussian):(1),(

0

202/),(

2

DnoteevuHDvuD

HP=−= − σ

52

PdW-SZ / 2016

Fac. EE SPS-VCA



2-D HPF

filters –

perform.

Ideal HPF

vs.

Gaussian

HPF perfor-

mance

27

53

PdW-SZ / 2016

Fac. EE SPS-VCA



Laplacian filtering - Concept

1. Laplacian can be implemented in frequency domain with

2. Or, with respect to center of frequency rectangle

3. Furthermore, it holds that for the filter in the freq. domain

4. Enhancement with subjective sharpness is obtained when adding this term (note the scaling), so that

),(4])2/()2/[(4),(

)(4),(

22222

222

vuDQvPuvuH

vuvuH

ππ

π

−=−+−−=

+−=

)},(),({),(2

vuFvuHIDFTyxf =∇

)},(),(),(2

yxfcyxfyxg ∇+=

)},()],(41{[)},()],(1{[

)},(),(),({),(

22vuFvuDIDFTvuFvuHIDFT

vuFvuHvuFIDFTyxg

π+=−

=−=

54

PdW-SZ / 2016

Fac. EE SPS-VCA



Laplacian filtering - performance

28

55

PdW-SZ / 2016

Fac. EE SPS-VCA



Unsharp masking & highboost filtering

1. Concept: sharpen images by subtracting an unsharp image from the original. The difference is the mask

2. Steps are:1. Blur the original image

2. Subtract the blurred image from original (difference is the mask)

3. Add mask to the original

3. More formally, the specification is

4. If k=1, it is unsharp masking, if k>1, we have highboost filtering, with if k<1, we have high-freq. emphasis

),(*),(),(

),(),(),(

yxgkyxfyxg

yxfyxfyxg

mask

mask

+=

−=

56

PdW-SZ / 2016

Fac. EE SPS-VCA



High-frequency emphasis filtering

1. For the unsharp mask, we can take the freq.-domain approach

2. Function fLP(x,y) is a low-pass smoothed image from the filtering with HLP(u,v).

3. The unsharp masking equation from the previous slide becomes now in the frequency domain

4. This result can be respecified as a high-pass filter result

)],(),([),( vuFvuHIDFTyxfLPLP

=

)},()]],(1[*1{[),( vuFvuHkIDFTyxgLP

−+=

)},()],(*1{[),( vuFvuHkIDFTyxgHP

+=

High-frequency emphasis filter

29

57

PdW-SZ / 2016

Fac. EE SPS-VCA



High-freq. emphasis filtering / Example

1. For more general form, take coefficients k1, k2 such that

2. Where k1>0 gives controls of the offset of the origin and k2 >0 the high-frequency contribution

3. In medical applications, ringing artifacts from filters are not accepted. Because spatial and freq.-domain Gaussian filters are transform pairs, these filters give smooth response and avoid ringing

4. In the following example, HF emphasis filtering helps

)},()],(*{[),(21

vuFvuHkkIDFTyxgHP

+=

58

PdW-SZ / 2016

Fac. EE SPS-VCA



High-freq. emphasis filtering / Example

Gaussian

filter with

D0=40

HF

emphasis

filter,

k1=0.5,

k2=0.75

HF em-

phasis &

histogram

equaliz.

Documents

Discrete Fourier Transform and Filteringvca.ele.tue.nl/sites/default/files/5XSF0 Mod 03... · (5XSF0), Module 03 Discrete Fourier Transform and Filtering Peter H.N. de With ([email protected]