Álgebra Linear e Aplicações

DISCRETE FOURIER TRANSFORM(AND THE FFT)

Motivation• Consider the propagation of sound from fixed

source to fixed destination in an environment• Sound is pressure variation through time• It is a vector space

• Each environment can be seen as an operator• Different environments, different operators

• What are the properties we can expect?

Linear, time-invariant systems• An LTI system is an operator

satisfying the two properties• Linearity

• Time invariance

• Sound can be discretized…

O : L2 ! L2

O{↵1x1 + x2} = ↵1O{x1}+O{x2}

Ta{u}(t) = u(t� a) ) O�Ta{x}

= Ta

�O{x}

Focus on the discrete, finite case• Linearity means linear operator• Time invariance could mean different things• Most powerful definition uses circular shifts• , with• T (u) = v, vi mod n = ui�1 mod n

T =

2

6664

11

. . .1

3

7775T

2

666664

u0

u1

u2...

un�1

3

777775=

2

666664

un�1

u0

u1...

un�2

3

777775

L�T (u)

�= T

�L(u)

�

L : Rn ! Rn

LT = TL

What can an LTI system do?

• Look at the effect of L on vector u

• It is completely defined by !

• What about matrix L?

vj =n�1X

i=0

ui

⇥T i(h)

⇤j =

n�1X

i=0

ui hj�i mod n

=

2

6664

h0 hn�1 · · · h1

h1 h0 · · · h2...

.... . .

...hn�1 hn�2 · · · h0

3

7775

Circulant matrix

Impulse

Circular convolution between u and h

Impulse response

v = L(u) = L

n�1X

i=0

uiei

!=

n�1X

i=0

ui L(ei) =n�1X

i=0

ui L�T i(e0)

�=

n�1X

i=0

ui Ti�L(e0)

�

h = L(e0)

L =⇥h T (h) · · · Tn�1(h)

⇤

Summary of LTI systems• Every discrete linear time-invariant system L

can be expressed as a circular convolution• The circular convolution between ,

denoted or , is

• is the impulse response of L• The corresponding matrix L is circulant• It commutes with the unit circular shift T

h = L(�) (� = e0)

vj =n�1X

i=0

ui hj�i mod n

u,h 2 Rn

u ⇤ h v 2 Rnh ⇤ u

=n�1X

k=0

uj�k mod n hk

Computational cost too high• Same as a matrix-vector product!• But matrix has only n degrees of freedom • Doesn’t make sense

• Vectors can be very large• Millions of entries• Need a faster alternative

• What if we found a better basis?

O(n2)

DiagonalizeT

• First notice that T has n distinct eigenvalues

• Find the eigenvector basis

det(T� �I) =

���������

�� 11 ��

. . .. . .1 ��

���������

= (��)n + (�1)1+n = 0

Tn = I ) (�k)n = 1

Tvk = �kvk

�k = !k, k 2 {0, 1, . . . , n� 1}, ! = e2⇡in

[Tvk]j = �k[vk]j = [vk]j�1 mod n

[vk]j = (�k)j = (!k)j

v⇤kvj =

n�1X

p=0

(!k)p(!j)p P⇤P = PP⇤ = I=n�1X

p=0

(!k�j)p =

(1�(!k�j)n

1�!k�j = 0, k 6= j

n, k = j

P =1pn

2

6664

1 1 1 · · · 11 ! !2 · · · !n�1

......

.... . .

...1 !n�1 (!2)n�1 · · · (!n�1)n�1

3

7775

Simultaneously diagonalizeL

• Eigenvectors of L and T are the same!

• Lv and v are eigenvectors of T associated to• But• So we must have • And therefore L is diagonalizable by same P!

�

dim�N(T� �I)

�= 1

Lv = �v

Tv = �vLT = TL

�) T(Lv) = LTv = �(Lv)

The Discrete Fourier Transform• Given a vector , the Discrete Fourier

Transform of v, denoted , is the vector

• The Inverse Discrete Fourier Transform, denoted , restores v

v 2 Rn

F (v)

v = P⇤v vj =1pn

n�1X

k=0

vke� 2⇡ijk

n

F�1(v)

v = Pv = PP⇤v vj =1pn

n�1X

k=0

vke2⇡ijk

n

The Convolution Theorem• If u,v are two vectors in Rn, then

• Proof

F (u ⇤ h) = F (u) � F (h)

[a � b]j = ajbj (Hadamard product, element-wise product)

F (u ⇤ h) = P⇤(u ⇤ h) = P⇤Lu = P⇤LPP⇤u = (P⇤LP)F (u)

= DF (u)

P⇤h = P⇤Le0 = P⇤LP(P⇤e0)

=⇥�0 · · · �n�1

⇤T� F (u)

= P⇤h � F (u) = F (h) � F (u)

=⇥�0 · · · �n�1

⇤T= D

⇥1 1 · · · 1

⇤T

Alternative to convolution

u,h u ⇤ hO(n2)

u � hO(n)

u, h

P⇤

FO(n2)

P

F�1O(n2)

Fast Fourier Transform (FFT)• Divide and conquer!

hj =n�1X

k=0

hk!�jkn

=

n/2�1X

k=0

h2k !�j2kn +

n/2�1X

k=0

h2k+1 !�j(2k+1)n

=m�1X

k=0

hek (!

2n)

�jk + !�jn

m�1X

k=0

hok (!

2n)

�jk

=m�1X

k=0

hek !

�jkm + !�j

n

m�1X

k=0

hok !

�jkm

= hej modm + !�j

n hoj modn

h =

2

666664

h0

h1...

hn�2

hn�1

3

777775

he =

2

666664

h0

h2...

hn�4

hn�2

3

777775

ho =

2

666664

h1

h3...

hn�3

hn�1

3

777775

Alternative to convolution

u,h u ⇤ h

u, h u � h

PP⇤

F F�1

O(n2)

O(n)

O(n log n) O(n log n)

Demo!