10. FAST ALGORITHMS FOR DISCRETE DiscreteFourierTransform

10. FAST ALGORITHMS FOR DISCRETE FOURIER TRANSFORM

10.2 Fast Fourier Transform (FFT)

Discrete Fourier Transform (DFT) is defined by

0. s ou e s o ( )

( ) y

21

, 0,1, , 1,N jnk N

N NX k x n W k N W e

or, as a matrix form,0n

X Wx

0 , 1 , , 1T

T

X X X N

X

X Wx 0 , 1 , , 1

1 1 1

Tx x x N

x

111 N

N NW W

W

1 1 11 N N N

N NW W


There are required a number of

0. s ou e s o ( )

q– N2 complex multiplications and

N(N 1) complex additions– N(N-1) complex additions. One complex multiplication = 4 multiplications

d 2 l dditiand 2 real additions Results a number of

– 4N2 real multiplications and – 2N2+2N(-N-1) = 4N2-2N real additions.2N 2N( N 1) 4N 2N real additions.

The arithmetic complexity is of order O(N2).


Twiddle factors WN properties

0. s ou e s o ( )

f p p

2j kNkN

1j

kN NNW e

N N 2 12 2 1N Nk j

N NW W e

2 kj nnk nNN N

k

W e W

k


There are algorithms that computes DFT with a

0. s ou e s o ( )

g psmaller number of operations.

An important case is where N=RM, R is called An important case is where N R , R is called radix.

There results FFT algorithms radix R There results FFT algorithms radix R. The index in the DFT formula can be expressed

i th f ll i l f lusing the following general formula:

1 1 2 2 modn K n K n N 1 1 2 2

3 1 4 2 modk K k K k N


For the radix R algorithms the indexes will be

0. s ou e s o ( )

grepresented as:

0 1 1 0 1 1;Nn Rn n n n R 1 2 1 2, 0,1, , 1, 0,1, , 1;

0 1 1 0 1 1;

n Rn n n n RR

N Nk k k k k R

that gives decimation in time algorithms and1 2 1 2, 0,1, , 1, 0,1, , 1;k k k k k R

R R

1 2 1 2, 0,1, , 1, 0,1, , 1;N Nn n n n n RR R

1 2 1 2, 0,1, , 1, 0,1, , 1;Nk Rk k k k RR

that gives decimation in frequency algorithms.

10.2.1 Radix 2 algorithms

Decimation in time (DIT) algorithms

0. . d go s

Radix 2 algorithms where introduced by Cooley and Tukey for N=2M, MN.

1 2 1 22 , 0,1, , 1, 0,1;2Nn n n n n

d l1 2 1 2, 0,1, , 1, 0,1;

2 2N Nk k k k k

and results:

NX k X k k 1 2 1 2

1 12 22

1 22

NNn n k k

Nx n n W

1 22X k X k k

1 2

1 20 0

2 Nn n

x n n W

1 12

N

n k n k n k

1 1 2 1 2 2

1 2

1 2 20 0 2

2 n k n k n kN N

n nx n n W W W

Decimation in time (DIT) radix 2 algorithms

By separating the last sum:

eci atio i ti e ( ) adix algo it s

1 1 1 2 1 1

1 12 2

1 2 1 2 12 2 12

N N

n k k k n kN N N

NX k k x n W W W x n W

and

1 10 02 22 n n

21 2 1 1 kk k kW W W and Separating for k2=0:

21 2 12 1k k k

N NW W W

1 1 1 1 1

1 12 2

1 1 10 0

2 2 1

N N

n k k n kN N NX k x n W W x n W

1 10 02 2n n

1kNW 1TFD 2 1N x n 1TFD 2N x n N 1

2N 1

2N

Decimation in time (DIT) radix 2 algorithms

By separating the last sum:

eci atio i ti e ( ) adix algo it s

1 1 1 2 1 1

1 12 2

1 2 1 2 12 2 12

N N

n k k k n kN N N

NX k k x n W W W x n W

and

1 10 02 22 n n

21 2 1 1 kk k kW W W and and for k2=1:

21 2 12 1k k k

N NW W W

1 1 1 1 1

1 12 2

1 1 12 2 12

N N

n k k n kN N N

NX k x n W W x n W

1 10 02 22 n n

1kNW 1TFD 2 1N x n 1TFD 2N x n N 1

2N 1

2N

Decimation in time (DIT) radix 2 algorithmseci atio i ti e ( ) adix algo it s

0

TFD

12nx 1k

TFD N/2 1kX

12 N

0

TFD

12 1 nx 1k

TFD N/2

1kNW

21 NkX -1

12 N

Decimation in time (DIT) radix 2 algorithmseci atio i ti e ( ) adix algo it s

Passing from transformations of order N/2 to the f f d N i d h htransform of order N is done through a structure

called butterfly:a k

Na bW DIT butterfly : one multiplication by one multiplication by,

2j kk NNW e

named twiddle factorDFT f d 2

N

-1 b k

Na bWk

NW

one DFT of order 2.

Example: an order 8 DIT transform.a p e: a o de 8 t a sfo .

(0) X(0)k 0x(0)

x(4) DFT

X(0)k1=0

X(1)k1=1

x(2)

DFTorder 4

1

X(2)k1=2

x(6)

( )0

8W( )

X(3)k1=3

x(1)

x(5) DFT-1

X(4)k1=01

8W1 X(5)k1=1

x(3)

DFTorder 4

-11

-1

28W

X(6)k1=23

x(7) -1

38W

X(7)k1=3


(0) X(0)k1=0

x(0)

x(4)

X(0)

X(1)

DFTorder 2

1

k1=1

x(2) X(2)DFT -1

08W

2

08W

x(6)

( )

X(3)

( )

order 2-1

28W

-11

8W1

x(1)

x(5)

X(4)

X(5)

DFTorder 2

-1

08W

2-1

-12

8W

3

x(3) X(6)DFT

-1

28W

-1

38W

x(7) X(7)order 2


(0) X(0)

1

08W

x(0)

x(4)

X(0)

X(1)-1

0-1

08W

2

x(2) X(2)

-1

08W

-1

28W

08W

x(6)

( )

X(3)

( )

1

08W

-11

8W1

x(1)

x(5)

X(4)

X(5)-1

0-1

08W

2-1

-12

8W

3

x(3) X(6)

-1

08W

-1

28W

-1

38W

x(7) X(7)

Example: an order 8 DIT transform. Observe that

h l iddl f k

a p e: a o de 8 t a sfo .

– The last stage twiddle factors are: W8k,

– For previous stage, the transforms having order 4, the twiddle factors are W4

k=W82k etc.

Evaluation of the arithmetic complexityvaluatio of t e a it etic co plexity

For a transform of order N=2M there are M stages, h h i N/2 b fli l f N/2 l Neach having N/2 butterflies, a total of N/2.log2N

butterflies. Each butterfly has a complex multiplication and

two complex additions. The complex multiplication is of form: cos sin cos sin sin cosx jy j x y j x y

there will be needed 4 real multiplications and 2real additions

cos sin cos sin sin cosx jy j x y j x y

real additions.

Evaluation of the arithmetic complexityvaluatio of t e a it etic co plexity

Results for one butterfly:– 4 real multiplications and– 6 real additions.

The total number of real multiplications (NMR) and real additions (NAR) for an order N transform will be:

2

2

NMR 2 logNAR 3 log

N N NN N N

Decimation in time (DIT) radix 2 algorithmseci atio i ti e ( ) adix algo it s Bit reversed reordering

Th i t d t d t l i th i t l d The input data does not apply in their natural order but in a reordered sequence known as the bit

d dreversed order.

Index Binary Bit-reversed New representation representation index

01

0 0 00 0 1

0 0 01 0 0

041

23

0 0 10 1 00 1 1

1 0 00 1 01 1 0

426

456

1 0 01 0 11 1 0

0 0 11 0 10 1 1

1536

71 1 01 1 1

0 1 11 1 1

37

Decimation in frequency (DIF) radix 2 algorithms

The indexes are represented of form:

eci atio i f eque cy ( ) adix algo it s

1 2 1 2, 0,1, , 1, 0,1;2 2N Nn n n n n

It results:1 2 1 22 , 0,1, , 1, 0,1;

2Nk k k k k

It results:

2X k X k k 1 2 1 2

1 12 22

NNn n k kNx n n W

1 22X k X k k

1 2

1 20 0 2 N

n nx n n W

1N

1 1 1 2 2 2

1 2

1 12

1 2 20 0 22

n k n k n kN N

n n

Nx n n W W W


from which results for k2=0 and k2=1:


1 1

12

1 1 1 22

N

n kNX k x n x n n W

1

1 1 1 20 2

22

TFD

Nn

X k x n x n n W

N

1 1 22

TFD2N x n x n n

N

1 1 1

1

12

1 1 1 20 2

2 12

N

n n kN N

n

NX k x n x n n W W

1

1

2

1 1 2TFD2

nN N

Nx n x n n W

2 2


It appears the decomposition of order N transformi N/2 d f


into two N/2 order transforms: 0

1n

TFD N/2

1nx 12X k

12 N

N/2

0

-1 1n

21 Nnx

1nNW

TFD N/2

12 1 kX

12 N

N


Like the previous algorithm, results log2N stages, h h i N/2 b fli


each having N/2 butterflies. The structure of a DIF butterfly:

a a+b

-1 nNW

b nNa b W

NW

Example: an order 8 DIF transform.a p e: a o de 8 t a sfo .

X(0)(0) 0 X(0)

X(4)TFD

x(0) n1=0

x(1) n1=1

X(2)

TFDde ordin 4

1

x(2) n1=2

X(6)

( )0

8W( )

x(3) n1=3

X(1)

X(5)TFD-1

x(4) n1=01

8W1x(5) n1=1

X(3)

TFDde ordin 4

-1 12

8W

-1x(6) n1=23

X(7)3

8W-1x(7) n1=3


(0) X(0)x(0)

x(1)

X(0)

X(4)

TFDde ordin 2

08W

2-1

x(2) X(2)TFD2

8W-1

08W

x(3)

( )

X(6)

( )

de ordin 2

18W

-1

1

x(4)

x(5)

X(1)

X(5)

TFDde ordin 2

08W

2-1

28W

3-1

-1

x(6) X(3)TFD2

8W-1

38W

-1x(7) X(7)de ordin 2


(0) X(0)0

8W1

x(0)

x(1)

X(0)

X(4)0

8W

2 0

-1

-1x(2) X(2)

28W 0

8W-1-1

08W

x(3)

( )

X(6)

( )0

8W1

18W

-1

1

x(4)

x(5)

X(1)

X(5)0

8W

2 0

-1

-1

28W

3-1

-1

x(6) X(3)2

8W 08W

-1-1

38W

-1x(7) X(7)


Fiecare din cele două grupuri de câte N/2 semnale i i i ă il f l î N/2


ce constituie intrările transformatoarelor în N/2 puncte se desparte în două, după acelaşi procedeu ( i l N/4 i ă l N/4)(primele N/4 şi următoarele N/4).

Complexitatea, exprimată prin numerele de înmulţiri reale şi de adunări reale, este aceeaşi ca în algoritmul precedent.

Eşantioanele semnalului se iau în ordinea naturală, iar ale ieşirii, rezultă în ordinea inversă a biţilor. ş , ţ

10.2.3 FFT optimization for faster computation

FFT implementation optimization can be done:

0. .3 op o o s e co pu o

p p– for reducing the calculation time.

to reduce the memory used by the program or– to reduce the memory used by the program or data.

Th t i t ll t di t The two requirements are generally contradictory. As memory costs are continuously decreasing,

optimization of the execution time is becoming more important.

Example: DIT Radix 2 FFTp e: adix

Trivial twiddle factors (no arithmetic is required)0 1NW / 4N

NW j

can be combined in the first and second stage of a 4th order butterfly:

A A’

-1

08 1W

A

B

A

B’

-1

08 1W

2W j

C

1

C’

-1 -18W j

D1

D’


x(0) X(0)

-1

08W

x(0)

x(4)

X(0)

X(1)

0W-1

08W

2W

x(2) X(2)

-18W

-18W

08W

x(6)

x(1)

X(3)

X(4)

-1

08W

-11

8W-1

x(1)

x(5)

X(4)

X(5)

0W-1

08W

2W-1

28W

3W

x(3) X(6)

-18W

-18W

-18W

x(7) X(7)


A A’

-1

08 1W

B B’

-1

08 1W

28W j

C

1

C’

'A A C B D

-1 -18W j

D1

D’

'A A C B D '

r r r r r

r r r r r

A A C B D

B A C B D

'

i i i i i

i i i i i

A A C B D

B A C B D

'

'

r r r i i

r r r i i

C A C B D

D A C B D

'

'

i i i r r

i i i r r

C A C B D

D A C B D

r r r i i i i i r r

Example: DIT Radix 2 FFT

8 real additions and 8 b i i d

p e: adix

'A A C B D subtractions are required For the first 2 stages of a

'

r r r r r

r r r r r

A A C B D

B A C B D

N order transform instead of computing 2.N/2

'

'

r r r i i

r r r i i

C A C B D

D A C B D

butterflies of order 2, there will be N/4 trivial

'

'

r r r i i

i i i i iA A C B D

B A C B D

butterflies of order 4. '

i i i i i

i i i r r

B A C B D

C A C B D

'i i i r rD A C B D

FFT optimization for faster computationop o o s e co pu o

Trivial twiddle factors exist in the remaining Th f i i l b flistages. There are two types of trivial butterflies:

– Type 1: 0 / 41, NN NW W j

– Type 2: /8 3 /8 2 22 2

N NN NW W j

First type don’t requires multiplicaions. Type 2 isn’t quite trivial but reduces the butterfly

2 2

yp q ycomplexity to 2 multiplications and 6 additions.

10.2.4 Real-data sequences FFT

For real-data input sequences the DFT is conjugate

0. . e d seque ces

p q j gsymmetric:

*( ) ( )X k X N k

h X(0) d X(N/2) l lRe{ ( )} Re{ ( )}X k X N k Im{ ( )} Im{ ( )}X k X N k

where X(0) and X(N/2) are real values. Therefore it is sufficient to calculate X(k) for

k=0,1,…,N/2 (a number of N real values).

Algorithms for a single real sequence

Considering the decimation in time algorithm:

lgo it s fo a si gle eal seque ce

g g

1 21 1 1 1

1 12 2

2( ) (2 ) (2 1)

N NNk kn k n kNX k k x n W W x n W

For k2=0 and k1=0 1 N/2–1:1 1

1 2 1 10 02 2

( ) (2 ) (2 1)2 N N N

n nX k k x n W W x n W

For k2 0 and k1 0,1,…,N/2 1:1

1 1 1( ) ( ) ( )kNX k X k W X k

where X'(k1) and X''(k1) are DFT of order N/2. Also: Also:

1* *1 1 12 2 2

kN

N N NX k X k W X k

1 1 12 2 2N

Algorithms for a single real sequence Expressing the real and imaginary parts:

lgo it s fo a si gle eal seque ce

1 1 1 1 1 1

1 1 1 1 1 1

Re ( ) Re ( ) cos Re ( ) sin Im ( )Im ( ) Re ( ) sin Re ( ) cos Im ( )

X k X k k X k k X kX k X k k X k k X k

for the samples of order 0,1,…,N/4N

2N

1 1 1 1 1 1Re Re ( ) cos Re ( ) sin Im ( )2NX k X k k X k k X k

N

1 1 1 1 1 1Im Im ( ) sin Re ( ) cos Im ( )2NX k X k k X k k X k

for the samples of order N/4+1,…,N/2.

Algorithms for a single real sequencelgo it s fo a si gle eal seque ce

0ReX’(k1) ReX[k1]

TFD : N/2 :

ImX’(k1)

x(2n)e [ 1]

ImX[k1] :

2N

-1

cos(k1)ReX[N/2-k1]-1x(2n+1)

0ReX’’(k1)

TFD :N/2

sin(k1)sin(k1)

-1ImX[N/2-k1]-1

N/2 :ImX’’(k1)

:-cos(k1)

sin(k1)

Algorithms for two real sequences

For simultaneously computing DFT for two input f l d ( ) d ( )

lgo it s fo two eal seque ces

sequences of real data x(n) and y(n), we may use complex data FFT algorithms.

The DFT is a linear transform. For the input sequence:

( ) ( ) ( )j we have:

( ) ( ) ( )z n x n j y n

( ) TFD{ ( )}( )Z k z n k( ) ( )X k jY k

TFD{ ( ) ( )}( )x n jy n k

Re ( ) Im ( ) Im ( ) Re ( )X k Y k j X k Y k

Re ( ) Im ( ) Re ( ) Im ( )X k j X k j Y k j Y k

( ) ( ) ( ) ( )j

Algorithms for two real sequences

Both sequences are of real data. DFT is conjugate i

lgo it s fo two eal seque ces

symmetric:Re{ ( )} Re{ ( )}X k X N k Im{ ( )} Im{ ( )}X k X N k

It lt

{ ( )} { ( )} { ( )} { ( )}Re{ ( )} Re{ ( )}Y k Y N k Im{ ( )} Im{ ( )}Y k Y N k

It results:

( ) Re ( ) Im ( ) Im ( ) Re ( )Z N k X k Y k j X k Y k

For each input sequence it is sufficient computing DFT for k=0 1 N/2

( ) ( ) ( ) ( ) ( )j

DFT for k 0,1,…N/2.

Algorithms for two real sequenceslgo it s fo two eal seque ces ( ) Re ( ) Im ( ) Im ( ) Re ( )Z k X k Y k j X k Y k

We can separate each input sequence DFT from ( ) Re ( ) Im ( ) Im ( ) Re ( )Z N k X k Y k j X k Y k

p p qZ(k) spectrum (k=1,…,N/2–1):

1 1( ) R ( ) R ( ) I ( ) I ( )X k Z k Z N k j Z k Z N k ( ) Re ( ) Re ( ) Im ( ) Im ( )2 2

X k Z k Z N k j Z k Z N k

1 1( ) Im ( ) Im ( ) Re ( ) Re ( )Y k Z N k Z k j Z N k Z k

Also for k=0 and k=N/2 we have:

( ) Im ( ) Im ( ) Re ( ) Re ( )2 2

Y k Z N k Z k j Z N k Z k

Also, for k 0 and k N/2 we have:

(0) Re (0) ( / 2) Re ( / 2)X Z X N Z N (0) Im (0) ( / 2) Im ( / 2)Y Z Y N Z N

Computing Inverse DFT using DFT

The Inverse Discrete Fourier transform (IDFT) is

Co pu g ve se us g

( )defined:

11( ) TFDI{ ( )}( ) ( )N

nkx n X k n X k W

0 1 1n N

IDFT b l l t d i th f ll i

0( ) TFDI{ ( )}( ) ( ) N

kx n X k n X k W

N

0,1,..., 1n N

IDFT can be calculated using the following relation involving DFT:

**1TFDI{ ( )} TFD{ ( )}X k j j X kN

Computing Inverse DFT using DFTCo pu g ve se us g

**1TFDI{ ( )} TFD{ ( )}X k j j X k

Indeed

TFDI{ ( )} TFD{ ( )}X k j j X kN

Indeed

*21** *TFD{ ( )} ( )

N j nkNj j X k j j X k e

0

TFD{ ( )} ( )k

j j X k j j X k e

*21*

N j nk

*21N j nk *

0( )

j nkN

kj X k je

0( )

j nkN

kX k j je

2 2

21

*

0( )

N j nkN

kX k j j e

21

0( )

N j nkN

kX k e

Computing Inverse DFT using DFT

Let: z = a + j.b.

Co pu g ve se us g

( )X kj then: j.z* will be:

*j a jb j a jb b ja

Inversareparte reală –

i i ă

Which is equivalent tol th l t b th

j a jb j a jb b ja parte imaginară

replace the real part by theimaginary part.

FFT(scalare cu 1/N)

Inversareparte reală –

parte imaginară

( ) IFFT{ ( )}x n X k

10.2.5 Radix 4 FFT algorithms

Considering N=4M, radix 4 algorithms may be li d (R 4)

0. .5 d go s

applied (R=4).– Decimation in time algorithms– Decimation in frequency algorithms

We will express the index according to decimation We will express the index according to decimation in time algorithms:

N1 2 1 24 , 0,1, , 1, 0,1,2,3;

4Nn n n n n

N N

1 2 1 2, 0,1, , 1, 0,1,2,3;4 4N Nk k k k k

10.2.5 Radix 4 FFT algorithms It results:

0. .5 d go s

1NN

1 24NX k X k k

1 2 1 2

1 2

1 34 44

1 20 0

4Nn n k k

Nn n

x n n W

N

1 1 2 1 2 2

1 34

1 2 40 0 4

4

N

n k n k n kN N

n nx n n W W W

Or by writing the sum over n2:1 20 0 4n n

N N

1 1

1

14

10 4

4

N

n kN

nx n W

1kNW 1 1

1

14

10 4

4 1

N

n kN

nx n W

1 24NX k k

2

4kW

1 0 4n

12kNW 13k

NW

1 4

1 1

14

14 2

N

n kNx n W

1 1

14

14 3

N

n kNx n W

224

kW 234

kWN N 1

10 4

Nn

1

10 4

Nn 4 4

10.2.5 Radix 4 FFT algorithms0. .5 d go s

x(4n1) TFD 0 TFD

0

TFDN/4 k1

N/4-1

TFD4

x(4n1+1) TFDN/4

0k1

1kNW

k2=0 X(k1)k1

N/4 N/4-1

x(4n1+2) TFD 0

TFD412kW

123

( 1)X(k1+N/4)X(k1+N/2)x(4n1 2) TFD

N/40

k1N/4-1

1NW 3 X(k1+3N/4)

N/4-1

x(4n1+3) TFD 0k

13kNW TFD

N/4 1

N/4 k1N/4-1 4

10.2.5 Radix 4 FFT algorithms0. .5 d go sk2=0 Each DFT of

d 4 b X(k1)order 4 may be computed:

1kNW

X(k1+N/4)-j

k2=1– directly 12 complex additions (24

12kW

( 1 )j

k2=2

additions (24 real additions)

– Fast algorithm

TFD41

NWX(k1+N/2)

-1

-1

k2 2Fast algorithm 16 real additions

13kNW

X(k1+3N/4)j-1

k2=3X(k1+3N/4)-j

10.2.5 Radix 4 FFT algorithms0. .5 d go sk2=0 For each butterfly

h ill b 3 X(k1)there will be 3 complex

l i li i d1k

NWX(k1+N/4)

-jk2=1multiplications and

16 real additions (12 l -1

12kW

( 1 )j

k2=2

(12 real multiplications and 22 l ddi i )

1

1NW

X(k1+N/2)-1

-1

k2 222 real additions)

13kNW

X(k1+3N/4)j-1

k2=3X(k1+3N/4)-j

10.2.5 Radix 4 FFT algorithms The arithmetic complexity:

0. .5 d go s

3 4 23NMR 3 log 5 8 log 5 8;2

11 3 8 11 3 8

N N N N N N N

where 1 complex ‘×’ 4 real ‘×’ and 2 real ‘+’

4 211 3 8 11 3 8NAR log log2 2 3 4 2 3

N N N N N N N

where 1 complex × 4 real × and 2 real +

Documents

10. FAST ALGORITHMS FOR DISCRETE DiscreteFourierTransform