The Fast Fourier Transform and

Applications to Multiplication

Prepared by

John Reif, Ph.D.

Analysis of Algorithms

Topics and Readings:

- The Fast Fourier Transform

Advanced Material :

- Using FFT to solve other Multipoint Evaluation Problems

- Applications to Multiplication

• Reading Selection:• CLR, Chapter 30

Nth Roots of Unity

• Assume Commutative Ring (R,+,·, 0,1)

is principal nth root of unity if

i 1 for i = 1, …, n-1

n = 1, and

• Example: for complex numbers

n-1jp

j=0

0 for 1 p n

2 i/ne

Example of nth Root of Unity for Complex Numbers

is the 8th root of unity2 i/8e

Fourier Matrix

1

2 2( 1)n

1 ( 1)( 1)

ijij

0

n-1

1 1 1

1

M ( ) = 1

1

so M( ) = for 0 i, j<n

a

given a =

a

n

n

n n n

Discrete Fourier Transform

Input a column n-vector a = (a0, …, an-1)T

Output an n-vector which is the product of the Fourier matrix times the input vector

n

0

n-1

n-1ik

i kk=0

DFT (a) = M( ) x a

f

= where

f

f = a

Inverse Fourier Transform-1 -1n

-1 -ijij

-1

n-1 n-1ik -kj k(i-j)

k=0 k=0

DFT (a) = M( ) x a

1 M( ) =

n

We must show M( ) M( ) = I

1 1 =

n n

0 if i-j 0 =

1 if i-j = 0

using i

Theorem

proof

n-1kp

k=0

dentity 0, for 1 p < n

Fourier Transform is Polynomial Evaluation at the Roots of Unity Input a column n-vector a = (a0, …, an-1)T

Output an n-vector (f0, …, fn-1)T which are the

values polynomial f(x)at the n roots of unity

0

n

n-1

ii

n-1j

jj=0

f

DFT (a) = where

f

f = f( ) and

f(x) = a x

Fast Fourier Transform

• Viewed as Evaluation Problem: naïve algorithm takes n2 ops

• Divide and Conquer gives FFT with O(n log n) ops for n a power of 2

• Key Idea:• If is nth root of unity

then 2 is n/2th root of unity• So can reduce the problem to two

subproblems of size n/2

Algorithm FFTn

• Input a = (a0, …, an-1)T, n a power of 2

T

' '0 n 0 2 2

12 2

T

" "0 n 1 3 1

12 2

' "i i i

[1] If n=1 then ouput

[2] f ,..., f (( , ,..., ) )

f ,..., f (( , ,..., ) )

n[3] For i=0, ..., 1 do f f f

2

Tn n

Tn n

i

FFT a a a

FFT a a a

' "n i i

i+2

0 1 n-1

f f f

[4] Output (f , f , ..., f )

i

FFT Circuit (also known as Butterfly Network)

• Total Recursion depth = log n• Communication Distance 2d at depth d

i 2i (n-1)ii 0 1 2 n-1

' i "i i i

i(n-2)' 2 i 2 2i 2 2i 0 2 4 n-2

i(n-2)" 2 i 2 2i 1 3 n-1

0'0

22n

2'n

1 22

f = a + a + a +...+a

f = f + f where

f = a + a ( ) + a ( ) +...+ a ( )

f = a + a ( ) +...+ a ( )

f

M ( )

fn

a

a

a

0 2 2

2

1"0

32n 1 3 1

2 2"n

1 12

(( , ,..., ) )

f

M ( ) (( , ,..., ) )

f

Tn n

Tn n

n

DFT a a a

a

aDFT a a a

a

' ' " "n i n i

i i2 2

n1n 2 n 2 i 2i2

' i "i i i

ni +' "2

n i ii +

2

nNote: f = f , f f , i=0, ..., 1

2

But =1, so ( ) = ( )

n for i=0, ..., 1

2n

Thus, f = f + f for i=0, ..., 12

and f = f + f

' i "i i

n n2 n2 2

n = f - f for i=0, ..., 1

2

since ( ) 1, so = -1

Operation Counts for FFT Algorithm

• Assume n = 2k

• # additionsAdd(n) = 2· Add(n/2) + n = n log n

• # multiplicationsMult(n) = 2· Mult(n/2) + n/2 = ½ n log n

• Total Time O(n log n)• Note in complex FFT,

# real ops is 5 n log n

Multipoint Polynomial Evaluation

• Input polynomial

• Problem evaluate f(x) at x0, x1, …, xn-1

• Easy Cases:FFT Case xi = i

= principal root of unity

1

0

( )n

ii

i

f x a x

Multipoint Polynomial Evaluation (cont’d)

2

Summary of FFT:

( ) '( ) "( )

where

'( ), "( ) both degree halved

needed to only evaluate at half as many points

method f x f y x f y

y x

f x f x

Other Polynomial Evaluation Problems Solved by FFT

Each costs O(n log n) time

• Evaluate at points Xi = bai + d for i=0,…, n-1(Chirp Transfom)– Reduced to FFT

• Single point evaluation of all derivatives of a polynomial– Solve by reduction to above Chirp Transform of case 2)

• Evaluate at points Xi = b(ai)2+ cai + d for i=0,…, n-1 – Solve by divide and conquer similar to FFT

Single Point Evaluation of all Derivatives of Polynomial

• Input

and point x0

• output

1

0

( )n

ii

i

f x a x

0

( )( ) for 0,..., 1

kk d f xf x x x k n

dx

Single Point Evaluation of all Derivatives of Polynomial (cont’d)• Taylor Series Representation of

Then

reduces to case of evaluation at points

• Solve this Chirp Transform problem by reduction FFT

1

00

( ) ( )n

ii

i

f x c x x

( )0( ) !k

kf x k c

for 0,..., 1iix ab i n

Advanced Material: Further Applications of FFT

1) Convolution: Products and Powers of Polynomials• Used for for Integer Multiplication

Algorithms • Also used for Filtering on infinite

input streams2) Division and Inverse of Polynomials3) Multipoint Evaluation and

Interpolation

Advanced Material: Products and Powers of Polynomials

• Input vectors a = (a0, a1, …, an-1)T

b = (b0, b1, …, bn-1)T

• Definition of Convolution c = a b

Where for i=0, …, 2n-1

define ak = bk = 0 if k< 0 or kn

n-1

i j i-jj=0

c = a b

Products and Powers of Polynomials (cont’d)

• Convolution Theorem

• Application to Polynomial Products:n-1

ii

i=0

n-1j

jj=0

2n-2 n-1i

i i j i-ji=0 j=0

p(x) = a x

q(x) = b x

p(x) q(x) = c x where c = a b

-12n 2n 2na b = FFT FFT (a) FFT (b)

Products of m Polynomials

• Generalized Convolution Theorem

k

n-1i

k k,ii=0

m(n-1)m mi

k i i k,jki=0k=1 k=1

j =1

for k=1, ..., m let P (x) = a x

P (x) = c x , where c = a

1 2 m

-1n m n m 1 n m 2 n m m

a a ... a =

FFT FFT (a ) FFT (a ) ... FFT (a )

Wrapped Convolutions

• a = (a0, a1, …, an-1)T , b = (b0, b1, …, bn-1)T

• Positive wrapped convolution is c = (c0, c1, …, cn-1)T

• Negative wrapped convolution is d = (d0, d1, …, dn-1)T

i n-1

i j i-j j n+i-jj=0 j=i+1

c = a b + a b

i n-1

i j i-j j n+i-jj=0 j=i+1

d = a b - a b

Application of Wrapped Convolution to Modular Polynomial Products

n-1i

ii=0

n-1j

jj=0

n

n-1i n n

ii=0

p(x) = a x

q(x) = b x

p(x) q(x) mod(x +1)

= d x since x = -1 mod(x +1)

Computing Positive Wrapped Convolution

• Let = principal nth root of unity• Assume n has multiplicative inverse,

Theorem

is the positive wrapped convolution of n-vectors a and b.

-1n n nc = FFT FFT (a) FFT (b)

Computing Negative Wrapped Convolution• Also

is the negatively wrapped convolution of n-vectors a and b

where

and 2 = = principal nth root of unity

-1n n n

ˆ ˆˆd = FFT FFT (a) FFT (b)

Tn-10 1 n-1

Tn-10 1 n-1

a = a , a , ..., a

b = b , b , ..., b

Integer Multiplication by Polynomial Product (solved via FFT)• Input n bit integers a,b

define polynomials degree k = n/L

k-1i L

i ii=0

k-1i L

i ii=0

L L

a(x) = a x , 0 a 2

b(x) = b x , 0 2

so a = a(2 ), b = b(2 )

b

Integer Multiplication by Polynomial Product (cont’d)

• Idea1) Compute c(x) = a(x)· b(x)

by convolution

2) Evaluate c(2L) = a· b

Integer Multiplication Algorithms using Reduction to Polynomial Product

• Pollard Mult Algorithm

• Karp Mult Algorithm

• Schönage-Strassen Mult Algorithm

2O(n(logn) )(loglogn) use L = logn

2O(n(logn) ) use L = n

O(n(logn)(loglogn)) use L = n

and wrapped convolution

Pollard Multiplication Algorithm

• n = kL, L = 1 + log k1) Choose primes P1, P2, P3 where

2) Compute C(x) by convolution over finite field Zpi for i =1,2,3 (requires k mults on 2L bit integers)

31 2 3

Li i i

P P P 4 k

and P = α 2 + 1, α = O(1)

Pollard Multiplication Algorithm (cont’d)

3) Evaluate C(2L)

• Time Bounds

2

2

T(n) = 3kT(2L) + O(k logk) O(L)

= 3kT (2(1 + logk)) + O(k(log k) )

O(n(log n) (log log n) ) for any > 0

recursive mults FFT

Korp Multiplication Algorithm

1) Compute C(x) modulo k by convolution2) Compute C(x) modulo (22L+1) by

convolution3) Compute C(x) coefficients from C(x)

mod k, C(x) mod (22L+1) by Chinese remaindering

s

s

2

(s-1)

2

n = 2 = kL

2 if s evenk =

2 else

Korp Multiplication Algorithm (cont’d)

4) Compute C(2L)

• Time

2

T(n) = 2kT(2L) + O(k logk)O(L)

= 2 nT (2 n ) + O(n log n)

O(n(log n) )

recursive mults FFT

Schönage-Strassen Multiplication Algorithm

(2’) Compute C(x) mod (xk+1) modulo (22L+1) by wrapped convolution requires only k recursive mults on 2L bit numbers

• Time

T(n) = kT(2L) + O(k logk)O(L)

= nT (2 n ) + O(n log n)

O(n log n)(log log n)

recursive mults FFT

Still Open Problem: How Fast Can You Multiply Integers?

• Can you mult n bit integers in O(n log n) time?

The Fast Fourier Transform and

Applications to Multiplication

Prepared by

John Reif, Ph.D.

Analysis of Algorithms