Embed Size (px)
Citation preview
Polynomial + Fast Fourier Transform
Michael Tsai2017/06/13
Polynomials
• Coefficients:• Degree: highest order term with nonzero coefficient (k if highest nonzero term is )• Degree-‐bound: any integer strictly greater than the degree
A(x) =n�1X
j=0
ajxj
A(x) = a0 + a1x+ a2x2 + · · ·+ an�1x
n�1
a0, a1, . . . , an�1
ak
Coefficient Representation
• Using it:• Evaluation:
• Addition:
A(x) =n�1X
j=0
ajxj (a0, a1, . . . , an�1)
Vector
A(x0) = a0 + x0(a1 + x0(a2 + · · ·+ x0(an�2 + x0(an�1)) . . . ))
Horner’s rule:
(a0, a1, . . . , an�1)
A(x) +B(x)
(b0, b1, . . . , bn�1)+ (a0 + b0, a1 + b1, . . . , an�1 + bn�1)
O(n)
O(n)
Prob.: Polynomial Multiplication• Example:
A(x) = 6x3 + 7x2 � 10x+ 9
B(x) = �2x3 + 4x� 5
C(x) = A(x)B(x)
Chapter 30 Polynomials and the FFT 899
mial C.x/, also of degree-bound n, such that C.x/ D A.x/C B.x/ for all x in theunderlying field. That is, if
A.x/ Dn!1X
j D0
aj xj
and
B.x/ Dn!1X
j D0
bj xj ;
then
C.x/ Dn!1X
j D0
cj xj ;
where cj D aj C bj for j D 0; 1; : : : ; n ! 1. For example, if we have thepolynomials A.x/ D 6x3 C 7x2 ! 10x C 9 and B.x/ D !2x3 C 4x ! 5, thenC.x/ D 4x3 C 7x2 ! 6x C 4.
For polynomial multiplication, if A.x/ and B.x/ are polynomials of degree-bound n, their product C.x/ is a polynomial of degree-bound 2n ! 1 such thatC.x/ D A.x/B.x/ for all x in the underlying field. You probably have multi-plied polynomials before, by multiplying each term in A.x/ by each term in B.x/and then combining terms with equal powers. For example, we can multiplyA.x/ D 6x3 C 7x2 ! 10x C 9 and B.x/ D !2x3 C 4x ! 5 as follows:
6x3 C 7x2 ! 10x C 9! 2x3 C 4x ! 5
! 30x3 ! 35x2 C 50x ! 4524x4 C 28x3 ! 40x2 C 36x
! 12x6 ! 14x5 C 20x4 ! 18x3
! 12x6 ! 14x5 C 44x4 ! 20x3 ! 75x2 C 86x ! 45
Another way to express the product C.x/ is
C.x/ D2n!2X
j D0
cj xj ; (30.1)
where
cj DjX
kD0
akbj !k : (30.2)
O(n2)
O(n)O(n)
O(n)…
Prob.: Polynomial Multiplication
• Degree(C)=degree(A)+degree(B)• Degree bound:
• !• Problem: can we reduce this time complexity?
C(x) =2n�2X
j=0
cjxj cj =
jX
k=0
akbj�k
na + nb
O(n2)
Point-‐Value Representation
• Computing a point-‐value representation:• Calculate each using Horner’s rule takes • Thus calculate all n values take• If we choose the points wisely, we can reduce this to !
A(x) =n�1X
j=0
ajxj
Degree-‐bound: n
{(x0, y0), (x1, y1), . . . , (xn�1, yn�1)}
yk = A(xk) k = 0, 1, . . . , n� 1
A(xk) O(n)
A(xk) O(n2)xk
O(n log n)
Interpolation• Interpolation is the inverse operation of evaluation• Interpolation is well-‐definedwhen the interpolating polynomial have a degree-‐bound == given number of point-‐value pairs
Theorem: For any set of n point-‐value pairs such that all the values are distinct, there is a unique polynomial A(x) of degree-‐bound n such that for
. (see p. 902 in Cormen for the proof)
{(x0, y0), (x1, y1), . . . , (xn�1, yn�1)}xk
k = 0, 1, . . . , n� 1
yk = A(xk)
Point-‐Value Representation
• Using it:• Addition:since ,
A(x) =n�1X
j=0
ajxj
Degree-‐bound: n
{(x0, y0), (x1, y1), . . . , (xn�1, yn�1)}
yk = A(xk) k = 0, 1, . . . , n� 1
Evaluation
Interpolation
C(x) = A(x) +B(x) C(xk) = A(xk) +B(xk)
{(x0, y0), (x1, y1), . . . , (xn�1, yn�1)}
{(x0, y00), (x1, y
01), . . . , (xn�1, y
0n�1)}
A:
B:
{(x0, y0 + y
00), (x1, y1 + y
01), . . . , (xn�1, yn�1 + y
0n�1)}
C:O(n)
Point-‐Value Representation: Multiplication• Multiplication:since ,
• Problem! We need 2n point-‐value pairs so that C(x) is well-‐defined!
C(x) = A(x)B(x) C(xk) = A(xk)B(xk)
A:
B:
C:
{(x0, y0), (x1, y1), . . . , (xn�1, yn�1)}
{(x0, y00), (x1, y
01), . . . , (xn�1, y
0n�1)}
{(x0, y0y00), (x1, y1y
01), . . . , (xn�1, yn�1y
0n�1)}
n pairs
n pairsn pairs
Point-‐Value Representation: Multiplication• Multiplication:since ,
• Solution: Extend A and B to 2n point-‐value pairs(add n zero coefficient high-‐order terms)• C(x) is now well-‐defined with 2n point-‐value pairs
C(x) = A(x)B(x) C(xk) = A(xk)B(xk)
A:
B:
C:
{(x0, y0y00), (x1, y1y
01), . . . , (x2n�1, y2n�1y
02n�1)}
{(x0, y0), (x1, y1), . . . , (x2n�1, y2n�1)}
{(x0, y00), (x1, y
01), . . . , (x2n�1, y
02n�1)}
2n pairs
2n pairs2n pairs
O(n)
904 Chapter 30 Polynomials and the FFT
a0; a1; : : : ; an!1
b0; b1; : : : ; bn!1
c0; c1; : : : ; c2n!2Ordinary multiplication
Time ‚.n2/
EvaluationTime ‚.n lg n/Time ‚.n lg n/
Interpolation
Pointwise multiplicationTime ‚.n/
A.!02n/; B.!0
2n/
A.!12n/; B.!1
2n/
A.!2n!12n /; B.!2n!1
2n /
::::::
C.!02n/
C.!12n/
C.!2n!12n /
Coefficient
Point-valuerepresentations
representations
Figure 30.1 A graphical outline of an efficient polynomial-multiplication process. Representationson the top are in coefficient form, while those on the bottom are in point-value form. The arrowsfrom left to right correspond to the multiplication operation. The !2n terms are complex .2n/th rootsof unity.
on whether we can convert a polynomial quickly from coefficient form to point-value form (evaluate) and vice versa (interpolate).
We can use any points we want as evaluation points, but by choosing the eval-uation points carefully, we can convert between representations in only ‚.n lg n/time. As we shall see in Section 30.2, if we choose “complex roots of unity” asthe evaluation points, we can produce a point-value representation by taking thediscrete Fourier transform (or DFT) of a coefficient vector. We can perform theinverse operation, interpolation, by taking the “inverse DFT” of point-value pairs,yielding a coefficient vector. Section 30.2 will show how the FFT accomplishesthe DFT and inverse DFT operations in ‚.n lg n/ time.
Figure 30.1 shows this strategy graphically. One minor detail concerns degree-bounds. The product of two polynomials of degree-bound n is a polynomial ofdegree-bound 2n. Before evaluating the input polynomials A and B , therefore,we first double their degree-bounds to 2n by adding n high-order coefficients of 0.Because the vectors have 2n elements, we use “complex .2n/th roots of unity,”which are denoted by the !2n terms in Figure 30.1.
Given the FFT, we have the following ‚.n lg n/-time procedure for multiplyingtwo polynomials A.x/ and B.x/ of degree-bound n, where the input and outputrepresentations are in coefficient form. We assume that n is a power of 2; we canalways meet this requirement by adding high-order zero coefficients.1. Double degree-bound: Create coefficient representations of A.x/ and B.x/ as
degree-bound 2n polynomials by adding n high-order zero coefficients to each.
⇥(n2) ⇥(n2)
Can we improve evaluation and interpolation time to
? ⇥(n log n)
Complex Roots of Unity
• A complex n-‐th root of unity is a complex number such that
• Exponential of a complex number:
• There are exactly n complex n-‐th roots of unity:
!n = 1
!
eiu = cos(u) + i sin(u)
e2⇡ik/n k = 0, 1, . . . , n� 1
Example30.2 The DFT and FFT 907
1!1
i
!i
!08 D !8
8
!18
!28
!38
!48
!58
!68
!78
Figure 30.2 The values of !08 ; !1
8 ; : : : ; !78 in the complex plane, where !8 D e2! i=8 is the prin-
cipal 8th root of unity.
!n D e2! i=n (30.6)is the principal nth root of unity;2 all other complex nth roots of unity are powersof !n.
The n complex nth roots of unity,!0
n; !1n; : : : ; !n!1
n ;
form a group under multiplication (see Section 31.3). This group has the samestructure as the additive group .Zn;C/ modulo n, since !n
n D !0n D 1 implies that
!jn!k
n D !j Ckn D !.j Ck/ mod n
n . Similarly, !!1n D !n!1
n . The following lemmasfurnish some essential properties of the complex nth roots of unity.
Lemma 30.3 (Cancellation lemma)For any integers n " 0, k " 0, and d > 0,!dk
dn D !kn : (30.7)
Proof The lemma follows directly from equation (30.6), since!dk
dn D!e2! i=dn
"dk
D!e2! i=n
"k
D !kn :
2Many other authors define !n differently: !n D e!2! i=n. This alternative definition tends to beused for signal-processing applications. The underlying mathematics is substantially the same witheither definition of !n.
Principle n-‐th root of unity:
n complex n-‐th roots of unity:
!n = e2⇡i/n
!0n,!
1n, . . . ,!
n�1n
Discrete Fourier Transform• Evaluate A(x) of degree-‐bound n at
• The vector is the discrete Fourier Transform (DFT) of coefficient vector
A(x) =n�1X
j=0
ajxj
!0n,!
1n, . . . ,!
n�1n
yk = A(!kn) =
n�1X
j=0
aj!kjn k = 0, 1, . . . , n� 1
y = (y0, y1, . . . , yn�1)
a = (a0, a1, . . . , an�1)
O(n2)still
Physical Meaning of DFT
yk = A(!kn) =
n�1X
j=0
aj!kjn
aj =1
n
n�1X
k=0
yk!�kjn Inverse DFT
Signal at different frequencies Weight of thatfrequency
Fast Fourier Transform
• Taking advantage of the special properties of the complex roots of unity, we can compute DFT in
!• Assumption: n is a power of 2.• Split A(x) into two parts:
⇥(n log n)
A(x) = a0 + a1x+ a2x2 + a3x
3 + · · ·+ an�2xn�2 + an�1x
n�1
A
[0](x) = a0 + a2x+ a4x2 + · · ·+ an�2x
n/2�1
A
[1](x) = a1 + a3x+ a5x2 + · · ·+ an�1x
n/2�1
A(x) = A
[0](x2) + xA
[1](x2)
Fast Fourier Transform
• How do we evaluate A(x) at ?1. Evaluate and at
2. Combine the result using (ㄅ)
A
[0](x) = a0 + a2x+ a4x2 + · · ·+ an�2x
n/2�1
A
[1](x) = a1 + a3x+ a5x2 + · · ·+ an�1x
n/2�1
A(x) = A
[0](x2) + xA
[1](x2)
!0n,!
1n, . . . ,!
n�1n
A
[0](x)A
[1](x)
(!0n)
2, (!1n)
2, . . . , (!0n�1)
2
(ㄅ)
What is Divide-‐and-‐Conquer?
• When dealing with a problem:1. Divide the problem into
smaller, but same type of, problems2. If the problem is small enough to solve (Conquer),
• then solve it• Else recursively call itself to solve smaller sub-‐problems
3. Combine the solutions of smaller sub-‐problems into the solution of the original, larger, problem
18
Base case
Recursive case
Fast Fourier Transform
• How do we evaluate A(x) at ?1. Evaluate and at
2. Combine the result using (ㄅ)
A
[0](x) = a0 + a2x+ a4x2 + · · ·+ an�2x
n/2�1
A
[1](x) = a1 + a3x+ a5x2 + · · ·+ an�1x
n/2�1
A(x) = A
[0](x2) + xA
[1](x2)
!0n,!
1n, . . . ,!
n�1n
A
[0](x)A
[1](x)
(!0n)
2, (!1n)
2, . . . , (!0n�1)
2
(ㄅ)
Divide and Conquer 2 n/2-‐sized problems
Combine the sub-‐problem solutions
Pseudo-‐code30.2 The DFT and FFT 911
RECURSIVE-FFT.a/
1 n D a: length // n is a power of 22 if n == 13 return a4 !n D e2! i=n
5 ! D 16 aŒ0" D .a0; a2; : : : ; an!2/7 aŒ1" D .a1; a3; : : : ; an!1/8 yŒ0" D RECURSIVE-FFT.aŒ0"/9 yŒ1" D RECURSIVE-FFT.aŒ1"/
10 for k D 0 to n=2 ! 1
11 yk D yŒ0"k C ! yŒ1"
k
12 ykC.n=2/ D yŒ0"k ! ! yŒ1"
k13 ! D ! !n
14 return y // y is assumed to be a column vector
The RECURSIVE-FFT procedure works as follows. Lines 2–3 represent the basisof the recursion; the DFT of one element is the element itself, since in this casey0 D a0 !0
1
D a0 " 1
D a0 :
Lines 6–7 define the coefficient vectors for the polynomials AŒ0" and AŒ1". Lines4, 5, and 13 guarantee that ! is updated properly so that whenever lines 11–12are executed, we have ! D !k
n . (Keeping a running value of ! from iterationto iteration saves time over computing !k
n from scratch each time through the forloop.) Lines 8–9 perform the recursive DFTn=2 computations, setting, for k D0; 1; : : : ; n=2 ! 1,yŒ0"
k D AŒ0".!kn=2/ ;
yŒ1"k D AŒ1".!k
n=2/ ;
or, since !kn=2 D !2k
n by the cancellation lemma,
yŒ0"k D AŒ0".!2k
n / ;
yŒ1"k D AŒ1".!2k
n / :
Divide: 2x n/2
Conquer
Combine
Evaluate at !kn
O(n)
O(n log n)
904 Chapter 30 Polynomials and the FFT
a0; a1; : : : ; an!1
b0; b1; : : : ; bn!1
c0; c1; : : : ; c2n!2Ordinary multiplication
Time ‚.n2/
EvaluationTime ‚.n lg n/Time ‚.n lg n/
Interpolation
Pointwise multiplicationTime ‚.n/
A.!02n/; B.!0
2n/
A.!12n/; B.!1
2n/
A.!2n!12n /; B.!2n!1
2n /
::::::
C.!02n/
C.!12n/
C.!2n!12n /
Coefficient
Point-valuerepresentations
representations
Figure 30.1 A graphical outline of an efficient polynomial-multiplication process. Representationson the top are in coefficient form, while those on the bottom are in point-value form. The arrowsfrom left to right correspond to the multiplication operation. The !2n terms are complex .2n/th rootsof unity.
on whether we can convert a polynomial quickly from coefficient form to point-value form (evaluate) and vice versa (interpolate).
We can use any points we want as evaluation points, but by choosing the eval-uation points carefully, we can convert between representations in only ‚.n lg n/time. As we shall see in Section 30.2, if we choose “complex roots of unity” asthe evaluation points, we can produce a point-value representation by taking thediscrete Fourier transform (or DFT) of a coefficient vector. We can perform theinverse operation, interpolation, by taking the “inverse DFT” of point-value pairs,yielding a coefficient vector. Section 30.2 will show how the FFT accomplishesthe DFT and inverse DFT operations in ‚.n lg n/ time.
Figure 30.1 shows this strategy graphically. One minor detail concerns degree-bounds. The product of two polynomials of degree-bound n is a polynomial ofdegree-bound 2n. Before evaluating the input polynomials A and B , therefore,we first double their degree-bounds to 2n by adding n high-order coefficients of 0.Because the vectors have 2n elements, we use “complex .2n/th roots of unity,”which are denoted by the !2n terms in Figure 30.1.
Given the FFT, we have the following ‚.n lg n/-time procedure for multiplyingtwo polynomials A.x/ and B.x/ of degree-bound n, where the input and outputrepresentations are in coefficient form. We assume that n is a power of 2; we canalways meet this requirement by adding high-order zero coefficients.1. Double degree-bound: Create coefficient representations of A.x/ and B.x/ as
degree-bound 2n polynomials by adding n high-order zero coefficients to each.
⇥(n2)
Can we improve evaluation and interpolation time to
? ⇥(n log n)
⇥(n log n)
How about interpolation?? (p. 912 on Cormen)
