The Fast Fourier Transform and Applications to Multiplication Analysis of Algorithms Prepared by John Reif, Ph.D. Topics and Readings: - The Fast Fourier Transform • Reading Selection: • CLR, Chapter 30 Advanced Material : - Using FFT to solve other Multipoint Evaluation Problems - Applications to Multiplication Nth Roots of Unity • Assume Commutative Ring (R,+,·, 0,1) • ω is principal nth root of unity if – ωk ≠ 1 for k = 1, …, n-1 – ωn = 1, and n-1 ∑ω jp = 0 for 1≤ p ≤ n j=0 2πi/n • Example: ω = e for complex numbers Example of nth Root of Unity for Complex Numbers ω = e 2 π i / 8 is the 8th root of unity Fourier Matrix #1 1 … 1 $ % n−1 & %1 ω … ω & M (ω) = %1 ω 2 … ω 2(n−1) & n % & % & % n−1 (n−1)(n−1) & '1 ω … ω ( ij so M(ω)ij = ω for 0 ≤ i, j<n #a0 $ % & given a =% & % & 'a n-1 ( Discrete Fourier Transform T Input a column n-vector a = (a0, …, an-1) Output an n-vector which is the product of the Fourier matrix times the input vector DFTn (a) = M(ω) x a "f0 # $ % = $ % where $ % &fn-1 ' n-1 f = a ik i ∑ kω k=0 Inverse Fourier Transform -1 -1 DFTn (a) = M(ω) x a 1 Theorem M(ω)-1 = ω -ij ij n proof We must show M(ω)⋅M(ω)-1 = I 1 n-1 1 n-1 ∑ωikω -kj = ∑ω k(i-j) n k=0 n k=0 $0 if i-j ≠ 0 = % &1 if i-j = 0 n-1 using identity ∑ω kp = 0, for 1≤ p < n k=0 Fourier Transform is Polynomial Evaluation at the Roots of Unity T Input a column n-vector a = (a0, …, an-1) T Output an n-vector (f0, …, fn-1) which are the values polynomial f(x)at the n roots of unity "f0 # DFT (a) = $ % where n $ % $ % &fn-1 ' i fi = f(ω ) and n-1 f(x) = a x j ∑ j ⋅ j=0 Fast Fourier Transform • Viewed as Evaluation Problem: naïve algorithm takes n2 ops • Divide and Conquer gives FFT with O(n log n) ops for n a power of 2 • Key Idea: • If ω is nth root of unity then ω2 is n/2th root of unity • So can reduce the problem to two subproblems of size n/2 Algorithm FFTn T • Input a = (a0, …, an-1) , n a power of 2 [1] If n=1 then ouput T ⎛⎞'' T [2] ⎜⎟f0n ,...,f← FFTnn (( a 022 , a ,..., a − ) ) −1 ⎝⎠22 T ⎛⎞"" T ⎜⎟f0n ,...,f← FFTnn (( a 131 , a ,..., a − ) ) −1 ⎝⎠22 n [3] For i=0, ...,−← 1 do f f'"+ωi f 2 ii i '"i fni←− fω f i i+ 2 [4] Output (f0 , f 1 , ..., f n-1 ) FFT Circuit (also known as Butterfly Network) • Total Recursion depth = log n • Communication Distance 2d at depth d i 2i (n-1)i fi = a 0 + a 1ωω + a 2 +...+a n-1 ω 'i" fii = f + ω f i where i(n-2) ' 2 i 2 2i 2 2 fi = a 0 + a 2 (ωω ) + a 4 ( ) +...+ a n-2 ( ω ) i(n-2) "2i22 fi = a 1 + a 3 (ωω ) +...+ a n-1 ( ) ⎡⎤' ⎡⎤a0 ⎢⎥f0 ⎢a ⎥ ⎢⎥ M = M()ω 2 ⎢ 2 ⎥ = DFT(( a , a ,..., a )T ) ⎢⎥ n ⎢ M ⎥ nn02− 2 ' 2 2 ⎢⎥f n ⎢ ⎥ −1 a ⎣⎦⎢⎥2 ⎣⎦n−2 ⎡⎤" ⎡⎤a1 ⎢⎥f0 ⎢⎥a ⎢⎥ M ==M (ω 2 )⎢⎥3 DFT (( a , a ,..., a )T ) ⎢⎥ n131⎢⎥ M nn− " 22 ⎢⎥f n ⎢⎥ −1 a ⎣⎦⎢⎥2 ⎣⎦n−1 ' ' " " n Note: f n = fi , f n = fi , i=0, ..., −1 +i +i 2 2 2 n +1 But ω n =1, so (ω 2 ) 2 = ω n ⋅(ω 2 )i = ω 2i n for i=0, ..., −1 2 n Thus, f = f ' + ωi f " for i=0, ..., −1 i i i 2 n i + ' 2 " and f n = fi +ω ⋅ fi i + 2 n = f ' - ωi f " for i=0, ..., −1 i i 2 n n since (ω 2 )2 = ω n =1, so ω 2 = -1 Operation Counts for FFT Algorithm • Assume n = 2k • # additions Add(n) = 2· Add(n/2) + n = n log n • # multiplications Mult(n) = 2· Mult(n/2) + n/2 = ½ n log n • Total Time O(n log n) • Note in complex FFT, # real ops is 5 n log n Multipoint Polynomial Evaluation n−1 f (x) a xi • Input polynomial = ∑ i i=0 • Problem evaluate f(x) at x0, x1, …, xn-1 • Easy Cases: eval at roots of unity k FFT Case xk = ω for k=0,…,n Multipoint Polynomial Evaluation (cont’d) Summary of FFT: method f() x=+ f '() y x f "() y where yx= 2 fxf'( ), "( x ) both degree halved ⇒ needed to only evaluate at half as many points Other Polynomial Evaluation Problems Solved by FFT Each costs O(n log n) time i • Evaluate at points Xi = ba + d for i=0,…, n-1(Chirp Transfom) – Reduced to FFT • Single point evaluation of all derivatives of a polynomial – Solve by reduction to above Chirp Transform of case 2) i 2 i • Evaluate at points Xi = b(a ) + ca + d for i=0,…, n-1 – Solve by divide and conquer similar to FFT Single Point Evaluation of all Derivatives of Polynomial • Input n−1 f ( x ) a x i and point x = ∑ i 0 i=0 • output d k f (x) f k (x) = x = x for k = 0,...,n −1 dx 0 Single Point Evaluation of all Derivatives of Polynomial (cont’d) • Taylor Series Representation of n−1 f (x) c (x x )i = ∑ i − 0 i=0 Then (k ) f (x0 ) = k!ck This reduces to case of evaluation at points i xi == ab for i0,..., n − 1 • Solve this Chirp Transform problem by reduction FFT Advanced Material: Further Applications of FFT 1) Convolution: Products and Powers of Polynomials • Used for for Integer Multiplication Algorithms • Also used for Filtering on infinite input streams 2) Division and Inverse of Polynomials 3) Multipoint Evaluation and Interpolation Advanced Material: Products and Powers of Polynomials • Input vectors T a = (a0, a1, …, an-1) T b = (b0, b1, …, bn-1) • Definition of Convolution c = a b ⊗ n-1 c = a b Where i ∑ j i- j for i=0, …, 2n-1 j=0 define ak = bk = 0 if k< 0 or k>n Products and Powers of Polynomials (cont’d) • Convolution Theorem a b = FFT-1 FFT (a) FFT (b) ⊗ 2n ( 2n ⋅ 2n ) • Application to Polynomial Products: n-1 p(x) = a xi ∑ i i=0 n-1 q(x) = b x j ∑ j j=0 2n-2 n-1 p(x) q(x) = c xi where c = a b ⋅ ∑ i i ∑ j i-j i=0 j=0 Products of m Polynomials n-1 for k=1, ..., m let P (x) = a xi k ∑ k,i i=0 m m(n-1) m P (x) = c xi , where c = a ∏ k ∑ i i ∑∏ k,jk k=1 i=0 k=1 ∑ jk =1 • Generalized Convolution Theorem a12⊗⊗⊗ a... a= m -1 FFTn m( FFT n m (a 1 )••• FFT n m (a 2 ) ... FFT n m (a m )) Wrapped Convolutions T T • a = (a0, a1, …, an-1) , b = (b0, b1, …, bn-1) • Positive wrapped convolution is T c = (c0, c1, …, cn-1) i n-1 c = a b + a b i ∑ j i-j ∑ j n+i-j j=0 j=i+1 • Negative wrapped convolution is T d = (d0, d1, …, dn-1) i n-1 d = a b - a b i ∑ j i-j ∑ j n+i-j j=0 j=i+1 Application of Wrapped Convolution to Modular Polynomial Products n-1 p(x) = a xi ∑ i i=0 n-1 q(x) = b x j ∑ j j=0 p(x)⋅q(x) mod(xn +1) n-1 = d xi since xn = -1 mod(xn +1) ∑ i i=0 Computing Positive Wrapped Convolution • Let ω = principal nth root of unity • Assume n has multiplicative inverse, Theorem -1 c = FFTn (FFTn (a)⋅FFTn (b)) is the positive wrapped convolution of n-vectors a and b. Computing Negative Wrapped Convolution • Also dˆ = FFT-1 FFT (aˆ) FFT (bˆ) n ( n ⋅ n ) is the negatively wrapped convolution of n-vectors a and b T where ˆ n-1 a = (a0 , Ψa1, ...,Ψ a n-1 ) T ˆ n-1 b = b0 , Ψb1, ...,Ψ bn-1 ( ) and Ψ2 = ω = principal nth root of unity Integer Multiplication by Polynomial Product (solved via FFT) • Input n bit integers a,b define polynomials degree k = n/L k-1 a(x) = a xi , 0 a 2L ∑ i ≤ i ≤ i=0 k-1 b(x) = b xi , 0 b 2L ∑ i ≤ i ≤ i=0 so a = a(2L ), b = b(2L ) Integer Multiplication by Polynomial Product (cont’d) • Idea 1) Compute c(x) = a(x)· b(x) by convolution 2) Evaluate c(2L) = a· b Integer Multiplication Algorithms using Reduction to Polynomial Product • Pollard Mult Algorithm O(n(logn)2 )(loglogn)∈ use L = logn • Karp Mult Algorithm O(n(logn)2 ) use L = n • Schönage-Strassen Mult Algorithm O(n(logn)(loglogn)) use L = n and wrapped convolution Pollard Multiplication Algorithm • n = kL, L = 1 + log k 1) Choose primes P1, P2, P3 where 3 P1 ⋅P2 ⋅P3 ≥ 4⋅k L and Pi = á i ⋅2 + 1, á i = O(1) 2) Compute C(x) by convolution over finite field Zpi for i =1,2,3 (requires k mults on 2L bit integers) Pollard Multiplication Algorithm (cont’d) 3) Evaluate C(2L) • Time Bounds recursive mults FFT T(n) = 3kT(2L) + O(k logk)⋅O(L) = 3kT (2(1 + logk)) + O(k(log k)2 ) ≤ O(n(log n)2 (log log n)∈) for any ∈ > 0 Korp Multiplication Algorithm n = 2s = kL ! s 22 if s even k = " # (s-1) " 2 $2 else 1) Compute C(x) modulo k by convolution 2) Compute C(x) modulo (22L+1) by convolution 3) Compute C(x) coefficients from C(x) mod k, C(x) mod (22L+1) by Chinese remaindering Korp Multiplication Algorithm (cont’d) 4) Compute C(2L) • Time recursive mults FFT T(n) = 2kT(2L) + O(k logk)O(L) = 2 nT (2 n ) + O(n log n) = O(n(log n)2 ) Schönage-Strassen Multiplication Algorithm (2’) Compute C(x) mod (xk+1) modulo (22L+1) by wrapped convolution Requires only k recursive mults on 2L bit numbers • Time recursive mults FFT T(n) = kT(2L) + O(k logk)O(L) = nT (2 n ) + O(n log n) = O(n log n)(log log n) Still Open Problem: How Fast Can You Multiply Integers? • Can you mult n bit integers in O(n log n) time? The Fast Fourier Transform and Applications to Multiplication Analysis of Algorithms Prepared by John Reif, Ph.D.