Lecture 5. FFT and Divide and Conquer

Lecture 5. FFT and Divide and Conquer

CS711008Z Algorithm Design and Analysis Lecture 5. FFT and Divide and Conquer Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China . 1 / 57 Outline DFT: evaluate a polynomial at n special points; FFT: an efficient implementation of DFT; Applications of FFT: multiplying two polynomials (and multiplying two n-bits integers); time-frequency transform; solving partial differential equations; Appendix: relationship between continuous and discrete Fourier transforms. 2 / 57 DFT: Discrete Fourier Transform n−1 DFT evaluates a polynomial A(x) = a0 + a1x + ::: + an−1x 2 n−1 2π i at n distinct points 1; !; ! ; :::; ! , where ! = e n is the n-th complex root of unity. Thus, it transforms the complex vector a0; a1; :::; an−1 into k another complex vector y0; y1; :::; yn−1, where yk = A(w ), i.e., y0 = a0 + a1 + a2 ::: + an−1 1 2 n−1 y1 = a0 + a1! + a2! ::: + an−1! ::: ::: ::: :::::: n−1 2(n−1) (n−1)2 yn−1 = a0 + a1! + a2! ::: + an−1! Matrix form: 2 3 2 3 2 3 1 1 1 ::: 1 y0 a0 6 7 6 1 !1 !2 :::!n−1 7 6 7 6 y1 7 6 7 6 a1 7 6 7 = 6 1 !2 !4 :::!2(n−1)7 6 7 4 . 5 6 7 4 . 5 . 4::::::::::::::: 5 . 2 yn−1 1 !n−1 !2(n−1) :::!(n−1) an−1 . 3 / 57 FFT: a fast way to implement DFT [Cooley and Tukey, 1965] Direct matrix-vector multiplication requires O(n2) operations when using the Horner’s method, i.e., A(x) = a0 + x(a1 + x(a2 + ::: + xan−1)). 2 FFT: reduce O(n ) to O(n log2 n) using divide-and-conquer technique. How does FFT achieve this? Or what calculations are redundant in the direct matrix-vector multiplication approach? Note: The idea of FFT was proposed by James Cooley and John Tukey in 1965 when analyzing earth-quake data, but the idea can be dated back to F. Gauss. 4 / 57 Let’s evaluate A(x) at four special points It is easy to evaluate any 1-degree polynomial A(x) = a0 + a1x at two points 1; −1. Now let’s evaluate a 2 3 3-degree polynomial A(x) = a0 + a1x + a2x + a3x at four special points 1; i; −1; −i. Divide: Break the polynomial into even and odd terms, i.e., Aeven(x) = a0 + a2x Aodd(x) = a1 + a3x Then we have the following equations: 2 2 2 2 A(x) = (a0 + a2x ) + x(a1 + a3x ) = Aeven(x ) + xAodd(x ) 2 2 2 2 A(−x) = (a0 + a2x ) − x(a1 + a3x ) = Aeven(x ) − xAodd(x ) Combine: For the 4 special points 1; i; −1; −i, we have A(1) = Aeven(1) + Aodd(1) A(i) = Aeven(−1) + iAodd(−1) A(−1) = Aeven(1) − Aodd(1) A(−i) = Aeven(−1) − iAodd(−1) In other words, the values of A(x) at 4 points 1; i; −1; −i can be calculated based on the values of Aeven(x); Aodd(x) at 2 points 1; −1. 5 / 57 An example: n = 4 y0 = a0 + a1 + a2 + a3 1 2 3 y1 = a0 + a1! + a2! + a3! 2 4 6 y2 = a0 + a1! + a2! + a3! 3 6 9 y3 = a0 + a1! + a2! + a3! Objective: Evaluate A(x) at 4 points: 1; !; !2;!3, where 1 2πi ! = e 4 . = ! +i 1 !2 −1 !4 < !3 −i . 6 / 57 Step 1: Simplification y0 = a0 + a1 + a2 + a3 1 2 3 y1 = a0 + a1! + a2! + a3! 2 2 y2 = a0 + a1! + a2 + a3! 3 2 1 y3 = a0 + a1! + a2! + a3! . 7 / 57 Step 2. Divide into odd- and even-terms y0 = a0 + a2 + a1 + a3 2 1 3 y1 = a0 + a2! + a1! + a3! 2 2 y2 = a0 + a2 + a1! + a3! 2 3 1 y3 = a0 + a2! + a1! + a3! . 8 / 57 Key observation: redundant calculations y0 = a0 + a2 + a1 + a3 2 1 3 y1 = a0 + a2! + a1! + a3! 2 2 y2 = a0 + a2 + a1! + a3! 2 3 1 y3 = a0 + a2! + a1! + a3! Note that the calculations in the two red frames are identical, and the calculations in the blue frames are also identical after 2π multiplying by !2. Here !2 = −1 as ! = e 4 i. 9 / 57 Step 3: Divide-and-conquer y0 = a0 + a2 + a1 + a3 2 1 3 y1 = a0 + a2! + a1! + a3! 2 2 y2 = a0 + a2 + a1! + a3! 2 3 1 y3 = a0 + a2! + a1! + a3! Thus the calculations in the top-left and bottom-right frames are × redundant. We need only 2 + 4 + 2 = 4 log2 4 calculations. 10 / 57 Another example: n = 8 y0 = a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7 1 2 3 4 5 6 7 y1 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 2 4 6 8 10 12 14 y2 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 3 6 9 12 15 18 21 y3 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 4 12 16 20 24 28 y4 = a0 + a1! + a2 + a3! + a4! + a5! + a6! + a7! 5 10 15 20 25 30 35 y5 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 6 12 18 24 30 36 42 y6 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 7 14 21 28 35 42 49 y7 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! Objective: Evaluate A(x) at 8 points: 1; !; !2; :::; !7, where 1 2πi ! = e 8 . = !2 +i !3 !1 !4 1 −1 !8 < . 5 7 . ! ! 11 / 57 !6 −i Step 1: Simplification y0 = a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7 1 2 3 4 5 6 7 y1 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 2 4 6 2 4 6 y2 = a0 + a1! + a2! + a3! + a4 + a5! + a6! + a7! 3 6 1 4 7 2 5 y3 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 4 8 4 4 4 y4 = a0 + a1! + a2! + a3! + a4 + a5! + a6 + a7! 5 2 7 4 1 6 3 y5 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! 6 4 2 6 4 2 y6 = a0 + a1! + a2! + a3! + a4 + a5! + a6! + a7! 7 6 5 4 3 2 1 y7 = a0 + a1! + a2! + a3! + a4! + a5! + a6! + a7! . 12 / 57 Step 2. Divide into odd- and even-terms y0 = a0 + a4 + a2 + a6 + a1 + a5 + a3 + a7 4 2 6 1 5 3 7 y1 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 2 2 6 6 y2 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 3 7 1 5 y3 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 4 4 y4 = a0 + a4 + a2 + a6 + a1! + a5! + a3! + a7! 4 2 6 5 1 7 3 y5 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 6 6 2 2 y6 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 7 3 5 1 y7 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! The specific order of these terms will be explained later. 13 / 57 Key observation: redundant calculations y0 = a0 + a4 + a2 + a6 + a1 + a5 + a3 + a7 4 2 6 1 5 3 7 y1 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 2 2 6 6 y2 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 3 7 1 5 y3 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 4 4 y4 = a0 + a4 + a2 + a6 + a1! + a5! + a3! + a7! 4 2 6 5 1 7 3 y5 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 6 6 2 2 y6 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 7 3 5 1 y7 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! Note that the calculations in the two red frames are identical, and the calculations in the blue frames are also identical after 2π multiplying by !4. Here !4 = −1 as ! = e 8 i. 14 / 57 Step 3: Divide-and-conquer y0 = a0 + a4 + a2 + a6 + a1 + a5 + a3 + a7 4 2 6 1 5 3 7 y1 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 2 2 6 6 y2 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 3 7 1 5 y3 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 4 4 y4 = a0 + a4 + a2 + a6 + a1! + a5! + a3! + a7! 4 2 6 5 1 7 3 y5 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 6 6 2 2 y6 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 7 3 5 1 y7 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! Thus the calculations in the top-left and bottom-right frames are redundant. 15 / 57 Step 3: Divide-and-conquer y0 = a0 + a4 + a2 + a6 + a1 + a5 + a3 + a7 4 2 6 1 5 3 7 y1 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 2 2 6 6 y2 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 3 7 1 5 y3 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 4 4 y4 = a0 + a4 + a2 + a6 + a1! + a5! + a3! + a7! 4 2 6 5 1 7 3 y5 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! 4 4 6 6 2 2 y6 = a0 + a4 + a2! + a6! + a1! + a5! + a3! + a7! 4 6 2 7 3 5 1 y7 = a0 + a4! + a2! + a6! + a1! + a5! + a3! + a7! × Finally, we need only 2 + 4 + 2 + 8 + 2 + 4 + 2 = 8 log2 8 calculations. 16 / 57 The final order 0; 1; 2; 3; 4; 5; 6; 7 0; 2; 4; 6 1; 3; 5; 7 0; 4 2; 6 1; 5 3; 7 0 4 2 6 1 5 3 7 000 100 010 110 001 101 011 111 .

View Full Text

Details

  • File Type
    pdf
  • Upload Time
    -
  • Content Languages
    English
  • Upload User
    Anonymous/Not logged-in
  • File Pages
    57 Page
  • File Size
    -

Download

Channel Download Status
Express Download Enable

Copyright

We respect the copyrights and intellectual property rights of all users. All uploaded documents are either original works of the uploader or authorized works of the rightful owners.

  • Not to be reproduced or distributed without explicit permission.
  • Not used for commercial purposes outside of approved use cases.
  • Not used to infringe on the rights of the original creators.
  • If you believe any content infringes your copyright, please contact us immediately.

Support

For help with questions, suggestions, or problems, please contact us