13 Fast Fourier Transform (FFT) The fast Fourier transform (FFT) is an algorithm for the efficient implementation of the discrete Fourier transform. We begin our discussion once more with the continuous Fourier transform. 13.1 Continuous and Discrete Fourier Transforms Revisited Let Ek be the complex exponential defined by ikx Ek(x) := e and consider the complex inner product 1 Z π hf, gi = f(x)g(x)dx. 2π −π Then Z π 1 ikx −ikx hEk,Eki = e e dx = 1, 2π −π | {z } =1 and for k 6= ` we have Z π 1 ikx −i`x hEk,E`i = e e dx 2π −π 1 Z π = ei(k−`)xdx 2π −π π 1 ei(k−`)x = 2π i(k − `) −π 1 h i = ei(k−`)π − e−i(k−`)π 2πi(k − `) " # 1 = ei(k−`)π 1 − e2π(`−k)i = 0. 2πi(k − `) | {z } =1 As we just saw, the complex exponentials are orthonormal on the interval [−π, π]. In fact, they are used as the basis functions for complex Fourier series ∞ X ikx f(x) ∼ cke k=−∞ with Fourier coefficients Z π 1 −ikt ˆ ck = f(t)e dt = hf, Eki = f(k). 2π −π ˆ As mentioned earlier, the coefficients ck = f(k) are also known as the Fourier transform of f. Remark Note that the notation used here is a bit different from that used in Chapter 11. In particular, earlier we used the interval (−∞, ∞). 135 In order to get to the discrete Fourier transform we first truncate the Fourier series and obtain n X P (x) = ckEk(x) k=0 — a trigonometric polynomial. This can be verified by writing n n X ikx X ixk P (x) = cke = ck e k=0 k=0 so that P is a polynomial of degree n in the complex exponential eix. We now also consider equally spaced data in the interval [0, 2π), i.e., 2πj (x , f(x )) with x = , j = 0, 1,...,N − 1, (122) j j j N and introduce a discrete (pseudo-)inner product N−1 1 X 2πj 2πj hf, gi = f g . (123) N N N N j=0 Remark This is only a pseudo-inner product since hf, fiN = 0 implies f = 0 only at 2πj the discrete nodes N , j = 0, 1,...,N − 1. N−1 Theorem 13.1 The set {Ek}k=0 of complex exponentials is orthonormal with respect to the inner product h , iN defined in (123). Proof First, N−1 1 X 2πj 2πj hE ,E i = E E k k N N k N k N j=0 N−1 1 X = eik2πj/N e−ik2πj/N = 1. N | {z } j=0 =1 Next, for k 6= ` we have N−1 1 X 2πj 2πj hE ,E i = E E k ` N N k N ` N j=0 N−1 1 X = eik2πj/N e−i`2πj/N N j=0 N−1 j 1 X i 2π(k−`) = e N . N j=0 Now, since k 6= `, i 2π(k−`) e N 6= 1 136 we can apply the formula for the sum of a finite geometric series to get N i 2π(k−`) 1 e N − 1 hEk,E`iN = . N i 2π(k−`) e N − 1 Finally, since N i 2π(k−`) e N = 1 the orthogonality result follows. This gives rise to a best approximation result (just as for standard Fourier series): n Corollary 13.2 Let data be given as in (122) and consider G = span{Ek}k=0 with n < N. Then the best least squares approximation to the data from G (with respect to the inner product (123)) is given by the discrete Fourier series n X g(x) = ckEk(x) k=0 with N−1 1 X 2πj −ik 2πj c = hf, E i = f e N , k = 0, 1, . , n. k k N N N j=0 Remark 1. The coefficients ck of the discrete Fourier series are called the discrete Fourier transform (DFT) of f. 2. It is our goal to compute the DFT (and also evaluate g) via the fast Fourier transform (FFT). 3. Note that, even if the data is real, the DFT will in general be complex. If, e.g., 2πj N = 4, and the values f N , j = 0, 1, 2, 3 are 4, 0, 3, 6 respectively, then the coefficients ck are c0 = 13 c1 = 1 + 6i c2 = 1 c3 = 1 − 6i. 13.2 The FFT Algorithm We start with the trigonometric polynomial N−1 X P (x) = ckEk(x) k=0 with DFT coefficients N−1 1 X 2πj −ik 2πj c = f e N k N N j=0 137 N−1 1 X = f(x )(λ )j N j k j=0 2πj where xj = N and we set −i 2πk λk = e N . Remark The values λk (or more naturally λ−k) are often referred to as the N-th roots of unity. This is illustrated in Figure 8 for N = 5. 1.0 l 0.8 4 0.6 l 3 0.4 0.2 Re −1.0 −0.8 −0.6 −0.4 −0.2 0.0 0.2 0.4 0.6 0.8 1.0 0.0 l −0.2 0 −0.4 l Im 2 −0.6 −0.8 l1 −1.0 Figure 8: Roots of unity λk, k = 0, 1,...,N − 1, for N = 5. The computation of each coefficient ck corresponds to evaluation of a polynomial of degree N − 1 in λk. Using nested multiplication (Horner’s method) this can be done in O(N) operations. Since we have N coefficients, the polynomial P can be constructed in O(N 2) operations. The major benefit of the fast Fourier transform is that it reduces the amount of work to O(N log2 N) operations. The FFT was discovered by Cooley and Tukey in 1965. However, Gauss seemed to already be aware of similar ideas. One of the most popular modern references is the ”DFT Owners Manual” by Briggs and Henson (published by SIAM in 1995). It is the dramatic reduction in computational complexity that earned the FFT a spot on the list of ”Top 10 Algorithms of the 20th Century” (see handout). The FFT is based on the following observation: kπ Theorem 13.3 Let data (xk, f(xk)), xk = n , k = 0, 1,..., 2n − 1, be given, and assume that p and q are exponential polynomials of degree at most n−1 which interpolate 138 part of the data according to p(x2j) = f(x2j), q(x2j) = f(x2j+1), j = 0, 1, . , n − 1, then the exponential polynomial of degree at most 2n−1 which interpolates all the given data is 1 1 π P (x) = 1 + einx p(x) + 1 − einx q x − . (124) 2 2 n Proof Since einx = eixn has degree n, and p and q have degree at most n − 1 it is clear that P has degree at most 2n − 1. We need to verify the interpolation claim, i.e., show that P (xk) = f(xk), k = 0, 1,..., 2n − 1. By assumption we have 1 1 π P (x ) = 1 + einxk p(x ) + 1 − einxk q x − k 2 k 2 k n where inx in kπ iπk k e k = e n = e = (−1) . Therefore ( p(x ) if k even, P (x ) = k k π q xk − n if k odd. Let k be even, i.e., k = 2j. Then, by the assumption on p P (x2j) = p(x2j) = f(x2j), j = 0, 1, . , n − 1. On the other hand, for k = 2j + 1 (odd), we have (using the assumption on q) π P (x ) = q x − = q(x ) = f(x ), j = 0, 1, . , n − 1. 2j+1 2j+1 n 2j 2j+1 This is true since π (2j + 1)π π 2jπ x − = − = = x . 2j+1 n n n n 2j The second useful fact tells us how to obtain the coefficients of P from those of p and q. Theorem 13.4 Let n−1 n−1 2n−1 X X X p = αjEj, q = βjEj, and P = γjEj. j=0 j=0 j=0 Then, for j = 0, 1, . , n − 1, 1 1 γ = α + e−ijπ/nβ j 2 j 2 j 1 1 γ = α − e−ijπ/nβ . j+n 2 j 2 j 139 Proof In order to use (124) we need to rewrite n−1 π X π q x − = β E x − n j j n j=0 n−1 X ij(x− π ) = βje n j=0 n−1 X ijx −ijπ/n = βje e . j=0 Therefore 1 1 π P (x) = 1 + einx p(x) + 1 − einx q x − 2 2 n n−1 n−1 1 X 1 X = 1 + einx α eijx + 1 − einx β eijxe−ijπ/n 2 j 2 j j=0 j=0 n−1 1 X h i = α + β e−ijπ/n eijx + α − β e−ijπ/n ei(n+j)x . 2 j j j j j=0 However, the terms in parentheses (together with the factor 1/2) lead precisely to the formulæ for the coefficients γ. Example We consider the data x 0 π . y f(0) f(π) Now we apply the DFT to find the interpolating polynomial P . We have n = 1 and 1 X 0 ix P (x) = γjEj(x) = γ0e + γ1e . j=0 According to Theorem 13.4 the coefficients are given by 1 γ = α + β e0 , 0 2 0 0 1 γ = α − β e0 .