Introduction to the FFT

1 Introduction Assume that we have a “signal” f(x) where x denotes “time”. We would like to represent f(x) as a linear combination of functions e2πiax (or, equivalently, sin(2πax) and cos(2πax)) of frequency a. Note that a˜ = 2πa is often called circular frequency. The frequency is the number of periods per unit interval, the circular frequency is the number of periods per interval of length 2π. Here we will always use frequencies and have 2π in all exponents. The resulting coefficients fˆ(a) describe “how strong frequency a is present in the signal” and are interesting in many applications. Often one wants to apply certain “filters” which are easier to apply in the “frequency domain”. So one first applies the Fourier transformation, then applies some other transformation, and then performs an inverse Fourier transformation. This approach is attractive since there exists a Fast Fourier Algorithm which takes only O(N log(N)) operations. This is in particular useful for convolution operations. We will always assume that the function f(x) is complex valued. In fact, the Fourier data fˆ will be complex valued even for a real valued function f(x).

2 Diﬀerent cases of “Fourier transforms” There are four basic cases, depending on whether one has periodic or nonperodic data, or continuous or discrete data. In each case the correspondence of f and fˆ is stated for the case where f, fˆ are square integrable or summable. Note that Fourier transforms can also be deﬁned for more general functions and even distributions.

2.1 Different conventions for definition of “Fourier transform” We will always put 2π in the exponent for consistency. Often Fourier transforms are defined without 2π in the exponent, giving factors involving 2π in the definition of the Fourier transform, its inverse, or 1 both. Also, often i is replaced with −i in the definitions. Furthermore, for discrete data factors of N or h occur which can be split up in different ways in the definition of the Fourier transform and its inverse. These different definitions are trivially related to each other. But one has to carefully check what the definition of “Fourier transform” is in a specific book or software package to make sure that one obtains correct results.

2.2 Periodic functions

Let f : R → C be a 1-periodic function, i.e., f(x) = f(x + 1) for all x ∈ R. We want to represent this P ˆ 2πikx ˆ R 1 2 function in the form f(x) = fke using coeﬃcients fk, k ∈ . If |f(x)| dx < ∞ there are k∈Z Z 0 ˆ P ˆ 2 unique coeﬃcients (fk)k∈ with fk < ∞ and we have the following relations: Z k∈Z

1 Z ˆ −2πikx X ˆ 2πikx fk = f(x)e dx ⇐⇒ f(x) = fke Inversion (1) 0 k∈Z 1 Z 2 X ˆ 2 |f(x)| dx = fk Parseval (2) 0 k∈Z

1 1 Z ˆ q(x) = f(t)g(x − t) dt ⇐⇒ qˆk = fkgˆk Convolution (3) 0 ˆ P ˆ 2πikx Proof: Assume ﬁrst that only ﬁnitely many fk are nonzero, and that f(x) = fke . Then the k∈Z result follows since 1 ( Z 1 if j = k e2πijxe−2πikxdx = (4) 2πi(j−k)x 2π c[e ]0 = 0 if j 6= k 0 This also implies Parseval’s identity (2). By using a density argument one can conclude that (1), (2) R 1 2 hold for all f with 0 |f(x)| dx < ∞. For the convolution formula we have 1 1 1 1−t Z Z Z Z −2πikx −2πiky −2πikt ˆ qˆk = f(t)g(x − t) dt e dx = f(t)g(y)e e dy dt = fkgˆk x=0 t=0 t=0 y=−t by changing the order of integration and a change of variables from x to y = x − t. 2

2.3 Discrete data of periodic functions j We consider the values f of a 1-periodic function at the points for j ∈ , i.e., we have f = f . j N Z j j+N We want to ﬁnd a function f˜(x) of the form N−1 ˜ X ˆ 2πikx f(x) = fke (5) k=0 ˜ j which passes through the given points, i.e., f( n ) = fj, j = 0,...,N − 1. For given values (f0, . . . , fN−1) ˆ ˆ there exists a unique solution (f0,..., fN−1) and we have the following relations: N−1 N−1 1 X X fˆ = f e−2πikj/N ⇐⇒ f = fˆ e2πikj/N Inversion (6) k N j j k j=0 k=0 N−1 N−1 1 X 2 X ˆ 2 |fj| = fk Parseval (7) N j=0 k=0 N−1 1 X q = f g ⇐⇒ qˆ = fˆ gˆ Convolution (8) j N ` j−` k k k `=0 Proof of (6): Assume that the right equation in (6) holds. Then N−1 N−1 N−1 1 X X 1 X f e−2πikj/N = fˆ e2πi(`−k)j/N N j ` N j=0 `=0 j=0 1 PN−1 2πi(k−`)j/N PN−1 2πi(k−`)j/N 1 N If ` = k we have N j=0 fje = 1. If ` 6= k we have j=0 e = N (q − 1)/(q − 1) with q = e2πi(`−k)/N , hence qN = 1 and the sum is zero. 2 The function f˜(x) in (5) uses values of k in the range 0,...,N − 1. Usually one prefers to have k values in a symmetric range about zero since the frequency of eikx is |k|, and we would like to include all frequencies below a certain bound. It turns out that we can construct a diﬀerent interpolating function ˆ ˆ 2πiNx j p˜(x) using (f0,..., fN−1). The key observation is that e = 1 at all points x = N . Hence we can divide some terms in (5) by e2πiNx and still have an interpolating function.

2 N−1 2πiNx N is odd: Here we divide the terms in (5) with k = 2 ,...,N − 1 by e and obtain

(N−1)/2 N−1 (N−1)/2 X ˆ 2πikx X ˆ 2πi(k−N)x X ˆ 2πikx p˜(x) = fke + fke = fke (9) k=0 k=(N−1)/2+1 k=−(N−1)/2 ˆ ˆ if we deﬁne fk := fk+N for k < 0. In other words, we use the coeﬃcients

N−1 N−1 − 2 ... −1 0 1 ... 2 ˆ ˆ ˆ ˆ ˆ f(N−1)/2+1 ... fN−1 f0 f1 ... f(N−1)/2 Using eit = cos t + i sin t we can rewritep ˜(x) in the form

(N−1)/2 X p˜(x) = a0 + ak cos(2πkx) + bk sin(2πkx) . k=1

If all fj are real, then ak, bk are real.

N 2πiNx N is even: Here we divide the terms in (5) with k = 2 + 1,...,N − 1 by e , and for k = N/2 we ˆ 2πi(N/2)x 1 ˆ 2πi(N/2)x 1 ˆ 2πi(N/2)x −2πiNx replace fN/2e with 2 fN/2e + 2 fN/2e e yielding

N/2−1 X ˆ 2πikx 1 ˆ −2πi(N/2)x −2πi(−N/2)x p˜(x) = fke + 2 fN/2 e + e (10) k=−N/2+1 ˆ ˆ if we deﬁne fk := fk+N for k < 0. In other words, we use the coeﬃcients

N N N N − 2 − 2 + 1 ... −1 0 1 ... 2 − 1 2 1 ˆ ˆ ˆ ˆ ˆ ˆ 1 ˆ 2 fN/2 fN/2+1 ... fN−1 f0 f1 ... fN/2−1 2 fN/2 Using eit = cos t + i sin t we can rewritep ˜(x) in the form

N/2−1 X N p˜(x) = a0 + ak cos(2πkx) + bk sin(2πkx) + aN/2 cos(2π 2 x). (11) k=1

If all fj are real, then ak, bk are real. ˆ ˆ We can use (5) to define the Fourier coefficients (f0,..., fN−1), but we should use (9), (10) to interprete ˆ them in terms of frequencies: The coefficients fj with j close to 0 or N −1 correspond to “low frequency” ˆ components of f, the coefficients fj with j close to N/2 correspond to high frequency components. We 1 N 1 also see that given data with a spacing d = N allows us to determine frequencies up to 2 = 2d . This 1 frequency 2d is called the Nyquist frequency.

−1 Matlab: The function fft in Matlab implements NFN and the function fft implements N FN . Assume we have the Matlab vectors f and fhat with ˆ ˆ f = [f0, . . . , fN−1], fhat = [f0,..., fN−1] then ˆ ˆ −1 fft(f) = N[f0,..., fN−1], ifft(fhat) = N [f0, . . . , fN−1]. Note that Matlab vectors of length N are indexed by 1,...,N (instead of 0,...,N − 1).

3 2.4 Manipulating the Fourier vector ˆ ˆ Recall that we should interprete the Fourier vector (f0,..., fN−1) as follows: For odd N = 2n + 1 the vector contains the coeﬃcients of the functions ˆ ˆ ˆ ˆ ˆ f0 f1 ... fn fN−n ... fN−1 1 e2πix . . . e2πinx e−2πinx . . . e−2πix For even N = 2n the vector contains the coeﬃcients of the functions ˆ ˆ ˆ ˆ ˆ ˆ f0 f1 ... fn−1 fn fN−(n−1) ... fN−1 2πix 2πi(n−1)x e2πinx+e−2πinx −2πi(n−1)x −2πix 1 e . . . e 2 e . . . e

ˆ ˆ Extending the Fourier vector by zeros: The Fourier vector (f0,..., fN−1) corresponds to a function cos p˜ ∈ Tn for N = 2n + 1, andp ˜ ∈ Tn for N = 2n. Note that the same function is represented by the longer Fourier vector of length M > N where we insert zeros for the higher frequency terms: For odd N = 2n + 1 we obtain the vector ˆ ˆ ˆ ˆ ˆ (f0, f1,..., fn, 0,..., 0 , fn+1,..., f2n) | {z } M − N zeros For even N = 2n we obtain the vector ˆ ˆ ˆ 1 ˆ 1 ˆ ˆ ˆ (f0, f1,..., fn−1, 2 fn, 0,..., 0 , 2 fn, fn+1,..., f2n) | {z } M − N − 1 zeros

−1 This operation allows us to evaluate the functionp ˜(x) at M > N equidistant points using FM , yielding an interpolation on a ﬁner grid of the original data.

ˆ ˆ Truncating the Fourier vector: The Fourier vector (f0,..., fM−1) corresponds to a functionp ˜(x). Sometimes we want to truncate the Fourier vector by removing the higher frequency terms, so that we cos get a Fourier vector of length N < M corresponding to a functionq ˜ ∈ Tn for N = 2n + 1, andq ˜ ∈ Tn for N = 2n. For odd N = 2n + 1 we obtain the vector ˆ ˆ ˆ ˆ ˆ (f0, f1,..., fn, fM−n,..., fM−1) For even N = 2n we obtain the vector ˆ ˆ ˆ ˆ ˆ ˆ ˆ (f0, f1,..., fn−1, fn + fM−n, fM−n+1,..., fM−1) j Let xj = M , j = 0,...,M −1. Note that q˜(xj) j=0,...,M−1 is the best discrete least squares approximation cos of the given data fj j=0,...,M−1 using Tn for N = 2n + 1, or using Tn for N = 2n. For M N the functionq ˜(x) approximates the truncated Fourier series for the function f(x) as can be seen by comparing (1) and (6). The truncated Fourier series is the best continuous least squares cos approximation of f(x) using Tn for N = 2n + 1, or using Tn for N = 2n.

2.5 The cosine transform In some applications it is useful to consider even or odd data, which allow to use series with only cosine or sine functions, respectively. The cosine transform is useful in image processing. There are a continuous version and two discrete versions of the cosine transform.

4 Continuous cosine transform Here we consider a 2-periodic function f (i.e., f(x) = f(x + 2)) which R 1 2 is even (i.e., f(x) = f(−x)). We assume that 0 |f(x)| dx is finite. Using the results of section 2.2 we can then find a unique coefficients ak (k = 0, 1, 2,...) and bk (k = 1, 2,...) such that

∞ X f(x) = a0 + ak cos(πkx) + bk sin(πkx) k=1

As f(x) is even the uniqueness implies that all bk = 0. The resulting transform f 7→ (2a0, a1, a2,..., ) =: ˆ ˆ ˆ (f0, f1, f2,...) is called cosine transform and has the following properties:

1 Z ∞ ˆ 1 ˆ X ˆ fk = 2 f(x) cos(πkx)dx ⇐⇒ f(x) = 2 f0 + fk cos(πkx) Inversion (12) 0 k=1 1 Z ∞ 2 1 ˆ 2 X ˆ 2 2 |f(x)| dx = 2 f0 + fk Parseval (13) 0 k=1 1 Z ˆ q(x) = f(t)g(x − t)dt ⇐⇒ qˆk = fkgˆk Convolution (14) −1

R 1 R 1 The integrals on the left in (12), (13) could also be written as −1(...), but they simplify to 2 0 (...) since f(x) is even. In the sum on the left in (14) we cannot make this simpliﬁcations as f(t)g(x − t) is not an even function of t. Note that the cosine transform of real data is real.

Discrete even data: interval-boundary and interval-center case There are two possibilities for j samples of an even, 2-periodic function: Either the samples are taken at the points N , or at the midpoints j+1/2 N . The two situations are shown here for the case N = 4:

F4 F3 F2 F1 F0 F1 F2 F3 F4

−1 uu u u 0u u u u 1 u

f3 f2 f1 f0 f0 f1 f2 f3

−1 u u u u0 u u u u 1

Note that in both cases we obtain discrete sequences Fj, fj with period 8, i.e., Fj+8 = Fj, fj+8 = fj for all j ∈ Z. In the first case we have N + 1 = 5 different values F0,...,FN whereas in the second case we have only N = 4 different values f0, . . . , fN−1.

Discrete cosine transform, values at endpoints of intervals Here we consider a 2-periodic function u (i.e., u(x) = u(x + 2)) which is even (i.e., u(x) = u(−x)) and which we sample at the endpoints j j j+1 j N of the intervals [ N , N ] j = 0,...,N. We denote the resulting discrete values at N by Fj for j ∈ Z. Therefore Fj satisﬁes Fj = Fj+2N , F−j = Fj for j ∈ Z.

5 Using the results of section 2.3 we can ﬁnd a unique interpolating function

N−1 ˜ X P (x) = a0 + ak cos(πkx) + bk sin(πkx) + aN cos(πNx) k=1

But since the data are even in [−1, 1] the function P˜(x) must have the same property (using the unique- ˇ ˇ ness) and all bk = 0. The resulting transform (F0,...,FN ) 7→ (2a0, a1, . . . , aN ) =: (F0,..., FN ) is called β β−1 0 X 1 X 1 discrete cosine transform. With the notation Cj := 2 Cα + Cj + 2 Cβ (which is reminiscent j=α j=α+1 of the trapezoid rule) we have the following properties:

N N 2 X0 j X0 j Fˇ = F cos πk ⇐⇒ F = Fˇ cos πk Inversion (15) k N j N j k N j=0 k=0 N N 2 X0 2 X0 ˇ 2 |Fj| = Fk Parseval (16) N j=0 k=0 N N−1 1 X0 1 X Q = F G = F G ⇐⇒ Qˇ = Fˇ Gˇ Convolution (17) j N ` j−` N ` j−` k k k `=−N `=−N

In (17) Fj, Gj are understood as inﬁnite 2N periodic sequences. The sums on the left in (15), (16) could N N 1 X0 2 X0 also be written as , but they simplify to since f is even, i.e., f = f . In the sum on the N N j −j j `=−N j=0 left in (17) we cannot make this simpliﬁcations as f`Gj−` is not even with respect to `. Note that the discrete cosine transform of real data is real.

Discrete cosine transform, values at centers of intervals Here we consider a 2-periodic function j+1/2 u (i.e., u(x) = u(x + 2)) which is even (i.e., u(x) = u(−x)) and which we sample at the midpoints N j j+1 j+1/2 of the intervals [ N , N ] j = 0,...,N −1. We denote the resulting discrete values at N by fj for j ∈ Z. Therefore fj satisﬁes fj = fj+2N , f−j−1 = fj for j ∈ Z. Using the results of section 2.3 we can ﬁnd a unique interpolating function

N−1 ˜ X P (x) = a0 + ak cos(πkx) + bk sin(πkx) + bN sin(πNx) k=1 (as the data are in the midpoints we have to shift (11), giving sine instead of cosine for the highest frequency). But since the data are even in [−1, 1] the function P˜(x) must have the same property (using the uniqueness) and all bk = 0. The resulting transform (f0, . . . , fN−1) 7→ (2a0, a1, . . . , aN−1) =: ˆ ˆ (f0,..., fN−1) is called discrete cosine transform and has the following properties:

N−1 N−1 2 X j + 1/2 X j + 1/2 fˆ = f cos πk ⇐⇒ f = 1 fˆ + fˆ cos πk Inversion (18) k N j N j 2 0 k N j=0 k=1 N−1 N−1 2 X 2 1 ˆ 2 X ˆ 2 |fj| = f0 + fk Parseval (19) N 2 j=0 k=1

6 N−1 ( ˆ 1 X ˇ fkgˆk k = 0,...,N − 1 Qj = f`gj−` ⇐⇒ Qk = Convolution (20) N 0 k = N `=−N N−1 1 X q = f G ⇐⇒ qˆ = fˆ Gˇ , k = 0,...,N − 1 (21) j N ` j−` k k k `=−N

Note that there are two cases for the convolution: You can convolve fj with another even, interval-center sequence gj which gives an even, interval-boundary sequence Qj. You can also convolve fj with an even, interval-boundary sequence gj which gives an even, interval-center sequence qj. The cosine transforms ˇ ˇ Qk, Gk of the interval-boundary sequences Qj, Gj are defined by (15). In (20), (21) fj, gj, Gj are understood as infinite 2N-periodic sequences. 1 PN−1 2 PN−1 The sums on the left in (18), (19) could also be written as N `=−N , but they simplify to N j=0 since fj is “even”, i.e., f−j−1 = fj. In the sum on the left in (20) we cannot make this simplifications as f`gj−` is not “even” with respect to `. Note that the discrete cosine transform of real data is real.

Image Compression The two dimensional version of the discrete Fourier transform is frequently used in image processing. The “image” is a function deﬁned on [0, 1]2 which gives the brightness at each point. A discretized image consists of “pixels” obtained by dividing [0, 1]2 in N by N subsquares, with a brightness value given in the center of each subsquare. By using a symmetric extension of the image values to R2 one avoids jumps at the edges, yielding faster convergence of interpolating functions and truncated series.

Matlab: The function dct implements the discrete cosine transform for interval-centered data as de- p ˆ ﬁned here, but with a factor N/2 and a diﬀerent factor for f0: Assume the Matlab vector f contains the data values, i.e.,f = [f0, . . . , fN−1]. Then dct(f) gives the vector r N dct([f , . . . , f ]) = [2−1/2fˆ ,..., fˆ ] 0 N−1 2 0 N−1 3 Fast Fourier Transform ˆ ˆ The discrete Fourier transform is the mapping (f0, . . . , fN−1) 7→ (f0,..., fN−1) for the case of periodic, discrete data. A straight forward implementation of the formulas

N−1 N−1 X 1 X f = e2πijk/N fˆ , fˆ = e−2πijk/N f (22) j k k N j k=0 j=0 would involve O(N 2) operations. However, for special values of N it is possible to give a much faster algorithm which requires only O(N log N) operations. The most important case is where N is a power of 2. 1 Note that the only diﬀerence between the two equations in (22) is a minus sign and a factor of . N ˆ ˆ Hence it is suﬃcient to describe the algorithm for one direction, e.g., FN :(f0,..., fN−1) 7→ (f0, . . . , fN−1). 2πi/N Here FN corresponds to a matrix containing powers of wN := e . Assume that N is even. Then we can split up the sum in even and odd terms:

N−1 N/2−1 N/2−1 N/2−1 N/2−1 X jk ˆ X j(2k) ˆ X j(2k+1) ˆ X jk ˆ j X jk ˆ fj = wN fk = wN f2k + wN f2k+1 = wN/2f2k + wN wN/2f2k+1 k=0 k=0 k=0 k=0 k=0

7 N using w = w2 . Note that PN/2−1 wjk fˆ for j = 0, − 1 is exactly F fê where fê := N/2 N k=0 N/2 2k 2 N/2 ˆ ˆ ˆ > ˆ (f0, f2,..., fN−2) contains the even components of the vector f. For j = N/2,...,N − 1 we can jk j1k N/2 write j = N/2 + j1 and have wN/2 = wN/2 because of wN/2 = 1. The second sum can be similarly ô ˆ ˆ ˆ > N/2 expressed in terms of the odd components f := (f1, f3,..., fN−1) . Since wN = −1 we get

 1  ye + yo wN f = , where ye := F fˆe, yo := . F fˆo (23) ye − yo N/2  ..  N/2 N/2−1 wN

Therefore a transform FN of length N requires two transforms FN/2 of length N/2 and N additional operations (additions and multiplications). Let cN denote number of operations needed for performing FN , then (23) implies cN = 2cN/2 +N. Since F1f = f we have c1 = 0 and therefore cN = Nm = N log2 N for N = 2m.