Fourier Transform of Real Discrete Data Notes

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Fourier transform of real discrete data Notes

Today we will discuss how to apply Fourier transform to real data, which is always sampled at discrete times and is finite in duration. Note: Fourier analysis also used in image processing. Images have finite resolution. Instead of discrete time steps, one has discrete points in space (pixels). Real data is sampled in finite time steps ∆t: y(t) → y(m∆t), where m = 0, 1, 2,... N − 1 and N is the number of points. Only consider equally spaced time steps. Data is of finite duration: T = N∆t Notation:

Time steps tm ≡ m∆t

Function/Signal at discrete times: y(m∆t) = y(tm) ≡ ym

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT How to discretize the Fourier transform Notes

Since our data is ﬁnite, cannot evaluate the Fourier integral below

1 Z ∞ g(ω) = √ f (t)e−iωt dt (1) 2π −∞ Task: Compute a measure from ﬁnite data sets sampled on discrete time steps that resembles/approximates the Fourier transform.

In the following we will develop the discrete Fourier transform.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Developing the discrete Fourier transform Notes

Let’s go back to the complex Fourier series for periodic functions with period T . Consider the Fourier representation of y(t) on an interval from 0 to T . (Before, in lecture 3 we considered −T /2 to T /2) ∞ X inωt y(t) = cne , where n=−∞ Z T Z T 1 −i2πnt/T 1 −in∆ωt cn = y(t)e dt = y(t)e dt, T 0 T 0 where the discrete frequencies of the Fourier series representation are 2π n∆ω ≡ n T

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Developing the discrete Fourier transform Notes

Let’s approximate the integral for cn by a summation: R T PN−1 0 f (x)dx ≈ ∆x i=0 f (i∆x).

N−1 1 Z T ∆t X c = y(t)e−in∆ωt dt ≈ y(m∆t)e−in∆ωm∆t n T T 0 m=0 N−1 1 X ≈ y(m∆t)e−i2πnm/N N m=0 | {z } Y (n∆ω)

2π where we used ∆t = T /N and ∆ω = T . Note that ∆t cancels out in the exponential. The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The discrete Fourier transform Notes

We deﬁne the discrete Fourier transform (DFT) as:

N−1 X Y (n∆ω) = y(m∆t)e−i2πnm/N , m=0 This can be written more concisely as

N−1 X −i2πnm/N Yn = yme , (2) m=0

since y(m∆t) = y(tm) = ym.

Still need the inverse discrete transform. Want it to be exact, such that we recover the yn’s from the Ym’s. Next we will ﬁnd the inverse transform by using orthogonality.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The inverse discrete Fourier transform Notes

Consider the sum N−1 X ikα SN = e k=0

If α = 0 then SN = N, since every term is 1. The sum above is just a geometric series:

N−1 X 1 − r N r k = r 0 + r 1 + ... + r N−1 = 1 − r k=0 Therefore, N−1 X 1 − eiαN S = eikα = N 1 − eiα k=0

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The inverse discrete Fourier transform Notes

Want SN to be zero for α 6= 0. This means

eiαN = 1

so α = 2πl/N, where l is integer Therefore, N−1 X N, l = 0 S = ei2πkl/N = N 0, l 6= 0 k=0

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The inverse discrete Fourier transform Notes

Now, let’s express integer l as diﬀerence between two integers m and n (l = m − n) to obtain orthogonality relation:

N−1 X i2πkm/N −i2πkn/N e e = Nδm,n (3) k=0 Let’s return to the discrete Fourier transform (eqn. (2))

N−1 X −i2πnm/N Yn = yme , m=0

and multiply both sides with ei2πkn/N and sum over n. The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The inverse discrete Fourier transform Notes

N−1 N−1 N−1 X i2πkn/N X X −i2πnm/N i2πkn/N Yne = yme e n=0 n=0 m=0 N−1 N−1 X X −i2πnm/N i2πkn/N = ym e e m=0 n=0 N−1 X = ymNδk,m m=0 = Nyk ,

where we used the orthogonality of the sums (eqn. (3))

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The inverse discrete Fourier transform Notes

Therefore, we can identify the inverse discrete Fourier transform

N−1 X −i2πnm/N Yn = yme , DFT (4) m=0 N−1 1 X y = Y ei2πnm/N , inverse DFT (5) m N n n=0 Similar to Fourier transform, prefactors and sign convention in the exponential can vary in diﬀerent books and code libraries. The product of the prefactors always has to be 1/N.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Comments on the DFT Notes

Discrete Fourier transform can be thought of as a truncated complex Fourier series. This means higher frequency components are missing. This is important when dealing with real signals (more later) → How well does the DFT approximate the Fourier transform ? Even though it arose from an approximation of the Fourier series, mathematically it is well deﬁned (orthogonality of the sums). Let’s look at the computational implementation

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Fast Fourier Transform Notes

To transform N data points, need to compute N summations over order N points. Therefore, computation time goes as N2. For higher dimensions d, it goes as N2d . The fast Fourier transform (Cooley and Tukey 1965), can 2 reduce the computational eﬀort dramatically: N → Nlog2N. e.g. for N = 216 = 65536 need ≈ 106 instead of ≈ 4 · 109 evaluations. The FFT is the most important and widely used algorithm! The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The Fast Fourier Transform Notes

Algorithm is based on regrouping terms in the summation. Let’s assume, that N is a power of 2. Now consider the discrete transform and decompose into even and odd terms:

N−1 X g(n∆ω) = f (m∆t)e−i2πmn/N m=0 N−2 N−1 X X = f (m∆t)e−i2πmn/N + f (m∆t)e−i2πmn/N m=0,even m=1,odd N/2−1 X = f (2j∆t)e−i2π(2j)n/N j=0 N/2−1 X + f ((2j + 1)∆t)e−i2π(2j+1)n/N j=0

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The Fast Fourier Transform Notes

The two sums can be rewritten as two DFT’s with N/2 points that run over even and odd numbered points, respectively.

N/2−1 X g(n∆ω) = f (2j∆t)e−i2πjn/(N/2) j=0 N/2−1 X +e−i2πn/N f ((2j + 1)∆t)e−i2πjn/(N/2) j=0 −i2πn/N = geven(n∆ω) +e godd (n∆ω) | {z } | {z } N/2 points N/2 points

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT The Fast Fourier Transform Notes

Instead of N2 operations, we now have 2(N/2)2 operations. Continue decomposition into even and odd points. If N = 2k , then after k steps, there will be N transforms each containing just one point. Therefore, need N · k = N log2 N operations. FFT The FFT requires N = 2k and the computing time goes as N log2 N Actual implementation a bit more tricky (see Giordano, Nakanishi Appendix C3).

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Properties of DFT - Periodicity Notes

The DFT is periodic in time and frequency space. Shift indices by N:

N−1 N−1 X −i2π(n+N)m/N X −i2πnm/N −i2πm Yn+N = yme = yme e = Yn | {z } m=0 m=0 =1 N−1 N−1 1 X i2π(m+N)n/N 1 X i2πnm/N i2πn ym+N = Yne = Yne e = ym N N | {z } n=0 n=0 =1

If one continues summation beyond (N − 1), then Yn and ym repeat. The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Properties of DFT - Parseval’s theorem Notes

Using the orthogonality of the sums (eqn. (3)) one can show

N−1 N−1 X 1 X x y ∗ = X Y ∗ (6) m m N n n m=0 n=0 analogous to what we had for Fourier transform. If x = y, then we recover Parseval’s theorem:

N−1 N−1 X 1 X |x |2 = |X |2 (7) m N n m=0 n=0

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Properties of DFT - Convolution Notes

Using the orthogonality of the sums (eqn. (3)) one can show the Convolution theorem for the DFT:

N−1 ∞ ! N−1 X X 1 X (x ⊗y) = x y = X Y ei2πkn/N (8) n l n−l−pN N k k l=0 p=−∞ k=0 | {z } | {z } periodic extension of yn−l inverse DFT of X ·Y

where p is an integer. Note that Convolution needs N2 operations. By performing it in frequency domain using FFT it is much faster! However, need to be careful since convolution “wraps around” due to periodicity.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Properties of DFT - Correlation Notes

Using the orthogonality of the sums (eqn. (3)) one can show the Correlation theorem for the DFT (Wiener-Khinchin):

N−1 ∞ ! N−1 X X 1 X (x y) = x∗ y = X ∗Y ei2πkn/N n l n+l−pN N k k l=0 p=−∞ k=0

| {z } | {z ∗ } periodic extension of yn+l inverse DFT of X ·Y (9) Again, get performance increase by doing correlation in frequency space using FFT, especially for higher dimensions.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Properties of DFT - Shifting Notes

Analogous to the Fourier transform, shifting the time axis or frequency axis by l gives rise to an additional phase factor:

−i2πkl/N DFT [ym−l ]k = Yk e i2πml/N DFT [yme ]k = Yk−l The discrete Fourier Transform (DFT) The fast Fourier transform - FFT What about the frequency cutoﬀ? Notes

Since we truncated the complex Fourier series to arrive at the DFT, the DFT has a frequency limit!

n2π n2π For N data points, the discrete frequencies are ωn = T = N∆t or n fn = N∆t , where n = 0 ... (N − 1).

N−1 However, fN−1 = N∆t is NOT the highest frequency that the DFT can resolve!! To illustrate that consider the symmetry of the DFT.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT DFT for a real data points Notes

The DFT is given by

N−1 X −i2πnm/N Yn = yme m=0

∗ Let’s assume that the ym’s are real (ym = ym). Then we can write

N−1 N−1 X −i2πm(N/2−n)/N X −iπm i2πmn/N YN/2−n = yme = yme e m=0 m=0 N−1 N−1 ∗ X i2πm(N/2+n)/N X iπm i2πmn/N YN/2+n = yme = yme e m=0 m=0

−iπm iπm ∗ but e = e = cos(πm). Therefore, YN/2−n = YN/2+n.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Nyquist frequency Notes

For a real signal, half of the points are redundant because of the N 1 1 symmetry around fN/2 = 2 N∆t = 2∆t . It’s enough to compute Yn from n = 0 ... (N/2 − 1) for even N. The highest independent frequency is therefore: Nyquist frequency N/2 − 1 1 f ≈ Nyquist N∆t 2∆t This is called the Nyquist frequency.

The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Nyquist frequency and sampling theorem Notes

The frequencies f ≥ fNyquist are actually the negative frequencies! (Remember that a simple sin/cos function was represented by a one positive and one negative frequency component in the Fourier transform/complex Fourier series. It is the same for the DFT).

Shannon-Nyquist sampling theorem

A function f (t) that contains frequencies less than fmax is completely determined if it is sampled at points that are ∆t = 1 apart. 2fmax The discrete Fourier Transform (DFT) The fast Fourier transform - FFT Nyquist frequency and sampling theorem Notes

Example: Human ear can hear up to 20, 000 Hz. In order to capture the whole audio spectrum, need a sampling rate that is at least twice as large: 2 · 20, 000 = 40, 000 Hz. CD’s use 44, 100 Hz, for example.

Notes