The Discrete Fourier Transform

The Discrete Fourier Transform:

Hayden Borg The Discrete Fourier Transform: From Hilbert Spaces to the FFT

Hayden Borg

University of Puget Sound

May 2021 Hilbert Spaces and Hand-waving

The Discrete Fourier Hilbert spaces generalize Euclidean spaces. Transform: What ”defines” a Euclidean space: Hayden Borg Vector space with the dot product Calculus We can generalize the dot product as the inner product Inner Product Define the inner product to be a bilinear functional acting on two elements of a vector space h~x, ~yi which is: Conjugate symmetric: h~x, ~yi = h~y, ~xi Linear in its first argument: hax~1 + bx~2, ~yi = ahx~1, ~yi + bhx~2, ~yi Positive definite: h~x, ~xi > 0 for all ~x 6= ~0 and h~x, ~xi = 0 if and only if ~x = ~0 Hilbert Space and Hand-waving: Inner Product Spaces

The Discrete Fourier Transform: A vector space V equipped with an inner product is an Inner Hayden Borg Product space. Norm We deﬁne the norm of a vector in an Inner Product space to be ||~v||2 = h~v, ~vi.

We have some familiar looking results: Theorem: Parallelogram Law 2 2 2 2 For v~1, v~2 ∈ V , ||~x + ~y|| + ||~x − ~y|| = 2||~x|| + 2||~y||

Theorem: Pythagoras For ~x, ~y ∈ V such that h~x, ~yi = 0, ||~x + ~y||2 = ||~x||2 + ||~y||2 Hilbert Spaces and Hand-waving: Metric Spaces

The Discrete Fourier A set M with metric µ is a Metric space if for all Transform: m1.m2, m3 ∈ M: Hayden Borg Identity of Indiscernibles: µ(s1, s2) = 0 if and only if s1 = s2

Symmetric: µ(s1, s2) = µ(s2, s1)

Triangle Inequality: µ(s1, s3) ≤ µ(s1, s2) + µ(s2, s3)

µ(s1, s2) is the ”distance” between s1 and s2. Completeness

A Cauchy sequence in Metric space M is a sequence {mi } for i ≥ 1 such that for every > 0 there exists an N such that µ(ml , mk ) < for l, k > N. A Metric space M is complete if every Cauchy sequence in M converges to a point in M Hilbert Spaces and Hand-waving

The Discrete Fourier Transform: Hayden Borg Theorem

Given an Inner Product space and v~1, v~2 ∈ V , the identity of indiscernibles, the triangle inequality, and symmetry hold for µ deﬁned: µ(v~1, v~2) = ||v~2 − v~1||.

An Inner Product space which is also a complete Metric space is called a Hilbert space.

We now have: Vector space with inner product Calculus Hilbert Spaces and Hand-waving: Central Result

The Discrete Fourier Transform:

Hayden Borg

Let {y~i } be an orthonormal set in Hilbert space H and ~v ∈ H P such that ~v = ak y~k . Then, ak = h~v, y~k i k Proof P P h~v, y~j i = h ak y~k , y~j i = ak hy~k , y~j i = ak k k Hilbert Spaces and Hand-waving

The Discrete Fourier Transform: 2 Hayden Borg Example: R  √1  2 " √1 # B = { √1  , 2 } is an orthonormal basis of 2.  2  − √1 R 0 2

" 1 # " 1 # " 1 # " 1 # 4 4 √ √ 4 √ √ = h , 2 i 2 + h , 2 i 2 8 8 √1 √1 8 − √1 − √1 2 2 2 2 " 1 # " 1 # 12 √ 4 √ 6 − 2 = √ 2 − √ 2 = √1 − √1 6 + 2 2 2 2 2 Hilbert Spaces and Hand-waving: L2([0, T ])

The Discrete The set of all complex valued functions with real input Fourier Transform: T R 2 Hayden Borg integrable on the interval [0, T ] such that |f (x)| dx < ∞. 0 Inner Product

T hf (x), g(x)i = R f (x)g(x)dx 0

1 i 2π kx Let’s examine { √ e T |k ∈ }. T Z First, T T Z Z 2 1 i 2π kx 2 1 |√ e T | dx = √ dx T T 0 0 = 1 < ∞ Hilbert Spaces and Hand-waving: L2([0, T ])

The Discrete Fourier Transform: Next, Hayden Borg Z 1 i 2π nx 1 i 2π mx 1 T i 2π nx i 2π mx h√ e T , √ e T i = 0 e T e T dx T T T Z 1 T i 2π (n−m)x = 0 e T dx T ( 1 R T T 0 1dx for n = m = 1 2π(n−m) 0 i2π(n−m) (e − e ) for n 6= m ( 1 for n = m = 0 for n 6= m

1 i 2π kx So, { √ e T |k ∈ } is orthonormal. T Z L2([0, T ]) and Complex Fourier Series

The Discrete Fourier Transform: 2 Hayden Borg We can rewrite any f ∈ L ([0, T ]) using our central result:

P i 2π kx f (x) = ak e T k∈Z Where:

i 2π kx ak = hf , e T i T Z −i 2π kx = f (x)e T dx 0 This is the complex Fourier series. Complex Fourier Series: Example

( 1 The Discrete 1 for 0 ≤ x < Fourier Square wave period 1: f (x) = 2 Transform: 1 −1 for 2 ≤ x < 1 Hayden Borg First we ﬁnd the ak : Z 1 −i2πkx ak = f (x)e dx 0 1 Z 2 Z 1 = e−i2πkx dx − e−i2πkx dx 1 0 2 −1 −1 = (e−iπk − 1) − (1 − e−iπk ) i2πk i2πk −1 = (e−ikπ − 1) inπ ( 0 if k is even = 2i − nπ if k is odd Fourier Series: Applications

The Discrete Fourier Transform:

Hayden Borg Wide variety of applications: Solving PDEs Probability theory and statistics NMR, IR, etc. spectroscopy X-ray crystallography MRI Image and signal processes Audio engineering Fourier Transform

The Discrete Fourier Transform: Generalization to aperiodic functions Hayden Borg 2 Need to consider: L (R)

Deﬁnition: Fourier Transform

∞ fˆ(ω) = R f (x)ei2πωx dx −∞ And ∞ f (x) = R fˆ(ω)ei2πωx dω −∞

i2πωx Orthonormal Basis: {e |ω ∈ R} Discrete Fourier Transform

The Discrete Fourier Transform:

Hayden Borg Digital signal and discretized data are common. Consider a function f sampled uniformly N times over an interval [0, T ]. T T T That’s at: 0, N , 2 N ,..., (N − 1) N “Package” in a vector:   f0  f1  f → ~v =    .   .  fN−1 Discrete Fourier Transform

The Discrete Fourier Transform:

Hayden Borg Sample basis vectors at the same points. Roots of Unity −i 2π The primitive Nth root of unity is ωN = e N

 0·k  ωN 1·k  ωN  i 2π kx  2·k  T  ωN  e → e~k =    .   .  (N−1)·k ωN

Note: We only need e~0, e~1,... e~N−1 Discrete Fourier Transform

The Discrete Fourier Transform: We’ll call ~vˆ the transform of ~v. Hayden Borg ˆ [~v]k = ak

Central result

Let {y~i } be an orthonormal set in Hilbert space H and ~v ∈ H P such that ~v = ak y~k . Then, ak = h~v, y~k i k

‘Do’ the Fourier series on our sampled function: ˆ ∗ [~v]k = h~v, e~k i = e~k ~v

Then, ˆ ∗ ~v = [e~0|e~1| ... |e~N−1] ~v The Discrete Fourier Transform

The Discrete Fourier Transform:

Hayden Borg So we deﬁne the DFT to be the linear transformation N N T : C → C deﬁned by the matrix vector product:

 0·0 0·1 0·(N−1)  ωN ωN . . . ωN  1·0 1·1 1·(N−1)  ωN ωN . . . ωN  T (~v) =  . . . .  ~v  . . .. .   . . .  N·0 N·1 N·(N−1) ωN ωN . . . ωN Call this matrix F. Theorem The matrix U = √1 F is unitary. N The Discrete Fourier Transform

The Discrete Fourier Transform: The DFT is primarily used to go from the ‘time’ to the Hayden Borg ‘frequency’ domain Spectral analysis Spectroscopy Filtering MRI Spatial information Artifacts Audio recording and engineering Can also be used in data compression JPEG mp3 Cooley-Tukey Algorithm

The Discrete Fourier Transform: Determining F and calculating the matrix vector product is Hayden Borg O(n2). We can exploit some symmetries to make this more eﬃcient. Danielson-Lanczos Lemma N−1 m ˆ P The DFT for N = 2 for some m ∈ N,[~v]i = [~v]k [FN ]ik , k=0 may be rewritten ˆ N [~v]i = [F N ~veven]i + [D N F N ~vodd]i for 0 ≤ i ≤ − 1 2 2 2 2 ˆ N [~v]i = [F N ~veven]i − [D N F N ~vodd]i for < i < N − 1 2 2 2 2 Cooley-Tukey Algorithm

The Discrete Fourier Transform: Hayden Borg Theorem k Given an N−point DTF, FN , where N = 2 where k ∈ N. Then, " #" # I N D N F N even-odd FN = 2 2 2 . I N −D N F N permutation 2 2 2 Then the FFT may be calculated

Calculate the ωN (O(N))

Recursively apply this decomposition log2 N times

This recursion gives log2 N operations for each ‘slot’ and there are N slots so we have O(N log N) Conclusions

The Discrete Fourier Transform:

Hayden Borg 1 Euclidean spaces can be generalized to Hilbert spaces 2 Square-integrable functions are vectors in the Hilbert 2 space L (R) and can be expressed as a linear combination of basis vectors 3 The Fourier series and Fourier Transform are vector decomposition with the special basis {ei2πωx } 4 The DFT can ‘do’ the Fourier Transform on discrete data and can be represented as a matrix vector product 5 The DFT can be more eﬃciently calculated using the Cooley-Tukey Algorithm References

The Discrete Halmos, P. Introduction to Hilbert Space and the Theory of Fourier Transform: Spectral Multiplicity, Masﬁeld Centre, CT: Martino Publishing, Hayden Borg 2013. Chp. 1. Choudhary, B. The Elements of Complex Analysis: Second Edition, New Delhi: New Age International Publishers, 2003. Chp. 2. Melrose, R. Functional Analysis: Lecture notes for 18.102. MIT, 2013. Chp. 3 Hern´andez,E. and Weiss, G. A First course on Wavelets, Boca Raton, FL: CRC Press, 1996. Chps. Intro. and 1 “Complex Fourier Series” May 18, 2020. https://eng.libretexts.org/@go/page/1614. Weisstein, Eric W. “Discrete Fourier Transform” From MathWorld–A Wolfram Web Resource. https://mathworld.wolfram.com/DiscreteFourierTransform.html Weisstein, Eric W. “Fourier Matrix.” From MathWorld–A Wolfram Web Resource. https://mathworld.wolfram.com/FourierMatrix.html