Math 994-002: Applied and Computational Harmonic Analysis, MSU, Spring 2020 Lecture 01: Fourier, heat diffusion, and shift-invariant operators January 14, 2020 Lecturer: Matthew Hirn 1 Sparse Representations Chapter 1 of A Wavelet Tour of Signal Processing [1]. Exercise 1. Read Chapter 1 (Sparse Representations) of A Wavelet Tour of Signal Process- ing. It gives a nice overview of the book and will give you a good perspective on computa- tional harmonic analysis heading into the course. Exercise 2. Read the appendices in A Wavelet Tour of Signal Processing, as we will not cover these in class. We will immediately need some of the material contained in them. Remark 1.1. The integral we use in this course will be the Lebesgue integral, which is usually taught in a first year graduate course in real analysis. However, if these are unfamiliar to you, you may replace most if not all of the results with Riemann integrals from Calculus and assume that the generic functions f, g, h, etc. are Schwartz class functions. For more details on the Schwartz class and Fourier integrals, see [2]. 2 The Fourier Kingdom Chapter 2 of A Wavelet Tour of Signal Processing [1]. 2.1 Linear time-invariant filtering Section 2.1 of A Wavelet Tour of Signal Processing [1]. Fourier analysis originates with the work of Joseph Fourier, who was studying the heat equation: @tF = ∆F F (u; 0) = f(u) Where F : Rd ×[0; 1) ! R and f : Rd ! R. This is a linear partial differential equation. In order to solve it, it helps to think about linear algebra. Suppose A is an n × n real valued, symmetric matrix, which maps vectors x 2 Rn to other vectors Ax 2 Rn. Then from the 1 spectral theorem, we know that A has a complete set of orthonormal eigenvectors, v1; : : : ; vn, such that Avk = λkvk n for some λk 2 R. Since fvkgk≤n forms an ONB, it allows us to write, for any x 2 R , n X x = hx; vkivk ; k=1 which in turn makes evaluating Ax very easy: n n X X Ax = hx; vkiAvk = λkhx; vkivk : k=1 k=1 Let us now try to apply the same ideas to the Laplacian, ∆. We may ask, what are the eigenfunctions of the Laplacian? If we consider complex valued functions, one can verify that ∆ei!·u = −|!j2ei!·u d i!·u for any ! 2 R . Thus the function e!(u) = e is an eigenfunction of ∆ for any !. Let us formally define Z −i!·u fb(!) = hf; e!i = f(u)e du : Rd This will be what we call the Fourier transform, but right now we see it as an analogue of basis coefficients in an ONB. Following the analogy, we may then be tempted to write: Z Z i!·u f(u) = hf; e!ie!(u) d! = fb(!)e d! : Rd Rd We this in hand, we then propose Z 2 F (u; t) = e−|!j tfb(!)ei!·u d! ; Rd as the solution to the heat equation. One can verify that, formally, F indeed is the so- lution. Fourier analysis was then born by trying to understanding when all of this makes mathematical sense. The reason Fourier analysis is used so often in signal processing, is that it turns out this analysis is not useful for just the Laplacian operator. In fact the Laplacian is just an example of a more general class operators, called shift invariant operators. Let us now work over R instead of Rd; we will use t to denote a value in R, since it is often useful to think of it as time. Let fτ (t) = f(t − τ) be the translation of f by τ; if t is time, then this is a time delay by τ. An operator L is shift invariant if it commutes with the time delay of any function, (Lfτ )(t) = (Lf)(t − τ) 2 As we shall see all linear, continuous shift invariant operators L are diagonalized by the i!t complex exponentials e!(t) = e . To see this, recall the convolution of two functions f; g: Z f ∗ g(t) = f(u)g(t − u) du R Now let δ(t) be a Dirac (centered at zero), and δu(t) = δ(t − u) be a Dirac centered at u. By definition this means f ∗ δ(t) = f(t). We have: Z Z f(t) = f ∗ δ(t) = f(u)δ(t − u) du = f(u)δu(t) du R R Since L is continuous and linear, Z Lf(t) = f(u)Lδu(t) du R Let h be the impulse response of L, defined as h(t) = Lδ(t) Since L is shift invariant, we have Lδu(t) = h(t − u) and therefore Z Lf(t) = f(u)h(t − u) du = f ∗ h(t) = h ∗ f(t) R Thus every continuous, linear shift invariant operator is equivalent to a convolution with an impulse response h. We can now use this fact to show our original goal, which was that the complex expo- i!t nential functions e!(t) = e diagonalize L. This will in turn motivate the study of Fourier integrals. We have: Z Z i!(t−u) it! −i!u Le!(t) = h(u)e du = e h(u)e du = bh(!)e!(t): R R | {z } bh(!) Thus e!(t) is an eigenfunction of L with eigenvalue bh(!), if bh(!) exists. The value bh(!) i!t is the Fourier transform of h at the frequency !. Since the functions e!(t) = e are the eigenfunctions of shift invariant operators, we would like to decompose any function f as a sum or integral of these functions. This will then allow us to write Lf directly in terms of the eigenvalues of L (as you do in linear algebra when you are able to diagonalize a matrix/operator on a finite dimensional vector space). We’ll try to undestand when this is possible. Exercise 3. Read Section 2.1 of A Wavelet Tour of Signal Processing. 3 Math 994-002: Applied and Computational Harmonic Analysis, MSU, Spring 2020 Lecture 02: The Fourier transform on L1(R) January 14, 2020 Lecturer: Matthew Hirn 2.2 Fourier integrals Section 2.2 of A Wavelet Tour of Signal Processing [1]. The Fourier transform is an operator F that maps a function f(u) to another function fb(!), which is defined as: Z F(f)(!) = fb(!) = f(t)e−i!t dt (1) R We will start by trying to understand what restrictions we need to place on f in order for this to make sense. In particular, if f is in some well defined space of functions, we will ask, does that imply fb is in some other well defined space of functions? We will start by considering the Lp spaces of functions. To that end, define: Z p p L (R) = f : R ! C : jf(t)j dt < +1 ; 0 < p < 1 R The space Lp(R) is a Banach space with norm: 1 Z p p kfkp = jf(t)j dt R The space L2(R) is special, as it is in fact a Hilbert space with inner product Z hf; gi = f(t)g∗(t) dt R where we use g∗(t) to denote the complex conjugate of g(t). We also define L1(R). Set: kfk1 = ess sup jf(t)j t2R The value kfk1 is the smallest number M, 0 ≤ M ≤ +1, such that jf(t)j ≤ M for almost every t 2 R; if f is continuous, it is the smallest number M such that jf(t)j ≤ M for all t 2 R. It thus measures whether f is bounded or not. The space L1(R) is the space of bounded functions: 1 L (R) = ff : kfk1 < +1g We then have: 1 Proposition 2.1. If f 2 L1(R), then fb 2 L1(R). Proof. Suppose f 2 L1(R). We have: Z Z Z −i!t −i!t jfb(!)j = f(t)e dt ≤ jf(t)e j dt = jf(t)j dt = kfk1 < +1 R R R Proposition 2.1 shows that F : L1(R) ! L1(R) is a well defined map using the definition (1). Later on we will extend the Fourier transform to other Lp spaces for 1 ≤ p ≤ 2, with particular interest in L2(R). For now recall from Section 2.1 that we would like to write f(t) in terms of fb(!). This requires a Fourier inversion formula. However, the above proposition only guarantees that fb 2 L1(R), which will not help with convergence issues. We thus assume that fb 2 L1(R) as well. Theorem 2.2 (Fourier inversion). If f 2 L1(R) and fb 2 L1(R) then Z 1 i!t f(t) = fb(!)e d!; for almost every t 2 R (2) 2π R To prove this theorem, we will need three standard results from graduate real analysis. We state them here, without proof. p Theorem 2.3. Suppose ffngn2N converges to f in L , meaning that lim kfn − fkp = 0 n!1 Then there exists a subsequency ffnk gk2N that converges to f almost everywhere, lim fn (t) = f(t) for almost every t k!1 k Theorem 2.4 (Dominated Convergence Theorem). Let ffngn2N be a sequence of functions such that limn!1 fn = f. If Z 8 n 2 N; jfn(t)j ≤ g(t) and g(t) dt < +1 R then f 2 L1(R) and Z Z lim fn(t) dt = f(t) dt n!1 R R Theorem 2.5 (Fubini’s Theorem).