Mathematics of Digitalization: Case Study Massimo Fornasier Chair of Applied Numerical Analysis Email: [email protected] ——————– First lecture Munich, Oct. 17 2017 – Fallstudien der Modellbildung 1 INTRODUCTION Main topics: 1. After a short introduction on the Hilbert spaces, we describe the concepts of Fourier series, Fourier transforms and their discrete implementation, in particular the algorithm of the Fast Fourier Transform (FFT). 2. To connect the continuous Fourier transforms and the discrete one, we need to address the Shannon sampling theory (analog-to-digital conversion). For that it will be fundamental to understand the Poisson summation formula which will allow us to estimate also the errors in the analog-to-digital conversion, the so-called aliasing errors. 3. We shall introduce systems for robust analog-to-digital INTRODUCTION 2 conversion called frames in Hilbert spaces. We address special cases of frames, in particular Gabor frames in their continuous and discrete versions and the implementation of their so-called canonical duals. Applications to audio signals and in image compression will be showed. 4. We address in more detail the study case of the fresco restoration and how to use the tools we considered so far to solve it. INTRODUCTION 3 THE IMPORTANCE OF HARMONIC ANALYSIS We can synthesize a variety of complicated functions by • means of pure sinusoids in the same way we can produce a chord of C major pushing the keyboard (C, E, G) of a piano. It was Joseph Fourier (http://www-gap.dcs.st- • and.ac.uk/ history/Mathematicians/Fourier.html) who ∼ first developed the modern methods of trigonometric series and integrals in the study of the heat conduction in the solids (Analytical Theory of Heat, 1815). THE IMPORTANCE OF HARMONIC ANALYSIS 4 SOME OF THE MILESTONES The most cited paper in mathematics • J. W. Cooley and J. W. Tukey, An algorithm for the machine computation of complex Fourier series, Math. Comp., 19 (1965), 297-301. Almost 3/4 of the Nobel prizes in Physics are given on • work using methods and tools from Fourier analysis. F. N. Magill, ed., The Nobel Prize Winners — Physics, Vol 1-3, Salem Press. Englewood Cliffs, NJ. 1989. Francis Crick, James Watson and Maurice Wilkins got the • Nobel prize for the medicine in 1962 for the discovery of the molecular structure of the DNA. This was the first example of the use of Fourier methods on data coming from X ray diffraction. SOME OF THE MILESTONES 5 Herbert Hauptmann (a mathematician) and Jerome • Karle shared in 1985 the Nobel prize for chemistry after having shown how to use systematically Fourier analysis in order to determine the structure of large molecules from X ray diffraction data. W. A. Hendrickson, the 1985 Nobel Prize in Chemistery, Science 231 (1986), 362-364. The starting of the time-frequency analysis and the • wavelets (1960-1990). A large number of technological products (CD-DVD, high • definition TV, digital phones, medical imaging instruments ...) are based on discrete Fourier analysis. SOME OF THE MILESTONES 6 HILBERT SPACES Let be a vector space. A scalar product u,v is a map H h i from to C such that: H×H (i) au + bv,z = a u,z + b v,z for all u,v,z and a,b C. h i h i h i ∈H ∈ (ii) u,v = v,u for all u,v . h i h i ∈H (iii) u,u R, u,u 0 for all u and u,u =0 if u =0. h i∈ h i≥ ∈H h i 6 6 HILBERT SPACES 7 We recall that a scalar product verify the Cauchy-Schwarz inequality (which will be proved in the exercises): u,v u,u 1/2 v,v 1/2 u,v . (1) |h i|≤h i h i ∀ ∈H and that u = u,u 1/2 define a norm for . k k h i H A Hilbert spaces is a vector space endowed with a scalar H product u,v and is complete with respect to the norm h i u,u 1/2. h i HILBERT SPACES 8 Example 0.1 Some relevant examples: Let Ω Rn. The vector space of functions • ⊂ L2(Ω) = f :Ω C f 2dx < is a Hilbert spaces with { → | Ω | | ∞} the scalar product: R u,v = u(x)v(x)dx. h i ZΩ Let Ω Zn. The vector space • d ⊂ 2 C 2 ℓ (Ωd)= f :Ωd k Ωd f(k) < is a Hilbert { → | ∈ | | ∞} spaces with the scalar product: P u,v = u(k)v(k). h i k Ωd X∈ In particular if Ω = d< then ℓ2(Ω )= Cd. | d| ∞ d HILBERT SPACES 9 Definition 0.2 Let E be a topological vector space. We say that E′ is the topological dual space of E, i.e., E′={ϕ from E to C ϕ is continuous and linear}. Given two Hilbert spaces | , , the set of linear operators T : , i.e., H K H→K T (au + bv)= aT (v)+ bT (v), which are also continuous is a vector space which, endowed with the norm Tu K k k is a complete normed space, T supu ,u=0 u H k| |kH→K ≡ ∈H 6 k k i.e., a Banach space, and we indicate it with ( , ). In L H K particular, if = the we set ( ) ( , ) and if = C H K L H ≡L H H K then we have ( , )= ′. L H K H HILBERT SPACES 10 Theorem 0.3 (Representation theorem of Riesz-Frechet) Let H be a Hilbert space. For every ϕ ′ there exists a unique ∈H f such that ∈H ϕ,v ϕ(v)= v,f v . h i≡ h i ∀ ∈H Moreover we have f = ϕ ′ sup ϕ,v . k k k kH ≡ v =1 |h i| k k HILBERT SPACES 11 Proposition 0.4 (Bessel inequality) Let uα α A be an { } ∈ orthonormal system in , i.e., uα,uβ = δα,β where δ , is the H h i · · Kronecker symbol. Then for x : ∈H x,u 2 x 2. |h αi| ≤k k α A X∈ In particular {α x,u =0} is countable. |h αi 6 HILBERT SPACES 12 Theorem 0.5 (Fourier) Let uα α A be a countable { } ∈ orthonormal system. Then the following are equivalent: (i) x = α A x,uα uα for all x . ∈ h i ∈H P 2 2 (ii) (Parseval idendity) x = α A x,uα for all x . k k ∈ |h i| ∈H (iii) (Completeness) If x andP if x,u =0 for all α, then ∈H h αi x =0. 2 The sequence x,uα is such that α A x,uα < , then h i ∈ |h i| ∞ the series α A x,uα uα is convergent in and converges to ∈ h i P H x. A orthonormal set which verifies one of the previous P conditions is called an orthonormal basis for the Hilbert space . H HILBERT SPACES 13 Theorem 0.6 Every Hilbert space has an orthonormal basis. Definition 0.7 A Hilbert space is separable if an only if it has H a countable orthonormal basis. HILBERT SPACES 14 FOURIER SERIES AND TRANSFORM A trigonometric series (or Fourier series) in complex form of period τ > 0 is a function series of the type: 1 T (x)= c e2πinx/τ (2) √τ n n Z X∈ Set a0 =2c0, an = cn + c n, bn = i(cn c n) (3) − − − we have, by the Euler formulas (eiw = cos(w)+ i sin(w)), that 1 a0 ∞ T (x)= + (ak cos(2πkx/τ)+ bk sin(2πkx/τ)) (4) √τ 2 ! kX=1 FOURIER SERIES AND TRANSFORM 15 1 2πinx/τ We observe now that e n Z e { √τ } ∈ τ τ cos(2πkx/τ) k N sin(2πkx/τ) k N,k=0 are an { 2 } ∈ { 2 } ∈ 6 orthonormal system in the Hilbert space = L2(0,τ). For p S p H example 1 1 1 τ e2πinx/τ , e2πimx/τ = e2πinx/τ e2πimx/τ dx h√τ √τ i τ 0 Z τ 1 2πi(n m)x/τ = e − dx τ Z0 1 2πi(n m)x = e − dx = δm,n. Z0 FOURIER SERIES AND TRANSFORM 16 We wonder whether they consitute systems of orthonormal bases. We consider the function πx x2 f(x)= . 2 − 4 Note that it is symmetric and therefore the coefficients b (f)= f, τ sin(2πkx/τ) vanish for all k. By computing k h 2 i (Exercise! Integrate twice by parts) instead the coefficients p a (f)= f, τ cos(2πkx/τ) we obtain the trigonometric series k h 2 i associated to f given by p π2 ∞ cos(kx) T (x)= . (5) f 6 − k2 kX=1 FOURIER SERIES AND TRANSFORM 17 Theorem 0.8 (Pointwise convergence) Let f : R C periodic → with period τ > 0 and locally integrable. Assume that in + x R the limits f(x ) e f(x−) exist and are finite, and that 0 ∈ 0 0 the left and right finite difference + f(x + h) f(x ) f(x h)+ f(x−) 0 − 0 , 0 − 0 , h h are bounded for any h> 0 small. Then the series Tf of f converges in x0 to the mean of the values of the right and left limits of f in x0: + 1 f(x ) f(x−) T (x )= f,e2πinξ/τ e2πinx/τ = 0 − 0 . f 0 τ h i 2 n Z X∈ In particular if f ′(x0) exists then Tf (x0)= f(x0). FOURIER SERIES AND TRANSFORM 18 2.5 2.5 2 2 1.5 1.5 1 1 0.5 0.5 0 0 1 2 3 4 5 6 0 1 2 3 4 5 6 2 2 πx x π cos (kx) Figure 1: f(x)= = ∞ 2 .