Computational Harmonic Analysis (Wavelet Tutorial) Part II Understanding Many Particle Systems with Machine Learning Tutorials Matthew Hirn Michigan State University Department of Computational Mathematics, Science & Engineering Department of Mathematics Wavelet Transform Wavelets 2 Wavelet L (R)satisfies: • 2 – Zero average: =0 – Normalized: R =1 k k2 – Centered around t =0 – Localized in time and frequency – Can be either real or complex valued Wavelet Transform f(t) Wavelet dictionary obtained by scaling and • 2 translating : 1 1 t u 0 = u,s +, u,s(t)= − t D { }u R,s R ps s 0 0.2 0.4 0.6 0.8 1 2 2 ✓ ◆ log2(s) −6 Wavelet transform: • −4 Wf(u, s)= f, u,s h + i = 1 f(t)s 1/2 (s 1(t u)) dt −2 − − − Z1 = f ˜s(u) 0 u ⇤ 0 0.2 0.4 0.6 0.8 1 Fig. 4.7. A Wavelet Tour of Signal Processing, 3rd ed. Real wavelet transform Wf(u, s) computed with a Mexican hat wavelet The where vertical axis represents log2 s. Black, grey and white points correspond respectively to positive, zero and negative wavelet coefficients. ˜ 1/2 1 s(t)=s− (s− t) Note: • 0.25 ˜ ( )=p ˆ( ) s ! s s! 0.2 Thus, since: b 0.15 f\ ˜s(!)=fˆ(!) ˜ (!) 0.1 ⇤ s the wavelet transform Wfb(u, s) captures 0.05 the frequency information of f organized 0 by the frequency bands of ˜s. −2 0 2 Fig. 5.1. A Wavelet Tour of Signal Processing, 3rd ed. Scaled Fourier transforms ˆ(2j!) 2, for 1 j 5 and ! [ ⇡, ⇡]. | | 6 6 2 − Real Wavelet Reconstruction Theorem (Calder´on,Grossman and Mor- • 2 let): Let L (R) be a real function such 2 that + ˆ 2 1 (!) C = | | d! < + Z0 ! 1 2 Then, for any f L (R): 2 + + 1 1 1 1/2 1 ds f(t)= Wf(u, s)s− (s− (t u)) du C 0 − s2 Z Z1 + + 2 1 1 1 2 ds f 2 = Wf(u, s) du . k k C 0 | | s2 Z Z1 C < is called the wavelet admissibility • 1 condition. C < + ˆ(0) = 0. This is almost • 1) sufficient. If additionally, ˆ C1, then C < + . • 2 1 Can insure this with sufficient time decay: K (t) | | 1+ t 2+✏ | | Scaling Function Numerically the wavelet transform is only • computed up to scales s<s0, which loses the low frequency information of f. φ The scaling function φ captures this infor- 1 • 0.8 mation. Defined by: 0.6 0.5 + ds 0.4 φˆ(!) 2 = 1 ˆ(s!) 2 0 0.2 | | Z1 | | s 0 −0.5 −5 0 5 −5 0 5 Denote: 1.5 • 1.5 1 t φ (t)= φ and φ˜ (t)=φ ( t) 1 s s s 1 ps ✓s◆ − 0.5 0.5 The low frequency approximation of f at 0 0 • −5 0 5 −5 0 5 scale s is: Fig. 4.6. A Wavelet Tour of Signal Processing, 3rd ed. Mexican hat wavelet for σ = 1 and its FourierFig. 4.8. transform. A Wavelet Tour of Signal Processing, 3rd ed. Scaling function associated to a Mexican hat wavelet and its Fourier transform. ˆ φˆ Af(u, s)= f,φu,s = f φ˜s(u) h i ⇤ Reconstruction still holds: • 1 s0 ds 1 ( )= ( ) ( ) + ( ) ( ) f t Wf ,s s t 2 Af ,s0 φs0 t C Z0 · ⇤ s C s0 · ⇤ Analytic Wavelets Complex valued, analytic wavelets admit a • time-frequency analysis, like the windowed Fourier transform. The wavelet is analytic if: • ! < 0, ˆ(!)=0 8 The wavelet transform Wf(u, s) of an an- • alytic wavelet satisfies very similar recon- struction and energy preservation formulas as the real wavelet transform. Analytic Wavelet Construction ψ^ (ω) Let g be a real, symmetric window. • ^ Define a wavelet as: g(ω ) • (t)=g(t)ei⌘t ˆ(!)=ˆg(! ⌘) ) − 0 η ω Fig. 4.10. A Wavelet Tour of Signal Processing, 3rd ed. Fourier transform ˆ(!) of a wavelet (t)=g(t) exp(i⌘t). Thus if ˆg(!)=0for ! > ⌘, then ˆ(!)=0 • | | for ! < 0, and is analytic. is centered in time at t = 0 and in fre- • quency at ! = ⌘. Gabor wavelets use a Gaussian window, and • so are not strictly analytic and do not have precisely zero average. However ˆ(!) 0 ⇡ for ! 0. Morlet wavelets also use a Gaussian win- • dow, but subtract a constant in order to have zero average: (t)=g(t)(ei⌘t C) − Analytic Wavelet Heisenberg Boxes Suppose is centered at t = 0 with central • frequency ! = ⌘. ω ^ 2 |ψ (ω)| The time variance σt and frequency vari- u,s • 2 ance σ! of are: + 2 1 2 2 η σω σt = t (t) dt s s Z | | 11 + 2 1 2 ˆ 2 σ! = (! ⌘) (!) d! 2⇡ Z0 − | | s σt ^ s σ |ψu ,s (ω)| 0 t 0 0 Scalogram: η σω • s 2 s0 0 PW f(u, ⌘/s)= Wf(u, s) ψ ψu ,s | | u,s 0 0 0 uu0 t Fig. 4.9. A Wavelet Tour of Signal Processing, 3rd ed. Heisenberg boxes of two wavelets. Smaller scales decrease the time spread but increase the frequency support, which is shifted towards higher frequencies. Time-Frequency Plane: Wavelets vs. Windowed Fourier Comparison of time-frequency tilings: Windowed Fourier Transform Wavelet Transform Hyperbolic Chirp Revisited f(t) 1 ↵ ↵ 0 f(t)=a cos 1 + a cos 2 1 β1 t 2 β2 t −1 • − − t ⇣ ⌘ ⇣ ⌘ 0 0.2 0.4 0.6 0.8 1 ξ / 2π 500 400 300 Spectrogram P f(u, ⇠) of windowed Fourier • S 200 transform 100 0 u 0 0.2 0.4 0.6 0.8 1 ξ / 2π 500 400 400 300300 200 Scalogram P f(u, ⌘/s) of analytic wavelet 200 • W 100 transform 100 0 u 0 0.2 0.4 0.6 0.8 1 0 rd u Fig. 4.14. A Wavelet Tour0 of Signal Processing,0.2 3 ed. Sum0.4 of two hyperbolic0.6 chirps. (a): Spectrogram0.8 PSf(u,1⇠). (b): Ridge support calculated from the spectrogram ξ / 2π 400 300 200 100 0 u 0 0.2 0.4 0.6 0.8 1 rd 1 Fig. 4.17. A Wavelet Tour of Signal Processing, 3 ed. (a): Normalized scalogram ⌘− ⇠PW f(u, ⇠) of two hyperbolic chirps. (b): Wavelet ridges. Hyperbolic Chirp Revisited f(t) 1 ↵ ↵ 0 f(t)=a cos 1 + a cos 2 1 β1 t 2 β2 t −1 • − − t ⇣ ⌘ ⇣ ⌘ 0 0.2 0.4 0.6 0.8 1 ξ / 2π 500 400 300 Local maxima of spectrogram P f(u, ⇠) 200 • S 100 0 u 0 0.2 0.4 0.6 0.8 1 ξ / 2π 500 400 300 200 Local maxima of scalogram PW f(u, ⌘/s) 100 • 0 u 0 0.2 0.4 0.6 0.8 1 rd Fig. 4.14. A Wavelet Tour of Signal Processing, 3 ed. Sum of two hyperbolic chirps. (a): Spectrogram PSf(u, ⇠). (b): Ridge support calculated from the spectrogram Parallel Linear Chirps f(t) 0.5 2 2 0 f(t)=a1 cos(bt + ct)+a2 cos(bt ) −0.5 • t 0 0.2 0.4 0.6 0.8 1 ξ / 2π ξ / 2π 500 400 400 300 300 200 200 100 100 0 u 0 u 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 ξ / 2π ξ / 2π 500 400 400 300 300 200 200 100 100 0 u 0 u 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 rd 1 rd 2 Fig. 4.16. A Wavelet Tour of Signal Processing, 3 ed. (a): Normalized scalogram ⌘− ⇠PW f(u, ⇠) of two parallel linear chirps. (b): Fig. 4.13. A Wavelet Tour of Signal Processing, 3 ed. Sum of two parallel linear chirps. (a): Spectrogram PSf(u, ⇠)= Sf(u, ⇠) . (b): Wavelet ridges. Ridge support calculated from the spectrogram. | | Spectrogram: P f(u, ⇠) Scalogram: P f(u, ⌘/s) • S • W Sparsity and Time- Frequency Resolution Lesson: Best transform depends on the • signal f time-frequency properties. A transform that is adapted to the sig- • nal time-frequency property has fewer local maxima, and is thus sparser. Transforms that are not adapted to the sig- • nal di↵use the signal’s energy over many atoms, leading to more local maxima and a less sparse representation. Thus sparsity is a natural criterion to guide • the construction of time-frequency trans- forms. Wavelet Zoom f(t) 2 1 0 t 0 0.2 0.4 0.6 0.8 1 log2(s) −6 −4 −2 0 u 0 0.2 0.4 0.6 0.8 1 Fig. 4.7. A Wavelet Tour of Signal Processing, 3rd ed. Real wavelet transform Wf(u, s) computed with a Mexican hat wavelet The vertical axis represents log2 s. Black, grey and white points correspond respectively to positive, zero and negative wavelet coefficients. Taylor’s Theorem We now turn to measuring the local regu- • larity of f at a point v. Suppose f is m times di↵erentiable in [v • − h, v + h]. Let pv be the Taylor polynomial of f in the • neighborhood of v: m 1 (k) − f (v) k pv(t)= (t v) k! − kX=0 Taylor’s Theorem: The residual "v(t)= • f(t) pv(t)satisfies t [v h, v + h]: − 8 2 − m t v (m) "v(t) | − | sup f (u) | | m! u [v h,v+h] | | 2 − Lipschitz Regularity Lipschitz Regularity: A function f is point • wise Lipschitz (H¨older) ↵ 0 at v, if there ≥ exists K>0 and a polynomial pv of degree m = ↵ such that b c ↵ t R, f(t) pv(t) K t v 8 2 | − | | − | f is uniformly Lipschitz ↵ over [a, b]ifit • satisfies the above for all v [a, b]witha 2 K independent of v.