Basics of Wavelets References: I. Daubechies (Ten Lectures on Wavelets; Orthonormal Bases of Compactly Supported Wavelets) Also: Y. Meyer, S. Mallat Outline: 1. Need for time-frequency localization 2. Orthonormal wavelet bases: examples 3. Meyer wavelet 4. Orthonormal wavelets and multiresolution analysis 1. Introduction Signal: fig 1 Interested in “frequency content” of signal, locally in time. E.G., what is the frequency content in the interval [.5, .6]? 1 Standard techniques: write in Fourier series as sum of sines and cosines: given function defined on Ò Pß PÓ as above: 0ÐBÑœ " _ + +cos 8BÐ11 ÎPÑ , sin 8BÐ ÎPÑ !8" 8 # 8œ" Ð+8ß , 8 constants) " P +œ8 .B0ÐBÑcos 8BÐÎPÑ1 P(P " P ,œ8 .B0ÐBÑsin 8BÐÎPÑ1 P(P (generally0+ß, is complex-valued and 88 are complex numbers). FOURIER SERIES: Consider function 0ÐBÑ defined on Ò Pß PÓ. Let PÒPßPÓœ# square integrable functions P œ 0ÀÒPßPÓÄ‚ .Bl0ÐBÑl_# Ÿº (P where ‚ œP complex numbers. Then # forms a Hilbert space. Basis for Hilbert space: _ "" cos8BÐ11 ÎPÑß sin 8BÐ ÎPÑ PP ŸÈÈRœ" (together with the constant function "ÎÈ #P). 2 # These vectors form an orthonormal basis for P"ÎP (constants È give length 1). 1. Recall the complex form of Fourier series: Equivalent representation: Can use Euler's formula /œ3, cos ,3, sin . Can show similarly that the family 8œ_ " /38BÐ1 ÎPÑ #P ŸÈ 8œ_ _ "3 œ8BÐÎPÑ8BÐÎPÑcos 11sin #P #P ŸÈÈ8œ_ is orthonormal basis for PÞ# Function 0ÐBÑ can be written _ 0ÐBÑœ -9 ÐBÑ " 88 8œ_ where P -88ß œ99 0 œ .B 8 ÐBÑ0ÐBÑß ¡(P where >2 " 38BÐ1 ÎPÑ 98ÐBÑ œ 8 basis element œ / . È#P 2. Recall derivation of the Fourier transform from Fourier series: We start with function 0ÐBÑ on Ð Pß PÑ: 3 fig 2 __ 0ÐBÑœ -9 ÐBÑœ - /38BÐ1 ÎPÑ Î #P ""88 8 È 8œ_ 8œ_ Let 018 œ8 ÎP; let ?01 œ ÎP; let -Ð01?0 Ñ œ - # Î #P . 88È Š‹È Then: _ 0ÐBÑœ - /38BÐ1 ÎPÑ Î #P " 8 È 8œ_ _ œÐ-Î#PÑ/3B08 " 8 È 8œ_ _ œ-Î#P/?03B08 ?0 . " 8 È 8œ_ Š‹ " _ œ-#Î#P/Þ1?0?03B08 " 8È Š‹È È#1 8œ_ " _ œ-ÐÑ/Þ0?03B08 " 8 È#1 8œ_ Note as PÄ_ßwe have ?0 Ä!, and 4 -Ð01?0 Ñ œ - # Î #P 88È Š‹È P œ.B0ÐBÑÐBц#Î#P91?08 È È (P Š‹ P #1 œ.B0ÐBÑ/È 38BÐ1 ÎPÑ (P #P?0 P #1 œ.B0ÐBÑÈ /38BÐ1 ÎPÑ (P #PÐ1 ÎPÑ " P œ.B0ÐBÑ/Þ3B08 È#1 (P Now (informally) take the limit PÄ_Þ The interval becomes Ò Pß PÓ Ä Ð _ß _ÑÞ We have " _ 0ÐBÑ œ -Ð0?0 Ñ/3B08 " 8 È#1 8œ_ " _ Ò00-Ð Ñ/3B0 . PÄ_ È#1 (_ Finally, from above " P -Ð0 Ñ œ .B0ÐBÑ /3B0 È#1 (P " _ Ò .B 0ÐBÑ /3B0. PÄ_ È#1 (_ Thus, the informal arguments give that in the limit, we can write 5 " _ 0ÐBÑœ -Ð00 Ñ/3B0 . , È#1 (_ where -Ð0 Ñ (called Fourier transform of 0 ) is " _ -Ð0 Ñ œ .B0ÐBÑ /3B0 È#1 (_ (like Fourier series with sums replaced by integrals over the real line). Note: can prove that writing 0ÐBÑ in the above integral form is valid for arbitrary 0−PÐ_ß_Ñ# . 3. FREQUENCY CONTENT AND GIBBS PHENOMENON Consider Fourier series of a function 0ÐBÑ which is discontinuous at lBl Bœ0. E.g. if 0ÐBÑœB À 2 1.5 1 0.5 -3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 the first few partial sums of the Fourier series of 0 look like this: 5terms: %B%$B%&Bsin sin sin 111$& 6 2 1.5 1 0.5 -3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 fig 3 10 terms of Fourier series: %B%$B%&Bsin sin sin %"!B sin á 111$& "! 1 2 1.5 1 0.5 -3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 20 terms: 7 2 1.5 1 0.5 -3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 40 terms: 2 1.5 1 0.5 -3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 fig 3 Note that there are larger errors appearing near the “singularity" Specifically: “overshoot” of about 9% of the jump near singularity no matter how many terms we take! In general, singularities (discontinuities in 0ÐBÑ or its derivatives) cause high frequency components so that the Fourier series _ 0ÐBÑœ - /38B Î #1 " 8 È 8œ_ 8 has large -8 for n large (bad for convergence). But notice that the singularities are in only one point, but cause all the -8 to be too large. Wavelets deal with the problem of localization of singularities, since they are localized. Advantages of Fourier series: ì “Frequency content" displayed in sizes of the coefficients +,55 and . ì Easy to write derivatives of 0 in terms of series (and use to solve differential equations) Fourier series are a natural for differentiation. Equivalently, sines and cosines are “eigenvectors" of the derivative operator . .B . Disadvantages: ì Usual Fourier transform or series not well-adapted for time-frequency analysis (i.e., if high frequencies are there, then we have large +,55 and for 5 œ "!!. But what part of the function has the high frequencies? Where B!? Where #B$? Possible solution: Sliding Fourier transform - fig 4 9 Thus first multiply 0ÐBÑ by a “window” 1ÐB5B! Ñ, and then take a look at the Fourier series or take Fourier transform: look at PP 34= B .B 0ÐBÑ 145 ÐBÑ œ .B 0ÐBÑ 1ÐB 5B ! Ñ/ ´ - 45 ((P P 34= B Note however: the functions 1ÐBÑœ1ÐB5BÑ/45 ! are not orthonormal like sines and cosines; do not form a nice basis as in Fourier series; need something better. 4. The wavelet transform Try: Wavelet transform - first fix an appropriate function 2ÐBÑ. Then form all possible translations by integers, and all possible “stretchings" by powers of 2: 4Î# 4 2ÐBÑœ#2Ð#B5Ñ45 (#4Î# is just a normalization constant) 10 fig. 5:2Ð#BÑ and 2Ð%B $Ñ Let -œ45 .B0ÐBÑ2ÐBÑÞ 45 ( If 20- chosen properly, then can get back from the 45: 0ÐBÑ œ - 2 ÐBÑ " 45 45 4ß5 These new functions and coefficients are easier to manage. Sometimes much better. Advantages over windowed Fourier transform: ñ Coefficients -45 are all real ñ For high frequencies (42Ð>Ñ large), the functions 45 have good localization (they get thinner as 4Ä_; see above diagram). Thus short lived (i.e. of small duration in B) high frequency components can be seen from wavelet analysis, but not from windowed Fourier transform. 4 4 Note 2#45 has width of order , and is centered about 5# (see diagram earlier). DISCRETE WAVELET EXPANSIONS: 11 Take a basic function 2ÐBÑ (the basic wavelet); fig 6 let 4Î# 4 2ÐBÑœ#2Ð#B5ÑÞ45 Form discrete wavelet coefficients: -45 œ .B0ÐBÑ2 45 ÐBÑ ´ Ø0ß 2 45 Ù. ( Questions: ñ Do the coefficients -045 characterize ? ñ Can we expand 0 in an expansion of the 245? ñ What properties must 2 have for this to happen? ñ How can we reconstruct 0 in a numerically stable way from knowing -45? We will show: It is possible to find a function 22 such that the functions 45 form such a perfect basis for the functions on ‘ . That is, the functions 245 are orthonormal: 12 2ß245 4ww 5 ´ 2ÐBÑ2ÐBÑ.Bœ! 45 4 ww 5 ¡( unless 4œ4ww and 5œ5Þ And any function 0ÐBÑ can be represented by the functions 245: 0ÐBÑ œ - 2 ÐBÑÞ " 45 45 4ß5 So: just like Fourier series, but the 245 have better properties (e.g., they are non-zero only on a small sub-interval, i.e., compactly supported) 5. A SIMPLE EXAMPLE: HAAR WAVELETS Motivation: suppose we have a basic function 1 if 0ŸBŸ 1 9ÐBÑ = œ basic “pixel". 0 otherwise We wish to build all other functions out of this pixel and translates 9ÐB 5Ñ fig 7: 9 and its translates Linear combinations of the 9ÐB 5Ñ: 0ÐBÑ œ #9999 ÐBÑ $ ÐB "Ñ # ÐB #Ñ % ÐB $Ñ 13 5 4 3 2 1 -1 1 2 3 4 5 -1 -2 -3 fig 8: a linear combination of 9ÐB 5Ñ [Note that any function which is constant on the integers can be written in such a form:] Given function 0ÐBÑ, approximate 0ÐBÑ by a linear combination of 9ÐB 5Ñ: fig 9: approximation of 0ÐBÑ using the pixel 9 ÐBÑ and its translates. Define Z! = all square integrable functions of the form 1ÐBÑ œ +9 ÐB 5Ñ " 5 5 = all square integrable functions which are constant on integer intervals 14 fig 10: a function in Z! To get better approximationsßÀ shrink the pixel fig 11: 99ÐBÑ, Ð#BÑ, and 9 Ð## BÑ fig 12: approximation of 0ÐBÑ by translates of 9 Ð#BÑ. Define Z" = all square integrable functions of the form 1ÐBÑ œ +9 Ð#B 5Ñ " 5 5 œ all square integrable functions which are constant on all half- integers 15 6 4 2 -4 -2 2 4 fig 13: Function in Z" Define Z# = sq. int. functions 1ÐBÑ œ +9 Ð## B 5Ñ " 5 5 = sq. int. fns which are constant on quarter integer intervals 6 4 2 -4 -2 2 4 -2 fig 14: function in Z# Generally define Z4 = all square integrable functions of the form 1ÐBÑ œ +9 Ð#4 B 5Ñ " 5 5 16 = all square integrable functions which are constant on #4 length intervals [note if 4 is negative the intervals are of length greater than 1].