12 The Haar wavelet
1. Definition
Now define desired wavelet <ÐBÑ
" if !ŸBŸ"Î# " if "Î#ŸB" ´ œ ! otherwise
60 2 1.5 1 0.5
-1 -0.5 0.5 1 1.5 -0.5 -1 -1.5 -2
fig 16: <ÐBÑ
Now define family of Haar wavelets by translating:
61 2 1.5 1 0.5
-1 1 2 3 4 5 6 -0.5 -1 -1.5 -2
fig 17 ÀÐB&Ñœ<
62 4 3 2 1
-0.5 0.5 1 1.5 2 -1 -2 -3 -4 $Î# $ fig 18: #Ð#B(Ñœ<<$ß(
63 In general:
4Î# 4 <<45 ´# Ð#B5Ñ Show Haar wavelets are orthogonal, i.e., ∞ <<45ß´.BÐBÑÐBÑœ! 4ww 5 < 45 < 4 ww 5 ¡(∞
if 4Á4ww or 5Á5 À
64 (i) if 4œ4ww , 5Á5 :
<<ßœ!ww ¡45 4 5 because <<45œ! wherever 45w Á! and vice-versa.
(ii) if 4Á4w :
<<ßœ!ww ¡45 4 5
65 because:
fig 19
Êß! integral < 66 2. Can any function be represented as a combination of Haar wavelets? [A general approach:] Recall: Zœsquare int. functions of form +9 Ð#B5Ñ4 45 5 œ square int. functions constant on dyadic intervals of length #Þ4 [note if j negative then intervals are of length > 1:] 67 Z" = functions constant on intervals of length 2 Zœ# functions constant on intervals of length 4 6 4 2 -4 -2 2 4 -2 fig. 20: function in ZÐ4œ#Ñ4 68 We have: (a) áZ#"!"#$ §Z §Z §Z §Z §Zá [i.e., piecewise constant on integers Ê piecewise constant on half-integers, etc.] 69 Fig. 21: Relationship of nested spaces Z4 70 (b) ∩ Z8 œ Ö!× (only ! function in all spaces) 8 [if a function is in all the spaces, then must be constant on arbitrarily large intervals Ê must be everywhere constant; also must be square integrable; so must be 0]. 71 2 (c) ∪Z8 is dense in L (‘ ) 8 [i.e. the collection of all functions of this form can approximate any function 0ÐBÑ ] Proof: First consider a function of the form 0ÐBÑ = 8 ;Ò+ß,ÓÐBÑ. Assume that + œ 5Î# +" , and 88 , œ jÎ# ,""", where + ß , "Î# . 72 fig 22: Relationship of +ß , with 5Î#88 and jÎ# . 73 Let 1ÐBÑ œ; 88 ÐBÑ − Z . Ò5Î# ßjÎ# Ó . 4 4 Then m0 1m œ .B Ð0 1Ñ#8 œarea under 0 1 Ÿ #Î# ( 74 fig 23: area under 01 75 Since 8²01² is arbitrary, can be made arbitrarily small. Thus arbitrary char. functions 0 can be well- approximated by functions 1ÐBÑ − Z3 Þ -3 76 Now if 0ÐBÑ is a step function: 4 2 -4 -2 2 4 -2 -4 fig 24: step function We can write 0ÐBÑœ -3;Ò+33 ß, Ó ÐBÑœ linear combination of char. 3 functionsÞ 77 So by above argument, step functions 0 can be approximated arbitrarily well by 1− ZÞ4 -4 Now step functions are dense in PÐ# ‘ Ñ (see R&S, # problem II.2), so that ZPÐÑÞ4 must be dense in ‘ -4 78 (d) 0ÐBÑ−Z88" Ê 0Ð#BÑ−Z [because function constant on intervals of length 28 when is constant on intervals of length #8" ] (e) 0ÐBÑ− Z!! Ê 0ÐB5Ñ − Z [i.e. translating function by integer does not change that it's constant on integer intervals] 79 (f) There is an orthogonal basis for the space V0 in the family of functions 99!5 ´ÐB5Ñ where 5 varies over the integers. This function 9 is (in this case) 9; = Ò!ß"ÓÐBÑ . 9 is called a scaling function. Definition: A sequence of spaces ÖZ4 × together with a scaling function 9 which generates Z! so that (a) - (f) above are satisfied, is called a multiresolution analysis. 80 3. Some more Hilbert space theory Recall: Two subspaces QQ"# and of an inner product vector space ZA−Q are orthogonal if every vector "" is perpendicular to every vector A−Q## . 81 Ex: Consider ZœPÐß# 11 Ñ . Then let ∞ QœÖ0ÐBÑÀ0ÐBÑœ +cos 8B× "8 8œ! be the set of Fourier cosine series. Let ∞ QœÖ0ÐBÑÀ0ÐBÑœ ,sin 8B× #8 8œ" be the set of Fourier sine series. 82 ∞∞ Then if 0œ"8 +cos 8B−Q " and if 0œ #8 , sin 8œ" 5œ" 5B − Q#, then using usual arguments: ∞∞ Ø0ß0Ùœ + , Øcos 8Bß sin 5BÙœ! "# 85 8œ" 5œ" Thus QQ"# is orthogonal to . Recall: A vector space ZŠQ is a direct sum Q"# of subspaces QßQ"# if every vector @−Z can be written uniquely as a sum of vectors A−Q"" and A−Q ##. 83 ZQŠQ is an orthogonal direct sum "# if the above holds and in addition QQ"# and are orthogonal. Ex: If Zœ‘$, and Q" œ B Cplane œ ÖÐBß Cß !Ñ À Bß C −‘ × Q# œDaxis œÖÐ!ß!ßDÑÀD −‘ ×ß then every ÐBßCßDÑ−Z can be written uniquely as sum of ÐBßCß!Ñ−Q"# and Ð!ß!ßDÑ−Q , so Z is orthogonal direct sum QŠQÞ"# 84 Ex: ZœPÒß# 11 Ó. Then every function 0ÐBÑ−Z can be written uniquely as ∞∞ 0ÐBÑœ +cos 8B , sin 5B 88 8œ! 5œ" [note first sum in QQ"# and second in ] # Thus PœQŠQ"# is orthogonal direct sum. Note: not hard to show # QP" = even functions in # QP# = odd functions in 85 [thus L2 œ orthogonal direct sum of even functions and odd functions] Theorem 1: If ZQ¼Q is a Hilbert space and if "# and ZœQQ"#, i.e., a @−Z b 7−Q 33 s.t. @œ7"# 7, then Z œQ " ŠQ # is an orthogonal direct sum of QQ"# and Pf: In exercises. Note: no assumption of uniqueness of @3 necessary above. 86 Def: If Zœ["# Š[ is an orthogonal direct sum, we also write [œZ‹[à"##" [œZ‹[Þ Recall: Given a subspace Q§Z , Qœ¼ vectors which are perpendicular to everything in Q œÖ@−ZÀ@¼AaA−[× 87 Ex: If Z œ‘$¼ ß and [ œBC - plane, then [ œD - axis Ex: If ZœPß#¼ then if [œ even functions, [ œ odd functions. Pf. exercise Recall (R&S, Theorem II.3): Given a complete inner product space ZQ and a complete subspace , then ZQQ is an orthogonal direct sum of and ¼ 88 4. Back to wavelets: Recall: ñ Zœ4 functions constant on dyadic intervalsÒ5#4 ß Ð5 "Ñ# 4 ÓÞ ñ áZ# §Z " §Z ! §Z " §Zá # Since Z§Z!" , there is a subspace [œZ‹Z !"! such that ZŠ[œZÞ!!" Indeed, we can make [œ ! perp space of ZZ!" as a subset of . That is we can define [œÖ@−Zl@¼Z×!"! ; follows easily that [! is closed. 89 fig 25: Schematic relationship of Z[ÀZ!!! and as the BC - plane and [! as the D axis ... $ ZŠ[œZœ!!"‘ Þ 90 Similarly define [œZ‹ZÞ"#" Generally: [œZ‹Z4" 4 4" Then relationships are: áZ#"!"# §Z §Z §Z §Zá [[[[# " ! " 91 Also note, say, for Z$ : ZœZŠ[$# # œ Z""# Š[ Š[ œ Z!!"# Š[ Š[ Š [ œ Z""!"# Š[ Š[ Š[ Š[ 92 So if @−Z$$ we have: @œ@A$## œ@""# A A œ @!!"# A A A œ@" A " A ! A " A # ß with @−Z33 and A−[Þ 3 3 [successively decomposing the @@A into another and a ]. 93 In general À # @œ@ AÞ (1) $85 5œ8 Now let 8Ä∞ . Since all vectors in above sum orthogonal, we have (see exercises): # m@ m## œ m@ m mA m # Þ $85 5œ8 94 Thus # mA m## Ÿ m@ m 5$ 5œ8 a8, so # # mA5 m ∞Þ 5œ∞ Lemma: In a Hilbert space LA , if 5 are orthogonal vectors and the sum # mA55 m ∞, then the sum A converges. 55 95 R Pf: We can show that the sum A5 forms a Cauchy 5œ" sequence by noting if RQ : RQ R R m A Amœm### Amœ mAmÒ !Þ 5555 RßQ Ä ∞ 5œ" 5œ" 5œQ" 5œQ" Thus we have a Cauchy sequence. The sequence must ∞ converge (LA is complete), and so 5 exists. 5œ" 96 From above: # @œ@ AÞ 8 $ 5 5œ8 # Letting 8Ä∞ß get @88Ä∞Ò @ $ A 5 . Thus 5œ∞ vectors @8 have limit as 8 Ä∞À @ 8 Ä@ ∞ Þ 97 But notice @−Z§Z§Zá8 8 8" 8# Ê ∞ @−Z∩Z∩ZᜠZ 8 8 8" 8#, 5 5œ8 Thus @−∞ intersection of all Zœ 8 's Ê@∞ œ! (by condition (b) on spaces Z 4 ). 98 Thus taking the limit as 8Ä∞ in (1) À # @œ@ A (1) $85 5œ8 get # @œ AÞ $5 5œ∞ 99 So by definition of direct sum: # Z œá[ Š[ Š[ Š[ Š[ œ [ $ #"!"#: 5 5œ∞ i.e., every vector in Z$ can be uniquely expressed as a sum of vectors in the [4 . Further this is an orthogonal direct sum since [Þ4 's orthogonal 100 Generally: Z8 œá[ #"!"#8" Š[ Š[ Š[ Š[ ŠáŠ[ 8" œ[5 5œ∞9 101 Now note #¼ PœZŠZ$ $ ¼ œZ% ŠZ% ¼ œZ$$ Š[ Š Z% Þ Thus comparing above get ¼¼ Zœ[ŠZ$%$ . 102 Similarlyß ¼¼ Zœ[ŠZÞ%&% So ¼¼ Z$& œ[Š[ŠZ$% . Generally: ¼ Z$ œ[Š[Š[ŠáŠ[Š$% & 8 ¼ Z8". 103 Letting 8Ä∞ and using same arguments, we see that ¼ the Z!8Ä∞8" components “go to ” as , so that ¼ Z$ œ[Š[Š[Šá$%& Thus: #¼ P œ Z$ ŠZ$ œá[ #"!"# Š [ Š[ Š[ Š[ Šá [Thus every function in P# can be uniquely written as a sum of functions in the [4 's]. 104 Thus: Theorem: Every vector @−PÐ∞ß∞Ñ# can be uniquely expressed as a sum ∞ AA−[Þ where 444 4œ∞ 105 Conclusion - relationship of Z[44 and : 106 5. What are the [4 spaces? Consider [!. Claim: [œE´! functions which are constant on half- integers and take equal and opposite values on half of each integer intervalÞ 107 4 3 2 1 -4 -2 2 4 -1 -2 -3 -4 fig 25: Typical function in E 108 Proof: Will show that with above definition of E , ZŠEœZ!", and that ZE! and are orthogonal. Then it will follow that EœZ"! ‹Z ´[ !, First to show ZE!! and are orthogonal: let 0−Z and 1−EÞ Then 0 looks like: 109 fig 25: 0ÐBÑ− Z!! à 1ÐBÑ− [ 110 Thus ∞ Ø0ß 1Ù œ 0ÐBÑ1ÐBÑ.B (∞ " ! " # œ á 0ÐBÑ1ÐBÑ.B œ ! ((((# " ! " since 0ÐBÑ1ÐBÑ takes on equal and opposite values on each half of every integer interval above, and so integrates to ! on each interval. 111 Thus 01 and orthogonal, and so Z! and E are orthogonal. Next will show that if 0−Z"!! , then 0œ0 1 , where 0−Z!! and 1−E ! (which is all that's left to show). 112 Let 0−Z" . Then 0 is constant on half integer intervals: 8 6 4 2 -4 -2 2 4 -2 -4 -6 -8 fig 26: 0ÐBÑ− Z" Define 0! to be the function which is constant on each integer interval, and whose value is the average of the two values of 0ÐBÑ on that interval: 113 fig 27: 0ÐBÑ! as related to 0ÐBÑ . Then clearly 0ÐBÑ! is constant on integer intervals, and so is in Z! . 114 Now define 1!! ÐBÑ œ 0ÐBÑ 0 ÐBÑ À 8 6 4 2 -4 -2 2 4 -2 -4 -6 -8 fig 28: 1!! ÐBÑ œ 0ÐBÑ 0 ÐBÑ Then clearly 1! takes on equal and opposite values on each half of every integer interval, and so is in E . Thus we have: for 0ÐBÑ− Z" ß 115 0ÐBÑœ 0!! ÐBÑ1 ÐBÑ , where 0−Z!! and 1−E ! . Thus ZœZŠE " ! by Theorem 1 above. Thus EœZ"! ‹Z œ[ !, so Eœ[ !. Thus [œ! functions which take on equal and opposite values on each half of an integer interval, as desired. 116 Similarly, can show: [4 = functions which take on equal and opposite values on each half of the dyadic interval of length #4" and are square integrable: 117 8 6 4 2 -4 -2 2 4 -2 -4 -6 -8 fig 29: typical function in [Ð4œ"Ñ4 118 6 What is a basis for the space [4 ? Consider [! = functions which take equal and opposite values on each integer interval What is a basis for this space? Let "if !ŸBŸ"Î# <ÐBÑ œ . œ "if "Î#BŸ" ∞ Claim a basis for [! is Ö< ÐB 5Ñ×5œ∞ . Note linear combinations of <ÐB 5Ñ look like: 1ÐBÑ œ #<<<< ÐBÑ $ ÐB "Ñ # ÐB #Ñ # ÐB $Ñ. 119 4 3 2 1 -4 -2 2 4 -1 -2 -3 -4 fig 30: graph of 1ÐBÑ . 120 I.E., linear combinations of translates <ÐB 5Ñ œ functions equal and opposite on each half of every integer interval. Can easily conclude: # [œ! functions in P equal and opposite on integer intervals œP functions in # which are linear combinations of translates <ÐB 5Ñ. 121 Also easily seen translates <ÐB 5Ñ are orthonormal. Conclude: ÖÐB5Ñ×< form orthonormal basis for [! . "Î# Similarly can show Ö#< Ð#B 5Ñ×5 form orthonormal basis for [" . #Î# # Ö#< Ð# B 5Ñ×5 form orthonormal basis for [#. 122 Generally, 4Î# 4 ∞ Ö#< Ð# B 5Ñ×5œ∞ form orthonormal basis for [4 . 4Î# 4 Define <<45ÐBÑ œ # Ð# B 5Ñ. 123 Recall every function 0−P# can be written 0œ A 4 4 where A−[44 . But each A 4 can be written Aœ +< ÐBÑ 4545 5 [note 4++ fixed above replace 545 by since need to keep track of 4 ]. 124 so: 0œ +< ÐBÑ. 45 45 4 5 Furthermore we have shown the <45 orthonormal. Conclude they form orthonormal basis for P# . 125 7. Example of a wavelet expansion: B# if !ŸB" Let 0ÐBÑœ . Find wavelet expansion. œ ! otherwise 126 8. Some more Fourier analysis: Recall Fourier transform (use =0 instead of for Fourier variable): " ∞ 0ÐBÑœs 0Ð== Ñ/3B= . È#1 (∞ " ∞ s0Ð= Ñ œ .B0ÐBÑ /3B=. È#1 (∞ [earlier had s0Ð== Ñ œ -Ð Ñ ] 127 Write ss0Ð==Y Ñ œ Fourier transform of 0Ð Ñ œ Ð0ÑÞ 9. Plancherel theorem: Plancharel Theorem: (i) The Fourier transform is a one to one correspondence from P# to itself. That is, for every function 0ÐBÑ − P## there is a unique P function which is its Fourier transform, and for every function s1Ð= Ñ − P## there is a unique P function which it is the Fourier transform of. 128 (ii) The Fourier transform preserves inner products, i.e., if s0011 is the FT of and s is the FT of , then Ø0Ðs == Ñßs 1Ð ÑÙ œ Ø0ÐBÑß 1ÐBÑÙ. (iii) Thus m0ÐBÑm## œ m0Ðs = Ñm Þ 129 Now for a function 0−PÒß# 11 Ó , consider the Fourier series of 0 , given by ∞ -/35B Þ 5 5œ∞ The above theorem has analog on Ò11 ß Ó . Theorem below follows immediately from fact that 38B ∞ # Ö/ ÎÈ #111 ×8œ∞ form orthonormal basis for P Ò ß ÓÞ 130 Plancharel Theorem for Fourier series: (i) The correspondence between functions 0 # −PÒ11 ß Ó and the coefficients Ö-×5 of their Fourier series is a one to one correspondence, if we ## restrict -∞5 . That is, for every 0−PÒßÓ11 5 there is a unique series of square summable Fourier # coefficients Ö-55 × of 0 such that l- l ∞Þ 5 Conversely for every square summable sequence # Ö-5 × there is a unique function 0 − P Ò 11 ß Ó such that Ö-5 × are the coefficients of the Fourier series of 0. 131 ##" (ii) Furthermore, m-5 ² œ#1 m0ÐBÑm 5 132