Basics of Wavelets

Basics of Wavelets

References: I. Daubechies (Ten Lectures on Wavelets ; Orthonormal Bases of Compactly Supported Wavelets)

Also: Y. Meyer , S. Mallat

Outline:

1. Need for time-frequency localization 2. Orthonormal wavelet bases: examples 3. Meyer wavelet 4. Orthonormal wavelets and multiresolution analysis

1. Introduction

Signal:

fig 1

Interested in “frequency content” of signal, locally in time. E.G., what is the frequency content in the interval [.5, .6]?

1 Standard techniques: write in Fourier series as sum of sines and cosines: given function defined on Ò  Pß PÓ as above:

0ÐBÑœ " _ +  +cos 8BÐ11 ÎPÑ  , sin 8BÐ ÎPÑ !8" 8 # 8œ"

Ð+8ß , 8 constants) " P +œ8 .B0ÐBÑcos 8BÐÎPÑ1 P(P

" P ,œ8 .B0ÐBÑsin 8BÐÎPÑ1 P(P

(generally0+ß, is complex-valued and 88 are complex numbers).

FOURIER SERIES:

Consider function 0ÐBÑ defined on Ò  Pß PÓ .

Let PÒPßPÓœ# square integrable functions

P œ 0ÀÒPßPÓÄ‚ .Bl0ÐBÑl_# Ÿº (P where ‚ œP complex numbers. Then # forms a Hilbert space.

Basis for Hilbert space:

_ "" cos8BÐ11 ÎPÑß sin 8BÐ ÎPÑ PP ŸÈÈRœ"

(together with the constant function "ÎÈ #P ).

2 # These vectors form an orthonormal basis for P"ÎP (constants È give length 1).

1. Recall the complex form of Fourier series:

Equivalent representation:

Can use Euler's formula /œ3, cos ,3, sin . Can show similarly that the family

8œ_ " /38BÐ1 ÎPÑ #P ŸÈ 8œ_ _ "3 œ8BÐÎPÑ8BÐÎPÑcos 11sin #P #P ŸÈÈ8œ_ is orthonormal basis for PÞ#

Function 0ÐBÑ can be written _ 0ÐBÑœ -9 ÐBÑ " 88 8œ_ where

P -88ß œ99 0 œ .B 8 ÐBÑ0ÐBÑß ¡(P where

>2 " 38BÐ1 ÎPÑ 98ÐBÑ œ 8 basis element œ / . È#P

2. Recall derivation of the Fourier transform from Fourier series:

We start with function 0ÐBÑ on Ð  Pß PÑ :

3

fig 2

__ 0ÐBÑœ -9 ÐBÑœ - /38BÐ1 ÎPÑ Î #P ""88 8 È 8œ_ 8œ_

Let 018 œ8 ÎP ; let ?01 œ ÎP ; let -Ð01?0 Ñ œ - # Î #P . 88È Š‹È

Then: _ 0ÐBÑœ - /38BÐ1 ÎPÑ Î #P " 8 È 8œ_ _ œÐ-Î#PÑ/3B08 " 8 È 8œ_ _ œ-Î#P/?03B08 ?0 . " 8 È 8œ_ Š‹ " _ œ-#Î#P/Þ1?0?03B08 " 8È Š‹È È#1 8œ_ " _ œ-ÐÑ/Þ0?03B08 " 8 È#1 8œ_ Note as PÄ_ß we have ?0 Ä! , and

4 -Ð01?0 Ñ œ - # Î #P 88È Š‹È P œ.B0ÐBÑÐBц#Î#P91?08 È È (P Š‹ P #1 œ.B0ÐBÑ/È 38BÐ1 ÎPÑ (P #P?0 P #1 œ.B0ÐBÑÈ /38BÐ1 ÎPÑ (P #PÐ1 ÎPÑ " P œ.B0ÐBÑ/Þ3B08 È#1 (P

Now (informally) take the limit PÄ_Þ The interval becomes Ò  Pß PÓ Ä Ð  _ß _ÑÞ We have " _ 0ÐBÑ œ -Ð0?0 Ñ/3B08 " 8 È#1 8œ_

" _ Ò00-Ð Ñ/3B0 . PÄ_ È#1 (_

Finally, from above

" P -Ð0 Ñ œ .B0ÐBÑ /3B0 È#1 (P

" _ Ò .B 0ÐBÑ /3B0. PÄ_ È#1 (_

Thus, the informal arguments give that in the limit, we can write

5 " _ 0ÐBÑœ -Ð00 Ñ/3B0 . , È#1 (_ where -Ð0 Ñ (called Fourier transform of 0 ) is " _ -Ð0 Ñ œ .B0ÐBÑ /3B0 È#1 (_ (like Fourier series with sums replaced by integrals over the real line).

Note: can prove that writing 0ÐBÑ in the above integral form is valid for arbitrary 0−PÐ_ß_Ñ# .

3. FREQUENCY CONTENT AND GIBBS PHENOMENON

Consider Fourier series of a function 0ÐBÑ which is discontinuous at lBl Bœ0. E.g. if 0ÐBÑœB À

2 1.5 1 0.5

-3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 the first few partial sums of the Fourier series of 0 look like this:

5terms:

%B%$B%&Bsin sin sin  111$&

6 2 1.5 1 0.5

-3 -2 -1 1 2 3 -0.5 -1 -1.5 -2

fig 3

10 terms of Fourier series: %B%$B%&Bsin sin sin %"!B sin á 111$& "! 1 2 1.5 1 0.5

-3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 20 terms:

7 2 1.5 1 0.5

-3 -2 -1 1 2 3 -0.5 -1 -1.5 -2 40 terms:

2 1.5 1 0.5

-3 -2 -1 1 2 3 -0.5 -1 -1.5 -2

fig 3

Note that there are larger errors appearing near the “singularity"

Specifically: “overshoot” of about 9% of the jump near singularity no matter how many terms we take!

In general, singularities (discontinuities in 0ÐBÑ or its derivatives) cause high frequency components so that the Fourier series _ 0ÐBÑœ - /38B Î #1 " 8 È 8œ_

8 has large -8 for n large (bad for convergence).

But notice that the singularities are in only one point , but cause all the -8 to be too large.

Wavelets deal with the problem of localization of singularities, since they are localized.

Advantages of Fourier series:

ì “Frequency content" displayed in sizes of the coefficients +,55 and .

ì Easy to write derivatives of 0 in terms of series (and use to solve differential equations)

Fourier series are a natural for differentiation.

Equivalently, sines and cosines are “eigenvectors" of the derivative operator . .B .

Disadvantages:

ì Usual Fourier transform or series not well-adapted for time-frequency analysis (i.e., if high frequencies are there, then we have large +,55 and for 5 œ "!! . But what part of the function has the high frequencies? Where B! ? Where #B$ ?

Possible solution:

Sliding Fourier transform -

fig 4

9 Thus first multiply 0ÐBÑ by a “window” 1ÐB5B! Ñ , and then take a look at the Fourier series or take Fourier transform: look at

PP 34= B .B 0ÐBÑ 145 ÐBÑ œ .B 0ÐBÑ 1ÐB  5B ! Ñ/ ´ - 45 ((P P 34= B Note however: the functions 1ÐBÑœ1ÐB5BÑ/45 ! are not orthonormal like sines and cosines; do not form a nice basis as in Fourier series; need something better.

4. The wavelet transform

Try: Wavelet transform - first fix an appropriate function 2ÐBÑ .

Then form all possible translations by integers, and all possible “stretchings" by powers of 2:

4Î# 4 2ÐBÑœ#2Ð#B5Ñ45

(#4Î# is just a normalization constant)

10

fig. 5:2Ð#BÑ and 2Ð%B  $Ñ

Let

-œ45 .B0ÐBÑ2ÐBÑÞ 45 (

If 20- chosen properly, then can get back from the 45 :

0ÐBÑ œ - 2 ÐBÑ " 45 45 4ß5

These new functions and coefficients are easier to manage. Sometimes much better.

Advantages over windowed Fourier transform: ñ Coefficients -45 are all real ñ For high frequencies (42Ð>Ñ large), the functions 45 have good localization (they get thinner as 4Ä_ ; see above diagram). Thus short lived (i.e. of small duration in B ) high frequency components can be seen from wavelet analysis, but not from windowed Fourier transform.

4 4 Note 2#45 has width of order , and is centered about 5# (see diagram earlier).

DISCRETE WAVELET EXPANSIONS:

11 Take a basic function 2ÐBÑ (the basic wavelet);

fig 6 let

4Î# 4 2ÐBÑœ#2Ð#B5ÑÞ45

Form discrete wavelet coefficients:

-45 œ .B0ÐBÑ2 45 ÐBÑ ´ Ø0ß 2 45 Ù. (

Questions:

ñ Do the coefficients -045 characterize ? ñ Can we expand 0 in an expansion of the 245 ? ñ What properties must 2 have for this to happen? ñ How can we reconstruct 0 in a numerically stable way from knowing -45?

We will show: It is possible to find a function 22 such that the functions 45 form such a perfect basis for the functions on ‘ .

That is, the functions 245 are orthonormal:

12 2ß245 4ww 5 ´ 2ÐBÑ2ÐBÑ.Bœ! 45 4 ww 5 ¡( unless 4œ4ww and 5œ5Þ

And any function 0ÐBÑ can be represented by the functions 245 :

0ÐBÑ œ - 2 ÐBÑÞ " 45 45 4ß5

So: just like Fourier series, but the 245 have better properties (e.g., they are non-zero only on a small sub-interval, i.e., compactly supported)

5. A SIMPLE EXAMPLE: HAAR WAVELETS

Motivation: suppose we have a basic function

1 if 0ŸBŸ 1 9ÐBÑ = œ basic “pixel".  0 otherwise We wish to build all other functions out of this pixel and translates 9ÐB  5Ñ

fig 7: 9 and its translates

Linear combinations of the 9ÐB  5Ñ : 0ÐBÑ œ #9999 ÐBÑ  $ ÐB  "Ñ  # ÐB  #Ñ  % ÐB  $Ñ

13 5 4 3 2 1

-1 1 2 3 4 5 -1 -2 -3 fig 8: a linear combination of 9ÐB  5Ñ

[Note that any function which is constant on the integers can be written in such a form:]

Given function 0ÐBÑ , approximate 0ÐBÑ by a linear combination of 9ÐB  5Ñ:

fig 9: approximation of 0ÐBÑ using the pixel 9 ÐBÑ and its translates.

Define Z! = all square integrable functions of the form 1ÐBÑ œ +9 ÐB  5Ñ " 5 5 = all square integrable functions which are constant on integer intervals

14

fig 10: a function in Z!

To get better approximationsßÀ shrink the pixel

fig 11: 99ÐBÑ, Ð#BÑ , and 9 Ð## BÑ

fig 12: approximation of 0ÐBÑ by translates of 9 Ð#BÑ .

Define

Z" = all square integrable functions of the form 1ÐBÑ œ +9 Ð#B  5Ñ " 5 5 œ all square integrable functions which are constant on all half- integers

15 6

4

2

-4 -2 2 4

fig 13: Function in Z"

Define Z# = sq. int. functions 1ÐBÑ œ +9 Ð## B  5Ñ " 5 5 = sq. int. fns which are constant on quarter integer intervals

6

4

2

-4 -2 2 4

-2

fig 14: function in Z#

Generally define Z4 = all square integrable functions of the form

1ÐBÑ œ +9 Ð#4 B  5Ñ " 5 5

16 = all square integrable functions which are constant on #4 length intervals

[note if 4 is negative the intervals are of length greater than 1].

17 2. Haar Wavelets, General Theory

1. The Haar wavelet

Now define the desired wavelet <ÐBÑ

" if !ŸBŸ"Î# " if "Î#ŸB" ´ œ ! otherwise

2 1.5 1 0.5

-1 -0.5 0.5 1 1.5 -0.5 -1 -1.5 -2

fig 15: <ÐBÑ

Now define family of Haar wavelets by translating:

2 1.5 1 0.5

-1 1 2 3 4 5 6 -0.5 -1 -1.5 -2

fig 16 ÀÐB&Ñœ<

18 4 3 2 1

-0.5 0.5 1 1.5 2 -1 -2 -3 -4 $Î# $ fig 17: #Ð#B(Ñœ<<$ß(

In general:

4Î# 4 <<45 ´# Ð#B5Ñ Show Haar wavelets are orthogonal, i.e., _ <<45ß´.BÐBÑÐBÑœ! 4ww 5 < 45 < 4 ww 5 ¡(_

if 4Á4ww or 5Á5 À

(i) if 4œ4ww , 5Á5 :

<<ßœ!ww ¡45 4 5 because <<45œ! wherever 45w Á! and vice-versa.

(ii) if 4Á4w :

<<ßœ!ww ¡45 4 5 because:

19

fig 18

Êß! integral <

2. Can any function be represented as a combination of Haar wavelets?

[A general approach:]

Recall:

Zœsquare int. functions of form +9 Ð#B5Ñ4 45" 5

œ square int. functions constant on dyadic intervals of length #Þ4 [note if j is negative the intervals are of length greater than 1:]

Z" = functions constant on intervals of length 2

Zœ# functions constant on intervals of length 4

20 6

4

2

-4 -2 2 4

-2

fig. 19: function in ZÐ4œ#Ñ4

We have:

(a) áZ#"!"#$ §Z §Z §Z §Z §Zá

[i.e., piecewise constant on integers Ê piecewise constant on half-integers, etc.]

Fig. 20: Relationship of the nested spaces Z4

(b)  Z8 œ Ö!× (only ! function in all spaces) 8

21 [if a function is in all the spaces, then it must be constant on arbitrarily large intervals Ê must be everywhere constant; also must be square integrable; so must be 0].

2 (c) Z8 is dense in L (‘ ) 8

[i.e. the collection of all functions of this form can approximate any function 0ÐBÑ]

Proof: First consider a function of the form 0ÐBÑ = ;Ò+ß,Ó ÐBÑ . Assume 88 8 that + œ 5Î#  +"""" , and , œ jÎ#  , , where + ß ,  "Î# .

fig 21: Relationship of +ß , with 5Î#88 and jÎ# .

Let

1ÐBÑ œ; 88 ÐBÑ − Z . Ò5Î# ßjÎ# Ó . 4 4

Then

m0  1m œ .B Ð0  1Ñ#8 œarea under 0  1 Ÿ #Î# (

22

fig 22: area under 01

Since 8²01² is arbitrary, can be made arbitrarily small. Thus arbitrary char. functions 0 can be well-approximated by functions 1ÐBÑ − Z3 Þ -3 Now if 0ÐBÑ is a step function:

4

2

-4 -2 2 4

-2

-4 fig 23: step function We can write

0ÐBÑœ -; ÐBÑœlinear combination of char. functions Þ " 3 Ò+33 ß, Ó 3 So by above argument, step functions 0 can be approximated arbitrarily well by 1− ZÞ4 -4 Now step functions are dense in PÐ# ‘ Ñ (see R&S, problem II.2), so # that ZPÐÑÞ4 must be dense in ‘ -4

(d) 0ÐBÑ − Z88" Ê 0Ð#BÑ − Z

23 [because a function constant on intervals of length 28 when shrunk is constant on intervals of length #8" ]

(e) 0ÐBÑ− Z!! Ê 0ÐB5Ñ − Z

[i.e. translating a function by an integer does not change that it is constant on integer intervals]

(f) There is an orthogonal basis for the space V0 in the family of functions 99!5 ´ÐB5Ñ where 5 varies over the integers. This function 99 is (in this case) = ;Ò!ß"ÓÐBÑ.

9 is called a scaling function.

Definition: A sequence of spaces ÖZ4 × together with a scaling function 9 which generates Z! so that (a) - (f) above are satisfied, is called a multiresolution analysis.

3. Some more Hilbert space theory

Recall: Two subspaces QQ"# and of vector space Z are orthogonal if every vector A−Q"" is perpendicular to every vector A−Q ## .

Ex: Consider ZœPÐß# 11 Ñ . Then let _ QœÖ0ÐBÑÀ0ÐBÑœ +cos 8B× "8" 8œ! be the set of Fourier series with cosine functions only. Let

_ QœÖ0ÐBÑÀ0ÐBÑœ ,sin 8B× #8" 8œ" be the set of Fourier series with sin functions only.

24 __ Then if 0œ"8 +cos 8B−Q " and if 0œ #8 , sin 5B−Q # , then using 8œ"!!5œ" usual arguments: _ Ø0ß0Ùœ + , Øcos 8Bß sin 5BÙœ! "#" 85 8œ"

Thus QQ"# is orthogonal to .

Recall: A vector space ZŠQQßQ is a direct sum M"# of subspaces "# if every vector @−Z can be written uniquely as a sum of vectors A"" −Q and A−Q##.

ZQŠQ is an orthogonal direct sum "# if the above holds and in addition QQ"# and are orthogonal to each other.

Ex: If Zœ‘$, and

Q" œ B  Cplane œ ÖÐBß Cß !Ñ À Bß C −‘ ×

Q# œDaxis œÖÐ!ß!ßDÑÀD −‘ ×ß then every vector ÐBßCßDÑ−Z can be written uniquely as a sum of ÐBßCß!Ñ−Q"# and Ð!ß!ßDÑ−Q , so that Z is an orthogonal direct sum QŠQÞ"#

Ex: ZœPÒß# 11 Ó. Then every function 0ÐBÑ can be written uniquely as __ 0ÐBÑœ +cos 8B , sin 5B ""88 8œ! 5œ"

[note first sum in QQ"# and second in ]

# Thus PœQŠQ"# is an orthogonal direct sum. Note: not hard to show that

# QP" = even functions in

# QP# = odd functions in

25 [thus L2 is an orthogonal direct sum of even functions and odd functions]

Theorem 1: If ZQ¼QZœQQ is a Hilbert space and if "# and "# , i.e., a@−Zb7−Q 33 s.t. @œ77 "# , then ZœQŠQ "# is an orthogonal direct sum of QQ"# and

Pf: In exercises.

Note: no assumption of uniqueness of @3 necessary above.

Def: If Zœ["# Š[ is an orthogonal direct sum, we also write

[œZ‹[à"##" [œZ‹[Þ Recall: Given a subspace Q§Z , Qœ¼ vectors which are perpendicular to everything in Q

œÖ@−ZÀ@¼AaA−[×

Ex: If Z œ‘$¼ ß and [ œBC - plane, then [ œD -axis

Ex: If ZœPß#¼ then if [œ even functions, [ œ odd functions.

Pf. exercise

Recall (R&S, Theorem II.3): Given a complete inner product space Z and a complete subspace QZ , then is an orthogonal direct sum of Q and Q ¼

4. Back to wavelets:

Recall: 4 4 ñ Z4 œ functions constant on dyadic intervals Ò5# ß Ð5  "Ñ# ÓÞ ñ áZ# §Z " §Z ! §Z " §Zá #

Since Z§Z! " , there is a subspace [œZ‹Z !"! such that ZŠ[œZÞ ! !"

26

fig 24: Relationship of Z[ÀZBC[!!! and as the - plane and ! as the $ DZŠ[œZœÞ axis ... !!"‘

Similarly define

[œZ‹ZÞ"#" Generally:

[œZ‹Z4" 4 4" Then relationships are: áZ#"!"# §Z §Z §Z §Zá [[[[# " ! "

Also note, say, for Z$ :

ZœZŠ[$# #

œ Z""# Š[ Š[

œ Z!!"# Š[ Š[ Š [

œ Z""!"# Š[ Š[ Š[ Š[

Thus if @−Z$$ , we have:

27 @œ@A$##

œ@""# A A

œ @!!"# A A A

œ@" A " A ! A " A # ß

with @−Z33 and A−[Þ 3 3

[successively decomposing the @@A into another and a ].

In general À

# @œ@  AÞ (1) $85" 5œ8 Now let 8Ä_ . Since all vectors in above sum orthogonal, we have (see exercises):

# m@ m## œ m@ m  mA m # Þ $85" 5œ8 Thus

# mA m## Ÿ m@ m " 5$ 5œ8 a8, so

# # mA5 m  _Þ 5œ_!

Lemma: In a Hilbert space LA , if 5 are orthogonal vectors and the sum # mA55 m  _, then the sum A converges. !!55

28 R Pf: We can show that the sum A5 forms a Cauchy sequence by noting 5œ"! if RQ:

RQ R R m A Amœm### Amœ mAmÒ !Þ ""5555 " " RßQ Ä _ 5œ" 5œ" 5œQ" 5œQ" Thus we have a Cauchy sequence. The sequence must converge (L is _ complete), and so A5 exists. 5œ"!

From above:

# @œ@ AÞ 8 $" 5 5œ8

# Letting 8 Ä _ß get @88Ä_Ò @ $  A 5 . Thus vectors @ 8 have 5œ_! limit as 8 Ä_À @8 Ä@ _ Þ

But notice @−Z§Z§Zá8 8 8" 8# Ê _ @−ZZZᜠZ 8 8 8" 8#, 5 5œ8

Thus @−_ intersection of all Zœ 8 's

Ê@_ œ! (by condition (b) on spaces Z 4 ).

Thus taking the limit as 8Ä_ in (1) À

# @œ@  A (1) $85" 5œ8 get

29 # @œ AÞ $5" 5œ_

So by definition of direct sum:

# Z œá[ Š[ Š[ Š[ Š[ œ [ $ #"!"#: 5 5œ_ i.e., every vector in Z$ can be uniquely expressed as a sum of vectors in the [[Þ44. Further this is an orthogonal direct sum since 's orthogonal

Generally:

8" Z œá[ Š[ Š[ Š[ Š[ ŠáŠ[ œ [ 8 #"!"#8"5: 5œ_ Now note

#¼ PœZŠZ$ $

¼ œZ% ŠZ%

¼ œZ$$ Š[ Š Z% Þ

Thus comparing above get

¼¼ Zœ[ŠZ$%$ . Similarlyß

¼¼ Zœ[ŠZÞ%&% So

¼¼ Z$& œ[Š[ŠZ$% . Generally:

30 ¼¼ Z$8" œ[Š[Š[ŠáŠ[ŠZ$% & 8 .

¼ Letting 8Ä_ and using same arguments, we see that the Z8" components “go to !8Ä_ ” as , so that

¼ Z$ œ[Š[Š[Šá$%& Thus:

#¼ P œ Z$ ŠZ$ œá[ #"!"# Š [ Š[ Š[ Š[ Šá

[Thus every function in P# can be uniquely written as a sum of functions in the [4's].

Thus:

Theorem: Every vector @−PÐ_ß_Ñ# can be uniquely expressed as a sum _ AA−[Þ where " 444 4œ_

Conclusion - relationship of Z[44 and :

5. What are the [4 spaces?

Consider [!.

31 Claim: [œE´! functions which are constant on half-integers and take equal and opposite values on half of each integer intervalÞ 4

3

2

1

-4 -2 2 4

-1

-2

-3

-4 fig 24: Typical function in E

Proof: Will show that with above definition of E ,

ZŠEœZ!", and that ZE! and are orthogonal. Then it will follow that

EœZ"! ‹Z ´[ !,

First to show ZE!! and are orthogonal: let 0−Z1−EÞ0 and Then looks like:

fig 24: 0ÐBÑ− Z!! à 1ÐBÑ− [

Thus

_"!"# Ø0ß 1Ù œ 0ÐBÑ1ÐBÑ.B œ     á 0ÐBÑ1ÐBÑ.B (((((_ # " ! "

œ!

32 since 0ÐBÑ1ÐBÑ takes on equal and opposite values on each half of every integer interval above, and so integrates to ! on each interval.

Thus 01 and orthogonal, and so Z! and E are orthogonal.

Next will show that if 0−Z"!!!!! , then 0œ0 1 , where 0 −Z and 1 −E (which is all that's left to show).

Let 0−Z" . Then 0 is constant on half integer intervals: 8 6 4 2

-4 -2 2 4 -2 -4 -6 -8 fig 25: 0ÐBÑ− Z"

Define 0! to be the function which is constant on each integer interval, and whose value is the average of the two values of 0ÐBÑ on that interval:

fig 26: 0! ÐBÑ as related to 0ÐBÑ .

Then clearly 0ÐBÑ!! is constant on integer intervals, and so is in Z .

Now define 1!! ÐBÑ œ 0ÐBÑ  0 ÐBÑ À

33 8 6 4 2

-4 -2 2 4 -2 -4 -6 -8 fig 27: 1!! ÐBÑ œ 0ÐBÑ  0 ÐBÑ

Then clearly 1! takes on equal and opposite values on each half of every integer interval, and so is in E0ÐBÑ−Zß . Thus we have: for "

0ÐBÑœ 0!! ÐBÑ1 ÐBÑ , where 0−Z!! and 1−E ! . Thus ZœZŠE " ! by Theorem 1 above.

Thus EœZ"! ‹Z œ[ !, so Eœ[ !.

Thus [œ! functions which take on equal and opposite values on each half of an integer interval, as desired.

Similarly, can show:

[4 = functions which take on equal and opposite values on each half of the dyadic interval of length #4 and are square integrable:

8

6

4

2

-4 -2 2 4

-2

-4

-6

-8 fig 28: typical function in [Ð4œ"Ñ4

34 6 What is a basis for the space [4 ?

Consider

[! = functions which take equal and opposite values on each integer interval

What is a basis for this space? Let "if !ŸBŸ"Î# <ÐBÑ œ . œ "if "Î#BŸ" _ Claim a basis for [! is Ö< ÐB  5Ñ×5œ_ .

Note linear combinations of <ÐB  5Ñ look like:

1ÐBÑ œ #<<<< ÐBÑ  $ ÐB  "Ñ  # ÐB  #Ñ  # ÐB  $Ñ.

4

3

2

1

-4 -2 2 4

-1

-2

-3

-4 fig 29: graph of 1ÐBÑ .

I.E., linear combinations of translates <ÐB  5Ñ œ functions equal and opposite on each half of every integer interval.

Can easily conclude:

# [œ! functions in P equal and opposite on integer intervals

œP functions in # which are linear combinations of translates < ÐB5Ñ .

35 Also easily seen translates <ÐB  5Ñ are orthonormal.

Conclude: ÖÐB5Ñ×< form orthonormal basis for [! .

"Î# Similarly can show Ö#< Ð#B  5Ñ×5" form orthonormal basis for [ .

#Î# # Ö#< Ð# B  5Ñ×5# form orthonormal basis for [ .

Generally,

4Î# 4 _ Ö#< Ð# B  5Ñ×5œ_ form orthonormal basis for [4 .

4Î# 4 Define <<45ÐBÑ œ # Ð# B  5Ñ.

Recall every function 0−P# can be written 0œ A " 4 4 where A−[44 . But each A 4 can be written Aœ +< ÐBÑ 4545" 5

[note 4++4 fixed above replace 545 by since need to keep track of ]. so:

0œ +< ÐBÑ. "" 45 45 4 5

Furthermore we have shown the <45 orthonormal. Conclude they form orthonormal basis for P# .

7. Example of a wavelet expansion:

B# if !ŸB" Let 0ÐBÑœ . Find wavelet expansion. œ ! otherwise

36 8. Some more Fourier analysis:

Recall Fourier transform (use =0 instead of for Fourier variable): " _ 0ÐBÑœs 0Ð== Ñ/3B= . È#1 (_

" _ s0Ð= Ñ œ .B0ÐBÑ /3B=. È#1 (_ [earlier had s0Ð== Ñ œ -Ð Ñ ]

Write ss0Ð==Y Ñ œ Fourier transform of 0Ð Ñ œ Ð0ÑÞ

9. Plancherel theorem:

Plancharel Theorem: (i) The Fourier transform is a one to one correspondence from P# to itself.

That is, for every function 0ÐBÑ − P## there is a unique P function which is its Fourier transform, and for every function s1Ð= Ñ − P# there is a unique P# function which it is the Fourier transform of.

(ii) The Fourier transform preserves inner products, i.e., if s00 is the FT of and ss1 is the FT of 1 , then Ø0Ðs == Ñß 1Ð ÑÙ œ Ø0ÐBÑß 1ÐBÑÙ .

(iii) Thus

m0ÐBÑm## œ m0Ðs = Ñm Þ

Now for a function 0−PÒß# 11 Ó , consider the Fourier series of 0 , given by _ -/35B Þ " 5 5œ_

37 The above theorem has analog on Ò11 ß Ó . Theorem below follows 38B _ immediately from fact that Ö/ ÎÈ #1 ×8œ_ form orthonormal basis for PÒß# 11 ÓÞ

Plancharel Theorem for Fourier series:

(i) The correspondence between functions 0−PÒß# 11 Ó and the coefficients Ö-5 × of their Fourier series is a one to one correspondence, if ## we restrict -_5 . That is, for every 0−PÒßÓ11 there is a unique !5 series of square summable Fourier coefficients Ö-5 × of 0 such that # l-5 l  _Þ Conversely for every square summable sequence Ö-5 × there !5 # is a unique function 0−PÒß11 Ó such that Ö-×5 are the coefficients of the Fourier series of 0 .

##" (ii) Furthermore, m-5 ² œ#1 m0ÐBÑm !5

38 3. General Wavelet Constructions

1. Other constructions: Suppose we use another “pixel” function 9ÐBÑ :

2

1.5

1

0.5

-1 1 2

-0.5

fig 30: another pixel function

Can we use this to build approximations to other functions? Consider linear combination: #99 ÐBÑ$ ÐB"Ñ# 9 ÐB#Ñ 9 ÐB$Ñ

4

3

2

1

-1 1 2 3 4 5 -1

-2

-3 fig 31: graph of linear combination of translates of 9

Note we can try to approximate functions with other pixel functions.

Question: Can we repeat the above process with this pixel (scaling) function? What would be the corresponding wavelet?

39 Assumptions: lÐBÑl99 has finite integral and ÐBÑ.BÁ! . ' More general construction:

# As before define Zœ! all P linear combinations of 9 and its translates:

œÖ0ÐBœ)(2) +9‘ ÐBÑ+ − à0 −P×Þ# " 5!5 5 5 ¹ with

99!5ÐBÑ œ ÐB  5ÑÞ and

ZœÖ0ÐBÑœ +9‘ ÐBÑl+− à0−P×Þ# (3) "5"55" 5

"Î# 99"5ÐBÑ œ # Ð#B  5Ñ etc.

We want the same theory as earlier.

[Note Z! no longer piecewise constant functions]

Recall condition

(d) 0ÐBÑ − Z88" Ê 0Ð#BÑ − Z

This is automatically true by definition of Z0ÐBÑ−Z8! , since if , then 00Ð#BÑ has the form of an element of (2). Then has form of an element of (3), and 0Ð#BÑ − Z" .

Similarly can be shown that (d) holds for any pair of spaces ZZ88" and of above form.

2Þ Some basic properties of F.T.:

40

Assume that s0 œY Ð0ÑÞ Then

(a) Y=Ð0ÐB  -ÑÑÐ Ñ œ /3= - s 0Ð = Ñ "s (b) Y=Ð0Ð-BÑÑ œ- 0Ð Î-Ñ

Proofs: Exercises.

3. Orthogonality of the 9 's:

Another property of ZÀ4

(f) The basis Ö999 ÐB5Ñ× for Z! is orthogonal, i.e. Ø ÐB5Ñß ÐBjÑÙœ! for 5ÁjÞ

Not automatic. Let Y=Ð0Ñ ´ F.T. of 0 ´s 0Ð Ñ .

Require a condition on 9= of the following sort: if 5Áj , then (note use as Fourier variable) À

! œ Ø9 ÐB  5Ñß 9 ÐB  jÑÙ œ Ø Y9 Ð ÐB  5ÑÑß Y9 Ð ÐB  jÑÑÙ

œ Ø/3== 59=ss Ð Ñß / 3 j 9= Ð ÑÙ _ œ/lÐÑl.3Ð5jÑ= 9=s # = (_ Thus conclude if 7Á!ß _ !œ /37=|9=s Ð Ñl. # = (_

#11 ! # 1 œá   /37= lÐÑl.9=s # = Œ(((%111 # !

_ Ð8"ц#1 œ/lÐÑl.37= 9=s # = " 8œ_ (8†#1

41 _ #1 œ/lÐ#8Ñl.37= 9=s 1 # = " 8œ_ (!

#1 _ œ/37= lÐ#8Ñl.9=s 1 # = " (! 8œ_ [since we can show that the integral of the absolute sum converges because _ lÐ9=s #8Ñl. 1# = absolutely integrable; see exercises] 8œ_!

_ Conclude function lÐ9=s #8Ñl 1## on Ò!ß#Ó 1 is in P because it has 8œ_! square summable Fourier coefficients (in fact they are !7Á! if ).

_ Further lÐ9=s #8Ñl 1# is # 1 - periodic in = , and has a Fourier series 8œ_! __ lÐ9=s #8Ñlœ 1 #37 - /= ß ""7 8œ_ 7œ_ where " #1 _ -œ /37= lÐ#8Ñl.œ!9=s 1 # = if 7Á!Þ 7 " #1 (! 8œ_ And __ ""37= # # -! œ / lÐÑl.9=ss = œ lÐÑl. 9= = ##11((_º7œ! _

""_ œÐBÑl.BœÞ |9 # ##11(_ Thus __" lÐ9=s #8Ñlœ 1 #37B -/ œ Þ ""7 8œ_ 7œ_ #1

42 This condition equivalent to orthonormality of ÖÐB5Ñ×9 .

VV01§À

Recall the condition

(a) Z§Z!"

What must be true of 9 for this to hold in general? This says that every function in ZZ!" is in . Thus since 99 ÐBÑ−Z ! , it follows ÐBÑ−Z " , i.e.

99ÐBÑ œ linear combination of translates of È # Ð#BÑ

= 2ÐBÑ9 (4) " 5"5 5

"Î# 99"5ÐBÑ œ # Ð#B  5Ñ

# [recall normalization constant È#PÞ is so we have unit norm] Ex: If9ÐBÑ œ Haar wavelet, then 99ÐBÑ œ Ð#BÑ  9 Ð#B  "Ñ

"" œÐBÑÐBÑ99"! "" ÈÈ##

œ2"!99 "! ÐBÑ2 "" "" ÐBÑ

43 2.5

2

1.5

1

0.5

-2 -1 1 2

fig 32: 99ÐBÑ œ Ð#BÑ  9 Ð#B  "Ñ

Thus in this case all 2! 's are except 2"! and 2 "" ;

"" 2œ"! à 2œ "" Þ ÈÈ##

Note in general that since this is an orthonormal expansion,

2œmÐBÑm_Þ##9 " 5 5

4. What must be true of the scaling function for (1) above to hold?

Thus in general we have:

_R 99ÐBÑ œ 25"5 ÐBÑ œlim 2 5"5 9 ÐBÑ (3) ""RÄ_ 5œ_ 5œR in P# norm. Denote

R 2ÐBÑ´JÐBÑ9 " 5"5 R 5œR

44 Specifically,

R m99 ÐBÑ  2 ÐBÑm Ä !Þ " 5"5 5œR [recall Y is Fourier transform]

Corollary of Plancherel Theorem:

Corollary: The Fourier transform is a bounded linear transformation. In # particular, if the sequence of functions ÖJR ÐBÑ× converges in P norm, then

Y=Y=Ðlim JRR ÑÐ Ñ œ lim ÐJ ÑÐ Ñ 8Ä_ RÄ_ in P# norm, i.e., Fourier transforms commute with limits.

Thus since _ sums are limits and Y is linear: __ Y92ÐBÑœ 2ÐÐÑÑ Y9= ""5"5 5 "5 ŒOœ_ 5œ_ [i.e., Y commutes with _ sums]

LetY9ÐÑÐÑœÐÑ = 9=s . Then generally:

4Î# 4 Y9Ð45 ÑÐ = Ñ œ Y Ð# 9 Ð# B  5ÑÑÐ = Ñ

œ #4Î#Y9 Ð Ð# 4 B  5ÑÑÐ = Ñ

[recall dilation properties of Fourier transform earlier]

4Î#" 4 œ ##4 Y9 Ð ÐB  5ÑÑÐ = Î# Ñ

[recall translation by5/ pulls out an3= 5 ]

œ #4Î# / 3= 5Î#4 Y9 Ð ÐBÑÑÐ = Î# 4 Ñ

œ#4Î# / 3= 5Î#4 9=s Ð Î#Ñ 4

45

Specifically for4œ" : " Y9Ð ÑÐ = Ñ œ # /3= 5Î# 9=s Ð Î#Ñ "5 È # Recall (3): _ 99ÐBÑ œ 2 ÐBÑ " 5"5 5œ_ Fourier transforming both sides:

9=sÐ Ñ œ Y9 Ð ÑÐBÑ (5) _ œ2ÐBÑY9 " 5"5 Œ5œ_ _ " œ 2 /35Ð= Î#Ñ 9=s Ð Î#Ñ " 5 5œ_ È# Define _ " 7Ð= Î#Ñ œ 2 /35Ð= Î#Ñ (6) " 5 5œ_ È# note 7 is #1= - periodic  Fourier series of 7Ð Î#Ñ given above.

## Note 7Ð=1 Ñ − P Ò!ß # Ó, since 25  _. !5 Thus by (5):

9=ssÐ Ñ œ 7Ð = Î#Ñ 9= Ð Î#ÑÞ with 7Ð † Ñ a #1 -periodic P# function.

[Note: This condition exactly summarizes our original demand that Z! §Z"!]

Note if Z§Z!" , then it follows (same arguments) that Z§Z "# , and Z§Z44" in general.

46 5. Some preliminaries:

Given a Hilbert space LZ0−L and a closed subspace , for write 0œ@@¼ where @−Z and @¼¼ −Z Þ

Definition: The operator T defined by T0 œ TÐ@@¼ Ñ œ @ is the orthogonal projection onto Z .

Note T is a bounded linear operator (see exercises).

Easy to check that mT m œ " if T Á ! (see exercises).

Ex: Z œ‘$ Þ TÐBßCßDÑœÐBßCß!Ñœ is the orthogonal projection onto the BC plane.

T ÐBß Cß DÑ œ Ð!ß !ß DÑ œ orthogonal projection onto D axis.

Ex: Z§PÒß##11 Ó is the even functions. Then for 0−P 0ÐBÑ  0ÐBÑ T0ÐBÑœ 0 ÐBÑœ even #

47 (see exercises).

6. How to construct the wavelet?

Recall we have now given conditions on the scaling function:

Condition

(a) áZ#"!"#$ §Z §Z §Z §Z §Zá is equivalent to:

(i) 9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñß where 7#! is a function of period 1 .

Condition

(f) There is an orthogonal basis for the space V0 in the family of functions 99!5 ´ÐB5Ñ is equivalent to:

" (ii) lÐ9=s #5Ñlœ 1 # " 5 #1 Condition

(b)  Z8 œ Ö!× 8 can also be shown to follow from (ii) as follows:

# Proposition: If 9‘− P Ð Ñ and satisfies (ii), then  Z4 œ Ö!×Þ 4−™ Proof: Denote G- to be compactly supported continuous functions. Let 0−ZÞ4 Let % ! be arbitrarily small. By arguments as in problem 4−™ # ˜ II.2 in R&S, GPÐÑ0−G-- is dense in ‘ , so that there exists an with m0  0m˜ % ß

48 with m†m denoting P# norm. Let

Tœ44orthogonal projection onto ZÞ

Then since 0−Z4 : ˜˜˜˜ m0 T4444 0m œ mT 0 T 0m œ mT Ð0 0ÑmŸ m0 0m Ÿ% Þ Thus by triangle inequality ˜˜ ˜ m0mŸm0T0mmT0mŸ44% mT0mÞ 4 (7) ˜ Since T0−Zß44 we have T 0˜ œ -9 ÐBÑÞ 44545" 5

_ where -œØß0Ù4599 45 (recall Ö 45 ÐBÑ×5œ_ is an orthonormal basis for Z 4 ).

Thus if m0m_ œsup l0ÐBÑlß B

mT0mœl-lœlØß0Ùl˜˜##9 # 44545"" 55

# œÐBÑ0ÐBÑ.B9 ˜ " 45 5 ºº( [assuming 0ÒVßVÓ˜ is supported in ]

# Ÿ#m0m4#˜ "†l9 Ð#B5Ñl.B 4 _" 5 (ÒVßVÓ [using Schwartz inequality Ø+ÐBÑ,ÐBÑÙ Ÿ m+ÐBÑmm,ÐBÑm ]

Ÿ #4# m0m˜ " # .B l9 Ð# 4 B  5Ñl # .B _" 5 ((ÒVßVÓ ÒVßVÓ

œ#m0m4#˜ #V l9 Ð#B5Ñl.B 4 # _ " 5 (ÒVßVÓ

49 Cœ#4 B5 ˜ ## œm0m#V_ lÐCÑl.C9 (WVß4

44 [where WœÒ5#Vß5#VÓVß4 5−™ (note we replaced 5Ä5 in the 4 " union) assuming 4#VÞ5 large and negative, so # Note that then the sum becomes a sum over disjoint intervals after the change of variables above, and we therefore replace a sum over 5 by a union over these intervals, as above]

## œ m0m #V;9W ÐCÑl ÐCÑl .C Ò ! _ ( Vß4 4Ä_ by the dominated convergence theorem, since if C™; , ÐCÑ Ò !Þ WVß4 4Ä_ Thus by (7), we have for 4!À large and negative and all % ˜˜ ˜ m0mŸm0T0mmT0mŸ44%% mT0mŸ 42. Thus m0m œ ! and 0 œ !Þ

Condition

2 (c) Z8 is dense in L (‘ ) 8 also follows from (ii):

## Proposition: If 9‘−PÐ Ñ and satisfies (ii), then Z4 œPÐ ‘ ÑÞ 4−™ Proof: Similarly technical proof.

Condition

(d) 0ÐBÑ − Z88" Ê 0Ð#BÑ − Z is automatic from the definition of the Z8 .

Condition

(e) 0ÐBÑ− Z!! Ê 0ÐB5Ñ − Z

50 is also automatic from definition.

Thus we conclude:

Theorem: Conditions (i) and (ii) above are necessary and sufficient for the spaces ÖZ4 × and scaling function 9 to form a multiresolution analysis.

Thus if (i), (ii) are satisfied for 9 and we define the spaces Z4 as usual, the spaces will satisfy properties (a) - (f) of a multiresolution analysis.

Recall: orthonormality of translates ÖÐB5Ñ×9 5−™ is equivalent to: " (ii) lÐ9=s #5Ñlœ 1 # " 5 #1 Rewrite (ii):

##s " l7ÐÎ#5ÑllÐÎ#5Ñlœ! =19=1 #1 !5

" w#w#s Ê#1 œ l7Ð5ÑllÐ5Ñl! =1 9=1 !5 [==w œÎ# ]

w#w# = l7Ð5ÑllÐ5Ñl! =1 9=1s 5 !even w#w#  l7! Ð=1  5Ñl l 9=1s Ð  5Ñl 5! odd

w#w# œ l7Ð†#5ÑllÐ†#5Ñl! =1 9=1s !5 w#w#  l7! Ð=1  Ð#5  "ÑÑl l 9=1s Ð  Ð#5  "Ñl !5 7! œ periodic l7 Ð=9=1w# Ñl lss Ð w  # 5Ñl #  l7 Ð =19=11 w  Ñl # l Ð w   # 5Ñl # !!"" 55

by (ii) w#"" w # œ l7!! Ð==1 Ñl †##11  l7 Ð  Ñl † .

This implies that

51 w# w # l7!! Ð==1 Ñl  l7 Ð  Ñl œ "Þ (8)

What about wavelets? Recall we define [œZ44"4 ‹Z . We now know that ÖÐBÑ×9<45 form basis for Z 4 . The wavelets 45 will form basis for [ 4 .

52 4. More on General Constructions

1. What are <45 ?

[Recall norms and inner products of functions are preserved when we take Fourier transform. Let's take FT to see.]

Note if we find [œZ‹Z!" ! , then we will be done.

[Let's look at Fourier transforms of functions in these spaces:]

Note that if 0−Zß! then

0ÐBÑ œ +99 ÐB5Ñ œ + ÐBÑ (9) ""55!5 55 gives by F.T.:

s0Ð=Y99==9= Ñ œ +() ÐBÑ œ + /35=ss Ð Ñ ´ 7 Ð Ñ Ð Ñ (10) ""5!5 5 0 55 where

7Ð= Ñ´ +/35= Þ 05" 5 is a 211 periodic PÒ!ß#Ó# function which depends on 0 . In fact reversing argument shows (9) and (10) are equivalent.

Similarly can show under Fourier transform that 1−Z" equivalent to:

s1Ð==9= Ñ œ 71 Ð Î#Ñs Ð Î#ÑÞ (11)

# with 71 Ð † Ñ some other #11 periodic function on P Ò!ß # ÓÞ

Notice functions 7701 and both have period #1 (look at their Fourier series). Also note above steps are reversible, so equation (10) implies (9) by reverse argument.

Thus:

53 0 − Z"0 Ís 0 œ 7 Ð=9= Î#Ñs Ð Î#Ñ

Recall: we want to characterize 0−[! ; such an 0 has the property that 0−Z"! and 0¼Z .

Now note: s s 0¼Z!!5 Í0¼99 a5Í0 ¼!5 ß

_ Í0ÐÑ/ÐÑ.œ!s =9==35= s (_

_#Ð7"Ñ1 Í !œss 0ÐÑ/=9==35==ss ÐÑ.œ 0ÐÑ/ =9= 35 ÐÑ " ((_7 #1 7

#1 œs 0Ð#7Ñ/=135Ð=1 # 7Ñ 9=1s Ð#7Ñ. = " 7 (! #1 œ/35= s 0Ð#7ÑÐ#7Ñ.Þ=19=1s = " (! 7 where above identities hold for all 5 .

Hence [viewing sum as some function of = ]

s0Ð=19=1 # 7Ñs Ð # 7Ñœ !Þ " 7 Thus:

!0Ð#7ÑÐ#7Ñœ s =19=1s !7

œ 70! ÐÐ=1  # 7ÑÎ#Ñ 9=1ss ÐÐ  # 7ÑÎ#Ñ7 ÐÐ =1  # 7ÑÎ#Ñ 9=1 ÐÐ  # 7ÑÎ#Ñ !7

œ 70! Ð=19= Î# 7Ñss Ð/) #7 7 Ð =19=1 Î# 7Ñ Ð Î# 7Ñ !7

54 œ + 70 Ð=19=1 Î#  7Ñ s Ð Î#  7Ñ 7!! even 7 odd

‚ 7ÐÎ#7ÑÐÎ#7Ñ! =19=1s

œ 7ÐÎ##7ÑÐÎ##7Ñ7ÐÎ##7ÑÐÎ##7Ñ0!=19=1ss =19=1 !7

7ÐÎ##7ÑÐ##7Ñ0 =119=11s /++ !7

‚7ÐÎ##7ÑÐÎ##7Ñ ! =119=11s

œ 70! Ð==9=19=1 Î#Ñ7 Ð Î#Ñss Ð Î#  # 7Ñ Ð Î#  # 7Ñ !7

7Ð#Ñ7ÐÎ#Ñ0!= / 1 = 1 9=ss ÐÎ##7ÑÐ##7Ñ 1 1 9= / 1 1 !7 # œ 70! Ð==9=1 Î#Ñ7 Ð Î#Ñ ls Ð Î#  # 7Ñl !7 # 7Ð#Ñ7ÐÎ#ÑlÐÎ##7Ñl0!=1= / 19=11s !7

"" œ Ð70! Ð= Î#Ñ7 Ð = Î#Ñ †##11  7 0 Ð =1=1 Î#  Ñ7 ! Ð Î#  ÑÑ †

wwww (3) Ê7Ð0!= Ñ7Ð = Ñ7Ð 0 =1  Ñ7Ð ! =1  Ñœ!

ww w Thus (note 7Ð!!==1 Ñ and 7Ð  Ñ cannot vanish together) à let == Ä À

7Ð0 =1  Ñ 70!! Ð==1-==1 Ñœ  7Ð  Ñ´ Ð Ñ7Ð  Ñß (12) 7Ð! = Ñ where 7Ð=1  Ñ -=ÐÑ´ 0 7Ð! = Ñ and so-=ÐÑ is # 1 periodic. Also,

55 7Ð=1  Ñ 7Ð = #Ñ 1 -=ÐÑÐ -=  1 Ñœ00  (13) 7Ð!!==1 Ñ 7Ð  Ñ

combining fractions and using Ð$Ñ œ! .

Define /=Ð# Ñ œ -= Ð Ñ /3= Þ

Then /=Ð#  # 1 Ñ œ -=1 Ð  Ñ /3Ð=1  Ñ

œ-= Ð Ñ/3=1 / 3 œ -= Ð Ñ/ 3 = œ / Ð# = Ñ so /1 has period # .

Thus s0Ð= Ñ œ 70! Ð = Î#Ñ 9=ss Ð Î#Ñ œ -= Ð Î#Ñ7 Ð = Î#  1 Ñ 9= Ð Î#Ñ

3Î#= œ/= Ð Ñ / 7! Ð = Î#  19= Ñs Ð Î#ÑÞ

Thus we define the wavelet <ÐBÑ by its Fourier transform:

3Î#= <=ssÐ Ñ œ / 7! Ð = Î#  1 Ñ 9= Ð Î#Ñ (14) Thus

s0ÐÑœÐÑÐÑÞ=/=<=s Going back in Fourier transform, we would get (compare with how we got s0Ð==9= Ñ œ 70 Ð Ñs Ð Ñ) 0ÐBÑ œ +< ÐB  5ÑÞ (15) " 5 5 where +ÐÑß5 are coefficients of the Fourier series of /= i.e.,

/=ÐÑœ +/35= Þ " 5 5 To justify process of Fourier transformation as above, need to also show that # the coefficients +l+l_55 are square summable (i.e. ), since we do not !5

56 know whether Fourier transform properties which we have used in getting (15) are valid otherwise.

Note since +5 are coefficients of Fourier series of // , we just need to show is square integrable on Ò!ß #1 Ó (recall this is equivalent to the +5 being square summable). To show that / is square integrable, note that with 70 as in (0) :

# use 7−PÒ!ß#Ó0 1 _#1 .l7ÐÑl==# '! 0

by (12) #1 .lÐÑll7ÐÑl=-=## = 1 œ '! ! 11# ## œ.lÐÑll7ÐÑl=-=! = 1 Œ''! 1

[substitute ==1 wwœ in second integral; then rename == œ again] 11 ## # # œ .=-= l Ð Ñl l7!! Ð =  1 Ñl  . =-= l Ð  1 Ñl l7 Ð =  # 1 Ñl ((!! ## [recall that by periodicityl7!! Ð=1  # Ñl œ l7 Ð = Ñl and use (13) ] 1 ### = .=-= l Ð Ñl Ðl7!! Ð =  1 Ñl  l7 Ð = Ñl Ñ (!

1 use (8) œ.lÐÑl =-= # (!

1 œ.lÐ#Ñl=/ = # (!

==w œ# " #1 .lÐÑl=/= # œ#(!

Thus we have that _#1 .=/= l Ð Ñlß# so that / is square integrable, as '! desired.

57 This was only thing left to show <Ð#B  5Ñ span [! . Wish to show also orthonormal. Use almost exactly the same argument as was used to show the same for 9ÐB  5Ñ :

l<=ss Ð  # 1 5Ñl###use œ (14) l7 Ð = Î#  1 5  1 Ñl l 9= Ð Î#  1 5Ñl ""! 55 [now break up the sum into even and odd 5 again and use the same method as before] ## œ + | 7! Ð=119=1 Î#  5  Ñl ls Ð Î#  5Ñl Œ55 !!even odd

## œ l7! Ð=1 Î#  † #5  19=1 Ñl ls Ð Î#  † #5Ñl !5 (16) ##  l7! Ð=1 Î#  † Ð#5  "Ñ  19=1 Ñl ls Ð Î#  † Ð#5  "ÑÑl !5

## œl7Ð! =1 Î# Ñl l 9=1s Ð Î# †#5Ñl !5

##  l7! Ð=9=1 Î#Ñl ls Ð Î#  † Ð#5  "ÑÑl !5

using (ii) above again ##" œ#Ðl7!! Ð=1 Î#  Ñl  l7 Ð = Î#Ñl Ñ † 1

" œ #1

By same arguments as used for 9<ÐB  5Ñ , it follows by (16) ÐB  5Ñ orthonormalÞ

This proves our choice of < gives a basis for [! as desired. Specifically,

<

58 In same way as for 9 , can show immediately that since functions in [4 are 4 functions in [#! stretched by factor , the functions

4Î# 4 <<45ÐBÑ œ # Ð# B  5Ñ form a basis for [44 ( fixed ß5 varies).

# Since Pœ direct sum of the [4 spaces, conclude functions _ # ÖÐBÑ×<45 4ß5œ_ over all integers 4 and 5 form orthonormal basis for P .

Conclusion: If we start with a pixel function 9ÐBÑ , which satisfies

(i) 9=ss ( )œ 7!! Ð = Î#Ñ 9= Ð Î#Ñ (with 7 some # 1 -periodic function)

# " (ii) lÐ9= #5lœ 1 #1 !5 then the set of spaces Z4 form a multiresolution analysis, i.e., satisfy properties (a) - (f) from earlier.

Further, if define function <ÐBÑ with Fourier transform:

3Î#= <=ssÐ Ñ œ / 7! Ð = Î#  1 Ñ 9= Ð Î#Ñ (17)

(here 7! is from (i) above), then 4Î# 4 <<45ÐBÑ œ # Ð# B  5Ñ. form orthonormal basis for P#

ÒÓ Next we'll construct some wavelets

2. Additional remarks:

Note further that (17) has another interpretation without Fourier transform À

Recall the two scale equation:

59 99ÐBÑ œ 2 ÐBÑÞ " 5"5 5 Also then we have (see eq. (5)) that if 2 7Ð= Ñœ5 /35= ß ! " 5 È# then:

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñ. Then we have from ÐÑ17 : _ " <=ss() = /2/3==1/ # 35Ð Î# Ñ 9= (/2) " 5 5œ_ È# _ " œ/ 3=1=/ # 2/ 35 / 35 Î# 9=s(/2) " 5 5œ_ È# _ " œ2Ð"Ñ/ 5 3Ð5+ "Ñ= Î# 9=s(/2) " 5 5œ_ È#

Inverse Fourier transforming:

60 where

5" 5"standard form 5" 1œ2Ð"Ñœ2Ð"Ñœ2Ð"Ñß5 "5 "5 "5 and (recall) 25 defined by 99ÐBÑ œ 2 ÐBÑÞ " 5"5 5 3. Some comments on the scaling function:

Recall

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñ from earlier. This stated that the Fourier transform of 9 and its stretched version are related by some function 7!! Ð= Î#Ñ , where 7 is a periodic function of period #1 .

Lemma: The Fourier transform of an integrable function is continuous.

Proof: exercise

Assumption: 9ÐBÑ (the scaling function) is integrable (i.e., its absolute value has a finite integral).

Fact: Under our assumptions, it can be shown that _ .B9 ÐBÑ œ " '_ [proof is an exercise]

Consequence: A consequence of the above assumption is that the Fourier transform 9=s ÐÑ satisfies: """__ 99sÐ!Ñ ´ .B ÐBÑ /3†!B œ .B 9 ÐBÑ œ Þ ÈÈÈ###111((_ _ Now recall we had

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñ (18) for some periodic function 7Î#! . Replacing == by above:

61 9=ssÐ Î#Ñ œ 7! Ð = Î%Ñ 9= Ð Î%Ñà Plugging into (18) :

9=ssÐ Ñ œ 7!! Ð = Î#Ñ7 Ð = Î%Ñ 9= Ð Î%ÑÞ (19) Now taking (18) and replacing == by Î% , and then plugging into (19) :

9=ssÐ Ñ œ 7!!! Ð = Î#Ñ7 Ð = Î%Ñ7 Ð = Î)Ñ 9= Ð Î)ÑÞ

Continuing this way 8 times, we get:

88 9====ssÐ Ñ œ 7!!! Ð Î#Ñ7 Ð Î%Ñ7 Ð Î)Ñá 7 ! Ð =9= Î# Ñ Ð Î# ÑÞ or:

8 9=ssÐÑœ 7ÐÎ#Ñ =48 9= ÐÎ#Ñ $ ! 4œ"

Ê s 8 9=ÐÑ 4 œ7ÐÎ#ÑÞ! = (20) s 8 $ 9=ÐÎ#Ñ 4œ"

Now let 8Ä_ on both sides of equation. Since 9s is continuous (above assumption), we get

ss8 " 9=ÐÎ#Ñ8Ä_ Ò 9 Ð!Ñœ Þ È#1 Since the left side of (20) converges as 8Ä_ , the right side also converges. After letting 8Ä_ on both sides of (20): 9=sÐÑ _ œ7ÐÎ#Ñß= 4 s $ ! 9Ð!Ñ 4œ"

Ê

62 " _ 9=sÐÑœ 7ÐÎ#ÑÞ = 4 $ ! È#1 4œ"

Conclusion: If we can find 7Ð! =9 ), we can find the scaling function .

4. Examples of wavelet constructions using this technique:

Haar wavelets: Recall that we chose the scaling function "if !ŸB" 9ÐBÑ œ , œ ! otherwise and then we defined spaces ZÞ4

From 9< we constructed the wavelet whose translates and dilates form a basis for P# .

Such constructions can be made automatic if we use above observations.

Note first in Haar case:

""/"/"" 3== B" 3 9=sÐÑœ /3= B .Bœ œ   333=== ÈÈÈ###111(! º! ”•

#//3== Î# 3 Î# œ /3= Î#  #3 #3 È#1= Œ

# œ /3= Î# sin = Î#Þ È#1=

For Haar wavelets we can find 7Ð! = Ñ from:

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñß so

63 s 9=ÐÑ "3= Î% sin = Î# 7! Ð= Î#Ñ œ œ / 9=sÐ Î#Ñ #Î%sin =

"sin Ð# †= Î%Ñ œ/3 = Î% #Î%sin =

" #sin== Î% cos Î% œ/3= Î% #Î%sin =

" œ/#3= Î% cos = Î% #

œ /3= Î% cos = Î%Þ Recall wavelet Fourier transform is:

3Î#= (4) <=ssÐ Ñ œ / 7! Ð = Î#  1 Ñ 9= Ð Î#Ñ

In this case % <=sÐ Ñ œ /3==1 Î# / 3Ð Î% Î#Ñcos Ð = Î%  1 Î#Ñ / 3 = Î% sin = Î%Þ È#1= [using cos Ð=1 Î% + Î#Ñ œ cos = Î% cos 1 Î#  sin = Î% sin 1 Î# œ  sin = Î% ] %3 œ  /3Î#= sin2 Ð= Î%Ñ È#1= Can check (below) this indeed is Fourier transform of usual Haar wavelet < , except the complex conjugate (which means the original wavelet is reflected about ![ , i.e., translated and negated, which still yields a basis for ! ).

To check this, recall Haar wavelet: "if !ŸB"Î# <ÐBÑ œ œ "if "Î#ŸB"

64 Thus: " _ <=sÐÑœ < ÐBÑ/3= B .B È#1 (_

" "Î# " œÐBÑ/.B< 3= B È#1 Œ((! "Î#

"""Î# " œ /3== B .B  / 3 B .B ÈÈ##11((! "Î#

"/3=== Î# " " / 3 / 3 Î# œ Š‹Š‹33== 3 = 3 = ÈÈ##11

#/3== Î# / 3  " œ  ÈÈ#31= #3 1=

#Ð//Ñ3== Î# 3 Î# œ//3== Î# 3 Î# # È#31=Œ

# œ//Î#3== Î# 3 Î# cos = È#31=Œ

# œ//#†Î%3== Î# 3 Î# cos = Š‹ È#31= [using cos #B œ "  # sin]2 B # œ  /3== Î#  / 3 Î# Ð"  #sin #= Î%Ñ Š‹ È#31=

% œ/Î%3= Î#sin #= Š‹ È#31=

65 %3 œ/Î%3= Î#sin #= Š‹ È#1=

66 5. Constructing Wavelets

1. Meyer wavelets: another example -

Scaling function: "llŸ#Î$ if =1 " Ú 9=sÐÑœ cos 1 /=$ ll" if #Î$ŸllŸ%Î$ 1 = 1 . Û ‘# ˆ‰21 È#1 ! otherwise Ü [error in DaubechiesÀ $Î%11/ instead of $Î# inside ] where / is any infinitely differentiable non-negative function satisfying

Ú !BŸ! if /ÐBÑ œ "B "if Û smooth transition in / from !"B to as goes from !" to Ü and //ÐBÑ  Ð"  BÑ œ ".

fig 33: //ÐBÑ and Ð"  BÑ

67

fig 34: Fourier transform 9=sÐÑ of the Meyer scaling function

Need to verify necessary properties for a scaling function: (i)

" lÐ9=s #5Ñlœ 1 # ÐÑ21 " 5 #1 To see this, consider the two possible ranges of values of = :

(a) l=1 #5lŸ#Î$"" 1 for some 5Þ In that case (see diagram above): " 9=ssÐ#5Ñœ 1"" à 9= Ð#5Ñœ!5Á5 1 if È#1 since if l=1 #5lŸ#Î$"" 1 , then l =1 #5l %Î$ 1 for 5Á5 . Thus (21) holds because there is only one non-zero term in that sum.

(b) #Î$Ÿ1=11 #5Ÿ%Î$"" for some 5 . In this case we also have

%1=1 Î$Ÿ #( 5" "ÑŸ # 1 Î$Þ

Also, for all values 5Á5"" or 5 " , can calculate that #111 5 Â Ò% Î$ß % Î$Óß so

9=sÐ#5Ñœ!Þ 1 So sum has only two non-zero terms:

68 #19=1 lsss Ð # 5Ñl## œ# 19=1 l Ð # 5 Ñl l 9=1 Ð # Ð5 "Ñl # Þ " "" 5 Š‹

##11$$ œcos/=1 l # 5"" l"  cos /=1 l # Ð5 "Ñl" ”•”•##ŒŒ11 ##  # 1 $ # 1 $ œcos /=$5"" " cos /=1  # Ð5 "Ñ " ”•##Œ1 ”•##Œ1 abab ##11$$ œcos/= $5"" "  cos /  = $5 # ”•”•##ŒŒ11 # # ##11$$ œ$5"""cos/=" cos / = $5" # ”•”##ŒŒ11 #ŒŒ #  • ##111$$ œcos/= $5"" "  cos  /= $5 " ”•”•##Œ11 ### Œ ##11$$ œcos/=  $5""  "  sin /=  $5  " ”•”•##ŒŒ11 ## œ" Note that above l=1 #Ð5"Ñlœ =1 #Ð5"Ñ , since quantity in ""ab parentheses always negative for our range of = . In next to last equality have used cos1 B œ sin BÞ ˆ‰# Note since cases (a), (b) cover all possibilities for = (since they cover a range of size ##51=1 for " ), we are finished proving (21) .

Also need to verify:

(ii)

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9= Ð Î#Ñ

for some #1= -periodic 7! Ð Î#Ñ . Indeed, looking at pictures:

69

fig 35: 9=ssÐÑ and 9= ÐÎ#Ñ (----) ratio of these two looks like:

s ss fig. 36: 9=ÐÑÎÐÎ#Ñœ 9=È # 19= ÐÑ in the interval Ò#ß#ÓÞ 1 1 ss s Note since ratio 9=ÐÑÎÐÎ#Ñœ 9=È # 19= ÐÑ in Ò#ß#Ó 1 1 , we can define s 9=ÐÑ s 7! Ð=19= Î#Ñ œ œÈ # Ð Ñ ÐÑ22 9=sÐ Î#Ñ if =11−Ò#ß# ÓÞ

Definition ambiguous when numerator and denominator are 0.

Definition also ambiguous for =11ÂÒ# , # Ó since numerator and denominator both ! . So define 7! Ð= Î#Ñ by periodic extension of above for all real = .

How to do that? Just add all possible translates of the bump 9=sÐÑ to make it 41 -periodic:

70 7 Ð=19=1 Î#Ñ œ #s Ð  % 5ÑÞ ! È " 5 Check:

7 Ð=9= Î#Ñsss Ð Î#Ñ œ # 1 9=19= Ð  % 5Ñ Ð Î#Ñ ! È " 5

ss œÈ #19 ÐÑÐÎ#Ñ = 9 =

œÐÑ9=s where we have used the fact that 9=ss ( % 1 5Ñ has no overlap with 9= ( Î#Ñ if 5Á!Þ

[So we expect a full MRA.]

2. Construction of the Meyer wavelet

Standard construction:

3Î#= <=ssÐ Ñ œ / 7! Ð = Î#  1 Ñ 9= Ð Î#Ñ

œ /3Î#= 9=ss Ð  # 1 Ð#5  "ÑÑ 9= Ð Î#Ñ " 5

œ /3Î#= 9=sss Ð  # 1 Ñ  9= Ð  # 1 Ñ 9= Ð Î#Ñ ’“ [supports of 2d and 3d factors do not overlap for other values of 5 ; note 99ssœ since 9s is real]

71

fig 37: 9=ssÐ  # 1 Ñ  9= Ð  # 1 Ñ and 9=s Ð Î#Ñ (dashed)

fig 38: 9=sssÐ  # 1 Ñ  9= Ð  # 1 Ñ 9= Ð Î#Ñ ’“

Thus have 2 distinct regions:

(a) For #Î$Ÿ1=1 Ÿ%Î$ we see in diagram that

72 /3= Î#<=ssss Ð Ñ œ # 1 9= Ð  # 1 Ñ  9= Ð  # 1 Ñ 9= Ð Î#Ñ È ’“ œÐ#Ñ9=s 1 "$1 œl#l"cos /=1 ##1 È#1 ”•Œ "$1 œÐ#Ñ"cos /=1 ##1 È#1 ”•Œ "$1 œ#cos /= ##1 È#1 ”•Œ " 1 $ œ cos "/= "  # ##1 È#1 ”•”•ŒŒ "$1 œ""cos /= ##1 È#1 ”•”•Œ "$1 œ"sin /= ##1 È#1 ”•Œ

So by symmetry same is true in #1= Î$ Ÿ Ÿ  % 1 Î$ , so replace == by l l above to get: "$1 /3= Î#<=s ÐÑœsin / ll" = for #Î$ŸllŸ%Î$ 1 = 1 ##1 È#1 ”•Œ

(b) For %Î$Ÿ1=1 Ÿ)Î$ , we see from diagram (note #1= Î$ Ÿ Î# Ÿ % 1 Î$Ñ:

/3= Î#<=ssss Ð Ñ œ # 1 9= Ð  # 1 Ñ  9= Ð  # 1 Ñ 9= Ð Î#Ñ È ’“ œ9=s Ð Î#Ñ "$1 œÎ#"cos /= ##1 È#1 ”•Œ "$1 œ"cos /= #%1 È#1 ”•Œ

73 Again by symmetry same is true in )1= Î$ Ÿ Ÿ  % 1 Î$ , so replace = by ll= : "$1 /3= Î#<=s ÐÑœcos / ll" = for %Î$ŸllŸ)Î$ 1 = 1 #%1 È#1 ”•Œ Thus:

Ú /3Î#= sin 1 /= Ð$ ll", if #Î$ŸllŸ%Î$ 1 = 1 " Ý ‘##1 <=sÐÑœ /Ðll"Ñ%Î$ŸllŸ)Î$3Î#= cos 1 /=$ , if 1 =1 #1 Û ‘#%1 È Ý ! otherwise Ü

Fig. 39: The wavelet Fourier transform lÐÑl<=s

Fig. 40: The Meyer wavelet <ÐBÑ

3. Properties of the Meyer wavelet

74 Note: If /1 is chosen as above and has all derivatives !Î# at , can check that <=sÐÑ is:

ì infinitely differentiable (since it is a composition of infinitely differentiable functionsÑ! , and one can check that all derivatives are from both sides at the breakÞ For example, the derivatives coming in from the left #1 at = œ $ are: 8 . s  8 <=ÐÑ œ! .= #1 º=œ $ and similarly 8 . s + 8 <=ÐÑ œ! .= #1 º=œ $ (proof in exercises).

ì supported (non-zero) on a finite interval

Lemma: (a) If a function <ÐBÑ has 8 derivatives which are integrable, then the Fourier transform satisfies

l<=s Ð Ñl Ÿ OÐ"  l = lÑ8 Þ ÐÑ23 Conversely, if (23) holds, then <ÐBÑ has at least 8  # derivatives.

(b) Equivalently, if <=sÐÑ has 8 integrable derivatives, then l< ÐBÑl Ÿ OÐ"  lBlÑ8 ÐÑ24

Conversely, if (24) holds, then <=sÐÑ has at least 8# derivatives.

Proof: in exercises.

Thus: <ÐBÑ

ì Decays at _B faster than any inverse power of

75 ì Is infinitely differentiable

Claim:

4Î# 4 <<45ÐBÑ œ # Ð# B  5Ñ form an orthonormal basis for PÐ# ‘ Ñ .

ì Check (only to verify above results - we already know this to be true from our theory): __ lÐBÑl.Bœ<<==## lÐÑl.s œ" ((_ _ Pf:

_ "$1 lÐÑl.<=s ## = œ . =sin / ll" = ###11#%11 ((_ $$ Ÿ||= Ÿ ”•Œ 1 $ .=/=cos# ll" %)11 #%1 ( $$ŸŸ||= ”•Œ

[getting rid of the |† | and doubling; changing vars. in second integral]

"$1 œ.=/= sin# " 11#%11 ## ( $$ŸŸ= ”•Œ

1 $ # .=/=cos# " 2411 ##1 ( $$ŸŸ= ”•Œ

"$$11 œ .=/=sin## " # cos /= " 111#%11 ## ## ( $$ŸŸ= œ”•”•Œ Œ

"$1 œ.""=/=cos# 11#%11 ## ( $$ŸŸ= œ”•Œ

76 $ [letting = œ#1 ==1  " Ê œ # Î$Ð=  "Ñ] # " 1 œ.="Ð=Ñcos# / $#(! Š‹’“

# "Î#11 " œ .="cos##// Ð=Ñ  .=" cos Ð=ÑÑ $#((!Š‹Š‹’“ "Î# ’“ #

# "Î#11 "Î# œ .= " cos##// Ð=Ñ  .= "  cos Ð=  "Î#Ñ $#((!!Š‹Š‹’“ ’ # “ [using //(=  "Î#Ñ œ "  Ð"Î#  =Ñ ]

# "Î#11 "Î# œ .="cos##// Ð=Ñ  .=" cos Ð"Ð"Î#=ÑÑ $#((!!Š‹Š’“ ’ # “ ‹

# "Î#11 "Î# œ .="cos##// Ð=Ñ  .=" sin Ð"Î#=Ñ $#((!!Š‹Š‹’“ ’ # “

"Î# "Î# =Ä"Î#= # 11 œ .="cos##// Ð=Ñ  .=" sin Ð=Ñ $#((!!Š‹Š‹’“ ’“ #

# "Î# œ.=Ð#"Ñœ" $(!

ì To show in another way that they form an orthonormal basis, sufficient to show that for arbitrary 0−PÐÑ# ‘ ,

_ _ lß0lœl0ÐBÑl.B< ## " ¡45 4ß5 (_

[this is a basic analytic theorem].

77 Now note:

__ # lß0lœ<<# .BÐBÑ0ÐBÑ.B "" ¡45 45 4ß5 4ß5 ºº(

_ # œ.0ÐÑÐÑÞ==<=s s " 45 4ß5 ºº(

Note if 4Î# 4 <<45ÐBÑ œ # Ð# B  5ÑÞ Then as usual: ss4Î# 4 3#4 5= <=45ÐÑœ# < Ð# = Ñ/ Þ Plug this in above and can do calculation to show (we won't do the calculation):

_ _ l0ß< l## œ .Bl0ÐBÑlß " ¡45 4ß5 (_ as desired.

CONCLUSION: The wavelets

4Î# 4 <<45ÐBÑ ´ #(2 B  5Ñ form an orthonormal basis for the square integrable functions on the real line.

4. Daubechies wavelets:

Recall that one way we have defined wavelets is by starting with the scaling (pixel) function 9sÐBÑ . Recall it satisfies:

9=ssÐ Ñ œ 7! Ð = Î#Ñ 9 Ð = Î#Ñ

78 for all == , where 7Ð!! Ñ is some periodic function. If we use 7 as the starting point, recall we can write " _ 9=sÐÑœ 7ÐÎ#ÑÞ = 4 (25) $ ! È#1 4œ"

Recall 7! is periodic, and so has Fourier series:

7Ð= Ñœ + /35= . !5" 5

## If7!!! satisfies l7ÐÑll7Ð==1 Ñlœ" , then it is a candidate for construction of wavelets and scaling functions.

3Î#= For Haar wavelets, recall 7! Ð== Ñ œ /cos Î#ß so we could plug into (25) to get 9

If we start with a function 7Ð! = Ñ , when does ÐÑ25 lead to a genuine wavelet? Check conditions: (1) " _ 9=sÐÑœ 7Ð#Ñ =/ 4 $ ! È#1 4œ"

" _ œ 7 Ð== Î#Ñ 7 Ð/ #4 Ñ !!$ È#1 4#=

" _ œ 7 Ð== Î#Ñ 7 Ð/ #4" Ñ !!$ È#1 j="

œ 7! Ð=9= Î#Ñs Ð Î#Ñ (26)

Recall this implies that Z§Z44" where

_ Zœ +9 ÐBÑ l+l_# 45455"" Ÿ5œ_º 5

4Î# 4 (usual definition) with 9945ÐBÑ œ # Ð# B  5Ñ

79 (2) The second condition we need to check is that translates of 9 orthonormal, i.e., " lÐ9=s #5Ñlœ 1 # Þ " 5 #1 If

R 7Ð= Ñœfinite Fourier series œ +/3= 5 œ trigonometric polynomial !5" 5œR there is a simple condition which guarantees condition (2) holds.

Theorem (Cohen, 1990): If the trigonometric polynomial7! satisfies 7Ð!Ñœ"! and ## l7!! Ð==1 Ñl  l7 Ð  Ñl œ " (27)

(our standard condition on 7Ñ!! , and also 7Ð==1 ÑÁ 0 for l lŸ Î$ , then condition (2) above is satisfied by " _ 9=sÐÑœ 7ÐÎ#Ñ = 4 $ ! È#1 4œ" Proof: Daubechies, Chapter 6.

Since condition (1) is also automatically satisfied, this means 9 is a scaling function which will lead to a full orthonormal basis using our algorithm for constructing wavelets.

Another choice of 7! is: " 7 Ð= Ñœ ÒÐ" $ÑÐ$ $Ñ/3== Ð$ $Ñ/ #3 Ð" $Ñ/ $3 = Ó ! ) ÈÈ È È

(Fourier series with finite number of terms).

80 2

1.5

1

0.5

-2 -1 1 2 -0.5

-1

-1.5 Fig 41: Real (symmetric) and imaginary (antisymmetric) parts of 7Ð! = Ñ

To check Cohen's theorem satisfied:

(i) Equation (27) satisfied (see exercises).

(ii) If 7Ð!!== ÑœRe 7Ð Ñ3 Im, 7Ð ! = Ñ ### l7ÐÑlœl!!!===Re 7ÐÑllIm 7ÐÑlÁ! for llŸÎ$=1 , as can be seen from graph above.

So: conditions of Cohen's theorem are satisfied.

In this case if we define scaling function 9 by computing infinite product (25) (perhaps numerically), and then use our standard procedure to construct wavelet <ÐBÑ , we get:

phi(t)

1.25

1

0.75

0.5

0.25

t 0.5 1 1.5 2 2.5 3

-0.25

81 psi_2(t)

1.5

1

0.5

t -1 -0.5 0.5 1 1.5 2

-0.5

-1

fig 42: pictures of 9< and

Note meaning of 7! : In terms of the original wavelet, this states

999ÐBÑ œ" ÒÐ"  $Ñ Ð#BÑ  Ð$  $Ñ Ð#B  "Ñ % ÈÈ  Ð$ ÈÈ $Ñ99 Ð#B  #Ñ  Ð"  $Ñ Ð#B  $ÑÓ

(see (26) above). Note this equation gives the information we need on 9 , since it determines 7Ð! = ÑÞ

82 6. Examples, applications

1. Other examples Note again it is possible to get other wavelets this way: If we demand 99ÐBÑ œ Þ##' Ð#BÑ  Þ)&% 9 Ð#B  "Ñ  "Þ#% 9 Ð#B  #Ñ

 Þ"*'999 Ð#B  $Ñ  "Þ%$% Ð#B  %Ñ  Þ!%' Ð#B  &Ñ

 Þ""!999 Ð#B  'Ñ  Þ!!) Ð#B  (Ñ  Þ!") Ð#B  )Ñ

 Þ!!%9 Ð#B  *Ñ (28)

Then this results with an 7Ð! = Ñ 3#3$=== *3 = 7! Ð= Ñ œ Þ""$  Þ%#( /  Þ&"# /  Þ!*)/  á  Þ!!#/ Þ

4

2

-2 -1 1 2

-2

-4 Fig 43: Real (symmetric) and imaginary parts of 7! ; note condition (ii) of Cohen's theorem is satisfied.

Can check it satisfies condition (ii) of Cohen's theorem and resulting 9 is obtained: _ 9=sÐÑœ 7ÐÎ#Ñ = 4 . $ ! 4œ"

83 It satisfies required properties (a) - (f) of a multiresolution analysis. Corresponding scaling function 559<ÐBÑ and wavelet ÐBÑ are below

Fig 44: Scaling function and wavelet for the above 9 choice

NOTE: Can show that if there is a finite number of terms on the right side of (28) , then corresponding wavelet and scaling function are compactly supported.

2. Numerical uses of wavelets Note that once we have an orthonormal wavelet basis Ö×<45 , can write any function:

0ÐBÑ œ +< ÐBÑß " 45 45 4ß5 with +œÐ0ßÑ45<< 45 . Numerically, can find +œØß0Ù 45 45 using numerical integration to evaluate inner productÞ

With Daubechies and other wavelets, there are no closed form for the wavelets, so above integrations must be performed on the computer.

84 But there are very efficient methods of doing this: in order to get all the wavelets <45 into the computer, we just need to input one - all others are rescalings and translations of the original one.

There are efficient algorithms to get coefficients +45 ; more details in Daubechies' book.

3. SOME GENERAL PROPERTIES OF ORTHONORMAL WAVELET BASES:

Theorem: If the basic wavelet <<ÐBÑ has exponential decay, then cannot be infinitely differentiable.

(in particular, if << has compact support, then cannot be infinitely differentiable).

Proof: Daubechies, Chapter 5.

Compactly Supported Wavelets:

So far we are able to get wavelets

4Î# 4 <<45ÐBÑ œ # Ð# B  5Ñ which form an orthonormal basis for PÞ# Note Haar wavelets had compact support. When will wavelets be compactly supported in general?

Recall we assume that given basic scale space Z! , that we have scaling (pixel) function 99 such that ÖÐB5Ñ×5! form basis for Z .

Recall

ì Z§Z!" , ì 99ÐBÑ − Z!" Ê ÐBÑ − Z ì È#Ð#BÑ−Z9 " _ form a basis for ì Ö#Ð#B5Ñ×È 9 5œ" Z"

85 Recall since 9ÐBÑ − Z"5 , we have for some choice of 2 : _ 99ÐBÑ œ 2 # Ð#B  5Ñ. " 5 È !

Constants 2ZZ5!" relate the space to .

We will see that:

Theorem:

finitely many 2Á!Í5 <9 , have compact support.

Proof: É#Ð#BjÑ : Assume 99 has compact support. Then note since È are orthonormal,

2œj #Ð#BjÑÐBÑ.B99 ( È œ!for all but a finite number of jÀ

fig 45À2œ Note 6 integral of product œ! for all but finite number of j To prove Ê : (rough sketch only)

Assume that 2!5 are for all but a finite number of 5 . Then need to show 9ÐBÑ has compact support. Strategy of proof: look at 9=sÐÑÞ

Recall we defined

86 2 7Ð= Ñœ5 /35= ! " 5 È# Recall: " _ 9=sÐÑœ 7Ð#4 = Ñ. $ ! È#1 4œ" ì From this show that 9=s ( ) extends to an analytic function of = in whole complex plane satisfying:

l9=s Ð Ñl Ÿ GÐ"  l = lÑQRll / Im = for constants QRÞ and

ì This implies by Paley-Wiener type theorems that 99ÐBÑ œ J" Ðs Ñ is compactly supported.

4. GENERIC PRESCRIPTION FOR COMPACTLY SUPPORTED WAVELETS:

ì Start with finite sequence of numbers 2Z5! (define how will be related to Z")

ì Construct 2 7Ð= Ñœ5 /35= ! " 5 È# check that it satisfies Cohen's theorem conditions À

l7Ð! ==1 ÑlÁ! for l lŸ Î$Þ and

## l7!! Ð==1 Ñl  l7 Ð  Ñl œ "Þ

ì Construct

87 " _ 7Ð#4=9= Ñœs Ð Ñ $ ! È#1 4œ" ì Construct Fourier transform of wavelet by:

3Î#= <=ssÐ Ñ ´ / 7! Ð = Î#  1 Ñ 9= Ð Î#Ñ, ì Take inverse Fourier transform to get <ÐBÑ œ wavelet

5. SOME FURTHER PROPERTIES OF WAVELET EXPANSIONS

QUESTION: Do wavelet expansions actually converge to the function being expanded at individual points B ?

Assume that scaling function 9 is bounded by an integrable decreasing function. Then:

Theorem: If 0 is a square integrable function, then the wavelet expansion of 0 _ 0ÐBÑœ +< ÐBÑ " 45 45 4ß5 converges to the function 0 almost everywhere (i.e., except on a set of measure !).

QUESTION: How fast do wavelet expansions converge to the function 0 ?

ANSWER: That depends on how “regular" the wavelet < is. More particularly it depends exactly on the Fourier transform of < :

Theorem: In . dimensions, the wavelet expansion

0ÐBÑœ +< ÐBÑ " 45 45 4ß5 converges to a smooth 0 in such a way that the partial sum

88 +ÐBÑ< " 45 45 4ŸRß5 differs from 0ÐBÑ at each B by at most G †#R= , iff

lÐÑlll<=s ##=. = . = _. (

6. CONTINUOUS WAVELET TRANSFORMS

Consider a function <<ÐBÑ − P# (i.e., is square integrable), such that < ÐBÑ decays fast enough at _ (faster than "ÎB# ), and such that _ <ÐBÑ .B œ !. (_ Then we can define an integral wavelet expansion (integrals instead of sums) using re-scalings of <ÐBÑ :

Define rescaled functions

"Î# <<+ß,ÐBÑ ´ l+l Ð+ÐB  ,ÑÑ. [note + Ä "Î+ in definition of Daubechies]

Here +ß , −‘< . Thus + measures how much has been stretched (dilation parameter), and , measures how much < has been moved to the right (translation parameter).

New point: dilation parameter +, and translation parameter can take on any real value.

Now define wavelet expansions in this case (analogous to Fourier transform -- called wavelet transform): given 0−PÐ# ‘ Ñ , we define the transform (assuming that < is real)

89 Ð[ 0ÑÐ+, ,Ñ œ .B 0ÐBÑ l+l"Î#< Ð+ÐB  ,ÑÑ (

œ.B0ÐBÑÐBÑ<+ß, (

œß0Ù< +ß, How to recover0 from Ð[ 0ÑÐ+ß ,Ñ ?

Claim: __ 0ÐBÑ œ G .+ ., Ð[ 0ÑÐ+ß ,Ñ<+ß, ÐBÑ ((_ _ where

Gœ#.lllÐÑlÞ"1==<= "s # ( Pf. of claim (sketch; details in Daubechies, Ch. 2):

We will show that for any 1ÐBÑ − P# ,

__ 1ÐBÑß 0ÐBÑ œ 1ÐBÑß G .+ ., Ð[ 0ÑÐ+ß ,Ñ<+ß, ÐBÑ ¡ ((_ _ ¡

To see this, note that _ 1ÐBÑß 0ÐBÑ œ 1ÐBÑ 0ÐBÑ .B ¡(_

_ œ.1ÐÑ0ÐÑ===s s (_ [use “Plancherel Theorem” for wavelet transforms]

œ G .+ ., Ð[ 1ÑÐ+ß ,ÑÐ[ 0ÑÐ+ß ,Ñ ((

90 œ G .+ ., Ø1ÐBÑß<+ß, ÐBÑÙÐ[ 0ÑÐ+ß ,ÑÐBÑ ((

œ 1ÐBÑß G .+ ., Ð[ 0ÑÐ+ß ,Ñ<+ß, ÐBÑ ß ¤¥(( as desired, completing the proof.

Thus we know how to recover 0ÐBÑ from [0Ð+ß,Ñ (analogous to recovering 0ÐBÑ from s 0Ð= Ñ in Fourier transform).

QUESTION: What sorts of functions are Ð[ 0ÑÐ+ß ,Ñ ? For some choices of <, these are spaces of analytic functions.

7. Convolutions:

Definition: The convolution of two functions 0ÐBÑ and 1ÐBÑ is defined to be _ 0ÐBч1ÐBÑ´ 0ÐBCÑ1ÐCÑ.CÞ (_

Theorem 2: The convolution is commutative: 0‡1œ 1‡0 Proof: Exercise.

Theorem 3: The Fourier transform of a convolution is a product. Specifically, s Y1==Ð0ÐBч1ÐBÑÑ œÈ # 0Ð Ñ1Ðs Ñ Proof: Exercise.

Lemma 4: For any function 0Ð0ÐBÑÑœ0ÐÑ , Y=s Proof: Exercise.

8. APPLICATION OF INTEGRAL WAVELET TRANSFORM: IMAGE RECONSTRUCTION (S. Mallat)

91 Dyadic wavelet transform: a variation on continuous wavelet transform.

Now define new dilation only by powers of 2; arbitrary translations:

44 <<4ß,ÐBÑ œ # Ð# ÐB  ,ÑÑ Define

44 <<4ÐBÑ œ # Ð# BÑ. (Still allow ,−‘ to take all values, but restrict +œ#4 .)

Define this dyadic (partially discrete) wavelet transform by:

Ð[ 0ÑÐ4ß ,Ñ œ 0ÐBÑ<4ß, ÐBÑ .B ( i.e., usual set of wavelet coefficients, except that , is continuous.

Note:

Ð[ 0ÑÐ4ß ,Ñ œ 0ÐBÑ<4ß, ÐBÑ .B (

œ .B 0ÐBÑ #44< Ð# ÐB  ,ÑÑ (

œ.B0ÐBÑÐB,Ñ<4 (

œ Ð0‡<4 ÑÐ,Ñ (a convolution) where as above

44 4 <<4ÐBÑœ# Ð#BÑœ shrinking of < by a factor # .

New assumption: Fourier transform <=s ( ) satisfies _ " lÐ#Ñlœ<=s 4# . " 4œ_ #1

92 Now: given 0ÐBÑ , consider dyadic wavelet transform; + œ #4 only:

Can show under our assumptions that can recover 0 in this case too: Recovery formula for 0 is: _ 0ÐBÑ œ Ð[ 0ÑÐ4ß Bч< ÐBÑ " 4 4œ_

(convolution in variable B ). It is easy to check that this is correct: if Y denotes Fourier transform:

__ Y

_ œY<< 0ÐBч ÐBч ÐBÑ " 44 4œ_ ab

_ œ#1s 0ÐÑÐÑÐÑ =<=<=ss " 44 4œ_

_ œ#1s 0Ð =<=<= Ñss Ð#4 Ñ Ð#4 Ñ. " 4œ_

_ œ #1=<=s 0Ð Ñls Ð#4 Ñl # " 4œ_

_ œs 0Ð=1 Ñ # l

œ0Ðs = Ñ. QUESTION: Given 0ÐBÑ , what sort of function is the wavelet transform Ð[ 0ÑÐ4ß ,Ñ, as a function of 4 and , ?

93 Let Z œ the collection of possible functions Ð[ 0ÑÐ4ß ,Ñ œ collection of possible wavelet transforms. When is an arbitrary function 1Ð4ß ,Ñ a wavelet transform?

Can check that 1 must satisfy a so-called reproducing kernel equation: 1Ð4ß ,Ñ is the wavelet transform of some function iff _ 1Ð4ß ,Ñ œ ÐO1ÑÐ4ß ,Ñ ´ << Ð,ч Ð  ,ч1Ðjß ,Ñ " 4j jœ_ [this equation defines O1 ; note convolution is in , .]

Back to recovering 0 from wavelet transform:

Thus we can recover 00 as a sum of at different scales: _ 0 œ Ð[ 0ÑÐ4ß Bч< Ð  BÑ. " 4 4œ_

Since < is a known function, we can recover 0 from the sequence of functions. Assume +ÐBÑ is a cubic B-spline:

Fig. 46: A cubic B-spline +ÐBÑ is a symmetric compactly supported piecewise cubic polynomial function whose transition points are twice continuously differentiable

94 . Now let the wavelet be its first derivative: <ÐBÑ œ.B +ÐBÑ

. Fig 47: <ÐBÑ œ.B +ÐBÑ is the wavelet

Using the wavelet <ÐBÑ À

Ð[ 0ÑÐ  #ß BÑ Ð[ 0ÑÐ  "ß BÑ Ð[ 0ÑÐ!ß BÑ Ð[ 0ÑÐ"ß BÑ Ð[ 0ÑÐ#ß BÑ Ð[ 0ÑÐ$ß BÑ

To see that these pieces of 00 represent at different scales, look at example:

95

So: one can recover 0 from knowing the functions Ð[ 0ÑÐ4ß BÑ. This is a lot of functions. What advantage of storing 0 in such a large number of functions? We can compress the data.

CONJECTURE: We can recover 0 not from knowing all of the functions [Ð4ßBÑ, but just from knowing their maxima and minima.

Meyer has proved this conjecture false strictly speaking certain choices of < (including the above derivative <ÐBÑ of the cubic spline). It has been proved true for another choice, the derivative of a Gaussian.

. # <ÐBÑ œ /B .B However, for either choice of < numerically it is possible to recover 0ÐBÑ from knowing only the maxima and minima of the functions [Ð4ßBÑÞ

Numerical method:

96 Assume that we are given only the maxima and minima points of the function [Ð4ßBÑ for each 4 . How to recover 0 ?

Given 0[Ð4ßBÑ , first take its wavelet transform; get . Define

> œ1Ð4ßBÑ set of all functions which have the same set of maxima and minima (in B[Ð4ßBÑ ) as for each 4 .

Zœ set of all 1Ð4ßBÑ which are wavelet transforms of some function of B .

Idea is: the true wavelet transform [ 0Ð4ß BÑ of our given function 0ÐBÑ is in > (i.e. has the same maxima as itself) and is in Z (i.e., in the collection of functions which are wavelet transforms).

Thus [0 −> Z. intuitive picture:

fig 48

Thus if we know just the maxima of [0Ð4ßBÑ , we can try to find [0Ð4ßBÑ

That is:

1. We know maxima of [0Ð4ßBÑ , so 2. know > œ[0Ð4ßBÑ all functions with same maxima as

97 3. Find [0Ð4ßBÑ as “unique” point in > which is also a wavelet transform, i.e., unique point in > Z :

Algorithm:

1. Start with only the maxima information about[Ð4ßBÑ . Call Q the maxima information.

2. Make initial guess using function 1Ð4ßBÑ" which has the same maxima as [Ð4ßBÑ.

3. Find closest function in Zœ set of wavelet transforms to 1Ð4ßBÑ" . Call this function 1Ð4ßBÑÞ#

4. Find closest function in > œQ functions with same maxima as to 1Ð4ßBÑ#$. Call this function 1Ð4ßBÑÞ

5. Find closest function in Z to 1$% Ð4ß BÑ ; call this 1 Ð4ß BÑ .

6. Find closest function in > to 11Þ%& ; call this

7. Continue this way: at each stage 411 find the closest function 44" to in the space Z or > (alternatingly).

Eventually the 1Ð4ßBÑÒ [0Ð4ßBÑ as desired. 4 4Ä_

98

CONCLUSION: We can recover the wavelet transform [0Ð4ßBÑ of a function just by knowing its maxima in B .

THE POINT: Compression. We can store the maxima of [0 using a lot less memory.

APPLICATION: Compression of images:

Fig. 49

99

Fig. 50

9. Wavelets and Wavelet Transforms in Two Dimensions

Multiresolution analysis and wavelets can be generalized to higher dimensions. Usual choice for a two-dimensional scaling function or wavelet is a product of two one-dimensional functions. For example,

999#ÐBß CÑ œ ÐBÑ ÐCÑ and scaling equation has form

99ÐBßCÑœ 2 †# Ð#B5ß#C6ÑÞ " 56 5ß6

Since 99ÐBÑ and ÐCÑ both satisfy the sclaing equation

99ÐBÑ œ 2 † # Ð#B  5Ñß " 5 È 5 we have 2œ22Þ56 5 6 Thus two dimensional scaling equation is product of two one dimensional scaling equations.

100 We can proceed analogously to construct wavelets using products of one-dimensional functions. However, unlike one-dimensional case, we have three rather than one basic wavelet. They are: <9<ÐMÑÐBß CÑ œ ÐBÑ ÐCÑ

<<9ÐMMÑÐBß CÑ œ ÐBÑ ÐCÑ

<<<ÐMMMÑÐBß CÑ œ ÐBÑ ÐCÑÞ The generalization of the one-dimensional wavelet equation leads to the following relations:

<9ÐMÑÐBßCÑœ 1ÐMÑ †# Ð#B5ß#C6Ñ " 56 5ß6

<9ÐMMÑÐBßCÑœ 1ÐMMÑ †# Ð#B5ß#C6Ñ " 56 5ß6

<9ÐMMMÑÐBßCÑœ 1ÐMMMÑ †# Ð#B5ß#C6Ñ " 56 5ß6

ÐMÑ ÐMMÑ ÐMMÑ where 1œ21ß1œ12ß5656 56 5 6and 1 56 œ11Þ 56

We can generate two-dimensional scaling functions and wavelets using the functions ScalingFunction and Wavelet then taking the product. For example, here we plot the Haar wavelets in two dimensions. Various translated and dilated versions of the wavelets can be plotted similarly.

101

Fig. 51: Two dimensional Haar scaling function 9ÐBß CÑ

Fig. 52: Haar wavelet <ÐMÑÐBß CÑ

102

Fig. 53: Second wavelet <<ÐMMÑÐBß CÑ œ ÐMÑ ÐCß BÑ

Fig. 54: Third wavelet <ÐMMMÑÐBß CÑ As example of another wavelet, here is so-called "least asymmetric wavelet" of order 8 in two dimensions À

103

Fig. 55: Least asymmetric wavelet of order 8

104