AN INTRODUCTION TO WAVELETS or: THE WAVELET TRANSFORM: WHAT'S IN IT FOR YOU? Andrew E. Yagle and Byung-Jae Kwak Dept. of Electrical Engineering and Computer Science The University of Michigan, Ann Arbor MI Presentation to Ford Motor Co. May 21, 1996 °c 1996 by Andrew E. Yagle 1 OUTLINE OF PRESENTATION 1. Signal Representation using Orthonormal Bases 1.1 De¯nitions and Properties 1.2 Example: Fourier Series 1.3 Example: Bandlimited Signals 1.4 Example: Wavelet Transform 2. Multiresolution Analysis 2.1 Multiresolution Subspaces 2.2 Wavelet Scaling Functions 2.3 Wavelet Basis Functions 2.4 Summary of Wavelet Design 3. Wavelet Transforms 3.1 Haar Function!Haar Transform 3.2 Sinc Function!LP Wavelet 3.3 Splines!Battle-Lemarie 3.4 General Properties 3.5 2-D Wavelet Transform 4. Applications of Wavelets 4.1 Sparsi¯cation of Operators 4.2 Compression of Signals 4.3 Localized Denoising 4.4 Tomography under Wavelet Constraints 2 1. SIGNAL REP. USING ORTHONORMAL BASES 1.1 De¯nitions 2 DEF: The (L ) inner product of x1(t) and x2(t) is Z 1 ¤ < x1(t); x2(t) >= x1(t)x2(t)dt ¡1 DEF: The (L2) norm of x(t) is s p Z 1 jjx(t)jj = < x(t); x(t) > = x(t)x¤(t)dt ¡1 DEF: x(t) 2 L2 if jjx(t)jj < 1 DEF: xi(t) and xj(t) are orthonormal if ½ 1; if i = j; < x (t); x (t) >= ± = i j i¡j 0; if i 6= j DEF: The space spanned by basis funcs fÁi(t)g P SP ANfÁi(t)g = f i ciÁi(t)g for any constants ci DEF: A set of basis functions is complete if any L2 function can be represented as a linear combination of the basis functions. 3 1.1 Properties Let fÁi(t)g be a complete orthonormal basis set. Then for any L2 signal x(t) we have: X1 x(t) = xiÁi(t) expansion i=¡1 Z 1 ¤ xi = x(t)Ái (t)dt orthonormal ¡1 XN xN (t) = xiÁi(t) ! i=¡N lim jjx(t) ¡ xN (t)jj = 0 complete N!1 1. Instead of processing the uncountably in¯nite x(t), we may process the countably in¯nite xi; 2. We may process the FINITE fxi; jij · Ng with arbitrarily small error if N su±ciently large; 3. xN (t) is the best approximation to x(t) using only 2N + 1 xi; 4. Each xi carries di®erent information about x(t){ no redundancy. 5. Easy to compute xi and update xN (t). 4 1.2 Example: Fourier Series Let x(t) be de¯ned for 0 < t < P (periodic?) Then x(t) has the orthonormal expansion X1 j2¼nt=P x(t) = xne n=¡1 Z P 1 ¡j2¼nt=P xn = x(t)e P 0 1. Convergence in L2 norm; Gibbs phenomenon at discontinuities; 2. Note basis functions orthogonal in SCALE. Approximation of square wave using N = 15: (uncountably in¯nite ! 15) 5 1.3 Example: Bandlimited Signals Let x(t) be bandlimited to j!j < B. Then x(t) has the orthonormal expansion X1 sin B(t ¡ n¢) ¼ x(t) = x ; ¢ = n B(t ¡ n¢) B n=¡1 Z 1 sin B(t ¡ n¢) 1 xn = x(t) dt = x(t = n¢) ¡1 B(t ¡ n¢) ¢ 1. Since x(t) and sinc(t) are both bandlimited x(t)¤sinc(t) = x(t); sinc(t)¤sinc(t) = sinc(t) Then sample; 2. Here xn happen to be sampled values of x(t); 3. Basis functions orthogonal in TRANSLATION: Z 1 sin B(t ¡ i¢) sin B(t ¡ j¢) dt = ¢±i¡j ¡1 B(t ¡ i¢) B(t ¡ j¢) 6 1.4 Example: Wavelet Transform Let x(t) 2 L2. Then x(t) has the orthonormal expansion X1 X1 i ¡i=2 ¡i x(t) = xj2 Ã(2 t ¡ j) i=¡1 j=¡1 Z 1 i ¡i=2 ¡i xj = x(t)2 Ã(2 t ¡ j) ¡1 i 1. Note double-indexed xj; 2. Basis functions areorthogonal in both SCALE AND TRANSLATION; 3. Therefore localized in time and frequency; 4. Note scale-dependent shift in t: 2ij 5. Need 2¡i=2 to make jj2¡i=2Ã(2¡it ¡ j)jj = 1. How do we ¯nd such basis functions? 7 2. MULTIRESOLUTION ANALYSIS 2.1 Multiresolution Subspaces DEF: A multiresolution analysis is a sequence of closed subspaces Vi such that: 2 1. f0g ½ ::: ½ V2 ½ V1 ½ V0 ½ V¡1 ½ ::: ½ L i 2. x(t) 2 Vi , x(2 t) 2 V0 3. x(t) 2 V0 , x(t ¡ j) 2 V0 4. 9orthonormal basis so V0 = SP ANfÁ(t ¡ j)g ¡i=2 ¡i Latter implies Vi = SP ANf2 Á(2 t ¡ j)g Example: Piecewise constant functions 1. Approximate function by piecewise constant steps 2. Smaller steps ! more accurate approximation i 3. Step size 2 ! project onto Vi (want small i) 8 2.2 Wavelet Scaling Functions ¡i=2 ¡i Vi = SP ANf2 Á(2 t ¡ j)g Á(t) is the wavelet scaling function. Á(t) 2 V0 ½ V¡1 ! Á(t) 2 V¡1 X1 1=2 ! Á(t) = gn2 Á(2t ¡ n) n=¡1 j! Taking Fourier transform and G(e ) = DT F T [gn] ! (for MEs: set backshift q = ej!) 1 ©(!) = p G(ej!=2)©(!=2) 2 Orthonormality of fÁ(t ¡ j)g ! jjG(ej!)jj2 + jjG(ej(!+¼))jj2 = 2 Suggests gn should be lowpass ¯lter. 9 2.3 Wavelet Basis Functions De¯ne fWig as orthogonal complements of fVig: 1. Wi=orthogonalL complementL of Vi in Vi¡1 2. Vi¡1 = Vi Wi, =direct sum M M M M 2 ::: W1 W0 W¡1 ::: = L i 1. Still have x(t) 2 Wi , x(2 t) 2 W0 2. Need basis Ã(t) so W0 = SP ANfÃ(t ¡ j)g 3. Compare the above to V0 = SP ANfÁ(t ¡ j)g Ã(t) 2 W0 ½ V¡1 ! Ã(t) 2 V¡1 X1 1=2 ! Ã(t) = hn2 Á(2t ¡ n) n=¡1 j! Taking Fourier transform and H(e ) = DT F T [hn] ! 1 ª(!) = p H(ej!=2)©(!=2) 2 Ã(t) 2 W0 ? V0 ! Ã(t) ? fÁ(t ¡ j)g ! H(ej!)G¤(ej!) + H(ej(!+¼))G¤(ej(!+¼)) = 0 10 2.3 Wavelet Basis Functions, continued n This leads to hn = g1¡n(¡1) 1. Now have Ã(t) in terms of Á(t) ¡i=2 ¡i 2. Wi = SP ANf2 Ã(2 t ¡ j)g 3. But fWig (unlike fVig) are orthogonal spaces 4. So f2¡i=2Ã(2¡it¡j)g is a complete orthonormal basis for L2. Note that gn is a lowpass ¯lter while n j! ¡j! ¤ j(!+¼) hn = g1¡n(¡1) ! H(e ) = ¡e G (e ) is a bandpass ¯lter. This shows the distinction between 1. wavelet scaling (Á(t)) and 2. wavelet basis (Ã(t)) functions: Scaled Ã(t) sweep out di®erent frequency bands. 11 2.4 Summary of Wavelet Design 1. Find scaling function Á(t), often by iterating 2-scale eqn X1 1=2 Á(t) = gn2 Á(2t ¡ n) n=¡1 a. Start with Á(t)=LPF b. Repeatedly convolve and upsample by 2 c. Converges if gn regular ¯lter: Q1 j!=2k k=1 G(e ) converges d. Á(t)=cont. limit discrete ¯lter gn: upsampling spreads spectrum so that it is periodic with 2k¼, k=iteration # 2. Find wavelet function Ã(t): X1 1=2 Ã(t) = hn2 Á(2t ¡ n) n=¡1 1 ª(!) = p H(ej!=2)©(!=2) 2 H(ej!) = ¡e¡j!G¤(ej(!+¼)) 12 3. WAVELET TRANSFORMS 3.1 Haar Function!Haar Transform 1. Vi = fpiecewise constant functionsg 2. x(t) changes levels at t = j2i 3. SMALLER i ! better resolution Consider DECREASE in resolution: In going from Vi to coarser Vi+1 ½ Vi: 1. Replace x(t) over the two intervals [2ij; 2i(j + 1)] and [2i(j + 1); 2i(j + 2)] 2. with x(t) over interval [2i+1(j=2); 2i+1(j=2 + 1)] i+1 i+1 3. Note that [2 (j=S2); 2 (j=2 + 1)] = [2ij; 2i(j + 1)] [2i(j + 1); 2i(j + 2)] How do we do this? Replace x(t) with its average (which is its projection onto Vi+1) and with its di®erence (which is its projection onto Wi+1) over the longer interval. 13 3.1 Haar Function!Haar Transform: Example x(t) = sin(t) Projection onto Vi: ½ sin(2ij); if 2ij < t < 2i(j + 1); x (t) = i sin(2i(j + 1)); if 2i(j + 1) < t < 2i(j + 2); Projection onto Vi+1: i i xi+1(t) = [sin(2 j) + sin(2 (j + 1))]=2, if 2i+1(j=2) < t < 2i+1(j=2 + 1) Same interval; replace 2 values with average. Projection onto Wi+1: 0 i i xi+1(t) = [sin(2 j) + sin(2 (j + 1))]=2, if 2i+1(j=2) < t < 2i+1(j=2 + 1) Replace 2 values with di®erence. 14 3.1 Haar Function!Haar Transform: Bases ½ 1; if 0 < t < 1; Á(t) = 0; otherwise. Á(t) = Á(2t) + Á(2t ¡ 1) ! g0 = g1 = 1 n hn = g1¡n(¡1) ! h0 = 1; h1 = ¡1 ! 8 < 1; if 0 < t < 1=2; Ã(t) = Á(2t) ¡ Á(2t ¡ 1) = ¡1; if 1=2 < t < 1; : 0; otherwise. Scaling function Á(t) Basis function Ã(t) 15 3.2 Sinc Function!LP Wavelet 1. V0 = ffunctions bandlimited to [¡¼; ¼] i i 2. V¡i = ffunctions bandlimited to [¡2 ¼; 2 ¼] 3.