Wavelet Transforms in Time Series Analysis
Total Page:16
File Type:pdf, Size:1020Kb
Wavelet Transforms in Time Series Analysis Andrew Tangborn Global Modeling and Assimilation Office, Goddard Space Flight Center [email protected] 301-614-6178 Outline 1. Fourier Transforms. 2. What is a Wavelet? 3. Continuous and Discrete Wavelet Transforms 4. Construction of Wavelets through dilation equations. 5. Example - Haar wavelets 6. Daubechies Compactly Supported wavelets. 7. Data compression, efficient representation. 8. Soft Thresholding. 9. Continuous Transform - Morlet Wavelet 10. Applications to approximating error correlations Fourier Transforms A good way to understand how wavelets work and why they are useful is by • comparing them with Fourier Transforms. The Fourier Transform converts a time series into the frequency domain: • Continuous Transform of a function f(x): ∞ iωx fˆ(ω) = f(x)e− dx Z −∞ where fˆ(ω) represents the strength of the function at frequency ω, where ω is continuous. Discrete Transform of a function f(x): ∞ ikx fˆ(k) = f(x)e− dx Z −∞ where k is a discrete number. for discrete data f(xj), j =1,...,N N ( i2π(k 1)(j 1)/N) fˆk = fje − − − jX=1 The Fast Fourier Transform (FFT) is o(NlogN) operations. • Example: Single Frequency Signal f(t) = sin(2πt) 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 0 1 2 3 4 5 6 7 8 9 10 Signal 0.5 Real Coefficient Imaginary Coefficient 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5 0 1 2 3 4 5 6 7 8 9 10 Fourier Transform Discrete Fourier Transform (DFT) locates the single frequency and re- • flection. Complex expansion is not exactly sine series - causes some spread. • Two Frequency Signal f(t) = sin(2πt)+ 2sin(4πt) 3 2 1 0 −1 −2 −3 0 1 2 3 4 5 6 7 8 9 10 Signal 1 Real Coefficients Imaginary Coefficients 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 0 1 2 3 4 5 6 7 8 9 10 Fourier Transform Both signal frequencies are represented by the Fourier Coefficients. • Intermittent Signal f(t) = sin(2πt)+ 2sin(4πt) when .37 <t<.55 f(t) = sin(2πt) otherwise 3 2 1 0 −1 −2 −3 0 1 2 3 4 5 6 7 8 9 10 Signal 0.5 Real coefficients Imaginary Coefficients 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 −0.5 0 1 2 3 4 5 6 7 8 9 10 Fourier Transform First frequency is found, but higher intermittent frequency appears as • many frequencies, and is not clearly identified. What is a Wavelet? A function that is localized in time and frequency, generally with a zero • mean. It is also a tool for decomposing a signal by location and frequency. • Consider the Fourier transform: A signal is only decomposed into its frequency components. No information is extracted about location and time. What happens when applying a Fourier transform to a signal that has • a time varying frequency? 1 0.8 0.6 0.4 0.2 0 −0.2 −0.4 −0.6 −0.8 −1 0 2 4 6 8 10 12 14 16 18 20 The Fourier transform will only give some information on which frequencies are present, but will give no information on when they occur. Schematic Representation of Decomposition Original Signal Frequency −> Time −> The signal is represented by an amplitude that is changing in time. • There is no explicit information on frequency. • Fourier Transform Frequency −> Time −> The Fourier transform results in a representation that depends only on frequency. • It gives no information on time. • Wavelet Transform Frequency −> Time −> The wavelet transform contains information on both the time location and fre- • quency of a signal. Some typical (but not required) properties of wavelets Orthogonality - Both wavelet transform matrix and wavelet functions can be • orthogonal. Useful for creating basis functions for computation. Zero Mean (admissibility condition) - Forces wavelet functions to wiggle (oscillate • between positive and negative). Compact Support - Efficient at representing localized data and functions. • How are Wavelets Defined? Families of basis functions that are based on • (1) dilations: ψ(x) ψ(2x) → (2) translations: ψ(x) ψ(x + 1) → of a given general ”mother wavelet” ψ(x). How do we use ψ(x)? • The general form is: ψj (x)=2j/2ψ(2jx k) k − where j: dilation index k: translation index 2k/2 needed for normalization How do we get ψ(x)? • Dilation Equations Construction of Wavelets We consider here only orthogonally/compactly supported • wavelets - Orthogonality means: ∞ j j′ ψk(x)ψk (x)dx = δkk δjj Z ′ ′ ′ −∞ Wavelets are constructed from scaling functions, φ(x) : • φ(x) come from the dilation equation: φ(x) = ckφ(2x k) Xk − ck: Finite set of filter coefficients General features: • - Fewer non-zero ck’s mean more compact and less smooth functions - More non-zero ck’s mean less compact and more smooth functions Restrictions on the Filter Coefficients Normality: • ∞ φ(x)dx =1 Z −∞ ∞ ckφ(2x k)dx =1 Z Xk − −∞ ∞ ck φ(2x k)dx =1 Xk Z − −∞ ∞ 1 ck φ(2x k)d(2x k)=1 Xk Z 2 − − −∞ ∞ 1 ck φ(ξ)d(ξ)=1 Xk Z 2 −∞ 1 ck( )(1) = 1 Xk 2 or ck =2 Xk Simple Examples • - Smallest number of c is 1: just get φ(x) = δ, zero support. k → - Haar Scaling Function: c0 = 1, c1 = 1. Haar Scaling Function The scaling function equation is: φ(x) = φ(2x) + φ(2x 1) − The only function that satisfies this is: φ(x)=1if0 x 1 ≤ ≤ φ(x) = 0 otherwise 1.5 1 y 0.5 0 −0.5 −1 −0.5 0 0.5 1 1.5 2 x Translation and Dilation of φ(x): • φ(2x)=1if0 x 1/2 ≤ ≤ φ(2x) = 0 elsewhere φ(2x 1)=1if1/2 x 1 − ≤ ≤ φ(2x 1) = 0 elsewhere − so the sum of the two functions is then: 1.5 1 y 0.5 0 −0.5 −1 −0.5 0 0.5 1 1.5 2 x Haar Wavelet Wavelets are constructed by taking differences of scaling • functions k ψ(x) = ( 1) c1 kφ(2x k) Xk − − − differencing is caused by the ( 1)k: − so the basic Haar wavelet is: 1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 and the family comes from dilating and translating: ψj (x)=2j/2ψ(2jx k) k − so that the j = 1 wavelets are: 1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 Orthogonality of Haar wavelets Translation no overlap • ⇒ 1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 Dilation cancellation • ⇒ 1.5 1 0.5 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 2 Daubechies’ Compactly Supported Wavelets The following plots are wavelets created using larger numbers of filter coefficients, but having all the properties of orthogonal wavelets. For example, D4 has coefficients: 1 1 1 1 ck = (1 + √3), (3 + √3), (3 √3), (1 √3) 4 4 4 − 4 − (Reference: Daubechies, 1988) 0.5 0.4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 0 20 40 60 80 100 120 D4 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 0 20 40 60 80 100 120 D12 0.3 0.2 0.1 0 −0.1 −0.2 −0.3 −0.4 0 20 40 60 80 100 120 D20 Continuous and Discrete Wavelet Transforms Continuous wav 1/2 t 1 (T f)(a,b) = a − dtf(t)ψ( − ) | | Z b where a: translation parameter b: dilation parameter Discrete wav m/2 m Tm,n (f) = ao− dtf(t)ψ(ao− t nbo) Z − m: dilation parameter n: translation parameter ao,bo depend on the wavelet used Fast Wavelet Transform (Reference: S. Mallat, 1989) Uses the discrete data: • f0 f1 f2 f3 f4 f5 f6 f7 h i Pyramid Algorithm o(N) !! - Start at finest scale and calculate differences • ⇒ j=0 a b 0,0 0,0 a j=1 1,0 b a b 1,0 1,1 1,1 j=2 a b a a 2,0 2,0 2,1 b 2,2 b a b 2,1 2,2 2,3 2,3 f f f f f f f f 1 2 3 4 5 6 7 8 and averages - Use Averages at next coarser scale to get new set of differences (bj,k) and averages (aj,k) And the coefficients are simply the differences (bj,k) and the average for the coarsest scale (a0,0): a0,0 b0,0 b1,0 b1,1 b2,0 b2,1 b2,2 b2,3 h i For Haar Wavelet: aj−1,k = c0aj,k + c1aj,k+1 bj−1,k = c0aj,k c1aj,k+1 − - The differences are the coefficients at each scale. Averages used for next scale. How do we store all this? (eg, Numerical Recipes algorithm). • Intermittent Signal - Wavelet Transform 3 2 1 0 −1 −2 −3 0 1 2 3 4 5 6 7 8 9 10 Signal 4 3 2 1 0 −1 j=3 j=4 −2 −3 j=5 −4 j=2 −5 0 20 40 60 80 100 120 Higher Frequency intermittent signal shows up in the j =5, 6 scales, and only in the • middle portion of the domain.