6 Splines and wavelets A fundamental aspect of our formulation is that the whitening operator L is nat- urally tied to some underlying B-spline function, which will play a crucial role in the sequel. The spline connection also provides a strong link with wavelets [UB03]. In this chapter, we review the foundations of spline theory and show how one can construct B-spline basis functions and wavelets that are tied to some specific oper- ator L. The chapter starts with a gentle introduction to wavelets that exploits the analogy with Legos blocks. This naturally leads to the formulation of a multiresolu- tion analysis of L2(R) using piecewise-constant functions and a de visu identification of Haar wavelets. We then proceed in Section 6.2 with a formal definition of our gen- eralized brand of splines—the cardinal L-splines—followed by a detailed discussion of the fundamental notion of Riesz basis. In Section 6.3, we systematically cover the first-order operators with the construction of exponential B-splines and wavelets, which have the convenient property of being orthogonal. We then address the the- ory in its full generality and present two generic methods for constructing B-spline basis functions (Section 6.4) and semi-orthogonal wavelets (Section 6.5). The pleas- ing aspect is that these results apply to the whole class of shift-invariant differential operators L whose null space is finite-dimensional (possibly trivial), which are pre- cisely those that can be safely inverted to specify sparse stochastic processes. 6.1 From Legos to wavelets It is instructive to get back to our introductory example of piecewise-contant splines in Chapter 1 (§1.3) and to show how these are naturally connected to wavelets. The fundamental idea in wavelet theory is to construct a series of fine-to-coarse approx- imations of a function s(r ) and to exploit the structure of the multiresolution approx- imation errors, which are orthogonal across scale. Here, we shall consider a series of approximating signals {si }i Z, where si is a piecewise-constant spline with knots 2 positioned on 2i Z. These multiresolution splines are represented by their B-spline expansion si (r ) ci [k]¡i,k (r ), (6.1) = k Z X2 6.1 From Legos to wavelets 113 where the B-spline basis functions (rectangles) are dilated versions of the cardinal ones by a factor of 2i r 2i k 1, for r 2i k,2i (k 1) ¡ (r ) Ø0 ° 2 + (6.2) i,k = i = 0, otherwise. + µ 2 ∂ ( £ ¢ The variable i is the scale index that specifies the resolution (or knot spacing) a 2i , = while the integer k encodes for the spatial location. The B-spline of degree 0, ¡ 0 = ¡0,0 Ø , is the scaling function of the representation. Interestingly, it is the iden- = + tification of a proper scaling function that constitutes the most fundamental step in the construction of a wavelet basis of L2(R). DEFINITION 6.1 (Scaling function) ¡ L (R) is a valid scaling function if and only 2 2 if it satisfies the following three properties: – Two-scale relation ¡(r /2) h[k]¡(r k), (6.3) = k Z ° X2 where the sequence h ` (Z) is the so-called refinement mask. 2 1 – Partition of unity ¡(r k) 1 (6.4) k Z ° = X2 – The set of functions {¡( k)}k Z forms a Riesz basis. · ° 2 In practice, a given brand of (orthogonal) wavelets (e.g., Daubechies or splines) is often summarized by its refinement filter h since the latter uniquely specifies ¡, subject to the admissibility constraints (6.4) and ¡ L (R). In the case of the B-spline 2 2 of degree 0, we have that h[k] ±[k] ±[k 1], where = + ° 1, for k 0 ±[k] = = ( 0, otherwise is the discrete Kronecker impulse. This translates into what we jokingly refer to as the Lego-Duplo relation 1 Ø0 (r /2) Ø0 (r ) Ø0 (r 1). (6.5) + = + + + ° The fact that Ø0 satisfies the partition of unity is obvious. Likewise, we already ob- + served in Chapter 1 that Ø0 generates an orthogonal system which is a special case + of a Riesz basis. By considering the rescaled version of such a basis, we specify the subspace of splines at scale i as Vi si (r ) ci [k]¡i,k (r ):ci `2(Z) L2(R) = ( = k Z 2 ) Ω X2 1. The Duplos are the larger-scale versions of the Lego building blocks and are more suitable for smal- ler children to play with. The main point of the analogy with wavelets is that Legos and Duplos are com- patible; they can be combined to build more complex shapes. The enabling property is that a Duplo is equivalent to two smaller Legos placed next to each other, as expressed by (6.5) 114 Splines and wavelets Wavelets: Haar transform revisited 4 s0(x) ψ(x) 3 Wavelet: 2 1 0 2 4 6 8 2 1 4 + s1(x) 3 2 4 6 8 -1 2 - -2 1 ri(x)=si−1(x) − si(x) 2 0 2 4 6 8 4 + 1 s2(x) 3 2 4 6 8 2 - -1 1 -2 2 0 2 4 6 8 4 + 1 s3(x) 3 2 4 6 8 2 - -1 1 -2 0 2 4 6 8 Figure 6.1 Multiresolution signal analysis using piecewise-constant splines with a dyadic scale progression. Left: multiresolution pyramid. Right: error signals between two successive levels of the pyramid. which, in our example, contains all the finite-energy functions that are piecewise- constant on the intervals 2i k,2i (k 1) with k Z. The two-scale relation (6.3) im- + 2 plies that the basis functions at scale i 1 are contained in V (the original space £ ¢= 0 of cardinal splines) and, by extension, in V for i 0. This translates into the gen- i ∑ eral inclusion property V V for any i 0 i, which is fundamental to the theory. i 0 Ω i > A subtler point is that the closure of i Z Vi is equal to L2(R), which follows from 2 the fact that any finite-energy function can be approximated arbitrarily well by a S piecewise-constant spline when the sampling step 2i tends to zero (i ). The !°1 necessary and sufficient condition for this asymptotic convergence is the partition of unity (6.4), which ensures that the representation is complete. Having set the notation and specified the underlying hierarchy of function spaces, we now proceed with the multiresolution approximation procedure starting from the fine-scale signal s (x), as illustrated in Figure 6.1. Given the sequence c [ ] of fine- 0 0 · scale coefficients, our task is to construct the best spline approximation at scale 1 which is specified by its B-spline coefficients c [ ] in (6.1) with i 1. It is easy to see 1 · = that the minimum-error solution (orthogonal projection of s0 onto V1) is obtained by taking the mean of two consecutive samples. The procedure is then repeated at the next coarser scale and so forth until one reaches the bottom of the pyramid, as shown on the left-hand side of Figure 6.1. The description of this coarsening algorithm is 1 1 ci [k] ci 1[2k] ci 1[2k 1] ci 1 h˜ [2k]. (6.6) = 2 ° + 2 ° + = ° § ° ¢ 6.1 From Legos to wavelets 115 It is run recursively for i 1,...,i where i denotes the bottom level of the pyr- = max max amid. The outcome is a multiresolution analysis of our input signal s0. In order to uncover the wavelets, it is enlightening to look at the residual signals ri (r ) si 1(r ) si (r ) Vi 1 on the right of Figure 6.1. While these are splines that = ° ° 2 ° live at the same resolution as si 1, they actually have half the apparent degrees of ° freedom. These error signals exhibit a striking sign-alternation pattern due to the fact that two consecutive samples (ci 1[2k],ci 1[2k 1]) are at an equal distance ° ° + from their mean value (ci [k]). This suggests rewriting the residuals more concisely in terms of oscillating basis functions (wavelets) at scale i, like ri (r ) si 1(r ) si (r ) di [k]√i,k (r ), (6.7) = ° ° = k Z X2 where the (non-normalized) Haar wavelets are given by r 2i k √ (r ) √ ° i,k = Haar i µ 2 ∂ with the Haar wavelet being defined by (1.19). The wavelet coefficients d [ ] are given i · by the consecutive half differences 1 1 di [k] ci 1[2k] ci 1[2k 1] ci 1 g˜ [2k]. (6.8) = 2 ° ° 2 ° + = ° § ° ¢ More generally, since the wavelet template at scale i 1, √ V , we can write = 1,0 2 0 √(r /2) g[k]¡(r k) (6.9) = k Z ° X2 which is the wavelet counterpart of the two-scale relation (6.3). In the present ex- ample, we have g[k] ( 1)k h[k], which is a general relation that is characteristic of = ° an orthogonal design. Likewise, in order to gain in generality, we have chosen to ex- press the decomposition algorithms (6.6) and (6.8) in terms of discrete convolution (filtering) and downsampling operations where the corresponding Haar analysis fil- ters are h˜[k] 1 h[ k] and g˜[k] 1 ( 1)k h[ k]. The Hilbert-space interpretation of = 2 ° = 2 ° ° this approximation process is that r W , where W is the orthogonal complement i 2 i i of Vi in Vi 1; that is, Vi 1 Vi Wi with Vi Wi (as a consequence of the orthogonal- ° ° = + ? projection theorem).