Non-orthogonal bases Although orthogonal bases have many useful properties, it is possi- ble (and sometimes desirable) to use a set of non-orthogonal basis functions for discretization. The main property we want from a basis is that the mapping from signal space to coefficient space is a stable bijection | each signal should have a different set of expansion coefficients, and each set of expansion coefficients should correspond to a different signal. Small changes in the expansion coefficients should not lead to large changes in the re-synthesized signal. We also want a concrete method for calculating the coefficients in a basis expansion. N We start our discussion in the familiar setting of R . N Bases in R N N Let 1;:::; N 2 R be a basis for R . We know from basic linear algebra that these vectors form a basis if and only if they are linearly N independent. For any x 2 R , we have N X x = αn n; (1) n=1 for some coefficient sequence 2 3 α1 6 α2 7 α = 6 . 7 : 4 . 5 αN How do we compute the αn? 149 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 The answer to this turns out to be straightforward as soon as we have everything written down the right way. We can write the de- composition in (1) as 2 3 2 3 j j j α1 6 7 6 α2 7 x = α1 + ··· + αN = 6 ··· 7 6 . 7 1 N 4 1 2 N 5 4 . 5 j j j αN = Ψα That is, the basis vectors are concatenated as columns in the N × N matrix Ψ. Since its columns are linearly independent, Ψ is invertible, and so we have the reproducing formula x = ΨΨ−1x N X = hx; e ni n; n=1 where e n is the nth row of the inverse of Ψ: 2 T 3 − e 1 − 6 T 7 −1 6− e −7 Ψ = 6 2 7 6 . 7 4 T 5 − e N − We have Transform: αn = hx; e ni; n = 1;:::;N; N X Inverse transform (synthesis): x = αn n n=1 150 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 So we compute the expansion coefficients by taking inner products against basis signals, just not the same basis signals as we are using to re-synthesize x. The e 1;:::; e N themselves are linearly indepen- dent, and are called the dual basis for 1;:::; N . Also note that while the f ng are not orthonormal and the f e ng are not orthonormal, jointly they obey the realtion ( 1; n = `; h ; e i = n ` 0; n =6 `: (This follows simply from the fact that ΨΨ−1 = I.) For this reason, f ng and f e ng are called biorthogonal bases. N Bases for subspaces of R N Suppose that T is a M-dimensional (M < N) subspace of R , and 1;:::; M 2 T is a basis for this space. Now 2 j j 3 6 7 Ψ = 6 ··· 7 4 1 M 5 j j is N ×M | it is not square, and so it is not invertible. But since the m are linearly independent, it does have a left inverse, and hence we can derive a reproducing formula. M Let x 2 T , so there exists an α 2 R such that x = Ψα. Then the reproducing formula is x = Ψ(ΨTΨ)−1ΨTx: 151 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 (The M × M matrix ΨTΨ is invertible by the linear independence of 1;:::; M .) To see that the formula above holds, simply plug x = Ψα into the expression on the right hand side. We can write M X x = hx; e mi m; m=1 where T −1 T e m = mth row of the M × N matrix (Ψ Ψ) Ψ : Notice that when M = N (and so Ψ is square and invertible), this agrees with the result in the previous section, as in this case (ΨTΨ)−1ΨT = Ψ−1(ΨT)−1ΨT = Ψ−1: Bases in finite dimensional spaces The construction of the dual basis (which tells us how to compute N the expansion coefficients for a basis) in R relied on manipulating matrices that contained the basis vectors. If our signals of interest are not length N vectors, but still live in a finite dimensional Hilbert space S, then we can proceed in a similar manner. Let 1(t); : : : ; N (t) be a basis for an N-dimensional inner product space S. Let x(t) 2 S be another arbitrary signal in this space. We know that the closest point to x in S is x itself, and from our work on the closest point problem, we know that we can write N X x(t) = αn n(t); n=1 152 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 where 2 3 2 3−1 2 3 α1 h 1; 1i h 2; 1i · · · h N ; 1i hx; 1i 6 α2 7 6 h 1; 2i h 2; 2i · · · h N ; 2i 7 6 hx; 2i 7 6 . 7 = 6 . 7 6 . 7 : 4 . 5 4 . .. 5 4 . 5 αN h 1; N i h 2; N i · · · h N ; N i hx; N i Use H to denote the inverse Gram matrix above. Then N * N + X X αn = Hn;`hx; `i = x; Hn;` ` : `=1 `=1 Thus N X x(t) = hx; e ni n(t); n=1 where N X en(t) = Hn;` `(t): `=1 153 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 Example: Let S be the space of all second-order polynomials on [0; 1]. Set 2 1(t) = 1; 2(t) = t; 3(t) = t : Then 2 1 1=2 1=33−1 2 9 −36 30 3 H = 41=2 1=3 1=45 = 4−36 192 −1805 ; 1=3 1=4 1=5 30 −180 180 and 2 2 2 e1(t) = 30t −36t+9; e2(t) = −180t +192t−36; e3(t) = 180t −180t+30 154 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 Non-orthogonal basis in infinite dimensions: Riesz Bases When S is infinite dimensional, we have to proceed with a little more caution. It is possible that we have a infinite set of vectors which are linearly independent and span S (after closure), but the representation is completely unstable. Recall that a (possibly infinite) set of vectors is called linearly inde- pendent if no finite subset is linearly dependent. Trouble can come when larger and larger sets are coming closer and closer to being linearly dependent. That is, if f n; 1 ≤ n ≤ 1g is a set of vectors, there might be no α2; : : : ; αL such that L X 1 = α` `; `=2 exactly, no matter how large L is. But there could be an infinite sequence fα`g such that L X lim k 1 − α` `k = 0: L!1 `=2 We will see an example of this below. Our definition of basis prevents sequences like the above from occur- ring. 1 1 Definition. We say f ngn=1 is a Riesz basis if there exists constants A; B > 0 such that 2 1 1 1 X 2 X X 2 A jαnj ≤ αn n ≤ B jαnj n=1 n=1 n=1 1This definition uses the natural numbers to index the set of basis functions, but of course it applies equally to any countably infinite sequence. 155 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 P1 2 uniformly for all sequences fαng with n=1 jαnj < 1. Note that if f ng is an orthonormal basis, then it is a Riesz basis with A = B = 1 (Parseval theorem). Our first example is something which is not a Riesz basis. Example: Multiscale Tent Functions Consider this set of continuous-time signals on [0; 1]. p p φ0(t) = 2(1 − t); φ1(t) = 2t; (p 3t; 0 ≤ t ≤ 1=2; (t) = p 0 3(1=2 − t); 1=2 ≤ t ≤ 1: j=2 j j j;n(t) = 2 0(2 t − n); j ≥ 1; n = 0;:::; 2 − 1: Sketch: From your sketch above, it should be clear that j Span(fφ0; φ1; 0; j;n; 1 ≤ j ≤ J; n = 0;:::; 2 − 1g) is the set of all continuous piecewise-linear functions on dyadic in- −J tervals of length 2 . Since this set is dense in L2([0; 1]), we can 156 Georgia Tech ECE 6250 Fall 2016; Notes by J. Romberg. Last updated 11:51, October 5, 2016 write X x(t) = b0φ0(t) + b1φ1(t) + cj;n j;n(t) j;n for some sequence of numbers fb0; b1; cj;ng. The problem is that this sequence of numbers might not be well-behaved. To see that this collection of functions cannot be a Riesz basis, notice that using the functions with 1 ≤ j ≤ J, we can match the samples of any function on the grid with spacing 2−J , with linear interpola- tion in between these samples. From this, we see that φ0(t) can be matched exactly on the interval [2−J−1; 1] with a linear combination of j;n; 0 ≤ j ≤ J. This means that there is a sequence of numbers fβj;ng such that 2j−1 X X φ0(t) = βj;n j;n(t): j≥0 n=0 This means that the non-zero sequence of numbers f1; 0; βj;n; j ≥ 0; n = 0;:::; 2j − 1g synthesizes the 0 signal, thus violating the condition that A > 0 uniformly.