Sparse Sets in Time and Frequency with Diophantine Problems and Integrable Systems D. V. Chudnovsky, G.V. Chudnovsky, T. Morgan IMAS, Polytechnic University 6 MetroTech Center Brooklyn, NY 11201 Overview of sparseness in space and fre- quency • Space and frequency domains - setting and motivation • Reconstruction and uncertainty in sparse signals • Ill conditioning–effects on reconstruction • Diophantine meaning of ill conditioning • Band limiting and related integrable systems • Combinatorial description of singular sparse sets The spaces over which we operate In this introduction, we give examples of the duality between conventional spatial and frequency representations of ‘sparse objects’, signals sparse in both space and frequency representations. We have a space of signals, and two basis representing signals. One basis consists of confined functions (delta functions),representing atoms of space and a second basis,consisiting of widely distributed functions (sines and cosines), having unbounded support, and the operator, , mapping between the basis. F Then we consider the behavior of the operator, , and the consequences of projecting F signals onto subspaces simultaneously in space and in frequency. The examples that follow will focus on Fourier representations; however other basis sharing the property of confined/unconfined support can be used as well For example, the standard basis can be paired with a polynomial basis. The Fourier matrix 00 10 (n 1)0 ζn ζn ... ζn − 0 01 11 1 1 ζn ζn ... ... n = √ B C F n B ... ... ... ... C B C B 0(n 1) 1(n 1) (n 1)(n 1) C B ζn − ζn − ... ζn − − C @ A 11 1 1 1 1 1 0 2iπ 4iπ 6iπ 6iπ 4iπ 2iπ 1 1 e 7 e 7 e 7 e− 7 e− 7 e− 7 B 4iπ 6iπ 2iπ 2iπ 6iπ 4iπ C B 1 e 7 e− 7 e− 7 e 7 e 7 e− 7 C 1 B 6iπ 2iπ 4iπ 4iπ 2iπ 6iπ C B 7 7 7 7 7 7 C √7 B 1 e e− e e− e e− C B 6iπ 2iπ 4iπ 4iπ 2iπ 6iπ C B C B 1 e− 7 e 7 e− 7 e 7 e− 7 e 7 C B 4iπ 6iπ 2iπ 2iπ 6iπ 4iπ C B 1 e 7 e 7 e 7 e 7 e 7 e 7 C B − − − C B 2iπ 4iπ 6iπ 6iπ 4iπ 2iπ C B 1 e− 7 e− 7 e− 7 e 7 e 7 e 7 C @ A The world (of human kind) is full of band limited signals F SF ←→ Sδ RW ↓ ↓ PT is in here: ? W ↔ T In the block diagram of a system, it’s not at all unusual for an early on block to be a band pass filter Further, there’s an enormous ’filter’ at any system’s outputs – namely, what’s interesting to the observer? Some examples of signals and their representations 500 400 ' Ordinary' images F 300 512 x512 have most energy ® 200 near zero 100 0 0 100 200 300 400 500 500 400 300 F512 x512 ¾¾¾¾¾¾¾¾¾¾® 200 100 0 0 100 200 300 400 500 500 400 F 300 One of a few ways to 51 x512 produce high ¾¾¾¾¾¾¾¾¾¾® 200 frequencies 100 0 0 100 200 300 400 500 Reconstruction/concentration S F S F ←→ δ R P W ↓ ↓ T ? W T ↔ • If the Fourier operator limited to the W, T subspaces is invertible, then signals can be reconstructed from partial frequency data. • If the Fourier operator limited to the W, T subspaces is *not* invertible, then it has a null space Vectors in this null space are interesting, since they exhibit concentration in both space (T) and frequency ( S W ) These are curious vectors, ones which are F \ running up against uncertainty principle limits. A visual example of reconstruction RW F ¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾® SF ←→ Sδ PT RW ↓ ↓ PT ? Reconstruct W ↔ T ¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾¾® The notation for the operators of in- terest, and what are their actions S F S F ←→ δ R P W ↓ ↓ T ? W T ↔ RW operator which projects to W W a subspace of frequencies − PT operator which projects to T T a subspace of spatial points − RWT the composition of RW PT The action of RWT , concretely, is straighforward: 00 10 (n 1)0 ζn ζn ... ζn − 0 01 11 1 1 ζn ζn ... ... n = B C F √n B ... ... ... ... C B C B 0(n 1) 1(n 1) (n 1)(n 1) C B ζn − ζn − ... ζn − − C @ A Non zero deteriminant of certain sub- matrices of the Fourier matrix A little more about the case with invertibility This opens an interesting possibility. For example, you might be able to quickly recover spatial data, if you know that the spatial data was of limited support (but not where it was) For this to work, more than ’non - zero’ is required to be known about the determinant of the the Fourier submatrices Some estimates of the conditioning of the Fourier submatrices are needed Statement of Chebotarev’s Theorem Chebotarev found a modular reduction of the value of the determinant of matrices of the form: a1x1 a1x2 a1xk ζp ζp ... ζp 0 a2x1 a2x2 a2xk 1 ζp ζp ... ζp det B C = B ... ... ... ... C B C B akx1 akx2 akxk C B ζp ζp ... ζp C @ A k(k−1) k(k−1) (xi xj ) (ai aj ) +1 Q − Q − 2 2 1!2!...(k 1)! π mod π − „ « Where π =(ζp 1), p is a prime and a and x are integers − i j In outline the proof proceeds by expanding the matrix entries in a series for the formal variable π , gathering the coefficients of the like terms of π and finding values for the coefficients for various powers of π in the expansion of the determinant. More about the reconstruction opera- tion S F S F ←→ δ R P W ↓ ↓ T ? W T ↔ Reconstruction consists of inverting the operator RWT on the subspace W We look at a signal, s, that is band limited to W (that is,the Fourier coefficients of S are zero outside of W). We want to reconstruct the signal, s everywhere, from it’s values on a subset T. For this to succeed, T must be at least as great as W . | | | | As a practical matter, the more observations points available in T, the better the reconstruction. For convenience, instead of reconstructing s, we reconstruct f= (s) ,knowing that f is F non zero only on W and recover s from f. 1 Writing explicitly the identity: s = − (s) 1 ( u x) F we have: − · √n f(u) ζn = s(x) x ǫ T u W · P∈ the sum may be restricted to W because f = (s) is zero outside W. In matrix notation, F this expression becomes: RWT f = s The pseudo-inverse and it’s applica- tion to recontruction Given a matrix,the Moore-Penrose generalized matrix inverse is a unique matrix pseudoinverse.This matrix was independently defined by Moore in 1920 and Penrose (1955),and variously known as the generalized inverse, pseudoinverse,or Moore-Penrose inverse. The psuedo inverse for a system A x = y : apply AH to both sides: AHA x = AH y AHA is square; invert it to find: 1 x = AH A − AH y The` psuedo´ inverse produces a useful result, even with inconsistent data; it provides a minimal L2 fit to the system of equations. For our system of equations, reconstruction depends on the conditioning of the matrix W x W matrix, Q = RWT RWTH t(u v) | | | | ( Q)u,v= tǫT ζn − for (u,v) in W The largerPT , the better conditioned the matrix Q. In particular, for T = n, Q isthe | | | | identity matrix of order W . | | The engineers point of view, part 1 The engineer imagines ’n’ as the discretization scale. The extent to which the number theoretic properties of ’n’ are involved is unsatisfactory, and surprising. Further ’non-singularity’ isn’t good enough. For practical reconstruction, well conditioning of the matrix Q is necessary. For these purposes, matrix conditioning is measured in a variety of ways 1.The size of the determinant 2.The size of the smallest eigenvalue 3.Ratio of the largest to the smallest eigenvalue 4.The size of the Frobenius norm of the inverted matrix 3 is a standard measure of matrix conditioning; 4 has an immediate consequence for the processing of quantized values Related by inequalities, any of these means provides an indication of reconstruction problems The distribution of the eigenvalues of the ma- trix Q shows extraordinary outliers Numerical evidence - subset pairs for n=23,d1=5 ,d2=6 Sidebar - on programming to gather numerical evidence • Problems amount to exhaustive search of all (most) subsets • Processing of equivalence classes to extend the search space • Optimizing programming execution time vs total time to result • Compiled versus interpretitive environments + very basic parallelism And then – How do you look at a billion numbers? A bound on det Q by means of Cheb- otarev’s theorem a1x1 a1x2 a1xk ζp ζp ... ζp 0 a2x1 a2x2 a2xk 1 ζp ζp ... ζp det B C = B ... ... ... ... C B C B akx1 akx2 akxk C B ζp ζp ... ζp C @ A k(k−1) k(k−1) (xi xj ) (ai aj ) +1 Q − Q − 2 2 1!2!...(k 1)! π modπ − „ « By Chebotarev’s theorm,the determinant of = w t CW,T ζn · w W,t T ∈ ∈ is non zero` for´ n a prime with W = T . We have: detQ = detC 2 | | | | | W,T | | W,T | How small can this determinant be for a given W and T and n a prime? (n 3)/2 It can be shown that: det C nd(d 1)/4 dd/2 − − W,T ≥ − · ˛ ` ´˛ ` ´ ˛ ˛ The engineers point of view, part 2 (n 3)/2 det C nd(d 1)/4 dd/2 − − W,T ≥ − · When˛ ` d is small´˛ and n is large,` this´ bound is exponential in n.