The Generalized Annihilation Property:A Tool for Solving Finite Rate of Innovation Problems Thierry Blu

The Generalized Annihilation Property:A Tool For Solving Finite Rate of Innovation Problems Thierry Blu

To cite this version:

Thierry Blu. The Generalized Annihilation Property:A Tool For Solving Finite Rate of Innovation Problems. SAMPTA’09, May 2009, Marseille, France. Special Session on sampling using finite rate of innovation principles. hal-00452220

HAL Id: hal-00452220 https://hal.archives-ouvertes.fr/hal-00452220 Submitted on 1 Feb 2010

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. The Generalized Annihilation Property A Tool For Solving Finite Rate of Innovation Problems

Thierry Blu

The Chinese University of Hong Kong, Shatin N.T., Hong Kong [email protected]

Abstract: At first sight, solving such a nonlinear system of equa- We describe a property satisfied by a class of nonlinear tions is a daunting task. Fortunately, if the matrix F(Ω) systems of equations that are of the form F(Ω)X = Y. satisfies a property that we shall call “Generalized Anni- Here F(Ω) is a matrix that depends on an unknown K- hilation Property” (GAP), this reduces to solving two lin- dimensional vector Ω, X is an unknown K-dimensional ear systems of equations sandwiching a nonlinear step that vector and Y is a vector of N ≥ K) given measure- amounts to polynomial root extraction in practical cases. ments. Such equations are encountered in superresolution The filters ϕ(t) that satisfy the GAP are thus especially in- or sparse signal recovery problems known as “Finite Rate teresting, since the related FRI problems enjoy a straight of Innovation” signal reconstruction. non-iterative solution. We show how this property allows to solve explicitly for the unknowns Ω and X by a direct, non-iterative, algo- 2. The Generalized Annihilation Property rithm that involves the resolution of two linear systems of (GAP) equations and the extraction of the roots of a polynomial and give examples of problems where this type of solu- We carry on with the previously identified general nonlin- tions has been found useful. ear problem, namely

F(Ω) X = Y, (3) 1. Introduction

where the unknowns are Ω = [ω1,ω2,...ωK ] and X = We consider the signal resulting from the convolution be- [x1, x2,...xK ], and where the measurements are Y = tween a window ϕ(t) andthe sum of K Diracs with ampli- [y1,y2,...yN ]. tude xk located at time tk. Given the N uniform samples This system is said to satisfy the Generalized Annihilation yn (T = sampling step ) Property whenever there exist K +1 constant matrices,

K Ak, and K +1 scalar functions of Ω, hk(Ω), such that we y = x ϕ(nT −t ) where n =1, 2,...,N, (1) n X k k have the identity k=1 K then FRI problems (see [1, 2]) consist in retrieving the pa- h (Ω) A F(Ω) = 0. (4) X k k rameters tk and xk. Solving such problems is conceptu- k=0 ally interesting because it shows how to break the standard for any vector of parameters Ω. By right multiplying with Nyquist-Shannon bandlimitation rule for the exact recon- X, the above equation implies that any solution Ω of (3) is struction of signals from their uniform samples [3]. also a solution of the (generalized) annihilation equation The system of consistent equations (1) can be expressed under the generic form of a nonlinear problem as shown K h (Ω)A Y=0. (5) in Fig. 1 (see next page), where the parameters Ω = X k k k=0 [ω1,ω2,...ωK ] are related unambiguously to the unknowns tk’s. Because of the variety of settings adapted This equation can be expressed in a matrix form AH=0 T to this general approach, it happens to be necessary to dis- where the unknown is H = [ h0(Ω),h1(Ω),...,hK (Ω) ] tinguish between the parameters ω —which we shall call and the matrix A = A Y, A Y,..., A Y . Thus, in k 0 1 K “abstract parameters”—and the locations tk: typically, the order to solve (3) for Ω and X, the idea consists in finding ωk’s will be the zeros of some polynomial and from these the scalar coefficients hk(Ω) that satisfy (5), then retriev- ωk’s, we will be able to retrieve the tk’s using a functional ing ω1,ω2,...,ωK from the knowledge of hk(Ω), and fi- relation of the form ωk = λ(tk) for some invertible func- nally finding X such that F(Ω) X = Y. Without elabo- tion λ(t). rating on the conditions that make this solution unique, a ϕ(T − t1) ϕ(T − t2) · · · ϕ(T − tK ) x1 y1       ϕ(2T − t1) ϕ(2T − t2) · · · ϕ(2T − tK ) x2 y2  . . .   .  =  .  (2)  . . .   .   .         ϕ(NT − t1) ϕ(NT − t2) · · · ϕ(NT − tK )   xK   yN 

F | {z(Ω) } | {zX } | {zY }

Figure 1: Algebraic equivalent of the consistency equations (1).

minimal requirement is that the matrices Ak have at least 3. Some GAP Kernels K rows.

In the simple case where the hk(Ω)’s are related to the The GAP is actually shared by many interesting filters ωk’s through a polynomial relation that can be used in sampling schemes, resulting in eas- ily solvable FRI problems. Among them, the first ones K K −k −1 to be identified were the periodized sinc, the infinite (i-e., hk(Ω)z = (1 − ωkz ), (6) X Y not periodized) sinc and the Gaussian kernels [1]. Even k=0 k=1 more interestingly, recent research indicates that this prop- solving (3) boils down to a three-step algorithm that can erty may somewhat be related to the Strang-Fix conditions be summarized as follows: which makes a very intriguing connection with approxi- mation theory [12], and considerably broadens the class T 1. Compute a solution H=[1,h1,...,hK−1,hK ] of of FRI-admissible kernels. In all cases investigated so far,

the scalar coefﬁcients hk(Ω) satisfy (6). A Y, A Y,..., A Y H=0; 0 1 K

2. Compute the roots ωk of the z-transform H(z) = 3.1 Periodized sinc (Dirichlet) ﬁlter K −k k=0 hkz ; P Solving the FRI problem in the case of a periodic stream 3. Compute a solution X of F(Ω) X = Y. of Diracs is equivalent to considering (1) where ϕ is a periodized sinc kernel, e.g., a Dirichlet kernel

Example—Spectral estimation problems boil down to a sin(πBt) ϕ(t)= sinc(B(t − k′τ)) = nonlinear problem of the form (3) involving the Vander- X Bτ sin(πt/τ) k′∈Z monde matrix: where τ is the period of the Dirac stream and B some ω1 ω2 · · · ωK  2 2 2  bandwith (chosen so that Bτ is an odd integer) [2]. This ω1 ω2 · · · ωK F(Ω) = problem can be reformulated using the annihilation equa-  . .   . .  tion (4) by deﬁning the following annihilation matrices  N N N   ω1 ω2 · · · ωK 

Ak = 0Bτ−K,k IBτ−K 0Bτ−K,Bτ−k W where the frequencies to retrieve, fk, are related to ωk j2πfk through ωk = e . This problem satisfies the GAP where W = [e−j2πmn/N ] for |m|≤⌊Bτ/2⌋ and 1 ≤ for band-diagonal matrices Ak which are more precisely given by: n ≤ N, is the N-DFT submatrix of size Bτ × N. Then, the abstract parameters ωk are related to the locations tk −j2πtk /τ Ak = 0N−K,k IN−K 0N−K,K−k , through ωk = e . This kernel has been found useful for the estimation of UWB channels [13] and for image where 0m,n is the m × n zero matrix and In is the n × n superresolution [14]. identity matrix. A minimal—yet not sufficient—condition for the unicity of the solution is N ≥ 2K. Since the Ak can be seen as shifting operators by k samples, the annihi- 3.2 Infinite sinc filter lation equation is analogous to a filtering equation—with an annihilating filter. The annihilation algorithm is then The filter ϕ(t) is given by ϕ(t) = sinc Bt with B =1/T . equivalent to Prony’s method [4]. Of course, spectral esti- When ϕ∗x (t) is sampled uniformly at frequency B, the mation in the presence of noise has been addressed by nu- nonlinear system of equations satisfies the GAP. The ab- merous researchers since the 1970’s [5, 6, 7, 8, 9, 10, 11]. stract parameters ωk are related to the locations tk through ωk = tk and the annihilation matrices are given by Additionally, there is a constraint on the minimal number of samples N for the GAP to hold, which is that N be K K · · · K 0 · · · 0  K K−1 0  larger than ⌈(S + max {t })/T ⌉. . . . . . k k ...... A  0 .  k =  .   ......  4. Conclusion  . . . . 0   0 ··· ··· K K · · · K   K K−1 0  We have shown how to unify the different techniques used 1 0 ··· ··· 0 in FRI signal reconstruction through an algebraic prop-   . . erty that we call the Generalized Annihilation property. In 0 2k .. .   essence, this property allows to solve nonlinear system of  . . . .  ×  . . k . .   . . 3 . .  equations within two noniterative steps. We hope that this    . .. ..  propertycan be used to solve other FRI problems(i.e, with  . . . 0    new kernels) in particular in dimensions higher than 1 (for  0 ··· ··· 0 N k  instance, like in [17]), and maybe to solve other types of 3.3 Gaussian filter problems not directly related to sampling. 2 2 The filter ϕ(t) is given by ϕ(t) = exp(−t /σ ). When References: ϕ ∗ x (t) is sampled uniformly at frequency T −1, the nonlinear system of equations satisfies the GAP. The ab- [1] M. Vetterli, P. Marziliano, and T. Blu, “Sampling sig- stract parameters ωk are related to the locations tk through nals with finite rate of innovation,” IEEE Transac- 2 ωk = exp(2tkT/σ ) and the annihilation matrices are tions on Signal Processing, vol. 50, pp. 1417–1428, given by June 2002. [2] T. Blu, P.-L. Dragotti, M. Vetterli, P. Marziliano, and Ak = 0N−K,k IN−K 0N−K,K−k L. Coulot, “Sparse sampling of signal innovations,” T 2  e σ2 0 ··· ··· 0  IEEE Signal Processing Magazine, vol. 25, no. 2, (2T )2 .. . pp. 31–40, 2008.  0 e σ2 . .  ×   [3] C. E. Shannon, “A mathematical theory of commu-  . .. ..   . . . 0  nication,” Bell System Technical Journal, vol. 27,  2   (NT )   0 ··· ··· 0 e σ2  pp. 379–423 and 623–656, July and October 1948. [4] R. Prony, “Essai expérimental et analytique,” An- A version of this solution (actually, for a Gabor kernel) nales de l’Ecole´ Polytechnique, vol. 1, no. 2, p. 24, was used in Optical Coherence Tomography, showing the 1795. possibility to resolve slices of a microscopic sample be- [5] P. Stoica and R. L. Moses, Introduction to Spectral low the coherence length of the illuminating reference Analysis. Upper Saddle River, NJ: Prentice Hall, light [15]. 1997. 3.4 Finite Support Strang-Fix filters [6] S. M. Kay, Modern Spectral Estimation—Theory and Application. Englewood Cliffs, NJ: Prentice Through linear combinations of its shifts, the finite sup- Hall, 1988. port filter ϕ(t) is assumed to reconstruct polynomials up [7] D. W. Tufts and R. Kumaresan, “Estimation of fre- to some degree L − 1 (standard Strang-Fix condition [16]) quencies of multiple sinusoids: Making linear pre- alt or exponentials e where al − a0 is linear with l = diction perform like maximum likelihood,” Proceed- 0, 1,...,L − 1. More precisely, in the standard Strang- ings of the IEEE, vol. 70, pp. 975–989, September Fix case, we denote by cl,n the coefficients such that 1982. c ϕ(nT − t)= tl where l =0, 1,...,L − 1, [8] S. M. Kay and S. L. Marple, “Spectrum analysis—a X l,n n∈Z modern perspective,” Proc. IEEE, vol. 69, pp. 1380– 1419, November 1981. by T the sampling step, and by [0,S] the support of ϕ(t). [9] Special Issue on Spectral Estimation, Proceedings of Then, the abstract parameters ωk are related to the loca- the IEEE, vol. 70, September 1982. tions tk through ωk = tk and the annihilation matrices are [10] V. F. Pisarenko, “The retrieval of harmonics from a given by covariance function,” Geophysical Journal, vol. 33, pp. 347–366, September 1973. ck,1 ck,2 · · · ck,N   [11] R. Roy and T. Kailath, “ESPRIT–estimation of sig- c − c − · · · c − A k 1,1 k 1,2 k 1,N nal parameters via rotational invariance techniques,” k =  . . .  .  . . .  IEEE Transactions on Acoustics, Speech, and Signal    ck−L+1,1 ck−L+1,2 · · · ck−L+1,N  Processing, vol. 37, pp. 984–995, July 1989. [12] P.-L. Dragotti, M. Vetterli, and T. Blu, “Sam- pling moments and reconstructing signals of finite rate of innovation: Shannon meets Strang-Fix,” IEEE Transactions on Signal Processing, vol. 55, pp. 1741–1757, May 2007. Part 1. [13] I. Maravić, J. Kusuma, and M. Vetterli, “Low- sampling rate UWB channel characterization and synchronization,” Journal of Communications and Networks, vol. 5, no. 4, pp. 319–327, 2003. [14] L. Baboulaz and P.-L. Dragotti, “Exact feature extraction using finite rate of innovation principles with an application to image super-resolution,” IEEE Transactions on Image Processing, vol. 18, pp. 281– 298, February 2009. [15] T. Blu, H. Bay, and M. Unser, “A new high- resolution processing method for the deconvolution of optical coherence tomography signals,” in Pro- ceedings of the First IEEE International Symposium on Biomedical Imaging: Macro to Nano (ISBI’02), vol. III, (Washington DC, USA), pp. 777–780, July 7-10, 2002. [16] G. Strangand G. Fix, “A Fourieranalysis ofthe finite element variational method,” in Constructive Aspects of Functional Analysis (G. Geymonat, ed.), pp. 793– 840, Rome: Edizioni Cremonese, 1973. [17] D. Kandaswamy, T. Blu, and D. Van De Ville, “Ana- lytic sensing: reconstructing pointwise sources from boundary Laplace measurements,” in Proceedings of the SPIE Conference on Mathematical Imaging: Wavelet XII, (San Diego CA, USA), August 26- August 30, 2007. To appear.