1 A New Algorithm for the Nearest Singular Toeplitz Matrix to a Given Toeplitz Matrix Andrew E. Yagle Department of EECS, The University of Michigan, Ann Arbor, MI 48109-2122 Abstract— Several signal processing applications can This problem arises in 1D array processing, spec- be formulated as the computation of the null vector tral estimation, beamforming, detection of resonance of a Hermitian Toeplitz matrix. These include ar- ray processing, spectral estimation, and beamform- frequencies, and many other applications. ing algorithms applied directly to data rather than In practice, the data Xk are noisy, so that the to its autocorrelation, and some blind deconvolution above matrix is not singular. A simple likelihood algorithms. When the data are noisy, the matrix is nonsingular, and the closest singular Toeplitz matrix function argument shows that if the noise is addi- (in the mean square norm) to the given matrix must tive zero-mean white Gaussian (in either time or fre- be computed. Two major approaches have been used quency), then the likelihood is maximized when the for this problem: (1) alternatingly subtracting off the given data {X } are perturbed as little as possible outer product of minimum singular vectors and aver- k aging along diagonals; and (2) structured total least (in the mean square norm sense) to make the ma- squares. Both require many iterations of computa- trix drop rank. The difficulty is that the Hermitian tionally intensive singular value decompositions. We Toeplitz structure of the matrix must be maintained. present a new algorithm that is: (1) non-iterative; and (2) requires only solution of a Toeplitz system of equa- B. Other Spectral Estimation Algorithms tions. Several interesting linear algebra issues arise. Numerical examples illustrate the new algorithm. Most spectral estimation algorithms, such as Pis- Keywords— Toeplitz matrices, spectral estimation. arenko method, MUSIC, and ESPRIT, operate not Phone: 734-763-9810. Fax: 734-763-1503. Email: [email protected]. EDICS: 2-REST. on the data but on the autocorrelation function es- timated from the data. This has the advantage that additive white noise tends to be concentrated in the I. INTRODUCTION subspace spanned by the singular vectors associated A. Background with the minimum singular values, since the autocor- relation of zero-mean white noise is an impulse. Several problems in signal processing can be However, autocorrelation-based methods can be formulated as computation of the null vector of inappropriate, for the following three reasons: a Hermitian Toeplitz matrix. For example, let ∗ {X−M ,...X0 ...XM } with X−k = Xk be a complex- • Only a small number of data points are available, valued time series modelled by finite Fourier series not a long time series of data; • M Estimation of autocorrelation from data, which is jωnk always inexact due to end effects, is impractical; Xk = xne , |k|≤ M (1) • In practice, the additive noise is often neither white nX=1 nor uncorrelated with the data. for unknown real {xn} and {ωn}. These unknown model parameters can be computed from the data Hence an approach that operates directly on the data, by computing the null vector of a Hermitian Toeplitz rather than on the autocorrelation, is desirable. TLS th matrix with (m,n) element Xm−n and computing Prony method is an example of this. However, TLS the roots of the polynomial whose coefficients are el- Prony uses the TLS solution (minimum singular vec- ements of its Hermitian symmetric null vector ~a: tor of the matrix), which does not preserver the Her- mitian Toeplitz structure of the matrix. The above X0 · · · XM a0 0 . problem is an improvement on TLS Prony. .. .. .. . = 0 Two major approaches are known for this problem. ∗ 0 X−m · · · X0 a0 First is an iterative algorithm alternating between: M .. jωn a0z + . + a0 =0 → z = e . (2) • Computing the nearest (in Frobenius norm) lower rank matrix using the singular value decomposition, Alternatively, the Xk might be real and even and the by subtracting the outer product of the minimum Fourier coefficients xn conjugate symmetric. singular vectors times the minimum singular value; 2 • Computing the nearest (in Frobenius norm) More precisely, suppose the ωi are all rational num- Toeplitz matrix by averaging along the diagonals. bers, and let N be the least common multiple of their denominators. Then hn is periodic with period N, The other is structured total least squares, which iter- and the ejωin are orthogonal functions. Parseval’s atively perturbs the matrix closer to singularity, aver- theorem then proves discarding the term with the aging along the diagonals in each iteration to preserve smallest |Ci| produces the minimum perturbation in Toeplitz structure (standard TLS is one iteration). the least-squares norm of a period of hn. Both of these approaches have been applied suc- There are three problems with this approach: cessfully. However, the enormous computation of re- peatedly computing the singular value decomposition • How to compute poles from a nonsingular matrix; of the matrix for possibly many iterations, suggests • There is no reason to believe the poles will lie on that a much simpler algorithm is desirable. the unit circle, as required by the above model; • Hence the terms are not orthogonal, and discarding C. New Algorithm the term with the smallest |Ci| may not be optimal. This paper proposes a new algorithm for finding the closest reduced-rank Hermitian Toeplitz matrix to a given Hermitian Toeplitz matrix. The error cri- B. Toeplitz Matrix Extension terion is not the Frobenius norm (sum of squared We can solve all of these problems by dealing magnitudes of matrix elements) of the perturbation, with Hermitian Toeplitz matrices instead of Hankel which weights lower-indexed values of Xk more than matrices; this explains the use of Toeplitz matrices higher-indexed values, since they occur more often in throughout this paper. We first fit a model to the the matrix. For example, X0 appears M+1 times Hermitian Toeplitz matrix (2) by extending it to along the main diagonal, while XM appears only ∗ X1 ... XK+1 X twice (once as XM and once as X−M = XM ). The new algorithm has three advantages over the .. .. .. .. (4) previous algorithms mentioned above: . ∗ ∗ .. X XK+1 X1 • It requires no singular value decompositions, only and choosing X so that this matrix is singular. This the solution of a single Toeplitz system of equations; clearly fits a model of order K+1 to (2). To reduce • It is non-iterative, hence no convergence issues; the rank of (2) from K+1 to K, we reduce the model • The least-squares error criterion applied to the order from K+1 to K by discarding the term with {Xk}, rather than to the Toeplitz matrix, is more smallest |Ci| in the polar expansion. suitable in many signal processing applications. But there is a problem: setting the determinant of this matrix to zero produces a quadratic equation in X if the X are real, and an equation of the form II. Noisy Data: Computation of Nearest k Singular Hermitian Toeplitz Matrix ∗ ∗ ∗ XX + A X + AX + B =0 → A. Background (X + A)(X + A)∗ = |X + A|2 = |A|2 − B (5) The algorithm is based on a result that goes back to Kronecker: A Hankel matrix has rank M if and for some constants A and B. The quadratic equation has two solutions for X; the above equation has an only if its elements hi+j−1 are Fourier coefficients of a strictly proper rational function whose denomina- infinite number of solutions! This is not surprising; tor has degree M. Suppose that this rational function there are not enough data points to uniquely deter- has M distinct poles ejωi all on the unit circle. Then mine the poles from the data. What is to be done? the elements hi+j−1 can be expanded as C. Structures of Null Vectors M Some matrix theory: Let J be the exchange matrix jωin hn = Cie . (3) with ones on the main antidiagonal and zeros else- Xi=1 where; note J 2 = I is the identity matrix. Then a for some constants Ci. The function hˆn closest (in Hankel matrix can be converted to a Toeplitz matrix the least-squares norm) to hn that reduces the matrix by pre or post-multiplying by J, which does not affect rank from M to M-1 is determined by discarding the the rank. However, the Hermitian Toeplitz structure term with the smallest |Ci|. This is similar to model imposes a certain type of Hermitian structure on the order reduction techniques in linear system theory. null vector. To see this note that for any Toeplitz 3 matrix T we have JT J = T T , while for Hermitian structure length fixed zeros Toeplitz matrices JT J = T ∗. The null vector ~a of a Hermitian odd none Hermitian Toeplitz matrix has the structure anti-Herm odd ±1 Hermitian even -1 anti-Herm even +1 0= T~a = JT ∗J~a → 0= T (CJ~a∗) → ~a = CJ~a∗. For odd lengths, we clearly prefer the Hermitian (6) structure, since anti-Hermitian constrains two zeros. But consistency requires that ~a satisfy For even lengths, we cannot avoid a zero constraint. The constrained zero is then the zero that is dis- carded (see the example in the next section). ~a = CJ~a∗ → CJ~a∗ = CC∗~a → |C| = 1 (7) For very large noise levels, there may be a recip- rocal complex conjugate quadruple. A root locus ar- gument shows that this arises from two poles on the so that the constant C must lie on the unit circle. unit circle coalescing and then moving off the unit Real symmetric Toeplitz matrices can have a null vec- circle.