Circulant decomposition of a matrix and the eigenvalues of Toeplitz type matrices Hariprasad M., Murugesan Venkatapathi Department of Computational & Data Sciences, Indian Institute of Science, Bangalore - 560012 Abstract We begin by showing that a n×n matrix can be decomposed into a sum of n cir- culant matrices with appropriate relaxations. We use Fast-Fourier-Transform (FFT) operations to perform a sparse similarity transformation representing only the dominant circulant components, to evaluate all eigenvalues of dense Toeplitz, block-Toeplitz and other periodic or quasi-periodic matrices, to a rea- sonable approximation in O(n2) arithmetic operations. This sparse similarity transformation can be exploited for other evaluations as well. Keywords: Eigenvalues, periodic entries, symbol. AMS subject classifications: 65F15, 15A18, 15B05, 15A04 1. Introduction Multiple approaches to efficiently and accurately evaluate eigenvalues of sparse and effectively-sparse matrices are known [1, 2, 3, 4, 5]. Efficient methods are equally desirable for dense matrices that occur frequently in the numerical solution of eigenvalue problems; and Toeplitz, block-Toeplitz, and other ma- trices with periodicity in the diagonals are such a class. O(n2) algorithms to evaluate the characteristic polynomial and its derivative were proposed a few decades ago for Toeplitz [6], block-Toeplitz [7, 8] and Hankel matrices [9]. These algorithms were amalgamated with the Newton methods applied to evaluate only the eigenvalues of interest. On the other hand, the asymptotic behavior of spectra of Toeplitz and block-Toeplitz operators in the limit of large dimen- arXiv:2105.14805v1 [math.NA] 31 May 2021 sions have been well studied [10],[11] and they can also be approximated by an appropriate circulant matrix. An approximation of a finite dimensional Toeplitz matrix using a circulant matrix for speed up of operations was suggested decades ago [12],[13]. Similarly, circulant preconditioners were constructed for Toeplitz matrices by minimizing Email addresses: [email protected] (Hariprasad M.), [email protected] (Murugesan Venkatapathi) the Frobenius norm of the residue [14]. Spectral preserving properties of this preconditioner were shown [15], and block circulant versions were also proposed and analyzed [16]. Note that this preconditioner is the first term in the circulant series representation of a matrix, discussed in this work. Since these precon- ditioners are a projection of the matrix on the low complexity matrix classes, they speed up iterative methods in regularization and optimization. But these preconditioners may not be sufficiently accurate as a similarity transformation of the given matrix. First, using the symbol of a Toeplitz matrix we highlight cases where this single-term circulant approximation is effective for Toeplitz matrices. Later, we broaden the scope of work to matrices with any periodicity along the diagonals. We first recall in remark 1 that any matrix can be decomposed into n cycles that generate its 2n-1 diagonals. These cycles are given by a n term power series of the cyclic permutation matrix with appropriate relaxations. We later show in lemma 1 that when this decomposition is instead applied on an appropriate sim- ilarity transformation of the matrix, it is equivalent to a decomposition of the given matrix into n circulant matrices with appropriate relaxations. By includ- ing only the dominant cycles of the similar matrix, we include the dominant circulant components of the given matrix. The sparse similar matrix can be operated by a non-symmetric Lanczos algorithm for (block) tridiagonalization, and one can evaluate all eigenvalues of such dense matrices in O(n2) arithmetic operations. Let In be the identity matrix of dimension n, and C be the permutation matrix corresponding to the adjacency matrix of a full cycle. 0 1 C = : I 0 n−1 n×n Remark 1. Any matrix A can be decomposed into n cycles given by a power Pn−1 j series in C such that A = j=0 ΛjC . In the above, we refer to the non-zero entries of Hadamard product A:Cj, supported on Cj, as the jth cycle of matrix A. Let the permutation matrix given by a flipped identity matrix be 2 3 0 ··· 0 0 1 6 7 60 ··· 0 1 07 6 7 J = 60 ··· 1 0 07 : 6. 7 6. :: 7 4. : 0 0 05 1 ··· 0 0 0 n×n Remark 2. Eigenvalues of a circulant matrix R are given by the discrete Fourier transform of the first row R(1; j). The corresponding eigenvectors are 1 2π p i n pq given by the columns of a matrix W with W (p; q) = n e where p; q = 1; 2 ··· n. The matrix W has the property W 2 = JC, so any circulant matrix has an eigen decomposition R = W ΛW y. R is also given by R = W yΛ~W where Λ~ = JCΛCT J T . 2 i 2πkp Lemma 1. Given diagonal matrices D with Dk(p; p) = e n and any circulant T k matrix R, the matrices RDk and Λ(~ C ) are similar. Here Λ is a diagonal matrix with the non zero entries given by eigenvalues of R, and C is the cyclic permutation matrix. Proof. Using W for a similarity transformation, y y y W RDkW = WW Λ~WDkW ; y = Λ~WDkW = Λ(~ CT )k: y T k The equality WDkW = (C ) can be deduced by evaluating its (p; l) entry given by y 1 X i 2πpq i 2πkq −i 2πql WD W (p; l) = e n e n e n ; k n q 1 X i 2π(p+k−l)q = e n ; n q = 1 when l = p + k and 0 otherwise: T k Hence, the eigenvalues of RDk are the eigenvalues of the matrix Λ(~ C ) . The matrix Λ(~ CT )k is sparse, and its eigenvalues are easy to compute. Mo- tivated by this result, we construct the decomposition of a matrix into these components. Theorem 1. Any n×n square matrix A can be represented as sum of n circulant matrices with relaxations taking values from the nth root of unity. It is of the form n X A = RkDk k=1 Where Rk is a circulant matrix and Dk is a diagonal matrix with Dk(p; p) = i 2πkp e n . Proof. Consider the matrix B = W AW y. Using remark 1 to decompose it into Pn ~ T k cycles, and recalling the proof of lemma 1, we have B = k=1 Λk(C ) = Pn ~ y ~ k=1 ΛkWDkW , where Λk are diagonal. The corresponding matrix A is then Pn y ~ Pn A = k=1 W ΛkWDk = k=1 RkDk. Corollary 1. If a matrix A has k particular dominant frequencies on each of its cycles, then the matrix W AW y has only k dominant cycles. Proof. Consider the decomposition of A into cycles. n−1 X q A = ΛqC q=0 3 where entries of each Λq are given by k dominant frequencies. Using the pro- posed similarity transformation, n−1 y X q y W AW = W ( ΛqC )W q=0 n−1 y X B = W AW = RqDq q=0 We know each circulant matrix Rq has eigenvalues given by the diagonal matrix Λq. As mentioned in remark 2, entries Rq(1; j) are given by the inverse FFT of Λ, and hence, all rows have k particular dominant entries, with the transformed similar matrix B having only k dominant cycles. Also note that Pn−1 j q=0 Rq(1; j)C Dq is the FFT of the sequence Rq(1; j) for q = 0; 1; 2; ··· n. By Parsevals theorem, a cycle supported on Cj in B has the same energy as the sequence corresponding entries of Rq(1; j) for q = 0; 1; 2; ··· n. 2. Single-term circulant approximation of a Toeplitz matrix and its relation with the symbol A Toeplitz matrix is of the form, 2 3 a0 a1 a2 ··· an−1 6 . 7 6 .. 7 6 a−1 a0 a1 a2 7 6 . 7 A = 6 .. .. 7. 6 a−2 a−1 a0 7 6 . 7 6 . .. .. .. .. 7 4 . 5 a−(n−1) a−(n−2) ······ a0 iθ Pn−1 2πik The function a(e ) = k=−(n−1) ake for θ 2 [0; 2π) is called the symbol of the matrix A [17]. In this section we consider different types of symbols, the corresponding eigenvalues of the single-term circulant approximation, and the actual spectrum of the Toeplitz matrices. The single-term approximation is given by only the diagonal entries of W AW y. The symbol may be exploited for computing efficiency, when applicable, as it can be evaluated in O(n) arithmetic operations. On the other hand, the single-term circulant approximation is more accurate and robust in general. 2.1. Case 1: Symbol is a product of a polynomial and a trigonometric polynomial Spectrum of A is said to be canonically distributed if the limiting spectrum approaches the range of the symbol. For symbols of the form a(eiθ) = p(θ)q(eiθ) with polynomials p and q, we consider the Toeplitz matrices whose entries are from its truncated Fourier series expansion. Several conditional theorems on the symbol such that the spectrum of T is canonically distributed have been pre- sented [17]. Figure 1 shows the spectrum, single-term circulant approximation, and the symbol for a(θ) = (1 + θ)eiθ. 4 1.5 Actual 1 Range(a) diag(WAW’) 0.5 0 Imag -0.5 -1 -1.5 -2 -2 -1 0 1 2 Real Figure 1: Range of the symbol, and diagonal entries of W AW y for a Toeplitz matrix of dimension 200 with a symbol a(θ) = (1 + θ)eiθ. 2.2. Case 2: Symbol of the banded Toeplitz matrix is a trigonometric polynomial For a banded Toeplitz matrix A, with first row a0; a1; a2; ··· al and first iθ Pl ikθ column entries a0; a−1; ··· a−m, the symbol is given by a(e ) = k=−m ake .