Singular Values using Cholesky Decomposition

Aravindh Krishnamoorthy, Kenan Kocagoez ST-Ericsson AT GmbH, Nuernberg, http://www.stericsson.com {aravindh.k, kenan.kocagoez}@stericsson.com

Abstract—In this paper two ways to compute singular values In the algorithm above, instead of comparing the diagonal are presented which use Cholesky decomposition as their basic element with zero, we compare it against a small tolerance operation. value  to account for numerical inaccuracies. I.INTRODUCTION 23rd January, 2012 Cholesky Decomposition is a fast and numerically stable II.CHOLESKY ITERATIONS matrix operation that finds wide use in various fields for linear system solving, inversion, and factorisation [1] and is a In this section, “Cholesky Iterations” based computation of common block for matrix operations in hardware and software singular-values is discussed for positive (semi-) definite, sym- based computing systems. In this paper we present two ways metric and arbitrary matrices. While the performance for pos- of finding singular values of a matrix which use Cholesky itive (semi-) definite and symmetric matrices are acceptable, decomposition as their basic operation, and may be useful the method for arbitrary matrices may have poor performance in constrained computing environments such as embedded for ill-conditioned matrices. computing systems. A. Positive (semi-) definite Matrices The first method “Cholesky Iterations” uses Cholesky decom- positions based iterations [2], and [10]. The second method Let A ∈ Cnxn be a positive (semi-) definite Hermitian uses Cholesky decomposition to compute the QR factors, , then the singular values (also eigenvalues) which are further used for “QR Iterations” [1], [4], [5], and are given using the following algorithm. [6]. Algorithm 2 Cholesky Iterations - Positive (semi-) definite matrices A. Cholesky Decomposition J (0) = A Cholesky Decomposition [1] factorises a complex (or real- for k := 1...iter do valued) positive-definite Hermitian symmetric matrix into a (k) = chol(J (k−1)) product of a lower and its hermitian trans- J (k) = R(k)(R(k))H pose. end for Σ = Λ = diag(J (iter)) A = RH R, or equivalently, R = chol(A) where A ∈ Cnxn is a positive-definite Hermitian symmetric matrix, R ∈ Cnxn is an upper triangular matrix, and RH is where the Hermitian tranpose of R.  A if i = j diag(A) = ii B. For Positive Semi-definite Matrices 0 if i 6= j Cholesky decomposition for positive semi-definite matrices is Singular values lie along the diagonals of the matrix Σ, and analysed in [3], and it is computed easily by setting the row they are the same as eigenvalues. to zero if the diagonal element is found to be zero. 1) Diagonalisation: If Algorithm 1 Cholesky Decomposition for Positive Semi-definite Matrix (k−1) H arXiv:1202.1490v1 [cs.MS] 7 Feb 2012 J = R R for i = 1q...N do Pi−1 ∗ Rii = Aii − k=1 RkiRki then (k−1) −1 H (k) if |Rii| >  then RJ R = RR = J for j = i + 1...N do 1  Pi−1 ∗  and so the matrix is diagonalised as: Rij = Aij − RkjR Rii k=1 ki end for (k) (k−1) (1) (0) (k−1) (1) −1 else J = (R ...R )J (R ...R ) Ri,i+1...N = 0 The matrix product (R(k−1)...R(1)) increases unboundedly end if with iterations, but the values of J (k) converge towards the end for diagonal singular-values matrix. B. Symmetric matrices nxn Let A ∈ be a Hermitian symmetric matrix, then the R = chol(AH A) (1) singular values (and eigenvalues) are given using the following Q = AR−1 (2) algorithm. If A is not full-column rank, we may find Q as Q = AR+ Algorithm 3 Cholesky Iterations - Symmetric Matrices where R+ is the pseudo-inverse of R. In that case, the last J (0) = A + µI m − n columns of Q, and the lower m − n rows of R are for k := 1...iter do zeros and may be ignored. (k) (k−1) R = chol(J ) Theorem III.1. Matrix Q = AR−1 is Unitary. J (k) = R(k)(R(k))H end for Proof: We have AH A = RH R; pre-and-post multiplying Λ = diag(J (iter)) − µI with R−H and R−1, we have, R−H AH AR−1 = I and so Σ = |Λ| (AR−1)H AR−1 = I, or QH Q = I, and hence Q is unitary.

Algorithm 5 (Pure) QR Iterations Where µ is a pivoting constant to ensure that the matrix J (0) is J (0) = A positive (semi-) definite matrix; µ may be found using variety for k := 1...iter do of pivoting techniques, for e.g. see [7], [8], and [9]. R(k) = chol((J (k−1))H J (k−1)) If the symmetric matrix is well-conditioned, the algorithm Q(k) = J (k−1)(R(k))−1 given below for Aribtrary Matrices may also be used, which J (k) = R(k)Q(k) avoids the computation of the pivoting constants. end for C. Arbitrary Matrices Σ = diag(J (iter)) Let A ∈ Cmxn be an arbitrary matrix, then the singular values are given using the following algorithm. For further information on“QR Iterations” refer [1], [4], [5], and [6]. Algorithm 4 Cholesky Iterations - Arbitrary Matrices J (0) = AH A REFERENCES for k := 1...iter do [1] Matrix Computations, 3rd Edition, Gene H. Golub, Charles F. van Van R(k) = chol(J (k−1)) Loan, 1996, ISBN: 0-8018-5413-X. [2] J.H. Wilkinson, The Algebraic Eigenvalues Problem, Oxford Science (k) (k) (k) H J = R (R ) Publications, Oxford, 1965. end for [3] Analysis of the Cholesky Decomposition of a Semi-definite Matrix, N. J. p Σ = diag(J (iter)) (or) diag(R(iter)) Higham, Manchester Institute for Mathematical Sciences, 1990, MIMS EPrint: 2008.56. [4] Accurate Singular Values of Bidiagonal Matrices, J. Demmel, W. Ka- han, SIAM J. Sci. Stat. Comput., v. 11, n. 5, pp. 873-912, 1990, If the matrix A is full row-rank, J (0) = AAH may also be www.netlib.org/lapack/lawnspdf/lawn03.pdf. used as an initial state for the iterations. [5] The Bidiagonal Singular Value Decomposition and Hamiltonian Mechan- ics, P. Deift, C. Li, C. Tomei, SIAM J. Numer. Anal., v. 18, n. 5, pp. III.QRITERATIONS 1463-1516, 1991, www.netlib.org/lapack/lawnspdf/lawn11.pdf. [6] Matrix factorizations and integrable systems, P. Deift, L. C. Li, and C. In this section, Cholesky decomposition is used to compute the Tomei, Comm. Pure Appl. Math., 42:443-521, 1989. QR factors of an aribtrary matrix. These QR factors are further [7] A New Modified Cholesky Factorization, R. B. Schnabel and E. Eskow, Department of Computer Science, University of Colorado, December used for “QR Iterations” to compute the singular values. 1988. While computationally the method of finding QR factors from [8] A Revised Modified Cholesky Factorization Algorithm, R. B. Schnabel, E. Cholesky decomposition may be expensive, its performance is Eskow, Department of Computer Science, University of Colorado, DOI: 10.1.1.27.319. acceptable and it may be advantageous to use this method in [9] A modifed Cholesky algorithm based on a symmetric indefnite fac- certain constrained computational platforms. torization, S. H. Cheng and N. J. Higham, Manchester Institute for If A ∈ Cmxn is an arbitrary matrix, then the QR Decomposi- Mathematical Sciences, 1998, MIMS EPrint: 2006.163. [10] Asymptotic Behavior of Iwasawa and Cholesky Iterations, R. R. Holmes, tion is given by: H. Huanh, and T. Tam, and its Applications (May 2011).