Matrix Inversion Using Cholesky Decomposition Aravindh Krishnamoorthy, Deepak Menon ST-Ericsson India Private Limited, Bangalore [email protected], [email protected]

Abstract—In this paper we present a method for matrix inversion Upper triangular elements, i.e. : based on Cholesky decomposition with reduced number of operations by avoiding computation of intermediate results; further, we use fixed point simulations to compare the numerical ( ∑ ) accuracy of the method. … (4)

Keywords-matrix, inversion, Cholesky, LDL. Note that since older values of aii aren’t required for computing newer elements, they may be overwritten by the I. INTRODUCTION value of rii, hence, the algorithm may be performed in-place Matrix inversion techniques based on Cholesky using the same memory for matrices A and . decomposition and the related LDL decomposition are efficient Cholesky decomposition is of order and requires techniques widely used for inversion of positive- operations. Matrix inversion based on Cholesky definite/symmetric matrices across multiple fields. decomposition is numerically stable for well conditioned Existing matrix inversion algorithms based on Cholesky matrices. decomposition use either equation solving [3] or operations [4] with most efficient implementation If , with is the linear system with requiring operations. variables, and satisfies the requirement for Cholesky decomposition, we can rewrite the linear system as In this paper we propose an inversion algorithm which … (5) reduces the number of operations by 16-17% compared to the existing algorithms by avoiding computation of some known By letting , we have intermediate results. … (6) In section 2 of this paper we review the Cholesky and LDL decomposition techniques, and discuss solutions to linear and systems based on them. In section 3 we review the existing … (7) matrix inversion techniques, and how they may be extended to non-Hermitian matrices. In section 4 we discuss the proposed These equations are solved using backward-substitution and matrix inversion method. require operations each for the solution.

II. CHOLESKY DECOMPOSITION A. LDL Decomposition If is a positive-definite , If is a , LDL decomposition Cholesky decomposition factorises it into a lower triangular factorises it into a lower triangular matrix, a matrix and its [3], [5] & [6]. and conjugate transpose of the lower triangular matrix [5]. … (1) … (8) Or equivalently, using an upper triangular matrix as Or equivalently, using an upper triangular matrix as … (2) … (9) In a software implementation the upper triangular matrix is This decomposition eliminates the need for square-root preferred as operations are row-wise and compatible with operation. programming language. The elements of , and diagonal elements The elements of , are given as follows. of the matrix are given as follows. Diagonal elements: Diagonal elements:

∑ √ ∑ … (10)

… (3) Upper triangular elements, i.e. :

… (19)

( ∑ ) When efficiently implemented, equation solving requires

… (11) and requires . The total number

When efficiently implemented, the complexity of the LDL of matrix multiplication including the Cholesky decomposition decomposition is same as Cholesky decomposition. is [4].

If , with is the linear system with C. Non-Hermitian Matrices variables, and satisfies the requirement for LDL decomposition, we can rewrite the linear system as Cholesky (or LDL) decomposition may be used for non- Hermitian matrices by creating an intermediate Hermitian

… (12) matrix as follows: By letting , we have For an arbitrary matrix , we may construct a

… (13) Hermitian matrix as . Once the inverse of A is found using Cholesky (or LDL) decomposition, we may find and as . … (14) This method requires a matrix transposition operation for

These equations are solved using backward-substitution and finding and matrix multiplication to find . when efficiently implemented, require operations each for the solution. IV. PROPOSED METHOD III. EXISTING TECHNIQUES The proposed method is a modification to the Equation Solving method described in section III., where the first A. Equation Solving equation’s solution (6, 13) is avoided. If , we may find , and by solving The method as applicable to Cholesky decomposition and LDL decomposition are described below.  … (15) A. Using Cholesky Decomposition th Where is the i column of the of order If and such that , we have [3]. This equation may be solved using either Cholesky or LDL based method as described above depending on the properties … (20) of . From Cholesky decomposition: In either case since is Hermitian, it is sufficient to solve … (21) for upper (or lower) half of and update the other half with the complex conjugate values as for . Solving for the upper half of the matrix requires two By letting , we have triangular matrix solutions with multiply operations each. … (22) The total number of multiply operations including the and decomposition is . … (23) B. Triangular Matrix Operations Let , for ; we note that the inverse If , we may find the inverse of , of the lower triangular matrix is lower triangular, and the using Cholesky decomposition, we have diagonal entries of are the reciprocals of diagonal entries of … (16) Therefore, the matrix B is lower triangular, with diagonal elements as reciprocals of the diagonal elements of . This implies: Now, given these properties, we construct a matrix such

… (17) that

, and is computed as { follows. … (24)

… (18) We note that the matrix is the correct solution to upper th diagonal elements of the matrix , i.e. ̃ Where is the i column of the identity matrix of order . , we do not compute B and instead use backward substitution Once is computed, is computed as to solve for using the equation , given:  We solve only for such that (upper triangle elements.) METHOD OPERATIONS

 where needed. Equation Solving

Equation solving requires multiply operations, the total number of multiply operations for matrix inverse including Triangular Matrix

Cholesky decomposition is . Operations

Proposed Method B. Using LDL Decomposition

If and such that , we have Table 1. Table summarising the number of operations for … (25) the methods described in the paper. From LDL decomposition: … (26) By letting ̃, we have ̃ … (27) and ̃ … (28)

From (27) we note that ̃ , for . In this case we construct the matrix ̃ as

̃ { … (29) We note that the matrix ̃ is the correct solution to upper Fig. 1. Fixed point error analysis of various techniques diagonal elements of the matrix ̃ , i.e. ̃ described in the paper. , we do not compute ̃ and instead use backward substitution to solve for as in Cholesky decomposition. REFERENCES

Equation solving requires multiply operations, the total number of multiply operations for matrix inverse including [1] Happonen, A.; Piirainen, O.; Burian, A: "GSM channel estimator using a

LDL decomposition is . fixed-point matrix inversion algorithm" IEEE International Symposium on Signals, Circuits and Systems, 2005, pp 119-122 Vol. 1. C. Numerical Accuracy [2] Lei Ma; Dickson, K.; McAllister, J.; McCanny, J.; Sellathurai, M: "Reduced-complexity MSGR-based matrix inversion" IEEE Workshop Fixed point simulations indicate that the proposed method on Signal Processing Systems, 2008. SiPS 2008, pp 124-128. has a good numerically accuracy. From figure 1 we may note [3] Gilbert W. Stewart: "Afternotes on ", SIAM, 1996. that the proposed method has the lowest average norm error [4] Burian, A.; Takala, J.; Ylinen, M: "A fixed-point implementation of compared to other methods described in this paper. matrix inversion using Cholesky decomposition" Proceedings of the 46th IEEE International Midwest Symposium on Circuits and Systems, 2003, pp 1431-1434 Vol. 3. [5] David S. Watkins: "Fundamentals of Matrix Computations", Second V. CONCLUSION Edition, Wiley, 2002. We presented a method for matrix inversion based on [6] William H. Press, Saul A. Teukolsky, William T. Vetterling, Brian P. Cholesky decomposition with reduced number of operations by Flannery: "Numerical Recipes, The Art of Scientific Computing", Third avoiding computation of intermediate results. The number of Edition, Cambridge University Press, 2007. operations for the methods is summarised in table 1. The results of fixed point simulations are in figure 1.