Matrix Inverse Computation of the Matrix Inverse n×n A square matrix S ∈ R is invertible if there exists a matrix We want to find the inverse of S ∈ Rn×n, that is we want to find a −1 n×n S ∈ R such that matrix X ∈ Rn×n such that SX = I. −1 −1 S S = I and SS = I. I Let X:,j denote the jth column of X, i.e., X = (X:,1,...,X:,n). Consider the matrix-matrix product SX. The jth column of SX is The matrix S−1 is called the inverse of S. the matrix-vector product SX:,j, i.e., SX = (SX:,1,...,SX:,n). I An invertible matrix is also called non-singular. The jth column of the identity I is the jth unit vector A matrix is called non-invertible or singular if it is not invertible. e = (0,..., 0, 1, 0,..., 0)T . n×n j I A matrix S ∈ cannot have two different inverses. R Hence SX = (SX:,1,...,SX:,n) = (e1, . . . , en) = I implies that we In fact, if X,Y ∈ n×n are two matrices with XS = I and SY = I, R can compute the columns X ,...,X of the inverse of S by then :,1 :,n X = XI = X(SY ) = (XS)Y = IY = Y. solving n systems of linear equations n×n −1 −1 I If S ∈ R is invertible, then Sx = f implies x = S Sx = S f, SX:,1 = e1, i.e., for every f the linear system Sx = f has a solution x = S−1f. . The linear system Sx = f cannot have more than one solution . because Sx = f and Sy = f imply S(x − y) = Sx − Sy = f − f = 0 and x − y = S−10 = 0. SX:,n = en. Hence if S is invertible, then for every f the linear system Sx = f has the unique solution x = S−1f. Note that if for every f the linear system Sx = f has a unique I We will see later that if for every f the linear system Sx = f has a solution x, then there exists a unique X = (X:,1,...,X:,n) with unique solution x, then S is invertible. SX = I.
M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 1 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 2
Want inverse of 1 0 2 LU-Decomposition S = 2 −1 3 . Consider 4 1 8 1 0 2 S = 2 −1 3 . Use Gaussian Elimination to solve the systems 4 1 8 SX = e ,SX = e ,SX = e for the three columns of X = S−1 :,1 1 :,2 2 :,3 3 Express Gaussian Elimination using Matrix-Matrix-multiplications 1 0 2 1 0 0 1 0 2 1 0 0 1 0 0 1 0 2 1 0 2 2 −1 3 0 1 0 → 0 −1 −1 −2 1 0 −2 1 0 2 −1 3 = 0 −1 −1 4 1 8 0 0 1 0 1 0 −4 0 1 −4 0 1 4 1 8 0 1 0 | {z } | {z } | {z } 1 0 2 1 0 0 =E1 =S =E1S → 0 −1 −1 −2 1 0 1 0 0 1 0 2 1 0 2 0 0 −1 −6 1 1 0 1 0 0 −1 −1 = 0 −1 −1 0 1 1 0 1 0 0 0 −1 1 0 0 −11 2 2 1 0 0 −11 2 2 | {z } | {z } | {z } → 0 −1 0 4 0 −1 → 0 1 0 −4 0 1 . =E2 =E1S =E2E1S=U 0 0 −1 −6 1 1 0 0 1 6 −1 −1 Inverses of E1 and E2 can be easily computed:
−11 2 2 1 0 0 1 0 0 −1 −1 −1 S = −4 0 1 . E1 = 2 1 0 ,E2 = 0 1 0 . 6 −1 −1 4 0 1 0 −1 1
M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 3 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 4 We have If we have computed the LU decomposition E2E1S = U S = LU, then we can use it to solve Hence Sx = f. −1 −1 −1 E1S = E2 U, and S = E1 E2 U. We replace S by LU, LUx = f, and introduce y = Ux. This leads to the two linear systems 1 0 2 1 0 0 1 0 0 1 0 2 Ly = f and Ux = y. 2 −1 3 = 2 1 0 0 1 0 0 −1 −1 4 1 8 4 0 1 0 −1 1 0 0 −1 Since L is lower triangular and U is upper triangular, these two systems can be easily solved. | {z } | {z } | {z } | {z } Example: =S =E−1 =E−1 =U 1 2 1 0 2 1 0 0 1 0 2 −4 1 0 0 1 0 2 2 −1 3 = 2 1 0 0 −1 −1 , f = −6 4 1 8 4 −1 1 0 0 −1 −15 = 2 1 0 0 −1 −1 | {z } | {z } | {z } 4 −1 1 0 0 −1 =S =L =U
| {z } | {z } 1 0 0 y −4 y −4 =E−1E−1=L =U 1 1 1 2 2 1 0 y2 = −6 ⇒ y2 = 2 4 −1 1 y3 −15 y3 3 | {z } | {z } Hence we have the LU-Decomposition of S, =L =f
S = LU, 1 0 2 x1 −4 x1 2 0 −1 −1 x2 = 2 ⇒ x2 = 1 0 0 −1 x3 3 x3 −3 where L is a lower triangular matrix and U is an upper triangular matrix. | {z } | {z } =U =y In Matlab compute using [L,U]=lu(S). M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 5 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 6
In Matlab the matrix inverse is computed using the LU decomposition. Execute the following: Given S, we want to compute S−1. Recall that the columns n = input(’Problem size = ’); X ,...,X of the inverse S−1 = X are the solutions of :,1 :,n S = rand(n,n); f = rand(n,1); SX:,1 = e1, . ntry = 50; .
SX:,n = en. tic for i = 1:ntry If we have computed the LU decomposition x = S\f; end S = LU, toc
tic then we can use it to solve the n linear systems SX:,j = ej, j = 1, . . . n. Use the LU decomposition to compute the inverse of for i = 1:ntry Sinv = inv(S); 1 0 2 1 0 0 1 0 2 x = Sinv*f; 2 −1 3 = 2 1 0 0 −1 −1 end toc 4 1 8 4 −1 1 0 0 −1 | {z } | {z } | {z } =S =L =U tic [L,U] =lu(S); What do you think of the following approach to solve Sx = f: for i = 1:ntry x = U\(L\f); >> Sinv = inv(S); end >> x = Sinv*f; toc
M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 7 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 8