Matrix Inverse Computation of the Matrix Inverse n×n A square matrix S 2 R is invertible if there exists a matrix We want to find the inverse of S 2 Rn×n, that is we want to find a −1 n×n S 2 R such that matrix X 2 Rn×n such that SX = I. −1 −1 S S = I and SS = I: I Let X:;j denote the jth column of X, i.e., X = (X:;1;:::;X:;n). Consider the matrix-matrix product SX. The jth column of SX is The matrix S−1 is called the inverse of S. the matrix-vector product SX:;j, i.e., SX = (SX:;1; : : : ; SX:;n). I An invertible matrix is also called non-singular. The jth column of the identity I is the jth unit vector A matrix is called non-invertible or singular if it is not invertible. e = (0;:::; 0; 1; 0;:::; 0)T . n×n j I A matrix S 2 cannot have two different inverses. R Hence SX = (SX:;1; : : : ; SX:;n) = (e1; : : : ; en) = I implies that we In fact, if X; Y 2 n×n are two matrices with XS = I and SY = I, R can compute the columns X ;:::;X of the inverse of S by then :;1 :;n X = XI = X(SY ) = (XS)Y = IY = Y: solving n systems of linear equations n×n −1 −1 I If S 2 R is invertible, then Sx = f implies x = S Sx = S f, SX:;1 = e1; i.e., for every f the linear system Sx = f has a solution x = S−1f. The linear system Sx = f cannot have more than one solution . because Sx = f and Sy = f imply S(x − y) = Sx − Sy = f − f = 0 and x − y = S−10 = 0. SX:;n = en: Hence if S is invertible, then for every f the linear system Sx = f has the unique solution x = S−1f. Note that if for every f the linear system Sx = f has a unique I We will see later that if for every f the linear system Sx = f has a solution x, then there exists a unique X = (X:;1;:::;X:;n) with unique solution x, then S is invertible. SX = I. M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 1 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 2 Want inverse of 01 0 21 LU-Decomposition S = 2 −1 3 : Consider @ A 0 1 4 1 8 1 0 2 S = @2 −1 3A : Use Gaussian Elimination to solve the systems 4 1 8 SX = e ; SX = e ; SX = e for the three columns of X = S−1 :;1 1 :;2 2 :;3 3 Express Gaussian Elimination using Matrix-Matrix-multiplications 0 1 0 1 1 0 2 1 0 0 1 0 2 1 0 0 0 1 0 0 1 0 1 0 2 1 0 1 0 2 1 @ 2 −1 3 0 1 0 A ! @ 0 −1 −1 −2 1 0 A @ −2 1 0 A @ 2 −1 3 A = @ 0 −1 −1 A 4 1 8 0 0 1 0 1 0 −4 0 1 −4 0 1 4 1 8 0 1 0 0 1 | {z } | {z } | {z } 1 0 2 1 0 0 =E1 =S =E1S 0 1 0 1 0 1 ! @ 0 −1 −1 −2 1 0 A 1 0 0 1 0 2 1 0 2 0 0 −1 −6 1 1 @ 0 1 0 A @ 0 −1 −1 A = @ 0 −1 −1 A 0 1 1 0 1 0 0 0 −1 0 1 0 0 −11 2 2 1 0 1 0 0 −11 2 2 1 | {z } | {z } | {z } ! @ 0 −1 0 4 0 −1 A ! @ 0 1 0 −4 0 1 A : =E2 =E1S =E2E1S=U 0 0 −1 −6 1 1 0 0 1 6 −1 −1 Inverses of E1 and E2 can be easily computed: 0 −11 2 2 1 0 1 0 0 1 0 1 0 0 1 −1 −1 −1 S = @ −4 0 1 A : E1 = @ 2 1 0 A ;E2 = @ 0 1 0 A : 6 −1 −1 4 0 1 0 −1 1 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 3 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 4 We have If we have computed the LU decomposition E2E1S = U S = LU; then we can use it to solve Hence Sx = f: −1 −1 −1 E1S = E2 U; and S = E1 E2 U: We replace S by LU, LUx = f; and introduce y = Ux. This leads to the two linear systems 01 0 21 0 1 0 0 1 0 1 0 0 1 0 1 0 2 1 Ly = f and Ux = y: @2 −1 3A = @ 2 1 0 A @ 0 1 0 A @ 0 −1 −1 A 4 1 8 4 0 1 0 −1 1 0 0 −1 Since L is lower triangular and U is upper triangular, these two systems can be easily solved. | {z } | {z } | {z } | {z } Example: =S =E−1 =E−1 =U 1 2 0 1 0 1 0 1 0 1 0 1 0 1 1 0 2 1 0 0 1 0 2 −4 1 0 0 1 0 2 @2 −1 3A = @ 2 1 0 A @ 0 −1 −1 A; f = @ −6 A 4 1 8 4 −1 1 0 0 −1 −15 = @ 2 1 0 A @ 0 −1 −1 A | {z } | {z } | {z } 4 −1 1 0 0 −1 =S =L =U | {z } | {z } 0 1 0 0 1 0 y 1 0 −4 1 0 y 1 0−41 =E−1E−1=L =U 1 1 1 2 @ 2 1 0 A @ y2 A = @ −6 A ) @ y2 A = @ 2 A 4 −1 1 y3 −15 y3 3 | {z } | {z } Hence we have the LU-Decomposition of S, =L =f 0 1 0 1 0 1 0 1 0 1 S = LU; 1 0 2 x1 −4 x1 2 @ 0 −1 −1 A @ x2 A = @ 2 A ) @ x2 A = @ 1 A 0 0 −1 x3 3 x3 −3 where L is a lower triangular matrix and U is an upper triangular matrix. | {z } | {z } =U =y In Matlab compute using [L,U]=lu(S). M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 5 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 6 In Matlab the matrix inverse is computed using the LU decomposition. Execute the following: Given S, we want to compute S−1. Recall that the columns n = input('Problem size = '); X ;:::;X of the inverse S−1 = X are the solutions of :;1 :;n S = rand(n,n); f = rand(n,1); SX:;1 = e1; . ntry = 50; . SX:;n = en: tic for i = 1:ntry If we have computed the LU decomposition x = S\f; end S = LU; toc tic then we can use it to solve the n linear systems SX:;j = ej, j = 1; : : : n. Use the LU decomposition to compute the inverse of for i = 1:ntry Sinv = inv(S); 01 0 21 0 1 0 0 1 0 1 0 2 1 x = Sinv*f; 2 −1 3 = 2 1 0 0 −1 −1 end @ A @ A @ A toc 4 1 8 4 −1 1 0 0 −1 | {z } | {z } | {z } =S =L =U tic [L,U] =lu(S); What do you think of the following approach to solve Sx = f: for i = 1:ntry x = U\(L\f); >> Sinv = inv(S); end >> x = Sinv*f; toc M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 7 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition { 8.