Matrix Inverse Computation of the Inverse n×n A square matrix S ∈ R is invertible if there exists a matrix We want to find the inverse of S ∈ Rn×n, that is we want to find a −1 n×n S ∈ R such that matrix X ∈ Rn×n such that SX = I. −1 −1 S S = I and SS = I. I Let X:,j denote the jth column of X, i.e., X = (X:,1,...,X:,n). Consider the matrix-matrix product SX. The jth column of SX is The matrix S−1 is called the inverse of S. the matrix-vector product SX:,j, i.e., SX = (SX:,1,...,SX:,n). I An is also called non-singular. The jth column of the identity I is the jth unit vector A matrix is called non-invertible or singular if it is not invertible. e = (0,..., 0, 1, 0,..., 0)T . n×n j I A matrix S ∈ cannot have two different inverses. R Hence SX = (SX:,1,...,SX:,n) = (e1, . . . , en) = I implies that we In fact, if X,Y ∈ n×n are two matrices with XS = I and SY = I, R can compute the columns X ,...,X of the inverse of S by then :,1 :,n X = XI = X(SY ) = (XS)Y = IY = Y. solving n systems of linear equations n×n −1 −1 I If S ∈ R is invertible, then Sx = f implies x = S Sx = S f, SX:,1 = e1, i.e., for every f the linear system Sx = f has a solution x = S−1f. . The linear system Sx = f cannot have more than one solution . because Sx = f and Sy = f imply S(x − y) = Sx − Sy = f − f = 0 and x − y = S−10 = 0. SX:,n = en. Hence if S is invertible, then for every f the linear system Sx = f has the unique solution x = S−1f. Note that if for every f the linear system Sx = f has a unique I We will see later that if for every f the linear system Sx = f has a solution x, then there exists a unique X = (X:,1,...,X:,n) with unique solution x, then S is invertible. SX = I.

M. Heinkenschloss - CAAM335 Matrix Inverse and LU Decomposition – 1 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 2

Want inverse of 1 0 2 LU-Decomposition S = 2 −1 3 . Consider     4 1 8 1 0 2 S = 2 −1 3 . Use to solve the systems 4 1 8 SX = e ,SX = e ,SX = e for the three columns of X = S−1 :,1 1 :,2 2 :,3 3 Express Gaussian Elimination using Matrix-Matrix-multiplications     1 0 2 1 0 0 1 0 2 1 0 0  1 0 0   1 0 2   1 0 2   2 −1 3 0 1 0  →  0 −1 −1 −2 1 0   −2 1 0   2 −1 3  =  0 −1 −1  4 1 8 0 0 1 0 1 0 −4 0 1 −4 0 1 4 1 8 0 1 0   | {z } | {z } | {z } 1 0 2 1 0 0 =E1 =S =E1S       →  0 −1 −1 −2 1 0  1 0 0 1 0 2 1 0 2 0 0 −1 −6 1 1  0 1 0   0 −1 −1  =  0 −1 −1  0 1 1 0 1 0 0 0 −1  1 0 0 −11 2 2   1 0 0 −11 2 2  | {z } | {z } | {z } →  0 −1 0 4 0 −1  →  0 1 0 −4 0 1  . =E2 =E1S =E2E1S=U 0 0 −1 −6 1 1 0 0 1 6 −1 −1 Inverses of E1 and E2 can be easily computed:

 −11 2 2   1 0 0   1 0 0  −1 −1 −1 S =  −4 0 1  . E1 =  2 1 0  ,E2 =  0 1 0  . 6 −1 −1 4 0 1 0 −1 1

M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 3 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 4 We have If we have computed the LU decomposition E2E1S = U S = LU, then we can use it to solve Hence Sx = f. −1 −1 −1 E1S = E2 U, and S = E1 E2 U. We replace S by LU, LUx = f, and introduce y = Ux. This leads to the two linear systems 1 0 2  1 0 0   1 0 0   1 0 2  Ly = f and Ux = y. 2 −1 3 =  2 1 0   0 1 0   0 −1 −1  4 1 8 4 0 1 0 −1 1 0 0 −1 Since L is lower triangular and U is upper triangular, these two systems can be easily solved. | {z } | {z } | {z } | {z } Example: =S =E−1 =E−1 =U 1 2             1 0 2 1 0 0 1 0 2 −4 1 0 0 1 0 2 2 −1 3 =  2 1 0   0 −1 −1 , f =  −6  4 1 8 4 −1 1 0 0 −1 −15 =  2 1 0   0 −1 −1  | {z } | {z } | {z } 4 −1 1 0 0 −1 =S =L =U

| {z } | {z }  1 0 0   y   −4   y  −4 =E−1E−1=L =U 1 1 1 2  2 1 0   y2  =  −6  ⇒  y2  =  2  4 −1 1 y3 −15 y3 3 | {z } | {z } Hence we have the LU-Decomposition of S, =L =f

          S = LU, 1 0 2 x1 −4 x1 2  0 −1 −1   x2  =  2  ⇒  x2  =  1  0 0 −1 x3 3 x3 −3 where L is a lower and U is an upper triangular matrix. | {z } | {z } =U =y In Matlab compute using [L,U]=lu(S). M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 5 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 6

In Matlab the matrix inverse is computed using the LU decomposition. Execute the following: Given S, we want to compute S−1. Recall that the columns n = input(’Problem size = ’); X ,...,X of the inverse S−1 = X are the solutions of :,1 :,n S = rand(n,n); f = rand(n,1); SX:,1 = e1, . ntry = 50; .

SX:,n = en. tic for i = 1:ntry If we have computed the LU decomposition x = S\f; end S = LU, toc

tic then we can use it to solve the n linear systems SX:,j = ej, j = 1, . . . n. Use the LU decomposition to compute the inverse of for i = 1:ntry Sinv = inv(S); 1 0 2  1 0 0   1 0 2  x = Sinv*f; 2 −1 3 = 2 1 0 0 −1 −1 end       toc 4 1 8 4 −1 1 0 0 −1 | {z } | {z } | {z } =S =L =U tic [L,U] =lu(S); What do you think of the following approach to solve Sx = f: for i = 1:ntry x = U\(L\f); >> Sinv = inv(S); end >> x = Sinv*f; toc

M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 7 M. Heinkenschloss - CAAM335 Matrix Analysis Matrix Inverse and LU Decomposition – 8