LU-Decomposition Example 1 Compare with Example 1 in .pdf. Consider I The Gaussian elimination for the solution of the linear system Sx = f transforms the augmented (S|f) into (U|c), where U  2 4 −2 is upper triangular. The transformations are determined by the S = 4 9 −3 . (1) matrix S only.   If we have to solve a system Sx = f˜ with a different right hand side −2 −3 7 f˜, then we start over. Most of the computations that lead us from We express Gaussian Elimination using Matrix-Matrix-multiplications (S|f˜) into (U|c˜), depend only on S and are identical to the steps that we executed when we applied Gaussian elimination to Sx = f.  1 0 0   2 4 −2   2 4 −2  We now express Gaussian elimination as a sequence of matrix-matrix I  −2 1 0   4 9 −3  =  0 1 1  multiplications. This representation leads to the decomposition of S 1 0 1 −2 −3 7 0 1 5 into a product of a lower L and an upper triangular | {z } | {z } | {z } matrix U, S = LU. This is known as the LU-Decomposition of S. =E1 =S =E1S I If we have computed the LU decomposition S = LU, then we can  1 0 0   2 4 −2   2 4 −2  use it to solve Sx = f.  0 1 0   0 1 1  =  0 1 1  I We replace S by LU, 0 −1 1 0 1 5 0 0 4 LUx = f, | {z } | {z } | {z } and introduce y = Ux. This leads to the two linear systems =E2 =E1S =E2E1S=U

Ly = f and Ux = y. The inverses of E1 and E2 can be easily computed:

Since L is lower triangular and U is upper triangular, these two  1 0 0   1 0 0  systems can be easily solved. −1 −1 E1 =  2 1 0  ,E2 =  0 1 0  . I The LU-decomposition of S can be re-used for the solution of linear −1 0 1 0 1 1 systems with other right hand sides.

M. Heinkenschloss - CAAM335 LU Decomposition (updated September 1, 2010) – 1 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 2

We have If we have computed the LU decomposition S = LU, then we can use it to solve Sx = f. E2E1S = U We replace S by LU, LUx = f, Hence −1 −1 −1 and introduce y = Ux. This leads to the two linear systems E1S = E2 U, and S = E1 E2 U. Ly = f and Ux = y.  2 4 −2  1 0 0   1 0 0   2 4 −2  Since L is lower triangular and U is upper triangular, these two systems can be easily solved.  4 9 −3 =  2 1 0   0 1 0   0 1 1  Example: −2 −3 7 −1 0 1 0 1 1 0 0 4         | {z } | {z } | {z } | {z } 2 4 −2 1 0 0 2 4 −2 2 =S =E−1 =E−1 =U  4 9 −3 =  2 1 0   0 1 1 , f =  8  1 2 −2 −3 7 −1 1 1 0 0 4 10     | {z } | {z } | {z } 1 0 0 2 4 −2 =S =L =U =  2 1 0   0 1 1            −1 1 1 0 0 4 1 0 0 y1 2 y1 2 | {z } | {z }  2 1 0   y2  =  8  ⇒  y2  =  8  −1 −1 =U =E1 E2 =L −1 1 1 y3 10 y3 10 | {z } | {z } =L =f Hence we have the LU-Decomposition of S,           2 4 −2 x1 2 x1 −1 S = LU,  0 1 1   x2  = 4 ⇒  x2  =  2  0 0 4 x3 8 x3 2 | {z } | {z } where L is a lower triangular matrix and U is an upper triangular matrix. =U =y

M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 3 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 4 The Matlab command inv(S) computes the inverse of S. Matlab does What do you think of the following approach to solve Sx = f: this using the LU decomposition. −1 Recall that the columns X:,1,...,X:,n of the inverse S = X are the >> Sinv = inv(S); solutions of >> x = Sinv*f;

SX:,1 = e1, . . Execute the following Matlab script which uses the approach above to

SX:,n = en. solve the linear systems, then uses the LU-decomposition to solve the linear system, and show the computing times needed by both approaches. To solve the n linear systems SX:,j = ej, j = 1, . . . n, the LU decomposition S = LU is used. This is essentially what Matlab does when you execute the command inv(S).

Use the LU decomposition to compute (by hand) the inverse of

 2 4 −2  1 0 0   2 4 −2   4 9 −3 =  2 1 0   0 1 1  . −2 −3 7 −1 1 1 0 0 4 | {z } | {z } | {z } =S =L =U Compare with Sinv = inv(S). M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 5 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 6

n = input(’Problem size = ’); Example 2 S = rand(n,n); Compare with Example 2 in gaussian elimination.pdf. Consider f = rand(n,1); 2 3 −2 ntry = 50; S = 1 −2 3  . (2) 4 −1 4 tic for i = 1:ntry We express Gaussian Elimination using Matrix-Matrix-multiplications x = S\f; end  1 0 0   2 3 −2   2 3 −2  toc  −1/2 1 0   1 −2 3  =  0 −7/2 4  −2 0 1 4 −1 4 0 −7 8 tic | {z } | {z } | {z } for i = 1:ntry =E1 =S =E1S Sinv = inv(S);  1 0 0   2 3 −2   2 3 −2  x = Sinv*f;  0 1 0   0 −7/2 4  =  0 −7/2 4  end 0 −2 1 0 −7 8 0 0 0 toc | {z } | {z } | {z } =E2 =E1S =E2E1S=U tic [L,U] =lu(S); The inverses of E1 and E2 can be easily computed: for i = 1:ntry  1 0 0   1 0 0  x = U\(L\f); −1 −1 end E1 =  1/2 1 0  ,E2 =  0 1 0  . toc 2 0 1 0 2 1

M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 7 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 8 We have E2E1S = U Numerical Computation of the LU-Decomposition Hence −1 −1 −1 E1S = E U, and S = E E U. 2 1 2 I The LU-decomposition of a matrix as introduced on the previous slides is fine if all calculations are performed exactly and if no row 2 3 −2  1 0 0   1 0 0   2 3 −2  interchanges are performed. 1 −2 3 = 1/2 1 0 0 1 0 0 −7/2 4         I If the LU decomposition is computed using computer programs in 4 −1 4 2 0 1 0 2 1 0 0 0 Matlab, C, Fortran,..., then computations are no lo longer performed | {z } | {z } | {z } | {z } =S −1 −1 =U exactly, but in floating point arithmetic. =E1 =E2  1 0 0   2 3 −2  I To avoid that errors due to floating point arithmetic are amplified too much, the LU decomposition in computed slightly differently. =  1/2 1 0   0 −7/2 4  2 2 1 0 0 0 For example, the LU decomposition of a matrix S can be computed | {z } | {z } in Matlab using [L,U]=lu(S), but the matrices L and U that we −1 −1 =U =E1 E2 =L compute by hand are in general different from the matrices L and U computed by Matlab’s [L,U]=lu(S). Hence we have the LU-Decomposition of S, I Such issues are discussed in CAAM 353 Computational Numerical S = LU, Analysis and CAAM 453 I. where L is a lower triangular matrix and U is an upper triangular matrix.

M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 9 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) – 10