LU-Decomposition Example 1 Compare with Example 1 in gaussian elimination.pdf. Consider I The Gaussian elimination for the solution of the linear system Sx = f transforms the augmented matrix (Sjf) into (Ujc), where U 0 2 4 −21 is upper triangular. The transformations are determined by the S = 4 9 −3 : (1) matrix S only. @ A If we have to solve a system Sx = f~ with a different right hand side −2 −3 7 f~, then we start over. Most of the computations that lead us from We express Gaussian Elimination using Matrix-Matrix-multiplications (Sjf~) into (Ujc~), depend only on S and are identical to the steps that we executed when we applied Gaussian elimination to Sx = f. 0 1 0 0 1 0 2 4 −2 1 0 2 4 −2 1 We now express Gaussian elimination as a sequence of matrix-matrix I @ −2 1 0 A @ 4 9 −3 A = @ 0 1 1 A multiplications. This representation leads to the decomposition of S 1 0 1 −2 −3 7 0 1 5 into a product of a lower triangular matrix L and an upper triangular | {z } | {z } | {z } matrix U, S = LU. This is known as the LU-Decomposition of S. =E1 =S =E1S I If we have computed the LU decomposition S = LU, then we can 0 1 0 0 1 0 2 4 −2 1 0 2 4 −2 1 use it to solve Sx = f. @ 0 1 0 A @ 0 1 1 A = @ 0 1 1 A I We replace S by LU, 0 −1 1 0 1 5 0 0 4 LUx = f; | {z } | {z } | {z } and introduce y = Ux. This leads to the two linear systems =E2 =E1S =E2E1S=U Ly = f and Ux = y: The inverses of E1 and E2 can be easily computed: Since L is lower triangular and U is upper triangular, these two 0 1 0 0 1 0 1 0 0 1 systems can be easily solved. −1 −1 E1 = @ 2 1 0 A ;E2 = @ 0 1 0 A : I The LU-decomposition of S can be re-used for the solution of linear −1 0 1 0 1 1 systems with other right hand sides. M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 1 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 2 We have If we have computed the LU decomposition S = LU, then we can use it to solve Sx = f. E2E1S = U We replace S by LU, LUx = f; Hence −1 −1 −1 and introduce y = Ux. This leads to the two linear systems E1S = E2 U; and S = E1 E2 U: Ly = f and Ux = y: 0 2 4 −21 0 1 0 0 1 0 1 0 0 1 0 2 4 −2 1 Since L is lower triangular and U is upper triangular, these two systems can be easily solved. @ 4 9 −3A = @ 2 1 0 A @ 0 1 0 A @ 0 1 1 A Example: −2 −3 7 −1 0 1 0 1 1 0 0 4 0 1 0 1 0 1 0 1 | {z } | {z } | {z } | {z } 2 4 −2 1 0 0 2 4 −2 2 =S =E−1 =E−1 =U @ 4 9 −3A = @ 2 1 0 A @ 0 1 1 A; f = @ 8 A 1 2 −2 −3 7 −1 1 1 0 0 4 10 0 1 0 1 | {z } | {z } | {z } 1 0 0 2 4 −2 =S =L =U = @ 2 1 0 A @ 0 1 1 A 0 1 0 1 0 1 0 1 0 1 −1 1 1 0 0 4 1 0 0 y1 2 y1 2 | {z } | {z } @ 2 1 0 A @ y2 A = @ 8 A ) @ y2 A = @ 8 A −1 −1 =U =E1 E2 =L −1 1 1 y3 10 y3 10 | {z } | {z } =L =f Hence we have the LU-Decomposition of S, 0 1 0 1 0 1 0 1 0 1 2 4 −2 x1 2 x1 −1 S = LU; @ 0 1 1 A @ x2 A = @4A ) @ x2 A = @ 2 A 0 0 4 x3 8 x3 2 | {z } | {z } where L is a lower triangular matrix and U is an upper triangular matrix. =U =y M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 3 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 4 The Matlab command inv(S) computes the inverse of S. Matlab does What do you think of the following approach to solve Sx = f: this using the LU decomposition. −1 Recall that the columns X:;1;:::;X:;n of the inverse S = X are the >> Sinv = inv(S); solutions of >> x = Sinv*f; SX:;1 = e1; . Execute the following Matlab script which uses the approach above to SX:;n = en: solve the linear systems, then uses the LU-decomposition to solve the linear system, and show the computing times needed by both approaches. To solve the n linear systems SX:;j = ej, j = 1; : : : n, the LU decomposition S = LU is used. This is essentially what Matlab does when you execute the command inv(S). Use the LU decomposition to compute (by hand) the inverse of 0 2 4 −21 0 1 0 0 1 0 2 4 −2 1 @ 4 9 −3A = @ 2 1 0 A @ 0 1 1 A : −2 −3 7 −1 1 1 0 0 4 | {z } | {z } | {z } =S =L =U Compare with Sinv = inv(S). M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 5 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 6 n = input('Problem size = '); Example 2 S = rand(n,n); Compare with Example 2 in gaussian elimination.pdf. Consider f = rand(n,1); 02 3 −21 ntry = 50; S = @1 −2 3 A : (2) 4 −1 4 tic for i = 1:ntry We express Gaussian Elimination using Matrix-Matrix-multiplications x = S\f; end 0 1 0 0 1 0 2 3 −2 1 0 2 3 −2 1 toc @ −1=2 1 0 A @ 1 −2 3 A = @ 0 −7=2 4 A −2 0 1 4 −1 4 0 −7 8 tic | {z } | {z } | {z } for i = 1:ntry =E1 =S =E1S Sinv = inv(S); 0 1 0 0 1 0 2 3 −2 1 0 2 3 −2 1 x = Sinv*f; @ 0 1 0 A @ 0 −7=2 4 A = @ 0 −7=2 4 A end 0 −2 1 0 −7 8 0 0 0 toc | {z } | {z } | {z } =E2 =E1S =E2E1S=U tic [L,U] =lu(S); The inverses of E1 and E2 can be easily computed: for i = 1:ntry 0 1 0 0 1 0 1 0 0 1 x = U\(L\f); −1 −1 end E1 = @ 1=2 1 0 A ;E2 = @ 0 1 0 A : toc 2 0 1 0 2 1 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 7 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 8 We have E2E1S = U Numerical Computation of the LU-Decomposition Hence −1 −1 −1 E1S = E U; and S = E E U: 2 1 2 I The LU-decomposition of a matrix as introduced on the previous slides is fine if all calculations are performed exactly and if no row 02 3 −21 0 1 0 0 1 0 1 0 0 1 0 2 3 −2 1 interchanges are performed. 1 −2 3 = 1=2 1 0 0 1 0 0 −7=2 4 @ A @ A @ A @ A I If the LU decomposition is computed using computer programs in 4 −1 4 2 0 1 0 2 1 0 0 0 Matlab, C, Fortran,..., then computations are no lo longer performed | {z } | {z } | {z } | {z } =S −1 −1 =U exactly, but in floating point arithmetic. =E1 =E2 0 1 0 0 1 0 2 3 −2 1 I To avoid that errors due to floating point arithmetic are amplified too much, the LU decomposition in computed slightly differently. = @ 1=2 1 0 A @ 0 −7=2 4 A 2 2 1 0 0 0 For example, the LU decomposition of a matrix S can be computed | {z } | {z } in Matlab using [L,U]=lu(S), but the matrices L and U that we −1 −1 =U =E1 E2 =L compute by hand are in general different from the matrices L and U computed by Matlab's [L,U]=lu(S). Hence we have the LU-Decomposition of S, I Such issues are discussed in CAAM 353 Computational Numerical S = LU; Analysis and CAAM 453 Numerical Analysis I. where L is a lower triangular matrix and U is an upper triangular matrix. M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 9 M. Heinkenschloss - CAAM335 Matrix Analysis LU Decomposition (updated September 1, 2010) { 10.