LU Factorization LU Decomposition To further improve the efficiency of solving linear systems Factorizations of matrix A : LU and QR A matrix A can be decomposed into a lower triangular matrix L and upper triangular matrix U so that LU Factorization Methods: Using basic Gaussian Elimination (GE) A = LU ◦ Factorization of Tridiagonal Matrix ◦ LU decomposition is performed once; can be used to solve multiple Using Gaussian Elimination with pivoting ◦ right hand sides. Direct LU Factorization ◦ Similar to Gaussian elimination, care must be taken to avoid roundoff Factorizing Symmetrix Matrices (Cholesky Decomposition) ◦ errors (partial or full pivoting) Applications Special Cases: Banded matrices, Symmetric matrices. Analysis ITCS 4133/5133: Intro. to Numerical Methods 1 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 2 LU/QR Factorization LU Decomposition: Motivation LU Decomposition Forward pass of Gaussian elimination results in the upper triangular ma- trix 1 u12 u13 u1n 0 1 u ··· u 23 ··· 2n U = 0 0 1 u3n . ···. 0 0 0 1 ··· We can determine a matrix L such that LU = A , and l11 0 0 0 l l 0 ··· 0 21 22 ··· L = l31 l32 l33 0 . ···. . l l l 1 n1 n2 n3 ··· ITCS 4133/5133: Intro. to Numerical Methods 3 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 4 LU/QR Factorization LU Decomposition via Basic Gaussian Elimination LU Decomposition via Basic Gaussian Elimination: Algorithm ITCS 4133/5133: Intro. to Numerical Methods 5 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 6 LU/QR Factorization LU Decomposition of Tridiagonal Matrix Banded Matrices Matrices that have non-zero elements close to the main diagonal Example: matrix with a bandwidth of 3 (or half band of 1) a11 a12 0 0 0 a21 a22 a23 0 0 0 a32 a33 a34 0 0 0 a43 a44 a45 0 0 0 a a 54 55 Efficiency: Reduced pivoting needed, as elements below bands are zero. Iterative techniques are generally more efficient for sparse matrices, such as banded systems. ITCS 4133/5133: Intro. to Numerical Methods 7 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 8 LU/QR Factorization LU Decomposition via GE with Pivoting LU Decomposition of Tridiagonal Matrix: Algorithm ITCS 4133/5133: Intro. to Numerical Methods 9 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 10 LU/QR Factorization LU Decomposition via GE with Pivoting: Algorithm Direct LU Decomposition l11 0 0 0 1 u12 u13 u1n a11 a12 a13 a1n l l 0 ··· 0 0 1 u ··· u a a a ··· a 21 22 ··· 23 ··· 2n 21 22 23 ··· 2n l31 l32 l33 0 0 0 1 u3n = a31 a32 a33 a2n . ···. ···. . ···. . . ln1 ln2 ln3 1 0 0 0 1 an1 an2 an3 ann ··· ··· ··· The above equation shows the coefficient matrix A can be decomposed into L and U; L and U can be determined by the computing the product and equating like terms. ITCS 4133/5133: Intro. to Numerical Methods 11 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 12 LU/QR Factorization Direct LU Decomposition: Example Determining L and U li1 = ai1, i = 1, 2, . , n a1j u1j = , j = 2, 3, . , n l11 j 1 − l = a l u , ij ij − ik kj k=1 for j X= 2, 3, . , n 1 and i = j, j + 1, . , n j 1 − a − l u u = ji − k=1 jk ki , ji l Pjj for j = 2, 3, . , n 1 and i = j + 1, j + 2, . , n n 1 − − l = a l u nn nn − nk kn k=1 X ITCS 4133/5133: Intro. to Numerical Methods 13 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 14 LU/QR Factorization Direct LU Decomposition: Algorithm Symmetric Matrices Symmetric square matrices - common in engineering, for example stiffness matrix (stiffness properties of structures). For a symmetric matrix A A = AT where AT is the transpose of A. Hence For a symmetric matrix A A = LLT ITCS 4133/5133: Intro. to Numerical Methods 15 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 16 LU/QR Factorization LU (Cholesky) Decomposition of Symmetric Matrices Symmetric Matrices: LU Decomposition We can use Cholesky Decomposition to compute the LU decompo- sition of A l11 = √a11 i 1 − l = a l2 , for i = 2, 3, . , n ii v ii − ik u k=1 u Xj 1 taij − likljk l = − k=1 , for j = 2, 3, . , i 1, and j < 1 ij l − Pjj ITCS 4133/5133: Intro. to Numerical Methods 17 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 18 LU/QR Factorization LU (Cholesky) Decomposition: Algorithm Applications of LU Decomposition:Solving Linear systems [A]X~ = [L][U][X~ ] = C = [L]~e = C where ~e = [U]X~ Solve for ~e using L (forward substitution) Solve for X~ using U (back substitution) ITCS 4133/5133: Intro. to Numerical Methods 19 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 20 LU/QR Factorization Solving Linear Systems: Example Solving Linear Systems Forward Substitution Solve for ~e, C1 e1 = l11 n Ci j=1 lijej ei = − , for i = 2, 3, . n Plii Back Substitution Solve for X~ Xn = en n X = e u X , for i = n 1, n 2,... 1 i i − ij j − − j=i+1 X ITCS 4133/5133: Intro. to Numerical Methods 21 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 22 LU/QR Factorization Solving Linear Systems: Algorithm Application 2: Tridiagonal Systems ITCS 4133/5133: Intro. to Numerical Methods 23 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 24 LU/QR Factorization Application 3: Matrix Inverse Analysis: LU Decomposition Why does LU decomposition work? Consider the first elimination step: 1 0 0 a11 a12 a13 a11 a12 a13 0 0 m21 1 0 a21 a22 a23 = 0 a22 a23 0 0 m31 0 1 a31 a32 a33 0 a32 a33 Define 1 0 0 1 0 0 M = m 1 0 ,M = 0 1 0 1 21 2 m31 0 1 0 m32 1 Hence 1 0 0 1 0 0 1 1 M1− = m21 1 0 ,M2− = 0 1 0 , −m 0 1 0 m 1 − 31 − 32 ITCS 4133/5133: Intro. to Numerical Methods 25 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 26 LU/QR Factorization Analysis:LU Decomposition with Pivoting Decomposition results in the factorization of PA, where P is the Analysis: LU Decomposition permutation matrix. 1 0 0 a11 a12 a13 a11 a12 a13 0 1 0 a21 a22 a23 = a21 a22 a23 Gradual tranformation of I towards LU: 0 0 1 a31 a32 a33 a31 a32 a33 I.A = A Assume no pivoting in column 1, pivoting in column 2: 1 I.M1− .M1.A = A 1 1 I.A = A I.M1− .M2− .M2.M1.A = A 1 I.M1− .M1.A = A 1 1 1 M2.M1.A is U and I.M1− .M2− is L, given by I.M1− .P.P.M1.A = A 1 1 I.M1− .P.M2− .M2.P.M1.A = A 1 0 0 m 1 0 1 1 21 M2.P.M1.A is upper triangular, but I.M1− .P.M2− is not lower tri- −m m 1 − 31 − 32 angular. Premultiply both sides by P : 1 1 PM1− .P.M2− .M2.P.M1.A = P.A ITCS 4133/5133: Intro. to Numerical Methods 27 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 28 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 29 LU/QR Factorization.