LU LU Decomposition  To further improve the efficiency of solving linear systems  of A : LU and QR  A matrix A can be decomposed into a lower L and upper triangular matrix U so that  LU Factorization Methods: Using basic (GE) A = LU ◦ Factorization of Tridiagonal Matrix ◦  LU decomposition is performed once; can be used to solve multiple Using Gaussian Elimination with pivoting ◦ right hand sides. Direct LU Factorization ◦  Similar to Gaussian elimination, care must be taken to avoid roundoff Factorizing Symmetrix Matrices () ◦ errors (partial or full pivoting)  Applications  Special Cases: Banded matrices, Symmetric matrices.  Analysis

ITCS 4133/5133: Intro. to Numerical Methods 1 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 2 LU/QR Factorization

LU Decomposition: Motivation LU Decomposition Forward pass of Gaussian elimination results in the upper triangular ma- trix

1 u12 u13 u1n 0 1 u ··· u 23 ··· 2n U = 0 0 1 u3n . . . ···. . . . . . .  0 0 0 1     ···  We can determine a matrix L such that

LU = A , and

l11 0 0 0 l l 0 ··· 0 21 22 ··· L = l31 l32 l33 0 . . . ···. .  . . . . . l l l 1  n1 n2 n3 ···  ITCS 4133/5133: Intro. to Numerical Methods 3 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 4  LU/QR Factorization LU Decomposition via Basic Gaussian Elimination LU Decomposition via Basic Gaussian Elimination: Algorithm

ITCS 4133/5133: Intro. to Numerical Methods 5 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 6 LU/QR Factorization

LU Decomposition of Tridiagonal Matrix Banded Matrices

 Matrices that have non-zero elements close to the main diagonal  Example: matrix with a bandwidth of 3 (or half band of 1)

a11 a12 0 0 0 a21 a22 a23 0 0  0 a32 a33 a34 0   0 0 a43 a44 a45  0 0 0 a a   54 55    Efficiency: Reduced pivoting needed, as elements below bands are zero.  Iterative techniques are generally more efficient for sparse matrices, such as banded systems.

ITCS 4133/5133: Intro. to Numerical Methods 7 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 8 LU/QR Factorization LU Decomposition via GE with Pivoting LU Decomposition of Tridiagonal Matrix: Algorithm

ITCS 4133/5133: Intro. to Numerical Methods 9 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 10 LU/QR Factorization

LU Decomposition via GE with Pivoting: Algorithm

Direct LU Decomposition

l11 0 0 0 1 u12 u13 u1n a11 a12 a13 a1n l l 0 ··· 0 0 1 u ··· u a a a ··· a  21 22 ···   23 ··· 2n  21 22 23 ··· 2n l31 l32 l33 0 0 0 1 u3n = a31 a32 a33 a2n  . . . ···. . . . . ···. .   . . . ···. .   . . . . . . . . . .   . . . . .        ln1 ln2 ln3 1 0 0 0 1  an1 an2 an3 ann  ···   ···   ···        The above equation shows the coefficient matrix A can be decomposed into L and U; L and U can be determined by the computing the product and equating like terms.

ITCS 4133/5133: Intro. to Numerical Methods 11 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 12 LU/QR Factorization Direct LU Decomposition: Example

Determining L and U

li1 = ai1, i = 1, 2, . . . , n a1j u1j = , j = 2, 3, . . . , n l11 j 1 − l = a l u , ij ij − ik kj k=1 for j X= 2, 3, . . . , n 1 and i = j, j + 1, . . . , n j 1 − a − l u u = ji − k=1 jk ki , ji l Pjj for j = 2, 3, . . . , n 1 and i = j + 1, j + 2, . . . , n n 1 − − l = a l u nn nn − nk kn k=1 X

ITCS 4133/5133: Intro. to Numerical Methods 13 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 14 LU/QR Factorization

Direct LU Decomposition: Algorithm Symmetric Matrices

 Symmetric square matrices - common in engineering, for example stiffness matrix (stiffness properties of structures).  For a symmetric matrix A A = AT

where AT is the transpose of A. Hence  For a symmetric matrix A A = LLT

ITCS 4133/5133: Intro. to Numerical Methods 15 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 16 LU/QR Factorization LU (Cholesky) Decomposition of Symmetric Matrices Symmetric Matrices: LU Decomposition

 We can use Cholesky Decomposition to compute the LU decompo- sition of A

l11 = √a11

i 1 − l = a l2 , for i = 2, 3, . . . , n ii v ii − ik u k=1 u Xj 1 taij − likljk l = − k=1 , for j = 2, 3, . . . , i 1, and j < 1 ij l − Pjj

ITCS 4133/5133: Intro. to Numerical Methods 17 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 18 LU/QR Factorization

LU (Cholesky) Decomposition: Algorithm Applications of LU Decomposition:Solving Linear systems

[A]X~ = [L][U][X~ ] = C = [L]~e = C

where ~e = [U]X~  Solve for ~e using L (forward substitution)  Solve for X~ using U (back substitution)

ITCS 4133/5133: Intro. to Numerical Methods 19 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 20 LU/QR Factorization Solving Linear Systems: Example Solving Linear Systems

Forward Substitution Solve for ~e,

C1 e1 = l11 n Ci j=1 lijej ei = − , for i = 2, 3, . . . n Plii Back Substitution Solve for X~

Xn = en n X = e u X , for i = n 1, n 2,... 1 i i − ij j − − j=i+1 X ITCS 4133/5133: Intro. to Numerical Methods 21 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 22 LU/QR Factorization

Solving Linear Systems: Algorithm Application 2: Tridiagonal Systems

ITCS 4133/5133: Intro. to Numerical Methods 23 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 24 LU/QR Factorization Application 3: Matrix Inverse Analysis: LU Decomposition  Why does LU decomposition work?  Consider the first elimination step:

1 0 0 a11 a12 a13 a11 a12 a13 0 0 m21 1 0 a21 a22 a23 = 0 a22 a23      0 0  m31 0 1 a31 a32 a33 0 a32 a33        Define 1 0 0 1 0 0 M = m 1 0 ,M = 0 1 0 1  21  2   m31 0 1 0 m32 1      Hence 1 0 0 1 0 0 1 1 M1− = m21 1 0 ,M2− = 0 1 0 , −m 0 1 0 m 1 − 31 − 32 ITCS 4133/5133: Intro. to Numerical Methods 25 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 26   LU/QR Factorization

Analysis:LU Decomposition with Pivoting  Decomposition results in the factorization of PA, where P is the Analysis: LU Decomposition . 1 0 0 a11 a12 a13 a11 a12 a13 0 1 0 a21 a22 a23 = a21 a22 a23  Gradual tranformation of I towards LU: 0 0 1 a31 a32 a33 a31 a32 a33 I.A = A  Assume no pivoting in column 1, pivoting in column 2: 1 I.M1− .M1.A = A 1 1 I.A = A I.M1− .M2− .M2.M1.A = A 1 I.M1− .M1.A = A 1 1 1  M2.M1.A is U and I.M1− .M2− is L, given by I.M1− .P.P.M1.A = A 1 1 I.M1− .P.M2− .M2.P.M1.A = A 1 0 0 m 1 0 1 1 21  M2.P.M1.A is upper triangular, but I.M1− .P.M2− is not lower tri- −m m 1 − 31 − 32 angular.    Premultiply both sides by P : 1 1 PM1− .P.M2− .M2.P.M1.A = P.A ITCS 4133/5133: Intro. to Numerical Methods 27 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 28 LU/QR Factorization ITCS 4133/5133: Intro. to Numerical Methods 29 LU/QR Factorization