Householder Matrix H( W )= E ( W , W ,2) = I − 2 Wwh for Wh W =1

Outline

 Elementary Matrix  Full-rank Decomposition  Triangular Matrix Decomposition  QR Decomposition  Schur Theorem and Normal Matrix

1 Elementary Matrix

 Elementary Matrix E(,, u vσσ )= I − uvH

Identity matrix plus a rank-one matrix  Properties on the Elementary Matrix  Determinant Euv(,,σσ )= 1 − vH u σ  inverse matrix Euv(,,σ )−1 = Euv (,, ) σ vuH −1  for any two nonzero vectors a and b, their exists an elementary matrix such that Euv(,,σ ) a= b

2 Elementary Matrix

 Determinant Euv(,,σσ )= 1 − vH u uH uuHH u H E(,, u v σσ )=−=−uvHH u σ v , u uu u H uuHH −=−uσσ v1, vu uu uH E(,,) u vσ = u HH if u u = 0. 0 00 σ  Inverse Matrix Euv(,,σ )−1 = Euv (,, ) σ vuH −1 brute-force proof  For any two nonzero vectors a and b, their exists an elementary matrix such that E(u,v,σ )a = b

3 Elementary Matrix

 For any two nonzero vectors a and b, their exists an elementary matrix such that Euv(,,σ ) a= b E(,, u vσσ ) a=−= a uvH a b ⇒=−σuvH a a b ⇒=−u a b,1σ vaH =

4 Elementary Matrix

 Householder Matrix H( w )= E ( w , w ,2) = I − 2 wwH for wH w =1

 Properties on the Householder Matrix  determinant det(Hw ( ))=−=− 1 2 wwH 1

 inverse Hw()−1 = Hw ()H = Hw ()

5 Full-Rank Decomposition

 Full-rank Decomposition Theorem: For m*n matrix A and det(A) = r, there exists m*r full- rank matrix B and r*n full-rank matrix C, such that A = BC. Proof: SVD of matrix A, A = UΣVH find the non-zero elements of Σ

6 Triangular Matrix Decomposition

 LU Decomposition: There exists a LU decomposition A = LU for a non-singular matrix A, where L is an unit lower triangular matrix and U is an upper triangular matrix, if and only if a11 a12 ... a1k    a a .. a A =  21 22 2k  ≠ 0 k  ......    ak1 ak 2 ... akk  for all k.

7 Triangular Matrix Decomposition

 LDU Decomposition: There exists a LDU decomposition A = LDU for a non-singular matrix A, where L is an unit lower triangular matrix, U is an unit upper triangular matrix, and D is a diagonal matrix if and only if

a11 a12 ... a1k    a a .. a A =  21 22 2k  ≠ 0 k  ......    for all k. ak1 ak 2 ... akk 

8 Triangular Matrix Decomposition

 LU Decomposition: Construction  Preliminary Row Transform for Matrix A, from bottom to top, converting it to a upper-triangular

−1 matrix NN ∏∏LAii=⇒= U A L U ii=11=  Preliminary Column Transform for Matrix U, from left to right, converting it to a diagonal matrix −−11 NN NN   ∏∏LAii U=⇒= D A ∏ Li  D ∏ U i  ii=11 = i=1  i= 1 

9 QR Decomposition

 QR Decomposition: For a non-singular matrix A, there exists an unitary matrix Q and an upper triangular matrix R, such that A = QR. Proof: the Gram-Schmidt orthogonalization on the columns of matrix A

10 Schur Theorem

 Unitary similarity: A and B are unitary similar, if and only if B = UHAU, for an unitary matrix U  Schur Theorem: For any n*n matrix A, there exists an unitary matrix U and an upper- triangular matrix R, such that UHAU = R. Proof: Mathematical induction, left for homework

11 Normal Matrix

 Normal Matrix: AAH = AHA  Matrix A is Unitary similarity to a diagonal matrix, if and only if A is a normal matrix. Proof: based on the Schur Theorem, prove that the upper triangular matrix is diagonal, left for homework

12 Normal Matrix

 Diagonization of Normal Matrix  Matrices A and B are normal matrices, and AB = BA, then there exists an unitary matrix U such that UHAU and UHBU are both diagonal. Proof: left for homework