Second Year Mathematics Revision Linear Algebra - Part 2
Total Page:16
File Type:pdf, Size:1020Kb
Second Year Mathematics Revision Linear Algebra - Part 2 Rui Tong UNSW Mathematics Society Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Today's plan 1 Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) 2 The Jordan Canonical Form Finding Jordan Forms The Cayley-Hamilton Theorem 3 Matrix Exponentials Computing Matrix Exponentials Application to Systems of Differential Equations Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) Singular Values (MATH2601 only section) Definition 1: Singular Values A singular value of a m × n matrix A is the square root of an eigenvalue of A∗A. Recall: A∗A denotes the adjoint of A. Definition 2: Singular Value Decomposition A SVD for an m × n matrix A is of the form A = UΣV ∗ where U is an m × m unitary matrix. V is an n × n unitary matrix. Σ has entries σii > 0. (These are determined by the singular values.) σij = 0 for all i 6= j. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) Nice properties of A∗A Lemma 1: Properties of A∗A 1 All eigenvalues of A∗A are real and non-negative (even if A has complex entries!) 2 ker(A∗A) = ker(A) 3 rank(A∗A) = rank(A) The first one is pretty much why everything works. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Algorithm Algorithm 1: Finding a SVD ∗ 1 Find all eigenvalues λi of A A and write in descending order. Also find their associated eigenvectors of unit length vi . 2 Find an orthonormal set of eigenvectors for A∗A. Automatically occurs when all eigenvalues are distinct, which will usually be the case. Otherwise require Gram-Schmidt for any eigenspace with dimension strictly greater than 1.. 1 3 Compute ui = p Avi for each non-zero eigenvalue. λi 4 State U and V from the vectors you found, and Σ from the singular values. Lemma 2: Used to speed up step 1 A∗A and AA∗ share the same non-zero eigenvalues. If rank(A) = r, then A∗A has r non-zero eigenvalues. All other eigenvalues are 0. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Example 1: MATH2601 2017 Q2 c) 3 1 1 For the matrix A = −1 3 1 1 Find the eigenvalues of AA∗. 2 Explain why the eigenvalues in part 1 are also eigenvalues of A∗A, and state any other eigenvalues of A∗A. 3 Find all eigenvectors of A∗A. 4 Find a singular value decomposition for A. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 1: We compute 03 −11 3 1 1 11 1 AA∗ = 1 3 = : −1 3 1 @ A 1 11 1 1 This matrix has two eigenvalues, and they sum to tr(AA∗) = 22 and ∗ multiply to det(AA ) = 120. By inspection, λ1 = 12 and λ2 = 10. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 2: Quoted word for word from the answers... "We know that A∗A and AA∗ have the same nonzero eigenvalues, so 12 and 10 are eigenvalues of A∗A. Also, all eigenvalues of A∗A are real and nonnegative, so its third eigenvalue is 0." Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 3: We compute 03 −11 010 0 21 3 1 1 A∗A = 1 3 = 0 10 4 @ A −1 3 1 @ A 1 1 2 4 2 For λ = 12: 0−2 0 2 1 ∗ A A − 12I = @ 0 −2 4 A : 2 4 −10 Looking at row 1, arbitrarily set first component to 1, and then the third component is 1. Equating row 2, the second component is 2. 011 ) v1 = t @2A : 1 Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 3: We compute 03 −11 010 0 21 3 1 1 A∗A = 1 3 = 0 10 4 @ A −1 3 1 @ A 1 1 2 4 2 For λ = 10: 00 0 2 1 ∗ A A − 10I = @0 0 4 A : 2 4 −8 Rows 1 and 2 force the third component to be 0. Looking at row 3, it is easier to set the second component to 1, and then the first component will be −2. 0−21 ) v2 = t @ 1 A : 0 Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 3: We compute 03 −11 010 0 21 3 1 1 A∗A = 1 3 = 0 10 4 @ A −1 3 1 @ A 1 1 2 4 2 For λ = 0, looking at A∗A itself, there really are many possibilities we can go about it. But I follow the answers, which arbitrarily set the first component to −1. See if you can then show that 0−11 v3 = t @−2A : 5 Note: In each case, t 2 R. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 4: In each case, choose the value of t that normalises the eigenvectors: 011 1 v1 = p @2A (λ1 = 12) 6 1 0−21 1 v2 = p @ 1 A (λ2 = 10) 5 0 0−11 1 v3 = p @−2A (λ3 = 0) 30 5 Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example 1 Part 4: Compute ui = p Avi for each non-zero eigenvector: λi 011 1 3 1 1 1 1 1 u = p p 2 = p 1 −1 3 1 @ A 1 12 6 1 2 0−21 1 3 1 1 1 1 −1 u = p p 1 = p 2 −1 3 1 @ A 1 10 5 0 2 Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) SVD Example Part 4: We conclude that a SVD for A is A = UΣV ∗, where 1 1 −1 U = p 2 1 1 p 12 0 0 Σ = p 0 10 0 0 p1 − p2 − p1 1 6 5 30 V = B p2 p1 − p2 C @ 6 5 30 A p1 0 p5 6 30 Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) Remark ∗ For A = UΣV , where A 2 Mn×n(C): The columns of U = u1 ::: um are called the left singular vectors. The columns of V = v1 ::: vn are called the right singular vectors. Word of advice: Write some of these numbers very quickly! SVDs are instructive when you know the method, but it always takes forever to do. Rui Tong Second Year Mathematics Revision Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) Reduced SVD I don't see these examined, but I should still mention them. 1 Obtain Σ^ by removing any zero columns in Σ 2 Obtain V^ by removing the corresponding columns in V . 3 Then, A = UΣ^V^∗. For the earlier example: p 12 0 Σ^ = p 0 10 0 p1 − p2 1 6 5 V^ = B p2 p1 C @ 6 5 A p1 0 6 Rui Tong Second Year Mathematics Revision The Jordan Canonical Form Today's plan 1 Eigenvalues and Eigenvectors Singular Value Decomposition (MATH2601) 2 The Jordan Canonical Form Finding Jordan Forms The Cayley-Hamilton Theorem 3 Matrix Exponentials Computing Matrix Exponentials Application to Systems of Differential Equations Rui Tong Second Year Mathematics Revision The Jordan Canonical Form Finding Jordan Forms Jordan Blocks Definition 3: Jordan blocks The k × k Jordan block for λ is the matrix 0λ 1 0 ::: 01 B0 λ 1 ::: 0C B C B0 0 λ : : : 0C J (λ) = B C 2 M ( ): k B . .. C k×k C B . C B C @0 0 0 ::: 1A 0 0 0 : : : λ That is, put λ on every entry along the main diagonal, and a 1 immediately above each λ wherever possible. Rui Tong Second Year Mathematics Revision The Jordan Canonical Form Finding Jordan Forms Jordan Blocks Quick examples: 0−4 1 0 1 J3(−4) = @ 0 −4 1 A 0 0 −4 00 1 0 01 B0 0 1 0C J4(0) = B C @0 0 0 1A 0 0 0 0 Rui Tong Second Year Mathematics Revision The Jordan Canonical Form Finding Jordan Forms Powers of Jordan Forms Find the pattern. n n J1(λ) = λ λn nλn−1 J (λ)n = 1 2 0 λn 0 n n n−1 n n−21 λ 1 λ 2 λ n n n n−1 J3(λ) = @ 0 λ 1 λ A 0 0 λn 0 n n n−1 n n−2 n n−31 λ 1 λ 2 λ 3 λ n n n−1 n n−2 n B 0 λ 1 λ 2 λ C J4(λ) = B n n n−1C @ 0 0 λ 1 λ A 0 0 0 λn Rui Tong Second Year Mathematics Revision The Jordan Canonical Form Finding Jordan Forms Powers of Jordan Forms Lemma 3: Computing powers of Jordan forms 1 Start with λn on every diagonal entry. 2 n n−1 n Put 1 λ wherever you can immediately above λ 3 n n−2 n n−1 Put 2 λ wherever you can immediately above 1 λ 4 Keep doing this, increasing the binomial coefficient and decreasing the power on λ. n Note: Not quite the above. If you ever bump into n , that's the last diagonal you fill. Just put 0's everywhere else above.