Eigenvalue Problems Chia-Ping Chen Department of Computer Science and Engineering National Sun Yat-sen University Linear Algebra 1/97 Outline Eigenvalues and eigenvectors Eigenvalue decomposition and diagonalization Difference equation Differential equation Complex matrices Similar matrices Change of basis 2/97 Chen P Eigenvalue Problems notation Ax = λx: the eigenvalue equation of A |A − λI| = 0: the characteristic equation of A λ0: an eigenvalue of a matrix Eλ0 : the eigenspace corresponding to eigenvalue λ0 {λ1, . , λk}: spectrum (the set of eigenvalues) of A si: an eigenvector corresponding to λi c∗: the complex conjugate of c AH : the Hermitian of A A ∼ B: similar matrices 3/97 Chen P Eigenvalue Problems Eigenvalue Equations 4/97 Chen P Eigenvalue Problems Definition (eigenvalue problem) Let A be a square matrix. The eigenvalue equation or eigenvalue problem of A is Ax = λx Solutions of λ and x are the eigenvalues and eigenvectors of A 5/97 Chen P Eigenvalue Problems Definition (characteristic equation/polynomial) Let A be a square matrix. The characteristic equation of A is |A − λI| = 0 The characteristic polynomial of A is f(λ) = |A − λI| 6/97 Chen P Eigenvalue Problems Lemma (eigenvalue and characteristic equation) λ0 is an eigenvalue of A if and only if |A − λ0I| = 0. |A − λ0I| = 0 ⇔ (A − λ0I) is singular ⇔ ∃ x 6= 0, (A − λ0I) x = 0 ⇔ ∃ x 6= 0, Ax = λ0x ⇔ λ0 is an eigenvalue of A 7/97 Chen P Eigenvalue Problems Definition (eigenspace) Let λ0 be an eigenvalue of square matrix A. The eigenspace of A corresponding to λ0 is the set of vectors Eλ0 = {x | Ax = λ0x} An eigenspace is a nullspace. Eλ0 = {x | Ax = λ0x} = {x | (A − λ0I) x = 0} = N (A − λ0I) Let A be a square matrix of order n and λ0 be an eigenvalue of A. Then the dimension of Eλ0 is dim(Eλ0 ) = dim(N (A − λ0I)) = n − rank(A − λ0I) 8/97 Chen P Eigenvalue Problems Definition (spectrum) The spectrum of A is the set of the eigenvalues of A, i.e. the set of solutions to |A − λI| = 0. Let A be a square matrix of order n. Then |A − λI| is a polynomial of order n, and |A − λI| = 0 is an equation of order n. By the fundamental theorem of algebra, we have spectrum(A) = {λ1, . , λk} where k ≤ n 9/97 Chen P Eigenvalue Problems Lemma (eigenvectors of distinct eigenvalues) Let A be a square matrix. Let s1 (resp. s2) be an eigenvector of A with eigenvalue λ1 (resp. λ2), where λ1 6= λ2. Then s1 and s2 are linearly independent. Suppose c1s1 + c2s2 = 0. Then λ1(c1s1 + c2s2) = 0 and A(c1s1 + c2s2) = c1λ1s1 + c2λ2s2 = 0 By subtraction, we have c2(λ2 − λ1)s2 = 0. It follows that c2 = 0 and c1 = 0. Hence s1 and s2 are linearly independent. 10/97 Chen P Eigenvalue Problems Example (eigenvalue problem) "4 −5# A = 2 −3 2 |A − λI| = 0 ⇒ λ − λ − 2 = 0 ⇒ λ1 = 2, λ2 = −1 For λ1 = 2, the eigenspace is " # ( " #) 2 −5 5 2 s1 = 0 ⇒ Eλ1 = s1 s1 = c 2 −5 1 For λ2 = −1, the eigenspace is " # ( " #) 5 −5 1 s2 = 0 ⇒ Eλ2 = s2 s2 = c 2 −2 1 11/97 Chen P Eigenvalue Problems Example (eigenvalue problem: projection matrix) " 1 1 # 2 2 P = 1 1 2 2 2 |P − λI| = 0 ⇒ λ − λ = 0 ⇒ λ1 = 1, λ2 = 0 " # ( " #) − 1 1 1 2 2 (P −λ1I)s1 = 1 1 s1 = 0 ⇒ Eλ1 = s1 s1 = c 2 − 2 1 " # ( " #) 1 1 −1 2 2 (P − λ2I)s2 = 1 1 s2 = 0 ⇒ Eλ2 = s2 s2 = c 2 2 1 12/97 Chen P Eigenvalue Problems Example (complex eigenvalues) "0 −1# K = 1 0 2 |K − λI| = 0 ⇒ λ + 1 = 0 ⇒ λ1 = i, λ2 = −i " # ( " #) −i −1 i (K−λ1I)s1 = s1 = 0 ⇒ Eλ1 = s1 s1 = c 1 −i 1 " # ( " #) i −1 −i (K−λ2I)s2 = s2 = 0 ⇒ Eλ2 = s2 s2 = c 1 i 1 13/97 Chen P Eigenvalue Problems Lemma (bound on the number of eigenvalues) Let A be a square matrix of order n. A has at most n eigen- values. Algebraic argument. The characteristic polynomial of A a − λ . a 11 1n . |A − λI| = . .. an1 . ann − λ has at most n roots (eigenvalues). Geometric argument. A space of dimension n can ac- commodate at most n linearly independent (eigen-)vectors. 14/97 Chen P Eigenvalue Problems Theorem (sum of eigenvalues = trace) Let A be a square matrix with distinct eigenvalues. The trace of a matrix is the sum of diagonal elements The sum of eigenvalues of A is equal to trace of A Factorize the polynomial |A − λI| by its roots a − λ . a 11 1n . .. = c(λ − λ1) ... (λ − λn) . an1 . ann − λ For the λn and λn−1 terms, the equality requires c = (−1)n and n−1 n−1 n n−1 (−1) (a11+···+ann)λ = (−1) ((−λ1)+···+(−λn))λ λ1 + ··· + λn = a11 + ··· + ann For repeated eigenvalues, the equality holds. 15/97 Chen P Eigenvalue Problems Theorem (product of eigenvalues = determinant) Let A be a square matrix with distinct eigenvalues. The product of the eigenvalues of A is equal to the determinant of A. The characteristic polynomial of A can be factorized by its roots n n Y |A − λI| = (−1) (λ − λi) i=1 Setting λ of both sides to 0, we get n n n Y Y |A| = (−1) (−λi) = λi i=1 i=1 For repeated eigenvalues, the equality holds. 16/97 Chen P Eigenvalue Problems tips for finding eigenvalues Let A be a square matrix. The eigenvalues of A may be found without solving the characteristic equation of A. If A is singular, 0 is an eigenvalue If A has a constant row sum (or column sum), that con- stant is an eigenvalue If A is a triangular matrix, the diagonal elements of A are eigenvalues 17/97 Chen P Eigenvalue Problems Eigenvalue Decomposition and Diagonalization 18/97 Chen P Eigenvalue Problems Definition (algebraic/geometric multiplicity) Let A be a square matrix with spectrum {λ1, . , λk}. The characteristic polynomial of A can be expressed as k Y γi |A − λI| = c (λ − λi) i=1 γi is called the algebraic multiplicity of λi λi corresponds to eigenspace Eλi . The dimension of Eλi is called the geometric multiplicity of λi, denoted by gi Suppose A is of order n × n. We have k k X X γi = n, gi ≤ n i=1 i=1 19/97 Chen P Eigenvalue Problems Definition (defective matrix) Let A be a square matrix of order n with spectrum {λ1, . , λk}. A is defective if g1 + ··· + gk < n "0 1# "0 0# Consider A = and B = . A is defective, since 0 0 0 0 dim N (A−λ1I) X z }| { n = 2, λ1 = 0, gi = g1 = n − rank(A − λ1I) = 2−1 = 1 < n i B is non-defective, since X n = 2, λ1 = 0, gi = g1 = n−rank(B−λ1I) = 2−0 = 2 = n i 20/97 Chen P Eigenvalue Problems Definition (eigenbasis) A basis in which every basis vector is an eigenvector of the same matrix is an eigenbasis. Existence. An eigenbasis consisting of eigenvectors of A exists if and only if A is non-defective. Construction. Let {λ1, . , λk} be the spectrum of A. Let B1,..., Bk be bases of eigenspaces Eλi ,..., Eλk and B = B1∪· · ·∪Bk. Then B is an eigenbasis since it is linearly Pk independent and contains i=1 gi = n eigenvectors of A. 21/97 Chen P Eigenvalue Problems Definition (eigenvector and eigenvalue matrix) Let A be a non-defective matrix of order n. An eigenvector matrix is S = s1 ... sn where {s1,..., sn} is an eigenbasis An eigenvalue matrix is Λ = diag(λ1, . , λn) where λi is the eigenvalue corresponding to si 22/97 Chen P Eigenvalue Problems Theorem (eigenvalue decomposition) Let A be a non-defective matrix. A can be decomposed by A = SΛS−1 A s1 ... sn = As1 ... Asn = λ1s1 . λnsn λ1 . = s1 ... sn .. λn It follows from AS = SΛ that A = SΛS−1. 23/97 Chen P Eigenvalue Problems Corollary (diagonalization of a matrix) A non-defective square matrix can be diagonalized by its eigen- vector matrix. It follows from eigenvalue decomposition A = SΛS−1 that −1 S AS = Λ = diag(λ1, . , λn) This is the diagonalization of A. 24/97 Chen P Eigenvalue Problems Example (diagonalization of matrix) " 1 1 # " #−1 " 1 1 #" # " # 2 2 1 −1 2 2 1 −1 1 0 P = 1 1 , 1 1 = 2 2 1 1 2 2 1 1 0 0 −1 "0 −1# "i −i# "0 −1#"i −i# "i 0 # K = , = 1 0 1 1 1 0 1 1 0 −i 25/97 Chen P Eigenvalue Problems Difference Equations 26/97 Chen P Eigenvalue Problems Definition (Fibonacci recurrence and numbers) The Fibonacci recurrence is Fk+1 = Fk + Fk−1 The initial Fibonacci numbers are F0 = 0,F1 = 1 The first few Fibonacci numbers are 0 1 1 2 3 5 8 13 ... 27/97 Chen P Eigenvalue Problems Definition (Fibonacci vectors and matrix) The Fibonacci vectors are " # Fk+1 uk = Fk The Fibonacci matrix is "1 1# A = 1 0 28/97 Chen P Eigenvalue Problems k-step recurrence The Fibonacci recurrence can be written by "F # "1 1#" F # k+1 = k Fk 1 0 Fk−1 That is uk = Auk−1 Repeating the Fibonacci recurrence, we get k-step recur- rence k−1 k uk = Auk−1 = A(Auk−2) = ··· = A u1 = A u0 That is k "F # "1 1# "1# k+1 = Fk 1 0 0 29/97 Chen P Eigenvalue Problems power of a non-defective matrix Let A be a non-defective matrix and A = SΛS−1 be an eigen- value decomposition of A.