CHAPTER 7 Eigenvalues and Eigenvectors 7.1 ELEMENTARY PROPERTIES OF EIGENSYSTEMS Up to this point, almost everything was either motivated by or evolved from the consideration of systems of linear algebraic equations. But we have come to a turning point, and from now on the emphasis will be different. Rather than being concerned with systems of algebraic equations, many topics will be motivated or driven by applications involving systems of linear differential equations and their discrete counterparts, difference equations. For example, consider the problem of solving the system of two first-order linear differential equations, du1/dt =7u1 4u2 and du2/dt =5u1 2u2. In matrix notation, this system is − − u1′ 7 4 u1 = − or, equivalently, u′ = Au, (7.1.1) u′ 5 2 u 2 − 2 ′ u1 7 −4 u1 where u′ = ′ , A = , and u = . Because solutions of a single u2 5 −2 u2 λt equation u′ = λu have the form u = αe , we are motivated to seek solutions of (7.1.1) that also have the form λt λt u1 = α1e and u2 = α2e . (7.1.2) Differentiating these two expressions and substituting the results in (7.1.1) yields λt λt λt α1λe =7α1e 4α2e α1λ =7α1 4α2 7 4 α1 α1 − − − =λ . α λeλt =5α eλt 2α eλt ⇒ α λ =5α 2α ⇒ 5 2 α α 2 1 − 2 2 1 − 2 − 2 2 490 Chapter 7 Eigenvalues and Eigenvectors In other words, solutions of (7.1.1) having the form (7.1.2) can be constructed provided solutions for λ and x = α1 in the matrix equation Ax = λx can α2 be found. Clearly, x = 0 trivially satisfies Ax = λx, but x = 0 provides no useful information concerning the solution of (7.1.1). What we really need are scalars λ and nonzero vectors x that satisfy Ax = λx. Writing Ax = λx as (A λI) x = 0 shows that the vectors of interest are the nonzero vectors in N (A −λI) . But N (A λI) contains nonzero vectors if and only if A λI is singular.− Therefore, the− scalars of interest are precisely the values of λ −that make A λI singular or, equivalently, the λ ’s for which det (A λI)=0. − − 66 These observations motivate the definition of eigenvalues and eigenvectors. Eigenvalues and Eigenvectors For an n n matrix A, scalars λ and vectors xn 1 = 0 satisfying Ax = λx ×are called eigenvalues and eigenvectors of× A , respectively, and any such pair, (λ, x), is called an eigenpair for A. The set of distinct eigenvalues, denoted by σ (A) , is called the spectrum of A. λ σ (A) A λI is singular det (A λI)=0. (7.1.3) • ∈ ⇐⇒ − ⇐⇒ − x = 0 x N (A λI) is the set of all eigenvectors associated • with λ. From∈ now on,− N (A λI) is called an eigenspace for A. − Nonzero row vectors y∗ such that y∗(A λI)=0 are called left- • hand eigenvectors for A (see Exercise− 7.1.18 on p. 503). Geometrically, Ax = λx says that under transformation by A, eigenvec- tors experience only changes in magnitude or sign—the orientation of Ax in n is the same as that of x. The eigenvalue λ is simply the amount of “stretch”ℜ or “shrink” to which the eigenvector x is subjected when transformed by A. Figure 7.1.1 depicts the situation in 2. ℜ Ax = λx x Figure 7.1.1 66 The words eigenvalue and eigenvector are derived from the German word eigen, which means owned by or peculiar to. Eigenvalues and eigenvectors are sometimes called characteristic values and characteristic vectors, proper values and proper vectors, or latent values and latent vectors. 7.1 Elementary Properties of Eigensystems 491 Let’s now face the problem of finding the eigenvalues and eigenvectors of 7 −4 the matrix A = 5 −2 appearing in (7.1.1). As noted in (7.1.3), the eigen- values are the scalars λ for which det (A λI)=0. Expansion of det (A λI) produces the second-degree polynomial − − 7 λ 4 p(λ) = det (A λI)= − − = λ2 5λ +6=(λ 2)(λ 3), − 5 2 λ − − − − − which is called the characteristic polynomial for A. Consequently, the eigen- values for A are the solutions of the characteristic equation p(λ) = 0 (i.e., the roots of the characteristic polynomial), and they are λ = 2 and λ =3. The eigenvectors associated with λ = 2 and λ = 3 are simply the nonzero vectors in the eigenspaces N (A 2I) and N (A 3I), respectively. But deter- mining these eigenspaces amounts− to nothing more− than solving the two homo- geneous systems, (A 2I) x = 0 and (A 3I) x = 0. For λ =2, − − 5 4 1 4/5 x = (4/ 5)x A 2I = − − = 1 2 − 5 4 −→ 00 ⇒ x2 is free − 4/5 = N (A 2I)= x x = α . ⇒ − 1 For λ =3, 4 4 1 1 x = x A 3I = − − = 1 2 − 5 5 −→ 00 ⇒ x2 is free − 1 = N (A 3I)= x x = β . ⇒ − 1 In other words, the eigenvectors of A associated with λ = 2 are all nonzero multiples of x =(4/5 1)T , and the eigenvectors associated with λ = 3 are all nonzero multiples of y =(1 1)T . Although there are an infinite number of eigenvectors associated with each eigenvalue, each eigenspace is one dimensional, so, for this example, there is only one independent eigenvector associated with each eigenvalue. Let’s complete the discussion concerning the system of differential equations u′ = Au in (7.1.1). Coupling (7.1.2) with the eigenpairs (λ1, x) and (λ2, y) of A computed above produces two solutions of u′ = Au, namely, 4/5 1 u =eλ1tx =e2t and u =eλ2ty =e3t . 1 1 2 1 It turns out that all other solutions are linear combinations of these two particular solutions—more is said in 7.4 on p. 541. Below is a summary of§ some general statements concerning features of the characteristic polynomial and the characteristic equation. 492 Chapter 7 Eigenvalues and Eigenvectors Characteristic Polynomial and Equation The characteristic polynomial of An n is p(λ) = det (A λI). • The degree of p(λ) is n, and the leading× term in p(λ) is ( 1)−nλn. − The characteristic equation for A is p(λ)=0. • The eigenvalues of A are the solutions of the characteristic equation • or, equivalently, the roots of the characteristic polynomial. Altogether, A has n eigenvalues, but some may be complex num- • bers (even if the entries of A are real numbers), and some eigenval- ues may be repeated. If A contains only real numbers, then its complex eigenvalues must • occur in conjugate pairs—i.e., if λ σ (A) , then λ σ (A) . ∈ ∈ Proof. The fact that det (A λI) is a polynomial of degree n whose leading term is ( 1)nλn follows from− the definition of determinant given in (6.1.1). If − 1 ifi = j, δ = ij 0 ifi = j, then det (A λI)= σ(p)(a δ λ)(a δ λ) (a δ λ) − 1p1 − 1p1 2p2 − 2p2 ··· npn − npn p is a polynomial in λ. The highest power of λ is produced by the term (a λ)(a λ) (a λ), 11 − 22 − ··· nn − so the degree is n, and the leading term is ( 1)nλn. The discussion given earlier contained the proof that the eigenvalues are− precisely the solutions of the characteristic equation, but, for the sake of completeness, it’s repeated below: λ σ (A) Ax = λx for some x = 0 (A λI) x = 0 for some x = 0 ∈ ⇐⇒ ⇐⇒ − A λI is singular det (A λI)=0. ⇐⇒ − ⇐⇒ − The fundamental theorem of algebra is a deep result that insures every poly- nomial of degree n with real or complex coefficients has n roots, but some roots may be complex numbers (even if all the coefficients are real), and some roots may be repeated. Consequently, A has n eigenvalues, but some may be complex, and some may be repeated. The fact that complex eigenvalues of real matrices must occur in conjugate pairs is a consequence of the fact that the roots of a polynomial with real coefficients occur in conjugate pairs. 7.1 Elementary Properties of Eigensystems 493 Example 7.1.1 1 −1 Problem: Determine the eigenvalues and eigenvectors of A = 11. Solution: The characteristic polynomial is 1 λ 1 det (A λI)= − − = (1 λ)2 +1=λ2 2λ +2, − 11λ − − − 2 so the characteristic equation is λ 2λ +2=0. Application of the quadratic formula yields − 2 √ 4 2 2√ 1 λ = ± − = ± − =1 i, 2 2 ± so the spectrum of A is σ (A)= 1+i, 1 i . Notice that the eigenvalues are complex conjugates of each other—as{ they must− } be because complex eigenvalues of real matrices must occur in conjugate pairs. Now find the eigenspaces. For λ =1+i, i 1 1 i i A λI = − − − = N (A λI)=span . − 1 i −→ 00 ⇒ − 1 − For λ =1 i, − i 1 1i i A λI = − = N (A λI)=span − . − 1i−→ 00 ⇒ − 1 In other words, the eigenvectors associated with λ1 = 1 + i are all nonzero T multiples of x1 =(i 1) , and the eigenvectors associated with λ2 =1 i T − are all nonzero multiples of x2 =( i 1). In previous sections, you could be successful by thinking only in terms− of real numbers and by dancing around those statements and issues involving complex numbers. But this example makes it clear that avoiding complex numbers, even when dealing with real matrices, is no longer possible—very innocent looking matrices, such as the one in this example, can possess complex eigenvalues and eigenvectors.