Section 5.1 Eigenvalues and Eigenvectors

At this point in the class, we understand that the operation of matrix multiplication is very different from scalar multiplication. Matrix multiplication is an operation that finds the product of a pair of matrices, whereas scalar multiplication finds the product of a scalar with a matrix.

More to the point, scalar multiplication of a scalar k with a matrix X is an easy operation to perform; indeed there’s a simple algorithm: multiply each entry of X by the scalar k. Matrix multiplication on the other hand cannot be expressed using a simple algorithm. Indeed, the operation is extremely complex: to multiply a pair of n×n matrices, it takes about n2 operations, a hugely inefficient process. Thus it is surprising that, for any square matrix A, certain matrix multiplications “collapse” (in a sense) into scalar multiplication. As an example, consider the matrices ( ) ( ) ( ) 3 2 −4 10 A = , w = , x = . 3 4 −1 15

The product Aw is not particularly interesting: ( ) −14 Aw = ; −16 indeed, there appears to be very little connection between A, w, and Aw. On the other hand, an we see an interesting phenomenon when we calculate Ax: ( ) ( ) 60 10 Ax = = 6 = 6x. 90 15

Forgetting the calculation in the middle, we have

Ax = 6x; in a sense, the matrix multiplication Ax collapsed into scalar multiplication 6x.

We will see below that the phenomenon described here is more common than one might think. Identifying products like Ax = 6x will allow us to extract a great deal of useful information about A; accordingly, the rest of the section will be devoted to this phenomenon, which we give a name in the next definition.

Definition. Let A be an n × n matrix and x any nonzero vector in Rn that satisfies the equation

Ax = λx, or alternatively TA(x) = λx for some scalar λ. We call λ an eigenvalue of A (and of the associated linear function TA), and call x an eigenvector corresponding to λ.

1 Section 5.1 In the example above, we saw that ( )( ) ( ) 3 2 10 10 = 6 . 3 4 15 15

In the language of our new definition, λ = 6 is an eigenvalue of ( ) 3 2 A = , 3 4 and ( ) 10 x = 15 is an eigenvector associated to λ = 6.

Geometrically, A “stretches” the vector x by a factor of 6, without changing x’s direction. Recall, however, that A did change the direction of the vector w; there is something unique about x with relation to multiplication by A.

As mentioned above, the eigenvalues and eigenvectors of a matrix can provide us with a great deal of information about the matrix. This brings us to several questions about the quantities:

1. Are we guaranteed that an n × n matrix A has eigenvalues?

2. What is the maximum number of distinct eigenvalues of an n × n matrix?

3. How do we find the eigenvalues of A?

The next theorem answers all of these questions:

Theorem 1.5.1. If A is an n × n matrix with real entries, then the number λ (which could be real or complex) is an eigenvalue of A if and only if it satisfies the equation

det(λI − A) = 0, which is called the characteristic equation of A.

The polynomial p(λ) = det(λI − A) is called the characteristic polynomial of A.

Before we answer the questions posed above, let’s think about what this theorem says about the matrix ( ) 3 2 A = : 3 4 Theorem 5.1.1 says that we can find the eigenvalues for A by finding the determinant of the matrix λI − A.

2 Section 5.1 Let’s find the matrix: ( ) ( ) 1 0 3 2 λI − A = λ − 0 1 3 4 ( ) ( ) λ 0 3 2 = − 0 λ 3 4 ( ) λ − 3 −2 = . −3 λ − 4

Now we need to find the determinant of the matrix above: ( ) λ − 3 −2 det(λI − A) = det −3 λ − 4

= (λ − 3)(λ − 4) − 6 = λ2 − 7λ + 6.

Before we proceed, we should note that

λ2 − 7λ + 6 = 0 is the characteristic equation of A, and

p(λ) = λ2 − 7λ + 6 is its characteristic polynomial; notice that it is a degree 2 polynomial.

Now according to the theorem, the numbers λ satisfying the equation above are the eigenvalues of A. Thus finding the roots of the polynomial will give us the desired eigenvalues: the polynomial factors as (λ − 6)(λ − 1) = 0, which means that the roots are λ = 6 and λ = 1. Thus the eigenvalues of the matrix A are λ = 6 and λ = 1. We have already seen that λ = 6 is an eigenvalue of A, since it satisfies the equation ( )( ) ( ) 3 2 10 10 = 6 . 3 4 15 15

We should verify( that λ) = 1 is also an eigenvalue by finding an eigenvector. In other words, we ⊤ 2 need a vector x = x1 x2 in R so that

Ax = 1x

3 Section 5.1 or ( )( ) ( ) 3 2 x x 1 = 1 . 3 4 x2 x2 This should not be too difficult to do; let’s make the calculation: ( )( ) 3 2 x Ax = 1 3 4 x2 ( ) 3x + 2x = 1 2 . 3x2 + 4x2 Now we want Ax = x, so we must have ( ) ( ) 3x + 2x x 1 2 = 1 , 3x1 + 4x2 x2 or

2x1 + 2x2 = 0

3x1 + 3x2 = 0. We can rewrite this system in matrix equation form as ( ) 2 2 x = 0. 3 3 Now it is easy to see that the coefficient matrix of this equation has determinant 0, so by Theorem 4.8.8, the system has at least one nontrivial solution; choosing, say, x1 = 1 gives us corresponding x2 = −1. So ( ) 1 x = −1 is an eigenvector corresponding to the eigenvalue λ = 1.

Key Point. Here are few facts about the characteristic equation of A, along with the answers to the questions posed above: • If A is an n × n matrix, then its characteristic polynomial

p(λ) = det(λI − A)

is a polynomial of degree n.

• The roots of the characteristic equation

det(λI − A) = 0

are precisely the eigenvalues of A. Finding the roots of the equation amounts to finding the eigenvalues of A.

4 Section 5.1 • By the Fundamental Theorem of Algebra, a polynomial of degree n has exactly n real or complex roots, which may or may not be repeated. Thus an n × n matrix A has as many as n distinct eigenvalues.

• The number 0 is allowed to be an eigenvalue for a matrix, but the vector 0 is not allowed to be an eigenvector, since A0 = k0 is true for any matrix A and any scalar k.

• If x is an eigenvector of A with associated eigenvalue λ, so that

Ax = λx,

then any nonzero scalar multiple kx of eigenvector x is also an eigenvector. To understand why, think about the product A(kx):

A(kx) = k(Ax) = k(λx) = λ(kx).

• If x and y are eigenvalues associated with the same eigenvector λ, so that

Ax = λx and Ay = λy,

then the vector x + y is also an eigenvector of A associated with λ, since

A(x + y) = Ax + Ay = λx + λy = λ(x + y).

Example Given   5 −2 0 A = 0 −3 0  , 1 4 −1 find:

1. The characteristic equation of A.

2. The eigenvalues of A.

3. An eigenvector corresponding to each eigenvalue of A.

1. The characteristic equation of A is given by

det(λI − A) = 0,

5 Section 5.1 so let’s begin by finding the matrix λI − A:     λ 0 0 5 −2 0 λI − A = 0 λ 0 − 0 −3 0  0 0 λ 1 4 −1   λ − 5 2 0 =  0 λ + 3 0  . −1 −4 λ + 1

Next, we need to calculate the determinant det(λI − A); we proceed by cofactor expansion along the last column:   λ − 5 2 0 det(λI − A) = det  0 λ + 3 0  −1 −4 λ + 1 ( ) = (λ + 1) (λ − 5)(λ + 3) − 2 · 0 = (λ + 1)(λ − 5)(λ + 3) = (λ + 1)(λ2 − 2λ − 15) = λ3 − λ2 − 17λ − 15.

Thus the characteristic equation of A is

λ3 − λ2 − 17λ − 15 = 0.

2. The roots of the characteristic equation above are the desired eigenvalues; fortunately we already know from the calculations above that the polynomial factors as

λ3 − λ2 − 17λ − 15 = (λ + 1)(λ − 5)(λ + 3).

So the roots of the equation (and the eigenvalues of A) are

λ1 = −1, λ2 = −3, and λ3 = 5.

3. The desired eigenvectors x1, x2, and x3 must satisfy the equations

Ax1 = −x1,Ax2 = −3x2, and Ax3 = 5x3,

6 Section 5.1 respectively. For a general vector x in R3, Ax is given by     5 −2 0 x1     Ax = 0 −3 0 x2 1 4 −1 x3   5x1 − 2x2   = −3x2 . x1 + 4x2 − x3

Starting with λ1 = −1, we want     5x1 − 2x2 −x1     −3x2 = −x2 , x1 + 4x2 − x3 −x3

so we see that

6x1 − 2x2 = 0

−2x2 = 0

x1 + 4x2 = 0.

Now we must choose x1 = x2 = 0, but x3 does not show up in the system, so we are free to choose a value for it as we wish, say x3 = 1. Thus   0   x1 = 0 1

is an eigenvector for A associated with the eigenvalue λ1 = −1.

Next, let’s look for an eigenvector associated with the eigenvalue λ2 = −3: we want     5x1 − 2x2 −3x1     −3x2 = −3x2 , x1 + 4x2 − x3 −3x3

so we have

8x1 − 2x2 = 0 0 = 0

x1 + 4x2 + 2x3 = 0.

Since we have two equations in three unknowns, we are free to choose one of the variables as we wish; we can easily satisfy the first equation by setting x2 = 4, so that x1 = 1. The third

7 Section 5.1 equation then dictates that x3 = −17/2. So   1   x2 = 4 − 17 2

is an eigenvector of A associated with the eigenvalue λ2 = −3. However, if we object to the fraction in the eigenvector above, we can multiply it by a suitable (nonzero) scalar and maintain its status as an eigenvector, as indicated above: so   2 ′   x2 = 8 −17

works just as well as an eigenvector associated with λ2 = −3.

Finally, let’s find an eigenvector associated with the eigenvalue λ3 = 5:     5x1 − 2x2 5x1     −3x2 = 5x2 , x1 + 4x2 − x3 5x3 which we rewrite as

−2x2 = 0

−8x2 = 0

x1 + 4x2 − 6x3 = 0.

Clearly we must choose x2 = 0; x3 is free, so we can choose x3 = 1 so that x1 = 6. Thus   6   x3 = 0 1

is an eigenvector of A associated with the eigenvalue λ3 = 5.

Key Point. We have mentioned many times that linear functions on vector spaces may be rep- resented as matrices. With this idea in mind, we can rephrase the result of the previous example: 3 3 the linear function TA : R → R defined by   5 −2 0   TA(x) = 0 −3 0 x 1 4 −1 has eigenvalues λ1 = −1, λ2 = −3, and λ3 = 5, with corresponding eigenvectors       0 2 6       x1 = 0 , x2 = 8 , and x3 = 0 . 1 −17 1

8 Section 5.1 Eigenvalues of Special Matrices Finding eigenvalues can be a tedious process, as it involves the determinant calculation; however, we have seen several special types of matrices whose determinants are quite easy to calculate. In fact, in Section 2.1, we saw that Theorem. The determinant of a diagonal, upper triangular, or lower triangular matrix is just the product of its diagonal entries. Let’s use this idea to think about finding the eigenvalues of diagonal, upper triangular, and lower triangular matrices: if A is diagonal or triangular, then so is λI − A, which means that det(λI − A) is just the product of the diagonal entries of (λI − A). So finding eigenvalues of one of these special kinds of matrices shouldn’t be too difficult. Let’s examine a specific example in order to get a conjecture for the values of the eigenvalues of a diagonal or triangular matrix.

Example

Find the eigenvalues of   −3 0 0 0 0    12 5 0 0 0   A =  .2 3 −1 0 0 . −10 3 1 −11 0 .5 −2 6 6 0

As mentioned above, the matrix λI − A is lower triangular:   λ + 3 0 0 0 0    −12 λ − 5 0 0 0    λI − A =  −.2 −3 λ + 1 0 0  .  10 −3 −1 λ + 11 0  −.5 2 −6 −6 λ − 0

Next, we calculate det(λI − A):

det(λI − A) = (λ + 3)(λ − 5)(λ + 1)(λ + 11)(λ − 0), so that the characteristic equation of A is

(λ + 3)(λ − 5)(λ + 1)(λ + 11)(λ − 0) = 0.

Now the roots of this equation are the eigenvalues of A; since the polynomial on the left-hand side of the equation is already factored, we see that the eigenvalues of A are

λ1 = −3, λ2 = 5, λ3 = −1, λ4 = −11, and λ5 = 0; these are precisely the diagonal entries of A, and it is not too hard to see that this phenomenon will occur for any diagonal or triangular matrix.

9 Section 5.1 Theorem 5.1.2. If A is a diagonal, upper triangular, or lower triangular matrix, then its eigen- values are its diagonal entries.

Eigenspaces Since eigenvectors satisfy the equation Ax = λx, they are solutions to the system (λI − A)x = 0; in other words, every eigenvector of A associated with eigenvalue λ is in the nullspace of the matrix λI − A, thus the set of all such x form a vector space. Definition 1. The vector space of solutions x to (λI − A)x = 0 is called the eigenspace of A corresponding to λ. In particular, an eigenspace consists of all of the vectors in Rn that are eigenvectors of A corresponding to λ. We can also think about eigenspaces in terms of linear functions; to do so, we must define a new term: Theorem. The kernel of a linear function f : Rm → Rn is the set of all vectors x in Rm so that

f(x) = 0.

n n With this definition in place, we point out that the linear function TλI−A : R → R , defined by TλI−A(x) = (λI − A)x, has kernel consisting of all of the eigenvectors of TλI−A; that is, the kernel of the linear function TλI−A is precisely the eigenspace of A corresponding to λ.

Example

Find bases for the eigenspaces of   5 −2 0 A = 0 −3 0  1 4 −1 corresponding to each of the eigenvalues

λ1 = −1, λ2 = −3, and λ3 = 5.

In our earlier investigation of this example, we found that, if x is an eigenvector corresponding to λ1 = −1, then     5x1 − 2x2 −x1     −3x2 = −x2 ; x1 + 4x2 − x3 −x3

10 Section 5.1 alternatively

6x1 − 2x2 = 0

−2x2 = 0

x1 + 4x2 = 0.

Of course, the only way to make these equations true is by choosing x1 = x2 = 0; as x3 does not show up in the system, we were free to choose the value x3 = 1. So   0   x1 = 0 1 and its multiples are the only eigenvectors corresponding to λ1. Thus the eigenspace of A cor- responding to λ1 is a one-dimensional subspace consisting of all multiples of x1, and {x1} is a perfectly acceptable basis for the eigenspace.

Using similar logic, we can see that the eigenspaces corresponding to λ2 and λ3 are both one- dimensional, with bases     { 2 } { 6 }  8  , 0 −17 1 respectively.

Example Find a basis for the eigenspace corresponding to the eigenvalue λ = 2 of   2 0 0 0 0 −1 0 0 A =   . 0 4 2 0 4 0 0 3

Since A is diagonal, it is easy to verify that 2 is indeed an eigenvalue; in fact, 2 appears twice as a diagonal entry of A, thus appears twice as an eigenvalue.

Regardless, we need to find a basis for the eigenspace of A corresponding to λ = 2; accordingly, we look for vectors satisfying       2 0 0 0 x1 2x1       0 −1 0 0 x2 2x2     =   : 0 4 2 0 x3 2x3 4 0 0 3 x4 2x4

11 Section 5.1 carrying out the multiplication on the left hand side of the equation, we have       2 0 0 0 x1 2x1       0 −1 0 0 x2  −x2      =   . 0 4 2 0 x3 4x2 + 2x3 4 0 0 3 x4 4x1 + 3x4

In system form, we want

2x1 = 2x1

−x2 = 2x2

4x2 + 2x3 = 2x3

4x1 + 3x4 = 2x4.

Clearly x2 = 0, so that x1 and x3 are both free; we parameterize them as x1 = s and x3 = t. The variable x4 must be chosen so that x4 = −4x1, so any eigenvector of A corresponding to λ = 2 must have form       s 1 0  0   0  0   = s   + t   .  t   0  1 −4s −4 0 Thus the eigenspace corresponding to λ = 2 consists of all linear combinations of vectors     1 0  0  0   and   ;  0  1 −4 0 the eigenspace is two-dimensional with basis     { 1 0 }  0  0   ,   .  0  1 −4 0

Eigenvalues and Invertibility The eigenvalues of a matrix A give us a wealth of information about A; one of the most important pieces of information they provide us is described in the next theorem:

Theorem 5.1.4. A square matrix A is invertible if and only if 0 is not one of its eigenvalues.

12 Section 5.1 Example Use Theorem 4 to determine if each of the following matrices is invertible:

1.   5 −2 0 A = 0 −3 0  1 4 −1

2.   −3 0 0 0 0    12 5 0 0 0   L =  .2 3 −1 0 0 −10 3 1 −11 0 .5 −2 6 6 0

1. We have already seen that A has eigenvalues λ1 = −1, λ2 = −3, and λ3 = 5. Since A has no 0 eigenvalues, it is invertible.

2. L has a 0 eigenvalue (again, because eigenvalues of lower triangular matrices are the diagonal entries), so it is not invertible.

With the information we have just discussed about eigenvalues, we can now expand our list of equivalent statements about square matrices.

Theorem 5.1.8. Let A be an n × n matrix. Then the following are equivalent:

(a) A is invertible.

(b) Ax = 0 has only the trivial solution.

(c) The reduced row echelon form of A is In. (d) A is a product of elementary matrices.

(e) Ax = b is consistent for every n × 1 matrix b.

(f) Ax = b has exactly one solution for every n × 1 matrix b.

(g) det A ≠ 0.

(h) The column vectors of A are linearly independent.

(i) The row vectors of A are linearly independent.

(j) The column vectors of A span Rn (column (A) = Rn).

(k) The row vectors of A span Rn (row (A) = Rn).

(l) The column vectors of A form a basis for Rn.

13 Section 5.1 (m) The row vectors of A form a basis for Rn.

(n) rank (A) = n.

(o) nullity (A) = 0.

(p) The orthogonal complement of the null space of A is Rn.

(q) The orthogonal complement of the row space of A is 0.

(r) The kernel of TA is {0}.

n (s) The range of TA is R .

(t) TA is one-to-one. (u) λ = 0 is not an eigenvalue of A.

14