Eigenvectors and Eigenvalues

Eigenvectors and eigenvalues

Brian Krummel March 30, 2020

We will now look at eigenvalues and eigenvectors of a square matrix. When solving a problem in linear algebra, it is often convenient to choose our basis, or coordinate system. For a square matrix A, it is often convenient to work with a basis of eigenvectors, which are nonzero vectors for which multiplication by A is the same by scaling by a number λ. λ is called an eigenvalue. Definition 1. Let A be an n × n matrix. We call a scalar λ an eigenvalue if there is a nonzero vector X ∈ Rn such that AX = λX. We call the nonzero vector X an eigenvector. Example 1. Let 2 0 A = . 0 3 T Then E1 = 1 0 satisfies AE1 = 2E1, so E1 is an eigenvector with corresponding eigenvalue T 2. E1 = 0 1 satisfies AE2 = 3E2, so E2 is an eigenvector with corresponding eigenvalue 3. Notice that this tells us something very important about A: A acts on E1 and E2 by scaling: x2

AE2

E1 AE1 x1

We of course have A0 = λ0 for every scalar λ, but this does not tell us anything insightful about A the way that 2, 3 being eigenvalues does. x2 AB2 Example 2. Show that B1 is an eigenvector of A and ﬁnd the corresponding eigenvalue, where AB1 5 −1 1 A = ,B = . −1 5 1 1 B2 B1

Answer. We compute x1 5 −1 1 4 1 AB = = = 4 = 4B . 1 −1 5 1 4 1 1

So B1 is an eigenvector with corresponding eigenvalue 4.

1 x2

Ae2

e2 The main reason we care about eigenvalues and eigenvectors is that some matrices are “diag- onalizable” in terms of their eigenvalues and eigenvectors.e1 Ae1 Forx1 instance, the other eigenvector of A is 1 B = 2 −1 with eigenvalue 6. A scales by 4 in the B1-direction and A scales by 6 in the B2-direction. x2

AB2 AB1

B2 B1

Let’s discuss how we can ﬁnd eigenvalues and eigenvectors. Let A be an n × n matrix. Notice that AX = λX for a scalar λ and nonzero vector X ∈ Rn if and only if (A − λI) X = 0 has a nontrivial solution X. Hence λ is an eigenvalue if and only if the matrix A − λI is a singular matrix. In particular, λ must satisfy the characteristic equation

det(A − λI) = 0.

We call det(A − λI) the characteristic polynomial. det(A − λI) is a degree n polynomial of λ. Solving the characteristic equation for an n × n matrix A will give us n eigenvalues counting real and complex roots and repeated roots.

Example 3. Let 1 3 A = . 2 2 Find all the eigenvalues for A and a basis for the corresponding eigenspaces. Answer. Finding eigenvalues of A. Solving the characteristic equation

1 − λ 3 2 det(A − λI) = = (1 − λ)(2 − λ) − 6 = λ − 3λ − 4 = (λ − 4)(λ + 1) = 0. 2 2 − λ Therefore the eigenvalues of A are λ = 4, −1.

Having found an eigenvalue λ of A, we recall that a corresponding eigenvector X is a solution to the homogeneous linear system (A − λI) X = 0.

2 The eigenspace of A corresponding to the eigenvalue λ is the solution set to (A − λI) X = 0. Equivalently, the eigenspace is the null space Nul(A − λI). Clearly the eigenspace is a subspace of Rn. The eigenspace consists of the zero vector 0 and all non-zero eigenvalues of A corresponding to the eigenvalue λ. Since the eigenvectors corresponding to λ form a subspace (together with 0), there are infinitely many eigenvectors corresponding to λ. For instance, if X is an eigenvector such that AX = λX, then cX is also an eigenvector such that A(cX) = λ(cX) for every nonzero scalar λ. More generally, any linear combinations of eigenvectors corresponding to the same eigenvalue λ is again an eigenvector corresponding to λ. Consequently, to “find the eigenvalues of A corresponding to λ”, we tend to want to find a basis for the eigenspace corresponding to λ. We do this by solving the homogeneous linear system

(A − λI) X = 0 and letting the resulting spanning vectors be the basis for the eigenspace. Note that each eigenspace has many bases. In particular, how one approaches the algebra to solving (A−λI) X = 0 might yield diﬀerent bases for the eigenspace. Example 3 continued. Finding eigenvector corresponding to 4. We solve (A − 4I)x = 0.

−3 3 1 1 A − 4I = −→ . 2 −2 0 0 x2 is a free variable and x1 is a pivot variable with x1 = −x2. Since there is only one free variable, the eigenspace corresponding to 4 has dimension 1. In general, the dimension of the eigenspace is equal to the number of free variables of (A − λI) X = 0. Since the eigenspace corresponding to 4 is one-dimensional, it suﬃces to ﬁnd one eigenvector which spans this eigenspace. In particular, recalling that x1 = −x2, an eigenvector of A corresponding to λ = 4 is

−1 . 1

Finding eigenvector corresponding to −1. We solve (A + I)x = 0.

2 3 1 3/2 A + I = −→ . 2 3 0 0 x2 is a free variable and x1 is a pivot variable with x1 = (−3/2) x2, so a basis for the eigenspace of A corresponding to λ = −1 is (by setting x2 = 2)

−3 . 2

Note that we set x2 = 2 to clear fractions. Any other choice of x2 would have simply yielded another eigenvector spanning the eigenspace.

Example 4. Find the eigenvalues of the rotation matrix

0 −1 A = . 1 0

3 Answer. Solving the characteristic polynomial gives us

−λ −1 2 det(A − λI) = = λ + 1 = 0 1 −λ √ so λ = ±i, where i = −1. Thus the eigenvalues of A are complex numbers. We will eventually discuss complex eigenvalues and eigenvectors.

Example 5. Find the eigenvalues and eigenvectors of

 3 4 4  A =  0 5 2  . 0 0 3

Answer. Finding eigenvalues of A. Finding the characteristic polynomial by computing the determinant of an upper triangular matrix

3 − λ 4 4 2 det(A − λI) = 0 5 − λ 2 = (3 − λ) (5 − λ).

0 0 3 − λ Therefore the eigenvalues of A are the diagonal entries λ = 3, 5. Note that λ = 3 is an eigenvalue with multiplicity two since it comes from the factor (3−λ)2 with exponent 3. λ = 5 is an eigenvalue with multiplicity one since it comes from the factor (5 − λ) with exponent 1. In general, every n × n matrix has exactly n eigenvalues counting multiplicity.

Example 5 illustrates the following important theorem.

Theorem 1. Let A be an n × n upper triangular matrix. Then the eigenvalues of A are the diagonal entries of A.

Example 5 continued. Finding eigenvector corresponding to 5. Using Gaussian elimination to solve (A − 5I)X = 0:

 −2 4 4   −2 4 0   1 −2 0  (−1/2) R1 7→ R1 A − 5I =  0 0 2  −→  0 0 1  −−−−−−−−−→  0 0 1  0 0 −2 0 0 0 0 0 0

Thus x2 is a free variables and x1, x2 are pivot variables with x1 = 2x2 and x3 = 0. That is, the eigenspace corresponding to λ = 5 is one-dimensional and is spanned by the eigenvector

 2   1  . 0

Finding eigenvector corresponding to 2. Using Gaussian elimination to solve (A − 3I)X = 0:

 0 4 4   0 1 1  A − 3I =  0 2 2  −→  0 0 0  . 0 0 0 0 0 0

4 Thus x1, x3 are free variables and x2 is a pivot variable with x2 = −x3. That is, the basis for the eigenspace corresponding to λ = 3 is      1 0   0  ,  −1  .  0 1 

Notice that the dimension of the eigenspace corresponding to λ = 3 is equal to the number of free variables, which is two. Together, all three eigenvectors        2 1 0   1  ,  0  ,  −1   0 0 1  form a basis of R3. Warning! Let A be a square matrix and suppose we use row reduction to find an echelon form U of A. The eigenvalues of A are not typically the diagonal entries / eigenvalues of U. For instance, suppose we had 2 0 (1/2) R1 7→ R1 1 0 A = −−−−−−−−→ = U 0 3 (1/3) R2 7→ R2 0 1 so that by scaling the rows we find that the reduced echelon form of U of A is the identity matrix. Then A has eigenvalues λ = 2, 3, but U has eigenvalue λ = 1 with multiplicity two. That is, A and U have different eigenvalues as a result of scaling the rows. Note that we may compute the determinant det(A − λI) using row reduction, but this involves applying row reduction to A − λI. However, it was important that we subtracted λ from the diagonal entries before applying row reduction. This is not the same as row reducing A to find U – with no λ in sight – and then computing the eigenvalues of U. Note that not only does scaling rows change eigenvalues. Any elementary row operation – including interchanging rows and adding a multiple of one row to another row – can change the eigenvalues of a matrix.

Example 6. Find the eigenvalues and eigenvectors of

 2 1 4 3   0 1 −4 1  A =   .  0 1 6 0  0 0 0 2

Answer. Finding eigenvalues of A. Computing the determinant of det(A − λI) using cofactor expansions over the ﬁrst column and last row:

2 − λ 1 4 3

0 1 − λ −4 1 2 1 − λ −4 det(A − λI) = = (2 − λ) 0 1 6 − λ 0 1 6 − λ

0 0 0 2 − λ = (2 − λ)2((1 − λ)(6 − λ) + 4) = (2 − λ)2(λ2 − 7λ + 10) = (2 − λ)2(λ − 2)(λ − 5) = (λ − 2)3(λ − 5).

5 Therefore, A has eigenvalues λ = 2 with multiplicity three and λ = 5 with multiplicity one. Warning. We cannot ﬁnd the eigenvalues by Gaussian elimination

 2 1 4 3   2 1 4 3   0 1 −4 1  R3−R2 7→ R3  0 1 −4 1  A =   −−−−−−−→    0 1 6 0   0 0 10 −1  0 0 0 2 0 0 0 2 and then taking the eigenvalues λ = 2, 1, 10 of the echelon form of A. The eigenvalues of A should be λ = 2, 5. Finding eigenvector corresponding to 2. Using Gaussian elimination to solve (A − 2I)X = 0:

 0 1 4 3   0 1 4 0   0 −1 −4 1   0 0 0 1  A − 2I =   −→   ,  0 1 4 0   0 0 0 0  0 0 0 0 0 0 0 0 so that x1, x3 are free variables and x2 = −4x3 and x4 = 0, the eigenspace corresponding to 2 has the basis where  1   0     0   −4    ,   .  0   1   0 0  Notice that λ = 2 has multiplicity three, but the eigenspace only has dimension two. This illustrates that the dimension of an eigenspace can be less than the multiplicity of an eigenvalue. Finding eigenvector corresponding to 5. Using Gaussian elimination to solve (A − 5I)X = 0:

 −3 1 4 3   −3 1 4 0   1 0 −1 0   0 −4 −4 1   0 1 1 0   0 1 1 0  A − 5I =   −→   −→    0 1 1 0   0 0 0 1   0 0 0 1  0 0 0 −3 0 0 0 0 0 0 0 0 so that x3 is a free variable and x1 = x3, x2 = −x3, and x4 = 0. Thus the eigenspace corresponding to λ = 5 is spanned by  1   −1    .  1  0 Together  1   0   1     0   −4   −1    ,   ,    0   1   1   0 0 0  is a linearly independent set of three vectors in R4.

6 Example 5 and Example 6 illustrate the following general principle.

Theorem 2. Let A be an n × n matrix. Let X1,X2,...,Xp be eigenvectors of A with distinct corresponding eigenvalues λ1, λ2, . . . , λp. In other words, assume that AXi = λiXi for every i and λi 6= λj for every i 6= j. Then S = {X1,X2,...,Xp} is linearly independent.

Consequence. Let A be an n×n matrix. Each eigenvalue λi of A has a corresponding eigenspace

Eλi = Nul(A − λiI) which itself has a basis Bi of eigenvectors. Putting the bases of eigenvectors together in a single collection B = B1 ∪ B2 ∪ · · · ∪ Bp gives us a linearly independent set of vectors in Rn. Note that this collection B might not be a basis for Rn.

Reason for Theorem 2 for two vectors. Let’s consider the special case where X1,X2 are eigenvectors of A with corresponding eigenvalues λ1, λ2 with λ1 6= λ2; that is

AX1 = λ1X1, AX2 = λ2X2, λ1 6= λ2.

Suppose to the contrary that {X1,X2} were linearly dependent. Then X2 would be a scalar multiple of X1: X2 = cX1 for some scalar c. Since A scales in the X1 direction by λ1 and in the X2 direction by a diﬀerent quantity λ2, this is impossible. In particular, by multiplying by A:

λ2X2 = AX2 = c AX1 = cλ1X1.

Substituting X2 = cX1: λ2cX1 = λ1cX1 or equivalently (λ1 − λ2) cX1 = 0. (1)

For (1) to be true, either λ1 = λ2 or c = 0 or X1 = 0. We assumed that λ1 6= λ2. Since X1,X2 are both eigenvectors, neither X1,X2 equals 0. In particular, since X2 = cX1, c 6= 0. Thus (1) is impossible. Reason for Theorem 2 for three vectors (not shown in lecture). In the case of three or more eigenvectors X1,X2,...,Xp, we can use the same computation as we did for two vectors. Suppose we had three eigenvectors X1,X2,X3 of A with distinct corresponding eigenvalues λ1, λ2, λ3; that is,

AX1 = λ1X1, AX2 = λ2X2, AX3 = λ2X3, λ1, λ2, λ3 distinct.

Suppose {X1,X2,X3} were linearly dependent. Then at least one vector Xi is redundant. Let’s suppose that X3 is redundant so that

X3 = c1X1 + c2X2 (2) for some scalars c1, c2. By multiplying by A

AX3 = c1 AX1 + c2 AX2 which since AXi = λiXi gives us

λ3X3 = λ1c1X1 + λ2c2X2.

7 Substituting (2) in for X3:

λ3(c1X1 + c2X2) = λ1c1X1 + λ2c2X2. or equivalently (λ1 − λ3) c1X1 + (λ2 − λ3) c2X2 = 0. (3)

From our previous discussion, {X1,X2} is linearly independent, for (3) to be true we must have

(λ1 − λ3) c1 = 0 (λ2 − λ3) c2 = 0.

Since we assumed that λ1 6= λ3 and λ2 6= λ3, we must have that c1 = 0 and c2 = 0. Substituting c1 = 0 and c2 = 0 into (2) gives us X3 = 0, which is impossible since X3 is an eigenvector. Therefore, {X1,X2,X3} must be linearly independent.