Lecture 12: Diagonalization

A square matrix D is called diagonal if all but diagonal entries are zero:

a1 0 0 ¢ ¢ ¢ 0 a2 0 D = 2 ¢ ¢ ¢ 3 . (1) ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ 6 0 0 an 7 6 ¢ ¢ ¢ 7n n 4 5 £ Diagonal matrices are the simplest matrices that are basically equivalent to vectors in Rn. Obviously, D has eigenvalues a1, a2, ..., an, and for each i = 1, 2, ..., n,

a1 0 0 0 0 . ¢ ¢ ¢ . ¢ ¢.¢ ...... 2 . . . . 3 2.3 2 . 3 D~ei = 0 ai 0 1 = ai = ai~ei. 6 . ¢ ¢.¢ . ¢.¢ ¢ . 7 6.7 6 . 7 6 ...... 7 6.7 6 . 7 6 7 6 7 6 7 6 0 0 a 7 607 6 0 7 6 n7 6 7 6 7 4 ¢ ¢ ¢ ¢ ¢ ¢ 5 4 5 4 5 We thus conclude that for any diagonal matrix, eigenvalues are all diagonal entries, and ~ei is an eigenvector associated with ai, the ith diagonal entry, i.e.,

~ei is an eigenvector associated with eigenvalue ai, for i = 1, 2, ..., n.

Due to the simplicity of diagonal matrices, one likes to know whether any matrix can be similar to a diagonal matrix. Diagonalization is a process of …nding a diagonal matrix that is similar to a given non-diagonal matrix. De…nition 12.1. An n n matrix A is called diagonalizable if A is similar to a diagonal matrix D. £ Example 12.1. Consider

7 2 5 0 1 1 A = , D = , P = . 4 1 0 3 1 2 ·¡ ¸ · ¸ ·¡ ¡ ¸ 1 (a) Verify A = P DP ¡ (b) Find Dk and Ak (c) Find eigenvalues and eigenvectors for A. Solution: (a) It su¢ces to show that AP = P D and that P is invertible. Direct calculations lead to det P = 1 = 0 = P is invertible ¡ 6 ) Again, direct computation leads to

5 3 5 3 AP = , P D = . 5 6 5 6 ·¡ ¡ ¸ ·¡ ¡ ¸ 1 Therefore AP = P D. (b) 5 0 5 0 52 0 5k 0 D2 = = , Dk = . 0 3 0 3 0 32 0 3k · ¸ · ¸ · ¸ · ¸

2 1 1 1 1 2 1 A = P DP ¡ P DP ¡ = P DP ¡ P DP ¡ = P D P ¡ k k 1 A = P D P ¡ ¡ ¢

1 ~e = is an eigenvectors for λ = 5 : D~e = 5~e (2) 1 0 1 1 1 · ¸ 0 ~e = is an eigenvectors for λ = 3 : D~e = 3~e . (3) 2 1 2 2 2 · ¸ Since AP = P D, from (2) and (3), respectively, we see

AP~e1 = P D~e1 = P (5~e1) = 5P~e1,

AP~e2 = P D~e2 = P (3~e2) = 3P~e2.

Therefore, 1 1 1 1 P~e = = 1 1 2 0 1 ·¡ ¡ ¸ · ¸ ·¡ ¸ is an eigenvector associated with eigenvalue λ= 5 for matrix A, and

1 1 0 1 P~e = = 2 1 2 1 2 ·¡ ¡ ¸ · ¸ ·¡ ¸ is an eigenvector of A associated with eigenvalue λ= 3. From this example, we observation that if A is diagonalizable and A is similar to a diagonal matrix D (as in (1)) through an invertible matrix P,

AP = P D.

Then P~ei is an eigenvector associated with ai, for i = 1, 2, ..., n. This generalization can be easily veri…ed in the manner analogous to Example 12.1. More- over, these n eigenvectors P~e1, P~e2, ..., P~en are linear independent. To see independence, we consider the linear system n

λiP~ei = ~0. i=1 X

2 Suppose the linear system admits a solution (λ1, ..., λn). Then

P λi~ei = ~0. Ã i=1 ! X Since P is invertible, the above equation leads to

λi~ei = ~0. i=1 X Since has ~e1,~e2, ...,~en are linear independent, it follows that the last linear system has only the trivial solution λi = 0 for all i = 1, 2, ..., n. Therefore, P~e1, P~e2, ..., P~en are linear independent. This observation leads to Theorem 12.1. (Diagonalization).A n n matrix A is diagonalizable i¤ A has n linearly independent eigenvectors. Furthermor£e, suppose that A has n linearly independent eigenvectors ~v1,~v2, ...,~vn . Set f g a 0 0 1 ¢ ¢ ¢ 0 a2 0 P = [~v1,~v2, ...,~vn] , D = 2 ¢ ¢ ¢ 3 ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ ¢ 6 0 0 an 7 6 ¢ ¢ ¢ 7 4 5 where ai is the eigenvalue associated with ~vi, i.e., A~vi = ai~vi.Then, P is invertible and

1 P ¡ AP = D.

Proof. We have demonstrated that if A is diagonalizable, then A has n linearly independent eigenvectors. We next show that the reverse is also true: if A has n linearly independent eigenvectors, then A must be diagonalizable. To this end, we only need to verify that AP = P D with P and D described above, since P is obviously invertible due to independence of eigenvectors. The proof of AP = P D is straightforward by calculation as follows:

AP = A [~v1,~v2, ...,~vn]

= [A~v1, A~v2, ..., A~vn]

= [a1~v1, a2~v2, ..., an~vn] ,

P D = P [a1~e1, a2~e2, ..., an~en]

= [a1P~e1, a2P~e2, ..., anP~en] . Now, 1 0 0 1 P~e1 = [~v1,~v2, ...,~vn] 2 3 = ~v1, P~e2 = [~v1,~v2, ...,~vn] 2 3 = ~v2, ... ¢ ¢ ¢ ¢ ¢ ¢ 6 0 7 6 0 7 6 7 6 7 4 5 4 5 3 This shows AP = P D. Example 12.2. Diagonalize

1 3 3 A = 3 5 3 . 2¡3 ¡3 ¡1 3 4 5 Solution: Diagonalization means …nding a diagonal matrix D and an invertible matrix P such that AP = P D. We shall follow Theorem 12.1 step by step. Step 1. Find all eigenvalues. From computations

1 λ 3 3 det (A λI) = det ¡3 5 λ 3 ¡ 2 ¡3 ¡ 3¡ 1¡ λ3 ¡ = λ34 3λ2 + 4 5 ¡ ¡ = λ3 + λ2 4λ2 + 4 ¡ ¡ = λ3 + λ2 + 4λ2 + 4 ¡ ¡ ¡¢ = λ2 (λ 1) 4 λ2 1 ¡ ¡ ¢ ¡¡ ¡ ¢ = (λ 1) λ2 + 4 (λ+ 1) ¡ ¡ ¡ ¢ = (λ 1) (λ+ 2)2 . ¡ ¡ £ ¤ we see that A has eigenvalues λ1 = 1, λ2 = 2 (this is a double root). ¡ Step 2. Find all eigenvalues – …nd a basis for each eigenspace Null (A λIi). ¡ For λ1 = 1,

1 3 3 1 0 0 A λ1I = 3 5 3 0 1 0 ¡ 2¡3 ¡3 ¡1 3 ¡ 20 0 13 4 0 3 3 5 4 3 5 3 0 R1 R2 = 3 6 3 ! 3 6 3 2¡3 ¡3 ¡0 3 ! 2¡0 ¡3 ¡3 3 4 3 3 5 0 4 1 0 1 5 R2+R1 R2 ¡ ! 0 3 3 0 1 1 . ! 20 ¡3 ¡3 3 ! 20 0 0 3 4 5 4 5 Therefore, linear system (A I) ~x = 0 ¡ reduces to

x1 x3 = 0 ¡ x2 + x3 = 0,

4 where x3 is the free variable. Choose x3 = 1, we obtain an eigenvector 1 ~x = 1 . 2¡1 3 For λ = 2, 4 5 2 ¡ 1 3 3 1 0 0 A λ2I = 3 5 3 ( 2) 0 1 0 ¡ 2¡3 ¡3 ¡1 3 ¡ ¡ 20 0 13 4 3 3 3 5 1 41 1 5 = 3 3 3 0 0 0 . 2¡3 ¡3 ¡3 3 ! 20 0 03 The corresponding linear system is4 5 4 5

x1 + x2 + x3 = 0

It follows that x2 and x3 are free variables. As we did before, we need to select (x2, x3) to be (1, 0) and (0, 1) . Choose x2 = 1, x3 = 0 = x1 = x2 x3 = 1; choose x2 = 0, ) ¡ ¡ ¡ x3 = 1 = x1 = x2 x3 = 1. We thus got two independent eigenvectors for λ2 = 2 : ) ¡ ¡ ¡ ¡ x1 1 x1 1 ¡ ¡ ~v = x2 = 1 , ~v = x2 = 0 . 2 2 3 2 3 3 2 3 2 3 x3 0 x3 1 Step 3. Assemble orderly D a4nd P5 as 4follow5s: there4are5seve4ral c5hoices to pair D and P. 1 0 0 1 1 1 ¡ ¡ Choice#1 : D = 0 2 0 , P = [~v1,~v2,~v3] = 1 1 0 20 ¡0 23 2¡1 0 1 3 ¡ 41 0 0 5 4 1 1 15 ¡ ¡ Choice#2 : D = 0 2 0 , P = [~v1,~v3,~v2] = 1 0 1 20 ¡0 23 2¡1 1 0 3 ¡ 4 2 0 05 4 1 1 1 5 ¡ ¡ ¡ Choice#3 : D = 0 2 0 , P = [~v2,~v3,~v1] = 1 0 1 . 2 0 ¡0 13 2 0 1 ¡1 3 Remark. Not every matr4ix is diagona5lizable. For instance,4 5 1 1 A = , det (A λI) = (λ 1)2 . 0 1 ¡ ¡ · ¸ The only eigenvalue is λ= 1; it has the multiplicity m = 2. From 0 1 A I = , ¡ 0 0 · ¸ 5 we see that (A I) has only one pivot. Thus r (A I) = 1. By Dimension Theorem, we have ¡ ¡ dim Null (A I) = 2 r (A) = 2 1 = 1. ¡ ¡ ¡ In other words, the basis consists of Null (A I) consists of only one vector. For instance, one may choose ¡ 1 0 · ¸ as a basis. According to the discussion above, if A is diagonalizable, i.e., AP = P D for a diagonal matrix a 0 D = 0 b · ¸ then

P~e1 is an eigenvector of A associated with a,

P~e2 is an eigenvector of A associated with b. Furthermore, since P is invertible,

P~e1 and P~e2 are linearly independent (why?). Since we have already demonstrated that there is no more than two linearly independent eigenvectors, it is impossible to diagonalize A. In general, if λ0 is an eigenvalue of multiplicity m (i.e., the characteristic polynomial det (A λI) = (λ λ )m Q (λ) , Q (λ ) = 0 ), ¡ ¡ 0 0 6 then dim (Null (A λI)) m. ¡ · Theorem 12.2. Let A be an n n matrix with distinct real eigenvalues λ1, λ2, ..., λp £ with multiplicity m1, m2, ..., mp, respectively. Then,

1. ni = dim (Null (A λiI)) mi and m1 + m2 + ... + mp n. ¡ · · 2. A is diagonalizable i¤ ni = mi for all i = 1, 2, ..., p, and

m1 + m2 + ... + mp = n.

1 In this case, we have P ¡ AP = D, where P and D can be obtained as follows. Let i B be a basis of Null (A λiI) for each i. Then ¡

λ1Im1 ... 0 P = [ , ..., ] , D = ...... , I = (m m ) identity B1 Bp 2 3 mi i £ i 0 ... λpImp 4 5 i.e., the …rst m1 columns of P form 1, the eigenvectors for λ1, the next m2 columns of P are , then , etc. The last mB columns of P are from ; the …rst m diagonal B2 B3 p Bp 1 entries of D are λ1, the next m2 diagonal entries of D are λ2, and so on.

6 3. In particular, if A has n distinct real eigenvalues, then A is diagonalizable.

4. Any symmetric matrix is diagonalizable.

Note that, as we saw before, there are multiple choices for assembling P. For instance, if A is 5 5, and A has two eigenvalues λ = 2, λ = 3 with a basis ~a ,~a for Null (A 2I) £ 1 2 f 1 2g ¡ and a basis ~b ,~b ,~b for Null (A 3I) , respectively, then, we have several choices to select 1 2 3 ¡ pairs of (P,nD) : o

~ ~ ~ 2I2 0 choice#1 : P = ~a1,~a2, b1, b2, b3 , D = 0 3I3 · ¸ 2h 0 i 2 0 0 0 0 0 0 2 0 2 0 0 0 = 2· ¸ 3 0 0 3 = 20 0 3 0 03 6 0 0 3 0 7 60 0 0 3 07 6 2 37 6 7 6 0 0 3 7 60 0 0 0 37 6 7 6 7 4 4 55 4 2 0 0 0 50 0 3 0 0 0 ~ ~ ~ 2 3 choice#2 : P = ~a1, b1, b3,~a2, b2 , D = 0 0 3 0 0 . 60 0 0 2 07 h i 6 7 60 0 0 0 37 6 7 4 5 Example 12.3. Diagonalize A

5 0 0 0 0 5 0 0 A = . 2 1 4 3 0 3 ¡ 6 1 2 0 37 6¡ ¡ ¡ 7 4 5 Solution: We need to …nd all eigenvalues and eigenvectors. Since A is an upper triangle matrix, we see that eigenvalues are λ = 5, m = 2 , λ = 3, m = 2. 1 1 2 ¡ 2 For λ1 = 5,

0 0 0 0 0 0 0 0

0 0 0 0 R4+R3 R4 0 0 0 0 A 5I = 2 3 ! 2 3 ¡ 1 4 8 0 ! 1 4 8 0 ¡ ¡ 6 1 2 0 87 60 2 8 87 6¡ ¡ ¡ 7 6 ¡ ¡ 7 4 0 0 0 0 5 0 0 04 0 5 0 0 0 0 0 0 0 0 2 3 2 3 ! 1 4 8 0 ! 1 0 8 16 ¡ 60 1 4 47 60 1 4 47 6 ¡ ¡ 7 6 ¡ ¡ 7 4 5 4 5

7 Therefore, x3 and x4 are free variable, and x = 8x 16x 1 ¡ 3 ¡ 4 x2 = 4x3 + 4x4.

Choose (x3, x4) = (1, 0) = x1 = 8, x2 = 4; Choose (x3, x4) = (0, 1) = x1 = 16, x2 = 4. We obtain two independe)nt eigenv¡ectors ) ¡ 8 16 ¡4 ¡4 , (for λ = 5). 2 1 3 2 0 3 1 6 0 7 6 1 7 6 7 6 7 For λ = 3, 4 5 4 5 2 ¡ 8 0 0 0 8 0 0 0

0 8 0 0 R4+R3 R4 0 8 0 0 A ( 3) I = 2 3 ! 2 3 ¡ ¡ 1 4 0 0 ! 1 4 0 0 6 1 2 0 07 60 2 0 07 6¡ ¡ 7 6 7 4 5 4 5 R3 2R4 R3 8 0 0 0 0 0 0 0 R ¡ 4R ! R 2 4 2 0 0 0 0 R1 8R3 R1 0 0 0 0 ¡ ! 2 3 ¡ ! 2 3 . ! 1 0 0 0 ! 1 0 0 0 60 2 0 07 60 2 0 07 6 7 6 7 Hence 4 5 4 5 x1 = 0, x2 = 0.

Choose (x3, x4) = (1, 0) and (x3, x4) = (0, 1) , respectively, we have eigenvectors 0 0 0 0 2 3 and 2 3 (for λ2 = 3). 1 0 ¡ 607 617 6 7 6 7 4 5 4 5 Assemble pairs (P, D) : There are several choice. For instance, 8 16 0 0 5 0 0 0 ¡4 ¡4 0 0 0 5 0 0 P = , D = 2 1 0 1 03 20 0 3 0 3 ¡ 6 0 1 0 17 60 0 0 37 6 7 6 ¡ 7 or 4 5 4 5 8 0 16 0 5 0 0 0 ¡4 0 ¡4 0 0 3 0 0 P = , D = . 2 1 1 0 03 20 ¡0 5 0 3 6 0 0 1 17 60 0 0 37 6 7 6 ¡ 7 4 5 4 5

8 Homework 12. ² 1. Suppose that A is similar to ² 1 0 0 0 ¡0 2 0 0 D = . 2 0 0 0 0 3 6 0 0 0 17 6 ¡ 7 4 5 (a) Find the characteristic polynomial and eigenvalues of A. 1 (b) Let P ¡ AP = D, where

2 1 0 5 0 3 0 2 P = . 2 1 0 1 23 ¡ 6 0 1 0 47 6 7 4 5 Find a basis for each eigenspace of A. 2. Diagonalize the following matrices. 2 2 1 (a) A = 1 3 ¡1 , λ= 1, 5. 2 1 2 ¡2 3 ¡ ¡ 42 0 0 0 5 4 3 0 0 (b) B = 21 ¡2 1 03 60 1 0 17 6 7 3. Show tha4t if A is diago5nalizable, so is A2. 4. Suppose that n n matrices A and B have exact the same linearly independent eigenvectors tha£t span Rn. Show that (a) both A and B can be simultaneously 1 1 diagonalized (i.e., there are the same P such that P ¡ AP and P ¡ BP are diagonal matrices), and (b) AB is also diagonalizable. 5. for each statement, determine and explain true or false. 1 (a) A is diagonalizable if A = P ¡ DP for some diagonal matrix D. (b) If A is diagonalizable, then A is invertible. (c) If AP = P D with D diagonal matrix, then nonzero columns of P are eigenvectors of A. (d) Let A be a symmetric matrix and λ be an eigenvalue of A with multiplicity 5. Then the eigenspace Null (A λI) has dimension 5. ¡