Eigenvalues & Eigenvectors Eigenvalues and Eigenvectors
Eigenvalue problem (one of the most important problems in the linear algebra) : If A is an n×n matrix, do there exist nonzero vectors x in Rn such that Ax is a scalar multiple of x？ (The term eigenvalue is from the German word Eigenwert , meaning “proper value”)
Eigenvalue Geometric Interpretation A: an n×n matrix λ: a scalar (could be zero ) x: a nonzero vector in Rn Eigenvalue Ax= λ x
Eigenvector 2 Eigenvalues and Eigenvectors
Example 1: Verifying eigenvalues and eigenvectors. 2 0 1 0 A = x = x = 0 −1 1 0 2 1 Eigenvalue 201 2 1 In fact, for each eigenvalue, it has infinitely many eigenvectors. Ax1 = ===2 2 x 1 010− 0 0 For λ = 2, [3 0]T or [5 0]T are both corresponding eigenvectors. Eigenvector Moreover, ([3 0] + [5 0]) T is still an eigenvector. Eigenvalue The proof is in Theorem. 7.1. 200 0 0 Ax= = =−1 =− ( 1) x 2 011− − 1 1 2
Eigenvector 3 Eigenspace Theorem 7.1: (The eigenspace corresponding to λ of matrix A) If A is an n×n matrix with an eigenvalue λ, then the set of all eigenvectors of λ together with the zero vector is a subspace of Rn. This subspace is called the eigenspace of λ.
Proof:
x1 and x2 are eigenvectors corresponding to λ
(i.e., Ax1 λ=λ x 1 , A x 2 = x 2 )
(1) A (xx12+=+ ) A xx 1 A 2 =+= λ λ λ xx 12 ( xx 12 + )
( i.e., x1+ x 2 is also an eigenvector corr esponding to λ)
(2)( Acx1 )= cA ( x 1 ) = c ( λλ x 1 ) = ( c x 1 )
( i.e., cx1 is also an eigenvector corre sponding to λ) Since this set is closed under vector addition and scalar multiplication, this set is a subspace of Rn according to Theorem 4.5. 4 Eigenspace Eamplex 3: Examples of eigenspaces on the xy -plane
For the matrix A as follows, the corresponding eigenvalues are λ1 = –1 and λ2 = 1: −1 0 A = 0 1
Solution
For the eigenvalue λ1 = –1 , corresponding vectors are any vectors on the x-axis Thus, the eigenspace corresponding to λ = –1 x−1 0 xxx − 2 A= = =− 1 is the x-axis, which is a subspace of R . 0 010 0 0
For the eigenvalue λ2 = 1, corresponding vectors are any vectors on the y-axis 0− 100 0 0 Thus, the eigenspace corresponding to λ = 1 A= = = 1 is the y-axis, which is a subspace of R2. y0 1 yyy
5 Eigenspace
Geometrically speaking, multiplying a vector (x, y) in R2 by the matrix A in the example, corresponds to a reflection to the y-axis.
x x0 x 0 AAAv = = += AA + y 0 y 0 y x 0 − x =−1 + 1 = 0 y y
6 Finding eigenvalues and eigenvectors
Theorem 7.2: ( Finding eigenvalues and eigenvectors of a matrix A∈Mn×n ) Let A be an n×n matrix. (1) An eigenvalue of A is a scalar λ such that det(λI− A ) = 0 . (2) The eigenvectors of A corresponding to λ are the nonzero solutions of (λI− A ) x = Θ .
7 Finding eigenvalues and eigenvectors Note: some definitions of the eigenvalue problem: homogeneous system A λxxxx λ = λ ⇒ AI= ⇒ ( IA− ) x = Θ (λI− A ) x = Θ has nonzero solutions for x iff det(λI− A ) = 0
Characteristic equation of A: det(λI− A ) = 0
Characteristic polynomial of A∈Mn×n: n n −1 λ λ det( λ λ λ IAIA−= ) ( −=+ ) cn−1 +++ cc 1 0
8 Finding eigenvalues and eigenvectors
Example 4: Finding eigenvalues and eigenvectors 2 −12 A = 1 − 5 Solution: Characteristic equation:
λ − 2 12 det(λI− A ) = −1λ + 5 = λ λ λ λ 2 +3 +=+ 2 ( 1)( += 2) 0
⇒ λ = − ,1 − 2
Eigenvalue: λ1 = − ,1 λ2 = −2
9 Finding eigenvalues and eigenvectors −312 x 0 λ = − ⇒ 1 (1) 1 1 (λ1I− A ) x = = −1 4 x2 0
−312 G.-J. E. 14 − ⇒ → −14 00
x1 4t 4 ⇒ = =t , t ≠ 0 x2 t 1
−4 12 x 0 ⇒ (λ I− A ) x =1 = (2) λ2 = − 2 2 −1 3 x2 0
−412 G.-J. E. 13 − ⇒ → −13 00
x1 3s 3 ⇒ = =s , s ≠ 0 x2 s 1
10 Finding eigenvalues and eigenvectors Example 5: Find the eigenvalues and corresponding eigenvectors for the matrix A. What is the dimension of the eigenspace of each eigenvalue? 2 1 0 A = 0 2 0 0 0 2
Solution: Characteristic equation:
λ −2 − 1 0 λ λ λ I−= A 0 − 2 0 =−= (2)03 0 0λ − 2
Eigenvalue: λ = 2
11 Finding eigenvalues and eigenvectors
The eigenspace of λ = 2:
010− x1 0 λ − = = (I A )000x x 2 0 000 x3 0
x1 s 1 0 x = 0 = s0 + t0, s,t ≠ 0 2 x3 t 0 1
1 0 ststR00,+ ∈ :the eigenspace of A corresponding to λ = 2 0 1 Thus, the dimension of its eigenspace is 2.
12 Finding eigenvalues and eigenvectors
Notes:
(1) If an eigenvalue λ1 occurs as a multiple root ( k times) for
the characteristic polynominal, then λ1 has multiplicity k. (2) The multiplicity of an eigenvalue is greater than or equal to the dimension of its eigenspace.
In Example. 5, k is 3 and the dimension of its eigenspace is 2.
13 Finding eigenvalues and eigenvectors Example 6: Find the eigenvalues of the matrix A and find a basis for each of the corresponding eigenspaces 1 0 0 0 0 1 5 −10 A = 1 0 2 0 1 0 0 3
Solution: Characteristic equation: According to the note on the previous slide, the dimension of λ −10 0 0 λ the eigenspace of 1 = 1 is at most 0λ − 1 − 5 10 to be 2. λI− A = λ λ −1 0λ − 20 For 2 = 2 and 3 = 3, the −10 0λ − 3 demensions of their eigenspaces are at most to be 1. =− λ( λ λ 1)2 ( − 2)( −= 3) 0
Eigenvalues: λ1 = ,1 λ2 = ,2 λ3 = 3
14 Finding eigenvalues and eigenvectors
0000 x1 0 00− 510x 0 (1) λ = 1 ⇒ (λ I− A ) x = 2 = 1 1 −1010 − x 0 3 −1002 − x4 0
x1 −2t 0 − 2 G.-J.E. x s 1 0 ⇒ 2 ==+ s t , st, ≠ 0 x 2t 0 2 3 x4 t 0 1
0 − 2 1 0 ⇒ , is a basis for the eigenspace corresponding to λ1 = 1. 0 2 0 1
λ The dimension of the eigenspace of 1 = 1 is 2.
15 Finding eigenvalues and eigenvectors
1000 x1 0 01− 510x 0 (2) λ = 2 ⇒ (λ I− A ) x = 2 = 2 2 −1000 x 0 3 −100 − 1 x4 0
x1 0 0 G.-J.E. x 5t 5 ⇒ 2 = =t , t ≠ 0 x t 1 3 x4 0 0
0 5 λ ⇒ is a basis for the eigenspace corresponding to 2 = 2 1 0 λ The dimension of the eigenspace of 2 = 2 is 1.
16 Finding eigenvalues and eigenvectors
2000 x1 0 0 2− 5 10x 0 (3) λ = 3 ⇒ (λ I− A ) x = 2 = 3 3 −1010 x 0 3 −1000 x4 0
x1 0 0 G.-J.E. x −5t − 5 ⇒ 2 = =t , t ≠ 0 x 0 0 3 x4 t 1
0 − 5 ⇒ λ is a basis for the eigenspace corresponding to 3 = 3. 0 1
λ The dimension of the eigenspace of 3 = 3 is 1.
17 Eigenvalues for triangular matrices Theorem 7.3: If A is an n×n triangular matrix, then its eigenvalues are the entries on its main diagonal.
18 Eigenvalues for triangular matrices Example 7: Finding eigenvalues for triangular and diagonal matrices −100 0 0 2 0 0 0 20 0 0 (a) A = − 1 1 0 (b) A = 0 00 0 0 5 3− 3 0 00− 40 0 00 0 3 Solution: λ − 2 0 0 λ λ λ (a) λ λ I−= A 1 − 1 0 =−−+= ( 2)( 1)( 3) 0 −5 − 3λ + 3 According to Theorem 3.2, the ⇒ λ λ λ =2, = 1, =− 3 determinant of a triangular 1 2 3 matrix is the product of the entries on the main diagonal . λ(b) λ λ λ λ 1=− 1, 234 = 2, = 0, =− 4, 5 = 3
19 Eigenvalues and eigenvectors of L.T.
A number λ is called an eigenvalue of a li near transformation TVV:→ if there is a nonzero vector x such that T ( x )= λ x . The vector x is called an eigenvector of T c orresponding to λ , and the set of all eigenvectors of λ (together with the zero vector) is called the eigenspace of λ .
20 Eigenvalues and eigenvectors of L.T.
The definition of linear transformation functions was introduced in Chapter 6. The typical example of a linear transformation function is that each component of the resulting vector is the linear combination of the components in the input vector x. An example for a linear transformation T: R3→R3
Txxx(,,)(123=+ x 1 3,3 xxx 212 +− ,2) x 3
21 Eigenvalues and eigenvectors of L.T. Example 8: Find eigenvalues and eigenvectors for standard matrices Find the eigenvalues and corresponding e igenvectors for 1 3 0 A is the standard matrix for A = 3 1 0 T(x1, x2, x3) = ( x1 + 3x2, 3x1 + x2, –2x3) 0 0− 2
Solution λ −1 − 3 0 2 λI − A = − 3 λ −1 0 =+( λλ 2) ( −= 4) 0 0 0 λ + 2
⇒ eigenvalues λ λ 1= 4, 2 = − 2
For λ1 = 4, the corresponding eigenvector is (1, 1, 0).
For λ2 = −2, the corresponding eigenvectors are (1, − 1 , 0) and (0, 0, 1).
22 Eigenvalues and eigenvectors of L.T.
Notes: The relationship among eigenvalues, eigenvectors, and diagonalization Let T : R3→ R 3 be the linear transformation whos e corresponding standard matrix is A in Example 8, and let B' be a basis of R 3 made up of three linearly independent eigenvectors of A found in Example 8,
i.e., B'= {v1 , v 2 , v 3 } = {(1, 1, 0),(1, − 1, 0),(0, 0, 1)} Then A' , the matrix of T relative to the ba sis B ' , defined as
[[T (v1' )]B [ T ( v 2' )] B [ T ( v 3' )] B ] is diagonal, and the main diagonal entries are correspond ing eigenvalu es.
λ = for λ =− 2 4 0 0 for 1 4 2 A'= 0 − 2 0 Eigenvalues of A B '= {(1, 1, 0),(1, − 1, 0),(0, 0, 1)} 0 0 − 2 Eigenvectors of A 23 Diagonalization
Diagonalization problem: For a square matrix A, does there exist an invertible matrix P such that P–1AP is diagonal? Diagonalizable matrix: Definition 1: A square matrix A is called diagonalizable if there exists an invertible matrix P such that P–1APis a diagonal matrix (i.e., P diagonalizes A) Definition 2: A square matrix A is called diagonalizable if A is similar to a diagonal matrix.
In Sec. 6.4, two square matrices A and B are similar if there exists an invertible matrix P such that B = P–1AP.
Notes: In this section, it is shown that the eigenvalue problem is related closely to the diagonalization problem.
24 Diagonalization Theorem. 7.4: Similar matrices have the same eigenvalues If A and B are similar n×n matrices, then they have the same eigenvalues. Proof: A and B are similar ⇒ B = P−1 AP Consider the characteristic equation of B: For any diagonal matrix in the form of D = λI, P–1DP = D
λ λ λ λ IB−=− IPAP−1 = P − 1 IPPAP − − 1 = P − 1 ( I − AP ) λ= λ P−1λ IAP −= PPIA − 1 −= PPIA − 1 − =λI − A
Since A and B have the same characteristic equation, they are with the same eigenvalues.
25 Diagonalization
Example 1: Eigenvalue problems and diagonalization programs 1 3 0 A = 3 1 0 0 0 − 2
Solution: Characteristic equation: λ −1 − 3 0 λ λ λ λ I−=− A 3 − 1 0 =− ( 4)( += 2)2 0 0 0λ + 2
The eigenvalues λ : λ λ 1= 4, 2 =− 2, 3 =− 2 1 ⇒ = (1) λ = 4 the eigenvector p1 1 0 26 Diagonalization 1 0 = − = (2) λ = − 2⇒ the eigenvector p21, p 3 0 0 1 110 400 = =−−1 =− P[p1 p 2 p 3 ] 1 1 0 , and P AP 0 2 0 001 002−
Note : If P = [p2 p 1 p 3 ] 110 − 200 −1 = − 11 0 ⇒ P AP = 0 4 0 001 002−
The above example can verify Theorem 7.4 since the eigenvalues for both A and P–1AP are the same to be 4, –2, and –2 The reason why the matrix P is constructed with the eigenvectors of A is demonstrated in Theorem 7.5 (see the next slide).
27 Diagonalization Theorem 7.5: Condition for diagonalization An n×n matrix A is diagonalizable if and only if it has n linearly independent eigenvectors. If there are n linearly independent eigenvectors, it does not imply that there are n distinct eigenvalues. In an extreme case, it is possible to have only one eigenvalue with the multiplicity n, and there are n linearly independent eigenvectors for this eigenvalue. However, if there are n distinct eigenvalues, then there are n linearly independent eivenvectors (see Thm. 7.6), and thus A must be diagonalizable.
Proof: ( ⇒) Since A is diagonalizable, there exists a n invertible P s.t. D= PAP−1 is diagonal.
Let P=[p12 p p n ] and D = diag (,,,),λ λ λ 12 n then
λ1 0 0 0λ 0 PD = [p p p ] 2 1 2 n 0 0 λn
= λ[λ λ 11p 2 p 2 n p n ] 28 Diagonalization
AP= PD (since D = PAP−1 )
[App12 A A pn][= λλ λ 1122 pp n p n ]
⇒ Api=λ i p i , i = 1, 2,… , n
(The above equations imply the column vec tors pi of P are eigenvectors
of A , and the diagonal entries λi in D are eigenvalues of A )
Because A is diagonalizable⇒ P is invertible ⇒ Columns in P , i.e., p1, p 2 , , p n , are linearly independ ent
Thus, A has n linearly independent eigenve ctors.
(⇐)
Since A has n l inearly independent eig envectors p1 , p 2 , p n
with corresponding eigenvalues λ λ1, 2 , n , then ⇒ Api=λ i p i , i = 1, 2,… , n
Let P = [p1 p 2 p n ] 29 Diagonalization
AP= A[pp12 pn][ = A pp 12 A A p n ]
= λ[λ λ 11p 2 p 2 n p n ]
λ1 0 0 0λ 0 =[p p p ] 2 = PD 1 2 n 0 0 λn
Since p1, p 2 , , p n are linearly independent ⇒ P is invertible ⇒ AP= PD⇒ P−1 AP= D ⇒ A is diagonalizable (according to the definition of the diag onalizable matrix)
Note that pi 's are linearly independent ei genvectors and the diagonal
entries λi in the resulting diagonalized D are eigenvalues of A .
30 Diagonalization Example 4: A matrix that is not diagonalizable Show that the following matrix is not di agonalizable 1 2 A = 0 1
Solution: Characteristic equation: λ −1 − 2 λ λ I−= A =−=( 1)2 0 0λ − 1
The eigenvalue λ λ 1=1, and then solve ( 1 I −=Θ A ) x for eige nvectors 0− 2 1 λ I− A = I − A = ⇒ eigenvector p = 1 0 01 0
Since A does not have two linearly independent eigenvectors, A is not diagonalizable.
31 Diagonalization
Steps for diagonalizing an n×n square matrix:
Step 1: Find n linearly independent eigenvectors p1, p 2 , p n
for A with corresponding eigenvalues λ λ λ 1, 2 ,… , n
Step 2: Let P = [p1 p 2 p n ]
Step 3: λ1 0 0 0 λ 0 P−1 AP = D = 2 0 0 λn
where Api=λ i p i , i = 1, 2,… , n
32 Diagonalization
Example 5: Diagonalizing a matrix 1 −1 −1 A = 1 3 1 − 3 1 −1 Find a matrix P such that P−1 AP is diagonal.
Solution: Characteristic equation:
λ −1 1 1 λ λ λ λ λ I−=− A 1 − 3 −=− 1 ( 2)( + 2)( −= 3) 0 3− 1λ + 1
The eigenvalues: λ λ λ 1=2, 2 =− 2, 3 = 3
33 Diagonalization
111 101 λ = ⇒ λ − = − − − →G.-J. E. 1 2 1I A 111 010 313− 000
x1 − t − 1 = ⇒ = x2 0 eigenvector p1 0 x3 t 1
−311 10 − 1 4 λ = − ⇒ λ − = − − − →G.-J. E. 1 2 2 2 I A 151 01 4 311− − 000
1 x1 t 1 4 = − 1 ⇒ = − x2 4 t eigenvector p2 1 x3 t 4 34 Diagonalization
211 101 ⇒ λ I−=− A 101 −→G.-J. E. 011 − λ3 = 3 3 314− 000
x1 − t − 1 = ⇒ = x2 t eigenvector p3 1 x3 t 1
−1 1 − 1 = = − P [p1 p 2 p 3 ] 0 1 1 and it follows that 1 4 1 2 0 0 −1 P AP =0 − 2 0 0 0 3
35 Diagonalization
Note: a quick way to calculate Ak based on the diagonalization technique k λ1 0 0 λ 0 0 1 0λ 0 0λ k 0 (1) D= 2 ⇒ D k = 2 k 0 0 λn 0 0 λn
−1k −−−− 1111 k (2) D= PAP ⇒ D= PAPPAP PAP = PAP repeat k times λ k 0 0 1 0λ k 0 Ak= PD k P−1 , where D k = 2 k 0 0 λn
36 Diagonalization Theorem 7.6: Sufficient conditions for diagonalization If an n×n matrix A has n distinct eigenvalues, then the corresponding eigenvectors are linearly independent and thus A is diagonalizable according to Theorem 7.5.
Proof: λ λ λ Let 1, 2, …, n be distinct eigenvalues and corresponding eigenvectors be x1, x2, …, xn. In addition, consider that the first m eigenvectors are linearly independent, but the first m+1 eigenvectors are linearly dependent, i.e.,
xm+1=c 11 xx + c 22 ++ c m x m , (1)
where ci’s are not all zero. Multiplying both sides of Eq. (1) by A yields
Axm+1= Ac 11 xx + Ac 22 ++ Ac m x m
λ λ λ λ m+1x m + 1=c 111 xx + c 222 ++ c m m x m (2)
37 Diagonalization
λ On the other hand, multiplying both sides of Eq. (1) by m+1 yields
λ λ λ λ mm+++11111212x=c m xx + c m + ++ c mmm + 1 x (3)
Now, subtracting Eq. (2) from Eq. (3) produces
λ λ c λ λ 11112122( λ λ m+−+)x c ( m + −++ )x c mmmm ( + 1 − )x = 0
Since the first m eigenvectors are linearly independent, we can infer that all coefficients of this equation should be zero, i.e.,
λ λ c111212 λ λ ( λ λ m+−= )( c m + −== ) c mmm ( + 1 −= )0
Because all the eigenvalues are distinct, it follows all ci’s equal to 0, which contradicts our assumption that xm+1 can be expressed as a linear combination of the first m eigenvectors. So, the set of n eigenvectors is linearly independent given n distinct eigenvalues, and according to Thm. 7.5, we can conclude that A is diagonalizable.
38 Diagonalization
Example 7: Determining whether a matrix is diagonalizable 1 − 2 1 A = 0 0 1 0 0 − 3 Solution: Because A is a triangular matrix, its eigenvalues are
λ λ λ 1=1, 2 = 0, 3 =− 3.
According to Theorem 7.6, because these three values are distinct, A is diagonalizable.
39 Diagonalization Example 8: Finding a diagonalized matrix for a linear transformation Let T : R3→ R 3 be the linear transformation given by
Tx,x,x(123 )(=−− x 1231 x x, x +3 x 23 +−+− x, 3 x 123 x x ) Find a basis B' for R3 such that the matrix for T relative to B ' is diagonal.
Solution: The standard matrix for T is gven by 1− 1 − 1 A = 1 3 1 −3 1 − 1 λ λ λ From Example 5 you know that 1 = 2, 2 = –2, 3 = 3 and thus A is diagonalizable. So, these three linearly independent eigenvectors found in Example 5 can be used to form the basis B′ . That is 40 Diagonalization
B '= {v1 , v 2 , v 3 } =− {( 1, 0, 1),(1, −− 1, 4),( 1, 1, 1)} The matrix for T relative to this basis is
AT'= [ [ (v1' )]B [ T ( v 2' )] B [ T ( v 3' )] B ] 2 0 0 =0 − 2 0 0 0 3
Note that it is not necessary to calculate A′ through the above equation, we already know that A′ is a diagonal matrix and its main diagonal entries are corresponding eigenvalues of A.
41 Symmetric Matrices and Orthogonal Diagonalization
Symmetric matrix: A square matrix A is symmetric if it is equal to its transpose: T A = A
Example 1: Symmetric matrices and nonsymetric matrices 0 1 − 2 A = 1 3 0 (symmetric) − 2 0 5 4 3 B = (symmetric) 3 1 3 2 1 C = 1 − 4 0 (nonsymmetric) 1 0 5
42 Theorem 7.7: Eigenvalues of symmetric matrices If A is an n×n symmetric matrix, then the following properties are true. (1) A is diagonalizable (symmetric matrices are guaranteed to have n linearly independent eigenvectors and thus be diagonalizable) . (2) All eigenvalues of A are real numbers. (3) If λ is an eigenvalue of A with the multiplicity to be k, then λ has k linearly independent eigenvectors. That is, the eigenspace of λ has dimension k. The above theorem is called the Real Spectral Theorem , and the set of eigenvalues of A is called the spectrum of A.
43 Example 2: Prove that a 2 × 2 symmetric matrix is diagonalizable. a c A = c b proof: Characteristic equation:
λ − a − c λI − A = = λ2 − (a + b)λ + ab − c2 = 0 − c λ − b
As a function in λ, this quadratic polynomial function has a nonnegative discriminant as follows:
(a+− b )2 4(1)( ab −=++−+ c 22 ) a 2 ab b 2 4 ab 4 c 2 =a2 −2 ab ++ b 2 4 c 2 =(a − b )2 + 4 c 2 ≥ 0
44 (1) ( a − b)2 + 4c2 = 0
⇒ a = b, c = 0
a c a 0 A = = itself is a diagonal matrix. c b 0 a
( )2( ( a − b)2 + 4c2 > 0
The characteristic polynomial of A has two distinct real roots, which implies that A has two distinct real eigenvalues. According to Thm. 7.6, A is diagonalizable.
Orthogonal matrix: A square matrix P is called orthogonal if it is invertible and P−1 = PT(or PP T = PPI T = )
46 Orthogonal matrix
Theorem 7.8: Properties of orthogonal matrices An n×n matrix P is orthogonal if and only if its column vectors form an orthonormal set.
Proof : Suppose the column vectors of P form an orthonormal set, i.e.,
P=[p1 p 2 pn] , where pp ij ⋅=≠⋅= 0 for i j and pp ii 1.
T T T pp pp pp pppp1112⋅ ⋅ pp 1 ⋅ n 11 12 1 n ppppT T pp T pppp⋅ ⋅ pp ⋅ PPT =21 22 21 =2122 21 = I n T T T ppppn1 n 2 pp nn ppppn⋅1 n ⋅ 2 pp nn ⋅
It implies that P–1 = PT and thus P is orthogonal.
47 Orthogonal matrix
Ex 5: Show that P is an orthogonal matrix. 1 2 2 3 3 3 P = −2 1 0 5 5 −2 −4 5 3 5 3 5 3 5
Solution: If P is a orthogonal matrix, then P−1 = PT ⇒ PP T = I
1 2 2 1− 2 − 2 3 5 3 5 1 0 0 3 3 3 PPT =−210 214 − = 010 = I 553 535 −2 − 45 2 0 5 0 0 1 353535 3 35
48 Orthogonal matrix
1 2 2 3 3 3 Moreover, let p=−2 , p = 1 , and p = 0, 15 2 5 3 −2 − 4 5 35 35 3 5
we can p roduce pppppp121323⋅=⋅=⋅= 0 and pppppp 112233 ⋅=⋅=⋅= 1.
So, {p1 , p 2 , p 3 } is an orthonormal set. (Theorem 7.8 can be verified by this example.)
49 Orthogonal matrix Theorem 7.9: Properties of symmetric matrices
Let A be an n×n symmetric matrix. If λ1 and λ2 are distinct eigenvalues of A, then their corresponding eigenvectors x1 and x2 are orthogonal. (Theorem 7.6 only states that eigenvectors corresponding to distinct eigenvalues are linearly independent)
T T T Proof: λ λ 112x,x= 112 x,x =A x,x 12 =() A xx 12 = ( x 1 A ) x 2
because A is symmetric T T T =()xxx1A 2 === 1 () A x 2 x λ 122λ ()λ x x,x 122 = 212 x,x
The above equation implies (λ λ1− 2)x 12 ,x = 0, and because
λ λ12≠, it follows that x,x 12 = 0. So, x 12 and x are orthogon al. For distinct eigenvalues of a symmetric matrix, their corresponding eigenvectors are orthogonal and thus linearly independent to each other.
Note that there may be multiple x1 and x2 corresponding to λ1 and λ2 . 50 Orthogonal matrix
Orthogonal diagonalization: A matrix A is orthogonally diagonalizable if there exists an orthogonal matrix P such that P–1AP = D is diagonal .
Theorem 7.10: Fundamental theorem of symmetric matrices Let A be an n×n matrix. Then A is orthogonally diagonalizable and has real eigenvalues if and only if A is symmetric . Proof: (⇒ ) A is orthogonally diagonalizable ⇒ DPAP= −1 is diagonal, and P is an orthogonal matrix s.t. PP− 1 = T ⇒ A= PDP−1 = PDPT⇒ A T=( PDP TT )() = P TTTT D P == PDP T A (⇐ ) See the next two slides 51 Orthogonal matrix Orthogonal diagonalization of a symmetric matrix: Let A be an n×n symmetric matrix . (1) Find all eigenvalues of A and determine the multiplicity of each. According to Theorem 7.9, eigenvectors corresponding to distinct eigenvalues are orthogonal.
(2) For each eigenvalue of multiplicity 1, choose the unit eigenvector.
(3) For each eigenvalue of the multiplicity to be k ≥ 2, find a set of k linearly independent eigenvectors. If this set {v1, v2, …, vk} is not orthonormal, apply the Gram-Schmidt orthonormalization process.
52 Orthogonal matrix It is known that G.-S. process is a kind of linear transformation, i.e., the
produced vectors can be expressed as c11v+ c 22 v + + c k v k ,
i. Since A λvvv1 λ =λ 12, A = v 2 ,,… A vk = v k , ⇒ Acc(11v+ 22 v ++ ck v k ) = λ ( c 11v+c 22 v + + c k v k ) ⇒ The produced vectors through the G. -S. process are still eigenvectors for λ
ii. Since v1, v 2 , , v k are orthogonal to eigenvectors corresponding to other
different eigenvalues (according to Theorem 7.9 ), c11v+ c 22 v + + c k v k is also orthogonal to eigenvectors corresponding to other different eigenvalues. (4) The composite of steps (2) and (3) produces an orthonormal set of n eigenvectors. Use these orthonormal and thus linearly independent eigenvectors as column vectors to form the matrix P. i. According to Thm. 7.8, the matrix P is orthogonal ii. Following the diagonalization process, D = P–1AP is diagonal
Therefore, the matrix A is orthogonally diagonalizable
53 Orthogonal matrix
Ex 7: Determining whether a matrix is orthogonally diagonalizable Symmetric Orthogonally 1 1 1 matrix diagonalizable A = 1 0 1 1 1 1 1 5 2 1 A = 2 1 8 2 −1 8 0
3 2 0 A = 3 2 0 1 0 0 A = 4 0 − 2 54 Orthogonal matrix
Example 9: Orthogonal diagonalization Find an orthogonal matrix P that diagonaliz es A . 2 2− 2 A =2 − 1 4 − − Solution: 2 4 1 )1( λI − A = (λ − )3 2 (λ + )6 = 0
λλ 1= −6, 2 = 3 ( has a multiplicity of 2 )
v ⇒ 1 1− 2 2 (2) λ1=− 6, v 1 = (1, − 2, 2) u 1 = = (,3 3 , 3 ) v1
(3) λ = 3, v = (2, 1, 0), v =− ( 2, 4, 5) 2 2 3 Verify Theorem 7.9 that
v1·v2 = v1·v3 = 0 orthogonal 55 Orthogonal matrix
If v2 and v3 are not orthogonal, the Gram-Schmidt Process should be performed. Here we simply normalize v2 and v3 to find the corresponding unit vectors
v v u==2 (21 , , 0), u ==3 (− 24 , , 5 ) 2 553 353535 v2 v 3
1 2 −2 − 6 0 0 3 5 3 5 P = −2 1 4 ⇒ P−1 AP = 0 3 0 3 5 3 5 2 0 5 0 0 3 3 3 5
u1 u 2 u 3
Note that there are some calculation error in the solution of Example 9 in the text book 56 Applications of Eigenvalues and Eigenvectors
The rotation for quadratic equation: ax2+bxy+cy2+dx+ey+f = 0 Example 5: Identify the graphs of the following quadratic equations (a) 4x2+ 9 y 2 − 36 = 0 (b) 13 x2−10 xy + 13 y 2 −= 72 0
solution x2 y 2 ()a In standard form, we can obtain + = 1. 32 2 2
Since there is no xy-term, it is easy to derive the standard form and it is apparent that this equation represents an ellipse.
57 Applications of Eigenvalues and Eigenvectors (b) 13 x2−10 xy + 13 y 2 −= 72 0 Since there is a xy-term, it is difficult to identify the graph of this equation. In fact, it is also an ellipse, which is oblique on the xy-plane.
There is a easy way to identify the graph of quadratic equation. The basic idea is to rotate the x- and y-axes to x′- and y′-axes such that there is no more x′y′-term in the new quadratic equation.
In the above example, if we rotate the x- and y-axes by 45 degree counterclockwise, the new quadratic (x ')2 ( y ') 2 equation + = 1 can be derived, which 32 2 2 represents an ellipse apparently.
In Section 4.8, the rotation of conics is achieved by changing basis, but here the diagonalization technique based on eigenvalues and eignvectors is applied to solving the rotation problem.
58 Applications of Eigenvalues and Eigenvectors Quadratic form: ax2 + bxy+ cy2 is the quadratic form associated with the quadratic equation ax2 + bxy+ cy2 + dx + ey+ f = 0.
Matrix of the quadratic form:
a b / 2 Note that A is a symmetric matrix. A = b/ 2 c x If we define X = , then X T AX = ax 2 + bxy + cy 2 . In fact, the y quadratic equation can be expressed in terms of X as follows: X T AX + [ d e] X + f = ax 2 + bxy + cy 2+ + dx + ey+ f = 0
59 Applications of Eigenvalues and Eigenvectors Principal Axes Theorem For a conic whose equation is ax2 + bxy+ cy2 + dx + ey+ f = 0, the rotation to eliminate the xy-term is achieved by X = PX′, where P is an orthogonal matrix that diagonalizes A (matrix of the quadratic form). That is, − λ1 0 P1 AP= PT AP = D = 0 λ2 x ' X ' = y ' λ λ where 1 and 2 are eigenvalues of A. The equation for the rotated conic is given by
2 2 λλ 1(x ')+ 2 ( y ') +[ dePXf] ′ += 0. 60 Applications of Eigenvalues and Eigenvectors
Proof: According to Theorem 7.10, since A is symmetric, we can conclude that there exists an orthogonal matrix P such that P–1AP = PTAP = D is diagonal. Replacing X with PX′ , the quadratic form becomes XT AX=( PX′ )( T A PX ′′ )() = X TT P APX ′ T 2 2 = (X′′ ) DX = λλ 1 () x ′ + 2 ( y ′ ). It is obvious that the new quadratic form in terms of X ′ has no x′y′-term, and the coefficients for (x′)2 and (y′)2 are the two eigenvalues of the matrix A. xx ′ xx ′ XPX= ′⇒ =[]vv = xy ′ vv + ′ ⇒ Since and are yy12 ′ 12 yy ′
the orignal and new coodinates, the role s of v1 and v 2 (the eigenvectors of A ) are like the basis vectors (or the axis vectors) in the new coordinate system.
61 Applications of Eigenvalues and Eigenvectors Example 6: Rotation of a conic Perform a rotation of axes to eliminate the xy-term in the following quadratic equation 13x2− 10 xy + 13 y 2 −= 72 0
Solution:
The matrix of the quadratic form associated with this equation is
13− 5 A = −5 13 λ λ The eigenvalues are 1 = 8 and 2 = 18, and the corresponding eigenvectors are
1 − 1 x= and x = 11 2 1
62 Applications of Eigenvalues and Eigenvectors After normalizing each eigenvector, we can obtain the orthogonal matrix P as follows.
1− 1 According to the results on p. 268 in 2 2 cos 45− sin 45 Ch4, X=PX′ is equivalent to rotate P = = 1 1 sin 45 cos 45 the xy-coordinates by 45 degree to form the new x′y′-coordinates . 2 2
Then by replacing X with PX’ , the equation of the rotated conic is
8(x′ )+ 18( y ′ )2 − 72 = 0,
which can be written in the standard form
()x′2 () y ′ 2 + = 1. 32 2 2 The above equation represents an ellipse on the x′y′-plane.
63 Applications of Eigenvalues and Eigenvectors In three-dimensional version: ax2 + by2 + cz2 + dxy+ exz+ fyz is the quadratic form associated with the equation of quadric surface: ax2 + by2 + cz2 + dxy+ exz+ fyz+ gx+ hy+ iz+ j = 0. Matrix of the quadratic form: a d/2 e /2 Ad= /2 b f /2 Note that A is a symmetric matrix. e/2 f /2 c If we define X = [ x y z]T, then XTAX= ax2 + by2 + cz2 + dxy+ exz+ fyz and the quadratic surface equation can be expressed as XAXT +[ ghiX ] + j = 0
64 Unitary and Hermitian Matrices Conjugate Transpose of a Complex Matrix The conjugate transpose of a complex matrix A denoted by A*, is given by: A* = Ā T where the entries of Ā are the complex conjugates of the corresponding entries of A.
Example: Determine A* for the matrix
Solution:
65 Unitary and Hermitian Matrices
Theorem 8.8: Properties of the Conjugate Transpose If A and B are complex matrices and k is a complex number, then the following properties are true. 1. (A*)* = A 2. (A* + B*)* = A + B Ŭ 3. (kA*)* = ͟A 4. (AB) *= B*A*
66 Unitary and Hermitian Matrices
Unitary Matrix: A complex matrix is unitary when A-1 = A*
Example 2: Show that the matrix A is unitary.
Solution:
67 Unitary and Hermitian Matrices
Theorem 8.9: Unitary Matrices An n×n complex matrix A is unitary if and only if its row (or column) vectors form an orthonormal set in C n.
68 Unitary and Hermitian Matrices
Hermitian Matrices: A square matrix A is Hermitian when A = A*
Example 3: A is Hermitian,
69 Unitary and Hermitian Matrices Theorem 8.10: The Eigenvalues of a Hermitian Matrix If A is a Hermitian matrix, then its eigenvalues are real numbers.
Proof:
70 Unitary and Hermitian Matrices
Example 4: Find the eigenvalues of the matrix A.
Solution:
⇒ λ1= -1, λ1= -2, λ1=6
71 Unitary and Hermitian Matrices
Theorem 8.11: Hermitian Matrices and Diagonalization If A is an n×n Hermitian matrix, then 1. eigenvectors corresponding to distinct eigenvalues are orthogonal. 2. A is unitarily diagonalizable.
proof:
72 Unitary and Hermitian Matrices
Example 5: Find a unitary matrix P such that P*AP is a diagonal matrix where
Solution: Eigenvalues of A are calculated in example 3. The normalized eigenvectors of A are:
73 Symmetric and Hermitian Matrices
A is a symmetric matrix A is a Hermitian matrix (real) (complex)
Eigenvalues of are real. Eigenvalues of are real. Eigenvectors corresponding to Eigenvectors corresponding to distinct eigenvalues are distinct eigenvalues are orthogonal. orthogonal. There exists an orthogonal There exists an unitary matrix matrix such that such that PTAP P*AP is diagonal. is diagonal.
74