sign in sign up
<<
Home , Diagonalizable matrix, Diagonal matrix, Orthogonalization, Orthogonal basis

W W L CHEN

c W W L Chen, 1997, 2008. This chapter is available free to all individuals, on the understanding that it is not to be used for financial gain, and may be downloaded and/or photocopied, with or without permission from the author. However, this document may not be kept on any information storage and retrieval system without permission from the author, unless such system is not accessible to any individuals other than its owners.

Chapter 10

ORTHOGONAL MATRICES

10.1. Introduction

Definition. A square A with real entries and satisfying the condition A−1 = At is called an .

Example 10.1.1. Consider the R2 with the euclidean inner product. The vectors u1 =(1, 0) and u2 =(0, 1) form an orthonormal B = {u1, u2}. Let us now rotate u1 and u2 anticlockwise by an angle θ to obtain v1 = (cos θ, sin θ) and v2 =(− sin θ, cos θ). Then C = {v1, v2} is also an .

u2

v2

v1

θ u1

Chapter 10 : Orthogonal Matrices page 1 of 11 <02> Linear Algebra c W W L Chen, 1997, 2008

The transition matrix from the basis C to the basis B is given by cos θ − sin θ P =([v ]B [v ]B )= . 1 2 sin θ cos θ

Clearly cos θ sin θ P −1 = P t = . − sin θ cos θ

In fact, our example is a special case of the following general result.

PROPOSITION 10A. Suppose that B = {u1,...,un} and C = {v1,...,vn} are two orthonormal bases of a real V . Then the transition matrix P from the basis C to the basis B is an orthogonal matrix.

Example 10.1.2. The matrix ⎛ ⎞ 1/3 −2/32/3 A = ⎝ 2/3 −1/3 −2/3 ⎠ 2/32/31/3 is orthogonal, since ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 1/32/32/3 1/3 −2/32/3 100 AtA = ⎝ −2/3 −1/32/3 ⎠ ⎝ 2/3 −1/3 −2/3 ⎠ = ⎝ 010⎠ . 2/3 −2/31/3 2/32/31/3 001

Note also that the row vectors of A, namely (1/3, −2/3, 2/3), (2/3, −1/3, −2/3) and (2/3, 2/3, 1/3) are orthonormal. So are the column vectors of A.

In fact, our last observation is not a coincidence.

PROPOSITION 10B. Suppose that A is an n × n matrix with real entries. Then (a) A is orthogonal if and only if the row vectors of A form an orthonormal basis of Rn under the euclidean inner product; and (b) A is orthogonal if and only if the column vectors of A form an orthonormal basis of Rn under the euclidean inner product.

Proof. We shall only prove (a), since the proof of (b) is almost identical. Let r1,...,rn denote the row vectors of A. Then ⎛ ⎞ r1 · r1 ... r1 · rn t ⎝ . . ⎠ AA = . . . rn · r1 ... rn · rn It follows that AAt = I if and only if for every i, j =1,...,n,wehave  1ifi = j, ri · rj = 0ifi = j, if and only if r1,...,rn are orthonormal.

PROPOSITION 10C. Suppose that A is an n × n matrix with real entries. Suppose further that the inner product in Rn is the euclidean inner product. Then the following are equivalent: (a) A is orthogonal. (b) For every x ∈ Rn, we have Ax = x. (c) For every u, v ∈ Rn, we have Au · Av = u · v.

Chapter 10 : Orthogonal Matrices page 2 of 11 <03> Linear Algebra c W W L Chen, 1997, 2008

Proof. ((a)⇒(b)) Suppose that A is orthogonal, so that AtA = I. It follows that for every x ∈ Rn,we have

Ax2 = Ax · Ax = xtAtAx = xtIx = xtx = x · x = x2.

((b)⇒(c)) Suppose that Ax = x for every x ∈ Rn. Then for every u, v ∈ Rn,wehave

A ·A 1 A A 2 − 1 A −A 2 1 A 2 − 1 A − 2 1  2 − 1  − 2 · . u v = 4 u+ v 4 u v = 4 (u+v) 4 (u v) = 4 u+v 4 u v = u v

((c)⇒(a)) Suppose that Au · Av = u · v for every u, v ∈ Rn. Then

Iu · v = u · v = Au · Av = vtAtAu = AtAu · v, so that

(AtA − I)u · v =0.

In particular, this holds when v =(AtA − I)u, so that

(AtA − I)u · (AtA − I)u =0, whence

(AtA − I)u = 0, (1) in view of Proposition 9A(d). But then (1) is a system of n homogeneous linear equations in n unknowns satisfied by every u ∈ Rn. Hence the coefficient matrix AtA−I must be the , and so AtA = I.

Proof of Proposition 10A. For every u ∈ V , we can write

u = β1u1 + ...+ βnun = γ1v1 + ...+ γnvn, where β1,...,βn,γ1,...,γn ∈ R, and where B = {u1,...,un} and C = {v1,...,vn} are two orthonormal bases of V . Then

n n n 2 2 u = u, u = β1u1 + ...+ βnun,β1u1 + ...+ βnun = βiβjui, uj = βi i=1 j=1 i=1

=(β1,...,βn) · (β1,...,βn).

Similarly,

n n n 2 2 u = u, u = γ1v1 + ...+ γnvn,γ1v1 + ...+ γnvn = γiγjvi, vj = γi i=1 j=1 i=1

=(γ1,...,γn) · (γ1,...,γn).

n It follows that in R with the euclidean , we have [u]B = [u]C, and so P [u]C = [u]C for every u ∈ V . Hence P x = x holds for every x ∈ Rn. It now follows from Proposition 10C that P is orthogonal.

Chapter 10 : Orthogonal Matrices page 3 of 11 <04> Linear Algebra c W W L Chen, 1997, 2008

10.2. Eigenvalues and Eigenvectors

In this section, we give a brief review on eigenvalues and eigenvectors first discussed in Chapter 7.

Suppose that ⎛ ⎞ a11 ... a1n ⎝ . . ⎠ A = . . an1 ... ann is an n × n matrix with real entries. Suppose further that there exist a number λ ∈ R and a non-zero vector v ∈ Rn such that Av = λv. Then we say that λ is an eigenvalue of the matrix A, and that v is an eigenvector corresponding to the eigenvalue λ. In this case, we have Av = λv = λIv, where I is the n × n , so that (A − λI)v = 0. Since v ∈ Rn is non-zero, it follows that we must have

det(A − λI)=0. (2)

In other words, we must have ⎛ ⎞ a11 − λa12 ... a1n ⎜ a a − λa⎟ ⎜ 21 22 2n ⎟ det ⎝ . . . ⎠ =0. . .. . an1 an2 ... ann − λ

Note that (2) is a polynomial equation. The polynomial det(A−λI) is called the characteristic polynomial of the matrix A. Solving this equation (2) gives the eigenvalues of the matrix A.

On the other hand, for any eigenvalue λ of the matrix A, the set

{v ∈ Rn :(A − λI)v = 0} (3) is the nullspace of the matrix A−λI, and forms a subspace of Rn. This space (3) is called the eigenspace corresponding to the eigenvalue λ.

Suppose now that A has eigenvalues λ1,...,λn ∈ R, not necessarily distinct, with corresponding n eigenvectors v1,...,vn ∈ R , and that v1,...,vn are linearly independent. Then it can be shown that

P −1AP = D, where ⎛ ⎞ λ1 ⎝ . ⎠ P =(v1 ... vn ) and D = .. . λn

In fact, we say that A is diagonalizable if there exists an P with real entries such that P −1AP is a with real entries. It follows that A is diagonalizable if its eigenvectors form a basis of Rn. In the opposite direction, one can show that if A is diagonalizable, then it has n linearly independent eigenvectors in Rn. It therefore follows that the question of diagonalizing a matrix A with real entries is reduced to one of of its eigenvectors.

We now summarize our discussion so far.

Chapter 10 : Orthogonal Matrices page 4 of 11 <05> Linear Algebra c W W L Chen, 1997, 2008

DIAGONALIZATION PROCESS. Suppose that A is an n × n matrix with real entries. (1) Determine whether the n roots of the characteristic polynomial det(A − λI) are real. (2) If not, then A is not diagonalizable. If so, then find the eigenvectors corresponding to these eigen- values. Determine whether we can find n linearly independent eigenvectors. (3) If not, then A is not diagonalizable. If so, then write ⎛ ⎞ λ1 ⎝ . ⎠ P =(v1 ... vn ) and D = .. , λn

n where λ1,...,λn ∈ R are the eigenvalues of A and where v1,...,vn ∈ R are respectively their corresponding eigenvectors. Then P −1AP = D.

In particular, it can be shown that if A has distinct eigenvalues λ1,...,λn ∈ R, with corresponding n eigenvectors v1,...,vn ∈ R , then v1,...,vn are linearly independent. It follows that all such matrices A are diagonalizable.

10.3. Orthonormal Diagonalization

We now consider the euclidean space Rn an as inner product space with the euclidean inner product. Given any n × n matrix A with real entries, we wish to find out whether there exists an orthonormal basis of Rn consisting of eigenvectors of A.

Recall that in the Diagonalization process discussed in the last section, the columns of the matrix P are eigenvectors of A, and these vectors form a basis of Rn. It follows from Proposition 10B that this basis is orthonormal if and only if the matrix P is orthogonal.

Definition. An n × n matrix A with real entries is said to be orthogonally diagonalizable if there exists an orthogonal matrix P with real entries such that P −1AP = P tAP is a diagonal matrix with real entries.

First of all, we would like to determine which matrices are orthogonally diagonalizable. For those that are, we then need to discuss how we may find an orthogonal matrix P to carry out the diagonalization.

To study the first question, we have the following result which gives a restriction on those matrices that are orthogonally diagonalizable.

PROPOSITION 10D. Suppose that A is a orthogonally with real entries. Then A is symmetric.

Proof. Suppose that A is orthogonally diagonalizable. Then there exists an orthogonal matrix P and a diagonal matrix D, both with real entries and such that P tAP = D. Since PPt = P tP = I and Dt = D,wehave

A = PDPt = PDtP t, so that

At =(PDtP t)t =(P t)t(Dt)tP t = PDPt = A, whence A is symmetric.

Our first question is in fact answered by the following result which we state without proof.

Chapter 10 : Orthogonal Matrices page 5 of 11 <06> Linear Algebra c W W L Chen, 1997, 2008

PROPOSITION 10E. Suppose that A is an n × n matrix with real entries. Then it is orthogonally diagonalizable if and only if it is symmetric.

The remainder of this section is devoted to finding a way to orthogonally diagonalize a with real entries. We begin by stating without proof the following result. The proof requires results from the theory of complex vector spaces.

PROPOSITION 10F. Suppose that A is a symmetric matrix with real entries. Then all the eigenvalues of A are real.

Our idea here is to follow the Diagonalization process discussed in the last section, knowing that since A is diagonalizable, we shall find a basis of Rn consisting of eigenvectors of A. We may then wish to orthogonalize this basis by the Gram-Schmidt process. This last step is considerably simplified in view of the following result.

PROPOSITION 10G. Suppose that u1 and u2 are eigenvectors of a symmetric matrix A with real entries, corresponding to distinct eigenvalues λ1 and λ2 respectively. Then u1 · u2 =0. In other words, eigenvectors of a symmetric real matrix corresponding to distinct eigenvalues are orthogonal.

Proof. Note that if we write u1 and u2 as column matrices, then since A is symmetric, we have

A · t A t At A t · A . u1 u2 = u2 u1 = u2 u1 =( u2) u1 = u1 u2

It follows that

λ1u1 · u2 = Au1 · u2 = u1 · Au2 = u1 · λ2u2, so that (λ1 − λ2)(u1 · u2) = 0. Since λ1 = λ2, we must have u1 · u2 =0.

We can now follow the procedure below.

ORTHOGONAL DIAGONALIZATION PROCESS. Suppose that A is a symmetric n×n matrix with real entries. (1) Determine the n real roots λ1,...,λn of the characteristic polynomial det(A−λI), and find n linearly independent eigenvectors u1,...,un of A corresponding to these eigenvalues as in the Diagonaliza- tion process. (2) Apply the Gram-Schmidt process to the eigenvectors u1,...,un to obtain orthogo- nal eigenvectors v1,...,vn of A, noting that eigenvectors corresponding to distinct eigenvalues are already orthogonal. (3) Normalize the orthogonal eigenvectors v1,...,vn to obtain orthonormal eigenvectors w1,...,wn of A. These form an orthonormal basis of Rn. Furthermore, write ⎛ ⎞ λ1 ⎝ . ⎠ P =(w1 ... wn ) and D = .. , λn

n where λ1,...,λn ∈ R are the eigenvalues of A and where w1,...,wn ∈ R are respectively their orthogonalized and normalized eigenvectors. Then P tAP = D.

Remark. Note that if we apply the Gram-Schmidt orthogonalization process to eigenvectors corre- sponding to the same eigenvalue, then the new vectors that result from this process are also eigenvectors corresponding to this eigenvalue. Why?

Chapter 10 : Orthogonal Matrices page 6 of 11 <07> Linear Algebra c W W L Chen, 1997, 2008

Example 10.3.1. Consider the matrix ⎛ ⎞ 221 A = ⎝ 252⎠ . 122

To find the eigenvalues of A, we need to find the roots of ⎛ ⎞ 2 − λ 21 det ⎝ 25− λ 2 ⎠ =0; 122− λ

2 in other words, (λ − 7)(λ − 1) = 0. The eigenvalues are therefore λ1 = 7 and (double root) λ2 = λ3 =1. An eigenvector corresponding to λ1 = 7 is a solution of the system ⎛ ⎞ ⎛ ⎞ −52 1 1 ⎝ ⎠ ⎝ ⎠ (A − 7I)u = 2 −22 u = 0, with root u1 = 2 . 12−5 1

Eigenvectors corresponding to λ2 = λ3 = 1 are solutions of the system ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 121 1 2 ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ (A − I)u = 242 u = 0, with roots u2 = 0 and u3 = −1 121 −1 0 which are linearly independent. Next, we apply the Gram-Schmidt orthogonalization process to u2 and u3, and obtain ⎛ ⎞ ⎛ ⎞ 1 1 ⎝ ⎠ ⎝ ⎠ v2 = 0 and v3 = −1 −1 1 which are now orthogonal to each other. Note that we do not have to do anything to u1 at this stage, in view of Proposition 10G. We now conclude that ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 1 1 1 ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ v1 = 2 , v2 = 0 , v3 = −1 1 −1 1 form an of R3. Normalizing each of these, we obtain respectively ⎛ √ ⎞ ⎛ √ ⎞ ⎛ √ ⎞ 1/√6 1/ 2 1/ √3 ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ w1 = 2/√6 , w2 = 0√ , w3 = −1/√ 3 . 1/ 6 −1/ 2 1/ 3

We now take ⎛ √ √ √ ⎞ 1/√61/ 21/ √3 ⎝ ⎠ P =(w1 w2 w3 )= 2/√60√ −1/√ 3 . 1/ 6 −1/ 21/ 3

Then ⎛ √ √ √ ⎞ ⎛ ⎞ 1/√62/ 61/ √6 700 −1 t ⎝ ⎠ t ⎝ ⎠ P = P = 1/√20√ −1/√ 2 and P AP = 010 . 1/ 3 −1/ 31/ 3 001

Chapter 10 : Orthogonal Matrices page 7 of 11 <08> Linear Algebra c W W L Chen, 1997, 2008

Example 10.3.2. Consider the matrix ⎛ ⎞ −16−12 A = ⎝ 0 −13 30 ⎠ . 0 −920

To find the eigenvalues of A, we need to find the roots of ⎛ ⎞ −1 − λ 6 −12 det ⎝ 0 −13 − λ 30 ⎠ =0; 0 −920− λ in other words, (λ + 1)(λ − 2)(λ − 5) = 0. The eigenvalues are therefore λ1 = −1, λ2 = 2 and λ3 =5. An eigenvector corresponding λ1 = −1 is a solution of the system ⎛ ⎞ ⎛ ⎞ 06−12 1 ⎝ ⎠ ⎝ ⎠ (A + I)u = 0 −12 30 u = 0, with root u1 = 0 . 0 −921 0

An eigenvector corresponding to λ2 = 2 is a solution of the system ⎛ ⎞ ⎛ ⎞ −36−12 0 ⎝ ⎠ ⎝ ⎠ (A − 2I)u = 0 −15 30 u = 0, with root u2 = 2 . 0 −918 1

An eigenvector corresponding to λ3 = 5 is a solution of the system ⎛ ⎞ ⎛ ⎞ −66−12 1 ⎝ ⎠ ⎝ ⎠ (A − 5I)u = 0 −18 30 u = 0, with root u3 = −5 . 0 −915 −3

Note that while u1, u2, u3 correspond to distinct eigenvalues of A, they are not orthogonal. The matrix A is not symmetric, and so Proposition 10G does not apply in this case.

Example 10.3.3. Consider the matrix ⎛ ⎞ 5 −20 A = ⎝ −262⎠ . 027

To find the eigenvalues of A, we need to find the roots of ⎛ ⎞ 5 − λ −20 det ⎝ −26− λ 2 ⎠ =0; 027− λ in other words, (λ − 3)(λ − 6)(λ − 9) = 0. The eigenvalues are therefore λ1 =3,λ2 = 6 and λ3 =9.An eigenvector corresponding λ1 = 3 is a solution of the system ⎛ ⎞ ⎛ ⎞ 2 −20 2 ⎝ ⎠ ⎝ ⎠ (A − 3I)u = −232u = 0, with root u1 = 2 . 024 −1

An eigenvector corresponding to λ2 = 6 is a solution of the system ⎛ ⎞ ⎛ ⎞ −1 −20 2 ⎝ ⎠ ⎝ ⎠ (A − 6I)u = −202u = 0, with root u2 = −1 . 021 2

Chapter 10 : Orthogonal Matrices page 8 of 11 <09> Linear Algebra c W W L Chen, 1997, 2008

An eigenvector corresponding to λ3 = 9 is a solution of the system ⎛ ⎞ ⎛ ⎞ −4 −20 −1 ⎝ ⎠ ⎝ ⎠ (A − 9I)u = −2 −32 u = 0, with root u3 = 2 . 02−2 2

Note now that the eigenvalues are distinct, so it follows from Proposition 10G that u1, u2, u3 are or- thogonal, so we do not have to apply Step (2) of the Orthogonal diagonalization process. Normalizing each of these vectors, we obtain respectively ⎛ ⎞ ⎛ ⎞ ⎛ ⎞ 2/3 2/3 −1/3 ⎝ ⎠ ⎝ ⎠ ⎝ ⎠ w1 = 2/3 , w2 = −1/3 , w3 = 2/3 . −1/3 2/3 2/3

We now take ⎛ ⎞ 2/32/3 −1/3 ⎝ ⎠ P =(w1 w2 w3 )= 2/3 −1/32/3 . −1/32/32/3

Then ⎛ ⎞ ⎛ ⎞ 2/32/3 −1/3 300 P −1 = P t = ⎝ 2/3 −1/32/3 ⎠ and P tAP = ⎝ 060⎠ . −1/32/32/3 009

Chapter 10 : Orthogonal Matrices page 9 of 11 <10> Linear Algebra c W W L Chen, 1997, 2008

Problems for Chapter 10

1. Prove Proposition 10B(b).

2. Let a + bb− a A = , a − bb+ a

where a, b ∈ R. Determine when A is orthogonal.

3. Suppose that A is an orthogonal matrix with real entries. Prove that a) A−1 is an orthogonal matrix; and b) det A = ±1.

4. Suppose that A and B are orthogonal matrices with real entries. Prove that AB is orthogonal.

5. Verify that for every a ∈ R, the matrix ⎛ ⎞ 1 −2a 2a2 1 A = ⎝ 2a 1 − 2a2 −2a ⎠ 1+2a2 2a2 2a 1

is orthogonal.

6. Suppose that λ is an eigenvalue of an orthogonal matrix A with real entries. Prove that 1/λ is also an eigenvalue of A.

7. Suppose that ab A = cd

is an orthogonal matrix with real entries. Explain why a2 + b2 = c2 + d2 = 1 and ac + bd = 0, and quote clearly any result that you use. Deduce that A has one of the two possible forms cos θ − sin θ cos θ − sin θ A = or A = , sin θ cos θ − sin θ − cos θ

where θ ∈ [0, 2π).

8. Consider the matrix ⎛ √ √ ⎞ √1 − 6 √3 ⎝ ⎠ A = −√ 62√ 2 . 3 23

a) Find the characteristic polynomial of A and show that A has eigenvalues 4 (twice) and −2. b) Find an eigenvector of A corresponding to the eigenvalue −2. c) Find two orthogonal eigenvectors of A corresponding to the eigenvalue 4. d) Find an orthonormal basis of R3 consisting of eigenvectors of A. e) Using the orthonormal basis in part (d), find a matrix P such that P tAP is a diagonal matrix.

Chapter 10 : Orthogonal Matrices page 10 of 11 <11> Linear Algebra c W W L Chen, 1997, 2008

9. Apply the⎛ Orthogonal⎞ diagonalization process to each of the⎛ following⎞ matrices: 50 6 020 a) A = ⎝ 0116⎠ b) A = ⎝ 201⎠ 66−2 010 ⎛ ⎞ ⎛ ⎞ 1 −42 2036 c) A = ⎝ −41−2 ⎠ d) A = ⎝ 030⎠ 2 −2 −2 36023 ⎛ ⎞ ⎛ ⎞ 1100 −7240 0 ⎜ 1100⎟ ⎜ 24700⎟ e) A = ⎝ ⎠ f) A = ⎝ ⎠ 0000 00−724 0000 0 0 24 7

10. Suppose that B is an m × n matrix with real entries. Prove that the matrix A = BtB has an orthonormal set of n eigenvectors.

Chapter 10 : Orthogonal Matrices page 11 of 11