sign in sign up
<<
Home , Orthogonal matrix

9.1 and Eigenvalues § We are living in the spaces Rn or Cn for this chapter. We assume all vectors are column vectors. Def 9.1: Let v(1), . . . , v(k) be a set of vectors. The set is linearly independent if { } (1) (k) 0 = α1v + + αkv , ··· then α1 = = αk = 0. Otherwise,··· v(1), . . . , v(k) is linearly dependent. You should{ make the connection} with the similar definition for functions in section 8.2. Theorem: Let v(1), . . . , v(n) be the set of n linearly independent vectors in Rn. Then the { } V v(1) . . . v(n) = 0. | | ≡ | | 6 Proof: By using method, we can rewrite V as V = LU, where L is nonsingular with nonzero determinant and U is an upper-triangular . If the determinant of U t is zero, i.e., at least one of its diagonal element is zero, then there is a nonzero vector α = (α1, . . . , αn) such that Uα = 0

(Set αi = 1 if no pivoting element in the i-th column. Then solve the system for other αk’s.) and thus V α = LUα = 0.

This implies the set of vk’s is linear dependent. Contradiction. Theorem 9.2: Let v(1), . . . , v(n) be the set of n linearly independent vectors in Rn. Then for n { } any vector x R , there are unique values α1, . . . , αn R such that ∈ ∈ (1) (n) x = α1v + + αnv . ··· Proof: This is an n n linear system with unknowns αk’s. Its coefficient matrix is nonsingular by the previous theorem.× Thus, this system has a unique solution. Ex: You can easily see that (1, 0, 0)t, ( 1, 1, 1)t, (0, 4, 2)t are linearly independent. − Theorem 9.3: If A is a matrix and λ1, . . . , λk are distinct eigenvalues of A with associated eigenvectors x(1), . . . , x(k), then x(1), . . . , x(k) is linearly independent. Proof: (By mathematical induction){ When} k = 1, a nonzero vector x(1) is linearly independent. Suppose that x(1), . . . , x(k) is linearly independent. We need to prove that x(1), . . . , x(k), x(k+1) { } { } is linearly independent. If there are constants a1, . . . , αk, αk+1 such that

(1) (k) (k+1) α1x + + αkx + αk+1x = 0. (1) ··· Then (1) (k) (k+1) A(α1x + + αkx + αk+1x ) = 0, ··· and (1) (k) (k+1) α1λ1x + + αkλkx + αk+1λk+1x = 0. ( ) ··· ∗ Note also that (1) (k) (k+1) λk+1(α1x + + αkx + αk+1x ) = 0, ··· 1 that is, (1) (k) (k+1) α1λk+1x + + αkλk+1x + αk+1λk+1x = 0. ( ) ··· ∗∗ (*)-(**) gives (1) (k) α1(λ1 λk+1)x + + αk(λk λk+1)x = 0. − ··· − But the first k vectors are linearly independent by the induction assumption, the coefficients in the above equation must be 0. Since the λk’s are distinct, this leads to

α1 = = αk = 0. ··· (1) (k+1) Then from (1), αk+1 = 0. Hence, x , . . . , x is linearly independent. By mathematical induction, the theorem is true. { } Def. 9.4: A set of vectors v(1), . . . , v(k) is called orthogonal if (v(i))tv(j) = 0 for all i = j (i.e., their inner product is < v(i{), v(j) >= 0.).} If, in addition, (v(i))tv(i) = 1 for all i, then the set6 is orthonormal. If v(1), . . . , v(k) is orthogonal, then v(1)/ v(1) , . . . , v(k)/ v(k) is orthonormal. Theorem{ 9.5:} An orthogonal set of{ vectorsk thatk doesn’tk containk} the zero vector is linearly independent. (We had similar results for orthogonal polynomials.) Proof: Use the similar technique to #14 on page 496, but replace integration with inner product. If there are constants α1, . . . , αn such that

(1) (n) α1v + + αnv = 0, ··· then for any 1 i n, ≤ ≤ (i) t (1) (n) 0 = (v ) (α1v + + αnv ) ··· (i) t (1) (i) t (n) (i) t (i) α1(v ) (v ) + + αn(v ) (v ) = αi(v ) (v ). ··· (i) t (i) Thus αi = 0 since (v ) (v ) = 0, 1 i n. 6 ≤ ≤ 1 t Def. 9.6: A matrix P is said to be an orthogonal matrix if P − = P . (so, P must be a .) Ex. If u(1), . . . , u(n) Rn is orthonormal and all u(i)’s are nonzero, then the matrix { } ⊂ P = [u(1) u(n)] ··· is orthogonal. (you can easily verify P tP = I.) Def. 9.7: Two matrices A and B are said to be similar if a nonsingular matrix S exists with 1 A = S− BS. 1 Theorem 9.8: If A and B are similar with A = S− BS and λ is an eigenvalue of A with associated eigenvector x, then λ is also an eigenvalue of B with associated eigenvector Sx. 1 Proof: S− BSx = Ax = λx. Then BSx = λSx. Since x = 0 and S is nonsingular, Sx is not zero. The result follows. 6 Ex. If A is a diagonal, upper-triangular or lower-triangular, then

0 = det(A λI) = (λ a11) (λ ann). − − ··· − Thus the diagonal elements of A are the eigenvalues of A.

2 Theorem 9.9: Let A be a square matrix. There is a nonsingular matrix U such that

1 T = U − AU, where T is an upper-. The diagonal elements of T are the eigenvalues of A. Remark: Find the matrix U is difficult. Theorem 9.10: If A is and D is a whose diagonal entries are 1 t the eigenvalues of A, then there exists an orthogonal matrix such that D = P − AP = P AP . Remark: It is easy to check that the columns of P are the eigenvectors of A (why?). But P is also hard to be found. Also, the eigenvalues of the symmetric matrix are all real. Theorem 9.12: A symmetric matrix A is positive definite if and only if all the eigenvalues of A are positives. Proof: (= ) If A is positive definite and λ is an eigenvalue of A with associated eigenvector x, then ⇒ xtAx > 0, xt(λx) > 0, λ(xtx) > 0, λ > 0. ( =) Let P = [x(1) . . . x(n)] be the matrix in Theorem 9.10, P tAP = D. Then the columns of P are⇐ the n eigenvectors of A. Since they are orthogonal, they are linearly independent. So, for any n t 0 = x R , there are constants 0 = β = (β1, . . . , βn) such that 6 ∈ 6 (1) (n) x = β1x + + βnx = P β. ··· If all eigenvalues of A are positive, then

xtAx = (P β)tA(P β) = βt(P tAP )β = βtDβ

2 2 = d11β + + dnnβ > 0 1 ··· n since d11, . . . , dnn are the eigenvalues of A which are positive. Hence, A is positive definite. Theorem 9.13: (Gerschgorin Circle Theorem) Let A be an n n matrix and Ri denote the n × circle in the complex plane with center aii and radius j=1,j=i aij , that is, P 6 | | n R = z C z a a , i ii X ij { ∈ | | − | ≤ j=1,j=i | |} 6 n where C is the complex plane. The eigenvalues of A are contained within R = i=1Ri. Moreover, the union of any k of these circles that do not intersect the remaining (n k) contains∪ precisely k (counting multiplicities) of the eigenvalues. − Proof: Let λ be an eigenvalue of A with associated eigenvector x satisfying x = 1. Then Ax = λx, that is, k k∞ n a x = λx = λx , i = 1, . . . , n. X ij j j i j=1

Let k be the integer with xk = 1, then | | n a x = λx = λx . X kj j j k j=1

3 And n a x = λx = (λ a )x . X kj j j kk k j=1,j=k − 6 Thus n n λ a = λ a x a x a . kk kk k X kj j X kj | − | | − || | ≤ j=1,j=k | | · | | ≤ j=1,j=k | | 6 6 Then λ Rk. This proves the first part of the theorem. The proof of the second part is omitted. ∈ Ex. If A = (aij)n n is strictly diagonally dominant, then the spetral radius of the iteration × matrix Tj of Jacobi method applied to Ax = b is less than 1. Sol. Note that for the matrix Tj of the Jacobi method, the radius of the circle Ri in the Gerschgorin theorem is n n   aij/aii = aij / aii < 1 X | | X | | | | j=1,j=i j=1,j=i  6 6 since A is strictly diagonally dominant. Note also that the diagonal elements of Tj are all zero. Thus, all the circles Ri are centered at 0 with radius less than 1. By Gerschgorin theorem, the absolute values of the eigenvalues of Tj are all less than 1. Thus, ρ(Tj) < 1.

4