Chapter 8

Eigenvalue Problems

8.1 Introduction

n n The focus of this chapter will be the eigenvalue problem: given A C ⇥ we 2 will be interested in finding pairs (, x) C Cn such that 2 ⇥ ? Ax = x.

Recall that, since (A I)x = 0,wemusthavethatdet(A I) = 0. Define the characteristic polynomial of the matrix A by

(t)=det(A tI). A This shows that (A) i↵ () = 0. To find the eigenvalues of a matrix, 2 A it is enough to find the roots of the characteristic polynomial. However, this is never done directly – that is, finding roots of the characteristic polynomial – because it is too expensive and very sensitive to perturbations on the entries of A. The following example shows the latter point:

Example 8.1.1 (instability of determinant). Suppose that

1 20 2 20 2 . . 3 A = .. .. . 6 7 6 9 207 6 7 6 107 6 7 4 5 243 244 CHAPTER 8. EIGENVALUE PROBLEMS

It follows that (A)= 1, , 10 .Now,for0 <" 1, consider also { ··· } ⌧ 1 20 2 20 2 . . 3 A" = .. .. . 6 7 6 9 207 6 7 6" 107 6 7 4 5 In any reasonable norm, A will be close to A . However, we can choose " k "k k k so that 0 (A ). To see this, consider 2 " 20 2 20 . . 2 20 .. .. 0 . . 1 det(A")=det0 1 " det .. .. 9 20 B C B C B 9 20 C B 10C B C B C B 10 20C @ A B C = 10! 209". @ A Therefore, if 6 10! 3 10 6 " = ⇥ 7 10 . 209 ⇡ 5 1011 ⇡ ⇥ ⇥ the matrix A" has zero as an eigenvalue. For a linear system Ax = b, or a even least-squares problem, we showed that there are always direct methods for finding the solution vector x. These may be quite expensive, requiring, say, (n3) operations; but, nonetheless, O direct methods were available. There is another important issue associated with eigenvalue problems. Direct methods are not typically available for find- ing eigen-pairs. This fundamental diculty can be seen from the following example. Consider the matrix

c1 c2 cn 1 cn ··· 10 00 2 . ···. . . 3 F = 0 ...... 6 7 6 . .. 7 6 . . 1007 6 7 6 0 0107 6 ··· 7 4 5 F has the characteristic polynomial

n n n 1 (t)=( 1) t c t c . F 1 ··· n 8.2. ESTIMATING EIGENVALUES USING GERSCHGORIN DISCS 245

In other words to any polynomial we can associate an eigenvalue problem, and vice versa, of course. Thus, since there is no general direct solution method for finding roots of polynomials (above order 5), there cannot be a direct method – an algorithmic method that completes in a finite number of steps – to find the eigenvalues of a matrix. In conclusion, the mantra of eigenvalue problems is the following:

Every eigenvalue solver must be iterative.

Now, you may argue that there are matrices for which the spectrum is easy to compute, namely diagonal and triangular matrices. True, for such matrices, we can simply read the eigenvalues from of the diagonal entries. The question is this: can we transform a generic matrix A into a diagonal or triangular matrix T via a similarity transformation. As guaranteed by the n n Schur factorization theorem, there exists a unitary matrix U C ⇥ such that 2 A = UTUH,whereT is triangular. So the question reduces to this one: can we find U using a direct method? The answer again is a resounding no. If there was a such a method, then we could find the roots to any polynomial of any degree in a finite number of steps. Since this is impossible, in general, we get a contradiction. Before we describe our iterative methods for finding eigen-pairs, let intro- duce some coarse approximation techniques.

8.2 Estimating Eigenvalues Using Gerschgorin Discs

If all that is wanted are coarse approximations of the eigenvalues, this can be obtained easily using Gerschgorin discs.

n n Definition 8.2.1 (Gershgorin discs). Let n 2 and A C ⇥ . The Gershgorin 2 discs Di of A are

n Di = z C z ai,i Ri ,Ri = ai,j ,i=1,...,n. { 2 || | } | | j=1 Xj=i 6 n n Theorem 8.2.2 (Gershgorin circle theorem). Let n 2 and A =[ai,j ] C ⇥ . 2 Then n (A) D . ⇢ i i[=1 246 CHAPTER 8. EIGENVALUE PROBLEMS

Proof. Suppose that (, w) is an eigen-pair of A. Then

n a w = w ,i=1, 2, ,n. i,j j i ··· Xj=1 Suppose that n wk = w = max wi . | | k k1 i=1 | | Observe that w = 0, since w = 0. Then k 6 6 a w = w a w | k,k |·| k | | k k,k k | n = a w a w k,j j k,k k j=1 X n = a w k,j j j=1 Xj=k 6 n a w  k,j j j=1 Xj=k 6 n a w = R w .  k,j ·| k | k | k | j=1 Xj=k 6 Thus D . 2 k n n Theorem 8.2.3 (Gerschgorin’s second theorem). Let n 2 and A C ⇥ . 2 Suppose that 1 p n 1 and that the Gerschgorin discs of the matrix A can   be divided into disjoint subsets D(p) and D(q) containing p and q = n p discs, respectively. Then the union of the discs in D(p) contains p eigenvalues, and the union of the discs in D(q) contains q eigenvalues, counting multiplicities. In particular, if one disc is dijoint from all the others, it contains exactly one eigenvalue. And, if all of the discs are disjoint, then each contains exactly one eigenvalue.

n Proof. Define the matrix B(")=[bi,j (")]i,j=1 as follows:

ai,i ,i= j bi,j (")= . "a ,i= j ( i,j 6 8.2. ESTIMATING EIGENVALUES USING GERSCHGORIN DISCS 247

Then B(1) = A and B(0) = diag(a , ,a ). Each eigenvalue of B(0) 1,1 ··· n,n is the center of one of the Gerschgorin discs of A. Thus, exactly p of the eigenvalues of B(0) lie in the union of the discs in D(p). The eigenvalues of the matrix B(") are the zeros of the characteristic polynomial of B("). The coecients of this characteristic polynomial are con- tiuous functions of the parameter ", and, in turn, the zeros of the polynomial are continuous functions of ". As " increases from 0 to 1, the eigenvalues of B(") move in continuous paths in the complex plane. Since the order of the characteristic polynomial of B(") is always exactly n, none of the zeros of the characteristic polynomial diverge to infinity. At the same time, the radii of the Gerschgorin discs increase from 0 to Ri , respectively. In particular, it is easy to see that they increase as

R = "R , 0 " 1,i=1, ,n. ",i i   ··· Since p of the eigenvalues of B(") lie in the union of the discs in D(p) when " = 0, and these discs are disjoint from those in D(q),thesep eigenvalues will stay within the union of the disc in D(p), for all values of ",0 " 1.   n n Theorem 8.2.4. Let n 2 and suppose that in the matrix A C ⇥ all o↵ 2 diagonal elements are smaller in modulus that ">0, i.e.,

a <", 1 i,j n, i = j. | i,j |   6 Suppose that for some r 1, ,n the diagonal element a satisfies 2{ ··· } r,r a a >, for all i = r. | r,r i,i | 6 Then, provided "< , 2(n 1) there is an eigenvalue (A) such that 2 2(n 1) a < "2 < . | r,r | 2(n 1) n n Proof. Suppose >0. Define K =[ki,j ] C ⇥ via 2 , i = j = r ki,j = . (i,j , otherwise 248 CHAPTER 8. EIGENVALUE PROBLEMS

1 Define the similar matrix A := KAK . For example, if r = 3, n = 4,

a1,1 a1,2 a1,3 a1,4 a2,1 a2,2 a2,3 a2,4 4 4 A = 2 3 C ⇥ , a3,1 a3,2 a3,3 a3,4 2 6a a a a 7 6 4,1 4,2 4,3 4,47 then 4 5 1 a1,1 a1,2  a1,3 a1,4 1 a2,1 a2,2  a2,3 a2,4 A = 2 3 . a3,1 a3,2 a3,3 a3,4 6 a a  1a a 7 6 4,1 4,2 4,3 4,4 7 4 5 The Gerschgorin disc of A, with respect to row r, has its center at ar,r ,and its radius is n n R,r = ai,j =  ai,j j=1 j=1 Xj=r Xj=r 6 6 n < " = (n 1)". j=1 Xj=r 6 The disc of A corresponding to row i = r has a as its center, and its radius  6 i,i is n 1 R = a +  a ,i i,j | i,r | j=1 Xj=i j=6 r 6 " < (n 2)" + .  2" Now, pick  = . Then, 2"2(n 1) R < , ,r and R < +(n 2)", i = r. ,i 2 6 Therefore, for i = r, 6 2"2(n 1) R + R < + +(n 2)" ,r ,i 2 <"+ +(n 2)"<, 2 8.3. STABILITY 249

on the assumption that "< 2(n 1) . Finally, we have < a a , | r,r i,i | but R,r + R,i <. Therefore, the two discs D,r and D,i must be disjoint. Hence, there is an eigenvalue (A )=(A), with D . In other 2  2 ,r words, 2"2(n 1) a R < < . | r,r | ,r 2(n 1)

8.3 Stability

In this section, we give a couple of results that are concerned with the stability of eigenvalue computation.

n n Theorem 8.3.1 (Bauer-Fike). Suppose that A C ⇥ is diagonalizable, i.e., n n 2 there is an invertible matrix P C ⇥ and a diagonal matrix D = diag(1, ,n) 1 2 n ··· such that A = PDP . Suppose that (A)= i i=1 is an eigenvalue of 62 n n { } the perturbed matrix A + A, with A C ⇥ . Then 2 n 1 min i P P A , i=1 | |k k k k where is an induced matrix norm. k·k Proof. There is some x Cn such that 2 ? (A + A) x = x.

It follows that (I A) x = Ax, n and 1 1 P (I D) P x = AP(P x). n Note that (In D) is invertible, since (A). Hence, 62 1 1 1 1 P x (I D) P AP P x .  n Observe that, for any induced norm,

1 n 1 1 (In D) = max = n . i=1 i mini=1 i | | | | 250 CHAPTER 8. EIGENVALUE PROBLEMS

Therefore, n 1 min i P P A , i=1 | |k k k k as desired. n n Corollary 8.3.2. If A C ⇥ is Hermitian, then 2 n min i A 2 . i=1 | |k k Proof. If A is Hermitian, it is unitarily diagonalizable. Since unitary matrices have norm 1, the result follows.

n n Corollary 8.3.3. Suppose that A C ⇥ is Hermitian, R,and is closest 2 2 to the eigenvalue r , i.e.,

r = argmin i . 1 i n | |  

Then, setting, = r + r ,

A . | r |k k2 Remark 8.3.4. In other words, for Hermitian matrices, the eigenvalue problem is stable to perturbations in the coecient matrix A.Inthegeneralcase,the eigenvalue problem is stable, but a type of condition number, P P 1 , k k appears on the left hand side. n n Theorem 8.3.5. Suppose that A C ⇥ is Hermitian and (A)= 1, ,n 2 { ··· }⇢ R. Suppose that x Cn and C are given and define 2 ? 2 w := Ax x. Then w 2 min i k k . 1 i n | | x   k k2 Proof. Suppose that S = w , , w is an orthonormal basis of eigenvec- { 1 ··· n} tors of A, with Aw i = i w i ,1 i n. There are unique constants ci C,   2 1 i n,suchthatx = n c w . It follows that x 2 = n c 2 > 0.   i=1 i i k k2 i=1 | i | Furthermore, P P n n n w =(A I) c w = c (A I) w = c ( ) w . i i i i i i i Xi=1 Xi=1 Xi=1 8.4. THE RAYLEIGH QUOTIENT FOR HERMITIAN MATRICES 251

Thus,

2 n 2 2 n 2 w 2 i=1 ci i 2 i=1 ci 2 k k2 = | n | | 2 | min i n | |2 = min i . x ci 1 i n | | ci 1 i n | | k k2 P i=1 | |   Pi=1 | |   The result followsP on taking square roots. P

Remark 8.3.6. The last result can be interpreted as follows: eigenvalue calcu- lations are stable with respect to perturbations in the associated eigenvector.

8.4 The Rayleigh Quotient for Hermitian Matrices

In this section, let us assume that A is Hermitian.

n n Definition 8.4.1 (Rayleigh quotient). Suppose that A C ⇥ is Hermitian. 2 The Rayleigh quotient of x Cn is 2 ? (Ax, x) R(x)= 2 . (x, x)2 Proposition 8.4.2 (properties of the Rayleigh quotient). Suppose that A n n n 2 C ⇥ is Hermitian with spectrum (A)= i R, where the following { }i=1 ⇢ ordering is imposed:

1 2 n 1 n.  ···  Then

1. R(x) R, for all x Cn. 2 2 ? n 2. For all x C?, 2 n n min j R(x) max j . j=1   j=1

3. R(x)=k i↵ x is an eigenvector associated to k .

4. For fixed x Cn, the function 2 ? 2 Q(↵):= Ax ↵x , ↵ C, k k2 8 2 has a unique global minimum, and, in fact, the Rayleigh quotient is the unique minimizer:

2 R(x) = argmin Ax ↵x 2 ↵ k k 2C 252 CHAPTER 8. EIGENVALUE PROBLEMS

Proof. Exercise.

n n Suppose for a moment that A R ⇥ is symmetric. Let us work over the 2 real vector space Rn. In light of the minimality property described in the last result, let us consider how the Rayleigh quotient behaves under perturbations of the vector x. For that, let us compute R(x). We see that r (A(x + x), x + x) (Ax, x) R(x + x) R(x)= 2 2 (x + x, x + x)2 (x, x)2 2 = (Ax R(x)x,x)2 + H.O.T. (x, x)2 where by H.O.T we mean “higher order terms” in the di↵erence vector x. This shows that 2 R(x)= (Ax R(x)x)2. r (x, x)2 For this reason, if (, w) is an eigen-pair of A we have that 2 R(w)= (Aw R(w)w)2 =0, r (w, w)2 so that eigenvalues are stationary points of the Rayleigh quotient. In light of this, using Taylor’s Theorem, we also have that

R(x) R(w)=( R(w), x w) + ( x w 2)= ( x w 2), x w, r 2 O k k2 O k k2 ! so that the Rayleigh quotient is a quadratically accurate approximation of . Let us make the last statement rigorous before moving on.

n n Theorem 8.4.3. Suppose that A C ⇥ is Hermitian with spectrum (A)= n 2 n i R and an associate orthonormal basis of eigenvectors S = w i . { }i=1 ⇢ { }i=1 Suppose that x Cn is a vector with the property that 2 ? x w <", x =1, k k k2 k k2 for some positive number ". Then

R(x) < 2⇢(A)"2. | k |

Proof. Suppose that x = w k . There are unique constants ↵i C,suchthat 6 2 n n x = ↵ w , x 2 = ↵ 2 =1. j j k k2 | j | Xj=1 Xj=1 8.4. THE RAYLEIGH QUOTIENT FOR HERMITIAN MATRICES 253

It follows from orthonormality that

n 0 < x w 2 = ↵ 2 + ↵ 1 2. k k k2 | j | | k | j=1 Xj=k 6 Therefore, using our assumptions,

n 0 < ↵ 2 + ↵ 1 2 = x w 2 <"2. (8.4.1) | j | | k | k k k2 j=1 Xj=k 6 Since x 2 = 1, it follows from (8.4.1) that k k2 n 0 ↵ 2 =1 ↵ 2 <"2. (8.4.2)  | j | | k | j=1 Xj=k 6 Finally, n 2 n j ↵j R(x)= j=1 | | = ↵ 2 + ↵ 2, n ↵ 2 k | k | j | j | P j=1 | j | j=1 Xj=k P 6 so that

n R(x) = ↵ 2 1 + ↵ 2 | k | k | k | j | j | j=1 Xj=k 6 n 1 ↵ 2 + ↵ 2 | k | | k | | j |·| j | j=1 Xj=k 6 n n 2 n 2 max j 1 ↵k + max j ↵j  j=1 | | | | j=1 | | | | j=1 Xj=k 6 ⇢(A)"2 + ⇢(A)"2 =2⇢(A)"2,  using (8.4.2).

This last property motivates the so-called power iterations to find the largest eigenvalue. 254 CHAPTER 8. EIGENVALUE PROBLEMS

8.5 Power Iteration Methods

n n n Definition 8.5.1. Suppose that A C ⇥ and q C? is given. The power 2 2 n method is an algorithm that generates a sequence of vectors v k 1 C { }k=0 2 ? according to the following recursive formula: q v := . 0 q k k2 For k 1 qk := Av k 1, and, provided q = 0, k 6 q v = k . k q k k k2 If qk = 0, the algorithm terminates at step k. Remark 8.5.2. The name of the method becomes clear once we realize that

1 1 k v k = qk = Av k 1 = = ck A v 0. q q ··· k k k2 k k k2 The convergence of the method is as follows.

n n Theorem 8.5.3 (convergence of the power iteration). Let A C ⇥ be Her- n 2 n mitian and q C be given. Suppose that the spectrum (A)= i has 2 ? { }i=1 the ordering 0 1 2 n 1 < n . | || |···| | | | Assume that S = w n is the associated basis of orthonormal eigenvectors { i }i=1 of A.If(q, w ) =0,wherew is the eigenvector corresponding to the n 6 n dominant eigenvalue n, then, for some constant C>0,

k 2k n 1 2 n 1 v s w C , R(v ) 2 C , k k k nk2  | k n| | n| n n when k is suciently large. Here sk is a sequence of modulus one complex numbers.

Proof. The vector v 0 is obtained by normalizing q.Letusthenexpandv 0 in the basis S: there exist unique constants ↵j C 2 n v 0 = ↵j w j . Xj=1 8.5. POWER ITERATION METHODS 255

From our assumptions ↵ =0and n ↵ 2 = 1. Now n 6 j=1 | j | P n

q1 = Av 0 = j ↵j w j . Xj=1 From this it follows that

1 n v 1 = j ↵j w j n ↵ 22 j=1 | j | j Xj=1 qP Proceeding this way we observe that

n 1 k v k = j ↵j w j n ↵ 22k j=1 | j | j Xj=1 qP n 1 1 k k = n↵nw n + j ↵j w j n ↵ 22k 0 1 j=1 | j | j Xj=1 @ A qPk n 1 k ↵nn 1 ↵j j = w n + w j . ↵ k 2 2k 0 ↵ 1 n n n 1 ↵j j n n | | 1+ j=1 ✓ ◆ j=1 ↵n n X r @ A P ⇣ ⌘ k ↵nn Define sk := k . Given the assumptions on (A), we observe that, for ↵nn j =1, ,n |1, that| ··· k j 0ask . ! !1 ✓ n ◆ It is clear that 1 v k w n 0 as k . sk ! !1 We leave it to the reader as an exercise to prove the estimate

k n 1 v s w C , k k k nk2  n for some C>0, provided k is suciently large. If this holds, Theorem 8.4.3 yields the second-order convergence rate of the eigenvalue approximations. 256 CHAPTER 8. EIGENVALUE PROBLEMS

8.5.1 Inverse iteration

Let us begin with an auxiliary result.

n n Proposition 8.5.4. Suppose that A C ⇥ and µ C has the the property 2 2 that µ (A). Then (A µI) 1 exists, and if (, w) is an eigen-pair of A, 62 then ( µ) 1, w is an eigen-pair of (A µI) 1. Proof. Exercise.

This motivates the following observation: if µ is close to (A)then r 2 ( µ) 1 is much larger that ( µ) 1 for any other eigenvalue. Thus, to r j approximate we can apply power iterations to the matrix (A µI) 1. This r is the idea of the inverse iteration method.

n n n Definition 8.5.5. Suppose that A C ⇥ , q C ,andµ C are given. 2 2 ? 2 Assume that A µI is invertible. The inverse iteration method is an algorithm n that generates a sequence of vectors v k 1 C according to the following { }k=0 2 ? recursive formula: q v := . 0 q k k2 For k 1 1 qk := (A µI) v k 1, and, provided qk = 0, 6 q v = k . k q k k k2 If qk = 0, the algorithm terminates at step k.

The convergence properties of this algorithm are as follows.

n n Theorem 8.5.6 (convergence of the inverse iteration). Suppose that A C ⇥ n n 2 is Hermitian with spectrum (A)= i R.LetS = w i be an { }i=1 ⇢ { }i=1 associated orthonormal basis of eigenvectors of A. Suppose that µ C is 2 given with the property that µ (A). Define 62 n n r := argmin j µ ,s := argmin j µ j=1 | | j=1 | | j=r 6 and assume that µ < µ µ , | r | | s || j | 8.5. POWER ITERATION METHODS 257 where j 1, ,n r,s .Ifq Cn is given with the property that 2{ ··· }\{ } 2 ? (q, w ) =0, then the inverse iterations converge. We have the following r 2 6 convergence estimates: there is some C>0 such that

µ k µ 2k v s w C r , R(v ) 2⇢(A)C2 r , k k k r k2  µ | k r | µ s s provided k is suciently large. As before, sk is a sequence of modulus one complex numbers.

Proof. The reader can easily show that

n ↵j k w j j=1 (j µ) v k = , 2 P n ↵j | | 2k j=1 j µ r | | P starting with the unique basis expansion

n v 0 = ↵j w j . Xj=1 From our assumptions, ↵ = 0. Separating the j = r terms in the sums, we r 6 have

↵r n ↵j k w r + k w (r µ) j=1 ( µ) j j=r j v k = 6 2 2 ↵r P n ↵j | | 2k + | | 2k r µ j=1 µ | | j=r | j | r 6 k k ↵r r µ P n ↵j r µ | | k w r + | | k w ↵r (r µ) j=1 ↵r ( µ) j | | j=r | | j = 6 . 2 2k n P ↵j r µ 1+ | |2| |2k j=1 ↵r µ j=r | | | j | r P 6 The remaining details are left to the reader.

8.5.2 Deflation The next result addresses the issue of computing a next-to-dominant eigen- value.

n n Theorem 8.5.7. Suppose that A C ⇥ is Hermitian, and its spectrum is 2 denoted (A)= 1,...,n R.LetS = w 1, , w n be an orthonormal { }⇢ { ··· } 258 CHAPTER 8. EIGENVALUE PROBLEMS basis of eigenvectors of A, with Aw = w ,fork =1, ,n. For simplicity, k k k ··· assume that the eigenvalues of A are ordered as follows:

0 1 2 n 2 < n 1 < n . | || |···| | | | | | Assume that n is known. If H ↵n 1 := w n 1v 0 =0, v k+1 = Av k , y k+1 = v k+1 nv k , 6 then H y k+1y k n 1 as k . H y k y k ! !1 Proof. There exist unique constants ↵j C, j =1, 2, ,n,suchthatv 0 = n 2 H··· j=1 ↵j w j . By assumption ↵n 1 = 0, since ↵j = w v 0. We observe that 6 j v = Ak v , and, consequently, Pk 0 n k k 1 k 1 k 1 y = A v A v = A (A I ) v = ↵ w . k 0 n 0 n n 0 j j j n j j=1 X Therefore,

H n 1 2 2k 1 2 y y ↵j j n k+1 k = j=1 | | j H n 1 2 2(k 1) 2 y k y k P ↵j j n j=1 | | j 2 2(k 1) 2 n 1 ↵j j j n P j=1 ↵n 1 j n 1 n 1 n = 2 2(k 1) 2 n 1 ↵j ⇣j ⌘ ⇣j n ⌘ P j=1 ↵n 1 n 1 n 1 n ⇣ 2⌘ ⇣ 2(k 1) ⌘ 2 P n 2 ↵j j j n + n 1 j=1 ↵n 1 j n 1 n 1 n = 2 2(k 1) 2 n 2 ↵j ⇣j ⌘ ⇣j n ⌘ 1+ P j=1 ↵n 1 n 1 n 1 n ⇣ ⌘ ⇣ ⌘ n 1, P ! 2 as k , since j < 1, for each j =1, 2, ,n 2. n 1 !1 ··· ⇣ ⌘ 8.6 Reduction to Hessenberg Form

We have seen that it is impossible, in general, to reduce a square matrix to a triangular or diagonal matrix through similarity transformation in a finite number of steps (via a direct method). We will see that it is possible to reduce to a Hessenberg matrix, one that has zeros below the first sub-diagonal. 8.6. REDUCTION TO HESSENBERG FORM 259

n n Definition 8.6.1. The square matrix A =[ai,j ] C ⇥ is called a lower Hes- 2 senberg matrix or is said to have lower Hessenberg form i↵ ai,j =0for all 1 i,j n satisfying i j +2. A is said to be upper Hessenberg i↵ A| is   lower Hessenberg.

We want to come up with a similarity transformation that e↵ects the following change:

⇥⇥ ⇥···. ⇥. 2 .. . 3 ⇥⇥ ⇥ H .. A Q AQ = 60 . 7 . ! 6 ⇥⇥ ⇥7 6 ...... 7 6 . . . . 7 6 ⇥7 60 0 7 6 ··· ⇥⇥7 4 5 Is it possible to obtain a similarity transformation using a direct method that gives as a lower Hessenberg matrix? The answer is yes. Suppose that Hn 1 is the unitary matrix that is the product of Houselholder matrices that leaves the first row of A unchanged and introduces zeros below the second row of the first column:

⇥⇥⇥···⇥ 2⌦⌦⌦···⌦3 H .. . A Hn 1A = 0 . . , ! 6 ⌦⌦ 7 6 . . . .. 7 6 . . . . 7 6 ⌦7 60 7 6 ⌦⌦···⌦7 4 5 where as before, by we denote entries of the matrix A that did not change ⇥ H and by those that did change. When we calculate Hn 1AHn 1 it leaves the ⌦ first column unchanged, so that we obtain

⇥⇥⇥···⇥ ⇥⌦⌦···⌦ 2⌦⌦⌦···⌦3 2⌦···3 H .. . H .. . A Hn 1A = 0 . . Hn 1AHn 1 0 . . . ! 6 ⌦⌦ 7 ! 6 7 6 . . . .. 7 6 . . . .. 7 6 . . . . 7 6 . . . . 7 6 ⌦7 6 7 60 7 60 7 6 ⌦⌦···⌦7 6 ···7 4 5 4 5 The symbol indicates that the entry has been changed twice. Repeating this idea we can reduce any matrix to Hessenberg form with the following algorithm. 260 CHAPTER 8. EIGENVALUE PROBLEMS

1. For k =1,...,n 2

• x = Ak+1:n,k . • v =sgn(x ) x e + x. k 1 k k2 1 1 • v k = v v k . k k k2 b • A = A 2v (v HA ). k+1:n,k:n k+1:n,k:n k k k+1:n,k:n • A = A 2(A v )v H. 1:n,k+1:n 1:n,k+1:n 1:n,k+1:n k k 2. EndFor

n n If A C ⇥ is Hermitian, then the Hessenberg transformation results in 2 a tridiagonal Hermitian matrix similar to A. We present a theorem on the existence of this transformation for the Hermitian case. We leave it to the reader to prove the existence in the more general setting.

n n Theorem 8.6.2. Suppose that A C ⇥ is Hermitian. There exists a unitary n 2 matrix Q C ⇥, the product of n 2 Householder matrices 2

Q = Hn 1 H2, ··· n n where Hk C ⇥ is a Householder matrix, such that 2 QHAQ = T,

n n where T C ⇥ is a Hermitian tridiagonal matrix. 2 Proof. Let us express A as

↵ bH A = , b C  n 1 (n 1) (n 1) where ↵ C, b C ,andC C ⇥ is Hermitian. Define 2 2 2 k k = v C v = e1, C . E1 2 9 2 n 1 (n 1) (n 1) If b C? , there is a Householder matrix Hn 1 C ⇥ such that 2 b 2 n 1 Hn 1b 1 . 2E n 1 If b = 0,thenHn 1b = 0 1 , regardless of our choice for Hn 1.Take 2E Hn 1 = I. 8.6. REDUCTION TO HESSENBERG FORM 261

Define H 1 0 n n Hn,n 1 := C ⇥ . (8.6.1) 0 Hn 1 2  This is also a Householder matrix as we have seen from a homework problem. It follows that

H H H H 1 0 ↵ b 1 0 Hn,n 1AHn,n 1 = H 0 Hn 1 b C 0 Hn 1    ↵ d H = , d D  H n 1 where d = Hn 1b = Hn 1b 1 and 2E H (n 1) (n 1) D = Hn 1CHn 1 C ⇥ , 2 which is clearly a Hermitian matrix. The proof from this point is by induction. 3 3 (k = 3): Suppose that A C ⇥ . The matrix 2 H 3 3 H AH3,2 C ⇥ 3,2 2 is already tridiagonal, since

H b = d 2. 3,2 2E1 This case follows upon taking Q = H .Observethat,foranyf 3,that 3 3,2 2E1 is f =[, 0, 0]|,wehave

H H Q3 f = H3,2f = H3,2f = f .

(k = ` 1): For the induction hypothesis, let us assume that for any D (` 1) (` 1) (` 1) (` 1)2 C ⇥ that is Hermitian, there is a unitary matrix Q` 1 C ⇥ , 2 the product of Householder matrices

Q` 1 = H` 1,` 2 H` 1,2, ··· such that the product

H (` 1) (` 1) Q` 1DQ` 1 = T` 1 C ⇥ , 2 is Hermitian and tridiagonal. Furthermore, let us assume that

H ` 1 ` 1 Q` 1f 1 , for all f 1 . 2E 2E 262 CHAPTER 8. EIGENVALUE PROBLEMS

` ` (k = `): Suppose that A C ⇥ is Hermitian. Let us inflate each H` 1,k (` 1) (` 1) 2 2 C ⇥ from the induction hypothesis step by H 1 0 ` ` H`,k := C ⇥ , 0 H` 1,k 2  for k =2, ,` 2. For k = ` 1, define H`,` 1 as in (8.6.1). Next, set ··· Q` := H`,` 1H`,` 2 H`,2. ··· It follows that H Q` AQ` = H`,2 H`,` 2H`,` 1AH`,` 1H`,` 2 H`,2 ··· ··· 1 0H ↵ d H 1 0H = H 0 Q` 1 d D 0 Q` 1    H ↵ d Q` 1 = H =: T`. Q` 1d T` 1  H ` 1 Since Q d , T must be tridiagonal. ` 1 2E1 ` Finally, for any f `,wehaveQHf `, since the (1, 1) entry of Q is 2E1 ` 2E1 ` 1 and the remaining entries in the first column of Qn are zeros. The proof by induction is complete.

n n Theorem 8.6.3. Suppose that A C ⇥ is Hermitian and tridiagonal. Assume 2 that its super and sub-diagonal entries are non-zero. Then the eigenvalues of A are distinct. Proof. Suppose that (A) R. Then A I is singular. Write 2 ⇢ c b 0 0 1 1 ··· .. . 2 b1 c2 b2 . . 3 .. A I = 6 0 b2 c3 . 0 7 , 6 7 6 ...... 7 6 . . . . bn 1 7 6 7 6 0 0 bn 1 cn 7 6 ··· 7 where c = a , b = 4a . To find the eigenspace5 associated to this i i,i i i+1,i eigenvalue, we solve the system c b 0 0 0 1 1 ··· .. . . 2 b1 c2 b2 . . . 3 .. [A I 0]=6 0 b2 c3 . 0 0 7 . | 6 7 6 ...... 7 6 . . . . bn 1 0 7 6 7 6 0 0 bn 1 cn 0 7 6 ··· 7 4 5 8.7. THE QR ALGORITHM 263

This is row-equivalent to b c b 0 0 0 1 2 2 ··· .. . 2 0 b2 c3 b3 . . 0 3 ...... 6 . . . . . 0 0 7 , 6 7 6 0 0 bn 2 cn 1 bn 1 0 7 6 ··· 7 6 0 00bn 1 cn 0 7 6 ··· 7 6 c b 0 000 7 6 1 1 ··· 7 4 5 by simple row interchanges. Since for every 1 i n 1, b = 0, we can use   i 6 these pivots to “zero out” all of the elements in the last row, except possibly the last: b c b 0 0 0 1 2 2 ··· .. . 2 0 b2 c3 b3 . . 0 3 ...... 6 . . . . . 0 0 7 . 6 7 6 0 0 bn 2 cn 1 bn 1 0 7 6 ··· 7 6 0 00bn 1 cn 0 7 6 ··· 7 6 000 0 r 0 7 6 ··· 7 But, of course, r4must be zero, since A I is singular.5 If r = 0, then the 6 only solution to (A I) x = 0 is the trivial solution x = 0. This implies that the nullity of A I is just one. In other words, the geometric multiplicity of is 1. Since the geometric and algebraic multiplicities of eigenvalues for a Hermitian matrix are equal, we have proven the result. Each eigenvalue of A is simple, since the geometric and algebraic multiplicities are both 1.

8.7 The QR Algorithm

Recall that every matrix has a (not necessarily unique) Schur decomposition

A = QHUQ, where Q is unitary and U is upper triangular. The following simple algorithm is called the QR iteration and, under suitable assumptions, computes the factor U in a Schur decomposition of A as the limit of a sequence of approximations.

1. A0 = A 2. For k =1, 2, 3, ··· • Compute the QR factorization Qk Rk = Ak 1 264 CHAPTER 8. EIGENVALUE PROBLEMS

• Set Ak := Rk Qk 3. EndFor

To minimize the cost of the QR algorithm, first one converts to Hessenberg form. The QR process will preserve this form, among other properties.

n n Proposition 8.7.1. Suppose that A C ⇥ . In the QR algorithm, the matrix 2 Ak is similar to Ak 1.IfAk 1 is Hermitian, so is Ak .IfAk 1 is Hessenberg, so is Ak . Finally, if Ak 1 is tridiagonal, so is Ak . H Proof. Since Rk = Qk Ak 1 and Ak = Rk Qk , it follows that H Ak = Rk Qk = Qk Ak 1Qk .

Since Ak = Rk Qk we have k Qk Ak = Qk Rk Qk = Ak 1Q which implies that H Ak = Qk Ak 1Qk . Thus, if Ak 1 is Hermitian, so is Ak . The last two statements are left as an exercise.

This leads us to the main result of convergence. For a generic matrix A, Ak converges (under the proper conditions) to an upper triangular matrix. n n Theorem 8.7.2. Suppose that A C ⇥ is invertible and all its eigenvalues are 2 distinct in modulus. Then, as we have seen before, there exists an invertible n n matrix P C ⇥ such that 2 1 A = PDP , where D = diag( , , ), i.e., A is diagonalizable, with 1 2 ··· n > > > > 0. | 1| | 2| ··· | n| 1 If P has an LU factorization – i.e., there exists a unit lower triangular matrix n n n n 1 L C ⇥ and an upper triangular matrix U C ⇥ ,suchthatP = LU 2 2 (Theorem 3.2.5) – then the sequence of matrices A produced by the { k }k1=1 QR algorithm is such that

lim [Ak ]i,i = i , 1 i n, k   !1 and lim [Ak ]i,j =0, 1 j

Proof. Let us define

:= Q Q , =: R R . Qk 1 ··· k Rk 1 ··· k It is not so hard to prove that

A = HA , Ak = . k+1 Qk Qk Qk Rk Now, since A has distinct eigenvalues, it is diagonalizable. Therefore, there 1 1 is an invertible matrix P,suchthatA = PDP . Assume that P has an LU factorization: 1 P = LU. According to Theorem 3.2.14, this factorization is unique, if it exists. Every matrix has a unique QR factorization of the following form:

P = QR,

n n n n where Q C ⇥ is unitary and R C ⇥ is upper triangular, with positive 2 2 diagonal entries. Thus

k k 1 k k k A = PD P = QR D LD D U.

Observe that, since L is unit lower triangular,

0 if i i if i>j ⇥ ⇤ j Hence, :> k k lim D LD = I. k !1 Now, let us set k k D LD = I + Fk , where lim Fk = O. k !1 It follows that k k 1 R D LD = I + RFk R R. 1 Since Fk O, there is some K N, such that if k K, RFk R < 1. ! 2 Thus, if k K, I + RF R 1 is invertible, and therefore admits a unique1 QR k factorization: 1 I + RFk R = Q˜ k R˜k , 266 CHAPTER 8. EIGENVALUE PROBLEMS

n n n n where Q˜ k C ⇥ is unitary and R˜k C ⇥ is upper triangular, with positive 2 2 ˜ 1 diagonal entries. Since the sequence Qk k=1 is bounded – in particular, Q˜ = 1 – there is a convergent subsequence Q˜ 1 Q˜ 1 .Set k 2 k` `=1 ✓ k k=1 ˜ ˜ Q := lim Qk` . ` !1 It is not dicult to see that Q˜ must be unitary. Next, we observe that

H 1 R˜ = Q˜ I + RF R R˜, k` k` k` ! where R˜ must be upper triangular. Its diagonal elements must be non-negative. Taking the limit of the subsequence that we constructed, we find

I = Q˜R˜, which implies that the diagonal elements of R˜ must be positive. By uniqueness of the QR factorization, I = Q˜ = R˜. We leave it to the reader to prove that, in fact, the original sequences Q˜ k and R˜k actually converge and

Q˜ I and R˜ I, as k . k ! k ! !1 Recall that we have

Ak = QQ˜ R˜ RDk U = . k k Qk Rk Note that QQ˜ is unitary and R˜ RDk U is upper triangular. Furthermore, id k k Qk unitary and is upper triangular. We do not assert here that the upper tri- Rk angular matrices have positive diagonal entries. Therefore, there is a diagonal matrix S = diag(s , ,s ) with s = 1, for i =1, ,n,suchthat k 1 ··· n | i | ··· = QQ˜ S . Qk k k This matrix, e↵ectively, accounts for the loss of uniqueness. Now, it may be proved, with some simple manipulations that

A = HA = SHQ˜ HRDQ˜ S . k+1 Qk Qk k k k k

Since, limk Q˜ k = I, !1

1 ⇥···. ⇥. .. . ˜ H ˜ 1 2 0 2 . 3 lim Qk RDQk = RDR = . k . .. .. !1 6 . . . 7 6 ⇥ 7 6 0 0 n7 6 ··· 7 4 5 8.7. THE QR ALGORITHM 267

Define := Q˜ HRDQ˜ . Dk k k Since the matrix Sk is diagonal

[A ] =¯s s [ ] . k+1 i,j i j Dk i,j Consequently, [A ] =[ ] as k . k+1 i,i Dk i,i ! i !1

Theorem 8.7.3 (convergence of the QR algorithm). Let the QR algorithm be n n applied to a Hermitian matrix A C ⇥ , whose eigenvalues satisfy 2 > > > | 1| | 2| ··· | n| and whose corresponding unitary eigenvector matrix Q has all nonsingular lead- ing principal sub-matrices. Then, as k , the matrices A converge linearly, !1 k with constant max j+1 , j j to ⇤= diag , , and Q converges, with the same rate, to Q. { 1 2 ··· n} k Proof. Extend the previous proof. Use the fact that A is unitarily diagonaliz- able, A = Q⇤QH,andQ has an LU factorization. The details are left as an exercise.

We can give a slightly di↵erent proof in the case that A is HPD, taking advantage of the Cholesky factorization.

n n Theorem 8.7.4. Suppose that A C ⇥ is HPD. Then the sequence of 2 matrices A produced by the QR algorithm converges to a diagonal { k }k1=1 matrix D = diag( , , ),where n ··· 1 0 < , 1 ··· n are the eigenvalues of A.

Proof. Suppose that B = A2,sothatB is HPD. Consider the following se- quence: set B = B, and for all k 0, if B = UHU is the unique Cholesky 0 k k k factorization of Bk ,then H Bk+1 := Uk Uk , 268 CHAPTER 8. EIGENVALUE PROBLEMS

It is left to the reader as a simple exercise to show that, if Bk is HPD, then Bk+1 is always HPD, and, therefore, will possess a unique Cholesky factorization. n n Here Uk C ⇥ is an upper triangular matrix with strictly positive real diagonal 2 entries. For the QR algorithm, recall that A = A, and for all k 0, if A = Q R , 0 k k k then Ak+1 = Rk Qk .

Each Ak is unitarily equivalent to A and is HPD. Furthermore,

2 H H H H Ak = Ak Ak = Rk Qk Qk Rk = Rk Rk . and 2 H H H H Ak+1 = Ak+1Ak+1 = Rk Qk Qk Rk = Rk Rk . It follows by uniquesness that

B = A2, U = R , for all k =1, 2, . k k k k ··· 2 Also, it must be that each Bk is unitarily equivalent to B = A . In particular, H since Rk+1 = Qk+1Rk Qk ,

B = QHRHQ QH R Q = QHB Q ,k=0, 1, . k+1 k k k+1 k+1 k k k k k ···

Now, since each Rk is upper triangular,

i

[Bk ]i,j = [Rk ]`,i [Rk ]`,j . X`=1 Similarly, n

[Bk+1]i,j = [Rk ]i,` [Rk ]j,`. X`=j With these formulae we observe that, for 1 m n,   m m m 2 [Bk ]p,p = [Rk ]i,j , p=1 i=1 j=1 X X X and m m n 2 [Bk+1]p,p = [Rk ]i,j . p=1 i=1 j=1 X X X 8.7. THE QR ALGORITHM 269

Thus, for 1 m

m 1 [B ] (8.7.2) 8 k p,p9 p=1

lim [Bk ]p,p =[B ]p,p . k 1 !1 Since this is the case, (8.7.1) implies that all of the o↵-diagonal elements of Rk must tend to zero. Thus also for the o↵-diagonal elements of Bk . B =: D 1 is a diagonal matrix. The theorem is proven.

8.7.1 QR with shifts Usually the following variant of the QR algorithm, known as QR with shifts, converges faster.

1. A0 := A 270 CHAPTER 8. EIGENVALUE PROBLEMS

2. For k =1, 2, 3, ···

• µk := [Ak ]n,n • Compute the QR factorization Qk Rk = Ak 1 µk In k • Set A := Rk Qk + µk In 3. EndFor

8.8 Problems

n n 1. Let A C ⇥ have eigenvalues (A)= 1, ,n and define 2 { ··· } n n Ri = ai,j ,Cj = ai,j . j=1 i=1 Xj=i Xi=j 6 6 (a) Prove that

n (A) z C z ai,i Ri . ✓ { 2 || | } i[=1 (b) Prove that

n (A) z C z aj,j Cj . ✓ 2  j=1 [ n n 2. For any A C ⇥ , define the Frobenius norm as 2 n n 2 A := ai,j . k kF v ui=1 j=1 uX X t n n Suppose that A, B C ⇥ are unitarily similar, i.e., there is a unitary n n 2 H matrix U C ⇥ such that A = U BU. Prove that A = B . 2 k kF k kF n n 3. Suppose that A C ⇥ is Hermitian and (A)= 1, ,n R. 2 { ··· }⇢ Suppose that x Cn and C are given and define 2 ? 2 w := Ax x. Prove that w 2 min i k k . 1 i n | | x   k k2 8.8. PROBLEMS 271

n n 4. Suppose that A C ⇥ is Hermitian and tridiagonal. Assume that its 2 super and sub-diagonal entries are non-zero. Prove that the eigenvalues of A are distinct.

n n 5. Suppose A C ⇥ is Hermitian and that the spectrum of A is denoted 2 (A)= 1,...,n R.LetS = w 1,...,w n be an orthonormal { }⇢ { } basis of eigenvectors of A, with Aw k = k w k ,fork =1,...,n. The Rayleigh quotient of x Cn is defined as 2 ? x HAx R(x):= . x Hx Prove the following facts:

(a) R(x) is always real-valued. (b) It satisfies the relation

n 2 j ↵j R(x)= j=1 | | , n ↵ 2 P j=1 | j | where ↵ = w Hx, j =1, ,n.P j j ··· (c) The quotient satisfies the estimates

min R(x) max . (A)   (A) 2 2 (d) The Rayleigh quotient is a minimizer:

2 R(x) = argmin Ax ↵x 2. ↵ k k 2C (e) Suppose w w ✏,where w = 1. Show that k r k2  k k2 R(w) 2⇢(A)✏2. | r | n n 6. Suppose that A C ⇥ . Suppose that Ak is computed according to 2 { } the QR algorithm. Prove the following:

(a) The matrix Ak is similar to Ak 1. (b) If Ak 1 is Hermitian, so is Ak . (c) If Ak 1 is Hessenberg, so is Ak . (d) If Ak 1 is tridiagonal, so is Ak . 272 CHAPTER 8. EIGENVALUE PROBLEMS

n n n 7. Suppose that A C ⇥ is Hermitian, with eigenvalues k and as- 2 { }k=1 sociated orthonormal basis of eigenvectors w n . Consider a pertur- { k }k=1 bation A + A, which induces a perturbation in the eigenvalues k + k and eigenvectors w k + w k . The purpose of this problem is to estimate the perturbation of the eigenvalues and eigenvectors.

(a) Fix p 1,...,n and consider and w . Expand the pertur- 2{ } p p bation vector w p in terms of the eigenvectors of A

n w p = ↵p,i w i . Xi=1 Prove that estimating the e↵ect of the perturbation A is reduced to estimating the sizes of the n +1numbers and ↵ , i = | p| | p,i | 1, ,n. ··· (b) Show that, up to higher order terms,

n n Aw p + ↵p,i i w i = ↵p,i pw i + pw p. (8.8.1) i=1 i=1 Xi=p Xi=p 6 6

(c) Take the inner product of (8.8.1) with w p to conclude that

µ A , | p| pk k2

determining the constant µp in the process. (d) Take the inner product of (8.8.1) with w , with j = p, to obtain j 6 µ ↵ j A , | p,j | k k2 | j p|

determining µj in the process. (e) What can you conclude from the computation above?

n n 8. Suppose that A C ⇥ is Hermitian with n distinct (real) eigenvalues. 2 Show that the eigenvalue problem is stable with respect to perturbations.

n n 9. Let n 2andA =[ai,j ] C ⇥ . Suppose that, for some r the diagonal 2 element ar,r satisfies

a a >, i= r. | r,r i,i | 6 8.8. PROBLEMS 273

Moreover, all the o↵-diagonal elements satisfy a < . | i,j | 2(n 1) Show that there is (A)suchthat 2 2(n 1) a < ✏2. | r,r | 1 Hint: Consider the similarity transformation K AK where

K = diag(1, , 1, , 1, , 1). ··· r,r ··· Choose  to obtain disjoint Gershgorin discs.

10. A Fr¨obenius matrix is a matrix that has the form

p1 p2 pn 1 pn ··· 10 00 2 ···. . . 3 F = 01 .. . . . 6 7 6 . .. .. 7 6 . . . 007 6 7 6 0 0107 6 ··· 7 4 5 Find F for a Fr¨obenius matrix?

n n 1 11. Suppose that A C ⇥ . Show that if µ/(A)then(A µIn) exists 2 2 and that if (A)then( µ) 1 ((A µI ) 1) with the same 2 2 n eigenvector.

n n n 12. Let A C ⇥ be Hermitian and q C be given. Suppose that the 2 2 ? spectrum (A)= n has the ordering { i }i=1

0 1 2 n 1 < n . | || |···| | | | Assume that S = w n is the associated basis of orthonormal eigen- { i }i=1 vectors of A.If(q, w ) = 0, where w is the eigenvector corresponding n 6 n to the dominant eigenvalue n, prove that, for some constant C>0,

k 2k n 1 2 n 1 v s w C , R(v ) 2 C , k k k nk2  | k n| | n| n n when k is suciently large. Here sk is a sequence of modulus one com- plex numbers. 274 CHAPTER 8. EIGENVALUE PROBLEMS

n n n 13. Suppose that A C ⇥ is Hermitian with spectrum (A)= i i=1 2n { } ⇢ R.LetS = w i be an associated orthonormal basis of eigenvectors { }i=1 of A. Suppose that µ C is given with the property that µ (A). 2 62 Define n n r := argmin j µ ,s := argmin j µ j=1 | | j=1 | | j=r 6 and assume that

µ < µ µ , | r | | s || j | where j 1, ,n r,s .Ifq Cn is given with the property that 2{ ··· }\{ } 2 ? (q, w ) = 0, prove that the inverse iterations converge, and we have r 2 6 the following convergence estimates: there is some C>0suchthat

µ k µ 2k v s w C r , R(v ) 2⇢(A)C2 r , k k k r k2  µ | k r | µ s s provided k is suciently large. As before, sk is a sequence of modulus one complex numbers.

n n 14. Suppose that A C ⇥ is Hermitian with spectrum (A)= 1,...,n 2 { }⇢ R.LetS = w 1, , w n be an orthonormal basis of eigenvectors of { ··· } A, with Aw = w ,fork =1, ,n. For simplicity, assume that the k k k ··· eigenvalues of A are ordered as follows:

0 1 2 n 2 < n 1 < n . | || |···| | | | | | n Assume that n is known and v 0 C is given. Show that if 2 ? H w n 1v 0 =0, v k+1 := Av k , y k+1 := v k+1 nv k ,k 0, 6 then H y k+1y k n 1. H y k y k ! More precisely, show that

H 2k y k+1y k n 2 H n 1 C , y y k  n 1 k for some constant C> 0, for k suciently large. 8.8. PROBLEMS 275

n n 15. Suppose that A C ⇥ is Hermitian with spectrum (A)= 1,...,n 2 { }⇢ R.LetS = w 1, , w n be an orthonormal basis of eigenvectors of { ··· } A, with Aw = w ,fork =1, ,n. For simplicity, assume that the k k k ··· eigenvalues of A are ordered as follows:

0 1 2 n 1 < n . | || |···| | | |

Show that, for a given x 0 C?, if 2 w Hx =0, x := Ax ,k 0, n 0 6 k+1 k then H x k+1x k H n as k . x k x k ! !1 More precisely, show that

H 2k x k+1x k n 1 H n C , x x k  n k for some constant C> 0, for k su ciently large. n n 16. Suppose that A C ⇥ is Hermitian and tridiagonal. Assume that its 2 super and sub-diagonal entries are non-zero. Prove that the eigenvalues of A are distinct. 276 CHAPTER 8. EIGENVALUE PROBLEMS