A HARMONIC RESTARTED ARNOLDI ALGORITHM FOR CALCULATING EIGENVALUES AND DETERMINING MULTIPLICITY RONALD B. MORGAN† AND MIN ZENG‡ Abstract. A restarted Arnoldi algorithm is given that computes eigenvalues and eigenvectors. It is related to implicitly restarted Arnoldi, but has a simpler restarting approach. Harmonic and regular Rayleigh- Ritz versions are possible. For multiple eigenvalues, an approach is proposed that first computes eigenvalues with the new harmonic restarted Arnoldi algorithm, then uses random restarts to determine multiplicity. This avoids the need for a block method or for relying on roundoff error to produce the multiple copies. Key words. eigenvalues, Arnoldi, Krylov subspaces, harmonic Ritz values AMS subject classifications. 65F15, 15A18 1. Introduction. We are interested in computing a few eigenvalues and eigen- vectors of a large, nonsymmetric matrix. We propose a new harmonic restarted Arnoldi method and also give a regular Rayleigh-Ritz version. These methods are closely related to other restarted eigenvalue methods such as implicit restarting and thick-restart Lanczos as will be discussed. Then an approach is given that checks the multiplicity of the eigenvalues. The first phase computes some eigenvalues with the new harmonic restarted Arnoldi algorithm. Then a second phase looks for double eigenvalues. A random starting vector is used for a second harmonic restarted Arnoldi iteration. The eigenvectors from the first phase are combined with approximate eigenvectors from the second iteration to give the multiple eigenvalues and eigenvectors. Convergence of the second phase appears to act as if the previously computed single eigenvalues have been removed from the problem. More phases can be added in order to search for higher multiplicities. The next section gives background information that will be needed. Section 3 presents the new restarted harmonic Arnoldi algorithm. Then multiple eigenvalues are dealt with in section 4. 2. Background. We review ways of restarting the Arnoldi algorithm and then mention the harmonic approach to extracting approximate eigenpairs from a subspace. 2.1. Restarted Arnoldi Methods. The Arnoldi method for eigenvalues [1, 16, 17] finds approximate eigenvalues using a Krylov subspace. The Arnoldi recurrence is T AVm = VmHm + hm+1,mvm+1em (2.1) = Vm+1H¯m, where Vm is the orthonormal matrix whose columns span the dimension m Krylov subspace and Hm is an m by m upper-Hessenberg matrix. H¯m is m +1 by m. The mth coordinate vector is em. The Rayleigh-Ritz procedure [15, 17] is used to compute †Department of Mathematics, Baylor University, Waco, TX 76798-7328 (Ronald [email protected]). The work of this author was partially supported by the National Science Foundation under grant NSF-DMS0310573. ‡Consumnes River College, 8401 Center Parkway, Sacramento, CA 95823. 1 2 RONALD B. MORGAN AND MIN ZENG approximate eigenpairs from the Krylov subspace generated by the Arnoldi recurrence. If we let the eigenpairs of Hm be (θi,gi), then the approximate eigenvalues, called Ritz values, are the θi’s. The approximate eigenvectors are yi ≡ Vgi and are called Ritz vectors. Storage and expense for Arnoldi grows as the size of the Krylov subspace increases. This makes restarting [17] necessary. However, restarting the Arnoldi recurrence is difficult. It is natural to start with only one vector, but several approximate eigenvec- tors need to be retained. That way more than one eigenvalue can be computed at a time. Also, convergence can be better to a specified eigenvector if nearby approximate eigenvectors are in the subspace and cause the corresponding eigenvalues to deflate from the problem. Block methods [17] are one possible remedy, but they can have disadvantages in storage and expense [9]. The implicitly restarted Arnoldi method (IRA) given by Sorensen [19] solves this problem by generating a subspace that contains several Ritz vectors. Specifically, if “exact shifts” are chosen (the unwanted Ritz values are shifts), it can be shown [9, 21] that the IRA subspace is 2 3 m−k−1 (2.2) Span{y1,y2,...yk, vm+1, Avm+1, A vm+1, A vm+1,...,A vm+1}, where y1,...,yk are Ritz vectors from the previous Arnoldi cycle and vm+1 is the m + 1st Arnoldi vector, also from the previous cycle. Subspace (2.2) has a Ritz vector portion and a Krylov portion, however, surprisingly, the entire subspace is a Krylov subspace. See [11, 22] for other proofs that (2.2) is a Krylov subspace, independent of implicit restarting. It is shown in [9] that subspace (2.2) is equivalent to 2 3 m−k (2.3) Span{y1,y2,...yk, Ayi, A yi, A yi,...,A yi}, for 1 ≤ i ≤ k. This helps explain the effectiveness of IRA, since for each Ritz vector, the IRA subspace contains a smaller Krylov subspace with the Ritz vector as starting vector. See also [19] and [6] for convergence of IRA. Another approach to restarting with several vectors is given in [9]. The subspace 2 3 m−k Span{y1, Ay1, A y1, A y1,...,A y1,y2,...yk}, is generated in a method called Arnoldi with eigenvectors (Arnoldi-E). It is equivalent to IRA at the end of each cycle, but is not as efficient. However, the vectors y2,...,yk can be replaced with any vectors, so initial estimates for the eigenvectors can be used if they are available. The whole subspace is then no longer a Krylov subspace, and there can be consequences; see subsection 4.3 in [11]. Wu and Simon [24] propose another restarted version of Arnoldi that is mathe- matically equivalent to IRA. Stewart [22] analyzes a related approach that is presented in a more general setting and thus can apply to various new methods. We discuss the Wu and Simon approach. They consider only the symmetric case and call their method thick-restart Lanczos. Instead of using implicit restarting, they orthonor- 2 3 m−k−1 malize the vectors y1,y2,...yk, vm+1, Avm+1, A vm+1, A vm+1,...,A vm+1 in order to generate an orthonormal matrix Vm with columns spanning subspace (2.2). This approach is like the Arnoldi-E method, except that the Ritz vectors are all put first. It ends up the same as IRA, except for the form of the first k basis vectors (and the QR approach is not needed to generate them). The good properties that were discussed for IRA still exist; namely the entire subspace is a Krylov subspace, and it includes smaller Krylov subspaces with Ritz vectors as starting vectors. HARMONIC RESTARTED ARNOLDI 3 2.2. Harmonic Arnoldi. Harmonic Ritz approximations [8, 4, 14, 18, 13] are an alternative to the regular Rayleigh-Ritz procedure. Given a subspace S and an orthonormal matrix V whose columns span S, regular Rayleigh-Ritz projects over the subspace using operator A. The small eigenvalue problem V T AV g = θg is solved. Meanwhile, harmonic Rayleigh-Ritz projects over subspace (A − σI)S using the oper- ator (A − σI)−1, where σ is a shift (possibly complex) in the region where eigenvalues are desired. The small eigenvalue problem becomes (2.4) V T (A − σI)H (A − σI)V g˜ = (θ˜ − σ)V T (A − σI)H V g.˜ Harmonic Ritz pairs are (θ˜i, y˜i), wherey ˜i = V g˜i. Stewart [23] shows from (2.4) that ify ˜i has unit norm, then (2.5) ||(A − σI)˜yi|| ≤ |θ˜i − σ|. So it is guaranteed that if θ˜i is near σ, then the corresponding harmonic Ritz pair has a small residual. This means that harmonic Ritz pairs near σ are meaningful, even when σ is in the interior of the spectrum. This is not always the case for regular Ritz pairs. The Rayleigh quotients of the harmonic Ritz vectors can be computed. We call these the harmonic Rayleigh quotients or ρ values. They often are more accurate than the harmonic Ritz values, particularly at early stages. However, the harmonic Rayleigh quotients do not have a property like the harmonic Ritz values do in (2.5), so they may not be as reliable. See [8, 13] for more on these ρ values. For harmonic Rayleigh-Ritz applied to the Arnoldi method, the small eigenvalue problem can be rewritten as 2 T ˜ (2.6) (Hm + hm+1,mfem)˜g = θg,˜ −H where f = (Hm − σI) em. So the difference computationally between regular and harmonic Arnoldi is that we replace finding eigenpairs of Hm with finding eigenpairs from (2.6). 3. New Restarted Arnoldi Algorithms. 3.1. Harmonic Restarted Arnoldi. We present a new restarted Arnoldi method that adapts Wu and Simon’s approach to nonsymmetric matrices and also uses har- monic Ritz approximations. We call it Harmonic Restarted Arnoldi (HRA). It is mathematically equivalent at the end of each cycle to the less efficient method called Interior Arnoldi with Eigenvectors in [13]. The subspace is 2 m−k−1 (3.1) Span{y˜1, y˜2,... y˜k, r, Ar, A r,...,A r}, wherey ˜1,... y˜k are harmonic Ritz vectors from the previous cycle and r is a multiple of the harmonic residual vectors. It is shown in [10] that the harmonic residuals are all multiples of each other, so we can say (3.2) Ay˜i − θ˜iy˜i = γir, for some scalars γi. At first glance, (3.1) has a Krylov portion and a separate aug- menting part made up of approximate eigenvectors. However, it is proved in [10] that the entire subspace is a Krylov subspace. The proof uses implicit restarting. See [11] 4 RONALD B. MORGAN AND MIN ZENG for a shorter proof in which the Krylov subspace is constructed and [3] for yet another approach. Subspace (3.1) is equivalent to 2 3 m−k (3.3) Span{y˜1, y˜2,..