SIAM J. MATRIX ANAL. APPL. c 2016 Society for Industrial and Applied Mathematics Vol. 37, No. 1, pp. 195–214 STABILITY ANALYSIS OF THE TWO-LEVEL ORTHOGONAL ARNOLDI PROCEDURE∗ DING LU†, YANGFENG SU†, AND ZHAOJUN BAI‡ Abstract. The second-order Arnoldi (SOAR) procedure is an algorithm for computing an orthonormal basis of the second-order Krylov subspace. It has found applications in solving quadratic eigenvalue problems and model order reduction of second-order dynamical systems among others. Unfortunately, the SOAR procedure can be numerically unstable. The two-level orthogonal Arnoldi (TOAR) procedure has been proposed as an alternative to SOAR to cure the numerical instability. In this paper, we provide a rigorous stability analysis of the TOAR procedure. We prove that under mild assumptions, the TOAR procedure is backward stable in computing an orthonormal basis of the associated linear Krylov subspace. The benefit of the backward stability of TOAR is demonstrated by its high accuracy in structure-preserving model order reduction of second-order dynamical systems. Key words. second-order Krylov subspace, second-order Arnoldi procedure, backward stability, model order reduction, dynamical systems AMS subject classifications. 65F15, 65F30, 65P99 DOI. 10.1137/151005142 1. Introduction. The second-order Krylov subspace induced by a pair of ma- trices and one or two vectors is a generalization of the well-known (linear) Krylov subspace based on a matrix and a vector. An orthonormal basis matrix Qk of the second-order Krylov subspace can be generated by a second-order Arnoldi (SOAR) procedure [3]. The SOAR procedure has found applications in solving quadratic eigenvalue problems [33, 38] and model order reduction of second-order dynamical systems [2, 6] and structural acoustic analysis [25, 26]. It is implemented in Omega3P for electromagnetic modeling of particle accelerators [18, 19], and in MOR4ANSYS for the model order reduction of ANSYS engineering models [28]. It has been known that SOAR is prone to numerical instability due to the fact that it involves solving potentially ill-conditioned triangular linear systems and im- plicitly generates a nonorthonormal basis matrix Vk of an associated (linear) Krylov subspace (see examples in section 5). The instability issue has drawn the attention of researchers since the SOAR procedure was proposed. For quadratic eigenvalue problems, Zhu [40] exploited the relations between the second-order Krylov subspace and the associated linear Krylov subspace described in [3] and proposed to represent the basis matrix Vk of the linear Krylov subspace by the product of two orthonormal matrices Qk and Uk,whereQk is an orthonormal basis of the second-order Krylov subspace and Uk is computed to maintain the orthonormality of Vk.Thesameidea ∗Received by the editors January 23, 2015; accepted for publication (in revised form) by T. Stykel December 11, 2015; published electronically February 16, 2016. http://www.siam.org/journals/simax/37-1/100514.html †School of Mathematical Sciences, Fudan University, Shanghai 200433, China ([email protected], [email protected]). Part of this work was done while the first author was visiting the University of California, Davis, supported by China Scholarship Council. The research of the second author was supported in part by the Innovation Program of Shanghai Mu- nicipal Education Commission 13zz007, E-Institutes of Shanghai Municipal Education Commission N.E303004, and NSFC key project 91330201. ‡Department of Computer Science and Department of Mathematics, University of California, Davis, CA 95616, USA ([email protected]). The research of the this author was supported in part by the NSF grants DMS-1522697 and CCF-1527091. 195 196 D. LU, Y. SU, AND Z. BAI was also independently studied about the same time and presented in [34] for the purposes of curing the numerical instability and finding a memory-efficient represen- tation of the basis matrix Vk in the context of using ARPACK [20] to solve high-order polynomial eigenvalue problems. The term “two-level orthogonal Arnoldi procedure”, TOAR in short, was coined in [34]. Recently, the notion of memory-efficient represen- tations of the orthonormal basis matrices of the second-order and higher-order Krylov subspaces was generalized to the model order reduction of time-delay systems [39], solving the linearized eigenvalue problem of matrix polynomials in the Chebyshev ba- sis [17] and implementing a rational Krylov method for solving nonlinear eigenvalue problems [35]. Unlike SOAR, TOAR maintains an orthonormal basis matrix Vk of the associated linear Krylov subspace. Therefore, it is generally believed to be numerically stable. This belief has been adopted in the literature [39, 17, 35]. The motivation of this paper is to provide a rigorous stability analysis of TOAR. We prove that under mild assumptions, TOAR with partial reorthogonalization is backward stable in computing the basis matrix Vk of the associated linear Krylov subspace. In addition, in this paper, we provide a different derivation of the TOAR procedure comparing to the ones in [40, 34] and remove unnecessary normalization steps. A comprehensive TOAR procedure, including the partial reorthogonalization and the treatments of deflation and breakdown, is presented. The advantages of the TOAR procedure are illustrated by an application to the structure-preserving model order reduction of second-order dynamical systems. The rest of this paper is organized as follows. In section 2, we review the definition and essential properties of the second-order Krylov subspace. In section 3, we derive the TOAR procedure. The backward stability of the TOAR procedure is proven in section 4. Numerical examples for the application of the TOAR procedure in the model order reduction of second-order dynamical systems are presented in section 5. Concluding remarks are in section 6. Notations. Throughout the paper, we use the upper case letter for matrices, lower case letter for vectors, particularly, I for the identity matrix with ei being the ith column, and the dimensions of these matrices and vectors conform with the dimensions used in the context. Following the convention of matrix analysis, we use ·T for transpose, and ·† for the pseudo-inverse, |·| for elementwise absolute value, ·2 and ·F for 2-norm and Frobenius norm, respectively, span{U, v} for the subspace spanned by the columns of matrices U and vector v.WealsouseMATLAB conventions v(i : j)fortheith to jth entries of vector v,andA(i : j, k : )forthe submatrix of matrix A by the intersection of row i to j and column k to .Other notations will be explained as used. 2. Second-order Krylov subspace. Let A and B be n×n matrices and r−1 and r0 be length-n vectors such that [r−1,r0] = 0. Then the sequence r−1,r0,r1,r2,... with (2.1) rj = Arj−1 + Brj−2 for j ≥ 1 is called a second-order Krylov sequence based on A, B, r−1,andr0. The subspace (2.2) Gk(A, B; r−1,r0) ≡ span{r−1,r0,r1,...,rk−1} is called a kth second-order Krylov subspace. If the vector rj lies in the subspace spanned by the vectors r−1,...rj−1, i.e., Gj+1(A, B; r−1,r0)=Gj (A, B; r−1,r0), then STABILITY ANALYSIS OF TOAR 197 a deflation occurs. We should stress that the deflation does not necessarily imply that rj+1 and all the following vectors still lie in Gj (A, B; r−1,r0). The latter case is referred to as a breakdown of the second-order Krylov sequence. The second-order Krylov subspace Gk(A, B; r−1,r0) can be embedded in the linear Krylov subspace 2 k−1 Kk(L, v0) ≡ span{v0,Lv0,L v0,...,L v0} r0 r1 r2 rk−1 (2.3) =span , , ,..., , r−1 r0 r1 rk−2 where AB 2n×2n r0 2n L = ∈ R and v0 = ∈ R , I 0 r−1 and I is an identity matrix of size n. Specifically, let Qk and Vk be basis matrices of the subspaces Gk(A, B; r−1,r0)andKk(L, v0), respectively. Then by (2.3), we know that (2.4) span{Vk(1 : n, :)} =span{r0,r1,...,rk−1}, (2.5) span{Vk(n+1 : 2n, :)} =span{r−1,r0,...,rk−2}, and (2.6) span{Qk} =span{Vk(1 : n, :),Vk(n+1 : 2n, :)}. By (2.4)–(2.6), the basis matrix Vk of the linear Krylov subspace Kk(L, v0)can be written in terms of the basis matrix Qk of the second-order Krylov subspace Gk(A, B; r−1,r0): Vk(1 : n, :) QkUk,1 Qk Uk,1 (2.7) Vk = = = ≡ Q[k]Uk. Vk(n+1 : 2n, :) QkUk,2 Qk Uk,2 Without loss of generality, we assume the linear Krylov subspace Kk(L, v0)isnot reduced, i.e., Vk is of dimension 2n × k.Otherwise,Kk(L, v0)=Kj (L, v0)forsome j<kwith Kj (L, v0) being unreduced, and Vk = Vj .NotethatQk is n × ηk, Uk,1 and Uk,2 are ηk × k,whereηk is the dimension of Gk(A, B; r−1,r0), and ηk ≤ k + 1 due to possible deflations. Equation (2.7) indicates a memory-efficient compact representation of the basis matrix Vk since the memory size 2nk required to store Vk is reduced to (n +2k)ηk for Qk and Uk. Equation (2.6) provides a means to compute an orthonormal basis matrix Qk of Gk(A, B; r−1,r0). One can first generate a basis matrix Vk of Kk(L, v0), say by the Arnoldi procedure [31, Chap.5], and then extract Qk by a rank-revealing orthogonal- ization procedure applied to the columns of [Vk(1 : n, :),Vk(n +1:2n, :)]. However, this scheme is too expensive. A computationally efficient algorithm exploits the com- pact representation (2.7) to directly compute Qk withoutfirstgeneratingVk explicitly.