Weighted Golub-Kahan-Lanczos Algorithms and Applications

Weighted Golub-Kahan-Lanczos algorithms and applications

Hongguo Xu∗ and Hong-xiu Zhong†

January 13, 2016

Abstract We present weighted Golub-Kahan-Lanczos algorithms for computing a bidiagonal form of a product of two invertible matrices. We give convergence analysis for the algorithms when they are used for computing extreme eigenvalues of a product of two symmetric positive definite matrices. We apply the algorithms for computing extreme eigenvalues of a double-sized matrix arising from linear response problem and provide convergence results. Numerical testing results are also reported. As another application we show a connection between the proposed algorithms and the conjugate gradient (CG) algorithm for positive definite linear systems. Keywords: Weighted Golub-Kahan-Lanczos algorithm, Eigenvalue, Eigenvector, Ritz value, Ritz vector, Linear response eigenvalue problem, Krylov subspace, Bidiagonal matrices. AMS subject classifications: 65F15, 15A18.

1 Introduction

The Golub-Kahan bidiagonalization procedure is a fundamental tool in the singular value decomposition (SVD) method ([8]). Based on such a procedure, Golub-Kahan-Lanczos algorithms were developed in [15] that provide powerful Krylov subspace methods for solving large scale singular value and related eigenvalue problems, as well as least squares and saddle- point problems, [15, 16]. Later, weighted Golub-Kahan-Lanczos algorithms were introduced for solving generalized least squares and saddle-point problems, e.g., [1, 4]. In this paper we propose a type of weighted Golub-Kahan-Lanczos bidiagonalization algorithms. The proposed algorithms are similar to the generalized Golub-Kahan-Lanczos bidiagonalization algorithms given in [1]. There, the algorithms are based on the Golub-Kahan bidiagonalizations of matrices of the form R−1AL−T , whereas ours are based the bidago- nalizations of RT L, where R,L are invertible matrices. When R and L are factors of real symmetric positive deﬁnite matrices: K = LLT , M = RRT , similar to those in [1], the proposed algorithms use the matrices K and M directly without computing the factors L and R. Therefore, the algorithms will be advantageous when K and M are large sparse while L and R may be not. We demonstrate that the algorithms can be simply employed for solving large scale eigenvalue problems of the matrices in the product form MK. As an application, we also use

1Department of Mathematics, University of Kansas, Lawrence, KS, 66046, USA. E-mail: [email protected]. 2Department of Mathematics, East China Normal University, Shanghai, 200241, P.R. China. E-mail: [email protected] and [email protected]. Supported by National Science Foundation of China under grant 11471122 and by China Scholarship Council (CSC) during a visit at University of Kansas.

1 the proposed algorithms for solving the eigenvalue problem of the matrices in the form 0 M H = with symmetric positive definite K and M. The eigenvalue problem of K 0 such a matrix arises from linear response problem in the time-dependent density functional theory [5, 11, 14, 19]. Related eigenvalue properties and numerical eigenvalue methods can be founded in [2, 3, 10, 12, 13, 20, 23, 24, 25]. For a positive definite linear system it is well known that the conjugate gradient (CG) method is equivalent to the standard Lanczos method, e.g., [21, Sec. 6.7] and [9]. As another application, we demonstrate that in the case when K or M is identity, the Krylov subspace linear system solver based on one of the proposed algorithms has a simpler and more direct connection to the CG method. In its original version (when neither K nor M is identity), the solver is mathematically equivalent to a weighted CG method. The paper is organized as follows. In Section 2 we present some basic results that are needed for deriving our algorithms and main results. In Section 3, we construct the weighted Golub-Kahan-Lanczos algorithms and show how the algorithms can be employed to solve the eigenvalue problem of a matrix as a product of two symmetric positive definite matrices. Application of the weighted Golub-Kahan-Lanczos algorithms to the eigenvalue problem of matrix H from linear response problem is described in Section 4, and numerical examples are given in Section 5. In Section 6, the relation between a weighted Golub-Kahan-Lanczos algorithm and a weighted CG method is provided. Section 7 contains some concluding remarks. m×n n Throughout the paper, R is the real field, R is the set of m × n real matrices, R is the n-dimensional real vector space, In is the n × n identity matrix, and ej is its jth column. The notation A > 0(≥ 0) means that the matrix A is symmetric and positive definite n×n n (semidefinite). For a given matrix A ∈ R and a vector b ∈ R , the kth Krylov subspace of A with b, i.e., the subspace spanned by the set of k vectors {b, Ab, . . . , Ak−1b}, is denoted by Kk(A, b). || · || is the spectral norm for matrices and 2-norm for vectors. For√ a symmetric T positive definite matrix A, || · ||A denotes the vector norm defined by ||x||A = x Ax for any vector x, and a matrix X is A-orthonormal if XT AX = I (it is A-orthogonal if X is a square matrix). A set of vectors {x1, . . . , xk} is also called A-orthonormal if X = x1 . . . xk is T A-orthonormal and A-orthogonal if xi Axj = 0 for i 6= j. σmax(A) and σmin(A) are the largest and the smallest singular values of matrix A, respectively; and κ2(A) = σmax(A)/σmin(A) (when σmin(A) > 0) is the spectral condition number of A. In the paper we restrict ourselves to the real case. All the results can be simply extended to the complex case.

2 Preliminaries

n×n Suppose K,M ∈ R are both symmetric and positive deﬁnite. There exist invertible matrices L and R such that K = LLT ,M = RRT . (1) Consider the eigenvalue problem of the matrix products MK or KM = (MK)T . Since

MK = RRT LLT = L−T [(RT L)T (RT L)]LT = R[(RT L)(RT L)T ]R−1,

MK is similar to the symmetric positive deﬁnite matrix (RT L)T (RT L) (and (RT L)(RT L)T ). Let RT L = ΦΛΨT (2)

2 be a singular value decomposition (SVD), where Φ and Ψ are real orthogonal and Λ = T diag(λ1, . . . , λn) with λ1 ≥ ... ≥ λn > 0 (since R L is invertible). Then MK = (L−T Ψ)Λ2(L−T Ψ)−1 = (RΦ)Λ2(RΦ)−1. (3) Therefore, we have the following basic properties. Lemma 1. Let RT L have an SVD (2). Then 2 2 (a) The eigenvalues of MK (and KM) are λ1, . . . , λn. (b) The columns of L−T Ψ and RΦ = L−T ΨΛ are the corresponding right (left) eigenvectors associated with MK (KM), and L−T Ψ is K-orthogonal and RΦ is M −1-orthogonal. (c) The columns of LΨ = R−T ΦΛ and R−T Φ are the corresponding left (right) eigenvectors associated with MK (KM), and LΨ is K−1-orthogonal and R−T Φ is M-orthogonal. Proof. (a) is simply from (3) and the fact KM = (MK)T . (b) is also from (3) and the fact that (L−T Ψ)T K(L−T Ψ) = I = (RΦ)T M −1(RΦ). The relation RΦ = L−T ΨΛ is from (2). (c) can be proved in the same way as for (b) by considering KM = (LΨ)Λ2(LΨ)−1 = (R−T Φ)Λ2(R−T Φ)−1, which is obtained by taking the transpose on (3). Lemma 1 shows that the matrix MK has a complete set of K-orthonormal right eigenvectors and a complete set of M −1-orthonormal right eigenvectors. It has also a complete set of K−1-orthonormal left eigenvectors and a complete set of M-orthonormal left eigenvectors. These four sets of eigenvectors are related and the relations can be established without using the orthogonal matrices Φ and Ψ, as shown in the following lemma. Lemma 2. Suppose W = ξ1 ξ2 . . . ξn and ξ1, . . . , ξn are K-orthonormal right eigen- 2 2 2 vectors of MK corresponding to the eigenvalues λ1, . . . , λn so that MKW = W Λ . Then −1 (a) the columns of W Λ = λ1ξ1 λ2ξ2 . . . λnξn form a complete set of M -orthonormal right eigenvectors of MK;

−1 2 (b) the columns of KW = Kξ1 Kξ2 . . . Kξn = M W Λ form a complete set of K−1-orthonormal left eigenvectors of MK;

−1 −1 −1 −1 −1 (c) the columns of KW Λ = λ1 Kξ1 λ2 Kξ2 . . . λn Kξn = M W Λ form a complete set of M-orthonormal left eigenvectors of MK. Proof. The claim in (a) follows from the equivalence relation MKW = W Λ2 if and only if MK(W Λ) = (W Λ)Λ2, and the fact that KW = M −1W Λ2 implies I = W T KW = W T M −1W Λ2, which is equivalent to (W Λ)T M −1(W Λ) = I. The claim in (b) follows from the equivalence relation MKW = W Λ2 if and only if KM(KW ) = (KW )Λ2 and the fact (KW )T K−1(KW ) = W T KW = I. The claim in (c) follows from the relations KM(KW Λ−1) = (KW Λ−1)Λ2 and (KW Λ−1)T M(KW Λ−1) = Λ−1W T K(MKW )Λ−1 = Λ−1W T KW Λ2Λ−1 = I.

3 Therefore, once one of the four complete sets is known, there other three sets can be simply constructed. From Lemma 1, one can choose W = L−T Ψ when Ψ is known. Then it is easily veriﬁed that

W = L−T Ψ,KW = LΨ,W Λ = RΦ,KW Λ−1 = R−T Φ. (4)

3 Weighted Golub-Kahan-Lanczos algorithms

In principle, the eigenvalue problem of MK or KM can be solved as follows: we ﬁrst compute the factorizations in (1), then compute an SVD of RT L as (2). Computing an SVD of RT L requires an initial step for determining real orthogonal matrices U and V such that

U T RT LV = B, (5)

where B is upper or lower bidiagonal ([8]). An SVD B = ΦΛ˜ Ψ˜ T can be computed by running SVD iterations. Then one has (2) with Φ = UΦ˜ and Ψ = V Ψ.˜ For large scale problem, the columns of U, V and the entries of B can be computed by the Golub-Kahan-Lanczos algorithm proposed in [15], and the singular values of the leading principal submatrices of B are used for approximating the singular values of RT L, therefore, also the eigenvalues of MK and KM. This algorithm requires the matrices L and R. If the matrices K and M are given, computing L and R is time consuming, and normally L and R may not be as sparse as K and M. This causes a numerical difficulty for using Lanczos-type methods. We propose weighted Golub-Kahan-Lanczos algorithms to overcome this difficulty. The algorithms apply to the matrices K and M directly instead of their factors. They are constructed based on the following observations. Define

X = R−T U, Y = L−T V, (6)

where U and V are given in (5). It is easily veriﬁed that

XT MX = U T R−1MR−T U = U T U = I,Y T KY = V T L−1KL−T V = V T V = I,

i.e., X is M-orthonormal and Y is K-orthonormal. The bidiagonal form (5) and its transpose can be written as LV = R−T UB,RU = L−T VBT . Since LV = LLT (L−T V ) = KY,RU = RRT (R−T U) = MX, we have KY = XB,MX = YBT . (7) Based on these two relations and the orthogonality of X and Y , Lanczos-type iteration procedures can be easily constructed. There are two Lanczos-type procedures depending on whether B is upper or lower bidiagonal. We ﬁrst consider the upper bidiagonal case and we call the procedure the upper bidiagonal version of weighted Golub-Kahan-Lanczos algorithm (WGKLU). Denote X = x1 ··· xn ,Y = y1 ··· yn ,

4 and   α1 β1  ..   α2 .  B =   .  ..   . βn−1  αn By comparing the columns of the relations in (7), one has

Ky1 = α1x1, Mx1 = α1y1 + β1y2,

Ky2 = β1x1 + α2x2, Mx2 = α2y2 + β2y3, . . . .

Kyk = βk−1xk−1 + αkxk, Mxk = αkyk + βkyk+1, . . . .

Kyn = βn−1xn−1 + αnxn, Mxn = αnyn.

T T Choosing an initial vector y1 satisfying y1 Ky1 = 1, and using the orthogonality xi Mxj = T yi Kyj = δij, where δij is 0 if i 6= j and is 1 if i = j, then the columns of X and Y , as well as the entries of B, can be computed by the following iterations

αj = ||Kyj − βj−1xj−1||M

xj = (Kyj − βj−1xj−1)/αj

βj = ||Mxj − αjyj||K

yj+1 = (Mxj − αjyj)/βj

with x0 = 0 and β0 = 1, for j = 1, 2,.... The following observations help to make the iterations more eﬃcient. Computing αj requires the vector fj := M(Kyj − βj−1xj−1), which equals αjMxj. The vector Mxj appears in Mxj − αjyj needed for computing βj and yj+1, which can now be computed using fj/αj. In this way, we save one matrix-vector multiplication. Similarly, computing βj needs the vector gj+1 := K(Mxj − αjyj) = βjKyj+1. The vector Kyj+1 involved in computing αj+1 and xj+1 in the next iteration can then be computed using gj+1/βj. So another matrix-vector multiplication can be saved. We now have the following WGKLU algorithm. Algorithm 1 (WGKLU). Choose y1 satisfying ky1kK = 1, and set β0 = 1, x0 = 0. Compute g1 = Ky1. For j = 1, 2, ··· sj = gj/βj−1 − βj−1xj−1 fj = Msj T 1 αj = (sj fj) 2 xj = sj/αj tj+1 = fj/αj − αjyj gj+1 = Ktj+1 T 1 βj = (tj+1gj+1) 2 yj+1 = tj+1/βj End

5 In each iteration, this algorithm requires two matrix-vector multiplications, and it needs 6 vectors fk, xk−1, xk, gk, yk, yk+1 to be involved. (xk, yk+1 may overwrite sk and tk+1.) Suppose Algorithm 1 runs k iterations we have x1, . . . , xk, y1, . . . , yk+1 and αj, βj for j = 1, . . . , k. For any j ≥ 0 deﬁne   α1 β1  .. ..   . .  Xj = x1 . . . xj ,Yj = y1 . . . yj ,Bj =   .  ..   . βj−1  αj

Then we have the relations

T T T KYk = XkBk,MXk = YkBk + βkyk+1ek = Yk+1 Bk βkek , (8)

and T T Xk MXk = Ik = Yk KYk. (9)

When k = n, it has to be βn = 0 and we have (5). Algorithm 1 can be implemented for computing the extreme eigenvalues of MK and KM directly. From (8), one has

T T T 2 T T MKYk = YkBk Bk + αkβkyk+1ek ,KMXk = Xk(BkBk + βkekek ) + αk+1βkxk+1ek . (10)

T 2 T T The matrices BkBk +βkekek and Bk Bk are symmetric tridiagonal. So WGKLU is equivalent to two weighted versions of Lanczos algorithm applied to the matrices MK and (MK)T , respectively. Clearly, x1 is parallel to the vector Ky1. From the relations in (10), one has

range Yk = Kk(MK, y1), range Xk = Kk(KM, Ky1) = KKk(MK, y1). (11)

T T Looking closely, suppose K,M are factorized as in (1), and let Uk = R Xk, Vk = L Yk. T T Then from (9), Uk Uk = Vk Vk = Ik, and the relations in (10) are equivalent to

T T T T T 2 T T L MLVk = VkBk Bk +αkβkvk+1ek ,R KRUk = Uk(BkBk +βkekek )+αk+1βkuk+1ek , (12) which are identical to the relations obtained by applying the standard Lanczos algorithm to the symmetric positive definite matrices LT ML and RT KR, respectively. The relations also show that WGKLU breaks down only when βk = 0 for some k. Otherwise, if αk = 0 T T T T while βk 6= 0, one has L MLVk = VkBk Bk, which implies that Vk L MLVk is singular, T contradicting to the positive definiteness of L ML. Going back to (10), if βk = 0, one has T T MKYk = YkBk Bk and KMXk = XkBkBk . Then the columns of Yk and Xk, respectively, span an invariant subspace of MK and KM, and the squares of the singular values of Bk are the eigenvalues of MK and KM. If βk > 0, we may still compute the singular values and singular vectors of Bk and Bk βkek , and use them to approximate the eigenvalues and eigenvectors of MK and KM. We first consider the approximations based on the first relation of (10). Suppose Bk has an SVD

˜ ˜ T ˜ ˜ Bk = ΦkΣkΨk , Φk = µ1 . . . µk , Ψk = ν1 . . . νk , Σk = diag(σ1, . . . , σk). (13)

6 2 Then from the ﬁrst relation in (10) for each j ∈ {1, . . . , k}, we may take σj as a Ritz value of MK and Ykνj as a corresponding right Ritz vector. Since Yk is K-orthonormal and Ψ˜ k is real orthogonal, Ykν1,...,Ykνk are K-orthonormal. Also, we have the residual

2 (MK − σj I)Ykνj = αkβkνjkyk+1,

where νjk is the kth component of νj, for j = 1, . . . , k. In view of Lemma 2 (c) or by pre- multiplying K to the the ﬁrst equation in (10), it is natural to take KYkνj or equivalently,

−1 −1 σj KYkνj = σj XkBkνj = Xkµj, j = 1, . . . , k (14)

as the corresponding left Ritz vectors. Note Xkµ1,...,Xkµk are M-orthonormal. For each j ∈ {1, . . . , k}, the corresponding residual, following (10), is the transpose of

2 (KM − σj I)Xkµj = βkµjk(βkxk + αk+1xk+1) = βkµjkKyk+1,

where µjk is the kth component of µj. In practice, we may use the residual norms

2 2 q 2 2 ||(MK − σj I)Ykνj||K = αkβk|νjk|, ||(KM − σj I)Xkµj||M = βk|µjk| βk + αk+1 to design a stopping criterion for WGKLU. The convergence properties can be readily established by employing the convergence theory of the standard Lanczos algorithm ([9, 18, 22]). We need the following deﬁnitions. For n n×n two vectors 0 6= x, y ∈ R and 0 < A ∈ R , we deﬁne the angles |xT y| |xT Ay| θ(x, y) = arccos , θA(x, y) = arccos . ||x||||y|| ||x||A||y||A

We also denote by Cj(x) the degree j Chebyshev polynomial of the ﬁrst kind. The following convergence results are based on the theory given in [22]. 2 2 2 Theorem 3. Let λ1 ≥ λ2 ≥ ... ≥ λn > 0 be the eigenvalues of MK with λj > 0 for j = 1, . . . , n, and ξ1, . . . , ξn be the corresponding K-orthonormal right eigenvectors, and fol- −1 lowing Lemma 2 (c) let ηj := λj Kξj,j = 1, . . . , n be the corresponding M-orthonormal left eigenvectors. Suppose that Bk has an SVD (13) with σ1 ≥ σ2 ≥ ... ≥ σk > 0. Consider the 2 2 Ritz values σ1, . . . , σk, the corresponding K-orthonormal right Ritz vectors Ykν1,...,Ykνk, and the M-orthonormal left Ritz vectors Xkµ1,...,Xkµk, associated with MK. Let

2 2 2 2 λj − λj+1 λn−k+j−1 − λn−k+j γj = 2 2 , γ˜j = 2 2 ; 1 ≤ j ≤ k, λj+1 − λn λ1 − λn−k+j−1 and y1 be the initial vector in Algorithm 1. Then for j = 1, . . . , k, 2 2 2 2 2 πj,k tan θK (y1, ξj) 0 ≤ λj − σj ≤ (λ1 − λn) (15) Ck−j(1 + 2γj) with j−1 Y σ2 − λ2 π = 1; π = i n , j > 1; 1,k j,k σ2 − λ2 i=1 i j

7 and 2 2 2 2 2 π˜j,k tan θK (y1, ξn−k+j) 0 ≤ σj − λn−k+j ≤ (λ1 − λn) (16) Cj−1(1 + 2˜γj) with n Y σ2 − λ2 π˜ = 1;π ˜ = i 1 , j < k; k,k j,k σ2 − λ2 i=n−k+j+1 i n−k+j The corresponding Ritz vectors have the following bounds: s 2 2 σj 2 σj sin θM (Xkµj, ηj) + 1 − λj λj

q 2 2 πj 1 + (αkβk) /δj = sin θK (Ykνj, ξj) ≤ sin θK (y1, ξj) (17) Ck−j(1 + 2γj)

2 2 with δj = mini6=j |λj − σi | and

j−1 Y λ2 − λ2 π = 1, π = i n , j > 1; 1 j λ2 − λ2 i=1 i j

s 2 2 σj σj sin θM (Xkµj, ηn−k+j) + 1 − λn−k+j λn−k+j q 2 ˜2 π˜j 1 + (αkβk) /δj = sin θK (Ykνj, ξn−k+j) ≤ sin θK (y1, ξn−k+j) (18) Cj−1(1 + 2˜γj) ˜ 2 2 with δj = mini6=j |λn−k+j − σi | and n Y λ2 − λ2 π˜ = 1, π˜ = i 1 , j < k. k j λ2 − λ2 i=n−k+j+1 i n−k+j Proof. We first prove (15) and (17). From Lemma 1, the K-orthonormal right eigenvectors T T ξ1, . . . , ξn are well defined. For any j the vector L ξj is a unit eigenvector of L ML corre- 2 T T sponding to the eigenvalue λj , and L ξ1,...,L ξn are orthonormal. From the first equation 2 2 T T of (12), σ1, . . . , σk are the Ritz values of L ML and for Vk = L Yk, Vkν1,...,Vkνk are the corresponding orthonormal right (left) Ritz vectors. Applying the standard Lanczos convergence results in [22, Sec. 6.6] to the first relation in (12), one has

T T 2 2 2 2 2 πj,k tan θ(L y1,L ξj) 0 ≤ λj − σj ≤ (λ1 − λn) Ck−j(1 + 2γj) q 2 2 πj 1 + (αkβk) /δ T j T T sin θ(Vkνj,L ξj) ≤ sin θ(L y1,L ξj), Ck−j(1 + γj) where πj,k, πj, δj, γj are deﬁned in the theorem. The bounds (15) and (17) can be derived simply by using the equalities

T T T T T θ(L y1,L ξj) = θK (y1, ξj), θ(Vkνj,L ξj) = θ(L Ykνj,L ξj) = θK (Ykνj, ξj).

8 We still need to prove the equality in (17). By (14), T T |νj Yk KMKξj| cos θM (ηj,Xkµj) = cos θM (Kξj/λj,KYkνj/σj) = σjλj

λj T T λj = |νj Yk Kξj| = cos θK (Ykνj, ξj). σj σj Hence σj cos θK (Ykνj, ξj) = cos θM (Xkµj, ηj), (19) λj and from which one has s 2 2 σj 2 σj sin θK (Ykνj, ξj) = sin θM (Xkµj, ηj) + 1 − . λj λj The bounds (16) and (18) can be proved by applying the proved results to the matrix −MK. The equality in (18) can be established by using the identity

σj cos θK (Ykνj, ξn−k+j) = cos θM (Xkµj, ηn−k+j), (20) λn−k+j which can be derived in the same way as for (19). The second relation in (10) can to used to approximate the eigenvalues and eigenvectors of MK too. Let   ρ1 0 0  .. ..  Bk βkek ω1 . . . ωk+1 = ζ1 . . . ζk  . .  (21) 0 ρk 0 2 2 be an SVD. Then ρ1, . . . , ρk are Ritz values and Xkζ1,...,Xkζk are corresponding M-orthonormal left (right) Ritz vectors of MK (KM). The residual (in transpose) formula is 2 (KM − ρj I)Xkζj = αk+1βkζjkxk+1, where ζjk is the kth component of ζj. Following Lemma 2 (c), or by pre-multiplying M to the second equation in (10) and using (8), the K-orthonormal vectors

−1 −1 T ρj MXkζj = ρj Yk+1 Bk βkek ζj = Yk+1ωj, j = 1, . . . , k, (22) can be served as the right Ritz vectors of MK. From the ﬁrst equation of (10) with k replaced by k + 1,

T T MKYk+1 = Yk+1 Bk βkek Bk βkek + αk+1(αk+1yk+1 + βk+1yk+2)ek+1 T T = Yk+1 Bk βkek Bk βkek + αk+1Mxk+1ek+1. So for each j ∈ {1, . . . , k}, the residual is 2 (MK − ρj I)Yk+1ωj = αk+1ωj,k+1Mxk+1 = αk+1ωj,k+1(αk+1yk+1 + βk+1yk+2),

where ωj,k+1 is the (k + 1)st component of ωj. Hence

2 2 q 2 2 ||(KM − ρj I)Xkζj||M = αk+1βk|ζjk|, ||(MK − ρj I)Yk+1ωj||K = αk+1|ωj,k+1| αk+1 + βk+1. The same type of convergence theory can be established.

9 2 2 2 Theorem 4. Let λ1 ≥ λ2 ≥ ... ≥ λn > 0 be the eigenvalues of MK with λj > 0 for j = 1, . . . , n, η1, . . . , ηn be the corresponding M-orthonormal left eigenvectors associated with MK, −1 and by Lemma 2 (c), ξj = λj Mηj, j = 1, . . . , n be the corresponding K-orthonormal right eigenvectors. Suppose ρ1 ≥ ... ≥ ρk are the singular values of Bk βkek , ζ1, . . . , ζk the corresponding orthonormal left singular vectors, and ω1, . . . , ωk the corresponding orthonormal right singular vectors, as deﬁned in (21). Let γj, γ˜j, πj, and π˜j be deﬁned in Theorem 3, and x1 = Ky1/||Ky1||M be generated by Algorithm 1. Then for j = 1, . . . , k, 2 2 2 2 2 κj,k tan θM (x1, ηj) 0 ≤ λj − ρj ≤ (λ1 − λn) (23) Ck−j(1 + 2γj) with j−1 Y ρ2 − λ2 κ = 1; κ = i n , j > 1; 1,k j,k ρ2 − λ2 i=1 i j and 2 2 2 2 2 κ˜j,k tan θM (x1, ηn−k+j) 0 ≤ ρj − λn−k+j ≤ (λ1 − λn) (24) Cj−1(1 + 2˜γj) with n Y ρ2 − λ2 κ˜ = 1;κ ˜ = i 1 , j < k; k,k j,k ρ2 − λ2 i=n−k+j+1 i n−k+j The corresponding Ritz vectors of MK have the following bounds: s 2 2 ρj 2 ρj sin θK (Yk+1ωj, ξj) + 1 − λj λj

q 2 2 πj 1 + (αk+1βk) /j = sin θM (Xkζj, ηj) ≤ sin θM (x1, ηj) (25) Ck−j(1 + 2γj)

2 2 with j = mini6=j |λj − ρi |,

s 2 2 ρj 2 ρj sin θK (Yk+1ωj, ξn−k+j) + 1 − λn−k+j λn−k+j

q 2 2 π˜j 1 + (αk+1βk) /˜j = sin θM (Xkζj, ηn−k+j) ≤ sin θM (x1, ηn−k+j) (26) Cj−1(1 + 2˜γj)

2 2 with ˜j = mini6=j |λn−k+j − ρi |. Proof. The bounds can be derived by applying the standard Lanczos convergence results to the second relation in (12). By (22),

−1 −1 T λj T λj cos θK (ξj,Yk+1ωj) = |λj ρj ηj MK(MXkζj)| = |ηj MXkζj| = cos θM (Xkζj, ηj), ρj ρj and with which the equality in (25) can be derived.

10 Similarly, one has

λn−k+j cos θK (ξn−k+j,Yk+1ωj) = cos θM (Xkζj, ηn−k+j), ρj which establishes the equality in (26).

Remark 1. Since σ1, . . . , σk are the singular values of Bk and ρ1, . . . , ρk are the singular values of Bk βkek , from the interlacing properties ([9, Theorem 8.6.2]) one has

ρ1 ≥ σ1 ≥ ρ2 ≥ σ2 ≥ ... ≥ ρk ≥ σk.

2 From Theorems 3 and 4, for approximating large eigenvalues of MK it is better to use ρj 2 2 while for small eigenvalues it is better to use σj . For instance, if we need to approximate λ1, 2 2 2 2 2 ρ1 is more accurate than σ1, and if we need to approximate λn, then σk is better than ρk. Remark 2. Theorems 3 and 4 provide the convergence results for both the left and right T eigenvectors of MK as well as KM = (MK) . The quantities sin θK (y1, ξj) and sin θM (x1, ηj) represent the inﬂuence of the initial vector y1 to the approximated eigenvalues and eigenvectors. In general, the angles θK (y1, ξj) and θM (x1, ηj) are diﬀerent, but they are close. Recall, −1 2 x1 = Ky1/||Ky1||M , ηj = λj Kξj and MKξj = λj ξj. So

T T |y1 KMKξj| λj T λj cos θM (x1, ηj) = |x1 Mηj| = = |y1 Kξj| = cos θK (y1, ξj). λj||Ky1||M ||Ky1||M ||Ky1||M Because

2 T T T T T T T T T ||Ky1||M = y1 KMKy1 = (L y1) (L ML)(L y1), (L y1) (L y1) = y1 Ky1 = 1,

2 2 T and λ1, . . . , λn are the eigenvalues of L ML, one has

λn ≤ ||Ky1||M ≤ λ1.

Therefore λj λj cos θK (y1, ξj) ≤ cos θM (x1, ηj) ≤ cos θK (y1, ξj). (27) λ1 λn

When the matrix B in (7) is lower bidiagonal, a corresponding lower bidiagonal version of weighted Golub-Kahan-Lanczos algorithm (WGKLL) can be easily derived by interchanging the roles of K and M, X and Y in (7). In this case, in order to avoid confusion we use X˜, Y˜ , B˜ instead of X,Y,B in (7) and we have   α˜1  ˜ ..  T  β1 .  KY˜ = X˜B,M˜ X˜ = Y˜ B˜ with B˜ =   . (28)  .. ..   . .  β˜n−1 α˜n

The K-orthogonal matrix Y˜ and the M-orthogonal matrix X˜ and the entries of B˜ can be computed by the following algorithm.

11 Algorithm 2 (WGKLL). ˜ Choose x˜1 satisfying kx˜1kM = 1, and set β0 = 1, y˜0 = 0. Compute g1 = Mx˜1. For j = 1, 2, ··· sj = gj/β˜j−1 − β˜j−1y˜j−1 fj = Ksj T 1 α˜j = (sj fj) 2 y˜j = sj/α˜j tj+1 = fj/α˜j − α˜jx˜j gj+1 = Mtj+1 ˜ T 1 βj = (tj+1gj+1) 2 x˜j+1 = tj+1/β˜j End Let   α˜1  ˜ ..  ˜ ˜ ˜  β1 .  Xj = x˜1 ... x˜j , Yj = y˜1 ... y˜j , Bj =   .  .. ..   . .  β˜j−1 α˜j Then one has ˜ ˜ ˜ ˜ ˜ T ˜ Bk ˜ ˜ ˜T KYk = XkBk + βkx˜k+1ek = Xk+1 ˜ T ,MXk = YkBk βkek and ˜ T ˜ ˜ T ˜ Xk MXk = I = Yk KYk. Then,

˜ ˜ ˜ ˜T ˜ T ˜ ˜ ˜T ˜ ˜2 T ˜ T KMXk = XkBkBk +α ˜kβkx˜k+1ek ,MKYk = Yk(Bk Bk + βkekek ) +α ˜k+1βky˜k+1ek , and range X˜k = Kk(KM, x˜1), range Y˜k = Kk(MK,Mx˜1) = MKk(KM, x˜1). ˜ ˜ Bk If we use the singular values and vectors of Bk and ˜ T to approximate the eigenval- βkek ues and eigenvectors of MK and KM, convergence theory as Theorems 3, 4 can be established in the same way. In the rest of this section we discuss the relations between the two algorithms. Denote U = RT X, V = LT Y , U˜ = RT X˜, V˜ = LT Y˜ , all of which are orthogonal matrices, where U˜ and V˜ satisfy U˜ T RT LV˜ = B˜. Note from (5), one has

RT L = UBV T = U˜B˜V˜ T .

Hence U˜ T UB = B˜V˜ T V.

If we choose y1 andx ˜1 = x1 = Ky1/||Ky1||M , then the ﬁrst columns of U˜ and U are identical, T T T T T T or the ﬁrst column of U˜ U is e1. Since U˜ UBB (U˜ U) = B˜B˜ is a tridiagonal reduction

12 T T of BB , if all βj, αj, β˜j, α˜j are positive, by the implicit-Q Theorem ([9]), U˜ U = I, i.e., U = U˜, or equivalently X = X˜. Then B˜ = BQ with Q = V T V˜ is an RQ factorization of the lower bidiagonal matrix B˜. Hence, when WGKLU starts with y1 and WGKLL starts with x˜1 = Ky1/||Ky1||M , if both algorithms can run n iterations, the generated matrices satisﬁes X˜ = X, Y˜ = YQ. Since B˜T B˜ = QT BT BQ, it is not diﬃcult to see that Q is just the orthogonal matrix generated by running one QR iteration from BT B to B˜T B˜ with zero shift ([9]). Clearly, one has BBT = B˜B˜T . For any integer 1 ≤ k ≤ n, by comparing the leading k × k principal submatrices of BBT and B˜B˜T , one has T ˜ ˜T Bk βkek Bk βkek = BkBk . So the singular values of B˜k and Bk βkek are identical. Now we have four blocks ˜ Bk ˜ ˜ T , Bk, Bk βkek ,Bk βkek and the singular values of each can be used for eigenvalue approximations. Following the B˜k same arguments given in Remark 1, the squares of large singular values of ˜ T are βkek closer to the large eigenvalues of MK than those of B˜k. So they are also closer than those of Bk βkek and Bk. Similarly, the squares of small singular values of Bk are the closest to the small eigenvalues of MK among those of the above four blocks. Similarly, when WGKLL starts withx ˜1 and WGKLU starts with y1 =y ˜1 = Mx˜1/||Mx˜1||K , we have Y˜ = Y and X˜ = XQ˜ with orthogonal matrix Q˜ satisfying B = Q˜B˜. It can be interpreted that Q˜ is obtained by performing one QR iteration on B˜B˜T with zero shift. In this case, among the above four blocks Bk βkek , will provide the best approximations to the large eigenvalues of MK and B˜k will provide the best approximations to the small eigenvalues of MK.

4 Application to linear response eigenvalue problem

Consider the matrix 0 M H = , 0 < K, M ∈ n×n. (29) K 0 R The eigenvalue problem of such a matrix arises from linear response problem ([2, 3, 5, 11, 14, 19]). In practice, the extreme eigenvalues of H, in particular a few small positive eigenvalues, are of interest. From (7) and the fact that X is M-orthogonal and Y is K-orthogonal, for

Y 0 K 0 X = , K = , 0 X 0 M

13 one has 0 BT HX = XB, B = ; XT KX = I . B 0 2n So H is similar to the symmetric matrix B with an K-orthogonal transformation matrix X. Moreover, suppose B = ΦΛ˜ Ψ˜ T is an SVD of B. Deﬁne the symmetric orthogonal matrix 1 In In Pn = √ . 2 In −In Then Y Ψ˜ 0 Y Ψ˜ 0 Λ 0 H P = P . (30) 0 XΦ˜ n 0 XΦ˜ n 0 −Λ

Hence, ±λ1,..., ±λn are the eigenvalues of H. From (5) and (2), one has

Φ = UΦ˜, Ψ = V Ψ˜ .

Following (4) and (6), one has

−T −T −1 Y Ψ˜ = L Ψ = W =: ξ1 . . . ξn ,XΦ˜ = R Φ = KW Λ =: η1 . . . ηn .

By Lemma 2, ξ1, . . . , ξn form a complete set of K-orthonormal right eigenvectors and η1, . . . , ηn form a complete set of M-orthonormal left eigenvectors of MK corresponding to the eigen- 2 2 values λ1, . . . , λn, as given in both Theorems 3 and 4. Hence ˜ Y Ψ 0 1 ξ1 . . . ξ1 ξ1 . . . ξn + + − − Pn = √ =: x1 ,..., xn , x1 ,... xn , 0 XΦ˜ 2 η1 . . . ηn −η1 . . . ηn

± and xj , j = 1, . . . , n form a complete set of K-orthonormal right eigenvectors of H corresponding to the eigenvalues ±λj, for j = 1, . . . , n. Since

0 I 0 I HT = n H n , (31) In 0 In 0

T one can show that y± := √1 ±ηT ξT , j = 1, . . . , n form a complete set of M-orthonormal j 2 j j M 0 left eigenvectors of H corresponding to the eigenvalues ±λ , where M = . j 0 K In this section we apply the algorithms WGKLU and WGKLL to solve the eigenvalue problem of H. We only consider WGKLU, since the results about WGKLL parallel to the following results about WGKLU can be established in the same way. Let Xj,Yj,Bj be generated by Algorithm 1 after j iterations. Deﬁne

T Yj 0 0 Bj Xj = , Bj = . 0 Xj Bj 0

Then from (8), y HX = X B + β k+1 eT . (32) k k k k 0 2k Let ˜ Pk = e1 ek+1 e2 ek+2 . . . ek e2k .

14 Then y H(X P˜ ) = (X P˜ )(P˜ T B P˜ ) + β k+1 eT , (33) k k k k k k k k 0 2k ˜ T ˜ where Pk BkPk is a symmetric tridiagonal matrix with zero diagonal entries and ˜ y1 0 y2 0 . . . yk−1 0 yk 0 XkPk = . 0 x1 0 x2 ... 0 xk−1 0 xk

Using (11), one has

y range X = range X P˜ = K H, 1 . (34) k k k 2k 0

So, running k iterations of WGKLU is just the same as running 2k iterations of weighted T T Lanczos algorithm with H and an initial vector of a special form y1 0 . Suppose Bk has an SVD (13) with σ1 ≥ σ2 ≥ ... ≥ σk > 0. From (31) and (32), we may take ±σ1,..., ±σk as Ritz values of H, ± 1 Ykνj vj = √ , j = 1, . . . , k, 2 ±Xkµj as corresponding K-orthonormal right Ritz vectors, and ± 1 ±Xkµj uj = √ , j = 1, . . . , k 2 Ykνj as corresponding M-orthonormal left Ritz vectors. From (32), for any j ∈ {1, . . . , k}, ± ± βkµkj yk+1 T ± ± βkµkj 0 Hvj = ±σjvj + √ , H uj = ±σjuj + √ , 2 0 2 yk+1

T where µjk = ek µj. In practice, we may use the residual norm

+ + T + + 1 βk|µjk| ||Hv − σjv ||K = ||H u − σju ||M = √ ||MXkµj − σjYkνj||K = √ (35) j j j j 2 2 to design a stopping criterion. ˜ T ˜ ˜ T ˜ Remark 3. For (33), if (θi, gi), i = 1 : 2k are eigenpairs of Pk BkPk, i.e., Pk BkPkgi = θigi, here gi is orthonormal, then (θi, qi), i = 1 : k are approximate eigenpairs of H, where ˜ qi = XkPkgi, and

y kHq − θ q k = kβ k+1 eT g k = β |g |. (36) i i i K k 0 2k i K k ki

15 The convergence results can be established by using Theorems 3. ˜ Theorem 5. Let γj, γ˜j, πj, π˜j, πj,k, π˜j,k, δj, and δj be deﬁned as in Theorem 3. Then for j = 1, . . . , k

2 2 2 λ1 − λn πj,k tan θK (y1, ξj) 0 ≤ λj − σj = (−σj) − (−λj) ≤ , λj + σj Ck−j(1 + 2γj) 2 2 2 λ1 − λn π˜j,k tan θK (y1, ξn−k+j) 0 ≤ σj − λn−k+j = (−λn−k+j) − (−σj) ≤ ; λn−k+j + σj Cj−1(1 + 2˜γj) and for the Ritz vectors one has the bounds,

± ± ± ± sin θK vj ,, xj = sin θM uj , yj v uπ2(1 + (α β )2/δ2) 1 u j k k j 2 2 ≤ t 2 sin θK (y1, ξj) − sin %j cos %j Ck−j(1 + 2γj)

± ± ± ± sin θK vj , xn−k+j = sin θM uj , yn−k+j v u 2 2 ˜2 π˜ (1 + (αkβk) /δ ) u 2 2 j j 2 ≤ tsin %˜j + cos %˜j 2 sin θK (y1, ξn−k+j), Cj−1(1 + 2˜γj) where 2σj σj + λn−k+j %j = arccos , %˜j = arccos . λj + σj 2σj Proof. The ﬁrst two bounds are simply from (15) and (16). For the last two relations, it is trivial to have the equalities. So we only need to prove the upper bounds. T T Following (19) and the fact that ξj KYkνj and ηj MXkµj have the same sign, which is shown right above (19), 1 1 cos θ v±, x± = |ξT KY ν + ηT MX µ | = (cos θ (Y ν , ξ ) + cos θ (X µ , η )) K j j 2 j k j j k j 2 K k j j M k j j λj + σj = cos θK (Ykνj, ξj). 2σj

Since 0 ≤ 2σ /(λ + σ ) ≤ 1, cos % = 2σj is well deﬁned. Then from j j j j λj +σj

± ± cos θK (Ykνj, ξj) = cos %j cos θK vj , xj , one has

2 2 2 ± ± 2 2 2 ± ± sin θK (Ykνj, ξj) = 1 − cos %j cos θK vj , xj = sin %j + cos %j sin θK vj , xj .

Hence q ± ± 2 2 sin θK vj , xj = sin θK (Ykνj, ξj) − sin %j/ cos %j. The third bound then follows from (17). The last bound can be proved in the same way, by using the relation (20).

16 Remark 4. It is straightforward to show that the relation (33) is equivalent to

T T 0 (R L) ˜ ˜ ˜ T ˜ vk+1 T Vk 0 T (ZkPk) = (ZkPk)(Pk BkPk) + βk e2k, Zk = R L 0 0 0 Uk

T T T T with Vk = L Yk, Uk = R Xk, Zk Zk = I2k, and vk+1 = L yk+1. This is a relation resulting T T T in the standard symmetric Lanczos algorithm with the initial vector v1 0 , v1 = L y1. So we can establish the following convergence results: For j = 1, . . . , k,

2 πˆj,k tan θK (y1, ξj) 0 ≤ λj − σj = (−σj) − (−λj) ≤ 2λ1 C2k−j(1 + 2ˆγj) q 2 ˆ2 πˆj 1 + (αkβk) /δ ± ± j sin θK vj , xj ≤ sin θK (y1, ξj), C2k−j(1 +γ ˆj) where

j−1 j−1 λj − λj+1 Y σi + λ1 Y λi + λ1 ˆ γˆj = , πˆ1,k =π ˆ1 = 1, πˆj,k = , πˆj = , δj = min |λj−σi|. λ + λ σ − λ λ − λ i6=j j+1 1 i=1 i j i=1 i j

However, it seems that it is not trivial to derive a bound for σj − λn−k+j, since the small positive eigenvalues of H are the interior eigenvalues.

In [24], another type of Lanczos algorithms was proposed for solving the eigenvalue problem of H. The algorithm is based on the factorizations

T KU = VT,MV = UD,U V = In with the assumption that M > 0, where T is symmetric tridiagonal and D is diagonal, so that U 0 U 0 0 D H = . 0 V 0 V T 0 The first Lanczos-type algorithm in [24] computes the columns of U, V and the entries of D and T by enforcing the columns of V to be unit vectors. By running k iterations with the first column of V as an initial vector, the leading principal k × k submatrices Dk and 0 Dk Tk of D and T , respectively, are computed. Then the eigenvalues of are used to Tk 0 approximate the eigenvalues of H. This algorithm works even when K is indefinite. On the other hand, when K > 0, Algorithms 1 and 2 exploit the symmetry of the problem and seem more natural.

5 Numerical examples

In this section, we test Algorithm 1 (WGKLU) with several numerical examples for solving the eigenvalue problem of H, where the initial vector y1 is randomly selected satisfying ||y1||K = 1. The numerical results are labeled with WGKLU . For comparison we tested the ﬁrst algorithm presented in [24] with the initial vector y1/||y1||. The numerical results are labeled with Alg-TL. We also ran the weighted Lanczos algorithm based on the relation (33) with H

17 being treated as a full matrix and Xk being a K-orthonormal matrix. The initial vector is T T y1 0 . The numerical results are labeled with Alg-Full. For each example we ran k iterations with Alg-Full using a given even number k and k/2 iterations with WGKLU and Alg-TL. For each numerical example the matrices M and K were scaled with a positive diagonal matrix ! r k r k S = diag 4 11 ,..., 4 nn m11 mnn so that all the corresponding diagonal entries of S−1KS−1 and SMS are identical. The Lanczos-type algorithms were then applied to the scaled symmetric positive deﬁnite matrix pairs, or equivalently, the diagonally scaled matrix

S 0 S−1 0 H := H , 0 S−1 0 S which is similar to H. For each example we computed the two largest and the two smallest positive eigenvalues of H (and H). We report the relative errors e(σj) := |λj − σj|/λj for j = 1, 2, e(σj) := + + |λn−k+j − σj|/λn−k+j for j = k − 1, k, and the residual norms r(σj) := ||Hvj − σjvj ||K, for j = 1, 2, k − 1, k. For WGKLU and Alg-Full, we use (35) and (36) as the formula of residual norm respectively. And for Alg-TL, we compute the above residual norms directly. The “exact” eigenvalues λj, j = 1, 2, k − 1, k, are computed with MATLAB code eig. All the numerical testing results are computed by using Matlab 7.0 on a computer with an Intel Core i5 2.3GHz and a 4GB memory. Example 1. In this example, we tested the three algorithms with an artificially constructed linear response eigenvalue problem given in [24]. The matrices K and M are ex- tracted from the University of Florida sparse matrix collection [6]: K is fv1 with n = 9604, and M is the n × n leading principal submatrix of finan512. Both K and M are symmetric positive definite. The two smallest eigenvalues are 1.15374812063587, 1.16693189795791, and the two largest ones are 9.80009365291714, 9.74987369865853. We set k = 200 for the smallest positive eigenvalues and k = 50 for the largest eigenvalues. The testing results associated with the two smallest positive eigenvalues are shown in Figure 1, and the results associated with the two largest eigenvalues are shown in Figure 2. For the two smallest positive eigenvalues, WGKLU runs about 12.5 seconds, Alg-Full about 13.2 seconds, and Alg-TL about 47.1 seconds. For the two largest eigenvalues the time is about 0.28, 0.30, 0.46 seconds, respectively. Alg-TL needs to compute the eigenvalues of 0 D k , which is treated as a general nonsymmetric matrix. This is the part that slows Tk 0 Alg-TL down. On the other hand, the numerical results of Alg-Full is less accurate than the other two algorithms. Example 2. In this example, we tested the problem with positive definite matrices of K and M given in [24] about sodium dimer Na2. The order n = 1862. The two smallest eigenvalues are 0.63651164987557, 0.69203520406320, and the two largest ones are 11.19304749521232, 11.16696857225674 We set k = 150 for the smallest positive eigenvalues and k = 20 for the largest eigenvalues. The numerical results are reported in Figures 3 and 4, respectively, associated with the

18 Residual K−norm Relative eigenvalue errors 1 1 10 10

0 10

−1 10

−1 −2 10 10 r(σ ) by WGKLU e(σ ) by WGKLU n n

r(σ ) by WGKLU −3 e(σ ) by WGKLU n−1 10 n−1 σ σ −2 r( ) by Alg−Full e( ) by Alg−Full 10 n n r(σ ) by Alg−Full e(σ ) by Alg−Full n−1 −4 n−1 10 r(σ ) by Alg−TL e(σ ) by Alg−TL n n r(σ ) by Alg−TL e(σ ) by Alg−TL n−1 n−1 −3 −5 10 10 0 50 100 150 200 0 50 100 150 200 Lanczos step k Lanczos step k

Figure 1: Error and convergence with the smallest 2 positive eigenvalues in Example 1

Residual K−norm Relative eigenvalue errors 1 0 10 10

0 −2 10 10

−1 −4 10 10

−2 −6 10 10 r(σ ) by WGKLU e(σ ) by WGKLU 1 1 σ −3 r( ) by WGKLU −8 e(σ ) by WGKLU 10 2 10 2 r(σ ) by Alg−Full e(σ ) by Alg−Full 1 1 r(σ ) by Alg−Full e(σ ) by Alg−Full −4 2 −10 2 10 10 r(σ ) by Alg−TL e(σ ) by Alg−TL 1 1 r(σ ) by Alg−TL e(σ ) by Alg−TL 2 2 −5 −12 10 10 0 10 20 30 40 50 0 10 20 30 40 50 Lanczos step k Lanczos step k

Figure 2: Error and convergence with the largest 2 eigenvalues in Example 1

Residual K−norm Relative eigenvalue errors 1 2 10 10

0 0 10 10

−2 10 −1 10

−4 10 σ σ −2 r( ) by WGKLU e( ) by WGKLU 10 n n r(σ ) by WGKLU −6 e(σ ) by WGKLU n−1 10 n−1 r(σ ) by Alg−Full e(σ ) by Alg−Full n n −3 r(σ ) by Alg−Full e(σ ) by Alg−Full 10 n−1 −8 n−1 10 r(σ ) by Alg−TL e(σ ) by Alg−TL n n r(σ ) by Alg−TL e(σ ) by Alg−TL n−1 n−1 −4 −10 10 10 0 50 100 150 0 50 100 150 Lanczos step k Lanczos step k

Figure 3: Error and convergence with the smallest 2 positive eigenvalues in Example 2 two smallest positive and two largest eigenvalues. For the two smallest positive eigenvalues, WGKLU runs about 5.5 seconds, Alg-Full about 15.8 seconds, and Alg-TL about 24.3 seconds.

19 Residual K−norm Relative eigenvalue errors 2 0 10 10

−2 10 0 10 −4 10

−6 −2 10 10

−8 10 r(σ ) by WGKLU e(σ ) by WGKLU −4 1 1 10 −10 r(σ ) by WGKLU 10 e(σ ) by WGKLU 2 2 σ σ r( ) by Alg−Full −12 e( ) by Alg−Full 1 10 1 −6 r(σ ) by Alg−Full e(σ ) by Alg−Full 10 2 2 r(σ ) by Alg−TL −14 e(σ ) by Alg−TL 1 10 1 r(σ ) by Alg−TL e(σ ) by Alg−TL 2 2 −8 −16 10 10 0 5 10 15 20 0 5 10 15 20 Lanczos step k Lanczos step k

Figure 4: Error and convergence with the largest 2 eigenvalues in Example 2

For the two largest eigenvalues the time is about 0.57, 1.65, 1.59 seconds, respectively. As in Example 1, WGKLU and Alg-TL provide the numerical results with comparable accuracy, whereas Alg-Full provides less accurate results.

6 Connection with conjugate gradient methods

Consider the system of linear equations

Mz = b, M > 0. (37)

−1 Let z0 be an initial guess of the solution ze = M b and r0 = b − Mz0 = M(ze − z0) be the corresponding residual. Assume that Xk,Yk and Bk are computed by WGKLU with M and another positive deﬁnite matrix K and y1 = r0/||r0||K . Then they satisfy (8) and (11). We approximate the solution ze with a vector zk ∈ z0 + KKk(MK, y1) for 1 ≤ k ≤ n. From (11), we may express zk = z0 + Xkwk k for some wk ∈ R . We take the approximation zk (or equivalently wk) as the minimizer of the minimization problem

T min J(wk),J(wk) = εk Mεk, εk = ze − zk = ε0 − Xkwk. wk Since

T T T T T T T T T T J(wk) = wk Xk MXkwk − 2wk Xk Mε0 + ε0 Mε0 = wk Xk MXkwk − 2wk Xk r0 + ε0 Mε0,

J(wk) is minimized when wk satisﬁes

T T Xk MXkwk = Xk r0.

T T Note that Xk MXk = Ik. Using r0 = ||r0||K y1, Yk KYk = Ik, and the ﬁrst relation of (8), one has T −1 T −T T −T Xk r0 = ||r0||K (KYkBk ) y1 = ||r0||K Bk Yk Ky1 = ||r0||K Bk e1.

20 Hence the minimizer is

−T zk = z0 + Xkwk, with wk = ||r0||K Bk e1.

The vector wk can be computed in an iterative way following the iterations of WGKLU. Note that   α1  β1 α2  BT 0 BT =   = k−1 . k  .. ..  T  . .  βk−1ek−1 αk βk−1 αk So " # B−T 0 B−T = k−1 k − βk−1 eT B−T α−1 αk k−1 k−1 k T and denoting wk = ϕ1 . . . ϕk , one has

" −T # −T Bk−1e1 wk−1 wk = ||r0||K Bk e1 = ||r0||K βk−1 T −T = , − e B e1 ϕk αk k−1 k−1

where ϕk follows the iteration

βk−1 T βk−1 ϕk = − ek−1wk−1 = − ϕk−1, k ≥ 1; β0 = 1, ϕ0 = −||r0||K . (38) αk αk Therefore, zk = z0 + Xkwk = zk−1 + ϕkxk, k ≥ 1, T and using Bk wk = ||r0||K e1 and the second relation in (8), the corresponding residual is T T rk = b − Mzk = r0 − MXkwk = r0 − (YkBk + βkyk+1ek )wk = r0 − ||r0||K Yke1 − βkϕkyk+1 = −βkϕkyk+1, k ≥ 0,. So we have the following algorithm for solving (37). Algorithm 3 (WGKLULin). Choose z0 and compute r0 = b − Mz0, ϕ0 = −||r0||K , and y1 = r0/||r0||K . Set β0 = 1, x0 = 0. Compute g1 = Ky1. For j = 1, 2, ··· sj = gj/βj−1 − βj−1xj−1 fj = Msj T 1 αj = (sj fj) 2 xj = sj/αj ϕj = −βj−1ϕj−1/αj zj = zj−1 + ϕjxj tj+1 = fj/αj − αjyj gj+1 = Ktj+1 T 1 βj = (tj+1gj+1) 2 yj+1 = tj+1/βj rj = −ϕjtj+1 End

21 We now show this algorithm is equivalent to a weighted conjugate gradient (CG) method. By introducing the vectors 2 pk−1 = αkϕkxk, k ≥ 1 with 2 p0 = α1ϕ1x1 = α1ϕ1Ky1 = ||r0||K Ky1 = Kr0. Then T 4 2 T 4 2 pk−1Mpk−1 = αkϕkxk Mxk = αkϕk. Since rk = −βkϕkyk+1, using (38) one has

T 2 2 T 2 2 2 2 rk Krk = βkϕkyk+1Kyk+1 = βkϕk = αk+1ϕk+1. We then have 4 2 T 2 αk+1ϕk+1 pk Mpk αk+1 = 2 2 = T , αk+1ϕk+1 rk Krk Now,

T −2 −2 rk−1Krk−1 zk = zk−1 + ϕkxk = zk−1 + αk pk−1 = zk−1 + γk−1pk−1, γk−1 = αk = T , pk−1Mpk−1 and rk = b − Mzk = rk−1 − γk−1Mpk−1.

By multiplying αk+1ϕk+1 to the equation

Kyk+1 = βkxk + αk+1xk+1,

and using (38) one has

2 pk = αk+1ϕk+1xk+1 = −αk+1βkϕk+1xk + αk+1ϕk+1Kyk+1 2 2 βk = βkϕkxk − βkϕkKyk+1 = 2 pk−1 + Krk. αk Since 2 2 2 T βk βkϕk rk Krk ϑk−1 := 2 = 2 2 = T , αk αkϕk rk−1Krk−1 we have T rk Krk pk = Krk + ϑk−1pk−1, ϑk−1 = T . rk−1Krk−1 So we have the simpliﬁed version of Algorithm 4. Algorithm 4 (WCG). Choose z0 and compute r0 = b − Mz0 and p0 = Kr0. For j = 0, 1, 2, ··· T T γj = rj Krj/pj Mpj zj+1 = zj + γjpj

22 rj+1 = rj − γjMpj T T ϑj = rj+1Krj+1/rj Krj pj+1 = Krj+1 + ϑjpj End We call Algorithm 4 the weighted CG algorithm, since for K = LLT if we set

T −1 −1 r˜j = L rj, p˜j = L pj, z˜j = L zj, then it is not diﬃcult to see that the above algorithm is nothing but the standard CG algorithm applied to the symmetric linear system

Az˜ = ˜b, A = LT ML, z˜ = L−1z, ˜b = LT b.

In particular, when K = I, Algorithm 4 is just the standard CG. Therefore, the algorithm WGKLU provides a simple way connecting to the standard CG algorithm compared to the common symmetric Lanczos algorithm connection. In this case the residual vectors r0, . . . , rk−1 are orthogonal and they are parallel to the columns of Yk. The search direction vectors p0, . . . , pk−1 are M-orthogonal and they are parallel to columns of Xk. WGKLU just provides an iterative way to compute both Xk and Yk. Similarly, the algorithm WGKLL can be employed to solve (37) as well.

7 Conclusions

We proposed two weighted Golub-Kahan-Lanczos bidiagonalization algorithms associated with two symmetric positive definite matrices K and M. The algorithms can be implemented 0 M naturally to solve the large scale eigenvalue problems of MK and the matrix H = . K 0 For these eigenvalue solvers, convergence results have been established. We also showed a simple connection between the algorithm WGKLU and the standard CG method. The proposed algorithms are still not ready for practical use. For instance, it is not clear how weighted orthogonality will affect convergence behavior, in particular in the presence of rounding errors. In order to make the algorithms more efficient and capable of dealing with different kind of related problems, other techniques have to be incorporated. All these require further investigations. Acknowledgment. The second author was supported in part by China Scholarship Council (CSC) during a visit at Department of Mathematics, University of Kansas, and the National Science Foundation of China under grant 11471122.

References

[1] M. Arioli, Generalized Golub-Kahan bidiagonalization and stopping criteria, SIAM J. Matrix Anal. Appl., 34:571–592, 2013.

[2] Z.-J. Bai and R.-C. Li, Minimization principles for the linear response eigenvalue problem I: Theory, SIAM J. Matrix Anal. Appl., 33:1075–1100, 2012.

[3] Z.-J. Bai and R.-C. Li, Minimization principles for the linear response eigenvalue problem II: Computation, SIAM J. Matrix Anal. Appl., 34: 392–416, 2013.

23 [4] S. J. Benbow, Solving generalized least-squares problems with LSQR. SIAM J. Matrix Anal. Appl. 21:166–177,1999.

[5] M. E. Casida, Time-dependent density-functional response theory for molecules, in Recent Advances in Density Functional Methods, D. P. Chong, ed., World Scientiﬁc, Singapore, pp:155–189, 1995.

[6] T. Davis and Y. Hu, The university of Florida sparse matrix collection, ACM Trans. Math. Software, 38: 1–25, 2011.

[7] A. Essai, Weighted FOM and GMRES for solving nonsymmetric linear systems, Numer. Algorithm, 18: 277–292, 1998.

[8] G. H. Golub and W. Kahan, Calculating the singular values and pseudo-inverse of a matrix, J. SIAM Ser. B Numer. Anal., 2 :205224, 1965.

[9] G. H. Golub and C. F. Van loan, Matrix Computations. 3rd Edition. Johns Hopkins University Press, Baltimore, MD, 1996.

[10] M. Gr¨uning, A. Marini, and X. Gonze, Exciton-plasmon states in nanoscale materials: Breakdown of the Tamm-Dancoﬀ approximation, Nano Letters, 9:2820–2824, 2009.

[11] M. J. Lucero, A. M. N. Niklasson, S. Tretiak, and M. Challacombe, Molecular-orbitalfree algorithm for excited states in time-dependent perturbation theory, J. Chem. Phys., 129:064114, 2008.

[12] M. T. Lusk and A. E. Mattsson, High-performance computing for materials design to advance energy science, MRS Bulletin, 36:169–174, 2011.

[13] C. Mehl, V. Mehrmann, and H. Xu, On doubly structured matrices and pencils that arise in linear response theory, Linear Algebra Appl., 380:3–51, 2004.

[14] G. Onida, L. Reining, and A. Rubio, Electronic excitations: Density-functional versus many-body Green’s function approaches, Rev. Modern Phys., 74:601–659, 2002.

[15] C. C. Paige, Bidiagonalization of matrices and solution of linear equations, SIAM J. Numer. Anal., 11:197–209, 1974.

[16] C. C. Paige and M. Saunders, LSQR: An algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software, 8:43–71, 1982.

[17] P. Papakonstantinou, Reduction of the RPA eigenvalue problem and a generalized Cholesky decomposition for real-symmetric matrices, EPL (Europhys. Lett.), 78:12001, 2007.

[18] B. N. Parlett, The Symmetric Eigenvalue Problem. SIAM, Philadelphia, 1998.

[19] D. Rocca, Time-dependent density functional perturbation theory: new algorithms with applications to molecular spectra. Ph.D. Thesis, The International School for Advanced Studies, Trieste, Italy, 2007.

24 [20] D. Rocca, D. Lu, and G. Galli, Ab initio calculations of optical absorpation spectra: Solution of the Bethe-Salpeter equation within density matrix perturbation theory, J. Chem. Phys., 133:164109, 2010.

[21] Y. Saad, Iterative Methods for Sparse Linear Systems. 2nd Edition. SIAM, Philadelphia, 2003.

[22] Y. Saad, Numerical Methods for Large Eigenvalue Problems. 2nd Edition. SIAM, Philadelphia, 2011.

[23] Y. Saad, J. R. Chelikowsky, and S. M. Shontz, Numerical methods for electronic structure calculations of materials, SIAM Rev., 52:3–54, 2010.

[24] Z.-M. Teng, and R.-C. Li, Convergence analysis of Lanczos-type methods for the linear response eigenvalue problem, J. Comput. Appl. Math., 247:17–33, 2013.

[25] E. V. Tsiper, A classical mechanics technique for quantum linear response, J. Phys. B: At. Mol. Opt. Phys., 34:L401–L407, 2001.