Weighted Golub-Kahan-Lanczos algorithms and applications Hongguo Xu∗ and Hong-xiu Zhongy January 13, 2016 Abstract We present weighted Golub-Kahan-Lanczos algorithms for computing a bidiagonal form of a product of two invertible matrices. We give convergence analysis for the al- gorithms when they are used for computing extreme eigenvalues of a product of two symmetric positive definite matrices. We apply the algorithms for computing extreme eigenvalues of a double-sized matrix arising from linear response problem and provide convergence results. Numerical testing results are also reported. As another application we show a connection between the proposed algorithms and the conjugate gradient (CG) algorithm for positive definite linear systems. Keywords: Weighted Golub-Kahan-Lanczos algorithm, Eigenvalue, Eigenvector, Ritz value, Ritz vector, Linear response eigenvalue problem, Krylov subspace, Bidiagonal matrices. AMS subject classifications: 65F15, 15A18. 1 Introduction The Golub-Kahan bidiagonalization procedure is a fundamental tool in the singular value decomposition (SVD) method ([8]). Based on such a procedure, Golub-Kahan-Lanczos al- gorithms were developed in [15] that provide powerful Krylov subspace methods for solving large scale singular value and related eigenvalue problems, as well as least squares and saddle- point problems, [15, 16]. Later, weighted Golub-Kahan-Lanczos algorithms were introduced for solving generalized least squares and saddle-point problems, e.g., [1, 4]. In this paper we propose a type of weighted Golub-Kahan-Lanczos bidiagonalization al- gorithms. The proposed algorithms are similar to the generalized Golub-Kahan-Lanczos bidi- agonalization algorithms given in [1]. There, the algorithms are based on the Golub-Kahan bidiagonalizations of matrices of the form R−1AL−T , whereas ours are based the bidago- nalizations of RT L, where R; L are invertible matrices. When R and L are factors of real symmetric positive definite matrices: K = LLT , M = RRT , similar to those in [1], the pro- posed algorithms use the matrices K and M directly without computing the factors L and R. Therefore, the algorithms will be advantageous when K and M are large sparse while L and R may be not. We demonstrate that the algorithms can be simply employed for solving large scale eigen- value problems of the matrices in the product form MK. As an application, we also use 1Department of Mathematics, University of Kansas, Lawrence, KS, 66046, USA. E-mail: [email protected]. 2Department of Mathematics, East China Normal University, Shanghai, 200241, P.R. China. E-mail: [email protected] and [email protected]. Supported by National Science Foundation of China under grant 11471122 and by China Scholarship Council (CSC) during a visit at University of Kansas. 1 the proposed algorithms for solving the eigenvalue problem of the matrices in the form 0 M H = with symmetric positive definite K and M. The eigenvalue problem of K 0 such a matrix arises from linear response problem in the time-dependent density functional theory [5, 11, 14, 19]. Related eigenvalue properties and numerical eigenvalue methods can be founded in [2, 3, 10, 12, 13, 20, 23, 24, 25]. For a positive definite linear system it is well known that the conjugate gradient (CG) method is equivalent to the standard Lanczos method, e.g., [21, Sec. 6.7] and [9]. As another application, we demonstrate that in the case when K or M is identity, the Krylov subspace linear system solver based on one of the proposed algorithms has a simpler and more direct connection to the CG method. In its original version (when neither K nor M is identity), the solver is mathematically equivalent to a weighted CG method. The paper is organized as follows. In Section 2 we present some basic results that are needed for deriving our algorithms and main results. In Section 3, we construct the weighted Golub-Kahan-Lanczos algorithms and show how the algorithms can be employed to solve the eigenvalue problem of a matrix as a product of two symmetric positive definite matrices. Application of the weighted Golub-Kahan-Lanczos algorithms to the eigenvalue problem of matrix H from linear response problem is described in Section 4, and numerical examples are given in Section 5. In Section 6, the relation between a weighted Golub-Kahan-Lanczos algo- rithm and a weighted CG method is provided. Section 7 contains some concluding remarks. m×n n Throughout the paper, R is the real field, R is the set of m × n real matrices, R is the n-dimensional real vector space, In is the n × n identity matrix, and ej is its jth column. The notation A > 0(≥ 0) means that the matrix A is symmetric and positive definite n×n n (semidefinite). For a given matrix A 2 R and a vector b 2 R , the kth Krylov subspace of A with b, i.e., the subspace spanned by the set of k vectors fb; Ab; : : : ; Ak−1bg, is denoted by Kk(A; b). jj · jj is the spectral norm for matrices and 2-norm for vectors. Forp a symmetric T positive definite matrix A, jj · jjA denotes the vector norm defined by jjxjjA = x Ax for any vector x, and a matrix X is A-orthonormal if XT AX = I (it is A-orthogonal if X is a square matrix). A set of vectors fx1; : : : ; xkg is also called A-orthonormal if X = x1 : : : xk is T A-orthonormal and A-orthogonal if xi Axj = 0 for i 6= j. σmax(A) and σmin(A) are the largest and the smallest singular values of matrix A, respectively; and κ2(A) = σmax(A)/σmin(A) (when σmin(A) > 0) is the spectral condition number of A. In the paper we restrict ourselves to the real case. All the results can be simply extended to the complex case. 2 Preliminaries n×n Suppose K; M 2 R are both symmetric and positive definite. There exist invertible matrices L and R such that K = LLT ;M = RRT : (1) Consider the eigenvalue problem of the matrix products MK or KM = (MK)T . Since MK = RRT LLT = L−T [(RT L)T (RT L)]LT = R[(RT L)(RT L)T ]R−1; MK is similar to the symmetric positive definite matrix (RT L)T (RT L) (and (RT L)(RT L)T ). Let RT L = ΦΛΨT (2) 2 be a singular value decomposition (SVD), where Φ and Ψ are real orthogonal and Λ = T diag(λ1; : : : ; λn) with λ1 ≥ ::: ≥ λn > 0 (since R L is invertible). Then MK = (L−T Ψ)Λ2(L−T Ψ)−1 = (RΦ)Λ2(RΦ)−1: (3) Therefore, we have the following basic properties. Lemma 1. Let RT L have an SVD (2). Then 2 2 (a) The eigenvalues of MK (and KM) are λ1; : : : ; λn. (b) The columns of L−T Ψ and RΦ = L−T ΨΛ are the corresponding right (left) eigenvectors associated with MK (KM), and L−T Ψ is K-orthogonal and RΦ is M −1-orthogonal. (c) The columns of LΨ = R−T ΦΛ and R−T Φ are the corresponding left (right) eigenvectors associated with MK (KM), and LΨ is K−1-orthogonal and R−T Φ is M-orthogonal. Proof. (a) is simply from (3) and the fact KM = (MK)T . (b) is also from (3) and the fact that (L−T Ψ)T K(L−T Ψ) = I = (RΦ)T M −1(RΦ): The relation RΦ = L−T ΨΛ is from (2). (c) can be proved in the same way as for (b) by considering KM = (LΨ)Λ2(LΨ)−1 = (R−T Φ)Λ2(R−T Φ)−1; which is obtained by taking the transpose on (3). Lemma 1 shows that the matrix MK has a complete set of K-orthonormal right eigen- vectors and a complete set of M −1-orthonormal right eigenvectors. It has also a complete set of K−1-orthonormal left eigenvectors and a complete set of M-orthonormal left eigenvectors. These four sets of eigenvectors are related and the relations can be established without using the orthogonal matrices Φ and Ψ, as shown in the following lemma. Lemma 2. Suppose W = ξ1 ξ2 : : : ξn and ξ1; : : : ; ξn are K-orthonormal right eigen- 2 2 2 vectors of MK corresponding to the eigenvalues λ1; : : : ; λn so that MKW = W Λ . Then −1 (a) the columns of W Λ = λ1ξ1 λ2ξ2 : : : λnξn form a complete set of M -orthonormal right eigenvectors of MK; −1 2 (b) the columns of KW = Kξ1 Kξ2 : : : Kξn = M W Λ form a complete set of K−1-orthonormal left eigenvectors of MK; −1 −1 −1 −1 −1 (c) the columns of KW Λ = λ1 Kξ1 λ2 Kξ2 : : : λn Kξn = M W Λ form a complete set of M-orthonormal left eigenvectors of MK. Proof. The claim in (a) follows from the equivalence relation MKW = W Λ2 if and only if MK(W Λ) = (W Λ)Λ2, and the fact that KW = M −1W Λ2 implies I = W T KW = W T M −1W Λ2, which is equivalent to (W Λ)T M −1(W Λ) = I. The claim in (b) follows from the equivalence relation MKW = W Λ2 if and only if KM(KW ) = (KW )Λ2 and the fact (KW )T K−1(KW ) = W T KW = I. The claim in (c) follows from the relations KM(KW Λ−1) = (KW Λ−1)Λ2 and (KW Λ−1)T M(KW Λ−1) = Λ−1W T K(MKW )Λ−1 = Λ−1W T KW Λ2Λ−1 = I: 3 Therefore, once one of the four complete sets is known, there other three sets can be simply constructed. From Lemma 1, one can choose W = L−T Ψ when Ψ is known.