DOCSLIB.ORG

A Compact Arnoldi Algorithm for Polynomial Eigenvalue Problems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Compact Arnoldi Algorithm for Polynomial Eigenvalue Problems A compact Arnoldi algorithm for polynomial eigenvalue problems Yangfeng Su, Junyi Zhang, Zhaojun Bai School of Mathematical Sciences Fudan University January, 2008, Taiwan RANMEP2008 Goals Polynomial Eigenvalue Problems (PEPs): d A0 + λA1 + ... + λ Ad x = 0 (λ, x) is called an eigenpair. d=1, SEP,GEP d=2, QEP ( Quadratic Eigenvalue Problem) Goals Solving large scale polynomial eigenvalue problems with Implicitly Restarted Arnoldi (IRA) algorithm with less memory. Goals Polynomial Eigenvalue Problems (PEPs): d A0 + λA1 + ... + λ Ad x = 0 (λ, x) is called an eigenpair. d=1, SEP,GEP d=2, QEP ( Quadratic Eigenvalue Problem) Goals Solving large scale polynomial eigenvalue problems with Implicitly Restarted Arnoldi (IRA) algorithm with less memory. Outline 1 Background 2 Algorithm 3 Numerical Comparison Outline 1 Background 2 Algorithm 3 Numerical Comparison Linearization Linearization ⇒ A larger GEP: A0 −A1 −A2 ... −Ad x I I λx − λ = 0 .. .. . . . . I I 0 λd−1x Sleijpen, van der Vorst, and van Gijzen. Quadratic eigenproblems are no problem, SIAM News, 1996 Can be solved with ARPACK Not true: structure is neither used nor preserved. size and therefore memory are d times! This work: explore the structure to save memory in ARPACK. It is a byproduct of SOAR [Bai & S. SIMAX 2005]. Linearization Linearization ⇒ A larger GEP: A0 −A1 −A2 ... −Ad x I I λx − λ = 0 .. .. . . . . I I 0 λd−1x Sleijpen, van der Vorst, and van Gijzen. Quadratic eigenproblems are no problem, SIAM News, 1996 Can be solved with ARPACK Not true: structure is neither used nor preserved. size and therefore memory are d times! This work: explore the structure to save memory in ARPACK. It is a byproduct of SOAR [Bai & S. SIMAX 2005]. Linearization Linearization ⇒ A larger GEP: A0 −A1 −A2 ... −Ad x I I λx − λ = 0 .. .. . . . . I I 0 λd−1x Sleijpen, van der Vorst, and van Gijzen. Quadratic eigenproblems are no problem, SIAM News, 1996 Can be solved with ARPACK Not true: structure is neither used nor preserved. size and therefore memory are d times! This work: explore the structure to save memory in ARPACK. It is a byproduct of SOAR [Bai & S. SIMAX 2005]. Linearization Linearization ⇒ A larger GEP: A0 −A1 −A2 ... −Ad x I I λx − λ = 0 .. .. . . . . I I 0 λd−1x Sleijpen, van der Vorst, and van Gijzen. Quadratic eigenproblems are no problem, SIAM News, 1996 Can be solved with ARPACK Not true: structure is neither used nor preserved. size and therefore memory are d times! This work: explore the structure to save memory in ARPACK. It is a byproduct of SOAR [Bai & S. SIMAX 2005]. Arnoldi decomposition An Arnoldi decomposition of order-j: T OPQj = Qj Hj + hj+1,j qj+1ej where OP: a matrix Qj = [q1, q2,..., qj ]: orthonormal Hj : upper Hessenberg Arnoldi process is used to compute the Arnoldi decomposition with numerical stability Implicitly Restarted Arnoldi (IRA) for SEP Sorensen [SIMAX92] Given an Arnoldi decomposition of order-p, T OPQp = QpHp + βpqp+1ep . 1 Extend Arnoldi decomposition from order p to order k: T AQk = Qk Hk + βk qk+1ek . 2 Divide the eigenvalues of Hk as “good” ones µ1, . , µp and “bad” ones µp+1, . , µk . 3 For Hk do implicitly QR steps with shifts µk+1, . , µk , get H Hk = UH˜k U 4 Take first p columns of ˜ T AQk U = Qk UHk + βk qk+1ek U as a restarted Arnoldi decomposition of order p. ARPACK ARPACK is an implementation of IRA algorithm a well-coded, well-documented package produced by Lehoucq, Sorensen and Yang during 1992-1997 used in MATLAB as eigs and arpackc IRA for QEP For simplicity, we only discuss QEPs. For QEP: (λ2M + λD + K)x = 0 1 shift-and-invert: for shift σ, let λ = σ + 1/µ (µ2I − µA − B)x = 0 where A = −(σ2M + σD + K)−1(2σM + D) B = −(σ2M + σD + K)−1M 2 linearize AB µx µx = µ I 0 x x 3 apply IRA IRA for QEP Easy use of ARPACK How to utilize the Frobenius structure to save memory? Outline 1 Background 2 Algorithm 3 Numerical Comparison Arnoldi decomposition for QEP An Arnoldi decomposition with order-j OPQj = Qj+1Hbj For QEP: AB Qj,1 Qj+1,1 = Hbj I 0 Qj,2 Qj+1,2 Since Qj,1 = Qj+1,2Hbj we have Theorem rank ([Qj,1, Qj,2]) ≡ rj ≤ j + 1 Observed by many people, e.g. Meerbergen [SIMAX06], Bai & S. [SIMAX05]. The key is how to use it with numerical stability. Arnoldi Decomposition for QEP Theorem rank ([Qj,1, Qj,2]) ≡ rj ≤ j + 1 Let n×rj Vj ∈ C = orth[Qj,1, Qj,2] then Qj,1 Vj Rj,1 Vj Rj,1 Qj = = = Qj,2 Vj Rj,2 Vj Rj,2 Two levels of orthonormality: Vj is orthonormal R j,1 is orthonormal Rj,2 Compact ARnoldi Decomposition (CARD) Compact ARnoldi Decomposition (CARD) Vj Rj,1 Vj+1 Rj+1,1 OP = Hbj Vj Rj,2 Vj+1 Rj+1,2 n×r Vj ∈ C j r ×j Rj ∈ C j j ≤ rj+1 ≤ j + 1 Memory cost: Arnoldi: 2n(j + 1) ( for PEPs: dn(j + 1)) CARD: nrj+1 ≤ n(j + 2) (for PEPs: ≤ n(j + d + 1)) CARD process CARD process is to compute the CARD with numerical stability! CARD of order j: VR1 VR1 Vˆ Rˆ1 OP = H + βqe1 = Hˆ VR2 VR2 Vˆ Rˆ2 Expand it to a CARD of order j + 1 ⇒ next two pages Expand CARD process 1 compute q1 = Aq1 + Bq2; 2 decompose q1 = Vˆ x + vα with MGS T x = Vˆ q1, v = q1 − Vˆ x, α = kvk, v = v/α 3 update h i Vˆ = Vˆ , v , rj+1 = rj + 1, Rˆ x Rˆ Rˆ (:, j + 1) Rˆ = 1 , Rˆ = 2 2 1 0 α 2 0 0 Expand CARD process Rˆ1 x ˆ R1 0 α := Rˆ2 Rˆ2 Rˆ1 (:, j + 1) 0 0 4 decompose with MGS: Rˆ (:, j + 2) Rˆ (:, 1 : j + 1) 1 = 1 H Rˆ (:, j + 2) Rˆ (:, 1 : j + 1) 1:j+1,j+1 2 old 2 Rˆ (:, j + 2) + 1 H Rˆ (:, j + 2) j+2,j+1 2 new 5 update the current Arnoldi vector q: q1 q = q2 q1 = Vˆ Rˆj+1,1 [:, j + 2] q2 = Vˆ Rˆj+1,2 [:, j + 2] Only GMS (with re-orthogonalization), no inversion CARD is numerically stable! IRA with CARD Given a CARD of k-order: VR1 Vˆ Rˆ1 H OP = ˆ ˆ T VR2 V R2 βek IRA does (m − p) QR steps on H with shifts µp+1, ..., µm, i.e. H = UHUe H Then VR1 Vˆ Rˆ1 UH˜ OP U = ˆ ˆ T VR2 V R2 βek U Denote U Uˆ = 1 IRA with CARD Then VR1U Vˆ Rˆ1Uˆ H˜ OP = ˆ ˆ ˆ T VR2U V R2U βek U Its first p columns , denoted by Vk R1,p Vk+1R1,p+1 Hp OP = ˜ T Vk R2,p Vk+1R2,p+1 βep still form an Arnoldi decomposition of order p However, the Vk has rk (instead of rp) columns, it is not a CARD! IRA with CARD Since Vk+1R1,p+1 Vk+1R2,p+1 is the orthonormal factor of an Arnoldi decomposition, from previous theorem, rank[Vk+1R1,p+1, Vk+1R2,p+1] = rank[R1,p+1, R2,p+1] = rp+1 ≤ p + 2, we have a compact SVD: rk+1×rp+1 rp+1×rp+1 rp+1×(p+1) rp+1×(p+1) [R1,p+1, R2,p+1] = P Σ [G1 , G2 ] ≡ P[R1, R2] IRA with CARD Therefore, n×r V k+1 R (V P)n×rp+1 R k+1 1,p+1 = k+1 1 Vk+1R2,p+1 (Vk+1P)R2 The Arnoldi decomposition is expressed in CARD again! This process can also be implemented by a compact “QR” decomposition, which is similar with the compact Arnoldi decomposition (CARD). Details omitted. POLYAR POLYAR: modified ARPARK for polynomial eigenvalue problems (not only QEPs) 1 znaitr p: compute CARD 2 znapps p: IRA with CARD; use LAPACK routine zgesdd to compute SVD decomposition 3 znaupd p, znaupd2 p, zgetv0 p: slightly revised (arguments, storage) 4 zgemip(added): compute inner product in compact form Outline 1 Background 2 Algorithm 3 Numerical Comparison Example 1: A random QEP Problem: QEP (pdeg=2) Size: n=500 Environment: PC (EMS memory: 512M) Randomized Matrix M,D,K (each matrix have about 24,000 non-zero elements.) We choose shift σ = 1 and use shift-invert mode to compute eigenvalues close to 1 LU factorization of Mσ2 + Dσ + K, L and U contain about 120,000 non-zero elements. To compute 8 eigenvalues close to 1; Use 30 Arnoldi base vectors, says, 31 CARD base vectors. A random QEP Computed eigenvalues: ARPACK POLYAR Real Imag Real Imag 1 1.02817D+00 1.38768D−01 1.02817D+00 1.38768D-01 2 1.11582D+00 3.61818D-02 1.11582D+00 3.61818D-02 3 1.11582D+00 -3.61818D-02 1.11582D+00 -3.61818D-02 4 1.05613D+00 -1.05380D-01 1.05613D+00 -1.05380D-01 5 1.05613D+00 1.05380D-01 1.05613D+00 1.05380D-01 6 9.34692D-01 -2.73028D-15 9.34692D-01 4.39496D-15 7 1.00023D+00 5.87804D-02 1.00023D+00 5.87804D-02 8 1.00023D+00 -5.87804D-02 1.00023D+00 -5.87804D-02 A random QEP Storage comparison: ARPACK POLYAR V 500 × 2 × 30 500 × 31 workd 3 × 2 × 500 (2 + 2) × 500 Resid 500 × 2 500 × 2 A random QEP Iteration and time comparison: ARPACK POLYAR update iteration 4 4 OP × x operations 86 86 reorthogonalization of V 84 84 reorthogonalization of R 0 84 user’s OP × x operations 1.375000 1.250000 naupd2 1.609375 1.515625 basic Arnoldi iteration loop 1.531250 1.437500 reorthogonalization phrase 0.093750 0.046875 Hessenberg eig subproblem 0.031250 0.031250 applying the shifts 0.046875 0.046875 calling gesdd 0 0.000000 Example 2: A QEP from SLAC Problem: QEP (pdeg=2) Size: n=5384 Environment: PC (EMS memory: 512M) Genuine Matrix M,D,K (M,D,K have 61425,1183,61425 non-zero elements respectively.) We choose shift σ = −10i and use shift-invert mode to compute eigenvalues close to −10i LU factorization of Mσ2 + Dσ + K, L and U have 749610 and 780229 non-zero elements.
Load more

Recommended publications
  • A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe Wu, Serge Petiton
    A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe Wu, Serge Petiton To cite this version: Xinzhe Wu, Serge Petiton. A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems. HPC Asia 2018 - International Conference on High Performance Computing in Asia-Pacific Region, Jan 2018, Tokyo, Japan. 10.1145/3149457.3154481. hal-01677110 HAL Id: hal-01677110 https://hal.archives-ouvertes.fr/hal-01677110 Submitted on 8 Jan 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe WU Serge G. Petiton Maison de la Simulation, USR 3441 Maison de la Simulation, USR 3441 CRIStAL, UMR CNRS 9189 CRIStAL, UMR CNRS 9189 Université de Lille 1, Sciences et Technologies Université de Lille 1, Sciences et Technologies F-91191, Gif-sur-Yvette, France F-91191, Gif-sur-Yvette, France [email protected] [email protected] ABSTRACT the iteration number. These methods are already well implemented Parallel Krylov Subspace Methods are commonly used for solv- in parallel to profit from the great number of computation cores ing large-scale sparse linear systems.
    [Show full text]
  • Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations
    NASA Contractor Report 198342 /" ICASE Report No. 96-40 J ICA IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen NASA Contract No. NASI-19480 May 1996 Institute for Computer Applications in Science and Engineering NASA Langley Research Center Hampton, VA 23681-0001 Operated by Universities Space Research Association National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-0001 IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen 1 Department of Computational and Applied Mathematics Rice University Houston, TX 77251 sorensen@rice, edu ABSTRACT Eigenvalues and eigenfunctions of linear operators are important to many areas of ap- plied mathematics. The ability to approximate these quantities numerically is becoming increasingly important in a wide variety of applications. This increasing demand has fu- eled interest in the development of new methods and software for the numerical solution of large-scale algebraic eigenvalue problems. In turn, the existence of these new methods and software, along with the dramatically increased computational capabilities now avail- able, has enabled the solution of problems that would not even have been posed five or ten years ago. Until very recently, software for large-scale nonsymmetric problems was virtually non-existent. Fortunately, the situation is improving rapidly. The purpose of this article is to provide an overview of the numerical solution of large- scale algebraic eigenvalue problems. The focus will be on a class of methods called Krylov subspace projection methods. The well-known Lanczos method is the premier member of this class. The Arnoldi method generalizes the Lanczos method to the nonsymmetric case.
    [Show full text]
  • Krylov Subspaces
    Lab 1 Krylov Subspaces Lab Objective: Discuss simple Krylov Subspace Methods for finding eigenvalues and show some interesting applications. One of the biggest difficulties in computational linear algebra is the amount of memory needed to store a large matrix and the amount of time needed to read its entries. Methods using Krylov subspaces avoid this difficulty by studying how a matrix acts on vectors, making it unnecessary in many cases to create the matrix itself. More specifically, we can construct a Krylov subspace just by knowing how a linear transformation acts on vectors, and with these subspaces we can closely approximate eigenvalues of the transformation and solutions to associated linear systems. The Arnoldi iteration is an algorithm for finding an orthonormal basis of a Krylov subspace. Its outputs can also be used to approximate the eigenvalues of the original matrix. The Arnoldi Iteration The order-N Krylov subspace of A generated by x is 2 n−1 Kn(A; x) = spanfx;Ax;A x;:::;A xg: If the vectors fx;Ax;A2x;:::;An−1xg are linearly independent, then they form a basis for Kn(A; x). However, this basis is usually far from orthogonal, and hence computations using this basis will likely be ill-conditioned. One way to find an orthonormal basis for Kn(A; x) would be to use the modi- fied Gram-Schmidt algorithm from Lab TODO on the set fx;Ax;A2x;:::;An−1xg. More efficiently, the Arnold iteration integrates the creation of fx;Ax;A2x;:::;An−1xg with the modified Gram-Schmidt algorithm, returning an orthonormal basis for Kn(A; x).
    [Show full text]
  • Iterative Methods for Linear System
    Iterative methods for Linear System JASS 2009 Student: Rishi Patil Advisor: Prof. Thomas Huckle Outline ●Basics: Matrices and their properties Eigenvalues, Condition Number ●Iterative Methods Direct and Indirect Methods ●Krylov Subspace Methods Ritz Galerkin: CG Minimum Residual Approach : GMRES/MINRES Petrov-Gaelerkin Method: BiCG, QMR, CGS 1st April JASS 2009 2 Basics ●Linear system of equations Ax = b ●A Hermitian matrix (or self-adjoint matrix ) is a square matrix with complex entries which is equal to its own conjugate transpose, 3 2 + i that is, the element in the ith row and jth column is equal to the A = complex conjugate of the element in the jth row and ith column, for 2 − i 1 all indices i and j • Symmetric if a ij = a ji • Positive definite if, for every nonzero vector x XTAx > 0 • Quadratic form: 1 f( x) === xTAx −−− bTx +++ c 2 ∂ f( x) • Gradient of Quadratic form: ∂x 1 1 1 f' (x) = ⋮ = ATx + Ax − b ∂ 2 2 f( x) ∂xn 1st April JASS 2009 3 Various quadratic forms Positive-definite Negative-definite matrix matrix Singular Indefinite positive-indefinite matrix matrix 1st April JASS 2009 4 Various quadratic forms 1st April JASS 2009 5 Eigenvalues and Eigenvectors For any n×n matrix A, a scalar λ and a nonzero vector v that satisfy the equation Av =λv are said to be the eigenvalue and the eigenvector of A. ●If the matrix is symmetric , then the following properties hold: (a) the eigenvalues of A are real (b) eigenvectors associated with distinct eigenvalues are orthogonal ●The matrix A is positive definite (or positive semidefinite) if and only if all eigenvalues of A are positive (or nonnegative).
    [Show full text]
  • Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations
    https://ntrs.nasa.gov/search.jsp?R=19960048075 2020-06-16T03:31:45+00:00Z NASA Contractor Report 198342 /" ICASE Report No. 96-40 J ICA IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen NASA Contract No. NASI-19480 May 1996 Institute for Computer Applications in Science and Engineering NASA Langley Research Center Hampton, VA 23681-0001 Operated by Universities Space Research Association National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-0001 IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen 1 Department of Computational and Applied Mathematics Rice University Houston, TX 77251 sorensen@rice, edu ABSTRACT Eigenvalues and eigenfunctions of linear operators are important to many areas of ap- plied mathematics. The ability to approximate these quantities numerically is becoming increasingly important in a wide variety of applications. This increasing demand has fu- eled interest in the development of new methods and software for the numerical solution of large-scale algebraic eigenvalue problems. In turn, the existence of these new methods and software, along with the dramatically increased computational capabilities now avail- able, has enabled the solution of problems that would not even have been posed five or ten years ago. Until very recently, software for large-scale nonsymmetric problems was virtually non-existent. Fortunately, the situation is improving rapidly. The purpose of this article is to provide an overview of the numerical solution of large- scale algebraic eigenvalue problems. The focus will be on a class of methods called Krylov subspace projection methods. The well-known Lanczos method is the premier member of this class.
    [Show full text]
  • The Influence of Orthogonality on the Arnoldi Method
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Linear Algebra and its Applications 309 (2000) 307±323 www.elsevier.com/locate/laa The in¯uence of orthogonality on the Arnoldi method T. Braconnier *, P. Langlois, J.C. Rioual IREMIA, Universite de la Reunion, DeÂpartement de Mathematiques et Informatique, BP 7151, 15 Avenue Rene Cassin, 97715 Saint-Denis Messag Cedex 9, La Reunion, France Received 1 February 1999; accepted 22 May 1999 Submitted by J.L. Barlow Abstract Many algorithms for solving eigenproblems need to compute an orthonormal basis. The computation is commonly performed using a QR factorization computed using the classical or the modi®ed Gram±Schmidt algorithm, the Householder algorithm, the Givens algorithm or the Gram±Schmidt algorithm with iterative reorthogonalization. For the eigenproblem, although textbooks warn users about the possible instability of eigensolvers due to loss of orthonormality, few theoretical results exist. In this paper we prove that the loss of orthonormality of the computed basis can aect the reliability of the computed eigenpair when we use the Arnoldi method. We also show that the stopping criterion based on the backward error and the value computed using the Arnoldi method can dier because of the loss of orthonormality of the computed basis of the Krylov subspace. We also give a bound which quanti®es this dierence in terms of the loss of orthonormality. Ó 2000 Elsevier Science Inc. All rights reserved. Keywords: QR factorization; Backward error; Eigenvalues; Arnoldi method 1. Introduction The aim of the most commonly used eigensolvers is to approximate a basis of an invariant subspace associated with some eigenvalues.
    [Show full text]
  • Power Method and Krylov Subspaces
    Power method and Krylov subspaces Introduction When calculating an eigenvalue of a matrix A using the power method, one starts with an initial random vector b and then computes iteratively the sequence Ab, A2b,...,An−1b normalising and storing the result in b on each iteration. The sequence converges to the eigenvector of the largest eigenvalue of A. The set of vectors 2 n−1 Kn = b, Ab, A b,...,A b , (1) where n < rank(A), is called the order-n Krylov matrix, and the subspace spanned by these vectors is called the order-n Krylov subspace. The vectors are not orthogonal but can be made so e.g. by Gram-Schmidt orthogonalisation. For the same reason that An−1b approximates the dominant eigenvector one can expect that the other orthogonalised vectors approximate the eigenvectors of the n largest eigenvalues. Krylov subspaces are the basis of several successful iterative methods in numerical linear algebra, in particular: Arnoldi and Lanczos methods for finding one (or a few) eigenvalues of a matrix; and GMRES (Generalised Minimum RESidual) method for solving systems of linear equations. These methods are particularly suitable for large sparse matrices as they avoid matrix-matrix opera- tions but rather multiply vectors by matrices and work with the resulting vectors and matrices in Krylov subspaces of modest sizes. Arnoldi iteration Arnoldi iteration is an algorithm where the order-n orthogonalised Krylov matrix Qn of a matrix A is built using stabilised Gram-Schmidt process: • start with a set Q = {q1} of one random normalised vector q1 • repeat for k =2 to n : – make a new vector qk = Aqk−1 † – orthogonalise qk to all vectors qi ∈ Q storing qi qk → hi,k−1 – normalise qk storing qk → hk,k−1 – add qk to the set Q By construction the matrix Hn made of the elements hjk is an upper Hessenberg matrix, h1,1 h1,2 h1,3 ··· h1,n h2,1 h2,2 h2,3 ··· h2,n 0 h3,2 h3,3 ··· h3,n Hn = , (2) .
    [Show full text]
  • Finding Eigenvalues: Arnoldi Iteration and the QR Algorithm
    Finding Eigenvalues: Arnoldi Iteration and the QR Algorithm Will Wheeler Algorithms Interest Group Jul 13, 2016 Outline Finding the largest eigenvalue I Largest eigenvalue power method I So much work for one eigenvector. What about others? I More eigenvectors orthogonalize Krylov matrix I build orthogonal list Arnoldi iteration Solving the eigenvalue problem I Eigenvalues diagonal of triangular matrix I Triangular matrix QR algorithm I QR algorithm QR decomposition I Better QR decomposition Hessenberg matrix I Hessenberg matrix Arnoldi algorithm Power Method How do we find the largest eigenvalue of an m × m matrix A? I Start with a vector b and make a power sequence: b; Ab; A2b;::: I Higher eigenvalues dominate: n n n n A (v1 + v2) = λ1v1 + λ2v2 ≈ λ1v1 Vector sequence (normalized) converged? I Yes: Eigenvector I No: Iterate some more Krylov Matrix Power method throws away information along the way. I Put the sequence into a matrix K = [b; Ab; A2b;:::; An−1b] (1) = [xn; xn−1; xn−2;:::; x1] (2) I Gram-Schmidt approximates first n eigenvectors v1 = x1 (3) x2 · x1 v2 = x2 − x1 (4) x1 · x1 x3 · x1 x3 · x2 v3 = x3 − x1 − x2 (5) x1 · x1 x2 · x2 ::: (6) I Why not orthogonalize as we build the list? Arnoldi Iteration I Start with a normalized vector q1 xk = Aqk−1 (next power) (7) k−1 X yk = xk − (qj · xk ) qj (Gram-Schmidt) (8) j=1 qk = yk = jyk j (normalize) (9) I Orthonormal vectors fq1;:::; qng span Krylov subspace n−1 (Kn = spanfq1; Aq1;:::; A q1g) I These are not the eigenvectors of A, but make a similarity y (unitary) transformation Hn = QnAQn Arnoldi Iteration I Start with a normalized vector q1 xk = Aqk−1 (next power) (10) k−1 X yk = xk − (qj · xk ) qj (Gram-Schmidt) (11) j=1 qk = yk =jyk j (normalize) (12) I Orthonormal vectors fq1;:::; qng span Krylov subspace n−1 (Kn = spanfq1; Aq1;:::; A q1g) I These are not the eigenvectors of A, but make a similarity y (unitary) transformation Hn = QnAQn Arnoldi Iteration - Construct Hn We can construct the matrix Hn along the way.
    [Show full text]
  • LARGE-SCALE COMPUTATION of PSEUDOSPECTRA USING ARPACK and EIGS∗ 1. Introduction. the Matrices in Many Eigenvalue Problems
    SIAM J. SCI. COMPUT. c 2001 Society for Industrial and Applied Mathematics Vol. 23, No. 2, pp. 591–605 LARGE-SCALE COMPUTATION OF PSEUDOSPECTRA USING ARPACK AND EIGS∗ THOMAS G. WRIGHT† AND LLOYD N. TREFETHEN† Abstract. ARPACK and its Matlab counterpart, eigs, are software packages that calculate some eigenvalues of a large nonsymmetric matrix by Arnoldi iteration with implicit restarts. We show that at a small additional cost, which diminishes relatively as the matrix dimension increases, good estimates of pseudospectra in addition to eigenvalues can be obtained as a by-product. Thus in large- scale eigenvalue calculations it is feasible to obtain routinely not just eigenvalue approximations, but also information as to whether or not the eigenvalues are likely to be physically significant. Examples are presented for matrices with dimension up to 200,000. Key words. Arnoldi, ARPACK, eigenvalues, implicit restarting, pseudospectra AMS subject classifications. 65F15, 65F30, 65F50 PII. S106482750037322X 1. Introduction. The matrices in many eigenvalue problems are too large to allow direct computation of their full spectra, and two of the iterative tools available for computing a part of the spectrum are ARPACK [10, 11]and its Matlab counter- part, eigs.1 For nonsymmetric matrices, the mathematical basis of these packages is the Arnoldi iteration with implicit restarting [11, 23], which works by compressing the matrix to an “interesting” Hessenberg matrix, one which contains information about the eigenvalues and eigenvectors of interest. For general information on large-scale nonsymmetric matrix eigenvalue iterations, see [2, 21, 29, 31]. For some matrices, nonnormality (nonorthogonality of the eigenvectors) may be physically important [30].
    [Show full text]
  • Comparison of Numerical Methods and Open-Source Libraries for Eigenvalue Analysis of Large-Scale Power Systems
    applied sciences Article Comparison of Numerical Methods and Open-Source Libraries for Eigenvalue Analysis of Large-Scale Power Systems Georgios Tzounas , Ioannis Dassios * , Muyang Liu and Federico Milano School of Electrical and Electronic Engineering, University College Dublin, Belfield, Dublin 4, Ireland; [email protected] (G.T.); [email protected] (M.L.); [email protected] (F.M.) * Correspondence: [email protected] Received: 30 September 2020; Accepted: 24 October 2020; Published: 28 October 2020 Abstract: This paper discusses the numerical solution of the generalized non-Hermitian eigenvalue problem. It provides a comprehensive comparison of existing algorithms, as well as of available free and open-source software tools, which are suitable for the solution of the eigenvalue problems that arise in the stability analysis of electric power systems. The paper focuses, in particular, on methods and software libraries that are able to handle the large-scale, non-symmetric matrices that arise in power system eigenvalue problems. These kinds of eigenvalue problems are particularly difficult for most numerical methods to handle. Thus, a review and fair comparison of existing algorithms and software tools is a valuable contribution for researchers and practitioners that are interested in power system dynamic analysis. The scalability and performance of the algorithms and libraries are duly discussed through case studies based on real-world electrical power networks. These are a model of the All-Island Irish Transmission System with 8640 variables; and, a model of the European Network of Transmission System Operators for Electricity, with 146,164 variables. Keywords: eigenvalue analysis; large non-Hermitian matrices; numerical methods; open-source libraries 1.
    [Show full text]
  • Eigenvalue Problems Last Time …
    Eigenvalue Problems Last Time … Social Network Graphs Betweenness Girvan-Newman Algorithm Graph Laplacian Spectral Bisection 휆2, 푤2 Today … Small deviation into eigenvalue problems … Formulation Standard eigenvalue problem: Given a 푛 × 푛 matrix 퐴, find scalar 휆 and a nonzero vector 푥 such that 퐴푥 = 휆푥 휆 is a eigenvalue, and 푥 is the corresponding eigenvector Spectrum = 휆(퐴) = set of eigenvalues of 퐴 Spectral radius = 퐴 = max 휆 ∶ 휆 ∈ 휆 퐴 Characteristic Polynomial Equation 퐴푥 = 휆푥 is equivalent to 퐴 − 휆퐼 푥 = 0 Eigenvalues of 퐴 are roots of the characteristic polynomial det 퐴 − 휆퐼 = 0 The characteristic polynomial is a powerful theoretical tool but usually not useful computationally Considerations Properties of eigenvalue problem affecting choice of algorithm Are all eigenvalues needed, or only a few? Are only eigenvalues needed, or are corresponding eigenvectors also needed? Is matrix real or complex? Is matrix relatively small and dense, or large and sparse? Does matrix have any special properties, such as symmetry? Problem Transformations Shift: If 퐴푥 = 휆푥 and is any scalar, then 퐴 − 퐼 푥 = 휆 − 푥 1 Inversion: If 퐴 is nonsingular and 퐴푥 = 휆푥, then 휆 ≠ 0 and 퐴−1푥 = 푥 휆 Powers: If 퐴푥 = 휆푥, then 퐴푘푥 = 휆푘푥 Polynomial: If 퐴푥 = 휆푥 and 푝(푡) is a polynomial, then 푝 퐴 푥 = 푝 휆 푥 Similarity Transforms 퐵 is similar to 퐴 if there is a nonsingular matrix 푇, such that 퐵 = 푇−1퐴푇 Then, 퐵푦 = 휆푦 ⇒ 푇−1퐴푇푦 = 휆푦 ⇒ 퐴 푇푦 = 휆 푇푦 Similarity transformations preserve eigenvalues and eigenvectors are easily recovered Diagonal form Eigenvalues
    [Show full text]
  • Lecture 33. the Arnoldi Iteration
    Lecture 33. The Arnoldi Iteration Despite the many names and acronyms that have proliferated in the field of Krylov subspaee rnatrix iterations, these algorithrIls are built upon a eOIIlIIlon foundation of a few fuudarnental ideas. One can take various approaehes to describing this foundation. Ours will be to consider the Arnoldi proeess, a. Gram-Schmidt-style iteration for transforming a matrix to Hessenberg form. The Arnoldi/Gram-Schmidt Analogy Suppose, to pass the time while marooned on a desert island, you challenged yourself t.o devise an algoritlnll to reduce a nOllhermitian Illatrix to Hes.senberg form by orthogonal similarity transformations, proceeding column by column from a prescribed first column q,. To your surprise, you would probably find you could solve this problem in an hour and still have time to gather coconuts for dinner. The method you would come up with goes by the name of the Arnoldi iteration. If A is hermitian, the Hessenberg matrix becomes tridiagonal, an n-tcrm recurrence relation becomes a three-term recurrence relation, and the name changes to the Lanc:zos iteration, to be discussed in Lecture 36. Here is an analogy. For computing the QR factorization A = (JR of a llWtrix A, \ve have discussed t\'w methods in this book: Householder reflec- tions, which triangularhe A by a succession of orthogonal operations, and GraIn-Schmidt orthogonalization, which A by a succession of triangular operations. Though Householder reflections lead to a more nearly 250 LECTURE 33. THE ARNOLDI ITERATION 251 orthogonal matrix Q in the presence of rounding errors, the Gram Schmidt process has the advantage that it can be stopped part-way, leaving one with a reduced QR factorization of the first.
    [Show full text]