7. Iterative Methods for Eigenvalue Problems
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
A Parallel Lanczos Algorithm for Eigensystem Calculation
A Parallel Lanczos Algorithm for Eigensystem Calculation Hans-Peter Kersken / Uwe Küster Eigenvalue problems arise in many fields of physics and engineering science for example in structural engineering from prediction of dynamic stability of structures or vibrations of a fluid in a closed cavity. Their solution consumes a large amount of memory and CPU time if more than very few (1-5) vibrational modes are desired. Both make the problem a natural candidate for parallel processing. Here we shall present some aspects of the solution of the generalized eigenvalue problem on parallel architectures with distributed memory. The research was carried out as a part of the EC founded project ACTIVATE. This project combines various activities to increase the performance vibro-acoustic software. It brings together end users form space, aviation, and automotive industry with software developers and numerical analysts. Introduction The algorithm we shall describe in the next sections is based on the Lanczos algorithm for solving large sparse eigenvalue problems. It became popular during the past two decades because of its superior convergence properties compared to more traditional methods like inverse vector iteration. Some experiments with an new algorithm described in [Bra97] showed that it is competitive with the Lanczos algorithm only if very few eigenpairs are needed. Therefore we decided to base our implementation on the Lanczos algorithm. However, when implementing the Lanczos algorithm one has to pay attention to some subtle algorithmic details. A state of the art algorithm is described in [Gri94]. On parallel architectures further problems arise concerning the robustness of the algorithm. We implemented the algorithm almost entirely by using numerical libraries. -
Exact Diagonalization of Quantum Lattice Models on Coprocessors
Exact diagonalization of quantum lattice models on coprocessors T. Siro∗, A. Harju Aalto University School of Science, P.O. Box 14100, 00076 Aalto, Finland Abstract We implement the Lanczos algorithm on an Intel Xeon Phi coprocessor and compare its performance to a multi-core Intel Xeon CPU and an NVIDIA graphics processor. The Xeon and the Xeon Phi are parallelized with OpenMP and the graphics processor is programmed with CUDA. The performance is evaluated by measuring the execution time of a single step in the Lanczos algorithm. We study two quantum lattice models with different particle numbers, and conclude that for small systems, the multi-core CPU is the fastest platform, while for large systems, the graphics processor is the clear winner, reaching speedups of up to 7.6 compared to the CPU. The Xeon Phi outperforms the CPU with sufficiently large particle number, reaching a speedup of 2.5. Keywords: Tight binding, Hubbard model, exact diagonalization, GPU, CUDA, MIC, Xeon Phi 1. Introduction hopping amplitude is denoted by t. The tight-binding model describes free electrons hopping around a lattice, and it gives a In recent years, there has been tremendous interest in utiliz- crude approximation of the electronic properties of a solid. The ing coprocessors in scientific computing, including condensed model can be made more realistic by adding interactions, such matter physics[1, 2, 3, 4]. Most of the work has been done as on-site repulsion, which results in the well-known Hubbard on graphics processing units (GPU), resulting in impressive model[9]. In our basis, however, such interaction terms are di- speedups compared to CPUs in problems that exhibit high data- agonal, rendering their effect on the computational complexity parallelism and benefit from the high throughput of the GPU. -
A Geometric Theory for Preconditioned Inverse Iteration. III: a Short and Sharp Convergence Estimate for Generalized Eigenvalue Problems
A geometric theory for preconditioned inverse iteration. III: A short and sharp convergence estimate for generalized eigenvalue problems. Andrew V. Knyazev Department of Mathematics, University of Colorado at Denver, P.O. Box 173364, Campus Box 170, Denver, CO 80217-3364 1 Klaus Neymeyr Mathematisches Institut der Universit¨atT¨ubingen,Auf der Morgenstelle 10, 72076 T¨ubingen,Germany 2 Abstract In two previous papers by Neymeyr: A geometric theory for preconditioned inverse iteration I: Extrema of the Rayleigh quotient, LAA 322: (1-3), 61-85, 2001, and A geometric theory for preconditioned inverse iteration II: Convergence estimates, LAA 322: (1-3), 87-104, 2001, a sharp, but cumbersome, convergence rate estimate was proved for a simple preconditioned eigensolver, which computes the smallest eigenvalue together with the corresponding eigenvector of a symmetric positive def- inite matrix, using a preconditioned gradient minimization of the Rayleigh quotient. In the present paper, we discover and prove a much shorter and more elegant, but still sharp in decisive quantities, convergence rate estimate of the same method that also holds for a generalized symmetric definite eigenvalue problem. The new estimate is simple enough to stimulate a search for a more straightforward proof technique that could be helpful to investigate such practically important method as the locally optimal block preconditioned conjugate gradient eigensolver. We demon- strate practical effectiveness of the latter for a model problem, where it compares favorably with two -
An Implementation of a Generalized Lanczos Procedure for Structural Dynamic Analysis on Distributed Memory Computers
NU~IERICAL ANALYSIS PROJECT AUGUST 1992 MANUSCRIPT ,NA-92-09 An Implementation of a Generalized Lanczos Procedure for Structural Dynamic Analysis on Distributed Memory Computers . bY David R. Mackay and Kincho H. Law NUMERICAL ANALYSIS PROJECT COMPUTER SCIENCE DEPARTMENT STANFORD UNIVERSITY STANFORD, CALIFORNIA 94305 An Implementation of a Generalized Lanczos Procedure for Structural Dynamic Analysis on Distributed Memory Computers’ David R. Mackay and Kincho H. Law Department of Civil Engineering Stanford University Stanford, CA 94305-4020 Abstract This paper describes a parallel implementation of a generalized Lanczos procedure for struc- tural dynamic analysis on a distributed memory parallel computer. One major cost of the gener- alized Lanczos procedure is the factorization of the (shifted) stiffness matrix and the forward and backward solution of triangular systems. In this paper, we discuss load assignment of a sparse matrix and propose a strategy for inverting the principal block submatrix factors to facilitate the forward and backward solution of triangular systems. We also discuss the different strategies in the implementation of mass matrix-vector multiplication on parallel computer and how they are used in the Lanczos procedure. The Lanczos procedure implemented includes partial and external selective reorthogonalizations and spectral shifts. Experimental results are presented to illustrate the effectiveness of the parallel generalized Lanczos procedure. The issues of balancing the com- putations among the basic steps of the Lanczos procedure on distributed memory computers are discussed. IThis work is sponsored by the National Science Foundation grant number ECS-9003107, and the Army Research Office grant number DAAL-03-91-G-0038. Contents List of Figures ii . -
Lanczos Vectors Versus Singular Vectors for Effective Dimension Reduction Jie Chen and Yousef Saad
1 Lanczos Vectors versus Singular Vectors for Effective Dimension Reduction Jie Chen and Yousef Saad Abstract— This paper takes an in-depth look at a technique for a) Latent Semantic Indexing (LSI): LSI [3], [4] is an effec- computing filtered matrix-vector (mat-vec) products which are tive information retrieval technique which computes the relevance required in many data analysis applications. In these applications scores for all the documents in a collection in response to a user the data matrix is multiplied by a vector and we wish to perform query. In LSI, a collection of documents is represented as a term- this product accurately in the space spanned by a few of the major singular vectors of the matrix. We examine the use of the document matrix X = [xij] where each column vector represents Lanczos algorithm for this purpose. The goal of the method is a document and xij is the weight of term i in document j. A query identical with that of the truncated singular value decomposition is represented as a pseudo-document in a similar form—a column (SVD), namely to preserve the quality of the resulting mat-vec vector q. By performing dimension reduction, each document xj product in the major singular directions of the matrix. The T T (j-th column of X) becomes Pk xj and the query becomes Pk q Lanczos-based approach achieves this goal by using a small in the reduced-rank representation, where P is the matrix whose number of Lanczos vectors, but it does not explicitly compute k column vectors are the k major left singular vectors of X. -
The Godunov–Inverse Iteration: a Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem
The Godunov{Inverse Iteration: A Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem Anna M. Matsekh a;1 aInstitute of Computational Technologies, Siberian Branch of the Russian Academy of Sciences, Lavrentiev Ave. 6, Novosibirsk 630090, Russia Abstract We present a new hybrid algorithm based on Godunov's method for computing eigenvectors of symmetric tridiagonal matrices and Inverse Iteration, which we call the Godunov{Inverse Iteration. We use eigenvectors computed according to Go- dunov's method as starting vectors in the Inverse Iteration, replacing any non- numeric elements of Godunov's eigenvectors with random uniform numbers. We use the right-hand bounds of the Ritz intervals found by the bisection method as Inverse Iteration shifts, while staying within guaranteed error bounds. In most test cases convergence is reached after only one step of the iteration, producing error estimates that are as good as or superior to those produced by standard Inverse Iteration implementations. Key words: Symmetric eigenvalue problem, tridiagonal matrices, Inverse Iteration 1 Introduction Construction of algorithms that enable to find all eigenvectors of the symmet- ric tridiagonal eigenvalue problem with guaranteed accuracy in O(n2) arith- metic operations has become one of the most pressing problems of the modern numerical algebra. QR method, one of the most accurate methods for solv- ing eigenvalue problems, requires 6n3 arithmetic operations and O(n2) square root operations to compute all eigenvectors of a tridiagonal matrix [Golub and Email address: [email protected] (Anna M. Matsekh ). 1 Present address: Modeling, Algorithms and Informatics Group, Los Alamos Na- tional Laboratory. P.O. Box 1663, MS B256, Los Alamos, NM 87544, USA Preprint submitted to Elsevier Science 31 January 2003 Loan (1996)]. -
Rcppeigen-Intro
Fast and Elegant Numerical Linear Algebra Using the RcppEigen Package Douglas Bates Dirk Eddelbuettel University of Wisconsin-Madison Debian Project Abstract The RcppEigen package provides access from R (R Core Team 2018a) to the Eigen (Guennebaud, Jacob et al. 2012) C++ template library for numerical linear algebra. Rcpp (Eddelbuettel et al. 2020a; Eddelbuettel 2013) classes and specializations of the C++ templated functions as and wrap from Rcpp provide the \glue" for passing objects from R to C++ and back. Several introductory examples are presented. This is followed by an in-depth discussion of various available approaches for solving least-squares problems, including rank-revealing methods, concluding with an empirical run-time comparison. Last but not least, sparse matrix methods are discussed. Keywords: linear algebra, template programming, R, C++, Rcpp. This vignette corresponds to a paper published in the Journal of Statistical Software. Currently still identical to the paper, this vignette version may over time receive minor updates. For citations, please use Bates and Eddelbuettel(2013) as provided by citation("RcppEigen"). This version corresponds to RcppEigen version 0.3.3.9.1 and was typeset on December 17, 2020. 1. Introduction Linear algebra is an essential building block of statistical computing. Operations such as matrix decompositions, linear program solvers, and eigenvalue/eigenvector computations are used in many estimation and analysis routines. As such, libraries supporting linear algebra have long been provided by statistical programmers for different programming languages and environments. Because it is object-oriented, C++, one of the central modern languages for numerical and statistical computing, is particularly effective at representing matrices, vectors and decompositions, and numerous class libraries providing linear algebra routines have been written over the years. -
A Note on Inverse Iteration
A Note on Inverse Iteration Klaus Neymeyr Universit¨atRostock, Fachbereich Mathematik, Universit¨atsplatz 1, 18051 Rostock, Germany; SUMMARY Inverse iteration, if applied to a symmetric positive definite matrix, is shown to generate a sequence of iterates with monotonously decreasing Rayleigh quotients. We present sharp bounds from above and from below which highlight inverse iteration as a descent scheme for the Rayleigh quotient. Such estimates provide the background for the analysis of the behavior of the Rayleigh quotient in certain approximate variants of inverse iteration. key words: Symmetric eigenvalue problem; Inverse iteration; Rayleigh quotient. 1. Introduction Inverse iteration is a well-known iterative procedure to compute approximations of eigenfunctions and eigenvalues of linear operators. It was introduced by Wielandt in 1944 in a sequence of five papers, see [1], to treat the matrix eigenvalue problem Axi = λixi for a real or complex square matrix A. The scalar λi is the ith eigenvalue and the vector xi denotes a corresponding eigenvector. Given a nonzero starting vector x(0), inverse iteration generates a sequence of iterates x(k) by solving the linear systems (A − σI)x(k+1) = x(k), k = 0, 1, 2,..., (1) where σ denotes an eigenvalue approximation and I is the identity matrix. In practice, the iterates are normalized after each step. If A is a symmetric matrix, then the iterates x(k) converge to an eigenvector associated with an eigenvalue closest to σ if the starting vector x(0) is not perpendicular to that vector. For non-symmetric matrices the issue of starting vectors is discussed in Sec. -
Linear Algebra Libraries
Linear Algebra Libraries Claire Mouton [email protected] March 2009 Contents I Requirements 3 II CPPLapack 4 III Eigen 5 IV Flens 7 V Gmm++ 9 VI GNU Scientific Library (GSL) 11 VII IT++ 12 VIII Lapack++ 13 arXiv:1103.3020v1 [cs.MS] 15 Mar 2011 IX Matrix Template Library (MTL) 14 X PETSc 16 XI Seldon 17 XII SparseLib++ 18 XIII Template Numerical Toolkit (TNT) 19 XIV Trilinos 20 1 XV uBlas 22 XVI Other Libraries 23 XVII Links and Benchmarks 25 1 Links 25 2 Benchmarks 25 2.1 Benchmarks for Linear Algebra Libraries . ....... 25 2.2 BenchmarksincludingSeldon . 26 2.2.1 BenchmarksforDenseMatrix. 26 2.2.2 BenchmarksforSparseMatrix . 29 XVIII Appendix 30 3 Flens Overloaded Operator Performance Compared to Seldon 30 4 Flens, Seldon and Trilinos Content Comparisons 32 4.1 Available Matrix Types from Blas (Flens and Seldon) . ........ 32 4.2 Available Interfaces to Blas and Lapack Routines (Flens and Seldon) . 33 4.3 Available Interfaces to Blas and Lapack Routines (Trilinos) ......... 40 5 Flens and Seldon Synoptic Comparison 41 2 Part I Requirements This document has been written to help in the choice of a linear algebra library to be included in Verdandi, a scientific library for data assimilation. The main requirements are 1. Portability: Verdandi should compile on BSD systems, Linux, MacOS, Unix and Windows. Beyond the portability itself, this often ensures that most compilers will accept Verdandi. An obvious consequence is that all dependencies of Verdandi must be portable, especially the linear algebra library. 2. High-level interface: the dependencies should be compatible with the building of the high-level interface (e. -
Inverse, Shifted Inverse, and Rayleigh Quotient Iteration As Newton’S Method⇤
INVERSE, SHIFTED INVERSE, AND RAYLEIGH QUOTIENT ITERATION AS NEWTON’S METHOD⇤ JOHN DENNIS JR.† AND RICHARD TAPIA‡ Abstract. Two-norm normalized inverse, shifted inverse, and Rayleigh quotient iteration are well-known algorithms for approximating an eigenvector of a symmetric matrix. In this work we establish rigorously that each one of these three algorithms can be viewed as a standard form of Newton’s method from the nonlinear programming literature, followed by the normalization. This equivalence adds considerable understanding to the formal structure of inverse, shifted inverse, and Rayleigh quotient iteration and provides an explanation for their good behavior despite the possible need to solve systems with nearly singular coefficient matrices; the algorithms have what can be viewed as removable singularities. A thorough historical development of these eigenvalue algorithms is presented. Utilizing our equivalences we construct traditional Newton’s method theory analysis in order to gain understanding as to why, as normalized Newton’s method, inverse iteration and shifted inverse iteration are only linearly convergent and not quadratically convergent, and why a new linear system need not be solved at each iteration. We also investigate why Rayleigh quotient iteration is cubically convergent and not just quadratically convergent. Key words. inverse power method, inverse iteration, shifted inverse iteration, Rayleigh quotient iteration, Newton’s method AMS subject classifications. 65F15, 49M37, 49M15, 65K05 1. Introduction. The initial -
C++ API for BLAS and LAPACK
2 C++ API for BLAS and LAPACK Mark Gates ICL1 Piotr Luszczek ICL Ahmad Abdelfattah ICL Jakub Kurzak ICL Jack Dongarra ICL Konstantin Arturov Intel2 Cris Cecka NVIDIA3 Chip Freitag AMD4 1Innovative Computing Laboratory 2Intel Corporation 3NVIDIA Corporation 4Advanced Micro Devices, Inc. February 21, 2018 This research was supported by the Exascale Computing Project (17-SC-20-SC), a collaborative effort of two U.S. Department of Energy organizations (Office of Science and the National Nuclear Security Administration) responsible for the planning and preparation of a capable exascale ecosystem, including software, applications, hardware, advanced system engineering and early testbed platforms, in support of the nation's exascale computing imperative. Revision Notes 06-2017 first publication 02-2018 • copy editing, • new cover. 03-2018 • adding a section about GPU support, • adding Ahmad Abdelfattah as an author. @techreport{gates2017cpp, author={Gates, Mark and Luszczek, Piotr and Abdelfattah, Ahmad and Kurzak, Jakub and Dongarra, Jack and Arturov, Konstantin and Cecka, Cris and Freitag, Chip}, title={{SLATE} Working Note 2: C++ {API} for {BLAS} and {LAPACK}}, institution={Innovative Computing Laboratory, University of Tennessee}, year={2017}, month={June}, number={ICL-UT-17-03}, note={revision 03-2018} } i Contents 1 Introduction and Motivation1 2 Standards and Trends2 2.1 Programming Language Fortran............................ 2 2.1.1 FORTRAN 77.................................. 2 2.1.2 BLAST XBLAS................................. 4 2.1.3 Fortran 90.................................... 4 2.2 Programming Language C................................ 5 2.2.1 Netlib CBLAS.................................. 5 2.2.2 Intel MKL CBLAS................................ 6 2.2.3 Netlib lapack cwrapper............................. 6 2.2.4 LAPACKE.................................... 7 2.2.5 Next-Generation BLAS: \BLAS G2"..................... -
Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2
ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU Abstract. We examine some numerical iterative methods for computing the eigenvalues and eigenvectors of real matrices. The five methods examined here range from the simple power iteration method to the more complicated QR iteration method. The derivations, procedure, and advantages of each method are briefly discussed. 1. Introduction Eigenvalues and eigenvectors play an important part in the applications of linear algebra. The naive method of finding the eigenvalues of a matrix involves finding the roots of the characteristic polynomial of the matrix. In industrial sized matrices, however, this method is not feasible, and the eigenvalues must be obtained by other means. Fortunately, there exist several other techniques for finding eigenvalues and eigenvectors of a matrix, some of which fall under the realm of iterative methods. These methods work by repeatedly refining approximations to the eigenvectors or eigenvalues, and can be terminated whenever the approximations reach a suitable degree of accuracy. Iterative methods form the basis of much of modern day eigenvalue computation. In this paper, we outline five such iterative methods, and summarize their derivations, procedures, and advantages. The methods to be examined are the power iteration method, the shifted inverse iteration method, the Rayleigh quotient method, the simultaneous iteration method, and the QR method. This paper is meant to be a survey over existing algorithms for the eigenvalue computation problem. Section 2 of this paper provides a brief review of some of the linear algebra background required to understand the concepts that are discussed. In section 3, the iterative methods are each presented, in order of complexity, and are studied in brief detail.