DOCSLIB.ORG

Lecture Notes on Solving Large Scale Eigenvalue Problems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Lecture Notes on Solving Large Scale Eigenvalue Problems Prof. Dr. Peter Arbenz Computer Science Department ETH Z¨urich Email: [email protected] Spring semester 2016 ii Contents 1 Introduction 1 1.1 What makes eigenvalues interesting? ...................... 1 1.2 Example 1: The vibrating string ........................ 2 1.2.1 Problem setting .............................. 2 1.2.2 The method of separation of variables ................. 5 1.3 Numerical methods for solving 1-dimensional problems ............ 6 1.3.1 Finite differences ............................. 6 1.3.2 The finite element method ........................ 7 1.3.3 Global functions ............................. 8 1.3.4 A numerical comparison ......................... 9 1.4 Example 2: The heat equation ......................... 9 1.5 Example 3: The wave equation ......................... 12 1.6 The 2D Laplace eigenvalue problem ...................... 13 1.6.1 The finite difference method ....................... 13 1.6.2 The finite element method (FEM) ................... 16 1.6.3 A numerical example ........................... 20 1.7 Cavity resonances in particle accelerators ................... 21 1.8 Spectral clustering ................................ 23 1.8.1 The graph Laplacian ........................... 24 1.8.2 Spectral clustering ............................ 25 1.8.3 Normalized graph Laplacians ...................... 27 1.9 Google’s PageRank ................................ 28 1.10 Other sources of eigenvalue problems ...................... 30 Bibliography ...................................... 31 2 Basics 33 2.1 Notation ...................................... 33 2.2 Statement of the problem ............................ 34 2.3 Similarity transformations ............................ 37 2.4 Schur decomposition ............................... 38 2.5 The real Schur decomposition .......................... 39 2.6 Normal matrices ................................. 40 2.7 Hermitian matrices ................................ 41 2.8 The Jordan normal form ............................. 43 2.9 Projections .................................... 45 2.10 The Rayleigh quotient .............................. 47 2.11 Cholesky factorization .............................. 49 2.12 The singular value decomposition (SVD) .................... 50 iii iv CONTENTS Bibliography ...................................... 52 3 Newton methods 53 3.1 Linear and nonlinear eigenvalue problems ................... 53 3.2 Zeros of the determinant ............................. 54 3.2.1 Algorithmic differentiation ........................ 55 3.2.2 Hyman’s algorithm ............................ 55 3.2.3 Computing multiple zeros ........................ 58 3.3 Newton methods for the constrained matrix problem ............. 58 3.4 Successive linear approximations ........................ 60 Bibliography ...................................... 61 4 The QR Algorithm 63 4.1 The basic QR algorithm ............................. 63 4.1.1 Numerical experiments .......................... 64 4.2 The Hessenberg QR algorithm ......................... 67 4.2.1 A numerical experiment ......................... 69 4.2.2 Complexity ................................ 70 4.3 The Householder reduction to Hessenberg form ................ 71 4.3.1 Householder reflectors .......................... 71 4.3.2 Reduction to Hessenberg form ..................... 71 4.4 Improving the convergence of the QR algorithm ................ 73 4.4.1 A numerical example ........................... 74 4.4.2 QR algorithm with shifts ........................ 75 4.4.3 A numerical example ........................... 76 4.5 The double shift QR algorithm ......................... 77 4.5.1 A numerical example ........................... 81 4.5.2 The complexity .............................. 83 4.6 The symmetric tridiagonal QR algorithm ................... 84 4.6.1 Reduction to tridiagonal form ...................... 84 4.6.2 The tridiagonal QR algorithm ...................... 85 4.7 Research ...................................... 87 4.8 Summary ..................................... 87 Bibliography ...................................... 88 5 Cuppen’s Divide and Conquer Algorithm 91 5.1 The divide and conquer idea ........................... 91 5.2 Partitioning the tridiagonal matrix ....................... 92 5.3 Solving the small systems ............................ 92 5.4 Deflation ..................................... 93 5.4.1 Numerical examples ........................... 94 5.5 The eigenvalue problem for D + ρvvT ..................... 95 5.6 Solving the secular equation ........................... 98 5.7 A first algorithm ................................. 99 5.7.1 A numerical example ...........................100 5.8 The algorithm of Gu and Eisenstat .......................103 5.8.1 A numerical example [continued] ....................104 Bibliography ......................................107 CONTENTS v 6 LAPACK and the BLAS 109 6.1 LAPACK .....................................109 6.2 BLAS .......................................110 6.2.1 Typical performance numbers for the BLAS ..............111 6.3 Blocking ......................................113 6.4 LAPACK solvers for the symmetric eigenproblems ..............114 6.5 Generalized Symmetric Definite Eigenproblems (GSEP) ...........116 6.6 An example of a LAPACK routines .......................116 Bibliography ......................................122 7 Vector iteration (power method) 125 7.1 Simple vector iteration ..............................125 7.2 Angles between vectors ..............................126 7.3 Convergence analysis ...............................127 7.4 A numerical example ...............................130 7.5 The symmetric case ...............................131 7.6 Inverse vector iteration ..............................135 7.7 The generalized eigenvalue problem .......................139 7.8 Computing higher eigenvalues ..........................139 7.9 Rayleigh quotient iteration ...........................140 7.9.1 A numerical example ...........................143 Bibliography ......................................144 8 Simultaneous vector or subspace iterations 145 8.1 Basic subspace iteration .............................145 8.2 Angles between subspaces ............................146 8.3 Convergence of basic subspace iteration ....................148 8.4 Accelerating subspace iteration .........................153 8.5 Relation between subspace iteration and QR algorithm ............158 8.6 Addendum ....................................161 Bibliography ......................................161 9 Krylov subspaces 163 9.1 Introduction ....................................163 9.2 Definition and basic properties .........................164 9.3 Polynomial representation of Krylov subspaces ................165 9.4 Error bounds of Saad ...............................168 Bibliography ......................................171 10 Arnoldi and Lanczos algorithms 173 10.1 An orthonormal basis for the Krylov space j(x) ...............173 K 10.2 Arnoldi algorithm with explicit restarts ....................175 10.3 The Lanczos basis ................................176 10.4 The Lanczos process as an iterative method ..................178 10.5 An error analysis of the unmodified Lanczos algorithm ............185 10.6 Partial reorthogonalization ...........................187 10.7 Block Lanczos ...................................190 10.8 External selective reorthogonalization .....................193 Bibliography ......................................194 vi CONTENTS 11 Restarting Arnoldi and Lanczos algorithms 195 11.1 The m-step Arnoldi iteration ..........................195 11.2 Implicit restart ..................................196 11.3 Convergence criterion ..............................198 11.4 The generalized eigenvalue problem .......................199 11.5 A numerical example ...............................201 11.6 Another numerical example ...........................206 11.7 The Lanczos algorithm with thick restarts ...................210 11.8 Krylov–Schur algorithm .............................213 11.9 The rational Krylov space method .......................214 Bibliography ......................................215 12 The Jacobi-Davidson Method 217 12.1 The Davidson algorithm .............................217 12.2 The Jacobi orthogonal component correction .................218 12.2.1 Restarts ..................................221 12.2.2 The computation of several eigenvalues ................221 12.2.3 Spectral shifts ..............................222 12.3 The generalized Hermitian eigenvalue problem ................224 12.4 A numerical example ...............................224 12.5 The Jacobi–Davidson algorithm for interior eigenvalues ............228 12.6 Harmonic Ritz values and vectors ........................229 12.7 Refined Ritz vectors ...............................231 12.8 The generalized Schur decomposition ......................233 12.9 JDQZ: Computing a partial QZ decomposition ................233 12.9.1 Restart ..................................235 12.9.2 Deflation .................................235 12.9.3 Algorithm .................................236 12.10Jacobi-Davidson for nonlinear eigenvalue problems ..............236 Bibliography ......................................239 13 Rayleigh quotient and trace minimization 241 13.1 Introduction ....................................241 13.2 The method of steepest descent .........................242 13.3 The conjugate gradient
Recommended publications
  • Conjugate Gradient Method (Part 4)  Pre-Conditioning  Nonlinear Conjugate Gradient Method

    Conjugate Gradient Method (Part 4)  Pre-Conditioning  Nonlinear Conjugate Gradient Method

    18-660: Numerical Methods for Engineering Design and Optimization Xin Li Department of ECE Carnegie Mellon University Pittsburgh, PA 15213 Slide 1 Overview Conjugate Gradient Method (Part 4) Pre-conditioning Nonlinear conjugate gradient method Slide 2 Conjugate Gradient Method Step 1: start from an initial guess X(0), and set k = 0 Step 2: calculate ( ) ( ) ( ) D 0 = R 0 = B − AX 0 Step 3: update solution D(k )T R(k ) X (k +1) = X (k ) + µ (k )D(k ) where µ (k ) = D(k )T AD(k ) Step 4: calculate residual ( + ) ( ) ( ) ( ) R k 1 = R k − µ k AD k Step 5: determine search direction R(k +1)T R(k +1) (k +1) = (k +1) + β (k ) β = D R k +1,k D where k +1,k (k )T (k ) D R Step 6: set k = k + 1 and go to Step 3 Slide 3 Convergence Rate k ( + ) κ(A) −1 ( ) X k 1 − X ≤ ⋅ X 0 − X κ(A) +1 Conjugate gradient method has slow convergence if κ(A) is large I.e., AX = B is ill-conditioned In this case, we want to improve convergence rate by pre- conditioning Slide 4 Pre-Conditioning Key idea Convert AX = B to another equivalent equation ÃX̃ = B̃ Solve ÃX̃ = B̃ by conjugate gradient method Important constraints to construct ÃX̃ = B̃ à is symmetric and positive definite – so that we can solve it by conjugate gradient method à has a small condition number – so that we can achieve fast convergence Slide 5 Pre-Conditioning AX = B −1 −1 L A⋅ X = L B L−1 AL−T ⋅ LT X = L−1B A X B ̃ ̃ ̃ L−1AL−T is symmetric and positive definite, if A is symmetric and positive definite T (L−1 AL−T ) = L−1 AL−T T X T L−1 AL−T X = (L−T X ) ⋅ A⋅(L−T X )> 0 Slide
  • A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe Wu, Serge Petiton

    A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe Wu, Serge Petiton

    A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe Wu, Serge Petiton To cite this version: Xinzhe Wu, Serge Petiton. A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems. HPC Asia 2018 - International Conference on High Performance Computing in Asia-Pacific Region, Jan 2018, Tokyo, Japan. 10.1145/3149457.3154481. hal-01677110 HAL Id: hal-01677110 https://hal.archives-ouvertes.fr/hal-01677110 Submitted on 8 Jan 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. A Distributed and Parallel Asynchronous Unite and Conquer Method to Solve Large Scale Non-Hermitian Linear Systems Xinzhe WU Serge G. Petiton Maison de la Simulation, USR 3441 Maison de la Simulation, USR 3441 CRIStAL, UMR CNRS 9189 CRIStAL, UMR CNRS 9189 Université de Lille 1, Sciences et Technologies Université de Lille 1, Sciences et Technologies F-91191, Gif-sur-Yvette, France F-91191, Gif-sur-Yvette, France [email protected] [email protected] ABSTRACT the iteration number. These methods are already well implemented Parallel Krylov Subspace Methods are commonly used for solv- in parallel to profit from the great number of computation cores ing large-scale sparse linear systems.
  • MAY the Bridges I BURN Light the WAY EXILE X Summer Camp

    MAY the Bridges I BURN Light the WAY EXILE X Summer Camp

    M ay the brid G es I burn L I G ht the way EXILE X summer camp June 13 - 17, 2018 Curated by María Inés Plaza Lazo In collaboration with Alina Kolar, Dalia Maini & Christian Siekmeier PARTICI PANTS in 2008. Within the last 10 years EXILE Del Vecchio is an artist and co-curator of Narine Arakelyan is a performance artist concluded an MA at Goldsmiths, University has operated out of five distinctly different flip project space, a collaborative project for that works in conjunction with a Moscow of London. Graduated in Psychology, her Albrecht Pischel is interested in the definition spaces in Berlin. In 2011 EXILE became critical experimentation. He attended the based group of producers known as laboratory ongoing research unveils post-historical identity, of a new internationality and its ghosts. Most a professionally operating gallery. Until Städelschule in Frankfurt am Main and a abc. Together they aim to unite progressive hospitality and intergenerational transmission as of his works behave as representing signifiers today, EXILE has hosted over 80 solo and Masters in Fine Art at The Glasgow School artists, musicians and curators. The group a means to inhabit but also perform/transform of the medium of the art exhibition itself, group exhibitions which have been reviewed of Art. He recently took part of CuratorLab is committed to promote new ideas and to history. In her work she reflects on human oscillating between the disarming of symbols extensively in relevant publications around at Konstfack Stockholm, a curatorial course develop the values of the cultural heritage. relationships and how to create and re-create of colonialism and alienation.
  • Periodic Staircase Matrices and Generalized Cluster Structures

    Periodic Staircase Matrices and Generalized Cluster Structures

    PERIODIC STAIRCASE MATRICES AND GENERALIZED CLUSTER STRUCTURES MISHA GEKHTMAN, MICHAEL SHAPIRO, AND ALEK VAINSHTEIN Abstract. As is well-known, cluster transformations in cluster structures of geometric type are often modeled on determinant identities, such as short Pl¨ucker relations, Desnanot–Jacobi identities and their generalizations. We present a construction that plays a similar role in a description of general- ized cluster transformations and discuss its applications to generalized clus- ter structures in GLn compatible with a certain subclass of Belavin–Drinfeld Poisson–Lie brackets, in the Drinfeld double of GLn, and in spaces of periodic difference operators. 1. Introduction Since the discovery of cluster algebras in [4], many important algebraic varieties were shown to support a cluster structure in a sense that the coordinate rings of such variety is isomorphic to a cluster algebra or an upper cluster algebra. Lie theory and representation theory turned out to be a particularly rich source of varieties of this sort including but in no way limited to such examples as Grassmannians [5, 18], double Bruhat cells [1] and strata in flag varieties [15]. In all these examples, cluster transformations that connect distinguished coordinate charts within a ring of regular functions are modeled on three-term relations such as short Pl¨ucker relations, Desnanot–Jacobi identities and their Lie-theoretic generalizations of the kind considered in [3]. This remains true even in the case of exotic cluster structures on GLn considered in [8, 10] where cluster transformations can be obtained by applying Desnanot–Jacobi type identities to certain structured matrices of a size far exceeding n.
  • Arxiv:Math/0010243V1

    Arxiv:Math/0010243V1

    FOUR SHORT STORIES ABOUT TOEPLITZ MATRIX CALCULATIONS THOMAS STROHMER∗ Abstract. The stories told in this paper are dealing with the solution of finite, infinite, and biinfinite Toeplitz-type systems. A crucial role plays the off-diagonal decay behavior of Toeplitz matrices and their inverses. Classical results of Gelfand et al. on commutative Banach algebras yield a general characterization of this decay behavior. We then derive estimates for the approximate solution of (bi)infinite Toeplitz systems by the finite section method, showing that the approximation rate depends only on the decay of the entries of the Toeplitz matrix and its condition number. Furthermore, we give error estimates for the solution of doubly infinite convolution systems by finite circulant systems. Finally, some quantitative results on the construction of preconditioners via circulant embedding are derived, which allow to provide a theoretical explanation for numerical observations made by some researchers in connection with deconvolution problems. Key words. Toeplitz matrix, Laurent operator, decay of inverse matrix, preconditioner, circu- lant matrix, finite section method. AMS subject classifications. 65T10, 42A10, 65D10, 65F10 0. Introduction. Toeplitz-type equations arise in many applications in mathe- matics, signal processing, communications engineering, and statistics. The excellent surveys [4, 17] describe a number of applications and contain a vast list of references. The stories told in this paper are dealing with the (approximate) solution of biinfinite, infinite, and finite hermitian positive definite Toeplitz-type systems. We pay special attention to Toeplitz-type systems with certain decay properties in the sense that the entries of the matrix enjoy a certain decay rate off the diagonal.
  • Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations

    Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations

    NASA Contractor Report 198342 /" ICASE Report No. 96-40 J ICA IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen NASA Contract No. NASI-19480 May 1996 Institute for Computer Applications in Science and Engineering NASA Langley Research Center Hampton, VA 23681-0001 Operated by Universities Space Research Association National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-0001 IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen 1 Department of Computational and Applied Mathematics Rice University Houston, TX 77251 sorensen@rice, edu ABSTRACT Eigenvalues and eigenfunctions of linear operators are important to many areas of ap- plied mathematics. The ability to approximate these quantities numerically is becoming increasingly important in a wide variety of applications. This increasing demand has fu- eled interest in the development of new methods and software for the numerical solution of large-scale algebraic eigenvalue problems. In turn, the existence of these new methods and software, along with the dramatically increased computational capabilities now avail- able, has enabled the solution of problems that would not even have been posed five or ten years ago. Until very recently, software for large-scale nonsymmetric problems was virtually non-existent. Fortunately, the situation is improving rapidly. The purpose of this article is to provide an overview of the numerical solution of large- scale algebraic eigenvalue problems. The focus will be on a class of methods called Krylov subspace projection methods. The well-known Lanczos method is the premier member of this class. The Arnoldi method generalizes the Lanczos method to the nonsymmetric case.
  • Krylov Subspaces

    Krylov Subspaces

    Lab 1 Krylov Subspaces Lab Objective: Discuss simple Krylov Subspace Methods for finding eigenvalues and show some interesting applications. One of the biggest difficulties in computational linear algebra is the amount of memory needed to store a large matrix and the amount of time needed to read its entries. Methods using Krylov subspaces avoid this difficulty by studying how a matrix acts on vectors, making it unnecessary in many cases to create the matrix itself. More specifically, we can construct a Krylov subspace just by knowing how a linear transformation acts on vectors, and with these subspaces we can closely approximate eigenvalues of the transformation and solutions to associated linear systems. The Arnoldi iteration is an algorithm for finding an orthonormal basis of a Krylov subspace. Its outputs can also be used to approximate the eigenvalues of the original matrix. The Arnoldi Iteration The order-N Krylov subspace of A generated by x is 2 n−1 Kn(A; x) = spanfx;Ax;A x;:::;A xg: If the vectors fx;Ax;A2x;:::;An−1xg are linearly independent, then they form a basis for Kn(A; x). However, this basis is usually far from orthogonal, and hence computations using this basis will likely be ill-conditioned. One way to find an orthonormal basis for Kn(A; x) would be to use the modi- fied Gram-Schmidt algorithm from Lab TODO on the set fx;Ax;A2x;:::;An−1xg. More efficiently, the Arnold iteration integrates the creation of fx;Ax;A2x;:::;An−1xg with the modified Gram-Schmidt algorithm, returning an orthonormal basis for Kn(A; x).
  • Accelerating the LOBPCG Method on Gpus Using a Blocked Sparse Matrix Vector Product

    Accelerating the LOBPCG Method on Gpus Using a Blocked Sparse Matrix Vector Product

    Accelerating the LOBPCG method on GPUs using a blocked Sparse Matrix Vector Product Hartwig Anzt and Stanimire Tomov and Jack Dongarra Innovative Computing Lab University of Tennessee Knoxville, USA Email: [email protected], [email protected], [email protected] Abstract— the computing power of today’s supercomputers, often accel- erated by coprocessors like graphics processing units (GPUs), This paper presents a heterogeneous CPU-GPU algorithm design and optimized implementation for an entire sparse iter- becomes challenging. ative eigensolver – the Locally Optimal Block Preconditioned Conjugate Gradient (LOBPCG) – starting from low-level GPU While there exist numerous efforts to adapt iterative lin- data structures and kernels to the higher-level algorithmic choices ear solvers like Krylov subspace methods to coprocessor and overall heterogeneous design. Most notably, the eigensolver technology, sparse eigensolvers have so far remained out- leverages the high-performance of a new GPU kernel developed side the main focus. A possible explanation is that many for the simultaneous multiplication of a sparse matrix and a of those combine sparse and dense linear algebra routines, set of vectors (SpMM). This is a building block that serves which makes porting them to accelerators more difficult. Aside as a backbone for not only block-Krylov, but also for other from the power method, algorithms based on the Krylov methods relying on blocking for acceleration in general. The subspace idea are among the most commonly used general heterogeneous LOBPCG developed here reveals the potential of eigensolvers [1]. When targeting symmetric positive definite this type of eigensolver by highly optimizing all of its components, eigenvalue problems, the recently developed Locally Optimal and can be viewed as a benchmark for other SpMM-dependent applications.
  • THREE STEPS on an OPEN ROAD Gilbert Strang This Note Describes

    THREE STEPS on an OPEN ROAD Gilbert Strang This Note Describes

    Inverse Problems and Imaging doi:10.3934/ipi.2013.7.961 Volume 7, No. 3, 2013, 961{966 THREE STEPS ON AN OPEN ROAD Gilbert Strang Massachusetts Institute of Technology Cambridge, MA 02139, USA Abstract. This note describes three recent factorizations of banded invertible infinite matrices 1. If A has a banded inverse : A=BC with block{diagonal factors B and C. 2. Permutations factor into a shift times N < 2w tridiagonal permutations. 3. A = LP U with lower triangular L, permutation P , upper triangular U. We include examples and references and outlines of proofs. This note describes three small steps in the factorization of banded matrices. It is written to encourage others to go further in this direction (and related directions). At some point the problems will become deeper and more difficult, especially for doubly infinite matrices. Our main point is that matrices need not be Toeplitz or block Toeplitz for progress to be possible. An important theory is already established [2, 9, 10, 13-16] for non-Toeplitz \band-dominated operators". The Fredholm index plays a key role, and the second small step below (taken jointly with Marko Lindner) computes that index in the special case of permutation matrices. Recall that banded Toeplitz matrices lead to Laurent polynomials. If the scalars or matrices a−w; : : : ; a0; : : : ; aw lie along the diagonals, the polynomial is A(z) = P k akz and the bandwidth is w. The required index is in this case a winding number of det A(z). Factorization into A+(z)A−(z) is a classical problem solved by Plemelj [12] and Gohberg [6-7].
  • Hybrid Algorithms for Efficient Cholesky Decomposition And

    Hybrid Algorithms for Efficient Cholesky Decomposition And

    Hybrid Algorithms for Efficient Cholesky Decomposition and Matrix Inverse using Multicore CPUs with GPU Accelerators Gary Macindoe A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy of UCL. 2013 ii I, Gary Macindoe, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. Signature : Abstract The use of linear algebra routines is fundamental to many areas of computational science, yet their implementation in software still forms the main computational bottleneck in many widely used algorithms. In machine learning and computational statistics, for example, the use of Gaussian distributions is ubiquitous, and routines for calculating the Cholesky decomposition, matrix inverse and matrix determinant must often be called many thousands of times for com- mon algorithms, such as Markov chain Monte Carlo. These linear algebra routines consume most of the total computational time of a wide range of statistical methods, and any improve- ments in this area will therefore greatly increase the overall efficiency of algorithms used in many scientific application areas. The importance of linear algebra algorithms is clear from the substantial effort that has been invested over the last 25 years in producing low-level software libraries such as LAPACK, which generally optimise these linear algebra routines by breaking up a large problem into smaller problems that may be computed independently. The performance of such libraries is however strongly dependent on the specific hardware available. LAPACK was originally de- veloped for single core processors with a memory hierarchy, whereas modern day computers often consist of mixed architectures, with large numbers of parallel cores and graphics process- ing units (GPU) being used alongside traditional CPUs.
  • The Generalized Triangular Decomposition

    The Generalized Triangular Decomposition

    MATHEMATICS OF COMPUTATION Volume 77, Number 262, April 2008, Pages 1037–1056 S 0025-5718(07)02014-5 Article electronically published on October 1, 2007 THE GENERALIZED TRIANGULAR DECOMPOSITION YI JIANG, WILLIAM W. HAGER, AND JIAN LI Abstract. Given a complex matrix H, we consider the decomposition H = QRP∗,whereR is upper triangular and Q and P have orthonormal columns. Special instances of this decomposition include the singular value decompo- sition (SVD) and the Schur decomposition where R is an upper triangular matrix with the eigenvalues of H on the diagonal. We show that any diag- onal for R can be achieved that satisfies Weyl’s multiplicative majorization conditions: k k K K |ri|≤ σi, 1 ≤ k<K, |ri| = σi, i=1 i=1 i=1 i=1 where K is the rank of H, σi is the i-th largest singular value of H,andri is the i-th largest (in magnitude) diagonal element of R. Given a vector r which satisfies Weyl’s conditions, we call the decomposition H = QRP∗,whereR is upper triangular with prescribed diagonal r, the generalized triangular decom- position (GTD). A direct (nonrecursive) algorithm is developed for computing the GTD. This algorithm starts with the SVD and applies a series of permu- tations and Givens rotations to obtain the GTD. The numerical stability of the GTD update step is established. The GTD can be used to optimize the power utilization of a communication channel, while taking into account qual- ity of service requirements for subchannels. Another application of the GTD is to inverse eigenvalue problems where the goal is to construct matrices with prescribed eigenvalues and singular values.
  • A Parallel Lanczos Algorithm for Eigensystem Calculation

    A Parallel Lanczos Algorithm for Eigensystem Calculation

    A Parallel Lanczos Algorithm for Eigensystem Calculation Hans-Peter Kersken / Uwe Küster Eigenvalue problems arise in many fields of physics and engineering science for example in structural engineering from prediction of dynamic stability of structures or vibrations of a fluid in a closed cavity. Their solution consumes a large amount of memory and CPU time if more than very few (1-5) vibrational modes are desired. Both make the problem a natural candidate for parallel processing. Here we shall present some aspects of the solution of the generalized eigenvalue problem on parallel architectures with distributed memory. The research was carried out as a part of the EC founded project ACTIVATE. This project combines various activities to increase the performance vibro-acoustic software. It brings together end users form space, aviation, and automotive industry with software developers and numerical analysts. Introduction The algorithm we shall describe in the next sections is based on the Lanczos algorithm for solving large sparse eigenvalue problems. It became popular during the past two decades because of its superior convergence properties compared to more traditional methods like inverse vector iteration. Some experiments with an new algorithm described in [Bra97] showed that it is competitive with the Lanczos algorithm only if very few eigenpairs are needed. Therefore we decided to base our implementation on the Lanczos algorithm. However, when implementing the Lanczos algorithm one has to pay attention to some subtle algorithmic details. A state of the art algorithm is described in [Gri94]. On parallel architectures further problems arise concerning the robustness of the algorithm. We implemented the algorithm almost entirely by using numerical libraries.