DOCSLIB.ORG

Solution Methods for the Generalized Eigenvalue Problem

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Solution Methods for the Generalized Eigenvalue Problem AA242B: MECHANICAL VIBRATIONS 1 / 18 AA242B: MECHANICAL VIBRATIONS Solution Methods for the Generalized Eigenvalue Problem These slides are based on the recommended textbook: M. G´eradinand D. Rixen, \Mechanical Vibrations: Theory and Applications to Structural Dynamics," Second Edition, Wiley, John & Sons, Incorporated, ISBN-13:9780471975465 1 / 18 AA242B: MECHANICAL VIBRATIONS 2 / 18 Outline 1 Eigenvector Iteration Methods 2 Subspace Construction Methods 2 / 18 AA242B: MECHANICAL VIBRATIONS 3 / 18 Eigenvector Iteration Methods Introduction Generalized eigenvalue problem associated with an n-degree-of-freedom system Kx = !2Mx n eigenvalues 2 2 2 0 ≤ !1 ≤ !2 ≤ · · · ≤ !n and n associated eigenvectors x1; x2; ··· ; xn 3 / 18 AA242B: MECHANICAL VIBRATIONS 4 / 18 Eigenvector Iteration Methods Introduction Dynamic flexibility matrix consider a system that has no rigid-body modes ) K is non-singular construct the dynamic flexibility matrix defined as D = K−1M the associated eigenvalue problem is 1 Kx = !2Mx ) Dx = λx with λ = !2 eigenvectors are same as for the generalized eigenproblem Kx = !2Mx eigenvalues are given by 1 λi = 2 ) λ1 ≥ λ2 ≥ · · · ≥ λn !i drawbacks of working with D building cost is O(n3) non-symmetric matrix and as such must be stored completely these drawbacks can be dealt with (for example, see next the inverse iteration form of the power iteration algorithm) 4 / 18 AA242B: MECHANICAL VIBRATIONS 5 / 18 Eigenvector Iteration Methods The Power Algorithm Determination of the fundamental eigenmode an iterative solution to Dx = λx can be obtained by considering the iteration zp+1 = Dzp proof: expand z0 in the basis of the eigenmodes of the system n X z0 = αi xi i=1 Then, at the p-th iteration, n n n p ! p X p X p p X λi zp = D z0 = αi D xi = αi λi xi = λ1 α1x1 + αi xi λ1 i=1 i=1 i=2 Recall that λ1 ≥ λ2 ≥ · · · ≥ λn and assume that λ1 > λ2 p kzp+1k =) zp ! λ1 α1x1 ) ! λ1 as p ! 1 kzpk 2 =) the power iteration algorithm converges to the lowest eigenvalue !1 5 / 18 AA242B: MECHANICAL VIBRATIONS 6 / 18 Eigenvector Iteration Methods The Power Algorithm Convergence analysis of the power iteration algorithm p p 8p, zp = λ1 x1 + λ2 x2 + ··· (recall that xi is defined up to a multiplicative constant) T p+1 T p+1 T p+1 p+1 T z ej λ x ej + λ x ej + ··· λ (x1 + r x2 + ··· ) ej =) p+1 = 1 1 2 2 = 1 T p T p T p p T zp ej λ1 x1 ej + λ2 x2 ej + ··· λ1 (x1 + r x2 + ··· ) ej p+1 T (x1 + r x2 + ··· ) e = λ j 1 p T (x1 + r x2 + ··· ) ej λ2 where r = λ1 xT e define a = 2 j and assume that jλ j < jλ j ) r < 1 j T 2 1 x1 ej T p+1 z ej λ1(1 + a r + ··· ) λ = p+1 = j ≈ λ (1 + a r p+1)(1 + a r p )−1 1p T p p!1 1 j j zp ej 1 + aj r + ··· p ≈p!1 λ1 1 + aj r (r − 1) number of iterations N after which λ1N is stabilized with t significant digits λ1N+1 − λ1N −t N −t t ≤ 10 ) aj r ≈ 10 ) N ≈ λ1 λ1N log λ2 =) N | and therefore the convergence rate of the power algorithm | depends only on the ratio of the first two eigenvalues 6 / 18 AA242B: MECHANICAL VIBRATIONS 7 / 18 Eigenvector Iteration Methods The Power Algorithm Determination of the higher modes by orthogonal deflation once x1 is determined, the following orthogonal projection operator 2 can be formed (orthogonal projection , P1 = P1 and Ker(P1) is M-orthogonal to span (P1)) T x1x1 M P1 = I − T () P1x1 = 0) x1 Mx1 next, the following starting vector can be constructed n ∗ X z0 = P1z0 = αi xi i=2 this vector does not have any component in the x1 \direction" ) ? the power method equipped with the starting vector z0 converges in principle to (λ2; x2) in practice, the mode x1 reappears in the iterations because of roundoff errors; to remediate this issue, the following algorithm can be used instead ∗ zp = P1zp ∗ zp+1 = Dzp this approach can be generalized to extract higher modes 7 / 18 AA242B: MECHANICAL VIBRATIONS 8 / 18 Eigenvector Iteration Methods The Power Algorithm Alternatively, one can construct a power iteration algorithm based on D = M−1K in which case the associated eigenvalue problem is Kx = !2Mx ) Dx = !2x the eigenvectors and eigenvalues of this problem are the same as for the generalized eigenproblem Kx = !2Mx the eigenvalues can be sorted and labeled as follows 2 2 2 !1 ≤ !2 ≤ · · · ≤ !n 2 the power iteration algorithm converges to !n { that is, to the highest eigenvalue the drawbacks of working with D are identical to those of working with D and can be dealt with in a similar way 8 / 18 AA242B: MECHANICAL VIBRATIONS 9 / 18 Eigenvector Iteration Methods The Inverse Iteration Method Problem: it is not desirable to construct the dynamical matrix D = K−1M (or D = M−1K) at the base of the power iteration algorithm Solution: power iteration approach using the more computationally −1 efficient iterate zp+1 = K (Mzp), which can be organized in two steps as follows yp = Mzp Kzp+1 = yp incurs the solution of an algebraic problem of the form Kq = g factorization-based algorithm 1 factorization of K (once) 2 forward and backward backward substitutions (repeated) Recall that convergence to the highest λ is convergence to the 1 lowest !2 (λ = ) !2 9 / 18 AA242B: MECHANICAL VIBRATIONS 10 / 18 Eigenvector Iteration Methods Inverse Iteration with Spectral Shifting Spectral shifting (by µ) of the eigenvalue problem 1 (K − µM)x = (!2 − µ)Mx ) (K − µM)−1Mx = x !2 − µ associated inverse iteration scheme yp = Mzp (K − µM)zp+1 = yp iterations n n n p X X −1 X αi z = α x ) z = α (K − µM) M x = x 0 i i p i i (!2 − µ)p i i=1 i=1 i=1 i 2 converge toward the eigenmode xr whose associated eigenvalue !r satisfies 2 n 2 o j!r − µj = min j!j − µj j and therefore is closest to µ, and this convergence accelerates as µ 2 approaches !r 10 / 18 AA242B: MECHANICAL VIBRATIONS 11 / 18 Subspace Construction Methods The Subspace Iteration Method The subspace concept let X denote the M-normalized solution of the generalized eigenvalue problem Kx = !2Mx T T 2 2 X MX = In; X KX = Λ = diag(!1 ; ··· ;!n) then span(X) = E of dimension n consider the first m n eigenmodes ? n×m X = x1 ··· xm 2 R ? ? ? span(X ) = E of dimension m; E ⊂ E m property 1: 8a 2 R ? ? z = X a 2 E m×m property 2: if A 2 R and det(A) 6= 0 span(X?A) = span(X?) 11 / 18 AA242B: MECHANICAL VIBRATIONS 12 / 18 Subspace Construction Methods The Subspace Iteration Method Interaction problem ? ? once any set Z of m n linearly independent vectors in E is ? n×m found (Z 2 R ) the first m eigensolutions are determined by forming and solving the reduced eigenproblem K?y = !2M?y ? ?T ? m×m ? ?T ? m×m where K = Z KZ 2 R and M = Z MZ 2 R m×m the matrix Y = y1 ··· ym 2 R is assembled ? ? n×m the matrix X = Z Y 2 R , which has the same span as the matrix Z?, is constructed 12 / 18 AA242B: MECHANICAL VIBRATIONS 13 / 18 Subspace Construction Methods The Subspace Iteration Method Using the inverse iteration algorithm, the subspace method n×m generates a sequence of matrices Z0; Z1; ··· ; Zk 2 R spanning ? ? ? the subspaces E0; E1; ··· ; Ek that converge toward ? ? ? E1 = E = span(X ) (iterative method on the Grassmann manifold G(m; n)) Algorithm iteration kernel Vk = MZk KZk+1 = Vk ? aim is convergence toward E of the subspace as a whole, and not of the individual vectors in Zk to make sure that the reduced-order bases Zi span subspaces of dimension m, orthogonality of the column vectors is ensured by one of two ways Gram-Schmidt orthogonalization re-orthogonalization with respect to the M-induced norm 13 / 18 AA242B: MECHANICAL VIBRATIONS 14 / 18 Subspace Construction Methods The Subspace Iteration Method Convergence analysis the subspaces spanned by the matrices Zk converge toward the ? subspace spanned by X , provided that Z0 is not orthogonal to any of the first m eigenvectors global convergence of the subspace iteration algorithm guarantees ? ? that subspace Ek ⊂ E converges toward E k the convergence rate toward each mode xj is of the order of O(rj ) where 2 !j rj = 2 !m+1 2 measures the ratio of the eigenvalue !j and that which is ? immediately outside the subspace E . This indicates that a higher convergence rate is obtained by using q > m iteration vectors (which increases computational cost however). A practical trade-off suggests to use q = min(2m; m + 8). considering the convergence rate mentioned above, multiple eigenvalues do not decrease the rate of convergence provided that 2 2 !q+1 > !m 14 / 18 AA242B: MECHANICAL VIBRATIONS 15 / 18 Subspace Construction Methods The Lanczos Method Principle of the method generate a subspace that includes the fundamental eigensolutions by applying the inverse iteration algorithm to a starting vector z0 construct the following Krylov sequence 2 −1 −1 z0; K Mz0; K M z0; ··· orthogonalize the terms of the sequence Properties (for the basic form of the algorithm) the search subspace is built column by column: its dimension is increased at each iteration fast convergence however, orthonormality rapidly deteriorates (and re-orthogonalization procedures can be computationally expensive) inability to detect multiple eigenvalues 15 / 18 AA242B: MECHANICAL VIBRATIONS 16 / 18 Subspace Construction Methods The Lanczos Method Algorithm iteration −1 γp+1xp+1 = K Mxp − αpxp − βp−1xp−1 where βp−1; αp; γp+1 are determined such that T xp+1Mxj = 0 j < p + 1 T xp+1Mxp+1 = 1 requiring M-orthogonality leads to 8 T −1 αp = xp MK Mxp < T −1 βp−1 = xp MK Mxp−1 : γp = βp−1 n×p the reduced-order basis X = x0 ··· xp 2 R satisfies −1 T K MX = XT + S = 0 ··· 0 γp+1xp+1 = γp+1xp+1ep where T = tridiag γ1 ··· γp ; α0 ··· αp ; γ1 ··· γp 16 / 18 AA242B: MECHANICAL VIBRATIONS 17 / 18 Subspace Construction Methods The Lanczos Method Interaction problem start with K−1MX = XT + S premultiply by YT = (MX)T associated to the Lanczos vectors X note that YT S = 0 ) YT K−1MX = YT XT = T hence, the tridiagonal matrix of the Lanczos coefficients results from the projection of D = K−1M onto the subspace of Lanczos vectors X.
Load more

Recommended publications
  • A Geometric Theory for Preconditioned Inverse Iteration. III: a Short and Sharp Convergence Estimate for Generalized Eigenvalue Problems
    A geometric theory for preconditioned inverse iteration. III: A short and sharp convergence estimate for generalized eigenvalue problems. Andrew V. Knyazev Department of Mathematics, University of Colorado at Denver, P.O. Box 173364, Campus Box 170, Denver, CO 80217-3364 1 Klaus Neymeyr Mathematisches Institut der Universit¨atT¨ubingen,Auf der Morgenstelle 10, 72076 T¨ubingen,Germany 2 Abstract In two previous papers by Neymeyr: A geometric theory for preconditioned inverse iteration I: Extrema of the Rayleigh quotient, LAA 322: (1-3), 61-85, 2001, and A geometric theory for preconditioned inverse iteration II: Convergence estimates, LAA 322: (1-3), 87-104, 2001, a sharp, but cumbersome, convergence rate estimate was proved for a simple preconditioned eigensolver, which computes the smallest eigenvalue together with the corresponding eigenvector of a symmetric positive def- inite matrix, using a preconditioned gradient minimization of the Rayleigh quotient. In the present paper, we discover and prove a much shorter and more elegant, but still sharp in decisive quantities, convergence rate estimate of the same method that also holds for a generalized symmetric definite eigenvalue problem. The new estimate is simple enough to stimulate a search for a more straightforward proof technique that could be helpful to investigate such practically important method as the locally optimal block preconditioned conjugate gradient eigensolver. We demon- strate practical effectiveness of the latter for a model problem, where it compares favorably with two
    [Show full text]
  • The Godunov–Inverse Iteration: a Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem
    The Godunov{Inverse Iteration: A Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem Anna M. Matsekh a;1 aInstitute of Computational Technologies, Siberian Branch of the Russian Academy of Sciences, Lavrentiev Ave. 6, Novosibirsk 630090, Russia Abstract We present a new hybrid algorithm based on Godunov's method for computing eigenvectors of symmetric tridiagonal matrices and Inverse Iteration, which we call the Godunov{Inverse Iteration. We use eigenvectors computed according to Go- dunov's method as starting vectors in the Inverse Iteration, replacing any non- numeric elements of Godunov's eigenvectors with random uniform numbers. We use the right-hand bounds of the Ritz intervals found by the bisection method as Inverse Iteration shifts, while staying within guaranteed error bounds. In most test cases convergence is reached after only one step of the iteration, producing error estimates that are as good as or superior to those produced by standard Inverse Iteration implementations. Key words: Symmetric eigenvalue problem, tridiagonal matrices, Inverse Iteration 1 Introduction Construction of algorithms that enable to find all eigenvectors of the symmet- ric tridiagonal eigenvalue problem with guaranteed accuracy in O(n2) arith- metic operations has become one of the most pressing problems of the modern numerical algebra. QR method, one of the most accurate methods for solv- ing eigenvalue problems, requires 6n3 arithmetic operations and O(n2) square root operations to compute all eigenvectors of a tridiagonal matrix [Golub and Email address: [email protected] (Anna M. Matsekh ). 1 Present address: Modeling, Algorithms and Informatics Group, Los Alamos Na- tional Laboratory. P.O. Box 1663, MS B256, Los Alamos, NM 87544, USA Preprint submitted to Elsevier Science 31 January 2003 Loan (1996)].
    [Show full text]
  • A Note on Inverse Iteration
    A Note on Inverse Iteration Klaus Neymeyr Universit¨atRostock, Fachbereich Mathematik, Universit¨atsplatz 1, 18051 Rostock, Germany; SUMMARY Inverse iteration, if applied to a symmetric positive definite matrix, is shown to generate a sequence of iterates with monotonously decreasing Rayleigh quotients. We present sharp bounds from above and from below which highlight inverse iteration as a descent scheme for the Rayleigh quotient. Such estimates provide the background for the analysis of the behavior of the Rayleigh quotient in certain approximate variants of inverse iteration. key words: Symmetric eigenvalue problem; Inverse iteration; Rayleigh quotient. 1. Introduction Inverse iteration is a well-known iterative procedure to compute approximations of eigenfunctions and eigenvalues of linear operators. It was introduced by Wielandt in 1944 in a sequence of five papers, see [1], to treat the matrix eigenvalue problem Axi = λixi for a real or complex square matrix A. The scalar λi is the ith eigenvalue and the vector xi denotes a corresponding eigenvector. Given a nonzero starting vector x(0), inverse iteration generates a sequence of iterates x(k) by solving the linear systems (A − σI)x(k+1) = x(k), k = 0, 1, 2,..., (1) where σ denotes an eigenvalue approximation and I is the identity matrix. In practice, the iterates are normalized after each step. If A is a symmetric matrix, then the iterates x(k) converge to an eigenvector associated with an eigenvalue closest to σ if the starting vector x(0) is not perpendicular to that vector. For non-symmetric matrices the issue of starting vectors is discussed in Sec.
    [Show full text]
  • Inverse, Shifted Inverse, and Rayleigh Quotient Iteration As Newton’S Method⇤
    INVERSE, SHIFTED INVERSE, AND RAYLEIGH QUOTIENT ITERATION AS NEWTON’S METHOD⇤ JOHN DENNIS JR.† AND RICHARD TAPIA‡ Abstract. Two-norm normalized inverse, shifted inverse, and Rayleigh quotient iteration are well-known algorithms for approximating an eigenvector of a symmetric matrix. In this work we establish rigorously that each one of these three algorithms can be viewed as a standard form of Newton’s method from the nonlinear programming literature, followed by the normalization. This equivalence adds considerable understanding to the formal structure of inverse, shifted inverse, and Rayleigh quotient iteration and provides an explanation for their good behavior despite the possible need to solve systems with nearly singular coefficient matrices; the algorithms have what can be viewed as removable singularities. A thorough historical development of these eigenvalue algorithms is presented. Utilizing our equivalences we construct traditional Newton’s method theory analysis in order to gain understanding as to why, as normalized Newton’s method, inverse iteration and shifted inverse iteration are only linearly convergent and not quadratically convergent, and why a new linear system need not be solved at each iteration. We also investigate why Rayleigh quotient iteration is cubically convergent and not just quadratically convergent. Key words. inverse power method, inverse iteration, shifted inverse iteration, Rayleigh quotient iteration, Newton’s method AMS subject classifications. 65F15, 49M37, 49M15, 65K05 1. Introduction. The initial
    [Show full text]
  • Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2
    ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU Abstract. We examine some numerical iterative methods for computing the eigenvalues and eigenvectors of real matrices. The five methods examined here range from the simple power iteration method to the more complicated QR iteration method. The derivations, procedure, and advantages of each method are briefly discussed. 1. Introduction Eigenvalues and eigenvectors play an important part in the applications of linear algebra. The naive method of finding the eigenvalues of a matrix involves finding the roots of the characteristic polynomial of the matrix. In industrial sized matrices, however, this method is not feasible, and the eigenvalues must be obtained by other means. Fortunately, there exist several other techniques for finding eigenvalues and eigenvectors of a matrix, some of which fall under the realm of iterative methods. These methods work by repeatedly refining approximations to the eigenvectors or eigenvalues, and can be terminated whenever the approximations reach a suitable degree of accuracy. Iterative methods form the basis of much of modern day eigenvalue computation. In this paper, we outline five such iterative methods, and summarize their derivations, procedures, and advantages. The methods to be examined are the power iteration method, the shifted inverse iteration method, the Rayleigh quotient method, the simultaneous iteration method, and the QR method. This paper is meant to be a survey over existing algorithms for the eigenvalue computation problem. Section 2 of this paper provides a brief review of some of the linear algebra background required to understand the concepts that are discussed. In section 3, the iterative methods are each presented, in order of complexity, and are studied in brief detail.
    [Show full text]
  • Preconditioned Inverse Iteration and Shift-Invert Arnoldi Method
    Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical Systems Magdeburg 3rd May 2011 joint work with Alastair Spence (Bath) 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions Outline 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Iterative solves Power method Simultaneous iteration Arnoldi method Jacobi-Davidson method repeated application of the matrix A to a vector Generally convergence to largest/outlying eigenvector Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Iterative solves Power method Simultaneous iteration Arnoldi method Jacobi-Davidson method The first three of these involve repeated application of the matrix A to a vector Generally convergence to largest/outlying eigenvector Shift-invert strategy Wish to find a few eigenvalues close to a shift σ σ λλ λ λ λ 3 1 2 4 n Shift-invert strategy Wish to find a few eigenvalues close to a shift σ σ λλ λ λ λ 3 1 2 4 n Problem becomes − 1 (A − σI) 1x = x λ − σ each step of the iterative method involves repeated application of A =(A − σI)−1 to a vector Inner iterative solve: (A − σI)y = x using Krylov method for linear systems.
    [Show full text]
  • Lecture 12 — February 26 and 28 12.1 Introduction
    EE 381V: Large Scale Learning Spring 2013 Lecture 12 | February 26 and 28 Lecturer: Caramanis & Sanghavi Scribe: Karthikeyan Shanmugam and Natalia Arzeno 12.1 Introduction In this lecture, we focus on algorithms that compute the eigenvalues and eigenvectors of a real symmetric matrix. Particularly, we are interested in finding the largest and smallest eigenvalues and the corresponding eigenvectors. We study two methods: Power method and the Lanczos iteration. The first involves multiplying the symmetric matrix by a randomly chosen vector, and iteratively normalizing and multiplying the matrix by the normalized vector from the previous step. The convergence is geometric, i.e. the `1 distance between the true and the computed largest eigenvalue at the end of every step falls geometrically in the number of iterations and the rate depends on the ratio between the second largest and the largest eigenvalue. Some generalizations of the power method to compute the largest k eigenvalues and the eigenvectors will be discussed. The second method (Lanczos iteration) terminates in n iterations where each iteration in- volves estimating the largest (smallest) eigenvalue by maximizing (minimizing) the Rayleigh coefficient over vectors drawn from a suitable subspace. At each iteration, the dimension of the subspace involved in the optimization increases by 1. The sequence of subspaces used are Krylov subspaces associated with a random initial vector. We study the relation between Krylov subspaces and tri-diagonalization of a real symmetric matrix. Using this connection, we show that an estimate of the extreme eigenvalues can be computed at each iteration which involves eigen-decomposition of a tri-diagonal matrix.
    [Show full text]
  • AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration
    AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Solving Eigenvalue Problems All eigenvalue solvers must be iterative Iterative algorithms have multiple facets: 1 Basic idea behind the algorithms 2 Convergence and techniques to speed-up convergence 3 Efficiency of implementation 4 Termination criteria We will focus on first two aspects Xiangmin Jiao Numerical Analysis I 2 / 10 Simplification: Real Symmetric Matrices We will consider eigenvalue problems for real symmetric matrices, i.e. T m×m m A = A 2 R , and Ax = λx for x 2 R p ∗ T T I Note: x = x , and kxk = x x A has real eigenvalues λ1,λ2, ::: , λm and orthonormal eigenvectors q1, q2, ::: , qm, where kqj k = 1 Eigenvalues are often also ordered in a particular way (e.g., ordered from large to small in magnitude) In addition, we focus on symmetric tridiagonal form I Why? Because phase 1 of two-phase algorithm reduces matrix into tridiagonal form Xiangmin Jiao Numerical Analysis I 3 / 10 Rayleigh Quotient m The Rayleigh quotient of x 2 R is the scalar xT Ax r(x) = xT x For an eigenvector x, its Rayleigh quotient is r(x) = xT λx=xT x = λ, the corresponding eigenvalue of x For general x, r(x) = α that minimizes kAx − αxk2. 2 x is eigenvector of A() rr(x) = x T x (Ax − r(x)x) = 0 with x 6= 0 r(x) is smooth and rr(qj ) = 0 for any j, and therefore is quadratically accurate: 2 r(x) − r(qJ ) = O(kx − qJ k ) as x ! qJ for some J Xiangmin Jiao Numerical Analysis I 4 / 10 Power
    [Show full text]
  • Randomized Numerical Linear Algebra: Foundations & Algorithms
    Randomized Numerical Linear Algebra: Foundations & Algorithms Per-Gunnar Martinsson, University of Texas at Austin Joel A. Tropp, California Institute of Technology Abstract: This survey describes probabilistic algorithms for linear algebra computations, such as factorizing matrices and solving linear systems. It focuses on techniques that have a proven track record for real-world problems. The paper treats both the theoretical foundations of the subject and practical computational issues. Topics include norm estimation; matrix approximation by sampling; structured and un- structured random embeddings; linear regression problems; low-rank approximation; sub- space iteration and Krylov methods; error estimation and adaptivity; interpolatory and CUR factorizations; Nystrom¨ approximation of positive semidefinite matrices; single- view (“streaming”) algorithms; full rank-revealing factorizations; solvers for linear sys- tems; and approximation of kernel matrices that arise in machine learning and in scientific computing. CONTENTS 1. Introduction 2 2. Linear algebra preliminaries 6 3. Probability preliminaries 9 4. Trace estimation by sampling 10 5. Schatten p-norm estimation by sampling 16 6. Maximum eigenvalues and trace functions 18 7. Matrix approximation by sampling 24 8. Randomized embeddings 30 9. Structured random embeddings 35 10. How to use random embeddings 40 arXiv:2002.01387v2 [math.NA] 15 Aug 2020 11. The randomized rangefinder 43 12. Error estimation and adaptivity 52 13. Finding natural bases: QR, ID, and CUR 57 14. Nystrom¨ approximation 62 15. Single-view algorithms 64 16. Factoring matrices of full or nearly full rank 68 17. General linear solvers 76 18. Linear solvers for graph Laplacians 79 19. Kernel matrices in machine learning 85 20. High-accuracy approximation of kernel matrices 94 References 106 Randomized Numerical Linear Algebra P.G.
    [Show full text]
  • Computing Selected Eigenvalues of Sparse Unsymmetric Matrices Using Subspace Iteration
    RAL-91-056 Computing selected eigenvalues of sparse unsymmetric matrices using subspace iteration by I. S. Duff and J. A. Scott ABSTRACT This paper discusses the design and development of a code to calculate the eigenvalues of a large sparse real unsymmetric matrix that are the right-most, left-most, or are of largest modulus. A subspace iteration algorithm is used to compute a sequence of sets of vectors that converge to an orthonormal basis for the invariant subspace corresponding to the required eigenvalues. This algorithm is combined with Chebychev acceleration if the right-most or left-most eigenvalues are sought, or if the eigenvalues of largest modulus are known to be the right-most or left-most eigenvalues. An option exists for computing the corresponding eigenvectors. The code does not need the matrix explicitly since it only requires the user to multiply sets of vectors by the matrix. Sophisticated and novel iteration controls, stopping criteria, and restart facilities are provided. The code is shown to be efficient and competitive on a range of test problems. Central Computing Department, Atlas Centre, Rutherford Appleton Laboratory, Oxon OX11 0QX. August 1993. 1 Introduction We are concerned with the problem of computing selected eigenvalues and the corresponding eigenvectors of a large sparse real unsymmetric matrix. In particular, we are interested in computing either the eigenvalues of largest modulus or the right-most (or left-most) eigenvalues. This problem arises in a significant number of applications, including mathematical models in economics, Markov chain modelling of queueing networks, and bifurcation problems (for references, see Saad 1984).
    [Show full text]
  • Arxiv:1803.03717V1 [Math.NA] 9 Mar 2018 Pression Techniques, the Iterates Are Forced to Stay in Low-Rank Format
    LOW-RANK SOLUTION METHODS FOR STOCHASTIC EIGENVALUE PROBLEMS∗ HOWARD C. ELMANy AND TENGFEI SUz Abstract. We study efficient solution methods for stochastic eigenvalue problems arising from discretization of self-adjoint partial differential equations with random data. With the stochastic Galerkin approach, the solutions are represented as generalized polynomial chaos expansions. A low-rank variant of the inverse subspace iteration algorithm is presented for computing one or sev- eral minimal eigenvalues and corresponding eigenvectors of parameter-dependent matrices. In the algorithm, the iterates are approximated by low-rank matrices, which leads to significant cost sav- ings. The algorithm is tested on two benchmark problems, a stochastic diffusion problem with some poorly separated eigenvalues, and an operator derived from a discrete stochastic Stokes problem whose minimal eigenvalue is related to the inf-sup stability constant. Numerical experiments show that the low-rank algorithm produces accurate solutions compared to the Monte Carlo method, and it uses much less computational time than the original algorithm without low-rank approximation. Key words. stochastic eigenvalue problem, inverse subspace iteration, low-rank approximation AMS subject classifications. 35R60, 65F15, 65F18, 65N22 1. Introduction. Approaches for solving stochastic eigenvalue problems can be broadly divided into non-intrusive methods, including Monte Carlo methods and stochastic collocation methods [1, 30], and intrusive stochastic Galerkin methods. The Galerkin approach gives parametrized descriptions of the eigenvalues and eigen- vectors, represented as expansions with stochastic basis functions. A commonly used framework is the generalized polynomial chaos (gPC) expansion [40]. A direct projec- tion onto the basis functions will result in large coupled nonlinear systems that can be solved by a Newton-type algorithm [6, 13].
    [Show full text]
  • AM 205: Lecture 22
    AM 205: lecture 22 I Final project proposal due by 6pm on Thu Nov 17. Email Chris or the TFs to set up a meeting. Those who have completed this will see four proposal points on Canvas. I Today: eigenvalue algorithms, QR algorithm Sensitivity of Eigenvalue Problems Weyl's Theorem: Let λ1 ≤ λ2 ≤ · · · ≤ λn and λ~1 ≤ λ~2 ≤ · · · ≤ λ~n be the eigenvalues of hermitian matrices A and A + δA, respectively. Then max jλi − λ~i j ≤ kδAk2. i=1;:::;n Hence in the hermitian case, each perturbed eigenvalue must be in the disk1 of its corresponding unperturbed eigenvalue! 1In fact, eigenvalues of a hermitian matrix are real, so disk here is actually an interval in R Sensitivity of Eigenvalue Problems The Bauer{Fike Theorem relates to perturbations of the whole spectrum We can also consider perturbations of individual eigenvalues n×n Suppose, for simplicity, that A 2 C is symmetric, and consider the perturbed eigenvalue problem (A + E)(v + ∆v) = (λ + ∆λ)(v + ∆v) Expanding this equation, dropping second order terms, and using Av = λv gives A∆v + Ev ≈ ∆λv + λ∆v Sensitivity of Eigenvalue Problems Premultiply A∆v + Ev ≈ ∆λv + λ∆v by v ∗ to obtain v ∗A∆v + v ∗Ev ≈ ∆λv ∗v + λv ∗∆v Noting that v ∗A∆v = (v ∗A∆v)∗ = ∆v ∗Av = λ∆v ∗v = λv ∗∆v leads to v ∗Ev v ∗Ev ≈ ∆λv ∗v; or∆ λ = v ∗v Sensitivity of Eigenvalue Problems Finally, we obtain ∗ jv Evj kvk2kEvk2 j∆λj ≈ 2 ≤ 2 = kEk2; kvk2 kvk2 so that j∆λj . kEk2 We observe that I perturbation bound does not depend on cond(V ) when we consider only an individual eigenvalue I this individual eigenvalue perturbation bound
    [Show full text]