DOCSLIB.ORG

Preconditioned Inverse Iteration and Shift-Invert Arnoldi Method

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Preconditioned Inverse Iteration and Shift-Invert Arnoldi Method Preconditioned inverse iteration and shift-invert Arnoldi method Melina Freitag Department of Mathematical Sciences University of Bath CSC Seminar Max-Planck-Institute for Dynamics of Complex Technical Systems Magdeburg 3rd May 2011 joint work with Alastair Spence (Bath) 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions Outline 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Iterative solves Power method Simultaneous iteration Arnoldi method Jacobi-Davidson method repeated application of the matrix A to a vector Generally convergence to largest/outlying eigenvector Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Iterative solves Power method Simultaneous iteration Arnoldi method Jacobi-Davidson method The first three of these involve repeated application of the matrix A to a vector Generally convergence to largest/outlying eigenvector Shift-invert strategy Wish to find a few eigenvalues close to a shift σ σ λλ λ λ λ 3 1 2 4 n Shift-invert strategy Wish to find a few eigenvalues close to a shift σ σ λλ λ λ λ 3 1 2 4 n Problem becomes − 1 (A − σI) 1x = x λ − σ each step of the iterative method involves repeated application of A =(A − σI)−1 to a vector Inner iterative solve: (A − σI)y = x using Krylov method for linear systems. leading to inner-outer iterative method. Shift-invert strategy This talk: Inner iteration and preconditioning Fixed shifts only Inverse iteration and Arnoldi method Outline 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions The algorithm Inexact inverse iteration for i =1 to . do choose τ (i) solve (A − σI)y(i) − x(i) = d(i)≤ τ (i), y(i) Rescale x(i+1) = , y(i) H Update λ(i+1) = x(i+1) Ax(i+1), Test: eigenvalue residual r(i+1) =(A − λ(i+1)I)x(i+1). end for The algorithm Inexact inverse iteration for i =1 to . do choose τ (i) solve (A − σI)y(i) − x(i) = d(i)≤ τ (i), y(i) Rescale x(i+1) = , y(i) H Update λ(i+1) = x(i+1) Ax(i+1), Test: eigenvalue residual r(i+1) =(A − λ(i+1)I)x(i+1). end for Convergence rates If τ (i) = Cr(i) then convergence rate is linear (same convergence rate as for exact solves). The inner iteration for (A − σI)y = x Standard GMRES theory for y0 = 0 and A diagonalisable x − (A − σI)yk≤ κ(W ) min max |p(λj )|x p∈Pk j=1,...,n −1 where λj are eigenvalues of A − σI and (A − σI)= W ΛW . The inner iteration for (A − σI)y = x Standard GMRES theory for y0 = 0 and A diagonalisable x − (A − σI)yk≤ κ(W ) min max |p(λj )|x p∈Pk j=1,...,n −1 where λj are eigenvalues of A − σI and (A − σI)= W ΛW . Number of inner iterations x k ≥ C + C log 1 2 τ for x − (A − σI)yk≤ τ. The inner iteration for (A − σI)y = x More detailed GMRES theory for y0 = 0 |λ2 − λ1| x − (A − σI)yk≤ κ˜(W ) min max |p(λj )|Qx λ1 p∈Pk−1 j=2,...,n where λj are eigenvalues of A − σI. Number of inner iterations ′ ′ Qx k ≥ C + C log , 1 2 τ where Q projects onto the space not spanned by the eigenvector. The inner iteration for (A − σI)y = x Good news! (i) ′ ′ C r k(i) ≥ C + C log 3 , 1 2 τ (i) where τ (i) = Cr(i). Iteration number approximately constant! The inner iteration for (A − σI)y = x Good news! (i) ′ ′ C r k(i) ≥ C + C log 3 , 1 2 τ (i) where τ (i) = Cr(i). Iteration number approximately constant! Bad news :-( For a standard preconditioner P − − (A − σI)P 1y˜(i) = x(i) P 1y˜(i) = y(i) (i) ′′ ′′ Q˜x ′′ ′′ C k(i) ≥ C + C log = C + C log , 1 2 τ (i) 1 2 τ (i) where τ (i) = Cr(i). Iteration number increases! Convection-Diffusion operator Finite difference discretisation on a 32 × 32 grid of the convection-diffusion operator 2 −∆u + 5ux + 5uy = λu on (0, 1) , with homogeneous Dirichlet boundary conditions (961 × 961 matrix). smallest eigenvalue: λ1 ≈ 32.18560954, Preconditioned GMRES with tolerance τ (i) = 0.01r(i), standard and tuned preconditioner (incomplete LU). Convection-Diffusion operator right preconditioning, 569 iterations right preconditioning, 569 iterations 0 45 10 40 −2 10 35 −4 30 10 25 −6 10 20 inner iterations 15 residual norms \|r^{(i)}\| −8 10 10 −10 5 10 2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations The inner iteration for (A − σI)P −1y˜ = x How to overcome this problem Use a different preconditioner, namely one that satisfies H (i) (i) (i) (i) Pix = Ax , Pi := P +(A − P )x x minor modification and minor extra computational cost, P−1 (i) (i) [A i ]Ax = Ax . The inner iteration for (A − σI)P −1y˜ = x How to overcome this problem Use a different preconditioner, namely one that satisfies H (i) (i) (i) (i) Pix = Ax , Pi := P +(A − P )x x minor modification and minor extra computational cost, P−1 (i) (i) [A i ]Ax = Ax . Why does that work? Assume we have found eigenvector x1 −1 λ1 − σ Ax1 = Px1 = λ1x1 ⇒ (A − σI)P x1 = x1 λ1 −1 and convergence of Krylov method applied to (A − σI)P y˜ = x1 in one iteration. For general x(i) (i) ′′ ′′ C r k(i) ≥ C + C log 3 , where τ (i) = Cr(i). 1 2 τ (i) Convection-Diffusion operator Finite difference discretisation on a 32 × 32 grid of the convection-diffusion operator 2 −∆u + 5ux + 5uy = λu on (0, 1) , with homogeneous Dirichlet boundary conditions (961 × 961 matrix). smallest eigenvalue: λ1 ≈ 32.18560954, Preconditioned GMRES with tolerance τ (i) = 0.01r(i), standard and tuned preconditioner (incomplete LU). Convection-Diffusion operator right preconditioning, 569 iterations right preconditioning, 569 iterations 0 45 10 40 −2 10 35 −4 30 10 25 −6 10 20 inner iterations 15 residual norms \|r^{(i)}\| −8 10 10 −10 5 10 2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations Convection-Diffusion operator right preconditioning, 569 iterations right preconditioning, 569 iterations 0 10 tuned right preconditioning, 115 iterations 45 tuned right preconditioning, 115 iterations 40 −2 10 35 −4 30 10 25 −6 10 20 inner iterations 15 residual norms \|r^{(i)}\| −8 10 10 −10 5 10 2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations Outline 1 Introduction 2 Inexact inverse iteration 3 Inexact Shift-invert Arnoldi method 4 Inexact Shift-invert Arnoldi method with implicit restarts 5 Conclusions The algorithm Arnoldi’s method Arnoldi method constructs an orthogonal basis of k-dimensional Krylov subspace (1) (1) (1) 2 (1) k−1 (1) Kk(A, q ) = span{q , Aq , A q ,..., A q }, H AQ = Q H + q h eH = Q k k k k k+1 k+1,k k k+1 h eH k+1,k k H Qk Qk = I. The algorithm Arnoldi’s method Arnoldi method constructs an orthogonal basis of k-dimensional Krylov subspace (1) (1) (1) 2 (1) k−1 (1) Kk(A, q ) = span{q , Aq , A q ,..., A q }, H AQ = Q H + q h eH = Q k k k k k+1 k+1,k k k+1 h eH k+1,k k H Qk Qk = I. Eigenvalues of Hk are eigenvalue approximations of (outlying) eigenvalues of A H rk = Ax − θx = (AQk − QkHk)u = |hk+1,k||ek u|, The algorithm Arnoldi’s method Arnoldi method constructs an orthogonal basis of k-dimensional Krylov subspace (1) (1) (1) 2 (1) k−1 (1) Kk(A, q ) = span{q , Aq , A q ,..., A q }, H AQ = Q H + q h eH = Q k k k k k+1 k+1,k k k+1 h eH k+1,k k H Qk Qk = I. Eigenvalues of Hk are eigenvalue approximations of (outlying) eigenvalues of A H rk = Ax − θx = (AQk − QkHk)u = |hk+1,k||ek u|, at each step, application of A to qk: Aqk =q ˜k+1 Example random complex matrix of dimension n = 144 generated in Matlab: G=numgrid(’N’,14);B=delsq(G);A=sprandn(B)+i*sprandn(B) eigenvalues of A 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 5 Arnoldi steps Arnoldi after 5 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 10 Arnoldi steps Arnoldi after 10 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 15 Arnoldi steps Arnoldi after 15 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 20 Arnoldi steps Arnoldi after 20 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 25 Arnoldi steps Arnoldi after 25 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 after 30 Arnoldi steps Arnoldi after 30 steps 3 2 1 0 −1 −2 −3 −4 −3 −2 −1 0 1 2 3 4 The algorithm: take σ = 0 Shift-Invert Arnoldi’s method A := A−1 Arnoldi method constructs an orthogonal basis of k-dimensional Krylov subspace −1 (1) (1) −1 (1) −1 2 (1) −1 k−1 (1) Kk(A , q ) = span{q ,A q , (A ) q ,..., (A ) q }, −1 H Hk A Qk = QkHk + qk+1hk+1,kek = Qk+1 H hk ,ke +1 k H Qk Qk = I.
Load more

Recommended publications
  • Overview of Iterative Linear System Solver Packages
    Overview of Iterative Linear System Solver Packages Victor Eijkhout July, 1998 Abstract Description and comparison of several packages for the iterative solu- tion of linear systems of equations. 1 1 Intro duction There are several freely available packages for the iterative solution of linear systems of equations, typically derived from partial di erential equation prob- lems. In this rep ort I will give a brief description of a numberofpackages, and giveaninventory of their features and de ning characteristics. The most imp ortant features of the packages are which iterative metho ds and preconditioners supply; the most relevant de ning characteristics are the interface they present to the user's data structures, and their implementation language. 2 2 Discussion Iterative metho ds are sub ject to several design decisions that a ect ease of use of the software and the resulting p erformance. In this section I will give a global discussion of the issues involved, and how certain p oints are addressed in the packages under review. 2.1 Preconditioners A go o d preconditioner is necessary for the convergence of iterative metho ds as the problem to b e solved b ecomes more dicult. Go o d preconditioners are hard to design, and this esp ecially holds true in the case of parallel pro cessing. Here is a short inventory of the various kinds of preconditioners found in the packages reviewed. 2.1.1 Ab out incomplete factorisation preconditioners Incomplete factorisations are among the most successful preconditioners devel- op ed for single-pro cessor computers. Unfortunately, since they are implicit in nature, they cannot immediately b e used on parallel architectures.
    [Show full text]
  • A Geometric Theory for Preconditioned Inverse Iteration. III: a Short and Sharp Convergence Estimate for Generalized Eigenvalue Problems
    A geometric theory for preconditioned inverse iteration. III: A short and sharp convergence estimate for generalized eigenvalue problems. Andrew V. Knyazev Department of Mathematics, University of Colorado at Denver, P.O. Box 173364, Campus Box 170, Denver, CO 80217-3364 1 Klaus Neymeyr Mathematisches Institut der Universit¨atT¨ubingen,Auf der Morgenstelle 10, 72076 T¨ubingen,Germany 2 Abstract In two previous papers by Neymeyr: A geometric theory for preconditioned inverse iteration I: Extrema of the Rayleigh quotient, LAA 322: (1-3), 61-85, 2001, and A geometric theory for preconditioned inverse iteration II: Convergence estimates, LAA 322: (1-3), 87-104, 2001, a sharp, but cumbersome, convergence rate estimate was proved for a simple preconditioned eigensolver, which computes the smallest eigenvalue together with the corresponding eigenvector of a symmetric positive def- inite matrix, using a preconditioned gradient minimization of the Rayleigh quotient. In the present paper, we discover and prove a much shorter and more elegant, but still sharp in decisive quantities, convergence rate estimate of the same method that also holds for a generalized symmetric definite eigenvalue problem. The new estimate is simple enough to stimulate a search for a more straightforward proof technique that could be helpful to investigate such practically important method as the locally optimal block preconditioned conjugate gradient eigensolver. We demon- strate practical effectiveness of the latter for a model problem, where it compares favorably with two
    [Show full text]
  • 16 Preconditioning
    16 Preconditioning The general idea underlying any preconditioning procedure for iterative solvers is to modify the (ill-conditioned) system Ax = b in such a way that we obtain an equivalent system Aˆxˆ = bˆ for which the iterative method converges faster. A standard approach is to use a nonsingular matrix M, and rewrite the system as M −1Ax = M −1b. The preconditioner M needs to be chosen such that the matrix Aˆ = M −1A is better conditioned for the conjugate gradient method, or has better clustered eigenvalues for the GMRES method. 16.1 Preconditioned Conjugate Gradients We mentioned earlier that the number of iterations required for the conjugate gradient algorithm to converge is proportional to pκ(A). Thus, for poorly conditioned matrices, convergence will be very slow. Thus, clearly we will want to choose M such that κ(Aˆ) < κ(A). This should result in faster convergence. How do we find Aˆ, xˆ, and bˆ? In order to ensure symmetry and positive definiteness of Aˆ we let M −1 = LLT (44) with a nonsingular m × m matrix L. Then we can rewrite Ax = b ⇐⇒ M −1Ax = M −1b ⇐⇒ LT Ax = LT b ⇐⇒ LT AL L−1x = LT b . | {z } | {z } |{z} =Aˆ =xˆ =bˆ The symmetric positive definite matrix M is called splitting matrix or preconditioner, and it can easily be verified that Aˆ is symmetric positive definite, also. One could now formally write down the standard CG algorithm with the new “hat- ted” quantities. However, the algorithm is more efficient if the preconditioning is incorporated directly into the iteration. To see what this means we need to examine every single line in the CG algorithm.
    [Show full text]
  • The Godunov–Inverse Iteration: a Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem
    The Godunov{Inverse Iteration: A Fast and Accurate Solution to the Symmetric Tridiagonal Eigenvalue Problem Anna M. Matsekh a;1 aInstitute of Computational Technologies, Siberian Branch of the Russian Academy of Sciences, Lavrentiev Ave. 6, Novosibirsk 630090, Russia Abstract We present a new hybrid algorithm based on Godunov's method for computing eigenvectors of symmetric tridiagonal matrices and Inverse Iteration, which we call the Godunov{Inverse Iteration. We use eigenvectors computed according to Go- dunov's method as starting vectors in the Inverse Iteration, replacing any non- numeric elements of Godunov's eigenvectors with random uniform numbers. We use the right-hand bounds of the Ritz intervals found by the bisection method as Inverse Iteration shifts, while staying within guaranteed error bounds. In most test cases convergence is reached after only one step of the iteration, producing error estimates that are as good as or superior to those produced by standard Inverse Iteration implementations. Key words: Symmetric eigenvalue problem, tridiagonal matrices, Inverse Iteration 1 Introduction Construction of algorithms that enable to find all eigenvectors of the symmet- ric tridiagonal eigenvalue problem with guaranteed accuracy in O(n2) arith- metic operations has become one of the most pressing problems of the modern numerical algebra. QR method, one of the most accurate methods for solv- ing eigenvalue problems, requires 6n3 arithmetic operations and O(n2) square root operations to compute all eigenvectors of a tridiagonal matrix [Golub and Email address: [email protected] (Anna M. Matsekh ). 1 Present address: Modeling, Algorithms and Informatics Group, Los Alamos Na- tional Laboratory. P.O. Box 1663, MS B256, Los Alamos, NM 87544, USA Preprint submitted to Elsevier Science 31 January 2003 Loan (1996)].
    [Show full text]
  • Numerical Linear Algebra Solving Ax = B , an Overview Introduction
    Solving Ax = b, an overview Numerical Linear Algebra ∗ A = A no Good precond. yes flex. precond. yes GCR Improving iterative solvers: ⇒ ⇒ ⇒ yes no no ⇓ ⇓ ⇓ preconditioning, deflation, numerical yes GMRES A > 0 CG software and parallelisation ⇒ ⇓ no ⇓ ⇓ ill cond. yes SYMMLQ str indef no large im eig no Bi-CGSTAB ⇒ ⇒ ⇒ no yes yes ⇓ ⇓ ⇓ MINRES IDR no large im eig BiCGstab(ℓ) ⇐ yes Gerard Sleijpen and Martin van Gijzen a good precond itioner is available ⇓ the precond itioner is flex ible IDRstab November 29, 2017 A + A∗ is strongly indefinite A 1 has large imaginary eigenvalues November 29, 2017 2 National Master Course National Master Course Delft University of Technology Introduction Program We already saw that the performance of iterative methods can Preconditioning be improved by applying a preconditioner. Preconditioners (and • Diagonal scaling, Gauss-Seidel, SOR and SSOR deflation techniques) are a key to successful iterative methods. • Incomplete Choleski and Incomplete LU In general they are very problem dependent. • Deflation Today we will discuss some standard preconditioners and we will • Numerical software explain the idea behind deflation. • Parallelisation We will also discuss some efforts to standardise numerical • software. Shared memory versus distributed memory • Domain decomposition Finally we will discuss how to perform scientific computations on • a parallel computer. November 29, 2017 3 November 29, 2017 4 National Master Course National Master Course Preconditioning Why preconditioners? A preconditioned iterative solver solves the system − − M 1Ax = M 1b. 1 The matrix M is called the preconditioner. 0.8 The preconditioner should satisfy certain requirements: 0.6 Convergence should be (much) faster (in time) for the 0.4 • preconditioned system than for the original system.
    [Show full text]
  • A Note on Inverse Iteration
    A Note on Inverse Iteration Klaus Neymeyr Universit¨atRostock, Fachbereich Mathematik, Universit¨atsplatz 1, 18051 Rostock, Germany; SUMMARY Inverse iteration, if applied to a symmetric positive definite matrix, is shown to generate a sequence of iterates with monotonously decreasing Rayleigh quotients. We present sharp bounds from above and from below which highlight inverse iteration as a descent scheme for the Rayleigh quotient. Such estimates provide the background for the analysis of the behavior of the Rayleigh quotient in certain approximate variants of inverse iteration. key words: Symmetric eigenvalue problem; Inverse iteration; Rayleigh quotient. 1. Introduction Inverse iteration is a well-known iterative procedure to compute approximations of eigenfunctions and eigenvalues of linear operators. It was introduced by Wielandt in 1944 in a sequence of five papers, see [1], to treat the matrix eigenvalue problem Axi = λixi for a real or complex square matrix A. The scalar λi is the ith eigenvalue and the vector xi denotes a corresponding eigenvector. Given a nonzero starting vector x(0), inverse iteration generates a sequence of iterates x(k) by solving the linear systems (A − σI)x(k+1) = x(k), k = 0, 1, 2,..., (1) where σ denotes an eigenvalue approximation and I is the identity matrix. In practice, the iterates are normalized after each step. If A is a symmetric matrix, then the iterates x(k) converge to an eigenvector associated with an eigenvalue closest to σ if the starting vector x(0) is not perpendicular to that vector. For non-symmetric matrices the issue of starting vectors is discussed in Sec.
    [Show full text]
  • Inverse, Shifted Inverse, and Rayleigh Quotient Iteration As Newton’S Method⇤
    INVERSE, SHIFTED INVERSE, AND RAYLEIGH QUOTIENT ITERATION AS NEWTON’S METHOD⇤ JOHN DENNIS JR.† AND RICHARD TAPIA‡ Abstract. Two-norm normalized inverse, shifted inverse, and Rayleigh quotient iteration are well-known algorithms for approximating an eigenvector of a symmetric matrix. In this work we establish rigorously that each one of these three algorithms can be viewed as a standard form of Newton’s method from the nonlinear programming literature, followed by the normalization. This equivalence adds considerable understanding to the formal structure of inverse, shifted inverse, and Rayleigh quotient iteration and provides an explanation for their good behavior despite the possible need to solve systems with nearly singular coefficient matrices; the algorithms have what can be viewed as removable singularities. A thorough historical development of these eigenvalue algorithms is presented. Utilizing our equivalences we construct traditional Newton’s method theory analysis in order to gain understanding as to why, as normalized Newton’s method, inverse iteration and shifted inverse iteration are only linearly convergent and not quadratically convergent, and why a new linear system need not be solved at each iteration. We also investigate why Rayleigh quotient iteration is cubically convergent and not just quadratically convergent. Key words. inverse power method, inverse iteration, shifted inverse iteration, Rayleigh quotient iteration, Newton’s method AMS subject classifications. 65F15, 49M37, 49M15, 65K05 1. Introduction. The initial
    [Show full text]
  • Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2
    ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU Abstract. We examine some numerical iterative methods for computing the eigenvalues and eigenvectors of real matrices. The five methods examined here range from the simple power iteration method to the more complicated QR iteration method. The derivations, procedure, and advantages of each method are briefly discussed. 1. Introduction Eigenvalues and eigenvectors play an important part in the applications of linear algebra. The naive method of finding the eigenvalues of a matrix involves finding the roots of the characteristic polynomial of the matrix. In industrial sized matrices, however, this method is not feasible, and the eigenvalues must be obtained by other means. Fortunately, there exist several other techniques for finding eigenvalues and eigenvectors of a matrix, some of which fall under the realm of iterative methods. These methods work by repeatedly refining approximations to the eigenvectors or eigenvalues, and can be terminated whenever the approximations reach a suitable degree of accuracy. Iterative methods form the basis of much of modern day eigenvalue computation. In this paper, we outline five such iterative methods, and summarize their derivations, procedures, and advantages. The methods to be examined are the power iteration method, the shifted inverse iteration method, the Rayleigh quotient method, the simultaneous iteration method, and the QR method. This paper is meant to be a survey over existing algorithms for the eigenvalue computation problem. Section 2 of this paper provides a brief review of some of the linear algebra background required to understand the concepts that are discussed. In section 3, the iterative methods are each presented, in order of complexity, and are studied in brief detail.
    [Show full text]
  • Lecture 12 — February 26 and 28 12.1 Introduction
    EE 381V: Large Scale Learning Spring 2013 Lecture 12 | February 26 and 28 Lecturer: Caramanis & Sanghavi Scribe: Karthikeyan Shanmugam and Natalia Arzeno 12.1 Introduction In this lecture, we focus on algorithms that compute the eigenvalues and eigenvectors of a real symmetric matrix. Particularly, we are interested in finding the largest and smallest eigenvalues and the corresponding eigenvectors. We study two methods: Power method and the Lanczos iteration. The first involves multiplying the symmetric matrix by a randomly chosen vector, and iteratively normalizing and multiplying the matrix by the normalized vector from the previous step. The convergence is geometric, i.e. the `1 distance between the true and the computed largest eigenvalue at the end of every step falls geometrically in the number of iterations and the rate depends on the ratio between the second largest and the largest eigenvalue. Some generalizations of the power method to compute the largest k eigenvalues and the eigenvectors will be discussed. The second method (Lanczos iteration) terminates in n iterations where each iteration in- volves estimating the largest (smallest) eigenvalue by maximizing (minimizing) the Rayleigh coefficient over vectors drawn from a suitable subspace. At each iteration, the dimension of the subspace involved in the optimization increases by 1. The sequence of subspaces used are Krylov subspaces associated with a random initial vector. We study the relation between Krylov subspaces and tri-diagonalization of a real symmetric matrix. Using this connection, we show that an estimate of the extreme eigenvalues can be computed at each iteration which involves eigen-decomposition of a tri-diagonal matrix.
    [Show full text]
  • Preconditioning the Coarse Problem of BDDC Methods—Three-Level, Algebraic Multigrid, and Vertex-Based Preconditioners
    Electronic Transactions on Numerical Analysis. Volume 51, pp. 432–450, 2019. ETNA Kent State University and Copyright c 2019, Kent State University. Johann Radon Institute (RICAM) ISSN 1068–9613. DOI: 10.1553/etna_vol51s432 PRECONDITIONING THE COARSE PROBLEM OF BDDC METHODS— THREE-LEVEL, ALGEBRAIC MULTIGRID, AND VERTEX-BASED PRECONDITIONERS∗ AXEL KLAWONNyz, MARTIN LANSERyz, OLIVER RHEINBACHx, AND JANINE WEBERy Abstract. A comparison of three Balancing Domain Decomposition by Constraints (BDDC) methods with an approximate coarse space solver using the same software building blocks is attempted for the first time. The comparison is made for a BDDC method with an algebraic multigrid preconditioner for the coarse problem, a three-level BDDC method, and a BDDC method with a vertex-based coarse preconditioner. It is new that all methods are presented and discussed in a common framework. Condition number bounds are provided for all approaches. All methods are implemented in a common highly parallel scalable BDDC software package based on PETSc to allow for a simple and meaningful comparison. Numerical results showing the parallel scalability are presented for the equations of linear elasticity. For the first time, this includes parallel scalability tests for a vertex-based approximate BDDC method. Key words. approximate BDDC, three-level BDDC, multilevel BDDC, vertex-based BDDC AMS subject classifications. 68W10, 65N22, 65N55, 65F08, 65F10, 65Y05 1. Introduction. During the last decade, approximate variants of the BDDC (Balanc- ing Domain Decomposition by Constraints) and FETI-DP (Finite Element Tearing and Interconnecting-Dual-Primal) methods have become popular for the solution of various linear and nonlinear partial differential equations [1,8,9, 12, 14, 15, 17, 19, 21, 24, 25].
    [Show full text]
  • The Preconditioned Conjugate Gradient Method with Incomplete Factorization Preconditioners
    Computers Math. Applic. Vol. 31, No. 4/5, pp. 1-5, 1996 Pergamon Copyright©1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0898-1221/96 $15.00 4- 0.00 0898-1221 (95)00210-3 The Preconditioned Conjugate Gradient Method with Incomplete Factorization Preconditioners I. ARANY University of Miskolc, Institute of Mathematics 3515 Miskolc-Egyetemv~iros, Hungary mat arany©gold, uni-miskolc, hu Abstract--We present a modification of the ILUT algorithm due to Y. Sand for preparing in- complete factorization preconditioner for positive definite matrices being neither diagonally dominant nor M-matrices. In all tested cases, the rate of convergence of the relevant preconditioned conjugate gradient method was at a sufficient level, when linear systems arising from static problem for bar structures with 3D beam elements were solved. Keywords--Preconditioned conjugate gradient method, Incomplete factorization. 1. INTRODUCTION For solving large sparse positive definite systems of linear equations, the conjugate gradient method combined with a preconditoning technique seems to be one of the most efficient ways among the simple iterative methods. As known, for positive definite symmetric diagonally dom- inant M-matrices, the incomplete factorization technique may be efficiently used for preparing preconditioners [1]. However, for more general cases, when the matrix is neither diagonally domi- nant nor M-matrix, but it is a positive definite one, some incomplete factorization techniques [2,3] seem to be quite unusable, since usually, during the process, a diagonal element is turning to zero. We present a modification to the ILUT algorithm of Sand [2] for preparing preconditioners for the above mentioned matrices.
    [Show full text]
  • AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration
    AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Solving Eigenvalue Problems All eigenvalue solvers must be iterative Iterative algorithms have multiple facets: 1 Basic idea behind the algorithms 2 Convergence and techniques to speed-up convergence 3 Efficiency of implementation 4 Termination criteria We will focus on first two aspects Xiangmin Jiao Numerical Analysis I 2 / 10 Simplification: Real Symmetric Matrices We will consider eigenvalue problems for real symmetric matrices, i.e. T m×m m A = A 2 R , and Ax = λx for x 2 R p ∗ T T I Note: x = x , and kxk = x x A has real eigenvalues λ1,λ2, ::: , λm and orthonormal eigenvectors q1, q2, ::: , qm, where kqj k = 1 Eigenvalues are often also ordered in a particular way (e.g., ordered from large to small in magnitude) In addition, we focus on symmetric tridiagonal form I Why? Because phase 1 of two-phase algorithm reduces matrix into tridiagonal form Xiangmin Jiao Numerical Analysis I 3 / 10 Rayleigh Quotient m The Rayleigh quotient of x 2 R is the scalar xT Ax r(x) = xT x For an eigenvector x, its Rayleigh quotient is r(x) = xT λx=xT x = λ, the corresponding eigenvalue of x For general x, r(x) = α that minimizes kAx − αxk2. 2 x is eigenvector of A() rr(x) = x T x (Ax − r(x)x) = 0 with x 6= 0 r(x) is smooth and rr(qj ) = 0 for any j, and therefore is quadratically accurate: 2 r(x) − r(qJ ) = O(kx − qJ k ) as x ! qJ for some J Xiangmin Jiao Numerical Analysis I 4 / 10 Power
    [Show full text]