Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions New Preconditioning Strategies for Eigenvalue Problems Melina Freitag Department of Mathematical Sciences University of Bath 22nd Biennial Conference on Numerical Analysis, Dundee Scotland, UK 26th June 2007 joint work with Howard Elman (Maryland) and Alastair Spence (Bath) Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions 1 The problem 2 Inverse iteration and Shift-invert Arnoldi method 3 Inexact inverse iteration 4 Inexact Shift-invert Arnoldi method 5 Conclusions Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Outline 1 The problem 2 Inverse iteration and Shift-invert Arnoldi method 3 Inexact inverse iteration 4 Inexact Shift-invert Arnoldi method 5 Conclusions Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Problem and iterative methods Find a small number of eigenvalues and eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn A is large, sparse, nonsymmetric Iterative solves Power method Simultaneous iteration Arnoldi method Jacobi-Davidson method Usually involves repeated application of the matrix A to a vector Generally convergence to largest/outlying eigenvector Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Outline 1 The problem 2 Inverse iteration and Shift-invert Arnoldi method 3 Inexact inverse iteration 4 Inexact Shift-invert Arnoldi method 5 Conclusions Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Shift-invert strategy Wish to find an eigenvalue close to a shift σ σ λλ λ λ λ 3 1 2 4 n Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Shift-invert strategy Wish to find an eigenvalue close to a shift σ σ λλ λ λ λ 3 1 2 4 n Problem becomes − 1 (A − σI) 1x = x λ − σ each step of the iterative method involves repeated application of (A − σI)−1 to a vector Inner iterative solve: (A − σI)y = x using Krylov method for linear systems. leading to inner-outer iterative method. Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Shift-invert strategy This talk: Inner Iteration and preconditioning fixed shifts only. Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Outline 1 The problem 2 Inverse iteration and Shift-invert Arnoldi method 3 Inexact inverse iteration 4 Inexact Shift-invert Arnoldi method 5 Conclusions Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The algorithm Inexact inverse iteration for i =1 to . do choose τ (i) solve k(A − σI)y(i) − x(i)k = kd(i)k≤ τ (i), y(i) Rescale x(i+1) = , ky(i)k H Update λ(i+1) = x(i+1) Ax(i+1), Test: eigenvalue residual r(i+1) =(A − λ(i+1)I)x(i+1). end for Convergence rates If τ (i) = Ckr(i)k then convergence rate is linear (same convergence rate as for exact solves). Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)y = x Standard GMRES theory for y0 = 0 and A diagonalisable kx − (A − σI)ykk≤ κ(W ) min max |p(λj )|kxk p∈Pk j=1,...,n −1 where λj are eigenvalues of A − σI and (A − σI)= W ΛW . Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)y = x Standard GMRES theory for y0 = 0 and A diagonalisable kx − (A − σI)ykk≤ κ(W ) min max |p(λj )|kxk p∈Pk j=1,...,n −1 where λj are eigenvalues of A − σI and (A − σI)= W ΛW . Number of inner iterations kxk k ≥ C + C log 1 2 τ for kx − (A − σI)ykk≤ τ. Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)y = x More detailed GMRES theory for y0 = 0 |λ2 − λ1| kx − (A − σI)ykk≤ κ˜(W ) min max |p(λj )|kQxk λ1 p∈Pk−1 j=2,...,n where λj are eigenvalues of A − σI. Number of inner iterations ′ ′ kQxk k ≥ C + C log , 1 2 τ where Q projects onto the space not spanned by the eigenvector. Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)y = x Good news! (i) ′ ′ C kr k k(i) ≥ C + C log 3 , 1 2 τ (i) where τ (i) = Ckr(i)k. Iteration number approximately constant! Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)y = x Good news! (i) ′ ′ C kr k k(i) ≥ C + C log 3 , 1 2 τ (i) where τ (i) = Ckr(i)k. Iteration number approximately constant! Bad news :-( For a standard preconditioner P − − (A − σI)P 1y˜(i) = x(i) P 1y˜(i) = y(i) (i) ′′ ′′ kQ˜x k ′′ ′′ C k(i) ≥ C + C log = C + C log , 1 2 τ (i) 1 2 τ (i) where τ (i) = Ckr(i)k. Iteration number increases! Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)P −1y˜ = x How to overcome this problem Use a different preconditioner, namely one that satisfies H (i) (i) (i) (i) Pix = Ax , Pi := P +(A − P )x x minor modification and minor extra computational cost, P−1 (i) (i) [A i ]Ax = Ax . Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions The inner iteration for (A − σI)P −1y˜ = x How to overcome this problem Use a different preconditioner, namely one that satisfies H (i) (i) (i) (i) Pix = Ax , Pi := P +(A − P )x x minor modification and minor extra computational cost, P−1 (i) (i) [A i ]Ax = Ax . Why does that work? Assume we have found eigenvector x1 −1 λ1 − σ Ax1 = Px1 = λ1x1 ⇒ (A − σI)P x1 = x1 λ1 −1 and convergence of Krylov method applied to (A − σI)P y˜ = x1 in one iteration. For general x(i) (i) ′′ ′′ C kr k k(i) ≥ C + C log 3 , where τ (i) = Ckr(i)k. 1 2 τ (i) Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Convection-Diffusion operator Finite difference discretisation on a 32 × 32 grid of the convection-diffusion operator 2 −∆u + 5ux + 5uy = λu on (0, 1) , with homogeneous Dirichlet boundary conditions (961 × 961 matrix). smallest eigenvalue: λ1 ≈ 32.18560954, Preconditioned GMRES with tolerance τ (i) = 0.01kr(i)k, standard and tuned preconditioner (incomplete LU). Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Convection-Diffusion operator right preconditioning, 569 iterations right preconditioning, 569 iterations 0 45 10 40 −2 10 35 −4 30 10 25 −6 10 20 inner iterations 15 residual norms \|r^{(i)}\| −8 10 10 −10 5 10 2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It. Inexact SI Arnoldi method Conclusions Convection-Diffusion operator right preconditioning, 569 iterations right preconditioning, 569 iterations 0 10 tuned right preconditioning, 115 iterations 45 tuned right preconditioning, 115 iterations 40 −2 10 35 −4 30 10 25 −6 10 20 inner iterations 15 residual norms \|r^{(i)}\| −8 10 10 −10 5 10 2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations Melina Freitag University of Bath Preconditioners for Eigenvalue Problems Outline The problem Inv. It. and SI Arnoldi Inexact Inv. It.