Rayleigh Quotient Iteration and Simplified Jacobi-Davidson Method

The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh Quotient iteration and simpliﬁed Jacobi-Davidson method with preconditioned iterative solves

Melina Freitag

Department of Mathematical Sciences University of Bath

10th Copper Mountain Conference on Iterative Methods, 2008 Copper Mountain, Colorado 10th April 2008

joint work with Alastair Spence (Bath)

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Problem and iterative methods

Find a small number of eigenvalues close to a shift σ and corresponding eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn

A is large, sparse, nonsymmetric

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Problem and iterative methods

Find a small number of eigenvalues close to a shift σ and corresponding eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn

A is large, sparse, nonsymmetric Iterative solves (e.g. Power method)

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Problem and iterative methods

Find a small number of eigenvalues close to a shift σ and corresponding eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Problem and iterative methods

Find a small number of eigenvalues close to a shift σ and corresponding eigenvectors of: Ax = λx, λ ∈ C,x ∈ Cn

A is large, sparse, nonsymmetric Iterative solves (e.g. Power method) Problem becomes 1 (A − σI)−1x = x λ − σ each step of the iterative method involves repeated application of (A − σI)−1 to a vector (inverse iteration) Inner iterative solve: (A − σI)y = x using Krylov or Galerkin-Krylov method for linear systems. leading to inner-outer iterative method.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Shift-invert strategy

This talk: Inner Iteration and preconditioning

Comparison of two methods: Inexact Rayleigh quotient iteration and Simpliﬁed Jacobi-Davidson method

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The algorithms

Inexact Rayleigh quotient iteration solve (A − ρ(x(i))I)y(i) = x(i), inexactly. (i) y H Rescale x(i+1) = and update ρ(x(i+1))= x(i+1) Ax(i+1), ky(i)k Test: eigenvalue residual r(i+1) =(A − ρ(x(i+1))I)x(i+1).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The algorithms

Inexact simpliﬁed Jacobi-Davidson method (without subspace expansion)

solve H H (I − x(i)x(i) )(A − ρ(x(i))I)(I − x(i)x(i) )s(i) = −r(i), (i) (i) x + s H Rescale x(i+1) = and update ρ(x(i+1))= x(i+1) Ax(i+1), kx(i) + s(i)k Test: eigenvalue residual r(i+1) =(A − ρ(x(i+1))I)x(i+1).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; FOM as inner solver

−1 10 Rayleigh Quotient iteration without preconditioner

−2 10

−3 10

−4 10 eigenvalue residual

−5 10

−6 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - without preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; FOM as inner solver

−1 −1 10 10 Rayleigh Quotient iteration without preconditioner simplified Jacobi−Davidson without preconditoner

−2 −2 10 10

−3 −3 10 10

−4 −4 10 10 eigenvalue residual eigenvalue residual

−5 −5 10 10

−6 −6 10 10 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 outer iteration outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - without preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; FOM as inner solver

−1 10 simplified Jacobi−Davidson without preconditoner Rayleigh Quotient iteration without preconditioner

−2 10

−3 10

−4 10 eigenvalue residual

−5 10

−6 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - without preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

Rayleigh Quotient iteration with standard preconditioner

−1.1 10

−1.2 10

−1.3 10 eigenvalue residual

−1.4 10

0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P1

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 10 Rayleigh Quotient iteration with standard preconditioner simplified Jacobi−Davidson without preconditoner −2 10

−1.1 10 −3 10

−4 10

−1.2 −5 10 10

−6 10

−1.3 −7 10 10 eigenvalue residual eigenvalue residual

−8 10

−9 −1.4 10 10

−10 10 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 outer iteration outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P1

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 10

−2 10

−3 10 simplified Jacobi−Davidson with standard preconditoner

−4 Rayleigh Quotient iteration with standard preconditioner 10

−5 10

−6 10

−7 10 eigenvalue residual

−8 10

−9 10

−10 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P1

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 10 Rayleigh Quotient iteration with tuned preconditioner −2 10

−3 10

−4 10

−5 10

−6 10

−7 10 eigenvalue residual

−8 10

−9 10

−10 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P2

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 −1 10 10 Rayleigh Quotient iteration with tuned preconditioner simplified Jacobi−Davidson without preconditoner −2 −2 10 10

−3 −3 10 10

−4 −4 10 10

−5 −5 10 10

−6 −6 10 10

−7 −7 10 10 eigenvalue residual eigenvalue residual

−8 −8 10 10

−9 −9 10 10

−10 −10 10 10 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 outer iteration outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P2

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 10 simplified Jacobi−Davidson with tuned preconditoner −2 Rayleigh Quotient iteration with tuned preconditioner 10

−3 10

−4 10

−5 10

−6 10

−7 10 eigenvalue residual

−8 10

−9 10

−10 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with preconditioner P2

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Exact solves

Rayleigh quotient iteration Solve

(A − ρ(x)I)y = x

at each outer iteration.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Exact solves

Jacobi-Davidson method Rayleigh quotient iteration Solve Solve (I − xxH )(A − ρ(x)I)(I − xxH )s = −r (A − ρ(x)I)y = x at each outer iteration, where at each outer iteration. r =(A − ρ(x)I)x is the eigenvalue residual and s ⊥ x.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Exact solves

Exact solves Sleijpen and van der Vorst (1996):

y = α(x + s)

for some constant α. Solutions are equivalent!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Exact solves

Jacobi-Davidson method Rayleigh quotient iteration Solve Solve (I − xxH )(A − ρ(x)I)s = −r (A − ρ(x)I)y = x at each outer iteration, where at each outer iteration. r =(A − ρ(x)I)x is the eigenvalue residual and s ⊥ x.

Exact solves Sleijpen and van der Vorst (1996):

y = α(x + s)

for some constant α. Solutions are equivalent!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Exact solves

Jacobi-Davidson method Rayleigh quotient iteration Solve Solve (A − ρ(x)I)s − γx = −(A − ρ(x)I)x (A − ρ(x)I)y = x at each outer iteration, where at each outer iteration. r =(A − ρ(x)I)x is the eigenvalue residual and s ⊥ x.

Exact solves Sleijpen and van der Vorst (1996):

y = α(x + s)

for some constant α. Solutions are equivalent!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Inexact solves

Galerkin-Krylov Solver Simoncini and Eld´en (2002), (Hochstenbach and Sleijpen (2003) for two-sided RQ iteration), based on result by Stathopoulos and Saad (1998) yk+1 = β(x + sk) for some constant β if both systems are solved using a Galerkin-Krylov subspace method

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment - Inexact solves

Matrix sherman5.mtx Rayleigh quotient shift; FOM as inner solver

−1 10 simplified Jacobi−Davidson without preconditoner Rayleigh Quotient iteration without preconditioner

−2 10

−3 10

−4 10 eigenvalue residual

−5 10

−6 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - without preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Rayleigh quotient iteration and Jacobi-Davidson: Preconditioned Solves

Preconditioning for JD method Preconditioning for RQ iteration Solve

Solve (I − xxH )(A − ρ(x)I)(I − xxH )P˜†s˜ = −r −1 (A − ρ(x)I)P y˜ = x, (with s = P˜†s˜) has to be solved. Note (with y = P −1y˜) at each iteration. the restricted preconditioner P˜ := (I − xxH )P (I − xxH ).

Equivalence does not hold!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

An experiment - with standard preconditioner

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

−1 10

−2 10

−3 10 simplified Jacobi−Davidson with standard preconditoner

−4 Rayleigh Quotient iteration with standard preconditioner 10

−5 10

−6 10

−7 10 eigenvalue residual

−8 10

−9 10

−10 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - with standard preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Preconditioners

Need a better preconditioner for Rayleigh quotient iteration.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The tuned preconditioner

Advantage of Jacobi-Davidson

(I − xxH )(A − ρ(x)I)(I − xxH )P˜†s˜ = −r (with s = P˜†s˜). The preconditioner

P˜ := (I − xxH )P (I − xxH ).

is restricted (to the subspace orthogonal to x).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The tuned preconditioner

Advantage of Jacobi-Davidson

(I − xxH )(A − ρ(x)I)(I − xxH )P˜†s˜ = −r (with s = P˜†s˜). The preconditioner

P˜ := (I − xxH )P (I − xxH ).

is restricted (to the subspace orthogonal to x).

The new (tuned) preconditioner for inexact Rayleigh quotient iteration Use P = IxxH + P (I − xxH ) which satisﬁes Px = x as well as P−1x = x

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The tuned preconditioner

Implementation

P = xxH + P (I − xxH )

P = P +(I − P )xxH

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

The tuned preconditioner

Implementation

P = xxH + P (I − xxH )

P = P +(I − P )xxH minor modiﬁcation and minor extra computational cost

(P −1x − x)xH P −1 P−1 = P −1 − xH P −1x

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Preconditioned Solves for inexact Rayleigh Quotient iteration and inexact Jacobi-Davidson

Tuned preconditioner in RQI ≡ preconditioned JD

Px = x

RQ iteration with preconditioner P: (A − ρ(x)I)P−1y = x Inner solves in RQ iteration build Krylov space

span{x, (A − ρ(x)I)P−1x, ((A − ρ(x)I)P−1)2x,...}

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Preconditioned Solves for inexact Rayleigh Quotient iteration and inexact Jacobi-Davidson

Tuned preconditioner in RQI ≡ preconditioned JD

Px = x

RQ iteration with preconditioner P: (A − ρ(x)I)P−1y = x Inner solves in RQ iteration build Krylov space

span{x, (A − ρ(x)I)P−1x, ((A − ρ(x)I)P−1)2x,...}

JD with preconditioner P : (I − xxH )(A − ρ(x)I)(I − xxH )P˜†s˜ = −r Inner solves in JD method build Krylov space

P −1 P −1 2 span{r, Π1(A − ρ(x)I)Π2 P r, (Π1(A − ρ(x)I)Π2 P ) r, . . .}

P −1xxH where Π =(I − xxH ) and ΠP = I − . 1 2 xH P −1x

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Consider subspaces

A ← A − ρ(x)I

Lemma The subspaces

−1 −1 2 −1 k Kk = span{x,AP x, (AP ) x,..., (AP ) x}

and

P −1 P −1 2 P −1 k−1 Lk = span{x, r, Π1AΠ2 P r, (Π1AΠ2 P ) r, . . . , (Π1AΠ2 P ) r} are equivalent.

Proof. Extension of result by Stathopoulos and Saad (1998).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Equivalence for inexact solves

Theorem Let both (A − ρ(x)I)P−1y˜ = x, y = P−1y˜ and (I − xxH )(A − ρ(x)I)(I − xxH)P˜†s˜ = −r, s = P˜†s˜ be solved with the same Galerkin-Krylov method. Then

RQ JD yk+1 = γ(x + sk ).

Proof. Extension of result by Simoncini and Eld´en (2002).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Example

sherman5.mtx; Rayleigh quotient shift; preconditioned FOM as inner solver

−1 −1 10 10 simplified Jacobi−Davidson without preconditoner Rayleigh Quotient iteration without preconditioner −2 10

−2 10 −3 10 simplified Jacobi−Davidson with standard preconditoner

−4 Rayleigh Quotient iteration with tuned preconditioner 10 −3 Rayleigh Quotient iteration with standard preconditioner 10 −5 10

−6 10 −4 10 −7 10 eigenvalue residual eigenvalue residual

−8 −5 10 10

−9 10

−6 −10 10 10 0 2 4 6 8 10 12 14 16 18 20 0 2 4 6 8 10 12 14 16 18 20 outer iteration outer iteration

Figure: Convergence history of the Figure: standard preconditioner for eigenvalue residuals; no JD, tuned preconditioner for RQ preconditioner. Iteration P2 = P.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Equivalence between preconditioned JD and preconditioned RQI

Technique applies to any preconditioner for linear systems.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Fixed shifts

Matrix sherman5.mtx Fixed shift; preconditioned FOM as inner solver

2 10 simplified Jacobi−Davidson with standard preconditoner Inexact inverse iteration with standard preconditioner 1 10 Inexact inverse iteration with tuned preconditioner

0 10

−1 10

−2 10

−3 10 eigenvalue residual −4 10

−5 10

−6 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Inexact inverse iteration and Jacobi-Davidson method - standard and tuned preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Petrov-Galerkin method (GMRES)

GMRES FOM krk k krk k = . GMRES GMRES 2 q1 − (krk k/krk−1 k) Matrix sherman5.mtx Fixed shift; preconditioned GMRES as inner solver

2 10 simplified Jacobi−Davidson with standard preconditoner Inexact inverse iteration with standard preconditioner 1 10 Inexact inverse iteration with tuned preconditioner

0 10

−1 10

−2 10

−3 10 eigenvalue residual −4 10

−5 10

−6 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Inexact inverse iteration and Jacobi-Davidson method - standard and tuned preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Petrov-Galerkin method (GMRES)

GMRES FOM krk k krk k = . GMRES GMRES 2 q1 − (krk k/krk−1 k) Matrix sherman5.mtx Fixed shift; preconditioned GMRES as inner solver

2 10 simplified Jacobi−Davidson with standard preconditoner difference between JD (standard prec) and RQI (tuned prec) Inexact inverse iteration with standard preconditioner −2 1 10 10 Inexact inverse iteration with tuned preconditioner

0 −3 10 10

−1 −4 10 10

−2 10 −5 10

−3 10 −6 10 eigenvalue residual −4 10 −7 10 −5 10 differences in the eigenvalue residual −8 10 −6 10 0 2 4 6 8 10 12 14 16 18 20 2 4 6 8 10 12 14 16 18 20 outer iteration outer iteration

Figure: Inexact inverse iteration and Jacobi-Davidson method - standard and tuned preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Improvement Px = Ax

The tuned preconditoner Px = Ax Instead of P = xxH + P (I − xxH ) use P = AxxH + P (I − xxH ) which satisﬁes Px = Ax

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Improvement Px = Ax

The tuned preconditoner Px = Ax Instead of P = xxH + P (I − xxH ) use P = AxxH + P (I − xxH ) which satisﬁes Px = Ax

P = P +(A − P )xxH

minor modiﬁcation and minor extra computational cost

(P −1Ax − x)xH P −1 P−1 = P −1 − xH P −1Ax

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Tuning with Px = Ax

Matrix sherman5.mtx Rayleigh quotient shift; preconditioned FOM as inner solver

0 10

−2 10 simplified Jacobi−Davidson with standard preconditoner Rayleigh Quotient iteration with standard preconditioner −4 10 Rayleigh Quotient iteration with tuned preconditioner

−6 10

−8 10 eigenvalue residual

−10 10

−12 10 0 2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - standard and tuned preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Generalised eigenproblem Ax = λMx

Solving (A − ρ(x)M)P−1y˜ = Mx, with y = P−1y,˜ and MxxH M H (I − )(A − ρ(x)M)(I − xuH )P˜†s˜ = −r, with s = P˜†s,˜ xH M H Mx using the same Galerkin-Krylov method and Px = Mx are equivalent.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Generalised eigenproblem Ax = λMx

Use Px = Mx Matrix sherman5.mtx with a pos. def. tridiagonal M Rayleigh quotient shift; preconditioned FOM as inner solver

−2 10 simplified Jacobi−Davidson with standard preconditoner Rayleigh Quotient iteration with standard preconditioner Rayleigh Quotient iteration with tuned preconditioner

−4 10

−6 10 eigenvalue residual

−8 10

2 4 6 8 10 12 14 16 18 20 outer iteration

Figure: Rayleigh quotient iteration and Jacobi-Davidson method - standard and tuned preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Another motivation for Px = Ax

More detailed GMRES theory for (A − σI)y = x

|λ2 − λ1| kx − (A − σI)ykk≤ κ˜(W ) min max |p(λj )|kQxk ≤ τ λ1 p∈Pk−1 j=2,...,n

where λj are eigenvalues of A − σI.

Number of inner iterations kQxk k ≥ C + C log , 1 2 τ where Q projects onto the space not spanned by the eigenvector.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Another motivation for Px = Ax

Good news!

(i) (i) C3kr k k ≥ C1 + C2 log , τ (i) where τ (i) = Ckr(i)k. Iteration number approximately constant!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Another motivation for Px = Ax

Good news!

(i) (i) C3kr k k ≥ C1 + C2 log , τ (i) where τ (i) = Ckr(i)k. Iteration number approximately constant!

Bad news :-( For a standard preconditioner P

(A − σI)P −1y˜(i) = x(i) P −1y˜(i) = y(i)

kQ˜x(i)k C k(i) ≥ C′ + C′ log = C′ + C′ log , 1 2 τ (i) 1 2 τ (i) where τ (i) = Ckr(i)k. Iteration number increases!

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Another motivation for Px = Ax

How to overcome this problem Use tuned preconditioner:

H (i) (i) (i) (i) Pix = Ax , Pi := P +(A − P )x x

Why does that work?

Assume we have found eigenvector x1

−1 λ1 − σ Ax1 = Px1 = λ1x1 ⇒ (A − σI)P x1 = x1 λ1

−1 and convergence of Krylov method applied to (A − σI)P y˜ = x1 in one iteration. For general x(i)

C kr(i)k k(i) ≥ C′ + C′ log 3 , where τ (i) = Ckr(i)k. 1 2 τ (i)

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Convection-Diﬀusion operator

Finite diﬀerence discretisation on a 32 × 32 grid of the convection-diﬀusion operator 2 −∆u + 5ux + 5uy = λu on (0, 1) , with homogeneous Dirichlet boundary conditions (961 × 961 matrix).

smallest eigenvalue: λ1 ≈ 32.18560954, Preconditioned GMRES with tolerance τ (i) = 0.01kr(i)k, standard and tuned preconditioner (incomplete LU).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Convection-Diﬀusion operator

right preconditioning, 569 iterations right preconditioning, 569 iterations 0 45 10

40 −2 10 35

−4 30 10

−6 10 20 inner iterations

residual norms \|r^{(i)}\| −8 10

−10 5 10

2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations

Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Convection-Diﬀusion operator

right preconditioning, 569 iterations right preconditioning, 569 iterations 0 10 tuned right preconditioning, 115 iterations 45 tuned right preconditioning, 115 iterations

40 −2 10 35

−4 30 10

−6 10 20 inner iterations

residual norms \|r^{(i)}\| −8 10

−10 5 10

2 4 6 8 10 12 14 16 18 20 22 100 200 300 400 500 600 outer iterations sum of inner iterations

Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Extensions

Possible Extensions (ongoing work) 1 subspace iteration 2 Shift-invert Arnoldi iteration using a tuned preconditioner

(i) (i) (i) (i)H PiQ = AQ ; given by Pi = P +(A − P )Q Q

is equivalent to the Jacobi-Davidson method with a standard preconditioner

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Numerical Example

sherman5.mtx matrix from the Matrix Market library (3312 × 3312). −2 smallest eigenvalue: λ1 ≈ 4.69 × 10 , Preconditioned GMRES as inner solver (ﬁxed tolerance and relaxation strategy), standard and tuned preconditioner (incomplete LU).

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

No tuning and standard preconditioner

Arnoldi tolerance 10e−14/m 0 10 25

20 || −5

(i) 10

−10 10

inner itertaions 10 residual norms ||r

−15 10 5

Arnoldi tolerance 10e−14/m

2 4 6 8 10 12 50 100 150 200 250 300 outer iterations sum of inner iterations

Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Tuning

Arnoldi tolerance 10e−14/m 0 10 Arnoldi tolerance 10e−14 tuned 25

20 || −5

(i) 10

−10 10

inner itertaions 10 residual norms ||r

−15 10 5 Arnoldi tolerance 10e−14/m Arnoldi tolerance 10e−14 tuned

2 4 6 8 10 12 50 100 150 200 250 300 outer iterations sum of inner iterations

Figure: Inner iterations vs outer Figure: Eigenvalue residual norms vs iterations total number of inner iterations

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

Conclusions

For eigencomputations it is advantageous to consider modiﬁed preconditioners (works for any preconditioner) Analysis provides further understanding of preconditioned Jacobi-Davidson method Numerical results on eigenvalue problems obtained from Mixed FEM Navier-Stokes with DD preconditioner show the same gains.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers The problem RQI and JD The tuned preconditioner Prec. RQI and JD Extensions Conclusions

M. A. Freitag and A. Spence, Convergence rates for inexact inverse iteration with application to preconditioned iterative solves, BIT, 47 (2007), pp. 27–44. , Convergence theory for inexact inverse iteration applied to the generalised nonsymmetric eigenproblem, Electron. Trans. Numer. Anal., 28 (2007), pp. 40–64. , Rayleigh quotient iteration and simpliﬁed Jacobi-Davidson method with preconditioned iterative solves, Linear Algebra Appl., 428 (2008), pp. 2049–2060. M. Robbe,´ M. Sadkane, and A. Spence, Inexact inverse subspace iteration with preconditioning applied to non-Hermitian eigenvalue problems, 2007. To appear in SIAM J. Matrix Anal. Appl. F. Xue and H. C. Elman, Convergence analysis of iterative solvers in inexact Rayleigh quotient iteration, Preprint UMCP-CSD:CS-TR-4902, University of Maryland, Department of Computer Science and Institute for Advanced Computer Studies, 2008.

Melina Freitag University of Bath RQI and JD method with tuned preconditioners for eigensolvers