A HISTORY OF INVERSE ITERATION y ILSE C F IPSEN The Metho d Inverse iteration is a metho d for computing eigenfunctions of linear op erators Here we consider inverse iteration for the solution of the algebraic eigenvalue problem Ax x where A is a real or complex square matrix the scalar is an eigenvalue and the non zero vector x is an eigenvector Inverse iteration is the metho d of choice when one has at ones disp osal approximations to a sp ecied subset of eigenvalues of A for which one wants to compute asso ciated eigenvectors The metho d is frequently used in structural mechanics for instance when the extreme eigenvalues and corresp onding eigenvectors of Hermitian p ositivesemidenite eigenproblems are sought Given an approximation to an eigenvalue of A inverse iteration generates a 1 sequence of vectors x from a given starting vector x by solving the linear systems k 0 A I x s x k k k k 1 The scalar s is resp onsible for normalising x Usually one choses s so that kx k k k k k in some norm If everything go es well the sequence of iterates x converges to k an eigenvector asso ciated with an eigenvalue closest to In particular when there is only a single eigenvalue closest to then the x converge to an eigenvector k asso ciated with provided the starting vector x contains a contribution of this 0 k eigenvector Since x is a multiple of the vector A I y and since k 0 1 is the largest eigenvalue of A I the contribution in x of the eigendirection k asso ciated with increases faster as k than the contributions asso ciated with the 2 other eigenvalues Inverse iteration is mathematically identical to the p ower metho d 1 applied to A I In Wilkinson remarked Inverse iteration is now the most widely used metho d for computing eigenvectors corresp onding to selected eigenvalues which have already b een computed more or less accurately p A lo ok at software in the public domain shows that this is still true to day Yet after so many years the b ehaviour of the metho d is still not completely understo o d neither in exact nor in oating p oint arithmetic As a consequence even highly sophisticated state of the art software like LAPACK still comes across matrices for which it cannot compute eigenvectors to high accuracy without a drastic sacrice in eciency 3 This article describ es the eorts undertaken on b ehalf of inverse iteration from its To app ear in Helmut Wielandt Mathematische Werke Mathematical Works Volume II Ma trix Theory and Analysis Hupp ert B and Schneider H eds Walter de Gruyter Berlin y Department of Mathematics North Carolina State University P O Box Raleigh NC USA ipsenmathncsuedu This work was supp orted in part by NSF grant CCR 1 Here I denotes the identity matrix 2 Mathematical identity means that equality holds in exact arithmetic but not necessarily in nite precision arithmetic b ecause the two metho ds determine their iterates by means of dierent op erations 3 We concentrate on the solution of the ordinary eigenvalue problem rather than the generalised eigenvalue problem Ax B x and we do not discuss techniques for convergence acceleration such as Rayleigh quotient iteration where the shift changes with each iteration inception to the present day Octob er Here are a few of the issues that prompted these eorts The iterates may not converge to an eigenvector b ecause the starting vector x do es not have a contribution in the desired eigenvector or b ecause there 0 may b e several dierent eigenvalues of A at the same distance from The eigenvalue closest to may b e a multiple eigenvalue and inverse iteration may have to compute a basis for the corresp onding invariant subspace Unless linear indep endence is enforced explicitly inverse iteration usually fails to compute the desired basis When the matrix is Hermitian linear indep endence is enforced by orthogonalising an iterate against the already computed eigenvectors But dep ending on the shift and the starting vector x iterates may still fail to converge The problem b ecomes even more pronounced 0 when happ ens to approximate a cluster of close but not identical eigenvalues Should one treat the cluster like a collection of separate eigenvalues or like a single multiple eigenvalue or like several multiple eigenvalues How close together must eigenvalues b e to qualify as a cluster Should an iterate b e orthogonalised against all eigenvectors in the cluster or against just a few Wielandt Started It Inverse iteration was intro duced by Wielandt in Although Peters and Wilkinson remark p without further elab oration that a numb er of p eople seem to have had the idea indep endently Wielandt is usually the one credited with the intro duction of inverse iteration He refers to inverse iteration as fractional iteration gebro chene Iteration in German 1 b ecause the matrix A I is a fractional linear function of A p He p oints out the b enet of inverse iteration in the stability analysis of vibrating systems that are small p erturbations of systems whose b ehaviour is known xI In this case go o d approximations to the eigenvalues of the p erturb ed system are available The computational asp ects of Wielandts work leading to the development of inverse iteration are describ ed in the companion pap er Adapted to the algebraic eigenvalue problem Wielandts justication for inverse iteration lo oks as follows xI I Ic Cho ose so that Then Ax x b ecomes A I x x Replacing x on the left by x and on the right by x and writing s instead of k k 1 k gives inverse iteration In Crandall presents inverse iteration for real symmetric matrices as an 4 instance of a particular relaxation metho d He do es not reference Wielandts pap er Wilkinsons Idea The p erson most closely asso ciated with inverse iteration for the algebraic eigenvalue problem is Jim Wilkinson Although Wilkinson gives the credit for inverse iteration to Wielandt he himself came up with it indep endently in as a result of trying to improve Givenss metho d for computing a single eigenvector of a symmetric tridiagonal matrix We mo dify Wilkinsons idea slightly and present it for a general matrix A rather than just a symmetric tridiagonal matrix Supp ose A has order n and the shift is 4 Crandall page gives the following characterisation of relaxation metho ds A unique feature of relaxation metho ds is that they are approximate pro cedures which are not rigidly prescrib ed in advance The precise pro cedure to b e followed is not dictated but is left to the intuition and accumulated exp erience of the computer The computers intelligence is thus an active or dynamic link in the computational chain It is this fact which has made relaxation so attractive to many computers Thus at the time Crandall wrote his pap er a computer was still a human b eing and for exploratory purp oses the relaxation metho ds were carried out with a slide rule p closer to the simple eigenvalue than to any other eigenvalue of A Wilkinsons idea is the following Since an eigenvector x is determined up to a multiple by n of the n equations A I x approximate x by a nonzero vector z that solves n equations of A I z Why should such a z b e a go o d approximation to x Partition A so as to distin guish its leading principal submatrix A of order n and partition z conformally A a z 1 A z a 2 Supp ose is also not an eigenvalue of A so A I is nonsingular If we x to a nonzero value then there exists a solution z to the smaller system A I z a 1 5 This implies that z is a nonzero solution to the larger system A I z e where a z n 2 If e contains a contribution of an eigenvector x then this contribution is amplied n in z by Since is the single closest eigenvalue to the contributions of all other eigenvectors in z are amplied by smaller amounts Thus z is closer to x than is e n Instead of solving the rst n equations one could solve any set of n equations If the ith equation is the one that is omitted then the righthand side is a multiple of e In general the righthand side can b e any vector x as long as it contains a i 0 contribution of a desired eigenvector x Then the solution x of A I x x is 1 1 0 closer to x than is x Hence as inverse iteration progresses the iterates x converge 0 k to an eigenvector x provided the asso ciated eigenvalue is simple Wilkinson Takes Charge The foremost issues on Wilkinsons mind were the choice of appropriate starting vectors and the misfortune of having to solve a linear system that is highly illconditioned when is an accurate approximation to an eigenvalue of A From to Wilkinson analysed the computation of a single eigenvector of a symmetric tridiagonal matrix determination of the convergence rate in exact arithmetic choice of a go o d starting vector in exact arithmetic and eect of the backward error due to the solution of the linear system by Gaussian elimination with partial pivoting on the size of the iterate and the convergence rate pp pp Regarding the computation of eigenvectors asso ciated with almost coincident eigenvalues he recommends a dierent p erturbation of the shift for each eigenvalue and the use of the GramSchmidt algorithm to orthogonalise the current iterate against previously computed eigenvectors p Wilkinson applies a similar treatment to computing a single eigenvector when the eigenvalue is semisimple and the matrix is real and nonsymmetric pp As Wielandt had already observed in the context of the p ower metho