Manchester University Press Series in Algorithms and Architectures for Advanced Scientic Computing NUMERICAL METHODS FOR LARGE EIGENVALUE PROBLEMS Y Saad Contents Preface vii I Background in Matrix Theory and Linear Alge bra Matrices Square Matrices and Eigenvalues Typ es of Matrices Vector Inner Pro ducts and Norms Matrix Norms Subspaces Orthogonal Vectors and Subspaces Canonical Forms of Matrices Reduction to the Diagonal Form The Jordan Canonical Form The Schur Canonical Form Normal and Hermitian Matrices Normal Matrices Hermitian Matrices Nonnegative Matrices II Sparse Matrices Intro duction Storage Schemes ii Basic Sparse Matrix Op erations Sparse Direct Solution Metho ds Test Problems Random Walk Problem Chemical Reactions The HarwellBo eing Collection SPARSKIT III Perturbation Theory and Error Analysis Pro jectors and their Prop erties Orthogonal Pro jectors Oblique Pro jectors Resolvent and Sp ectral Pro jector Relations with the Jordan form Linear Perturbations of A APosteriori Error Bounds General Error Bounds The Hermitian Case The KahanParlettJiang Theorem Conditioning of Eigenproblems Conditioning of Eigenvalues Conditioning of Eigenvectors Conditioning of Invariant Subspaces Lo calization Theorems IV The Tools of Sp ectral Approximation Single Vector Iterations The Power Metho d The Shifted Power Metho d Inverse Iteration Deation Techniques Wielandt Deation with One Vector OptimalityinWieldants Deation Deation with Several Vectors Partial Schur Decomp osition Practical Deation Pro cedures iii General Pro jection Metho ds Orthogonal Pro jection Metho ds The Hermitian Case Oblique Pro jection Metho ds Chebyshev Polynomials Real Chebyshev Polynomials Complex Chebyshev Polynomials V Subspace Iteration Simple Subspace Iteration Subspace Iteration with Pro jection Practical Implementations Lo cking Linear Shifts Preconditionings VI Krylov Subspace Metho ds Krylov Subspaces Arnoldis Metho d The Basic Algorithm Practical Implementations Incorp oration of Implicit Deation The Hermitian Lanczos Algorithm The Algorithm Relation with Orthogonal Polynomials NonHermitian Lanczos algorithm The Algorithm Practical Implementations Blo ck Krylov Metho ds Convergence of the Lanczos Pro cess Distance between K and an Eigenvector m Convergence of the Eigenvalues Convergence of the Eigenvectors Convergence of the Arnoldi Pro cess iv VI I Acceleration Techniques and Hybrid Metho ds The Basic Chebyshev Iteration Convergence Prop erties ArnoldiChebyshev Iteration Purication by Arnoldis Metho d The Enhanced Initial Vector Approach Computing an Optimal Ellipse Starting the Chebyshev Iteration Cho osing the Parameters m and k Deated ArnoldiChebyshev Chebyshev Subspace Iteration Getting the Best Ellipse Parameters k and m Deation Least Squares Arnoldi The Least Squares Polynomial Use of Chebyshev Bases The Gram Matrix Computing the Best Polynomial Least Squares Arnoldi Algorithms VI I I Preconditioning Techniques Shiftandinvert Preconditioning General Concepts Dealing with Complex Arithmetic ShiftandInvert Arnoldi Polynomial Preconditioning Davidsons Metho d Generalized Arnoldi Algorithms IX NonStandard Eigenvalue Problems Intro duction Generalized Eigenvalue Problems General Results Reduction to Standard Form Deation v ShiftandInvert Pro jection Metho ds The Hermitian Denite Case Quadratic Problems From Quadratic to Generalized Problems X Origins of Matrix Eigenvalue Problems Intro duction Mechanical Vibrations Electrical Networks Quantum Chemistry Stability of Dynamical Systems Bifurcation Analysis Chemical Reactions Macroeconomics Markov Chain Mo dels References Index vi Preface Matrix eigenvalue problems arise in a large numb er of disciplines of sciences and engineering They constitute the basic to ol used in designing buildings bridges and turbines that are resistent to vibrations They allow to mo del queueing networks and to analyze stability of electrical networks or uid ow They also allow the scientist to understand lo cal physical phenonema or to study bifurcation patterns in dynamical systems In fact the writing of this b o ok was motivated mostly by the second class of problems Several b o oks dealing with numerical metho ds for solving eigen value problems involving symmetric or Hermitian matrices have been written and there are a few software packages b oth public and commercial available The book by Parlett is an ex cellent treatise of the problem Despite a rather strong demand by engineers and scientists there is little written on nonsymmetric problems and even less is available in terms of software The book by Wilkinson still constitutes an imp ortant reference Certainly science has evolved since the writing of Wilkinsons b o ok and so has the computational environment and the demand for solving large matrix problems Problems are b ecoming larger and more complicated while at the same time computers are able to deliver ever higher p erformances This means in particular that metho ds that were deemed to o demanding yesterday are now in the realm of the achievable I hop e that this b o ok will b e a small step in bridging the gap between the literature on what is avail able in the symmetric case and the nonsymmetric case Both viii Preface the Hermitian and the nonHermitian case are covered although nonHermitian problems are given more emphasis This book attempts to achieve a go o d balance between the ory and practice I should comment that the theory is esp ecially imp ortant in the nonsymmetric case In essence what dierenti ates the Hermitian from the nonHermitian eigenvalue problem is that in the rst case we can always manage to compute an ap proximation whereas there are nonsymmetric problems that can b e arbitrarily dicult to solve and can essentially makeany algo rithm fail Stated more rigorouslythe eigenvalue of a Hermitian matrix is always wellconditioned whereas this is not true for non symmetric matrices On the practical side I tried to give a general view of algorithms and to ols that have proved ecient Many of the algorithms describ ed corresp ond to actual implementations of research software and have b een tested on realistic problems I have tried to convey our exp erience from the practice in using these techniques As a result of the partial emphasis on theory there are a few chapters that may be found hard to digest for readers inexp eri enced with linear algebra These are Chapter III and to some extent a small part of Chapter IV Fortunately Chapter III is basically indep endent of the rest of the b o ok The minimal back ground needed to use the algorithmic part of the b o ok namely Chapters IV through VIII is calculus and linear algebra at the undergraduate level The b o ok has b een used twice to teachaspe cial topics course once in a Mathematics department and once in a Computer Science department In a quarter period represent ing roughly weeks of hours lecture p er week Chapter I I I I and IV to VI have been covered without much diculty In a semester p erio d weeks of hours lecture weeklyallchapters canbecovered with various degrees of depth Chapters II and X need not b e treated in class and can b e given as remedial reading Finally I would like to extend my appreciation to a number of people to whom I am indebted Francoise Chatelin who was my thesis adviser intro duced me to numerical metho ds for eigen Preface ix value problems Her inuence on my way of thinking is certainly reected in this book Beresford