DOCSLIB.ORG

Eigenvalue Problems

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

Eigenvalue Problems Eigenvalue Problems Existence, Uniqueness, and Conditioning Existence, Uniqueness, and Conditioning Computing Eigenvalues and Eigenvectors Computing Eigenvalues and Eigenvectors Outline Scientific Computing: An Introductory Survey Chapter 4 – Eigenvalue Problems 1 Eigenvalue Problems Prof. Michael T. Heath 2 Existence, Uniqueness, and Conditioning Department of Computer Science University of Illinois at Urbana-Champaign 3 Computing Eigenvalues and Eigenvectors Copyright c 2002. Reproduction permitted for noncommercial, educational use only. Michael T. Heath Scientific Computing 1 / 87 Michael T. Heath Scientific Computing 2 / 87 Eigenvalue Problems Eigenvalue Problems Eigenvalue Problems Eigenvalue Problems Existence, Uniqueness, and Conditioning Eigenvalues and Eigenvectors Existence, Uniqueness, and Conditioning Eigenvalues and Eigenvectors Computing Eigenvalues and Eigenvectors Geometric Interpretation Computing Eigenvalues and Eigenvectors Geometric Interpretation Eigenvalue Problems Eigenvalues and Eigenvectors Eigenvalue problems occur in many areas of science and Standard eigenvalue problem : Given n × n matrix A, find engineering, such as structural analysis scalar λ and nonzero vector x such that Eigenvalues are also important in analyzing numerical A x = λ x methods λ is eigenvalue, and x is corresponding eigenvector Theory and algorithms apply to complex matrices as well as real matrices λ may be complex even if A is real With complex matrices, we use conjugate transpose, AH , Spectrum = λ(A) = set of eigenvalues of A T instead of usual transpose, A Spectral radius = ρ(A) = maxfjλj : λ 2 λ(A)g Michael T. Heath Scientific Computing 3 / 87 Michael T. Heath Scientific Computing 4 / 87 Eigenvalue Problems Eigenvalue Problems Eigenvalue Problems Eigenvalue Problems Existence, Uniqueness, and Conditioning Eigenvalues and Eigenvectors Existence, Uniqueness, and Conditioning Eigenvalues and Eigenvectors Computing Eigenvalues and Eigenvectors Geometric Interpretation Computing Eigenvalues and Eigenvectors Geometric Interpretation Geometric Interpretation Examples: Eigenvalues and Eigenvectors 1 0 1 0 A = : λ = 1; x = ; λ = 2; x = 0 2 1 1 0 2 2 1 Matrix expands or shrinks any vector lying in direction of 1 1 1 1 A = : λ = 1; x = ; λ = 2; x = eigenvector by scalar factor 0 2 1 1 0 2 2 1 Expansion or contraction factor is given by corresponding 3 −1 1 1 A = : λ = 2; x = ; λ = 4; x = eigenvalue λ −1 3 1 1 1 2 2 −1 Eigenvalues and eigenvectors decompose complicated 1:5 0:5 1 −1 A = : λ = 2; x = ; λ = 1; x = behavior of general linear transformation into simpler 0:5 1:5 1 1 1 2 2 1 actions 0 1 1 i A = : λ = i; x = ; λ = −i; x = −1 0 1 1 i 2 2 1 p where i = −1 Michael T. Heath Scientific Computing 5 / 87 Michael T. Heath Scientific Computing 6 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Characteristic Polynomial Example: Characteristic Polynomial Equation Ax = λx is equivalent to Characteristic polynomial of previous example matrix is (A − λI)x = 0 3 −1 1 0 which has nonzero solution x if, and only if, its matrix is det − λ = singular −1 3 0 1 Eigenvalues of A are roots λi of characteristic polynomial 3 − λ −1 det = det(A − λI) = 0 −1 3 − λ in λ of degree n (3 − λ)(3 − λ) − (−1)(−1) = λ2 − 6λ + 8 = 0 Fundamental Theorem of Algebra implies that n × n matrix A always has n eigenvalues, but they may not be real nor so eigenvalues are given by distinct p 6 ± 36 − 32 Complex eigenvalues of real matrix occur in complex λ = ; or λ = 2; λ = 4 2 1 2 conjugate pairs: if α + iβpis eigenvalue of real matrix, then so is α − iβ, where i = −1 Michael T. Heath Scientific Computing 7 / 87 Michael T. Heath Scientific Computing 8 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Companion Matrix Characteristic Polynomial, continued Monic polynomial n−1 n p(λ) = c0 + c1λ + ··· + cn−1λ + λ Computing eigenvalues using characteristic polynomial is is characteristic polynomial of companion matrix not recommended because of 2 3 0 0 ··· 0 −c0 work in computing coefficients of characteristic polynomial 61 0 ··· 0 −c1 7 sensitivity of coefficients of characteristic polynomial 6 7 60 1 ··· 0 −c2 7 Cn = 6 7 work in solving for roots of characteristic polynomial 6. .. 7 4. 5 0 0 ··· 1 −cn−1 Characteristic polynomial is powerful theoretical tool but usually not useful computationally Roots of polynomial of degree > 4 cannot always computed in finite number of steps So in general, computation of eigenvalues of matrices of order > 4 requires (theoretically infinite) iterative process Michael T. Heath Scientific Computing 9 / 87 Michael T. Heath Scientific Computing 10 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Example: Characteristic Polynomial Multiplicity and Diagonalizability Consider Multiplicity is number of times root appears when 1 A = polynomial is written as product of linear factors 1 p Eigenvalue of multiplicity 1 is simple where is positive number slightly smaller than mach Exact eigenvalues of A are 1 + and 1 − Defective matrix has eigenvalue of multiplicity k > 1 with fewer than k linearly independent corresponding Computing characteristic polynomial in floating-point eigenvectors arithmetic, we obtain Nondefective matrix A has n linearly independent 2 2 2 det(A − λI) = λ − 2λ + (1 − ) = λ − 2λ + 1 eigenvectors, so it is diagonalizable which has 1 as double root X−1AX = D Thus, eigenvalues cannot be resolved by this method even though they are distinct in working precision where X is nonsingular matrix of eigenvectors Michael T. Heath Scientific Computing 11 / 87 Michael T. Heath Scientific Computing 12 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Eigenspaces and Invariant Subspaces Relevant Properties of Matrices Properties of matrix A relevant to eigenvalue problems Eigenvectors can be scaled arbitrarily: if Ax = λx, then Property Definition A(γx) = λ(γx) for any scalar γ, so γx is also eigenvector diagonal aij = 0 for i 6= j corresponding to λ tridiagonal aij = 0 for ji − jj > 1 triangular a = 0 for i > j (upper) Eigenvectors are usually normalized by requiring some ij a = 0 for i < j (lower) norm of eigenvector to be 1 ij Hessenberg aij = 0 for i > j + 1 (upper) Eigenspace = Sλ = fx : Ax = λxg aij = 0 for i < j − 1 (lower) n n Subspace S of R (or C ) is invariant if AS ⊆ S orthogonal AT A = AAT = I unitary AH A = AAH = I For eigenvectors x ··· x , span([x ··· x ]) is invariant 1 p 1 p symmetric A = AT subspace Hermitian A = AH normal AH A = AAH Michael T. Heath Scientific Computing 13 / 87 Michael T. Heath Scientific Computing 14 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Examples: Matrix Properties Examples, continued T 1 2 1 3 p p Transpose: = 0 1 −1 0 2=2 2=2 3 4 2 4 Orthogonal: ; ; p p 1 0 0 −1 − 2=2 2=2 H p p 1 + i 1 + 2i 1 − i 2 + i i 2=2 2=2 Conjugate transpose: = Unitary: p p 2 − i 2 − 2i 1 − 2i 2 + 2i − 2=2 −i 2=2 1 2 Symmetric: 1 1 2 3 Nonorthogonal: 1 2 1 3 Nonsymmetric: 21 2 03 2 4 Normal: 40 1 25 1 1 + i Hermitian: 2 0 1 1 − i 2 1 1 1 1 + i Nonnormal: NonHermitian: 0 1 1 + i 2 Michael T. Heath Scientific Computing 15 / 87 Michael T. Heath Scientific Computing 16 / 87 Eigenvalue Problems Characteristic Polynomial Eigenvalue Problems Characteristic Polynomial Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Existence, Uniqueness, and Conditioning Relevant Properties of Matrices Computing Eigenvalues and Eigenvectors Conditioning Computing Eigenvalues and Eigenvectors Conditioning Properties of Eigenvalue Problems Conditioning of Eigenvalue Problems Properties of eigenvalue problem affecting choice of algorithm and software Condition of eigenvalue problem is sensitivity of Are all eigenvalues needed, or only a few? eigenvalues and eigenvectors to changes in matrix Are only eigenvalues needed, or are corresponding Conditioning of eigenvalue problem is not same as eigenvectors also needed? conditioning of solution to linear system for same matrix Is matrix real or complex? Different eigenvalues and eigenvectors are not necessarily Is matrix relatively small and dense, or large and sparse? equally sensitive to perturbations
Recommended publications
  • Efficient Algorithms for High-Dimensional Eigenvalue

    Efficient Algorithms for High-Dimensional Eigenvalue

    Efficient Algorithms for High-dimensional Eigenvalue Problems by Zhe Wang Department of Mathematics Duke University Date: Approved: Jianfeng Lu, Advisor Xiuyuan Cheng Jonathon Mattingly Weitao Yang Dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Mathematics in the Graduate School of Duke University 2020 ABSTRACT Efficient Algorithms for High-dimensional Eigenvalue Problems by Zhe Wang Department of Mathematics Duke University Date: Approved: Jianfeng Lu, Advisor Xiuyuan Cheng Jonathon Mattingly Weitao Yang An abstract of a dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Mathematics in the Graduate School of Duke University 2020 Copyright © 2020 by Zhe Wang All rights reserved Abstract The eigenvalue problem is a traditional mathematical problem and has a wide appli- cations. Although there are many algorithms and theories, it is still challenging to solve the leading eigenvalue problem of extreme high dimension. Full configuration interaction (FCI) problem in quantum chemistry is such a problem. This thesis tries to understand some existing algorithms of FCI problem and propose new efficient algorithms for the high-dimensional eigenvalue problem. In more details, we first es- tablish a general framework of inexact power iteration and establish the convergence theorem of full configuration interaction quantum Monte Carlo (FCIQMC) and fast randomized iteration (FRI). Second, we reformulate the leading eigenvalue problem as an optimization problem, then compare the show the convergence of several coor- dinate descent methods (CDM) to solve the leading eigenvalue problem. Third, we propose a new efficient algorithm named Coordinate descent FCI (CDFCI) based on coordinate descent methods to solve the FCI problem, which produces some state-of- the-art results.
  • Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations

    Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations

    https://ntrs.nasa.gov/search.jsp?R=19960048075 2020-06-16T03:31:45+00:00Z NASA Contractor Report 198342 /" ICASE Report No. 96-40 J ICA IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen NASA Contract No. NASI-19480 May 1996 Institute for Computer Applications in Science and Engineering NASA Langley Research Center Hampton, VA 23681-0001 Operated by Universities Space Research Association National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-0001 IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen 1 Department of Computational and Applied Mathematics Rice University Houston, TX 77251 sorensen@rice, edu ABSTRACT Eigenvalues and eigenfunctions of linear operators are important to many areas of ap- plied mathematics. The ability to approximate these quantities numerically is becoming increasingly important in a wide variety of applications. This increasing demand has fu- eled interest in the development of new methods and software for the numerical solution of large-scale algebraic eigenvalue problems. In turn, the existence of these new methods and software, along with the dramatically increased computational capabilities now avail- able, has enabled the solution of problems that would not even have been posed five or ten years ago. Until very recently, software for large-scale nonsymmetric problems was virtually non-existent. Fortunately, the situation is improving rapidly. The purpose of this article is to provide an overview of the numerical solution of large- scale algebraic eigenvalue problems. The focus will be on a class of methods called Krylov subspace projection methods. The well-known Lanczos method is the premier member of this class.
  • Course Introduction

    Course Introduction

    Spectral Graph Theory Lecture 1 Introduction Daniel A. Spielman September 2, 2015 Disclaimer These notes are not necessarily an accurate representation of what happened in class. The notes written before class say what I think I should say. I sometimes edit the notes after class to make them way what I wish I had said. There may be small mistakes, so I recommend that you check any mathematically precise statement before using it in your own work. These notes were last revised on September 3, 2015. 1.1 First Things 1. Please call me \Dan". If such informality makes you uncomfortable, you can try \Professor Dan". If that fails, I will also answer to \Prof. Spielman". 2. If you are going to take this course, please sign up for it on Classes V2. This is the only way you will get emails like \Problem 3 was false, so you don't have to solve it". 3. This class meets this coming Friday, September 4, but not on Labor Day, which is Monday, September 7. 1.2 Introduction I have three objectives in this lecture: to tell you what the course is about, to help you decide if this course is right for you, and to tell you how the course will work. As the title suggests, this course is about the eigenvalues and eigenvectors of matrices associated with graphs, and their applications. I will never forget my amazement at learning that combinatorial properties of graphs could be revealed by an examination of the eigenvalues and eigenvectors of their associated matrices.
  • Accelerated Stochastic Power Iteration

    Accelerated Stochastic Power Iteration

    Accelerated Stochastic Power Iteration CHRISTOPHER DE SAy BRYAN HEy IOANNIS MITLIAGKASy CHRISTOPHER RE´ y PENG XU∗ yDepartment of Computer Science, Stanford University ∗Institute for Computational and Mathematical Engineering, Stanford University cdesa,bryanhe,[email protected], [email protected], [email protected] July 11, 2017 Abstract Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, requires O(1=∆) full-data passes to recover the principal component of a matrix withp eigen-gap ∆. Lanczos, a significantly more complex method, achieves an accelerated rate of O(1= ∆) passes. Modern applications, however, motivate methods that only ingest a subset of available data, known as the stochastic setting. In the online stochastic setting, simple 2 2 algorithms like Oja’s iteration achieve the optimal sample complexity O(σ =p∆ ). Unfortunately, they are fully sequential, and also require O(σ2=∆2) iterations, far from the O(1= ∆) rate of Lanczos. We propose a simple variant of the power iteration with an added momentum term, that achieves both the optimal sample and iteration complexity.p In the full-pass setting, standard analysis shows that momentum achieves the accelerated rate, O(1= ∆). We demonstrate empirically that naively applying momentum to a stochastic method, does not result in acceleration. We perform a novel, tight variance analysis that reveals the “breaking-point variance” beyond which this acceleration does not occur. By combining this insight with modern variance reduction techniques, we construct stochastic PCAp algorithms, for the online and offline setting, that achieve an accelerated iteration complexity O(1= ∆).
  • Numerical Linear Algebra Program Lecture 4 Basic Methods For

    Numerical Linear Algebra Program Lecture 4 Basic Methods For

    Program Lecture 4 Numerical Linear Algebra • Basic methods for eigenproblems. Basic iterative methods • Power method • Shift-and-invert Power method • QR algorithm • Basic iterative methods for linear systems • Richardson’s method • Jacobi, Gauss-Seidel and SOR • Iterative refinement Gerard Sleijpen and Martin van Gijzen • Steepest decent and the Minimal residual method October 5, 2016 1 October 5, 2016 2 National Master Course National Master Course Delft University of Technology Basic methods for eigenproblems The Power method The eigenvalue problem The Power method is the classical method to compute in modulus largest eigenvalue and associated eigenvector of a Av = λv matrix. can not be solved in a direct way for problems of order > 4, since Multiplying with a matrix amplifies strongest the eigendirection the eigenvalues are the roots of the characteristic equation corresponding to the in modulus largest eigenvalues. det(A − λI) = 0. Successively multiplying and scaling (to avoid overflow or underflow) yields a vector in which the direction of the largest Today we will discuss two iterative methods for solving the eigenvector becomes more and more dominant. eigenproblem. October 5, 2016 3 October 5, 2016 4 National Master Course National Master Course Algorithm Convergence (1) The Power method for an n × n matrix A. Let the n eigenvalues λi with eigenvectors vi, Avi = λivi, be n ordered such that |λ1| ≥ |λ2|≥ . ≥ |λn|. u0 ∈ C is given • Assume the eigenvectors v ,..., vn form a basis. for k = 1, 2, ... 1 • Assume |λ1| > |λ2|. uk = Auk−1 Each arbitrary starting vector u0 can be written as: uk = uk/kukk2 e(k) ∗ λ = uk−1uk u0 = α1v1 + α2v2 + ..
  • Arxiv:1012.3835V1

    Arxiv:1012.3835V1

    A NEW GENERALIZED FIELD OF VALUES RICARDO REIS DA SILVA∗ Abstract. Given a right eigenvector x and a left eigenvector y associated with the same eigen- value of a matrix A, there is a Hermitian positive definite matrix H for which y = Hx. The matrix H defines an inner product and consequently also a field of values. The new generalized field of values is always the convex hull of the eigenvalues of A. Moreover, it is equal to the standard field of values when A is normal and is a particular case of the field of values associated with non-standard inner products proposed by Givens. As a consequence, in the same way as with Hermitian matrices, the eigenvalues of non-Hermitian matrices with real spectrum can be characterized in terms of extrema of a corresponding generalized Rayleigh Quotient. Key words. field of values, numerical range, Rayleigh quotient AMS subject classifications. 15A60, 15A18, 15A42 1. Introduction. We propose a new generalized field of values which brings all the properties of the standard field of values to non-Hermitian matrices. For a matrix A ∈ Cn×n the (classical) field of values or numerical range of A is the set of complex numbers F (A) ≡{x∗Ax : x ∈ Cn, x∗x =1}. (1.1) Alternatively, the field of values of a matrix A ∈ Cn×n can be described as the region in the complex plane defined by the range of the Rayleigh Quotient x∗Ax ρ(x, A) ≡ , ∀x 6=0. (1.2) x∗x For Hermitian matrices, field of values and Rayleigh Quotient exhibit very agreeable properties: F (A) is a subset of the real line and the extrema of F (A) coincide with the extrema of the spectrum, σ(A), of A.
  • Spectral Graph Theory: Applications of Courant-Fischer∗

    Spectral Graph Theory: Applications of Courant-Fischer∗

    Spectral graph theory: Applications of Courant-Fischer∗ Steve Butler September 2006 Abstract In this second talk we will introduce the Rayleigh quotient and the Courant- Fischer Theorem and give some applications for the normalized Laplacian. Our applications will include structural characterizations of the graph, interlacing results for addition or removal of subgraphs, and interlacing for weak coverings. We also will introduce the idea of “weighted graphs”. 1 Introduction In the first talk we introduced the three common matrices that are used in spectral graph theory, namely the adjacency matrix, the combinatorial Laplacian and the normalized Laplacian. In this talk and the next we will focus on the normalized Laplacian. This is a reflection of the bias of the author having worked closely with Fan Chung who popularized the normalized Laplacian. The adjacency matrix and the combinatorial Laplacian are also very interesting to study in their own right and are useful for giving structural information about a graph. However, in many “practical” applications it tends to be the spectrum of the normalized Laplacian which is most useful. In this talk we will introduce the Rayleigh quotient which is closely linked to the Courant-Fischer Theorem and give some results. We give some structural results (such as showing that if a graph is bipartite then the spectrum is symmetric around 1) as well as some interlacing and weak covering results. But first, we will start by listing some common spectra. ∗Second of three talks given at the Center for Combinatorics, Nankai University, Tianjin 1 Some common spectra When studying spectral graph theory it is good to have a few basic examples to start with and be familiar with their spectrums.
  • Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2

    Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2

    ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU Abstract. We examine some numerical iterative methods for computing the eigenvalues and eigenvectors of real matrices. The five methods examined here range from the simple power iteration method to the more complicated QR iteration method. The derivations, procedure, and advantages of each method are briefly discussed. 1. Introduction Eigenvalues and eigenvectors play an important part in the applications of linear algebra. The naive method of finding the eigenvalues of a matrix involves finding the roots of the characteristic polynomial of the matrix. In industrial sized matrices, however, this method is not feasible, and the eigenvalues must be obtained by other means. Fortunately, there exist several other techniques for finding eigenvalues and eigenvectors of a matrix, some of which fall under the realm of iterative methods. These methods work by repeatedly refining approximations to the eigenvectors or eigenvalues, and can be terminated whenever the approximations reach a suitable degree of accuracy. Iterative methods form the basis of much of modern day eigenvalue computation. In this paper, we outline five such iterative methods, and summarize their derivations, procedures, and advantages. The methods to be examined are the power iteration method, the shifted inverse iteration method, the Rayleigh quotient method, the simultaneous iteration method, and the QR method. This paper is meant to be a survey over existing algorithms for the eigenvalue computation problem. Section 2 of this paper provides a brief review of some of the linear algebra background required to understand the concepts that are discussed. In section 3, the iterative methods are each presented, in order of complexity, and are studied in brief detail.
  • Lecture 12 — February 26 and 28 12.1 Introduction

    Lecture 12 — February 26 and 28 12.1 Introduction

    EE 381V: Large Scale Learning Spring 2013 Lecture 12 | February 26 and 28 Lecturer: Caramanis & Sanghavi Scribe: Karthikeyan Shanmugam and Natalia Arzeno 12.1 Introduction In this lecture, we focus on algorithms that compute the eigenvalues and eigenvectors of a real symmetric matrix. Particularly, we are interested in finding the largest and smallest eigenvalues and the corresponding eigenvectors. We study two methods: Power method and the Lanczos iteration. The first involves multiplying the symmetric matrix by a randomly chosen vector, and iteratively normalizing and multiplying the matrix by the normalized vector from the previous step. The convergence is geometric, i.e. the `1 distance between the true and the computed largest eigenvalue at the end of every step falls geometrically in the number of iterations and the rate depends on the ratio between the second largest and the largest eigenvalue. Some generalizations of the power method to compute the largest k eigenvalues and the eigenvectors will be discussed. The second method (Lanczos iteration) terminates in n iterations where each iteration in- volves estimating the largest (smallest) eigenvalue by maximizing (minimizing) the Rayleigh coefficient over vectors drawn from a suitable subspace. At each iteration, the dimension of the subspace involved in the optimization increases by 1. The sequence of subspaces used are Krylov subspaces associated with a random initial vector. We study the relation between Krylov subspaces and tri-diagonalization of a real symmetric matrix. Using this connection, we show that an estimate of the extreme eigenvalues can be computed at each iteration which involves eigen-decomposition of a tri-diagonal matrix.
  • AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration

    AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration

    AMS526: Numerical Analysis I (Numerical Linear Algebra) Lecture 16: Rayleigh Quotient Iteration Xiangmin Jiao SUNY Stony Brook Xiangmin Jiao Numerical Analysis I 1 / 10 Solving Eigenvalue Problems All eigenvalue solvers must be iterative Iterative algorithms have multiple facets: 1 Basic idea behind the algorithms 2 Convergence and techniques to speed-up convergence 3 Efficiency of implementation 4 Termination criteria We will focus on first two aspects Xiangmin Jiao Numerical Analysis I 2 / 10 Simplification: Real Symmetric Matrices We will consider eigenvalue problems for real symmetric matrices, i.e. T m×m m A = A 2 R , and Ax = λx for x 2 R p ∗ T T I Note: x = x , and kxk = x x A has real eigenvalues λ1,λ2, ::: , λm and orthonormal eigenvectors q1, q2, ::: , qm, where kqj k = 1 Eigenvalues are often also ordered in a particular way (e.g., ordered from large to small in magnitude) In addition, we focus on symmetric tridiagonal form I Why? Because phase 1 of two-phase algorithm reduces matrix into tridiagonal form Xiangmin Jiao Numerical Analysis I 3 / 10 Rayleigh Quotient m The Rayleigh quotient of x 2 R is the scalar xT Ax r(x) = xT x For an eigenvector x, its Rayleigh quotient is r(x) = xT λx=xT x = λ, the corresponding eigenvalue of x For general x, r(x) = α that minimizes kAx − αxk2. 2 x is eigenvector of A() rr(x) = x T x (Ax − r(x)x) = 0 with x 6= 0 r(x) is smooth and rr(qj ) = 0 for any j, and therefore is quadratically accurate: 2 r(x) − r(qJ ) = O(kx − qJ k ) as x ! qJ for some J Xiangmin Jiao Numerical Analysis I 4 / 10 Power
  • Math 504 (Fall 2011)

    Math 504 (Fall 2011)

    Instructor: Emre Mengi Math 504 (Fall 2011) Study Guide for Weeks 11-14 This homework concerns the following topics. • Basic definitions and facts about eigenvalues and eigenvectors (Trefethen&Bau, Lecture 24) • Similarity transformations (Trefethen&Bau, Lecture 24) • Power, inverse and Rayleigh iterations (Trefethen&Bau, Lecture 27) • QR algorithm with and without shifts (Trefethen&Bau, Lecture 28&29) • Simultaneous power iteration and its equivalence with the QR algorithm (Trefethen&Bau, Lectures 28&29) • The implicit QR algorithm • The Arnoldi Iteration (Trefethen&Bau, Lectures 33&34) • GMRES (Trefethen&Bau, Lecture 35) Homework 5 (Assigned on Dec 26th, Mon; Due on Jan 10th, Tue by 11:00) Please turn in the solutions only to the questions marked with (*). The rest is to practice on your own. Attach Matlab output, m-files, print-outs whenever they are necessary. The Dec 10, 11:00 deadline is tight. 1. (*) Consider the matrices 2 2 4 −5 3 1 7 A = and A = 0 1 3 1 3 5 2 4 5 0 0 1 (a) Find the eigenvalues of A1 and the eigenspace associated with each of its eigenvalues. (b) Find the eigenvalues of A2 together with their algebraic and geometric multiplicities. (c) Find a Schur factorization for A1. (d) Let v0 and v1 be two linearly independent eigenvectors of A1. Suppose also that fqkg denotes the sequence of vectors generated by the inverse iteration with shift σ = 2 and 2 starting with an initial vector q0 = α0v0 + α1v1 2 C where α0; α1 are nonzero scalars. Determine the subspace that spanfqkg is approaching as k ! 1.
  • Lecture 8: Eigenvalues, Eigenvectors and Spectral Theorem Lecturer: Shayan Oveis Gharan April 20Th Scribe: Sudipto Mukherjee

    Lecture 8: Eigenvalues, Eigenvectors and Spectral Theorem Lecturer: Shayan Oveis Gharan April 20Th Scribe: Sudipto Mukherjee

    CSE 521: Design and Analysis of Algorithms I Spring 2016 Lecture 8: Eigenvalues, Eigenvectors and Spectral Theorem Lecturer: Shayan Oveis Gharan April 20th Scribe: Sudipto Mukherjee Disclaimer: These notes have not been subjected to the usual scrutiny reserved for formal publications. 8.1 Introduction Let M 2 Rn×n be a symmetric matrix. We say λ is an eigenvalue of M with eigenvector v, if Mv = λv: Theorem 8.1. If M is symmetric, then all its eigenvalues are real. Proof. Suppose Mv = λv: We want to show that λ has imaginary value 0. For a complex number x = a + ib, the conjugate of x, is defined as follows: x∗ = a − ib. So, all we need to show is that λ = λ∗. The conjugate of a vector is the conjugate of all of its coordinate. Taking the conjugate transpose of both sides of the above equality, we have v∗M = λ∗v∗; (8.1) where we used that M T = M. So, on one hand, v∗Mv = v∗(Mv) = v∗(λv) = λ(v∗v): and on the other hand, by (8.1) v∗Mv = λ∗v∗v: So, we must have λ = λ∗. 8.2 Characteristic Polynomial If M does not have 0 as one of its eigenvalues, then det(M) 6= 0. An equivalent statement is that, if M all columns of M are linearly independent, then det(M) 6= 0. If λ is an eigenvalue of M, then Mv = λv, so (M − λI)v = 0. In other words, M − λI has a zero eigenvalue and det(M − λI) = 0.