DOCSLIB.ORG

Iterative Methods for the Eigenvalue Problem

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Iterative Methods for the Eigenvalue Problem 15/11/2013 Iterative methods for eigenvalue problem Project Report Group members: Radharaman Roy, Mouktik Chattopadhyay Necessity of an Iterative Method for Eigenvalues:In general, any method for computing eigenvalues necessarily involves an infinite number of steps. This is because finding eigenvalues of an n × n matrix is equivalent to finding the roots of its characteristic polynomial of degree n and Computing coefficients of characteristic polynomial requires computation of the determinant, however, the problem of finding the roots of a polynomial can be very ill-conditioned and for n > 4 such roots cannot be found (By Abel's theorem), in general, in a finite number of steps. In other words, we must consider iterative methods producing, at step k, an approximated eigenvector xk associated with an approximated eigenvalue λk that converge to the desired eigenvector x and eigenvalue λ as the number of iterations becomes larger. Here we present some iterative methods called the Schur factorization, QR method, power method, Bisection method, Jacobi's method, and Divide and conquer method. Jacobi eigenvalue algorithm: Jacobi eigenvalue algorithm is an iterative method for calculation of the eigenvalues and eigenvectors of a symmetric matrix. ••Givens rotation: A Givens rotation is represented by a matrix of the form 01 ··· 0 ··· 0 ··· 01 B. C B. C B C B0 ··· c · · · −s ··· 0C B C B. C G(i; j; θ) =B. C B C B0 ··· s ··· c ··· 0C B C B. C @. A 0 ··· 0 ··· 0 ··· 1 where c = cos θ, s = sin θ appear intersection of i th and j th rows and columns. That is the non zero elements of Givens matrix is given by gkk = 1 for k6=i; j gii=c gjj=-s gij = s for i > j. • Description of Jacobi algorithm: S0 = GT SG where G be a Givens rotation matrix & S be a symmetric matrix then S0 is also a symmetric matrix. S0 has entries 0 0 2 2 Sij = Sji = (c − s )Sij + sc(Sii − Sjj); 0 0 Sik = Ski = cSik − sSjk for k6=i,j 0 0 Sjk = Skj = sSik + cSjk for k6=j,i 0 2 2 Sii = c Sii − 2scSij + s Sjj; 0 2 2 Sjj = s Sii + 2scSij + c Sjj; ;where c = cos θ ; s = sin θ, G is orthogonal,S and S0 have the same Frobenious norm , however we can 0 0 choose θ s.t Sij = 0, in which case S has a large sum of squares on the diagonal. 0 1 Sij = (cos 2θ)SiJ + 2 (sin 2θ)(Sii − Sjj) Set this equal to 0 ; tan 2θ = 2Sij . Sjj −Sii The Jacobi eigenvalue method repeatedly performs rotation until the matrix becomes almost diagonal. Then the diagonal elements are approximations of the eigenvalues of S. • Convergence of Jacobi's method: Let od(A)is the root-sum-of-squares of the (upper) off diagonal entries of A, so A is diagonal if and only if od(A) = 0. Our goal is to make od(A) approach 0 quickly. The next lemma tells us that od(A) decreases monotonically with every Jacobi rotation. { Lemma: Let A0 be the matrix after calling Jacobi-Rotation (A, j, k)for any j,k. Then od2(A0) = 2 2 od (A) − ajk. Proof: Here A0 = A except in rows and columns j and k. Write 2 1 2 od (A)= 2 ×sum of square of all off diagonal elements of A except ajk + ajk 2 0 02 0 2 02 0 and similarly od (A ) = S + ajk = S , since ajk = 0 after calling Jacobi- Rotation(A,j,k). Since 2 02 k X kF = k QX kF and k X kF = k XQ kF for any X and any orthogonal Q, we can show S = S . 2 0 2 2 Thus od (A ) = od (A)- ajk as desired. 0 p 1 { Theorem: After one Jacobi's rotation in the classical Jacobi's algorithm, we have od(A ) ≤ (1 − N ), n(n−1) where N = 2 =the no. of sub-per diagonal entries of A. After k Jacobi rotation od(.) is no more k 1 2 than (1 − N ) od(A). 2 0 2 2 ( n(n−1) 2 Proof: od (A ) = od (A) − ajk, where ajk is the largest off diagonal entry. Thus od A) ≤ 2 ajk. 2 2 1 2 So od (A) − ajk ≤ (1 − N )od (A). So the classical Jacobi's algorithm converges at least linearly with the error (measured by od(A)) p 1 decreasing by a factor at least (1 − N ) at time. It eventually converges quadratically. • Algorithm: This method determines the eigenvalues and eigenvectors of a real symmetric matrix A, by converting A into a diagonal matrix by similarity transformation. { Step 1. Read the symmetric matrix A. { Step 2. Initialize D = A and S = I, a unit matrix. { Step 3. Find the largest off-diagonal element (in magnitude) from D = [dij] and let it be dij . { Step 4.Find the rotational angle . If dii = djj then π π if dij > 0 then θ = 4 else θ = − 4 endif; else ! θ = 1 tan1 2dij ; 2 dii−djj endif; { Step 5. Compute the matrix S1 = [spq] Set spq = 0 for all p; q = 1; 2; :::; n skk = 1, k = 1; 2; :::; n and sii = sjj = cos , sij = sin , sji = sin. T { Step 6. Find D = S1 DS1 and S = SS1; { Step 7. Repeat steps 3 to 6 until D becomes diagonal. { Step 8. Diagonal elements of D are the eigenvalues and the columns of S are the corresponding eigenvectors. • Program: This program finds all the eigenvalues and the corresponding eigenvectors of a real symmetric matrix. Assume that the given matrix is real symmetric. #include < stdio:h > #include < math:h > void main() f int n,i,j,p,q,flag; float a[10][10],d[10][10],s[10][10],s1[10][10],s1t[10][10]; float temp[10][10],theta,zero=1e-4,max,pi=3.141592654; printf("Enter the size of the matrix "); scanf("%d",&n); printf("Enter the elements row wise "); for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) scanf("%f"; &a[i][j]); printf("The given matrix is"); for(i = 1; i <= n; i + +) f for(j = 1; j <= n; j + +) printf("%8:5f"; a[i][j]); printf("nn"); g printf("nn"); for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) f d[i][j]=a[i][j]; s[i][j]=0; g for(i = 1; i <= n; i + +) s[i][i]=1; do f flag=0; i=1; j=2; max=fabs(d[1][2]); for(p = 1; p <= n; p + +) for(q = 1; q <= n; q + +) f if(p!=q) if(max < fabs(d[p][q])) f max=fabs(d[p][q]); i=p; j=q; g g if(d[i][i]==d[j][j]) f if(d[i][j] > 0) theta=pi/4; else theta=-pi/4; g else f theta=0.5*atan(2*d[i][j]/(d[i][i]-d[j][j])); g for(p = 1; p <= n; p + +) for(q = 1; q <= n; q + +) s1[p][q]=0; s1t[p][q]=0; for(p = 1; p <= n; p + +) s1[p][p]=1; s1t[p][p]=1; s1[i][i]=cos(theta); s1[j][j]=s1[i][i]; s1[j][i]=sin(theta); s1[i][j]=-s1[j][i]; s1t[i][i]=s1[i][i]; s1t[j][j]=s1[j][j]; s1t[i][j]=s1[j][i]; s1t[j][i]=s1[i][j]; for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +)f temp[i][j]=0; for(p = 1; p <= n; p + +) temp[i][j]+=s1t[i][p]*d[p][j]; g for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) f d[i][j]=0; for(p = 1; p <= n; p + +) d[i][j]+=temp[i][p]*s1[p][j]; g for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) f temp[i][j]=0; for(p = 1; p <= n; p + +) temp[i][j]+=s[i][p]*s1[p][j]; g for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) s[i][j]=temp[i][j]; for(i = 1; i <= n; i + +) for(j = 1; j <= n; j + +) f if(i!=j) if(fabs(d[i][j] > zero)) flag=1; g g while(flag==1); printf("The eigenvalues are nn"); for(i = 1; i <= n; i + +) printf("%8.5f ",d[i][i]); printf("nnThe corresponding eigenvectors are nn"); for(j = 1; j <= n; j + +) f for(i = 1; i < n; i + +) printf("% 8.5f,",s[i][j]); printf("%8.5fnn",s[n][j]) g g Divide and conquer method eigenvalue algorithm: This method we can use for symmetric tridiagonal matrix. • Divide: The divide part of the divide conquer algorithm from the realization that a tridiagonal matrix is almost block diagonal. ! T A T = 1 .where A and B be a matrix whose elements are 0 except (n; 1)th element of A and (1; n)th B T2 element of B and these elements are a. The size of sub-matrix T1 is n × n and then T2 is (m − n) × (m − n) 0 ! ! T1 0 a a T = 0 + 0 T2 a a 0 0 The only difference between T1 and T1 is that the lower right entry in T1 has been replaced with tnn - a and 0 similarly in T 2 . t t t t t • Conquer: First define z = (q1; q2) where q1 is the last row of Q1 and q2 is the first row of Q2 then ! ! ! t ! Q1 D1 t Q1 T = + azz t ) Q2 D2 Q2 Now we have to find the eigenvalues of a diagonal matrix plus a rank one correction .
Load more

Recommended publications
  • Efficient Algorithms for High-Dimensional Eigenvalue
    Efficient Algorithms for High-dimensional Eigenvalue Problems by Zhe Wang Department of Mathematics Duke University Date: Approved: Jianfeng Lu, Advisor Xiuyuan Cheng Jonathon Mattingly Weitao Yang Dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Mathematics in the Graduate School of Duke University 2020 ABSTRACT Efficient Algorithms for High-dimensional Eigenvalue Problems by Zhe Wang Department of Mathematics Duke University Date: Approved: Jianfeng Lu, Advisor Xiuyuan Cheng Jonathon Mattingly Weitao Yang An abstract of a dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Department of Mathematics in the Graduate School of Duke University 2020 Copyright © 2020 by Zhe Wang All rights reserved Abstract The eigenvalue problem is a traditional mathematical problem and has a wide appli- cations. Although there are many algorithms and theories, it is still challenging to solve the leading eigenvalue problem of extreme high dimension. Full configuration interaction (FCI) problem in quantum chemistry is such a problem. This thesis tries to understand some existing algorithms of FCI problem and propose new efficient algorithms for the high-dimensional eigenvalue problem. In more details, we first es- tablish a general framework of inexact power iteration and establish the convergence theorem of full configuration interaction quantum Monte Carlo (FCIQMC) and fast randomized iteration (FRI). Second, we reformulate the leading eigenvalue problem as an optimization problem, then compare the show the convergence of several coor- dinate descent methods (CDM) to solve the leading eigenvalue problem. Third, we propose a new efficient algorithm named Coordinate descent FCI (CDFCI) based on coordinate descent methods to solve the FCI problem, which produces some state-of- the-art results.
    [Show full text]
  • 2 Homework Solutions 18.335 " Fall 2004
    2 Homework Solutions 18.335 - Fall 2004 2.1 Count the number of ‡oating point operations required to compute the QR decomposition of an m-by-n matrix using (a) Householder re‡ectors (b) Givens rotations. 2 (a) See Trefethen p. 74-75. Answer: 2mn2 n3 ‡ops. 3 (b) Following the same procedure as in part (a) we get the same ‘volume’, 1 1 namely mn2 n3: The only di¤erence we have here comes from the 2 6 number of ‡opsrequired for calculating the Givens matrix. This operation requires 6 ‡ops (instead of 4 for the Householder re‡ectors) and hence in total we need 3mn2 n3 ‡ops. 2.2 Trefethen 5.4 Let the SVD of A = UV : Denote with vi the columns of V , ui the columns of U and i the singular values of A: We want to …nd x = (x1; x2) and such that: 0 A x x 1 = 1 A 0 x2 x2 This gives Ax2 = x1 and Ax1 = x2: Multiplying the 1st equation with A 2 and substitution of the 2nd equation gives AAx2 = x2: From this we may conclude that x2 is a left singular vector of A: The same can be done to see that x1 is a right singular vector of A: From this the 2m eigenvectors are found to be: 1 v x = i ; i = 1:::m p2 ui corresponding to the eigenvalues = i. Therefore we get the eigenvalue decomposition: 1 0 A 1 VV 0 1 VV = A 0 p2 U U 0 p2 U U 3 2.3 If A = R + uv, where R is upper triangular matrix and u and v are (column) vectors, describe an algorithm to compute the QR decomposition of A in (n2) time.
    [Show full text]
  • A Fast Algorithm for the Recursive Calculation of Dominant Singular Subspaces N
    View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Journal of Computational and Applied Mathematics 218 (2008) 238–246 www.elsevier.com/locate/cam A fast algorithm for the recursive calculation of dominant singular subspaces N. Mastronardia,1, M. Van Barelb,∗,2, R. Vandebrilb,2 aIstituto per le Applicazioni del Calcolo, CNR, via Amendola122/D, 70126, Bari, Italy bDepartment of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium Received 26 September 2006 Abstract In many engineering applications it is required to compute the dominant subspace of a matrix A of dimension m × n, with m?n. Often the matrix A is produced incrementally, so all the columns are not available simultaneously. This problem arises, e.g., in image processing, where each column of the matrix A represents an image of a given sequence leading to a singular value decomposition-based compression [S. Chandrasekaran, B.S. Manjunath, Y.F. Wang, J. Winkeler, H. Zhang, An eigenspace update algorithm for image analysis, Graphical Models and Image Process. 59 (5) (1997) 321–332]. Furthermore, the so-called proper orthogonal decomposition approximation uses the left dominant subspace of a matrix A where a column consists of a time instance of the solution of an evolution equation, e.g., the flow field from a fluid dynamics simulation. Since these flow fields tend to be very large, only a small number can be stored efficiently during the simulation, and therefore an incremental approach is useful [P. Van Dooren, Gramian based model reduction of large-scale dynamical systems, in: Numerical Analysis 1999, Chapman & Hall, CRC Press, London, Boca Raton, FL, 2000, pp.
    [Show full text]
  • New Fast and Accurate Jacobi Svd Algorithm: I
    NEW FAST AND ACCURATE JACOBI SVD ALGORITHM: I. ZLATKO DRMAC· ¤ AND KRESIMIR· VESELIC¶ y Abstract. This paper is the result of contrived e®orts to break the barrier between numerical accuracy and run time e±ciency in computing the fundamental decomposition of numerical linear algebra { the singular value decomposition (SVD) of a general dense matrix. It is an unfortunate fact that the numerically most accurate one{sided Jacobi SVD algorithm is several times slower than generally less accurate bidiagonalization based methods such as the QR or the divide and conquer algorithm. Despite its sound numerical qualities, the Jacobi SVD is not included in the state of the art matrix computation libraries and it is even considered obsolete by some leading researches. Our quest for a highly accurate and e±cient SVD algorithm has led us to a new, superior variant of the Jacobi algorithm. The new algorithm has inherited all good high accuracy properties, and it outperforms not only the best implementations of the one{sided Jacobi algorithm but also the QR algorithm. Moreover, it seems that the potential of the new approach is yet to be fully exploited. Key words. Jacobi method, singular value decomposition, eigenvalues AMS subject classi¯cations. 15A09, 15A12, 15A18, 15A23, 65F15, 65F22, 65F35 1. Introduction. In der Theorie der SÄacularstÄorungenund der kleinen Oscil- lationen wird man auf ein System lineÄarerGleichungen gefÄurt,in welchem die Co- e±cienten der verschiedenen Unbekanten in Bezug auf die Diagonale symmetrisch sind, die ganz constanten Glieder fehlen und zu allen in der Diagonale be¯ndlichen Coe±cienten noch dieselbe GrÄo¼e ¡x addirt ist.
    [Show full text]
  • Implicitly Restarted Arnoldi/Lanczos Methods for Large Scale Eigenvalue Calculations
    https://ntrs.nasa.gov/search.jsp?R=19960048075 2020-06-16T03:31:45+00:00Z NASA Contractor Report 198342 /" ICASE Report No. 96-40 J ICA IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen NASA Contract No. NASI-19480 May 1996 Institute for Computer Applications in Science and Engineering NASA Langley Research Center Hampton, VA 23681-0001 Operated by Universities Space Research Association National Aeronautics and Space Administration Langley Research Center Hampton, Virginia 23681-0001 IMPLICITLY RESTARTED ARNOLDI/LANCZOS METHODS FOR LARGE SCALE EIGENVALUE CALCULATIONS Danny C. Sorensen 1 Department of Computational and Applied Mathematics Rice University Houston, TX 77251 sorensen@rice, edu ABSTRACT Eigenvalues and eigenfunctions of linear operators are important to many areas of ap- plied mathematics. The ability to approximate these quantities numerically is becoming increasingly important in a wide variety of applications. This increasing demand has fu- eled interest in the development of new methods and software for the numerical solution of large-scale algebraic eigenvalue problems. In turn, the existence of these new methods and software, along with the dramatically increased computational capabilities now avail- able, has enabled the solution of problems that would not even have been posed five or ten years ago. Until very recently, software for large-scale nonsymmetric problems was virtually non-existent. Fortunately, the situation is improving rapidly. The purpose of this article is to provide an overview of the numerical solution of large- scale algebraic eigenvalue problems. The focus will be on a class of methods called Krylov subspace projection methods. The well-known Lanczos method is the premier member of this class.
    [Show full text]
  • Deflation by Restriction for the Inverse-Free Preconditioned Krylov Subspace Method
    NUMERICAL ALGEBRA, doi:10.3934/naco.2016.6.55 CONTROL AND OPTIMIZATION Volume 6, Number 1, March 2016 pp. 55{71 DEFLATION BY RESTRICTION FOR THE INVERSE-FREE PRECONDITIONED KRYLOV SUBSPACE METHOD Qiao Liang Department of Mathematics University of Kentucky Lexington, KY 40506-0027, USA Qiang Ye∗ Department of Mathematics University of Kentucky Lexington, KY 40506-0027, USA (Communicated by Xiaoqing Jin) Abstract. A deflation by restriction scheme is developed for the inverse-free preconditioned Krylov subspace method for computing a few extreme eigen- values of the definite symmetric generalized eigenvalue problem Ax = λBx. The convergence theory for the inverse-free preconditioned Krylov subspace method is generalized to include this deflation scheme and numerical examples are presented to demonstrate the convergence properties of the algorithm with the deflation scheme. 1. Introduction. The definite symmetric generalized eigenvalue problem for (A; B) is to find λ 2 R and x 2 Rn with x 6= 0 such that Ax = λBx (1) where A; B are n × n symmetric matrices and B is positive definite. The eigen- value problem (1), also referred to as a pencil eigenvalue problem (A; B), arises in many scientific and engineering applications, such as structural dynamics, quantum mechanics, and machine learning. The matrices involved in these applications are usually large and sparse and only a few of the eigenvalues are desired. Iterative methods such as the Lanczos algorithm and the Arnoldi algorithm are some of the most efficient numerical methods developed in the past few decades for computing a few eigenvalues of a large scale eigenvalue problem, see [1, 11, 19].
    [Show full text]
  • Numerical Linear Algebra Revised February 15, 2010 4.1 the LU
    Numerical Linear Algebra Revised February 15, 2010 4.1 The LU Decomposition The Elementary Matrices and badgauss In the previous chapters we talked a bit about the solving systems of the form Lx = b and Ux = b where L is lower triangular and U is upper triangular. In the exercises for Chapter 2 you were asked to write a program x=lusolver(L,U,b) which solves LUx = b using forsub and backsub. We now address the problem of representing a matrix A as a product of a lower triangular L and an upper triangular U: Recall our old friend badgauss. function B=badgauss(A) m=size(A,1); B=A; for i=1:m-1 for j=i+1:m a=-B(j,i)/B(i,i); B(j,:)=a*B(i,:)+B(j,:); end end The heart of badgauss is the elementary row operation of type 3: B(j,:)=a*B(i,:)+B(j,:); where a=-B(j,i)/B(i,i); Note also that the index j is greater than i since the loop is for j=i+1:m As we know from linear algebra, an elementary opreration of type 3 can be viewed as matrix multipli- cation EA where E is an elementary matrix of type 3. E looks just like the identity matrix, except that E(j; i) = a where j; i and a are as in the MATLAB code above. In particular, E is a lower triangular matrix, moreover the entries on the diagonal are 1's. We call such a matrix unit lower triangular.
    [Show full text]
  • Massively Parallel Poisson and QR Factorization Solvers
    Computers Math. Applic. Vol. 31, No. 4/5, pp. 19-26, 1996 Pergamon Copyright~)1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0898-1221/96 $15.00 + 0.00 0898-122 ! (95)00212-X Massively Parallel Poisson and QR Factorization Solvers M. LUCK£ Institute for Control Theory and Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrluck@savba, sk M. VAJTERSIC Institute of Informatics, Slovak Academy of Sciences DdbravskA cesta 9, 840 00 Bratislava, P.O. Box 56, Slovak Republic kaifmava©savba, sk E. VIKTORINOVA Institute for Control Theoryand Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrevka@savba, sk Abstract--The paper brings a massively parallel Poisson solver for rectangle domain and parallel algorithms for computation of QR factorization of a dense matrix A by means of Householder re- flections and Givens rotations. The computer model under consideration is a SIMD mesh-connected toroidal n x n processor array. The Dirichlet problem is replaced by its finite-difference analog on an M x N (M + 1, N are powers of two) grid. The algorithm is composed of parallel fast sine transform and cyclic odd-even reduction blocks and runs in a fully parallel fashion. Its computational complexity is O(MN log L/n2), where L = max(M + 1, N). A parallel proposal of QI~ factorization by the Householder method zeros all subdiagonal elements in each column and updates all elements of the given submatrix in parallel. For the second method with Givens rotations, the parallel scheme of the Sameh and Kuck was chosen where the disjoint rotations can be computed simultaneously.
    [Show full text]
  • Accelerated Stochastic Power Iteration
    Accelerated Stochastic Power Iteration CHRISTOPHER DE SAy BRYAN HEy IOANNIS MITLIAGKASy CHRISTOPHER RE´ y PENG XU∗ yDepartment of Computer Science, Stanford University ∗Institute for Computational and Mathematical Engineering, Stanford University cdesa,bryanhe,[email protected], [email protected], [email protected] July 11, 2017 Abstract Principal component analysis (PCA) is one of the most powerful tools in machine learning. The simplest method for PCA, the power iteration, requires O(1=∆) full-data passes to recover the principal component of a matrix withp eigen-gap ∆. Lanczos, a significantly more complex method, achieves an accelerated rate of O(1= ∆) passes. Modern applications, however, motivate methods that only ingest a subset of available data, known as the stochastic setting. In the online stochastic setting, simple 2 2 algorithms like Oja’s iteration achieve the optimal sample complexity O(σ =p∆ ). Unfortunately, they are fully sequential, and also require O(σ2=∆2) iterations, far from the O(1= ∆) rate of Lanczos. We propose a simple variant of the power iteration with an added momentum term, that achieves both the optimal sample and iteration complexity.p In the full-pass setting, standard analysis shows that momentum achieves the accelerated rate, O(1= ∆). We demonstrate empirically that naively applying momentum to a stochastic method, does not result in acceleration. We perform a novel, tight variance analysis that reveals the “breaking-point variance” beyond which this acceleration does not occur. By combining this insight with modern variance reduction techniques, we construct stochastic PCAp algorithms, for the online and offline setting, that achieve an accelerated iteration complexity O(1= ∆).
    [Show full text]
  • Numerical Linear Algebra Program Lecture 4 Basic Methods For
    Program Lecture 4 Numerical Linear Algebra • Basic methods for eigenproblems. Basic iterative methods • Power method • Shift-and-invert Power method • QR algorithm • Basic iterative methods for linear systems • Richardson’s method • Jacobi, Gauss-Seidel and SOR • Iterative refinement Gerard Sleijpen and Martin van Gijzen • Steepest decent and the Minimal residual method October 5, 2016 1 October 5, 2016 2 National Master Course National Master Course Delft University of Technology Basic methods for eigenproblems The Power method The eigenvalue problem The Power method is the classical method to compute in modulus largest eigenvalue and associated eigenvector of a Av = λv matrix. can not be solved in a direct way for problems of order > 4, since Multiplying with a matrix amplifies strongest the eigendirection the eigenvalues are the roots of the characteristic equation corresponding to the in modulus largest eigenvalues. det(A − λI) = 0. Successively multiplying and scaling (to avoid overflow or underflow) yields a vector in which the direction of the largest Today we will discuss two iterative methods for solving the eigenvector becomes more and more dominant. eigenproblem. October 5, 2016 3 October 5, 2016 4 National Master Course National Master Course Algorithm Convergence (1) The Power method for an n × n matrix A. Let the n eigenvalues λi with eigenvectors vi, Avi = λivi, be n ordered such that |λ1| ≥ |λ2|≥ . ≥ |λn|. u0 ∈ C is given • Assume the eigenvectors v ,..., vn form a basis. for k = 1, 2, ... 1 • Assume |λ1| > |λ2|. uk = Auk−1 Each arbitrary starting vector u0 can be written as: uk = uk/kukk2 e(k) ∗ λ = uk−1uk u0 = α1v1 + α2v2 + ..
    [Show full text]
  • A Generalized Eigenvalue Algorithm for Tridiagonal Matrix Pencils Based on a Nonautonomous Discrete Integrable System
    A generalized eigenvalue algorithm for tridiagonal matrix pencils based on a nonautonomous discrete integrable system Kazuki Maedaa, , Satoshi Tsujimotoa ∗ aDepartment of Applied Mathematics and Physics, Graduate School of Informatics, Kyoto University, Kyoto 606-8501, Japan Abstract A generalized eigenvalue algorithm for tridiagonal matrix pencils is presented. The algorithm appears as the time evolution equation of a nonautonomous discrete integrable system associated with a polynomial sequence which has some orthogonality on the support set of the zeros of the characteristic polynomial for a tridiagonal matrix pencil. The convergence of the algorithm is discussed by using the solution to the initial value problem for the corresponding discrete integrable system. Keywords: generalized eigenvalue problem, nonautonomous discrete integrable system, RII chain, dqds algorithm, orthogonal polynomials 2010 MSC: 37K10, 37K40, 42C05, 65F15 1. Introduction Applications of discrete integrable systems to numerical algorithms are important and fascinating top- ics. Since the end of the twentieth century, a number of relationships between classical numerical algorithms and integrable systems have been studied (see the review papers [1–3]). On this basis, new algorithms based on discrete integrable systems have been developed: (i) singular value algorithms for bidiagonal matrices based on the discrete Lotka–Volterra equation [4, 5], (ii) Pade´ approximation algorithms based on the dis- crete relativistic Toda lattice [6] and the discrete Schur flow [7], (iii) eigenvalue algorithms for band matrices based on the discrete hungry Lotka–Volterra equation [8] and the nonautonomous discrete hungry Toda lattice [9], and (iv) algorithms for computing D-optimal designs based on the nonautonomous discrete Toda (nd-Toda) lattice [10] and the discrete modified KdV equation [11].
    [Show full text]
  • Arxiv:1105.1185V1 [Math.NA] 5 May 2011 Ento 2.2
    ITERATIVE METHODS FOR COMPUTING EIGENVALUES AND EIGENVECTORS MAYSUM PANJU Abstract. We examine some numerical iterative methods for computing the eigenvalues and eigenvectors of real matrices. The five methods examined here range from the simple power iteration method to the more complicated QR iteration method. The derivations, procedure, and advantages of each method are briefly discussed. 1. Introduction Eigenvalues and eigenvectors play an important part in the applications of linear algebra. The naive method of finding the eigenvalues of a matrix involves finding the roots of the characteristic polynomial of the matrix. In industrial sized matrices, however, this method is not feasible, and the eigenvalues must be obtained by other means. Fortunately, there exist several other techniques for finding eigenvalues and eigenvectors of a matrix, some of which fall under the realm of iterative methods. These methods work by repeatedly refining approximations to the eigenvectors or eigenvalues, and can be terminated whenever the approximations reach a suitable degree of accuracy. Iterative methods form the basis of much of modern day eigenvalue computation. In this paper, we outline five such iterative methods, and summarize their derivations, procedures, and advantages. The methods to be examined are the power iteration method, the shifted inverse iteration method, the Rayleigh quotient method, the simultaneous iteration method, and the QR method. This paper is meant to be a survey over existing algorithms for the eigenvalue computation problem. Section 2 of this paper provides a brief review of some of the linear algebra background required to understand the concepts that are discussed. In section 3, the iterative methods are each presented, in order of complexity, and are studied in brief detail.
    [Show full text]