DOCSLIB.ORG

Applied Numerical Linear Algebra. Lecture 14

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Applied Numerical Linear Algebra. Lecture 14 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Applied Numerical Linear Algebra. Lecture 14. 1/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) 5.3.3. Divide-and-Conquer This method is the fastest now available if you want all eigenvalues and eigenvectors of a tridiagonal matrix whose dimension is larger than about 25. (The exact threshold depends on the computer.) It is quite subtle to implement in a numerically stable way. Indeed, although this method was first introduced in 1981 [J. J. M. Cuppen. A divide and conquer method for the symmetric tridiagonal eigenproblem. Numer. Math., 36:177-195, 1981], the ”right” implementation was not discovered until 1992 [M. Gu and S. Eisenstat. A stable algorithm for the rank-1 modification of the symmetric eigenproblem. Computer Science Dept. Report YALEU/DCS/RR-916, Yale University, September 1992;M. Gu and S. C. Eisenstat. A divide-and-conquer algorithm for the symmetric tridiagonal eigenproblem. SIAM J. Matrix Anal Appl, 16:172-191, 1995]). 2/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) a1 b1 0 ... ... 0 b1 a2 b2 0 ... 0 ... ... ... ... ... ... ... am 1 bm 1 ... ... 0 − − 0 bm 1 am bm 0 0 T = − 0 0 b a b 0 m m+1 m+1 0 0 0 b ... ... m+1 ... ... ... ... ... ... 0 0 0 ... ... bn 1 − 00 0 0 bn 1 an − 3/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) a1 b1 0 ... ... 0 b1 a2 b2 0 ... 0 ... ... ... ... ... ... ... am 1 bm 1 ... ... 0 ... ... − − 0 bm 1 am bm 0 0 0 ...bm bm... T = − − + 0 0 0 a b b 0 ...b b ... m+1 − m m+1 m m 0 0 0 b ... ... ... ... m+1 ... ... ... ... ... ... 0 0 0 ... ... bn 1 − 00 0 0 bn 1 an − 0 ... 0 T1 0...0 1 = + bm 0...0 1 1 0....0 0...0 T2 · 1 · 0 .... 0 4/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Assume that we have eigendecomposition of T1, T2 such that T T T1 = Q1Λ1Q1 and T2 = Q2Λ2Q2 . Then we can write that T T1 0...0 T Q1Λ1Q1 0...0 T T = + bmv v = T + bmv v 0...0 T2 · 0...0 Q2Λ2Q · 2 T Q1 0...0 Λ1 0...0 T Q1 0...0 = + bmu u T 0...0 Q2 · 0...0 Λ2 · · 0...0 Q2 Let define the diagonal matrix Λ 0...0 D = 1 0...0 Λ 2 We observe that the eigenvalues of T are the same as of D + b u uT = D + ρu uT . m · · 5/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) 1. We want to find eigenvalues of D + b u uT = D + ρu uT m · · 2. We assume that diagonal elements of D are sorted such that d ... d and D + λI is nonsingular 1 ≥ ≥ n 3. To find eigenvalues we compute the characteristic polynomial D + ρu uT λI noting that · T − 1 T D + ρu u λI =(D λI )(I + ρ(D λI )− u u ) · − − − · By assumption (2) we have det(D λI ) = 0 and thus − 6 1 T det(I + ρ(D λI )− u u )=0. − · 6/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) LEMMA 5.1. If x and y are vectors, det(I + xy T )=1+ y T x. Thus, using this lemma we can get that 2 1 T T 1 n ui det(I + ρ(D λI )− u u )=1+ u ρ(D λI )− u =1+ ρ = i=1 di λ f (λ). − · − − We see that eigenvalues of T are roots of the secular equationPf (λ)=0. The secular equation can be solved using Newton’s method with starting point (di , di+1). T 1 LEMMA 5.2. If α is an eigenvalue of D + ρuu , then (D αI )− u is its eigenvector. Since D αI is diagonal, this costs O(n) flops− to compute. − 7/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) ALGORITHM 5.2. Finding eigenvalues and eigenvectors of a symmetric tridiagonal matrix using divide-and-conquer: proc dc eig (T , Q, Λ) ..... from input T compute outputs Q and Λ where T = QΛQT if T is 1-by-1 return Q = 1, Λ= T else T 0 form T = 1 + b υυT 0 T m 2 call dc eig (T1, Q1, Λ1) call dc eig (T2, Q2, Λ2) T form D + ρuu from Λ1, Λ2, Q1, Q2 find eigenvalues Λ and eigenvectors Q′ of D + ρuuT Q 0 form Q = 1 Q′ = eigenvectors of T 0 Q2 · return Q and Λ end if 8/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Computing the Eigenvectors Stably Lemma 5.2 provides formula for the computing of eigenvectors. This formula is not stable in the case when two eigenvalues are close to each other. Let αi ,αi+1 are close to each other. Then (D α )−1u and (D α )−1u are inaccurate and far − i − i+1 from orthogonal. An alternative formula was found which is based on the L¨owner’s theorem. 9/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) THEOREM 5.10. (L¨owner). Let D = diag(d1,..., dn) be diagonal with dn <...< d1. Let αn <...<α1 be given, satisfying the interlacing property d <α < < d <α < d <α < < d <α . n n ··· i+1 i+1 i i ··· 1 1 Then there is a vectoru ˆ such that the αi are the exact eigenvalues of Dˆ D +u ˆuˆT . The entries ofu ˆ are given by ≡ n 1/2 j=1(αj di ) uˆi = n − . | | " j=1,j=i (dj di )# Q 6 − Q 10/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Proof. The characteristic polynomial of Dˆ can be written both as ˆ n ˆ T det(D λI )= j=1(αj λ) and as det(D λI ) = det(D +u ˆuˆ λI )= − 1 T − − 1 T − det(D(I + D− uˆuˆ ) λI ) = det((D λI )(I +(D λI )− uˆuˆ )) or Q − − − n n 2 uˆj det(Dˆ λI ) = (dj λ) 1+ − − · dj λ Yj=1 Xj=1 − n n 2 uˆj = (dj λ) 1+ − · dj λ j=1 j = 1 − Y X j = i 6 n 2 + (dj λ) uˆi . − · j = 1 Y j = i 6 11/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Setting λ = d in the above eq. and in det(Dˆ λI )= n (α λ), i − j=1 j − equating both expressions for det(Dˆ λI ) yield − Q n n (α d ) =u ˆ2 (d d ) j − i i · j − i Yj=1 j Y= 1 j = i 6 or n 2 j=1(αj di ) uˆi = n − . j=1,j=i (dj di ) Q 6 − Using the interlacing property,Q we can show that the fraction on the right is positive, so we can take its square root to get the desired expression for uˆi . 12/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) Here is the stable algorithm for computing the eigenvalues and eigenvectors (where we assume for simplicity of presentation that ρ = 1). ALGORITHM 5.3. Compute the eigenvalues and eigenvectors of D + uuT . u2 Solve the secular equation 1+ n i = 0 to get the eigenvalues i=1 di λ α of D + uuT − i P Use L¨owner’s theorem to compute uˆ so that the αi are ”exact” eigenvalues of D +u ˆuˆT . Use Lemma 5.2 to compute the eigenvectors of Dˆ = D +u ˆuˆT . 13/54 Algorithms for the Symmetric Eigenproblem: Divide-and-Conquer Algorithms for the Symmetric Eigenproblem: Bisection and Inverse Iteration Algorithms for the Symmetric Eigenproblem: Jacobi’s Method Algorithms for the Singular Value Decomposition(SVD) 5.3.4. Bisection and Inverse Iteration The Bisection algorithm exploits Sylvester’s inertia theorem (Theorem 5.3) to find only those k eigenvalues that one wants, at cost O(nk).
Load more

Recommended publications
  • New Fast and Accurate Jacobi Svd Algorithm: I
    NEW FAST AND ACCURATE JACOBI SVD ALGORITHM: I. ZLATKO DRMAC· ¤ AND KRESIMIR· VESELIC¶ y Abstract. This paper is the result of contrived e®orts to break the barrier between numerical accuracy and run time e±ciency in computing the fundamental decomposition of numerical linear algebra { the singular value decomposition (SVD) of a general dense matrix. It is an unfortunate fact that the numerically most accurate one{sided Jacobi SVD algorithm is several times slower than generally less accurate bidiagonalization based methods such as the QR or the divide and conquer algorithm. Despite its sound numerical qualities, the Jacobi SVD is not included in the state of the art matrix computation libraries and it is even considered obsolete by some leading researches. Our quest for a highly accurate and e±cient SVD algorithm has led us to a new, superior variant of the Jacobi algorithm. The new algorithm has inherited all good high accuracy properties, and it outperforms not only the best implementations of the one{sided Jacobi algorithm but also the QR algorithm. Moreover, it seems that the potential of the new approach is yet to be fully exploited. Key words. Jacobi method, singular value decomposition, eigenvalues AMS subject classi¯cations. 15A09, 15A12, 15A18, 15A23, 65F15, 65F22, 65F35 1. Introduction. In der Theorie der SÄacularstÄorungenund der kleinen Oscil- lationen wird man auf ein System lineÄarerGleichungen gefÄurt,in welchem die Co- e±cienten der verschiedenen Unbekanten in Bezug auf die Diagonale symmetrisch sind, die ganz constanten Glieder fehlen und zu allen in der Diagonale be¯ndlichen Coe±cienten noch dieselbe GrÄo¼e ¡x addirt ist.
    [Show full text]
  • The Singular Value Decomposition: Anatomy of Optimizing an Algorithm for Extreme Scale
    The University of Manchester Research The singular value decomposition DOI: 10.1137/17M1117732 Document Version Final published version Link to publication record in Manchester Research Explorer Citation for published version (APA): Dongarra, J., Gates, M., Haidar, A., Kurzak, J., Luszczek, P., Tomov, S., & Yamazaki, I. (2018). The singular value decomposition: Anatomy of optimizing an algorithm for extreme scale. SIAM REVIEW, 60(4), 808-865. https://doi.org/10.1137/17M1117732 Published in: SIAM REVIEW Citing this paper Please note that where the full-text provided on Manchester Research Explorer is the Author Accepted Manuscript or Proof version this may differ from the final Published version. If citing, it is advised that you check and use the publisher's definitive version. General rights Copyright and moral rights for the publications made accessible in the Research Explorer are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights. Takedown policy If you believe that this document breaches copyright please refer to the University of Manchester’s Takedown Procedures [http://man.ac.uk/04Y6Bo] or contact [email protected] providing relevant details, so we can investigate your claim. Download date:06. Oct. 2021 SIAM REVIEW \bigcirc c 2018 Society for Industrial and Applied Mathematics Vol. 60, No. 4, pp. 808–865 The Singular Value Decomposition: Anatomy of Optimizing an Algorithm \ast for Extreme Scale Jack Dongarray z Mark Gates Azzam Haidarz Jakub Kurzakz Piotr Luszczekz Stanimire Tomovz Ichitaro Yamazakiz Abstract. The computation of the singular value decomposition, or SVD, has a long history with many improvements over the years, both in its implementations and algorithmically.
    [Show full text]
  • Singular Value Decomposition in Embedded Systems Based on ARM Cortex-M Architecture
    electronics Article Singular Value Decomposition in Embedded Systems Based on ARM Cortex-M Architecture Michele Alessandrini †, Giorgio Biagetti † , Paolo Crippa *,† , Laura Falaschetti † , Lorenzo Manoni † and Claudio Turchetti † Department of Information Engineering, Università Politecnica delle Marche, Via Brecce Bianche 12, I-60131 Ancona, Italy; [email protected] (M.A.); [email protected] (G.B.); [email protected] (L.F.); [email protected] (L.M.); [email protected] (C.T.) * Correspondence: [email protected]; Tel.: +39-071-220-4541 † These authors contributed equally to this work. Abstract: Singular value decomposition (SVD) is a central mathematical tool for several emerging applications in embedded systems, such as multiple-input multiple-output (MIMO) systems, data an- alytics, sparse representation of signals. Since SVD algorithms reduce to solve an eigenvalue problem, that is computationally expensive, both specific hardware solutions and parallel implementations have been proposed to overcome this bottleneck. However, as those solutions require additional hardware resources that are not in general available in embedded systems, optimized algorithms are demanded in this context. The aim of this paper is to present an efficient implementation of the SVD algorithm on ARM Cortex-M. To this end, we proceed to (i) present a comprehensive treatment of the most common algorithms for SVD, providing a fairly complete and deep overview of these algorithms, with a common notation, (ii) implement them on an ARM Cortex-M4F microcontroller, in order to develop a library suitable for embedded systems without an operating system, (iii) find, through a comparative study of the proposed SVD algorithms, the best implementation suitable for a low-resource bare-metal embedded system, (iv) show a practical application to Kalman filtering of an inertial measurement unit (IMU), as an example of how SVD can improve the accuracy of existing algorithms and of its usefulness on a such low-resources system.
    [Show full text]
  • New Fast and Accurate Jacobi Svd Algorithm: I. Lapack Working Note 169
    NEW FAST AND ACCURATE JACOBI SVD ALGORITHM: I. LAPACK WORKING NOTE 169 ZLATKO DRMACˇ ∗ AND KRESIMIRˇ VESELIC´ † Abstract. This paper is the result of contrived efforts to break the barrier between numerical accuracy and run time efficiency in computing the fundamental decomposition of numerical linear algebra – the singular value decomposition (SVD) of a general dense matrix. It is an unfortunate fact that the numerically most accurate one–sided Jacobi SVD algorithm is several times slower than generally less accurate bidiagonalization based methods such as the QR or the divide and conquer algorithm. Despite its sound numerical qualities, the Jacobi SVD is not included in the state of the art matrix computation libraries and it is even considered obsolete by some leading researches. Our quest for a highly accurate and efficient SVD algorithm has led us to a new, superior variant of the Jacobi algorithm. The new algorithm has inherited all good high accuracy properties, and it outperforms not only the best implementations of the one–sided Jacobi algorithm but also the QR algorithm. Moreover, it seems that the potential of the new approach is yet to be fully exploited. Key words. Jacobi method, singular value decomposition, eigenvalues AMS subject classifications. 15A09, 15A12, 15A18, 15A23, 65F15, 65F22, 65F35 1. Introduction. In der Theorie der S¨acularst¨orungen und der kleinen Oscil- lationen wird man auf ein System line¨arer Gleichungen gef¨urt, in welchem die Co- efficienten der verschiedenen Unbekanten in Bezug auf die Diagonale symmetrisch sind, die ganz constanten Glieder fehlen und zu allen in der Diagonale befindlichen Coefficienten noch dieselbe Gr¨oße x addirt ist.
    [Show full text]
  • Arxiv:1301.5758V3
    Generalized Householder Transformations for the Complex Symmetric Eigenvalue Problem J. H. Noble,1 M. Lubasch,2 and U. D. Jentschura1, 3 1Department of Physics, Missouri University of Science and Technology, Rolla, Missouri 65409, USA 2Max–Planck–Institut f¨ur Quantenoptik, Hans–Kopfermann–Straße 1, 85748 Garching, Germany 3MTA–DE Particle Physics Research Group, P.O.Box 51, H-4001 Debrecen, Hungary We present an intuitive and scalable algorithm for the diagonalization of complex symmet- ric matrices, which arise from the projection of pseudo–Hermitian and complex scaled Hamilto- nians onto a suitable basis set of “trial” states. The algorithm diagonalizes complex and sym- metric (non–Hermitian) matrices and is easily implemented in modern computer languages. It is based on generalized Householder transformations and relies on iterative similarity transformations ′ − T → T = QT T Q, where Q is a complex and orthogonal, but not unitary, matrix, i.e, QT = Q 1 − but Q+ 6= Q 1. We present numerical reference data to support the scalability of the algorithm. We construct the generalized Householder transformations from the notion that the conserved scalar product of eigenstates Ψn and Ψm of a pseudo–Hermitian quantum mechanical Hamiltonian can be reformulated in terms of the generalized indefinite inner product R dx Ψn(x,t)Ψm(x,t), where the integrand is locally defined, and complex conjugation is avoided. A few example calculations are described which illustrate the physical origin of the ideas used in the construction of the algorithm. PACS numbers: 03.65.Ge, 02.60.Lj, 11.15.Bt, 11.30.Er, 02.60.-x, 02.60.Dc I.
    [Show full text]
  • New Fast and Accurate Jacobi Svd Algorithm: Ii. Lapack Working Note 170
    NEW FAST AND ACCURATE JACOBI SVD ALGORITHM: II. LAPACK WORKING NOTE 170 ZLATKO DRMACˇ ∗ AND KRESIMIRˇ VESELIC´ † Abstract. This paper presents new implementation of one–sided Jacobi SVD for triangular matrices and its use as the core routine in a new preconditioned Jacobi SVD algorithm, recently proposed by the authors. New pivot strategy exploits the triangular form and uses the fact that the input triangular matrix is the result of rank revealing QR factorization. If used in the preconditioned Jacobi SVD algorithm, described in the first part of this report, it delivers superior performance leading to the currently fastest method for computing SVD decomposition with high relative accuracy. Furthermore, the efficiency of the new algorithm is comparable to the less accurate bidiagonalization based methods. The paper also discusses underflow issues in floating point implementation, and shows how to use perturbation theory to fix the imperfectness of machine arithmetic on some systems. Key words. Jacobi method, singular value decomposition, eigenvalues AMS subject classifications. 15A09, 15A12, 15A18, 15A23, 65F15, 65F22, 65F35 1. Introduction. Jacobi iteration is one of the time-honored methods to com- pute the spectral decomposition H = V ΛV T of a real symmetric matrix H. The early discovery in 1846. is certainly due to the simplicity and the elegance of the method as well as to the geniality of the 19’th century computing matador C. G. J. Jacobi who called it ’Ein leichtes Verfahren’ and applied it to compute the secular pertur- bations of the planets. Jacobi’s original article [23] is a very masterpiece of applied mathematics and may even today be read with profit by both students and scientists.
    [Show full text]
  • Parallelization and Performance Optimization of Bioinformatics and Biomedical Applications Targeted to Advanced Computer Architectures
    PARALLELIZATION AND PERFORMANCE OPTIMIZATION OF BIOINFORMATICS AND BIOMEDICAL APPLICATIONS TARGETED TO ADVANCED COMPUTER ARCHITECTURES by Yanwei Niu A dissertation submitted to the Faculty of the University of Delaware in partial fulfillment of the requirements for the degree of Doctor of Philosophy in Electrical and Computer Engineering Summer 2005 c 2005 Yanwei Niu All Rights Reserved UMI Number: 3181852 UMI Microform 3181852 Copyright 2005 by ProQuest Information and Learning Company. All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. ProQuest Information and Learning Company 300 North Zeeb Road P.O. Box 1346 Ann Arbor, MI 48106-1346 PARALLELIZATION AND PERFORMANCE OPTIMIZATION OF BIOINFORMATICS AND BIOMEDICAL APPLICATIONS TARGETED TO ADVANCED COMPUTER ARCHITECTURES by Yanwei Niu Approved: Gonzalo R. Arce, Ph.D. Chairperson of the Department of Electrical and Computer Engineering Approved: Eric W. Kaler, Ph.D. Dean of the College of Engineering Approved: Conrado M. Gempesaw II, Ph.D. Vice Provost for Academic and International Programs I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: Guang R. Gao, Ph.D. Professor in charge of dissertation I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy. Signed: Kenneth E. Barner, Ph.D. Professor in charge of dissertation I certify that I have read this dissertation and that in my opinion it meets the academic and professional standard required by the University as a dissertation for the degree of Doctor of Philosophy.
    [Show full text]
  • HP MLIB User's Guide
    HP MLIB User’s Guide Seventh Edition HP Part No. B6061-96027 and B6061-96028 HP MLIB December 2004 Printed in USA Edition: Seventh Document Numbers: B6061-96027 and B6061-96028 Remarks: Released Dcember 2004 with HP MLIB software version9.0. User’s Guide split into two volumes with this release. Edition: Sixth Document Number: B6061-96023 Remarks: Released September 2003 with HP MLIB software version 8.50. Edition: Fifth Document Number: B6061-96020 Remarks: Released September 2002 with HP MLIB software version 8.30. Edition: Fourth Document Number: B6061-96017 Remarks: Released June 2002 with HP MLIB software version 8.20. Edition: Third Document Number: B6061-96015 Remarks: Released September 2001 with HP MLIB software version 8.10. Edition: Second Document Number: B6061-96012 Remarks: Released June 2001 with HP MLIB software version 8.00. Edition: First Document Number: B6061-96010 Remarks: Released December 1999 with HP MLIB software version B.07.00. This HP MLIB User’s Guide contains both HP VECLIB and HP LAPACK information. This document supercedes both the HP MLIB VECLIB User’s Guide, fifth edition (B6061-96006) and the HP MLIB LAPACK User’s Guide, sixth edition (B6061-96005). Notice Copyright 1979-2004 Hewlett-Packard Development Company. All Rights Reserved. Reproduction, adaptation, or translation without prior written permission is prohibited, except as allowed under the copyright laws. The information contained in this document is subject to change without notice. Hewlett-Packard makes no warranty of any kind with regard to this material, including, but not limited to, the implied warranties of merchantability and fitness for a particular purpose.
    [Show full text]
  • 1. Introduction
    Neural, Parallel, and Scientific Computations 19 (2011) 129-164 AN OVERVIEW ON THE EIGENVALUE COMPUTATION FOR MATRICES MOHAN K. KADALBAJOO1 AND ANKIT GUPTA2 1Department of Mathematics and Statistics, Indian Institute of Technology, Kanpur - 208016 Email Id: [email protected] 2Institute of Technology, Banaras Hindu University, Varanasi – 221005 Email Id: [email protected] Abstract. This paper sketches the research developments in the area of computational methods for solving the eigenvalue problems and how the methods developed relate to each other as they evolve over centuries. This is an attempt to write a complete overview on the research on computational aspects of eigenvalue problem, emphasize the history of methods that still play a role and some of those that no longer are considered to be on the main track but are somehow related to the present techniques in some smaller steps. This contribution brings out the state-of-the-art of the algorithms for solving large-scale eigenvalue problems for both symmetric and nonsymmetric matrices separately, thereby clearly drawing a comparison between the differences in the algorithms in practical use for the two. Some of the methods or modifications to the earlier methods that have been notable at the turn of the 21st century will also be covered through this paper under ―Modern Approaches‖. Also as the standard eigenvalue problem has become better understood, in a numerical sense, progress has been made in the generalized eigenvalue problem and this will briefly cover developments in solving these problems too. 1. INTRODUCTION The terminology eigenvalue comes from the German word Eigenvert which means proper or characteristic value.
    [Show full text]
  • Using Reconfigurable Computing Technology to Accelerate Matrix Decomposition and Applications Xinying Wang Iowa State University
    Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations 2016 Using reconfigurable computing technology to accelerate matrix decomposition and applications Xinying Wang Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/etd Part of the Computer Engineering Commons Recommended Citation Wang, Xinying, "Using reconfigurable computing technology to accelerate matrix decomposition and applications" (2016). Graduate Theses and Dissertations. 15834. https://lib.dr.iastate.edu/etd/15834 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Using reconfigurable computing technology to accelerate matrix decomposition and applications by Xinying Wang A dissertation submitted to the graduate faculty in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Major: Computer Engineering Program of Study Committee: Joseph A. Zambreno, Major Professor Phillip H. Jones Tien N. Nguyen Diane T. Rover Zhao Zhang Iowa State University Ames, Iowa 2016 Copyright c Xinying Wang, 2016. All rights reserved. ii DEDICATION I would like to dedicate this dissertation to my family for their continuous encouragement and financial assistance in my education. I would also like to thank my friends for their help and support during the writing of this work. iii TABLE OF CONTENTS LIST OF TABLES . ix LIST OF FIGURES . x ACKNOWLEDGEMENTS . xv ABSTRACT . xvi CHAPTER 1. INTRODUCTION . 1 1.1 Problem: matrix decomposition .
    [Show full text]
  • Parallel Jacobi-Type Algorithms for the Singular and the Generalized Singular Value Decomposition
    Faculty of Science Department of Mathematics Vedran Novaković Parallel Jacobi-type algorithms for the singular and the generalized singular value decomposition DOCTORAL THESIS Zagreb, 2017 Faculty of Science Department of Mathematics Vedran Novaković Parallel Jacobi-type algorithms for the singular and the generalized singular value decomposition DOCTORAL THESIS Supervisor: prof. Sanja Singer, PhD Zagreb, 2017 Prirodoslovno–matematički fakultet Matematički odsjek Vedran Novaković Paralelni algoritmi Jacobijeva tipa za singularnu i generaliziranu singularnu dekompoziciju DOKTORSKI RAD Mentor: prof. dr. sc. Sanja Singer Zagreb, 2017. A note on the supervisor and the committees (O mentoru i komisijama) Sanja Singer was born in Zagreb on December 27th, 1963, where she earned BSc. (1986), MSc. (1993) and PhD. (1997) in mathematics at the University of Zagreb, Faculty of Science, Department of Mathematics. From December 1987 to September 1989 she was employed as a junior assistant at the University of Zagreb, Faculty of Economics. Since September 1989 she has been affiliated with the University of Zagreb, Faculty of Mechanical Engineering and Naval Architecture, first as a research assistant, then assistant professor (2001– 2007), associate professor (2007–2013) and full professor (2013–). She has co-authored 27 papers, 17 of them published in journals. Her research interests are the algorithms for the dense matrix factorizations, eigenvalues and singular values, and efficient parallelization of those algorithms. Sanja Singer rođena je u Zagrebu 27. prosinca 1963., gdje je diplomirala (1986.), magistrirala (1993.) i doktorirala (1997.) na Matematičkom odsjeku Prirodoslovno– matematičkog fakulteta Sveučilišta u Zagrebu. Od prosinca 1987. do rujna 1989. bila je zaposlena kao asistentica pripravnica na Ekonomskom fakultetu Sveučilišta u Zagrebu.
    [Show full text]
  • Arxiv:1907.08697V5 [Math.NA] 23 Mar 2021 Matrices [40, 44] and Fast Slepian Transforms [27]
    Fast approximation of orthogonal matrices and application to PCA Cristian Rusu Faculty of Automatic Control and Computers, University Politehnica Bucharest [email protected] Lorenzo Rosasco LCSL, Universitá di Genova Massachusetts Institute of Technology and Istituto Italiano di Tecnologia Abstract We study the problem of approximating orthogonal matrices so that their appli- cation is numerically fast and yet accurate. We find an approximation by solving an optimization problem over a set of structured matrices, that we call extended orthogonal Givens transformations, including Givens rotations as a special case. We propose an efficient greedy algorithm to solve such a problem and show that it strikes a balance between approximation accuracy and speed of computation. The approach is relevant to spectral methods and we illustrate its application to PCA. 1 Introduction Orthonormal transformations play a key role in most matrix decomposition techniques and spectral methods [4]. As such, manipulating them in an efficient manner is essential to many practical applications. While general matrix-vector multiplications with orthogonal matrices take O(d2) space and time, it is natural to ask whether faster approximate computations (say O(d log d)) can be achieved while retaining enough accuracy. Approximating an orthonormal matrix with just a few building blocks is hard in general. The standard decomposition technique meant to reduce complexity is a low-rank approximation. Unfortunately, for an orthonormal matrix, which is perfectly conditioned, this approach is meaningless. In this work, we are inspired by the fact that several orthonormal/unitary transformations that exhibit low numerical complexity are known. The typical example is the discrete Fourier transform with its efficient implementation as the fast Fourier transform [47] together with other Fourier- related algorithms: fast Walsh-Hadamard transforms [17], fast cosine transforms [33], and fast Hartley transforms [9].
    [Show full text]