Improvement to Hessenberg Reduction
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
2 Homework Solutions 18.335 " Fall 2004
2 Homework Solutions 18.335 - Fall 2004 2.1 Count the number of ‡oating point operations required to compute the QR decomposition of an m-by-n matrix using (a) Householder re‡ectors (b) Givens rotations. 2 (a) See Trefethen p. 74-75. Answer: 2mn2 n3 ‡ops. 3 (b) Following the same procedure as in part (a) we get the same ‘volume’, 1 1 namely mn2 n3: The only di¤erence we have here comes from the 2 6 number of ‡opsrequired for calculating the Givens matrix. This operation requires 6 ‡ops (instead of 4 for the Householder re‡ectors) and hence in total we need 3mn2 n3 ‡ops. 2.2 Trefethen 5.4 Let the SVD of A = UV : Denote with vi the columns of V , ui the columns of U and i the singular values of A: We want to …nd x = (x1; x2) and such that: 0 A x x 1 = 1 A 0 x2 x2 This gives Ax2 = x1 and Ax1 = x2: Multiplying the 1st equation with A 2 and substitution of the 2nd equation gives AAx2 = x2: From this we may conclude that x2 is a left singular vector of A: The same can be done to see that x1 is a right singular vector of A: From this the 2m eigenvectors are found to be: 1 v x = i ; i = 1:::m p2 ui corresponding to the eigenvalues = i. Therefore we get the eigenvalue decomposition: 1 0 A 1 VV 0 1 VV = A 0 p2 U U 0 p2 U U 3 2.3 If A = R + uv, where R is upper triangular matrix and u and v are (column) vectors, describe an algorithm to compute the QR decomposition of A in (n2) time. -
QR Decomposition: History and Its Applications
Mathematics & Statistics Auburn University, Alabama, USA QR History Dec 17, 2010 Asymptotic result QR iteration QR decomposition: History and its EE Applications Home Page Title Page Tin-Yau Tam èèèUUUÎÎÎ JJ II J I Page 1 of 37 Æâ§w Go Back fÆêÆÆÆ Full Screen Close email: [email protected] Website: www.auburn.edu/∼tamtiny Quit 1. QR decomposition Recall the QR decomposition of A ∈ GLn(C): QR A = QR History Asymptotic result QR iteration where Q ∈ GLn(C) is unitary and R ∈ GLn(C) is upper ∆ with positive EE diagonal entries. Such decomposition is unique. Set Home Page a(A) := diag (r11, . , rnn) Title Page where A is written in column form JJ II J I A = (a1| · · · |an) Page 2 of 37 Go Back Geometric interpretation of a(A): Full Screen rii is the distance (w.r.t. 2-norm) between ai and span {a1, . , ai−1}, Close i = 2, . , n. Quit Example: 12 −51 4 6/7 −69/175 −58/175 14 21 −14 6 167 −68 = 3/7 158/175 6/175 0 175 −70 . QR −4 24 −41 −2/7 6/35 −33/35 0 0 35 History Asymptotic result QR iteration EE • QR decomposition is the matrix version of the Gram-Schmidt orthonor- Home Page malization process. Title Page JJ II • QR decomposition can be extended to rectangular matrices, i.e., if A ∈ J I m×n with m ≥ n (tall matrix) and full rank, then C Page 3 of 37 A = QR Go Back Full Screen where Q ∈ Cm×n has orthonormal columns and R ∈ Cn×n is upper ∆ Close with positive “diagonal” entries. -
A Fast Algorithm for the Recursive Calculation of Dominant Singular Subspaces N
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Journal of Computational and Applied Mathematics 218 (2008) 238–246 www.elsevier.com/locate/cam A fast algorithm for the recursive calculation of dominant singular subspaces N. Mastronardia,1, M. Van Barelb,∗,2, R. Vandebrilb,2 aIstituto per le Applicazioni del Calcolo, CNR, via Amendola122/D, 70126, Bari, Italy bDepartment of Computer Science, Katholieke Universiteit Leuven, Celestijnenlaan 200A, 3001 Leuven, Belgium Received 26 September 2006 Abstract In many engineering applications it is required to compute the dominant subspace of a matrix A of dimension m × n, with m?n. Often the matrix A is produced incrementally, so all the columns are not available simultaneously. This problem arises, e.g., in image processing, where each column of the matrix A represents an image of a given sequence leading to a singular value decomposition-based compression [S. Chandrasekaran, B.S. Manjunath, Y.F. Wang, J. Winkeler, H. Zhang, An eigenspace update algorithm for image analysis, Graphical Models and Image Process. 59 (5) (1997) 321–332]. Furthermore, the so-called proper orthogonal decomposition approximation uses the left dominant subspace of a matrix A where a column consists of a time instance of the solution of an evolution equation, e.g., the flow field from a fluid dynamics simulation. Since these flow fields tend to be very large, only a small number can be stored efficiently during the simulation, and therefore an incremental approach is useful [P. Van Dooren, Gramian based model reduction of large-scale dynamical systems, in: Numerical Analysis 1999, Chapman & Hall, CRC Press, London, Boca Raton, FL, 2000, pp. -
Numerical Linear Algebra Revised February 15, 2010 4.1 the LU
Numerical Linear Algebra Revised February 15, 2010 4.1 The LU Decomposition The Elementary Matrices and badgauss In the previous chapters we talked a bit about the solving systems of the form Lx = b and Ux = b where L is lower triangular and U is upper triangular. In the exercises for Chapter 2 you were asked to write a program x=lusolver(L,U,b) which solves LUx = b using forsub and backsub. We now address the problem of representing a matrix A as a product of a lower triangular L and an upper triangular U: Recall our old friend badgauss. function B=badgauss(A) m=size(A,1); B=A; for i=1:m-1 for j=i+1:m a=-B(j,i)/B(i,i); B(j,:)=a*B(i,:)+B(j,:); end end The heart of badgauss is the elementary row operation of type 3: B(j,:)=a*B(i,:)+B(j,:); where a=-B(j,i)/B(i,i); Note also that the index j is greater than i since the loop is for j=i+1:m As we know from linear algebra, an elementary opreration of type 3 can be viewed as matrix multipli- cation EA where E is an elementary matrix of type 3. E looks just like the identity matrix, except that E(j; i) = a where j; i and a are as in the MATLAB code above. In particular, E is a lower triangular matrix, moreover the entries on the diagonal are 1's. We call such a matrix unit lower triangular. -
(Hessenberg) Eigenvalue-Eigenmatrix Relations∗
(HESSENBERG) EIGENVALUE-EIGENMATRIX RELATIONS∗ JENS-PETER M. ZEMKE† Abstract. Explicit relations between eigenvalues, eigenmatrix entries and matrix elements are derived. First, a general, theoretical result based on the Taylor expansion of the adjugate of zI − A on the one hand and explicit knowledge of the Jordan decomposition on the other hand is proven. This result forms the basis for several, more practical and enlightening results tailored to non-derogatory, diagonalizable and normal matrices, respectively. Finally, inherent properties of (upper) Hessenberg, resp. tridiagonal matrix structure are utilized to construct computable relations between eigenvalues, eigenvector components, eigenvalues of principal submatrices and products of lower diagonal elements. Key words. Algebraic eigenvalue problem, eigenvalue-eigenmatrix relations, Jordan normal form, adjugate, principal submatrices, Hessenberg matrices, eigenvector components AMS subject classifications. 15A18 (primary), 15A24, 15A15, 15A57 1. Introduction. Eigenvalues and eigenvectors are defined using the relations Av = vλ and V −1AV = J. (1.1) We speak of a partial eigenvalue problem, when for a given matrix A ∈ Cn×n we seek scalar λ ∈ C and a corresponding nonzero vector v ∈ Cn. The scalar λ is called the eigenvalue and the corresponding vector v is called the eigenvector. We speak of the full or algebraic eigenvalue problem, when for a given matrix A ∈ Cn×n we seek its Jordan normal form J ∈ Cn×n and a corresponding (not necessarily unique) eigenmatrix V ∈ Cn×n. Apart from these constitutional relations, for some classes of structured matrices several more intriguing relations between components of eigenvectors, matrix entries and eigenvalues are known. For example, consider the so-called Jacobi matrices. -
Massively Parallel Poisson and QR Factorization Solvers
Computers Math. Applic. Vol. 31, No. 4/5, pp. 19-26, 1996 Pergamon Copyright~)1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0898-1221/96 $15.00 + 0.00 0898-122 ! (95)00212-X Massively Parallel Poisson and QR Factorization Solvers M. LUCK£ Institute for Control Theory and Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrluck@savba, sk M. VAJTERSIC Institute of Informatics, Slovak Academy of Sciences DdbravskA cesta 9, 840 00 Bratislava, P.O. Box 56, Slovak Republic kaifmava©savba, sk E. VIKTORINOVA Institute for Control Theoryand Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrevka@savba, sk Abstract--The paper brings a massively parallel Poisson solver for rectangle domain and parallel algorithms for computation of QR factorization of a dense matrix A by means of Householder re- flections and Givens rotations. The computer model under consideration is a SIMD mesh-connected toroidal n x n processor array. The Dirichlet problem is replaced by its finite-difference analog on an M x N (M + 1, N are powers of two) grid. The algorithm is composed of parallel fast sine transform and cyclic odd-even reduction blocks and runs in a fully parallel fashion. Its computational complexity is O(MN log L/n2), where L = max(M + 1, N). A parallel proposal of QI~ factorization by the Householder method zeros all subdiagonal elements in each column and updates all elements of the given submatrix in parallel. For the second method with Givens rotations, the parallel scheme of the Sameh and Kuck was chosen where the disjoint rotations can be computed simultaneously. -
Etna the Block Hessenberg Process for Matrix
Electronic Transactions on Numerical Analysis. Volume 46, pp. 460–473, 2017. ETNA Kent State University and c Copyright 2017, Kent State University. Johann Radon Institute (RICAM) ISSN 1068–9613. THE BLOCK HESSENBERG PROCESS FOR MATRIX EQUATIONS∗ M. ADDAMy, M. HEYOUNIy, AND H. SADOKz Abstract. In the present paper, we first introduce a block variant of the Hessenberg process and discuss its properties. Then, we show how to apply the block Hessenberg process in order to solve linear systems with multiple right-hand sides. More precisely, we define the block CMRH method for solving linear systems that share the same coefficient matrix. We also show how to apply this process for solving discrete Sylvester matrix equations. Finally, numerical comparisons are provided in order to compare the proposed new algorithms with other existing methods. Key words. Block Krylov subspace methods, Hessenberg process, Arnoldi process, CMRH, GMRES, low-rank matrix equations. AMS subject classifications. 65F10, 65F30 1. Introduction. In this work, we are first interested in solving s systems of linear equations with the same coefficient matrix and different right-hand sides of the form (1.1) A x(i) = y(i); 1 ≤ i ≤ s; where A is a large and sparse n × n real matrix, y(i) is a real column vector of length n, and s n. Such linear systems arise in numerous applications in computational science and engineering such as wave propagation phenomena, quantum chromodynamics, and dynamics of structures [5, 9, 36, 39]. When n is small, it is well known that the solution of (1.1) can be computed by a direct method such as LU or Cholesky factorization. -
Computing the Jordan Structure of an Eigenvalue∗
SIAM J. MATRIX ANAL.APPL. c 2017 Society for Industrial and Applied Mathematics Vol. 38, No. 3, pp. 949{966 COMPUTING THE JORDAN STRUCTURE OF AN EIGENVALUE∗ NICOLA MASTRONARDIy AND PAUL VAN DOORENz Abstract. In this paper we revisit the problem of finding an orthogonal similarity transformation that puts an n × n matrix A in a block upper-triangular form that reveals its Jordan structure at a particular eigenvalue λ0. The obtained form in fact reveals the dimensions of the null spaces of i (A − λ0I) at that eigenvalue via the sizes of the leading diagonal blocks, and from this the Jordan structure at λ0 is then easily recovered. The method starts from a Hessenberg form that already reveals several properties of the Jordan structure of A. It then updates the Hessenberg form in an efficient way to transform it to a block-triangular form in O(mn2) floating point operations, where m is the total multiplicity of the eigenvalue. The method only uses orthogonal transformations and is backward stable. We illustrate the method with a number of numerical examples. Key words. Jordan structure, staircase form, Hessenberg form AMS subject classifications. 65F15, 65F25 DOI. 10.1137/16M1083098 1. Introduction. Finding the eigenvalues and their corresponding Jordan struc- ture of a matrix A is one of the most studied problems in numerical linear algebra. This structure plays an important role in the solution of explicit differential equations, which can be modeled as n×n (1.1) λx(t) = Ax(t); x(0) = x0;A 2 R ; where λ stands for the differential operator. -
11.5 Reduction of a General Matrix to Hessenberg Form
482 Chapter 11. Eigensystems is equivalent to the 2n 2n real problem × A B u u − =λ (11.4.2) BA·v v T T Note that the 2n 2n matrix in (11.4.2) is symmetric: A = A and B = B visit website http://www.nr.com or call 1-800-872-7423 (North America only),or send email to [email protected] (outside North America). readable files (including this one) to any servercomputer, is strictly prohibited. To order Numerical Recipes books,diskettes, or CDROMs Permission is granted for internet users to make one paper copy their own personal use. Further reproduction, or any copying of machine- Copyright (C) 1988-1992 by Cambridge University Press.Programs Numerical Recipes Software. Sample page from NUMERICAL RECIPES IN C: THE ART OF SCIENTIFIC COMPUTING (ISBN 0-521-43108-5) if C is Hermitian.× − Corresponding to a given eigenvalue λ, the vector v (11.4.3) −u is also an eigenvector, as you can verify by writing out the two matrix equa- tions implied by (11.4.2). Thus if λ1,λ2,...,λn are the eigenvalues of C, then the 2n eigenvalues of the augmented problem (11.4.2) are λ1,λ1,λ2,λ2,..., λn,λn; each, in other words, is repeated twice. The eigenvectors are pairs of the form u + iv and i(u + iv); that is, they are the same up to an inessential phase. Thus we solve the augmented problem (11.4.2), and choose one eigenvalue and eigenvector from each pair. These give the eigenvalues and eigenvectors of the original matrix C. -
The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals
The Sparse Matrix Transform for Covariance Estimation and Analysis of High Dimensional Signals Guangzhi Cao*, Member, IEEE, Leonardo R. Bachega, and Charles A. Bouman, Fellow, IEEE Abstract Covariance estimation for high dimensional signals is a classically difficult problem in statistical signal analysis and machine learning. In this paper, we propose a maximum likelihood (ML) approach to covariance estimation, which employs a novel non-linear sparsity constraint. More specifically, the covariance is constrained to have an eigen decomposition which can be represented as a sparse matrix transform (SMT). The SMT is formed by a product of pairwise coordinate rotations known as Givens rotations. Using this framework, the covariance can be efficiently estimated using greedy optimization of the log-likelihood function, and the number of Givens rotations can be efficiently computed using a cross- validation procedure. The resulting estimator is generally positive definite and well-conditioned, even when the sample size is limited. Experiments on a combination of simulated data, standard hyperspectral data, and face image sets show that the SMT-based covariance estimates are consistently more accurate than both traditional shrinkage estimates and recently proposed graphical lasso estimates for a variety of different classes and sample sizes. An important property of the new covariance estimate is that it naturally yields a fast implementation of the estimated eigen-transformation using the SMT representation. In fact, the SMT can be viewed as a generalization of the classical fast Fourier transform (FFT) in that it uses “butterflies” to represent an orthonormal transform. However, unlike the FFT, the SMT can be used for fast eigen-signal analysis of general non-stationary signals. -
Generalized Hessenberg Matricesୋ Miroslav Fiedler a ,∗,Zdenekˇ Vavrínˇ B Aacademy of Sciences of the Czech Republic, Institute of Computer Science, Pod Vodáren
View metadata, citation and similar papers at core.ac.uk brought to you by CORE provided by Elsevier - Publisher Connector Linear Algebra and its Applications 380 (2004) 95–105 www.elsevier.com/locate/laa Generalized Hessenberg matricesୋ Miroslav Fiedler a ,∗,Zdenekˇ Vavrínˇ b aAcademy of Sciences of the Czech Republic, Institute of Computer Science, Pod vodáren. vˇeží 2, 182 07 Praha 8, Czech Republic bAcademy of Sciences of the Czech Republic, Mathematics Institute, Žitná 25, 115 67 Praha 1, Czech Republic Received 10 December 2001; accepted 31 January 2003 Submitted by R. Nabben Abstract We define and study generalized Hessenberg matrices, i.e. square matrices which have subdiagonal rank one. Here, subdiagonal rank means the maximum order of a nonsingular submatrix all of whose entries are in the subdiagonal part. We prove that the property of being generalized Hessenberg matrix is preserved by post- and premultiplication by a nonsingular upper triangular matrix, by inversion (for invertible matrices), etc. We also study a special kind of generalized Hessenberg matrices. © 2003 Elsevier Inc. All rights reserved. AMS classification: 15A23; 15A33 Keywords: Hessenberg matrix; Structure rank; Subdiagonal rank 1. Introduction As is well known, Hessenberg matrices are square matrices which have zero en- tries in the lower triangular part below the second diagonal. Formally, for a matrix A = (aik), aik = 0 whenever i>k+ 1. These matrices occur e.g. in numerical linear algebra as a result of an orthogo- nal transformation of a matrix using Givens or Householder transformations when solving the eigenvalue problem for a general matrix. ୋ Research supported by grant A1030003. -
Matrix Theory
Matrix Theory Xingzhi Zhan +VEHYEXI7XYHMIW MR1EXLIQEXMGW :SPYQI %QIVMGER1EXLIQEXMGEP7SGMIX] Matrix Theory https://doi.org/10.1090//gsm/147 Matrix Theory Xingzhi Zhan Graduate Studies in Mathematics Volume 147 American Mathematical Society Providence, Rhode Island EDITORIAL COMMITTEE David Cox (Chair) Daniel S. Freed Rafe Mazzeo Gigliola Staffilani 2010 Mathematics Subject Classification. Primary 15-01, 15A18, 15A21, 15A60, 15A83, 15A99, 15B35, 05B20, 47A63. For additional information and updates on this book, visit www.ams.org/bookpages/gsm-147 Library of Congress Cataloging-in-Publication Data Zhan, Xingzhi, 1965– Matrix theory / Xingzhi Zhan. pages cm — (Graduate studies in mathematics ; volume 147) Includes bibliographical references and index. ISBN 978-0-8218-9491-0 (alk. paper) 1. Matrices. 2. Algebras, Linear. I. Title. QA188.Z43 2013 512.9434—dc23 2013001353 Copying and reprinting. Individual readers of this publication, and nonprofit libraries acting for them, are permitted to make fair use of the material, such as to copy a chapter for use in teaching or research. Permission is granted to quote brief passages from this publication in reviews, provided the customary acknowledgment of the source is given. Republication, systematic copying, or multiple reproduction of any material in this publication is permitted only under license from the American Mathematical Society. Requests for such permission should be addressed to the Acquisitions Department, American Mathematical Society, 201 Charles Street, Providence, Rhode Island 02904-2294 USA. Requests can also be made by e-mail to [email protected]. c 2013 by the American Mathematical Society. All rights reserved. The American Mathematical Society retains all rights except those granted to the United States Government.