DOCSLIB.ORG

Matrices with Hierarchical Low-Rank Structures

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Matrices with Hierarchical Low-Rank Structures Matrices with hierarchical low-rank structures Jonas Ballani and Daniel Kressner ANCHP, EPF Lausanne, Station 8, CH-1015 Lausanne, Switzerland e-mail: [email protected], [email protected] The first author has been supported by an EPFL fellowship through the European Union’s Seventh Framework Programme under grant agreement no. 291771. Summary. Matrices with low-rank off-diagonal blocks are a versatile tool to perform matrix compression and to speed up various matrix operations, such as the solution of linear systems. Often, the underlying block partitioning is described by a hierarchical partitioning of the row and column indices, thus giving rise to hierarchical low-rank structures. The goal of this chap- ter is to provide a brief introduction to these techniques, with an emphasis on linear algebra aspects. 1 Introduction 1.1 Sparsity versus data-sparsity An n × n matrix A is called sparse if nnz(A), the number of its nonzero entries, is much smaller than n2. Sparsity reduces the storage requirements to O(nnz(A)) and, likewise, the complexity of matrix-vector multiplications. Unfortunately, sparsity is fragile; it is easily de- stroyed by operations with the matrix. For example, the factors of an LU factorization of A usually have (many) more nonzeros than A itself. This so called fill-in can be reduced, sometimes significantly, by reordering the matrix prior to the factorization [25]. Based on existing numerical evidence, it is probably safe to say that such sparse factorizations consti- tute the methods of choice for (low-order) finite element or finite difference discretizations of two-dimensional partial differential equations (PDEs). They are less effective for three- dimensional PDEs. More severely, sparse factorizations are of limited use when the inverse of A is explicitly needed, as for example in inverse covariance matrix estimation and matrix iter- ations for computing matrix functions. For nearly all sparsity patterns of practical relevance, A−1 is a completely dense matrix. In certain situations, this issue can be addressed effectively by considering approximate sparsity: A−1 is replaced by a sparse matrix M such that kA−1 − Mk ≤ tol for some prescribed tolerance tol and an appropriate matrix norm k · k. To illustrate the concepts mentioned above, let us consider the tridiagonal matrix 2 Jonas Ballani and Daniel Kressner 0 2 −1 1 B .. C 2 B−1 2 . C 2 A = (n + 1) B C + s(n + 1) In; (1) B . C @ .. .. −1A −1 2 −1 where the shift s > 0 controls the condition number k(A) = kAk2kA k2. The intensity plots in Figure 1 reveal the magnitude of the entries of A−1; entries of magnitude below 10−15 are white. It can be concluded that A−1 can be very well approximated by a banded matrix if its (a) s = 4;k(A) ≈ 2 (b) s = 1;k(A) ≈ 5 (c) s = 0;k(A) ≈ 103 Fig. 1. Approximate sparsity of A−1 for the matrix A from (1) with n = 50 and different values of s. condition number is small. This is an accordance with a classical result [26, Proposition 2.1], which states for any symmetric positive definite tridiagonal matrix A that p 2ji− jj −1 k − 1 −1 −1 p 2 [A ]i j ≤ C p ; C = max l ;(2lmax) (1 + k(A)) ; (2) k + 1 min where lmin;lmax denote the smallest/largest eigenvalues of A; see also [15] for recent exten- sions. This implies that the entries of A−1 decay exponentially away from the diagonal, but this decay may deteriorate as k(A) increases. Although the bound (2) can be pessimistic, it correctly predicts that approximate sparsity is of limited use for ill-conditioned matrices as they arise from the discretization of (one-dimensional) PDEs. An n × n matrix is called data-sparse if it can be represented by much less than n2 pa- rameters with respect to a certain format. Data-sparsity is much more general than sparsity. For example, any matrix of rank r n can be represented with 2nr (or even less) parameters by storing its low-rank factors. The inverse of the tridiagonal matrix (1) is obviously not a low-rank matrix, simply because it is invertible. However, any of its off-diagonal blocks has rank 1. One of the many ways to see is to use the Sherman-Morrison formula. Consider the following decomposition of a general symmetric positive definite tridiagonal matrix: T A11 0 en1 en1 n1×n1 n2×n2 A = − an1;n1+1 ; A11 2 R ; A22 2 R ; 0 A22 −e1 −e1 where e j denotes a jth unit vector of appropriate length. Then −1 a T −1 A11 0 n1;n1+1 w1 w1 A = −1 + ; (3) 0 A 1 + eT A−1e + eT A−1e w2 w2 22 n1 11 n1 1 22 1 Matrices with hierarchical low-rank structures 3 −1 −1 −1 where w1 = A11 en1 and w2 = −A22 e1. Hence, the off-diagonal blocks of A have rank at most 1. Assuming that n is an integer multiple of 4, let us partition 0 1 0 1 A(2) A(2) B(2) B(2) 11 12 A 11 12 B B (2) (2) 12 C B (2) (2) 12 C B A A C − B B B C A = B 21 22 C; A 1 = B 21 22 C; (4) B A(2) A(2) C B B(2) B(2) C @ 33 34 A @ 33 34 A A21 (2) (2) B34 (2) (2) A43 A44 B43 B44 (2) (2) n=4×n=4 such that Ai j ;Bi j 2 R . By storing only the factors of the rank-1 off-diagonal blocks −1 (2) of A and exploiting symmetry, it follows that – additionally to the 4 diagonal blocks B j j of size n=4 × n=4 – only 2n=2 + 4n=4 = 2n entries are needed to represent A−1. Assuming that n = 2k and partitioning recursively, it follows that the whole matrix A−1 can be stored with k k 2n=2 + 4n=4 + ··· + 2 n=2 + n = n(log2 n + 1) (5) parameters. The logarithmic factor in (5) can actually be removed by exploiting the nestedness of the (2) n=4×2 low-rank factors. To see this, we again consider the partitioning (4) and let Uj 2 R , j = 1;:::;4, be orthonormal bases such that n (2) −1 (2) −1 o (2) span (A j j ) e1;(A j j ) en=4 ⊆ range(Uj ): (2) (2)! A11 A12 Applying the Sherman-Morrison formula (3) to A11 = (2) (2) shows A21 A22 (2) ! (2) ! −1 U1 0 −1 U1 0 A11 e1 2 range (2) ; A11 en=2 2 range (2) : 0 U2 0 U2 Similarly, (2) ! (2) ! −1 U3 0 −1 U3 0 A22 e1 2 range (2) ; A22 en=2 2 range (2) : 0 U4 0 U4 n=2×2 If we let Uj 2 R , j = 1;2, be orthonormal basis such that n −1 −1 o span A j j e1;A j j en=2 ⊆ range(Uj); 4×2 then there exists matrices Xj 2 R such that (2) ! U2 j−1 0 Uj = (2) Xj: (6) 0 U2 j n=2×2 (2) Hence, there is no need to store the bases U1;U2 2 R explicitly; the availability of Uj −1 and the small matrices X1;X2 suffices. In summary, we can represent A as 0 1 B(2) U(2)S(2)(U(2))T 11 1 12 2 U S UT B (2) (2) T (2) T (2) 1 12 2 C B U (S ) (U ) B C B 2 12 1 22 C (7) B B(2) U(2)S(2)(U(2))T C @ U ST UT 33 3 34 4 A 2 12 1 (2) (2) T (2) T (2) U4 (S34 ) (U3 ) B44 4 Jonas Ballani and Daniel Kressner (2) 2×2 for some matrices S12;Si j 2 R . The number of entries needed for representing the off- diagonal blocks A−1 is thus 4 × 2n=4+2 × 8+ (2 + 1) × 4 : | {z } |{z} | {z } (2) for Xj (2) (2) for Uj for S12;S12 ;S34 Assuming that n = 2k and recursively repeating the described procedure k − 1 times, we find that at most O(n) total storage is needed for representing A−1. Thus we have removed the logarithmic factor from (5), at the expense of a larger constant due to the fact that we work with rank 2 instead of rank 1. Remark 1. The format (7) together with (6) is a special case of the HSS matrices that will be discussed in Section 4.1. The related but different concepts of semi-separable matrices [65] and sequentially semi-separable matrices [21] allow to represent the inverse of a tridiago- nal matrix A with nested rank-1 factors and consequently reduce the required storage to two vectors of length n. For a number of reasons, the situation discussed above is simplistic and not fully rep- resentative for the type of applications that can be addressed with the hierarchical low-rank structures covered in this chapter. First, sparsity of the original matrix can be beneficial but it is not needed in the context of low-rank techniques. Second, exact low rank is a rare phenomenon and we will nearly always work with low-rank approximations instead. Third, the recursive partitioning (4) is closely tied to banded matrices and related structures, which arise, for ex- ample, from the discretization of one-dimensional PDEs. More general partitionings need to be used when dealing with structures that arise from the discretization of higher-dimensional PDEs.
Load more

Recommended publications
  • Extremely Accurate Solutions Using Block Decomposition and Extended Precision for Solving Very Ill-Conditioned Equations
    Extremely accurate solutions using block decomposition and extended precision for solving very ill-conditioned equations †Kansa, E.J.1, Skala V.2, and Holoborodko, P.3 1Convergent Solutions, Livermore, CA 94550 USA 2Computer Science & Engineering Dept., Faculty of Applied Sciences, University of West Bohemia, University 8, CZ 301 00 Plzen, Czech Republic 3Advanpix LLC, Maison Takashima Bldg. 2F, Daimachi 10-15-201, Yokohama, Japan 221-0834 †Corresponding author: [email protected] Abstract Many authors have complained about the ill-conditioning associated the numerical solution of partial differential equations (PDEs) and integral equations (IEs) using as the continuously differentiable Gaussian and multiquadric continuously differentiable (C ) radial basis functions (RBFs). Unlike finite elements, finite difference, or finite volume methods that lave compact local support that give rise to sparse equations, the C -RBFs with simple collocation methods give rise to full, asymmetric systems of equations. Since C RBFs have adjustable constent or variable shape parameters, the resulting systems of equations that are solve on single or double precision computers can suffer from “ill-conditioning”. Condition numbers can be either the absolute or relative condition number, but in the context of linear equations, the absolute condition number will be understood. Results will be presented that demonstrates the combination of Block Gaussian elimination, arbitrary arithmetic precision, and iterative refinement can give remarkably accurate numerical salutations to large Hilbert and van der Monde equation systems. 1. Introduction An accurate definition of the condition number, , of the matrix, A, is the ratio of the largest to smallest absolute value of the singular values, { i}, obtained from the singular value decomposition (SVD) method, see [1]: (A) = maxjjminjj (1) This definition of condition number will be used in this study.
    [Show full text]
  • A Study of Convergence of the PMARC Matrices Applicable to WICS Calculations
    A Study of Convergence of the PMARC Matrices applicable to WICS Calculations /- by Amitabha Ghosh Department of Mechanical Engineering Rochester Institute of Technology Rochester, NY 14623 Final Report NASA Cooperative Agreement No: NCC 2-937 Presented to NASA Ames Research Center Moffett Field, CA 94035 August 31, 1997 Table of Contents Abstract ............................................ 3 Introduction .......................................... 3 Solution of Linear Systems ................................... 4 Direct Solvers .......................................... 5 Gaussian Elimination: .................................. 5 Gauss-Jordan Elimination: ................................ 5 L-U Decompostion: ................................... 6 Iterative Solvers ........................................ 6 Jacobi Method: ..................................... 7 Gauss-Seidel Method: .............................. .... 7 Successive Over-relaxation Method: .......................... 8 Conjugate Gradient Method: .............................. 9 Recent Developments: ......................... ....... 9 Computational Efficiency .................................... 10 Graphical Interpretation of Residual Correction Schemes .................... 11 Defective Matrices ............................. , ......... 15 Results and Discussion ............ ......................... 16 Concluding Remarks ...................................... 19 Acknowledgements ....................................... 19 References ..........................................
    [Show full text]
  • Applied Linear Algebra
    APPLIED LINEAR ALGEBRA Giorgio Picci November 24, 2015 1 Contents 1 LINEAR VECTOR SPACES AND LINEAR MAPS 10 1.1 Linear Maps and Matrices . 11 1.2 Inverse of a Linear Map . 12 1.3 Inner products and norms . 13 1.4 Inner products in coordinate spaces (1) . 14 1.5 Inner products in coordinate spaces (2) . 15 1.6 Adjoints. 16 1.7 Subspaces . 18 1.8 Image and kernel of a linear map . 19 1.9 Invariant subspaces in Rn ........................... 22 1.10 Invariant subspaces and block-diagonalization . 23 2 1.11 Eigenvalues and Eigenvectors . 24 2 SYMMETRIC MATRICES 25 2.1 Generalizations: Normal, Hermitian and Unitary matrices . 26 2.2 Change of Basis . 27 2.3 Similarity . 29 2.4 Similarity again . 30 2.5 Problems . 31 2.6 Skew-Hermitian matrices (1) . 32 2.7 Skew-Symmetric matrices (2) . 33 2.8 Square roots of positive semidefinite matrices . 36 2.9 Projections in Rn ............................... 38 2.10 Projections on general inner product spaces . 40 3 2.11 Gramians. 41 2.12 Example: Polynomial vector spaces . 42 3 LINEAR LEAST SQUARES PROBLEMS 43 3.1 Weighted Least Squares . 44 3.2 Solution by the Orthogonality Principle . 46 3.3 Matrix least-Squares Problems . 48 3.4 A problem from subspace identification . 50 3.5 Relation with Left- and Right- Inverses . 51 3.6 The Pseudoinverse . 54 3.7 The Euclidean pseudoinverse . 63 3.8 The Pseudoinverse and Orthogonal Projections . 64 3.9 Linear equations . 66 4 3.10 Unfeasible linear equations and Least Squares . 68 3.11 The Singular value decomposition (SVD) .
    [Show full text]
  • Matrix Computations
    Chapter 5 Matrix Computations §5.1 Setting Up Matrix Problems §5.2 Matrix Operations §5.3 Once Again, Setting Up Matrix Problems §5.4 Recursive Matrix Operations §5.5 Distributed Memory Matrix Multiplication The next item on our agenda is the linear equation problem Ax = b. However, before we get into algorithmic details, it is important to study two simpler calculations: matrix-vector mul- tiplication and matrix-matrix multiplication. Both operations are supported in Matlab so in that sense there is “nothing to do.” However, there is much to learn by studying how these com- putations can be implemented. Matrix-vector multiplications arise during the course of solving Ax = b problems. Moreover, it is good to uplift our ability to think at the matrix-vector level before embarking on a presentation of Ax = b solvers. The act of setting up a matrix problem also deserves attention. Often, the amount of work that is required to initialize an n-by-n matrix is as much as the work required to solve for x. We pay particular attention to the common setting when each matrix entry aij is an evaluation of a continuous function f(x, y). A theme throughout this chapter is the exploitation of structure. It is frequently the case that there is a pattern among the entries in A which can be used to reduce the amount of work. The fast Fourier transform and the fast Strassen matrix multiply algorithm are presented as examples of recursion in the matrix computations. The organization of matrix-matrix multiplication on a ring of processors is also studied and gives us a nice snapshot of what algorithm development is like in a distributed memory environment.
    [Show full text]
  • Solving a Low-Rank Factorization Model for Matrix Completion by a Nonlinear Successive Over-Relaxation Algorithm
    SOLVING A LOW-RANK FACTORIZATION MODEL FOR MATRIX COMPLETION BY A NONLINEAR SUCCESSIVE OVER-RELAXATION ALGORITHM ZAIWEN WEN †, WOTAO YIN ‡, AND YIN ZHANG § Abstract. The matrix completion problem is to recover a low-rank matrix from a subset of its entries. The main solution strategy for this problem has been based on nuclear-norm minimization which requires computing singular value decompositions – a task that is increasingly costly as matrix sizes and ranks increase. To improve the capacity of solving large-scale problems, we propose a low-rank factorization model and construct a nonlinear successive over-relaxation (SOR) algorithm that only requires solving a linear least squares problem per iteration. Convergence of this nonlinear SOR algorithm is analyzed. Numerical results show that the algorithm can reliably solve a wide range of problems at a speed at least several times faster than many nuclear-norm minimization algorithms. Key words. Matrix Completion, alternating minimization, nonlinear GS method, nonlinear SOR method AMS subject classifications. 65K05, 90C06, 93C41, 68Q32 1. Introduction. The problem of minimizing the rank of a matrix arises in many applications, for example, control and systems theory, model reduction and minimum order control synthesis [20], recovering shape and motion from image streams [25, 32], data mining and pattern recognitions [6] and machine learning such as latent semantic indexing, collaborative prediction and low-dimensional embedding. In this paper, we consider the Matrix Completion (MC) problem of finding a lowest-rank matrix given a subset of its entries, that is, (1.1) min rank(W ), s.t. Wij = Mij , ∀(i,j) ∈ Ω, W ∈Rm×n where rank(W ) denotes the rank of W , and Mi,j ∈ R are given for (i,j) ∈ Ω ⊂ {(i,j) : 1 ≤ i ≤ m, 1 ≤ j ≤ n}.
    [Show full text]
  • Low-Rank Incremental Methods for Computing Dominant Singular Subspaces
    Low-Rank Incremental Methods for Computing Dominant Singular Subspaces C. G. Baker∗ K. A. Gallivan∗ P. Van Dooren† Abstract This paper describes a generic low-rank incremental method for computing the dominant singular triplets of a matrix via a single pass through the matrix. This work unifies several efforts previously described in the literature. We tie the operation of the proposed method to a particular optimization- based eigensolver. This allows the description of novel methods exploiting multiple passes through the matrix. 1 Introduction Given a matrix A Rm×n,m n, the Singular Value Decomposition (SVD) of A is: ∈ ≥ Σ A = U V T , 0 where U and V are m m and n n orthogonal matrices, respectively, and Σ is a diagonal matrix whose × × elements σ1,...,σn are real, non-negative and non-increasing. The σi are the singular values of A, and the first n columns of U and V are the left and right singular vectors of A, respectively. Given k n, the singular vectors associated with the largest k singular values of A are the rank-k dominant singular≤ vectors. The subspaces associated with these vectors are the rank-k dominant singular subspaces. The components of the SVD are optimal in many respects [1], and these properties result in the occurrence of the SVD in numerous analyses, e.g., principal component analysis, proper orthogonal decomposition, the Karhunen-Loeve transform. These applications of the SVD are frequently used in problems related to, e.g., signal processing, model reduction of dynamical systems, image and face recognition, and information retrieval.
    [Show full text]
  • Block Matrices in Linear Algebra
    PRIMUS Problems, Resources, and Issues in Mathematics Undergraduate Studies ISSN: 1051-1970 (Print) 1935-4053 (Online) Journal homepage: https://www.tandfonline.com/loi/upri20 Block Matrices in Linear Algebra Stephan Ramon Garcia & Roger A. Horn To cite this article: Stephan Ramon Garcia & Roger A. Horn (2020) Block Matrices in Linear Algebra, PRIMUS, 30:3, 285-306, DOI: 10.1080/10511970.2019.1567214 To link to this article: https://doi.org/10.1080/10511970.2019.1567214 Accepted author version posted online: 05 Feb 2019. Published online: 13 May 2019. Submit your article to this journal Article views: 86 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=upri20 PRIMUS, 30(3): 285–306, 2020 Copyright # Taylor & Francis Group, LLC ISSN: 1051-1970 print / 1935-4053 online DOI: 10.1080/10511970.2019.1567214 Block Matrices in Linear Algebra Stephan Ramon Garcia and Roger A. Horn Abstract: Linear algebra is best done with block matrices. As evidence in sup- port of this thesis, we present numerous examples suitable for classroom presentation. Keywords: Matrix, matrix multiplication, block matrix, Kronecker product, rank, eigenvalues 1. INTRODUCTION This paper is addressed to instructors of a first course in linear algebra, who need not be specialists in the field. We aim to convince the reader that linear algebra is best done with block matrices. In particular, flexible thinking about the process of matrix multiplication can reveal concise proofs of important theorems and expose new results. Viewing linear algebra from a block-matrix perspective gives an instructor access to use- ful techniques, exercises, and examples.
    [Show full text]
  • Contributions of Issai Schur to Analysis
    Contributions of Issai Schur to Analysis Harry Dym and Victor Katsnelson∗ November 1, 2018 The name Schur is associated with many terms and concepts that are widely used in a number of diverse fields of mathematics and engineering. This survey article focuses on Schur’s work in analysis. Here too, Schur’s name is commonplace: The Schur test and Schur-Hadamard multipliers (in the study of estimates for Hermitian forms), Schur convexity, Schur complements, Schur’s results in summation theory for sequences (in particular, the fundamental Kojima-Schur theorem), the Schur-Cohn test, the Schur al- gorithm, Schur parameters and the Schur interpolation problem for functions that are holomorphic and bounded by one in the unit disk. In this survey, we shall discuss all of the above mentioned topics and then some, as well as some of the generalizations that they inspired. There are nine sections of text, each of which is devoted to a separate theme based on Schur’s work. Each of these sections has an independent bibliography. There is very little overlap. A tenth section presents a list of the papers of Schur that focus on topics that are commonly considered to be analysis. We shall begin with a review of Schur’s less familiar papers on the theory of commuting differential operators. Acknowledgement: The authors extend their thanks to Bernd Kirstein for carefully reading the manuscript and spotting a number of misprints. 1 . Permutable differential operators and fractional powers of differential operators. Let arXiv:0706.1868v1 [math.CA] 13 Jun 2007 dny dn−1y P (y)= p (x) + p (x) + + p (x)y (1.1) n dxn n−1 dxn−1 ··· 0 and dmy dm−1y Q(y)= q (x) + q (x) + + q (x)y, (1.2) m dxm m−1 dxm−1 ··· 0 ∗HD thanks Renee and Jay Weiss and VK thanks Ruth and Sylvia Shogam for endowing the chairs that support their respective research.
    [Show full text]
  • Package 'Matrixcalc'
    Package ‘matrixcalc’ July 28, 2021 Version 1.0-5 Date 2021-07-27 Title Collection of Functions for Matrix Calculations Author Frederick Novomestky <[email protected]> Maintainer S. Thomas Kelly <[email protected]> Depends R (>= 2.0.1) Description A collection of functions to support matrix calculations for probability, econometric and numerical analysis. There are additional functions that are comparable to APL functions which are useful for actuarial models such as pension mathematics. This package is used for teaching and research purposes at the Department of Finance and Risk Engineering, New York University, Polytechnic Institute, Brooklyn, NY 11201. Horn, R.A. (1990) Matrix Analysis. ISBN 978-0521386326. Lancaster, P. (1969) Theory of Matrices. ISBN 978-0124355507. Lay, D.C. (1995) Linear Algebra: And Its Applications. ISBN 978-0201845563. License GPL (>= 2) Repository CRAN Date/Publication 2021-07-28 08:00:02 UTC NeedsCompilation no R topics documented: commutation.matrix . .3 creation.matrix . .4 D.matrix . .5 direct.prod . .6 direct.sum . .7 duplication.matrix . .8 E.matrices . .9 elimination.matrix . 10 entrywise.norm . 11 1 2 R topics documented: fibonacci.matrix . 12 frobenius.matrix . 13 frobenius.norm . 14 frobenius.prod . 15 H.matrices . 17 hadamard.prod . 18 hankel.matrix . 19 hilbert.matrix . 20 hilbert.schmidt.norm . 21 inf.norm . 22 is.diagonal.matrix . 23 is.idempotent.matrix . 24 is.indefinite . 25 is.negative.definite . 26 is.negative.semi.definite . 28 is.non.singular.matrix . 29 is.positive.definite . 31 is.positive.semi.definite . 32 is.singular.matrix . 34 is.skew.symmetric.matrix . 35 is.square.matrix . 36 is.symmetric.matrix .
    [Show full text]
  • Math 448/548 Cpt S 430/530 Assignment 2 Due February 9 1. Let
    Math 448/548 Cpt S 430/530 Assignment 2 due February 9 1. Let A be a real n × n matrix. n (a) Let x 2 R and y = Ax. Show that for i = 1, . , n, yi (the ith component of y) satisfies n ! X jyi j ≤ jaij j kxk1: j=1 (b) Deduce that n ! X ky k ≤ max jaij j kxk 1 i 1 j=1 and n kAxk1 X max ≤ max jaij j: (1) x6=0 kxk i 1 j=1 (c) Show that equality holds in (1) by exhibiting an x for which n kAxk1 X = max jaij j: kxk i 1 j=1 (Hint: Choose an x whose entries are all ±1, with the signs care- fully chosen.) n ! X (d) Deduce that kAk = max jaij j 1 i j=1 2. Let 2 1 2 3 3 A = 4 4 5 6 5 : 7 8 9 (a) Prove by hand computation that A is singular. For example, you can show that the rows of A are linearly dependent. (b) Demonstrate that MATLAB, which uses floating-point arithmetic, does not notice that A is singular (although it does suspect that something is wrong). Use MATLAB commands like inv (inverse) and cond (condition number). 3. The Hilbert matrices are the most famous family of ill-conditioned ma- trices. The (i; j) entry of the n×n Hilbert matrix is hij = 1=(i+j −1). Using MATLAB, do the following for n = 4, 8, and 12. Generate the n×n Hilbert matrix. (MATLAB has a convenient command hilb that generates a Hilbert matrix.) Let z denote the vector of all ones, and let b = Az, where A is the Hilbert matrix.
    [Show full text]
  • Math 240 Summer 2019 Notes
    Math 240 Summer 2019 Notes Yao-Rui Lecture notes might contain typos. Contents 1 Lecture 1 { Introduction 4 1.1 Brief Overview of the Course . 4 1.2 Separable ODEs . 6 1.3 First Order Linear ODEs . 7 1.4 Bernoulli ODEs . 8 1.5 Ricatti ODEs . 8 1.6 Reduction of Order . 9 2 Lecture 2 { Matrices 10 2.1 Row Echelon Form . 10 2.2 Determinants and Invertible Matrices . 11 2.3 More Determinant Facts . 13 3 Lecture 3 { Linear Maps 15 3.1 Basis . 15 3.2 Matrix Representation of Linear Maps . 15 3.3 Change of Basis . 17 3.4 Subspaces and the Rank-Nullity Theorem . 18 3.5 An Overview of Abstract Vector Spaces . 19 4 Lecture 4 { Eigenvalues and Eigenvectors 21 4.1 Eigenvalues and Eigenvectors . 21 4.2 The Cayley-Hamilton Theorem . 24 5 Lecture 5 { Matrix Exponentials 26 5.1 Definition and Example Computations . 26 5.2 Analytic Function on Matrices . 28 5.3 Nilpotent Matrices and Exponentiation . 29 6 Lecture 6 { First Order Constant Linear ODEs Part 1 30 6.1 The General Solution . 30 6.2 Examples on Diagonalizable Matrices . 30 6.3 Duhamel's Formula . 31 7 Lecture 7 { First Order Constant Linear ODEs Part 2 33 7.1 Generalized Eigenvectors . 33 7.2 The Jordan Canonical Form for Nilpotent Matrices . 35 7.3 The Jordan Canonical Form in General . 36 1 8 Lecture 8 { First Order Constant Linear ODEs Part 3 42 8.1 An Application of Invariant Subspaces . 42 8.2 Buchheim's Algorithm and Some Formulas for Exponentiating Matrices .
    [Show full text]
  • 1. Matrix Rank
    L. Vandenberghe ECE133B (Spring 2020) 1. Matrix rank • subspaces, dimension, rank • QR factorization with pivoting • properties of matrix rank • low-rank matrices • pseudo-inverse 1.1 Subspace a nonempty subset V of Rm is a subspace if αx + βy 2 V for all vectors x; y 2 V and scalars α; β • all linear combinations of elements of V are in V • V is nonempty and closed under scalar multiplication and vector addition Examples • f0g, Rm • the span of a set S ⊆ Rm: all linear combinations of elements of S span¹Sº = fβ1a1 + ··· + βkak j a1;:::; ak 2 S; β1; : : : ; βk 2 Rg if S = fa1;:::; ang is a finite set, we write span¹Sº = span¹a1;:::; anº (the span of the empty set is defined as f0g) Matrix rank 1.2 Operations on subspaces three common operations on subspaces (V and W are subspaces) • intersection: V\W = fx j x 2 V; x 2 Wg • sum: V + W = fx + y j x 2 V; y 2 Wg if V\W = f0g this is called the direct sum and written as V ⊕ W • orthogonal complement: V? = fx j yT x = 0 for all y 2 Vg the result of each of the three operations is a subspace Matrix rank 1.3 Range of a matrix × T T suppose A is an m n matrix with columns a1;:::; an and rows b1;:::; bm: 2 bT 3 6 :1 7 A = a1 ··· an = 6 : 7 6 7 6 bT 7 4 m 5 Range (column space): the span of the column vectors (a subspace of Rm) range¹Aº = span¹a1;:::; anº n = fx1a1 + ··· + xnan j x 2 R g = fAx j x 2 Rng the range of AT is called the row space of A (a subspace of Rn): T range¹A º = span¹b1;:::; bmº m = fy1b1 + ··· + ymbm j y 2 R g = fAT y j y 2 Rmg Matrix rank 1.4 Nullspace of a matrix × T T suppose
    [Show full text]