Matrices with Hierarchical Low-Rank Structures
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Extremely Accurate Solutions Using Block Decomposition and Extended Precision for Solving Very Ill-Conditioned Equations
Extremely accurate solutions using block decomposition and extended precision for solving very ill-conditioned equations †Kansa, E.J.1, Skala V.2, and Holoborodko, P.3 1Convergent Solutions, Livermore, CA 94550 USA 2Computer Science & Engineering Dept., Faculty of Applied Sciences, University of West Bohemia, University 8, CZ 301 00 Plzen, Czech Republic 3Advanpix LLC, Maison Takashima Bldg. 2F, Daimachi 10-15-201, Yokohama, Japan 221-0834 †Corresponding author: [email protected] Abstract Many authors have complained about the ill-conditioning associated the numerical solution of partial differential equations (PDEs) and integral equations (IEs) using as the continuously differentiable Gaussian and multiquadric continuously differentiable (C ) radial basis functions (RBFs). Unlike finite elements, finite difference, or finite volume methods that lave compact local support that give rise to sparse equations, the C -RBFs with simple collocation methods give rise to full, asymmetric systems of equations. Since C RBFs have adjustable constent or variable shape parameters, the resulting systems of equations that are solve on single or double precision computers can suffer from “ill-conditioning”. Condition numbers can be either the absolute or relative condition number, but in the context of linear equations, the absolute condition number will be understood. Results will be presented that demonstrates the combination of Block Gaussian elimination, arbitrary arithmetic precision, and iterative refinement can give remarkably accurate numerical salutations to large Hilbert and van der Monde equation systems. 1. Introduction An accurate definition of the condition number, , of the matrix, A, is the ratio of the largest to smallest absolute value of the singular values, { i}, obtained from the singular value decomposition (SVD) method, see [1]: (A) = maxjjminjj (1) This definition of condition number will be used in this study. -
A Study of Convergence of the PMARC Matrices Applicable to WICS Calculations
A Study of Convergence of the PMARC Matrices applicable to WICS Calculations /- by Amitabha Ghosh Department of Mechanical Engineering Rochester Institute of Technology Rochester, NY 14623 Final Report NASA Cooperative Agreement No: NCC 2-937 Presented to NASA Ames Research Center Moffett Field, CA 94035 August 31, 1997 Table of Contents Abstract ............................................ 3 Introduction .......................................... 3 Solution of Linear Systems ................................... 4 Direct Solvers .......................................... 5 Gaussian Elimination: .................................. 5 Gauss-Jordan Elimination: ................................ 5 L-U Decompostion: ................................... 6 Iterative Solvers ........................................ 6 Jacobi Method: ..................................... 7 Gauss-Seidel Method: .............................. .... 7 Successive Over-relaxation Method: .......................... 8 Conjugate Gradient Method: .............................. 9 Recent Developments: ......................... ....... 9 Computational Efficiency .................................... 10 Graphical Interpretation of Residual Correction Schemes .................... 11 Defective Matrices ............................. , ......... 15 Results and Discussion ............ ......................... 16 Concluding Remarks ...................................... 19 Acknowledgements ....................................... 19 References .......................................... -
Applied Linear Algebra
APPLIED LINEAR ALGEBRA Giorgio Picci November 24, 2015 1 Contents 1 LINEAR VECTOR SPACES AND LINEAR MAPS 10 1.1 Linear Maps and Matrices . 11 1.2 Inverse of a Linear Map . 12 1.3 Inner products and norms . 13 1.4 Inner products in coordinate spaces (1) . 14 1.5 Inner products in coordinate spaces (2) . 15 1.6 Adjoints. 16 1.7 Subspaces . 18 1.8 Image and kernel of a linear map . 19 1.9 Invariant subspaces in Rn ........................... 22 1.10 Invariant subspaces and block-diagonalization . 23 2 1.11 Eigenvalues and Eigenvectors . 24 2 SYMMETRIC MATRICES 25 2.1 Generalizations: Normal, Hermitian and Unitary matrices . 26 2.2 Change of Basis . 27 2.3 Similarity . 29 2.4 Similarity again . 30 2.5 Problems . 31 2.6 Skew-Hermitian matrices (1) . 32 2.7 Skew-Symmetric matrices (2) . 33 2.8 Square roots of positive semidefinite matrices . 36 2.9 Projections in Rn ............................... 38 2.10 Projections on general inner product spaces . 40 3 2.11 Gramians. 41 2.12 Example: Polynomial vector spaces . 42 3 LINEAR LEAST SQUARES PROBLEMS 43 3.1 Weighted Least Squares . 44 3.2 Solution by the Orthogonality Principle . 46 3.3 Matrix least-Squares Problems . 48 3.4 A problem from subspace identification . 50 3.5 Relation with Left- and Right- Inverses . 51 3.6 The Pseudoinverse . 54 3.7 The Euclidean pseudoinverse . 63 3.8 The Pseudoinverse and Orthogonal Projections . 64 3.9 Linear equations . 66 4 3.10 Unfeasible linear equations and Least Squares . 68 3.11 The Singular value decomposition (SVD) . -
Matrix Computations
Chapter 5 Matrix Computations §5.1 Setting Up Matrix Problems §5.2 Matrix Operations §5.3 Once Again, Setting Up Matrix Problems §5.4 Recursive Matrix Operations §5.5 Distributed Memory Matrix Multiplication The next item on our agenda is the linear equation problem Ax = b. However, before we get into algorithmic details, it is important to study two simpler calculations: matrix-vector mul- tiplication and matrix-matrix multiplication. Both operations are supported in Matlab so in that sense there is “nothing to do.” However, there is much to learn by studying how these com- putations can be implemented. Matrix-vector multiplications arise during the course of solving Ax = b problems. Moreover, it is good to uplift our ability to think at the matrix-vector level before embarking on a presentation of Ax = b solvers. The act of setting up a matrix problem also deserves attention. Often, the amount of work that is required to initialize an n-by-n matrix is as much as the work required to solve for x. We pay particular attention to the common setting when each matrix entry aij is an evaluation of a continuous function f(x, y). A theme throughout this chapter is the exploitation of structure. It is frequently the case that there is a pattern among the entries in A which can be used to reduce the amount of work. The fast Fourier transform and the fast Strassen matrix multiply algorithm are presented as examples of recursion in the matrix computations. The organization of matrix-matrix multiplication on a ring of processors is also studied and gives us a nice snapshot of what algorithm development is like in a distributed memory environment. -
Solving a Low-Rank Factorization Model for Matrix Completion by a Nonlinear Successive Over-Relaxation Algorithm
SOLVING A LOW-RANK FACTORIZATION MODEL FOR MATRIX COMPLETION BY A NONLINEAR SUCCESSIVE OVER-RELAXATION ALGORITHM ZAIWEN WEN †, WOTAO YIN ‡, AND YIN ZHANG § Abstract. The matrix completion problem is to recover a low-rank matrix from a subset of its entries. The main solution strategy for this problem has been based on nuclear-norm minimization which requires computing singular value decompositions – a task that is increasingly costly as matrix sizes and ranks increase. To improve the capacity of solving large-scale problems, we propose a low-rank factorization model and construct a nonlinear successive over-relaxation (SOR) algorithm that only requires solving a linear least squares problem per iteration. Convergence of this nonlinear SOR algorithm is analyzed. Numerical results show that the algorithm can reliably solve a wide range of problems at a speed at least several times faster than many nuclear-norm minimization algorithms. Key words. Matrix Completion, alternating minimization, nonlinear GS method, nonlinear SOR method AMS subject classifications. 65K05, 90C06, 93C41, 68Q32 1. Introduction. The problem of minimizing the rank of a matrix arises in many applications, for example, control and systems theory, model reduction and minimum order control synthesis [20], recovering shape and motion from image streams [25, 32], data mining and pattern recognitions [6] and machine learning such as latent semantic indexing, collaborative prediction and low-dimensional embedding. In this paper, we consider the Matrix Completion (MC) problem of finding a lowest-rank matrix given a subset of its entries, that is, (1.1) min rank(W ), s.t. Wij = Mij , ∀(i,j) ∈ Ω, W ∈Rm×n where rank(W ) denotes the rank of W , and Mi,j ∈ R are given for (i,j) ∈ Ω ⊂ {(i,j) : 1 ≤ i ≤ m, 1 ≤ j ≤ n}. -
Low-Rank Incremental Methods for Computing Dominant Singular Subspaces
Low-Rank Incremental Methods for Computing Dominant Singular Subspaces C. G. Baker∗ K. A. Gallivan∗ P. Van Dooren† Abstract This paper describes a generic low-rank incremental method for computing the dominant singular triplets of a matrix via a single pass through the matrix. This work unifies several efforts previously described in the literature. We tie the operation of the proposed method to a particular optimization- based eigensolver. This allows the description of novel methods exploiting multiple passes through the matrix. 1 Introduction Given a matrix A Rm×n,m n, the Singular Value Decomposition (SVD) of A is: ∈ ≥ Σ A = U V T , 0 where U and V are m m and n n orthogonal matrices, respectively, and Σ is a diagonal matrix whose × × elements σ1,...,σn are real, non-negative and non-increasing. The σi are the singular values of A, and the first n columns of U and V are the left and right singular vectors of A, respectively. Given k n, the singular vectors associated with the largest k singular values of A are the rank-k dominant singular≤ vectors. The subspaces associated with these vectors are the rank-k dominant singular subspaces. The components of the SVD are optimal in many respects [1], and these properties result in the occurrence of the SVD in numerous analyses, e.g., principal component analysis, proper orthogonal decomposition, the Karhunen-Loeve transform. These applications of the SVD are frequently used in problems related to, e.g., signal processing, model reduction of dynamical systems, image and face recognition, and information retrieval. -
Block Matrices in Linear Algebra
PRIMUS Problems, Resources, and Issues in Mathematics Undergraduate Studies ISSN: 1051-1970 (Print) 1935-4053 (Online) Journal homepage: https://www.tandfonline.com/loi/upri20 Block Matrices in Linear Algebra Stephan Ramon Garcia & Roger A. Horn To cite this article: Stephan Ramon Garcia & Roger A. Horn (2020) Block Matrices in Linear Algebra, PRIMUS, 30:3, 285-306, DOI: 10.1080/10511970.2019.1567214 To link to this article: https://doi.org/10.1080/10511970.2019.1567214 Accepted author version posted online: 05 Feb 2019. Published online: 13 May 2019. Submit your article to this journal Article views: 86 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=upri20 PRIMUS, 30(3): 285–306, 2020 Copyright # Taylor & Francis Group, LLC ISSN: 1051-1970 print / 1935-4053 online DOI: 10.1080/10511970.2019.1567214 Block Matrices in Linear Algebra Stephan Ramon Garcia and Roger A. Horn Abstract: Linear algebra is best done with block matrices. As evidence in sup- port of this thesis, we present numerous examples suitable for classroom presentation. Keywords: Matrix, matrix multiplication, block matrix, Kronecker product, rank, eigenvalues 1. INTRODUCTION This paper is addressed to instructors of a first course in linear algebra, who need not be specialists in the field. We aim to convince the reader that linear algebra is best done with block matrices. In particular, flexible thinking about the process of matrix multiplication can reveal concise proofs of important theorems and expose new results. Viewing linear algebra from a block-matrix perspective gives an instructor access to use- ful techniques, exercises, and examples. -
Contributions of Issai Schur to Analysis
Contributions of Issai Schur to Analysis Harry Dym and Victor Katsnelson∗ November 1, 2018 The name Schur is associated with many terms and concepts that are widely used in a number of diverse fields of mathematics and engineering. This survey article focuses on Schur’s work in analysis. Here too, Schur’s name is commonplace: The Schur test and Schur-Hadamard multipliers (in the study of estimates for Hermitian forms), Schur convexity, Schur complements, Schur’s results in summation theory for sequences (in particular, the fundamental Kojima-Schur theorem), the Schur-Cohn test, the Schur al- gorithm, Schur parameters and the Schur interpolation problem for functions that are holomorphic and bounded by one in the unit disk. In this survey, we shall discuss all of the above mentioned topics and then some, as well as some of the generalizations that they inspired. There are nine sections of text, each of which is devoted to a separate theme based on Schur’s work. Each of these sections has an independent bibliography. There is very little overlap. A tenth section presents a list of the papers of Schur that focus on topics that are commonly considered to be analysis. We shall begin with a review of Schur’s less familiar papers on the theory of commuting differential operators. Acknowledgement: The authors extend their thanks to Bernd Kirstein for carefully reading the manuscript and spotting a number of misprints. 1 . Permutable differential operators and fractional powers of differential operators. Let arXiv:0706.1868v1 [math.CA] 13 Jun 2007 dny dn−1y P (y)= p (x) + p (x) + + p (x)y (1.1) n dxn n−1 dxn−1 ··· 0 and dmy dm−1y Q(y)= q (x) + q (x) + + q (x)y, (1.2) m dxm m−1 dxm−1 ··· 0 ∗HD thanks Renee and Jay Weiss and VK thanks Ruth and Sylvia Shogam for endowing the chairs that support their respective research. -
Package 'Matrixcalc'
Package ‘matrixcalc’ July 28, 2021 Version 1.0-5 Date 2021-07-27 Title Collection of Functions for Matrix Calculations Author Frederick Novomestky <[email protected]> Maintainer S. Thomas Kelly <[email protected]> Depends R (>= 2.0.1) Description A collection of functions to support matrix calculations for probability, econometric and numerical analysis. There are additional functions that are comparable to APL functions which are useful for actuarial models such as pension mathematics. This package is used for teaching and research purposes at the Department of Finance and Risk Engineering, New York University, Polytechnic Institute, Brooklyn, NY 11201. Horn, R.A. (1990) Matrix Analysis. ISBN 978-0521386326. Lancaster, P. (1969) Theory of Matrices. ISBN 978-0124355507. Lay, D.C. (1995) Linear Algebra: And Its Applications. ISBN 978-0201845563. License GPL (>= 2) Repository CRAN Date/Publication 2021-07-28 08:00:02 UTC NeedsCompilation no R topics documented: commutation.matrix . .3 creation.matrix . .4 D.matrix . .5 direct.prod . .6 direct.sum . .7 duplication.matrix . .8 E.matrices . .9 elimination.matrix . 10 entrywise.norm . 11 1 2 R topics documented: fibonacci.matrix . 12 frobenius.matrix . 13 frobenius.norm . 14 frobenius.prod . 15 H.matrices . 17 hadamard.prod . 18 hankel.matrix . 19 hilbert.matrix . 20 hilbert.schmidt.norm . 21 inf.norm . 22 is.diagonal.matrix . 23 is.idempotent.matrix . 24 is.indefinite . 25 is.negative.definite . 26 is.negative.semi.definite . 28 is.non.singular.matrix . 29 is.positive.definite . 31 is.positive.semi.definite . 32 is.singular.matrix . 34 is.skew.symmetric.matrix . 35 is.square.matrix . 36 is.symmetric.matrix . -
Math 448/548 Cpt S 430/530 Assignment 2 Due February 9 1. Let
Math 448/548 Cpt S 430/530 Assignment 2 due February 9 1. Let A be a real n × n matrix. n (a) Let x 2 R and y = Ax. Show that for i = 1, . , n, yi (the ith component of y) satisfies n ! X jyi j ≤ jaij j kxk1: j=1 (b) Deduce that n ! X ky k ≤ max jaij j kxk 1 i 1 j=1 and n kAxk1 X max ≤ max jaij j: (1) x6=0 kxk i 1 j=1 (c) Show that equality holds in (1) by exhibiting an x for which n kAxk1 X = max jaij j: kxk i 1 j=1 (Hint: Choose an x whose entries are all ±1, with the signs care- fully chosen.) n ! X (d) Deduce that kAk = max jaij j 1 i j=1 2. Let 2 1 2 3 3 A = 4 4 5 6 5 : 7 8 9 (a) Prove by hand computation that A is singular. For example, you can show that the rows of A are linearly dependent. (b) Demonstrate that MATLAB, which uses floating-point arithmetic, does not notice that A is singular (although it does suspect that something is wrong). Use MATLAB commands like inv (inverse) and cond (condition number). 3. The Hilbert matrices are the most famous family of ill-conditioned ma- trices. The (i; j) entry of the n×n Hilbert matrix is hij = 1=(i+j −1). Using MATLAB, do the following for n = 4, 8, and 12. Generate the n×n Hilbert matrix. (MATLAB has a convenient command hilb that generates a Hilbert matrix.) Let z denote the vector of all ones, and let b = Az, where A is the Hilbert matrix. -
Math 240 Summer 2019 Notes
Math 240 Summer 2019 Notes Yao-Rui Lecture notes might contain typos. Contents 1 Lecture 1 { Introduction 4 1.1 Brief Overview of the Course . 4 1.2 Separable ODEs . 6 1.3 First Order Linear ODEs . 7 1.4 Bernoulli ODEs . 8 1.5 Ricatti ODEs . 8 1.6 Reduction of Order . 9 2 Lecture 2 { Matrices 10 2.1 Row Echelon Form . 10 2.2 Determinants and Invertible Matrices . 11 2.3 More Determinant Facts . 13 3 Lecture 3 { Linear Maps 15 3.1 Basis . 15 3.2 Matrix Representation of Linear Maps . 15 3.3 Change of Basis . 17 3.4 Subspaces and the Rank-Nullity Theorem . 18 3.5 An Overview of Abstract Vector Spaces . 19 4 Lecture 4 { Eigenvalues and Eigenvectors 21 4.1 Eigenvalues and Eigenvectors . 21 4.2 The Cayley-Hamilton Theorem . 24 5 Lecture 5 { Matrix Exponentials 26 5.1 Definition and Example Computations . 26 5.2 Analytic Function on Matrices . 28 5.3 Nilpotent Matrices and Exponentiation . 29 6 Lecture 6 { First Order Constant Linear ODEs Part 1 30 6.1 The General Solution . 30 6.2 Examples on Diagonalizable Matrices . 30 6.3 Duhamel's Formula . 31 7 Lecture 7 { First Order Constant Linear ODEs Part 2 33 7.1 Generalized Eigenvectors . 33 7.2 The Jordan Canonical Form for Nilpotent Matrices . 35 7.3 The Jordan Canonical Form in General . 36 1 8 Lecture 8 { First Order Constant Linear ODEs Part 3 42 8.1 An Application of Invariant Subspaces . 42 8.2 Buchheim's Algorithm and Some Formulas for Exponentiating Matrices . -
1. Matrix Rank
L. Vandenberghe ECE133B (Spring 2020) 1. Matrix rank • subspaces, dimension, rank • QR factorization with pivoting • properties of matrix rank • low-rank matrices • pseudo-inverse 1.1 Subspace a nonempty subset V of Rm is a subspace if αx + βy 2 V for all vectors x; y 2 V and scalars α; β • all linear combinations of elements of V are in V • V is nonempty and closed under scalar multiplication and vector addition Examples • f0g, Rm • the span of a set S ⊆ Rm: all linear combinations of elements of S span¹Sº = fβ1a1 + ··· + βkak j a1;:::; ak 2 S; β1; : : : ; βk 2 Rg if S = fa1;:::; ang is a finite set, we write span¹Sº = span¹a1;:::; anº (the span of the empty set is defined as f0g) Matrix rank 1.2 Operations on subspaces three common operations on subspaces (V and W are subspaces) • intersection: V\W = fx j x 2 V; x 2 Wg • sum: V + W = fx + y j x 2 V; y 2 Wg if V\W = f0g this is called the direct sum and written as V ⊕ W • orthogonal complement: V? = fx j yT x = 0 for all y 2 Vg the result of each of the three operations is a subspace Matrix rank 1.3 Range of a matrix × T T suppose A is an m n matrix with columns a1;:::; an and rows b1;:::; bm: 2 bT 3 6 :1 7 A = a1 ··· an = 6 : 7 6 7 6 bT 7 4 m 5 Range (column space): the span of the column vectors (a subspace of Rm) range¹Aº = span¹a1;:::; anº n = fx1a1 + ··· + xnan j x 2 R g = fAx j x 2 Rng the range of AT is called the row space of A (a subspace of Rn): T range¹A º = span¹b1;:::; bmº m = fy1b1 + ··· + ymbm j y 2 R g = fAT y j y 2 Rmg Matrix rank 1.4 Nullspace of a matrix × T T suppose