DOCSLIB.ORG

1. Matrix Rank

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
1. Matrix Rank L. Vandenberghe ECE133B (Spring 2020) 1. Matrix rank • subspaces, dimension, rank • QR factorization with pivoting • properties of matrix rank • low-rank matrices • pseudo-inverse 1.1 Subspace a nonempty subset V of Rm is a subspace if αx + βy 2 V for all vectors x; y 2 V and scalars α; β • all linear combinations of elements of V are in V • V is nonempty and closed under scalar multiplication and vector addition Examples • f0g, Rm • the span of a set S ⊆ Rm: all linear combinations of elements of S span¹Sº = fβ1a1 + ··· + βkak j a1;:::; ak 2 S; β1; : : : ; βk 2 Rg if S = fa1;:::; ang is a finite set, we write span¹Sº = span¹a1;:::; anº (the span of the empty set is defined as f0g) Matrix rank 1.2 Operations on subspaces three common operations on subspaces (V and W are subspaces) • intersection: V\W = fx j x 2 V; x 2 Wg • sum: V + W = fx + y j x 2 V; y 2 Wg if V\W = f0g this is called the direct sum and written as V ⊕ W • orthogonal complement: V? = fx j yT x = 0 for all y 2 Vg the result of each of the three operations is a subspace Matrix rank 1.3 Range of a matrix × T T suppose A is an m n matrix with columns a1;:::; an and rows b1;:::; bm: 2 bT 3 6 :1 7 A = a1 ··· an = 6 : 7 6 7 6 bT 7 4 m 5 Range (column space): the span of the column vectors (a subspace of Rm) range¹Aº = span¹a1;:::; anº n = fx1a1 + ··· + xnan j x 2 R g = fAx j x 2 Rng the range of AT is called the row space of A (a subspace of Rn): T range¹A º = span¹b1;:::; bmº m = fy1b1 + ··· + ymbm j y 2 R g = fAT y j y 2 Rmg Matrix rank 1.4 Nullspace of a matrix × T T suppose A is an m n matrix with columns a1;:::; an and rows b1;:::; bm: 2 bT 3 6 :1 7 A = a1 ··· an = 6 : 7 6 7 6 bT 7 4 m 5 Nullspace: the orthogonal complement of the row space (a subspace of Rn) null¹Aº = range¹ATº? n T T = fx 2 R j b1 x = ··· = bmx = 0g = fx 2 Rn j Ax = 0g the nullspace of AT is the orthogonal complement of range¹Aº (a subspace of Rm) null¹ATº = range¹Aº? m T T = fy 2 R j a1 y = ··· = an y = 0g = fy 2 Rm j AT y = 0g Matrix rank 1.5 Exercise • directed graph with m vertices, n arcs (directed edges) • node–arc incidence matrix is m × n matrix A with 8 1 if arc j enters node i ><> Aij = −1 if arc j leaves node i > 0 otherwise : 3 2 3 2 −1 −1 0 1 0 3 6 7 6 1 0 −1 0 0 7 1 4 5 A = 6 7 6 0 0 1 −1 −1 7 6 7 6 0 1 0 0 1 7 4 5 2 1 4 describe in words the subspaces null¹Aº and range¹ATº Matrix rank 1.6 Linearly independent vectors vectors a1,..., an are linearly independent if x1a1 + x2a2 + ··· + xnan = 0 =) x1 = x2 = ··· = xn = 0 • the zero vector cannot be written as a nontrivial linear combination of a1,..., an • no vector ai is a linear combination of the other vectors • in matrix–vector notation: Ax = 0 holds only if x = 0, where A = a1 a2 ··· an • linear (in)dependence is a property of the set of vectors fa1;:::; ang (by convention, the empty set is linearly independent) recall from 133A: if a1;:::; an are linearly independent m-vectors, then n ≤ m Matrix rank 1.7 Basis of a subspace fv1;:::; vk g is a basis for the subspace V if two conditions are satisfied 1. V = span¹v1;:::; vkº 2. v1,..., vk are linearly independent • condition 1 means that every x 2 V can be expressed as x = β1v1 + ··· + βkvk • condition 2 means that the coefficients β1,..., βk are unique: x = β1v1 + ··· + βkvk =) ¹β1 − γ1ºv1 + ··· + ¹βk − γkºvk = 0 x = γ1v1 + ··· + γkvk =) β1 = γ1; : : : ; βk = γk Matrix rank 1.8 Dimension of a subspace • every basis of a subspace V contains the same number of vectors • this number is called the dimension of V (notation: dim¹Vº) Proof: let fv1;:::; vk g be a basis of V containing k vectors and define B = v1 ··· vk • suppose a1;:::; an are linearly independent vectors in V • each ai can be expressed as a linear combination of the basis vectors: a1 = Bx1; a2 = Bx2;:::; an = Bxn • the k-vectors x1;:::; xn are linearly independent: β1x1 + ··· + βnxn = 0 =) B¹β1x1 + ··· + βnxnº = β1a1 + ··· + βnan = 0 =) β1 = ··· = βn = 0 • therefore n ≤ k; this shows that no basis can contain more than k vectors Matrix rank 1.9 Completing a basis • suppose fv1;:::; vj g ⊂ V is a linearly independent set (possibly empty) • then there exists a basis of V of the form fv1;:::; vj; vj+1;:::; vk g we complete the basis by adding the vectors vj+1,..., vk Proof • if fv1;:::; vj g is not already a basis, its span is not V • then there exist vectors in V that are not linear combinations of v1,..., vj • choose one of those vectors, call it vj+1, and add it to the set • the set fv1;:::; vj+1g is a linearly independent subset of V with j + 1 elements • repeat this process until it terminates • it terminates because a linearly independent set in Rm has at most m elements Consequence: every subspace of Rm has a basis Matrix rank 1.10 Rank of a matrix Rank: the rank of a matrix is the dimension of its range rank¹Aº = dim¹range¹Aºº this is also the maximal number of linearly independent columns Example: a 4 × 4 matrix with rank 3 2 1 −1 3 1 3 6 7 6 −1 2 0 0 7 A = a1 a2 a3 a4 = 6 7 6 1 −1 3 0 7 6 7 6 −1 2 0 1 7 4 5 • fa1g is linearly independent (a1 is not zero) • fa1; a2g is linearly independent • fa1; a2; a3g is linearly dependent: a3 = 6a1 + 3a2 • fa1; a2; a4g is a basis for range¹Aº: linearly independent and spans range¹Aº Matrix rank 1.11 Rank-r matrices in factored form we will often encounter matrices in the product form A = BC, where • B is m × r with linearly independent columns • C is r × n with linearly independent rows the matrix A has rank r • range¹Aº ⊆ range¹Bº: each column of A is a linear combination of columns of B • range¹Bº ⊆ range¹Aº: y = Bx =) y = B¹CDºx = A¹Dxº where D is a right inverse of C (for example, D = Cy) • therefore range¹Aº = range¹Bº and rank¹Aº = rank¹Bº • since the columns of B are linearly independent, rank¹Bº = r Matrix rank 1.12 Exercises Exercise 1 V and W are subspaces in Rm; show that dim¹V + Wº + dim¹V \ Wº = dim¹Vº + dim¹Wº Exercise 2 • A and B are matrices with the same number of rows; find a matrix C with range¹Cº = range¹Aº + range¹Bº • A and B are matrices with the same number of columns; find a matrix C with null¹Cº = null¹Aº \ null¹Bº Matrix rank 1.13 Outline • subspaces, dimension, rank • QR factorization with pivoting • properties of matrix rank • low-rank matrices • pseudo-inverse QR factorization A is an m × n matrix with linearly independent columns (hence, m ≥ n) QR factorization A = QR • R is n × n, upper triangular, with positive diagonal elements • Q is m × n with orthonormal columns (QTQ = I) • several algorithms, including Gram–Schmidt algorithm Full QR factorization (QR decomposition) R A = Q Q˜ 0 • R is n × n, upper triangular, with positive diagonal elements • Q Q˜ is orthogonal: square with orthonormal columns • several algorithms, including Householder triangularization Matrix rank 1.14 QR factorization with column pivoting A is an m × n matrix (may be wide or have linearly dependent columns) QR factorization with column pivoting (column reordering) A = QRP • Q is m × r with orthonormal columns • R is r × n, leading r × r submatrix is upper triangular with positive diagonal: 2 R11 R12 ··· R1r R1;r+1 ··· R1n 3 6 7 6 0 R22 ··· R2r R2;r+1 ··· R2n 7 R = 6 : : : : : : 7 6 : : : : : : : 7 6 7 6 0 0 ··· Rrr Rr;r+1 ··· Rrn 7 4 5 • can be chosen to satisfy R11 ≥ R22 ≥ · · · ≥ Rrr > 0 • P is an n × n permutation matrix • r is the rank of A: apply the result on page 1.27 with B = Q, C = RP Matrix rank 1.15 Interpretation • columns of APT = QR are the columns of A in a different order • the columns are divided in two groups: T AP = Aˆ1 Aˆ2 = Q R1 R2 Aˆ1 is m × r, R1 is r × r • Aˆ1 = QR1 is m × r with linearly independent columns: ˆ −1 T ˆ A1x = QR1x = 0 =) R1 Q A1x = x = 0 • Aˆ2 = QR2 is m × ¹n − rº: columns are linear combinations of columns of Aˆ1 ˆ ˆ −1 A2 = QR2 = A1R1 R2 the factorization provides two useful bases for range¹Aº • columns of Q are an orthonormal basis • columns of Aˆ1 are a basis selected from the columns of A Matrix rank 1.16 Computing the pivoted QR factorization we first describe the modified Gram–Schmidt algorithm • a variation of the standard Gram–Schmidt algorithm for QR factorization A = QR of a matrix A with linearly independent columns • has better numerical properties than the standard Gram–Schmidt algorithm we then extend the modified GS method to the pivoted QR factorization Matrix rank 1.17 Modified Gram–Schmidt algorithm computes QR factorization of an m × n matrix A with linearly independent columns 2 R11 R12 ··· R1n 3 6 7 6 0 R22 ··· R2n 7 a1 a2 ··· an = q1 q2 ··· qn 6 : : : : 7 6 : : : : : 7 6 7 6 0 0 ··· Rnn 7 4 5 • after k steps, the algorithm has computed a partial factorization 2 R ··· R R ··· R 3 6 11 1k 1;k+1 1n 7 6 : : : : : : : : : : 7 ··· 6 7 A = q1 qk Bk 6 0 ··· R R ··· R 7 6 kk k;k+1 kn 7 6 7 6 0 I 7 4 5 where Bk has size m × ¹n − kº with columns orthogonal to q1,..., qk • we start the algorithm with B0 = A Matrix rank 1.18 Modified Gram–Schmidt update careful inspection of the update at step k shows that " # Rkk Rk;¹k+1º:n Bk−1 = qk Bk 0 I partition Bk−1 as Bk−1 = b Bˆ with b the first column and Bˆ of size m × ¹n − kº: b = qk Rkk; Bˆ = qk Rk;¹k+1º:n + Bk • from the first equation, and the required properties kqk k = 1 and Rkk > 0: 1 Rkk = kbk; qk = b Rkk • T from the second equation, and the requirement that qk Bk = 0: T ˆ ˆ Rk;¹k+1º:n = qk B; Bk = B − qk Rk;¹k+1º:n Matrix rank 1.19 Summary: modified Gram–Schmidt algorithm Algorithm define B0 = A and repeat for k = 1 to n: • compute Rkk = kbk and qk = ¹1/Rkkºb where b is the first column of Bk−1 • compute T ˆ ˆ Rk;k+1 ··· Rkn = qk B; Bk = B −
Load more

Recommended publications
  • Solving a Low-Rank Factorization Model for Matrix Completion by a Nonlinear Successive Over-Relaxation Algorithm
    SOLVING A LOW-RANK FACTORIZATION MODEL FOR MATRIX COMPLETION BY A NONLINEAR SUCCESSIVE OVER-RELAXATION ALGORITHM ZAIWEN WEN †, WOTAO YIN ‡, AND YIN ZHANG § Abstract. The matrix completion problem is to recover a low-rank matrix from a subset of its entries. The main solution strategy for this problem has been based on nuclear-norm minimization which requires computing singular value decompositions – a task that is increasingly costly as matrix sizes and ranks increase. To improve the capacity of solving large-scale problems, we propose a low-rank factorization model and construct a nonlinear successive over-relaxation (SOR) algorithm that only requires solving a linear least squares problem per iteration. Convergence of this nonlinear SOR algorithm is analyzed. Numerical results show that the algorithm can reliably solve a wide range of problems at a speed at least several times faster than many nuclear-norm minimization algorithms. Key words. Matrix Completion, alternating minimization, nonlinear GS method, nonlinear SOR method AMS subject classifications. 65K05, 90C06, 93C41, 68Q32 1. Introduction. The problem of minimizing the rank of a matrix arises in many applications, for example, control and systems theory, model reduction and minimum order control synthesis [20], recovering shape and motion from image streams [25, 32], data mining and pattern recognitions [6] and machine learning such as latent semantic indexing, collaborative prediction and low-dimensional embedding. In this paper, we consider the Matrix Completion (MC) problem of finding a lowest-rank matrix given a subset of its entries, that is, (1.1) min rank(W ), s.t. Wij = Mij , ∀(i,j) ∈ Ω, W ∈Rm×n where rank(W ) denotes the rank of W , and Mi,j ∈ R are given for (i,j) ∈ Ω ⊂ {(i,j) : 1 ≤ i ≤ m, 1 ≤ j ≤ n}.
    [Show full text]
  • Low-Rank Incremental Methods for Computing Dominant Singular Subspaces
    Low-Rank Incremental Methods for Computing Dominant Singular Subspaces C. G. Baker∗ K. A. Gallivan∗ P. Van Dooren† Abstract This paper describes a generic low-rank incremental method for computing the dominant singular triplets of a matrix via a single pass through the matrix. This work unifies several efforts previously described in the literature. We tie the operation of the proposed method to a particular optimization- based eigensolver. This allows the description of novel methods exploiting multiple passes through the matrix. 1 Introduction Given a matrix A Rm×n,m n, the Singular Value Decomposition (SVD) of A is: ∈ ≥ Σ A = U V T , 0 where U and V are m m and n n orthogonal matrices, respectively, and Σ is a diagonal matrix whose × × elements σ1,...,σn are real, non-negative and non-increasing. The σi are the singular values of A, and the first n columns of U and V are the left and right singular vectors of A, respectively. Given k n, the singular vectors associated with the largest k singular values of A are the rank-k dominant singular≤ vectors. The subspaces associated with these vectors are the rank-k dominant singular subspaces. The components of the SVD are optimal in many respects [1], and these properties result in the occurrence of the SVD in numerous analyses, e.g., principal component analysis, proper orthogonal decomposition, the Karhunen-Loeve transform. These applications of the SVD are frequently used in problems related to, e.g., signal processing, model reduction of dynamical systems, image and face recognition, and information retrieval.
    [Show full text]
  • Block Matrices in Linear Algebra
    PRIMUS Problems, Resources, and Issues in Mathematics Undergraduate Studies ISSN: 1051-1970 (Print) 1935-4053 (Online) Journal homepage: https://www.tandfonline.com/loi/upri20 Block Matrices in Linear Algebra Stephan Ramon Garcia & Roger A. Horn To cite this article: Stephan Ramon Garcia & Roger A. Horn (2020) Block Matrices in Linear Algebra, PRIMUS, 30:3, 285-306, DOI: 10.1080/10511970.2019.1567214 To link to this article: https://doi.org/10.1080/10511970.2019.1567214 Accepted author version posted online: 05 Feb 2019. Published online: 13 May 2019. Submit your article to this journal Article views: 86 View related articles View Crossmark data Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=upri20 PRIMUS, 30(3): 285–306, 2020 Copyright # Taylor & Francis Group, LLC ISSN: 1051-1970 print / 1935-4053 online DOI: 10.1080/10511970.2019.1567214 Block Matrices in Linear Algebra Stephan Ramon Garcia and Roger A. Horn Abstract: Linear algebra is best done with block matrices. As evidence in sup- port of this thesis, we present numerous examples suitable for classroom presentation. Keywords: Matrix, matrix multiplication, block matrix, Kronecker product, rank, eigenvalues 1. INTRODUCTION This paper is addressed to instructors of a first course in linear algebra, who need not be specialists in the field. We aim to convince the reader that linear algebra is best done with block matrices. In particular, flexible thinking about the process of matrix multiplication can reveal concise proofs of important theorems and expose new results. Viewing linear algebra from a block-matrix perspective gives an instructor access to use- ful techniques, exercises, and examples.
    [Show full text]
  • Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications
    JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2015 1 Structured Low-Rank Matrix Factorization: Global Optimality, Algorithms, and Applications Benjamin D. Haeffele, Rene´ Vidal, Fellow, IEEE Abstract—Recently, convex formulations of low-rank matrix factorization problems have received considerable attention in machine learning. However, such formulations often require solving for a matrix of the size of the data matrix, making it challenging to apply them to large scale datasets. Moreover, in many applications the data can display structures beyond simply being low-rank, e.g., images and videos present complex spatio-temporal structures that are largely ignored by standard low-rank methods. In this paper we study a matrix factorization technique that is suitable for large datasets and captures additional structure in the factors by using a particular form of regularization that includes well-known regularizers such as total variation and the nuclear norm as particular cases. Although the resulting optimization problem is non-convex, we show that if the size of the factors is large enough, under certain conditions, any local minimizer for the factors yields a global minimizer. A few practical algorithms are also provided to solve the matrix factorization problem, and bounds on the distance from a given approximate solution of the optimization problem to the global optimum are derived. Examples in neural calcium imaging video segmentation and hyperspectral compressed recovery show the advantages of our approach on high-dimensional
    [Show full text]
  • Multidimensional Butterfly Factorization
    JID:YACHA AID:1193 /COR [m3L; v1.215; Prn:11/05/2017; 9:36] P.1 (1-22) Appl. Comput. Harmon. Anal. ••• (••••) •••–••• Contents lists available at ScienceDirect Applied and Computational Harmonic Analysis www.elsevier.com/locate/acha Letter to the Editor Multidimensional butterfly factorization Yingzhou Li b,∗, Haizhao Yang c, Lexing Ying a,b a Department of Mathematics, Stanford University, United States b ICME, Stanford University, United States c Department of Mathematics, Duke University, United States a r t i c l e i n f o a b s t r a c t Article history: This paper introduces the multidimensional butterfly factorization as a data-sparse Received 26 September 2015 representation of multidimensional kernel matrices that satisfy the complementary Received in revised form 30 March low-rank property. This factorization approximates such a kernel matrix of size 2017 N × N with a product of O(log N)sparse matrices, each of which contains Accepted 11 April 2017 O(N) nonzero entries. We also propose efficient algorithms for constructing this Available online xxxx Communicated by Joel A. Tropp factorization when either (i) a fast algorithm for applying the kernel matrix and its adjoint is available or (ii) every entry of the kernel matrix can be evaluated MSC: in O(1) operations. For the kernel matrices of multidimensional Fourier integral 44A55 operators, for which the complementary low-rank property is not satisfied due to a 65R10 singularity at the origin, we extend this factorization by combining it with either 65T50 a polar coordinate transformation or a multiscale decomposition of the integration domain to overcome the singularity.
    [Show full text]
  • Notes APPM 5720 — P.G
    Course notes APPM 5720 — P.G. Martinsson January 22, 2016 Matrix factorizations and low rank approximation The first section of the course provides a quick review of basic concepts from linear algebra that we will use frequently. Note that the pace is fast here, and assumes that you have seen these concepts in prior course-work. If not, then additional reading on the side is strongly recommended! 1. NOTATION, ETC n n 1.1. Norms. Let x = [x1; x2; : : : ; xn] denote a vector in R or C . Our default norm for vectors is the Euclidean norm 1=2 0 n 1 X 2 kxk = @ jxjj A : j=1 We will at times also use `p norms 1=p 0 n 1 X p kxkp = @ jxjj A : j=1 Let A denote an m × n matrix. For the most part, we allow A to have complex entries. We define the spectral norm of A via kAxk kAk = sup kAxk = sup : kxk=1 x6=0 kxk We define the Frobenius norm of A via 1=2 0 m n 1 X X 2 kAkF = @ jA(i; j)j A : i=1 j=1 Observe that p kAk ≤ kAkF ≤ min(m; n) kAk: 1.2. Transpose and adjoint. Given an m × n matrix A, the transpose At is the n × m matrix B with entries B(i; j) = A(j; i): The transpose is most commonly used for real matrices. It can also be used for a complex matrix, but more typically, we then use the adjoint, which is the complex conjugate of the transpose A∗ = At: 1.3.
    [Show full text]
  • Direct Methodsmethods Forfor Solvingsolving Linearlinear Systemssystems (3H.)(3H.)
    NumericalNumerical MethodsMethods RafaRafałł ZdunekZdunek DirectDirect methodsmethods forfor solvingsolving linearlinear systemssystems (3h.)(3h.) ((GaussianGaussian eliminationelimination andand matrixmatrix decompositionsdecompositions)) Introduction • Solutions to linear systems • Gaussian elimination, • LU factorization, • Gauss-Jordan elimination, • Pivoting Bibliography [1] A. Bjorck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996, [2] G. Golub, C. F. Van Loan, Matrix Computations, The John Hopkins University Press, (Third Edition), 1996, [3] J. Stoer R. Bulirsch, Introduction to Numerical Analysis (Second Edition), Springer-Verlag, 1993, [4] C. D. Meyer, Matrix Analysis and Applied Linear Algebra, SIAM, 2000, [5] Ch. Zarowski, An Introduction to Numerical Analysis for Electrical and Computer Engineers, Wiley, 2004, [6] G. Strang, Linear Algebra and Its Applications, Harcourt Brace & Company International Edition, 1998, Applications There are two basic classes of methods for solving a system of linear equations: direct and iterative methods. Direct methods theoretically give an exact solution in a (predictable) finite number of steps. Unfortunately, this does not have to be true in computational approach due to rounding errors: an error made in one step spreads in all following steps. Classical direct methods usually involve a variety of matrix factorization techniques such as the LU, LDU, LUP, Cholesky, QR, EVD, SVD, and GSVD. Direct methods are computationally efficient only for a specific class of problems where the system matrix is sparse and reveals some patterns, e.g. it is a banded matrix. Otherwise, iterative methods are usually more efficient. Solutions to linear systems m: ixrtro ma fingollow fd in thexses een beprations cqua era linem ofste syA Ax= b (1) , MN× M where =A [aij ] ∈ ℑ is a coefficient mtarix, =b []bi ∈ ℑ is a dat vector, and a N MN×( 1+ ) =x []x j ∈ ℑt ed.
    [Show full text]
  • Generalized Inverses and Generalized Connections with Statistics
    Generalized Inverses and Generalized Connections with Statistics Tim Zitzer April 12, 2007 Creative Commons License c 2007 Permission is granted to others to copy, distribute, display and perform the work and make derivative works based upon it only if they give the author or licensor the credits in the manner specified by these and only for noncommercial purposes. Generalized Inverses and Generalized Connections with Statistics Consider an arbitrary system of linear equations with a coefficient matrix A ∈ Mn,n, vector of constants b ∈ Cn, and solution vector x ∈ Cn : Ax = b If matrix A is nonsingular and thus invertible, then we can employ the techniques of matrix inversion and multiplication to find the solution vector x. In other words, the unique solution x = A−1b exists. However, when A is rectangular, dimension m × n or singular, a simple representation of a solution in terms of A is more difficult. There may be none, one, or an infinite number of solutions depending on whether b ∈ C(A) and whether n − rank(A) > 0. We would like to be able to find a matrix (or matrices) G, such that solutions of Ax = b are of the form Gb. Thus, through the work of mathematicians such as Penrose, the generalized inverse matrix was born. In broad terms, a generalized inverse matrix of A is some matrix G such that Gb is a solution to Ax = b. Definitions and Notation In 1920, Moore published the first work on generalized inverses. After his definition was more or less forgotten due to cumbersome notation, an algebraic form of Moore’s definition was given by Penrose in 1955, having explored the theoretical properties in depth.
    [Show full text]
  • On the Complexity of the Block Low-Rank Multifrontal Factorization Patrick Amestoy, Alfredo Buttari, Jean-Yves L’Excellent, Théo Mary
    On the Complexity of the Block Low-Rank Multifrontal Factorization Patrick Amestoy, Alfredo Buttari, Jean-Yves l’Excellent, Théo Mary To cite this version: Patrick Amestoy, Alfredo Buttari, Jean-Yves l’Excellent, Théo Mary. On the Complexity of the Block Low-Rank Multifrontal Factorization. SIAM Journal on Scientific Computing, Society for Industrial and Applied Mathematics, 2017, 39 (4), pp.34. 10.1137/16M1077192. hal-01322230v3 HAL Id: hal-01322230 https://hal.archives-ouvertes.fr/hal-01322230v3 Submitted on 10 Oct 2019 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ON THE COMPLEXITY OF THE BLOCK LOW-RANK MULTIFRONTAL FACTORIZATION PATRICK AMESTOY∗, ALFREDO BUTTARIy , JEAN-YVES L'EXCELLENTz , AND THEO MARYx Abstract. Matrices coming from elliptic Partial Differential Equations have been shown to have a low- rank property: well defined off-diagonal blocks of their Schur complements can be approximated by low-rank products and this property can be efficiently exploited in multifrontal solvers to provide a substantial reduction of their complexity. Among the possible low-rank formats, the Block Low-Rank format (BLR) is easy to use in a general purpose multifrontal solver and has been shown to provide significant gains compared to full-rank on practical applications.
    [Show full text]
  • RANK FACTORIZATION and MOORE-PENROSE INVERSE Gradimir V
    RANK FACTORIZATION AND MOORE-PENROSE INVERSE Gradimir V. Milovanovic´ and Predrag S. Stanimirovic´ Abstract. In this paper we develop a few representations of the Moore- Penrose inverse, based on full-rank factorizations of matrices. These rep- resentations we divide into the two different classes: methods which arise from the known block decompositions and determinantal representation. In particular cases we obtain several known results. 1. Introduction Cm×n The set of m × n complex matrices of rank r is denoted by r = {X ∈ Cm×n |r : rank(X) = r}. With A and A|r we denote the first r columns of A and the first r rows of A, respectively. The identity matrix of the order k is denoted by Ik, and O denotes the zero block of an appropriate dimensions. We use the following useful expression for the Moore-Penrose generalized inverse A†, based on the full-rank factorization A = PQ of A [1-2]: A† = Q†P † = Q∗(QQ∗)−1(P ∗P )−1P ∗ = Q∗(P ∗AQ∗)−1P ∗. We restate main known block decompositions [7], [16-18]. For a given Cm×n matrix A ∈ r there exist regular matrices R, G, permutation matrices E, F and unitary matrices U, V , such that: 1991 Mathematics Subject Classification. 15A09, 15A24. Typeset by AMS-TEX 1 2 G. V. Milovanovi´cand P. S. Stanimirovi´c Ir O B O (T1) RAG = O O = N1, (T2) RAG = O O = N2, I K I O (T ) RAF = r = N , (T ) EAG = r = N , 3 O O 3 4 K O 4 Ir O Ir O (T5) UAG = O O = N1, (T6) RAV = O O = N1, B O B K (T7) UAV = O O = N2, (T8) UAF = O O = N5, B O (T ) EAV = = N , 9 K O 6 A11 A12 (T10) EAF = = N7, where rank(A11)=rank(A).
    [Show full text]
  • Matrix Factorizations and Low Rank Approximation
    CHAPTER 1 Matrix factorizations and low rank approximation We will start with a review of basic concepts from linear algebra that will be used frequently. Note that the pace is fast here, and assumes that the reader has some level of familiarity with the material. The one concept that is perhaps less well known is the material on the interpolative decomposition (ID) in Section 1.5. 1.1. Notation, etc n n 1.1.1. Norms. Let x = [x1; x2; : : : ; xn] denote a vector in R or C . Our default norm for vectors is the Euclidean norm 1=2 0 n 1 X 2 kxk = @ jxjj A : j=1 We will at times also use `p norms 1=p 0 n 1 X p kxkp = @ jxjj A : j=1 Let A denote an m × n matrix. For the most part, we allow A to have complex entries. We define the spectral norm of A via kAxk kAk = sup kAxk = sup : kxk=1 x6=0 kxk We define the Frobenius norm of A via 1=2 0 m n 1 X X 2 kAkF = @ jA(i; j)j A : i=1 j=1 Observe that the spectral norm and the Frobenius norms satisfy the inequalities p kAk ≤ kAkF ≤ min(m; n) kAk: 1.1.2. Transpose and adjoint. Given an m × n matrix A, the transpose At is the n × m matrix B with entries B(i; j) = A(j; i): The transpose is most commonly used for real matrices. It can also be used for a complex matrix, but more typically, we then use the adjoint, which is the complex conjugate of the transpose A∗ = At: 1 1.1.3.
    [Show full text]
  • Low Rank Approximation Lecture 1
    Low Rank Approximation Lecture 1 Daniel Kressner Chair for Numerical Algorithms and HPC Institute of Mathematics, EPFL [email protected] 1 Organizational aspects I Lectures: Tuesday 8-10, MA A110. First: September 25, Last: December 18. I Exercises: Tuesday 8-10, MA A110. First: September 25, Last: December 18. I Exam: Miniproject + oral exam. I Webpage: https://anchp.epfl.ch/lowrank. I [email protected], [email protected] 2 From http://www.niemanlab.org ... his [Aleksandr Kogan’s] message went on to confirm that his approach was indeed similar to SVD or other matrix factorization meth- ods, like in the Netflix Prize competition, and the Kosinki-Stillwell- Graepel Facebook model. Dimensionality reduction of Facebook data was the core of his model. 3 Rank and basic properties For field F, let A 2 F m×n. Then rank(A) := dim(range(A)): For simplicity, F = R throughout the lecture and often m ≥ n. Lemma Let A 2 Rm×n. Then 1. rank(AT ) = rank(A); 2. rank(PAQ) = rank(A) for invertible matrices P 2 Rm×m, Q 2 Rn×n; 3. rank(AB) ≤ minfrank(A); rank(B)g for any matrix B 2 Rn×p. A11 A12 m ×n 4. rank = rank(A11) + rank(A22) for A11 2 R 1 1 , 0 A22 m ×n m ×n A12 2 R 1 2 ,A22 2 R 2 2 . Proof: See Linear Algebra 1 / Exercises. 4 Rank and matrix factorizations m Let B = fb1;:::; br g ⊂ R with r = rank(A) be basis of range(A). Then each of the columns of A = a1; a2;:::; an can be expressed as linear combination of B: 2 3 ci1 ;:::; 6 .
    [Show full text]