Bilinear Factor Matrix Norm Minimization for Robust PCA: Algorithms and Applications
Total Page:16
File Type:pdf, Size:1020Kb
Load more
Recommended publications
-
Schatten Class and Berezin Transform of Quaternionic Linear Operators
Schatten class and Berezin transform of quaternionic linear operators Fabrizio Colombo Jonathan Gantner Politecnico di Milano Politecnico di Milano Dipartimento di Matematica Dipartimento di Matematica Via E. Bonardi, 9 Via E. Bonardi, 9 20133 Milano, Italy 20133 Milano, Italy [email protected] [email protected] Tim Janssens University of Antwerp Department of Mathematics Middelheimlaan, 1 2020 Antwerp, Belgium [email protected] Abstract In this paper we introduce the Schatten class of operators and the Berezin transform of operators in the quaternionic setting. The first topic is of great importance in operator theory but it is also necessary to study the second one because we need the notion of trace class operators, which is a particular case of the Schatten class. Regarding the Berezin transform, we give the general definition and properties. Then we concentrate on the setting of weighted Bergman spaces of slice hyperholomorphic functions. Our results are based on the S-spectrum for quaternionic operators, which is the notion of spectrum that appears in the quaternionic version of the spectral theorem and in the quaternionic S-functional calculus. AMS Classification: 47A10, 47A60, 30G35. Key words: Schatten class of operators, Spectral theorem and the S-spectrum, Berezin transform of quater- nionic operators, Weighted Bergman spaces of slice hyperholomorphic functions. arXiv:1511.07308v1 [math.FA] 23 Nov 2015 1 Introduction The study of quaternionic linear operators was stimulated by the celebrated paper of G. Birkhoff and J. von Neumann [13] on the logic of quantum mechanics, where they proved that there are essentially two possible ways to formulate quantum mechanics: the well known one using complex numbers and also one using quaternions. -
On Sketching Matrix Norms and the Top Singular Vector
On Sketching Matrix Norms and the Top Singular Vector Yi Li Huy L. Nguy˜ên University of Michigan, Ann Arbor Princeton University [email protected] [email protected] David P. Woodruff IBM Research, Almaden [email protected] Abstract 1 Introduction Sketching is a prominent algorithmic tool for processing Sketching is an algorithmic tool for handling big data. A large data. In this paper, we study the problem of sketching matrix norms. We consider two sketching models. The first linear skech is a distribution over k × n matrices S, k is bilinear sketching, in which there is a distribution over n, so that for any fixed vector v, one can approximate a pairs of r × n matrices S and n × s matrices T such that for function f(v) from Sv with high probability. The goal any fixed n×n matrix A, from S ·A·T one can approximate is to minimize the dimension k. kAkp up to an approximation factor α ≥ 1 with constant probability, where kAkp is a matrix norm. The second is The sketch Sv thus provides a compression of v, and general linear sketching, in which there is a distribution over is useful for compressed sensing and for achieving low- n2 k linear maps L : R → R , such that for any fixed n × n communication protocols in distributed models. One n2 matrix A, interpreting it as a vector in R , from L(A) one can perform complex procedures on the sketch which can approximate kAkp up to a factor α. We study some of the most frequently occurring matrix would be too expensive to perform on v itself, and this norms, which correspond to Schatten p-norms for p ∈ has led to the fastest known approximation algorithms {0, 1, 2, ∞}. -
Input-Sparsity Low Rank Approximation in Schatten Norm
Input-Sparsity Low Rank Approximation in Schatten Norm Yi Li * 1 David P. Woodruff * 2 Abstract lar value decomposition (SVD) of A, which is an expensive We give the first input-sparsity time algorithms operation. for the rank-k low rank approximation problem For large matrices A this is too slow, so we instead allow for in every Schatten norm. Specifically, for a given randomized approximation algorithms in the hope of achiev- m×n (m ≥ n) matrix A, our algorithm computes ing a much faster running time. Formally, given an approx- Y 2 Rm×k, Z 2 Rn×k, which, with high proba- imation parameter " > 0, we would like to find a rank-k bility, satisfy kA − YZT k ≤ (1 + ")kA − A k , p k p matrix X for which kA − XkF ≤ (1 + ") kA − AkkF Pn p 1=p where kMkp = ( i=1 σi(M) ) is the Schat- with large probability. For this relaxed problem, a num- ten p-norm of a matrix M with singular values ber of efficient methods are known, which are based on σ1(M); : : : ; σn(M), and where Ak is the best dimensionality reduction techniques such as random projec- rank-k approximation to A. Our algorithm runs tions, importance sampling, and other sketching methods, 1;2 in time O~(nnz(A) + mnαp poly(k=")), where with running times O~(nnz(A) + m poly(k=")), where αp = 0 for p 2 [1; 2) and αp = (! − 1)(1 − 2=p) nnz(A) denotes the number of non-zero entries of A. This for p > 2 and ! ≈ 2:374 is the exponent of matrix is significantly faster than the SVD, which takes Θ(~ mn!−1) multiplication. -
Low Rank Approximation Lecture 1
Low Rank Approximation Lecture 1 Daniel Kressner Chair for Numerical Algorithms and HPC Institute of Mathematics, EPFL [email protected] 1 Organizational aspects I Lectures: Tuesday 8-10, MA A110. First: September 25, Last: December 18. I Exercises: Tuesday 8-10, MA A110. First: September 25, Last: December 18. I Exam: Miniproject + oral exam. I Webpage: https://anchp.epfl.ch/lowrank. I [email protected], [email protected] 2 From http://www.niemanlab.org ... his [Aleksandr Kogan’s] message went on to confirm that his approach was indeed similar to SVD or other matrix factorization meth- ods, like in the Netflix Prize competition, and the Kosinki-Stillwell- Graepel Facebook model. Dimensionality reduction of Facebook data was the core of his model. 3 Rank and basic properties For field F, let A 2 F m×n. Then rank(A) := dim(range(A)): For simplicity, F = R throughout the lecture and often m ≥ n. Lemma Let A 2 Rm×n. Then 1. rank(AT ) = rank(A); 2. rank(PAQ) = rank(A) for invertible matrices P 2 Rm×m, Q 2 Rn×n; 3. rank(AB) ≤ minfrank(A); rank(B)g for any matrix B 2 Rn×p. A11 A12 m ×n 4. rank = rank(A11) + rank(A22) for A11 2 R 1 1 , 0 A22 m ×n m ×n A12 2 R 1 2 ,A22 2 R 2 2 . Proof: See Linear Algebra 1 / Exercises. 4 Rank and matrix factorizations m Let B = fb1;:::; br g ⊂ R with r = rank(A) be basis of range(A). Then each of the columns of A = a1; a2;:::; an can be expressed as linear combination of B: 2 3 ci1 ;:::; 6 . -
T-Schatten-P Norm for Low-Rank Tensor Recovery Hao Kong, Xingyu Xie, Student Member, IEEE, Zhouchen Lin, Fellow, IEEE
JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING 1 t-Schatten-p Norm for Low-Rank Tensor Recovery Hao Kong, Xingyu Xie, Student Member, IEEE, Zhouchen Lin, Fellow, IEEE Abstract—In this paper, we propose a new definition of tensor Minimizing the rank function directly is usually NP-hard and Schatten-p norm (t-Schatten-p norm) based on t-SVD [1], and is difficult to be solved within polynomial time, hence we prove that this norm has similar properties to matrix Schatten- often replace the function rank(X ) by its convex/non-convex p norm. More importantly, the t-Schatten-p norm can better surrogate function f(X ): approximate the `1 norm of the tensor multi-rank [2] with 0 < p < 1. Therefore it can be used for the Low-Rank min f(X ); Tensor Recovery problems as a tighter regularizer. We further X (2) prove the tensor multi-Schatten-p norm surrogate theorem and s:t: Ψ(X ) = T : give an efficient algorithm accordingly. By decomposing the target tensor into many small-scale tensors, the non-convex The main difference among the present LRTR models is the optimization problem (0 < p < 1) is transformed into many choice of surrogate function f(·). Based on different definitions convex sub-problems equivalently, which can greatly improve the of tensor singular values, various tensor nuclear norms or tensor computational efficiency when dealing with large-scale tensors. 1 Finally, we provide the theories on the conditions for exact Schatten-p norms are proposed as the rank surrogates [1], [13], recovery in the noiseless case and give the corresponding error [14]. -
Matrix Analysis
Appendix A Matrix Analysis This appendix collects useful background from linear algebra and matrix analysis, such as vector and matrix norms, singular value decompositions, Gershgorin’s disk theorem, and matrix functions. Much more material can be found in various books on the subject including [47, 51, 231, 273, 280, 281, 475]. A.1 Vector and Matrix Norms We work with real or complex vector spaces X, usually X = Rn or X = Cn.We n usually write the vectors in C in boldface, x, while their entries are denoted xj , j ∈ [n],where[n]:={1,...,n}. The canonical unit vectors in Rn are denoted by e1,...,en. They have entries 1 if j = , (e ) = δ = j ,j 0 otherwise. Definition A.1. A nonnegative function ·: X → [0, ∞) is called a norm if (a) x =0if and only if x = 0 (definiteness). (b) λx = |λ|x for all scalars λ and all vectors x ∈ X (homogeneity). (c) x + y≤x + y for all vectors x, y ∈ X (triangle inequality). If only (b) and (c) hold, so that x =0does not necessarily imply x = 0,then· is called a seminorm. If (a) and (b) hold, but (c) is replaced by the weaker quasitriangle inequality x + y≤C(x + y) for some constant C ≥ 1,then·is called a quasinorm. The smallest constant C is called its quasinorm constant. S. Foucart and H. Rauhut, A Mathematical Introduction to Compressive Sensing, 515 Applied and Numerical Harmonic Analysis, DOI 10.1007/978-0-8176-4948-7, © Springer Science+Business Media New York 2013 516 A Matrix Analysis A space X endowed with a norm ·is called a normed space.