A Fast and Accurate Matrix Completion Method based on QR Decomposition and L 2,1-Norm Minimization Qing Liu, Franck Davoine, Jian Yang, Ying Cui, Jin Zhong, Fei Han To cite this version: Qing Liu, Franck Davoine, Jian Yang, Ying Cui, Jin Zhong, et al.. A Fast and Accu- rate Matrix Completion Method based on QR Decomposition and L 2,1-Norm Minimization. IEEE Transactions on Neural Networks and Learning Systems, IEEE, 2019, 30 (3), pp.803-817. 10.1109/TNNLS.2018.2851957. hal-01927616 HAL Id: hal-01927616 https://hal.archives-ouvertes.fr/hal-01927616 Submitted on 20 Nov 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Page 1 of 20 1 1 2 3 A Fast and Accurate Matrix Completion Method 4 5 based on QR Decomposition and L -Norm 6 2,1 7 8 Minimization 9 Qing Liu, Franck Davoine, Jian Yang, Member, IEEE, Ying Cui, Zhong Jin, and Fei Han 10 11 12 13 14 Abstract—Low-rank matrix completion aims to recover ma- I. INTRODUCTION 15 trices with missing entries and has attracted considerable at- tention from machine learning researchers. Most of the ex- HE problem of recovering an incomplete matrix with 16 isting methods, such as weighted nuclear-norm-minimization- missing values has recently attracted considerable atten- 17 based methods and QR-decomposition-based methods, cannot tionT from researchers in the image processing [1-10], signal 18 provide both convergence accuracy and convergence speed. To processing [11-13], and machine learning [14-20] fields. Con- 19 investigate a fast and accurate completionFor method, Peer an iterative Reviewventional methods formulate this task as a low-rank matrix QR-decomposition-based method is proposed for computing an m n 20 minimization problem. Suppose that M(M R × ,m approximate Singular Value Decomposition (CSVD-QR). This ∈ ≥ 21 method can compute the largest r(r>0) singular values of n>0) is an incomplete matrix; then, the traditional low-rank 22 a matrix by iterative QR decomposition. Then, under the frame- minimization problem is formulated as follows: 23 work of matrix tri-factorization, a CSVD-QR-based L2,1-norm minimization method (LNM-QR) is proposed for fast matrix com- min rank(X), s.t.Xi,j = Mi,j, (i, j) Ω, (1) 24 X ∈ 25 pletion. Theoretical analysis shows that this QR-decomposition- based method can obtain the same optimal solution as a nuclear m n where X R × is the considered low-rank matrix, rank(X) 26 norm minimization method, i.e., the L2,1-norm of a submatrix ∈ is the rank of X, and Ω is the set of locations corresponding 27 can converge to its nuclear norm. Consequently, an LNM-QR- 28 based iteratively reweighted L2,1-norm minimization method to the observed entries. The problem in Eq. (1) is NP- 29 (IRLNM-QR) is proposed to improve the accuracy of LNM-QR. hard and is difficult to optimize. Fortunately, the missing 30 Theoretical analysis shows that IRLNM-QR is as accurate as an values in a matrix can be accurately recovered by a nuclear iteratively reweighted nuclear norm minimization method, which 31 norm minimization under broad conditions [21, 22]. The most is much more accurate than the traditional QR-decomposition- widely used methods based on the nuclear norm are singular 32 based matrix completion methods. Experimental results obtained 33 on both synthetic and real-world visual datasets show that our value thresholding (SVT) [23] and accelerated proximal gra- 34 methods are much faster and more accurate than the state-of- dient [24]. These methods are not fast because of the high 35 the-art methods. computational cost of singular value decomposition (SVD) 36 Index Terms—Matrix Completion, QR Decomposition, Ap- iterations. Moreover, these methods are not very accurate 37 proximate SVD, Iteratively Reweighted L2,1-Norm. when recovering matrices with complex structures. One of 38 the reasons is that the nuclear norm may not be a good 39 approximation of the rank function [28] in these cases. 40 This work was supported in part by the National Natural Science Foundation To improve the accuracies of nuclear-norm-based methods, 41 of China under Grant Nos. U1713208, 61672287, 61602244, 91420201, some improved methods based on the Schatten p-norm [25, 35, 61472187, and 61572241 and in part by the Natural Science Foundation 36], weighted nuclear norm [27], γ-norm [33], and arctangent 42 of Zhejiang Province (LQ18F030014) and National Basic Research Program 43 of China under Grant No. 2014CB349303 and Innovation Foundation from rank [34], have been proposed. In 2015, F. Nie et al. [25] 44 Key Laboratory of Intelligent Perception and Systems for High-Dimensional proposed a joint Schatten p-norm and Lp-norm robust matrix Information of Ministry of Education (JYB201706). This work was also completion method. This method can obtain a better conver- 45 carried out in the framework of the Labex MS2T, program Investments for the 46 future, French ANR (Ref. ANR-11-IDEX-0004-02). (Corresponding author: gence accuracy than that of SVT. However, it may become 47 Zhong Jin.) slow when addressing large-scale matrices because of using Q. Liu is with the School of Computer Science and Engineering, Nanjing SVD in each iteration. C. Lu et al. [26] proposed an iteratively 48 University of Science and Technology, Nanjing, 210094, China and is with 49 the School of Software, Nanyang Institute of Technology, Nanyang, 473004, reweighted nuclear norm minimization (IRNN) method [26] in 50 China (e-mail: [email protected]). 2016. By using nonconvex functions to update the weights for F. Davoine is with Sorbonne Universites,´ Universite´ de technologie de the singular values, IRNN is much more accurate than SVT. 51 Compiegne,` CNRS, Heudiasyc, UMR 7253, Compiegne,` France (e-mail: 52 [email protected]). However, it still relies on SVD to obtain the singular values 53 J. Yang and Z. Jin are with the School of Computer Science and Engineer- for recovering incomplete matrices, which may cause it to be ing, Nanjing University of Science and Technology, Nanjing, 210094, China slow when applied to real-world datasets. Some other methods 54 (e-mail: [email protected]; [email protected]). 55 Y. Cui is with the College of Computer Science and Technology, Zhe- in references [33] and [34] also face the same difficulty. 56 jiang University of Technology, Hang’zhou, 310023, China (e-mail: cuiy- To improve the speed of SVT, some methods based on ma- [email protected]). trix factorization [13, 17, 18, 31] have recently been proposed. 57 F. Han is with the School of Computer Science and Communication 58 Engineering, Jiangsu University, Zhenjiang, Jiangsu, 212013, China (e-mail: In 2013, A fast tri-factorization (FTF) method [32] based on 59 [email protected]). Qatar Riyal (QR) decomposition [29, 30] was proposed. FTF 60 Page 2 of 20 2 1 2 relies on the cheaper QR decomposition as a substitute for II. RELATED WORK 3 SVD to extract the orthogonal bases of rows and columns of In this section, the SVD of a matrix and some widely 4 an incomplete matrix and applies SVD to a submatrix whose used matrix completion methods based on SVD and QR 5 size can be set in advance. FTF is very fast when applied decomposition are respectively introduced. 6 to low-rank data matrices. However, it will become slow if 7 the test matrices are not of low rank. Moreover, the FTF 8 method is not as accurate as a weighted nuclear-norm-based A. Singular Value Decomposition m n 9 method, such as IRNN [26]. A more recent work, i.e., the Suppose that X R × is an arbitrary real matrix; then, ∈ 10 robust bilinear factorization (RBF) method [18], is slightly the SVD of X is as follows: 11 more accurate than FTF. However, it is still not fast and not X = UΛV T , (2) 12 accurate enough for real applications. Some other methods 13 based on matrix factorization proposed in references [13], m n U =(u1, ,um) R × , (3) 14 [17], and [31] also have similar characteristics. Thus, the ··· ∈ n n 15 traditional methods based on the weighted nuclear norm and V =(v1, ,vn) R × , (4) ··· ∈ 16 matrix factorization cannot provide both convergence speed where U and V are column orthogonal matrices, the columns 17 and convergence accuracy. Recently, the L2,1-norm was successfully applied to feature of which are the left and right singular vectors of X, re- 18 m n spectively. Λ R × is a diagonal matrix with diagonal 19 selection [37, 38], optimal mean robust principle component ∈ For Peer Reviewentries, where Λ = σ (X), that are assumed to be in order 20 analysis [53], and low-rank representation [39-42]. The feature ii i of decreasing magnitude. σ (X) is the ith singular value of 21 selection methods and the method in [53] use a combination i X. 22 of the nuclear norm and L2,1-norm as their loss function Many papers on how to compute the singular values of 23 to extract the subspace structures of test datasets. Because X exist [48-50]. Here, we introduce a simple SVD method 24 they use the L2,1-norm, they are more robust with respect to (SVD-SIM), which was proposed by Paul Godfrey [51] in 25 outliers.