arXiv:1607.01404v2 [cs.MS] 24 Jan 2017 n olret(mlet iglrvle r eerda ags (sm largest compu be as always referred can (SVD) are Decomposition values Value Singular singular The (smallest) triplets. largest to ing matrices oun.( columns. hr h iglrvle r aee nacnigorder, ascending in labeled are values singular the where feteesnua ausadcrepniglf n ih singular right and left matrix corresponding sparse and values singular extreme of (1.1) n ipa reua rso ovrec ncsso lsee spec clustered of implementation research cases MATLAB in in available convergence only slow are they or portantly, irregular w display challenge and this mode address with [ to problems methods attempt iterative for efforts even research methods, recent iterative c speed Many of the the to accuracy particular, both the In challenges to memory. presents triplets limited singular under ite smallest triplets the decades, singular two few last a the [ pute developed Over been have methods. SVD iterative the of [ use inverse the matrix sitate the m of a trace [ the of problems computing rank squares for least and e.g., total space, mechanism, the null for co range, learning as the such machine applications determining s algebra and linear few in a dospectra, role Calculating important tool. an plays filtering triplets important appr an matrix as recognition low-rank pattern a computing for [ sing learning largest machine the of in [ few role a of critical computation a The areas. engineering and pa good its and scaling. software experimen state-of-the-art numerical other The to compared preconditioning. f with PRIMME especially package PRIMME eigensolver state-of-the-art values. the pr PRIMME on library, o we based number high-performance Recently, small a a present compute we problems. search, accurately and large effectively to required th method Furthermore, the address triplets. by singular cannot plagued still smallest are computing methods for iterative metho scale iterative large in in interest vances an rekindled has problems value 75 ...([email protected] lyc.meu andreas [email protected], ([email protected], U.S.A. 8795, 10 43 V OVRFRACRT AG-CL COMPUTATIONS LARGE-SCALE ACCURATE FOR SOLVER SVD PRIMME ,i ttsisfrpicplcmoetaayi [ analysis component principal for statistics in ], , ∗ Abstract. o ag cl rbes eoydmnsadcmuainlcomp computational and demands memory problems, scale large For variou in kernels computational used widely most the of one is SVD The Introduction. 1. eateto optrSine olg fWlimadMary and William of College Science, Computer of Department 46 , 8 U n h ula om[ norm nuclear the and ] σ i [ = u , IGE WU LINGFEI h nraignme fapiain eurn h soluti the requiring applications of number increasing The A i VS IHPROMNEPRECONDITIONED HIGH-PERFORMANCE A SVDS: u v , VSfil a npouto ee otaefrcmuigth computing for software level production in gap a fills SVDS ∈ 1 u , . . . , i scle iglrtiltof triplet singular a called is ) 3 R , m 2 × , n 23 ecnie h rbe ffidn ml number, small a finding of problem the consider We n ] ( , LYROMERO ELOY , ∗ m ∈ 24 ≥ R , 12 20 m n × .Teesnua ausadvcossatisfy, vectors and values singular These ). , .Teemtossillc h eesr robustness necessary the lack still methods These ]. n 36 37 and Av , ,i aamnn o aetsmni indexing semantic latent for mining data in ], 26 i , V = 9 1 , NRA STATHOPOULOS ANDREAS , ∗ σ [ = 11 i u , i v , 3 1 VS htipeet u yrdmethod hybrid our implements that SVDS, , A sfrteSD oepoiigrcn ad- recent promising Some SVD. the for ds v , . . . , lwcnegneadacrc limitations accuracy and convergence slow i urn mlmnain nMATLAB in implementations current eir allpromneudrsrn n weak and strong under performance rallel 2 sdmntaeisspro performance superior its demonstrate ts h iglrtilt correspond- triplets singular The . @cs.wm.edu) , 25 23 rbt ags n mletsingular smallest and largest both or xrm iglrtilt.I hsre- this In triplets. singular extreme f sne rcniind two-stage preconditioned, a esented ,adi inlpoesn and processing signal in and ], 13 , n 24 ] ,ada eea deflation general a as and ], ilasug igna23187- Virginia Williamsburg, , ∈ , σ 20 1 R ≤ , n 45 × no ag cl singular scale large of on 44 σ n t e rmodified or new ith oeetvl com- effectively to ] 2 fcnegneand convergence of aiemtosfor methods rative ]. aeorthonormal have aeconditioning. rate lrtilt plays triplets ular ≤ etr falarge a of vectors ti [ atrix als singular mallest let singular allest) r.Ms im- Most tra. . . . ptn pseu- mputing muainof omputation ∗ htcannot that s ata SVD, partial e eiyneces- lexity e [ ted ≤ 14 scientific s oximation σ , n 12 k 42 The . ≪ ]. ,in ], n , solve large-scale problems. In our previous work [45], we presented a hybrid, two-stage method that achieves both efficiency and accuracy for both largest and smallest singu- lar values under limited memory. In addition, the method can take full advantage of preconditioning to significantly accelerate the computation, which is the key towards large-scale SVD computation for certain real-world problems. Our previous results showed substantial improvements in efficiency and robustness over other methods. With all this algorithmic activity it is surprising that there is a lack of good quality software for computing the partial SVD, especially with preconditioning. SVDPACK [7] and PROPACK [27] can efficiently compute largest singular triplets but they are either single-threaded or only support shared-memory parallel computing. In addi- tion, they are slow and unreliable for computing the smallest singular triplets and cannot directly take advantage of preconditioning. SLEPc offers a more robust imple- mentation of the thick restarted Lanczos bidiagonalization method (LBD) [18] which can be used in both distributed and shared memory environments. SLEPc also of- fers some limited functionality for computing the partial SVD through its eigensolvers [19]. However, neither approach is tuned for accurate computation of smallest singular triplets, and only the eigensolver approach can use preconditioning which limits the desired accuracy or the practical efficiency of the package. There is a clear need for high quality, high performance SVD software that addresses the above shortcomings of current software and that provides a flexible interface suitable for both black-box and advanced usage. In this work we address this need by developing a high quality SVD library, PRIMME SVDS, based on the state-of-the-art package PRIMME (PReconditioned Iterative MultiMethod Eigensolver [40]). We highlight three main contributions: We present the modifications and functionality extensions of PRIMME that • are required to implement our two stage PHSVDS method [45] so that it achieves full accuracy and high-performance, for both smallest and largest singular values and with or without preconditioning. We provide intuitive user interfaces in C, MATLAB, Python, and R for both • ordinary and advanced users to fully exploit the power of PRIMME SVDS. We also discuss distributed and shared memory interfaces. We demonstrate with numerical experiments on large scale matrices that • PRIMME SVDS is more efficient and significantly more robust than the two most widely used software packages, PROPACK and SLEPc, even without a preconditioner. We also demonstrate its good parallel scalability. Since this is a software paper, the presentation of our method is given in a more abstract way that relates to the implementation. Further theoretical and algorithmic details can be found in [45]. 2. Related work. The partial SVD problem can be solved as a Hermitian eigen- value problem or directly using bidiagonalization methods [14]. We introduce these approaches and discuss the resulting state-of-the-art SVD methods. Next, we review the state-of-practice software, both eigensolver and dedicated SVD software. 2.1. Approaches and Methods for Computing SVD Problems. The first T n n approach computes eigenpairs of the normal equations matrix C = A A C × T m m ∈ (or AA C × if m
PRIMME SVDS: a High-Performance Preconditioned SVD Solver For
![PRIMME SVDS: a High-Performance Preconditioned SVD Solver For](http://data.docslib.org/img/6b8acfcd8c98da389c8d0b3509afd486-1.webp)