Computing RankRevealing QR Factorizations of Dense Matrices Christian H Bischof Mathematics and Computer Science Division Argonne National Lab oratory S Cass Ave Argonne IL bischofmcsanlgov and Gregorio QuintanaOrt Departamento de InformaticaUniversidad Jaime I Campus Penyeta Roja Castellon Spain gquintaninfujies We develop algorithms and implementations for computing rankrevealing QR RRQR factor izations of dense matrices First we develop an ecient blo ck algorithm for approximating an RRQR factorization employing a windowed version of the commonly used Golub pivoting strategy safeguarded by incremental condition estimation Second we develop eciently implementable variants of guaranteed reliable RRQR algorithms for triangular matrices originally suggested by Chandrasekaran and Ipsen and by Pan and Tang We suggest algorithmic improvements with re sp ect to condition estimation termination criteria and Givens up dating By combining the blo ck algorithm with one of the triangular p ostpro cessing steps we arrive at an ecient and reliable algorithm for computing an RRQR factorization of a dense matrix Exp erimental results on IBM RS and SGI R platforms show that this approach p erforms up to three times faster than the less reliable QR factorization with column pivoting as it is currently implemented in LA PACK and comes within of the p erformance of the LAPACK blo ck algorithm for computing a QR factorization without any column exchanges Thus we exp ect this routine to b e useful in many circumstances where numerical rank deciency cannot b e ruled out but currently has b een ignored b ecause of the computational cost of dealing with it Categories and Sub ject Descriptors G Numerical Analysis Numerical Linear Algebra G Mathematical Software Additional Key Words and Phrases rankrevealing orthogonal factorization numerical rank blo ck algorithm QR factorization leastsquares systems This work was supp orted by the Applied and Computational Mathematics Program Advanced Research Pro jects Agency under contracts DME and P Bischof was also supp orted by the Mathematical Information and Computational Sciences Divi sion subprogram of the Oce of Computational and Technology Research U S Department of Energy under Contract WEng Quintana also received supp ort through the Europ ean ESPRIT Pro ject GEPPCOM and the Spanish Research Agency CICYT under grant TICC During part of this work Quintana was a research fellow of the Spanish Ministry of Education and Science of the Valencian Government at the Universidad Politecnicade Valencia and a visiting scientist at the Mathematics and Computer Science Division at Argonne National Lab oratory 2 INTRODUCTION We briey summarize the prop erties of a rankrevealing QR RRQR factorization Let A b e an m n matrix wlog m n with singular values n and dene the numerical rank r of A with resp ect to a threshold as follows r r Also let A have a QR factorization of the form R R AP QR Q R where P is a p ermutation matrix Q has orthonormal columns R is upp er triangu lar and R is of order r Further let A denote the twonorm condition number of a matrix A We then say that is an RRQR factorization of A if the following prop erties are satised R and kR k R r max r Whenever there is a welldetermined gap in the singularvalue sp ectrum b etween r and and hence the numerical rank r is well dened the RRQR factorization r reveals the numerical rank of A by having a wellconditioned leading submatrix R and a trailing submatrix R of small norm We also note that the matrix R R T P I which can b e easily computed from is usually a go o d approximation of the nullvectors and a few steps of subspace iteration suce to compute nullvectors that are correct to working precision Chan and Hansen The RRQR factorization is a valuable to ol in numerical linear algebra b ecause it provides accurate information ab out rank and numerical nullspace Its main use arises in the solution of rankdecient leastsquares problems for example in geo desy Golub et al computeraided design Grandine nonlinear leastsquares problems More the solution of integral equations Elden and Schreiber and the calculation of splines Grandine Other applications arise in b eam forming Bischof and Shro sp ectral estimation Hsieh et al and regularization Hansen Hansen et al Walden Stewart suggested another alternative to the singular value decomp osition a complete orthogonal decomp osition called URV decomp osition This factorization decomp oses R R T V A U R where U and V are orthogonal and b oth kR k and kR k are of the order r In particular compared with RRQR factorizations URV decomp ositions employ a general orthogonal matrix V instead of the p ermutation matrix P URV decomp o sitions are more exp ensive to compute but they are well suited for nullspace up dat ing RRQR factorizations on the other hand are more suited for the leastsquares 3 setting since one need not store the orthogonal matrix V the other orthogonal matrix is usually applied to the righthand side on the y Of course RRQR factorizations can b e used to compute an initial URV decomp osition where U Q and V P We briey review the history of RRQR algorithms From the interlacing theorem for singular values Golub and Loan Corollary we have R k k A and Rk n k n A min k max k Hence to satisfy condition we need to pursue two tasks Task Find a p ermutation P that maximizes R min Task Find a p ermutation P that minimizes R max Golub suggested what is commonly called the QR factorization with column pivoting Given a set of already selected columns this algorithm chooses as the next pivot column the one that is farthest away in the Euclidean norm from the subspace spanned by the columns already chosen Golub and Loan p P This intuitive strategy addresses task While this greedy algorithm is known to fail on the socalled Kahan matri ces Golub and Loan p Example it works well in practice and forms the basis of the LINPACK Dongarra et al and LAPACK Anderson et al a Anderson et al b implementations Recently QuintanaOrtSun and Bischof developed an implementation of the Golub algorithm that allows half of the work to b e p erformed with BLAS kernels Bischof also had developed restrictedpivoting variants of the Golub strategy to enable the use of BLAS type kernels Bischof for almost all of the work and to reduce communication cost on distributedmemory machines Bischof One approach to task is based in essence on the following fact which is proved in Chan and Hansen W nn np Lemma For any R IR and any W IR with a nonsingular W pp W IR we have kRn p n n p nk kRW k kW k This means that if we can determine a matrix W with p linearly indep endent columns all of which lie approximately in the nullspace of R ie kRW k is small and if W is well conditioned such that W kW k is not large we min are guaranteed that the elements of the b ottom right p p blo ck of R will b e small Algorithms based on computing wellconditioned nullspace bases for A include these by Golub Klema and Stewart Chan and Foster Other algorithms addressing task are these by Stewart and Gragg and Stewart Algorithms addressing task include those of Chan and Hansen and Golub Klema and Stewart In fact the latter achieves b oth task and task and therefore reveals the rank but it is to o exp ensive in comparison with the others Bischof and Hansen combined a restrictedpivoting strategy with Chans algo rithm Chan to arrive at an algorithm for sparse matrices Bischof and Hansen 4 and also developed a blo ck variant of Chans algorithm Bischof and Hansen A Fortran implementation of Chans algorithm was provided by Reichel and Gragg Chans algorithm Chan guaranteed i p R i i min i ni nn i and p ni nn i Ri n i n i max i That is as long as the rank of the matrix is close to n the algorithm is guaranteed to pro duce reliable b ounds but reliability may decrease with the rank of the matrix Hong and Pan then showed that there exists a p ermutation matrix P such that for the triangular factor R partitioned as in we have jjR jj Ap r n r and R A min r p r n where p and p are loworder p olynomials in n and r versus an exp onential factor in Chans algorithm Chandrasekaran and Ipsen were the rst to develop RRQR algorithms that satisfy and Their pap er also reviews and provides a common framework for the previously devised strategies In particular they introduce the socalled unication principle which says that running a task algorithm on the rows of the inverse of the matrix yields a task algorithm They suggest hybrid algorithms that alternate b etween task and task steps to rene the separation of the singular values of R Pan and Tang and Gu and Eisenstat presented dierent classes of algorithms for achieving and addressing the p ossibility of nontermination of the algorithms b ecause of oatingp oint inaccuracies The goal of our work was to develop an ecient and reliable RRQR algorithm and implementation suitable for inclusion in a numerical library such as LAPACK Sp ecically we wished to develop an implementation that was b oth reliable and close in p erformance to the QR factorization without any pivoting Such an imple mentation would provide algorithm developers with an ecient to ol for addressing p otential numerical rank deciency by minimizing the computational p enalty for addressing potential rank deciency Our strategy involves