Faster Inversion and Other Black Box Matrix Computations Using Efficient Block Projections Wayne Eberly1, Mark Giesbrecht2, Pascal Giorgi2;4, Arne Storjohann2, Gilles Villard3 (1) Department of Computer Science, University of Calgary, Calgary, Alberta, Canada [email protected] (2) David R. Cheriton School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada [email protected], [email protected], [email protected] (3) CNRS, LIP, Ecole´ Normale Superieure´ de Lyon, Lyon, France [email protected] (4) IUT — Universite´ de Perpignan, Perpignan, France [email protected] ABSTRACT rank, are obtained at the same cost. An application of this E±cient block projections of non-singular matrices have re- technique to Kaltofen and Villard's Baby-Steps/Giant-Steps cently been used by the authors in [10] to obtain an e±cient algorithms for the determinant and Smith Form of an inte- ger matrix is also sketched, yielding algorithms requiring algorithm to ¯nd rational solutions for sparse systems of lin- 2:66 ear equations. In particular a bound of O~(n2:5) machine op- O~(n ) machine operations. More general bounds involv- erations is presented for this computation assuming that the ing the number of black-box matrix operations to be used input matrix can be multiplied by a vector with constant- are also obtained. sized entries using O~(n) machine operations. Somewhat The derived algorithms are all probabilistic of the Las Ve- more general bounds for black-box matrix computations are gas type. They are assumed to be able to generate random also derived. Unfortunately, the correctness of this algo- elements | bits or ¯eld elements | at unit cost, and always rithm depends on the existence of e±cient block projections output the correct answer in the expected time given. of non-singular matrices, and this was only conjectured. In this paper we establish the correctness of the algorithm Categories and Subject Descriptors from [10] by proving the existence of e±cient block pro- I.1.2 [Symbolic and Algebraic Manipulation]: Algo- jections for arbitrary non-singular matrices over su±ciently rithms|Algebraic algorithms, analysis of algorithms large ¯elds. We further demonstrate the usefulness of these projections by incorporating them into existing black-box matrix algorithms to derive improved bounds for the cost General Terms of several matrix problems. We consider, in particular, ma- Algorithms trices that can be multiplied by a vector using O~(n) ¯eld operations: We show how to compute the inverse of any such non-singular matrix over any ¯eld using an expected Keywords 2:27 number of O~(n ) operations in that ¯eld. A basis for Sparse integer matrix, structured integer matrix, linear sys- the null space of such a matrix, and a certi¯cation of its tem solving, black box linear algebra ¤This material is based on work supported in part by the French National Research Agency (ANR Gecko, Villard), 1. INTRODUCTION and by the Natural Sciences and Engineering Research Council (NSERC) of Canada (Eberly, Giesbrecht, Storjo- In our paper [10] we presented an algorithm which pur- hann). portedly solved a sparse system of rational equations consid- erably more e±ciently than standard linear equations solv- ing. Unfortunately, its e®ectiveness in all cases was conjec- tural, even as its complexity and actual performance were Permission to make digital or hard copies of all or part of this work for very appealing. This e®ectiveness relied on a conjecture re- personal or classroom use is granted without fee provided that copies are garding the existence of so-called e±cient block projections. £ not made or distributed for profit or commercial advantage and that copies Given a matrix A Fn n over any ¯eld F, these projections bear this notice and the full citation on the first page. To copy otherwise, to should be block vectors2 u Fn£s (where s is a blocking republish, to post on servers or to redistribute to lists, requires prior specific factor dividing n, so n = ms2) such that we can compute uv permission and/or a fee. £ or vtu quickly for any v Fn s, and such that the sequence ISSAC'07,July29–August1,2007,Waterloo,Ontario,Canada. m2¡1 Copyright 2007 ACM 978-1-59593-743-8/07/0007 ...$5.00. of vectors u, Au,..., A u has rank n. In this paper, we 143 prove the existence of a class of such e±cient block projec- lel in [12, 13], and this is further developed in [8, 18]. See tions for non-singular n n matrices over su±ciently large the references in these papers for a more complete history. ¯elds; we require that the£size of the ¯eld F exceed n(n + 1). For sparse systems over a ¯eld, the seminal work is that This can be used to establish a variety of results concern- of Wiedemann [22] who shows how to solve sparse n n ing matrices A Zn£n with e±cient matrix-vector prod- systems over a ¯eld with O(n) matrix-vector products £and ucts | in particular,2 such that a matrix-vector product O(n2) other operations. This research is further developed Ax mod p can be computed for a given integer vector x and a in [4, 16, 17] and many other works. The bit complexity of small (word-sized) prime p using O~(n) bit operations. Such similar operations for various families of structured matrices matrices include all \sparse" matrices having O(n) nonzero is examined by Emiris and Pan [11]. entries, assuming these are appropriately represented. They also include a variety of \structured" matrices, having con- 2. EFFICIENT BLOCK PROJECTIONS stant \displacement rank" (for one de¯nition of displace- For now we will consider an arbitrary invertible matrix ment rank or another) studied in the recent literature. n£n £ A F over a ¯eld F, and s an integer, the blocking factor, In particular, our existence result implies that if A Zn n 2 2 that divides n exactly. Let m = n=s. For a so-called block is non-singular and has an e±cient matrix-vector product n£s projection u F and 1 k m, we denote by k(A; u) then the Las Vegas algorithm for system solving given in the block Krylov2 matrix [u;·Au;·: : : ; Ak¡1u] Fn£Kks. We [10] can be used to solve a system Ax = b for a given integer n£n 2 wish to show that m(A; u) F is non-singular for a vector b using an expected number of matrix-vector products K 2 1:5 particularly simple and sparse u, assuming some properties modulo a word-sized prime that is O~(n log( A + b )) of A. together with an expected number of additionalk bitk opkera-k 2:5 Our factorization uses the special projection (which we tions that is O~(n log( A + b )). If A has an e±cient will refer to as an e±cient block projection) matrix-vector product thenk thek totalk k expected number of bit operations used by this algorithm is less than that used by Is £ any previously known algorithm, at least when \standard" u = 2 . 3 Fn s; (2.1) (i.e., cubic) matrix arithmetic is used. 2 6 I 7 Consider, for example, the case when the cost of a matrix- 4 s 5 vector product by A modulo a word-sized prime is O~(n) which is comprised of m copies of Is and thus has exactly operations, and the entries in A are constant size. The cost n non-zero entries. We suggest a similar projection in [10] 2:5 of our algorithm will be O~(n ) bit operations. This im- without proof of its reliability (i.e., that the corresponding proves upon the p-adic lifting method of Dixon [6], which block Krylov matrix is non-singular). We establish here that 3 requires O~(n ) bit operations for sparse or dense matrices. it does yield a block Krylov matrix of full rank, and hence This theoretical e±ciency was reected in practice in [10] at can be used for an e±cient inverse of a sparse A. least for large matrices. Let = diag(±1; : : : ; ±1; ±2; : : : ; ±2; : : : ; ±m; : : : ; ±m) be an We present several other rather surprising applications of n n Ddiagonal matrix whose entries consist of m distinct £ this technique. Each incorporates the technique into an ex- indeterminates ±i, each ±i occurring s times. isting algorithm in order to reduce the asymptotic complex- Theorem ity for the matrix problem to be solved. In particular, given 2.1. If the leading ks ks minor of A is non- £ F n£n Fn£n F zero, for 1 k m, then m( A ; u) [±1; : : : ; ±m] a matrix A over an arbitrary ¯eld , we are able to · · K D D 2 compute the2complete inverse of A with O~(n3¡1=(!¡1)) op- is non-singular. 2¡1=(!¡1) Proof. erations in F plus O~(n ) matrix-vector products by Let = A . For 1 k m, de¯ne k as B D D · · B A. Here ! is such that we can multiply two n n matrices the specialization of obtained by setting ±k+1; ±k+2; : : : ; ±m ! £ B with O(n ) operations in F. Standard matrix multiplication to zero. Thus k is the matrix constructed by setting to B gives ! = 3, while the best known matrix multiplication of zero the last n ks rows and columns of . Similarly, for ¡ Fn£s B Coppersmith and Winograd [5] has ! = 2:376. If again we 1 k m we de¯ne uk to be the matrix constructed · · 2 can compute v Av with O~(n) operations in F, this implies from u by setting to zero the last n ks rows.