mathematics of computation, volume 28, number 125, January, 1974 Triangular Factorization and Inversion by Fast Matrix Multiplication* By James R. Bunch and John E. Hopcroft Abstract. The fast matrix multiplication algorithm by Strassen is used to obtain the triangular factorization of a permutation of any nonsingular matrix of ordern in <Cxnlos'7 operations, and, hence, the inverse of any nonsingular matrix in <Cürtlog'7 operations. 1. Introduction. Strassen [3] has given an algorithm using noncommutative multiplication which computes the product of two matrices of order 2 by 7 multi- plications and 18 additions. Then the product of two matrices of order mlk could be computed by m3lk multiplications and (5 + m)m2lk — 6(m2k)2 additions. Let an operation be a multiplication, division, addition, or subtraction. Strassen showed that the product of two matrices could be computed by < (4.7)«logl 7 opera- tions. The usual method for matrix multiplication requires 2n3 operations. Thus, (4.7)n'°g*7 < 2«3 if and only if n3-'oga7 > 2.35, i.e. if n > (2.35)5 « 100. Strassen uses block LDU factorization (Householder [2, p. 126]) recursively to compute the inverse of a matrix of order «22* by mlk divisions, g (6/5)m37* — mlk multiplications, and ;£ (6/5)(5 + m)m2lk — 7(m2k)2 additions. The inverse of a matrix of order n could then be computed by ^ (5.64)«logi'7 arithmetic operations. Let A as Axx AX2 Axx 0 1 Axx AX2 _A2X A22_ _A2X Ax 0 A 0 / where A = A22 — A2XAxx'1Al2, and / 0~ A~ = I Axx Ax2 An 0 _0 / 0 A"1 A%\ ^11 /_ Axx -- Axx Ax2 A A2XAX1 — Axx AX2 A -A^AnAÏÎ A-1 if Axx and A are nonsingular. Since the algorithm is applied recursively, it will fail whenever the inversion of a singular principal submatrix in any of the reduced matrices is required. For example, the block LDU factorization fails to exist for a matrix as simple as Received February 26, 1973. AMS (MOS) subject classifications(1970). Primary 15A06, 15A09; Secondary 65F05, 68A20. Key words and phrases. LU decomposition, Strassen's method, fast matrix inversion, linear equations, Gaussian elimination, computational complexity. * This research was supported by ONR Grant N0O014-67-A-O077-O021. Copyright © 1974, American Mathematical Society 231 License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 232 JAMES R. BUNCH AND JOHN E. HOPCROFT 0 0 0 1 0 0 10 0 10 0 _1 0 0 0_ Every principal submatrix in every reduced matrix is nonsingular if A is symmetric positive definite, strictly diagonally dominant, or irreducibly diagonally dominant (Varga [4, p. 23]). However, if A is only nonsingular, then we must, in general, pivot (i.e., interchange rows or columns) in order to obtain a (point or block) LDU factori- zation. If A is nonsingular, then there exist permutation matrices Px, P2, Qx, Q2 such that APX, QXA,Q2AP2 have (point or block) LDU factorizations or LU factori- zations (cf. Forsythe and Moler [1, p. 36]). We shall ignore lower order terms. Then the usual method for inversion requires 2«3 operations [1, pp. 77-79]. Thus, (5.64)nlog!7 < 2n3 if n > (2.82)5 Srf 180. However, one rarely needs the inverse of a matrix; rather, one usually wants to solve systems of linear equations, and here Gaussian elimination (i.e. obtaining the LU decomposition of a permutation of A) is more efficient [1, p. 79], since the LU decomposition requires fn3 operations (and solving the two triangular systems requires In2 operations). Thus, a fairer comparison is (5.64)n,og!7 < fn3 if n > 35(2.82)5 « 43,340. If A is symmetric positive definite, then one should compare with symmetric Gaussian elimination or Cholesky's method. In Sections 2 and 3, we show that, by employing pivoting, we can use Strassen's fast matrix multiplication algorithm to obtain the triangular factorization (LU de- composition) of a permutation of any nonsingular matrix of order n = lk in < (3.64)nlog*7 operations, and hence its inverse in < (ÎO.IS)«1"8'7 operations, where an operation is defined to be a multiplication, division, addition, or subtraction. In Section 4, we modify the algorithm so that we can find triangular factorizations in < (2.04)nIogl 7 operations and inverses in < (5.70)nlogs 7 operations when n = 2k. Then, for arbitrary n, we can find triangular factorizations in < (2.45)«logs 7 operations and inverses in < (6.84)/ilog!!7operations. In Section 5 we show that matrix multiplication, triangular factorization, and inversion are equivalent in computational complexity. 2. The Basic Algorithm. For simplicity, let M be of order « = 2* with det M^O- Let M° = M. We shall construct a sequence P1, P2, ■■ ■ , P"'1 of permutation matrices so that M = LUP, i.e., MP'1 = LU, where P = PlP2 ■■ ■ P""1 is a permutation matrix, L= LiL2 ■■ • Ln~ ' is unit lower triangular, U is upper triangular, and det M = (det P) det U = ± ]I"-i «•<•Since (P*')_1 = P' here, p-1 = p"-1 ... p*p\ and License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use triangular factorization and inversion 233 M'1 = P'1 V'L'1 = P" PV U-'iL"-1)-1 'L'rwr1, where (LT1 = 11- V. We define the algorithm sequentially for 1 g i ^ n — 1 as follows. Let Bi = \j : U = X, i = i^l"-1 + /t_22*"2 + • • • + Í.2' + /„20}; let \t if S 5¿ Í, t = maxjy : y*G 5,}, s = min{j : j £ #¿ ) and r = i ;' [t - 1 if S = f. Then Mir1 m;2 0 Aft. M^r1 where Mxx'~ is a nonsingular upper triangular matrix of order / — 1, MX2~ is (i - 1) X (n - i + 1), 0 is the (2r+1 - / + 1) X (i - 1) zero matrix, Af«'"1 is (« - 2r+1) X (i - 1), M22i_1 is (h - i + 1) X (n - i + 1), and M'"1 is nonsingular. Since 2r+1 — / + 1 > 0 and M'_1 is nonsingular, there exists a nonzero element in the first row of M22*-1. Hence, there exists a permutation matrix P' such that N{ m M'"1?', nu* ?¿ 0, and A7*can be partitioned as t/1 É \ F N* = G' \ H' X' r where [P is (i - T) X (i - T), V*is (i - 2') X (n - i + 2'), F and G* are 2' X 2*, F' and H* are 2' X (« - i). 0 is the (2r+1 - i + T) X (i - T) zero matrix, X¿ is (n - 2r+1) X (i - T), and Y" is (« - i - T) X (n - i + V). Further, U* and F are nonsingular upper triangular. LetZ¿ = G'iET1 and /.- /■• L' Z' /„-._ where /, is the identity matrix of order j. License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use 234 JAMES R. BUNCH AND JOHN E. HOPCROFT Define M' ■ (L'y'N'. Then if F' £* F 0 M' - f where 7' = H{ - z'f' X' r At the last step, U = M" 1 is nonsingular and upper triangular. 3. Operation Count. Finding the permutation P' requires at most n — i com- parisons, and, if Pu = 0, then the permutation involves n element interchanges. Hence, at most n(n — l)/2 comparisons and at most n'n — 1) element interchanges are required to obtain M = LU P. The computation of M"1 would require at most an additional n'n — 1) element interchanges: Let an operation be a multiplication, division, addition, or subtraction. Let M'ri), MT(ri), and IT(ri) be the number of operations required to multiply two « X « matrices, to multiply an n X « matrix by an upper triangular « X « matrix, and to invert an « X « nonsingular upper triangular matrix (we shall ignore lower order terms). Then M(l) = 1 and M(2*) = 7*+1for k ^ 1. Since Mr(2*) = 4MT(2k~1)+ 2MÍ2*"1) + 2""1 and IAX) = 2/T(2*-1) + IM Al"'1), Mri2k) = 2 Z 4,M(2*-'-1) < (yj/ and ITi2k)= 2 Z 2'Mr(2*-'-1)< (11)7*. Inverting all the U* for 1 t% i ^ n — 1 requires operations. Forming all the multipliers Z* for 1 1%i á « — 1 requires 1 2"-1 Z t}j Mt'2¡) *<Ê> i-a ¿ operations. Forming all the reduced matrices J' for 1 ^ / g n — 1 requires t-l r<n-2>')/2' + ' I2k -Í21+ 1)2'],^^MX) 22<t-u £ Af(2') z\ z 2'' -a>"§(0'<-I^G)XD-(^- Hence, for n = 2*, triangular factorization requires < (91/25)7* = (3.64)nlog'7 operations. Inverting U requires < (28/15)7* operations and U~1L~1 requires License or copyright restrictions may apply to redistribution; see https://www.ams.org/journal-terms-of-use TRIANGULAR FACTORIZATION AND INVERSION 235 ^§^-(i)^§(7)'<G2W7)'(D»(^ operations. Hence, for n = 2", inversion requires (763/75)7* < (10.18)nlogï7 operations. If M is a nonsingular matrix of order n, where 2* < n < 2*+1, then let 3TC= M © I2k+,_„. We can find the triangular factorization of a permutation of 2(11,and hence of a permutation of M, by < (91/25)7*+1 = (637/25)7* < (25.48)nlog*7opera- tions, and the inverse of 9TC,and hence of M, by < (763/75)7*+1= (5341/75)7* < (71.22)n'°g*7. 4. A Modified Algorithm. We can modify the algorithm in Section 2 so that the coefficient of nlogî7 is smaller in the operation counts of Section 3.