Vol. 45 No. 6 IPSJ Journal June 2004 Regular Paper A Practical Implementation of Modular Algorithms for Frobenius Normal Forms of Rational Matrices
Shuichi Moritsugu†
Modular algorithms for computing the Frobenius normal forms of integer and rational ma- trices are presented and their implementation is reported. These methods compute Frobenius normal forms over Zpi , where pi’s are distinct primes, and then construct the normal forms over Z or Q by the Chinese remainder theorem. Our implementation includes: (1) detection of unlucky primes, (2) a new formula for the efficient computation of a transformation ma- trix, and (3) extension of our preceding algorithm over Z to one over Q. Through experiments using a number of test matrices, we confirm that our modular algorithm is more efficient in practical terms than the straightforward implementation of conventional methods.
• the structure of eigenvectors or generalized 1. Introduction eigenvectors. In a previous paper19), we studied a mod- In addition, algorithms for symbolic computa- ular algorithm for the exact computation of tion of Jordan normal forms can be realized by Frobenius normal forms of integer matrices, and transforming the Frobenius normal forms9),20). substantiated its efficiency by computational Therefore, Frobenius normal forms are the most experiments. In this paper, we extend the suitable for constructing linear algebra algo- above algorithm to the case of rational matri- rithms by means of computer algebra systems. ces. We propose several algorithms and com- They also have important applications such as pare their efficiency and effectiveness through the eigenvalue method for solving systems of experiments. algebraic equations22),23),30). The motivation for these studies is the fact Algorithms for Frobenius normal forms have that computing various types of normal forms been discussed in a number of papers, but these of matrices is a very important subject in sym- have been concerned mainly with the theoreti- bolic linear algebra. In particular, the Jordan cal computational complexity. As an improve- normal form is theoretically fundamental, but it ment over the classical O(n4) algorithm16),26), requires an algebraic extension field to which all the O(n3) algorithm was proposed in Storjo- the eigenvalues belong. Hence, we have to han- hann 29), and a series of algorithms by Gies- dle algebraic numbers efficiently in computer brecht6)∼8) are known to give the best complex- algebra systems, but this is often so difficult ity. Nevertheless, implementations of such algo- that computing Jordan normal forms becomes rithms have rarely been reported, and Maple2) generally inefficient. is the only existing commercial computer alge- In contrast, the Frobenius normal form is bra system to provide a function for Frobenius computed only by rational operations of the normal forms. matrix elements without algebraic extensions, The purpose of this paper is to describe the and it is absolutely unique, while the Jordan implementation of computing normal forms and normal form is unique up to the permutation of transformation matrices, comparing algorithms Jordan blocks. Moreover, the Frobenius normal from a practical point of view. The algorithm form provides the same detailed information on adopted here belongs to the O(n3 log n) class1) the matrix as the Jordan normal form; for ex- and is not necessarily optimal. However, in- ample24): stead of precise analysis of the complexity, we • the characteristic polynomial, focus on the practical application of modular • the minimal polynomial, methods for the computation of normal forms. • the algebraic and geometric multiplicities The basic idea of the modular algorithm of the eigenvalues, and was first explained by Howell12), where only Frobenius normal forms are computed, with- † Institute of Library and Information Science, Uni- out transformation matrices. In Mathieu and versity of Tsukuba Ford18) and Mukhopadhyay25), p-adic methods
1630 Vol. 45 No. 6 Implementation of Modular Algorithms for Frobenius Normal Forms 1631 for the computation of characteristic polyno- 01 mials were proposed, in which only block di- .. agonalized forms or upper triangular forms are 00 . O computed. The latest paper by Giesbrecht and C = . . . . (1) ...... Storjohann8) also describes a modular method 00··· 01 and an algorithm for a transformation ma- c c ··· c c trix. The detection of unlucky primes is treated 0 1 n−2 n−1 in both Howell12) and Giesbrecht and Storjo- is called the companion matrix associated with 8) n n−1 hann , but since the notions in those stud- the polynomial f(x)=x − cn−1x −···− ies are slightly different, the procedure is eluci- c1x − c0. In particular, the companion matrix dated in this paper from the viewpoint of actual associated with the polynomial of degree one implementation. f(x)=x − c0 is the 1 × 1 square matrix [c0]. As another approach for Frobenius normal Lemma 2. The characteristic polynomial forms, we previously proposed a fraction-free ϕC (x) and the minimal polynomial φC (x) of the algorithm21) for polynomial matrices. This al- companion matrix C are equal to the associated n n−1 gorithm is also applicable to integer matrices, polynomial f(x)=x −cn−1x −· · ·−c1x−c0. but it does not necessarily yield a dramatic im- provement in efficiency. In this paper, there- Theorem 3 (Frobenius Normal Form). fore, we apply modular methods to integer and (i) Using a suitable regular matrix S, every rational matrices, and focus particularly on the n × n square matrix A can be transformed following: into a block diagonal matrix as follows: −1 • detection of unlucky primes and its imple- F = S AS = C1 ⊕ C2 ⊕···⊕Ct. (2) mentation, This is called the Frobenius normal form (or • efficient computation of transformation rational normal form) of A. Each block ma- matrices with simple elements, and trix Ci (i =1, ···,t)isadi × di compan- • recovery of rational numbers and algo- ion matrix in the form of Eq.(1), and the rithms for rational matrices. associated polynomial ϕi+1(x)ofCi+1 di- The implementation for integer matrices was vides the associated polynomial ϕi(x)ofCi reported in our previous paper19). Theories (i =1,...,t− 1). necessary for both integer matrices and ratio- (ii) For every given matrix A, its Frobenius nal matrices are summarized here in sections 2 normal form F of Eq.(2) is uniquely deter- and 3. We show an extension of the method mined, while a transformation matrix S is to rational matrices in section 4, and discuss not unique. Furthermore, the minimal poly- its efficiency with reference to the experimental nomial of A is given by φA(x)=ϕ1(x), and results in section 5. Most other papers discuss the characteristic polynomial of A is given by only integer matrices, because it is theoretically ϕA(x)=ϕ1(x) · ϕ2(x) ···ϕt(x). sufficient to extract the common denominator For the computation of characteristic poly- of rational matrices. However, our results show nomials, there is a similar algorithm using ma- that such an approach is quite inefficient in ac- trix block diagonalization, as in Eq.(2), known tual implementation. as Danilevskii’s method3),4), where the condi- tion ϕt(x) | ϕt− (x) |···|ϕ (x) is not required. 2. Frobenius Normal Forms of Matri- 1 1 Hence, the result given by Danilevskii’s method ces is not unique and ϕ1(x) is not necessarily iden- In the following sections, we assume that the tical to the minimal polynomial of A. elements of a given matrix are rational num- The normal form in the strict sense of bers; that is, A =[aij] ,aij ∈ Q. Refer to text- Theorem 3 is deduced by the constructive books by Iri and Kan13) and Kan and Iri14) for proof of the “cyclic decomposition theorem details of the normal forms theory. of a finite-dimensional vector space”5),11),13). Definition 1 (Companion Matrix). The computational complexity is analyzed as The following n × n square matrix O(n3 log n)1), where O(n3) corresponds to basic matrix operations such as multiplication, and O(log n) is interpreted as the number of com- panion blocks t in the normal form of Eq.(2). However, instead of the “constructive proof” 1632 IPSJ Journal June 2004 above, this paper adopts the algorithm ex- adopt the form in Definition 1 for the symbolic pressed by the composition of elementary trans- formulation of eigenvectors. Let S−1AS = C formations14), which is the conversion by the in Eq.(1), and let λ be a root of the associated authors of 13) themselves. To our knowledge, polynomial f(x)ofC. If we put there is nothing in the English literature that n− T u := 1,λ,λ2,...,λ 1 , gives such an algorithm explicitly, even though the formulation in Iri and Kan13) is essentially then we have Cu = λu, A(Su)=λ(Su). identical to that in Hoffman and Kunze11). These relations give the symbolic expression of According to the formulation in Kan and an eigenvector of C and A with the eigenvalue Iri14), we eliminate the elements step by λ. (Hence, we do not compute S−1 in the actual step analogously to the Gaussian elimination implementation.) method, and finally obtain the normal form F Remark 2. If all the elements of A are in- as well as a transformation matrix S. The num- tegers, then its Frobenius normal form F con- ber of loop counts is not stable, but depends on sists only of integers. On the other hand, we can the arrangement of the elements in the original take the transformation matrices S and S−1 so matrix A. However, this algorithm, whose com- that only one of them is integral, even though plexity is also considered to be O(n3 log n), is A is rational. There exists no known algorithm certainly deterministic. In order to compute a that yields both S and S−1 as integer matrices. similar transformation A → S−1AS, we apply the following elementary transformations suc- 3. Modular Algorithm for Integer Ma- cessively: trices Definition 4 (Elementary Transforma- tions). The following three operations are In this section, we summarize the modular called the elementary transformations. algorithm for integer matrices given in our pre- op1(k, l): Exchange the k-th row and the - vious paper19). We assume that the straightfor- th row of A, and then exchange the k-th ward program based on procedure (3) over Q is column and the -th column of A. already implemented. We propose an algorithm op2(k, c): Multiply the k-th row of A by c−1, that consists of the following two steps: and then multiply the k-th column of A by (i) compute the normal form F s.t. AS = c. SF, using modular arithmetic, and op3(k, l, c): Add the -th row multiplied by c (ii) compute a transformation matrix S with to the k-th row of A, and then subtract the simpler integer elements. k-th column multiplied by c from the -th 3.1 Chinese Remainder Theorem column of A. The elementary transformation op2(k, c) in- In these transformations, the operations on cludes division by c, but this step is per- −1 rows correspond to the transformation matrix formed in Zp as the multiplication by s ≡ c S−1, and the operations on columns correspond (mod p), where cs + pt = 1 from the Euclidean to S. When we apply suitable transformations algorithm. Therefore, every rational operation ···S−1 S−1 S−1AS S S ··· successively 3 2 1 1 2 3 , in the elementary transformations in Definition we finally obtain F, S, S−1 in Eq.(2). This pro- 4 can be executed modulo a prime p, and the cedure is carried out by performing only ratio- same procedure as over Q yields the Frobenius nal operations in Q, and can be realized by normal form over Zp such that ApSp = SpFp. preparing two unit matrices with the same size In order to recover F over Z from several im- F ,F ,... for A, as follows: ages p1 p2 , we apply the Chinese remain- der theorem (CRT). Since the elements of S in E A E −→ S−1 F S (3) procedure (3) generally produce much longer Remark 1. In some of the literature, in- integers than those of F , recovery of S by the cluding Kan and Iri14), a companion matrix is CRT is very inefficient. Hence, we abandon this S defined as the transposed form of Eq.(1), and procedure and do not reserve each pi . the Frobenius normal form as the transposed Let us denote by F (k) the normal form F form of Eq.(2). Both definitions are related by (mod p1 ···pk). Applying the Newtonian solu- F T = S−1AT S ⇐⇒ F = ST AS−T ; hence, tion to the CRT (Theorem 9 in the appendix), they are essentially identical. However, we we compute F (1) → F (2) → ··· for the moduli Vol. 45 No. 6 Implementation of Modular Algorithms for Frobenius Normal Forms 1633 p1,p1p2,... successively. call the prime number p “strictly unlucky.” In the lifting procedure, when we obtain In this definition, we do not care about trans- (k−1) (k) F = F for (mod p1 ···pk−1) and (mod formation matrices S and Sp. It is obvious that p1 ···pk−1pk), we check the condition ϕ(A)=0 F ≡ Fp (mod p) ⇒ S ≡ Sp (mod p); how- over Z, where ϕ(x) is the minimal polynomial ever, if we have F ≡ Fp (mod p) and S ≡ Sp of F (k). Since the minimal polynomial of A is (mod p), then we do not regard p as strictly unique, if we have the above condition, F (k) unlucky. In order to detect such strictly un- coincides with the normal form over Z. lucky primes, instead of the pivoting pattern, We can also estimate the bound for the ele- the block structure and their sizes (d1,...,dt) ments of F and compute Fp’s using a sufficient are noted in the normal form F = C1 ⊕···⊕Ct. number of primes to recover the integer8).How- Lemma 8 (Giesbrecht6),8)). If and only ever, we selected the above incremental formula if we have lexicographically so that the minimum necessary number of Fp’s (p) (p) (d ,...,dt) (d ,...,d ), would be computed in the framework of sequen- 1 1 t tial computation. We should also note that it then p is strictly unlucky. is difficult to extend the bound to the case of According to this lemma, we can apply the rational matrices. CRT using every Fp that has the same block 3.2 Detecting and Avoiding Unlucky structure as F ,evenifS ≡ Sp (mod p). The Primes existence of such p’s is discussed later with ref- In this section, we discuss the criterion for de- erence to an example. tecting unlucky primes, arranging various defi- Nevertheless, we implemented the history of nitions used in several previous studies. First, pivoting on the basis of the criterion in Lemma we introduce the following broad definition: 6, because we also implemented the recovery Definition 5 (Unlucky Prime). of S by means of the CRT, using Sp’s for the Let AS = SF be the normal form and a comparison of algorithms19). If we recover only transformation matrix computed over Z, and F by means of the CRT, then we can adopt the ApSp = SpFp be those computed over Zp.Ifwe criterion of Lemma 8, but the program has not have F ≡ Fp (mod p)orS ≡ Sp (mod p), yet been optimized. then we call the prime number p “unlucky.” The difficulty in detecting unlucky primes lies The following lemma shows that unlucky in the fact that neither the correct pivoting primes (in the sense of Definition 5) are de- pattern over Z nor the block structure in F tected by recording and comparing the history is known beforehand. In our implementation, of pivoting in the elimination process. we presume the correct pattern by a majority Lemma 6 (Howell12)). The pivoting decision, comparing the results for first three pattern for a lucky prime p is identical to that primes p1, p2, and p3. As explained in Morit- over Z. sugu19), this presumption is correct with very If an unlucky prime p divides the pivot ele- high probability if we use prime numbers of ap- ment at some step and the pivot is reduced to propriate magnitude; for example, word size. 0, we have to apply op1(k, ) to choose a non- Cases where the presumption is incorrect will zero pivot, but such an exchange of rows and rarely occur unless we purposely use very small columns does not occur in the computation over primes; for example, those with values less than Z. Hence, we discard the result over such mod- 500. Therefore, the detection of unlucky primes ulus p, and apply the CRT to pi’s that hold the is practically resolved. same pivoting pattern as that over Z. Since the 3.3 Construction of a Transformation number of prime factors of the pivot element Matrix for each step is finite, only a finite number of Applying similar transformations (3) succes- unlucky primes exist for a certain matrix. sively, we can compute a transformation matrix In contrast, other papers6),8) that also pro- S together with the normal form F , but the re- pose a modular algorithm using the CRT adopt sult S = S1S2 ··· has very long integers as its the following narrow definition: elements. If we also recover S by means of the Definition 7 (Strictly Unlucky Prime). CRT, we have to use a larger number of moduli Let A, F , Ap, and Fp be the same as in Defini- pi’s than for the recovery of F . Experimen- 19) tion 5. If we have F ≡ Fp (mod p), then we tal results have shown that this approach is 1634 IPSJ Journal June 2004 fairly inefficient. formation Matrix). However, since transformation matrices S are % input: an integer matrix A and its Frobe- not unique, we have only to obtain one regu- nius normal form F = C1 ⊕ C2 ⊕···⊕Ct lar matrix S such that AS = SF. Hence, we % Let the associated polynomial of each Ck F ϕ x xdk − c xdk−1− first recover by means of the CRT, and then be k( )= k,dk−1 we compute S with elements that are as simple ···−ck,1x − ck,0. AS SF possible, so that = is satisfied. This ap- % output: an integer matrix proach is also proposed in Giesbrecht and Stor- S = s , ··· s ,d ··· st, ··· st,d johann8), using another formulation. 1 1 1 1 1 t For the sake of simplicity, we first consider the s.t. AS = SF case where F has one companion block. The (Step) for k =1tot do s relation Compute k,dk by ϕ A s 1 solving k( ) k,dk = 0; j d − .. for = k 1downto1do AS S . s As − c s = k,j = k,j+1 k,j k,dk ; 1 If the initial vector sk,d is inappropriately c c ··· c k 0 1 n−1 chosen, this algorithm might return a singular n n−1 ϕ(x)=x − cn−1x −···−c1x − c0 matrix S even when the condition AS = SF is S S s s ··· s sian elimination, then we try this algorithm = 1 2 n : s again, choosing another initial vector k,dk .A As1 = c0sn necessary condition for such appropriate initial 19) As2 = s1 + c1sn vectors is discussed in our previous paper , ··· which guarantees the existence of an appropri- As s c s s n−1 = n−2 + n−2 n ate k,dk . In our actual implementation, we set s Asn = sn−1 + cn−1sn. the vector k,dk with randomly chosen simple but generic elements. Hence, Algorithm 1 is By eliminating sn− ,...,s in turn upward 1 1 probabilistic at this step, but experiments indi- from the lowest equation, we obtain cate that it is practical enough. n n−1 A − cn−1A −···−c1A − c0E sn 3.4 Modular Algorithm for Integer = 0;i.e., ϕ(A)sn = 0. (4) Matrices The whole algorithm for integer matrices is When the normal form F has more than one given below. For the sake of simplicity, the pro- block, we have the following conditions for sd, gram will simply stop when 10 unlucky primes where we let d be the size of the target block: are detected, where the first presumption might • For the first companion block, ϕ(x) is the be incorrect. Otherwise, the CRT process is minimal polynomial of A. Then we have terminated when F (k−1) = F (k), and we check ϕ(A)=O according to Cayley-Hamilton’s ϕ(A)=O, where ϕ(x) is the minimal polyno- theorem, and sd can be arbitrary. mial of F (k). Since the minimal polynomial is • For the second block downward, we need unique, F (k) is exactly equal to the Frobenius to solve the system of linear equations (4) normal form of A when ϕ(A)=O. actually, where 0 ≤ rankϕ(A) % To simplify the following description, we Remark 3. Let us compute the bound for assume a majority decision; for example, F using Lemma 2.1 in 8): p p the patterns for 1 and 3 are identical but γ =2nen/2Annn/2, p2 gives a different pattern. (Refer to the (5) where A = max|Aij|. discussion in §3.2 for details.) ij F (3) F F n A (S3) Construct from p1 and p3 Substituting = 10 and = 10, we obtain 20 by the CRT; γ 1.5 × 10 , while the final modulus pi ≤ (S4) k := 3; count := 1; 1014 was used in the actual computation of the (S5) Do until (F (k−1) = F (k)) CRT. k := k +1; F A S ≡ S F 4. Algorithms for Rational Matrices Compute pk s.t. pk pk pk pk (mod pk); Two ways can be considered for treating ra- If pk does not yield tional matrices, as shown in the following sub- the presumed pivoting pattern sections. then { count := count +1; 4.1 An Algorithm Executeed over the If count = 10 then STOP; } Integers (k) else construct F from We let A˜ = kA (aij ∈ Q,k,a˜ij ∈ Z), F (k−1) F A k and pk by the CRT; where is a rational matrix and is the com- (S6) If ϕ(A) = O then goto (S5); mon denominator of aij’s. We can perform the % ϕ(x) = min.pol. of F (k) computation over Z as far as possible by ap- (S7) Construct S from A, F (k) plying Algorithm 2 to A˜. The relation between by Algorithm 1 ( over Z ); the Frobenius normal forms of A and A˜ is illus- (S8) While rank(S)