Characteristic polynomials of p-adic matrices Xavier Caruso, David Roe, Tristan Vaccon

To cite this version:

Xavier Caruso, David Roe, Tristan Vaccon. Characteristic polynomials of p-adic matrices. The Inter- national Symposium on Symbolic and Algebraic Computation (ISSAC), University of Kaisersiautern, Jul 2017, Kaiserslautern, Germany. pp.389-396. ￿hal-01458027￿

HAL Id: hal-01458027 https://hal.archives-ouvertes.fr/hal-01458027 Submitted on 6 Feb 2017

HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Characteristic polynomials of p-adic matrices

Xavier Caruso David Roe Tristan Vaccon Université Rennes 1 University of Pittsburg Université de Limoges [email protected] [email protected] [email protected]

ABSTRACT and O˜(n2+ω/2) operations. Second, while the lack of divi- N We analyze the precision of the characteristic polynomial sion implies that the result is accurate modulo p as long as Z χ(A) of an n × n p-adic matrix A using differential pre- M ∈ Mn( p), they still do not yield the optimal precision. cision methods developed previously. When A is integral A faster approach over a field is to compute the Frobe- N nius normal form of M, which is achievable in running time with precision O(p ), we give a criterion (checkable in time ω O˜(nω)) for χ(A) to have precision exactly O(pN ). We also O˜(n ) [15]. However, the fact that it uses division fre- give a O˜(n3) algorithm for determining the optimal preci- quently leads to catastrophic losses of precision. In many sion when the criterion is not satisfied, and give examples examples, no precision remains at the end of the calcula- when the precision is larger than O(pN ). tion. Instead, we separate the computation of the precision CCS Concepts of χM from the computation of an approximation to χM . Given some precision on M, we use [3, Lem. 3.4] to find the •Computing methodologies → Algebraic algorithms; best possible precision for χM . The analysis of this preci- Keywords sion is the subject of much of this paper. With this precision known, the actual calculation of χM may proceed by lifting Algorithms, p-adic precision, characteristic polynomial, eigen- M to a temporarily higher precision and then using a suffi- value ciently stable algorithm (see Remark 5.3). One benefit of this approach is that we may account for 1. INTRODUCTION diffuse precision: precision that is not localized on any sin- The characteristic polynomial is a fundamental invariant gle coefficient of χM . For example, let 0 ≤ α1 ≤ α2 ≤ of a matrix: its roots give the eigenvalues, and the trace and ···≤ αn, consider a diagonal matrix D with diagonal entries α1 αn determinant can be extracted from its coefficients. In fact, (p ,...,p ), let P,Q ∈ GLn(Zp) and set M = PDQ. The n−k k the best known division-free algorithm for computing deter- valuation of the coefficient of X in χM will be i=1 αi, N and if α − > 0 and M is known with precision O(p ) then minants over arbitrary rings [10] does so using the charac- n 1 P teristic polynomial. Over p-adic fields, computing the char- the constant term of χM will be known with precision larger N acteristic polynomial is a key ingredient in algorithms for than O(p ) (see [4, Prop. 3.2]). counting points of varieties over finite fields (see [7, 8, 11]. As long as none of the eigenvalues of M are congruent to When computing with p-adic matrices, the lack of infinite −1 modulo p, then none of the coefficients of the charac- memory implies that the entries may only be approximated teristic polynomial of 1 + M will have precision larger than N N at some finite precision O(p ). As a consequence, in design- O(p ). But χ1+M (X)= χM (X−1), so the precision content ing algorithms for such matrices one must analyze not only of these two polynomials should be equivalent. The solution the running time of the algorithm but also the accuracy of is that the extra precision in χ1+M is diffuse and not visible the result. on any individual coefficient. We formalize this phenomenon N Let M ∈ Mn(Qp) be known at precision O(p ). The sim- using lattices; see Section 2.1 for further explanation, and plest approach for computing the characteristic polynomial [2, §3.2.2] for a specific example of the relationship between of M is to compute det(X − M) either using recursive row χM and χ1+M . expansion or various division free algorithms [10,14]. There are two issues with these methods. First, they are slower Previous contributions. than alternatives that allow division, requiring O(n!), O(n4) Since the description of Kedlaya’s algorithm in [11], the com- putation of characteristic polynomials over p-adic numbers has become a crucial ingredient in many counting-points al- gorithms. For example, [7, 8] use p-adic cohomology and Publication rights licensed to ACM. ACM acknowledges that this contribution was the characteristic polynomial of Frobenius to compute zeta authored or co-authored by an employee, contractor or affiliate of a national govern- ment. As such, the Government retains a nonexclusive, royalty-free right to publish or functions of hyperelliptic curves. reproduce this article, or to allow others to do so, for Government purposes only. In most of these papers, the precision analysis usually ISSAC ’17, July 25-28, 2017, Kaiserslautern, Germany deals with great details on how to obtain the matrices (e.g. c 2017 ACM. ISBN ***-*-****-****-0/17/07. . . $15.00 of action of Frobenius) that are involved in the point-counting DOI: http://dx.doi.org/10.1145/*******.******* schemes. However, the computation of their characteristic polynomials is often a little bit less thoroughly studied: some and χ the characteristic polynomial map, χM ∈ K[X] for the refer to fast algorithms (using division), while others apply characteristic polynomial of M and dχM for the differential division-free algorithms. of χ at M, as a linear map from Mn(K) to the space of In [4], the authors have begun the application of the the- polynomials of degree less than n. Wefixan ω ∈ R such that ory of differential precision of [3] to the stable computation the multiplication of two matrices over a ring is in O(nω ) of characteristic polynomials. They have obtained a way to operations in the ring. Currently, the smallest known ω is express the optimal precision on the characteristic polyno- less than 2.3728639 thanks to [13]. We will denote by In mial, but have not given practical algorithms to attain this the identity matrix of rank n in Mn(K). When there is no optimal precision. ambiguity, we will drop this In for scalar matrices, e.g. for λ ∈ K and M ∈ Mn(K), λ − M denotes λIn − M. Finally, The contribution of this paper. we write σ1(M),...,σn(M) for the elementary divisors of Thanks to the application the framework of differential pre- M, sorted in increasing order of valuation. cision of [3] in [4], we know that the precision of the char- acteristic polynomial χM of a matrix M ∈ Mn(Qp) is de- 2. THEORETICAL STUDY termined by the comatrix Com(X − M). In this article, we provide: 2.1 The theory of p-adic precision 1. Proposition 2.7: a factorization of Com(X − M) as a We recall some of the definitions and results of [3] as a product of two rank-1 matrices (when M has a cyclic foundation for our discussion of the precision for the charac- vector), computable in O˜(nω) operations by Theorem teristic polynomial of a matrix. We will be concerned with 4.1 two K-manifolds in what follows: the space Mn(K) of n × n matrices with entries in K and the space Kn[X] of monic 2. Corollary 2.4: a simple, O˜(nω) criterion to decide degree n polynomials over K. Given a matrix M ∈ Mn(K), whether χM is defined at precision higher than the the most general kind of precision structure we may attach precision of M (when M ∈ Mn(Zp)). to M is a lattice H in the tangent space at M. However, rep- 3. Theorem 3.11: a O˜(n3) algorithm with operations in resenting an arbitrary lattice requires n2 basis vectors, each 2 Zp to compute the optimal precision on each coefficient with n entries. We therefore frequently work with certain of χM (when M is given with uniform precision on its classes of lattices, either jagged lattices where we specify a entries). precision for each matrix entry or flat lattices where every N ω entry is known to a fixed precision O(p ). Similarly, pre- 4. Proposition 5.6: a O˜(n ) algorithm to compute the cision for monic polynomials can be specified by giving a optimal precision on each eigenvalue of M lattice in the tangent space at f(X) ∈ Kn[X], or restricted Organization of the article. to jagged or flat precision in the interest of simplicity. Let χ : Mn(K) → Kn[X] be the characteristic polynomial In Section 2, we review the differential theory of precision map. Our analysis of the precision behavior of χ rests upon developed in [3] and apply it to the specific case of the char- the computation of its derivative dχ, using [3, Lem. 3.4]. For acteristic polynomial, giving conditions under which the dif- a matrix M ∈ Mn(K), we identify the tangent space V at ferential will be surjective (and thus provide a good measure M with Mn(K) itself, and the tangent space W at χM with of precision). We also give a condition based on reduction the space Kcompanion matrix) optimal coefficient-wise precision for the characteristic poly- (iii) the eigenspaces of M over the algebraic closure K¯ of nomial. We also give the results of some experiments demon- K all have dimension 1 strating that these methods can lead to dramatic gains in precision over standard interval arithmetic. We close with (iv) the characteristic polynomial of M is equal to the min- results describing the precision associated to eigenvalues of imal polynomial of M. a matrix. Proof. The equivalence of (ii), (iii), and (iv) is standard; see [9, §7.1] for example. We now show (ii) ⇒ (i) and (i) ⇒ Notation (iii) Throughout the paper, K will refer to a complete, discrete For any A ∈ GLn(K), the image of dχ at M will be the −1 valuation field, val : K ։ Z ∪ {+∞} to its valuation, OK same as the image of dχ at AMA , so we may assume that its ring of integers and π a uniformizer. We will write that M is a companion matrix. For a companion matrix, the f(n)= O˜(g(n)) if there exists some k ∈ N such that f(n)= bottom row of Com(X − M) consists of 1,X,X2,...,Xn−1 k O(g(n) log(n) ). We will write M for an n×n matrix over K, so dχM is surjective. Now suppose that M has a repeated eigenvalue λ over Proposition 2.5. Let A be the subring of K[X] consisting K¯ . After conjugating into Jordan normal form over K¯ , the of polynomials P for which P (M) ∈ Mn(OK ), and V = entries of Com(X − M) will also be block diagonal, and di- dχM Mn(OK ) as a submodule of K[X]/χM (X). Then V visible within each block by the product of (X −µ)dµ , where is stable under multiplication by A. µ, d ranges over the eigenvalues and dimensions of the other
µ Proof. Jordan blocks. Since λ occurs in two Jordan blocks, X − λ Let C = Com(X − M) and P ∈ A. By(1), V is given by the OK -span of the entries of C. Using the fact will divide every entry of Com(X − M) and dχM will not be surjective. that the product of matrix with its comatrix is the determi- nant, (X − M) · C = χM and thus P (X) · C ≡ P (M) · C We also have an analogue of Proposition 2.1 for integral (mod χM (X)). The span of the entries of the left hand side matrices. is precisely P (X)·V , while the span of the entries of the right hand side is contained within V since P (M) ∈ Mn(OK ). Proposition 2.2. For M ∈ Mn(OK ), the following condi- tions are equivalent: Corollary 2.6. If M ∈ Mn(OK ), then dχM Mn(OK ) is stable under multiplication by X and hence is a module over (i) the image of Mn(OK ) under dχM is OK [X]∩Kinvertible matrix Y − Q ∈ GL (K) such that M t = QC Q 1. Then, for all ρ ∈ (0, 1] and all r ∈ (0, α−1 · ρ−1), any lattice n − H such that BV (ρr) ⊂ H ⊂ BV (r) satisfies: Proposition 2.7. We keep the previous notations and as- sumptions. Let V be the row vector (1,X,...,Xn−1). Then χ(M + H)= χ(M)+ dχM (H). (2) t t Proof. Recall [4, Def. 3.3] that the precision polygon of Com(X−M)= α · PV · VQ mod χM (4) M is the lower convex hull of the Newton polygons of the en- tries of Com(X −M). By [4, Prop. 3.4], the endpoints of the for some α ∈ K[X]. n−1 precision polygon occur at height 0 and i=1 val(σi(M)). Proof. −1 Write C = Com(X−M). From (X−M) · C ≡ By convexity, BW (1) ⊂ dχM (BV (α )). C −1 P 0 (mod χM ), we deduce (X− ) · P C ≡ 0 (mod χM ). Since the coefficients of χM are given by polynomials in Therefore each column of P −1C lies in the right kernel of the entries of M with integral coefficients, [4, Prop. 2.2] X−C modulo χM . On the other hand, a direct computation implies the conclusion. shows that every column vector W lying in the right kernel of X−C modulo χ can be written as W = w · V t for some The relationship between precision and the images of lat- M w ∈ K[X]/χ . We deduce that C ≡ P · V tB (mod χ ) tices under dχ allows us to apply Proposition 2.2 to de- M M M for some row vector B. Applying the same reasoning with termine when the precision of the characteristic polynomial M t, we find that B can be written B = αV Qt for some is the minimum possible. α ∈ K[X]/χM and we are done.

Corollary 2.4. Suppose that M ∈ GLn(OK ) is known with precision O(πm). Then the characteristic polynomial of M Proposition 2.7 shows that Com(X−M) can be encoded m has precision lattice strictly contained in O(π ) if and only by the datum of the quadruple (α,P,Q,χM ) whose total 2 if the reduction of M modulo π does not have a cyclic vector. size stays within O(n ) : the polynomials α and χM are determined by 2n coefficients while we need 2n2 entries to ω Note that this criterion is checkable using O˜(n ) opera- write down the matrices P and Q. We shall see moreover tions in the residue field [15]. in Section 4 that interesting information can be read off of this short form (α,P,Q,χM ). 2.2 Stability under multiplication by X

By definition, the codomain of dχM is KHessenberg matrix is a matrix M ∈ flat precision of O(π ) on M then M can be computed at m−2 cond(M) Mn(K) with precision O(π ) by a SNF computation and this lower-bound is optimal, at least when m is large. Mi,j = 0 for j ≤ i − 2. Proof. The smallest valuation of a coefficient of M −1 is Given integers n , an approximate Hessenberg matrix is a − i,j − cond(M). It is −2 cond(M) for M 2 and it is then clear matrix M ∈ M (K) with n that m−2 cond(M) can be obtained as the valuation of a co- ni,j Mi,j = O(π ) for j ≤ i − 2. efficient of duM (dM) and the smallest that can be achieved this way for dM in a precision lattice of flat precision. Hence If M ∈ Mn(K) and H ∈ Mn(K) is an (approximate) Hessen- the optimality of the bound given, at least when m is large berg matrix similar to M, we say that H is an (approximate) [3, Lem. 3.4]. Hessenberg form of M. Now, the computation of the Smith normal form was de- It is not hard to prove that every matrix over a field ad- scribed in [16]. From M known at flat precision O(πm), we mits a Hessenberg form. We prove here that over K, if a can obtain an exact ∆, and P and Q known at precision at m−cond(M) matrix is known at finite (jagged) precision, we can com- least O(π ), with coefficients in OK and determi- × pute an approximate Hessenberg form of it. Moreover, we nant in OK realizing an Smith normal form of M. There is − − can provide an exact change of basis matrix. It relies on the no loss in precision when computing P 1 and Q 1. Since − following algorithm. the smallest valuation occurring in ∆ 1 is − cond(M), we see that M −1 = Q−1∆−1P −1 is known at precision at least Algorithm 1: Approximate Hessenberg form computation O(πm−2 cond(M)), which concludes the proof. Input: a matrix M in Mn(K). 3.3 The comatrix of X−H 0. P := In.H := M. 1. for j = 1,...,n − 1 do In this section, we compute Com(X −H) for a Hessenberg matrix H using the Smith normal form computation of the 2. swap the row j + 1 with a row imin (imin ≥ 2) s.t. previous section. The entries of Com(X − H) lie in K[X], val(Himin,j ) is minimal. 3. for i = j + 2,...,n do which is not a CDVF, so we may not directly apply the methods of the previous section. However, we may relate 4. Eliminate the significant digits of Hi,j by pivoting with row j + 1 using a matrix T. Com(X − H) to Com(1 − XH), whose entries lie in the 5. H := H × T −1. P := T × P. CDVF K((X)). In this way, we compute Com(X − H) using 6. Return H,P. an SNF method, with no division in K. First, we need a lemma relating comatrices of similar ma- trices: Proposition 3.2. Algorithm 1 computes H and P realizing Lemma 3.6. If M1, M2 ∈ Mn(K) and P ∈ GLn(K) are an approximate Hessenberg form of M. P is exact over fi- −1 such that M1 = P M2P , then: nite extensions of Qp and k((X)), and the computation is in 3 −1 O(n ) operations in K at precision the maximum precision Com(X − M1)= P Com(X − M2)P . of a coefficent in M. The second ingredient we need is reciprocal polynomials. Proof. Let us assume that K is a finite extensions of Qp We extend its definition to matrices of polynomials. or k((X)). Inside the nested for loop, if we want to eliminate uy ny ux nx π εy + O(π ) with pivot π εx + O(π ), with the ε’s be- Definition 3.7. Let d ∈ N and P ∈ K[X] of degree at ing units, the corresponding coefficient of the corresponding most d. We define the reciprocal polynomial of order d of rec,d d shear matrix is the lift(in Z, Fq[X], Q[X] or adequate exten- P as P = X P (1/X) . Let A ∈ Mn(K[X]) a matrix of rec,d uy −ux −1 uy−ux min(nx−ux,ny −uy ) sion) of π εyε mod π . Exact- polynomials of degree at most d. We denote by A the x rec,d rec,d ness follows directly. Over other fields, we can not lift, but matrix with (A )i,j =(Ai,j ) . the computations are still valid. The rest is clear. We then have the following result :

Remark 3.3. From a Hessenberg form of M, it is well Lemma 3.8. Let M ∈ Mn(K). Then: known that one can compute the characteristic polynomial rec,n−1 of M in O(n3) operations in K [5, pp. 55–56]. However, this Com(1 − XM) = Com(X − M), rec,n computation involves division, and its precision behavior is (χM In) = (1 − XM) Com(1 − XM). not easy to quantify. Proof. It all comes down to the following result: let A ∈ throughout the algorithm. Line 9 is costless, and the result Md(K[X]) a matrix of polynomials of degree at most 1, then is then proved. det(Arec,1) = det(A)rec,d. Indeed, one can use multilinearity d Remark 3.10. If M ∈ Mn(K) does not have coefficients in of the determinant on X det(A(1/X)) to prove this result. v OK , we may apply Algorithms 1 and 2 to p M ∈ Mn(OK ) It directly implies the second part of the lemma; the first 3 in O˜(n ) operations in OK , and then divide the coefficient part follows from the fact that the entries of Com(X − M) k kv and of Com(1 − XM) are determinants of size n − 1. of X in the resulting polynomial by p . We will see in Section 5 that for an entry matrix with This lemma allows us to compute Com(1 − XM) instead coefficients known at flat precision, Algorithms 1 and 2 are of Com(X − M). This has a remarkable advantage: the piv- enough to know the optimal jagged precision on χ . ots during the computation of the SNF of Com(1 − XM) M are units of OK [[X]], and are known in advance to be on the 3.4 The comatrix of X−M diagonal. This leads to a very smooth precision and com- In this section, we combine Proposition 2.7 with Algo- plexity behaviour when the input matrix lives in Mn(OK ). rithm 2 to compute the comatrix of X − M when χM is squarefree. Note that this condition on χ is equivalent Algorithm 2: M Approximate Com(X − H) to M being diagonalizable under the assumption that dχ Input: M an approximate Hessenberg matrix H in Mn(OK ). is surjective. The result is the following O˜(n3) algorithm, 0. U := 1 − XH. U0 := 1 − XH. where the only divisions are for gcd and modular inverse 1. While updating U, track P and Q so that U0 = PUQ is computations. always satisfied. 2. for i = 1,...,n − 1 do Algorithm 3: Approximate Com(X−M) n+1 3. Eliminate, modulo X the coefficients Ui,j , for j ≥ Input: an approx. M ∈ Mn(OK ), with Disc(χM ) 6= 0. n+1 i + 1 using the invertible pivot Ui,i =1+ XLi,i mod X 0. Find P ∈ GLn(OK ) and H ∈ Mn(OK ), approximate (with Li,i ∈OK [X]). Hessenberg, such that M = P HP −1, using Algorithm 1. 4. for i = 1,...,n − 1 do n+1 1. Compute A = Com(X − H) and χM = χH using Algo- 5. Eliminate, modulo X the coefficients Ui+1,i, using rithm 2. the invertible pivot Ui,i. n 2. Do row(A, 1) ← row(A, 1)+ i=2 µi row(A, i), for random 6. ψ := i Ui,i. n+1 µi ∈OK , by doing T × A for some T ∈ GLn(OK ). Compute 7. Rescale to get U = In mod X . −1 P Q n+1 1 B := TAT . 8. V := ψ × P × Q mod X . −1 rec,n−1 rec,n 3. Similarily compute C := S BS for S ∈ GLn(OK ) cor- 9. Return V , ψ . responding to adding a random linear combination of the columns of index j ≥ 2 to the first column of B. 4. If gcd(C1,1,χM ) 6= 1, then go to 2. Theorem 3.9. Let H ∈ Mn(OK ) be an approximate Hes- senberg matrix. Then, using Algorithm 2, one can compute 5. Let F be the inverse of C1,1 mod χM . 3 6. Let U := col(C, 1) and V := F · row(C, 1) mod χM . Com(X − H) and χH in O˜(n ) operations in OK at the −1 −1 −1 precision given by H. 7. Return Com(X−M):=(PT SU×VS T P ) mod χM . Proof. First, the operations of the lines 2 and 3 use 3 O˜(n ) operations in OK at the precision given by H. In- Theorem 3.11. For M ∈ Mn(OK ) such that Disc(χM ) 6= deed, since H is an approximate Heisenberg matrix, when 0, Algorithm 3 computes Com(X−M) (mod χM ) in average 3 we use Ui,i as pivot the only other nonzero coefficient in its complexity O˜(n ) operations in K. The only divisions occur column is Ui+1,i. As a consequence, when performing this in taking gcds and inverses modulo χM . column-pivoting, only two rows (i and i + 1) lead to opera- Proof. As we have already seen, completing Steps 0 and tions in OK [[X]] other than checking precision. Hence, line 3 1 is in O˜(n3). Multiplying by T or S or their inverse cor- 2 costs O˜(n ) for the computation of U. Following line 1, the responds to n operations on rows or columns over a matrix computation of Q is done by operations on rows, starting with coefficients in OK [X] of degree at most n. Thus, it is from the identity matrix. The order in which the entries in O˜(n3). Step 5 is in O˜(n), Step 6 in O˜(n2) and Step 7 of U are cleared implies that Q is just filled in as an upper in O˜(n3). All that is to prove is that the set of P and S to triangular matrix: no additional operations in OK [[X]] are avoid is of dimension at most n−1. The idea is to work mod- required. Thus the total cost for lines 2 and 3 is indeed ulo X − λ for λ a root of χ(M) (in an algebraic closure) and 3 O˜(n ) operations. then apply Chinese Remainder Theorem. The goal of the For lines 4 and 5, there are only n − 1 eliminations, re- Step 2 is to ensure the first row of B contains an invertible 2 sulting in a O˜(n ) cost for the computation of U.. Rather entry modulo χM . Since A(λ) is of rank one, the µi’s have to than actually construct P , we just track the eliminations avoid an affine hyperplane so that row(B, 1) mod (X − λ) performed in order to do the corresponding row operations is a non-zero vector. Hence for row(B, 1) mod χ(M) to on Q, since we only need the product P × Q. contain an invertible coefficient, a finite union of affine hy- 2 3 Line 6 is in O˜(n ) and 7 in O˜(n ). perplane is to avoid. Similarly, the goal of Step 3 is to put an Thanks to the fact that the P only corresponds to the invertible coefficient (modulo χM ) on C1,1, and again, only product of n − 1 shear matrices, the product on line 8 is a finite union of affine hyperplane is to avoid. Hence, the set 3 in O˜(n ). We emphasize that no division has been done that the µi’s have to avoid is a finite union of hyperplane, 1The product P × Q should be implemented by sequential and hence, is of dimension at most n − 1. Thus, almost any row operations corresponding to the eliminations in Step 5 choice of µi leads to a matrix C passing the test in Step 4. in order to avoid a product of two matrices in Mn(OK [X]). This concludes the proof. Remark 3.12. As in the previous section, it is possible to write the relation: scale M ∈ M (K) so as to get coefficients in O and apply n K Com(X−C )= α · V t · V Rt mod χ (5) the previous algorithm. M which comes from Eq. (4) after multiplication on the left Remark 3.13. We refer to [1] for the handling of the preci- by P −1 and multiplication on the right by P . We observe sion of gcd and modular inverse computations. In this arti- moreover that the first row of R is (1, 0,..., 0). Evaluating cle, ways to tame the loss of precision coming from divisions the top left entry of Eq. (5), we end up with the relation: are explored, following the methods of [3]. n−2 n−1 α = a1 + a2X + · · · + an−1X + X . 4. DIFFERENTIAL No further computation are then needed to derive the value VIA FROBENIUS FORM of α. We summarize this section with the following theorem: Theorem 4.1. The algorithm designed in the previous section computes Given M ∈ Mn(K) such that dχM is surjec- tive, then one can compute (α,P,Q,χM ) in K[X] such that the differential dχM of χ at a given matrix M ∈ Mn(K) for t t ω a cost of O(n3) operations in K. This seems to be optimal Com(X−N)= α · V · V R mod χM in O˜(n ) operations 3 in K. given that the (naive) size of the dχM is n : it is a matrix of size n × n2. It turns out however that improvements are still possible! Indeed, thanks to Proposition 2.7, the matrix 5. OPTIMAL JAGGED PRECISION of dχM admits a compact form which can be encoded using In the previous Sections, 3 and 4, we have proposed algo- only O(n2) coefficients. The aim of this short section is to rithms to obtain the comatrix of X − M. Our motivation design a fast algorithm (with complexity O˜(nω)) for com- for these computations is to then be able to understand puting this short form. The price to pay is that divisions in what is the optimal precision on χM . In this section, we K appear, which can be an issue regarding to precision in provide some answers to this question, along with numerical particular cases. In this section, we only estimate the num- evidence. We also show that it is then possible to derive ber of operations in K and not their behaviour on precision. optimal precision of eigenvalues of M. From now on, we fix a matrix M ∈ Mn(K) for which dχM is surjective. Let (α,P,Q,χM ) be the quadruple encoding 5.1 On the characteristic polynomial the short form of dχM ; we recall that they are related by For 0 ≤ k < n, let πk : K[X] → K be the mapping the relations: taking a polynomial to its coefficients in Xk. By applying [3, Lem. 3.4] to the composite πk ◦χM , one can figure out the dχM (dM) = Tr(Com(X−M) · dM) optimal precision on the k-th coefficient of the characteristic t t Com(X−M)= α · PV · VQ mod χM . polynomial of M (at least if M is given at enough precision). Let us consider more precisely the case where M is given ω An approximation to χM can be computed in O˜(n ) oper- at jagged precision: the (i, j) entry of M is given at precision Ni,j ations in K (e.g. as a by-product of [15]). O(π ) for some integers Ni,j . Lemma 3.4 of [3] then shows n The matrix P can be computed as follows. Pick c ∈ K . that the optimal precision on the k-th coefficient of χM is ′ i ′ Define ci = M c for all i ≥ 1. The ci’s can be computed O(πNk ) where N is given by the formula: ω k in O˜(n ) operations in K, e.g. using the first algorithm ′ Nk = min Nj,i + val(πk(Ci,j )), (6) of [12]. Let Pinv be the n × n matrix whose rows are the 1≤i,j≤n ci’s for 1 ≤ i ≤ n. Remark that Pinv is invertible if and n only if (c0, c1,...,cn−1) is a basis of K if and only if c where Ci,j is the (i, j) entry of the comatrix Com(X−M). is a cyclic vector. Moreover after base change to the basis Proposition 5.1. If M ∈ Mn(OK ) is given at (high enough) (c0,...,cn−1), the matrix M takes the shape (3). In other −1 jagged precision, then we can compute the optimal jagged words, if Pinv is invertible, then P = P is a solution of 3 inv precision on χM in O˜(n ) operations in K. M = P C P −1, where C is the companion matrix similar to Proof. We have seen in §3 and §4 that the computation M. Moreover, observe that the condition “Pinv is invertible” is open for the Zariski topology. It then happens with high of the matrix C = Com(X−M) can be carried out within 3 probability as soon as it is not empty, that is as soon as M O˜(n ) operations in K (either with the Hessenberg method admits a cyclic vector, which holds by assumption. or the Frobenius method). We conclude by applying Eq. (6) which requires no further operation in K (but n3 evaluations The characteristic polynomial χM can be recovered thanks of valuations and n3 manipulations of integers). to the relation a0c0 + a1c1 + · · · + an−1cn−1 = −cn−1 · P . Now, instead of directly computing Q, we first compute a Remark 5.2. C t C −1 If M ∈ Mn(OK ) is given at (high enough) matrix R with the property that = R R . To do so, we flat precision, then we can avoid the final base change step apply the same strategy as above except that we start with in the Hessenberg method. Indeed, observe that, thanks to the vector e = (1, 0,..., 0) (and not with a random vector). Lemma 3.6, we can write: A simple computation shows that, for 1 ≤ i ≤ n−1, the vector C ie has the shape: Tr(Com(X−M) · dM) = Tr(Com(X−H) · P −1dMP )

i C e = (0,..., 0, −a0,⋆,...,⋆) where P lies in GLn(OK ). Moreover, the latter condition −1 implies that P dMP runs over Mn(OK ) when P runs over C i ′ with n−i starting zeros. Therefore the e’s form a basis of Mn(OK ). As a consequence, the integer Nk giving the op- Kn, i.e. e is always a cyclic vector of C . Once R has been timal precision on the k-th coefficient of M is also equal to t H H computed, we recover Q using the relation Q = PinvR. N +min1≤i,j≤n val(πk(Ci,j )) where Ci,j is the (i, j) entry of It remains to compute the scaling factor α. For this, we Com(X−H), where H is the Hessenberg form of M. Remark 5.3. As a consequence of the previous discussion, Average loss of accuracy once the optimal jagged precision is known, it is possible to Optimal CR FP lift the entries of M to a sufficiently large precision, rescale 0 them to have entries in O and then use Algorithm 2 to X 3.17 196 189 K (det.) dev: 1.76 dev: 240 dev: 226 compute the characteristic polynomial. The output might 1 then need to be rescaled and truncated at the optimal pre- X 2.98 161 156 3 dev: 1.69 dev: 204 dev: 195 cision. This requires O˜(n ) operations in OK and unfortu- nately, for several instances, may require to increase a lot X2 2.75 129 126 the precision. dev: 1.57 dev: 164 dev: 164 X3 2.74 108 105 Numerical experiments. We have made numerical ex- dev: 1.73 dev: 144 dev: 143 periments in SageMath [6] in order to compare the opti- X4 2.57 63.2 60.6 mal precision obtained with the methods explained above dev: 1.70 dev: 85.9 dev: 85.8 with the actual precision obtained by the software. For do- X5 2.29 51.6 49.7 ing so, we picked a sample of 1000 random matrices M in dev: 1.66 dev: 75.3 dev: 74.9 Q M9( 2) where all the entries are given at the same relative X6 2.07 9.04 8.59 precision. We recall that, in SageMath, random elements dev: 1.70 dev: 26.9 dev: 26.4 x ∈ Q are generated as follows. We fix an integer N — the p X7 1.64 5.70 5.38 so-called relative precision — and generate elements of Qp dev: 1.65 dev: 15.3 dev: 14.7 of the shape X8 0.99 0.99 0.99 x = pv · a + O pN+vp(a) (trace) dev: 1.37 dev: 1.37 dev: 1.37 where v is a random integer generated according

to the dis- Results for a sample of 1000 instances tribution: 1 2 Figure 1: Average loss of accuracy on the coefficients P[v =0]= ; P[v = n]= for |n|≥ 1 of the characteristic polynomial of a random 9 × 9 5 5 · |n| · (|n| + 1) matrix over Q2 and a is an integer in the range [0,pN ), selected uniformly at random. Once this sample has been generated, we computed, for • the column “FP” is the average of the quatities (FPk−v)− each k ∈ {0, 1,..., 8}, the three following quantities: N where FPk is the first position of an incorrect digit on • the optimal precision on the k-th coefficient of the char- the k-th coefficient of χM . acteristic polynomial of M given by Eq. (6) We observe that the loss of relative accuracy stays under 2 • in the capped relative model , the precision gotten on control in the “Optimal”column whereas it has a very erratic the k-th coefficient of the characteristic polynomial of M behavior — very large values and very large deviation as computed via the call: well — in the two other columns. These experiments thus M.charpoly(algorithm="df") demonstrate the utility of the methods developed in this • in the model of floating-point arithmetic (see [2, §2.3]), paper. the number of correct digits of the k-th coefficient of the 5.2 On eigenvalues characteristic polynomial of M. 3 Let M ∈ Mn(K) and λ ∈ K be a simple eigenvalue of Remark 5.4. The keyword algorithm="df" forces Sage- M. We are interesting in quantifying the optimal precision Math to use the division free algorithm of [14]. It is likely on λ when M is given with some uncertainty. that, proceeding so, we limit the loss of precision. To do so, we fix an approximation Mapp ∈ Mn(K) of M and suppose that the uncertainty of M is “jagged” in The table of Figure 1 summarizes the results obtained. It the sense that each entry of M is given at some precision should be read as follows. First, the acronyms CR and FP Ni,j O(π ). Let λapp be the relevant eigenvalue of Mapp. We refers to “capped relative” and “floating-point” respectively. remark that it is possible to follow the eigenvalue λapp on a The numbers displayed in the table are the average loss of small neighborhood U of M. More precisely, there exists a relative precision. More precisely, if N is the relative preci- unique continuous function f : U → K such that: sion at which the entries of the input random matrix M have been generated and v is the valuation of the k-th coefficient • f(Mapp)= λapp, and of χ , then: ′ ′ ′ M • f(M ) is an eigenvalue of M for all M ∈U. • the column “Optimal” is the average of the quantities ′ ′ ′ (Nk−v) − N (where Nk is defined by Eq. (6)): Nk−v Lemma 5.5. The function f is strictly differentiable on a is the optimal relative precision, so that the difference neighborhood of Mapp. The differential of f at M is the ′ (Nk−v) − N is the loss of relative precision; linear mapping: • the column“CR”is the average of the quatities (CRk−v)− Tr(Com(λ − M) · dM) dM 7→ dλ = − N where CRk is the computed (absolute) precision on the ′ χM (λ) k-th coefficient of χM ; where χ′ is the usual derivative of χ . 2Each coefficient carries its own precision which is updated M M after each elementary arithmetical operation. 3the corresponding generalized eigenspace has dimension 1 Proof. The first assertion follows from the implicit func- References tion Theorem. Differentiating the relation χM (λ) = 0, we [1] Xavier Caruso, Resultants and subresultants of p-adic ′ get χM (λ) · dλ + Tr(Com(X − M) · dM)(λ) = 0, from which polynomials, arxiv:1507.06502 (2015). the Lemma follows. [2] , Computations with p-adic numbers, arxiv:1701.06794 (2017). Lemma 3.4 of [3] now implies that, if the Ni,j ’s are large enough and sufficiently well balanced, the optimal precision [3] Xavier Caruso, David Roe, and Tristan Vaccon, Track- ′ on the eigenvalue λ is O(πN ) with: ing p-adic precision, LMS J. Comput. Math. 17 (2014), no. suppl. A, 274–294. MR3240809 ′ ′ N = min Nj,i + val(Ci,j (λ)) − val(χM (λ)) 1≤i,j≤n [4] , p-Adic Stability In Linear Algebra, Proceedings of the 2015 ACM on International Symposium on Sym-
where Ci,j denotes as above the (i, j) entry of Com(X−M). bolic and Algebraic Computation, ISSAC 2015, Bath, t t Writing Com(X−M)= α · PV · VQ mod χM as in Propo- United Kingdom, July 06 - 09, 2015 (2015), 101–108. sition 2.7, we find: [5] Henri Cohen, A course in computational algebraic num- ′ ′ ber theory, Vol. 138, Springer Science & Business Media, N = val(α(λ)) − val(χM (λ)) 2013. t t + min Nj,i + val(PiV (λ) ) + val(V (λ)Qj ) (7) [6] The Sage Developpers, Sage Mathematics Software 1≤i,j≤n (Version 7.5), 2017. http://www.sagemath.org.
where Pi denotes the i-th row of P and, similarly, Qj denotes [7] David Harvey, Kedlaya’s algorithm in larger character- the j-th row of Q. Note moreover that V (λ) is the row vector istic, International Mathematics Research Notices 2007 (1,λ,...,λn−1). By the discussion of §4, the exact value of (2007), rnm095. ′ ω N can be determined for a cost of O˜(n ) operations in K [8] , Counting points on hyperelliptic curves in av- and O(n2) operations on integers. erage polynomial time, Annals of Mathematics 179 When M is given at flat precision, i.e. the Ni,j ’s are all (2014), no. 2, 783–803. ′ equal to some N, the formula for N may be rewritten: [9] Kenneth Hoffman and Ray Kunze, Linear algebra, 2nd ′ ′ ed., Prentice-Hall, Englewood Cliffs, New Jersey, 1971. N = N + val(α(λ)) − val(χM (λ)) t t [10] Erich Kaltofen, On computing determinants of matrices + min val(PiV (λ) )+ min val(V (λ)Qj ) (8) 1≤i≤n 1≤j≤n without division, Proc. 1992 Internat. Symp. Symbolic Algebraic Computation. (ISSAC’92), 1992, pp. 342– and can therefore now be evaluated for a cost of
O˜(nω) 349. operations in K and only O(n) operations with integers. [11] Kiran S. Kedlaya, Counting points on hyperelliptic To conclude, let us briefly discuss the situation where we curves using monsky–washnitzer cohomology, J. Ra- manujan Math. Soc. 16 (2001), 323–338. want to figure out the optimal jagged precision on a tuple (λ ,...,λ ) of simple eigenvalues. Applying (7), we find [12] Walter Keller-Gehrig, Fast algorithms for the charac- 1 s 36 that the optimal precision on λ is teristics polynomial, Theoretical Computer Science k (1985), 309 –317. ′ ′ Nk = val(α(λk)) − val(χM (λk)) [13] Fran¸cois Le Gall, Powers of tensors and fast matrix t t multiplication, Proceedings of the 39th international + min Nj,i + val(PiV (λk) ) + val(V (λk)Qj ) . 1≤i,j≤n symposium on symbolic and algebraic computation, ′
ω 2014, pp. 296–303. Proposition 5.6. The Nk’s can be all computed in O˜(n ) operations in K and O(n2s) operations with integers. [14] T. R. Seifullin, Computation of determinants, adjoint matrices, and characteristic polynomials without divi- If the Ni,j ’s are all equal, the above complexity can be ω sion, Cybernetics and Systems Analysis 38 (2002), lowered to O˜(n ) operations in K and O(ns) operations no. 5, 650–672. with integers. [15] Arne Storjohann, Deterministic computation of the Proof. ′ The α(λk)’s and the χM (λk)’s can be computed Frobenius form, Proceedings of the 42nd IEEE sympo- for a cost of O˜(ns) operations in K using fast multipoint sium on foundations of computer science, 2001, pp. 368– evaluation methods (see 10.7 of [17]). On the other hand, 377. t we observe that PiV (λk) is nothing but the (i, k) entry of [16] Tristan Vaccon, p-adic precision, Theses, 2015. the matrix: [17] Joachim Von Zur Gathen and Jurgen¨ Gerhard, Modern λ1 · · · λs computer algebra, Cambridge university press, 2013. 2 2 λ1 · · · λs P ·  . .  . . .   λn−1 · · · λn−1  1 s  The latter product can be computed inO˜(nω) operations 4 t in K . Therefore all the PiV (λk) ’s (for i and k varying) can be determined with the same complexity. Similarly all t the V (λ)Qj are computed for the same cost. The first as- sertion of Proposition 5.6 follows. The second assertion is now proved similarly to the case of a unique eigenvalue. 4It turns out that O˜(n2) is also possible because the right factor is a structured matrix (a truncated Vandermonde): computing the above product reduces to evaluating a poly- nomial at the points λ1,...,λs.