Differential Qd Algorithm for Totally Nonnegative Hessenberg Matrices

JSIAM Letters Vol.2 (2010) pp.69–72 c 2010 Japan Society for Industrial and Applied Mathematics

Diﬀerential qd algorithm for totally nonnegative Hessenberg matrices: introduction of origin shifts and relationship with the discrete hungry Lotka-Volterra system

Yusaku Yamamoto1 and Takeshi Fukaya2

1 Department of Computer Science and Systems Engineering, Kobe University, 1-1 Rokkodai- cho, Nada-ku, Kobe 657-8501, Japan 2 Department of Computational Science and Engineering, Nagoya University, Furo-cho, Chikusa-ku, Nagoya 464-8603, Japan E-mail yamamoto cs.kobe-u.ac.jp Received February 27, 2010, Accepted April 21, 2010 Abstract We propose an approach for introducing the origin shift into the multiple dqd algorithm for computing the eigenvalues of a totally nonnegative matrix. Numerical experiments show that the shift speeds up the convergence while retaining the accuracy of the computed eigenvalue. Keywords eigenvalue, totally nonnegative, Hessenberg matrix, qd algorithm, origin shift Research Activity Group Algorithms for Matrix / Eigenvalue Problems and their Applications

1. Introduction theoretical proof of high relative accuracy has yet to be Let A be an m × m matrix. A is called totally non- established. We also point out the close relationship be- negative (TN) if all of its minors are nonnegative. TN tween our shifted multiple dqd algorithm and the dis- matrices have many applications in areas such as com- crete hungry Lotka-Volterra system [5]. binatorics and statistics [1]. Recently, there is a growing interest in numerical algorithms for TN matrices, and 2. The multiple dqd algorithm several algorithms for eigenvalue computation, singular In this section, we define our target problem and re- value decomposition and linear equation solving with view the unshifted multiple dqd algorithm. TN coefficient matrices have been developed [1–3]. Let L and Ri (i = 1,...,M) be m × m lower and In [2], we proposed an algorithm for computing the upper bidiagonal matrices defined by eigenvalues of a totally nonnegative band matrix. The q algorithm, called the multiple dqd (differential quotient- 1  1 q  difference) algorithm, is a natural extension of the dqd 2  1 q3  algorithm for computing the eigenvalues of a symmetric L =   and  . .  positive-definite tridiagonal matrix [4]. It has two fea-  .. ..    tures: first, it exploits the fact that TN matrices can be  1 qm  represented as a product of positive bidiagonal factors 1 e and works directly on these bidiagonal factors. Second, i1  e  it preserves the total nonnegativity throughout the iter- 1 i2  ..  ations. These features enable us to show that the algo- Ri =  1 .  , (1)   rithm can compute all the eigenvalues of a TN matrix  ..   . ei,m−1  to high relative accuracy [2]. Unfortunately, due to the   structure of the algorithm, it seemed difficult to intro-  1 

duce the origin shift to accelerate the convergence into respectively, where qk (1 ≤ k ≤ m) and eik (1 ≤ i ≤ M, the multiple dqd algorithm. For this reason, the order of 1 ≤ k ≤ m − 1) are some positive numbers. We consider convergence of this algorithm remained only linear. the problem of computing the eigenvalues of a matrix In this paper, we consider the case where A is a TN deﬁned as the product of these bidiagonal factors: Hessenberg matrix. For this type of matrices, we show that we can introduce the origin shift while retaining the A = LR1R2 · · · RM . (2) above two features. Our preliminary numerical experi- A is a Hessenberg matrix with upper bandwidth M. Fur- ment shows that the resulting algorithm exhibits faster thermore, A is totally nonnegative because it is a prod- convergence and can compute the smallest eigenvalue to uct of positive bidiagonal factors [1]. Then, from the gen- the same accuracy as the shiftless algorithm, although eral theory of the TN matrix, we know that all of the

– 69 – JSIAM Letters Vol. 2 (2010) pp.69–72 Yusaku Yamamoto et al.

eigenvalues of A are simple, real and positive. 3.2 Introduction of the origin shift into the multiple dqd The multiple dqd algorithm for A is a variant of the algorithm LR algorithm that performs the LR step solely on the Assume that A is given as in (2). In considering the bidiagonal factors. Let M = 3 and consider applying an shifted version of the multiple dqd algorithm, it would be LR step to A. Noting that (2) already gives the LR de- natural to require that the next iterate Aˆ is also given as composition of A, we can compute the next iterate Aˆ by a product of bidiagonal factors: making the product R R R L and computing its LR de- 1 2 3 Aˆ LˆRˆ Rˆ · · · Rˆ , composition. In the multiple dqd algorithm, we do this = 1 2 M (8) by repeating the LR decomposition of a product of upper where Lˆ is a lower bidiagonal matrix whose lower subdi- and lower bidiagonal factors as follows: agonal elements are all 1’s and Rî is a unit upper bidiagonal matrix. We denote the diagonal elements of Lˆ by Aˆ = R1R2R3L qˆk and the upper subdiagonal elements of Rî bye îk. (1) = R1R2L Rˆ3 We want to design the shifted algorithm in such a way that the conditions (i) and (ii) explained in Section R L(2)Rˆ Rˆ = 1 2 3 2 are satisfied. This is because then we will be able to (3) perform relative error analysis of the algorithm as in the = L Rˆ1Rˆ2Rˆ3 ≡ LˆRˆ1Rˆ2Rˆ3. (3) unshifted case. In the present subsection, we concentrate Here, the products such as R1R2R3L are not computed on constructing the shifted algorithm so that the condi- explicitly and the LR transformations such as R3L = tion (i) is satisfied. Positivity of the variables will be dis- (1) ˆ L R3 are done with the dqd algorithm [4]. This algo- cussed in the next subsection. Also, in this subsection, rithm has the following two features: we assume that breakdown of the algorithm, such as di- (i) It works on the bidiagonal factors directly and never vision by zero, does not occur. A sufficient condition for forms their products explicitly. this will be given in the next subsection. Inserting (2) and (8) into (6) and (7), we have (ii) Positivity of the bidiagonal factors is preserved due (0) (0) to the characteristics of the dqd algorithm. LR1 · · · RM − sI = L R , (9)

By combining these features with mixed error analysis of (0) −1 (0) LˆRˆ · · · RˆM = (L ) LR · · · RM L . (10) the dqd algorithm [4] and relative perturbation theory 1 1 on the eigenvalues of a TN matrix [1], one can show In (9), L(0) is a lower bidiagonal matrix whose subdi- that the multiple dqd algorithm can compute all the agonal elements are all 1’s. Now, suppose that we know (0) eigenvalues of a TN matrix to high relative accuracy [2]. L . Then we can compute L,ˆ Rˆ1,..., RˆM by applying the LR transformations repeatedly to the right-hand side 3. Introduction of the origin shift of (10) as follows: (0) (1) 3.1 The shifted LR algorithm for a general matrix RM L = L RˆM , (11) As a preparation for introducing the origin shift into (1) (2) ˆ the multiple dqd algorithm, we first explain the shifted RM−1L = L RM−1, (12) LR algorithm. Let s be some properly chosen shift and . denote the m × m identity matrix by I. Then, one step . of the shifted LR algorithm can be written as follows: (M−1) (M) R1L = L Rˆ1, (13) (0) (0) A − sI = L R , (4) LL(M) = L(0)L.ˆ (14) ˆ (0) (0) (0) −1 (0) A = R L + sI = (L ) AL . (5) Here, the last equation is obtained by rewriting the equation Lˆ = (L(0))−1LL(M). Eqs. (11) through (14) show Here, we define the LU decomposition in (4) so that the that once we know L(0), all the subsequent computations (0) diagonals of R are 1’s, in accordance with (1). can be done by working only on the bidiagonal factors. Now, assume that A is given in the factored form as The remaining problem is how to compute L(0). From ˆ ˆ A = LR and we want A also in the factored form A = (9), it seems that one needs to form LR1 · · · RM − sI LˆRˆ. Then, the above formulae can be rewritten as explicitly and compute its LR decomposition. But this LR − sI = L(0)R(0), (6) approach would violate the condition (i). Fortunately, we can compute L(0) simultaneously with (0) −1 (0) LˆRˆ = (L ) LRL . (7) L,ˆ Rˆ1,..., RˆM by rearranging the computations. To see this, first compare the (1, 1) elements of both sides of (9) ˆ ˆ This shows that the computation of L and R from L and to obtain R can be done in the following two steps. q(0) = q − s. (15) • Obtain the lower triangular factor L(0) from the LR 1 1 decomposition of LR − sI. Next, by writing down (11) through (13) element by el- • Apply similarity transformation by L(0) to LR. ement, we have the following set of equalities: (0) Based on this observation, we introduce the origin shift qk eˆ − = e − (2 ≤ k ≤ m), (16) into the multiple dqd algorithm in the next subsection. M,k 1 M,k 1 (1) qk−1

– 70 – JSIAM Letters Vol. 2 (2010) pp.69–72 Yusaku Yamamoto et al.

(1) (0) 1:k 1:k qk = eM,k + qk − eˆM,k−1 (1 ≤ k ≤ m), (17) values of C1:k = A1:k − sIk are positive and accordingly C1:k > 0 (1 ≤ k ≤ m). In this case, the left-hand side (1) 1:k qk e − − e − − ≤ k ≤ m , of (9) is LR-decomposable and the diagonal elements of ˆM 1,k 1 = M 1,k 1 (2) (2 ) (18) qk−1 the lower triangular factor are given as (2) (1) 1:k (0) C k qk = eM−1,k + qk − eˆM−1,k−1 (1 ≤ k ≤ m), (19) q = 1: > 0, (26) k 1:k− 1 . C1:k−1 . 1:0 (0) (M−1) where C1:0 ≡ 1. This shows the positivity of L . qk (i) e − e − k m , Next, we show the positivity of L and Rî (1 ≤ i ≤ ˆ1,k 1 = 1,k 1 M (2 ≤ ≤ ) (20) ( ) (1) qk−1 M). First, observe that L and RˆM are computed from (0) (M) (M−1) L and RM by the LR transformation (11). It is well q = e + q − eˆ − (1 ≤ k ≤ m), (21) k 1,k k 1,k 1 known that the LR transformation can be done with the where we adopted the convention that differential qd algorithm that uses only addition, multi- plication and division. Hence if the input bidiagonal fac- eˆ =e ˆ − = · · · =e ˆ = 0, (22) M,0 M 1,0 1,0 tors are positive, there is no breakdown and the output (0) eM,m = eM−1,m = · · · = e1,m = 0. (23) bidiagonal factors are also positive. Since both L and R L(1) Finally, we write down (14) element-wise to obtain M are positive in (11), we know that both and RˆM are positive. By repeating this reasoning, we know (M) (2) (M) ˆ ˆ qk that L ,...,L and RM−1,..., R1 are positive. qˆk = qk (1 ≤ k ≤ m), (24) (0) (M) q(0) Finally, since L, L and L are positive, we know k from (24) that Lˆ is also positive. Also, the positivity (0) (M) (0) (1) (M) qk+1 = qk+1 + qk − qˆk (1 ≤ k ≤ m − 1). (25) of L , L ,...,L guarantees the absence of breakdown, because the denominators on the right-hand sides (0) ˆ ˆ ˆ Eqs. (15) through (25) determine L , L, R1,..., RM of (16), (18), (20) and (24) are positive. (0) (QED) completely. Indeed, starting from q1 given by (15) and using (17), (19), (21), (24) and (25), we can compute From Theorem 1, we know that the shifted multiple (1) (2) (M) (0) q1 , q1 ,...,q1 , qˆ1 and q2 in this order (Note that dqd algorithm also satisfies the condition (ii) if s<λm. (0) we need not computee î,k−1 when k = 1). Once q2 has been computed, we can proceed to the second round; 4. Relationship with the discrete hungry (1) (2) (M) we computee ˆM,1, q2 , eˆM−1,1, q2 ,..., eˆ1,1, q2 , qˆ2 and Lotka-Volterra system (0) q3 in this order using (16) through (25). By continuing The shifted multiple dqd algorithm given in the pre- this process, all the elements of L,ˆ Rˆ1,..., RˆM , along vious section has a close relationship with the discrete with the elements of L(0), can be computed. We call this hungry Lotka-Volterra system. To see this, we rename process the shifted multiple dqd algorithm. Obviously, the variables in (15) through (25) as follows: this algorithm obviates the need to form the product (n) qk → U , (27) LR1R2 · · · RM explicitly and works only on the bidiago- (k−1)(M+1)+1 nal factors. Thus the condition (i) is satisfied. (n) − eM i+1,k → U(k−1)(M+1)+i+1, (28) 3.3 The condition for positivity of the variables (n+1) In the previous subsection, we derived the equations qˆk → U(k−1)(M+1)+1, (29) (15) through (25) of the shifted multiple dqd algorithm (n+1) − without asking whether the variables are positive or not. eˆM i+1,k → U(k−1)(M+1)+i+1, (30) We also assumed that breakdown of the algorithm does (0) ¯ (n) not occur. In this subsection, we show that all the vari- qk → V(k−1)(M+1)+1, (31) ables remain positive and breakdown does not occur if q(i) → V¯ (n) , the shift satisfies a certain condition. More specifically, k (k−1)(M+1)+i+1 (32) we have the following theorem. 1 s → − . (33) Theorem 1 Let A be a TN matrix given by (2) and δ(n) denote its smallest eigenvalue by λm. If s<λm, then Then (16) through (21), (24) and (25) can be rewritten breakdown does not occur in one step of the shifted mul- as follows: tiple dqd algorithm. Moreover, all the elements of L(i) U (n) V¯ (n) (0 ≤ i ≤ M), Rî (1 ≤ i ≤ M) and Lˆ are positive. (n+1) (k−2)(M+1)+2 (k−1)(M+1)+1 U − = , (34) (0) (k 2)(M+1)+2 ¯ (n) Proof We first show the positivity of L . Let C ≡ A V(k−2)(M+1)+2 −sI and denote the k × k leading principal submatrices (n) (n) (n) 1:k 1:k V¯ = U + V¯ of A and C by A1:k and C1:k , respectively. Since A is (k−1)(M+1)+2 (k−1)(M+1)+2 (k−1)(M+1)+1 a TN matrix, its eigenvalues are all real and positive. (n+1) − U , (35) Moreover, from the interlace inequality concerning the (k−2)(M+1)+2 eigenvalues of a leading principal submatrix of a TN ma- (n) ¯ (n) U k− M V k− M trix [6], the smallest eigenvalue of A1:k is larger than or U (n+1) = ( 2)( +1)+3 ( 1)( +1)+2 , (36) 1:k (k−2)(M+1)+3 ¯ (n) equal to the smallest eigenvalue of A. Hence all the eigen- V(k−2)(M+1)+3

– 71 – JSIAM Letters Vol. 2 (2010) pp.69–72 Yusaku Yamamoto et al. (n) (n) (n) (0) Iterations V¯ = U + V¯ Σei,m −1 (k−1)(M+1)+3 (k−1)(M+1)+3 (k−1)(M+1)+2 1 2 3 4 5 6 (n+1) 1 − U(k−2)(M+1)+3, (37) . . 10 −10 (n) (n) U − V¯ − U (n+1) (k 1)(M+1) (k 1)(M+1)+M , (k−1)(M+1) = n (38) ¯ ( ) −20 V(k−1)(M+1) 10 unshifted s = 0.9 λm (n) (n) (n) (n+1) ¯ ¯ s m Vk(M+1) = Uk(M+1) + V(k−1)(M+1)+M − U(k−1)(M+1), = 0.99 λ −30 (39) 10 s = 0.999 λm U (n) V¯ (n) (n+1) (k−1)(M+1)+1 k(M+1) 3 (0) U = , (40) Fig. 1. Convergence of P e − . (k−1)(M+1)+1 ¯ (n) i=1 i,m 1 V(k−1)(M+1)+1 ¯ (n) (n) ¯ (n) (n+1) Vk(M+1)+1 = Uk(M+1)+1 + Vk(M+1) − U(k−1)(M+1)+1. or 0.999λm here, because we have not devised an ef- (41) ficient shifting strategy yet. So the convergence of the shifted algorithm should still be linear, though it should Here, the ranges of k in (34) through (41) are the same as be faster than that of the unshifted algorithm. those in (16) through (21), (24) and (25). Corresponding In Fig. 1, we plotted the sum of ei,m− (1 ≤ i ≤ 3) as to (15), (22) and (23), we make the following definitions: 1 a function of the iteration number. As can be seen, this (n) (n) 1 quantity decays faster as s approaches λ , showing the V¯ = U + , (42) m 1 1 δ(n) effect of the shift in accelerating the convergence. (n+1) (n+1) In the shifted algorithm, e and q remained positive U = · · · = U = 0, (43) ik k −M+1 0 throughout the computation. This confirms the predic- (n) (n) tion of Theorem 1. Also, the computed smallest eigen- U(m−1)(M+1)+2 = · · · = Um(M+1) = 0. (44) value agreed to the correct value to 15 decimal digits, Eqs. (34) through (41) can be written succinctly as indicating relative accuracy of the shifted algorithm. U (n) V¯ (n) ¯ (n) (n) ¯ (n) k−M−1 k−1 6. Conclusion Vk = Uk + Vk−1 − , (45) V¯ (n) k−M−1 In this paper, we showed that the origin shift can be (n) ¯ (n) incorporated into the multiple dqd algorithm when the (n+1) Uk Vk M U = + . (46) target matrix is a TN Hessenberg matrix. It remains as k ¯ (n) Vk the subject of our future research to prove the relative where 1 ≤ k ≤ m(M + 1) in (45) and 1 ≤ k ≤ (m − accuracy of the algorithm theoretically. 1)(M + 1) + 1 in (46). Moreover, (42) and (43) can be transformed into the following conditions: Acknowledgments 1 We are grateful to the anonymous referee, whose com- V¯ (n) = V¯ (n) = · · · = V¯ (n) = , (47) −M −M+1 0 δ(n) ments helped us to improve the quality of this paper. (n) (n) (n) We also would like to thank Prof. Masashi Iwasaki, U−M = U−M+1 = · · · = U0 = 0. (48) Prof. Satoshi Tsujimoto, Prof. Yoshimasa Nakamura, Eqs. (45) through (48) are nothing but the discrete hun- Ms. Akiko Fukuda and Mr. Kensuke Aishima for valu- gry Lotka-Volterra (dhLV) system [5], if we set V (n) = able discussion. This work is partially supported by k the Ministry of Education, Science, Sports and Culture, δ(n)V¯ (n) k . However, in the context of the dhLV system, Grant-in-aid for Scientific Research. the parameter δ(n) has the meaning of step size and must be positive. On the other hand, in the multiple dqd al- References gorithm, we want to set δ(n) negative (that is, set s positive) to accelerate the convergence. We showed that pos- [1] P. Koev, Accurate eigenvalues and SVDs of totally nonnega- 27 itivity of the bidiagonal factors is retained in this case if tive matrices, SIAM J. Matrix Anal. Appl., (2005), 1–23. [2] Y. Yamamoto and T. Fukaya, Differential qd algorithm for to- s is smaller than the smallest eigenvalue of A. tally nonnegative band matrices: convergence properties and error analysis, JSIAM Letters, 1 (2009), 56–59. 5. Numerical results [3] M. Gasca and J. M. Peña, Total positivity and Neville elimi- nation, Lin. Alg. Appl., 165 (1992), 25–44. To confirm our analysis in Section 3, we performed a [4] K. V. Fernando and B. N. Parlett, Accurate singular values preliminary numerical experiment. We set m = 10 and and differential qd algorithms, Numer. Math., 67 (1994), 191– M = 3 and generated eik and qk using random num- 229. bers in (0, 1]. For this matrix, the smallest eigenvalue is [5] A.Fukuda, E.Ishiwata, M.Iwasaki and Y.Nakamura, The dis- −4 crete hungry Lotka-Volterra system and a new algorithm for λm = 1.24375120694785 × 10 [2]. We applied the mul- computing matrix eigenvalues, Inverse Problems, 25 (2009), tiple dqd algorithm with and without the origin shift to 015007. this matrix. Ideally, the shift s should be updated at each [6] C. Li and R. Mathias, Interlacing inequalities for totally non- iteration. However, we fixed s to one of 0.9λm, 0.99λm, negative matrices, Lin. Alg. Appl., 341 (2002), 35–44.

– 72 –