arXiv:math/0612309v1 [math.RA] 12 Dec 2006 iain 7 ,9 0 1 2)o h usa dmoetsho,adm and school, idempotent Russian fra the the in of considered 12]) be 11, to 10, are 9, papers 8, These [7, 6]. lications [5, in Maslova G.L by E.V. introduced and was semirings. semiring over over algorithm universal equations of of notion The systems linear under the elimination, to G Gaussian applied and be the [3] can that Gondran proving M. by graphs. precise on algorithms observation some for pivo prototype without a elimination as Gaussian the Carr´e[1, 2], B.A. by observed As Introduction 1 ocwSaeUiest,199 eisi oy S-,Moscow, GSP-2, Gory, Leninskie 119992 University, State Moscow nvra loihsfrgnrlzddsrt discrete generalized for algorithms Universal ∗ † ela qain ihsmercTelt matrix Toeplitz symmetric with equations Bellman eerhspotdb FRgat0-1084adjitRFBR/ joint and 05-01-00824 grant RFBR by supported Research eateto hsc,SbDprmn fQatmSaitc a Statistics Quantum of Sub-Department Physics, of Department ftowl-nw ehd ovn opizssesi h or the in systems Toeplitz solving methods a well-known algorithms two The of matrix. Toeplitz symmetric with equations generaliz for algorithms universal two presents paper This Serge˘ı Sergeev uy2,2018 28, July Abstract 1 † usa -al [email protected] E-mail: Russia. ddsrt arxBellman matrix discrete ed dFedTer,MV Lomonosov M.V. Theory, Field nd NSgat05-01-02807. grant CNRS esmrn extensions semiring re iaylna algebra. linear dinary ivnv ..Maslov V.P. Litvinov, . oe[]md this made [4] Rote . eti conditions, certain igcnb viewed be can ting r eeal,in generally, ore eoko pub- of mework ∗ the framework of idempotent and tropical mathematics, see [13, 14, 15] and references therein. Essentially, an algorithm is called universal if it does not depend on the com- puter representation of data and on a specific realization of algebraic operations involved in the algorithm [6]. Linear algebraic universal algorithms include generalized bordering method, LU- and LDM-decompositions for solving matrix equations. These methods are basically due to B.A. Carr´e, see also [6]. It was observed in [5, 6] that universal algorithms can be implemented by means of objective-oriented programming supported by C++, MATLAB, Scilab, Maple and other computer systems and languages. Such universal programs can be instrumental in many areas including the problems of linear algebra, optimization theory, and interval analysis over positive semirings, see [5, 11, 12, 16]. This paper presents new universal algorithms based on the methods of Durbin and Levin- son, see [17], Sect. 4.7. These algorithms solve systems of linear equations with symmet- ric Toeplitz matrices. Our universal algorithms have the same computational complexity O(n2) as their prototypes which beats the complexity O(n3) of the LDM-decomposition method. All algorithms are described as MATLAB-programs, meaning that they can be actually implemented. The author is grateful to G.L. Litvinov and A.N. Sobolevski˘ıfor drawing his attention to this problem and for valuable discussions.

2 Semirings and universal algorithms

A set S equipped with addition ⊕ and multiplication ⊙ is a semiring (with unity) if the following axioms hold:

1) (S, ⊕) is a commutative semigroup with neutral element 0;

2) (S, ⊙) is a semigroup with neutral element 1 =6 0;

3) a ⊙ (b ⊕ c)=(a ⊙ b) ⊕ (a ⊙ c), (a ⊕ b) ⊙ c = (a ⊙ c) ⊕ (b ⊙ c) for all a, b, c ∈ S (distributivity);

2 4) 0 ⊙ a = a ⊙ 0 = 0 for all a ∈ S.

In the sequel, we omit the notation ⊙ whenever this is convenient and does not lead to confusion.

The semiring S is called idempotent if a ⊕ a = a for any a ∈ S. In this case ⊕ induces the canonical partial order relation

a  b ⇔ a ⊕ b = b. (1)

The semiring S is called complete (cf. [18]), if any subset {xα} ⊂ S is summable and the infinite distributivity

c ⊙ ( xα)= (c ⊙ xα), α α (2) x c x c . ( αLα) ⊙ = Lα( α ⊙ ) holds for all c ∈ S and {xα} ⊂ LS. This propertyL is natural in idempotent semirings and also in the theory of partially ordered spaces (cf. G. Birkhoff [19]) with partial order (1). Complete idempotent semirings are called a-complete (cf. [9]).

Consider the closure operation ∞ a∗ = ai. (3) i=0 M In the complete semirings it is defined for all elements. The property

a∗ = 1 ⊕ aa∗ = 1 ⊕ a∗a, (4) reveals that the closure operation is a natural extension of (1 − a)−1.

We give some examples of semirings living on the set of reals R totally ordered by ≤: the semiring R+ with customary operations ⊕ = +, ⊙ = · and neutral elements 0 = 0 and

1 = 1; the semiring Rmax = R ∪ {−∞} with operations ⊕ = max ⊙ = +, and neutral elements 0 = −∞, 1 = 0; the semiring Rmax = Rmax ∪ {∞}, which is a completion of

Rmax with the element ∞ satisfying a ⊕∞ = ∞ for all a, a ⊙∞ = ∞ ⊙ a = ∞ for a =6 0 b and 0 ⊙∞ = ∞ ⊙ 0 = 0; the semiring Rmax,min = R ∪ {∞} ∪ {−∞} with ⊕ = max, ⊙ = min, 0 = −∞, and 1 = ∞.

3 ∗ −1 Consider operation (3) for the examples above. In R+ the closure a equals (1 − a) if a< 1 and is undefined otherwise; in Rmax it equals 1 if a ≤ 1 and is undefined otherwise; ∗ ∗ ∗ in Rmax we have a = 1 for a ≤ 1 and a = ∞ for a> 1; in Rmax,min we have a = 1 for all a. Note that Rmax and Rmax are a-complete, so the closure is defined for any element b of these semirings. b The matrix operations ⊕ and ⊙ are defined analogously to their counterparts in linear algebra. Denote by Matmn(S) the set of all m × n matrices over the semiring S. By In we denote the n × n unity matrix, that is, the matrix with 1 on the diagonal and 0 off the 0 diagonal. As usual, we have AIn = InA = A and A = In for any A ∈ Matnn(S). The set

Matnn(S) of all n × n square matrices is a semiring. Its unity is In and its zero is 0n, the with all entries equal to 0. If S is complete and/or idempotent, then so is ∗ the semiring Matnn(S). If S (and hence Matnn(S)) is complete, the closure A is defined for any matrix A and it satisfies (4). Note that if S is partially ordered, then Matmn(S) is ordered elementwise: A  B iff Aij  Bij for all i = 1,...,m and j = 1,...,n. If S is idempotent and canonically ordered (1), then the elementwise order of Matmn(S) also satisfies (1).

The closure operation of matrices is important for the (discrete stationary) matrix Bellman equations X = AX ⊕ B. (5)

If the closure of A exists and (4) holds, then X = A∗B is a solution to (5). In a-complete idempotent semirings the matrix A∗B is the least solution of this equation with respect to (1).

Since A∗ is a generalization of (I − A)−1, the known universal algorithms for A∗ are generalizations of the methods for matrix inverses, and the known algorithms for Bellman equations are generalizations of the methods for AX = B. Further we consider the generalized bordering method.

Let A be a square matrix. Closures of its main submatrices Ak can be found inductively. ∗ The base of induction is A1, the closure of the the first diagonal entry. Generally, we

4 represent Ak+1 as Ak gk A = , k+1 T hk ak+1 ! assuming that we have found the closure of Ak. In this representation, gk and hk are ∗ columns with k entries and ak+1 is a scalar. We also represent Ak+1 as

U v A∗ k k . k+1 = T wk uk+1 !

Using (4) we obtain that

T ∗ ∗ uk+1 =(hk Akgk ⊕ ak+1) , ∗ vk = A gkuk , k +1 (6) T T ∗ wk = uk+1hk Ak, ∗ T ∗ ∗ Uk = Akgkuk+1hk Ak ⊕ Ak.

Consider the bordering method for finding the solution x = A∗b to (5), where X = x and (1) ∗ (k) B = b are column vectors. Firstly, we have x = A1b1. Let x be the vector found after (k − 1) steps, and let us write

z x(k+1) = . xk+1 !

Using (6) we obtain that T (k) xk = uk (h x ⊕ bk ), +1 +1 k +1 (7) (k) ∗ (k+1) z = x ⊕ Akgkxk+1 .

∗ We have to compute Akgk. In general it makes a problem, but not in the case of the next section when A is symmetrical Toeplitz.

We also note that the bordering method described by (6) and (7) is valid more generally over Conway semirings, see [18] for the definition.

5 3 Universal algorithms for Toeplitz linear systems

Formally, a matrix A ∈ Matnn(S) is called (generalized) Toeplitz if there exist scalars r−n+1,...,r0,...,rn−1 such that Aij = rj−i for all i and j. Informally, Toeplitz matrices are such that their entries are constant along any line parallel to the main diagonal (and along the main diagonal itself). For example,

r0 r1 r2 r4

 r−1 r0 r1 r2  A = (8)  r−2 r−1 r0 r1     r−3 r−2 r−1 r0      is Toeplitz. Such matrices are not necessarily symmetric. However, they are always persymmetric, that is, symmetric with respect to the inverse diagonal. This property is T algebraically expressed as A = EnA En, where En = [en,...,e1]. By ei we denote the 2 column whose ith entry is 1 and other entries are 0. The property En = In (where In is the n × n ) implies that the product of two persymmetric matrices is persymmetric. Hence any degree of a persymmetric matrix is persymmetric, and so is the closure of a persymmetric matrix. Thus, if A is persymmetric, then

∗ ∗ T EnA =(A ) En. (9)

Further we deal only with symmetric Toeplitz matrices. Consider the equation y = (n) (n) T Tny ⊕ r , where r = (r1,...rn) and Tn is defined by the scalars r0,r1,...,rn−1 so that Tij = r|j−i| for all i and j. This is a generalization of the Yule-Walker problem [17]. (k) (k) Assume that we have obtained a solution y to the system y = Tky ⊕ r for some k such that 1 ≤ k ≤ n − 1, where Tk is the main k × k submatrix of Tn. We write Tk+1 as

(k) Tk Ekr T = . (k+1) (k)T r Ek r0 ! We also write y(k+1) and r(k+1) as z r(k) y(k+1) = , r(k+1) = . αk ! rk+1 !

6 ∗ (k) (k) Using (7), (9) and the identity Tk r = y , we obtain that

(k)T (k) ∗ (k)T (k) αk =(r ⊕ r y ) (r Eky ⊕ rk ), 0 +1 (10) (k) (k) z = Eky αk ⊕ y .

(k)T (k) Denote βk = r0 ⊕r y . The following argument shows that βk can be found recursively ∗ −1 if (βk−1) exists.

(k−1) (k−1) (k−1)T Ek−1y αk−1 ⊕ y βk = r0 ⊕ [r rk] = αk−1 ! (11) (k−1)T (k−1) (k−1)T (k−1) = r0 ⊕ r y ⊕ (r Ek−1y ⊕ rk)αk−1 = ∗ −1 2 = βk−1 ⊕ (βk−1) ⊙ (αk−1) .

The argument above is not always valid and this will make us write two versions of our algorithm, the first one involving (11) and the second one not involving it. We will write these two versions in one program and mark the expressions which refer only to the first or only to the second version by the MATLAB-style comments %1 or %2, respectively.

Collecting the expressions for βk,αk and z, we obtain the following recursive expression for y(k): (k)T (k) βk = r0 ⊕ r y , %2 ∗ −1 2 βk = βk−1 ⊕ (βk−1) ⊙ (αk−1) , %1 ∗ (k)T (k) αk =(βk) ⊙ ((r Eky ⊕ rk+1), (12) (k) (k) Eky αk ⊕ y y(k+1) = . αk ! Recursive expression (12) is a generalized version of the Durbin method for the Yule- Walker problem [17]. Using this expression we obtain the following algorithm.

Algorithm 1 The Yule-Walker problem for the Bellman equations with symmetric . function y = durbin(r0,r) n = size(r)+1 ∗ y(1) = r0 ⊙ r(1)

7 β = r0 %1 ∗ α = r0 ⊙ r(1) %1 for k = 1 : n − 1

β = r0 ⊕ r(1 : k) ⊙ y(1 : k) %2 β = β ⊕ (β∗)−1 ⊙ α2 %1 α = β∗ ⊙ (r(k : −1:1) ⊙ y(1 : k) ⊕ r(k + 1)) z(1 : k)= y(1 : k) ⊕ α ⊙ y(k : −1:1) y(1 : k)= z(1 : k) y(k +1)= α end (n) ∗ (n) (n) Now we consider the problem of finding x = Tn b where Tn is as above and b = T (n) (b1,...,bn) is arbitrary. We also introduce the column y which solves the Yule-Walker (n) ∗ (n) (k+1) ∗ (k+1) problem: y = Tn r . The main idea is to find the expression for x = Tk+1b involving x(k) and y(k). We write x(k+1) and b(k+1) as

v b(k) x(k+1) = , b(k+1) = . µk ! bk+1 !

∗ ∗ (k) ∗ (k) Making use of the persymmetry of Tk and of the identities Tk bk = x and Tk rk = y , we specialize expressions (7) and obtain that

(k)T (k) ∗ (k)T (k) µk =(r ⊕ r y ) ⊙ ((r Ekx ⊕ bk ), 0 +1 (13) (k) (k) v = Eky µk ⊕ x .

(k)T (k) ∗ −1 The coefficient r0 ⊕ r y = βk is again to be expressed as βk = βk−1 ⊕ (βk−1) ⊙ 2 ∗ (αk−1) , if the closure (βk−1) is invertible. Using this we obtain the following recursive expression: (k)T (k) βk = r0 ⊕ r y , %2 ∗ −1 2 βk = βk−1 ⊕ (βk−1) ⊙ (αk−1) , %1 ∗ (k)T (k) µk =(βk) ⊙ ((r Ekx ⊕ bk+1), (14) (k) (k) Eky µk ⊕ x x(k+1) = . µk !

8 This expression yields the following generalized version of the Levinson algorithm for solving linear symmetric Toeplitz systems [17]:

Algorithm 2 Bellman system with symmetric Toeplitz matrix. function y = levinson(r0, r, b) n = size(b) ∗ ∗ y(1) = r0 ⊙ r(1); x(1) = r0 ⊙ b(1)

β = r0 %1 ∗ α = r0 ⊙ r(1) %1 for k = 1 : n − 1

β = r0 ⊕ r(1 : k) ⊙ y(1 : k) %2 β = β ⊕ (β∗)−1 ⊙ α2 %1 µ =(r(k : −1:1) ⊙ x(1 : k) ⊕ b(k + 1)) ⊙ β∗ v(1 : k)= x(1 : k) ⊕ µ ⊙ y(k : −1:1) x(1 : k)= v(1 : k) x(k +1) = µ if k < n − 1 α =(r(k : −1:1) ⊙ y(1 : k) ⊕ r(k + 1)) ⊙ β∗ z(1 : k)= y(1 : k) ⊕ α ⊙ y(k : −1:1) y(1 : k)= z(1 : k) y(k +1)= α end end The computational complexity of all methods described in this section is O(n2).

References

[1] B.A. Carr´e. An algebra for network routing problems. J. of the Inst. of Maths. and Applics, 7:273–299, 1971.

9 [2] R.C. Backhouse and B.A. Carr´e. Regular algebra applied to path-finding problems. J. of the Inst. of Maths. and Applics, 15:161–186, 1975.

[3] M. Gondran. Path algebra and algorithms. In B. Roy, editor, Combinatorial pro- gramming: methods and applications, pages 137–148. Reidel, Dordrecht, 1975.

[4] G. Rote. A systolic array algorithm for the algebraic path problem. Computing, 34:191–219, 1985.

[5] G.L. Litvinov and V.P. Maslov. Correspondence principle for idempotent calculus and some computer applications. In J. Gunawardena, editor, Idempotency, Publications of the I. Newton Institute, pages 420–443. Cambridge Univ. Press, 1998.

[6] G. Litvinov and E. Maslova. Universal numerical algorithms and their software implementation. Programming and Computer Software, 26(5):275–380, 2000. E-print arXiv:math.NA/0102144.

[7] V.P. Maslov. M´ethods op´eratorielles. Editions´ MIR, Moscow, 1987.

[8] V.P. Maslov and V.N. Kolokoltsov. Idempotent analysis and its applications. Kluwer Acad. Publ., Dordrecht et al., 1997.

[9] G.L. Litvinov, V.P. Maslov, and G.B. Shpiz. Idempotent functional anal- ysis. An algebraical approach. Math. Notes, 69(5):696–729, 2001. E-print arXiv:math.FA/0009128.

[10] G. Litvinov and G. Shpiz. Dequantization of mathematics and group representations. Vestnik Tamb. Univ., 10(4):390–411, 2005. in Russian.

[11] G.L. Litvinov and A.N. Sobolevski˘ı. Exact interval solutions to the discrete Bell- man equation and polynomial complexity of problems in interval idempotent linear algebra. Doklady Mathematics, 62(2):199–201, 2000.

[12] G. Litvinov and A. Sobolevski˘ı.Idempotent interval analysis and optimization prob- lems. Reliable Computing, 7(5):353–377, 2001. E-print arXiv:math.NA/0101152.

10 [13] G. Litvinov and V. Maslov, editors. Idempotent Mathematics and Mathematical Physics, volume 377 of Contemporary Mathematics. American Mathematical Society, Providence, 2005.

[14] F.L. Baccelli, G. Cohen, G.J. Olsder, and J.P. Quadrat. Synchronization and Lin- earity. Wiley, Chichester, New York, 1992.

[15] R.A. Cuninghame-Green. Minimax Algebra, volume 166 of Lecture Notes in Eco- nomics and Mathematical Systems. Springer, Berlin, 1979.

[16] P. Loreti and M. Pedicini. An object oriented approach to idempotent analysis: Integral equations as optimal control problems. In [13], pages 187–208.

[17] G.H. Golub and C. Van Loan. Matrix computations. The John Hopkins University Press, Baltimore and London, 1989.

[18] J. Golan. Semirings and their applications. Kluwer, Dordrecht, 2000.

[19] G. Birkhoff. Lattice theory. Providence, Rhode Island, 1967.

11