Geometrical aspects of linear prediction algorithms Franc¸ois Desbouvries D´epartement Signal et Image Institut National des T´el´ecommunications 9 rue Charles Fourier, 91011 Evry, France [email protected] Keywords: Schur and Levinson-Szeg¨o algorithms, Schur On the other hand, Toeplitz forms were studied indepen- complements, spherical trigonometry, analytic interpolation dently by Szeg¨o, who introduced a set of orthogonal poly- theory, Lobachevski geometry nomials with respect to an (absolutely continuous) positive measure on the unit circle. These polynomials obey a two- terms recursion [2] involving a set of parameters of modulus Abstract bounded by one, which later on were recognized to be equal to the Schur parameters [3]. In the 1940s, Toeplitz forms re- In this paper, an old identity of G. U. Yule among partial cor- ceived a revived interest in view of their natural occurrence relation coefficients is recognized as being equal to the cosine in the Kolmogorov-Wiener prediction and interpolation the- law of spherical trigonometry. Exploiting this connection en- ory of stationary processes (see e.g., [4, ch. 10], as well as ables us to derive some new (and potentially useful) relations the survey paper [5] and the references therein). Working on among partial correlation coefficients. Moreover, this obser- Wiener’s solution of the continuous time prediction problem, vation provides new (dual) non-Euclidean geometrical inter- Levinson proposed a fast algorithm for solving Toeplitz sys- pretations of the Schur and Levinson-Szeg¨o algorithms. tems; later on, the Levinson recursions were recognized as being the recurrence relations of Szeg¨o. 1 Introduction Finally, there was an intense activity in these fields beginning in the late 70s, mainly towards the development Linear prediction and interpolation is a major tool in time se- of fast algorithms for numerical linear algebra, on the one ries analysis and in signal processing. In this context, the hand, and in the domain of analytic interpolation theory, Schur and Levinson-Szeg¨o algorithms compute the partial on the other hand. Through these new developments and autocorrelation function of a wide-sense stationary process. extensions, new connections with other mathematical topics As such, they have found a large variety of electrical engi- and disciplines were developed, including among others neering applications. displacement rank theory, J lossless transfer functions, Let us briefly recall the history of these algorithms. At the modern analytic function theory and operator theory. The beginning of the century, Schur, Carath´eodory and Toeplitz literature on these connections and extensions is vast; the were active in such fields as analytic function theory, Toeplitz reader may refer for instance to the papers [6] [7] [8] [9] and forms and moment problems. In 1917, Schur developped books [10] [11]. a recursive algorithm for checking whether a given func- P 1 k The mathematical environment of these algorithms is thus s(z )= s z tion k is analytic and bounded by one k =0 in the unit disk [1]. Such functions are characterized by very rich, and these various interactions have already been a sequence of parameters of modulus less than one (the thoroughly investigated by many researchers. In this wealthy Schur parameters) which are computed recursively from the context, our contribution in this paper consists in exhibiting new unnoticed connections with spherical trigonometry. s power series coefficients k by an elegant algorithm. On the (z )= other hand, Carath´eodory and Toeplitz showed that c P 1 k As far as geometry is concerned, the Lobachevski geom- c +2 c z 0 k is analytic and has positive real part for k =1 P n etry was already known to be a natural environment of the jz j < 1 a b c j i if and only if the Toeplitz forms i , j i;j =0 Schur and Levinson-Szeg¨o algorithms, since the core of these c = c n n with , are positive for all . Let n algorithms mainly consists in a linear fractional transforma- c(z ) c 1+s(z ) 0 tion leaving the unit circle invariant. However, a new point of s(z )= () c(z )=c ; 0 (1) (z )+c 1 s(z ) c view is obtained when considering the algorithms (via posi- 0 tive definite Toeplitz forms) in the particular context of their s(z )j 1 c(z ) since j if and only if has positive real part, the application to linear prediction. Then, up to an appropriate Schur algorithm implicitely enables to test whether a Toeplitz normalization, the Schur and Levinson-Szeg¨o algorithms be- form is positive. come trigonometric identities in a spherical triangle. Since 2 P the real projective 2-space I is the quotient space obtained from the sphere by identifying antipodal points, we see that the alternate non-Euclidean geometry with constant curva- ture (i.e., the elliptic one) is indeed another natural geometri- AAAAAA cal environment of the Schur and Levinson-Szeg¨o algorithms as well. A AAAAAAA A Let us briefly outline the underlying mechanisms lead- X g f AAAAAAAAAAAAAAAAAAAAAAAA ing to this new interpretation. Let i be zero-mean A 1:n ^ square-integrable random variables, X the best lin- j b n AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA X fX g i ear mean-square estimate of j in terms of , i=1 1:n 1:n ^ ~ c X = X X and j the corresponding estima- O j j tion error. The partial correlation coefficient (or par- AAAAA AAA AA n X fX g X C n+1 i cor) of 0 and , given , is defined as =1 i AAAAAA 1:n 1:n 2 1=2 1:n 1:n 1:n 2 1=2 ~ ~ ~ ~ C =[E (X ) ] E (X X )[E (X ) ] . ;n+1 0 0 n+1 n+1 0 B It is bounded by 1 in magnitude and is classicaly interpreted AAAAAA B a X X n+1 as the correlation coefficient of 0 and , once the in- n fX g fluence of i has been removed. In 1907, G. U. Yule =1 i AAAAAA [12] showed that the parcors could be computed recursively : AAAAAAAAAAA 1:n1 1:n1 1:n1 0;n+1 0;n n;n+1 1:n q q : = (2) 0;n+1 1:n1 1:n1 2 2 1 ( ) 1 ( ) 0;n n;n+1 It happens that this well known formula is formally equal to the fundamental cosine law of spherical trigonometry : Figure 1: The spherical triangle ABC. cos a cos b cos c cos A = ; (3) b sin c sin Let us now turn to the organization of this paper. Non- which gives an angle of a spherical triangle, in terms of its Euclidean hyperbolic aspects of the Schur algorithm are im- three sides (see figure 1). This observation establishes an plicit in [15] but do not seem otherwise to be well known. unexpected link between statistics and time-series analysis, Yet the Lobachevski geometry is, by construction, an essen- on the one hand, and spherical trigonometry (a branch of tial feature of the algorithm, which deserves to be better ap- trigonometry), on the other hand. preciated. More precisely, we show in section 2 that Schur’s In former papers [13] [14], spherical trigonometry was layer-peeling type solution to the Carath´eodory problem nec- shown also to admit a close connection with the electrical essarily makes use of automorphisms of the unit disk which, engineering topic of recursive least-squares adaptive filtering on the other hand, happen to be the direct isometries of the which, as linear regression analysis, is a mean square ap- Lobachevski plane. proximation problem. Now, the Schur and Levinson-Szeg¨o The last two sections are devoted to the new geometrical algorithms can be written as algebraic recursions within a co- interpretations in terms of spherical trigonometry. So in sec- n +1) variance matrix or its inverse; due to the identification (2) = tion 3, we relate recursive regressions within a set of ( (3), they admit a connection with spherical trigonometry as random variables, algebraic manipulations in a covariance well. matrix or in its inverse, and spherical trigonometry. We show Indeed, the source of such analogies is that (time- or that adding (resp. removing) a new variable in the regres- order-) recursive least-squares algorithms can be devel- sion problem which, in terms of Schur complements on the oped from projection identities. In linear regression, one covariance matrix (resp. on its inverse), amounts to using recognizes that the mean-square error to be minimized the quotient property [16, p. 279], corresponds in terms of is a distance, so the projection theorem can be applied spherical trigonometry to applying the law of cosines (resp. in the Hilbert space generated by the random variables. the polar law of cosines). Introducing a new variable in the regression problem Lastly in section 4, we further assume that the random amounts to updating a projection operator, and the problem variables are taken out of a stationary time series, and we use can indeed be described in terms of projections in a space the results of section 3 to interpret in parallel the Schur and generated by three vectors. But three unit-length vectors Levinson-Szeg¨o algorithms in terms of spherical trigonom- form a tetrahedron in 3D-space, and deriving projective etry. The Schur (resp. Levinson-Szeg¨o) relations consist in identities in a normalized tetrahedron results in deriving two Schur complement recursions (in the forward and back- trigonometric relations in the spherical triangle determined ward sense) in the original covariance matrix (resp. in its by this tetrahedron (see figure 1, and [14] for details). inverse), and can indeed be interpreted in dual spherical tri- angles.