Characterization of Discrete Linear Shift-Invariant Systems

2017 25th European Signal Processing Conference (EUSIPCO)

Michael Clausen and Frank Kurth Fraunhofer FKIE Communication Systems 53343 Wachtberg Germany Email: [email protected], [email protected]

Abstract—Linear time-invariant (LTI) systems are of funda- i.e., C(A) = C[A]. Note that in this case, the centralizer is a mental importance in classical digital signal processing. LTI sys- commutative algebra. If however A is the N × N unit matrix, tems are linear operators commuting with the time-shift operator. then C(A) = CN×N , which is non-commutative for N ≥ 2. For N-periodic discrete time series the time-shift operator is a circulant N × N permutation matrix. Sandryhaila and Moura Thus centralizers can be non-commutative. In fact, we prove developed a linear discrete signal processing framework and in Section IV, that the centralizer of A is commutative if and corresponding tools for datasets arising from social, biological, only if all eigenspaces of A are one-dimensional. Thus if we and physical networks. In their framework, the circulant permu- are in the degenerate case, then not all LSI systems w.r.t. A N ×N A tation matrix is replaced by a network-specific matrix , are of the form h(A), for a polynomial h. Hence the argument called a shift matrix, and the linear shift-invariant (LSI) systems are all N × N matrices H over C commuting with the shift [1] (p. 1647, following Theorem 2) reducing graph filters for matrix: HA = AH. Sandryhaila and Moura described all those the degenerate case to filters in the non-degenerate case does H for the non-degenerate case, in which all eigenspaces of A not hold. are one-dimensional. Then the authors reduced the degenerate In this paper, we give a complete description of LSI sys- case to the non-degenerate one. As we show in this paper this tems by describing the centralizer of arbitrary complex-valued reduction does, however, not generally hold, leaving open one gap in the proposed argument. In this paper we are able to N × N matrices A. We start in Section II with two examples close this gap and propose a complete characterization of all illustrating the degenerate case. To keep this paper to some (i.e., degenerate and non-degenerate) LSI systems. Finally, we extent self-contained, we introduce in Section III notation describe the corresponding spectral decompositions. required in the following and discuss basic tools: associative algebras and their morphisms, minimal and characteristic poly- I.INTRODUCTION nomials, upper triangular Toeplitz matrices, Jordan canonical Linear time-invariant (LTI) systems are linear operators form, and Hermite interpolation. After compiling these tools, commuting with the time-shift operator. Such systems are of we present in Section IV the characterization of centralizers. fundamental importance in classical digital signal processing. Finally, in Section V we describe the spectral decompositions For N-periodic discrete time series the time-shift operator is of the signal space CN w.r.t. C[A] and C(A). the circulant permutation matrix corresponding to the cyclic II.ILLUSTRATING THE DEGENERATE CASE permutation (0, 1,...,N − 1). In [1], Sandryhaila and Moura There are a number of network-specific matrices A with developed a linear discrete signal processing framework and repeated eigenvalues. Here are two examples. corresponding tools for datasets arising from social, biological, and physical networks. In their framework, the circulant Example 1. Figure 1 shows the Petersen graph. permutation matrix is replaced by a network-specific matrix bc CN×N bc A ∈ , the shift matrix, and the linear shift-invariant bc bc bc bc (LSI) systems are all matrices in the centralizer of A: bc bc C(A) := {H ∈ CN×N | HA = AH}. bc bc Fig. 1. Peterson graph This centralizer has an additional algebraic structure, briefly N×N reviewed in Subsection III-A: C(A) is a subalgebra of C . The characteristic polynomial pA(X) := det(XI − A) of its Moreover, C(A) contains the algebra C[A] of all polynomial adjacency matrix A equals (X − 1)5 · (X + 2)4 · (X − 3). N×N expressions in A. Thus C[A] ⊆ C(A) ⊆ C . Thus λ1 = 1 is a 5-fold eigenvalue, whereas λ2 = −2 is a 4- In [1], the authors provide an explicit description of the fold eigenvalue of A. A is symmetric, hence diagonalizable. centralizer of A in the non-degenerate case in which the Thus the dimension of C[A] equals the number of distinct eigenspace of every eigenvalue of A is one-dimensional. More eigenvalues of A, which is 3, whereas the centralizer C(A) is precisely, they showed that in this case the centralizer of A isomorphic to the non-commutative algebra C5×5 ⊕ C4×4 ⊕ coincides with the algebra of all polynomial expressions in A, C1×1, which is of dimension 42. ♦

ISBN 978-0-9928626-7-1 © EURASIP 2017 366 2017 25th European Signal Processing Conference (EUSIPCO)

Example 2. As a second example we consider the adjacency B. Upper Triangular Toeplitz Matrices matrix A of a clique with nodes. Here, m×n = (ai,j ) N ai,i = 0 Recall that a matrix T = (ti,j ) ∈ C is a Toeplitz matrix, and ai,j = 1, for all i and all j 6= i. Figure 2 illustrates the if each descending diagonal from left to right is constant, i.e., case N = 6. ti,j = ti+1,j+1, whenever both sides are defined. Thus every m × n Toeplitz matrix is completely specified by the entries in its leftmost column and its topmost row. We are mainly interested in upper triangular Toeplitz matrices. Such matrices are specified by its topmost row. For a = (a0, . . . , ad−1) ∈ Fig. 2. Clique for N = 6. d d×d C define T d(a) = T d(a0, . . . , ad−1) = (ti,j ) ∈ C by t := 0, if i > j, and t := a , if i ≤ j. Thus A is a circulant matrix. (Recall that an N × N matrix of i,j i,j j−i the form (aj−i mod N )i,j is called circulant.) It is well- a0 a1 a2 . . . ad−2 ad−1 pq a0 a1 a2 ad−2 known that the N × N DFT matrix F = (ω )0≤p,qconvolution. Via expressions in A. d FFT, this multiplication can be performed with O(d log d) III.PRELIMINARIES arithmetic operations. Below, we need the following gener- A. Review of Associative Algebras alization of Td. An associative C-algebra A is a vector space over C with Definition 4. For positive integers d, e let Td,e denote the an associative multiplication and a unit element 1A. Moreover, set of all d × e matrices T = (ti,j )0≤ikernel of the evalua- the entries a , . . . , a in its first row. We indicate this by −1 C 0 e−1 tion map T is defined by T [0] := {h ∈ [X] | h(A) = 0}. writing It consists of all multiples of a uniquely determined monic T = T d,e(a0, . . . , ae−1). polynomial of smallest degree in T −1[0], the so-called minimal polynomial mA(X) of A. By the Cayley-Hamilton Theorem, Thus for i ∈ [0, d − 1] and j ∈ [0, e − 1] the following holds: the characteristic polynomial pA(X) := det(XI − A) of A a , if j − i ≥ max(0, e − d) t = j−i (1) is a multiple of the minimal polynomial. i,j 0 otherwise.

ISBN 978-0-9928626-7-1 © EURASIP 2017 367 2017 25th European Signal Processing Conference (EUSIPCO)

The following observation is obvious but crucial. Proof. Although this is a well-known result, see, e.g., [3], we recall its short proof for the reader’s convenience. Lemma 6. Let d, e, f be positive integers. Then T ·T ⊆ d,e e,f Uniqueness: Let p and q be polynomials of degree at most T . More precisely, if a , b ∈ C then d,f i j N satisfying (2). Then r := p − q is a polynomial of degree T d,e(a0, . . . , ae−1) · T e,f (b0, . . . , bf−1) at most N, having each xi as root of multiplicity di + 1. Thus r has at least (d0 + 1) + (d1 + 1) + ... + (dn + 1) > N roots, T k equals d,f (c0, . . . , cf−1), where ck = j=0 aj · bk−j. In hence r = 0, i.e., p = q. particular, if d < e or e < f, then ck P= 0, for all k ∈ N j [0, max(e − d − 1, f − e − 1)]. Existence: With the ansatz p(x) = j=0 aj x , Equation (2) is equivalent to the following systemP of linear equations: C. Review of Jordan Canonical Form Special upper triangular Toeplitz matrices are involved in Xa = y, the Jordan canonical form of a matrix. The Jordan matrix ⊤ N+1 where a = (a0, . . . , aN ) ∈ C and corresponding to λ ∈ C and d ∈ N is deﬁned by ⊤ CN+1 y = (y0,0, . . . , y0,d0 , . . . , yn,0, . . . , yn,dn ) ∈ . J λ,d := T d(λ, 1, 0,..., 0) ∈ Td. With y in mind, we index the rows of X C(N+1)×(N+1) 1×1 ∈ Note that J λ,1 = (λ) ∈ C . n by the elements in i=0{i} × [0, di] and the columns by j ∈ Theorem 7 (Jordan canonical form). For each A ∈ CN×N [0,N]. Then the entryS at position ((i, k), j) in X satisﬁes j−k there exist an invertible N × N matrix V as well as pairs X(i,k),j = j(j − 1) ··· (j − k + 1) · xi . (λ1, d1),..., (λI , dI ) such that I Consider the special case, where y = 0. With the same −1 0 V · A · V = J λ ,d . reasoning as in the proof of uniqueness we see that a = i i 0 Mi=1 is the unique solution of the homogeneous system Xa = . Thus X is invertible. Hence Xa = y has the unique solution Up to reordering, the so-called Jordan blocks J λi,di are −1 I a = X y. uniquely determined by A. i=1 J λi,di is called the Jordan decomposition of A. Moreover,L each λi is an eigenvalue of Fast (parallel) algorithms for Hermite interpolation via A. divided differences are presented in [4]. Proof. See, e.g., [2]. IV. LINEAR SHIFT-INVARIANT SYSTEMS The next result describes the characteristic polynomial In this section we describe the centralizer of an arbitrary pA(X) and the minimal polynomial mA(X) of A in terms matrix A ∈ CN×N . Conjugate matrices A and B, i.e., of the Jordan canonical form of A. B = V AV −1 for some invertible matrix V , have conjugate centralizers: Theorem 8 (Characteristic versus minimal polynomial). If − I C(B) = V · C(A) · V 1. i=1 J λi,di is the Jordan decomposition of A, then L I Thus we can assume that A is already in Jordan canonical di rλ I pA(X) = (X − λi) and mA(X) = (X − λ) , form: A = i=1 J λi,di . Suppose HA = AH. Writing H = di×dj iY=1 Yλ (Hi,j ) withL blocks Hi,j ∈ C , for all i, j we obtain the where the product runs through all distinct eigenvalues λ of conditions J H H J A and rλ := max{di | λi = λ}. λi,di i,j = i,j λj ,dj . Proof. See, e.g., [2]. To study such conditions more closely, we introduce intertwin- ing spaces: for λ, µ ∈ C and d, e ∈ N let In particular, the minimal polynomial divides the character- d×e istic polynomial. Moreover, both polynomials coincide if and Int(J λ,d, J µ,e) := {H ∈ C | J λ,dH = HJ µ,e}. only if λ1, . . . , λI are pairwise different, i.e., the eigenspace Lemma 10. Let λ, µ ∈ C and d, e ∈ N. of each eigenvalue λi of A is one-dimensional. (a) Int(J λ,d, J λ,e) = Td,e. D. Review of Hermite Interpolation (b) Int(J λ,d, J λ,d) = C(J λ,d) = Td. Theorem 9 (Hermite Interpolation). Let x0, . . . , xn ∈ C be (c) If λ 6= µ, then Int(J λ,d, J µ,e) = {0}. pairwise different, and let d , . . . , d ∈ N. Then for every 0 n Proof. (a) Let H = (h ) ∈ Cd×e. Then HJ = J H if family of complex numbers, y ∈ C, where i ∈ [0, n] and i,j λ,e λ,d ik and only if for all a ∈ [0, d−1] and all b ∈ [0, e−1] the entries k ∈ [0, di], there exists a unique polynomial p of degree at at position (a, b) in HJ λ,e and J λ,dH coincide. Putting most N := n + d0 + ... + dn such that for all (i, k) ha,−1 := 0 and hd,b := 0, for all a and b, a straightforward (k) p (xi) = yik, (2) computation shows that the above condition is equivalent to (k) where p denotes the k-fold derivative of the polynomial p. ha,b−1 + λha,b = λha,b + ha+1,b.

In particular, H is a Toeplitz matrix. Furthermore, for a = d− Step 1: We prove this claim for the special case, where A I 1 we obtain hd−1,b−1 = 0. Hence hd−1,j = 0 = h0,j−(d−1), equals its Jordan decomposition: A = i=1 J λi,di . for all j ∈ [0, e − 2]. Thus H ∈ Td,e. L (b) is a special case of (a). Lemma 15. Evaluating a polynomial h(X) at a Jordan block (c) Now let λ 6= µ. Keeping the above notation and J λ,d yields an upper triangular Toeplitz matrix: conventions we get (0) (d−1) h(J λ,d) = T d h (λ)/0!, . . . , h (λ)/(d − 1)! . (λ − µ) · h = h − h , (3) a,b a,b−1 a+1,b Proof. Using Lemma 3 an easy induction on n proves the for all a ∈ [0, d − 1] and b ∈ [0, e − 1]. In particular, for following formula for the nth power of a Jordan block: a = d − 1, this yields (λ − µ) · hd−1,b = hd−1,b−1. Thus n n n−j J λ,d = T d(a0, . . . , ad−1), where aj = λ . (4) hd−1,0 = 0 and hence hd−1,b = 0 for all b ∈ [0, e − 1]. The j other extreme case, b = 0, yields (λ − µ) · ha,0 = −ha+1,0. L n If h(X) = n=0 hnX , then Thus hd−2,0 = 0 and hence ha,0 = 0 for all a ∈ [0, d−1]. Thus P L the leftmost column and the last row in the matrix (λ − µ)H h(j)(λ) n = h λn−j. (5) are zero. This vanishing of the border in combination with j! nj Equation (3) triggers a domino effect, proving H = 0. nX=0 Hence Now we are prepared to describe the centralizer for matrices m n in Jordan canonical form. h(J λ,d) = hnJ λ,d nX=0 Theorem 11. Let A = J λ,dλ ⊕ ... ⊕ J λ,dλ denote λ 1 mλ L an matrix in Jordan canonical form. Here runs (4) n n − N × N L λ = h T λn, λn 1,... through all distinct eigenvalues of A. Then the centralizer of n d 0 1 nX=0 C C A satisﬁes (A) = λ (A)λ, where L L n n n n−1 L = T d hn λ , hn λ ,... T λ λ ... T λ λ 0 1 d1 ,d1 d1 ,dm λ nX=0 nX=0 C . .. . (A)λ = . . . . (5) T (0) (d−1) Tdλ ,dλ ... Tdλ ,dλ = d h (λ)/0!, . . . , h (λ)/(d − 1)! . mλ 1 mλ mλ Proof. Apply Lemma 10. This proves Lemma 15. I According to Corollary 13 (a), H = i=1 Hi with Example 12. Let A = J λ,3 ⊕J λ,3 ⊕J λ,2 ⊕J µ,4 with λ 6= µ. Hi ∈ Tdi . Thus for suitable yi,k we canL write Hi = Then a typical element of C(A)λ reads as follows T di (yi,0, . . . , yi,di−1). We are looking for a polynomial h(X) a0 a1 a2 b0 b1 b2 c0 c1 of minimum degree, such that for all i ∈ [1,I] 0 a0 a1 0 b0 b1 0 c0 0 0 a0 0 0 b0 0 0 Hi = T di (yi,0, . . . , yi,di−1) d0 d1 d2 e0 e1 e2 f0 f1 . (0) (di−1) 0 d0 d1 0 e0 e1 0 f0 = T di h (λi)/0!, . . . , h (λi)/(di − 1)! . 0 0 d0 0 0 e0 0 0 0 g1 g2 0 h1 h2 i0 i1 By Hermite interpolation, such a polynomial exists and is 0 0 g1 0 0 h1 0 i0 uniquely determined by these conditions. With Lemma 15 we

Two extreme cases deserve special mention. get Hi = h(J λi,di ), thus H = h(J), thereby concluding the I proof of Step 1. Corollary 13. Let A = i=1 J λi,di . C I Step 2: We prove the claim for the general case, where (a) If all λi are pairwiseL different, then (A) = i=1 Tdi . −1 I I×I A = VJV and J = J with pairwise different If J J for all , then C A , i.e., C A i=1 λi,di (b) λi,di = λ,d i ( ) = Td L ( ) −1 is the algebra of all I × I matrices with coefﬁcients in λi. Suppose that HA = LAH. Put H := V HV . Then Td. HJ = V −1HVJV −1V =fV −1HAV −1 −1 −1 If all λi are pairwise different, then all LSI systems w.r.t. f = V AHV = V VJV HV = JH. A are given by polynomial expressions in A: According to Step 1, there exists a polynomial h(fX) = Theorem 14 ([1]). Assume that the characteristic and minimal L n hnX such that H = h(J). Then polynomials of the matrix A are equal. Then the centralizer of n=0 P −1f n −1 A consists of all polynomial expressions in the shift matrix: H = V HV = V hnJ V C(A) = C[A]. Xn f −1 n n Proof. Obviously, if h ∈ C[X] and H = h(A), then HA = = hn(VJV ) = hnA = h(A). AH. Thus it remains to prove that HA = AH implies H = Xn Xn h(A), for some polynomial h. We proceed in two steps. This proves Theorem 14.

CN mλ Our next goal is to characterize the non-degenerate case. (a) Obviously, = λ i=1 Sλ,i is a decomposition of CN into C[J]-invariant subspaces. Thus it remains to show Theorem 16. For A ∈ CN×N the following statements are L L that every S is indecomposable. Suppose, X is a non-zero equivalent: λ,i C[J]-invariant subspace of Sλ,i and let 0 6= x ∈ X. Then λ (a) All eigenspaces of A are one-dimensional. di x = ξj eλ,i,j . Let ξk 6= 0 but ξj = 0, for all j > k. A A j=1 (b) p (X) = m (X). k−1 (c) C(A) = C[A]. Then,P by Equation (6), (J − λI) x = ξkeλ,i,1 ∈ X. Thus C (d) The centralizer of A is commutative. eλ,i,1 is contained in every non-zero [J]-invariant subspace of Sλ,i. This proves the indecomposability of Sλ,i with respect Proof. (a) → (b) follows from Theorem 8. (b) → (c) follows to the algebra C[A]. from Theorem 14. (c) (d): trivial. CN CN → (b) Obviously, = λ Sλ is a decomposition of (d) → (a): We show that ¬ (a) implies ¬ (d). W.l.o.g. let into C(J)-invariant subspaces.L Thus it remains to show that A = J λ,d ⊕ J λ,e ⊕ ... Then, by Theorem 11, every Sλ is indecomposable. Suppose, X is a non-zero C(J)- invariant subspace of Sλ and let 0 6= x ∈ X. Write Td,d Td,e ... 0 T T ... x = i xi, with xi ∈ Sλ,i. Suppose, xi 6= . Project x C(A)λ = e,d e,e . . . . via aP suitable element in C(J) onto xi. Then xi ∈ X, and . .. . as C[J] ⊆ C(J) we know by (a), that also eλ,i,1 ∈ X. Now a straightforward computation shows that the upper left Combining Theorem 11 with the constellation of the bold entries in Example 12, it follows that eλ,k,1 ∈ X, for all 2 × 2 block Td,d Td,e is not commutative. Te,d Te,e k corresponding to Jordan blocks J λ of maximal size, λ,dk i.e., dλ = max{dλ | i ∈ [1, m ]}. As those e are The results of this section yield a complete picture of all k i λ λ,k,1 independent of x, we see that those vectors are contained in possible LSI systems. Moreover, Theorem 16 yields a precise every non-zero C(J)-invariant subspace of S . This proves separation of the non-degenerate and the degenerate cases. λ the indecomposability of Sλ w.r.t. the algebra C(J). V. SPECTRAL DECOMPOSITIONS Note that in the non-degenerate case (all mλ = 1) both Let A ∈ CN×N . Then both C[A] and C(A) are sub- spectral decompositions coincide. N×N algebras of C . By the Krull-Remak-Schmidt Theorem VI.CONCLUSIONS CN (see, e.g., [2]), the signal space can be written (in an We have characterized all linear shift invariant systems cor- essentially unique way) as a direct sum of indecomposable responding to a given shift matrix. Our result extends the result C C [A]-invariant and (A)-invariant subspaces, respectively. As of Sandryhaila and Moura, who described the non-degenerate C C C [A] is a subalgebra of (A), the indecomposable (A)- case. It turned out that in general the centralizer of a matrix invariant subspaces further decompose into the direct sum of A in Jordan canonical form has a nested block structure: the indecomposable C[A]-invariant subspaces. We are going to −1 macro blocks correspond to the distinct eigenvalues λ of A. describe these spectral decompositions. Let A = VJV ∈ The micro blocks within the λ-block correspond to pairs of CN×N with Jordan canonical form J J , where, for = λ λ Jordan blocks of A corresponding to the same λ. Each micro each eigenvalue λ of A, J λ = J λ,dλ ⊕ ...L⊕ J λ,dλ . In view 1 mλ block is a space of rectangular upper Toeplitz matrices. The of the structure of J, the columns of V can be parametrized examples in Section II indicate that in the degenerate case λ as V λ,i,j , where i ∈ [1, mλ] and j ∈ [1, di ]. W.l.o.g. we can the algebra C(A) of all LSI systems can be of much higher CN assume that the columns of V are a canonical basis of dimension than the subalgebra of all polynomial expressions (also called a graph Fourier basis in [1]), i.e., these columns in A. This opens new and versatile possibilities for the design are composed entirely of Jordan chains, see, e.g., [2]. This of linear shift invariant systems in Digital Signal Processing. means that for all λ, i, j, and all p ≤ 0 The spectral decomposition in Theorem 17 (b) unifies the non-degenerate and the degenerate case. In both cases there V λ,i,j−1 = (A − λI)V λ,i,j , and V λ,i,1 6= 0 =: V λ,i,p. (6) is only one spectral component of frequency λ, for each λ mλ Let Sλ,i := span{V λ,i,j | j ∈ [1, di ]} and Sλ := i=1 Sλ,i. eigenvalue λ of A. The spectral decomposition in Theorem 17 Theorem 17 (Spectral decompositions). With theL above no- (a) seems to be problematic for the degenerate case, for there tation, the following holds. are several spectral components corresponding to the same CN mλ CN frequency. (a) = λ i=1 Sλ,i is the decomposition of into REFERENCES C indecomposableL L [A]-invariant subspaces. [1] Aliaksei Sandryhaila and Jose´ M. F. Moura. Discrete signal processing CN CN (b) = λ Sλ is the decomposition of into indecom- on graphs. IEEE Trans. Signal Processing, 61(7):1644–1656, 2013. posable C(A)-invariant subspaces. [2] Serge Lang. Algebra, volume 211 of Graduate Texts in Mathematics. L Springer, revised third edition, 2002. Proof. W.l.o.g. it suffices to prove our claims for the special [3] Richard L. Burden and J. Douglas Faires. Numerical analysis. Thomson Brooks/Cole, 2005. case when A is already in Jordan canonical form: A = J. [4] Omer¨ Egecioglu, Efstratios Gallopoulos, and Çetin Kaya Koç. Fast Thus V = I is the unit matrix and the unit vectors form a computation of divided differences and parallel Hermite interpolation. N J. Complexity, 5(4):417–437, 1989. canonical basis of C . We will write eλ,i,j instead of V λ,i,j .