An Algorithm for Polynomial Matrix SVD Based on Generalised

18th European Signal Processing Conference (EUSIPCO-2010) Aalborg, Denmark, August 23-27, 2010

AN ALGORITHM FOR POLYNOMIAL MATRIX SVD BASED ON GENERALISED KOGBETLIANTZ TRANSFORMATIONS John G McWhirter School of Engineering, Cardiff University Queen’s Buildings, CF24 3AA, Cardiff, Wales phone: + (44) (0)2920870627, fax: + (44) (0)2920874716, email: [email protected]

ABSTRACT for conventional matrices, basing an algorithm entirely on An algorithm is presented for computing the singular value the SBR philosophy would not be computationally efficient. decomposition (SVD) of a polynomial matrix. It takes the Given a matrix X ∈ Cm×n, where we assume that m > n, it is form of a sequential best rotation (SBR) algorithm and con- more efficient to begin the SVD by performing a QR decom- stitutes a generalisation of the Kogbetliantz technique for position of the matrix since this only requires a fixed number computing the SVD of conventional scalar matrices. It of carefully structured steps. In this case, a unitary matrix avoids ”squaring” the matrix to be factorised, uses only uni- Q ∈ Cm×m is computed s.t. tary and paraunitary operations, and therefore exhibits a high degree of numerical stability. R = (1) QX 0 1. INTRODUCTION Polynomial matrices have been used for many years in the where R ∈ Cn×n is an upper triangular matrix with real diag- area of control [1]. They play an important role in the re- onal elements. Computation of the SVD can then proceed by alisation of multi-variable transfer functions associated with treating R as a general square matrix and performing an itera- multiple-input multiple-output (MIMO) systems. Over the tive sequence of Kogbetliantz transformations which is guar- last few years they have become more widely used in the anteed to converge and reduce it to diagonal form. The Kog- context of digital signal processing (DSP) and communica- betliantz transformation may be viewed as a generalisation of tions [2]. Typical areas of application include broadband the classical Jacobi algorithm to non-symmetric (square) ma- adaptive sensor array processing [3, 4], MIMO communica- trices and also belongs to the class of SBR algorithms [18]. tion channels [5–7], and digital filter banks for subband cod- It results in a transformation of the form ing [8] or data compression [9]. Just as orthogonal or unitary matrix decomposition tech- Σ niques such as the QR decomposition (QRD), eigenvalue Q1RV = (2) decomposition (EVD), and singular value decomposition Cn×n Cn×n (SVD) [10] are important for narrowband adaptive sensor ar- where Q1 ∈ and V ∈ are unitary matrices and rays [11], corresponding paraunitary polynomial matrix de- Σ ∈ Rn×n is diagonal. In combination, we have compositions are proving beneficial for broadband adaptive arrays [12–14] and also for filterbank design [15, 16]. In Σ = (3) a previous paper [17], we described a generalisation of the UXV 0 EVD for conventional Hermitian matrices to para-Hermitian polynomial matrices. This technique will be referred to as Cm×m PEVD while the underlying algorithm is known as the 2nd where U ∈ is a unitary matrix given by order sequential best rotation (SBR2) algorithm. In order to minimise the number of iterative steps re- Q 0 U = 1 Q (4) quired, and so prevent unnecessary growth in the order of 0 I the polynomial matrix being diagonalised, the philosophy adopted for the algorithm was one of sequential best rotation and so this constitutes the SVD of X [10]. (SBR). In the context of conventional Hermitian matrices, this corresponds to the classical Jacobi algorithm [18]. The The first stage in developing a PSVD algorithm based SBR2 algorithm is, in effect, a generalisation of the classi- on the SBR philosophy, is to generate an SBR algorithm for cal Jacobi algorithm to the third (time) dimension associated the case of conventional matrices. In effect, it is necessary with polynomial matrices. A similar approach was subse- to merge the QR decomposition stage of the SVD algorithm quently adopted for polynomial matrix QR decomposition described above, into the iterative Kogbetliantz process. A (PQRD) [19]. novel algorithm of this type is developed in the next section For similar reasons, the PSVD algorithm outlined here for complex (scalar) matrices. Note that the complex case is is also based on the SBR principle. It can be viewed as the more involved than its real counterpart because of the need generalisation to polynomial matrices of an SBR algorithm to ensure that the diagonal elements remain real throughout for computing the SVD of conventional matrices. Unlike the the process. This is vital for the Jacobi transformation step case of Hermitian matrix EVD, no such algorithm seems to which relies on Hermitian symmetry of the 2 × 2 sub-matrix exist already in the literature. This is not surprising since, to which it is applied.

© EURASIP, 2010 ISSN 2076-1465 457 2. MODIFIED KOGBETLIANTZ ALGORITHM matrix except for the (k,k),(k, j),( j,k) and ( j, j) elements FOR COMPLEX NON-SQUARE MATRICES which serve to embed the 2 × 2 rotation matrix in equation (6). This completes the elementary transformation for the m > n Assuming that , we seek to compute the SVD of a case j > n. In preparation for the next iteration the input ma- matrix ∈ Cm×n as defined by equation (3). Note that for X trix is now updated by setting X ← X0. Note that kX0k = kXk square matrices, it is not necessary to perform the initial QR 2 decomposition as indicated in equation (1) so the basic Kog- and that for each iteration, kdiag(X)k (the on-diagonal en- 2 betliantz method is sufficient. The corresponding PSVD al- ergy) increases while koffdiag(X)k (the off-diagonal en- gorithm can easily be deduced from the one developed below ergy) decreases by the same amount. The operator G(θ,φ) and will not be treated separately in this paper. in equation(11) contributes to the overall transformation U in equation (3). 2.1 Initial phase adjustment The algorithm begins by transforming the matrix to one with 2.3 Complex Kogbetliantz transformation real elements on the principal diagonal. This can be achieved If j ≤ n (i.e. if the dominant off-diagonal element of X lies by multiplying the jth column by exp(−iα j) where α j is the within the upper n × n sub-matrix), perform the following phase of x j j i.e. x j j = x j j exp(iα j). The same procedure complex Kogbetliantz transformation designed to eliminate is applied to every column and so the entire process may be both x and x [18]. This transformation may be broken jk k j written in the form down into three simple steps - a Givens rotation and a symmetrisation followed by a standard Jacobi transformation. X ← XT (5) The Givens rotation step, which would not be required in the where T ∈ Cn×n is a diagonal matrix of phase rotations and is real case, is needed here to ensure that the symmetrisation therefore unitary. Note that kXk is not affected by this initial step leaves the diagonal elements entirely real. First, swap phase adjustment (where k.k denotes the Frobenius norm of the indices j and k, if necessary, so that j > k. It is advisable a matrix or of a polynomial matrix as defined in [17]). The also to swap rows j and k, and columns j and k, so that the algorithm ensures that each subsequent transformation main- Givens rotation which follows, is never used to eliminate a tains the real property of the diagonal elements of the matrix. zero element and thus become numerically ill-defined. This occurs naturally with the Jacobi transformation which preserves Hermitian symmetry, but requires more effort in 2.3.1 Givens rotation other cases. Compute and apply a Givens rotation of the form The new SBR algorithm begins by locating the dominant off-diagonal element of X (i.e. the one with greatest magnitude) denoted by x . The next step depends on whether or iφ 0 0 jk c se xkk xk j xkk xk j not j > n. −iφ = 0 (12) −se c x jk x j j 0 x j j 2.2 Givens rotation 0 R where x j j, xkk and xkk ∈ . This requires the same choice of If j > n (i.e. if the dominant off-diagonal element lies outside rotation parameters as the Givens rotation applied if j > n. the upper n × n sub-matrix), compute and apply a complex However we also have Givens rotation with rotation parameters defined by 0 iω iφ 0 x j j = −se xk j + cx j j (13) c se xkk xkk −iφ = (6) −se c x jk 0 This quantity is not necessarily real, so a further phase ad- 0 0 iβ 0 R justment is required. Denote x j j = x j j e and multiply the where c = cosθ, s = sinθ, and xkk, xkk∈ . This requires jth row of the matrix by e−iβ . This ensures that the matrix −iφ −se xkk + cx jk = 0 (7) still has real elements on the diagonal. Denote the embed- while ded Givens rotation, together with any phase adjusment and interchange of rows and columns, by 0 iφ xkk = cxkk + se x jk (8) X0 = F(θ,φ,β)X (14) Denoting x = x eiω , it is clear that we must have φ = −ω jk jk ( , , ) ∈ Cm×m to ensure that x0 is real. Equation (7) then becomes where F θ φ β . Once again, since this combined k k transformation is unitary, it can be seen that kX0k = kXk and

iω iω 2 2 −se xkk + c x jk e = 0 (9) that kdiag(X)k increases while koffdiag(X)k decreases by the same amount. which is satisﬁed by

2.3.2 Symmetrisation tanθ = x jk /xkk (10) Next, compute and apply a plane rotation with parameters We denote this transformation by chosen s.t. X0 = G(θ,φ)X (11) 0 m×n m×m 0 0 iφ 0 0 0 00 00 where X ∈ C and G(θ,φ) ∈ C takes the form of a c s e xkk xk j xkk xk j 0 = 00 00 (15) suitably embedded Givens rotation [10] - i.e. an m × m unit −s0e−iφ c0 0 x0 x x j j jk j j

458 0 0 0 0 0 0 00 00 R 00 where c = cosθ , s = sinθ , xkk,x j j,xkk,x j j ∈ and xk j = x00∗. This requires jk X000 = J0(θ 00,φ 00)Q(θ 0,φ 0)F(θ,φ,β)XJ(θ 00,φ 00) (26) 0 0 0 0 + 0 iφ 0 = − 0 iφ 0 c xk j s e x j j s e xkk (16) This completes the elementary transformation for the case We also have j ≤ n. In preparation for the next iteration the input matrix is now updated by setting X ← X000. Note, once again, 00 0 0 that k 000k = k k and that, as a result of the Kogbetliantz xkk = c xkk (17) X X transformation, kdiag( )k2 increases while koffdiag( )k2 and X X decreases by the same amount. The combined operator 0 0 J (θ 00,φ 00)Q(θ 0,φ 0)F(θ,φ,β) applied from the left in equa- x00 = −s0e−iφ x0 + c0x0 (18) j j k j j j tion (26) contributes to the overall transformation U in equation (3) while the operator J(θ 00,φ 00) applied from the right Clearly x00 is real. Denoting x0 = x0 eiγ , it can be seen kk k j k j contributes to V. that we require φ 0 = γ to ensure that x00 is real. Equation The process of locating the dominant off-diagonal ele- j j (16) then takes the form ment, and then eliminating it using either a Givens rotation (for j > n ) or a complex Kogbetliantz transformation (for 0 0 iγ 0 iγ 0 0 iγ 0 c xk j e + s e x j j = −s e xkk (19) j ≤ n), is repeated until the dominant off-diagonal element is sufﬁciently small. Again, as for the conventional Kog- which is satisﬁed by betliantz algorithm [18], this process is guaranteed to con- 0 0 0 0 0 verge so that c xk j = −s (xkk + x j j) (20) Σ i.e X → (27) 0 0 0 0 0 tanθ = − xk j /(xkk + x j j) (21) 3. GENERALISATION TO POLYNOMIAL We denote the embedded rotation of equation (15) by MATRICES X00 = Q(θ 0,φ 0)X0 (22) Assume, without loss of generality, that we are given a polynomial matrix of the form where X00 ∈ Cm×n and Q(θ 0,φ 0) ∈ Cm×m. Note once again that kX00k = kX0k. T X(z) = ∑ z−t X(t) (28) 2.3.3 Jacobi transformation t=0 Cm×n Finally, given the matrix X00 resulting from the symmetri- where X(t) (t = 0,1,...T) ∈ and m > n. As before, the sation, compute and apply a Jacobi transformation with the algorithm for square polynomial matrices may be deduced rotation parameters chosen s.t. very simply as a special case of what follows. We seek to compute and apply a transformation of the form

00 00 iφ 00 00 00 00 00 iφ 00 c s e xkk xk j c −s e D(z) 00 00 00 00 U(z)X(z)V(z) = (29) −s00e−iφ c00 x x s00e−iφ c00 0 jk j j 000 xkk 0 where D(z) is an n × n diagonal polynomial matrix and U(z) = 000 0 x j j and V(z) are paraunitary polynomial matrices of dimension m × m and n × n respectively i.e. 00 00 000 000 00 00∗ where xkk,x j j,xkk and x j j are real and xk j = x jk . Since the 000 000 U(z)U(z) = U(z)U(z) = Im (30) Hermitian symmetry is preserved, xkk and x j j are guaranteed to be real. The required rotation parameters are given by and e e 00 00 00 00 tan2θ = 2 xk j /(xkk − x j j) (23) V(z)V(z) = V(z)V(z) = In (31) The tilde notation denotes paraconjugation of a polynomial 00 = 00 = 00 i δ and φ δ where xk j xk j e [10]. We denote the Jacobi matrix as deﬁned in [2].e e transformation by The algorithm described here generalises the modiﬁed

Kogbetliantz algorithm, detailed above, to polynomial ma- X000 = J0(θ 00,φ 00)X00J(θ 00,φ 00) (24) trices in a manner similar to that in which the SBR2 algorithm generalises the classical Jacobi algorithm to para- where J ∈ Cn×n is the embedded transformation represented 000 0 Hermitian polynomial matrices [17]. It begins by transform- above, X ∈ Cm×n and J ∈ Cm×m takes the form ing the polynomial matrix X(z) so that X(0), the coefﬁcient H matrix of order zero (referred to here as the zero-plane ma- 0 J 0 J = (25) 0 I trix), has real diagonal elements. This is accomplished using a diagonal phase rotation matrix of the type represented by The complex Kogbetliantz transformation (including the ad- equation (5). The algorithm continues by locating the domi- ditional triangularisation step) may be expressed in the form nant off-diagonal coefﬁcient in X(z), denoted here by x jk(t),

459 and shifting it to the zero-plane matrix using a pure delay transformation of the form 2 2 2 1 1 1 0 0 0 0 (k,t) (k,t) 0 1 2 0 1 2 0 1 2 X (z) = B1 (z)X(z)B2 (z) (32) 2 2 2 where 1 1 1 0 0 0 0 1 2 0 1 2 0 1 2 Ik−1 0 0 2 2 2 (k,t)(z) = 0 z−t 0 1 1 1 B1 (33) 0 0 0 " 0 0 Im−k # 0 1 2 0 1 2 0 1 2 2 2 2 and 1 1 1 0 0 0 0 1 2 0 1 2 0 1 2 Ik−1 0 0 2 2 2 (k,t) B (z) = 0 zt 0 (34) 1 1 1 2 0 0 0 " 0 0 In−k # 0 1 2 0 1 2 0 1 2 Note that the diagonal elements are not affected by this trans- 0 formation which results in x (0) = x jk(t). jk Figure 1: Input 5 × 3 Polynomial Matrix X(z). Having shifted the dominant off-diagonal coefficient to the zero-plane matrix X0(0), a modified Kogbetliantz transformation, as defined in section 2, is computed with respect to this matrix in order to drive the dominant coefficient to 5 5 5 zero. The sequence of transformations required for the mod- 0 0 0 ified Kogbetliantz transformation is then applied to the entire 0 10 20 0 10 20 0 10 20 polynomial matrix X0(z), i.e. to every coefficient matrix in 5 5 5 0 0 X (z) including X (0). In preparation for the next iteration, 0 0 0 the transformed polynomial matrix X0(z) is then redesignated 0 10 20 0 10 20 0 10 20 as X(z). 5 5 5 The process of locating the dominant off-diagonal co- 0 0 0 0 10 20 0 10 20 0 10 20 efficient, moving it to the zero-plane, and eliminating it by 5 5 5 means of a modified Kogbetliantz transformation, is repeated iteratively until the dominant off-diagonal coefficient is suffi- 0 0 0 0 10 20 0 10 20 0 10 20 ciently small. Noting that kdiag(X(0))k increases monoton- 5 5 5 ically throughout the entire iterative process while kX(z)k remains invariant, it can easily be shown that this procedure 0 0 0 converges to produce a diagonalised matrix Γ(z) of the form 0 10 20 0 10 20 0 10 20 specified on the right hand side of equation (29). The proof follows a similar line of argument to that used to prove the convergence proof of the SBR2 algorithm [17]. Figure 2: Output matrix Γ(z) produced by PSVD algorithm

4. NUMERICAL EXAMPLE 4.1 Comparison with alternative SBR2 method An alternative approach to computing the PSVD in equa- The performance of the PSVD algorithm outlined in section tion (29) is to form the polynomial matrix products P (z) = 3 is illustrated here by means of a simple numerical example. 1 (z) (z) P (z) = (z) (z) × A 5 × 3 polynomial matrix (z) of order 2 was generated X X and 2 X X of dimension 5 5 and X × with the real and imaginary components of its complex co- 3 3 respectively. Using the SBR2 algorithm to compute the PEVD of P (z) and P (z) respectively, serves to gener- efficients selected randomly from a normal distribution with e 1 e2 (z) (z) mean 0 and variance 1. This matrix is depicted in figure 1, ate the paraunitary matrices U and V required for the Γ(z) where the absolute values of the polynomial coefficients are transformation in equation (29). The output matrix produced using this alternative method is depicted in figure 4. plotted in the 5 × 3 array of stem plots. Figure 2 provides a similar representation of the diagonalised polynomial matrix Once again this can be seen to take the required diagonal Γ 2 Γ(z) obtained using the PSVD algorithm after 318 iterations, form. However, koffdiag( (z))k now takes the value 2.32 when the magnitude of the dominant off-diagonal coefficient which is much greater than that achieved using the PSVD al- was < 0.005. The value of this quantity as a function of it- gorithm of section 3. This illustrates the reduced numerical eration number is plotted in figure 3 which shows that the precision which arises from forming the product matrices in convergence of the algorithm is quite rapid but, as expected, the PEVD method. non-monotonic [17]. From figure 2 it can be seen that the output matrix Γ(z) takes the required form. The value of 5. CONCLUSIONS Γ 2 koffdiag( (z))k was found to be 0.0005 as compared to the An algorithm for computing the PSVD of a complex poly- value of kΓ(z)k2 which was 70.81. nomial matrix, based on a generalisation of the Kogbetliantz

460 [4] R. H. Lambert, M. Joho and H. Mathis. Polynomial 2.5 singular values for number of wideband sources esti- mation and principal component analysis. Proc. Int. Conf. Independent Component Analysis, pages 379– 2 383, 2001. [5] S.Y. Kung, Y. Wu and X. Zhang. Bezout space-time 1.5 precoders and equalisers for MIMO channels. IEEE Trans. Signal Processing, 50:2499–2514, Oct 2002. [6] A. Medles and D.T.M. Slock. Linear precoding for spa- 1 tial multiplexing MIMO systems: Blind channel esti- mation aspects. Proc. IEEE Int Conf on Communica- tions, pages 401–404, April 2002. 0.5 [7] L. Rota, P. Comon and S. Icart. Blind MIMO paraunitary equalizer. Proc. IEEE Int Conf on Acoustics, 0 Speech and Signal Processing, April 2003. 0 50 100 150 200 250 300 350 [8] P.P. Vaidyanathan. Theory of optimal orthonormal subband coders. IEEE Trans Signal Processing, 46(6):1528–1543, 1998. Figure 3: Convergence of PSVD algorithm: magnitude of [9] P. Moulin and M.K. Mihcak. Theory and design of the dominant off-diagonal coefﬁcient plotted as a function of signal-adapted FIR paraunitary ﬁlter banks. IEEE iteration number. Trans Signal Processing, 46:920–929, April 1998. [10] G.H. Golub and C.F. Van Loan. Matrix Computations 5 5 5 (Third Edition). The John Hopkins University Press, 1996. 0 0 0 0 10 20 30 0 10 20 30 0 10 20 30 [11] S. Haykin. Adaptive Filter Theory (Fourth Edition). 5 5 5 Prentice Hall, 2002. [12] M. Davies, S. Lambotharan, J.A. Chambers and J.G. 0 0 0 0 10 20 30 0 10 20 30 0 10 20 30 McWhirter. Broadband MIMO beamforming for fre- 5 5 5 quency selective channels using the sequential best rotation algorithm. Proc 67th Vehicular Technology Conf, 0 0 0 0 10 20 30 0 10 20 30 0 10 20 30 Singapore, 2008. 5 5 5 [13] M. Davies, S. Lambotharan, J.A. Foster, J.A. Chambers

0 0 0 and J.G. McWhirter. A polynomial QR decomposition 0 10 20 30 0 10 20 30 0 10 20 30 5 5 5 based turbo equalization technique for frequency selective MIMO channels. Proc 69th Vehicular Technology 0 0 0 Conf, Barcelona, 2009. 0 10 20 30 0 10 20 30 0 10 20 30 [14] C.H. Ta and S. Weiss. A design of precoding and equalisation for broadband MIMO transmission. Proc 15th Int Conf on Digital Signal Processing, Cardiff, 2007. Figure 4: Output matrix Γ(z) produced by SBR2 algorithm [15] S. Weiss, C.H. Ta and C. Liu. A wiener filter approach to the design of filter bank based single-carrier precoding and equalisation. Power Line Communications and method for complex scalar matrices, has been presented. The Its Applications, pages 493–498, 2007. results obtained for a simple numerical example demonstrate that the new PSVD algorithm can work well in practice. A [16] S. Redif and T. Cooper. Paraunitary filterbank design comparison with the alternative PEVD approach based on the via a polynomial singular value decomposition. Proc. SBR2 algorithm demonstrated, as expected, that the PSVD IEEE Int Conf on Acoustics, Speech and Signal Pro- algorithm generates a more accurate decomposition for the cessing, 2005. same example. [17] J.G. McWhirter, P.D. Baxter, T. Cooper, S. Redif and J.A. Foster. An EVD algorithm for para-Hermitian REFERENCES polynomial matrices. IEEE Trans Signal Processing, 55(6):2158–2169, 2007. [1] T. Kailath. Linear Systems. Prentice-Hall International [18] G.E. Forsythe and P. Henrici. The cyclic jacobi method Inc., 1980. for computing the principal values of a complex matrix. [2] P.P. Vaidyanathan. Multirate Systems and Filter Banks. Trans. Amer. Math. Soc., 94:1–23, 1960. Prentice Hall, 1993. [19] J.A. Foster, J.G. McWhirter, M.R. Davies and J.A. [3] N. Delfosse and P. Loubaton. Adaptive blind separa- Chambers. An algorithm for calculating the QR and tion of independent sources: A second order stable al- singular value decompositions of polynomial matrices. gorithm for the general case. IEEE Trans. Circuits and IEEE Trans Signal Processing, 58(6), March, 2010. Systems, 47:1056–1070, July 2000.

461