Signal Processing 74 (1999) 253—277 On the efficient use of Givens rotations in SVD-based subspace tracking algorithms Philippe Pango*, Benoıˆt Champagne INRS-Te& le& communications, Universite& du Que& bec, 16 place du Commerce, Verdun, Que&bec, Canada H3E 1H6 Received 28 July 1997; received in revised form 23 October 1998 Abstract In this paper, the issue of the efficient use of Givens rotations in SVD-based QR Jacobi-type subspace tracking algorithms is addressed. By relaxing the constraint of upper triangularity on the singular value matrix, we show how even fewer Givens rotations can achieve a better diagonalization and provide more accurate singular values. Then, we investigate the efficient use of Givens rotations as a vector rotation tool. The cancellation of cross-terms is presented as an efficient signal/noise separation technique which guarantees a better updating of the subspaces basis. Regarding the choice between inner and outer rotations, we properly use the permutation properties of Givens rotations to maintain the decreasing ordering of the singular values throughout the updating process and analyze the consequences on the tracking performance of QR Jacobi-type algorithms. Finally, based on the developed theory, we propose two efficient subspace tracking algorithms which outperform existing QR Jacobi-type algorithms. Comparative simulation experiments validate the concepts. ( 1999 Elsevier Science B.V. All rights reserved. Zusammenfassung In diesem Beitrag wird die effiziente Verwendung von Givens rotations in SVD-basierten subspace tracking Algorith- men vom QR Jacobi-Typ diskutiert. Wir zeigen, dass durch die Aufgabe der Einschra¨ nkung auf obere Dreiecksstruktur der Singula¨ rwertmatrix mit weniger Givens rotations eine bessere Diagonalisierung and genauere Singula¨ rwerte erzielt werden ko¨ nnen. Weiters wird die effiziente Verwendung von Givens rotations als ein vector rotation Werkzeug diskutiert. Die gegenseitige Aufhebung von Kreuztermen wird als effizientes Signal/Gerau¨ sch Trennungsverfahren pra¨ sentiert, welches ein besseres updating der Unterraumbasis garantiert. Wir demonstrieren den Einfluss der Rotations- art (innere oder au¨ ssere) auf die Reihenfolge der Singula¨ rwerte wa¨ hrend des gesamten Aktualisierungsprozesses. Weiters werden die Konsequenzen der Rotationsart auf die tracking performance diskutiert. Schliesslich werden zwei effiziente subspace tracking Algorithmen vorgeschlagen, die auf den gewonnenen Ergebnissen basieren und besser sind als existierende Algorithmen vom QR Jacobi-Typ. Vergleichende Simulationsergebnisse besta¨ tigen die vorgestellen Kon- zepte. ( 1999 Elsevier Science B.V. All rights reserved. Re´sume´ Dans cet article, nous abordons le proble`me de l’utilisation optimale des rotations de Givens dans les algorithmes de type QR Jacobi pour le suivi de sous-espaces. En enlevant la contrainte de triangularite´ supe´rieure, nous montrons * Corresponding author. Tel.: 514-761-8635; fax: 514-761-8501; e-mail: [email protected] 0165-1684/99/$ — see front matter ( 1999 Elsevier Science B.V. All rights reserved. PII: S 0 1 6 5 - 1 6 8 4 ( 9 8 ) 0 0 2 1 5 - 1 254 P. Pango, B. Champagne / Signal Processing 74 (1999) 253—277 comment un nombre re´duit de rotations de Givens rotations peuvent produire une meilleure diagonalisation de la matrice des valeurs singulie`res et donner des valeurs singulie`res plus pre´cises. Puis, nous analysons l’utilisation efficace des rotations de Givens en tant qu’outils de rotation de vecteurs. L’annulation des termes croise´s est pre´sente´e comme une technique efficace de se´paration des sous-espaces signal et bruit qui garanti une meilleure mise a` jour des bases des sous-espaces. A© propos du choix entre rotations internes et externes, nous exploitons ade´quatement les proprie´te´s permutatives de ces rotations pour maintenir l’ordre de´croissant des valeurs singulie`res lors de la mise a` jour, et analysons les conse´quences sur les performances de suivi des algorithmes de type QR Jacobi. Finalement, en se basant sur la the´orie de´veloppe´e, nous proposons deux algorithmes efficaces de suivi de sous-espaces dont les performances exce`dent celles des algorithmes de type QR Jacobi existant. Des simulation expe´rimentales valident les concepts pro- pose´s. ( 1999 Elsevier Science B.V. All rights reserved. Keywords: Subspace tracking; Invariant subspace updating; Singular-value decomposition; Givens rotation; Cross- terms; Direction of arrival Notation various decompositions, including the eigenvalue a angle of arrival decomposition (EVD) of the sample correlation A data matrix matrix, the QR factorization [2] or the singular- *» 1 time variation of signal subspace value decomposition (SVD) of the data matrix. In * J convergence step induced by the lth rota- the case of non-stationary signals, these decomposi- tion tions need to be updated each time new informa- c efficiency of Givens rotations (EGR) tion is provided by the most recent measurements. GH h GF Givens rotation in i—j plane with angle The quality of the updating process affects the k time index accuracy of the parameter estimates, and its low j forgetting factor complexity may allow a real-time implementation. M number of pairs of rotations at refinement These considerations have led to an intensive re- step search on the development of the so-called sub- N number of sensors space tracking algorithms. Beginning with Owsley off[ ] off-norm of its argument [18], many adaptive subspace tracking algorithms r number of sources have been developed and can be grouped in fami- p G ith singular value lies, depending on the specific technique they use: p , average noise singular value stochastic gradient [35], recursive least squares R singular value matrix [34], perturbation approach [3], to quote a few. º left singular vectors The introduction in [5] is a good source of refer- » right singular vectors ences concerning the development of subspace u electrical angle tracking algorithms. x measurement vector Generally, an SVD-based subspace tracking con- » y update vector, i.e. projection of x on sists in computing the SVD of a data matrix of yG update vector after the ith QR rotation growing dimension, defined recursively as (jA(k!1) A(k)" , (1) 1. Introduction C x&(k) D In the application of subspace methods to array where k is the discrete time index, 0(j(1 is the signal processing, projection techniques like forgetting factor, and x(k)3", is the measurement MUSIC [25], root-MUSIC [1], ESPRIT [24] or vector at time index k. The SVD of A(k) can be minimum-norm [15] are often used to estimate expressed as the signal parameters. These methods require a subspace information which can be provided by A(k)"º(k)R(k)»(k)&, (2) P. Pango, B. Champagne / Signal Processing 74 (1999) 253—277 255 where »(k)isanN;N unitary matrix, º(k)is many questions remain unanswered regarding a k;N matrix with orthonormal columns and R(k) these algorithms. Since they use exclusively Givens is an N;N diagonal matrix. Usually, the left singu- rotations throughout the updating process, one lar vectors º(k) need not be tracked to provide the must analyze the efficiency of these rotations on the subspace information. So, one is interested in track- singular values and singular vectors tracking per- +p *p *2*p , ing only the singular values , formance, and also on the accuracy of the signal along the diagonal of R(k) and the right singular parameter estimates they provide. vectors, i.e. the columns of »(k). Computing this SVD Regarding the singular values, QR Jacobi-type from scratch at each iteration is computationally algorithms usually track an approximate SVD de- expensive and time-consuming. The main issue composition (e.g. URV) in which the singular value consists then in using the information contained in matrix R(k) is given a specific structure; then, Givens the measurement x(k) to update the previous de- rotations are applied throughout the updating algo- composition obtained at time k!1. rithm in order not to destroy that specific structure. In this work, we focus our attention on a specific It might be argued that such a structural constraint family of SVD-based subspace tracking algorithms introduces limitations in the way Givens rotations which we define as QR Jacobi-type algorithms couldbeusedtoachieveabetterupdatingofthe [12,17,23]. QR Jacobi-type algorithms intensively singular values. For example, the above QR Jacobi- make use of Givens rotations in the subspace updat- type algorithms impose R(k)inEq.(2)tobeupper- ing process, the main matrix operations consisting in triangular. The efficiency of Givens rotations assum- products with Givens rotations; thus, they track the ing this specific structure (and eventually others) has subspaces with a low complexity and a rather simple not been investigated yet. Therefore, one notes that formulation. Also, Givens rotations have the ad- the above algorithms use Givens rotations in a way vantage to maintain the orthonormality of matrices, which is not proved to be optimal. assuming an infinite precision. The direct conse- Regarding the singular vectors, QR Jacobi-type quence is that QR Jacobi-type algorithms are good algorithms use Givens rotations to rotate the ap- candidates for implementation in a rather regular proximate singular vectors in the direction of the structure, e.g. using CORDIC processors [27,32]. exact ones. This re-orientation of the singular vec- While the exact SVD-based tracking of »(k)and tors is performed to update a subspace basis which R(k)requiresO(N) operations at each iteration, inputs MUSIC-like estimators and provide the de- Moonen et al. have proposed a QR Jacobi-type sired signal parameter estimates. The accuracy of algorithm with a complexity of O(N) operations per the parameter estimates is linked to the ability of update [17]. In this paper, we shall refer to Moon- the Givens rotations to rotate the proper singular en’s algorithm as MA.