FAST QRS DECOMPOSITION OF AND ITS APPLICATIONS TO NUMERICAL OPTIMIZATION

JIAN-KANG ZHANG∗ AND KON MAX WONG

Abstract. In this paper, we extend our recently developed equal diagonal QRS decomposition of a matrix to a general case in which the diagonal entries of the R-factor are not necessarily equal but satisfy a majority relationship with the singular values of the matrix. A low-complexity quadratic recursive algorithm to characterize all the eligible S-factors is derived. In particular, a pair of fast closed-form QRS decompositions for both a matrix and its inverse is obtained. Finally, we show how to apply this generalized QRS decomposition to efficiently solving two classes of numerical opti- mization problems that usually occur in the joint design of transceiver pairs with decision feedback detection in communication systems via semi-definite and geometrical programming.

Key words. QR decomposition, majority, QRS decomposition, Q-R-S-factors, semi-definite and geometrical programming

1. Introduction. The QR decomposition and the singular value decomposition (SVD) are commonly used tools in various signal processing applications. The QR decomposition of a matrix A is a factorization [1, 2] A = QR, where Q is a and R is an upper , while the SVD decomposition of an M × N matrix A is a factorization [1, 2] µ ¶ Λ1/2 0 A = U r×(N−r) VH , (1.1) 0(M−r)×r 0(M−r)×(N−r) where r is the of A, U and V are unitary and Λ = diag(λ1, λ2, ··· , λr) with λ1 ≥ λ2 ≥ · · · λr > 0. To preserve the virtues of both decompositions, an intermediary between the QR decomposition and the QR decomposition, a two-sided orthogonal decomposition URV was developed in [2, 3, 4]; i.e., A = URVH , where U and V are unitary matrices and µ ¶ R 0 R = r×r r×(N−r) (1.2) 0(M−r)×r 0(M−r)×(N−r) with Rr×r being a r × r upper triangular matrix. Recently, we have developed the equal diagonal QRS decomposition of a ma- trix [5, 6, 7, 8]; i.e., any matrix A can be decomposed into AS = QR, where Q and S are unitary matrices and R has the same structure as that given in (1.2) but Qr 1/2r diagonal entries of Rr×r are all equal to ( i=1 λi) . There are two main differ- ences between the URV decomposition and the QRS decomposition. (a) Methodology is totally different. In the URV decomposition, for a given data matrix A, perform trangularlization to the left corner by orthonormal row and column transformations. Such unitary matrices U and V always exist [2, 3, 4]. However, in our equal diagonal QRS decomposition, the existence of the Q and S factors is problematic in mathe- matics literature. (b) The task of designing the S-factor in the QRS decomposition is totally different from that of designing the V-factor in the URV decomposition. The goal of designing the V-factor is rank-revealing and updating for subspace track- ing [2, 3, 4], while the principal purpose to design the S-factor is to optimize the

∗Contact Author: Department of Electrical and Computer Engineering, McMaster University, 1280 Main Street West, Hamilton, Ontario, L8S 4K1, Canada. Phone: +1 905 525 9140, Ext. 27599. Fax: +1 905 521 2922. ([email protected]). 1 2 J.-K. ZHANG AND K. M. WANG signal to noise (or plus interference) ratio of the worst case R-factor value subchannel or block error probability for decision feedback device in communication signal pro- cessing [5, 6, 7, 8]. Essentially, our equal diagonal QRS decomposition is based on the Schur’s decomposition [1] that the of a positive definite matrix is equal to the product of the determinant of an arbitrarily given principal submatrix and the determinant of its schur complement. Therefore, it actually describes a process for successively and evenly distributing the total information over each one-dimensional subspace [8]. Very interestingly, such decomposition simultaneously enjoys many im- portant optimality properties in communication signal processing and therefore, has played a vital role in jointly designing transceiver with decision feedback detection in many block-based communication systems [5, 6, 7, 9]. However, in some environ- ments where different R-factor value subchannels (users) require different the qualities of service (Q◦S) [10] or have different space freedom such as random MIMO chan- nels [11, 12], this would demand an unequal diagonal QRS decomposition. Therefore, in this paper we extend the equal diagonal QRS decomposition [5, 6, 7] to a general case where the diagonal entries of the R-factor are not necessarily equal but satisfy a majority relationship [13, 14, 10]. A low-complexity generalized quadratic recursive algorithm to characterize all the S-factors is developed. In particular, a pair of fast specific closed-form QRS decompositions is derived for both an and its inverse. Finally, we show how to apply this general QRS decomposition to effi- ciently solving two classes of numerical optimization problems that usually occur in the joint design of transceiver pairs with decision feedback detection in communication systems via semi-definite and geometrical programming [15]. It should be mentioned that Mirsky [13] and Guess [10] gave their constructive proofs to find a specific positive definite matrix with a pair of the prescribed eigen- values and the Cholesky values. Although their methods can be exploited to find a unitary S-factor via the eigenvalue decomposition of the obtained positive definite matrix, these are not direct and efficient, since many applications require directly computing the S-factor. In addition, their methods [13, 10] cannot characterize all the eligible positive definite matrices. Notation: Matrices are denoted by uppercase boldface characters (e.g., A), while column vectors are denoted by lowercase boldface characters (e.g., b). The (i, j)-th entry of A is denoted by Ai,j. The i-th entry of b is denoted by bi. The columns of an M × N matrix A are denoted by a1, a2, ··· , aN . Notation Ak denotes a matrix consisting of the first k columns of A, i.e., Ak = [a1, a2, ··· , ak]. By convention,

A0 = 1. The remaining matrix after deleting columns ak1 , ak2 , ··· , aki from A is denoted by Ak1,k2··· ,ki . The j-th diagonal entry of a A is denoted ⊥ by [A]j. Notation A denotes the orthonormal complement of a matrix A. The transpose of A is denoted by AT . The Hermitian transpose of A (i.e., the conjugate and transpose of A) is denoted by AH . 2. QRS decomposition of matrix. Our task in this section is to extend the equal diagonal QRS decomposition in [5, 6, 7, 8] and the quadratic recursive algorithm to a general case and then, derive a pair of a specific closed-form QRS decompositions for both an invertible matrix and its inverse. 2.1. QRS decomposition. First, we give a quadratic recursive algorithm to determine whether a positive sequence is a valid candid for the diagonal entries of the R-factor in the QR decomposition of a given matrix. For discussion convenience, we assume hereafter that the singular value decomposition of an M × N matrix A is given by (1.1). In order to judge whether a positive sequence constitutes a diagonal FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 3 entries of the R-factor of a given matrix, we recall the notion of majorization and some key results [13, 16, 10] we will require in our derivation. K Definition 2.1. For any a ∈ R , let a[1] ≥ a[2] ≥ · · · ≥ a[K] denote the components of a in decreasing order (also termed order statistics of a). Majorization makes precise the vague notion that the components of a vector a are “less spread out” or “more nearly equal” than the components of a vector b. Definition 2.2. Let a, b ∈ RK . Then, vector b majorizes a if Xk Xk a[i] ≤ b[i] i=1 i=1 for all k = 1, 2, ··· ,K, and with equality when k = K, which is denoted by a ≺ b. We also need the following definition. K K Definition 2.3. Let {ak}k=1 and {bk}k=1 be two positive real-valued sequences. K K It is said that {ak}k=1 majorities {bk}k=1 in the product sense if Yk Yk a[i] ≥ b[i] (2.1) i=1 i=1

⊗ for all k = 1, 2, ··· ,K, and with equality when k = K, which is denoted by a > b. The following lemma [17, 16, 13, 10] gives a very simple necessary and sufficient condition to check whether a pair of positive sequences constitute a pair of the valid singular values and the R-factor values of some matrix, r r Lemma 2.4. Let {λi}k=1 and {di}i=1 be the eigenvalue sequence and the Cholesky r value sequence of a positive definite matrix A, respectively. Then, {λi}i=1 majorities r r r {di}i=1 in the product sense. Conversely, if {λi}i=1 majorities {dk}i=1 in the product r sense, then, for an arbitrarily given desired permutation of {di}i=1, dj1 , dj2 , ··· , djr , r r there exists a matrix A such that {λi}i=1 and {dji }i=1 are the eigenvalues and the Cholesky values of A, respectively. In [13], Mirsky gave a constructive proof to find some positive definite matrix with the given valid eigenvalues and the Cholesky values. Recently, Guess [10] generalized this approach to cover arbitrary orderings of the desired Cholesky values and to make the construction more explicit. However, these approaches and Lemma 2.4 do not provide us a systematic technique to characterize all such positive definite matrices with the prescribed eigenvalues and the Cholesky values. In order to resolves this issues, we introduce the following√ definition. √ r r Definition√ 2.5. Let { λk }k=1√and { dk }k=1 be two positive real-valued se- r r quences and { λk }k=1 majorities { dk }k=1 in the product sense. Then, the follow- ing equations are said to be Canonical Information Distribution Equations (CIDE) generated by Λ and Dp for the QR successive cancelation detector, 1. Initialization. The first column vector w1 of W satisfies H w1 Λw1 = dj1 (2.2a) H w1 w1 = 1. (2.2b) ⊥ 2. Recursion. There exists zi such that wi+1 = Wi zi and H (i) zi C zi = dji+1 (2.2c) H zi zi = 1, (2.2d) ¡ ¢ (i) ⊥ H −1 ⊥ −1 where C = (Wi ) Λ Wi . 4 J.-K. ZHANG AND K. M. WANG

For discussion simplicity, we use notation CIDE(Λ, Dp) to shortly write the above recursion equations with Dp = diag(dj1 , dj2 , ··· , djr ) and p = [j1, j2, ··· , jr] being an arbitrarily given desired permutation of 1, 2, ··· , r. The canonical information distribution equations (2.2) essentially describes a pro- cess for successively distributing the total information quantity D = det(Λ) in the whole r-dimensional space over each one dimensional subspace (or subchannel), which holds a key to characterize all matrices with the prescribed singular values and the R-factor values. In order to extend the equal diagonal QRS decomposition in [5, 6, 7] to a general case, we establish the following lemma. Lemma 2.6. Let a1 ≥ a2 ≥ · · · ≥ aK and

1 Xi b = a . (2.3) i i j j=1

Then, b1 ≥ b2 ≥ · · · ≥ bK . Proof : First, we notice that

Pi Pi+1 Pi (i + 1) aj − i aj aj − iai+1 b − b = j=1 j=1 = j=1 (2.4) i i+1 i(i + 1) i(i + 1)

Pi Since a1 ≥ a2 · · · ≥ aK , we get aj ≥ ai+1 for j = 1, 2, ··· , i. As a result, j=1 aj ≥ iai+1. Combining this with (2.4) yields bi ≥ bi+1. ¤ Now, we are in a position to formally state our generalized QRS decomposition. Theorem 2.7. (QRS decomposition) Let the singular value decomposition of matrix A be given by (1.1). Let di for i = 1, 2, ··· , r be an arbitrarily given positive numbers. Then, the following three statements are equivalent. r r 1. { λi }i=1 majorities { di }i=1 in the product sense. r 2. For an arbitrarily given desired permutation of {di}i=1, dj1 , dj2 , ··· , djr , there exists a unitary matrix S such that AS = QR, where Q is an M × r column- wise orthonormal matrix and R = [ R 0 ] with R being a r×r pr×(N−r) r×r r × r upper triangular matrix and [Rr×r]i = dji for i = 1, ··· , r 3. There exist a unitary matrix W and a vector sequence zi for i = 1, 2, ··· , r−1 such that the canonical information distribution equation (2.2) holds. In particular, the equal diagonal QRS decomposition always exists; i.e., there exists a unitary matrix S such that AS = QR, where Q is an M ×r column-wise orthonormal matrix and R = [ Rr×r 0r×(N−r) ] with Rr×r being a r ×r upper triangular matrix 1/2r and [Rr×r]i = (λ1λ2 ··· λr) for i = 1, ··· , r. Proof : The proof map is as follows: Statement 1⇔Statement 2⇔Statement 3. a) Statement 1⇔Statement 2. Let Ae = AVr, i.e., µ ¶ Λ1/2 Ae = U . (2.5) 0(M−r)×r

r r By Lemma 2.8, that sequence {λi}i=1 majorities sequence {di}i=1 is equivalent to a r fact that for an arbitrarily given desired permutation of {di}i=1, dj1 , dj2 , ··· , djr , there exists a positive definite matrix P with eigenvalue sequence {λ1, λ2, ··· , λr} such that H P = E DpE, where E is an upper triangular matrix with unit diagonal entries and e eH e Dp = diag(dj1 , dj2 , ··· , djr ). Then, P can be decomposed into P = SΛS with S being a r × r unitary matrix. As a result, SeH PSe = Λ = Ae H Ae ; i.e., P = SeH Ae H Ae Se. FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 5

H If we let the QR decomposition of Ae Se = QRr×r, then, we have that P = E DpE = RH R . Using the uniqueness of the , we can get R = r×r r×r p r×r 1/2 e Dp E. Therefore, [Rr×r]ji = dji . Let S = [VrS, V1,2,··· ,r]. Then, S is an N × N e e unitary matrix satisfying AS = [AS, AV1,2,··· ,r] = Q[Rr×r, 0r×(N−r)] = QR. Thus far, we have proved that there exists a positive definite matrix with the prescribed singular values λi for i = 1, 2, ··· , r and the Cholesky valuesp if and only if A has a QRS decomposition with the prescribed R-factor values dij for j = 1, 2, ··· , r; i.e., Statement 1 is equivalent to Statement 2. b) Statement 2⇔Statement 3. First, we note thatp the fact that there exists a unitary matrix S such that AS = QR with [R]i = dji for i = 1, 2, ··· , r is e e e equivalentp to the fact that there exists a unitary matrix S such that AS = QRr×r with [R ] = d , which can be seen in the previous proof of Statement 1⇔Statement r×r i ji ¡ ¢ e e H 2. On³ the other´ hand, we notice that if let B = AS, then we have det Bi Bi = eH e det Si ΛSi . Now applying Schur’s decomposition ([1], pp. 21-22) to

h iH h i eH e e e Si+1ΛSi+1 = Si,esi+1 Λ Si,esi+1 Ã ! SeH ΛSe SeH Λes = i i i i+1 , H e H esi+1ΛSi esi+1Λesi+1 we get ¡ ¢ ³ ´ H eH e det Bi+1Bi+1 = det Si+1ΛSi+1 µ ¶ ³ ´−1 eH e H 1/2 1/2 e eH e eH 1/2 1/2 = det(Si ΛSi)esi+1Λ I − Λ Si Si ΛSi Si Λ Λ esi+1

¡ H ¢ H 1/2 (i) 1/2 = det Bi Bi esi+1Λ P Λ esi+1 (2.6) where

(i) 1/2 e⊥ e⊥ H e⊥ −1 1/2 e⊥ H P = I − Λ Si ((Si ) ΛSi ) (Λ Si ) .

H e⊥ H Since esi+1Si = 0, there exists a vector zi+1 with zi+1zi+1 = 1 such that esi+1 = e⊥ Si zi+1. Substituting this into (2.6) yields ¡ ¢ det BH B H (i) ¡i+1 i+1¢ zi+1C zi+1 = H , (2.7) det Bi Bi where C(i) = Λ1/2P(i)Λ1/2. On the other hand, we notice that P(i) can be rewritten as

(i) 1/2 e⊥ ⊥ 1/2 e⊥ ⊥ H 1/2 e⊥ ⊥ −1 1/2 e⊥ ⊥ H P = (Λ Si ) (((Λ Si ) ) (Λ Si ) ) ((Λ Si ) ) . (2.8)

1/2 e⊥ ⊥ Now we choose (Λ Si ) to be the Q-factor of the QR decomposition of matrix −1/2 e⊥ −1/2 e⊥ 1/2 e⊥ ⊥ −1/2 e⊥ −1 Λ Si ; i.e., if we let Λ Si = QR, then, set (Λ Si ) = Q = Λ Si R . Therefore, matrix P(i) can be further simplified as

(i) H −1/2 e⊥ −1 −H e⊥ H −1/2 P = QQ = Λ Si R R (Si ) Λ . (2.9) 6 J.-K. ZHANG AND K. M. WANG

As a result, we have

³ ´−1 (i) −1 −H e⊥ H −1 e⊥ C = R R = (Si ) Λ Si (2.10)

In addition, using our notation, we notice that there is the following relationship between the determinant¡ ¢ of the¡ submatrix¢ of B and that of the submatrix of its H H Qi 2 R-factor R, det Bi Bi = det Ri Ri = j=1[R]j . Therefore, Eq. (2.7) can be H (i) 2 rewritten as zi+1C zi+1 = [R]i . We therefore conclude that the matrix A has the given R-factor values if and only if there exist a unitary matrix W = Se and a vector zi+1 that satisfies the canonical information distribution equations (2.2); i.e., Statement 2 is equivalent to Statement 3. Finally, we examine whether Statement 2 includes the equal diagonal QRS de- composition as a special case. All we need to do is to check whether the eigenvalue r r Qr 1/r sequence {λi}i=1 majorities the constant sequence {D}i=1 with D = ( i=1 λi) . This can be verified by using Lemma 2.6 with ai = log λi. ¤ Theorem 2.7 not only provides a criterion to judge if a positive sequence is the diagonal elements of the R-factor in the QR decomposition of a given matrix, but also provides a quadratic recursive algorithm to systematically characterize and construct all unitary matrices with a given R-factor values. This recursive algorithm is obtained by successively constructing the vectors w1, w2, ··· , wr to satisfy equations (2.2a) - (2.2d). 2.2. Construction of the S-factor. The key to obtaining the QRS decompo- sition is to find the unitary matrix S. Once we have had S, we can apply the QR decomposition [1] to matrix AS so as to obtain the generalized QRS decomposition. The following recursive algorithm is to find the S-factor of the QRS decomposition AS = QR. Algorithm 1 (Construction of the S-factor): 1. SVD. Perform the SVD (1.1) of A. 2. Initialization. Determine the first column of Se such that constraints

H es1 Λes1 = dj1 (2.11) H es1 es1 = 1 (2.12) are satisfied. 3. Recursion (reduce the dimension and decouple constraints). Set esi+1 = e⊥ Si zi+1, where zi+1 is any vector that satisfies

H (i) zi+1C zi+1 = dji (2.13) H zi+1zi+1 = 1, (2.14)

³ ´−1 (i) e⊥ H −1 e⊥ where C = (Si ) Λ Si

4. Complete the S-factor. S = [VrSe, V1,··· ,r]. We would like to make the following comments. • Algorithm 3 tells us that a problem of finding the S-factor for a matrix A is essentially reduced to a problem of finding the S-factor for its singular value diagonal matrix Λ1/2. • Information decomposition. Actually, the quadratic recursive algorithm is the Schur’s decomposition that the determinant of a positive definite matrix FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 7

is equal to the product of the determinant of an arbitrarily given principal submatrix and the determinant of its Schur complement [1]. Therefore, the algorithm essentially describes a process for successively distributing the total information quantity D = det(Λ) in the whole r-dimensional space over each one dimensional subspace (or R-factor values subchannel). • Characterization of the S-factor. The quadratic recursive algorithm (2.11)– (2.14) characterizes all S-factors such that the resulting matrices AS possess the prescribed diagonal R-factors. By properly choosing a particular solutions for each recursion one can significantly simplify the complexity of computing a specific S-factor, which we show in the ensuring subsection. • Characterization of the Q-factor. Notice that if Q, R and S are the Q-R-S factors of an invertible matrix A, then, S, R−1 and Q are the Q-R-S factors of its inverse A−1. This observation implies that there is a companion algorithm for A−1 which has the same structure as Algorithm 1 This inverse algorithm actually characterizes the Q-factor in the QRS decomposition of matrix A, which leads to an efficient algorithm for finding the S and Q factors.

2.3. Specific closed-form QRS decompositions. In this subsection, we will give a pair of closed-form QRS decompositions for both a matrix and its inverse by carefully choosing such specific solutions that all the matrices C(i) are diagonalized simultaneously. To this end, we first define canonical eigen-diagonal matrix sequences from the original singular value matrix. To do that, we first establish the following lemma. r r Lemma 2.8. Let sequence {di}i=1 be majorized by sequence {λi}i=1. For an r−1 arbitrarily given 1 ≤ j ≤ r, define a new sequence {µi}i=1 as follows: µ1 = λ1, µ2 = λ`λ`+1 λ2, ··· , µ`−1 = λ`−1, µ` = , µ`+1 = λ`+2, ··· , µr−1 = λr, where the ` is a dj1 maximum positive integer such that λ` ≥ dj1 ≥ λ`+1. Then, µ1 ≥ µ2 ≥ · · · ≥ µr−1 and µ majorities dj1 . λ`λ`+1 Proof : First, we note that λ` ≥ ≥ λ`+1, since λ` ≥ dj1 ≥ λ`+1 by the dj1 definition of sequence `. As a result, we have that µ1 = λ1 ≥ λ2 = µ2 ≥ · · · ≥ λ`−1 = λ`λ`+1 µ`−1 ≥ λ` ≥ = µ` ≥ λ`+1 ≥ λ`+2 = µ`+1 ≥ · · · ≥ λr = µr−1. Therefore, dj1 r−1 sequence {µi}i=1 is decreasing.

In the following, we will prove that sequence µ majorities dj1 . For notation simplicity, let dj1 = d[q] for some q, 1 ≤ q ≤ r. We consider the following two cases. Qm Qm Qm (a) ` < q. In this case, we have i=1 µi = i=1 λi ≥ i=1 d[i] for 1 ≤ m < ` and Qm −1 Qm+1 −1 Qm+1 Qm i=1 µi = d[q] i=1 λi ≥ d[q] i=1 d[i] ≥ i=1 d[i] for ` ≤ m < q, since λ majorities Qm −1 Qm+1 d and d[m+1] ≥ d[q]. In addition, for q ≤ m ≤ r we have i=1 µi = d[q] i=1 λi ≥ −1 Qm+1 Qr−1 −1 Qr −1 Qr d[q] i=1 d[i]. It is clear that i=1 µi = d[q] i=1 λi = d[q] i=1 d[i]. Therefore, in (2) this case, µ majorities dj1 in the product sense. Qm Qm Qm (b) ` ≥ q. In this case, we have i=1 µi = i=1 λj ≥ i=1 d[i] for 1 ≤ m < q, and Qm Qm Qm i=1 µi = i=1 λi ≥ i=1 d[i] ≥ d[1]d[2] ··· d[q−1]d[q+1] ··· d[m+1] for q ≤ m ≤ ` − 1 Qm −1 Qm+1 −1 Qm+1 and i=1 µi = d[q] i=1 λi ≥ d[q] i=1 d[i] for ` ≤ m ≤ r − 1, since λ majorities d.

Therefore, in this case, µ also majorities dj1 in the product sense. ¤ Lemma 2.8 tells us that we can derive a new pair of sequences with the majority relationship in a small dimension of space from a given pair of sequences with the majority relationship in a large dimension of space. Using Lemma 2.8, we can induc- tively define the following canonical eigen-diagonal sequence, which will be the central 8 J.-K. ZHANG AND K. M. WANG positive definite matrix C(i) in the canonical information distribution equation; i.e., C(i) = Λ(i), and thus, define a pair of rotation sequences. Definition 2.9. Let the singular value decomposition of matrix A be given by (1.1). Let dj for k = 1, 2, ··· , r be an arbitrarily positive numbers such that d < λ. r For an arbitrarily given desired permutation of {di}i=1, dj1 , dj2 , ··· , djr , a canonical (i) (i) (i) (i) r−1 eigen-diagonal matrix sequence {`i, Λ = diag(λ1 , λ2 , ··· , λr−i+1)}i=1 is defined as follows: 1. Λ(1) = Λ. 2. Let ` be a maximum positive integer such that λ(i) ≥ d ≥ λ(i) . i `i ji `i+1 Then, we define λ(i+1) = λ(i), λ(i+1) = λ(i), ··· , λ(i+1) = λ(i) , λ(i+1) = 1 1 2 2 `i−1 `i−1 `i λ(i)λ(i) `i `i+1 (i+1) (i) (i+1) (i) , λ` +1 = λ` +2, ··· , λr−i = λr−i+1. dji i i (i) −1 r−1 The sequence {`i, (Λ ) }i=1 is called its canonical inverse eigen-diagonal matrix r−1 sequence. Correspondingly, a basic rotation sequence {αi, βi}i=1 and its inverse basic e r rotation sequence {αek, βk}k=1 are defined, respectively, by v v u (i) u (i) u dj − λ u λ − dj t i `i+1 t `i i αi = , βi = , (2.15) λ(i) − λ(i) λ(i) − λ(i) `i `i+1 `i `i+1 and v v u e(i) e u e e(i) u λ − dj u dj − λ t `i+1 i e t i `i αei = , βi = , (2.16) λe(i) − λe(i) λe(i) − λe(i) `i+1 `i `i+1 `i

e(i) (i) −1 where, for notation simplicity and symmetrical structure, we denote λj = (λj ) e −1 and dji = (dji ) . The canonical eigen-diagonal matrix sequence and rotation sequences have the following property, which will paly key role in deriving our fast closed-form QRS decomposition. (i) r−1 Lemma 2.10. The eigen-diagonal matrix sequence {`i, Λ }i=1 and rotation sequences are defined in Definition 2.9. Then, we have (i) (i) (i) 1. Decreasing property: λ1 ≥ λ2 ≥ · · · ≥ λr−i+1 (i) ⊗ 2. Majority relationship: λ > dj1,j2,··· ,ji−1 . r e r 3. Sequences {αi, βi}i=1 and {αei, βi}i=1 satisfy conditions:

2 2 αi + βi = 1 (2.17a) λ(i)α2 + λ(i) β2 = d (2.17b) `i i `i+1 i ji 2 e2 αei + βi = 1 (2.17c) e(i) 2 e(i) e2 e λ αei + λ βi = dji (2.17d) s `i `is+1 λ(i) λ(i) `i e `i+1 αei = αi, βi = βi. (2.17e) dji dji Proof : Statement 1 and Statement 2 can be examined directly by the definition of the eigen-diagonal matrix sequences and the rotation sequences. Inductively using Lemma 2.8, we can prove Statement 2. ¤ FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 9

Now we are in a position to formally state a family of fast closed-form QRS decompositions for both a invertible matrix and its inverse. 1/2 Theorem√ √ 2.11.√ (Closed-form ideal QRS decomposition) Let Λ = diag( λ1, λ2, ··· , λr) with λ1 ≥ λ2 ≥ · · · λr > 0 of matrix A be given by (1.1). Also,√ let di for i = 1, 2√, ··· , r be an arbitrarily given positive numbers such that r r { λi }i=1 majorities { di }i=1 in the product sense and the rotation sequences r e r {αk, βk}k=1 and {αek, βk}k=1 be defined by (2.15) and (2.16), respectively. Then, r for an arbitrarily given desired permutation of {di}i=1, dj1 , dj2 , ··· , djr , we have the following closed-form QRS decompositions,

1/2 Λ Sα, β = Sαe, βe Rα, Λ, Dp, β (2.18) −1/2 Λ S e = Sα, β R −1 −1 e , (2.19) αe, β αe, Λ , Dp , β

where Rα, Λ, D , β and R −1 −1 e are upper triangular matrices with p p αe, Λ , Dp , β 1 [Rα, Λ, D , β]i = dj and [R −1 −1 e ]i = √ for i = 1, 2, ··· , r and Sα, β = p i αe, Λ , Dp , β dji Qr−1 i=1 Sαi, `i, βi with each Sαi, `i, βi be determined by

  Ii−1 0(i−1)×1 0(i−1)×(`i−1) 0(i−1)×1 0(i−1)×(r−`i−i)  0 0 I 0 0   (`−1)×(i−1) (`i−1)×1 `i−1 (`i−1)×1 (`i−1)×(r−`i−i)  0 α 0 −β 0  Sαi, `i, βi= 1×(i−1) i 1×(`i−1) i 1×(r−`i−i)    01×(i−1) βi 01×(`i−1) αi 01×(r−`i−i)

0(r−`i−i)×(i−1) 0(r−`i−i)×1 0(r−`i−i)×(`i−1) 0(r−`i−i)×1 Ir−`i−i

r−1 The same results hold for Sαe, βe by replacing the basic rotation sequences {αk, βk}k=1 e r−1 in Sα, β by its inverse basic rotation sequences {αek, βk}k=1.

Proof : Since each Sαi, `i, βi is unitary, so does Sα, β. Therefore, in the following we only need to prove that SH Λ1/2S is an upper triangular matrix with a given αe, βe α, β diagonal entries. To this end, we first prove the following statement using induction on J : 1 ≤ J ≤ r − 1.

à !H YJ YJ µ ¶ 1/2 RJ×J RJ×(r−J) S e Λ Sα , ` , β = √ . (2.20) αei, `i, βi i i i 0 Λ(J+1) i=1 i=1 (r−J)×J

When J = 1, it can be verified by computation that

µ p ¶ 1/2 dj1 R1×(r−1) S Λ Sα , ` , β = √ , (2.21) αe1, `1, βe1 1 1 1 (2) 0(r−1)×1 Λ where we have used the definition of the eigen-diagonal matrices and the rotation sequences and Lemma 2.10. Now we assume that (2.20) is true for J = L. When 10 J.-K. ZHANG AND K. M. WANG

J = L + 1, exploiting the induction assumption, we have

à !H LY+1 LY+1 1/2 S Λ Sα , ` , β αei, `i, βei i i i i=1 i=1 µ ¶ RL×L R√L×(r−L) = Sαe , ` , βe (L+1) Sαe , ` , βe L+1 L+1 L+1 0(r−L)×L Λ L+1 L+1 L+1   RL×L pRL×1 RL×(r−L−1)  0 d R  = 1×L jL+1 √1×(r−L−1) (L+2) 0(r−L−1)×L 0(r−L−1)×1 Λ µ ¶ R(L+1)×(L+1) R(L√+1)×(r−L−1) = (L+2) 0(r−L−1)×(L+1) Λ This shows that (2.20) is also true for J = L + 1. ¤ We would like to make the following comments. 1. Different eigen-diagonal matrix sequences result in different rotation se- quences and thus, different specific Q-R-S factors. 2. Combining Theorem 2.7, Algorithm 3 with Theorem 2.11, we have a specific Q-R-S factors in the QRS decomposition of a general matrix A; i.e., Q =

UrSα,e βe, Rr×r = Rα,Λ,Dp,β and S = [VrSα,β, V1,2,··· ,r]. 3. Applications to numerical optimization. In this section, we apply our QRS decomposition to solving two families of optimization problems, which are often encountered in designing optimal transceiver pairs in communication signal process- ing [5, 6, 7, 9, 8]. (k) Problem 1. Find a sequence of matrices A of size Mk × Nk such that

(k) K n=Nk,k=K {Aopt}k=1 = arg min F({[RB(k)A(k) ]n}n=1,k=1 ). subject to the following constraints

(k) K F`({P }k=1) ≤ 0 for ` = 1, 2, ··· , L, (k) (k) (k) H (k) (i) K where P = A (A ) , B is some matrix function with respect to {P }i=1, (k) (k) and RB(k)A(k) denotes the R-factor of B A . In a general case, Problem 1 is not easy to be solved. However, in many appli- cations, we find that Problem 1 can be formulated such that the following conditions are satisfied. K e (k) K 1. F({RB(k)A(k) }k=1) ≥ F({P }k=1) with equality when

[RB(k)A(k) ]1 = [RB(k)A(k) ]2 = ··· = [RB(k)A(k) ]Nk (3.1) for k = 1, 2, ··· ,K. e (k) K K 2. F({P }k=1) and F`({P}k=1) are convex over all semidefinite positive matrix domain. In this case, Problem 1 can be efficiently solved using the following two stages. Stage 1. Find a sequence of matrices P(k) such that

(k) K e (k) K {Popt}k=1 = arg min F({P }k=1). subject to the following constraints

(k) K F`({P }k=1) ≤ 0 for ` = 1, 2, ··· , L, FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 11

This optimization problem can be efficiently solved by using the semidefinite pro- gramming (SDP) [15]. Stage 2. Find a sequence of unitary matrices in the remaining free variables to (k) meet Condition (3.1). Note that some Popt would be zero matrices. Therefore, let (ki) Popt 6= 0 for i = 1, 2, ··· , K be the optimal solutions from Stage 1. Perform the (ki) eigenvalue decomposition to Popt ; i.e., Ã ! (ki) (ki) (k ) ∆r ×r 0ki×(M−rk ) (k ) H P = V i ki ki i (V i ) . (3.2) opt 0 0 (M−rki )×rki (M−rki )×(M−rki )

Then, the optimal solution is q (ki) (ki) (ki) T (ki) A = V [ ∆ , 0r ×(M−r )] S , opt rki ×rki ki ki

(k ) where S i qis the S-factor of the equal diagonal QRS decomposition of matrices (ki) (ki) (ki) T B V [ ∆ , 0r ×(M−r )] , which can be efficiently obtained by using Al- opt rki ×rki ki ki gorithm 1 and Theorem 2.11 with the eigen diagonal matrix Λ = ∆(ki) and rki ×rki (ki) 1/2rki dj = det(∆ ) for j = 1, 2, ··· , rki . rki ×rki Problem 2. Find a positive diagonal matrix Ξ = diag(µ1, µ2, ··· , µr) and a r × r unitary matrix U such that

Xr {Ξopt, Uopt} = arg min ckG([RΞ1/2U]k) k=1 subject to the following constraints

r r G`({[RΞ1/2U]i}i=1, {µi}i=1) ≤ 0 for ` = 1, 2, ··· , L, where constant sequence ck and function G(x) are both decreasing xi r yi r and each G`({2 }i=1, {2 }i=1) is convex with respect to variables x1, x2, ··· , xr, y1, y2, ··· , yr. In order to efficiently solve this problem, we establish the following lemma. Lemma 3.1. Let a1 ≤ a2 ≤ · · · ≤ ar and function g(t) be decreasing. For an arbitrarily given r real numbers, let

Xr f(t1, t2, ··· , tr) = aig(ti). (3.3) i=1

Then, for any pair of positive integers i < j and ti ≤ tj, we have

f(t1, t2, ··· , ti, ··· , tj, ··· , tK ) ≤ f(t1, t2, ··· , tj, ··· , ti, ··· , tr) (3.4)

Proof : Since for any pair of positive integers i < j and ti ≤ tj,

f(t1, t2, ··· , ti, ··· , tj, ··· , tr) − f(t1, t2, ··· , tj, ··· , ti, ··· , tr)

= (ai − aj)(g(ti) − g(tj)) ≤ 0. (3.5) 12 J.-K. ZHANG AND K. M. WANG due to assumptions that ai ≤ aj and g(ti) ≥ g(tj), we have

f(t1, t2, ··· , ti, ··· , tj, ··· , tr) ≤ f(t1, t2, ··· , tj, ··· , ti, ··· , tr).

¤ 2 e Now, by introducing new variables bi = log([RΞ1/2U]N−r+i−1) and bi = log µi for i = 1, 2, ··· , r and considering a majority relationship between the singular values and the R-factor values of matrix Ξ1/2U with Lemma 3.1, Problem 2 can be equivalently transformed into the following convex geometrical programming [15]. T e e e e T Formulation 1. Find a vector b = [b1, b2, ··· , br] and b = [b1, b2, ··· , br] such that Xr e {bopt, bopt} = arg min ckG([RΞ1/2U]k) k=1 subject to the following constraints,

Xi Xi e bj ≤ bj for i = 1, 2, ··· , r − 1 j=1 j=1 Xr Xr e bj = bj j=1 j=1

bi r ebi r G`({2 }i=1, {2 }i=1) ≤ 0 for ` = 1, 2, ··· , L.

Hence, the optimization problem in Formulation 1 can be efficiently solved by using the interior point method [15]. Once we have had bopt, we substitute it back to get bi/2 [RΞ1/2U]N−r+i−1 = 2 and furthermore, all the corresponding optimal unitary ma- trices can be characterized by the canonical information distribution equations (2.2)

eb1 eb2 ebr br b2 b1 with Λ = diag(2 , 2 , ··· , 2 ) and Dp = diag(2 , 2 , ··· , 2 ). A specific S-factor can be efficiently found by using our Theorem 2.11.

REFERENCES

[1] R. Horn and C. Johnson, Matrix Analysis. Cambridge, MA: Cambridge University Press, 1985. [2] G. H. Golub and C. F. Van Loan, Matrix Computations. Baltimore: The Johns Hopkins University Press, 1983. [3] G. W. Steward, “An updating algorithm for subspace tracking,” IEEE Trans. Signal Processing, vol. 40, pp. 1535–1541, June 1992. [4] M. Steward and P. V. Dooren, “Updating a generalized URV decomposition,” SIAM J. Matrix Anal. Appl., vol. 22, pp. 479–500, 2000. [5] J.-K. Zhang, A. Kacvic, X. Ma, and K. M. Wong, “Unitary precoder design for ISI channels,” in Int. Conf. Acoust., Speech, Signal Process., (Orlando USA), pp. 2265–2268, May 2002. [6] J.-K. Zhang and K. M. Wong, “An optimal QR decomposition,” in International Symposium on Inform. Theory and Its Applications, (Xi’an PRC), Oct. 2002. [7] J.-K. Zhang, A. Kav˘cic, and K. M. Wong, “Equal-daigonal QR decomposition and its applica- tion to precoder design for successive cancellation detection,” IEEE Trans. Inform. Theory, pp. 154–171, Jan. 2005. [8] J.-K. Zhang, T. N. Davidson, and K. M. Wong, “Uniform decomposition of mutul information with decision feedback detection,” in the Proceedings of the 2005 International Symposium on Information Theory, (Adelaide, Australia), Sept. 2005 (Submitted). [9] F. Xu, T. N. Davidson, J.-K. Zhang, and K. M. Wong, “Joint design of tranceiver for block transmission with decision feedback detection,” to appear in IEEE Trans. Signal Process- ing. FAST QRS DECOMPOSITION OF MATRIX AND ITS APPLICATIONS 13

[10] T. Guess, “Optimal sequences for CDMA with decision-feedback receivers,” IEEE Trans. In- form. Theory, vol. 49, pp. 886–900, Apr. 2003. [11] R. J. Muirhead, Aspects of multivariate statistical theory. New York: John Wiley & Sons, INC, 1982. [12] N. Prasad and M. K. Varanasi, “Analysis of decision feedback detection for MIMO Rayleigh- fading channels and the optimization of power and rate allocations,” IEEE Trans. Informat. Theory, vol. 50, pp. 1009–1025, June 2004. [13] A. W. Marshall and I. Olkin, Inequality: theory of majority and its applications. San Diego CA: Academic Press, 1979. [14] P. Viswanath and V. Anantharam, “Optimal sequences and sum capacity of synchronous CDMA systems,” IEEE Trans. Inform. Theory, vol. 45, pp. 1984–1991, Sept. 1999. [15] S. Boyd and L. Vandenberghe, Convex Optimization. The Edinburgh Building, Cambridge: Cambridge University Press, 2004. [16] A. Horn, “On the eigenvalues of a matrix with prescribed singular values,” Proc. Amer. Math. Soc., vol. 5, pp. 4–7, 1954. [17] H. Weyl, “Inequalities between the two kinds of eigenvalues of a linear transformation,” Proc. Nat. Acad. Sci., vol. 35, pp. 408–411, July 1949.