Computing the Polar Decomposition---With Applications
Total Page:16
File Type:pdf, Size:1020Kb
This 1984 technical report was published as N. J. Higham. Computing the polar decomposition---with applications. SIAM J. Sci. Stat. Comput., 7(4):1160-1174, 1986. The appendix contains an alternative proof omitted in the published paper. Pages 19-20 are missing from the copy that was scanned. The report is mainly of interest for how it was produced. - The cover was prepared in Vizawrite 64 on a Commodore 64 and was printed on an Epson FX-80 dot matrix printer. - The main text was prepared in Vuwriter on an Apricot PC and printed on an unknown dot matrix printer. - The bibliography was prepared in Superbase on a Commodore 64 and printed on an Epson FX-80 dot matrix printer. COMPUTING THE POLAR DECOMPOSITION - WITH APPLICATIONS N.J. Higham • Numerical Analysis Report No. 94 November 1984 This work was carried out with the support of a SERC Research Studentship • Department of Mathematics University of Manchester Manchester M13 9PL ENGLAND Univeriity of Manchester/UMIST Joint Nu•erical Analysis Reports D•p•rt••nt of Math••atic1 Dep•rt•ent of Mathe~atic1 The Victoria Univ•rsity University of Manchester Institute of of Manchester Sci•nce •nd Technology Reque1t1 for individu•l t•chnical reports •ay be addressed to Dr C.T.H. Baker, Depart•ent of Mathe••tics, The University, Mancheiter M13 9PL ABSTRACT .. A quadratically convergent Newton method for computing the polar decomposition of a full-rank matrix is presented and analysed. Acceleration parameters are introduced so as to enhance the initial rate of convergence and it is shown how reliable estimates of the optimal parameters may be computed in practice. To add to the known best approximation property of the unitary polar factor, the Hermitian polar factor of a Hermitian matrix A is shown to be a good positive <semi-> definite approximation to A. Perturbation bounds for the polar factors are derived. Applications of the polar decomposition to factor analysis, aerospace computations and optimisation are outlined; and a new method is derived for computing the square root of a symmetric positive definite matrix. Keywords: polar decomposition, singular value decomposition, Newton·s method, matrix square root. CONTENTS 1. Introduction 1 2. Properties of the Polar Decomposition 3 2.1 Elementary Properties 3 2.2 The Unitary Polar Factor 4 2.3 The Hermitian Polar Factor 6 2.4 Perturbation Bounds for the Polar Factors 9 3. Computing the Polar Decomposition 12 3.1 Using the Singular Value Decomposition 12 3.2 A Newton Method 12 3.3 Accelerating Convergence 15 3.4 The Practical Algorithm 17 4. Backward Error Analysis 21 5. Relation to Matrix Sign and Square Root Iterations 23 6. Applications 25 6.1 Factor Analysis 25 6.2 Aerospace Computations 25 6.3 Optimisation 27 6.4 Matrix Square Root 28 • 7. Numerical Examples 30 8. Conclusions 32 Appendix 33 .. ,. 1 • Int;roduct.ion The polar decomposition is a generalisation to matrices of the familiar complex nwnber representation z = reie, r > o. The decomposition is well- known and can be found in many textbooks, for example, (10), (11), (14), (23], (25], (27]. An early reference is (l]. The polar decomposition is readily derived from the singular value decOllll(lOsition. Let A E cmxn, m > n, have the singular value decomposition (14, p.16] • ( 1.1) where P ~ cm><111 and Q ~ cnxn are unitary and Partitioning we have which yields the polar decanposition A= UH, ( 1. 2) where u ( 1. 3) has orthono.nnal columns (U*U = In) and • ( 1.4) is Hermitian positive semi-definite. From (1.2), A*A = H2; since A*A is Hermitian positive semi- de£inite it follows that H ( 1. 5) - 2- 1 where c tt denotes the unique Hermitian positive semi-definite square root of the He:rmitian positive semi-definite matrix c (18), (25). Rank (A) = rank (H), so if rank (A) = n then B is positive dP.finite and u = Afrl is uniquely determined. Sunnarising, Theorem 1.1. Polar Decomposition. Let A E cm><n, m > n . Then there exists a matrix u E cmxn and a unique He.nnitian positive semi-definite matrix H E cnxn such that A = UH, u*u = In· If rank (A) = n then H is positive definite and u is uniquely deter.ined. 0 It is well- known that the polar factor u possesses a .best approximation property (see section 2). Less attention has .been paid in the literature to the Hermitian polar factor H. We derive some interesting properties of H which show that it is a good Hermitian positive (semi-) definite aproxiJnation to A when A is Hermitian. In view of the properties possessed by the polar factors of a matrix, techniques for computing the polar decomposition are of interest. Wh ile U and H can .be obtained via the singular value decomposition (as shown above), this approach is not always the most efficient (if A~ U, as explained in j6.2) or the most convenient (a library routine for computi ng the singular value decomposition might not .be available, on a microcomputer for example ) . In section 3 we present and analyse a Newton method for computing the polar decomposition which involves only matrix additions and matri x inversions. The taethod is shown to .be quadratically convergent . Acceleration parameters are introduced so as to enhance the initi al rate of convergence and it is shown how reliable estimates of the optimal parameters may be computed in practice. The stability of the method i s considered in section 4. In section 5 the relationship of the method -3- to two well-known iterations is described. In section 6 we describe applications of the polar decomposition to factor analysis, aerospace computations and optimisation. We show how our algorithlll 1nay be ell!ployed in these applications and compare it with other methods in use currently. A new method for computing the square root of a synnetric positive definite matrix is derived. " 2. 2 .1 ltllment:ary P.r<prt::ies • • We begin by sunwnarising some elementar:y properties of the polar deco.position. our notation is as follows. For A ~ cnxn A(A) and a(A) denote, respectively, the set of eigenvalues and the set of singular condition nWllber. A is normal if A*A =AA* [l4, p.193). 1.-a 2.1. Let. A t= cnxn have the polar decomposition A = UH. Then (i) If H has the spectral decomposition H then A = PJ:Q* is a singular value decomposition of A, where p = UQ. (ii) A( R) = a( R) = a( A). (iii) t<z(H) = t<z(A). (iv) A is nonnal if and only if UH HU. Proof. P UQ and Q are unitary, and E is diagonal with nonnegative d .iagonal ele.ents since H is positive semi-definite; so A = UH = PI.:Q* constitutes a singular value decOll'lposition of A. This gives the first part, from which the second and third parts follow. For the last part, if UH HU then HUU*H = UH HU* AA*, while A*A AA* implies a2 = UH2u*, -4- that is, e2u = CJH2, which i.Jnplies that H commutes with u, since 1 1 H = (A*Aj C? = (ff2) z (see ( -1.5)) is a polynomial in ff2 (10), (19]. our int.creel 1.n the polar decomposit. ion stems from a beat approxi- mat.ion property posssessed by the unitary factor. The following result displays this property, it ie the generalisation to complex matrices of a result in (6), (13], (14], [15], [24], (29]. The Probeniue matrix norm i• defined by •Alp= (trace (A..-A))1z. 'Jbeonm 2.2. Let A, 8 E cmxn and let B*A E cnxn have the polar decomposition Then for any unitary z E cnxn, llA- BUllp ( llA-BZNp ( llA+BUllp. (2.1) 2 n NA t BUNp E (cri(A)2 t 2cri(B*A) + cri(B)2 ). (2.2) i = l Pn)Qf. (Cf. [14, p.425].) P9r any unitary z E cnxn, 2 NA - BZllp trace ((A - BZ)* (A - BZ)) n n ~ E cr·(A)2 + .E cri(B)2 - trace (A*Bz + z*e*A). i=1 ]. 1=1 Since the first two terms are independent of z it suffices to consider the last tent. Prom Lenaa 2.1 (i) B*A has the singular value decomposition ( 2. 3) where p UQ. (2.4) -5- Thus, using (2 . 3) trace (A*ez + z*e*A) trace (QI:P*z + z*p~*) trace (I:P*ZQ + Q*z*Pr) trace (Oi + w*I:) n - I: ai(Wii + wii>• i-1 where W = P*ZQ ( 2. 5) is unitary. Since the columns of W have unit 2- norm, IWiil ' 1, so - 2 ( Wii + Wii ( 2 for al1 i. Hence n n - 2 I: oi ' trace (A*ez + z*e*A> ' 2 r oi, i=l i =1 with equality on the left when w =-I, that is, from (2.5) and (2.4), z = - u, and equality on the right when W = I, that is z u. 0 An i.Jlportant special case of Theorem 2.2 is obtained by taking m = n and 5 , ,,.. I: COJ"Ollary 2 • 3'. Let A E cnxn have the polar decomposition A = UH. Then for any unitary z E cnxn, n n 2 ' ( I: (a · (A) - 1) )z = IA - Ullp' IA - Zllp' AA+ UHp ( I: (ai(A) + 1)2~z . O i =l .1 i =1 Thus if distance is measured in the Frobenius norm, the nearest unitary matrix to A E cnxn is the unitary factor in the polar decomposition of A; and the furthest unitary matrix from A is minus this unitary factor.