Chapter 7

QR-Decomposition for Arbitrary Matrices

7.1 Orthogonal Reflections

Orthogonal symmetries are a very important example of isometries. First let us review the definition of a (linear) projection.

Given a vector space E,letF and G be subspaces of E that form a direct sum E = F G. Since every u E can be written uniquely as u = v + w,where2 v F and w G,wecandefinethe two projections p : E2 F and2p : E G,suchthat F ! G ! pF (u)=v and pG(u)=w.

465 466 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES

It is immediately verified that pG and pF are linear maps, and that p2 = p , p2 = p , p p = p p =0,and F F G G F G G F pF + pG =id.

Definition 7.1. Given a vector space E,foranytwo subspaces F and G that form a direct sum E = F G, the symmetry with respect to F and parallel to G, or reflection about F is the linear map s: E E,defined such that ! s(u)=2p (u) u, F for every u E. 2

Because pF + pG =id,notethatwealsohave s(u)=p (u) p (u) F G and s(u)=u 2p (u), G s2 =id,s is the identity on F ,ands = id on G. 7.1. ORTHOGONAL REFLECTIONS 467 We now assume that E is a of finite dimension.

Definition 7.2. Let E be a Euclidean space of finite dimension n.ForanytwosubspacesF and G,ifF and G form a direct sum E = F G and F and G are orthogonal, i.e. F = G?,theorthogonal symmetry with respect to F and parallel to G, or orthogonal reflection about F is the linear map s: E E,definedsuchthat ! s(u)=2p (u) u, F for every u E. 2 When F is a hyperplane, we call s an hyperplane symme- try with respect to F or reflection about F ,andwhen G is a , we call s a flip about F .

It is easy to show that s is an isometry. 468 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES Using Proposition 6.7, it is possible to find an orthonor- mal basis (e1,...,en)ofE consisting of an of F and an orthonormal basis of G.

Assume that F has dimension p,sothatG has dimension n p.

With respect to the orthonormal basis (e1,...,en), the symmetry s has a of the form

Ip 0 0 In p ✓ ◆ 7.1. ORTHOGONAL REFLECTIONS 469

n p Thus, det(s)=( 1) ,ands is a rotation i↵ n p is even.

In particular, when F is a hyperplane H,wehave p = n 1, and n p =1,sothats is an improper orthogonal transformation.

When F = 0 ,wehaves = id, which is called the symmetry with{ } respect to the origin.Thesymmetry with respect to the origin is a rotation i↵ n is even, and an improper orthogonal transformation i↵ n is odd.

When n is odd, we observe that every improper orthogo- nal transformation is the composition of a rotation with the symmetry with respect to the origin. 470 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES When G is a plane, p = n 2, and det(s)=( 1)2 =1, so that a flip about F is a rotation.

In particular, when n =3,F is a line, and a flip about the line F is indeed a rotation of measure ⇡.

When F = H is a hyperplane, we can give an explicit for- mula for s(u)intermsofanynonnullvectorw orthogonal to H.

We get (u w) s(u)=u 2 · w. w 2 k k Such reflections are represented by matrices called House- holder matrices,andtheyplayanimportantroleinnu- merical matrix analysis. Householder matrices are sym- metric and orthogonal. 7.1. ORTHOGONAL REFLECTIONS 471

Over an orthonormal basis (e1,...,en), a hyperplane re- flection about a hyperplane H orthogonal to a nonnull vector w is represented by the matrix

WW> WW> H = In 2 2 = In 2 , W W >W k k where W is the column vector of the coordinates of w.

Since (u w) pG(u)= · w, w 2 k k the matrix representing pG is

WW >, W >W and since pH + pG =id,thematrixrepresentingpH is

WW> In . W >W 472 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES The following fact is the key to the proof that every isom- etry can be decomposed as a product of reflections.

Proposition 7.1. Let E be any nontrivial Euclidean space. For any two vectors u, v E, if u = v , then there is an hyperplane H such2 thatk thek reflec-k k tion s about H maps u to v, and if u = v, then this reflection is unique. 6

We now show that Hyperplane reflections can be used to obtain another proof of the QR-decomposition. 7.2. QR-DECOMPOSITION USING HOUSEHOLDER MATRICES 473 7.2 QR-Decomposition Using Householder Matrices

First, we state the result geometrically. When translated in terms of Householder matrices, we obtain the fact ad- vertised earlier that every matrix (not necessarily invert- ible) has a QR-decomposition.

Proposition 7.2. Let E be a nontrivial Euclidean space of dimension n. Given any orthonormal basis (e1,...,en), for any n-tuple of vectors (v1,...,vn), there is a sequence of n isometries h1,...,hn, such that hi is a hyperplane reflection or the identity, and if (r1,...,rn) are the vectors given by r = h h h (v ), j n ··· 2 1 j then every rj is a linear combination of the vectors (e ,...,e ), (1 j n). Equivalently, the matrix 1 j   R whose columns are the components of the rj over the basis (e1,...,en) is an upper . Furthermore, the hi can be chosen so that the diagonal entries of R are nonnegative. 474 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES

Remarks.(1)Sinceeveryhi is a hyperplane reflection or the identity, ⇢ = h h h n ··· 2 1 is an isometry.

(2) If we allow negative diagonal entries in R,thelast isometry hn may be omitted.

(3) Instead of picking r = u00 ,whichmeansthat k,k k kk w = r e u00, k k,k k k where 1 k n,itmightbepreferabletopick   2 r = u00 if this makes w larger, in which case k,k k kk k kk wk = rk,k ek + uk00.

Indeed, since the definition of hk involves division by 2 wk ,itisdesirabletoavoiddivisionbyverysmallnum- bers.k k

Proposition 7.2 immediately yields the QR-decomposition in terms of Householder transformations. 7.2. QR-DECOMPOSITION USING HOUSEHOLDER MATRICES 475 Theorem 7.3. For every real n n-matrix A, there ⇥ is a sequence H1,...,Hn of matrices, where each Hi is either a Householder matrix or the identity, and an upper triangular matrix R, such that R = H H H A. n ··· 2 1 As a corollary, there is a pair of matrices Q, R, where Q is orthogonal and R is upper triangular, such that A = QR (a QR-decomposition of A). Furthermore, R can be chosen so that its diagonal entries are non- negative.

Remarks.(1)Letting A = H H H A, k+1 k ··· 2 1 with A1 = A,1 k n,theproofofProposition7.2 can be interpreted in terms of the computation of the sequence of matrices A1,...,An+1 = R. 476 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES

The matrix Ak+1 has the shape

k+1 u1 ⇥⇥⇥0 . . ⇥⇥⇥⇥. . . . 0 ⇥ k+1 1 00 uk ⇥ k+1 ⇥⇥⇥⇥ B 000uk+1 C Ak+1 = B ⇥⇥⇥⇥C B 000uk+1 C B k+2 ⇥⇥⇥⇥C B ...... C B C B 000uk+1 C B n 1 C B 000uk+1 ⇥⇥⇥⇥C B n C @ ⇥⇥⇥⇥A where the (k +1)thcolumnofthematrixisthevector u = h h h (v ), k+1 k ··· 2 1 k+1 and thus k+1 k+1 uk0 +1 =(u1 ,...,uk ), and k+1 k+1 k+1 uk00+1 =(uk+1,uk+2,...,un ).

If the last n k 1entriesincolumnk +1areallzero, there is nothing to do and we let Hk+1 = I. 7.2. QR-DECOMPOSITION USING HOUSEHOLDER MATRICES 477 Otherwise, we kill these n k 1entriesbymultiplying Ak+1 on the left by the Householder matrix Hk+1 sending k+1 k+1 (0,...,0,uk+1,...,un )to(0,...,0,rk+1,k+1, 0,...,0), where k+1 k+1 rk+1,k+1 = (uk+1,...,un ) . (2) If we allow negative diagonal entries inR,thematrix Hn may be omitted (Hn = I).

(3) If A is invertible and the diagonal entries of R are positive, it can be shown that Q and R are unique. 478 CHAPTER 7. QR-DECOMPOSITION FOR ARBITRARY MATRICES (4) The method allows the computation of the determi- nant of A.Wehave det(A)=( 1)mr r , 1,1 ··· n,n where m is the number of Householder matrices (not the identity) among the Hi.

(5) The of the matrix A is preserved. This is very good for .