Math 344 Lecture #18 3.4 QR with Householder Transformations We detail a different algorithm for finding a QR-decomposition of a matrix through left multiplication by special orthonormal matrices knows as Householder transformations. 3.4.1 The Geometry of Householder Transformations Definition 3.4.1. For a nonzero vector v 2 Fn, the orthogonal complement of v, denoted by v?, is the set of all vectors x 2 Fn that are orthogonal to v, i.e., ? n v = fx 2 F : hv; xi = 0g: n For each x 2 F , the residual x − projv(x) is orthogonal to v, and so ? x − projv(x) 2 v : This means that the map x 7! x − projv(x) is the projection of x onto the orthogonal complement v?, i.e., v vvHx vvH proj ? (x) = x − proj (x) = x − hv; xi = x − = I − x: v v kvk2 vHv vHv The idea of a Householder transformation is to double the vector projv(x) subtracted from x, resulting in a reflection of vectors across an orthogonal complement of a vector. Definition 3.4.3. For a fixed nonzero v 2 Fn the Householder transformation of x 2 Fn is the map Hv given by vvH H (x) = x − 2proj (x) = I − 2 x: v v vHv Proposition 3.4.4. Assume v 2 Fn is nonzero. Then, (i) Hv(x) or an orthonormal transformation, and ? (ii) Hv(x) = x for all x 2 v . The proof of these is HW (Exercise 3.20). 3.4.2 Computing the QR Decomposition via Householder The first step in finding a QR decomposition of a matrix is to find for each nonzero n n x 2 F the nonzero v 2 F for which Hv(x) 2 span(e1), where e1 is the first standard basis vector of Fn. n Lemma 3.4.5. For x 2 R , if v = x + kxke1 is nonzero, then Hv(x) = −kxke1, and if v = x − kxke1 is nonzero, then Hv(x) = kxke1. Proof. The argument for the two cases is similar; we show the argument for v = x + kxke1 6= 0. For x1 the first entry of x we have T T v v = (x + kxke1) (x + kxke1) T T T 2 T = x x + kxkx e1 + kxke1 x + kxk e1 e1 2 = 2kxk + 2kxkx1 = 2kxk(kxk + x1); and T T vv x = (x + kxke1)(x + kxke1) x T T T 2 T = xx x + kxkxe1 x + kxke1x x + kxk e1e1 x 2 3 2 = kxk x + kxkx1x + kxk e1 + kxk x1e1 2 = kxk(kxkx + x1x + kxk e1 + kxkx1e1) = kxk (kxk + x1)x + kxk(kxk + x1)e1 = kxk(kxk + x1)(x + kxke1): Then vvT x H (x) = x − 2 v vT v kxk(kxk + x )(x + kxke ) = x − 2 1 1 2kxk(kxk + x1) = x − (x + kxke1) = −kxke1: This gives the result. The complex sign of a complex number z is sign(z) = z=jzj when z 6= 0, and sign(0) = 1. n Lemma 3.4.7. Let x1 be the first entry of a nonzero x 2 C . If v = x + sign(x1)kxke1, then Hv(x) = −sign(x1)kxke1: The proof of this is HW (Exercise 3.21). Remark 3.4.8. The vector v = x + sign(x1)kxke1 is never zero unless x = 0. For if x + sign(x1)kxke1 = 0 when x 6= 0, then x = −sign(x1)kxke1 which implies that x1 = −sign(x1)jx1j, but since x1 = sign(x1)jx1j we would get sign(x1) = −sign(x1), which implies that sign(x1) = 0, which is a contradiction. The algorithm for computing a QR decomposition by means of Householder transforma- tions is repeated applications of Lemma 3.4.5 and/or Lemma 3.4.7. n Theorem 3.4.9. For A 2 Mm×n(F) with m ≥ n there are vectors v1; v2; : : : ; vl 2 F where l = minfm − 1; ng such that R = Hvl ··· Hv2 Hv1 A is upper triangular matrix and Q = HH HH ··· HH is an orthonormal matrix that satisfy A = QR. v1 v2 vl The proof of this is quite long and involved. We will demonstrate the proof by way of an example. 3.4.3 A Complete Worked Example Recall that previously we found a QR decomposition for 21 −1 4 3 61 4 −27 A = 6 7 41 4 2 5 1 −1 0 by means of the Gram-Schmidt process, obtaining 21=2 −1=2 1=2 −1=23 22 3 2 3 61=2 1=2 −1=2 −1=27 60 5 −27 Q = 6 7 ;R = 6 7 : 41=2 1=2 1=2 1=2 5 40 0 4 5 1=2 −1=2 −1=2 1=2 0 0 0 To find a QR decomposition of A by means of Householder transformations, we start with the first column x1 of A, set 213 213 2−13 617 607 6 1 7 v1 = x1 − sign(x11)kx1ke1 = 6 7 − 2 6 7 = 6 7 ; 415 405 4 1 5 1 0 1 and get the Householder transformation 21=2 1=2 1=2 1=2 3 61=2 1=2 −1=2 −1=27 Hv1 = 6 7 : 41=2 −1=2 1=2 −1=25 1=2 −1=2 −1=2 1=2 Multiplying A on the left by Hv1 gives 22 3 23 60 0 07 Hv1 A = 6 7 : 40 0 45 0 −5 2 Notice that the first column of Hv1 A is a positive scalar multiple of e1 as Lemma 3.4.5 predicted. We now consider the second column x2 of Hv1 A which can we written as 233 2 0 3 0 00 607 6 0 7 x2 = x + x = 6 7 = 6 7 2 2 405 4 0 5 0 −5 0 00 ? where x2 2 span(fe1g) and x2 2 e1 . 00 00 Taking x22 to be the second entry of x2 we set 2 0 3 203 2 0 3 00 00 00 6 0 7 657 6−57 v2 = x − sign(x )kx ke2 = 6 7 + (−1) 6 7 = 6 7 : 2 22 2 4 0 5 405 4 0 5 −5 0 −5 The Householder transformation associated to v2 is 21 0 0 0 3 60 0 0 −17 Hv2 = 6 7 : 40 0 1 0 5 0 −1 0 0 00 Since x2 and e2 are both orthogonal to e1, the vector v2 is orthogonal to e1. 0 0 0 Since x2 2 span(fe1g), we obtain by Proposition 3.4.4 part (ii) that Hv2 (x2) = x2. 0 00 0 00 Thus Hv2 (x2) = Hv2 (x2 + x2) = x2 + Hv2 (x2). 00 By the appropriate adaptation of Lemma 3.4.7 the vector Hv2 (x2) belongs to span(fe2g). Thus Hv2 (x2) belongs to span(fe1; e2g), and so the second column of Hv2 (Hv1 A) belongs to span(fe1; e2g). We verify this: 22 3 2 3 60 5 −27 Hv2 (Hv1 A) = 6 7 : 40 0 4 5 0 0 0 This is R and 21=2 −1=2 1=2 −1=23 T T T 61=2 1=2 −1=2 −1=27 Q = (Hv2 Hv1 ) = H H = 6 7 : v1 v2 41=2 1=2 1=2 1=2 5 1=2 −1=2 −1=2 1=2 [All the matrix computations, including the entries of the Householder matrices, were performed in Maple.].