LAPACK Working Note Generalized QR Factorization and its Applications Work in Progress E. Anderson, Z. Bai and J. Dongarra Decemb er 9, 1991 August 9, 1994 Abstract The purp ose of this note is to re-intro duce the generalized QR factorization with or without pivoting of two matrices A and B having the same numberofrows, and whenever B is square and nonsingular, the factorization implicitly gives the orthogonal 1 factorization with or without pivoting of B A. The GQR factorization was intro- duced early by Hammarling[6] and Paige[9]. But from the general-purp ose software development p oint of view, we prop osed the di erent factorization forms. In addition to the factorization forms and implementation details, we show the applications of GQR factorization in solving the linear equality constraint least square problem, generalized linear mo del. It is intended to show the p ossible usage of LAPACK co des for solving a class of generalized least square problems who arise from optimization and statistics on high-p erformance machines. 1 Intro duction QR factorization of an n by m matrix A assumes the form A = QR T where Q is an n by n orthogonal matrix, R = Q A is zero b elow its diagonal. If n m, T then Q A can b e written in the form " R T Q A = 0 where R is an n by n upp er triangular. If n<m, then the QR factorization of A assumes the form h i T Q A = R S where R is an n by n upp er triangular matrix. However, in practical applications, it is more convenient to represent the factorization in this case as h i A = 0 R Q; This work was supp orted in part by NSF grant ASC-xxxx 1 The Generalized QR Decomp osition 2 which is known as the RQ factorization. As the variants of the QR and RQ factorization of matrix A,we also have QL and LQ factorization, which are orthogonal-lower triangular and lower triangular-orthogonal factorization, resp ectively. Moreover, it is well-known that the orthogonal factors of A provide information ab out its column and row spaces [4]. A column pivoting option in the QR factorization allows the user to detect dep endencies among the columns of matrix A.IfAhas rank k , then there are orthogonal matrix Q and a p ermutation matrix P such that " R R k 11 12 T Q AP = 0 0 n k k m k where R is a k by k upp er triangular and nonsingular[4]. 11 Householder transformation matrix or Givens rotation matrix provide numerical stable numerical metho ds to compute these factorizations with or without pivoting. The software for computing the QR factorization on sequential machines is available from public linear algebra library LINPACK[7 ]. Redesigned co des in blo ck algorithm fashion that are b etter suited for to day's high-p erformance architectures can b e found in LAPACK. The terminology generalized QR factorizations GQR factorization, which has b een intro duced by Hammarling[6] and Paige[9], is to refer to orthogonal transformations that apply to n by m matrix A and n by p matrix B to transform them to triangular forms, 1 resp ectively, but which corresp onds to the QR factorization of B A in the case whenever B is square and nonsingular. For example, if n m, n p, then the GQR factorization of A and B assumes the form " h i R T T Q A = ; Q BV = 0 S ; 0 where Q is an n by n orthogonal matrix, V is a p by p upp er triangular matrix, R is a m by m upp er triangular, S is a p by p upp er triangular. If B is square and nonsingular, then 1 the QR factorization of B A is given by " " T R T 1 1 V B A= = S ; 0 0 i.e., the upp er triangular part T of QR factorization can b e determined by solving the triangular matrix equation ST = R: 1 The advantage of this implicit determination of the QR factorization of B A is obvious. 1 1 Weavoid the p ossible numerical diculties to form B and B A. As the p owerful to ol of QR factorization in least square and related linear regression problems, as examples, we shall show that the GQR factorization can b een used to solve linear equality constrained least square problem min kAx bk; Bx=d where A and B are m by n and p by m matrices, resp ectively, and generalized linear regression mo del T min u u; sub ject to b = Ax + Bu x;u The Generalized QR Decomp osition 3 where A is an n by m matrix, B is an n by p matrix. Indeed, the QR factorization approachs have b een used for solving these problems, see Lawson and Hanson [8], Paige[10]. We will see that the GQR factorization of A and B provides an uniform approach to these problems. The b ene t of this approach is threefold. First, it uses a single GQR factorization concept to solve these problems directly. Second, from software development p oint of view, it allows us to develop one subroutine that can b e used for solving these problems. Third, as the QR factorization provides imp ortant information of conditioning of linear least square, classical linear regression mo del, we will show that the GQR factorization is the same. The condition numb ers of these problems can b e exploited from the triangular factors of the factorization. The principle concepts ab out the GQR factorization discussed in this note have b een presented in Paige's work on GQR[10]. However, from general-purp ose software develop- ment p oint of view, we will take a di erent approach for the GQR factorization. The de nition of the GQR factorization is di erent from the one presented in Paige's pap er. As a guideline of the development of the GQR factorization for LAPACK library, in this note, we consider the di erent p ossible cases of the factorizations and practical implementation of the factorizations. The outline of this LAPACK working note is as follows: In next two sections, we shall showhow to use existing QR factorization and its variants to construct the GQR factoriza- tion with or without pivoting strategies of two matrices A and B having the same number of rows. The implementation details of the di erent factorizations are discussed in section 4. Then we show the applications of the GQR factorization in solving the linear equality constrained least square problem, generalized linear mo del problem, and estimating the conditioning of these problems. Notations: .... 2 Generalized QR Factorization In this section, we rst intro duce the GQR factorization of n by m matrix A and n by p matrix B when n m, the most frequently o ccurring case. Then for the case n<m,we intro duce the GRQ factorization of A and B . GQR factorization. Let A beann by m matrix, B beann by p matrix, n m, then thereare orthogonal matrices Qn n and V p p such that T T Q A = R; Q BV = S 1 assumes one of the fol lowing forms: if n p, " h i R m 11 R = S = 0 S n 11 0 n m ; ; p n n m where m by m matrix R and n by n matrix S are upper triangular, and if n>p, 11 11 " " R m S n p 11 11 R= S = 0 n m ; S p ; 21 m p The Generalized QR Decomp osition 4 where m by m matrix R and p by p matrix S are upper triangular. 11 21 Proof: The pro of is constructive. By the QR factorization of A wehave " m R 11 T Q A= 0 n m : m T Let Q premultiply on B , then the desired factorizations follow up on the RQ factorization T of Q B ;ifnp, i h T n Q BV = 0 S 11 : p n n 1 otherwise, the RQ factorization of B A is of the form " n p S 11 T Q B V = p : S 21 p 2. 1 Occasionally, one wishes to compute the QR factorizations of B A, for example, to solveweighted least square problem 1 min kB Ax bk: x 1 1 Toavoid forming B and B A,we note that the GQR factorization 1 of A and B 1 implicitly gives the QR factorization B A: " " R T 11 1 T 1 =S ; V B A= 11 0 0 1 i.e, the upp er triangular part T of the QR factorization of B A can b e determined by S T = R : 11 11 Hence, the p ossible numerical diculties to use, explicitly or implicitly, the QR factorization 1 of B A is con ned to the condition number of S . 11 h i Moreover, if we partition V = V V where V has m columns, then 1 2 1 1 1 B A = V S R : 1 11 11 This shows that if A is of rank m, the columns of V form an orthonormal basis for the 1 1 T space spanned by the columns of B A. The matrix V V is the orthogonal pro jection on 1 1 1 the column space of B A. When A is n by m matrix with n<m, although it still can b e presented the similar GQR factorization form of A and B , it is more useful in applications to represent the factorization as the following: GRQ factorization: Let A bean nby m matrix, B beann by p matrix, n<m, then thereare orthogonal matrices Qn n and U m m such that T T Q AU = R; Q B = S 2 The Generalized QR Decomp osition 5 assumes one of the fol lowing forms if n p i i h h n n S = S S R = 0 R 11 12 11 ; ; n p n m n n and if n>p " i h p S 11 n S = R= 0 R 11 0 n p ; ; m n n p where n by n matrix R and min n; p by min n; p matrix S are upper triangular.