LAPACK Working Note
Generalized QR Factorization and its Applications
Work in Progress
E. Anderson, Z. Bai and J. Dongarra
Decemb er 9, 1991
August 9, 1994
Abstract
The purp ose of this note is to re-intro duce the generalized QR factorization with
or without pivoting of two matrices A and B having the same numberofrows, and
whenever B is square and nonsingular, the factorization implicitly gives the orthogonal
1
factorization with or without pivoting of B A. The GQR factorization was intro-
duced early by Hammarling[6] and Paige[9]. But from the general-purp ose software
development p oint of view, we prop osed the di erent factorization forms. In addition
to the factorization forms and implementation details, we show the applications of GQR
factorization in solving the linear equality constraint least square problem, generalized
linear mo del. It is intended to show the p ossible usage of LAPACK co des for solving a
class of generalized least square problems who arise from optimization and statistics on
high-p erformance machines.
1 Intro duction
QR factorization of an n by m matrix A assumes the form
A = QR
T
where Q is an n by n orthogonal matrix, R = Q A is zero b elow its diagonal. If n m,
T
then Q A can b e written in the form
"
R
T
Q A =
0
where R is an n by n upp er triangular. If n the form h i T Q A = R S where R is an n by n upp er triangular matrix. However, in practical applications, it is more convenient to represent the factorization in this case as h i A = 0 R Q; This work was supp orted in part by NSF grant ASC-xxxx 1 The Generalized QR Decomp osition 2 which is known as the RQ factorization. As the variants of the QR and RQ factorization of matrix A,we also have QL and LQ factorization, which are orthogonal-lower triangular and lower triangular-orthogonal factorization, resp ectively. Moreover, it is well-known that the orthogonal factors of A provide information ab out its column and row spaces [4]. A column pivoting option in the QR factorization allows the user to detect dep endencies among the columns of matrix A.IfAhas rank k , then there are orthogonal matrix Q and a p ermutation matrix P such that " R R k 11 12 T Q AP = 0 0 n k k m k where R is a k by k upp er triangular and nonsingular[4]. 11 Householder transformation matrix or Givens rotation matrix provide numerical stable numerical metho ds to compute these factorizations with or without pivoting. The software for computing the QR factorization on sequential machines is available from public linear algebra library LINPACK[7 ]. Redesigned co des in blo ck algorithm fashion that are b etter suited for to day's high-p erformance architectures can b e found in LAPACK. The terminology generalized QR factorizations GQR factorization , which has b een intro duced by Hammarling[6] and Paige[9], is to refer to orthogonal transformations that apply to n by m matrix A and n by p matrix B to transform them to triangular forms, 1 resp ectively, but which corresp onds to the QR factorization of B A in the case whenever B is square and nonsingular. For example, if n m, n p, then the GQR factorization of A and B assumes the form " h i R T T Q A = ; Q BV = 0 S ; 0 where Q is an n by n orthogonal matrix, V is a p by p upp er triangular matrix, R is a m by m upp er triangular, S is a p by p upp er triangular. If B is square and nonsingular, then 1 the QR factorization of B A is given by " " T R T 1 1 V B A = = S ; 0 0 i.e., the upp er triangular part T of QR factorization can b e determined by solving the triangular matrix equation ST = R: 1 The advantage of this implicit determination of the QR factorization of B A is obvious. 1 1 Weavoid the p ossible numerical diculties to form B and B A. As the p owerful to ol of QR factorization in least square and related linear regression problems, as examples, we shall show that the GQR factorization can b een used to solve linear equality constrained least square problem min kAx bk; Bx=d where A and B are m by n and p by m matrices, resp ectively, and generalized linear regression mo del T min u u; sub ject to b = Ax + Bu x;u The Generalized QR Decomp osition 3 where A is an n by m matrix, B is an n by p matrix. Indeed, the QR factorization approachs have b een used for solving these problems, see Lawson and Hanson [8], Paige[10]. We will see that the GQR factorization of A and B provides an uniform approach to these problems. The b ene t of this approach is threefold. First, it uses a single GQR factorization concept to solve these problems directly. Second, from software development p oint of view, it allows us to develop one subroutine that can b e used for solving these problems. Third, as the QR factorization provides imp ortant information of conditioning of linear least square, classical linear regression mo del, we will show that the GQR factorization is the same. The condition numb ers of these problems can b e exploited from the triangular factors of the factorization. The principle concepts ab out the GQR factorization discussed in this note have b een presented in Paige's work on GQR[10]. However, from general-purp ose software develop- ment p oint of view, we will take a di erent approach for the GQR factorization. The de nition of the GQR factorization is di erent from the one presented in Paige's pap er. As a guideline of the development of the GQR factorization for LAPACK library, in this note, we consider the di erent p ossible cases of the factorizations and practical implementation of the factorizations. The outline of this LAPACK working note is as follows: In next two sections, we shall showhow to use existing QR factorization and its variants to construct the GQR factoriza- tion with or without pivoting strategies of two matrices A and B having the same number of rows. The implementation details of the di erent factorizations are discussed in section 4. Then we show the applications of the GQR factorization in solving the linear equality constrained least square problem, generalized linear mo del problem, and estimating the conditioning of these problems. Notations: .... 2 Generalized QR Factorization In this section, we rst intro duce the GQR factorization of n by m matrix A and n by p matrix B when n m, the most frequently o ccurring case. Then for the case n intro duce the GRQ factorization of A and B . GQR factorization. Let A beann by m matrix, B beann by p matrix, n m, then thereare orthogonal matrices Q n n and V p p such that T T Q A = R; Q BV = S 1 assumes one of the fol lowing forms: if n p, " h i R m 11 R = S = 0 S n 11 0 n m ; ; p n n m where m by m matrix R and n by n matrix S are upper triangular, and if n>p, 11 11 " " R m S n p 11 11 R= S = 0 n m ; S p ; 21 m p The Generalized QR Decomp osition 4 where m by m matrix R and p by p matrix S are upper triangular. 11 21 Proof: The pro of is constructive. By the QR factorization of A wehave " m R 11 T Q A= 0 n m : m T Let Q premultiply on B , then the desired factorizations follow up on the RQ factorization T of Q B ;ifnp, i h T n Q B V = 0 S 11 : p n n 1 otherwise, the RQ factorization of B A is of the form " n p S 11 T Q B V = p : S 21 p 2. 1 Occasionally, one wishes to compute the QR factorizations of B A, for example, to solveweighted least square problem 1 min kB Ax b k: x 1 1 Toavoid forming B and B A,we note that the GQR factorization 1 of A and B 1 implicitly gives the QR factorization B A: " " R T 11 1 T 1 =S ; V B A = 11 0 0 1 i.e, the upp er triangular part T of the QR factorization of B A can b e determined by S T = R : 11 11 Hence, the p ossible numerical diculties to use, explicitly or implicitly, the QR factorization 1 of B A is con ned to the condition number of S . 11 h i Moreover, if we partition V = V V where V has m columns, then 1 2 1 1 1 B A = V S R : 1 11 11 This shows that if A is of rank m, the columns of V form an orthonormal basis for the 1 1 T space spanned by the columns of B A. The matrix V V is the orthogonal pro jection on 1 1 1 the column space of B A. When A is n by m matrix with n factorization form of A and B , it is more useful in applications to represent the factorization as the following: GRQ factorization: Let A bean nby m matrix, B beann by p matrix, n thereare orthogonal matrices Q n n and U m m such that T T Q AU = R; Q B = S 2 The Generalized QR Decomp osition 5 assumes one of the fol lowing forms if n p i i h h n n S = S S R = 0 R 11 12 11 ; ; n p n m n n and if n>p " i h p S 11 n S = R= 0 R 11 0 n p ; ; m n n p where n by n matrix R and min n; p by min n; p matrix S are upper triangular. 11 11 Pro of : The pro of is similar to the pro of of the GQR factorization. In short, one rst T do es the QR factorization of B : B = QS , then follows by the RQ factorization of Q A. 2 From the GRQ factorization form of A and B ,we see that if B is square and nonsingular, 1 then the RQ factorization of B A is given by h i h i 1 1 0 R : B A U = 0 T = S 11 11 With the decomp osition forms 1 and 2 , we present the formal de nition for the generalized QR factorizations. De nition: For n by m matrix A, n by p matrix B ,we call 1 as the generalized QR factorization in short the GQR factorization of A and B , 2 as the generalizedRQ factorization in short the GRQ factorization of A and B . Discussion: the other p ossible generalized QR factorization forms. For example, the QR T factorizaiton of B A. | will write later. 3 Generalized QR Factorization with Pivoting The previous section intro duces the generalized QR factorization. As in the QR factorization of a matrix, we can also incorp orate the pivoting technique into the GQR factorization to deal with the rank de cient case of the factorization. GQR factorization with column pivoting: Let A bean n by m matrix, B beann by p matrix, then thereare orthogonal matrices Q n n , V p p , and n by n permutation matrix P such that T T Q AP = R; Q BV = S 3 assumes one of the fol lowing forms: if n p 2 3 2 3 R R q 0 S S q 11 12 11 12 6 7 6 7 R = 0 0 k S = 0 0 S k 4 5 4 5 22 ; ; 0 0 n q k n q k 0 0 0 q m q p n q n q The Generalized QR Decomp osition 6 where R and S , if which exists, are ful l row rank upper trapezoidal matrices. q by q 11 22 matrix S is upper triangular. And if n>p, 11 3 2 3 2 q R R q S S 11 12 11 12 7 6 7 6 k R= 0 0 k S = 0 S 5 4 5 4 22 ; ; 0 0 n q k 0 0 n q k p n + q n q q m q where q by q matrix R are nonsingular and upper triangular, and k by n q matrix S 11 22 1 is ful l row rank upper trapezoidal . Proof : The pro of is also constructive. By the QR factorization with pivoting of A,we have " q R R 11 12 T Q AP = 1 0 0 n q : q m q where q = rank A . If n p, let " 0 S S q 11 12 T Q B V = 1 1 0 0 S n q 22 p n q n q T b e the RQ factorization of matrix Q B . Then by the QR factorization with pivoting of 1 submatrix S ,wehave 22 " S k 22 T Q S P = 22 2 2 0 n q k : n q " " I 0 I 0 The result for this case follows by setting Q = Q , V = V . 1 1 0 Q 0 P 2 2 T If n>p, let Q premultiply on B , denote as 1 " S q 11 T Q B = B = 1 S n q : 21 p if p n q , then by the QR factorization with pivoting of S ,wehave 21 " S k 21 T S P = Q 21 2 2 0 n q k p where k = rank S . The desired factorization forms are obtained by setting 21 " I 0 Q = Q and V = P . 1 2 0 Q 2 1 if p The Generalized QR Decomp osition 7 If p>n q,by the RQ factorization of B wehave " n p S 11 T Q B V = 1 S p : 21 p The conclusion for this case follows up on the QR factorization with pivoting of n q by n q b ottom right corner submatrix of S . 2 21 We note that in this case, if B is square and nonsingular, then the QR factorization 1 with column pivoting of B A is given by " R 11 T 1 1 : V B A P = S 11 0 Hence we present De nition: For n by m matrix A, n by m matrix B ,we call 3 as the generalizedQR factorization with column pivoting in short the GQR with pivoting of matrices A and B . 4 Implementations In this section, we shall discuss the algorithms and implementation details of GQR factor- ization and its variants. We will see that the implementations of all these factorizations can b e easily kept in the blo ck algorithm fashion as QR factorization do es. The \MATLAB" style notation A i : j; p : q is used in this section to sp ecify the ith to j th rows and pth to q th columns submatrix of matrix A. As was mentioned in section 2, the existence pro ofs of the GQR factorization and its variants are constructive. They strongly dep ends on the regular QR factorization and its variants. At the b eginning of this note we brie y reviewed the QR factorizations and its variants, the corresp ondent LAPACK subroutine names in typewriter font and its functions are listed in the followings: SGEQRF: QR factorization of a n by m matrix. SGEQPF: QR factorization of a n by m matrix with column pivoting. SGERQF:RQ factorization of a n by m matrix. SGELQF: LQ factorization of a n by m matrix. All these subroutines are the implementations of the blo ck algorithms. That is to say, instead of traditional algorithms to generate and apply Householder transformations one by one BLAS 2 matrix-vector op erations , the blo ck algorithms aggragate a series saya blo ck with size b of Householder transformations, represent in a blo ck matrix form and apply them to reduce a blo ck at the same time BLAS 3 matrix-matrix op erations . There is a common parameters called blo cksize NB of these subroutines that allows user to choice to nd the b est blo cksize one their machines. In particularly, if blo cksize is one, then they are just traditional QR factorizations and its variants. To call this routine, the input array A contains the n by m matrix. On the return of these subroutines, the upp er lower triangle of the array contains the triangular part of The Generalized QR Decomp osition 8 the factorizations, the orthogonal matrix can b e recovered from the elements b elow ab ove the diagonal with another one dimensional array. Turning to compute GQR factorization, we rst present the GQR factorization of n by m matrix A and n by m matrix B with n m, then the GRQ factorizations for the case n m and GQR factorization with column pivoting. Algorithm 1 GQR factorization, n m T T 1 . QR factoriztion of A: Q A = R and B = Q B . T 2 . RQ factorization of B : BV = S . To implement Algorithm 1, we note that the blo ck transformations can b e applied to B at the same time while A is reduced to upp er triangular form blo ckby blo ck. At the end of step 1, the upp er diagonal line part of A stores the factor R, the strictly lower diagonal line of A and another vector stores Householder vectors from which the orthogonal matrix Q can b e recovered later if necessary.At step 2, we just need to implement the blo ckRQ factorization on up dated B . When A is n by m with n m, B is n by p matrix, the following algorithm computes the GRQ factorization. Algorithm 2 GRQ factorization, n m T T 1. QR factorization of B : Q B = S , and A = Q A. 2. RQ factorization of A: A = RU . The remarks for algorithm 1 can b e established similarly in this case. Finally, to treat with ill-conditione d or rank de cient cases, we can use the GQR factorization with column pivoting. Algorithm 3 GQR factorization with pivoting T T 1 . QR factorization with pivoting of A: Q AP = R and B = Q B . 2 . determine the rank q of A. 3 . if n p then T RQ factorization of B : BV = S QR factorization with pivoting of S q +1 : n; p n + q +1 : p else if p n q then QR factorization with pivoting of B q +1 : n; 1:p . else T RQ factorization of B : BV = S , QR factorization with pivoting of S q +1 : n; q +1;p . endif endif Up date the q +1 to n columns of Q corresp ondently. The Generalized QR Decomp osition 9 Tonumerically determine the rank of A, according to the prop erties of the QR factor- ization with column pivoting, we can use the the following simple strategy: q =0; k = min n; m ; for i =1;m if jr j tol jr j q = q +1 ii 11 where tol is a user given tolerance value, for example, tol can b e chosed as machine precision. As a more careful metho d to determine the rank of a matrix, the so-called rank revealing QR factorization can also b e used to replace the normal column pivoting QR scheme[4, 2]. The MATLAB co des of algorithm 1, 2 and 3 are in App endix. 5 Applications In the ab ove sections, weintro duced the GQR factorization with or without pivoting. In this section, as examples, we will show that the GQR factorization provide a simplier and more ecientway to solve the linear equality constrained linear least square problem and generalized linear regression problem, and to assess the conditioning of these problems. The results presented in this section show that the GQR factorization for solving these general- ized optimization or linear regression problems is just as p ower as the QR factorization for solving least square and linear regression problems. 5.1 Linear Equality Constrained Linear Least Squares Linear equality constrained linear least squares problem Problem LSE arises in constrained surface tting, constrained optimizing, geo detic least-squares adjustment, and b eamforming etc. The problem is stated as follows: nd a n-vector x that solves min kAx bk; 4 Bx=d where A is m by n matrix, m n, B is p by n matrix, p n, b is n column vector, d is p column vector. Clearly, the Problem LSE has a solution if and only if the equation Bx = d is consistent. In addition, for simplicity,we will also assume that rank B =p 5 and that the null spaces of A and B intersect only trivially: " ! A N A \N B =f0g rank = n: 6 B These conditions ensure that the problem LSE has a unique solution whichwe denote by x . e Several metho ds for solving the problem LSE are discussed in Lawson and Hanson[8, Chaps.20-22], Van Loan[12]. For large sparse matrices case, see Barlow et al [1]. The QR style approach is one of the most common approach. Now this approach can b e more easily presented in terms of the GQR factorization of A and B . The Generalized QR Decomp osition 10 T T By the GQR factoization of B and A ,we know that there are orthogonal matrices Q and U " " T T T p p S 0 R R 11 11 12 T T T T Q B = Q A U = T 0 n p 0 0 R n p ; 22 p m n p n p T T and moreover, from the assumptions 4 and 6 , we know S and R are upp er triangular 11 22 and nonsingular. If we partition i i h h ; ; U = U U U Q = Q Q 1 2 3 1 2 where Q has p columns, U has m n columns, and U has p columns, and set 1 1 2 3 2 " m n c 1 p y 7 6 1 T T ; c = U b = c p ; y = Q x = 5 4 2 n p y 2 c n p 3 T T where y = Q x; i =1;2, c = U b; i =1;2;3. The Problem LSE is then transformed to i i i i 2 3 2 3 " 0 0 c 1 y 6 7 6 7 1 min R 0 c 4 5 4 5 11 2 y 2 R R c 12 22 3 sub ject to " h i y 1 S 0 = d: 11 y 2 Hence we can compute y from constrained equalityby solving the triangular system 1 S y = d: 11 1 Then the Problem LSE is truncated to the ordinary linear least square problem min kR y c R y k 22 2 3 12 1 y 2 Since R is nonsingular and lower triangular, y is given by 22 2 1 y = R c R y ; 2 3 12 1 22 a triangular solver. The solution of Problem LSE is then given by x = Qy = Q y + Q y e 1 1 2 2 or in a more straightforward form " I 1 T 1 x = Q R U b + Q S d e 2 1 3 22 11 R R 12 22 2 2 2 and the residual sum of squares = kr k = kAx bk is given by e e 2 2 2 = kc k + kR y c k : 1 11 1 2 The Generalized QR Decomp osition 11 The Sensitivity of Problem LSE: The condition numb ers of A and B were intro duced by Elden [3] to assess the p erturbation b ehavior of the Problem LSE. Sp eci call y, let E be an error matrix of A, F b e an error matrix of B , e and f b e errors of b and d, resp ectively. We assume that B + F also has full row rank and N A + E \N B +F =f0g. Letx e b e the solution of the same problem with A; B ; b; d replaced by A + E , B + F , b + e, d + f , resp ectively. Elden uses the numb ers + + A = kAkk AG k; B =kBkkB k B A IA to measure the sensitivity of the problem LSE, where + + + + G = I B B; B = I AG A B : IA His asymptotic p erturbation b ound, mo di ed slightly here, can b e presented as following, 2 where the b ound has a +O term i.e., the higher order of the p erturbation matrices E and F has not b een written. Problem LSE Perturbation Bound: kx x k kE k kF k kE k kF k e e 2 A + + + B + B + A B e e A A e B kx k kAk kB k kAk kB k e where kfk kek ; = ; = e e kAkkx k kBkkx k e e kr k e = ; r =Ax b: e e e kAkkx k e The interpretation of this result is that the sensitivityofx is measured by A and e B 2 B if the residual r is zero or relatively small, and otherwise by A B + 1 . A e A B We note that if matrix B is zero hence F = 0 , then the problem LSE is just the ordinary linear least square problem. The p erturbation b ound for the problem LSE is then reduced to kx x k kE k kek kEk kr k e e e 2 2 A + + A +O kx k kAk kAkkx k kAkkAkkx k e e e + where A = A =kAkkA k. This is just the p erturbation b ound of the linear least B square problem eariest othe btained by Golub and Wilkinson 1966 [4]. Esitmation of the condition numb ers: The condition numb ers of Problem LSE + + + A and B involve B , B B , AG etc, and computing these matrices can b e B A relatively exp ensive. Fortunately, it is p ossible to compute inexp ensive estimates of A B + + + and B without forming B , B B or AG . This can b e done using a metho d of A Hager 1984 and Higham 1988 [5] that computes a lower b ound for kB k , where B is a 1 T matrix, given a mean for evluating matrix-vector pro ducts Bu and B u.Typically, 4 or 5 pro ducts are required, and the lower b ound is almost always within a factor 3 of kB k . The 1 corresp onding subroutine, named as SLACON,isavailable in LAPACK. To estimate A B + and B , we apparently need to estimate vector norms kKzk , where K = AG ,or A 1 + K =B and z 0isavector that is readily computed. Given GQR factorization of A IA and B , after tedious computations, wehave 1 + T AG z = Q R U z 2 3 22 The Generalized QR Decomp osition 12 " I + 1 B z = Q S z 1 11 IA R R 12 22 1 1 where we do not form R or S but rather solve the triangular system and do matrix- 22 11 vector op erations. 5.2 Solve Generalized Linear Mo del The generalized linear regression mo del Problem GLM can b e written as b = Ax + w; 7 where the vector b, the n by m matrix A, n by n symmetric nonnegative de nite matrix W 2 are known, while w is a random error with mean 0 and covariance W . The problem is that of estimating the unknown parameters x on the basis of observation b.IfW has rank p, then W has a factorization T W = BB ; where the n by p matrix B has linearly indep endent columns, for example, the Cholesky factorization of W could b e carried out to get B . In some practical problems, the matrix B mightbeavailable directly.Form numerical computational reasons it is preferable to use B rather than W . Since W could b e ill-conditi oned in the sene discussed by Golub and Styan 1973 . Then the condition of B will b e much b etter. Thus we replace 7 by b = Ax + Bu; 8 where A is a n by m matrix, B is n by p matrix having full column rank, while u is a random 2 error with mean 0 and covariance I . Then the estimator of x in 8 is the solution to the following algebraic generalized linear least squares problem: T min u u sub ject to b = Ax + Bu 9 x;u For convenience, we assume that n m, n p, and rank B = p, the most frequently o curring case. When B = I , 9 is just an ordinary linear regression problem. The Problem GLM can b e formulated as the problem LSE which is discussed in the last subsection: " " h i h i x x min 0 I sub ject to A B = b: u u Hence, it is easy to see that the problem GLM has a unique solution under the following conditions: h i rank A B = n and rank A = m: Toavoid the overhead of computation cost, storage and p ossible numerical dicultyof the combination of the matrices A and B , it is not suggested to solve the problem GLM directly by the metho d of the problem LSE. Early 1979, Paige [10] prop osed two steps QR decomp osition approach to the problem GLM to treat with A and B separately.Now, his approach can b e simpli ed with GQR factorization terminology.For sake of arguments, we assume that the Problem GLM has unique solutions of x and u whichwe denote by x and e u . e The Generalized QR Decomp osition 13 By the GQR factorization with pivoting of A and B , under the uniqueness assumption, wehave orthogonal matrices Q n n , V p p , and m by m p ermutation matrix P such that " " R R q S S q 11 12 11 12 T T Q AP = Q BV = 0 0 n q ; 0 S n q 22 q m q p n + q n q where q by q matrix R , n q by n q matrix S are nonsingular upp er triangular. If we 11 22 partition i i h i h h ; ; P = P P ; V = V V Q = Q Q 1 2 1 2 1 2 where Q has q columns, V has n q columns, P has q columns, and set 1 2 1 " " " c v y 1 1 1 T T T c = Q b ; v = V u ; y = P x c v y 2 2 2 T T T i.e., c = Q b, v = V u, y = P x, i =1;2, then the problem GLM is transformed to i i i i i i " " " " " c R R y S S v 1 11 12 1 11 12 1 = + : 10 c 0 0 y 0 S v 2 2 22 2 Hence v can b e determined from the \b ottom" equation by solving a triangular system: 2 1 v = S c : 2 2 22 Then from the \top" equation, wehave c =R y + R y + S v + S v : 1 11 1 12 2 11 1 12 2 It is obvious that the rest of solutions are given by 1 v =0; y =0; y =R c S v : 1 2 1 1 12 2 11 The solution of the original problem GLM can b e written in the forms: x = P y ; u = V v e 1 1 e 2 2 or in a more straightforward form 1 1 T T x = P R Q S S Q b e 1 12 1 2 11 22 and 1 T u = V S Q b: e 2 2 22 The Sensitivity of the Problem GLM: Regarding to the sensitivity of the problem solutions to p erturbations, we shall consider the e ects of the p erturbations in the vector b and p erturbations in the matrices A and B . Let the p erturb ed problem GLM b e de ned as T min u u; sub ject to b + e = A+E x + B +F u; x; u The Generalized QR Decomp osition 14 and also assume that the p erturbated system has unique solution, i.e., rank A + E; B +F = n, and rank A+E =m. The solutions are denoted asx andu . Then wehave the following e e b ounds on the relative error inx andu due to the p erturbations of b, A and B , where e e 2 each asymptotic b ound has a +O term i.e., the higher order of the p erturbation data E , F etc which is not written. Problem GLM Perturbation Bounds: 2 kek kEk kF k kx x k kE k kBk kpk 1 e e A + + A A + ; 11 B B B kx k kAk kAkkx k kAk kBk kAkkx k e e e and 2 kE k B kx k kek kpk kpk ku u k kB kkpk e e e A A + kEk + +kF k +kFk ; B 1 kbk kAk kbk kBk kbk kbk kbk kbk 12 where + + A = kAkkA k; B = kBkk GB k; B A IB + + + + T + T T G = I AA , A = A I B GB and p = GB GB b, F = BF + F B. 1 IB The pro of is going to b e presented later. The b ounds are quite complicated. If we note that 2 2 kB k kpk B kbk: A then the b ounds 11 and 12 can b e simpli ed a little bit. We see that the sensitivities of x andu dep end on the A and B . For this reason, A and B are de ned e e B A B A as the condition numbers of Problem GLM. They can b e used to predict the e ects of errors in the regression variables on regression co ecients. As sp ecial case, we note that if B = I , then the problem GLM is reduced to the classical + linear regression problem. u is just the residual vector, u = p = r = b Ax = I AA b, e e e e u =r= b+e A+E x, F = 0, and + A = A = kAkkA k; B =1: B A Hence wehave kx x k kEk kek kEk kr k e e e 2 2 A + + A +O kxk kAk kAkkx k kAkkAkkx k e e and kr r k kE k kr k kx k kek e e e e 2 A + kE k + + O kbk kAk kbk kbk kbk These are the well-known p erturbation results for the solution and residual of ordinary linear least regression problem [11, 4]. Estimation of condition numb ers: Concerning ab out the estimating of the condition numb ers A and B of the Problem GLM, we still can use the Hager and Higham's B A + + or GB .To use their metho d, the required vector metho d without forming exp ensive A + norms kKzk can b e computed by GQR factorization of A and B , where K = GB ,or 1 + K =A and z 0isavector that is readily computed. After tedious computations, we IB have 1 + T GB z = V S Q z 2 2 22 The Generalized QR Decomp osition 15 and + 1 T 1 T A z = P R Q z S S Q z : 1 12 1 2 11 22 IB Hence, we can just use triangular system solver and matrix-vector op erations to give the estimation of condition numb ers of the Problem GLM. 5.3 Other p ossible applications: Will brie y mention the p ossible use of GQR in Prepro cessing step of computing the generalized singular value decomp osition. Structural equations: T T f = A t; e = BB t; e = Ad; where f is given, and we wish to nd d. The Generalized QR Decomp osition 16 A Matlab Co des In this app endix, we list the MATLAB co des for computing the GQR factorizations with and without pivoting. function [Q,V,R,S] = gqr A,B GQR ------ Compute the generalized QR factorization of A nxm and B nxp : A = Q*R; B = Q*S*V; where n >= m, Q nxn , V pxp are orthogonal matrices, R and S assumes one of the forms: ifn<=p R=[R11]m S=[0 S11 ] n [ 0 ] n-m p-n n m where m by m matrix R11 and n by n matrix S11 are upper triangular, and ifn>p R=[R11]m S = [ S11 ] n-p [ 0 ] n-m [ S21 ] p m p where m by m matrix R11 and p by p matrix S21 are upper triangular. ------ [n,m] = size A ; [n1,p] = size B ; if n1 ~= n error 'Matrices A and B have different rows' end ifn error 'suggest to use GRQ factorization' end QR factorization of A: Q'*A = RA [Q,R] = qr A ; The Generalized QR Decomp osition 17 RQ factorization of Q'*B: Q'*B = S*V; [S,V] = rq Q'*B ; Test the backward error resida = norm A - Q*R residb = norm B - Q*S*V function [Q,U,R,S] = grq A,B GRQ ------ Compute the generalized RQ factorization of A nxm and B nxp : A = Q*R; B = Q*S*V; where n <= m, Q nxn , V pxp are orthogonal matrices, R and S assumes one of the forms: ifn<=p R=[0 R11 ] n S = [ S11 S12 ] n m-n n n p-n ifn>p R=[0 R11 ] n S=[S11]p m-n n [ 0 ] n-p p where n by n matrix R11 and min n,p by min n,p matrix S11 are upper triangular. ------ [n,m] = size A ; [n1,p] = size B ; if n1 ~= n error 'Matrices A and B have different rows' end ifn>m The Generalized QR Decomp osition 18 error 'suggest to use GQR factorization' end QR decomposition of B: Q'*B = S [Q,S] = qr B ; RQ decomposition of Q'*A: Q'*A = R*U; [R,U] = rq Q'*A ; Test the backward error resida = norm A - Q*R*U residb = norm B - Q*S function [Q,P,V,R,S] = gqrp A,B GQRP ------ Generalized QR factorization with partial pivoting of matrices A nxm and B nxp . Find orthogonal matrices Q, V and permutation matrix P such that A = Q*R*P'; B = Q*S*V'; where R = [ R11 R12 ] q [ 0 0 ] n-q q m-q R11 qxq is nonsingular upper trinagular, and if n <= p, then S=[ 0 S11 S12 ]q [ 0 0 S22 ]k [ 0 0 0 ] n-k-q p-n q n-q S22 kx n-q is full row rank upper trapezoidal, if exists. The Generalized QR Decomp osition 19 else if n > p, then S = [ S11 S12 ] q [ 0 S22 ] k [ 0 0 ] n-q-k p-n+q n-q where S21 kxp is full row rank upper trapezoidal form. ------ [n,m] = size A ; [n1,p] = size B ; if n1 ~= n error 'Matrices A and B have different rows' end QR decomposition with column pivoting of A: Q'*A*P = R [Q,R,P] = qr A ; q = rank R ; if n<= p [S,V] = rq Q'*B ; V = V'; [Q1,S q+1:n,p-n+q+1:p ,P1] = qr S q+1:n,p-n+q+1:p ; S 1:q,p-n+q+1:p = S 1:q,p-n+q+1:p *P1; Q 1:n,q+1:n = Q 1:n,q+1:n *Q1; V 1:p,p-n+q+1:p = V 1:p,p-n+q+1:p *P1; else n > p case if p <= n-q S = Q'*B; [Q1,S q+1:n,1:p ,P1] = qr S q+1:n,1:p ; S 1:q,1:p = S 1:q,1:p *P1; Q 1:n,q+1:n = Q 1:n,q+1:n *Q1; V = P1; else p > n-q [S,V] = rq Q'*B ; V = V'; [Q1,S q+1:n,p- n-q +1:p ,P1] = qr S q+1:n,p- n-q +1:p ; S 1:q,p- n-q +1:p = S 1:q,p- n-q +1:p *P1; Q 1:n,q+1:n = Q 1:n,q+1:n *Q1; V 1:p,p- n-q +1:p = V 1:p,p- n-q +1:p *P1; end end The Generalized QR Decomp osition 20 Test the backward error if wanted. resida = norm A - Q*R*P' residb = norm B - Q*S*V' function [R,Q] = rq A RQ ------ Compute RQ decomposition of matrix A. A = R*Q ------ [n,m] = size A ; Q = eye m ; ifn<=m for i = n:-1:1 x = A i,1:m-n+i ; v = housen x ; v = v'; A 1:i,1:m-n+i = colhouse A 1:i,1:m-n+i ,v ; Q 1:m,1:m-n+i = colhouse Q 1:m,1:m-n+i ,v ; end else for i = n:-1:n-m+2 x = A i,1:m-n+i ; v = housen x ; v = v'; A 1:i,1:m-n+i = colhouse A 1:i,1:m-n+i ,v ; Q 1:m,1:m-n+i = colhouse Q 1:m,1:m-n+i ,v ; end end R=A; Q = Q'; function A = colhouse A,v Apply the Householder matrix from right left A <== A*H beta = -2/ v'*v ; The Generalized QR Decomp osition 21 w = beta*A*v; A = A + w*v'; function v = house x HOUSE computes the Householdr vectors n = max size x ; mu = norm x ; v= x; if mu ~= 0, beta = x 1 + sign x 1 *mu; v 2:n = v 2:n /beta; end v 1 = 1; The Generalized QR Decomp osition 22 References [1] J. L. Barlow, N. K. Nichols, and R. J. Plemmons, Iterative metho ds for equality con- strained least squares problems, SIAM J. Sci. and Stat. Comp. 9, pp.892-906, 1988. [2] C. H. Bischof and P. C. Hansen Structure-Perserving and Rank-Revealing QR- Factorizations, Argonal National Lab oratory, priprint MCS-P100-0989, 1989 [3] L. Elden, Perturbation theory for the least squares problem with linear equality con- straints, SIAM J. Numer. Anal. Vol.17, pp.338-350, 1980 [4] G. H. Golub and C. F. Van Loan, Matrix computations, The Johns Hopkins Univ. Press, Baltimore, MD, 1989 [5] N. J. Higham, FORTRAN codes for estimating the one-norm of a real or complex matrix, with applications to condition estimation,ACM TOMS, 14: 381-396 1988 . [6] S. Hammarling, The numerical solution of the general Gauss-Markov linear mo del, NAGTechnical Rep ort, TR2/85, 1985. [7] J. J. Dongarra, J. R. Bunch, C. B. Moler and G. W. Stewart, LINPACK User's Guide, So ciety for Industrial and Applied Mathematics, Philadelphia, 1978. [8] C. L. Lawson and R. J. Hanson, Solving least squares problems, Prentice-Hall, Engle- wo o d Cli s, NJ, 1974. [9] C. Paige, Some asp ects of generalized QR factorization. [10] C. Paige, Computer solution and p erturbation analysis of generalized linear least squares problems. Math. Comp. Vol. 33, pp. 171-183, 1979 [11] G. W. Stewart, On the p erturbation of pseudo-inverses, pro jections and linear least squares problems, SIAM Rev. Vol.19, pp.634-662, 1977 [12] C. F. Van Loan, On the metho d of weighting for equality-constrained least-squares problems, SIAM J. Numer. Anal. 22, pp.851-864. 1985.