DOCSLIB.ORG

QR Decomposition on Gpus

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
QR Decomposition on Gpus QR Decomposition on GPUs Andrew Kerr, Dan Campbell, Mark Richards Georgia Institute of Technology, Georgia Tech Research Institute {andrew.kerr, dan.campbell}@gtri.gatech.edu, [email protected] ABSTRACT LU, and QR decomposition, however, typically require ¯ne- QR decomposition is a computationally intensive linear al- grain synchronization between processors and contain short gebra operation that factors a matrix A into the product of serial routines as well as massively parallel operations. Achiev- a unitary matrix Q and upper triangular matrix R. Adap- ing good utilization on a GPU requires a careful implementa- tive systems commonly employ QR decomposition to solve tion informed by detailed understanding of the performance overdetermined least squares problems. Performance of QR characteristics of the underlying architecture. decomposition is typically the crucial factor limiting problem sizes. In this paper, we focus on QR decomposition in particular and discuss the suitability of several algorithms for imple- Graphics Processing Units (GPUs) are high-performance pro- mentation on GPUs. Then, we provide a detailed discussion cessors capable of executing hundreds of floating point oper- and analysis of how blocked Householder reflections may be ations in parallel. As commodity accelerators for 3D graph- used to implement QR on CUDA-compatible GPUs sup- ics, GPUs o®er tremendous computational performance at ported by performance measurements. Our real-valued QR relatively low costs. While GPUs are favorable to applica- implementation achieves more than 10x speedup over the tions with much inherent parallelism requiring coarse-grain native QR algorithm in MATLAB and over 4x speedup be- synchronization between processors, methods for e±ciently yond the Intel Math Kernel Library executing on a multi- utilizing GPUs for algorithms computing QR decomposition core CPU, all in single-precision floating-point. We present remain elusive. throughput for real-valued implementations of this QR im- plementation executing on several GPU architectures and In this paper1, we discuss the architectural characteristics believe this is the fastest strictly GPU implementation yet of GPUs and explain how a high-performance implementa- presented. Our implementation performs all processing on tion of QR decomposition may be implemented. We provide the selected GPU, enabling threads on the host processor detailed performance analysis of the resulting implementa- to perform other computations and to devote PCI-Express tion for real-valued matrices and o®er recommendations for bandwidth to other tasks. achieving high performance to future developers of dense lin- ear algebra procedures for GPUs. Our implementation sus- tains 143 GFLOP/s, and we believe this is the fastest an- 2. BACKGROUND nounced QR implementation executing entirely on the GPU. Commodity GPUs are inexpensive resources for delivering very high computing throughput for certain classes of ap- plications. GPUs are sold primarily as an integrated com- 1. INTRODUCTION ponent in display adapters for desktop personal computers. Graphics Processing Units are massively parallel processors High-throughput GPUs are primarily aimed for the video capable of completing hundreds of floating point operations game market. The primary application of GPUs has a very in parallel. Their large register ¯les and high-performance large degree of potential parallelism at the most computa- scratchpad memories are well-suited to streaming execution tionally intensive step: applying ¯nal shading e®ects to each models. Matrix factorization algorithms such as Cholesky, pixel in a polygon as it is rendered into the display bu®er. This fact has allowed GPU vendors to exploit microarchi- 1This work was supported in part by DARPA and AFRL under contracts FA8750-06-1-0012 and FA8650-07-C-7724. tecture parallelism for increased performance without con- The opinions expressed are those of the authors. straint by the application and without requiring much archi- tectural infrastructure to facilitate parallel execution. The volume of the GPU market provides tremendous competi- tive pressure to improve performance and keep prices low Permission to make digital or hard copies of all or part of this work for over successive product generations. personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies Simultaneously, GPU execution models have grown in flex- bear this notice and the full citation on the first page. To copy otherwise, to ibility in response to the needs of graphics programmers republish, to post on servers or to redistribute to lists, requires prior specific thereby enabling a wide range of computing tasks. GPU permission and/or a fee. GPGPU '09, March 8, 2009, Washington, D.C., USA. vendors have consequently developed graphics-agnostic pro- Copyright 2009 ACM 978-1-60558-517-8/09/03 ... $5.00 gramming models such as NVIDIA's Compute Uni¯ed De- vice Architecture (CUDA) [1] and Open Compute Layer · ¸ · ¸ · ¸ · ¸ (OpenCL) [2] to facilitate general purpose computing on f c s f r G(θ) = = GPUs. Nevertheless, fully exploiting the peak performance g ¡s¤ c g 0 capacity of GPUs has remained a challenge. Algorithms with very high arithmetic intensity, very little need to syn- chronize between execution paths, and very few scatter oper- The algorithm begins by choosing elements from the bottom ations typically perform well on GPUs without the need for two rows in A in the left-most column. These determine the careful optimization, but many computing tasks do not ¯t the Givens rotation Gm;1(θ) which may then be used to these idealized constraints. Several algorithms for fast QR multiply in-place the submatrix of A formed by the bottom H decomposition exhibit a high degree of parallelism, but have two rows. Then, Gm;1(θ) may be used to multiply a pair of low arithmetic intensity and are highly coupled between ex- columns Q(:; m¡1 : m) in place. The multiplication in A has ecution paths, requiring synchronization between elements the desired e®ect of overwriting the bottom left element with after small numbers of arithmetic operations. As a result, 0 while preserving the invariants in the algorithm. Next, the attempts to exploit GPUs to accelerate QR decomposition algorithm proceeds up one row in A and repeats, computing have been moderately successful achieving 4x speedup [3] Gm¡1;1(θ) and applying it to A(m ¡ 2 : m ¡ 1; :) and Q(: over a reference implementation distributed with Lincoln' ; m¡2 : m¡1). This proceeds throughout the entire column Laboratory's HPEC Challenge [4]. In this paper, we base stopping at the main diagonal at which point all of the left speedup numbers on the highly optimized Intel Math Ker- column in A except the ¯rst element is overwritten with nel Library which is among the fastest QR implementations zeros. The algorithm moves right one column and repeats available for multicore CPUs. from the bottom row, again stopping at the main diagonal. When all columns have been visited, A is in upper triangular form, and the QR decomposition is complete. 3. QR ALGORITHMS QR decomposition factors an m-by-n matrix A into the G(θ) may be computed from f and g without explicitly com- product A = QR, where Q is an m-by-m unitary matrix puting θ or evaluating any trigonometic functions [7]. More- and R is an m-by-n upper triangular matrix. Several one- over, Sameh and Kuck [8] demonstrate a pattern in which sided factorization methods compute the QR decomposition. Givens rotations may be computed in parallel. This method Two of these, Givens and Householder [5], apply a set of or- is adaptable to streaming architectures and systolic arrays thogonal transformations to the input matrix to bring it into such as MIT RAW [9]. In general, this approach achieves upper triangular form. By concatenating these orthogonal good performance on MIMD architectures that support low- transforms, the matrix Q is formed, all while preserving the latency communication and synchronization. GPUs, on the H invariants A = QR and Q Q = I. Modi¯ed Gram-Schmidt other hand, do not o®er on-chip mechanisms for synchroniz- computes the QR factorization via projection operations. ing the several SIMD processors and require either invoking Each of these methods has favorable numerical properties a sequence of kernels with implicit synchronization barriers and o®ers some parallelism. The suitability of each algo- between each invocation or assuming a consistency model rithm for GPU implementation depends on memory access for global memory and implementing barriers in the shared patterns, the frequency of inter-processor synchronization, global address space. Performance results for Givens QR and the scalability of parallelism within the algorithm. implemented on CUDA-compatible GPUs have been pre- sented for the HPEC Challenge benchmarks [3]. However, attempts to develop a high performance implementation of 3.1 Modified Gram-Schmidt this algorithm in CUDA were met with limited success and The modi¯ed Gram-Schmidt QR (MGS) method computes do not scale well to arbitrarily large matrix sizes. A = Q1R1 where A is m-by-n, Q1 is m-by-n with orthonor- mal columns, and R1 is n-by-n. This factorization di®ers from what we have de¯ned in that Q is not square. MGS of- 3.3 Householder QR fers fairly good numerical properties, but it consists of oper- Householder QR computes the upper triangular matrix R ations that are not favorable to high performance on GPUs: from an m-by-n matrix A by applying a sequence of House- computing each element of the output matrix R requires holder reflections to A in place [5]. A Householder reflection large vector inner products and many synchronizations [5]. is an orthogonal transform of the form Implementing MGS on a GPU would incur prohibitive over- head. A blocked method exists in which the matrix to be factored is ¯rst partitioned into a number of su±ciently 2 P = I ¡ vvH small submatrices which are independently factored.
Load more

Recommended publications
  • The Multishift Qr Algorithm. Part I: Maintaining Well-Focused Shifts and Level 3 Performance∗
    SIAM J. MATRIX ANAL. APPL. c 2002 Society for Industrial and Applied Mathematics Vol. 23, No. 4, pp. 929–947 THE MULTISHIFT QR ALGORITHM. PART I: MAINTAINING WELL-FOCUSED SHIFTS AND LEVEL 3 PERFORMANCE∗ KAREN BRAMAN† , RALPH BYERS† , AND ROY MATHIAS‡ Abstract. This paper presents a small-bulge multishift variation of the multishift QR algorithm that avoids the phenomenon ofshiftblurring, which retards convergence and limits the number of simultaneous shifts. It replaces the large diagonal bulge in the multishift QR sweep with a chain of many small bulges. The small-bulge multishift QR sweep admits nearly any number ofsimultaneous shifts—even hundreds—without adverse effects on the convergence rate. With enough simultaneous shifts, the small-bulge multishift QR algorithm takes advantage ofthe level 3 BLAS, which is a special advantage for computers with advanced architectures. Key words. QR algorithm, implicit shifts, level 3 BLAS, eigenvalues, eigenvectors AMS subject classifications. 65F15, 15A18 PII. S0895479801384573 1. Introduction. This paper presents a small-bulge multishift variation of the multishift QR algorithm [4] that avoids the phenomenon of shift blurring, which re- tards convergence and limits the number of simultaneous shifts that can be used effec- tively. The small-bulge multishift QR algorithm replaces the large diagonal bulge in the multishift QR sweep with a chain of many small bulges. The small-bulge multishift QR sweep admits nearly any number of simultaneous shifts—even hundreds—without adverse effects on the convergence rate. It takes advantage of the level 3 BLAS by organizing nearly all the arithmetic workinto matrix-matrix multiplies. This is par- ticularly efficient on most modern computers and especially efficient on computers with advanced architectures.
    [Show full text]
  • The SVD Algorithm
    Jim Lambers CME 335 Spring Quarter 2010-11 Lecture 6 Notes The SVD Algorithm Let A be an m × n matrix. The Singular Value Decomposition (SVD) of A, A = UΣV T ; where U is m × m and orthogonal, V is n × n and orthogonal, and Σ is an m × n diagonal matrix with nonnegative diagonal entries σ1 ≥ σ2 ≥ · · · ≥ σp; p = minfm; ng; known as the singular values of A, is an extremely useful decomposition that yields much informa- tion about A, including its range, null space, rank, and 2-norm condition number. We now discuss a practical algorithm for computing the SVD of A, due to Golub and Kahan. Let U and V have column partitions U = u1 ··· um ;V = v1 ··· vn : From the relations T Avj = σjuj;A uj = σjvj; j = 1; : : : ; p; it follows that T 2 A Avj = σj vj: That is, the squares of the singular values are the eigenvalues of AT A, which is a symmetric matrix. It follows that one approach to computing the SVD of A is to apply the symmetric QR algorithm T T T T to A A to obtain a decomposition A A = V Σ ΣV . Then, the relations Avj = σjuj, j = 1; : : : ; p, can be used in conjunction with the QR factorization with column pivoting to obtain U. However, this approach is not the most practical, because of the expense and loss of information incurred from computing AT A. Instead, we can implicitly apply the symmetric QR algorithm to AT A. As the first step of the symmetric QR algorithm is to use Householder reflections to reduce the matrix to tridiagonal form, we can use Householder reflections to instead reduce A to upper bidiagonal form 2 3 d1 f1 6 d2 f2 7 6 7 T 6 .
    [Show full text]
  • QR Decomposition: History and Its Applications
    Mathematics & Statistics Auburn University, Alabama, USA QR History Dec 17, 2010 Asymptotic result QR iteration QR decomposition: History and its EE Applications Home Page Title Page Tin-Yau Tam èèèUUUÎÎÎ JJ II J I Page 1 of 37 Æâ§w Go Back fÆêÆÆÆ Full Screen Close email: [email protected] Website: www.auburn.edu/∼tamtiny Quit 1. QR decomposition Recall the QR decomposition of A ∈ GLn(C): QR A = QR History Asymptotic result QR iteration where Q ∈ GLn(C) is unitary and R ∈ GLn(C) is upper ∆ with positive EE diagonal entries. Such decomposition is unique. Set Home Page a(A) := diag (r11, . , rnn) Title Page where A is written in column form JJ II J I A = (a1| · · · |an) Page 2 of 37 Go Back Geometric interpretation of a(A): Full Screen rii is the distance (w.r.t. 2-norm) between ai and span {a1, . , ai−1}, Close i = 2, . , n. Quit Example: 12 −51 4 6/7 −69/175 −58/175 14 21 −14 6 167 −68 = 3/7 158/175 6/175 0 175 −70 . QR −4 24 −41 −2/7 6/35 −33/35 0 0 35 History Asymptotic result QR iteration EE • QR decomposition is the matrix version of the Gram-Schmidt orthonor- Home Page malization process. Title Page JJ II • QR decomposition can be extended to rectangular matrices, i.e., if A ∈ J I m×n with m ≥ n (tall matrix) and full rank, then C Page 3 of 37 A = QR Go Back Full Screen where Q ∈ Cm×n has orthonormal columns and R ∈ Cn×n is upper ∆ Close with positive “diagonal” entries.
    [Show full text]
  • Lecture 4: Applications of Orthogonality: QR Decompositions Week 4 UCSB 2014
    Math 108B Professor: Padraic Bartlett Lecture 4: Applications of Orthogonality: QR Decompositions Week 4 UCSB 2014 In our last class, we described the following method for creating orthonormal bases, known as the Gram-Schmidt method: ~ ~ Theorem. Suppose that V is a k-dimensional space with a basis B = fb1;::: bkg. The following process (called the Gram-Schmidt process) creates an orthonormal basis for V : 1. First, create the following vectors f ~v1; : : : ~vkg: • ~u1 = b~1: • ~u2 = b~2 − proj(b~2 onto ~u1). • ~u3 = b~3 − proj(b~3 onto ~u1) − proj(b~3 onto ~u2). • ~u4 = b~4 − proj(b~4 onto ~u1) − proj(b~4 onto ~u2) − proj(b~4 onto ~u3). ~ ~ ~ • ~uk = bk − proj(bk onto ~u1) − ::: − proj(bk onto k~u −1). 2. Now, take each of the vectors ~ui, and rescale them so that they are unit length: i.e. ~ui redefine each ~ui as the rescaled vector . jj ~uijj In this class, I want to talk about a useful application of the Gram-Schmidt method: the QR decomposition! We define this here: 1 The QR decomposition Definition. We say that an n×n matrix Q is orthogonal if its columns form an orthonor- n mal basis for R . As a side note: we've studied these matrices before! In class on Friday, we proved that for any such matrix, the relation QT · Q = I held; to see this, we just looked at the (i; j)-th entry of the product QT · Q, which by definition was the i-th row of QT dotted with the j-th column of A.
    [Show full text]
  • Numerical Linear Algebra Revised February 15, 2010 4.1 the LU
    Numerical Linear Algebra Revised February 15, 2010 4.1 The LU Decomposition The Elementary Matrices and badgauss In the previous chapters we talked a bit about the solving systems of the form Lx = b and Ux = b where L is lower triangular and U is upper triangular. In the exercises for Chapter 2 you were asked to write a program x=lusolver(L,U,b) which solves LUx = b using forsub and backsub. We now address the problem of representing a matrix A as a product of a lower triangular L and an upper triangular U: Recall our old friend badgauss. function B=badgauss(A) m=size(A,1); B=A; for i=1:m-1 for j=i+1:m a=-B(j,i)/B(i,i); B(j,:)=a*B(i,:)+B(j,:); end end The heart of badgauss is the elementary row operation of type 3: B(j,:)=a*B(i,:)+B(j,:); where a=-B(j,i)/B(i,i); Note also that the index j is greater than i since the loop is for j=i+1:m As we know from linear algebra, an elementary opreration of type 3 can be viewed as matrix multipli- cation EA where E is an elementary matrix of type 3. E looks just like the identity matrix, except that E(j; i) = a where j; i and a are as in the MATLAB code above. In particular, E is a lower triangular matrix, moreover the entries on the diagonal are 1's. We call such a matrix unit lower triangular.
    [Show full text]
  • Communication-Optimal Parallel and Sequential QR and LU Factorizations
    Communication-optimal parallel and sequential QR and LU factorizations James Demmel Laura Grigori Mark Frederick Hoemmen Julien Langou Electrical Engineering and Computer Sciences University of California at Berkeley Technical Report No. UCB/EECS-2008-89 http://www.eecs.berkeley.edu/Pubs/TechRpts/2008/EECS-2008-89.html August 4, 2008 Copyright © 2008, by the author(s). All rights reserved. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission. Communication-optimal parallel and sequential QR and LU factorizations James Demmel, Laura Grigori, Mark Hoemmen, and Julien Langou August 4, 2008 Abstract We present parallel and sequential dense QR factorization algorithms that are both optimal (up to polylogarithmic factors) in the amount of communication they perform, and just as stable as Householder QR. Our first algorithm, Tall Skinny QR (TSQR), factors m × n matrices in a one-dimensional (1-D) block cyclic row layout, and is optimized for m n. Our second algorithm, CAQR (Communication-Avoiding QR), factors general rectangular matrices distributed in a two-dimensional block cyclic layout. It invokes TSQR for each block column factorization. The new algorithms are superior in both theory and practice. We have extended known lower bounds on communication for sequential and parallel matrix multiplication to provide latency lower bounds, and show these bounds apply to the LU and QR decompositions.
    [Show full text]
  • Massively Parallel Poisson and QR Factorization Solvers
    Computers Math. Applic. Vol. 31, No. 4/5, pp. 19-26, 1996 Pergamon Copyright~)1996 Elsevier Science Ltd Printed in Great Britain. All rights reserved 0898-1221/96 $15.00 + 0.00 0898-122 ! (95)00212-X Massively Parallel Poisson and QR Factorization Solvers M. LUCK£ Institute for Control Theory and Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrluck@savba, sk M. VAJTERSIC Institute of Informatics, Slovak Academy of Sciences DdbravskA cesta 9, 840 00 Bratislava, P.O. Box 56, Slovak Republic kaifmava©savba, sk E. VIKTORINOVA Institute for Control Theoryand Robotics, Slovak Academy of Sciences DdbravskA cesta 9, 842 37 Bratislava, Slovak Republik utrrevka@savba, sk Abstract--The paper brings a massively parallel Poisson solver for rectangle domain and parallel algorithms for computation of QR factorization of a dense matrix A by means of Householder re- flections and Givens rotations. The computer model under consideration is a SIMD mesh-connected toroidal n x n processor array. The Dirichlet problem is replaced by its finite-difference analog on an M x N (M + 1, N are powers of two) grid. The algorithm is composed of parallel fast sine transform and cyclic odd-even reduction blocks and runs in a fully parallel fashion. Its computational complexity is O(MN log L/n2), where L = max(M + 1, N). A parallel proposal of QI~ factorization by the Householder method zeros all subdiagonal elements in each column and updates all elements of the given submatrix in parallel. For the second method with Givens rotations, the parallel scheme of the Sameh and Kuck was chosen where the disjoint rotations can be computed simultaneously.
    [Show full text]
  • A Fast and Accurate Matrix Completion Method Based on QR Decomposition and L 2,1-Norm Minimization Qing Liu, Franck Davoine, Jian Yang, Ying Cui, Jin Zhong, Fei Han
    A Fast and Accurate Matrix Completion Method based on QR Decomposition and L 2,1-Norm Minimization Qing Liu, Franck Davoine, Jian Yang, Ying Cui, Jin Zhong, Fei Han To cite this version: Qing Liu, Franck Davoine, Jian Yang, Ying Cui, Jin Zhong, et al.. A Fast and Accu- rate Matrix Completion Method based on QR Decomposition and L 2,1-Norm Minimization. IEEE Transactions on Neural Networks and Learning Systems, IEEE, 2019, 30 (3), pp.803-817. 10.1109/TNNLS.2018.2851957. hal-01927616 HAL Id: hal-01927616 https://hal.archives-ouvertes.fr/hal-01927616 Submitted on 20 Nov 2018 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Page 1 of 20 1 1 2 3 A Fast and Accurate Matrix Completion Method 4 5 based on QR Decomposition and L -Norm 6 2,1 7 8 Minimization 9 Qing Liu, Franck Davoine, Jian Yang, Member, IEEE, Ying Cui, Zhong Jin, and Fei Han 10 11 12 13 14 Abstract—Low-rank matrix completion aims to recover ma- I. INTRODUCTION 15 trices with missing entries and has attracted considerable at- tention from machine learning researchers.
    [Show full text]
  • Notes on Orthogonal Projections, Operators, QR Factorizations, Etc. These Notes Are Largely Based on Parts of Trefethen and Bau’S Numerical Linear Algebra
    Notes on orthogonal projections, operators, QR factorizations, etc. These notes are largely based on parts of Trefethen and Bau’s Numerical Linear Algebra. Please notify me of any typos as soon as possible. 1. Basic lemmas and definitions Throughout these notes, V is a n-dimensional vector space over C with a fixed Her- mitian product. Unless otherwise stated, it is assumed that bases are chosen so that the Hermitian product is given by the conjugate transpose, i.e. that (x, y) = y∗x where y∗ is the conjugate transpose of y. Recall that a projection is a linear operator P : V → V such that P 2 = P . Its complementary projection is I − P . In class we proved the elementary result that: Lemma 1. Range(P ) = Ker(I − P ), and Range(I − P ) = Ker(P ). Another fact worth noting is that Range(P ) ∩ Ker(P ) = {0} so the vector space V = Range(P ) ⊕ Ker(P ). Definition 1. A projection is orthogonal if Range(P ) is orthogonal to Ker(P ). Since it is convenient to use orthonormal bases, but adapt them to different situations, we often need to change basis in an orthogonality-preserving way. I.e. we need the following type of operator: Definition 2. A nonsingular matrix Q is called an unitary matrix if (x, y) = (Qx, Qy) for every x, y ∈ V . If Q is real, it is called an orthogonal matrix. An orthogonal matrix Q can be thought of as an orthogonal change of basis matrix, in which each column is a new basis vector. Lemma 2.
    [Show full text]
  • GRAM SCHMIDT and QR FACTORIZATION Math 21B, O. Knill
    GRAM SCHMIDT AND QR FACTORIZATION Math 21b, O. Knill GRAM-SCHMIDT PROCESS. Let ~v1; : : : ;~vn be a basis in V . Let ~w1 = ~v1 and ~u1 = ~w1=jj~w1jj. The Gram-Schmidt process recursively constructs from the already constructed orthonormal set ~u1; : : : ; ~ui−1 which spans a linear space Vi−1 the new vector ~wi = (~vi − projVi−1 (~vi)) which is orthogonal to Vi−1, and then normalizes ~wi to get ~ui = ~wi=jj~wijj. Each vector ~wi is orthonormal to the linear space Vi−1. The vectors f~u1; : : : ; ~un g form then an orthonormal basis in V . EXAMPLE. 2 2 3 2 1 3 2 1 3 Find an orthonormal basis for ~v1 = 4 0 5, ~v2 = 4 3 5 and ~v3 = 4 2 5. 0 0 5 SOLUTION. 2 1 3 1. ~u1 = ~v1=jj~v1jj = 4 0 5. 0 2 0 3 2 0 3 3 1 2. ~w2 = (~v2 − projV1 (~v2)) = ~v2 − (~u1 · ~v2)~u1 = 4 5. ~u2 = ~w2=jj~w2jj = 4 5. 0 0 2 0 3 2 0 3 0 0 3. ~w3 = (~v3 − projV2 (~v3)) = ~v3 − (~u1 · ~v3)~u1 − (~u2 · ~v3)~u2 = 4 5, ~u3 = ~w3=jj~w3jj = 4 5. 5 1 QR FACTORIZATION. The formulas can be written as ~v1 = jj~v1jj~u1 = r11~u1 ··· ~vi = (~u1 · ~vi)~u1 + ··· + (~ui−1 · ~vi)~ui−1 + jj~wijj~ui = r1i~u1 + ··· + rii~ui ··· ~vn = (~u1 · ~vn)~u1 + ··· + (~un−1 · ~vn)~un−1 + jj~wnjj~un = r1n~u1 + ··· + rnn~un which means in matrix form 2 3 2 3 2 3 j j · j j j · j r11 r12 · r1n A = 4 ~v1 ···· ~vn 5 = 4 ~u1 ···· ~un 5 4 0 r22 · r2n 5 = QR ; j j · j j j · j 0 0 · rnn where A and Q are m × n matrices and R is a n × n matrix which has rij = ~vj · ~ui, for i < j and vii = j~vij THE GRAM-SCHMIDT PROCESS PROVES: Any matrix A with linearly independent columns ~vi can be decomposed as A = QR, where Q has orthonormal column vectors and where R is an upper triangular square matrix with the same number of columns than A.
    [Show full text]
  • How Unique Is QR? Full Rank, M = N
    LECTUREAPPENDIX · purdue university cs 51500 Kyle Kloster matrix computations October 12, 2014 How unique is QR? Full rank, m = n In class we looked at the special case of full rank, n × n matrices, and showed that the QR decomposition is unique up to a factor of a diagonal matrix with entries ±1. Here we'll see that the other full rank cases follow the m = n case somewhat closely. Any full rank QR decomposition involves a square, upper- triangular partition R within the larger (possibly rectangular) m × n matrix. The gist of these uniqueness theorems is that R is unique, up to multiplication by a diagonal matrix of ±1s; the extent to which the orthogonal matrix is unique depends on its dimensions. Theorem (m = n) If A = Q1R1 = Q2R2 are two QR decompositions of full rank, square A, then Q2 = Q1S R2 = SR1 for some square diagonal S with entries ±1. If we require the diagonal entries of R to be positive, then the decomposition is unique. Theorem (m < n) If A = Q1 R1 N 1 = Q2 R2 N 2 are two QR decom- positions of a full rank, m × n matrix A with m < n, then Q2 = Q1S; R2 = SR1; and N 2 = SN 1 for square diagonal S with entries ±1. If we require the diagonal entries of R to be positive, then the decomposition is unique. R R Theorem (m > n) If A = Q U 1 = Q U 2 are two QR 1 1 0 2 2 0 decompositions of a full rank, m × n matrix A with m > n, then Q2 = Q1S; R2 = SR1; and U 2 = U 1T for square diagonal S with entries ±1, and square orthogonal T .
    [Show full text]
  • QR Factorization for the CELL Processor 1 Introduction
    QR Factorization for the CELL Processor { LAPACK Working Note 201 Jakub Kurzak Department of Electrical Engineering and Computer Science, University of Tennessee Jack Dongarra Department of Electrical Engineering and Computer Science, University of Tennessee Computer Science and Mathematics Division, Oak Ridge National Laboratory School of Mathematics & School of Computer Science, University of Manchester ABSTRACT ScaLAPACK [4] are good examples. These imple- The QR factorization is one of the most mentations work on square or rectangular submatri- important operations in dense linear alge- ces in their inner loops, where operations are encap- bra, offering a numerically stable method for sulated in calls to Basic Linear Algebra Subroutines solving linear systems of equations includ- (BLAS) [5], with emphasis on expressing the compu- ing overdetermined and underdetermined sys- tation as level 3 BLAS (matrix-matrix type) opera- tems. Classic implementation of the QR fac- tions. torization suffers from performance limita- The fork-and-join parallelization model of these li- tions due to the use of matrix-vector type op- braries has been identified as the main obstacle for erations in the phase of panel factorization. achieving scalable performance on new processor ar- These limitations can be remedied by using chitectures. The arrival of multi-core chips increased the idea of updating of QR factorization, ren- the demand for new algorithms, exposing much more dering an algorithm, which is much more scal- thread-level parallelism of much finer granularity. able and much more suitable for implementa- This paper presents an implementation of the QR tion on a multi-core processor. It is demon- factorization based on the idea of updating the QR strated how the potential of the CELL proces- factorization.
    [Show full text]