DOCSLIB.ORG

A Block Orthogonalization Procedure with Constant Synchronization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Block Orthogonalization Procedure with Constant Synchronization A BLOCK ORTHOGONALIZATION PROCEDURE WITH CONSTANT SYNCHRONIZATION REQUIREMENTS ࣿ y ANDREAS STATHOPOULOS AND KESHENG WU Abstract. We prop ose an alternative orthonormalization metho d that computes the orthonor- mal basis from the right singular vectors of a matrix. Its advantage are: aङ all op erations are matrix-matrix multiplications and thus cache-ecient, bङ only one synchronization p oint is required in parallel implementations, cङ could b e more stable than Gram-Schmidt. In addition, we consider the problem of incremental orthonormalization where a blo ckofvectors is orthonormalized against a previously orthonormal set of vectors and among itself. We solve this problem by alternating itera- tively b etween a phase of Gram-Schmidt and a phase of the new metho d. We provide error analysis and use it to derive b ounds on how accurately the two successive orthonormalization phases should b e p erformed to minimize total work p erformed. Our exp eriments con rm the favorable numerical b ehavior of the new metho d and its e ectiveness on mo dern parallel computers. Key words. Gram-Schmidt, orthogonalization, Householder, QR factorization, singular value decomp osition, Poincare AMS Sub ject Classi cation. 65F15 1. Intro duction. Computing an orthonormal basis from a given set of vectors is a basic computation, common in most scienti c applications. Often, it is also one of the most computationally demanding pro cedures b ecause the vectors are of large dimension, and b ecause the computation scales as the square of the number of vectors involved. Further, among several orthonormalization techniques the ones that ensure high accuracy are the more exp ensive ones. In many applications, orthonormalization o ccurs in an incremental fashion, where a new set of vectors घwe call this internal setङ is orthogonalized against a previously orthonormal set of vectors घwe call this externalङ, and then among themselves. This computation is typical in blo ck Krylov metho ds, where the Krylov basis is expanded by a blo ckofvectors [12, 11]. It is also typical when certain external orthogonalization constraints have to be applied to the vectors of an iterative metho d. Lo cking of converged eigenvectors in eigenvalue iterative metho ds is such an example [19,22]. This problem di ers from the classical QR factorization in that the external set of vectors should not be mo di ed. Therefore, atwo phase pro cess is required; rst orthogonalizing the internal vectors against the external, and second the internal among themselves. Usually, the numb er of the internal vectors is much smaller than the external ones, and signi cantly smaller than their dimension. Another imp ortant di erence is that the accuracy of the R matrix of the QR factorization is not of pri- mary interest, but rather the orthonormalityof the pro duced vectors Q. A variety of orthogonalization techniques exist for b oth phases. For the external phase, Gram- Schmidt घGSङ and its mo di ed version घMGSङ are the most comp etitivechoices. For the internal phase, QR factorization using Householder transformations is the most stable, alb eit more exp ensive metho d [11]. When the number of vectors is signi - cantly smaller than their dimension, MGS or GS with reorthogonalization are usually preferred. Computationally, MGS, GS and Housholder transformations are based on level 1 or level 2 BLAS op erations [15,9,8]. These basic kernel computations, dot pro d- ucts, vector up dates and sometimes matrix-vector op erations, cannot fully utilize the ࣿ Department of Computer Science, College of William and Mary, Williamsburg, Virginia 23187- 8795, घ[email protected]ङ. y NERSC, Lawrence Berkeley National Lab oratory, Berkeley, California 94720, घ[email protected]ङ. 1 pro cessor cache, with signi cant p erformance shortcomings on most of to days com- puters. In addition, these metho ds have high synchronization requirements when implemented on parallel computers. The numb er of synchronization p oints of the GS is linear to the number of vectors, while for the rest metho ds it is quadratic [14]. When the parallel platform involves a high latency network but fast pro cessors, as in the case of the increasingly p opular clusters of workstations [21], the number of synchronization p oints can haveanadverse e ect in the scalability of the metho d. Most of the previous e orts to address these problems considered blo cks of vectors, and used hybrids of the more scalable GS for orthogonalization b etween blo cks, and the more accurate MGS for orthogonalization within blo cks [14, 2]. By adjusting the blo ck size appropriately, these techniques can pro duce accurate orthonormal sets while reducing the synchronization requirements of MGS. However, the number of synchronization p oints is still linear to the number of vectors, and BLAS level 2 kernels are still dominant despite blo cking. Finally, most of such e orts have fo cused on a full QR factorization of a set of vectors, rather than the two phase problem. In this pap er, weintro duce a metho d based on the singular value decomp osition घSVDङ, that uses the right singular vectors to pro duce an orthonormal basis for a given subspace. The idea itself is not new, dating back at least to Poincare, and it is sometimes encountered in chemistry and wavelet literature [5, 20,16,17, 6]. How- ever, it has not received any attention as a computationally viable orthogonalization metho d, and to our knowledge there is no analysis of its numerical prop erties. Beyond theoretical considerations, the metho d, whichwe call SVQB, has some very attractive characteristics. It uses primarily level 3 BLAS kernels, and it has a constantnumber of synchronization p oints. As we show in this pap er, it is not as numerically accurate as MGS, but it is often b etter than GS. More interestingly,we show that more stable alternatives, such as MGS or Householder, are an overkill for our two phase problem. Coupling the SVQB metho d for the internal phase with a blo ck GS with reorthogonal- ization for the external phase results in a metho d also with constant synchronization requirements. The pap er is organized as follows. We rst describ e the basic idea of the SVQB metho d, and we couple it with a blo ck GS for the two phase problem. The numerical stability of the metho d is analyzed in the following section, and some numerical exp er- iments and comparisons with other metho ds verify its e ectiveness. We then analyze the numerical interaction b etween the metho ds used in each of the two phases, and based on this theory we tune the two phases so that they do not p erform unneces- sary reorthogonalizations. Following, we present timings from a series of exp eriments on the Cray T3E, and on the IBM SP-2. These verify that b oth the blo ck computa- tions and the small numb er of synchronizations help the new metho d achieve accurate orthogonality, faster than other comp etitive metho ds. nࣾk 2. Orthogonalization metho ds. Let V 2< b e a set of orthonormal vec- nࣾm tors, and W 2< b e a set of vectors, where k + m ऊ n. In practice, we usually exp ect m < k and m n. The problem is to obtain an orthonormal set of vec- tors Q, such that spanघQङ=spanघW ङ and Q ? V . This problem can be viewed as consisting of two phases: obtaining a Q which is orthogonal to V घexternal phaseङ, and orthonormalizing the Q vectors among themselves घinternal phaseङ. The internal phase problem is most commonly solved through various metho ds for QR factoriza- tion [11,23]. Gram-Schmidt घGSङ is the most well known and also the least computationally exp ensive metho d for QR factorization. In addition, GS is based primarily on level 2 2 BLAS kernels, and it is p ossible to implement it in parallel with a mo dest num- ber of m + 1 synchronization p oints. Although computationally attractive, GS do es not always pro duce numerically orthogonal vectors [2]. However, when used with reorthogonalization, a second GS iteration is typically enough for pro ducing orthog- onality to machine precision [7]. Since a second iteration is not always necessary,GS with reorthogonalization is often preferred over other metho ds. Householder re ections, on the other hand, yield a QR factorization that is nu- merically accurate. By construction, the resulting matrix Q is orthogonal to machine precision घङ, and the computed upp er triangular matrix R di ers from the exact one by less than kAk [11]. Despite its excellentnumerical characteristics, this technique involves twice the arithmetic of the GS. For a slight increase in arithmetic, a blo ck version is also p ossible that facilitates the use of level 3 BLAS kernels. However, when m n, this blo cking is harder to obtain and it is less e ective. Moreover, the number of synchronization p oints it requires grows quadratically with m. Mo di ed Gram-Schmidt घMGSङ improves the numerical stability of the GS by orthogonalizing individual pairs of vectors rather than a vector against a blo ck. For MGS, the error in the orthogonalityof Q can b e b ounded by ऊघW ङ, where ऊघW ङis the condition number of W [1, 3]. The op eration count for MGS is the same with GS, but working on individual vectors allows only for level 1 BLAS kernels, which impairs signi cantly the actual p erformance on cache based computers. Finally, the number of synchronization p oints grows quadratically with m. In the context of the two phase problem, pro ducing an internal set of vectors Q with orthogonality close to machine precision is unnecessary, b ecause of the interde- p endence of the phases.
Load more

Recommended publications
  • 21. Orthonormal Bases
    21. Orthonormal Bases The canonical/standard basis 011 001 001 B C B C B C B0C B1C B0C e1 = B.C ; e2 = B.C ; : : : ; en = B.C B.C B.C B.C @.A @.A @.A 0 0 1 has many useful properties. • Each of the standard basis vectors has unit length: q p T jjeijj = ei ei = ei ei = 1: • The standard basis vectors are orthogonal (in other words, at right angles or perpendicular). T ei ej = ei ej = 0 when i 6= j This is summarized by ( 1 i = j eT e = δ = ; i j ij 0 i 6= j where δij is the Kronecker delta. Notice that the Kronecker delta gives the entries of the identity matrix. Given column vectors v and w, we have seen that the dot product v w is the same as the matrix multiplication vT w. This is the inner product on n T R . We can also form the outer product vw , which gives a square matrix. 1 The outer product on the standard basis vectors is interesting. Set T Π1 = e1e1 011 B C B0C = B.C 1 0 ::: 0 B.C @.A 0 01 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 0 . T Πn = enen 001 B C B0C = B.C 0 0 ::: 1 B.C @.A 1 00 0 ::: 01 B C B0 0 ::: 0C = B. .C B. .C @. .A 0 0 ::: 1 In short, Πi is the diagonal square matrix with a 1 in the ith diagonal position and zeros everywhere else.
    [Show full text]
  • Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases
    Week 11 Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases 11.1 Opening Remarks 11.1.1 Low Rank Approximation * View at edX 463 Week 11. Orthogonal Projection, Low Rank Approximation, and Orthogonal Bases 464 11.1.2 Outline 11.1. Opening Remarks..................................... 463 11.1.1. Low Rank Approximation............................. 463 11.1.2. Outline....................................... 464 11.1.3. What You Will Learn................................ 465 11.2. Projecting a Vector onto a Subspace........................... 466 11.2.1. Component in the Direction of ............................ 466 11.2.2. An Application: Rank-1 Approximation...................... 470 11.2.3. Projection onto a Subspace............................. 474 11.2.4. An Application: Rank-2 Approximation...................... 476 11.2.5. An Application: Rank-k Approximation...................... 478 11.3. Orthonormal Bases.................................... 481 11.3.1. The Unit Basis Vectors, Again........................... 481 11.3.2. Orthonormal Vectors................................ 482 11.3.3. Orthogonal Bases.................................. 485 11.3.4. Orthogonal Bases (Alternative Explanation).................... 488 11.3.5. The QR Factorization................................ 492 11.3.6. Solving the Linear Least-Squares Problem via QR Factorization......... 493 11.3.7. The QR Factorization (Again)........................... 494 11.4. Change of Basis...................................... 498 11.4.1. The Unit Basis Vectors,
    [Show full text]
  • Lecture 4: April 8, 2021 1 Orthogonality and Orthonormality
    Mathematical Toolkit Spring 2021 Lecture 4: April 8, 2021 Lecturer: Avrim Blum (notes based on notes from Madhur Tulsiani) 1 Orthogonality and orthonormality Definition 1.1 Two vectors u, v in an inner product space are said to be orthogonal if hu, vi = 0. A set of vectors S ⊆ V is said to consist of mutually orthogonal vectors if hu, vi = 0 for all u 6= v, u, v 2 S. A set of S ⊆ V is said to be orthonormal if hu, vi = 0 for all u 6= v, u, v 2 S and kuk = 1 for all u 2 S. Proposition 1.2 A set S ⊆ V n f0V g consisting of mutually orthogonal vectors is linearly inde- pendent. Proposition 1.3 (Gram-Schmidt orthogonalization) Given a finite set fv1,..., vng of linearly independent vectors, there exists a set of orthonormal vectors fw1,..., wng such that Span (fw1,..., wng) = Span (fv1,..., vng) . Proof: By induction. The case with one vector is trivial. Given the statement for k vectors and orthonormal fw1,..., wkg such that Span (fw1,..., wkg) = Span (fv1,..., vkg) , define k u + u = v − hw , v i · w and w = k 1 . k+1 k+1 ∑ i k+1 i k+1 k k i=1 uk+1 We can now check that the set fw1,..., wk+1g satisfies the required conditions. Unit length is clear, so let’s check orthogonality: k uk+1, wj = vk+1, wj − ∑ hwi, vk+1i · wi, wj = vk+1, wj − wj, vk+1 = 0. i=1 Corollary 1.4 Every finite dimensional inner product space has an orthonormal basis.
    [Show full text]
  • Orthonormality and the Gram-Schmidt Process
    Unit 4, Section 2: The Gram-Schmidt Process Orthonormality and the Gram-Schmidt Process The basis (e1; e2; : : : ; en) n n for R (or C ) is considered the standard basis for the space because of its geometric properties under the standard inner product: 1. jjeijj = 1 for all i, and 2. ei, ej are orthogonal whenever i 6= j. With this idea in mind, we record the following definitions: Definitions 6.23/6.25. Let V be an inner product space. • A list of vectors in V is called orthonormal if each vector in the list has norm 1, and if each pair of distinct vectors is orthogonal. • A basis for V is called an orthonormal basis if the basis is an orthonormal list. Remark. If a list (v1; : : : ; vn) is orthonormal, then ( 0 if i 6= j hvi; vji = 1 if i = j: Example. The list (e1; e2; : : : ; en) n n forms an orthonormal basis for R =C under the standard inner products on those spaces. 2 Example. The standard basis for Mn(C) consists of n matrices eij, 1 ≤ i; j ≤ n, where eij is the n × n matrix with a 1 in the ij entry and 0s elsewhere. Under the standard inner product on Mn(C) this is an orthonormal basis for Mn(C): 1. heij; eiji: ∗ heij; eiji = tr (eijeij) = tr (ejieij) = tr (ejj) = 1: 1 Unit 4, Section 2: The Gram-Schmidt Process 2. heij; ekli, k 6= i or j 6= l: ∗ heij; ekli = tr (ekleij) = tr (elkeij) = tr (0) if k 6= i, or tr (elj) if k = i but l 6= j = 0: So every vector in the list has norm 1, and every distinct pair of vectors is orthogonal.
    [Show full text]
  • Inner Product Spaces
    CHAPTER 6 Woman teaching geometry, from a fourteenth-century edition of Euclid’s geometry book. Inner Product Spaces In making the definition of a vector space, we generalized the linear structure (addition and scalar multiplication) of R2 and R3. We ignored other important features, such as the notions of length and angle. These ideas are embedded in the concept we now investigate, inner products. Our standing assumptions are as follows: 6.1 Notation F, V F denotes R or C. V denotes a vector space over F. LEARNING OBJECTIVES FOR THIS CHAPTER Cauchy–Schwarz Inequality Gram–Schmidt Procedure linear functionals on inner product spaces calculating minimum distance to a subspace Linear Algebra Done Right, third edition, by Sheldon Axler 164 CHAPTER 6 Inner Product Spaces 6.A Inner Products and Norms Inner Products To motivate the concept of inner prod- 2 3 x1 , x 2 uct, think of vectors in R and R as x arrows with initial point at the origin. x R2 R3 H L The length of a vector in or is called the norm of x, denoted x . 2 k k Thus for x .x1; x2/ R , we have The length of this vector x is p D2 2 2 x x1 x2 . p 2 2 x1 x2 . k k D C 3 C Similarly, if x .x1; x2; x3/ R , p 2D 2 2 2 then x x1 x2 x3 . k k D C C Even though we cannot draw pictures in higher dimensions, the gener- n n alization to R is obvious: we define the norm of x .x1; : : : ; xn/ R D 2 by p 2 2 x x1 xn : k k D C C The norm is not linear on Rn.
    [Show full text]
  • Geometric Algebra Techniques for General Relativity
    Geometric Algebra Techniques for General Relativity Matthew R. Francis∗ and Arthur Kosowsky† Dept. of Physics and Astronomy, Rutgers University 136 Frelinghuysen Road, Piscataway, NJ 08854 (Dated: February 4, 2008) Geometric (Clifford) algebra provides an efficient mathematical language for describing physical problems. We formulate general relativity in this language. The resulting formalism combines the efficiency of differential forms with the straightforwardness of coordinate methods. We focus our attention on orthonormal frames and the associated connection bivector, using them to find the Schwarzschild and Kerr solutions, along with a detailed exposition of the Petrov types for the Weyl tensor. PACS numbers: 02.40.-k; 04.20.Cv Keywords: General relativity; Clifford algebras; solution techniques I. INTRODUCTION Geometric (or Clifford) algebra provides a simple and natural language for describing geometric concepts, a point which has been argued persuasively by Hestenes [1] and Lounesto [2] among many others. Geometric algebra (GA) unifies many other mathematical formalisms describing specific aspects of geometry, including complex variables, matrix algebra, projective geometry, and differential geometry. Gravitation, which is usually viewed as a geometric theory, is a natural candidate for translation into the language of geometric algebra. This has been done for some aspects of gravitational theory; notably, Hestenes and Sobczyk have shown how geometric algebra greatly simplifies certain calculations involving the curvature tensor and provides techniques for classifying the Weyl tensor [3, 4]. Lasenby, Doran, and Gull [5] have also discussed gravitation using geometric algebra via a reformulation in terms of a gauge principle. In this paper, we formulate standard general relativity in terms of geometric algebra. A comprehensive overview like the one presented here has not previously appeared in the literature, although unpublished works of Hestenes and of Doran take significant steps in this direction.
    [Show full text]
  • Inner Product Spaces Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (March 2, 2007)
    MAT067 University of California, Davis Winter 2007 Inner Product Spaces Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (March 2, 2007) The abstract definition of vector spaces only takes into account algebraic properties for the addition and scalar multiplication of vectors. For vectors in Rn, for example, we also have geometric intuition which involves the length of vectors or angles between vectors. In this section we discuss inner product spaces, which are vector spaces with an inner product defined on them, which allow us to introduce the notion of length (or norm) of vectors and concepts such as orthogonality. 1 Inner product In this section V is a finite-dimensional, nonzero vector space over F. Definition 1. An inner product on V is a map ·, · : V × V → F (u, v) →u, v with the following properties: 1. Linearity in first slot: u + v, w = u, w + v, w for all u, v, w ∈ V and au, v = au, v; 2. Positivity: v, v≥0 for all v ∈ V ; 3. Positive definiteness: v, v =0ifandonlyifv =0; 4. Conjugate symmetry: u, v = v, u for all u, v ∈ V . Remark 1. Recall that every real number x ∈ R equals its complex conjugate. Hence for real vector spaces the condition about conjugate symmetry becomes symmetry. Definition 2. An inner product space is a vector space over F together with an inner product ·, ·. Copyright c 2007 by the authors. These lecture notes may be reproduced in their entirety for non- commercial purposes. 2NORMS 2 Example 1. V = Fn n u =(u1,...,un),v =(v1,...,vn) ∈ F Then n u, v = uivi.
    [Show full text]
  • Basics of Euclidean Geometry
    This is page 162 Printer: Opaque this 6 Basics of Euclidean Geometry Rien n'est beau que le vrai. |Hermann Minkowski 6.1 Inner Products, Euclidean Spaces In a±ne geometry it is possible to deal with ratios of vectors and barycen- ters of points, but there is no way to express the notion of length of a line segment or to talk about orthogonality of vectors. A Euclidean structure allows us to deal with metric notions such as orthogonality and length (or distance). This chapter and the next two cover the bare bones of Euclidean ge- ometry. One of our main goals is to give the basic properties of the transformations that preserve the Euclidean structure, rotations and re- ections, since they play an important role in practice. As a±ne geometry is the study of properties invariant under bijective a±ne maps and projec- tive geometry is the study of properties invariant under bijective projective maps, Euclidean geometry is the study of properties invariant under certain a±ne maps called rigid motions. Rigid motions are the maps that preserve the distance between points. Such maps are, in fact, a±ne and bijective (at least in the ¯nite{dimensional case; see Lemma 7.4.3). They form a group Is(n) of a±ne maps whose corresponding linear maps form the group O(n) of orthogonal transformations. The subgroup SE(n) of Is(n) corresponds to the orientation{preserving rigid motions, and there is a corresponding 6.1. Inner Products, Euclidean Spaces 163 subgroup SO(n) of O(n), the group of rotations.
    [Show full text]
  • INNER PRODUCTS on N-INNER PRODUCT SPACES
    SOOCHOW JOURNAL OF MATHEMATICS Volume 28, No. 4, pp. 389-398, October 2002 INNER PRODUCTS ON n-INNER PRODUCT SPACES BY HENDRA GUNAWAN Abstract. In this note, we show that in any n-inner product space with n ≥ 2 we can explicitly derive an inner product or, more generally, an (n − k)-inner product from the n-inner product, for each k 2 f1; : : : ; n − 1g. We also present some related results on n-normed spaces. 1. Introduction Let n be a nonnegative integer and X be a real vector space of dimension d ≥ n (d may be infinite). A real-valued function h·; ·|·; : : : ; ·i on X n+1 satisfying the following five properties: (I1) hx1; x1jx2; : : : ; xni ≥ 0; hx1; x1jx2; : : : ; xni = 0 if and only if x1; x2; : : : ; xn are linearly dependent; (I2) hx1; x1jx2; : : : ; xni = hxi1 ; xi1 jxi2 ; : : : ; xin i for every permutation (i1; : : : ; in) of (1; : : : ; n); (I3) hx; yjx2; : : : ; xni = hy; xjx2; : : : ; xni; (I4) hαx, yjx2; : : : ; xni = αhx; yjx2; : : : ; xni; α 2 R; 0 0 (I5) hx + x ; yjx2; : : : ; xni = hx; yjx2; : : : ; xni + hx ; yjx2; : : : ; xni; is called an n-inner product on X, and the pair (X; h·; ·|·; : : : ; ·i) is called an n-inner product space. For n = 1, the expression hx; yjx2; : : : ; xni is to be understood as hx; yi, which denotes nothing but an inner product on X. The concept of 2-inner product spaces was first introduced by Diminnie, G¨ahler and White [2, 3, 7] in 1970's, Received March 26, 2001; revised August 13, 2002. AMS Subject Classification. 46C50, 46B20, 46B99, 46A19.
    [Show full text]
  • Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality
    MATH 532: Linear Algebra Chapter 5: Norms, Inner Products and Orthogonality Greg Fasshauer Department of Applied Mathematics Illinois Institute of Technology Spring 2015 [email protected] MATH 532 1 Outline 1 Vector Norms 2 Matrix Norms 3 Inner Product Spaces 4 Orthogonal Vectors 5 Gram–Schmidt Orthogonalization & QR Factorization 6 Unitary and Orthogonal Matrices 7 Orthogonal Reduction 8 Complementary Subspaces 9 Orthogonal Decomposition 10 Singular Value Decomposition 11 Orthogonal Projections [email protected] MATH 532 2 Vector Norms Vector[0] Norms 1 Vector Norms 2 Matrix Norms Definition 3 Inner Product Spaces Let x; y 2 Rn (Cn). Then 4 Orthogonal Vectors n T X 5 Gram–Schmidt Orthogonalization & QRx Factorizationy = xi yi 2 R i=1 6 Unitary and Orthogonal Matrices n X ∗ = ¯ 2 7 Orthogonal Reduction x y xi yi C i=1 8 Complementary Subspaces is called the standard inner product for Rn (Cn). 9 Orthogonal Decomposition 10 Singular Value Decomposition 11 Orthogonal Projections [email protected] MATH 532 4 Vector Norms Definition Let V be a vector space. A function k · k : V! R≥0 is called a norm provided for any x; y 2 V and α 2 R 1 kxk ≥ 0 and kxk = 0 if and only if x = 0, 2 kαxk = jαj kxk, 3 kx + yk ≤ kxk + kyk. Remark The inequality in (3) is known as the triangle inequality. [email protected] MATH 532 5 Vector Norms Remark Any inner product h·; ·i induces a norm via (more later) p kxk = hx; xi: We will show that the standard inner product induces the Euclidean norm (cf.
    [Show full text]
  • Projection and Its Importance in Scientific Computing ______
    Projection and its Importance in Scientific Computing ________________________________________________ Stan Tomov EECS Department The University of Tennessee, Knoxville March 22, 2017 COCS 594 Lecture Notes Slide 1 / 41 03/22/2017 Contact information office : Claxton 317 phone : (865) 974-6317 email : [email protected] Additional reference materials: [1] R.Barrett, M.Berry, T.F.Chan, J.Demmel, J.Donato, J. Dongarra, V. Eijkhout, R.Pozo, C.Romine, and H.Van der Vorst, Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods (2nd edition) http://netlib2.cs.utk.edu/linalg/html_templates/Templates.html [2] Yousef Saad, Iterative methods for sparse linear systems (1st edition) http://www-users.cs.umn.edu/~saad/books.html Slide 2 / 41 Topics as related to high-performance scientific computing‏ Projection in Scientific Computing Sparse matrices, PDEs, Numerical parallel implementations solution, Tools, etc. Iterative Methods Slide 3 / 41 Topics on new architectures – multicore, GPUs (CUDA & OpenCL), MIC Projection in Scientific Computing Sparse matrices, PDEs, Numerical parallel implementations solution, Tools, etc. Iterative Methods Slide 4 / 41 Outline Part I – Fundamentals Part II – Projection in Linear Algebra Part III – Projection in Functional Analysis (e.g. PDEs) HPC with Multicore and GPUs Slide 5 / 41 Part I Fundamentals Slide 6 / 41 Projection in Scientific Computing [ an example – in solvers for PDE discretizations ] A model leading to self-consistent iteration with need for high-performance diagonalization
    [Show full text]
  • In Defense of Orthonormality Constraints for Nonrigid Structure from Motion
    In Defense of Orthonormality Constraints for Nonrigid Structure from Motion Ijaz Akhter1 Yaser Sheikh2 Sohaib Khan1 [email protected] [email protected] [email protected] 1LUMS School of Science and Engineering, Lahore, Pakistan 2The Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, USA http://cvlab.lums.edu.pk Abstract using these constraints. The ambiguity theorem introduced by Xiao et al. has In factorization approaches to nonrigid structure from greatly influenced research on NRSFM. They proposed ad- motion, the 3D shape of a deforming object is usually mod- ditional constraints termed basis constraints to resolve this eled as a linear combination of a small number of basis ambiguity to get a unique linear solution. In addition, sev- shapes. The original approach to simultaneously estimate eral new constraints have also been proposed. For exam- the shape basis and nonrigid structure exploited orthonor- ple, Torresani et al. [8] proposed a Gaussian prior for the mality constraints for metric rectification. Recently, it has shape coefficients. Del Bue et al. [5] assumed that nonrigid been asserted that structure recovery through orthonormal- shape contained a significant number of points which be- ity constraints alone is inherently ambiguous and cannot re- have in a rigid fashion. Recently, Bartoli et al. [2] proposed sult in a unique solution. This assertion has been accepted that we can iteratively find the shape basis based on the as- as conventional wisdom and is the justification of many re- sumption that each basis shape should decrease the repro- medial heuristics in literature. Our key contribution is to jection error left due to the earlier basis.
    [Show full text]