Accurate and Efficient Sums and Dot Products in Julia Chris Elrod, François Févotte To cite this version: Chris Elrod, François Févotte. Accurate and Efficient Sums and Dot Products in Julia. 2019. hal- 02265534v2 HAL Id: hal-02265534 https://hal.archives-ouvertes.fr/hal-02265534v2 Preprint submitted on 14 Aug 2019 (v2), last revised 21 Aug 2019 (v3) HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Accurate and Efficient Sums and Dot Products in Julia Chris Elrod Franc¸ois Fevotte´ Eli Lilly TriScale innov Email: [email protected] Palaiseau, France Email: [email protected] Abstract—This paper presents an efficient, vectorized im- all cases. For programs that already use double precision (64- plementation of various summation and dot product algo- bit FP numbers), increasing the precision is not an option, rithms in the Julia programming language. These implemen- since quadruple precision (128-bit FP numbers) remains too tations are available under an open source license in the 1 AccurateArithmetic.jl Julia package. expensive for most purposes . On the other hand, hitting the Besides naive algorithms, compensated algorithms are im- memory wall means that many algorithms (and in particular plemented: the Kahan-Babuska-Neumaierˇ summation algorithm, BLAS1-like algorithms such as summation and dot product) and the Ogita-Rump-Oishi simply compensated summation and are limited by the memory bandwidth in a wide range of vector dot product algorithms. These algorithms effectively double the lengths on modern systems. In turn, this means that CPUs working precision, producing much more accurate results while incurring little to no overhead, especially for large input vectors. are idling for a fraction of the computing time, which could This paper also tries and builds upon this example to make instead be used to perform additional operations for free, as a case for a more widespread use of Julia in the HPC com- long as these operations do not involve additional memory munity. Although the vectorization of compensated algorithms accesses. Much work has therefore been devoted to computing is no particularly simple task, Julia makes it relatively easy sums and dot products in a way that is both fast and accurate, and straightforward. It also allows structuring the code in small, composable building blocks, closely matching textbook leading to efficient algorithms implemented in a variety of algorithms yet efficiently compiled. programming languages. Index Terms—Vectorization, Compensated Algorithms, Julia The work presented here comes from the observation2 that, programming language as of November 2018, no such algorithm was currently imple- mented in an efficient way in the Julia programming language. I. INTRODUCTION This paper describes an attempt made to implement accurate Computing the dot product of two vectors and, perhaps to a summation and dot product algorithms in Julia, in a way that is lesser extent, summing the elements of a vector, are two very as efficient as possible, and in particular makes use of SIMD common basic building blocks for more complex linear algebra vector units. All the code described here is available under algorithms. As such, any change in their performance is likely an open source license in the AccurateArithmetic.jl to affect the overall performance of scientific computing codes; package. This paper will also try to make the argument any change in their accuracy is likely to induce a loss of that writing such efficient implementations of compensated reproducibility in overall computed results. algorithms is a relatively easy and straightforward task thanks Such a lack of reproducibility is becoming more and more to the nice properties of the Julia language. common as the performance of supercomputers increase, due Part II of this paper takes the viewpoint of computational to a combination of multiple factors. First, an increase in sciences and discusses some aspects related to the use of computational power allows for ever more expensive models vector instructions in fast implementations of summation al- (such as, for example, finer grids for the discretization of gorithms. Part III then takes the point of view of floating- partial differential equations), which in turn generates larger point arithmetic, and reviews a subclass of accurate summation vectors, whose summation is often more ill-conditioned. Sec- and dot product algorithms: compensated algorithms. Nothing ond, the order in which operations are performed in modern particularly new is presented in these parts, except maybe systems is far from being deterministic and reproducible: some specific details about interactions between vectorization vectorization, parallelism, and compiler optimizations are only and compensation (paragraph III-C), that have to be dealt with a few examples of possible causes for unexpected changes in order to gain the full benefit of both worlds: speed and in the execution order of a given program. Since floating- accuracy. Part IV gives some insight about the Julia imple- point (FP) arithmetic is not associative, such non-determinism mentation of such compensated algorithms, and discusses the in execution order entails non-reproducibilities in computed performance obtained on some test systems. These results are results. For these reasons, accurate summation and dot product algorithms are sometimes useful. 1at least on x86-64 architectures, where quadruple precision is implemented However, increasing the algorithms accuracy at the price in software of an increase in their computing cost is not acceptable in 2https://discourse.julialang.org/t/floating-point-summation/17785 Algorithm 1 Scalar, naive summation Algorithm 2 Vectorized, naive summation 1: fInitialization:g 1: fInitialization:g 2: a 0 2: a 0 3: fLoop on vector elements:g 3: fLoop on full packs:g N 4: for i 2 1 : N do 4: for j 2 1 : W do 5: a a ⊕ xi 5: i W (j − 1) + 1 6: end for 6: p (xi; xi+1; xi+2; xi+3) 7: a a ⊕ p 7: return a 8: end for 9: fReduction of SIMD accumulator:g summarized in part V, along with some closing notes and 10: a vsum(a) outlooks. 11: fLoop on remaining elements:g II. THE COMPUTATIONAL SCIENCE POINT OF VIEW N 12: for j 2 W W + 1 : N do For the sake of simplicity, let us only discuss here the case 13: a a ⊕ xi of the summation algorithm: 14: end for N X 15: return a sum(x) = xi; i=1 N vsum where x 2 R . Since a dot product can also be expressed as to be reduced to a scalar: this is the purpose of the a summation, no loss of generality is entailed by this choice: function, which sums all W components of a SIMD register: N vsum(a) = a[1] ⊕ a[2] ⊕ ::: ⊕ a[W ]; X dot(x; y) = x y ; i i where a[i] represents the i-th element in a SIMD pack a. i=1 Although such an implementation is already much more where x 2 RN and y 2 RN . complex than its scalar version, it is not enough to obtain op- Algorithm 1 presents the most naive possible algorithm timal performance on modern hardware. Because of the loop- for a summation, in its scalar version. An accumulator a carried dependency of the accumulator and instruction latency stores partial sums. During each loop iteration, a new vector in superscalar hardware architectures, full performance can element xi is added to this accumulator. Note that this accu- only be obtained if multiple instructions can be running at the mulation is performed using the computer arithmetic addition same time. This means breaking that sequential dependency operation ⊕, which operates on FP numbers, rounds its result, between one iteration and the next; a traditional way to do that and is therefore only an approximation of the mathematical + consists in partially unrolling the main loop. An illustration operator which operates on real numbers. In the remainder of of this technique is given in Algorithm 3, where the loop is this paper, similar notations will be used for other operators, unrolled twice (U = 2). A consequence of this technique is such as denoting the FP subtraction. that there are now as many SIMD accumulators as the level of In Algorithm 2, this algorithm is refined to make use partial unrolling U (2 in this case). These accumulators need of vector instructions. In the following, the system will be to be summed together and reduced before proceeding with assumed to efficiently implement SIMD instructions on vectors the accumulation of remaining scalar elements. N (hereafter referred to as “packs” so that the “vector” term If the number of packs W is not a multiple of U, an can unambiguously refer to mathematical vectors such as the additional type of remainder needs to be handled: the full operand x) of width W . For x86_64 architectures (which are packs that were not handled in the partially unrolled loop. In the main target of the algorithms described here), the AVX the algorithm presented here, SIMD accumulator a1 is used extension for example supports efficiently operating on 256- to accumulate these. bit registers holding W = 4 double-precision FP numbers. Figure 1 illustrates the gains to be expected from such All variables denoted in bold font represent such packs, and a technique: for various vector sizes, the total summation are meant to be stored and manipulated as SIMD registers. time (normalized by the vector size) is plotted against the The main loop in Alg.