DOCSLIB.ORG

Libflame the Complete Reference ( Version 5.1.0-56 )

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

libflame The Complete Reference ( version 5.1.0-56 ) Field G. Van Zee The University of Texas at Austin Copyright c 2011 by Field G. Van Zee. 10 9 8 7 6 5 4 3 2 1 All rights reserved. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, contact either of the authors. No warranties, express or implied, are made by the publisher, authors, and their employers that the programs contained in this volume are free of error. They should not be relied on as the sole basis to solve a problem whose incorrect solution could result in injury to person or property. If the programs are employed in such a manner, it is at the user's own risk and the publisher, authors, and their employers disclaim all liability for such misuse. Trademarked names may be used in this book without the inclusion of a trademark symbol. These names are used in an editorial context only; no infringement of trademark is intended. Library of Congress Cataloging-in-Publication Data not yet available Draft, November 2008 This \Draft Edition" allows this material to be used while we sort out through what mechanism we will publish the book. Contents 1. Introduction 1 1.1. What's provided...........................................1 1.2. What's not provided.........................................6 1.3. Acknowledgments...........................................7 2. Setup for GNU/Linux and UNIX9 2.1. Before obtaining libflame .....................................9 2.1.1. System software requirements................................9 2.1.2. System hardware support.................................. 10 2.1.3. License............................................ 10 2.1.4. Source code.......................................... 11 2.1.5. Tracking source code revisions............................... 11 2.1.6. If you have problems..................................... 11 2.2. Preparation.............................................. 11 2.3. Configuration............................................. 12 2.3.1. configure options...................................... 12 2.3.2. Running configure ..................................... 19 2.4. Compiling............................................... 20 2.4.1. Parallel make ......................................... 21 2.5. Installation.............................................. 22 2.6. Linking against libflame ...................................... 22 2.6.1. Linking with the lapack2flame compatibility layer.................... 24 3. Setup for Microsoft Windows 25 3.1. Before obtaining libflame ..................................... 25 3.1.1. System software requirements................................ 25 3.1.2. System hardware support.................................. 26 3.1.3. License............................................ 26 3.1.4. Source code.......................................... 26 3.1.5. Tracking source code revisions............................... 26 3.1.6. If you have problems..................................... 26 3.2. Preparation.............................................. 26 3.3. Configuration............................................. 28 3.3.1. IronPython.......................................... 28 3.3.2. Running configure.cmd .................................. 30 3.4. Compiling............................................... 30 3.5. Installation.............................................. 32 3.6. Dynamic library generation..................................... 32 3.7. Linking against libflame ...................................... 34 i 4. Using libflame 37 4.1. FLAME/C examples......................................... 37 4.2. FLASH examples........................................... 39 4.3. SuperMatrix examples........................................ 40 5. User-level Application Programming Interfaces 45 5.1. Conventions.............................................. 45 5.1.1. General terms......................................... 45 5.1.2. Notation............................................ 46 5.1.3. Objects............................................ 49 5.2. FLAME/C Basics.......................................... 50 5.2.1. Initialization and finalization................................ 50 5.2.2. Object creation and destruction............................... 51 5.2.3. General query functions................................... 53 5.2.4. Interfacing with conventional matrix arrays........................ 55 5.2.5. More query functions.................................... 62 5.2.6. Assignment/Update functions................................ 67 5.2.7. Math-related functions.................................... 70 5.2.8. Miscellaneous functions................................... 80 5.2.9. Advanced query routines.................................. 82 5.3. Managing Views........................................... 84 5.3.1. Vertical partitioning..................................... 84 5.3.2. Horizontal partitioning.................................... 85 5.3.3. Bidirectional partitioning.................................. 86 5.3.4. Merging views........................................ 88 5.4. FLASH................................................ 89 5.4.1. Motivation.......................................... 89 5.4.2. Concepts........................................... 90 5.4.3. Interoperability with FLAME/C.............................. 91 5.4.4. Object creation and destruction............................... 92 5.4.5. Interfacing with flat matrix objects............................. 95 5.4.6. Interfacing with conventional matrix arrays........................ 101 5.4.7. Object query functions.................................... 104 5.4.8. Managing Views....................................... 107 5.4.8.1. Vertical partitioning................................ 107 5.4.8.2. Horizontal partitioning.............................. 108 5.4.8.3. Bidirectional partitioning............................. 109 5.4.9. Utility functions....................................... 110 5.4.9.1. Miscellaneous functions.............................. 110 5.5. SuperMatrix............................................. 110 5.5.1. Overview........................................... 110 5.5.2. API.............................................. 111 5.5.3. Integration with FLASH front-ends............................. 115 5.6. Front-ends............................................... 115 5.6.1. BLAS operations....................................... 115 5.6.1.1. Level-1 BLAS................................... 115 5.6.1.2. Level-2 BLAS................................... 130 5.6.1.3. Level-3 BLAS................................... 143 5.6.2. LAPACK operations..................................... 154 5.6.3. Utility functions....................................... 191 5.7. External wrappers.......................................... 213 5.7.1. BLAS operations....................................... 214 5.7.1.1. Level-1 BLAS................................... 214 5.7.1.2. Level-2 BLAS................................... 223 5.7.1.3. Level-3 BLAS................................... 231 5.7.2. LAPACK operations..................................... 238 5.7.3. LAPACK-related utility functions............................. 249 5.8. LAPACK compatibility (lapack2flame).............................. 249 5.8.1. Supported routines...................................... 250 A. FLAME Project Related Publications 259 A.1. Books................................................. 259 A.2. Dissertations............................................. 259 A.3. Journal Articles............................................ 259 A.4. Conference Papers.......................................... 260 A.5. FLAME Working Notes....................................... 262 A.6. Other Technical Reports....................................... 265 B. License 267 B.1. BSD 3-clause license......................................... 267 List of Contributors A large number of people have contributed, and continue to contribute, to the FLAME project. For a complete list, please visit http://www.cs.utexas.edu/users/flame/ Below we list the people who have contributed directly to the knowledge and understanding that is summa- rized in this text. Paolo Bientinesi The University of Texas at Austin Ernie Chan The University of Texas at Austin John A. Gunnels IBM T.J. Watson Research Center Kazushige Goto The University of Texas at Austin Tze Meng Low The University of Texas at Austin Margaret E. Myers The University of Texas at Austin Enrique S. Quintana-Ort´ı Universidad Jaume I Gregorio Quintana-Ort´ı Universidad Jaume I Robert A. van de Geijn The University of Texas at Austin v Chapter 1 Introduction In past years, the FLAME project, a collaborative effort between The University of Texas at Austin and Universidad Jaime I de Castellon, developed a unique methodology, notation, and set of APIs for deriving and representing linear algebra libraries. In an effort to better promote the techniques characteristic to the FLAME project, we have implemented a functional prototype library that demonstrates findings and insights from the last decade of research. We call this library libflame.1 The primary purpose of libflame is to provide the scientific and numerical computing communities with a modern, high-performance dense linear algebra library that is extensible, easy to use, and available under an open source license. Its developers have published numerous papers and working notes over the last decade documenting the challenges and motivations
Recommended publications
  • Linpack Evaluation on a Supercomputer with Heterogeneous Accelerators

    Linpack Evaluation on a Supercomputer with Heterogeneous Accelerators

    Linpack Evaluation on a Supercomputer with Heterogeneous Accelerators Toshio Endo Akira Nukada Graduate School of Information Science and Engineering Global Scientific Information and Computing Center Tokyo Institute of Technology Tokyo Institute of Technology Tokyo, Japan Tokyo, Japan [email protected] [email protected] Satoshi Matsuoka Naoya Maruyama Global Scientific Information and Computing Center Global Scientific Information and Computing Center Tokyo Institute of Technology/National Institute of Informatics Tokyo Institute of Technology Tokyo, Japan Tokyo, Japan [email protected] [email protected] Abstract—We report Linpack benchmark results on the Roadrunner or other systems described above, it includes TSUBAME supercomputer, a large scale heterogeneous system two types of accelerators. This is due to incremental upgrade equipped with NVIDIA Tesla GPUs and ClearSpeed SIMD of the system, which has been the case in commodity CPU accelerators. With all of 10,480 Opteron cores, 640 Xeon cores, 648 ClearSpeed accelerators and 624 NVIDIA Tesla GPUs, clusters; they may have processors with different speeds as we have achieved 87.01TFlops, which is the third record as a result of incremental upgrade. In this paper, we present a heterogeneous system in the world. This paper describes a Linpack implementation and evaluation results on TSUB- careful tuning and load balancing method required to achieve AME with 10,480 Opteron cores, 624 Tesla GPUs and 648 this performance. On the other hand, since the peak speed is ClearSpeed accelerators. In the evaluation, we also used a 163 TFlops, the efficiency is 53%, which is lower than other systems.
  • Red Hat Enterprise Linux 6 Developer Guide

    Red Hat Enterprise Linux 6 Developer Guide

    Red Hat Enterprise Linux 6 Developer Guide An introduction to application development tools in Red Hat Enterprise Linux 6 Dave Brolley William Cohen Roland Grunberg Aldy Hernandez Karsten Hopp Jakub Jelinek Developer Guide Jeff Johnston Benjamin Kosnik Aleksander Kurtakov Chris Moller Phil Muldoon Andrew Overholt Charley Wang Kent Sebastian Red Hat Enterprise Linux 6 Developer Guide An introduction to application development tools in Red Hat Enterprise Linux 6 Edition 0 Author Dave Brolley [email protected] Author William Cohen [email protected] Author Roland Grunberg [email protected] Author Aldy Hernandez [email protected] Author Karsten Hopp [email protected] Author Jakub Jelinek [email protected] Author Jeff Johnston [email protected] Author Benjamin Kosnik [email protected] Author Aleksander Kurtakov [email protected] Author Chris Moller [email protected] Author Phil Muldoon [email protected] Author Andrew Overholt [email protected] Author Charley Wang [email protected] Author Kent Sebastian [email protected] Editor Don Domingo [email protected] Editor Jacquelynn East [email protected] Copyright © 2010 Red Hat, Inc. and others. The text of and illustrations in this document are licensed by Red Hat under a Creative Commons Attribution–Share Alike 3.0 Unported license ("CC-BY-SA"). An explanation of CC-BY-SA is available at http://creativecommons.org/licenses/by-sa/3.0/. In accordance with CC-BY-SA, if you distribute this document or an adaptation of it, you must provide the URL for the original version. Red Hat, as the licensor of this document, waives the right to enforce, and agrees not to assert, Section 4d of CC-BY-SA to the fullest extent permitted by applicable law.
  • Atmospheric Modelling and HPC

    Atmospheric Modelling and HPC

    Atmospheric Modelling and HPC Graziano Giuliani ICTP – ESP Trieste, 1 October 2015 NWP and Climate Codes ● Legacy code from the '60 : FORTRAN language ● Path and Dynamic Libraries ● Communication Libraries ● I/O format Libraries ● Data analysis tools and the data deluge http://clima-dods.ictp.it/Workshops/smr2761 Materials ● This presentation: – atmospheric_models_and_hpc.pdf ● Codes: – codes.tar.gz ● Download both, uncompress codes: – cp codes.tar.gz /scratch – cd /scratch – tar zxvf codes.tar.gz Legacy code ● Numerical Weather Prediction uses mathematical models of the atmosphere and oceans to predict the weather based on current weather conditions. ● MOST of the NWP and climate codes can push their roots back in the 1960-1970 ● The NWP problem was one of the target problem which led to the Computer Era: the ENIAC was used to create the first weather forecasts via computer in 1950. ● Code grows by ACCRETION FORTRAN and FortranXX ● Fortran Programming Language is a general-purpose, imperative programming language that is especially suited to numeric computation and scientific computing. ● Originally developed by IBM in the 1950s for scientific and engineering applications ● Its standard is controlled by an ISO comitee: the most recent standard, ISO/IEC 1539-1:2010, informally known as Fortran 2008, was approved in September 2010. ● Needs a compiler to translate code into executable Fortran Compilers ● http://fortranwiki.org ● Most of the High Performance Compilers are Commercial, and “SuperComputer” vendors usually provide their HIGH OPTIMIZED version of Fortran Compiler. ● Some part of the Standard are left “COMPILER SPECIFIC”, opening for incompatibility among the binary formats produced by different compilers. The Fortran .mod files ● Fortran 90 introduced into the language the modules: module my_mod contains subroutine mysub(a,b) real , intent(in) :: a real , intent(out) :: b end subroutine mysub end module my_mod ● The compiler creates the object file and a .mod file compiling the above code.
  • Key Benefits Key Features

    Key Benefits Key Features

    With the MapleSim LabVIEW®/VeriStand™ Connector, you can • Includes a set of Maple language commands, which extend your LabVIEW and VeriStand applications by integrating provides programmatic access to all functionality as an MapleSim’s high-performance, multi-domain environment alternative to the interactive interface and supports into your existing toolchain. The MapleSim LabVIEW/ custom application development. VeriStand Connector accelerates any project that requires • Supports both the External Model Interface (EMI) and high-fidelity engineering models for hardware-in-the-loop the Simulation Interface Toolkit (SIT). applications, such as component testing and electronic • Allows generated block code to be viewed and modified. controller development and integration. • Automatically generates an HTML help page for each block for easy lookup of definitions and parameter Key Benefits defaults. • Complex engineering system models can be developed and optimized rapidly in the intuitive visual modeling environment of MapleSim. • The high-performance, high-fidelity MapleSim models are automatically converted to user-code blocks for easy inclusion in your LabVIEW VIs and VeriStand Applications. • The model code is fully optimized for high-speed real- time simulation, allowing you to get the performance you need for hardware-in-the-loop (HIL) testing without sacrificing fidelity. Key Features • Exports MapleSim models to LabVIEW and VeriStand, including rotational, translational, and multibody mechanical systems, thermal models, and electric circuits. • Creates ANSI C code blocks for fast execution within LabVIEW, VeriStand, and the corresponding real-time platforms. • Code blocks are created from the symbolically simplified system equations produced by MapleSim, resulting in compact, highly efficient models. • The resulting code is further optimized using the powerful optimization tools in Maple, ensuring fast execution.
  • CCPP Technical Documentation Release V3.0.0

    CCPP Technical Documentation Release V3.0.0

    CCPP Technical Documentation Release v3.0.0 J. Schramm, L. Bernardet, L. Carson, G. Firl, D. Heinzeller, L. Pan, and M. Zhang Jun 17, 2019 For referencing this document please use: Schramm, J., L. Bernardet, L. Carson, G. Firl, D. Heinzeller, L. Pan, and M. Zhang, 2019. CCPP Technical Documentation Release v3.0.0. 91pp. Available at https://dtcenter.org/GMTB/v3.0/ccpp_tech_guide.pdf. CONTENTS 1 CCPP Overview 1 1.1 How to Use this Document........................................4 2 CCPP-Compliant Physics Parameterizations7 2.1 General Rules..............................................8 2.2 Input/output Variable (argument) Rules.................................9 2.3 Coding Rules............................................... 10 2.4 Parallel Programming Rules....................................... 11 2.5 Scientific Documentation Rules..................................... 12 2.5.1 Doxygen Comments and Commands.............................. 12 2.5.2 Doxygen Documentation Style................................. 13 2.5.3 Doxygen Configuration..................................... 18 2.5.4 Using Doxygen......................................... 20 3 CCPP Configuration and Build Options 23 4 Constructing Suites 27 4.1 Suite Definition File........................................... 27 4.1.1 Groups............................................. 27 4.1.2 Subcycling........................................... 27 4.1.3 Order of Schemes........................................ 28 4.2 Interstitial Schemes........................................... 28 4.3 SDF Examples.............................................
  • Towards a Fully Automated Extraction and Interpretation of Tabular Data Using Machine Learning

    Towards a Fully Automated Extraction and Interpretation of Tabular Data Using Machine Learning

    UPTEC F 19050 Examensarbete 30 hp August 2019 Towards a fully automated extraction and interpretation of tabular data using machine learning Per Hedbrant Per Hedbrant Master Thesis in Engineering Physics Department of Engineering Sciences Uppsala University Sweden Abstract Towards a fully automated extraction and interpretation of tabular data using machine learning Per Hedbrant Teknisk- naturvetenskaplig fakultet UTH-enheten Motivation A challenge for researchers at CBCS is the ability to efficiently manage the Besöksadress: different data formats that frequently are changed. Significant amount of time is Ångströmlaboratoriet Lägerhyddsvägen 1 spent on manual pre-processing, converting from one format to another. There are Hus 4, Plan 0 currently no solutions that uses pattern recognition to locate and automatically recognise data structures in a spreadsheet. Postadress: Box 536 751 21 Uppsala Problem Definition The desired solution is to build a self-learning Software as-a-Service (SaaS) for Telefon: automated recognition and loading of data stored in arbitrary formats. The aim of 018 – 471 30 03 this study is three-folded: A) Investigate if unsupervised machine learning Telefax: methods can be used to label different types of cells in spreadsheets. B) 018 – 471 30 00 Investigate if a hypothesis-generating algorithm can be used to label different types of cells in spreadsheets. C) Advise on choices of architecture and Hemsida: technologies for the SaaS solution. http://www.teknat.uu.se/student Method A pre-processing framework is built that can read and pre-process any type of spreadsheet into a feature matrix. Different datasets are read and clustered. An investigation on the usefulness of reducing the dimensionality is also done.
  • Compiling Maplesim C Code for Simulation in Vissim

    Compiling Maplesim C Code for Simulation in Vissim

    Compiling MapleSim C Code for Simulation in VisSim 1 Introduction MapleSim generates ANSI C code from any model. The code contains the differential equations that describe the model dynamics, and a solver. Moreover, the code is royalty-free, and can be used in any simulation tool (or development project) that accepts external code. VisSim is a signal-flow simulation tool with strength in embedded systems programming, real-time data acquisition and OPC. This document will describe the steps required to • Generate C code from a MapleSim model of a DC Motor. The C code will contain a solver. • Implement the C code in a simulation DLL for VisSim. VisSim provides a DLL Wizard that sets up a Visual Studio C project for a simulation DLL. MapleSim code will be copied into this project. After a few modifications, the project will be compiled to a DLL. The DLL can then be used as a block in a VisSim simulation. The techniques demonstrated in this document can used to implement MapleSim code in any other environment. MapleSim’s royalty -free C code can be implemented in other modeling environment s, such as VisSim MapleSim’s C code can also be used in Mathcad 2 API for the Maplesim Code The C code generated by MapleSim contains four significant functions. • SolverSetup(t0, *ic, *u, *p, *y, h, *S) • SolverStep(*u, *S) where SolverStep is EulerStep, RK2Step, RK3Step or RK4Step • SolverUpdate(*u, *p, first, internal, *S) • SolverOutputs(*y, *S) u are the inputs, p are subsystem parameters (i.e. variables defined in a subsystem mask), ic are the initial conditions, y are the outputs, t0 is the initial time, and h is the time step.
  • Supermatrix: a Multithreaded Runtime Scheduling System for Algorithms-By-Blocks

    Supermatrix: a Multithreaded Runtime Scheduling System for Algorithms-By-Blocks

    SuperMatrix: A Multithreaded Runtime Scheduling System for Algorithms-by-Blocks Ernie Chan Field G. Van Zee Paolo Bientinesi Enrique S. Quintana-Ort´ı Robert van de Geijn Department of Computer Science Gregorio Quintana-Ort´ı Department of Computer Sciences Duke University Departamento de Ingenier´ıa y Ciencia de The University of Texas at Austin Durham, NC 27708 Computadores Austin, Texas 78712 [email protected] Universidad Jaume I {echan,field,rvdg}@cs.utexas.edu 12.071–Castellon,´ Spain {quintana,gquintan}@icc.uji.es Abstract which led to the adoption of out-of-order execution in many com- This paper describes SuperMatrix, a runtime system that paral- puter microarchitectures [32]. For the dense linear algebra opera- lelizes matrix operations for SMP and/or multi-core architectures. tions on which we will concentrate in this paper, many researchers We use this system to demonstrate how code described at a high in the early days of distributed-memory computing recognized that level of abstraction can achieve high performance on such archi- “compute-ahead” techniques could be used to improve parallelism. tectures while completely hiding the parallelism from the library However, the coding complexity required of such an effort proved programmer. The key insight entails viewing matrices hierarchi- too great for these techniques to gain wide acceptance. In fact, cally, consisting of blocks that serve as units of data where oper- compute-ahead optimizations are still absent from linear algebra ations over those blocks are treated as units of computation. The packages such as ScaLAPACK [12] and PLAPACK [34]. implementation transparently enqueues the required operations, in- Recently, there has been a flurry of interest in reviving the idea ternally tracking dependencies, and then executes the operations of compute-ahead [1, 25, 31].
  • Benchmark of C++ Libraries for Sparse Matrix Computation

    Benchmark of C++ Libraries for Sparse Matrix Computation

    Benchmark of C++ Libraries for Sparse Matrix Computation Georg Holzmann http://grh.mur.at email: [email protected] August 2007 This report presents benchmarks of C++ scientific computing libraries for small and medium size sparse matrices. Be warned: these benchmarks are very special- ized on a neural network like algorithm I had to implement. However, also the initialization time of sparse matrices and a matrix-vector multiplication was mea- sured, which might be of general interest. WARNING At the time of its writing this document did not consider the eigen library (http://eigen.tuxfamily.org). Please evaluate also this very nice, fast and well maintained library before making any decisions! See http://grh.mur.at/blog/matrix-library-benchmark-follow for more information. Contents 1 Introduction 2 2 Benchmarks 2 2.1 Initialization.....................................3 2.2 Matrix-Vector Multiplication............................4 2.3 Neural Network like Operation...........................4 3 Results 4 4 Conclusion 12 A Libraries and Flags 12 B Implementations 13 1 1 Introduction Quite a lot open-source libraries exist for scientific computing, which makes it hard to choose between them. Therefore, after communication on various mailing lists, I decided to perform some benchmarks, specialized to the kind of algorithms I had to implement. Ideally algorithms should be implemented in an object oriented, reuseable way, but of course without a loss of performance. BLAS (Basic Linear Algebra Subprograms - see [1]) and LA- PACK (Linear Algebra Package - see [3]) are the standard building blocks for efficient scientific software. Highly optimized BLAS implementations for all relevant hardware architectures are available, traditionally expressed in Fortran routines for scalar, vector and matrix operations (called Level 1, 2 and 3 BLAS).
  • Using Machine Learning to Improve Dense and Sparse Matrix Multiplication Kernels

    Using Machine Learning to Improve Dense and Sparse Matrix Multiplication Kernels

    Iowa State University Capstones, Theses and Graduate Theses and Dissertations Dissertations 2019 Using machine learning to improve dense and sparse matrix multiplication kernels Brandon Groth Iowa State University Follow this and additional works at: https://lib.dr.iastate.edu/etd Part of the Applied Mathematics Commons, and the Computer Sciences Commons Recommended Citation Groth, Brandon, "Using machine learning to improve dense and sparse matrix multiplication kernels" (2019). Graduate Theses and Dissertations. 17688. https://lib.dr.iastate.edu/etd/17688 This Dissertation is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Using machine learning to improve dense and sparse matrix multiplication kernels by Brandon Micheal Groth A dissertation submitted to the graduate faculty in partial fulfillment of the requirements for the degree of DOCTOR OF PHILOSOPHY Major: Applied Mathematics Program of Study Committee: Glenn R. Luecke, Major Professor James Rossmanith Zhijun Wu Jin Tian Kris De Brabanter The student author, whose presentation of the scholarship herein was approved by the program of study committee, is solely responsible for the content of this dissertation. The Graduate College will ensure this dissertation is globally accessible and will not permit alterations after a degree is conferred. Iowa State University Ames, Iowa 2019 Copyright c Brandon Micheal Groth, 2019. All rights reserved. ii DEDICATION I would like to dedicate this thesis to my wife Maria and our puppy Tiger.
  • Anatomy of High-Performance Matrix Multiplication

    Anatomy of High-Performance Matrix Multiplication

    Anatomy of High-Performance Matrix Multiplication KAZUSHIGE GOTO The University of Texas at Austin and ROBERT A. VAN DE GEIJN The University of Texas at Austin We present the basic principles which underlie the high-performance implementation of the matrix- matrix multiplication that is part of the widely used GotoBLAS library. Design decisions are justified by successively refining a model of architectures with multilevel memories. A simple but effective algorithm for executing this operation results. Implementations on a broad selection of architectures are shown to achieve near-peak performance. Categories and Subject Descriptors: G.4 [Mathematical Software]: —Efficiency General Terms: Algorithms;Performance Additional Key Words and Phrases: linear algebra, matrix multiplication, basic linear algebra subprogrms 1. INTRODUCTION Implementing matrix multiplication so that near-optimal performance is attained requires a thorough understanding of how the operation must be layered at the macro level in combination with careful engineering of high-performance kernels at the micro level. This paper primarily addresses the macro issues, namely how to exploit a high-performance “inner-kernel”, more so than the the micro issues related to the design and engineering of that “inner-kernel”. In [Gunnels et al. 2001] a layered approach to the implementation of matrix multiplication was reported. The approach was shown to optimally amortize the required movement of data between two adjacent memory layers of an architecture with a complex multi-level memory. Like other work in the area [Agarwal et al. 1994; Whaley et al. 2001], that paper ([Gunnels et al. 2001]) casts computation in terms of an “inner-kernel” that computes C := AB˜ + C for some mc × kc matrix A˜ that is stored contiguously in some packed format and fits in cache memory.
  • DD2358 – Introduction to HPC Linear Algebra Libraries & BLAS

    DD2358 – Introduction to HPC Linear Algebra Libraries & BLAS

    DD2358 – Introduction to HPC Linear Algebra Libraries & BLAS Stefano Markidis, KTH Royal Institute of Technology After this lecture, you will be able to • Understand the importance of numerical libraries in HPC • List a series of key numerical libraries including BLAS • Describe which kind of operations BLAS supports • Experiment with OpenBLAS and perform a matrix-matrix multiply using BLAS 2021-02-22 2 Numerical Libraries are the Foundation for Application Developments • While these applications are used in a wide variety of very different disciplines, their underlying computational algorithms are very similar to one another. • Application developers do not have to waste time redeveloping supercomputing software that has already been developed elsewhere. • Libraries targeting numerical linear algebra operations are the most common, given the ubiquity of linear algebra in scientific computing algorithms. 2021-02-22 3 Libraries are Tuned for Performance • Numerical libraries have been highly tuned for performance, often for more than a decade – It makes it difficult for the application developer to match a library’s performance using a homemade equivalent. • Because they are relatively easy to use and their highly tuned performance across a wide range of HPC platforms – The use of scientific computing libraries as software dependencies in computational science applications has become widespread. 2021-02-22 4 HPC Community Standards • Apart from acting as a repository for software reuse, libraries serve the important role of providing a