MAX PLANCK INSTITUTE FOR DYNAMICS OF COMPLEX TECHNICAL SYSTEMS MAGDEBURG Lecture Notes “Scientific Computing I” winter term 2014/2015 Dr. Jens Saak Dipl.-Math. Martin Köhler jens.saak martin.koehler @mpi-magdeburg.mpg.de @mpi-magdeburg.mpg.de 1 for j :“ i; : : : ; n do 2 for k :“ 1; : : : ; i ´ 1 do 3 Ai;j ´ “ Ai;kAk;j version from March 26, 2015 2 Contents 1 Linux and the Commandline1 1.1 A short History of an Accidental Revolution............2 1.2 The Linux Shell and Basic Commands for Handling Files.....4 1.3 Getting Help............................. 15 1.4 Manipulation of Simple Commands................ 15 1.5 Script File Basics.......................... 17 1.6 Simple Automatic File Manipulation................ 18 1.7 Remote Computing on Encrypted Connections.......... 24 1.8 Screen an Online/Offline Terminal................. 25 1.9 The Toolchain............................ 27 References and Further Reading..................... 28 2 Revision Control 29 2.1 Types of Revision Control Systems................. 30 2.1.1 Local Revision Control................... 30 2.1.2 Central Revision Control.................. 31 2.1.3 Distributed Revision Control................ 31 2.2 Collaborative Work on Projects................... 32 2.2.1 Conflicts........................... 32 2.2.2 Branches........................... 32 2.2.3 Tags............................. 32 3 Introduction to C and the GNU Toolchain 33 3.1 The Programming Environment................... 35 3.2 C Statements, Types and Operators................ 38 3.3 Control Structures.......................... 43 3.4 Complex Data Types and Arrays.................. 48 i ii Contents 3.5 Functions.............................. 54 3.6 An Introduction to the Standard Library.............. 56 3.6.1 stdio.h and stdlib.h ................. 56 3.6.2 math.h and complex.h ................. 59 3.6.3 string.h .......................... 61 3.7 File Input and Output........................ 62 3.8 The Preprocessor and Header Files................ 63 3.9 Makefiles............................... 66 3.10 Writing Own Libraries........................ 69 3.11 Interfacing Fortran.......................... 71 3.12 Automatic Generation of Documentations Using DOXYGEN... 73 References and Further Reading..................... 75 4 Error Analysis and Machine Numbers 77 4.1 Machine Numbers.......................... 77 4.2 Rounding Errors and Error Propagation.............. 81 4.2.1 Rounding Rules....................... 81 4.2.2 Computer Arithmetic.................... 85 4.2.3 Error Propagation...................... 86 4.2.4 The IEEE Standard 754.................. 90 4.3 Error Analysis............................ 92 References and Further Reading..................... 103 5 Memory Architecture and Memory Management 105 5.1 Virtual Memory Concept...................... 107 5.1.1 Paging............................ 108 5.1.2 Memory Related Error Signals............... 108 5.2 Volatile memory........................... 109 5.2.1 Registers........................... 109 5.2.2 Cache............................ 109 5.2.3 Main Memory........................ 110 5.3 Non-Volatile Storage........................ 111 5.3.1 Local Storage Media.................... 111 5.3.2 Local Network........................ 111 5.3.3 Cloud and Remote Network Services........... 112 5.4 Non Uniform Memory Access.................... 112 5.4.1 Cache Coherence...................... 112 5.4.2 Memory Consistency.................... 113 References and Further Reading..................... 113 6 Basic Operations, Formats and Matrix-Norms 115 6.1 Vector Norms and Inner Products................. 116 6.2 Linear Operators, Operator and Matrix Norms........... 118 6.2.1 Spectral Norm and Spectral Radius............ 124 Contents iii 6.2.2 Condition Number and Singular Values.......... 126 6.2.3 Some Remarks on κ2pAq .................. 128 6.3 Matrix Storage Formats....................... 129 6.3.1 Dense Matrices....................... 130 6.3.2 Sparse Matrices....................... 132 6.3.3 Complex Matrices...................... 137 6.4 Linear Algebra Software...................... 138 6.4.1 Basic Linear Algebra Subroutines (BLAS)......... 138 6.4.2 Linear Algebra PACKage (LAPACK)............ 142 6.4.3 SuiteSparse......................... 144 6.4.4 ITPACK........................... 145 6.4.5 Trilinos............................ 145 6.4.6 Native Packages for other Programming Environments and Languages....................... 145 References and Further Reading..................... 146 7 The Solution of Moderate Size Dense Linear Systems 149 7.1 Important Preliminaries....................... 149 7.2 Cache/BLAS Exploitation...................... 152 7.2.1 Triangular System...................... 152 7.2.2 Triangular Systems with Multiple Right Hand Sides and BLAS Level-3 formulation.................. 153 7.2.3 BLAS Level-3 based Gaussian Elimination......... 154 7.3 Iterative Refinement......................... 155 References and Further Reading..................... 157 8 Solving Linear Systems With Sparse Matrices 159 8.1 Preconditioning........................... 162 8.1.1 Diagonal Preconditioning.................. 162 8.1.2 Splitting Methods...................... 163 8.1.3 Multigrid approaches.................... 163 8.1.4 Incomplete Factorizations.................. 163 8.1.5 Sparse Approximate Inverses (SPAI)............ 164 8.2 Krylov Subspaces and Projection Methods............ 164 8.3 Conjugate Gradients........................ 166 8.4 Direct Solvers for Sparse Symmetric Systems........... 168 8.4.1 The Elimination Graph Model for Symmetric Matrices... 169 8.4.2 The filled graph G`pAq ................... 171 8.4.3 Characterization of Fill-in.................. 171 8.4.4 Heuristic Fill Reduction................... 172 8.4.5 Related Software...................... 178 References and Further Reading..................... 179 iv Contents Preface German Die Vorlesung “Wissenschaftliches Rechnen 1” verfolgt das Ziel, Ver- fahren und Algorithmen der Numerischen Mathematik praktisch umzusetzen. Sie soll Wissen und Strategien vermitteln, welche notwendig sind, um Ideen aus der Theorie in praktisch nutzbare Programme zu übersetzen und diese effizient zu implementieren. Dies soll mehrheitlich mit Hilfe der Programmier- sprache C geschehen, da sie eine der am meisten eingesetzten Sprachen ist1 und auch im Bereich von eingebetteten System unverzichtbar ist. Die rein mathematische Betrachtung von Problemstellung reicht in vielen Fällen dem Urheber des Problems nicht mehr aus. Viel mehr sind Industrie und Tech- nik an praktisch nutzbaren Ergebnissen für die Anwendung in Informatik, Inge- nieurwesen und Alltagsproblemen interessiert. Neben der Umsetzungen von mathematischen Verfahren soll der Umgang mit unixoiden Betriebssystemen (in diesem Fall Linux) erlernt werden. Diese bilden die hauptsächlich eingesetzte Klasse von Betriebssystemen auf den großen Compute-Clustern in modernen Rechenzentren. Neben den Betriebssystem- Spezifika werden auch Hilfsmittel vorgestellt, die den Arbeitsablauf im Umfeld des wissenschaftlichen Rechnens erleichtern. English This lecture aims at the practical implementation of methods and algortihms in numerical mathematics. Its main purpose is to convey the knowl- edge and strategies necessary to transfer and efficiently implement theoretical ideas into computer programs for practical application. We will focus on the C 1http://www.tiobe.com/index.php/content/paperinfo/tpci/index. html v vi Contents programming language since this is one of the most comonly used languages, which is especially invaluable in the environment of embedded systems. The purely mathematical consideration of problem settings often is no longer sufficient. Today partners from industry and technology are interested in prac- tically usable results for applications in computer and engineering sciences. Along with the practical implementation of mathematical methods the usage of unixoidal operating systems (in our case Linux) is to be learned. Those operat- ing systems form the most important class of operating systems used on large compute clusters in modern high performance computing centers. Besides op- erating system specifics we also present a couple of tools that help simplifying work in a scientific computing environment. Layout and Style We have put some effort into creating a unique reading experience that visually supports the reader in identifying contributions to the content. Examples are typeset inside light gray background boxes to find them easily in the document. They follow a chapter-wise numbering scheme, that is also used for Theorem- like environments (i.e. definitions, theorems, lemmas, corollaries and remarks). These environments are all displayed as framed boxes where definitions are marked by a -symbol. Theorems, corollaries and lemmas can be identified by the -symbol and remarks show a . Equation numbers follow their own chapter-wise scheme. Commands, program variables and alike are displayed in typewriter style throughout the document. When an appropriate portion of code is presented, we use color coding (of the background color) to identify the type of code that is displayed. We distinguish the following: C sources Fortran sources Shell scripts (especially BASH) Makefiles Acknowledgments We would like to thank a couple of people that helped us in preparing this manuscript. Some of them had major contributions. First of all Peter Benner Contents vii provided the German basis for Chapter4, which we slightly modified with mate- rial from the seminal book on