LecturE Notes “ScientifiC Computing I” WINTER TERM 2018/2019 Dr. Jens Saak Dipl.-Math. Martin Köhler jens.saak martin.koehler @mpi-magdeburg.mpg.de @mpi-magdeburg.mpg.de 1 FOR j :“ i; : : : ; n DO 2 FOR k :“ 1; : : : ; i ´ 1 DO 3 Ai;j ´ “ Ai;kAk;j VERSION FROM January 23, 2019 2 Contents 1 Linux AND THE Commandline1 1.1 A SHORT History OF AN Accidental ReVOLUTION............ 2 1.2 The Linux Shell AND Basic Commands FOR Handling Files..... 4 1.3 Getting Help ............................. 15 1.4 Manipulation OF Simple Commands ................ 15 1.5 Script File Basics........................... 17 1.6 Simple Automatic File Manipulation ................ 18 1.7 Remote Computing ON Encrypted Connections.......... 23 1.8 ScrEEN AN Online/OfflINE TERMINAL ................. 24 1.9 The TOOLCHAIN............................. 27 BibliogrAPHY ................................ 28 2 INTRODUCTION TO C AND THE GNU TOOLCHAIN 29 2.1 The PrOGRAMMING EnvirONMENT................... 30 2.2 C Statements, TYPES AND OperATORS ................ 34 2.3 ContrOL StructurES.......................... 39 2.4 CompleX Data TYPES AND ArrAYS .................. 44 2.5 Functions............................... 51 2.6 An INTRODUCTION TO THE StandarD LibrARY.............. 53 2.6.1 stdio.h AND stdlib.h . 53 2.6.2 math.h AND complex.h . 56 2.6.3 string.h .......................... 58 2.7 File INPUT AND Output ........................ 59 2.8 The PrEPROCESSOR AND Header Files................. 60 2.9 MakEfiLES............................... 63 2.10 WRITING Own LibrARIES........................ 66 I II Contents 2.11 INTERFACING FortrAN.......................... 68 2.12 Automatic GenerATION OF Documentations Using DOXYGEN . 70 BibliogrAPHY ................................ 71 3 ReVISION ContrOL 73 3.1 TYPES OF ReVISION ContrOL Systems................. 74 3.1.1 Local ReVISION ContrOL.................... 74 3.1.2 CentrAL ReVISION ContrOL .................. 75 3.1.3 Distributed ReVISION ContrOL................ 75 3.2 CollaborATIVE WORK ON PrOJECTS................... 76 3.2.1 ConflICTS ........................... 76 3.2.2 BrANCHES........................... 76 3.2.3 TAGS.............................. 76 3.3 ReVISION ContrOL MEETS Social Networking............. 77 3.3.1 ISSUES............................. 77 3.3.2 Pull Request / MerGE Request ............... 77 3.3.3 Forks............................. 78 3.3.4 A GENERIC WORKflOW ..................... 78 4 ErrOR Analysis AND Machine Numbers 79 4.1 Machine Numbers.......................... 80 4.2 Rounding ErrORS AND ErrOR PrOPAGATION .............. 83 4.2.1 Rounding Rules ....................... 83 4.2.2 Computer Arithmetic.................... 87 4.2.3 ErrOR PrOPAGATION...................... 88 4.2.4 The IEEE StandarD 754 ................... 92 4.3 ErrOR Analysis............................. 95 4.3.1 Conditioning/Condition Number.............. 96 4.3.2 Stability............................ 96 4.3.3 ForwarD ErrOR Analysis ................... 97 4.3.4 BackwarD ErrOR Analysis .................. 98 4.3.5 Perturbation Analysis....................101 BibliogrAPHY ................................105 5 Memory ArCHITECTURE AND Memory Management 107 5.1 Virtual Memory Concept.......................109 5.1.1 Paging ............................110 5.1.2 Memory Related ErrOR Signals . 110 5.2 VOLATILE MEMORY...........................111 5.2.1 Registers...........................111 5.2.2 Cache.............................111 5.2.3 Main Memory ........................112 5.3 Non-VOLATILE StorAGE.........................113 5.3.1 Local StorAGE Media.....................113 III Contents 5.3.2 Local Network........................113 5.3.3 Cloud AND Remote Network Services . 114 5.4 Non Uniform Memory Access....................114 5.4.1 Cache CoherENCE ......................114 5.4.2 Memory Consistency ....................114 BibliogrAPHY ................................115 6 Basic Operations, Formats AND Matrix-Norms 117 6.1 VECTOR Norms AND INNER PrODUCTS . 118 6.2 Linear Operators, OperATOR AND Matrix Norms . 120 6.2.1 SpectrAL Norm AND SpectrAL Radius . 125 6.2.2 Condition Number AND Singular VALUES . 128 6.2.3 Some Remarks ON κ2pAq . 129 6.3 Matrix StorAGE Formats.......................131 6.3.1 Dense Matrices .......................132 6.3.2 Sparse Matrices.......................135 6.3.3 CompleX Matrices......................139 6.4 Linear AlgebrA SoftwarE.......................140 6.4.1 Basic Linear AlgebrA SubrOUTINES (BLAS) . 141 6.4.2 Linear AlgebrA PACKage (LAPACK) . 145 6.4.3 SuiteSparse .........................147 6.4.4 ITPACK............................147 6.4.5 TRILINOS............................147 6.4.6 Native Packages FOR OTHER PrOGRAMMING EnvirONMENTS AND Languages .......................148 BibliogrAPHY ................................148 7 The Solution OF ModerATE Size Dense Linear Systems 151 7.1 IMPORTANT PrELIMINARIES.......................151 7.2 Cache/BLAS Exploitation.......................154 7.2.1 TRIANGULAR System......................154 7.2.2 TRIANGULAR Systems WITH Multiple Right Hand Sides AND BLAS Level-3 FORMULATION..................155 7.2.3 BLAS Level-3 BASED Gaussian Elimination . 156 7.3 ITERATIVE RefiNEMENT.........................157 BibliogrAPHY ................................159 8 Solving Linear Systems With Sparse Matrices 161 8.1 PrECONDITIONING ...........................164 8.1.1 Diagonal PrECONDITIONING..................164 8.1.2 Splitting Methods......................165 8.1.3 Multigrid APPROACHES....................165 8.1.4 INCOMPLETE Factorizations . 165 8.1.5 Sparse ApprOXIMATE INVERSES (SPAI) . 166 IV Contents 8.2 KryloV Subspaces AND PrOJECTION Methods . 166 8.3 Conjugate GrADIENTS.........................168 8.4 DirECT Solvers FOR Sparse Symmetric Systems . 170 8.4.1 The Elimination GrAPH Model FOR Symmetric Matrices . 171 8.4.2 The fiLLED GRAPH G`pAq . 172 8.4.3 CharACTERIZATION OF Fill-in..................173 8.4.4 Heuristic Fill Reduction...................174 8.4.5 Related SoftwarE ......................180 BibliogrAPHY ................................180 PrEFACE Die VORLESUNG “Wissenschaftliches Rechnen 1” VERFOLGT DAS Ziel, Ver- German FAHREN UND Algorithmen DER Numerischen Mathematik PRAKTISCH umzusetzen. Sie SOLL Wissen UND StrATEGIEN vermitteln, WELCHE NOTWENDIG sind, UM IDEEN AUS DER Theorie IN PRAKTISCH NUTZBARE PrOGRAMME ZU übersetzen UND DIESE EffiZIENT ZU IMPLEMENTIERen. Dies SOLL MEHRHEITLICH MIT Hilfe DER PrOGRAMMIERSPRACHE C geschehen, DA SIE EINE DER AM MEISTEN EINGESETZTEN SprACHEN IST1 UND AUCH IM BerEICH VON EINGEBETTETEN System UNVERZICHTBAR ist. Die REIN MATHEMATISCHE BetrACHTUNG VON PrOBLEMSTELLUNG REICHT IN VIELEN Fäl- LEN DEM Urheber DES PrOBLEMS NICHT MEHR aus. Viel MEHR SIND INDUSTRIE UND TECHNIK AN PRAKTISCH NUTZBAREN ErGEBNISSEN für DIE Anwendung IN Informatik, INGENIEURWESEN UND AlltagsprOBLEMEN INTERessiert. Neben DER Umsetzungen VON MATHEMATISCHEN VERFAHREN SOLL DER Umgang MIT UNIXOIDEN Betriebssystemen (in DIESEM Fall Linux) ERLERNT WERden. Diese BILDEN DIE hauptsächlich EINGESETZTE Klasse VON Betriebssystemen AUF DEN GRoßen UND SEHR GRoßen Installationen, WIE Compute-Clustern IN MODERNEN Rechenzentren. Neben DEN Betriebssystem-SpeZIfiKA WERDEN AUCH Hilfsmittel VORgestellt, DIE DEN Arbeitsablauf IM Umfeld DES WISSENSCHAFTLICHEN Rechnens erleichtern. This LECTURE AIMS AT THE PRACTICAL IMPLEMENTATION OF METHODS AND al- English GORTIHMS IN NUMERICAL mathematics. ITS MAIN PURPOSE IS TO CONVEY THE KNOwl- EDGE AND STRATEGIES NECESSARY TO TRANSFER AND EffiCIENTLY IMPLEMENT THEORETICAL IDEAS INTO COMPUTER PROGRAMS FOR PRACTICAL application. WE WILL FOCUS ON THE C 1http://www.tiobe.com/index.php/content/paperinfo/tpci/index. html V VI Contents PROGRAMMING LANGUAGE SINCE THIS IS ONE OF THE MOST COMONLY USED languages, WHICH IS ESPECIALLY INVALUABLE IN THE ENVIRONMENT OF EMBEDDED systems. The PURELY MATHEMATICAL CONSIDERATION OF PROBLEM SETTINGS OFTEN IS NO LONGER SUfficient. TODAY PARTNERS FROM INDUSTRY AND TECHNOLOGY ARE INTERESTED IN PRac- TICALLY USABLE RESULTS FOR APPLICATIONS IN COMPUTER AND ENGINEERING sciences. Along WITH THE PRACTICAL IMPLEMENTATION OF MATHEMATICAL METHODS THE USAGE OF UNIXOIDAL OPERATING SYSTEMS (in OUR CASE Linux) IS TO BE learned. Those OPERat- ING SYSTEMS FORM THE MOST IMPORTANT CLASS OF OPERATING SYSTEMS USED ON LARGE COMPUTE CLUSTERS IN MODERN HIGH PERFORMANCE COMPUTING centers. Besides op- ERATING SYSTEM SPECIfiCS WE ALSO PRESENT A COUPLE OF TOOLS THAT HELP SIMPLIFYING WORK IN A SCIENTIfiC COMPUTING ENVIRonment. Layout AND Style WE HAVE PUT SOME EffORT INTO CREATING A UNIQUE READING EXPERIENCE THAT VISUALLY SUPPORTS THE READER IN IDENTIFYING CONTRIBUTIONS TO THE content. Examples ARE TYPESET INSIDE LIGHT GRAY BACKGROUND BOXES TO fiND THEM EASILY IN THE document. TheY FOLLOW A chapter-wise NUMBERING scheme, THAT IS ALSO USED FOR Theorem- LIKE ENVIRONMENTS (i.e. DEfinitions, THEORems, lemmas, COROLLARIES AND Remarks). These ENVIRONMENTS ARE ALL DISPLAYED AS FRAMED BOXES WHERE DEfiNITIONS ARE MARKED BY A -symbol. Theorems, COROLLARIES AND LEMMAS CAN BE IDENTIfiED BY THE -symbol AND REMARKS SHOW A . Equation NUMBERS FOLLOW THEIR OWN chapter-wise scheme. Commands, PROGRAM VARIABLES AND ALIKE ARE DISPLAYED IN typewriter STYLE THROUGHOUT THE document. When AN APPROPRIATE PORTION OF CODE IS PResented, WE USE COLOR CODING (of THE BACKGROUND color) TO IDENTIFY THE TYPE OF CODE THAT IS displayed. WE DISTINGUISH THE FOLLOwing: C sources Fortran sources Shell scripts (especially BASH) Makefiles AcknoWLEDGMENTS WE WOULD LIKE TO THANK A COUPLE OF PEOPLE THAT HELPED