COMPUTING FOR SCIENCE AND ENGINEERING Revised through end of Chapter 6 Robert D. Skeel This work is licensed under the Creative Commons Attribution 3.0 United StatesLicense. To view a copy of this license, visit http://creativecommons.org/licenses/by/3.0/us/ or send a letter to Creative Commons, 444 Castro Street, Suite 900, Mountain View, California, 94041, USA. Contents 1 Introduction 1 1.1 Role of Computing in Science and Engineering.....................1 1.2 Using Unix.........................................2 1.2.1 Text editors.....................................5 1.2.2 Integrated development environments......................5 1.2.3 Shell scripts.....................................5 1.2.4 Unix tools......................................6 1.3 Installation of Software..................................8 1.3.1 Unix, C compiler, and OpenMP.........................9 1.3.2 The Anaconda Python Distribution and MPI.................. 10 1.3.3 ∗ The Enthought Python Distribution...................... 11 1.3.4 ∗ Separate Installation of Python and Packages................. 12 1.3.5 ∗ MPI for Python................................. 12 1.4 Use of Rosen Center Linux Cluster............................ 13 1.4.1 Remote access and file transfer.......................... 13 1.4.2 Python....................................... 13 2 Python 14 2.1 Interactive Computing................................... 14 2.1.1 Importing a module................................ 15 2.1.2 Documentation................................... 15 2.1.3 Lists......................................... 15 2.1.4 Strings........................................ 16 2.1.5 Data......................................... 16 2.1.6 Operators...................................... 17 2.2 Developing a Script..................................... 18 2.2.1 Control structures................................. 19 2.2.2 Objects....................................... 20 2.2.3 Reading and writing................................ 20 2.3 Variables and Functions.................................. 22 2.3.1 Variables and data................................. 22 2.3.2 Assignment and copy............................... 22 i 2.3.3 Functions...................................... 23 2.4 Developing a Module.................................... 25 2.4.1 Exception handling................................. 26 2.4.2 Verification..................................... 26 2.4.3 Side effects..................................... 27 2.4.4 Debugging...................................... 27 2.4.5 Coding standards.................................. 27 2.4.6 Documentation................................... 27 2.5 Operating System Commands............................... 28 2.6 Object-oriented Programming............................... 29 3 Python Extensions 33 3.1 NumPy........................................... 33 3.1.1 Array constructors................................. 33 3.1.2 Indexing and slicing................................ 34 3.1.3 Views........................................ 34 3.1.4 Array operations.................................. 35 3.1.5 Fancy indexing................................... 35 3.1.6 Input/output functions.............................. 36 3.1.7 Matrix operations................................. 37 3.2 NumPy Modules...................................... 38 3.2.1 Pseudo-random numbers.............................. 38 3.2.2 Linear algebra functions.............................. 38 3.3 Matplotlib.......................................... 38 3.4 Other Extensions...................................... 40 3.4.1 Computer algebra................................. 40 3.4.2 SciPy........................................ 41 4 C Programming 42 4.1 Declarations and Control Structures........................... 42 4.2 Pointers, Arrays, and malloc ............................... 44 4.2.1 Pointers....................................... 44 4.2.2 One-dimensional arrays.............................. 44 4.2.3 Multidimensional arrays.............................. 45 4.2.4 malloc ....................................... 46 4.3 Functions and Programs.................................. 48 4.3.1 Definitions vs. declarations............................ 48 4.3.2 Function pointers.................................. 48 4.3.3 Scope........................................ 49 4.4 Strings and Input/Output................................. 50 4.4.1 Strings........................................ 50 4.4.2 I/O functions.................................... 50 4.5 Structures.......................................... 51 ii 4.5.1 Abstract data types................................ 52 4.5.2 Data structures................................... 54 4.6 Compiling and Debugging................................. 57 4.6.1 Creating dynamically loaded libraries...................... 58 4.6.2 Dynamic loading.................................. 58 4.6.3 Improving performance.............................. 59 4.7 Calling Compiled Code from Python........................... 60 4.7.1 The Python ctypes module............................ 60 4.7.2 The NumPy ctypeslib module.......................... 62 5 Parallel Computing 64 5.1 Computer System Basics.................................. 64 5.1.1 Computer organization.............................. 64 5.1.2 Processes...................................... 65 5.1.3 Python as glue................................... 65 5.2 Distributed Memory Programming............................ 66 5.2.1 MPI......................................... 67 5.2.2 Point-to-point operations............................. 67 5.2.3 Collective operations................................ 69 5.2.4 Grid computing and Condor........................... 69 5.3 Computing on Scholar and PBS Job Submission.................... 70 5.4 Shared Memory Programming............................... 72 5.4.1 POSIX Threads.................................. 72 5.4.2 Cilk Plus...................................... 74 5.4.3 OpenMP...................................... 74 5.4.4 Coprocessors, including graphics processing units................ 75 5.4.5 OpenACC...................................... 76 5.4.6 Python multiprocessing module.......................... 76 6 Survey of Other Topics 78 6.1 Software Development................................... 78 6.1.1 Version control................................... 78 6.1.2 Automated builds................................. 79 6.2 Algorithms......................................... 82 6.2.1 Loop invariants................................... 82 6.2.2 Computational complexity............................. 83 6.2.3 Parallel algorithms................................. 83 6.3 Floating-point Computation................................ 85 6.3.1 Floating-point numbers.............................. 85 6.3.2 Rounding...................................... 86 6.3.3 Floating-point operations............................. 87 6.3.4 Other types of arithmetic............................. 87 6.4 Regular Expressions.................................... 88 iii 6.5 GUI Programming..................................... 90 6.5.1 Tkinter....................................... 91 6.5.2 Web interfaces................................... 91 6.6 Programming Languages.................................. 92 6.6.1 Other scripting languages............................. 92 6.6.2 Computer algebra systems............................. 92 6.6.3 Matlab and Octave................................. 92 6.6.4 R........................................... 93 6.6.5 Julia......................................... 93 6.6.6 Fortran, C, and C++............................... 93 6.7 Data............................................. 94 6.7.1 XML......................................... 94 6.7.2 Relational databases................................ 94 A Functions 95 A.1 Polynomial approximation................................. 95 A.1.1 Polynomial representation............................. 95 A.1.2 Existence and uniqueness............................. 96 A.1.3 Lagrange form................................... 96 A.1.4 Interpolation.................................... 97 A.1.5 Bivariate linear polynomials............................ 97 A.2 Piecewise Polynomial Approximation........................... 99 A.2.1 Piecewise polynomials............................... 99 A.2.2 Multivariate piecewise linear polynomials.................... 100 A.3 Trigonometric Approximation............................... 101 A.3.1 Fast Fourier Transform.............................. 102 B Matrices 104 B.1 Matrix Computations................................... 104 B.1.1 Partitioned matrices................................ 104 B.1.2 Structured matrices................................ 105 B.1.3 Efficient matrix operations............................ 105 B.2 Triangular Factorizations................................. 106 B.2.1 Division by a triangular matrix.......................... 107 B.2.2 LU factorization.................................. 108 B.2.3 Symmetric matrices................................ 109 B.2.4 NumPy and LAPACK functions......................... 109 B.3 Orthogonal Factorizations................................. 111 B.3.1 Full and reduced QR factorizations........................ 112 B.3.2 Singular value decomposition........................... 113 B.3.3 NumPy and LAPACK functions......................... 113 B.4 Spectral Factorizations................................... 115 B.4.1 General matrices.................................. 116 iv B.4.2 Symmetric matrices................................ 117 B.4.3