Scientific Computing with Python Dr. A.A.N. Ridder February 22, 2019 Abstract This document reports Python procedures, data structures, file manipulations, algo- rithms, modules, etc for usage in the Numerical Methods course of the BSc EOR program. This document assumes that you have basic knowledge of Python programming. If not, see Section 18 for links to online tutorials, pdf tutorials and many more documenta- tions. Contents 1 Floating Point Representation 1 1.1 Python Functions .................................. 2 2 Math: Standard Mathematical Functions 3 3 NumPy for More Mathematical Functions 4 4 NumPy for Arrays 4 4.1 Creating Arrays ................................... 4 4.2 Array Indexing and Slicing ............................ 7 4.3 Memory Management ............................... 7 4.4 Numerical Indexing ................................. 8 4.5 Logical Indexing .................................. 9 4.6 Finding and Searching ............................... 9 4.7 Array Arithmetic Operations ........................... 10 4.8 Printing ....................................... 10 4.9 Array Ufuncs .................................... 11 4.10 Array Routines ................................... 12 4.11 Array Linear Algebra ................................ 15 5 Dictionaries 17 5.1 Creating a Dictionary ............................... 17 5.2 Using a Dictionary ................................. 18 5.3 Dictionary Functions ................................ 18 5.4 Application: Dijkstra Algorithm ......................... 19 1 6 Files 19 6.1 TXT Files for String Manipulations ....................... 20 6.2 TXT and CSV Files Using NumPy ........................ 20 6.2.1 Reading ................................... 21 6.2.2 Change Type ................................ 22 6.2.3 Missing Data ................................ 22 6.2.4 Writing to File ............................... 23 6.3 JSON Files ..................................... 24 7 Pandas 24 7.1 The Series Structure ................................ 24 7.2 The DataFrame Structure ............................. 26 7.3 Using Pandas .................................... 28 7.3.1 Inspecting the Data ............................ 28 7.3.2 Indexing ................................... 30 7.3.3 Selecting, Finding and Slicing ....................... 30 7.4 DataFrames and Dictionaries ........................... 31 8 File Input and Output Using Pandas 32 8.1 Read and Write Data From TXT and CSV ................... 32 8.2 Missing Data .................................... 32 8.3 Read and Write Data From Excel ......................... 34 9 matplotlib: Graphics 35 9.1 Plotting the Graph of a Function ......................... 35 9.1.1 Axis ..................................... 36 9.1.2 Axes ..................................... 37 9.2 Plotting Multiple Graphs in one Figure ..................... 37 9.3 Creating Multiple Figures ............................. 38 9.4 Subplots ....................................... 39 9.5 Scatter Plot ..................................... 41 9.6 Pie Chart ...................................... 42 9.7 Histogram ...................................... 42 9.7.1 Single Data Set ............................... 42 9.7.2 Two Data Sets ............................... 44 9.7.3 Probabilities ................................ 45 9.8 Bar Plot ....................................... 46 9.9 3D Plots ....................................... 47 9.10 LaTeX and Substitution .............................. 47 10 Dealing with Times and Dates 48 10.1 The datetime module ............................... 51 10.2 matplotlib and datetime ............................. 52 11 Random Numbers and Simulation 53 11.1 Random Number Generator ............................ 53 11.2 Simple Random Data ............................... 54 11.3 Random Sample from a Finite Distribution ................... 55 11.4 Random Permutations ............................... 55 2 11.5 Random Finite Probability Distribution ..................... 56 11.6 Probability Distributions .............................. 56 11.6.1 Uniform (continue) on (a; b) ........................ 56 11.6.2 Uniform (discrete) on fa; : : : ; bg ...................... 56 11.6.3 Normal (µ, σ2) ............................... 56 11.6.4 Bernoulli(p) on fFalse,Trueg ....................... 57 11.6.5 Logistic(µ, s) ................................ 57 12 Numerical Methods 57 12.1 Polynomials ..................................... 57 12.1.1 Interpolation and Regression ....................... 58 12.2 Splines ........................................ 60 12.3 Roots of Functions ................................. 60 12.4 Solving Linear Equations ............................. 62 12.5 Solving Non-Linear Equations ........................... 62 12.6 Minimization of Univariate Functions ....................... 65 12.7 Minimalization of Multivariate Functions .................... 66 12.8 Numerical Integration ............................... 66 13 Graphs and Networks 67 14 Statistics 67 14.1 Empirical PMF of Discrete Data ......................... 68 14.2 Continuous Data .................................. 69 14.3 A Goodness-of-Fit Test .............................. 71 15 Operations Research 72 15.1 linprog ........................................ 72 15.2 pulp ......................................... 73 15.3 cvxopt ........................................ 74 15.4 cvxpy ........................................ 74 15.5 Gurobi ........................................ 75 16 Markov Chains 75 16.1 Equilibrium Distribution of Discrete-Time MC’s ................ 76 16.2 Equilibrium Distribution of Continuous-Time MC’s ............... 78 16.3 Simulation of a Discrete-Time Markov chain ................... 80 16.4 Simulation of a Continuous-Time Markov chain ................. 82 17 Splitting Your Program in Multiple Files 82 18 Further Information 83 18.1 Python for Absolute Beginners .......................... 83 18.2 Python for Econometrics .............................. 83 18.3 Numerical Programming with Python ...................... 84 18.4 Modules Docmentation ............................... 84 18.4.1 NumPy (version 1.15.0) .......................... 84 18.4.2 SciPy (version 1.10.0) ........................... 84 18.4.3 Matplotlib (version 2.2.3) ......................... 84 3 18.4.4 Pandas (version 0.23.4) .......................... 84 1 Floating Point Representation Because the computer is a finite machine, it can not represent exactly all real numbers, and thus it can not make exact computations!. Real numbers are represented by the IEEE-754 floating point format, similarly as in most other computer languages. Recall that anyreal number other than zero can be written mathematically by ( 1 ) X β x = 2k × 1 + i ; 2i i=1 where k 2 Z is the exponent, and βi 2 f0; 1g are called bits. In the IEEE-754 system this becomes ( ) X52 β x = 2k × 1 + i ; 2i i=1 with k 2 {−1022;:::; 1023g. This set of finite real numbers are the machine numbers. Any other real number that you enter in a computer program is rounded to its nearest machine number. P 52 i The finite sum f = i=1 βi=2 is the fraction of the number. Hence, in short-hand notation, machine numbers have the form x = sign × 2exponent × (1 + fraction): For storing such numbers we (the computer) need 64 bits (8 bytes): • 1 bit for the sign. • 11 bits for the exponent. This 11-bit binary number is converted to a decimal number in f0; 1;:::; 2047g, and then 1023 is subtracted. • 52 bits for the fraction. Note that a (artificial) 1 is added. This makes the fraction a normalized fraction. Thus, you get something like x = 0 10011010010| {z } |1100100010100001001100001101110011110010101001110001{z } exponent fraction ( ) 1 4 6 7 10 1 1 1 1 = +22 +2 +2 +2 +2 −1023 × 1 + + + + ··· + : 2 22 25 252 Note that there are ‘only’ 264 of such machine numbers, each representing a unique real number. Actually, some of these machine numbers are reserved for 0, inf and NaN. For a clearer insight, the 52 bits of the fraction are grouped in 13 numbers of consecutive 4 bits, where these 4-bit numbers are hexadecimal digits: 0; 1; 2; 3; 4; 5; 6; 7; 8; 9; a; b; c; d; e; f: Thus, the above 64-bit number becomes x = 0 10011010010| {z } |1100{z} |1000{z} |1010{z} |0001{z} |0011{z} |0000{z} |1101{z} |1100{z} |1111{z} |0010{z} |1010{z} |0111{z} |0001{z} 21234−1023 c 8 a 1 3 0 d c f 2 a 7 1 = 2211 × 1:c8a130dcf2a71 4 1.1 Python Functions Python prints the floating point number representation as 0x1.hhhhhhhhhhhhhp+eeee where • The prefix 0x shows that it is a hexadecimal constant. • The 1 represents the leading 1 bit of a normalized binary fraction. • 13 hex digits h 2 f0; 1;:::; 9; a;:::; fg representing the 52 bits of the the binary fraction. • p+ or p- represents the power 2 with a positive or negative exponent. • At most 4 decimal digits for the exponent; i.e. e 2 f0; 1;:::; 9g and eeee 2 f0; 1;:::; 1023g (in case of negative power, p- it goes up to 1022. Example: Python prints the number 2211 × 1:c8a130dcf2a71 as 0x1.c8a130dcf2a71p+211. Conversion functions from decimal to floating point and vice versa are available. • <decimal>.hex() print a decimal number as hexadecimal floating point. • float.fromhex('<floating point>') print a hexadecimal floating point number as reugular decimaal number. Examples: In []: float.fromhex('0x1.c8a130dcf2a71p +211') Out[]: 5.870204477445384e+63 In []: x=5.870204477445384e+63 In []: x.hex() Out[]: '0x1.c8a130dcf2a71p +211' In []: (15.125).hex() Out[]: '0x1.e400000000000p+3' In []: (-math.sqrt(2)).hex() Out[]: '-0x1.6a09e667f3bcdp+0'