Python in High-Performance Computing Martin Sandve Alnæs, Are Magnus Bruaset, Xing Cai, Hans Petter Langtangen, Kent-Andre Mardal, Halvard Moe, Ola Skavhaug, Åsmund Ødegård Simula Research Laboratory c 2006 Topics Crash Course in Python 4 Scientific Computing with Python 17 Visualization in with Python 35 Performance Issues – Tips and Tricks 58 Parallel Computing via Python 64 Extend Your Favorite Library with Python 82 Various Packages for Scientific Computing 121 2 Why Scientific Computing with Python The old days Today FORTRAN C C++ Perl Python Java • Languages have moved from speed and efficiency to flexibility and convenience • Python is a powerful yet easy to use language • Python has a rich set of libraries and extensions • Python is an ideal glue between your applications • Using wrapping techniques, your legacy code may get another life • Python is suitable for computational steering • In this tutorial we also do number crunching with Python! 3 Crash Course in Python We begin with a crash course in Python. In this section we cover: • Basic variables, containers and control structures • Functions, lambda functions and callable objects • Object oriented features like classes and operator overloading • String editing and file handling • For more information on the Python language, please check out: • The Slides "Scripting for Computational Science" [35] • The Book "Python Scripting for Computational Science" [36] • The Python tutorial on python.org [82] • The Introductory Material on Python [32] 4 5 ¤ ¤ § ¦ ¤ ¤ ¦ ¨ £¥¤ ¨ © ¨ §©¨ ! ¦ ¤ © ¦ §©¨ § § ¤ £¥¤ ¦ ¦ ¦ ¢ ¢ ¢ ¦ ¢ ¢ ¢ £¥¤ ¢ ¢ ¢ a in t lines star with on IDLE. t appear segments either or lines ] to star in 33 interactive prefix ¡ code appear sessions: ¡ uing , line ¡ file Sessions the h supposed with Eac Contin Output Run sessions without IPython[ ogram • • • • pr are Interactive Otherwise • • Interactive 6 s les math umber orld, to a n W modules w ariab y v call and ho xt Hello command-line with librar and te guments ork function Load Read ar Load W Print • • • • • Scientific A demonstrates § ¦ ¥ ¤ ¦ ¦ ¤ ¤ ¥ ¤ ¦ ¦ § © ¦ § ¨ £ £ ¥ ¤ ¤ ¦ £ ¤ xample ¤ £ ¥ ¥ ¤ e § ¤ ¤ ¤ ¡ ¥ ¤ ¦ § ¤ ¤ ¤ ¤ ¢ ¦ ¡ ¤ £ £ ¥ ¥ ¤ ¤ ¦ ¤ ¡ ¥ Python A Basic Types • Python is a dynamically typed language, meaning that the type of every variable is determined runtime • Everything is an object, even integers, modules and functions • Python has the following basic variable types: • int: 1, 2, 3 • long (arbitrary length!): 1L, 2L, 3L • complex: 1j, 4+5j • float (only double precision): 0.1, 0.2, 0.3 • bool: True or False • str: "hello world" ¨ © ¢ ¢ ¢¤£ ¥ ¥ ¡ ¦ ¡ ¦ ¡ ¦ £ • Casting is done like § 7 8 ¨ ¨ s £ ¢ of £ © ¢ ¥ ¡ ¨ wrapper ¡ ¦ ¢ objects ¡ are £ ¢ ¡ h ¦ ¢ hold ¦ ¢ © whic ¡ ys): can ¡ © h ¢ ¢ ¨ arra ¡ © later), ¡ ¢ whic ¡ ¦ (see ¥ ¢ ¡ © ¢ types, ys © lists): ¢ © ¡ associative ¡ ¢ arra ¡ or ¥ ¢ ¥ § ¨ © ys. container § maps ¤ ¢ ¡ NumPy in ¦ ¥ arra le/constant ¦ s ¥ C use uilt ¥ ¤ ¨ b (hash utab £ ¤ § will § § (imm three ¢ ¢ we ¥ © . ¢ ¢ has Container contiguous ¡ ¡ ¡ uples Lists: T Dictionaries dition, type ad • • • y ound an ar Python In Basic 9 lists empty and ver o ¤ ¦ strings ¤ iterated indentation ¨ be empty ¥ ¤ ¦ 0, can , equal ¤ ¤ y £ ¤ b if that ¥ ¤ ¤ None ¤ £ to ¦ . ¤ ¥ ¤ ¥ ¤ ¦ ything type ¤ defined ¤ y an ¦ similar £ are © an ¡ on ¤ £ ¥ ¤ ¦ £ ork ¨ § ks ¥ on Structures ¨ w ¤ ¤ ¦ ¢ false ork ¦ loc ¤ ¤ ol w ¤ £ b ork to ¦ ¤ w ¨ ¤ £ ¤ ¦ Contr -loops aluate ogram ¤ § v or ¦ ¦ Pr f if-tests while-loops e ¤ ¤ ¥ • • • • ¦ ¤ Basic ¤ ¤ § 10 ¥ ¤ ¤ £ ¥ £ ¥ £ ¤ ¤ £ ¦ § ¤ ¤ § ¥ ¥ ¥ ¤ ¤ ¤ ¤ ¦ ¤ § ¤ ¤ ¤ § £ ¥ ¤ £ ¥ ¦ ¦ ¦ § § £ ¤ § ¤ ¤ ¤ ¤ ¥ © ¥ ¤ § ¥ ¤ ¤ ¢ ¦ ¥ ¤ ¤ ¤ ¥ ¦ ¦ ¦ ¦ ¦ tools x.: one joins .e str str F ¢ creates y in into b © xpressions werful ¢ e ds ¢ Handling list po or ¡ a strings. w ¨ ¢ ¨ © of ¡ File regular ¢ the some separated with ¡ of © ¡ ¡ items and has ¦ ¦ ¢ ¢ list erl-like the a P string, orking © © w • • • or Python f String 11 orms: f defining y defined aluated b veral v e simple se operator or be in f objects, functions, functions s can al le v come e def __call__ venient with the with con Regular Lambda Strings Callab oneliner • • • • Functions ¨ ¢ ¦ ¡ ¡ ¤ ¨ § ¢ ¡ § © ¢ § ¢ © ¤ ¡ ¡ ¨ ¤ ¤ ¡ ¡ ¤ § § ¦ ¦ ¡ ¤ ¢ ¡ ¦ ¤ ¤ ¤ ¢ ¢ ¢ ¦ ¢ ¡ ! § § ¤ ¡ ¦ ¦ ¤ ¨ ¡ ¦ ¦ ¤ ¤ ¢ ¡ ! ¡ § ¤ ¤ ¡ § © ¤ £ ¢ ¨ ¡ § ¦ ¦ ¢ ¡ § § ¥ ¡ ¦ ¦ ¦ § ¤ ¤ ¤ ¤ ¤ ¤ ¦ ¦ ¤ ¤ ¦ ¦ § ¤ £ £ £ ¥ § § ¤ ¤ ¦ £ ¤ ¦ ¦ ¤ £ ¤ £ £ ¤ ¤ ¤ ¤ £ £ ¢ ¤ ¦ ¦ ¤ £ ¤ ¤ £¥¤ Functions Object Oriented Numerics in Python • Python is a powerful object oriented language • Everything in Python is an object • There are no protected or private variables or methods, but the ¢ ¡ ¥ ¦ ¦ £ ¥ effect can be “simulated” with underscore prefix ( ¥ ¦ ¦ £ £ ) • Python supports multiple inheritance • Dynamic typing implies support for generic programming (as with C++ templates) 12 13 ¡ ¡ ¡ y b ve ¢ ¡ ¤ © beha statements ¡ lasses or c ¨ , £ £ print types wn ¤ ¤ o ¦ ¢ ¡ in wn ¡ o ., our © and § § y y i.e , ¢ like our , ¡ ¡ in y ¡ ¢ s ¢ returning ¢ or let f ¨ © Overloading) © to ¦ ¢ © iterating or respectivel f © or ¡ Used , f operator ¡ or ¨ xamples: f : ¦ or e f Used ¥ Used Comparison, : methods, or ¥ ¡ Used : : ¡ ¢ § Used (Operator Some : ¡ ¡ ¥ § : ¥ ¥ ¡ ¢ ¢ ¨ © verloading ¢ ¢ ¢ o , ¡ special ¥ ¨ © ¨ © ts types. ¨ © ¨ © ¨ © ¢ okes ¢ v ¦ ¢ Methods some ¦ ¢ ¢ in uiltin ¥ ¡ £ b suppor mean ¡ © to the laring • • • • • • Python dec like Special 14 y the of of in y onl rue es. T g cop to a is element h langua h manipulation containing aluates ¡ eac ¡ v whic ¨ § list e ¨ § to ¤ ¦ w ¤ ¦ ¤ list, simple ¤ ne functional w et a ¡ y ¨ in ne ¨ function ¡ a function § ¨ like ¨ werful creates ¡ ¨ ¤ ¨ where £ ¤ ¨ ¦ po £ ¢ ¡ ¢ ¥ creates ¤ ¦ § ¢ passed ¥ ¤ list do ¤ ¤ ¦ © Python, § ¢ ¢ § the ¥ ¤ ou in old ¥ © ¦ y ¤ ¢ ¥ ¤ ¢ ¥ ¦ ¥ ogramming ¡ let the ¦ ¤ ¨ ¡ ¥ ¤ ¦ Pr £ applies ¤ ¡ ¢ ¤ £ om £ ¨ ¡ £ datatype fr £ ¡ and ¢ £ ¡ ¥ Style functions ¨ list, in central ¦ ¥ list ¢ ¨ a old elements ¢ uilt w ¥ b are the ne the w e • • f Lists A lists. Functional Introspection § ¦ ¤ ¤ £ ¢ £ £ ¤ ¤ ¤ ¤ ¦ ¤ Python lets you examine and edit objects and their properties runtime. ¨ ¢ © ¥ £ • ¦ ¡ ¡ returns a list of the names of all the properties of the object, both variables and functions ¨ ¢ ¢ © ¥ £ ¡ ¡ • ¡ returns the type of the object ¨ ¢ ¢ ¢ ¢ £ ¥ ¥ © ¥ £ • ¡ ¡ tells you if an object is something that can be called like a function ¢ ¢ ¢ © ¥ • The function ¦ lets you add new variables or functions to a class ¡ £ • All objects have a variable that can hold a documentation string 15 Python has a comprehensive library We mention a few: • A portable interface to the operating system, e.g., file and process ¡ ¢ © ¡ ¥ © ¦ management ( ), file compression ( ), threads ( ) £ ¤ £ ¢ ¢ ¢ ¡ ¦ ¢ ¡ • GUIs: Qt ( ¡ ), Gtk ( ), Tk ( ), WxWindows ¡ ¢ ¡ ¡ ( ¥ ¡ ), . ¢ © • String handling ( ¦ ¡ ), regular expressions (¦ ) • Many applications with Python interface: Word/OpenOffice, Excel/Gnumeric, Oracle, Gimp . ¡ ¢ ¢ ¢ ¢ ¢ ¢ £ ¡ ¡ © • Web modules: , , , 16 Scienti®c Computing with Python • Computing with Numerical Python is introduced • Constructing arrays • Vectoring expressions • Slicing • Solving a linear system • Some available modules for Scientific Computing are then presented • The packages covered in this tutorial are chosen on basis of our research experiences with numerical solution of Partial Differential Equations. • The Vaults of Parnassus[83] have an extensive list of Python modules 17 Numeric vs numarray vs. numpy • There are actually three different implementations of Numerical Python (NumPy) ¦¢§ ¢¡ ¤ • £ ¥ is the original and hence widely used © © ¨ £ ¥ ¥ • ¡ was a reimplementation with some new features ¨ £ • ¡ is a blend of the other two, again with improvements • The three packages have almost the same interface • The performance differs greatly: numpy is fastest for vectorized operations, while Numeric is fastest for indexing and scalar operations • Now it seems best to use a common interface to Numerical Python such that any of the three packages can be used as backend • A common interface can be found at http://folk.uio.no/hpl/scripting/src/tools/py4cs/numpytools.py 18 Example: Solving a Differential Equation • Python can be used as a scientific calculator, like Matlab, Octave, R, etc. • Applications: −u00(x) = f (x) • heat conductivity 2 [ ] x 0, 1 • string deflection u(0) = u(1) = 0 • fluid flow f (x) user given • electrostatics • elasticity, ... • Goal: Compute u using the Numerical Python module (aka NumPy) • NumPy is an extension module for Python that enables efficient numerical computing 19 Numerical Solution Procedure • Divide [0, 1] into n + 1 cells h = 1=(n + 1) • Discretization by the Finite xi = h ∗ i, i = 0, 1, . , n + 1 Difference method u = u(x ) i i • 2 Result: bi = h f (xi) linear system 1 0 · · · 0 2 . 3 . Au = b 60 2 −1 0 · · · . 7 6 7 6 . 7 6 . 7 6 . −1 2 −1 7 6 7 A = 6 . 7 6 .. .. .. 7 6 7 6 7 6 − − 7 6 1 2 1 7 6 7 6 0 −1 2 07 6 7 6 7 60 · · · 0 17 4 5 20 § 21 £ ¨ & : ¡ ¢ ¡ ¢ ¨ £ ¡ $ ¥ ¢ 9 ¢ ¢ ¢ ¨ $ '& : © § § § $ ¥ ¡ ¡ $ " ¨ $ '& : $ ¨