Introduction to Python Part 3: Advanced Topics Michael Kraus ([email protected]) Max-Planck-Institut fur¨ Plasmaphysik, Garching 1. December 2011 Advanced Topics calling and embedding C and Fortran code: Weave: inline C/C++ code, translation of Python code to C++ Cython: extension of the Python language providing static typed functions and variables, generating efficient C code for fast computations ctypes: call functions in C libraries f2py: wrap Fortran code parallelisation: threading multiprocessing parallel python (PP) mpi4py GUI programming with PyQt and PySide symbolic computing with Sage Calling and Embedding C and Fortran Code in Python Python is very fast when writing code, but not necessarily so fast when executing code (especially for numerical applications) implement time-consuming parts of your program in C/C++/Fortran example: solving the 2D Laplace equation by an iterative finite difference scheme (500x500 grid, 100 iterations) Type of Solution Time Taken (secs) Python 1500.0 Numpy 29.3 Weave (Blitz) 9.5 Weave (Inline) 4.3 f2py 2.9 Cython 2.5 Matlab 29.0 Pure C++ 2.2 [Numbers from the Beginners Guide to Using Python for Performance Computing: http://www.scipy.org/PerformancePython] Sample Problem: Laplace Equation solving the 2D Laplace equation using an iterative finite difference scheme (four point averaging, Gauss-Seidel or Gauss-Jordan) solve for some unknown function u(x; y) such that r2u = 0 with some boundary condition specified discretise the domain into an (nx × ny ) grid of points the function u can be represented as a two-dimensional array u(nx ; ny ) the values of u along the sides of the domain are given (and stay fixed) the solution can be obtained by iterating in the following manner: fori in range(1, nx-1): forj in range(1, ny-1): u[i,j] = ( (u[i-1, j] + u[i+1, j])*dy**2 + \ (u[i, j-1] + u[i, j+1])*dx**2 \ ) / (2.0*(dx**2 + dy**2)) Sample Problem: Laplace Equation in NumPy the for loop of the Laplace solver can be readily expressed by a much simpler NumPy expression: u[1:-1, 1:-1] = ( (u[0:-2, 1:-1] + u[2:, 1:-1])*dy**2 + \ (u[1:-1, 0:-2] + u[1:-1, 2:])*dx**2 \ ) / (2.0*(dx**2 + dy**2)) the advantage of this expression is that it is completely done in C speedup of a factor of 50x over the pure Python loop (another factor of 5 or so if you link NumPy with Intel MKL or ATLAS) (slight) drawback: this expression uses temporary arrays during one iteration, the computed values at an already computed location will not be used in the original for loop, once the value of u[1,1] is computed, the next value for u[1,2] will use the newly computed u[1,1] and not the old one since the NumPy expression uses temporary arrays internally, only the old value of u[1,1] will be used the algorithm will still converge but in twice as much time reduction of the benefit by a factor of 2 Weave Weave is a subpackage of SciPy and has two modes of operation weave.blitz accelerates Python code by translating it to C++ code which it compiles into a Python module weave.inline allows to embed C/C++ code directly into Python code mainly used to speed up calculations on arrays fast/efficient: directly operates on NumPy arrays (no temporary copies) the first time you run a blitz or inline function, it gets compiled into a Python module, the next time it is called, it will run immediately References: http://www.scipy.org/Weave http://www.scipy.org/Cookbook/Weave Sample Problem: Laplace Equation in weave.blitz to use weave.blitz, the accelerated code has to be put into a string which is passed to the weave.blitz function: from scipy import weave expr = """ u[1:-1, 1:-1]=((u[0:-2, 1:-1]+u[2:, 1:-1])*dy**2+\ (u[1:-1, 0:-2]+u[1:-1, 2:])*dx**2\ )/ (2.0*(dx**2+ dy**2)) """ weave.blitz(expr, check_size=0) the first time the code is called, weave.blitz converts the NumPy expression into C++ code, builds a Python module, and invokes it for the array expressions, weave.blitz uses Blitz++ speedup of 100-200x over the Python loop weave.blitz does not use temporary arrays for the computation (the computed values are re-used immediately) and therefore behaves more like the original for loop Sample Problem: Laplace Equation in weave.inline in weave.inline the C/C++ code has to be put into a string which is passed to the weave.inline function, together with the variables used: from scipy.weave import converters, inline code = """ for(inti=1;i<nx-1; ++i){ for(intj=1;j<ny-1; ++j){ u(i,j)=((u(i-1,j)+u(i+1,j))*dy*dy+ (u(i,j-1)+u(i,j+1))*dx*dx )/ (2.0*(dx*dx+ dy*dy)); } } """ inline(code, ['u','dx','dy','nx','ny'], type_converters=converters.blitz, compiler ='gcc') here we use Blitz++ arrays (speedup 250-500x over the Python loop) with pointer arithmetic, you can get an additional speedup of a factor 2 weave.inline does not use temporary arrays for the computation Cython Cython is a programming language based on Python provides extra syntax allowing for static type declarations (remember: Python is generally dynamically typed) the source code gets translated into optimised C/C++ code and compiled as Python extension modules Cython can compile (most) regular Python code but the optional static type declarations usually achieve major speed improvements allows for very fast program execution and tight integration with external C libraries combines the benefits of Python with the speed of C References: http://cython.org/ http://docs.cython.org/ Cython: Compilation Cython code must, unlike Python, be compiled: a .pyx source file is compiled by Cython to a .c file, containing the code of a Python extension module the .c file is compiled by a C compiler to a .so (shared object library) file which can be imported directly into a Python session several ways to build Cython code: use pyximport, importing Cython .pyx files as if they were .py files (using distutils to compile and build the background) write a distutils setup.py run the cython command-line utility manually to produce the .c file from the .pyx file, then manually compile the .c file into a shared object library use the Sage notebook which allows Cython code inline Cython: Compilation imagine a simple \hello world" script: hello.pyx def say_hello_to(name): print("Hello%s!" % name) implicit compilation using pyximport: >>> import pyximport >>> pyximport.install() >>> import hello >>> hello.say_hello_to("Mike") Hello Mike ! this allows to automatically run Cython on every .pyx that Python is trying to import use this for simple Cython builds where no extra C libraries and no special building setup is needed Cython: Compilation the following could be a corresponding setup.py script: setup.py from distutils.core import setup from distutils.extension import Extension from Cython.Distutils import build_ext ext_modules = [Extension("hello",["hello.pyx"])] setup ( name ='Hello World App', cmdclass = {'build_ext': build_ext}, ext_modules = ext_modules ) you can specify additional build settings (include dirs, linker flags, ...) ext_modules = [Extension("hello",["hello.pyx"], include_dirs = [numpy.get_include()], extra_compile_args = ['-fopenmp'], extra_link_args = ['-fopenmp'])] Cython: Compilation the script can be called with: > python setup.py build_ext --inplace after compilation, Cython modules can be imported like every other Python module: >>> import hello >>> hello.say_hello_to("Mike") Hello Mike ! Cython: Static Types Cython can compile normal Python code without changes (with a few exceptions of some as-yet unsupported language features), but for performance critical code, it is often helpful to add static type declarations they allow Cython to step out of the dynamic nature of the Python code and generate simpler and faster C code (sometimes faster by orders of magnitude) however, type declarations can make the source code more verbose and thus less readable it is discouraged to use them without good reason, i.e. only in performance critical sections where they really make the code substantially faster Cython: Static Types consider the following pure Python code: def f(x): return x**2-x def integrate_f(a, b, N): s = 0 dx = (b-a)/N fori in range(N): s += f(a+i*dx) returns*dx simply compiling this in Cython gives a 35% speedup adding some static types can make a much larger difference Cython: Static Types with additional type declarations, the former example might look like: def f(double x): return x**2-x def integrate_f(double a, double b, int N): cdef int i cdef double s, dx s = 0 dx = (b-a)/N fori in range(N): s += f(a+i*dx) returns*dx typing the iterator variable i with C semantics, tells Cython to compile the for-loop to pure C code typing a, s and dx is important as they are involved in arithmetic within the for-loop this results in a 4x speedup over the pure Python version Cython: Static Types Python function calls can be expensive: one needs to convert to and from Python objects to do the call in our example, the argument is assumed to be a C double both inside f() and in the call to it, yet a Python float object must be constructed around the argument in order to pass it Cython provides the cdef keyword for declaring a C-style function: cdef double f(double x): return x**2-x speedup: 150x over pure Python but: now the function is no longer available from Python-space, as Python wouldn't know how to call it using the cpdef keyword instead, a Python wrapper is created as well Cython: Static Types and NumPy Arrays you can use NumPy from Cython exactly the same as in regular Python, but by doing so you are loosing potentially high speedups Cython has support for fast access to NumPy arrays: import numpy as np cimport numpy as np def update(np.ndarray[double, ndim=2] f): cdef unsigned int i, j cdef np.ndarray[double, ndim=2] h = \ np.zeros([xmax, ymax], dtype=DTYPE) fori in range(0, f.shape[0]): forj in range(0, f.shape[1]): ..