1 The State of Python

Francisco Santos, Nuno Cerqueira, Ricardo Barreira, Rui Martins

FEUP - Faculdade de Engenharia da Universidade do Porto Rua Dr. Roberto Frias, s/n, 4200-465 Porto, Portugal [email protected], [email protected], [email protected], [email protected]

Abstract. In this article we describe the current state of the relatively recent Python programming language, which is an interpreted language with a simple syntax. Although often referred as a scripting language, it has a degree of functionality similar to the mainstream languages. Python’s philosophy is based on adaptability, flexibility and dynamism, supporting multiple programming paradigms. Some of the most useful and well known languages’ libraries and frame- works are then described. Among others, we present: SciPy, a that offers useful scientific functions (e.g. Fast Fourier Transform); Tur- boGears, a framework that allows rapid front-to-back Web Development, including automatic mapping of ; and wxPython, a framework to implement applications that use multi-platform native GUI widgets. In the end, we present a set of conclusions regarding the benefits of using Python as a general-purpose programming language.

1 Introduction

Computer Science is a very recent subject. Despite that, it has suffered an ex- ceptional growth, since in just half a century1 it became the centre of the In- formation Society, having an immense number of daily-use applications. With a growth rate of this magnitude, it is normal that several branches appear in its development, as a reaction to the different types of people involved, as well as their different needs. Nowadays there are millions of developers all around the world, with a multi- tude of cultures, education and needs. This variety is mirrored in the existing pro- gramming languages. There are some mainstream programming languages (like , ++ and C#), used widely due to their enterprise nature or capabili- ties in certain key domains. However, in other tasks these languages are quite inefficient2 and are outperformed by languages of another kind. Python [16] is not a very well known language and it is not a mainstream one, although its features allow the same degree of functionality of these lan- guages, but frequently in an easier way (for with the same degree of familiarity). 1 The first high-level programming languages were invented in the 1950s: COBOL, and Lisp. 2 In this case, the performance of a programming language is not given only by its computational efficiency, but also by its capabilities, productivity. . .

F. Santos, N. Cerqueira, R. Barreira, R. Martins 2 F. Santos, N. Cerqueira, R. Barreira, R. Martins

With this article we intend to “spread the word” about Python, describing its current state. In this section (Sect. 1) we present the context on which Python is inserted, the goals and structure of this document. In The Python Programming Language (Sect. 2) we describe the language features, its philosophy, history and future. In Libraries (Sect. 3) we describe some important libraries that extend the language, adding usefulness and power to it. In Frameworks (Sect. 4), we describe some Python frameworks that allow to easily create domain specific applications. Finally, in Conclusions (Sect. 5) we present a small summary of Python’s current state and issue some comments on its usage and evolution.

2 The Python Programming Language

2.1 History

Python is a relatively recent language. It was created in 1990 by Guido van Rossum at the CWI 3 in the Netherlands. Guido van Rossum was an experienced programmer and had been previously involved in the creation of the ABC programming language [5], also at the CWI. The lack of flexibility of ABC, along with his general dissatisfaction with other programming languages made him create Python. In its development he took advice and suggestions from several programmers, and a small Internet based community started to grow. After some problems with the licensing of the language, finally Python be- came completely Open . This event took place even before of the global establishment of the word Open Source. During this process, and due to the language’s features, the Python community had had an astounding growth as lots of people were contributing with very useful modules. The Python Foundation [14] was then created to satisfy the need for guidance that this evolution had created. This foundation is “(. . . ) a non-profit membership organization devoted to advancing open source technology related to the Python programming language”. One of its main purposes is to organize the PyCon – an annual conference where the state and evolution of Python is discussed, as well as the applications using it. Nowadays, pythoneers4 are very involved and actively developing applica- tions, frameworks and modules which contribute to the language’s growth. As a final remark, the name Python is often associated with snakes, which is wrong. As the author asserts frequently Python is really all about “spam, ham and eggs”, in a reference to Monty Python, the famous british comedian group. It is very common to see references to the series in Python’s code, tutorials and reference materials.

3 CWI stands for Centrum voor Wiskunde en Informatica, or National Research In- stitute for Mathematics and . 4 A common name used for the members of the Python community. The State of Python 3

2.2 Philosophy

Python is a high level dynamic programming language designed to ease the pro- grammer’s work, allowing him to spend most of the time with application prob- lems rather than dealing with the artifacts of the language. It also has some syntax constraints that force a more readable code, like rigid whitespace based indentation. It also aims to be an unambiguous language, meaning its code should be explicit about what it is doing, and also that there should be one, and only one, obvious way to do something. Python is a multi-paradigm language which means that the programmer has a wide range of paradigms to choose from when solving a problem, he could use object orientation, functional programming as well as aspect orientation or even design by contract. Another important goal of the language is its extensibility, integrating na- tively new C or C++ modules (as will be described in Sect. 2.5).

2.3 Language Features

The Python programming language combines the most common features of mod- ern programming languages, like modules, classes, exceptions, high-level dynamic data types and dynamic typing. Modules permit a better code organization for large projects and also open the road for continuous expansion of the language by adding new libraries with new functionalities. Classes add the power of Object Oriented Programming (OOP), like in C++ or Java, giving more flexibility and easing changes to programs. OOP is consid- ered to be easier to learn by newcomers and it leads to a better direct analysis, coding, understanding of complex situations and procedures than other program- ming methods. Exceptions are a programming construct designed to handle runtime errors or other problems which occur during the execution of a computer program. They help the programmer to deal with unexpected situations that could be harder to spot otherwise. Also, because exceptions should be exceptional, the language doesn’t force the programmer to catch every single thrown exception as Java does. Should an exception occur, the language assumes that the application is better off by immediately exiting and printing the exception stack trace. Python supports the common high-level programming language data types: basic types like integers, decimals, characters and strings; collection types like lists, tuples and dictionaries; and object types that can be dynamically created as complex structures of other data types. Dynamic typing is a feature that permits a variable to change type during the execution of the program. It’s possible to start using a variable as an integer number and later use it as a string. Dynamic typing is a key language feature and one of the greatest causes for the reduction of code length in the application comparing to other static typed languages such as C++ or Java. 4 F. Santos, N. Cerqueira, R. Barreira, R. Martins

2.4 Python Versus Other Programming Languages

It is not always easy to make a clear classification on how good a language is, when compared to others. Most of the comparisons available on the web are strongly biased because they’re written by “zealots”. Python has its own fair share of this. Like “VI versus Emacs”, “KDE versus Gnome” or “BSD versus ”, also Python has its own neverending debates titled “Python versus ”. Python, Perl and Ruby are similar when it comes to language power compar- ison. Some people might argue that language A has this feature which language B doesn’t, but in the end, they just have different ways of getting things done. Perl language emphasizes strings and regular expression manipulation, so it is expected that if you’re looking for these features, you’ll get you’re goals done easier by using Perl. This however doesn’t mean that it can’t be done in Python or Ruby. It simply is done in a different way. A common acceptance point is that Python code is easier to read than other languages. Because indentation is part of the language syntax, Python code generated in different projects always tends to look the same. This is undoubtedly a major maintainability asset for large projects and you can’t get this feature for free in other languages unless you impose well defined rules regarding the project’s code layout. It is also very usual to see Python compared to Java or C/C++ as far as performance is concerned. Well, it is a fact that Python runs slower than Java and much slower than C/C++. Its speed can be greatly increased using Psyco but you can’t expect to get the performance obtained with C/C++. Anyhow, the speed you are looking for when choosing Python is development speed. In average, the code developed in Python for the same set of functionalities, is 5 times smaller than in Java and up to 10 times compared to C/C++. A common approach is to develop a prototype of the desired application in Python, profile the application and optimize the parts that are slowing down the application in C/C++.

2.5 Interoperability with Other Languages

C/C++. Since Python programming language’s first and main implementation is written in C code, it is natural that C is the most supported language when it comes to bridging Python code with non-Python code. Many Python libraries are implemented in C code as external modules, and it is easy to implement new modules, which can interact with Python in virtually every way, including the construction of objects consisting of complicated data types. For this end, there is a C library which is easy to use and well documented. When there’s a need for calling pre-existing C/C++ functions from Python code (as opposed to building a C module specifically for Python), the easiest solution is probably to use SWIG (Simplified Wrapper and Interface Generator)[13]. SWIG works by reading the function declarations contained in C/C++ header files and use this information to generate Python wrapper code which handles the low-level details such as converting data types. The State of Python 5

Java. Python doesn’t directly support interoperability with the Java program- ming language, but there is a very good standalone implementation of the Python 2.1 programming language written in pure Java, which is called Jython[8]. Be- sides being a way to easily run Python on any platform which supports Java, it consistently integrates Python code with the JVM and all its resources, including the standard packages and it provides. In terms of running pure Python programs, it supports the whole core of the language, and a large majority of the standard Python library. It doesn’t yet support (or supports partially) some of the built-in extension modules which are, as of now, written in C and haven’t been ported to Java. However it supports calling those external modules through Java’s Native Interface (JNI). Jython has its origin in JPython[6], which was started in 1997 by Jim Hugunin. JPython has the distinct honor of having been the first existent Python imple- mentation besides the seminal CPython implementation, which, for a long time, was the only complete standard for the Python language. So JPython and later Jython have brought a number of advantages to the Python world:

– Portability, meaning the ability to run Python programs in any of the nu- merous platforms which can run Java; – Robustness, in the sense that the Java programming language enforces strict restrictions on the behavior of Java programs, making it virtually impossi- ble for typical runtime problems such as segmentation faults to affect a pro- gram’s execution – an understandable exception will be generated instead; – Simplicity of Python’s virtual machine implementation, since many of Python’s features map directly into Java features: automatic memory management, true exception handling and object-orientation.

In the near future, it’s expected that Jython will catch up with the current version of the Python language, since Jython’s developers are actively working on support for the latest features introduced in CPython 2.2 and 2.3.

.NET Framework. Exactly as in the case of Java, Python doesn’t directly support interoperability with the .NET framework. However, an implementa- tion analogous with JPython for Java already exists for this framework – Iron- Python[7], having been programmed by the same author of JPython, Jim Hugunin. Its main features, as highlighted by the author, are:

– Performance, having attained a 70% improvement in speed over CPython 2.3 on the standard Python benchmark (pystone); – Integration, since it enables Python code to directly call .NET’s libraries or extend classes from those libraries; – Fully dynamic, as it supports on-the-fly interpretation and compilation of static or dynamic code, as CPython does; – Managed and verifiable, as it generates code which fulfills the defined .NET requirements for both managed and verifiable code; 6 F. Santos, N. Cerqueira, R. Barreira, R. Martins

– Interoperability with other languages, since it can make Python interact with any of the languages supported by the .NET Common Language Runtime, which include C#, C++, Fortran, Java and many others. Jim Hugunin is currently working at Microsoft, where, besides other activities related to the .NET CLR, he’s continuing IronPython’s development.

2.6 Python 3000 Python 3000 is the name used by pythoneers for referring to the forthcoming Python version 3.0.0. For the first time in Python’s history, version 3.0 can break up compatibility with earlier versions in order to bring the language even closer to its philosophy. This is a major issue amidst the community and there has been great discussion about which features should be removed and which should be kept. Some core feature changes aim to remove ambiguity in the language. One example on this is the split of the division operator in two distinct operators. In actual python, the division operator returns a value based on the type of its arguments. For int or long arguments, a truncated mathematical division result is return, while for float arguments the the actual approximation of the mathematical division is return. Other changes correct minor problems or try to simplify/unify the way the language is used. The main reason why some old known issues about the lan- guage haven’t been corrected in new version releases is that they would raise compatibility issues.

3 Libraries

As far as programming languages are concerned, a library is the definition used to describe a bundle of classes/methods/functions which operate on one or more subjects, e.g., a library which handles all sort of geometric operations such as transformations and rotations on polygons. Nowadays, having a very complete set of libraries is more than a requirement for new programming languages: it is a standard. Python is no different in this aspect. The rich set of libraries available allow Python programmers to increase their productivity. Moreover, due to Python’s easy interface with other program- ming languages (as it was described in Sect. 2.5), some non-Python libraries can be easily used. In Python, libraries are composed of one or more modules which most fre- quently are Python source code files.

3.1 SciPy SciPy[10] is an open source library of tools for scientific programming. It’s quite broad, covering areas such as: graphics and plotting, optimization, calculus, sig- nal and image processing, genetic algorithms, statistics and linear algebra. The State of Python 7

SciPy is a very efficient library, since it relies a lot on fast C code to implement the most time-critical sections of code. Despite this fact, it’s still a very portable library, having been successfully used in other platforms such as SunOS and MAC OS X.

3.2 PIL

PIL[11] acronym stands for Python Imaging Library. It maintained by Secret Labs AB and comprises a powerful set of classes and methods to manipulate im- ages. Hiding all the complexity of different image formats and operations behind its API, PIL enables Python programmers to easily add image processing capa- bilities to their applications easily. As an example, the following code produces the transformations visualized in Fig. 1. import Image, ImageDraw monaImage = Image.open("mona_lisa.jpg") monaImage = monaImage.crop( (80, 0, 320, 373) ).resize( monaImage.size, Image.ANTIALIAS ) ImageDraw.Draw(monaImage).ellipse((132, 243, 167, 277), fill="red", outline= "darkred") ImageDraw.Draw(monaImage).ellipse((137, 250, 144, 258), fill="white" ) monaImage.show()

Fig. 1. Image processing using PIL.

3.3

Pygame[12] is a set of Python modules designed for writing games. It is built over the Simple DirectMedia Layer (SDL) library, with the intention of allowing real-time computer game development without the restraints and low-level me- chanics of the C programming language and its derivatives. This is based on the assumption that the most consuming functions inside games (mainly the graph- ics part) can be completely abstracted from the game logic in itself, making it possible to use a high-level programming language, like Python, or Perl to struc- ture the game. Pygame is highly portable and runs on nearly every platform and . 8 F. Santos, N. Cerqueira, R. Barreira, R. Martins

4 Frameworks

A framework can be seen as a library that takes active control over the global ap- plication flow. This means that programmers hook up their methods and classes to the framework, and it will take care of calling them in the proper time. In the late years, frameworks have become the trend of choice in . Because they tend to characterize domains of concepts and their functionalities, they provide great code reusability. Web frameworks are clearly the ones sprouting the fastest nowadays. More and more they minimize the code a programmer has to write to a fully functional dynamic website with a backend . In this section we will be focusing on some of the most important frameworks developed in Python.

4.1 PyUnit Unit testing is a procedure of automatic testing of software parts. It is currently accepted as a good practice in several Software Engineering processes, allowing easier refactoring and integration. PyUnit [9] is a framework that allows the use of unit testing in Python program development. It is a Python language version of the original JUnit for Java. An hint of its importance is given by the fact that it’s currently part of the Python standard library.

4.2 CherryPy CherryPy[1] is an open source web framework developed in Python. The project has been around for three years now and it’s currently in version 2.1.0. Its main feature is to allow programmers to develop web applications the same way they would when programming a normal object-oriented program. Since CherryPy is bundled up with its own multi-threaded web server, applications using it are standalone and can be deployed in any system that can run Python. The frame- work also supports sessions out of the box with multiple storage and handles static files, cookies, file uploads and everything required for web development. Along these years CherryPy has proven its stability and there are already several websites running atop it.

4.3 TurboGears ”TurboGears is the rapid web development megaframework you’ve been looking for.“ – from TurboGears[15] website.

If there’s a twilight zone of web frameworks, then TurboGears is definitively in it. There is great expectation in the Python community regarding the future release of TurboGears 1.0. For the time being, version 0.9 should give a nice preview of the project’s features. TurboGears uses AJAX5 and can be seen as 5 Asynchronous JavaScript and XML The State of Python 9 the Python version of the framework. It strives to make it easier and cleaner for web programmers to create new web applications. Instead of reinventing the wheel, TurboGears uses already stable libraries and frameworks and blends them together to achieve its goals. Each of these used components represents a layer in TurboGears’s architecture. Going from the user interface down to the database access layer it is composed by: – MochiKit — a well documented Javascript library needed for AJAX; – Kid — a designer/programmer friendly template engine; – CherryPy — a Python webserver that eases the input/output process; – SQLObject — to access the relational database in a object-oriented way. TurboGears code joins the components above together without obscuring them from the developer. A demonstration of this powerful mixture is provided in the form of a realtime video where the developer creates a Wiki step-by-step in 20 minutes using TurboGears[2]. Most recently, another similar screencast was produced, only this time around the goal was to build an online music store. Both videos are quite impressive and worthwhile watching. Even programmers who haven’t been in contact with TurboGears receive the message of how simple and clean is to get a web application working using the framework.

4.4 Pyro Pyro[3] (which stands for PYthon Remote Objects) is the leading Python frame- work for programming applications which need remote method invocation. It features a very powerful API, which is nevertheless designed to be easy to use. It resembles the one found in the quite well known Java RMI framework, which should make it easier for many programmers to use. Pyro’s main features are: – Written in 100% Python code, which means that the library will work on virtually every platform which can run Python; – Persistent (on-disk) naming service; – Mobile objects, which means that Pyro will transfer the objects’ bytecode through the network automatically if such an object is passed across via remote method invocation; – Remote exception passing support; – Very good documentation and example code.

4.5 wxPython wxPython[4] is an an open source cross-platform GUI toolkit for the Python programming language. It allows Python programmers to create programs with a robust, highly functional graphical user interface, simply and easily. It is im- plemented as a Python extension module (native code) that wraps the popular wxWidgets cross platform GUI library, which is written in C++. Currently sup- ported platforms are 32-bit , most Unix or unix-like systems, and Macintosh OS X. 10 F. Santos, N. Cerqueira, R. Barreira, R. Martins

5 Conclusions

Nowadays, Python is starting to pierce into the enterprise world and this comes as no surprise. Above all features, Python focuses on making the most out of the programmers time increasing their productivity. This is achieved by allowing the developer to focus on the application logic instead of continuously having to interrupt his reasoning dealing with language glitches. Python integrates easily with C/C++, Java and .NET making it possible for programmers to fine tune and speed up specially slow sections of code in their application, or even use features made available by such language/framework’s libraries. Moreover, Python has to offer a wide range of fully functional libraries and frameworks making development more about the ideas and less about how to translate them into code.

References

1. CherryPy Team. CherryPy Home Page, 2005. http://www.cherrypy.org/ [online] consulted on 10 Dec 2005. 2. Dangoor, K. The 20 minute wiki, 2005. http://www.turbogears.org/docs/wiki20/ [online] consulted on 10 Dec 2005. 3. de Jong, I. Pyro - about, 2005. http://pyro.sourceforge.net/ [online] consulted on 12 Dec 2005. 4. Dunn, R. wxPython, 2001. http://www.wxpython.org [online] consulted on 12 Dec 2005. 5. Geurts, L., Meertens, L., and Pemberton, S. The ABC Programmer’s Hand- book. Prentice-Hall, 1990. 6. Hugunin, J. Python and java: The best of both worlds, 1997. http://www.python.org/workshops/1997-10/proceedings/hugunin.html [online] consulted on 12 Dec 2005. 7. Hugunin, J. Ironpython home page, 2004. http://www.ironpython.org/ [online] consulted on 12 Dec 2005. 8. Jython Developers. Jython home page, 2005. http://www.jython.org/ [online] consulted on 12 Dec 2005. 9. Purcell, S. PyUnit - The Standard Unit Testing Framework for Python, 2001. http://pyunit.sourceforge.net/ [online] consulted on 27 Nov 2005. 10. SciPy Developers. Scipy scientific tools for python, 2005. http://www.scipy.org/ [online] consulted on 12 Dec 2005. 11. Secret Labs AB. Python Imaging Library (PIL), 2005. http://www.pythonware.com/products/pil/ [online] consulted on 27 Nov 2005. 12. Shinners, P. Pygame - Python Game Development, 2004. http://www.pygame.org [online] consulted on 12 Dec 2005. 13. SWIG Developers. Simplified wrapper and interface generator, 2005. http://www.swig.org/ [online] consulted on 12 Dec 2005. 14. The Python Software Foundation. The Python Software Foundation, 2005. http://www.python.org/psf/ [online] consulted on 6 Dec 2005. 15. TurboGears Team. Turbogears home page, 2005. http://www.turbogears.org/ [online] consulted on 10 Dec 2005. 16. van Rossum, G. The Python Language Reference Manual. Network Theory Ltd., 2003.