Evaluating the Haxe Programming Language Performance comparison between Haxe and platform-specific languages Bachelor of Science Thesis in Software Engineering and Management STEPAN STEPASYUK YAVOR PAUNOV University of Gothenburg Chalmers University of Technology Department of Computer Science and Engineering Göteborg, Sweden, June 2013 The Author grants to Chalmers University of Technology and University of Gothenburg the non- exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Author warrants that he/she is the author to the Work, and warrants that the Work does not contain text, pictures or other material that violates copyright law. The Author shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Author has signed a copyright agreement with a third party regarding the Work, the Author warrants hereby that he/she has obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. Evaluating the Haxe Programming Language Performance comparison between Haxe and platform-specific languages STEPAN STEPASYUK, YAVOR PAUNOV. © STEPAN STEPASYUK, June 2013. © YAVOR PAUNOV, June 2013. Examiner: RICHARD TORKAR University of Gothenburg Chalmers University of Technology Department of Computer Science and Engineering SE-412 96 Göteborg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Göteborg, Sweden June 2013 Evaluating the Haxe Programming Language Performance comparison between Haxe and platform-specific languages Stepan Stepasyuk Yavor Paunov Gothenburg University Gothenburg University Department of Computer Science and Engineering Department of Computer Science and Engineering Gothenburg, Sweden Gothenburg, Sweden [email protected] [email protected] Abstract 2. Background The goal of this paper is to evaluate the performance of Haxe, a cross-platform programming language. By applying an experi- 2.1 Haxe mental method and a statistical test, we measure and analyze the 2 performance associated with compiling Haxe source code to five Haxe is a strictly typed object-oriented language whose standard of its targets. We find a significant difference in performance libraries are licensed under the “two-clause” BSD license while the compiler is under GPLv2+ in order to “keep it free open soft- between Haxe-compiled and target-specific language code on four 3 out of five targets, two of which in favour of Haxe and two ware” . Advertised mainly for its multiplatform nature Haxe against it. Our findings are useful for developers who are consid- compiles to a number of different programming platforms, includ- ering adopting the language with the intent of replacing their ing JavaScript, Flash, PHP, C++, and NekoVM and provides current development toolbox with it. experimental support for C# and Java. The practical implications of this are that Haxe code can be compiled for applications run- Keywords: evaluation, performance, language design, experi- ning on desktop, mobile and web platforms. Haxe comes with a ment, statistical testing standard library that can be used on all supported targets and platform-specific libraries for each of them. 1. Introduction 2.1.1 Multiplatform The recent years have seen a growing demand for cross-platform development solutions1, especially in the mobile sector where new The biggest selling point of Haxe is its multiplatform nature, a platforms come out rather often and a big part the market is frag- noticeable benefit of which is saving the trouble of switching mented. In order to reach a wider audience, applications have to languages. For example, in a web-development project where target multiple platforms and thus must be developed multiple PHP is used for back-end and JavaScript for front-end. By using times. This requires more effort from the developers and creates Haxe, it is possible to develop for both platforms using the same additional development and maintenance costs [1]. Tools that language. simplify this process already exist in great numbers and focus on Moreover, Haxe provides features missing in the languages of different domains such as web, desktop and mobile. some of the target platforms. For instance, it provides enumera- One such tool is Haxe, a language with the ability to compile tions that are missing from ActionScript 3 and PHP and type to a number of different platforms. Although not a complete safety missing in loosely typed languages such as PHP and JavaS- cross-platform solution by itself, Haxe can greatly ease the pro- cript. cess. For instance, platform-specific code still needs to be written but conditional compilation makes an abstraction layer easy to 2.1.2 Comprehensive type system implement. Haxe can be useful when targeting a single platform, The Haxe type system comes with several different groups of as well because many of its features are lacking on its targets types. For example, basic types like class or enum. The class type This paper aims to test how the performance of programs is similar to that in other object-oriented languages. It supports compiled from Haxe compares to code written in the platform- inheritance and can implement interfaces. The enum type which specific languages. This led to the following research question: works in a similar way to that in Java. Is there a significant overhead related to compiling Haxe source Haxe also allows for more advanced types like the anonymous code? structure that represents a list of fields, which do not refer to a specific class. Structures can be made reusable by using the The targets used during the testing were Flash, C++, PHP, Ja- vaScript, and Java. The architecture and the inner workings of the typedef keyword. Haxe is a strictly typed language. However, there are several compiler, although of potential interest for a future study, are not ways to get around compile-time type checking. One of them is by within the scope of the paper. using the untyped keyword, which disables type checking for the specified block of code. Another option is to use the Dynamic 2 1http://software.intel.com/en-us/blogs/2013/03/07/cross-platform- http://haxe.org/doc/intro [Accessed: 25 May 2013] development-what-the-stats-say [Accessed: 25 May 2013] 3 http://haxe.org/doc/license [Accessed: 25 May 2013] type when declaring variables. A variable of the Dynamic type 3.1 Experimental setup can hold a value of any other type. Moreover, an infinite number The experimental consisted of comparing the execution time of of fields can be added to a Dynamic variable. All of those will Haxe-compiled and platform-specific code on the following tar- also be Dynamic unless a type parameter is given, in which case gets: all fields will be of the specified type. Flash 2.1.3 Code embedding C++ It is possible to use code written in some of the targeted languages Java while writing Haxe. This can be done with JavaScript, PHP, C# and Java. It is important to note that since C# and Java are exper- PHP imental targets, code embedding for these targets is discouraged4. JavaScript Code embedding can be achieved by utilizing Haxe magic func- tions. For example, the functions __js__ and __php__ will embed The Mann–Whitney U test [6] was used in order to distinguish the presence of significant statistical difference between the re- JavaScript and PHP code respectively. Using external libraries on sults. This particular test was selected due to the nature of data to the other targets is possible as well but not as straightforward and requires certain workarounds5,6. be compared. Samples of execution times are independent since execution time of one implementation does not affect the execu- tion time of another in any way. Moreover, the obtained data is of 2.1.4 Compiler metadata a rational level9 because it can be ranked (from shortest to longest Haxe allows metadata annotation using the @ sign, in which it time of execution and vice versa). It is meaningful because we can shares similarity to that of Java. What makes metadata in Haxe compare two given executions and find out which one was the powerful is that under certain circumstances it can affect the way quickest. It also has an absolute minimum of 0 seconds execution the compiler works. In order to mark compiler metadata, a colon time, and it is possible to calculate ratios based on it. However, character (‘:’) is added in front of the identifier. A number of the absolute minimum is a subject for argument since no execu- general identifiers can be used on all platforms in addition to a tion time can be exactly 0 seconds in practice. It can be 0.01 or few platform-specific ones. 0.000001 and the amount of zeros between the decimal point and Examples of platform independent identifiers include the 1 can span infinitely. On the other hand, no algorithm can execute @:final annotation, preventing a class from being further extend- in less than 0 seconds, for instance -0.1 seconds. ed. This is similar to the final keyword in ActionScript 37 and The performed test was two-tailed which means that finding Java8 , providing the same feature. Although, Haxe itself does not the presence of significant statistical difference and not its direc- support method overloading, the @:overload identifier allows tion is the main goal of the test [6]. Thus, hypotheses were formu- overloading of external methods. Other examples of compiler lated accordingly. Null hypothesis was stating that overhead in metadata are the @:macro and @:build identifiers, used to create performance of compiling from Haxe is statistically significant in macros. comparison to compiling from platform-specific languages.