Performance of Programs Written in Different Programming Languages Ryan Stansifer Florida Institute of Technology The Languages
• Ada (GNAT) • Haskell (GHC) • C (GNU) • Java • C++ (GNU) • Lua • C# (Mono) • Pascal • Fortran (GNU) • Python 2 • Go • Python 3 • Scala A is for Ada Ada is a structured, statically typed, imperative, high-level computer programming language, extended from Pascal and other languages. It has language support for design-by-contract, strong typing, explicit concurrency, offering tasks, synchronous message passing, protected objects, and non-determinism. C is for C C is a general-purpose, imperative computer programming language, supporting structured programming, lexical variable scope and recursion. C provides constructs that map efficiently to typical machine instructions. C is for C++ C++ is a middle-level programming language developed by Bjarne Stroustrup starting in 1979 at Bell Labs. C++ runs on a variety of platforms, such as Windows, Mac OS, and the various versions of UNIX. C is for C# C# is a general-purpose, object-oriented programming language. It was developed by Microsoft within its .NET initiative. F is for Fortran Fortran is an imperative programming language suited for numeric and scientific computation that has gone through many decades of revisions. F is for F# F# is developed by the F# Software Foundation, Microsoft and open contributors. An open source, cross-platform compiler for F# is available. F# is also a fully supported language in Visual Studio and Xamarin Studio. Other tools supporting F# development include Mono. F# originated as .NET implementation of a core of the OCaml programming language. G is for Go Go is an open source programming language created at Google in 2007. It is a compiled, statically typed language in the tradition of Algol and C, with garbage collection, memory safety features and CSP-style concurrent programming features added. H is for Haskell Haskell is a standardized, general-purpose purely functional programming language, with non-strict semantics and strong static typing. It is named after logician Haskell Curry. J is for Java Java is an imperative, object-oriented programming language designed as an improvement over C++. L is for Lua Lua (LOO-ə, from Portuguese: meaning moon) is a lightweight multi- paradigm programming language designed primarily for embedded systems and clients. O is for OCaml OCaml, originally known as Objective Caml, is the main implementation of the Caml programming language. OCaml is a free open source project managed and principally maintained by INRIA. Many newer languages have drawn elements from OCaml, most notably F# and Scala. P is for Pascal Pascal is a historically influential imperative and procedural programming language, designed in 1968–69 and published in 1970 by Niklaus Wirth as a small and efficient language intended to encourage good programming practices using structured programming and data structuring. P is for Python Python is a widely used high-level, general-purpose, interpreted, dynamic programming language. Its design philosophy emphasizes code readability, and its syntax allows programmers to express concepts in fewer lines of code than would be possible in other. R is for Rust Rust is a general-purpose, multi-paradigm, compiled programming language sponsored by Mozilla Research. It is designed to be a "safe, concurrent, practical language", supporting pure-functional, imperative- procedural, and object-oriented styles. S is for Scala Scala is object-oriented, and uses a curly-brace syntax reminiscent of the C programming language. Scala has many features of functional programming languages like Scheme, Standard ML and Haskell, including currying, type inference, immutability, lazy evaluation, and pattern matching. It also has an advanced type system supporting algebraic data types, covariance and contravariance, higher-order types, and anonymous types. Hardware on hc210-059044
Processors 8 Model name Intel(R) Core(TM) i7-4790 CPU @ 3.60GHz Cpu cores 4 Bogomips 7183.69 Hardware on ip-172-31-11-81 Amazon Cloud
Processors 2 Model name Intel(R) Xeon(R) CPU E5-2666 v3 @ 2.90GHz Cpu cores 1 Bogomips 5800.13 Versions on hc210-059044 Language Version Ada (Gnat) GNATMAKE GPL 2015 (20150428-49) C/C++/Fortran (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4 C# (Mono) Mono C# compiler version 3.2.8.0 Go go version go1.2.1 linux/amd64 Haskell (ghc) The Glorious Glasgow Haskell Compilation System, version 7.6.3 Java javac 1.8.0_91 Java(TM) SE Runtime Environment (build 1.8.0_91-b14) Java HotSpot(TM) 64-Bit Server VM (build 25.91-b14, mixed mode) Lua Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio Pascal (fpc) 2014/01/22 2.6.2-8 Python 2 Python 2.7.6 Python 3 Python 3.4.3 Scala Scala compiler version 2.12.0-M2 -- Copyright 2002-2013, LAMP/EPFL Scala code runner version 2.12.0-M2 -- Copyright 2002-2013, LAMP/EPFL Versions on ip-172-31-11-81 Language Version Ada (Gnat) GNATMAKE 4.6 C/C++/Fortran (Ubuntu 4.8.4-2ubuntu1~14.04.1) 4.8.4 C# (mono) Mono C# compiler version 3.2.8.0 Go go version go1.2.1 linux/amd64 Haskell (ghc) The Glorious Glasgow Haskell Compilation System, version 7.6.3 Java javac 1.8.0_66 Java(TM) SE Runtime Environment (build 1.8.0_66-b17) Java HotSpot(TM) 64-Bit Server VM (build 25.66-b17, mixed mode) Lua Lua 5.2.3 Copyright (C) 1994-2013 Lua.org, PUC-Rio Pascal (fpc) 2014/01/22 2.6.2-8 Python 2 Python 2.7.6 Python 3 Python 3.4.3 Scala Scala compiler version 2.11.7-M2 -- Copyright 2002-2013, LAMP/EPFL Scala code runner version 2.11.7-M2 -- Copyright 2002-2013, LAMP/EPFL Compiler Flags/Options Ada (Gnat) gnatmake –f is –O2 {exec} {file} C (clang) clang –o {exec} {file} C (GNU) gcc –std=gnul11 –O2 –w –o {exec} {file} C++ (GNU) g++ -O2 –w –o {exec} {file} C# (Mono) mcs –out:{exec} {file} Fortran (GNU) gfortran –O2 –o {exec} {file} Go go build –o {exec} {file} Haskell (ghc) ghc –fforce-recomp –O2 –outputdir ./build –o {exec} {file} Java javac –d ./build {file}; jar cevf {base} {exec} –C ./build java –jar {exec} Pascal (fpc) fpc –O2 –Sa –o{exec} {file} Scala scalac –deprecation –d {exec} {file} scala {exec} Methodology: Python wall-clock time
• Python script start_time = time.time() subprocess.call (
Tag Source Title Note balancing USACO, February 2016 Load Balancing O(n^3) / scan / Fenwick Trees (617,619,624) heart ACM ICPC SER, 2008 Heart of the County Iterative fixed point lawrence ACM ICPC SER, 2008 Lawrence of Arabia DP, precomputation, partial sums rare ACM ICPC World Finals, 1990 Rare Order Topological sort cheese ACM ICPC World Finals, 2015 Cutting Cheese Binary search, 3D geometry sort O(n log n) quadratic O(n^2) cubic O(n^3)
Results
Old Results Generally consistent even though completely different version and platform Time Compared with to C
512
256
128
64 Python3 Python2
32 Lua Haskell C# 16 Scala Pascal
8 Java
Ada Number of times Numbertimes of compared C to C 4 C++ Fortran
2
1
0.5 200 250 300 350 400 450 500 N, the size of the problem instance "Lawrence of Arabia" Time Compared to C for "Heart of the Country"
128
64
Haskell
Python3 32 Lua
Java
16 Fortran Python2
Scala
8 Ada
C++ times times the C program C# 4 C
Pascal
2
1
0.5 0 100 200 300 400 500 600 700 800 900 1000 "Heart of the Country" Ratio Program Time (seconds)
1.00 C gcc 1.73
1.00 Pascal Free Pascal #3 1.73
1.00 Rust 1.76
The Computer Language Benchmarks Game 1.10 Fortran Intel #3 1.92 pidigits 1.10 Lua #5 1.94
1.30 Racket #2 2.18
1.30 Python 3 #5 2.20
1.30 C++ g++ #3 2.29
1.30 Ada 2005 GNAT #2 2.29
1.30 C gcc #4 2.30
1.60 Haskell GHC #4 2.83
1.70 Go 2.85
1.70 Scala #4 2.97
1.80 Java #2 3.11
6.50 F# Mono #3 11.19 The Computer Language Benchmarks Game: Python 3 versus C (Gnu)
Benchmark Python C Ratio Pidigits 2.20 1.73 1.28 Regex-dna 10.58 2.46 4.30 Reverse-complement 3.11 0.62 5.02 K-nucleotide 76.50 12.53 6.11 Binary-trees 152.06 3.26 46.64 Mandlebrot 293.92 5.92 49.65 Fannkuch-redux 567.35 8.97 63.25 Fasta 111.44 1.36 81.94 N-body 923.74 9.56 96.63 Future Work
• F#, Rust, OCaml, … • Java 8 – streams • Haskell – lazy dynamic programming • Lazy versus eager functional programming • Clang • Compiler options On-line Repository
• A trusting person can repeat the experiment wget https://ser.cs.fit.edu/testing/time.sh | /bin/sh
• (This takes a very long time.) • This also downloads all the source code and all the test cases • John Clevenger, “Performance Analysis of Virtual vs. Physical Machines for Automated Judging in Programming Contests”, Competitive Learning Institute, 2015. • The Computer Language Benchmarks Game, http://benchmarksgame.alioth.debian.org/