Iowa State University Capstones, Theses and Creative Components Dissertations Spring 2020 Benchmarking AssemblyScript for Faster Web Applications Nischay Venkatram Follow this and additional works at: https://lib.dr.iastate.edu/creativecomponents Part of the Numerical Analysis and Scientific Computing Commons, Programming Languages and Compilers Commons, and the Software Engineering Commons Recommended Citation Venkatram, Nischay, "Benchmarking AssemblyScript for Faster Web Applications" (2020). Creative Components. 558. https://lib.dr.iastate.edu/creativecomponents/558 This Creative Component is brought to you for free and open access by the Iowa State University Capstones, Theses and Dissertations at Iowa State University Digital Repository. It has been accepted for inclusion in Creative Components by an authorized administrator of Iowa State University Digital Repository. For more information, please contact [email protected]. Benchmarking AssemblyScript for Faster Web Applications by Nischay Venkatram A report submitted to the graduate faculty in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Major: Computer Engineering Program of Study Committee: Joseph Zambreno, Major Professor The student author, whose presentation of the scholarship herein was approved by the program of study committee, is solely responsible for the content of this dissertation/thesis. The Graduate College will ensure this dissertation/thesis is globally accessible and will not permit alterations after a degree is conferred. Iowa State University Ames, Iowa 2020 Copyright c Nischay Venkatram, 2020. All rights reserved. ii TABLE OF CONTENTS Page LIST OF FIGURES . iii ACKNOWLEDGMENTS . iv ABSTRACT . .v CHAPTER 1. Introduction . .1 CHAPTER 2. Related Work . .2 2.1 Javascript Performance . .2 2.2 Webassembly . .3 2.3 Benchmarks . .3 CHAPTER 3. Methodology . .5 3.1 Research Questions . .5 3.2 Benchmark Selection . .5 3.2.1 Breadth First Search . .6 3.2.2 Fast Fourier Transform . .6 3.2.3 Lower Upper Decomposition . .7 3.2.4 Sparse Matrix Vector Multiplication . .7 3.2.5 Page Rank . .7 3.3 Experimental Platform . .7 3.3.1 Setup . .7 3.3.2 Implementation . .7 3.3.3 Execution . .9 CHAPTER 4. Results . 10 4.1 Javascript vs Assemblyscript . 10 4.2 Key Insights . 12 CHAPTER 5. Conclusion and Future Work . 16 BIBLIOGRAPHY . 17 iii LIST OF FIGURES Page 4.1 Average Runtime . 11 4.2 Breadth First Search . 11 4.3 Fast Fourier Transform . 12 4.4 Lower Upper Decomposition . 13 4.5 Page Rank . 14 4.6 Sparse Matrix Vector Multiplication . 14 iv ACKNOWLEDGMENTS Firstly, I would like to thank my parents and family for believing in me and being a constant source of motivation throughout my academic and personal endeavors. I would also like to thank Dr. Zambreno for his guidance and support throughout the research and writing of this paper. I am genuinely grateful to everyone who has helped and inspired me in any way throughout my career as a student and a software engineer. v ABSTRACT As web applications are becoming increasingly complex, it is crucial now more than ever to be able to develop web apps with an emphasis on performance to ensure a responsive and smooth user experience. Since the introduction of Webassembly as a compilation target for the web, the promise of writing programs that can run at native speed seemed revolutionary in theory. But the real world performance benefits of Webassembly in comparison to Javascript is not clearly understood. This paper evaluates the current performance of Assemblyscript - a strict subset of TypeScript that compiles to Webassembly, and Javascript in the areas of numerical computing across multiple browsers. A set of benchmarks were developed in Assemblyscript that includes numerical computing problems from the Ostrich Benchmark suite. The tests were executed across Chrome and Firefox. After studying the results from the benchmarks that were created, we find that Assemblyscript demonstrates speedups that range between 1.1-7.2x. It is also noticed that writing idiomatic Typescript can slow down Assemblyscript in certain scenarios. In conclusion, this study suggests that Assemblyscript (and Webassembly) provides far more consistent and predictable performance in comparison to Javascript. 1 CHAPTER 1. Introduction In the early 90's, web pages were static and did not support any dynamic behavior after a page was loaded in the browser. To enable scripting on the web, Brendan Eich and the Netscape team developed Javascript in 1995 [1]. This allowed developers to make web pages dynamic and perform simple tasks like form validation or HTML manipulation in the browser, without requiring any interaction with the server. Since then, Javascript has become one of the most prominent languages in the software engineering world. It is used across multiple different platforms and devices, and for a plethora of different applications. Over the past two decades, these applications have become incredibly complex and to ensure that users have a good experience, performance has become a key factor. Continuous performance improvements to browser engines and compilers eventually led to the creation of Webassembly - a new binary format that is secure, portable, and serves as a compilation target for typed languages like C/C++ and Rust, allowing faster computation times than Javascript [2]. This would allow developers to write the computationally intensive parts of an application in languages like C and compile it to Webassembly for faster performance. But software engineers predominantly working with Javascript may not have the expertise in languages like C/C++ and Rust and it is not ideal to learn another language to harness the benefits of Webassembly. This is why Assemblyscript was developed [3]. It is a strict subset of Typescript that compiles down to Webassembly, allowing developers to improve the performance of their application without having to become proficient in another language. Since the benchmarks for analyzing the performance of Assemblyscript for computationally intensive tasks are limited, that will be the contribution of this paper. In this study, five scientific computing problems will be used to measure and analyze the relative performance of Webassembly (compiled from Assemblyscript) and Javascript. 2 CHAPTER 2. Related Work Since its inception in 1995 to be used as glue code within HTML, Javascript has become a widespread language being used across client side web applications, server side applications, mobile applications, and even desktop applications. 2.1 Javascript Performance The increasing use of Javascript has prompted web browser developers to make significant improvements in Javascript compilers and engines over the years [4,5]. Modern browser engines make use of JIT (Just-in-time) compilers [6,7] to achieve optimizations while executing Javascript. In 2014, the performance of these engines was recorded to be close to 1.5-2 times that of native code [8]. The continuous evolution and improvement of these engines has made it possible for developers to build more complex applications in Javascript. Browsers are being used for computationally intensive tasks like numerical computing, cryptocurrency mining, and even machine learning and AI applications [9,10]. The web has also become a target for graphics and game development over the last two decades due to the advancements made in browser engines [11{13]. Since providing users with a seamless user experience in any web application is crucial, research has been done to analyze and benchmark the performance of different graphics rendering techniques [14{17]. Additional efforts have been made to improve the performance of intensive Javascript applications and to avoid common pitfalls [18, 19]. A new specification that allowed Javascript as a compilation target for low level languages like C and C++ called asm.js emerged [20]. It is intended to have performance characteristics closer to native code. This was possible because it was a strict subset of Javascript, which allowed for ahead of time optimizations. Despite the performance improvements, asm.js inherits various Javascript issues like inconsistent performance, and overhead from parsing and 3 compiling the source. This is because asm.js needs to be transpiled to Javascript using a source- to-source compiler to run on the web. 2.2 Webassembly All these efforts eventually led to the development of Webassembly which serves as a compilation target for typed languages like C/C++ and Rust with the intention of providing near native speeds [2]. Currently, the Emscripten toolchain built using LLVM [21, 22] compiles C and C++ to Webassembly, essentially enabling developers to run C and C++ in browsers. Binaryen is another compiler that allows developers to compile Assemblyscript to Webassembly [3,23]. Similar compilers exist to cross compile Javascript or Typescript to Webassembly [24]. Webassembly is already faster and smaller than asm.js, and will only improve with features like multi-threading and SIMD support in development [25{27]. 2.3 Benchmarks To properly understand the performance benefits of using Webassembly, existing or new bench- marks will have to be utilized. Octane, Kraken, and Speedometer are suites provided by browser vendors to test the performance of browser engines [28{30]. Octane is no longer being maintained. In addition to Speedometer, the Webkit team has also created JetStream2 [30], a suite that also contains some benchmark tests on Webassembly. JSBench [31] suggests that benchmarks may not accurately measure the performance of real world applications and instead introduces a technique to generate benchmarks based on popular websites that match the behavior of real world applications. This was one of the major reasons why the Chrome V8 team retired octane, as micro-benchmarks were not a good representation of how browsers will perform while running modern web applica- tions [28]. The Ostrich benchmark suite developed in the Sable lab at McGill University contains a set of 12 numerical computing problems [32] implemented in Javascript and C. Another bench- mark suite was developed [33] using the Ostrich suite to compare the performance of Webassmebly compiled from C and Javascript.