Bachelor Degree Project WebAssembly vs. its predecessors A comparison of technologies Author: Stefan Fredriksson Supervisor: Jesper Andersson Semester: VT 2020 Subject: Computer Science Abstract For many years it has only been HTML, CSS, and JavaScript that have been native to the Web. In December 2019, WebAssembly joined them as the fourth language to run natively on the Web. This thesis compared WebAssembly to the technologies ActiveX, Java applets, Asm.js, and Portable Native Client (PNaCl) in terms of their performance, security, and browser support. The reason why this was an interesting topic to investigate was to determine in what areas WebAssembly is an improvement over previous similar technologies. Another goal was to provide companies that still use older technologies with an indication as to whether or not it is worth upgrading their system with newer technology. To answer the problem, the thesis mainly focused on comparing the performance of the technologies through a controlled experiment. The thesis also aimed at getting a glimpse of the security differences between the technologies by the use of a literature study. The thesis showed that PNaCl was the technology with the best performance. However, WebAssembly had better browser support. Also, PNaCl is deprecated while WebAssembly is heavily supported and could potentially be further optimized. Keywords: WebAssembly, wasm, ActiveX, Java applet, applet, Asm.js, Portable Native Client, PNaCl, Performance, Security, Browser support, Dynamic Web Preface I would like to thank my supervisor during this thesis, Jesper Andersson, for guiding me and coming up with ideas I would not have had without him. I would also like to thank my peers who reviewed the thesis during its different stages. Contents List of Figures List of Tables List of Listings 1 Introduction1 1.1 Background.................................1 1.2 Related work................................2 1.3 Problem formulation............................3 1.4 Objectives..................................4 1.5 Scope/Limitation..............................4 1.6 Target group.................................4 1.7 Outline...................................4 2 Method5 2.1 Scientific Approach.............................5 2.1.1 Literature study...........................5 2.1.2 Controlled Experiment.......................5 2.2 Reliability and Validity...........................6 3 Dynamic web performance7 3.1 Dynamic web................................7 3.2 Technologies server-side..........................8 3.3 Technology overview............................8 3.3.1 Java Applets............................8 3.3.2 ActiveX...............................9 3.3.3 Asm.js............................... 10 3.3.4 Portable Native Client....................... 11 3.3.5 WebAssembly........................... 12 3.4 Performance Testing............................ 12 4 Data Collection 14 4.1 Design.................................... 14 4.1.1 Design for collecting performance data.............. 14 4.1.2 Design for collecting data on security............... 17 4.2 Performance experiment preparations................... 18 4.2.1 Browser settings.......................... 18 4.2.2 Compilation tools.......................... 18 4.3 Performance test execution......................... 19 4.3.1 Readying machine for testing................... 19 4.3.2 Running the tests.......................... 20 4.3.3 Hardware and Software....................... 20 5 Results 22 5.1 Performance experiment.......................... 22 5.1.1 Result explanation......................... 22 5.1.2 Fibonacci application........................ 22 5.1.3 Array application.......................... 25 5.1.4 Numeric application........................ 28 5.2 Qualitative results.............................. 31 5.2.1 Security............................... 31 5.2.2 Browser support.......................... 34 6 Analysis 35 6.1 Performance experiment.......................... 35 6.1.1 Execution time........................... 35 6.1.2 Load time.............................. 38 6.1.3 CPU usage............................. 41 6.1.4 Memory usage........................... 44 6.2 Qualitative results.............................. 47 6.2.1 Security............................... 47 6.2.2 Browser support.......................... 48 7 Discussion 49 7.1 Execution time & Load time........................ 49 7.2 CPU & RAM Usage............................ 50 7.3 Security................................... 50 7.4 Browser support............................... 51 7.5 Summary.................................. 51 8 Conclusions & Future work 53 References 55 A Appendix 1A B Appendix 2B C Appendix 3C List of Figures 3.1 High-level flow of a server-side scripting application............7 3.2 High-level flow of a client-side scripting application............8 3.3 High-level flow of a Java applet application.................9 3.4 High-level flow of an ActiveX application.................. 10 3.5 High-level flow of an Asm.js application.................. 11 3.6 High-level flow of a Portable Native Client application........... 11 3.7 High-level flow of a WebAssembly application............... 12 4.1 Design of the test process flow........................ 17 5.1 Execution times of the Fibonacci application................ 23 5.2 Load times of the Fibonacci application................... 23 5.3 CPU usage of the Fibonacci application running on the desktop...... 24 5.4 CPU usage of the Fibonacci application running on the laptop....... 24 5.5 Memory usage of the Fibonacci application running on the desktop.... 25 5.6 Memory usage of the Fibonacci application running on the laptop..... 25 5.7 Execution times of the Array application.................. 26 5.8 Load times of the Array application..................... 26 5.9 CPU usage of the Array application running on the desktop........ 27 5.10 CPU usage of the Array application running on the laptop......... 27 5.11 Memory usage of the Array application running on the desktop...... 28 5.12 Memory usage of the Array application running on the laptop....... 28 5.13 Execution times of the Numeric application................. 29 5.14 Load times of the Numeric application................... 29 5.15 CPU usage of the Numeric application running on the desktop....... 30 5.16 CPU usage of the Numeric application running on the laptop........ 30 5.17 Memory usage of the Numeric application running on the desktop..... 31 5.18 Memory usage of the Numeric application running on the laptop...... 31 6.1 Box plot showing the variance of execution times for the different tech- nologies running the Fibonacci application................. 36 6.2 Box plot showing the variance of execution times for the different tech- nologies running the array application.................... 37 6.3 Box plot showing the variance of execution times for the different tech- nologies running the numeric application.................. 38 6.4 Box plot showing the variance of load times for the different technologies running the Fibonacci application...................... 39 6.5 Box plot showing the variance of load times for the different technologies running the array application......................... 40 6.6 Box plot showing the variance of load times for the different technologies running the numeric application....................... 41 6.7 Box plot showing the variance of CPU usage for the different technologies running the Fibonacci application...................... 42 6.8 Box plot showing the variance of CPU usage for the different technologies running the array application......................... 43 6.9 Box plot showing the variance of CPU usage for the different technologies running the numeric application....................... 44 6.10 Box plot showing the variance of memory usage for the different tech- nologies running the Fibonacci application................. 45 6.11 Box plot showing the variance of memory usage for the different tech- nologies running the array application.................... 46 6.12 Box plot showing the variance of memory usage for the different tech- nologies running the numeric application.................. 47 C.1 Execution times of the Fibonacci application showing the execution time per browser..................................C C.2 Load times of the Fibonacci application showing the load time per browser.C C.3 Execution times of the Array application showing the execution time per browser....................................D C.4 Load times of the Array application showing the load time per browser..D C.5 Execution times of the Numeric application showing the execution time per browser..................................E C.6 Load times of the Numeric application showing the load time per browser.E List of Tables 4.1 Keywords used during the literature study.................. 18 4.2 Specification of the machines that the performance tests were executed on. 20 4.3 The versions of the software used in this study............... 21 5.1 Browser support for each technology.................... 34 Listings 1 Installing the Emscripten SDK.......................A 2 WebAssembly compilation command for the array application......A 3 Makefile of the array PNaCl application..................A 4 Installing the NaCl SDK..........................B 5 Make.bat file content of a PNaCl application................B 6 PNaCl compilation commands.......................B 7 Installing curl for Visual Studio 2017....................B This thesis is the final product of a bachelor degree