A Concurrent Trace-based Just-In-Time Compiler for Single-threaded JavaScript Jungwoo Ha Mohammad R. Haghighat Shengnan Cong Department of Computer Software Solution Group Software Solution Group Sciences Intel Corporation Intel Corporation The University of Texas at Austin [email protected] [email protected] [email protected] Kathryn S. McKinley Department of Computer Sciences The University of Texas at Austin [email protected] Abstract design also improves throughput by 6% on average and JavaScript is emerging as the ubiquitous language up to 34%, because it delivers optimized application of choice for web browser applications. These ap- code faster. This design provides a better end-user ex- plications increasingly execute on embedded mobile perience by exploiting multicore hardware to improve devices, and thus demand responsiveness (i.e., short responsiveness and throughput. pause times for system activities, such as compilation Categories and Subject Descriptors D.3.4 [Program- and garbage collection). To deliver responsiveness, web ming Languages]: Processors—Incremental compilers, browsers, such as Firefox, have adopted trace-based code generation. Just-In-Time (JIT) compilation. A trace-based JIT re- stricts the scope of compilation to a short hot path of General Terms Design, Experimentation, Performance, instructions, limiting compilation time and space. Al- Measurement though the JavaScript limits applications to a single- Keywords Just-In-Time Compilation, Multicore, Con- thread, multicore embedded and general-purpose ar- currency chitectures are now widely available. This limitation presents an opportunity to reduce compiler pause times 1. Introduction further by exploiting cores that the application is guar- JavaScript is emerging as the scripting language of anteed not to use. While method-based concurrent JITs choice for client-side web browsers [10]. Client-side have proven useful for multi-threaded languages such JavaScript applications initially performed simple HTML as Java, trace-based JIT compilation for JavaScript of- web page manipulations to aid server-side web applica- fers new opportunities for concurrency. tions, but they have since evolved to use asynchronous This paper presents the design and implementa- and XML features to perform sophisticated, interactive tion of a concurrent trace-based JIT that uses novel dynamic content manipulation on the client-side. This lock-free synchronization to trace, compile, install, and style of JavaScript programming is called AJAX (for stitch traces on a separate core such that the interpreter Asynchronous JavaScript and XML). Companies, such essentially never needs to pause. Our evaluation shows as Google and Yahoo, are using it to implement inter- that this design reduces the total, average, and maxi- active desktop applications such as mail, messaging, mum pause time by 89%, 97%, and 93%, respectively and collaborative spreadsheets, word processors, and compared to the base single-threaded JIT system. Our calendars. Because Internet usage on mobile platforms 47 is growing rapidly, the performance of JavaScript is concurrently with the interpreter reducing pause times critical for both desktops and embedded mobile de- to nearly zero. vices. To speed up the processing of JavaScript appli- Our mechanism piggybacks a single word, called cations, many web browsers are adopting Just-In-Time the compiled state variable (CSV), on each trace, us- (JIT) compilation, including Firefox TraceMonkey [5], ing it as a synchronization variable. Comparing with Google V8 [11], and WebKit SFE [19]. CSV synchronizes all of the compilation actions, in- Generating efficient machine code for dynamic lan- cluding checking for the native code, preventing dupli- guages, such as JavaScript, is more difficult than for cate traces, and allowing the interpretation to proceed, statically typed languages. For dynamic languages, the without using any lock. compiler must generate code that correctly executes We introduce lock-free dynamic trace stitching in all possible runtime types. Gal et al. recently intro- which the compiler patches new native code to the ex- duced a trace-based JIT compilation for dynamic lan- isting code. Dynamic trace stitching prevents the com- guages to address this problem and to provide respon- piler from waiting for trace stitching while the inter- siveness (i.e., low compiler pause times and memory preter is executing the native code, and reduces the po- requirements) [7]. Responsiveness is critical, because tential overhead of returning from native code to the JavaScript runs on client-side web browsers. Pause interpreter. times induced by the JIT must be short enough not to We implement our design in the open source Tamar- disturb the end-user experience. Therefore, Gal et al.’s inTracing VM, and evaluate our implementation us- system interprets until it detects a hot path in a loop. ing the SunSpider JavaScript benchmark suite [20] on The interpreter then traces, recording instructions and three different hardware platforms. The experiments variable types along a hot path. The JIT then special- show that our concurrent trace-based JIT implemen- izes the trace by type and translates it into native code tation reduces the total pause time by 89%, the max- in linear time. The JIT sacrifices code quality for linear imum pause time by 93%, and the average pause time compile times, rather than applying heavy weight op- by 97% on Linux. Moreover, the design improves the timizations. This trace-based JIT provides fast, light- throughput by an average of 6%, with improvements up weight compilation with a small memory footprint, to 34%. Our concurrent trace-based JIT virtually elim- which make it suitable for resource-constrained de- inates compiler pause times and increases application vices. throughput. Because tracing overlaps with compilation, On the hardware side, multicore processors are pre- the interpreter prepares the trace earlier for subsequent vailing in embedded and general purpose systems. The compilation, thus the JIT delivers the native code more JavaScript language however lacks a thread model, and quickly. This approach also opens up the possibility of thus all JavaScript applications are single-threaded. increasing the code quality with compiler optimizations This limitation provides the opportunity to perform the without sacrificing the application pause time. JIT and other VM services concurrently on another core, transparently to the application, since the appli- 2. Related Work cation is guaranteed not to be using it.Unfortunately, Gal et al. proposed splitting trace tree compilation steps state-of-the-art trace-based JIT compilers are sequen- into multiple pipeline stages to exploit parallelism [6]. tial [7, 5, 18], and have not exploited concurrency to This is the only work we can find seeking parallelism improve responsiveness. in the trace-based compilation. There are a total of 19 In this paper, we present the design and implemen- compilation pipeline stages, and each pipeline stage tation of a concurrent trace-based JIT compiler for runs on a separate thread. Because of data dependency JavaScript that combines responsiveness and through- between each stage and the synchronization overhead, put for JavaScript applications. We address the syn- the authors failed to achieve any speedup in compila- chronization problem specific to the trace-based JIT tion time. We show having a parallel compiler thread compiler, and present novel lock-free synchronization operating on an independent trace provides more bene- mechanisms for wait-free communication between the fit than pipelining compilation stages. With proper syn- interpreter and the compiler. Hence, the compiler runs chronization mechanisms, our work successfully ex- ploited parallelism in the trace-based JIT by allowing 48 tracing to happen concurrently with the compilation, byte code native code even when only one compiler thread was used. trunk trunk trace Kulkarni et al. explored maximizing throughput of trace trunk trace guard background compilation by adjusting the CPU utiliza- guard side tion level of the compiler thread [15]. This technique exit side exit is useful when the number of application threads ex- branch trace ceeds the number of physical processors and the com- branch trace piler thread cannot fully utilize a processor resource. They conducted their evaluation on method-based com- after trunk trace after branch trace pilation, though the same technique can be applied to compilation compilation trace-based compilation. However, because JavaScript Figure 1. Byte code and native code transition in the is single-threaded, it is less likely that all the cores are trace-based JIT. Initially, the interpreter interprets on fully utilized in today’s multicore hardware. Hence, the the byte code. First detected hot path (thick path) effect of adjusting CPU usage levels will not be as sig- is traced forming a trunk trace. Following hot paths nificant as it is in multi-threaded Java programs. guarded and installed in a side-exit. The compiler at- A number of previous efforts have sought to re- tach the branch trace, which begins from the hot side- duce compilation pause time in method-based JIT. exit to the loop header, to the trunk trace. SELF-93 VM introduced adaptive compilation strate- gies for minimizing application pause time [13]. When ically. Furthermore, the type of JavaScript variables a method is invoked for the first time, the VM compiles can change over time as