Just-in-time Compilation Techniques

Timo Lilja

January 25, 2010

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 1 / 19 Course arrangements

Course code: T-106.5800 Giving one presentation gives you 3 credits, two gives 5 credits Assignment: presentation and 4-page extended abstract Meetings on Mondays, 16-18 at T4 Mandatory attendance: 90% presence required

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 2 / 19 Introduction Contents

1 Introduction

2 Conclusion

3 Topics

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 3 / 19 Introduction Just-in-time compilation

A technique that compiles virtual machine byte code into native code during run-time Run-time compilation can give several benets Better information of the specic CPU (i.e., SSE2 instructions) Can use proling information to produce better compilation Better global optimizations even with dynamically linked libraries Can rearrange memory for better cache utilization System start-up can be slower than purely interpreted environments Usually only hot-spot code is JIT-compiled while the rest is only interpreted JIT compilation can be done step-by-step gradually improving hot-spot code

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 4 / 19 Introduction History

Earliest JIT compilation in McCarthy's 1960s Lisp systems Thompson's method to compile regular expressions to native code in 1960s LC2: the interpreted code was stored during execution works only for code that is executed Mixed code: program is a mixture of interpreted and native code Throw-away compiling: the end result is not stored permanently

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 5 / 19 Introduction Fortran

One of the rst hot-spot JIT compiler by Hansen in 1974 Each block of code maintained a frequency counter with which it was decided wheter to JIT compile the block The more frequent the execution of the block, the more ecient and time consuming optimizations were used Not always faster than statically compiled counterparts

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 6 / 19 Introduction Smalltalk

Presented by Schiman and Deutsch in 1984 Smalltalk virtual machine was slow, thus need for optimization JIT was one of the chosen optimization techniques Procedures were compiled lazily when they were entered Native code was thrown away and regenerated if the memory was run out

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 7 / 19 Introduction Self

Self is a dynamically typed prototype based high-level language akin to Javascript JIT environment presented in 1898 by Chambers In the rst generation generic functions' code was compiled only in specialized contexts Second generation: deferred compilation of uncommon cases but the system was much slower Third generation: Adaptive compilation were blocks will be compiled with better optimizations based on their execution frequency An observation: the most frequently executed code block is not necessary the best choice for optimization In an object-oriented language it might be worthwhile to JIT compile the method's caller and in-line the most frequently executed method In adaptive compilation there can be multiple versions of the same code executing at the same time

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 8 / 19 Introduction Slim binaries and Oberon

A program that is compiled to the least common subset of a processor architecture doesn't use all of the features available in run-time In 1994 Franz presented slim binaries which provide a high-level machine-independent description of a module Entire module was compiled at once instead of a method like in Smalltalk or Clean which resulted better code Fast code generation: code that could be reused was stored and used again instead of recompiling it Kistler 1999: continuous optimization were the code was optimized ad inntum used to optimize data access patterns for better cache locality Burger 1997: Scheme implementation which improved branch prediction and code locality

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 9 / 19 Introduction Staged compilation

At rst stage, the code is compiled to templates which are then in the second stage compiled to actual code Templates are specied by user annotations Used in some ML and C implementations

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 10 / 19 Introduction OCaml

Dynamically combined sequences of instructions as new virtual machine opcodes (i.e., macro opcodes) in 1999 by Rémy Reduced the overhead of instruction dispatch Provided optimization opportunities which would not have been possible if opcodes were xed Implementation didn't use the run-time execution information and was most suitable for low-level instruction sets where instruction dispatch was using considerable time

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 11 / 19 Introduction Simulation

Running native code of one architecture in another Binary translation: transferring a architectures' code into another during run-time These techniques used in most virtual machine nowadays Entire blocks are translated at once and the execution paths and corresponding control ow is considered when doing JIT compilation Execution is proled and only hot code (code that is run more often) is compiled Host and target machines can have the same architecture dynamic compilation provides opportunities for run-time optimizations Translation of legacy code to VLIW architectures provide better instruction level parallelism

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 12 / 19 Introduction Java

Cramer 1997: Only 68% of the execution time was used in interpretation Mere translation is not enough but optimizations are needed Statically used compiler optimization algorithms are too slow to be used in JIT compilation Various compilation strategies tried: no interpreter but everything is compiled, annotations, continuous compilation

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 13 / 19 Introduction Feedback directed code generation

JIT code generation where the optimization decisions are based on dynamically observed execution prole Inlining procedure calls Code layout organization to maximize code locality frequently executed code paths laid out contiguously Instruction scheduling maximize the ow of instructions through processor pipeline Multiversioning Compile several versions of a code sequence and choose the appropriate in run-time

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 14 / 19 Introduction Classication of JIT systems

According to Aycock, JITs can be classied to three categories 1 Invocation: explicit invocation is when the user invokes the JIT compilation manually. Automatic invocation is refered as implicit. 2 Executability: JITs involve source and target language. If they both can be run in the target system, then the system is polyexecutable, if only the target language is applicable, the system is refered as monoexecutable 3 Concurrency: if the JIT compilation can be run without stopping the system, the system is concurrent Hard real-time: an open question whether they constitute their own category Work on concurrent compilation is only beginning

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 15 / 19 Introduction JIT Tools

LLVM: compile to low-level virtual machine code which is JIT compiled later can optimized unnecessary static branches out Gnu lightning Lightweight library which compiles its own RISC-like language to native SPARC, x86 or PowerPC architectures Used by MzSchedme, Gnu Smalltalk and CLISP libJIT own intermediate representation and type system opcode form is close to static single assignment Used in DotGNU and some other dot-net implementations Using an existing tool forces the xing of the intermediate language and might not be suitable for all situations On the other hand, an error-prone JIT compiler implementation can be avoided

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 16 / 19 Conclusion Conclusion

Virtual machine architectures getting more common Without JIT VM would not be suciently fast even with the fastest hardware available JIT compilation allows one to use dynamic run-time information when optimizing the code Hard real-time requirements and concurrent compilation of JIT-code can be fruitful research topics

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 17 / 19 Conclusion References

Aycock, J. A Brief History of Just-in-Time. ACM Computing Surveys, Vol 35. No 2, June 2003 Arnold, M. A Survey of Adaptive Optimization in Virtual Machines. Proceeding of the IEEE, VOL 93. NO 2, February 2005

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 18 / 19 Topics Topics for presentation

Virtual machines and JIT by Arnold et al 2005 JIT and the programming languages How does the source language aect the JIT design? What about the target language? What are the eects of high-levelness of a language in JIT design? Overview of actual JIT usages: Java, JavaScript, Scheme, Python Comparison of JITs vs. interpretation vs. static compiling vs. binary translation (i.e., in QEMU) JIT tools LLVM JIT, JVM, Gnu lighting Future and advanced topics concurrent JIT, real-time, trace tree methods

Timo Lilja () Just-in-time Compilation Techniques January 25, 2010 19 / 19