The Java HotSpot VM Under the Hood Tobias Hartmann Compiler Group – Java HotSpot Virtual Machine Oracle Corporation May 2018 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. | About me • Software engineer in the HotSpot JVM Compiler Team at Oracle – Based in Baden, Switzerland • Master’s degree in Computer Science from ETH Zurich • Worked on various compiler-related projects – Currently working on future Value Type support for Java Copyright © 2018, Oracle and/or its affiliates. All rights reserved. Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 3 Outline • Intro: Why virtual machines? • Part 1: The Java HotSpot VM – JIT compilation in HotSpot – Tiered Compilation • Part 2: What's new in Java – Segmented Code Cache – Compact Strings – Ahead-of-time Compilation – Value Types Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 4 A typical computing platform User Applications Application Software Java SE Java EE Java Virtual Machine System Software Operating system Hardware Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 5 A typical computing platform User Applications Application Software Java SE Java EE Java Virtual Machine System Software Operating system Hardware Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 6 A typical computing platform User Applications Application Software Java SE Java EE Java Virtual Machine System Software Operating system Hardware Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 7 Programming language implementation Programming language C Compiler CompilerCompiler Standard libraries StandardCompiler libraries Language StandardStandard libraries libraries implementation Debugger DebuggerDebugger Memory management MemoryDebugger management MemoryMemory management management Operating Windows Linux system LinuxSolaris Hardware Intel x86 Intel x86 ARMSPARC Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 8 (Language) virtual machine Programming language Java JavaScript Scala Python Virtual machine HotSpot VM Operating Windows Linux Mac OS X Solaris system Hardware Intel x86 PPC ARM SPARC Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 9 Outline • Intro: Why virtual machines? • Part 1: The Java HotSpot VM – JIT compilation in HotSpot – Tiered Compilation • Part 2: What's new in Java – Segmented Code Cache – Compact Strings – Ahead-of-time Compilation – Value Types Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 10 The JVM: An application developer’s view Java source code Bytecodes HotSpot int i = 0; compile 0: iconst_0 execute Java VM do { 1: istore_1 i++; 2: iinc } while (i < f()); 5: iload_1 6: invokestatic f 9: if_icmplt 2 12: return • Ahead-of-time • Using javac • Instructions for an abstract machine • Stack-based machine (no registers) Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 11 The JVM: A VM engineer’s view Bytecodes HotSpot Java VM 0: iconst_0 1: istore_1 compile Compilation produce Compiled method Garbage 2: iinc system collector 5: iload_1 6: invokestatic f 9: if_icmplt 2 C1 Machine code 12: return C2 Debug info manage Object maps execute access Heap Interpreter access Stack Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 12 Outline • Intro: Why virtual machines? • Part 1: The Java HotSpot VM – JIT compilation in HotSpot – Tiered Compilation • Part 2: What's new in Java – Segmented Code Cache – Compact Strings – Ahead-of-time Compilation – Value Types Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 13 Interpretation vs. compilation in HotSpot • Template-based interpreter – Generated at VM startup (before program execution) – Maps a well-defined machine code sequence to every bytecode instruction Bytecodes 0: iconst_0 Machine code 1: istore_1 mov -0x8(%r14), %eax Load local variable 1 2: iinc movzbl 0x1(%r13), %ebx 5: iload_1 inc %r13 Dispatch next instruction 6: invokestatic f mov $0xff40,%r10 9: if_icmplt 2 jpmq *(%r10, %rbx, 8) 12: return • Compilation system – Speedup relative to interpretation: ~100X – Two just-in-time compilers (C1, C2) – Aggressive optimistic optimizations Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 14 Ahead-of-time vs. just-in-time compilation • AOT: Before program execution • JIT: During program execution – Tradeoff: Resource usage vs. performance of generated code Performance Bad performance Good performance Bad performance due to interpretation due to good selection of due to compilation compiled methods and overhead applied optimizations Amount of compilation Interpretation Compile everything Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 15 JIT compilation in HotSpot • Resource usage vs. performance – Getting to the “sweet spot” 1. Selecting methods to compile 2. Selecting compiler optimizations Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 16 1. Selecting methods to compile • Hot methods (frequently executed methods) • Profile method execution – # of method invocations, # of backedges • A method’s lifetime in the VM Compiler’s optimistic assumptions proven wrong Deoptimization # method invocations > THRESHOLD1 # of backedges > THRESHOLD2 Interpreter Compiler (C1 or C2) Code cache Gather profiling information Compile bytecode to native code Store machine code Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 17 Example optimization: Hot path compilation Control flow graph Generated code S1; guard(x > 3) S2; S1; S ; S3; 2 Deoptimize if (x > 3) S3; S4; S8; 10’000 0 S ; True False 9 S4; S5; S6; S7; S8; S9; Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 18 Example optimization: Virtual call inlining Class hierarchy Method to be compiled class A { void foo() { void bar() { A a = create(); // return A or B Compiler: S1; loaded a.bar(); } } Inline call? } Yes. class B extends A { void bar() { S2; not loaded } } Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 19 Example optimization: Virtual call inlining Class hierarchy Method to be compiled class A { void foo() { void bar() { A a = create(); // return A or B Compiler: S1; loaded S1; } } Inline call? } Yes. class B extends A { • Benefits of inlining void bar() { – Virtual call avoided S2; not loaded } – Code locality } • Optimistic assumption: only A is loaded – Note dependence on class hierarchy – Deoptimize if hierarchy changes Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 20 Example optimization: Virtual call inlining Class hierarchy Method to be compiled class A { void foo() { void bar() { A a = create(); // return A or B Compiler: S1; loaded a.bar(); } } Inline call? } No. class B extends A { void bar() { S2; loaded } } Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 21 Deoptimization • Compiler’s optimistic assumption proven wrong – Assumptions about class hierarchy – Profile information does not match method behavior • Switch execution from compiled code to interpretation – Reconstruct state of the interpreter at runtime – Complex implementation • Compiled code – Possibly thrown away – Possibly reprofiled and recompiled Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 22 Performance effect of deoptimization • Follow the variation of a method’s performance Interpreted Compiled Interpreted Compiled Performance Time VM Startup Compilation Deoptimization Compilation Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 23 JIT compilation in HotSpot • Resource usage vs. performance – Getting to the “sweet spot” 1. Selecting methods to compile 2. Selecting compiler optimizations Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 24 2. Selecting compiler optimizations • C1 compiler – Limited set of optimizations Client VM – Fast compilation – Small footprint Tiered Compilation • C2 compiler (enabled since JDK 8) – Aggressive optimistic optimizations Server VM – High resource demands – High-performance code • Graal – Part of HotSpot for AOT since JDK 9 – Available as experimental C2 replacement in JDK 11 Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 25 Outline • Why virtual machines? • Part 1: The Java HotSpot VM – JIT compilation in HotSpot – Tiered Compilation • Part 2: What's new in Java – Segmented Code – Compact Strings – Ahead-of-Time Compilation – Value Types Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 26 Tiered Compilation • Introduced in JDK 7, enabled by default in JDK 8 • Combines the benefits of – Interpreter: Fast startup – C1: Fast compilation – C2: High peak performance • Within the sweet spot – Faster startup – More profile information Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 27 Benefits of Tiered Compilation Client VM (C1 only) warm-up time Performance Interpreted C1-compiled Time VM Startup Compilation Copyright © 2018, Oracle and/or its affiliates. All rights reserved. 28 Benefits of Tiered Compilation