Project Panama Status Update Vladimir Ivanov JVM Compiler team, JPG Oracle JVM Compiler team Offsite, January, 22, 2017 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Safe Harbor Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle. Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 2 Project Panama: What’s in there? http://hg.openjdk.java.net/panama/dev $ hg branches vectorIntrinsics 47814:8eafa3e1dd22 vectorSnippets 47748:8f8232cadaff nicl 47744:8499209102d4 linkToNative 47740:8b90565dcbfe array2 47726:1fd0974b1d4b … Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 3 Project Panama: What’s in there? http://hg.openjdk.java.net/panama/dev $ hg branches vectorIntrinsics 47814:8eafa3e1dd22 Vector API vectorSnippets 47748:8f8232cadaff nicl 47744:8499209102d4 JNI 2.0 linkToNative 47740:8b90565dcbfe array2 47726:1fd0974b1d4b Arrays 2.0 … Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 4 Project Panama: Recent Activity http://hg.openjdk.java.net/panama/dev $ hg branches vectorIntrinsics 47814:8eafa3e1dd22 Vector API vectorSnippets 47748:8f8232cadaff nicl 47744:8499209102d4 JNI 2.0 linkToNative 47740:8b90565dcbfe array2 47726:1fd0974b1d4b Arrays 2.0 … Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 5 Better JNI “Codes like Java, works like native” $ hg branches vectorIntrinsics 47814:8eafa3e1dd22 Vector API vectorSnippets 47748:8f8232cadaff nicl 47744:8499209102d4 JNI 2.0 linkToNative 47740:8b90565dcbfe array2 47726:1fd0974b1d4b Arrays 2.0 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 6 What’s in there? • nicl – jextract, libclang bindings (Henry, 2014-2016; Maurizio, Sundar, 2017-2018) – Universal adapter (Mikael, 2015-2016) – Binder, vectors, etc. (Mikael Vidstedt, 2015-2016) – MemoryRegion, Scope, Pointer (Henry, Mikael, 2016) • linkToNative – MethodHandle-based native invoker (Vladimir Ivanov, 2015) Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 7 JNI JNR NICL User-defined Interfaces produced User-defined Interfaces by jextract generated generated bindings on-the-fly bindings on-the-fly Java JNR Java j.l.i Native libffi Native JVM stubs Library Target Target Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 8 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 9 jextract - Header file -> Java interface unistd.class package mypackage; unistd.h #ifndef _UNISTD_H @Header(path=”unistd.h”) #define _UNISTD_H public interface unistd { @C(file=”unistd.h”, line=3, column=1, USR=”c:@F@getpid”) pid_t getpid(void); @NativeType(ctype=”pid_t (void)”, size=1) @CallingConvention(1) #endif // _UNISTD_H public int getpid(); } Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10 Example: getpid() // Load native library Library lib = NativeLibrary.load(“c”); // Weave a class for the unistd interface, // and return an instance unistd unistd = NativeLibrary.create(lib, unistd.class) // Call the system getpid() function int pid = unistd.getpid(); Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11 Native Method Handles MethodType mt = MethodType.methodType(int.class); MethodHandle mh = NativeLookup.findNative(“getpid", mt); int pid = mh.invokeExact(); Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 12 Native Method Handles Java Native Construction Lookup.findVirtual() et al Lookup.findNative() Reference DirectMethodHandle NativeMethodHandle (typed) Reference MemberName NativeEntryPoint (direct) Linker MH.linkToVirtual() et al MH.linkToNative() Invocation indy, MH.invoke(), MH.invokeExact() “Making native calls from the JVM” by John Rose http://cr.openjdk.java.net/~jrose/panama/native-call-primitive.html Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 13 // int pid = mh.invokeExact(); @ 8 LambdaForm$MH::invokeExact_MT (29 bytes) force inline by annotation @ 11 Invokers::checkExactType (17 bytes) force inline by annotation @ 1 MethodHandle::type (5 bytes) accessor @ 15 Invokers::checkCustomized (23 bytes) force inline by annotation @ 1 MethodHandleImpl::isCompileConstant (2 bytes) (intrinsic) @ 25 LambdaForm$NMH::invokeNative_I (27 bytes) force inline by … @ 7 NativeMethodHandle::internalNativeEntryPoint (8 bytes) force inline … @ 23 MethodHandle::linkToNative(L)I (0 bytes) direct native call Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 14 Native Method Handles callq 0x1057b2eb0 ; native method entry getpid JNI 13.7 ± 0.5 ns Direct call 3.4 ± 0.2 ns Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 15 Calling Convention Java vs C hotspot/src/cpu/x86/vm/sharedRuntime_x86_64.cpp: “The Java calling convention is a "shifted" version of the C ABI. By skipping the first C ABI register we can call non-static jni methods with small numbers of arguments without having to shuffle the arguments at all. Since we control the java ABI we ought to at least get some advantage out of it.“ System V AMD64 ABI arg 1st 2nd 3rd 4th 5th 6th … C RDI RSI RDX RCX R8 R9 stack Java RSI RDX RCX R8 R9 RDI stack Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 16 “Universal Adapter” • jdk.internal.nicl.NativeInvoker – static native void invokeNative(long[] args, long[] rets, long[] recipe, long addr); – alternative to MH.linkToNative() • Very powerful mechanism – fully programmable calling conventions through “recipe” • Rich functionality support – varargs – struct-by-value argument + return – callbacks (function pointers) • Hard to optimize by JITs Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 17 Optimize checks void run() { nativeFunc1(); // checks & state trans. nativeFunc2(); // checks & state trans. nativeFunc3(); // checks & state trans. } Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 18 Java Heap Thread State Thread Java Native VM Native Memory Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 19 Optimize checks void run() { if (!performChecks()) jump failed; transJavaToNative(); call nativeFunc1(); call nativeFunc2(); call nativeFunc3(); transNativeToJava(); } Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 20 Java Heap Thread State Thread Java Native VM Native Memory Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 21 Better JNI Easier, Safer, Faster • Native access between the JVM and native APIs – Native code via FFIs – Native data via safely-wrapped access functions – Tooling for header file API extraction and API metadata storage • Wrapper interposition mechanisms, based on JVM interfaces – add (or delete) wrappers for specialized safety invariants • Basic bindings for selected native APIs Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 22 Contributors • John Rose • Henry Jen • Mikael Vidstedt • Maurizio Cimadamore • Sundararajan Athijegannathan • Tobi Ajila (IBM) Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 23 Vector API $ hg branches vectorIntrinsics 47814:8eafa3e1dd22 Vector API vectorSnippets 47748:8f8232cadaff nicl 47744:8499209102d4 JNI 2.0 linkToNative 47740:8b90565dcbfe array2 47726:1fd0974b1d4b Arrays 2.0 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 24 Motivation Expose data-parallel operations through a cross-platform API Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 25 Motivation Int8Vector x = ..., y = ...; // packed vectors of 8 ints Int8Vector z = x.add(y); // element-wise addition vpaddd %ymm1,%ymm0,%ymm0 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 26 Goals 1. Maximally expressive and portable API – “principle of least astonishment” – uniform coverage operations and data types – type-safe 2. Performant – predictable performance – high quality of generated code – competitive with existing facilities for auto-vectorization 3. Graceful performance degradation – fallback for "holes" in native architectures Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 27 Draft API (by John Rose, c. 2015) interface Vector<E,S extends Shape<Vector<?,S>>> { Vector<E,S> add(Vector<E,S> v2); } Vector<Integer,S256Bit> x = ..., y = ...; // vectors of 8 ints Vector<Integer,S256Bit> z = x.add(y); // element-wise addition vpaddd %ymm1,%ymm0,%ymm0 Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 28 Implementation Challenges 1. How to represent vector operations on JVM level? 2. Optimize away vector boxes – required for mapping Vector instances to vector registers in generated code – Vector<Integer,S256Bits> => ymm regs on x86/AVX 3. Primitive specialization – Vector<Integer,S256Bits> vs Vector<int,S256Bits> 4. Vectorize higher-order operations – Vector<E,S> map(UnaryOperator<E> op); – “You can fit a universe in a lambda” Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 29 Vector API: Ultimate Goal Vector<int,?>.map(arr, i -> i + 1) (with value types, generic specialization, and crackable lambdas) Copyright © 2017, Oracle and/or its affiliates.