A Post-Apocalyptic sun.misc.Unsafe World http://www.superbwallpapers.com/fantasy/post-apocalyptic-tower-bridge-london-26546/ Chris Engelbert Twitter: @noctarius2k Jatumba! 2014, 2015, 2016, … Disclaimer This talk is not going to be negative! Disclaimer But certain things are highly speculative and APIs or ideas might change by tomorrow! sun.misc.Scissors http://www.underwhelmedcomic.com/wp-content/uploads/2012/03/runningdude.jpg sun.misc.Unsafe - What you (don’t) know sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE // Unsafe mechanics private static final sun.misc.Unsafe U; private static final long QBASE; private static final long QLOCK; private static final int ABASE; private static final int ASHIFT; static { try { U = sun.misc.Unsafe.getUnsafe(); Class<?> k = WorkQueue.class; Class<?> ak = ForkJoinTask[].class; example: QBASE = U.objectFieldOffset (k.getDeclaredField("base")); java.util.concurrent.ForkJoinPool QLOCK = U.objectFieldOffset (k.getDeclaredField("qlock")); ABASE = U.arrayBaseOffset(ak); int scale = U.arrayIndexScale(ak); if ((scale & (scale - 1)) != 0) throw new Error("data type scale not a power of two"); ASHIFT = 31 - Integer.numberOfLeadingZeros(scale); } catch (Exception e) { throw new Error(e); } } } sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well (good // Deserialization public Object readObject(InputStream is) { SomeType element = unsafe.allocateInstance(SomeType.class); element.deserializeValues(is); return element; } sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well (good, bad) # # A fatal error has been detected by the Java Runtime Environment: # # SIGSEGV (0xb) at pc=0x00000001067314b5, pid=25244, tid=5891 # # JRE version: Java(TM) SE Runtime Environment (8.0_05-b13) (build 1.8.0_05-b13) # Java VM: Java HotSpot(TM) 64-Bit Server VM (25.5-b02 mixed mode bsd-amd64 compressed oops) # Problematic frame: # V [libjvm.dylib+0x5314b5] Unsafe_SetNativeAddress+0x36 # # An error report file with more information is saved as: # /Users/noctarius/git/hazelcast-only.idea/hs_err_pid25244.log # # If you would like to submit a bug report, please visit: # http://bugreport.sun.com/bugreport/crash.jsp # sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well (good / bad) • Never intended for outside use sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well (good / bad) • Never intended for outside use sun.misc.Unsafe - What you (don’t) know • Internal class (sun.misc Package) • Used inside the JVM / JRE • Used outside as well (good / bad) • Never intended for outside use • Used by popular: • Applications • Frameworks • Libraries CapLogic ActivePivot Apache Ignite JRuby Liquibase Apache Spark Neo4j Presto Gson Disruptor Metrics (AOP) GWT Mockito Scala Robolectric Hibernate OrientDB MapDB EasyMock XStream jol Grails JMock Spock Ehcache Apache HBase Agrona Akka Netty Apache Continuum Apache Flink Hazelcast Kryo Zookeeper Cassandra XRebel LWJGL Apache Storm Apache Kafka Chronicle Byte Buddy Aeron Simple Binary Encoding Apache Wink XAP Infinispan Spring Framework Dropwizard Apache Hadoop WildFly CapLogic ActivePivot Apache Ignite JRuby Liquibase Apache Spark Neo4j Presto Gson Disruptor Metrics (AOP) GWT Mockito Scala Robolectric Hibernate OrientDB MapDB EasyMock XStream jol Grails JMock Spock Ehcache Apache HBase Agrona Akka Netty Apache Continuum Apache Flink Hazelcast Kryo Zookeeper Cassandra XRebel LWJGL Apache Storm Apache Kafka Chronicle Byte Buddy Aeron Simple Binary Encoding Apache Wink XAP Infinispan Spring Framework Dropwizard Apache Hadoop WildFly sun.misc.Unsafe is only supported on HotSpot JVMs anyways, isn’t it? No! sun.misc.Unsafe is only supported on HotSpot JVMs anyways, isn’t it? Language URL sun.misc.Unsafe Avian C++ https://github.com/ReadyTalk/avian Yes BicaVM JavaScript https://github.com/nurv/BicaVM No Cacao C http://www.cacaojvm.org Yes Excelsior JVM C / C++ ? http://www.excelsiorjet.com Yes Dalvik / ART / Android C / C++ https://source.android.com/source/index.html Yes Graal mainly Java http://openjdk.java.net/projects/graal Yes GCJ C / C++ https://gcc.gnu.org/java Yes HaikuVM C http://haiku-vm.sourceforge.net No IBM J9 C / C++ ? http://www.ibm.com/developerworks/java/jdk/index.html Yes IcedTea C / C++ http://icedtea.classpath.org/wiki/Main_Page Yes IKVM.NET C# http://www.ikvm.net Yes Jamiga C https://sourceforge.net/projects/jamiga2 No JamVM C http://jamvm.sourceforge.net Yes JatoVM C http://jatovm.org Yes JikesRVM C microkernel, Java http://www.jikesrvm.org Yes JNode C microkernel, Java http://www.jnode.org Yes JVM.go Go https://github.com/zxh0/jvm.go Yes JX C http://www4.cs.fau.de/Projects/JX/index.html No Kaffe C https://github.com/kaffe/kaffe Yes NanoVM C http://harbaum.org/till/nanovm/index.shtml No OpenJDK / Oracle / Zulu C / C++ http://openjdk.java.net Yes RoboVM C / C++ / ObjC https://robovm.com Yes SAPJVM C / C++ ? http://help.sap.com/saphelp_nwce10/helpdata/en/47/ Yes Waratek Multitenant JVM C / C++ ? dc90b4ef17452289f9128b8c2bbd77/content.htmhttp://www.waratek.com Yes Zing JVM C / C++ ? https://www.azul.com/products/zing Yes Use Cases - The Good Ones! Use Cases - The Good Ones! • Low Latency, GC Overhead Use Cases - The Good Ones! • Low Latency, GC Overhead Unsafe unsafe = findUnsafe(); long baseAddress = unsafe.allocateMemory(8); class Point { long offsetX = 0; long offsetY = 4; int getX(long baseAddress) { return unsafe.getInt(baseAddress + offsetX); } int getY(long baseAddress) { return unsafe.getInt(baseAddress + offsetY); } } Caution: Overly simplified! Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization // Deserialization public Object readObject(InputStream is) { SomeType element = unsafe.allocateInstance(SomeType.class); element.deserializeValues(is); return element; } Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions long baseAddress = unsafe.allocateMemory(Long.MAX_VALUE); unsafe.putInt(baseAddress + Long.MAX_VALUE - 4, Integer.MAX_VALUE); Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations unsafe.compareAndSwapInt(this, offsetField, oldValue, newValue); unsafe.compareAndSwapLong(this, offsetField, oldValue, newValue); unsafe.compareAndSwapObject(this, offsetField, oldObject, newObject); Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout Use Cases - The Good Ones! Low Latency, GC Overhead class Point { • long offsetX = 0; • Fast (de-)serialization long offsetY = 4; • 64bit Memory regions } • Atomic memory operations • Efficient memory layout offsetX offsetY Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory access Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory access long versionFieldOffset = unsafe.objectFieldOffset(versionField); long version = unsafe.getLong(this, versionFieldOffset); unsafe.putLong(this, versionFieldOffset, 10L); Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory access • Access to array with volatile semantics Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory access • Access to array with volatile semantics long base = unsafe.arrayBaseOffset(int[].class); long index = unsafe.arrayIndexScale(int[].class); unsafe.getIntVolatile(array, base + (index * 3)); Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory access • Access to array with volatile semantics • Custom memory fences Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization unsafe.fullFence(); 64bit Memory regions • unsafe.storeFence(); • Atomic memory operations • Efficient memory layout unsafe.loadFence(); • Fast field / memory access • Access to array with volatile semantics • Custom memory fences Use Cases - The Good Ones! • Low Latency, GC Overhead • Fast (de-)serialization • 64bit Memory regions • Atomic memory operations • Efficient memory layout • Fast field / memory