QuickCheck: Using Speculation to Reduce the Overhead of Checks in NVM Frameworks Thomas Shull Jian Huang Josep Torrellas University of Illinois at University of Illinois at University of Illinois at Urbana-Champaign Urbana-Champaign Urbana-Champaign [email protected] [email protected] [email protected] Abstract ACM Reference Format: Byte addressable, Non-Volatile Memory (NVM) is emerging Thomas Shull, Jian Huang, and Josep Torrellas. 2019. QuickCheck: as a revolutionary technology that provides near-DRAM per- Using Speculation to Reduce the Overhead of Checks in NVM Frameworks. In Proceedings of the 15th ACM SIGPLAN/SIGOPS formance and scalable memory capacity. To facilitate the International Conference on Virtual Execution Environments (VEE usability of NVM, new programming frameworks have been ’19), April 14, 2019, Providence, RI, USA. ACM, New York, NY, USA, proposed to automatically or semi-automatically maintain 15 pages. https://doi.org/10.1145/3313808.3313822 crash-consistent data structures, relieving much of the bur- den of developing persistent applications from programmers. 1 Introduction While these new frameworks greatly improve program- Recently, there have been massive technological advances mer productivity, they also require many runtime checks for towards providing fast and byte addressable Non-Volatile correct execution on persistent objects, which significantly Memory (NVM), such as Intel 3D XPoint [38], Phase-Change affect the application performance. With a characterization Memory (PCM) [42, 52], and Resistive RAM (ReRAM) [5]. study of various workloads, we find that the overhead of These memory technologies promise near-DRAM perfor- these persistence checks in these programmer-friendly NVM mance, scalable memory capacity, and data durability, which frameworks can be substantial and reach up to 214%. Further- provides opportunities for boosting the performance of soft- more, we find that programs nearly always access exclusively ware systems and applications [19, 34, 64]. either a persistent or a non-persistent object at a given site, To enable applications to take advantage of NVM while making the behavior of these checks highly predictable. also improving their programmability, many frameworks for In this paper, we propose QuickCheck, a technique that bi- NVM have been proposed, including those for C/C++ [3, 15, ases persistence checks based on their expected behavior, and 16, 19, 20, 22, 24, 29, 33, 40, 47, 62] and Java [3, 55, 56, 64] ap- exploits speculative optimizations to further reduce the over- plications. A trend in this research area is to identify and per- heads of these persistence checks. We evaluate QuickCheck sist data structures automatically or semi-automatically, to with a variety of data intensive applications such as a key- relieve the burden on developers. Instead of having to either value store. Our experiments show that QuickCheck im- use custom persistent libraries or explicitly mark all actions proves the performance of a persistent Java framework on needed for crash consistency, developers mark only a few average by 48.2% for applications that do not require data core objects. Then, the runtime ensures that the transitive- persistence, and by 8.0% for a persistent memcached imple- closure of these core objects is stored in NVM and operated mentation running YCSB. on in a crash-consistent manner. CCS Concepts • Hardware → Non-volatile memory; • While such an approach greatly reduces the load on pro- Software and its engineering → Just-in-time compil- grammers, it requires a large number of software checks, ers; Source code generation. which we call persistence checks, to be inserted alongside ob- Keywords Java, Non-Volatile Memory, JIT Compilation ject accesses. These checks guard runtime actions necessary for correctly handling persistent objects during execution. A Permission to make digital or hard copies of all or part of this work for persistence check’s action may be activated, meaning that ex- personal or classroom use is granted without fee provided that copies tra actions need to be performed, or bypassed, meaning that are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights no extra work is needed. The overhead of these persistence for components of this work owned by others than the author(s) must checks can be substantial. This is because these checks must be honored. Abstracting with credit is permitted. To copy otherwise, or be inserted for both persistent and non-persistent objects, as republish, to post on servers or to redistribute to lists, requires prior specific the compiler does not know which objects will be persistent permission and/or a fee. Request permissions from [email protected]. until execution time, and these checks must be conservative. VEE ’19, April 14, 2019, Providence, RI, USA Through our profiling with various Java-based workloads, © 2019 Copyright held by the owner/author(s). Publication rights licensed to ACM. we find that the overhead of persistence checks is 55.4%on ACM ISBN 978-1-4503-6020-3/19/04...$15.00 average, and can be as high as 214%, for applications that do https://doi.org/10.1145/3313808.3313822 not require data persistence. VEE ’19, April 14, 2019, Providence, RI, USA Thomas Shull, Jian Huang, and Josep Torrellas However, we also find that whether the check at a given Compiler Traits Compilation Cycle code location is activated or bypassed is highly predictable. Baseline Compiler Specifically, we find that, over 99% of the time, a given site “Hot” Method Recompilation + Fast Compilation Times Baseline Compiler accesses exclusively either persistent or non-persistent ob- — Generates Lower Quality Code + Collects Profiling Information jects. Given this insight, we propose to bias each persistence check site to match its expected behavior. We leverage the Optimizing Compiler multi-tier execution of a Java Virtual Machine (JVM) to pro- — Slower Compilation Times file and categorize each persistence check into one ofthe + Generates High Quality Code Optimizing Compiler four categories: likely, unbiased, unlikely, and very_unlikely, + Utilizes Profiling Information Speculative Optimization Failure based on how likely its action will be activated. According to the persistence check state, the compiler decides what Figure 1. JVM compilation overview. optimized code to generate. Options range from generating where each tier offers a different tradeoff between the timeto code to favor the likely persistence check execution path, generate code and the quality of code generated. Figure1 pro- to performing speculative optimizations that eliminate the vides a high-level overview of JVM compilation. The initial check activation path entirely. compiler, commonly known as the baseline compiler, gen- We develop our persistence check biasing technique named erates code quickly. However, the code is not efficient. The QuickCheck in a real JVM framework running on a server “hot” methods in which most of the execution time is spent system with Intel Optane DC persistent memory [35]. We are later recompiled by a more advanced compiler, which evaluate QuickCheck with a variety of applications. Our ex- in the figure we call the optimizing compiler. The optimiz- perimental results demonstrate that QuickCheck improves ing compiler produces high-quality code. However, the code the performance of the persistent Java framework on average takes longer to generate. by 48.2% for applications that do not require data persistence, An optimization commonly performed during multi-tiered and by 8.0% for a persistent memcached implementation run- compilation is to perform speculative optimizations within ning the Yahoo! Cloud Serving Benchmarks (YCSB). Overall, the optimizing compiler. Instead of generating code which we make the following contributions in this paper: can correctly handle all corner cases, the optimizing compiler • We conduct a thorough study on the checking of persis- chooses to speculate on which behaviors will be encountered tent objects in NVM frameworks, and identify that the during runtime, and only generates code that covers these overhead of persistence checks is substantial. behaviors. By performing this speculation, the optimizing • We provide a detailed characterization of the behavior compiler is able to better optimize the code paths. However, of persistence checking on a diverse set of applications if an unexpected behavior is encountered, i.e., a mispecula- that require data persistence, and show that persistence tion occurs, the optimized code will not be able to handle it checking is highly predictable. correctly. In this case, the execution is transferred back to • We propose QuickCheck, a persistence check biasing tech- the code generated by the baseline compiler, which handles nique that uses profiling and speculation to reduce the all corner cases. This process of transferring code execution overhead of persistence checks. back to a more conservative version of generated code is known as on-stack-replacement [31] fallback. 2 Background Performing an on-stack-replacement fallback on a mis- 2.1 Java Application Execution peculation can be an expensive operation. Therefore, the optimizing compiler must be careful in choosing what spec- Java programs are executed on top of a Java Virtual Machine ulative optimizations to perform. To assist with this process, (JVM) [44]. The JVM receives as input .class files consist- multi-tiered