Adaptive Lock-Free Data Structures in Haskell: A General Method for Concurrent Implementation Swapping Chao-Hong Chen Vikraman Choudhury Ryan R. Newton Indiana University, USA Indiana University, USA Indiana University, USA [email protected] [email protected] [email protected] Abstract 1 Introduction A key part of implementing high-level languages is providing built- High-level, productivity languages are equipped with rich built-in in and default data structures. Yet selecting good defaults is hard. data structures, such as mutable and immutable dictionaries. If we A mutable data structure’s workload is not known in advance, and envision parallel-by-default languages of the future, programmers it may shift over its lifetime—e.g., between read-heavy and write- may expect that built-in data structures support concurrency too. heavy, or from heavy contention by multiple threads to single- This will require intelligent selections of default implementations threaded or low-frequency use. One idea is to switch implementa- to not incur undue overhead under either contended or single- tions adaptively, but it is nontrivial to switch the implementation threaded access patterns. of a concurrent data structure at runtime. Performing the transition If you need a one-size-fits all choice, automatically selecting or requires a concurrent snapshot of data structure contents, which swapping between data structure implementations at runtime has normally demands special engineering in the data structure’s de- a natural appeal [Bolz et al. 2013; Xu 2013]. These swaps can match sign. However, in this paper we identify and formalize an relevant implementations to observed workloads: e.g., relative frequency of property of lock-free algorithms. Namely, lock-freedom is sufficient methods, degree of contention, or data structure contents. In the to guarantee that freezing memory locations in an arbitrary order sequential case, adaptive data structures are already used to good will result in a valid snapshot. effect by tracing JIT compilers for dynamic languages such asPyPy Several functional languages have data structures that freeze and Pycket [Bauman et al. 2015], which, for instance, optimistically and thaw, transitioning between mutable and immutable, such assumes that an array of floats will continue to contain only floats, as Haskell vectors and Clojure transients, but these enable only and fall back automatically to a more general representation only single-threaded writers. We generalize this approach to augment if this is violated. an arbitrary lock-free data structure with the ability to gradually But such adaptive data structures are much harder to achieve in freeze and optionally transition to a new representation. This aug- concurrent settings than in sequential. While implementations of mentation doesn’t require changing the algorithm or code for the adaptive data structures exist in Java and other languages [De Wael data structure, only replacing its datatype for mutable references et al. 2015; Kusum et al. 2016; Xu 2013], no techniques exist to apply with a freezable variant. In this paper, we present an algorithm this form of adaptation in a multithreaded scenario. How can we for lifting plain to adaptive data and prove that the resulting hy- convert a data structure that is being mutated concurrently? The brid data structure is itself lock-free, linearizable, and simulates the problem is similar to that of concurrent garbage collection, where a original. We also perform an empirical case study in the context of collector tries to move an object graph while the mutator modifies heating up and cooling down concurrent maps. it. In this paper we introduce a general approach for freezing a CCS Concepts • Computing methodologies → Concurrent concurrent, lock-free, mutable data structure—subject only to a algorithms; • Software and its engineering → Functional lan- few restrictions on the shape of methods and which instructions guages; linearize. We do this using the simple notion of a freezable reference Keywords lock-free algorithms, concurrent data structures, con- that replaces the regular mutable reference. Remarkably, it is possi- currency, parallelism ble to freeze the references within a data structure in an arbitrary order, and still have a valid snapshot. The snapshot is valid because arXiv:1708.02318v1 [cs.PL] 7 Aug 2017 ACM Reference Format: all operations on the original structure are linearizable, which pre- Chao-Hong Chen, Vikraman Choudhury, and Ryan R. Newton. 2017. Adap- vents intermediate modifications due to half-completed methods tive Lock-Free Data Structures in Haskell: A General Method for Concurrent from corrupting the structure or affecting its logical contents. Implementation Swapping. In Proceedings of Haskell’17, Oxford, United King- Using freezable references, we show how to build a hybrid data dom, September 7–8, 2017, 15 pages. https://doi.org/10.1145/3122955.3122973 structure that transitions between two lock-free representations, while in turn preserving lock-freedom and linearizability for the Permission to make digital or hard copies of all or part of this work for personal or complete structure. The end result is a hybrid exposing a special classroom use is granted without fee provided that copies are not made or distributed method to trigger transition between representations, with user- for profit or commercial advantage and that copies bear this notice and the full citation exposed snapshotting being a special case of transitioning to an on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or immutable data structure inside a reference that allows O¹1º exact republish, to post on servers or to redistribute to lists, requires prior specific permission snapshots. and/or a fee. Request permissions from [email protected]. The contributions of this paper are: Haskell’17, September 7–8, 2017, Oxford, United Kingdom • © 2017 Copyright held by the owner/author(s). Publication rights licensed to Associa- We prove a property of lock-free data structures that has tion for Computing Machinery. not previously been formalized: that the memory locations ACM ISBN 978-1-4503-5182-9/17/09...$15.00 making up the structure can be frozen in arbitrary order, https://doi.org/10.1145/3122955.3122973 Haskell’17, September 7–8, 2017, Oxford, United Kingdom Chao-Hong Chen, Vikraman Choudhury, and Ryan R. Newton providing a valid snapshot state. We apply the tools of oper- upon detecting contention (“heating up”). The combined, hybrid ational semantics in this concurrency proof, which enables data structure was proven to preserve lock-freedom. There are sev- more precision than is standard for such proofs. eral drawbacks of this approach, however1. Most importantly, the • We present a novel algorithm for building hybrid, adaptive algorithm assumed a starting state of a pure data structure inside data structures which we prove is lock-free, linearizable, and a single mutable reference. What about adapting from a lock-free correctly models the original structures (§6). Our formal concurrent data structure as the starting state? model uses an abstract machine that models clients interact- Example: Cooling down data: As a motivating example, consider ing with a lock free data structure. a cloud document stored on a server that experiences concurrent • We demonstrate how to apply the method in a Haskell library writes while it is being created, and then read-only accesses for that transitions between a concurrent Ctrie [Prokopec et al. the rest of its lifetime. Likewise, applications that use time series 2012] and a purely functional hashmap (§ 7.1). We perform data (e.g., analytics) handle concurrent write-heavy operations, a case study measuring the benefit of transitioning from a followed by a cool-down phase—a read-only workload, such as concurrent data structure to a representation optimized for running machine learning algorithms. But in order to support these read-heavy workloads (§8). scenarios, we first need to introduce our basic building block— freezable references. 2 Background and Related Work At first glance, the problem we pose would seem to bear similar- 3 Prerequisite: Freezable IORefs ity to previous work on multi-word transactions (MCAS or STM). In this section, we describe the interface to a freezable reference. For instance, Harris et al. [2002] propose a way to perform multi- This API could be implemented in any language, but because we word CAS (MCAS). However this approach is effective only for use Haskell for our experiments (§8), we follow the conventions of small, fixed numbers of locations rather than large, lock-free data the Haskell data type IORef. structures—it must create a descriptor proportional to all addresses newIORef :: a ! IO ( IORef a) being changed, which we cannot do for a data structure which readIORef :: IORef a ! IO a grows during transitioning, and would be inefficient in any case. writeIORef :: IORef a ! a ! IO () Software transactional memory [Herlihy et al. 2003b] can be used Ultimately, we need only one new bit of information per IORef—a on dynamic-sized data to implement obstruction-free data struc- frozen bit. The reference is frozen with a call to freezeIORef: tures [Herlihy et al. 2003a]. However, STM’s optimistic approach freezeIORef :: IORef a ! IO () in general cannot scale to large data structures. After this call, any further attempts to writeIORef