Fast, Dynamically-Sized Concurrent Hash Table? J. Barnat, P. Roˇckai, V. Still,ˇ and J. Weiser Faculty of Informatics, Masaryk University Brno, Czech Republic fbarnat,xrockai,xstill,xweiser1g@fi.muni.cz Abstract. We present a new design and a C++ implementation of a high-performance, cache-efficient hash table suitable for use in implemen- tation of parallel programs in shared memory. Among the main design criteria were the ability to efficiently use variable-length keys, dynamic table resizing to accommodate data sets of unpredictable size and fully concurrent read-write access. We show that the design is correct with respect to data races, both through a high-level argument, as well as by using a model checker to prove crucial safety properties of the actual implementation. Finally, we provide a number of benchmarks showing the performance characteristics of the C++ implementation, in comparison with both sequential-access and concurrent-access designs. 1 Introduction Many practical algorithms make use of hash tables as a fast, compact data struc- ture with expected O(1) lookup and insertion. Moreover, in many applications, it is desirable that multiple threads can access the data structure at once, ide- ally without causing execution delays due to synchronisation or locking. One such application of hash tables is parallel model checking, where the hash table is a central structure, and its performance is crucial for a successful, scalable implementation of the model checking algorithm. Moreover, in this context, it is also imperative that the hash table is compact (has low memory overhead), because the model checker is often primarily constrained by available memory: therefore, a more compact hash table can directly translate into the ability to model-check larger problem instances. Another desirable property is an ability to dynamically resize (grow) the hash table, in accordance with changing needs of the model checking algorithm as it explores the state space. Finally, it is often the case that the items (state vectors) stored in the hash table by the model checker have a dynamic size, for which it is difficult to predict an upper bound. Hence, we need to be able to efficiently store variable-length keys in the hash table. While the outlined use-case from parallel model checking was our original motivation, a data structure with the same or similar properties is useful in many other applications. ? This work has been partially supported by the Czech Science Foundation grant No. 15-08772S. 1.1 Related Work As we demonstrate in Section 4, our design is highly competitive, improving on the state of the art in parallel data structures, as represented by the venerable Intel Threading Building Blocks library [1]. The design presented in this pa- per offers faster sequential access, better multi-threaded scalability and reduced memory overhead. Most of these attributes can be derived from the fact that our design is based on an open hashing scheme, in contrast to almost all existing concurrent hash tables. Often, concurrent hash table designs take advantage of the simplicity of concurrent access to linked lists (eg. [2], but also the designs in Intel TBB [1]), leading to a closed hashing scheme. Alternatively, a concurrent, open-hashing table based on our earlier (sequential) design has been described in [3], but while providing very good performance and scalability, it was limited to statically pre-allocated hash tables (i.e. with a fixed number of slots). Our de- sign, however, does not explicitly deal with key removal: a standard `tombstone' approach can be used, although it may also be possible to leverage the scheme proposed in [4], where authors focus on deletion in a concurrent (but fixed size) hash table with open addressing. A more closely related design (without an implementation, however) was presented in [5]. In this paper, the authors present a concurrent hash table based on open hashing and arrive to solution that are in many cases similar to ours. Especially the approach to ensuring that resize operations do not interfere with running inserts is very similar { in this particular case, we believe that the extensive and detailed correctness proofs done in [5] would transfer to our design with only minor adjustments. Our present paper, however, places more emphasis on the implementation and its practical consequences. By comparing notes with existing work on the subject, we can conclude that the design approach is sound in principle; while we did basic correctness analysis on the design, our main concern was correctness of the implementation. Unlike existing work, we make use of software model checking to ascertain that the implementation (and by extension, the design) is indeed correct. 2 Design There are many considerations that influence the design of a data structure. Our first priorities were performance and scalability of concurrent access; in both cases, it is important to consider the hardware which will execute the code. First, we need to realize that modern multi-core and SMP systems exhibit a deep memory hierarchy, with many levels of cache. Some of this cache is shared by multiple cores, some is private to a particular core. This translates into a complex memory layout. To further complicate matters, multi-CPU computers nowadays often use non-uniform access architecture even for the main memory: different parts of RAM have different latency towards different cores. Most of this complexity is implicitly hidden by the architecture, but performance-wise, this abstraction is necessarily leaky. Moreover, the gap between the first and the last rungs of the hierarchy is huge: this means that compact data structures often vastly outperform asymptotically equivalent, but sparse structures. Due to cache organisation constraints, memory cells that live close to each other are usually fetched and flushed together, as part of a single \cache line". They are also synchronised together between core-private caches. A modern data structure should therefore strive to reduce to an absolute minimum the number of cache lines it needs to access in order to perform a particular operation. On the other hand, when concurrency is involved, there is a strong preference to have threads use non-overlapping sets of cache-line-sized chunks of memory, especially in hot code paths. 2.1 Hash Functions A hash table is represented as a vector of values in memory, associated with a function that maps keys to indices within this vector. The function is known as a hash function and should posses a number of specific properties: the distribution of key images should be uniform across the entire length of the vector, a small change in the key should produce a large change in the value, the function should be fast to compute and such a function should be available for arbitrary index range. In practice, to implement the last criterion, hash functions for hash tables are usually implemented over the range of all 32 (64, 128 bit) integers in such a way that the remainder of division by an arbitrary integer n (or at least a power of two) will yield a uniform distribution in f1; :::; ng. The current practice is to use a purpose-built lookup function, either providing 64 (lookup3 [6] is a good candidate) or even 128 bits of output (the currently best available are spooky hash [7] and the city hash [8]). 2.2 Open vs Closed Hashing Even with the best lookup function, hash collisions, and more importantly, index collisions will happen in a dynamic hash table. Hence, an important part of the hash table design is dealing with such collisions, and there are two main options: open and closed hashing (also known as open and closed addressing). With a closed hashing scheme, each position in the hash table is a \bucket" { capable of holding multiple values at the same time. This is implemented using an auxiliary data structure, usually a linked list. While closed hashing is easier to implement and to predict, it usually gives poor performance. An alternative is to make each position in the table only hold at most one value at a time, using alternate positions for items that cause a collision. Instead of using a single fixed position for each value, the hash table has a list of candidate indices. The most common such series are h + ai + b where i is the sequence number of the index, h is the index assigned by a lookup function and a; b are arbitrary constants (a linear probing scheme). Another common choice is h + ai2 + bi + c, obviously known as quadratic probing. An important property of a probing scheme is that it does not (significantly) disrupt the uniform distribution of values across indices. In case of a quadratic function and a hash table with a size that is a power of 2, a simple set of constraints can be shown to give a good distribution [9]. 2.3 Cache Performance There are many considerations when choosing a good hash table implementation for a particular application. In model checking, as well as many other use cases, the hash table becomes very big, and as such, it cannot fit in the CPU cache entirely. For that reason, it is very important that all hash table operations have as much spatial and temporal locality as possible, to make best possible use of the CPU cache. The very nature of a hash table means that insert or lookup operations on different keys will access entirely different memory regions: this is unavoidable. However, with a naive implementation, even a single lookup or insert can cause many cache misses: a closed-hashing scheme, for example, will need to traverse a linked list during collision resolution, which is a notoriously cache-inefficient operation.