Bringing Modular Concurrency Control to the Next Level Chunzhi Suy Natacha Crooksy Cong Dingz Lorenzo Alvisiyz Chao Xiey∗ yThe University of Texas at Austin zCornell University ABSTRACT should allow developers to reason about the correctness of any given This paper presents Tebaldi, a distributed key-value store that explores CC mechanism in isolation, without being aware of other coexisting new ways to harness the performance opportunity of combining CC mechanisms. The solution should also be general: it should be different specialized concurrency control mechanisms (CCs) within capable of federating a large set of diverse techniques—optimistic the same database. Tebaldi partitions conflicts at a fine granularity and and pessimistic, single-version as well as multiversion. matches them to specialized CCs within a hierarchical framework Prior approaches go some way towards achieving these goals as that is modular, extensible, and able to support a wide variety of they enable different concurrency controls to execute on disjoint sub- concurrency control techniques, from single-version to multiversion sets of either data [42, 51], or transactions [13, 56]. However, some and from lock-based to timestamp-based. When running the TPC-C are restricted to specific CC combinations [13, 17, 34, 45, 51], while benchmark, Tebaldi yields more than 20× the throughput of the basic others, though more general, assume that the database/application can two-phase locking protocol, and over 3:7× the throughput of Callas, be partitioned such that conflicts across partitions are rare and incon- a recent system that, like Tebaldi, aims to combine different CCs. sequential to the end-to-end performance of the system [42, 56]. As a result, most solutions simply handle every cross-partition conflict 1. INTRODUCTION using a single, undifferentiated mechanism. We find that the assumption behind this conclusion is flawed: This paper introduces Tebaldi, a transactional key-value store that cross-partition conflicts can in fact throttle the performance benefits takes significant steps towards harnessing the performance opportu- of federation (§2) as a perfect partitioning of conflicts is, in general, federation nities offered by a of optimized concurrency controls by unfeasible. In practice, there is often an inescapable tension between hierarchically allowing these mechanisms to be composed . minimizing cross-partition conflicts and supporting aggressive CC The rationale behind federating CC mechanisms [17, 34, 42, 45, mechanisms within each partition. 56] is straightforward: any single CC technique is bound to make Tebaldi, the new transactional key-value store that this paper in- trade-offs or rely on assumptions that cause it to perform extremely troduces, seeks to resolve this tension with a simple, but powerful, well in some cases but poorly in others. For instance, pessimistic insight: the mechanism by which different concurrency controls are techniques such as two-phase locking [10] perform well in highly- federated should itself be a federation. This flexibility, applied hier- contended workloads, but may lead to write transactions unnecessar- archically at the level of cross-partition CC mechanisms, is key to ily stalling read transactions, while multiversioned CC algorithms realizing the performance potential of federation. Tebaldi’s starting improve read performance, but introduce non-serializable behaviors point is Modular Concurrency Control (MCC) [56], perhaps the most that are difficult to detect [24, 35]. Since concurrent transactions recent expression of the federated-CC approach. MCC proposes to interact in fundamentally different ways across these scenarios, these partition transactions in groups, giving each group the flexibility to trade-offs appear unavoidable. A promising approach is instead to run its own private concurrency control mechanism. Charged with federate different CC mechanisms within the same database, apply- regulating concurrency only for the transactions within their own ing each given CC only to the transactions or workloads where it groups, these mechanisms can be very aggressive and yield more con- shines, while maintaining the overall correctness of the database. currency while still upholding safety. MCC imposes no restrictions In practice, however, realizing the performance potential of a fed- on the kind of transactions or CCs that it can handle, and emphasizes erated solution is challenging. Such a solution should be modular: it modularity: as long as any given isolation property holds within each ∗Chao Xie is currently working at Google. group, MCC guarantees that it will also hold among transactions in Permission to make digital or hard copies of all or part of this work for personal or different groups. The question at the core of this paper is then simply: classroom use is granted without fee provided that copies are not made or distributed how should this guarantee be enforced? for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM Callas [56], the distributed database that represents the current must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, embodiment of the MCC vision, offers a conservative answer: an to post on servers or to redistribute to lists, requires prior specific permission and/or a 1 fee. Request permissions from [email protected]. inflexible mechanism based on two-phase locking. Tebaldi aims SIGMOD ’17, May 14–19, 2017, Chicago, IL, USA. 1 c 2017 ACM. ISBN 978-1-4503-4197-4/17/05. $15.00 Renata Tebaldi was Maria Callas’s main rival on the opera stage. DOI: http://dx.doi.org/10.1145/3035918.3064031 to take MCC to a new level—both figuratively and, indeed, literally. Customer Conflict Instead of handling cross-group conflicts through a single mecha- Data Warehouse nism, Tebaldi regulates them by applying MCC recursively, adding dependency additional levels to its tree-like structure of federated CC mecha- Item District nisms. Refining the management of cross-group conflicts in this way Stock increases flexibility in how conflicts are handled, which, in turn, im- District Order Line proves performance. Tebaldi can, for example, combine the benefits Order of multiversioning [9, 12] with aggressive single version techniques Stock Preferred Execution Order New-Order such as runtime pipelining [56] at the cross-group layer. Read/Write Realizing this vision in Tebaldi presents two main technical hur- Order Line Read dles. First, directly applying CCs hierarchically does not guarantee New Order Stock level serializability (§4). We derive a sufficient condition—consistent or- dering—that CCs must enforce to ensure correctness, and highlight Figure 1: TPC-C new order/stock level transactions. how this property can be achieved in practice. Second, federating ability to aggressively handle conflicts. Mechanisms like two-phase different CC mechanisms requires seamlessly managing the differ- locking [10] or optimistic concurrency control [27] make few as- ent expectations upon which their correctness depends (in terms of sumptions about the application or the system, but are often overly protocol, storage, failure recovery, etc.): Tebaldi allows CCs to inde- pessimistic in dealing with conflicts. Most optimizations, however, pendently implement the execution logic (including maintaining the rely on properties that are unlikely to hold in general, such as full necessary metadata) for making ordering decisions, but provides a knowledge of the read and write set [23, 47], lack of SC cycles [44], general framework for composing the CCs’ execution hierarchically the ability to statically determine a total order of tables [56], or and determining the appropriate version of data to read or write. access locality [25, 29]. By federating these mechanisms, one can We show that Tebaldi’s hierarchical MCC yields significant per- restrict the scope of these optimizations only to the portions of the formance gains: running the TPC-C benchmark under serializability application for which their assumptions hold, allowing for higher shows that Tebaldi yields more than 20× throughput improvement performance without sacrificing generality. Consider, for example, over the basic two-phase locking protocol, and a 3:7× throughput runtime pipelining (RP) [56] and deterministic concurrency control increase over Callas [56], the state-of-the-art federated system. (DCC) [23, 47]. Runtime pipelining efficiently pipelines transac- In summary, this paper makes the following contributions: tions by allowing operations to observe the result of uncommitted • It introduces a hierarchical approach to federating concurrency writes, maintaining correctness through a combination of static anal- controls that allows conflicts to be handled more efficiently, ysis and runtime techniques. RP is most effective when transactions while preserving modularity. generate few circular dependencies when accessing tables. As the • It identifies a condition for the correct composition of concur- number of transactions grows, however, such dependencies are in- rency controls within Tebaldi’s hierarchy and shows that sev- creasingly likely. RP is, for instance, of limited use when applied to eral existing concurrency controls can be modified to enforce it. the full TPC-C (Figure 1), as there exists a circular dependency in the • It presents the design and evaluation of Tebaldi, a transactional new order and stock level transactions