Low Latency for Cloud Data Management Dissertation with the aim of achieving a doctoral degree at the Faculty of Mathematics, Informatics, and Natural Sciences Submitted at the University of Hamburg by Felix Gessert, 2018 Day of oral defense: December 18th, 2018 The following evaluators recommend the admission of the dissertation: Prof. Dr. Norbert Ritter Prof. Dr. Stefan Deßloch Prof. Dr. Mathias Fischer There are only two hard things in Computer Science: cache invalidation, naming things, and off-by-one errors. – Anonymous ii iii Acknowledgments This dissertation would not have been possible without the support and hard work of numerous other people. First and foremost, I would like to thank my advisor Prof. Norbert Ritter for his help and mentoring that enabled this research. Not only has he always given me the freedom and patience to execute my ideas in different directions, but he has formed my perception that academic research should eventually be practically applicable. Therefore, he is one of the key persons that enabled building a startup from this research. I also deeply enjoyed our joint workshops, talks, tutorials, and conference presentations with the personal development these opportunities gave rise to. I am convinced that without his mentoring and pragmatic attitude neither my research nor entrepreneurial efforts would have worked out this well. I would also like to express my gratitude to my co-advisor Prof. Stefan Deßloch. His insightful questions and feedback on different encounters helped me improve the overall quality of this work. My sincerest thanks also go to my co-founders Florian Bücklers, Hannes Kuhlmann, and Malte Lauenroth. The extensive discussions with Florian and our relentless efforts to build useful software are the cornerstone this work is built upon. Without this set of technically skilled and personally generous co-founders, this research would never have led to Baqend. I am excitedly looking forward to what more we will be able to achieve together. I was fortunate to work with exceptional colleagues and co-researchers that made this work not only fruitful, but enjoyable. Wolfram Wingerath provided not only the most critical feedback, but also the most valuable one. At many occasions, our brainstorming sessions sparked pivotal new approaches. Michael Schaarschmidt offered an enthusiastic stream of helpful ideas and new perspectives. Erik Witt contributed numerous improve- ments to this work as well as inspirations for clarity of thought and writing. Fabian Panse was always available for stimulating conversations and advice. Steffen Friedrich could be counted on to keep the teaching and organizational matters on course. The same is true for Anne Awizen, who never got tired of reminding me about deadlines, forms, and other things that I would surely have missed otherwise. Many others provided helpful feedback or essential work on Orestes during the course of writing this dissertation, including Konstantin Möllers, Kevin Twesten, Sven Decken, Jörn iv Domnik, Julian Tiemann, Julian Schenkemeyer, Nils Gessert, Dirk Bade, Uta Störl, Meike Klettke, and Stefanie Scherzinger. Most of all, I would like to thank my wife for her encouragement throughout these chal- lenging and thrilling times. Finally, I am also deeply grateful for the support from my family and friends. Felix Gessert Hamburg, September 24th, 2018 v Abstract English With the rise of scalable, distributed web applications, latency has become a fundamental challenge for cloud data management. The delays caused by accessing data from cloud services often dominate the performance of web and mobile applications. While modern data management systems address the need for higher scalability and fault tolerance, low latency remains an open issue. How can low-latency queries and reads be enabled without sacrificing central data management abstractions such as consistency levels and transactional isolation? In this thesis, we investigate caching in cloud data management for dynamic data ranging from database objects to complex query results. In the context of distributed systems, achieving reads that are both fast and consistent is tied to the challenge of maintaining fresh replicated data in proximity to clients. Therefore, we propose the data manage- ment platform Orestes that enhances existing NoSQL database systems with low latency. Orestes introduces several new techniques to substantially improve latency in cloud data management. First, it leverages the expiration-based model of web caches available all over the world through a novel cache coherence scheme – Cache Sketches. Our approach thus makes caching applicable to highly volatile cloud data while maintaining rigorous consistency levels. Second, transactions are a key concept often sacrificed in state-of-the- art systems for performance reasons. Therefore, we propose an approach for horizontally scalable, low-latency ACID transactions that can be added on top of existing database systems. Third, to enable polyglot persistence, we survey the field of scalable data man- agement and derive a novel classification scheme that relates database implementation techniques to functional and non-functional guarantees. By combining these findings in a unified data management interface, Orestes can provide existing systems as a scalable, low-latency Database-as-a-Service. Fourth, with the design of a polyglot persistence me- diator, we argue that the selection of suitable database systems for a given set of require- ments can be automated based on service level agreements. Finally, we provide evidence that for typical web applications and database workloads, our approach can improve la- tency by more than an order of magnitude compared to traditional cloud-hosted backends and database systems. vi Abstract German Mit der Verbreitung skalierbarer und verteilter Webanwendungen sind Zugriffslatenzen zu einer grundlegenden Herausforderung für das Cloud Data Management geworden. Die Verzögerungen bei der Abfrage von Daten aus Cloud-Diensten dominieren oft die Perfor- mance von Web- und mobilen Anwendungen. Während moderne Datenmanagementsys- teme den Bedarf nach höherer Skalierbarkeit und Fehlertoleranz adressieren, bleibt die Latenz eine offene Herausforderung. Wie können Lesezugriffe und Queries mit geringer Latenz beantwortet werden, ohne dabei zentrale Abstraktionen des Datenmanagements wie Konsistenzstufen und transaktionale Isolation aufzugeben? In dieser Arbeit untersuchen wir Caching im Cloud Data Management für dynamische Daten von Datenbankobjekten bis hin zu komplexen Query-Ergebnissen. Im Kontext verteilter Systeme sind schnelle und korrekte Lesezugriffe mit der Herausforderung ver- bunden, replizierte Daten konsistent in physischer Nähe zu Usern vorzuhalten. Aus diesem Grund führen wir die Data Management-Plattform Orestes ein, um die Latenzen beste- hender NoSQL-Datenbanksysteme zu verringern. Orestes verwendet mehrere neue Tech- niken, mit denen die Latenzen lesender Operationen im Cloud Data Management erhe- blich verbessert werden. Erstens nutzt es das expirationsbasierte Modell von Web-Caches, die über ein neues Cachekohärenz-Verfahren namens Cache Sketches aktuell gehalten wer- den. Unser Ansatz macht Caching somit auch für sehr volatile Cloud-Daten anwendbar und stellt dabei konfigurierbare Konsistenzgarantien sicher. Zweitens sind Transaktionen ein Kernkonzept des Datenmanagements, auf das in modernen Systemen oft aus Perfor- mancegründen verzichtet wird. Daher schlagen wir einen Ansatz für horizontal skalier- bare ACID-Transaktionen mit geringen Latenzen vor, der auf bestehende Datenbanksys- teme anwendbar ist. Drittens leiten wir für polyglotte Persistenz durch eine genaue Anal- yse verfügbarer Ansätze ein Klassifikationsschema ab, das die Implementierungstechniken der Datenbanksysteme mit funktionalen und nicht-funktionalen Garantien in Beziehung setzt. Durch die Anwendung der Systematik auf eine vereinheitlichte Datenmanagement- Schnittstelle kann Orestes bestehende Systeme als skalierbares Database-as-a-Service mit geringer Latenz anbieten. Viertens zeigen wir mit dem Design eines Polyglot Persistence Mediators, dass die Auswahl geeigneter Datenbanksysteme auf Basis von Service Level Agreements automatisiert werden kann. Abschließend belegen wir quantitativ, dass unser Ansatz für typische Webanwendungen und Datenbank-Workloads die Latenz um mehr als eine Größenordnung gegenüber herkömmlichen Backends und Datenbanksystemen verbessert. vii Contents Abstract v 1 Introduction 1 1.1 Problem Statement . 4 1.2 Challenges . 6 1.3 Primary Contributions . 7 1.3.1 Web Caching of Records and Query Results with Rich Consistency Guarantees . 9 1.3.2 A Database-as-a-Service Middleware for Scalable Web Applications . 9 1.3.3 Polyglot Persistence Mediation with Database-Independent Caching and ACID Transactions . 10 1.4 Outline and Previously Published Work . 11 1.5 List of Own Publications . 12 2 Background 15 2.1 End-to-End Latency in Cloud-based Architectures . 15 2.1.1 Three-Tier Architectures: Server-Side Rendering . 16 2.1.2 Two-Tier Architectures: Client-Side Rendering . 20 2.1.3 Latency and Round-Trip Time . 24 2.1.4 Cloud Computing as a Source of Latency . 25 2.2 Backend Performance: Scalable Data Management . 29 2.2.1 NoSQL Database Systems . 30 2.2.2 Different Data Models . 31 2.2.3 Latency, Consistency, and Availability: Trade-Offs . 33 2.2.4 Relaxed Consistency Models . 35 2.2.5 Polyglot Persistence . 41 2.2.6 Cloud Data Management: Database- and Backend-as-a-Service . 47 2.2.7 Latency