Pocket: Elastic Ephemeral Storage for Serverless Analytics Ana Klimovic1 Yawen Wang1 Patrick Stuedi 2 Animesh Trivedi2 Jonas Pfefferle2 Christos Kozyrakis1 1 Stanford University 2 IBM Research Abstract Pareto frontier Serverless computing is becoming increasingly popu- lar, enabling users to quickly launch thousands of short- lived tasks in the cloud with high elasticity and fine- grain billing. These properties make serverless comput- ing appealing for interactive data analytics. However exchanging intermediate data between execution stages in an analytics job is a key challenge as direct commu- nication between serverless tasks is difficult. The nat- ural approach is to store such ephemeral data in a re- mote data store. However, existing storage systems are not designed to meet the demands of serverless applica- tions in terms of elasticity, performance, and cost. We present Pocket, an elastic, distributed data store that au- Figure 1: Example of performance-cost trade-off for tomatically scales to provide applications with desired a serverless video analytics job using different storage performance at low cost. Pocket dynamically rightsizes technologies and VM types in Amazon EC2 resources across multiple dimensions (CPU cores, net- work bandwidth, storage capacity) and leverages multi- able parallelism in an analytics job and achieve near real- ple storage technologies to minimize cost while ensuring time performance [32, 45, 27]. In contrast to traditional applications are not bottlenecked on I/O. We show that serverless applications that consist of a single function Pocket achieves similar performance to ElastiCache Re- executed when a new request arrives, analytics jobs typ- dis for serverless analytics applications while reducing ically consist of multiple stages and require sharing of cost by almost 60%. state and data across stages of tasks (e.g., data shuffling). Most analytics frameworks (e.g., Spark) implement 1 Introduction data sharing with a long-running framework agent on each node buffering intermediate data in local stor- Serverless computing is becoming an increasingly popu- age [78]. This enables tasks from different execution lar cloud service due to its high elasticity and fine-grain stages to directly exchange intermediate data over the billing. Serverless platforms like AWS Lambda, Google network. However, in serverless deployments, there is Cloud Functions, and Azure Functions enable users to no long-running application framework agent to manage quickly launch thousands of light-weight tasks, as op- local storage. Furthermore, serverless applications have posed to entire virtual machines. The number of server- no control over task scheduling or placement, making di- less tasks scales automatically based on application de- rect communication among tasks difficult. As a result of mands and users are charged only for the resources their these limitations, the natural approach for data sharing in tasks consume, at millisecond granularity [17, 36, 56]. serverless applications is to use a remote storage service. While serverless platforms were originally developed For instance, early frameworks for serverless analytics for web microservices and IoT applications, their elas- either use object stores (e.g., S3 [16]), databases (e.g., ticity and billing advantages make them appealing for CouchDB [1]) or distributed caches (e.g., Redis [51]). data intensive applications such as interactive analytics. Unfortunately, existing storage services are not a good Several recent frameworks launch large numbers of fine- fit for sharing short-lived intermediate data in server- grain tasks on serverless platforms to exploit all avail- less applications. We refer to the intermediate data as ephemeral data to distinguish it from input and out- In summary, our contributions are as follows: put data which requires long-term storage. File sys- tems, object stores and NoSQL databases prioritize pro- • We identify the key characteristics of ephemeral viding durable, long-term, and highly-available storage data in serverless analytics and synthesize require- rather than optimizing for performance and cost. Dis- ments for storage platforms used to share such data tributed key-value stores offer good performance, but among serverless tasks. burden users with managing the storage cluster scale and • We introduce Pocket, a distributed data store whose configuration, which includes selecting the appropriate control, metadata and data planes are designed compute, storage and network resources to provision. for sub-second response times, automatic resource The availability of different storage technologies (e.g., scaling and intelligent data placement across stor- DRAM, NVM, Flash, and HDD) increases the complex- age tiers. To our knowledge, Pocket is the first plat- ity of finding the best cluster configuration for perfor- form targeting data sharing in serverless analytics. mance and cost. However, the choice of storage tech- • We show that Pocket’s data plane delivers sub- nology is critical since jobs may exhibit different stor- millisecond latency and scalable bandwidth while age latency, bandwidth and capacity requirements while the control plane rightsizes resources based on the different storage technologies vary significantly in terms number of jobs and their attributes. For a video an- of their performance characteristics and cost [48]. As alytics job, Pocket reduces the average time server- an example, Figure1 plots the performance-cost trade- less tasks spend on ephemeral I/O by up to 4.1× off for a serverless video analytics application using a compared to S3 and achieves similar performance distributed ephemeral data store configured with differ- to ElastiCache Redis while saving 59% in cost. ent storage technologies, number of nodes, compute re- sources per node, and network bandwidth (see §6.1 for Pocket is open-source software. The code is available our AWS experiment setup). Each resource configura- at: https://github.com/stanford-mast/pocket. tion leads to different performance and cost. Finding Pareto efficient storage allocations for a job is non-trivial 2 Storage for Serverless Analytics and gets more complicated with multiple jobs. We present Pocket, a distributed data store designed Early work in serverless analytics has identified the chal- for efficient data sharing in serverless analytics. Pocket lenge of storing and exchanging data between hundreds offers high throughput and low latency for arbitrary size of fine-grain, short-lived tasks [45, 32]. We build on our data sets, automatic resource scaling, and intelligent data study of ephemeral storage requirements for serverless placement across multiple storage tiers such as DRAM, analytics applications [49] to synthesize essential prop- Flash, and disk. The unique properties of Pocket result erties for an ephemeral data storage solution. We also from a strict separation of responsibilities across three discuss why current systems are not able to meet the planes: a control plane which determines data placement ephemeral I/O demands of serverless analytics applica- policies for jobs, a metadata plane which manages dis- tions. Our focus is on ephemeral data as the original in- tributed data placement, and a ‘dumb’ (i.e., metadata- put and final output data of analytics jobs typically has oblivious) data plane responsible for storing data. Pocket long-term availability and durability requirements that scales all three planes independently at fine resource and are well served by the variety of file systems, object time granularity based on the current load. Pocket uses stores, and databases available in the cloud. heuristics, which take into account job characteristics, to allocate the right storage media, capacity, bandwidth and CPU resources for cost and performance efficiency. The 2.1 Ephemeral Storage Requirements storage API exposes deliberately simple I/O operations High performance for a wide range of object sizes: for sub-millisecond access latency. We intend for Pocket Serverless analytics applications vary considerably in the to be managed by cloud providers and offered to users way they store, distribute, and process data. This diver- with a pay-what-you-use cost model. sity is reflected in the granularity of ephemeral data that We deploy Pocket on Amazon EC2 and evaluate the is generated during a job. Figure2 shows the ephemeral system using using three serverless analytics workloads: object size distribution for a distributed lambda compi- video analytics, MapReduce sort, and distributed source lation of the cmake program, a serverless video analyt- code compilation. We show that Pocket is capable ics job using the Thousand Island (THIS) video scan- of rightsizing the type and number of resources such ner [63], and a 100 GB MapReduce sort job on lamb- that jobs achieve similar performance compared to us- das. The key observation is that ephemeral data ac- ing ElastiCache Redis, a DRAM-based key-value store, cess granularity varies greatly in size, ranging from hun- while saving almost 60% in cost. dreds of bytes to hundreds of megabytes. We observe a 1.0 1.0 λ-cc cmake 0.8 video analytics 0.8 sort100GB 0.6 0.6 CDF CDF 0.4 0.4 λ-cc cmake 0.2 0.2 video-analytics sort 100 GB 0.0 1 2 3 4 5 6 7 8 0.0 10 10 10 10 10 10 10 10 0 20 40 60 80 100 Ephemeral Object Size (bytes) Ephemeral Object Lifetime (seconds) Figure 2: Objects are 100s of bytes to 100s of MBs. Figure 3: Objects have short lifetime. straight line for sorting as its ephemeral data size is equal The variety of storage media available in the cloud al- to the partition size. However, with a different dataset low for different performance-cost trade-offs, as shown size and/or number of workers, the location of the line in Figure1. Each storage technology differs in terms of changes. Applications that read/write large objects de- I/O latency, throughput and IOPS per GB of capacity, mand high throughput (e.g., we find that sorting 100 GB and the cost per GB. The optimal choice of storage me- with 500 lambdas requires up to 7.5 GB/s of ephemeral dia for a job depends on its characteristics.