Scalable Analytics on Serverless Infrastructure
Total Page：16
File Type：pdf, Size：1020Kb
Shuffling, Fast and Slow: Scalable Analytics on Serverless Infrastructure Qifan Pu (UC Berkeley), Shivaram Venkataraman (UW Madison), Ion Stoica (UC Berkeley) Abstract workload requirements can be dynamically matched with Serverless computing is poised to fulfill the long-held continuous scaling. promise of transparent elasticity and millisecond-level pric- Fine-grained elasticity in serverless platforms is natu- ing. To achieve this goal, service providers impose a fine- rally useful for on-demand applications like creating image grained computational model where every function has a thumbnails [18] or processing streaming events [26]. How- maximum duration, a fixed amount of memory and no persis- ever, we observe such elasticity also plays an important role tent local storage. We observe that the fine-grained elasticity for data analytics workloads. Consider for example an ad- of serverless is key to achieve high utilization for general hoc data analysis job exemplified by say TPC-DS query computations such as analytics workloads, but that resource 95 [34] (See section 5 for more details). This query con- limits make it challenging to implement such applications as sists of eight stages and the amount of input data at each they need to move large amounts of data between functions stage varies from 0.8MB to 66GB. With a cluster of virtual that don’t overlap in time. In this paper, we present Locus, machines users would need to size the cluster to handle the a serverless analytics system that judiciously combines (1) largest stage leaving resources idle during other stages. Us- cheap but slow storage with (2) fast but expensive storage, ing a serverless platform can improve resource utilization as to achieve good performance while remaining cost-efficient. resources can be immediately released after use. Locus applies a performance model to guide users in select- However, directly using a serverless platform for data an- ing the type and the amount of storage to achieve the desired alytics workloads could lead to extremely inefficient execu- cost-performance trade-off. We evaluate Locus on a number tion. For example we find that running the CloudSort bench- of analytics applications including TPC-DS, CloudSort, Big mark [40] with 100TB of data on AWS Lambda, can be up Data Benchmark and show that Locus can navigate the cost- to 500 slower (Section 2.3) when compared to running on ⇥ performance trade-off, leading to 4 -500 performance im- a cluster of VMs. By breaking down the overheads we find ⇥ ⇥ provements over slow storage-only baseline and reducing re- that the main reason for the slowdown comes from slow data source usage by up to 59% while achieving comparable per- shuffle between asynchronous function invocations. As the formance with running Apache Spark on a cluster of virtual ephemeral, stateless compute units lack any local storage, machines, and within 2 slower compared to Redshift. and as direct transfers between functions is not always feasi- ⇥ ble1, intermediate data between stages needs to be persisted 1 Introduction on shared storage systems like Amazon S3. The character- istics of the storage medium can have a significant impact The past decade has seen the widespread adoption of cloud on performance and cost. For example, a shuffle from 1000 computing infrastructure where users launch virtual ma- map tasks to 1000 reduce tasks leads to 1M data blocks being chines on demand to deploy services on a provisioned clus- created on the storage system. Therefore, throughput limits ter. As cloud computing continues to evolve towards more of object stores like Amazon S3 can lead to significant slow elasticity, there is a shift to using serverless computing, downs (Section 2.3). where storage and compute is separated for both resource Our key observation is that in addition to using elas- provisioning and billing. This trend was started by ser- tic compute and object storage systems we can also pro- vices like Google BigQuery [9], and AWS Glue [22] that vision fast memory-based resources in the cloud, such as provide cluster-free data warehouse analytics, followed by in-memory Redis or Memcached clusters. While naively services like Amazon Athena[5] that allow users to per- putting all data in fast storage is cost prohibitive, we can ap- form interactive queries against a remote object storage with- propriately combine fast, but expensive storage with slower out provisioning a compute cluster. While the aforemen- but cheaper storage, similar to the memory and disk hierar- tioned services mostly focus on providing SQL-like analyt- chy on a local machine, to achieve the best of both worlds: ics, to meet the growing demand, all major cloud providers approach the performance of a pure in-memory execution at now offer “general” serverless computing platforms, such as a significantly lower cost. However, achieving such a sweet AWS Lambda, Google Cloud Functions, Azure Functions spot is not trivial as it depends on a variety of configuration and IBM OpenWhisk. In these platforms short-lived user- parameters, including storage type and size, degree of task defined functions are scheduled and executed in the cloud. parallelism, and the memory size of each serverless func- Compared to virtual machines, this model provides more 1Cloud providers typically provide no guarantees on concurrent execu- fine-grained elasticity with sub-second start-up times, so that tion of workers. tion. This is further exacerbated by the various performance 2 Background limits imposed in a serverless environment (Section 2.4). In this paper we propose Locus, a serverless analytics sys- We first present a brief overview of serverless computing and tem that combines multiple storage types to achieve better compare it with the traditional VM-based instances. Next we performance and resource efficiency. In Locus, we build a discuss how analytics queries are implemented on serverless performance model to aid users in selecting the appropriate infrastructure and present some of the challenges in execut- storage mechanism, as well as the amount of fast storage ing large scale shuffles. and parallelism to use for map-reduce like jobs in server- less environments. Our model captures the performance and 2.1 Serverless Computing: What fits? cost metrics of various cloud storage systems and we show how we can combine different storage systems to construct Recently, cloud providers and open source projects [25, 32] hybrid shuffle methods. Using simple micro-benchmarks, have proposed services that execute functions in the cloud we model the performance variations of storage systems as or providing Functions-as-a-Service. As of now, these func- other variables like serverless function memory and paral- tions are subject to stringent resource limits. For example, lelism change. AWS Lambda currently imposes a 5 minute limit on function We evaluate Locus on a number of analytics applications duration and 3GB memory limit. Functions are also assumed including TPC-DS, Daytona CloudSort and the Big Data to be stateless and are only allocated 512MB of ephemeral Benchmark. We show that using fine-grained elasticity, Lo- storage. Similar limits are applied by other providers such cus can reduce cluster time in terms of total core seconds as Google Cloud Functions and Azure Functions. Regard- · by up to 59% while being close to or beating Spark’s query less of such limitations, these offerings are popular among completion time by up to 2 . We also show that with a small users for two main reasons: ease of deployment and flexi- ⇥ amount of fast storage, for example, with fast storage just ble resource allocation. When deploying a cluster of virtual large enough to hold 5% of total shuffle data, Locus matches machines, users need to choose the instance type, number Apache Spark in running time on CloudSort benchmark and of instances, and make sure these instances are shutdown is within 13% of the cost of the winning entry in 2016. While when the computation finishes. In contrast, serverless of- we find Locus to be 2 slower when compared to Ama- ferings have a much simpler deployment model where the ⇥ zon Redshift, Locus is still a preferable choice to Redshift functions are automatically triggered based on events, e.g., since it requires no provisioning time (vs. minutes to setup a arrival of new data. Redshift cluster) or knowing an optimal cluster size before- Furthermore, due to their lightweight nature, containers hand. Finally, we also show that our model is able to accu- used for serverless deployment can often be launched within rately predict shuffle performance and cost with an average seconds and thus are easier to scale up or scale down when error of 15.9% and 14.8%, respectively, which allows Locus compared to VMs. The benefits of elasticity are especially to choose the most appropriate shuffle implementation and pronounced for workloads where the number of cores re- other configuration variables. quired varies across time. While this naturally happens for event-driven workloads for example where say users upload In summary, the main contributions of this paper are: a photo to a service that needs to be compressed and stored, we find that elasticity is also important for data analytics We study the problem of executing general purpose data workloads. In particular, user-facing ad-hoc queries or ex- • analytics on serverless platforms to exploit fine-grained ploratory analytics workloads are often unpredictable yet elasticity and identify the need for efficient shuffles. have more stringent responsiveness requirements, making it more difficult to provision a traditional cluster compared to We show how using a small amount of memory-based • recurring production workloads. fast storage can lead to significant benefits in perfor- We present two common scenarios that highlight the im- mance while remaining cost effective. portance of elasticitiy. First, consider a stage of tasks being To aid users in selecting the appropriate storage mech- • run as a part of an analytics workload.