Session 6A: Datacenter/cloud power/performance — Managing ASPLOS’20, March 16–20, 2020, Lausanne, Switzerland the beast. Physics Experiments (with a particular eye on CERN LHC) Catalyzer: Sub-millisecond Startup for Serverless Computing with Initialization-less Booting Dong Duyz, Tianyi Yuyzx, Yubin Xiayz, Binyu Zangyz, Guanglu Yanx, Chenggang Qinx, Qixuan Wux, Haibo Chenyz y Shanghai Key Laboratory of Scalable Computing and Systems, Shanghai Jiao Tong University z Institute of Parallel and Distributed Systems, Shanghai Jiao Tong University x Ant Financial Services Group Abstract Catalyzer has been adopted by Ant Financial, and we also Serverless computing promises cost-efficiency and elasticity present lessons learned from industrial development. for high-productive software development. To achieve this, CCS Concepts • Computer systems organization → the serverless sandbox system must address two challenges: Cloud computing; • Software and its engineering → strong isolation between function instances, and low startup Operating systems. latency to ensure user experience. While strong isolation can be provided by virtualization-based sandboxes, the ini- Keywords serverless computing; startup latency; check- tialization of sandbox and application causes non-negligible point and restore; operating system startup overhead. Conventional sandbox systems fall short in ACM Reference Format: low-latency startup due to their application-agnostic nature: Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Cheng- they can only reduce the latency of sandbox initialization gang Qin, Qixuan Wu, Haibo Chen. 2020. Catalyzer: Sub-millisecond through hypervisor and guest kernel customization, which Startup for Serverless Computing with Initialization-less Booting. is inadequate and does not mitigate the majority of startup In Proceedings of the Twenty-Fifth International Conference on Archi- overhead. tectural Support for Programming Languages and Operating Systems This paper proposes Catalyzer, a serverless sandbox sys- (ASPLOS ’20), March 16–20, 2020, Lausanne, Switzerland. ACM, New tem design providing both strong isolation and extremely York, NY, USA, 15 pages. https://doi.org/10.1145/3373376.3378512 fast function startup. Instead of booting from scratch, Cat- alyzer restores a virtualization-based function instance from 1 Introduction a well-formed checkpoint image and thereby skips the ini- Serverless computing, the new trending paradigm in cloud tialization on the critical path (init-less). Catalyzer boosts the computing, liberates developers from the distraction of man- restore performance by on-demand recovering both user-level aging servers and has already been supported by many memory state and system state. We also propose a new OS platforms, including Amazon Lambda [2], IBM Cloud Func- primitive, sfork (sandbox fork), to further reduce the startup tion [1], Microsoft Azure Functions [3] and Google Cloud latency by directly reusing the state of a running sandbox in- Functions [7]. In serverless computing, the unit of compu- stance. Fundamentally, Catalyzer removes the initialization tation is a function. When a service request is received, the cost by reusing state, which enables general optimizations serverless platform allocates an ephemeral execution sand- for diverse serverless functions. The evaluation shows that box and instantiates a user-defined function to handle the Catalyzer reduces startup latency by orders of magnitude, request. This computing model shifts the responsibility of achieves <1ms latency in the best case, and significantly dynamically managing cloud resources to cloud providers, reduces the end-to-end latency for real-world workloads. allowing the developers to focus purely on their application logic. Besides, cloud providers can manage their resources more efficiently. Permission to make digital or hard copies of all or part of this work for The ephemeral execution sandboxes are typically con- personal or classroom use is granted without fee provided that copies tainers [1], virtual machines [20, 44] or recently proposed are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights lightweight virtualization designs [6, 8, 19, 35, 37, 41, 45]. for components of this work owned by others than the author(s) must However, container instances suffer from isolation issues be honored. Abstracting with credit is permitted. To copy otherwise, or since they share one kernel, which is error-prone. Virtual republish, to post on servers or to redistribute to lists, requires prior specific machines can achieve better isolation but are too heavy to permission and/or a fee. Request permissions from [email protected]. run serverless functions. Lightweight virtualization designs ASPLOS ’20, March 16–20, 2020, Lausanne, Switzerland like Google gVisor [8] and Amazon FireCracker [6] achieve © 2020 Copyright held by the owner/author(s). Publication rights licensed to ACM. high performance, easy resource management and strong iso- ACM ISBN 978-1-4503-7102-5/20/03...$15.00 lation by customizing the host-guest interfaces, e.g., gVisor https://doi.org/10.1145/3373376.3378512 uses a process abstraction interface. 467 Session 6A: Datacenter/cloud power/performance — Managing ASPLOS’20, March 16–20, 2020, Lausanne, Switzerland the beast. Physics Experiments (with a particular eye on CERN LHC) 1 gVisor The result shows the Catalyzer can achieve <1ms startup la- Catalyzer 0.8 tency on C-hello (best case), and <2ms to boot Java SPECjbb, 0.6 1000x speedup over baseline gVisor. We also present eval- 0.4 uations on server machines and share our lessons learned 0.2 from industrial development at Ant Financial. CDF of serverless functions 0 0 10 20 30 40 50 60 65.54 Execution Time/Latency (%) The main contributions of this paper are as follows: • Figure 1. Distribution of Execution/Overall latency ratio in A detailed analysis of latency overhead on serverless serverless computing. The ratio of all functions in gVisor can not computing (§2). even achieve 65.54%. The startup is cold boot. • A general design of Init-less booting that boosts startup of diverse serverless applications (§3 and §4). • An implementation of Catalyzer on a state-of-the-art Executing serverless functions with low latency is critical serverless sandbox system, Google gVisor (§5). for user experience [21, 24, 28, 32, 38], and is still a signifi- • An evaluation with micro-benchmarks and real-world cant challenge for virtualization-based sandbox design. To serverless applications proving the efficiency and prac- explain the severity, we conduct an end-to-end evaluation on ticability of Catalyzer (§6). three benchmarks, DeathStar [22], E-business microservices, • The experience of deploying Catalyzer on real plat- and image processing functions, and divide the latency into forms (§6.9). “execution” part and “boot” part (§6.4). We calculate the “Ex- ecution/Overall” ratio of the tested 14 serverless functions, 2 Serverless Function Startup Breakdown and present the CDF in Figure 1. The ratio of 12 functions In this section, we evaluate and analyze the startup latency (out of 14) in gVisor can not even achieve 30%, indicating of serverless platforms with different system sandboxes (i.e., that the startup dominates the overall latency. Long startup gVisor, FireCracker, Hyper Container, and Docker) and dif- latency, especially for virtualization-based sandbox, has be- ferent language runtimes. Based on evaluation and analysis, come a significant challenge for serverless platforms. we present our motivation that serverless functions should Existing VM-based sandboxes [6, 8, 37] reduce the startup be executed with an initialization-less approach. latency through hypervisor customization, e.g., FireCracker can boot a virtual machine (microVM) and a minimized Linux 2.1 Background kernel in 100ms. However, none of them can reduce the appli- Serverless Platform. In serverless computing, the devel- cation initialization latency like JVM or Python interpreter oper sends a function to the serverless platform to execute. setup time. Our studies on serverless functions (written by We use the term handler function to represent the target five programming languages) show that most of the startup function, which could be written in different languages. The latency comes from application initialization (Insight I). handler function is compiled offline together with a wrapper, This paper proposes Catalyzer, a general design to boost which does initialization and invokes the handler function. startup for serverless computing. The key idea of Catalyzer is Wrapped programs (consist of the wrapper and handler func- to restore an instance from a well-formed checkpoint image tion) execute safely within sandboxes, which can be con- and thereby skip the initialization on the critical path. The tainers [5, 40] or virtual machines (VM) [6, 8, 10]. There is a design is based on two additional insights: First, a server- gateway program running on each server as a daemon, which less function in execution stage typically accesses only a small accepts “invoke function” requests, and starts a sandbox with fraction of memory and files used in the initialization stage two arguments: a configuration file and a rootfs containing (Insight II), thus we can on-demand recover both appli- both the wrapped program and runtime libraries. The argu- cation state (e.g., data in memory) and system state (e.g., file ments are