Deadline-Driven Serverless for the Edge
Total Page:16
File Type:pdf, Size:1020Kb
SledgeEDF: Deadline-driven Serverless for the Edge by Sean Patrick McBride B.S. in German, May 2007, United States Military Academy M.A.S. in Information Technology and Management, May 2013, Illinois Institute of Technology A Thesis submitted to The Faculty of The School of Engineering and Applied Science of The George Washington University in partial satisfaction of the requirements for the degree of Master of Science January 8, 2021 Thesis directed by Gabriel Parmer Associate Professor of Computer Science c Copyright 2021 by Sean Patrick McBride All rights reserved ii Dedication This thesis is dedicated to the many educators, mentors, and battle buddies that have helped me grow as a technologist, software engineer, and computer scientist. Sankaran Iyer, Susan Schwartz, Chris Okasaki, Christa Chewar, Ray Trygstad, Jeremy Hajek, Jeffrey Kimont, Robert Hendry, Carol Davids, Bill Slater, Bonnie Goins, David Gaertner, Andy Quintana, Patricia Schatz, Wayne Bucek, Pat Medo, Lih Wang, Tony Liu, Bill Seubert, Marty Horan, Fred Bader, Mitch Green, Bob Kaas, Richard Lewis, Gwen Dente, Ray Mullins, Frank DeGilio, Paul Novak, Bruce Hayden, Art Breslau, Chris Ganim, Mark Woehrer, Will Dory, Steve Payne, Walt Melo, Mark Davis, Omri Bernstein, Eliot Szwajkowski, Dani Young-Smith, Conrad Holloman, David Tillery, Garth Hershfield, Daniel Cox, Doug Fort, Jeff Hemminger, Josh Rutherford, Hiromi Suenaga, Kait Moreno, Howie Huang, Ben Bowman, Yuede Ji, Pradeep Kumar, Nahid Ghalaty, Roozbeh Haghnazar, Morris Lancaster, Gabe Parmer, Phani Kishoreg, and the unnamed others I’ve forgotten. I am mostly an ambulatory accumulation of the investments others have made in me over the years. I hope that you consider me a worthy investment, and I pledge to pay this forward! iii Abstract SledgeEDF: Deadline-driven Serverless for the Edge Serverless Computing has gained mass popularity by offering lower cost, improved elasticity, and improved ease of use. Driven by the need for efficient low latency computation on resource-constrained infrastructure, it is also becoming a common execution model for edge computing. However, hyperscale cloud mitigations against the serverless cold start problem do not cleanly scale down to tiny 10-100kW edge sites, causing edge deployments of existing VM and container-based serverless runtimes to suffer poor tail latency (the slowest latency in the distribution, typically expressed as the 99th percentile). This is particularly acute considering that future edge computing workloads are expected to have latency requirements ranging from microseconds to seconds. SledgeEDF is the first runtime to apply the traditional real-time systems techniques of admissions control and deadline-driven scheduling to the serverless execution model. It extends previous research on aWsm, an ahead-of-time (AOT) WebAssembly compiler, and Sledge, a single-process WebAssembly-based serverless runtime designed for the edge, yielding a runtime that targets efficient execution of mixed-criticality edge workloads. Evaluations demonstrate that SledgeEDF prevents backpressure due to excessive client requests and eliminates head-of-line blocking, allowing latency-sensitive high-criticality requests to preempt executing tasks and complete within 10% of their optimal execution time. Taken together, SledgeEDF’s admissions controller and deadline-driven scheduler enable it to provide limited guarantees around latency deadlines defined by client service level objectives. iv Table of Contents Dedication ...................................... iii Abstract ........................................ iv 1 Introduction .................................... 1 1.1 Contributions . .2 2 Background .................................... 3 2.1 Serverless . .3 2.2 Edge Computing . .5 2.3 WebAssembly . .7 3 Related Work ................................... 11 3.1 Serverless Research . 11 3.2 Edge Computing Research . 12 3.3 WebAssembly Research . 13 3.4 Comparisons with SledgeEDF . 13 4 Design ....................................... 15 4.1 Baseline Functionality . 16 4.2 Design Changes . 17 5 Implementation .................................. 18 5.1 Enabling Refactors . 18 5.1.1 Processor Aware . 18 5.1.2 Sandbox State Machine . 18 5.1.3 Polymorphic Scheduling Interface . 19 5.1.4 Module Specification Changes . 19 5.1.5 Removal of libuv . 20 5.2 EDF Scheduling . 20 5.3 Admissions Control . 22 5.4 SledgeEDF . 24 6 Evaluation ..................................... 25 6.1 Deadline-based Scheduling . 25 6.2 Admissions Control . 28 6.3 System Overheads . 29 7 Conclusion ..................................... 31 7.1 Future Work . 31 Bibliography ..................................... 32 v Chapter 1: Introduction Serverless Computing or Functions-as-a-Service (FaaS) is the first novel cloud-native service to have gained mass popularity, largely by offering lower cost, improved elasticity, and improved ease of use. Certain classes of startups are adopting the so-called Backend-as-a- Service (BaaS) pattern, whereby a company focuses on highly differentiated web or mobile apps and implements all backend logic using serverless functions orchestrating managed cloud services, allowing them to scale with minimal distributed systems or operations expertise. Inspired by the needs of IOT and smart devices, cloud services are migrating from hyperscale data centers to the edge of the network, resulting in a class of services called edge computing. Given the event-driven nature of certain classes of IOT devices, serverless is one of the more common cloud services to be deployed to the edge. However, while existing serverless services typically use container or VM-based deployment units in hyperscale cloud data centers, this approach is ill suited to the edge due to the cold start problem, a situation where, in the worst-case, all caches are cold, and a request must stall waiting for the deployment of an operating system and language runtime before being able to execute, resulting in unfavorable tail latency. Newer serverless runtimes have solved the cold-start problem by using user-level sandboxing technologies, most commonly WebAssembly. Some research and open-source systems also solve the research questions of how to efficiently perform stateful computation, such as big data or machine learning, using stateless functions. This thesis views these research questions as largely solved and examines an area as yet unexplored in the recent serverless research literature: scheduling. Generally, existing serverless runtimes use simple schedulers that provide weighted- fairness and optimize for efficient throughput at the expense of quality of service guarantees around request latency. This likely makes sense for serverless running in a hyperscale data center, as the risk of exhausting hardware capacity is effectively nil and users are accustomed 1 to tolerating occasional 100ms stalls caused my the cold start problem in exchange for low cost. However, serverless running on the edge deals with different constraints. Small scale edge infrastructure is unable to guarantee the illusion of infinite capacity, so edge serverless must be able to handle request outstripping available supply. Additionally, to be able to support hard and soft realtime systems that require low roundtrip latency, a runtime needs to be able to handle differentiated quality of service guarantees for different sorts of requests. A serverless request from an aerial quadcopter requiring 20ms roundtrip latency cannot indefinitely sit on a request queue because all processors are occupied executing long-running non-realtime tasks, such as data compression of footage from nearby video cameras that is subsequently uploaded to cloud object storage. This thesis presents SledgeEDF, a single-process WebAssembly-based serverless runtime designed to support differentiated quality of service guarantees in a resource-constrained edge environment. The theoretical basis for this work is derived from decades-old research into real-time scheduling and the deadline-driven execution of sporadic jobs [84]. However, after a thorough review of the existing literature, the author believes this to be the first application of these traditional techniques in the context of a serverless runtime. 1.1 Contributions This thesis presents SledgeEDF, a single-node serverless runtime optimized for running mixed-criticality workloads at the edge. Contributions include: • An Earliest Deadline First (EDF) scheduler designed in the context of a single-process serverless runtime. • A lightweight Admissions Controller for a serverless runtime running on limited edge infrastructure that cannot sustain the illusion of infinite capacity. • An evaluation demonstrating the efficacy of these components. 2 Chapter 2: Background 2.1 Serverless The value and use case of SledgeEDF is largely an extension of the serverless paradigm. This section provides a brief overview of serverless by contextualizing it in the larger ecosystem of cloud services, providing a high level overview, and discussing several common criticisms. In the fifteen years since Jeff Bezos’ announcement that Amazon would provide "pay by the drink" web services [95], most corporate IT departments have begun to migrate their critical software systems from on-premise data centers to the cloud. However, to date, much of this cloud adoption has simply involved the "lifting and shifting" of virtual machines from on-premise VMWare vCenter clusters to managed Infrastructure as a Service (IaaS), such as AWS EC2. Adoption of higher-level Platform