Enabling End-host Network Functions ∗ Hitesh Ballani, Paolo Costa, Christos Gkantsidis, Matthew P. Grosvenor, y Thomas Karagiannis, Lazaros Koromilas, and Greg O’Shea Microsoft Research Cambridge, UK ABSTRACT Keywords Many network functions executed in modern datacen- Software Defined Networking; SDN; Network Manage- ters, e.g., load balancing, application-level QoS, and ment; Data-plane programming; Network Functions congestion control, exhibit three common properties at the data plane: they need to access and modify state, to 1 Introduction perform computations, and to access application seman- Recent years have seen a lot of innovation in functional- tics | this is critical since many network functions are ity deployed across datacenter networks. Network func- best expressed in terms of application-level messages. tions range from management tasks like load balanc- In this paper, we argue that the end hosts are a natural ing [65, 4, 26, 1, 40] and Quality of Service (QoS) [9, enforcement point for these functions and we present 39, 10, 33, 52, 61, 6, 28] to data-plane tasks like (central- Eden, an architecture for implementing network func- ized) congestion control and network scheduling [48, 27, tions at end hosts with minimal network support. 64, 45, 30] to application-specific tasks like replica selec- Eden comprises three components, a centralized con- tion [17]. Today, such functions are implemented using troller, an enclave at each end host, and Eden-compliant a mishmash of techniques and at a variety of places|at applications called stages. To implement network func- network switches using SDN and OpenFlow, at physi- tions, the controller configures stages to classify their cal or virtual middleboxes using NFV, and at end hosts data into messages and the enclaves to apply action through Open vSwitch or custom implementations. functions based on a packet's class. Our Eden pro- totype includes enclaves implemented both in the OS Despite their differences, three common requirements kernel and on programmable NICs. Through case stud- characterize a large fraction of network functions: i) ies, we show how application-level classification and the they maintain state at the data plane, ii) they perform ability to run actual programs on the data-path allows computation at the data plane, and iii) they operate on Eden to efficiently support a broad range of network application semantics. The last feature is particularly functions at the network's edge. important since many network functions are best ex- pressed in terms of application data units or\messages". CCS Concepts For example, a load balancing function for memcached, a popular key-value store, may put its messages into •Networks ! Programmable networks; Network two \classes" (GETs and PUTs) and treat them differ- management; Data center networks; Cloud comput- ently. It may even use message-specific details like the ing; key being accessed for load balancing [40]. ∗Work performed while an intern with Microsoft Re- Traditional network management approaches imple- search; currently at Cambridge University, UK ment network functions completely decoupled from ap- yWork performed while an intern with Microsoft Re- plications. They infer the application message a packet search; currently at University of Crete, Greece belongs to using deep packet inspection or through other heuristics. Instead, we propose messages and classes as Permission to make digital or hard copies of all or part of this work for personal first-order abstractions at the network layer. A mes- or classroom use is granted without fee provided that copies are not made or sage refers to an (arbitrary) application data unit while distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work a class is the set of messages (and consequent network owned by others than ACM must be honored. Abstracting with credit is per- packets) to which the same network function should be mitted. To copy otherwise, or republish, to post on servers or to redistribute to applied. Applications can provide the class and message lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. information for any traffic they generate. SIGCOMM ’15, August 17 - 21, 2015, London, United Kingdom In this paper, we present Eden, an architecture for c 2015 ACM. ISBN 978-1-4503-3542-3/15/08. $15.00 implementing network functions at end hosts. End hosts DOI: http://dx.doi.org/10.1145/2785956.2787493 are a natural enforcement point for such functions| 493 they have plentiful resources that allow for complex Overall, this paper makes the following contributions: computation and large amounts of state to be main- • We highlight that a large class of network functions tained, and they are ideally placed for fine-grained vis- feature three key requirements: data-plane compu- ibility into application semantics. Finally, in single ad- tation, data-plane state, and operate on application ministrator environments like enterprises and datacen- semantics (x2). ters, some part of end hosts can be trusted. We show • We design and implement Eden, an architecture that that a large number of diverse and interesting network enables end host network functions through data plane functions can be efficiently realized at the end hosts programmability (x3). with minimal support from the network. • We present a flexible scheme for application-level clas- Eden comprises three components: a logically cen- sification of network traffic (x3.3). tralized controller, stages, and end host enclaves.A • We present a language, compiler and runtime for ac- stage is any application or library that is Eden-compliant. tion functions. The compiler decouples state man- Stages bridge the gap between network functions ex- agement from the function, thus providing a clean pressed in terms of application-level messages and the programming abstraction to administrators (x3.4). enclave operating on packets. To achieve this, stages classify their network traffic, associating application mes- The idea of end hosts participating in the implemen- sages with a class and a message identifier that is carried tation of network functions is not new [49, 16, 21, 34, with it down the host's network stack. The enclave re- 59, 57]. These state of the art approaches, however, sides along the end host network stack, either in the still encourage a low-level, packet-based API for pro- OS or the NIC. It extends and replaces functionality gramming the data plane, often a variant of OpenFlow. typically performed by the end host virtual switch. An This ignores end host capabilities and restricts the func- enclave has a set of match-action tables that, based on tions that can be implemented. Instead, Eden adopts a a packet's class, determine an action function to apply. different philosophy by introducing a data plane inter- The action function can modify both the packet and face that is wide and rich enough to allow for general, the enclave's global state. Enclaves and stages each ex- application-informed data-plane computation. pose an API through which they can be programmed. Enclaves and stages, taken together, enable application- 2 Network Functions aware data plane programmability. Given a network function, the controller can imple- This paper is motivated by the observation that three ment it by programming stages and enclaves across the common data-plane requirements underlie many net- network. Hence, Eden achieves a careful division of work functions. First, they need to create, access and functionality; the controller provides global visibility, modify state, in many cases on a per-packet basis. This stages provide application visibility while enclaves pro- allows for \stateful network functions" where a packet vide a pragmatic enforcement point at the data plane. influences the processing of subsequent packets. Sec- A key challenge posed by our design is efficient and ond, they require computation on the data path. safe execution of action functions at enclaves while al- Finally, they require visibility into application seman- lowing for the functions to be dynamically updated by tics. This introduces a mismatch { data-plane elements the controller without impacting data plane performance. like the end host network stack and switches operate With Eden, action functions are written in a high-level at the granularity of packets. Instead, each application domain specific language using F# code quotations. has its own application data unit or \message". For They are compiled to bytecode which is then interpreted example, for memcached, a popular key-value store, a through a stack-based interpreter within the enclave. message is a GET or a PUT request or response. For This approach allows Eden to execute the same com- a HTTP library, a message is an HTTP request or re- putation across multiple platforms and avoids the com- sponse. These messages are fragmented into packets plexities of dynamically loading code in the OS or the before being sent across the network. Many network NIC. Indeed, our interpreter can execute the same ac- functions however are best expressed in terms of appli- tion function inside the OS or in a programmable NIC. cation messages. Implementing such network functions We have implemented the Eden enclave across two thus requires a mapping between messages and the con- platforms: Hyper-V and Netronome's programmable sequent network packets. NICs [46]. We evaluate Eden through case studies across these platforms. These studies highlight how the Eden 2.1 Examples architecture can implement diverse functions spanning application-aware load-balancing, quality of service and To make the discussion concrete, we now use a few net- weighted path selection. We show that Eden's inter- work functions proposed in recent literature as case- preter-based data plane computation incurs reasonable studies to highlight the three requirements mentioned overheads with negligible impact on application perfor- above.