Merlin: A Language for Provisioning Network Resources Robert Soulé* Shrutarshi Basuy Parisa Jalili Marandi* Fernando Pedone* Robert Kleinbergy Emin Gün Sirery Nate Fostery *University of Lugano yCornell University ABSTRACT As a result, there is widespread interest in academia and indus- This paper presents Merlin, a new framework for managing re- try in higher-level programming languages and “northbound” (i.e., sources in software-defined networks. With Merlin, administra- application-facing) APIs that provide convenient control over the tors express high-level policies using programs in a declarative lan- full set of resources available in a network. guage. The language includes logical predicates to identify sets Unfortunately, despite several notable advances, there is still a of packets, regular expressions to encode forwarding paths, and wide gap between the capabilities of existing SDN APIs and the arithmetic formulas to specify bandwidth constraints. The Merlin realities of network management. Current programming languages compiler maps these policies into a constraint problem that deter- focus mostly on packet forwarding and largely ignore functionality mines bandwidth allocations using parameterizable heuristics. It such as bandwidth and packet-processing functions that can only then generates code that can be executed on the network elements be implemented on middleboxes, end hosts, or with custom hard- to enforce the policies. To allow network tenants to dynamically ware [20, 46, 65,3, 50]. Network orchestration frameworks pro- adapt policies to their needs, Merlin provides mechanisms for del- vide powerful mechanisms that handle a larger set of concerns in- egating control of sub-policies and for verifying that modifications cluding middlebox placement and bandwidth [22, 34, 55, 58], but made to sub-policies do not violate global constraints. Experiments they either fail to provide a programmable API to those mecha- demonstrate the expressiveness and effectiveness of Merlin on real- nisms, or expose APIs that are extremely simple (e.g., sequences world topologies and applications. Overall, Merlin simplifies net- of middleboxes). Overall, the challenges of managing real-world work administration by providing high-level abstractions for spec- networks using existing SDN APIs remain unmet. ifying network policies that provision network resources. This paper presents a new SDN programming language designed to fill this gap. This language, called Merlin, provides a collection of high-level programming constructs for (i) classifying packets; Categories and Subject Descriptors (ii) controlling forwarding paths; (iii) specifying packet-processing C.2.4 [Distributed Systems]: Network operating systems functions; and (iv) provisioning bandwidth in terms of maximum ; D.3.2 [Language Classifications]: Specialized application lan- limits and minimum guarantees. These features go far beyond what guages can be realized just using SDN switches or with existing languages like Frenetic [20], Pyretic [47], and Maple [65]. As a result, imple- menting Merlin is non-trivial because it involves determining allo- Keywords cations of network-wide resources such as bandwidth—the simple Software-defined networking, resource management, delegation, compositional translations used in existing SDN compilers cannot verification, Merlin. be readily extended to handle the new features provided in Merlin. The Merlin compiler uses a variety of techniques to determine forwarding paths, map packet-processing functions to network el- 1. INTRODUCTION ements, and allocate bandwidth. These techniques are based on a Network operators today must deal with a wide range of man- unified logical representation of the network that encodes the con- agement challenges from increasingly complex policies to a prolif- straints of the physical topology as well as the constraints expressed eration of heterogeneous devices to ever-growing traffic demands. by the policy. For traffic with bandwidth constraints, the com- Software-defined networking (SDN) provides tools that could be piler uses a mixed-integer program formulation to solve a variant of used to address these challenges, but existing APIs for SDN pro- the multi-commodity flow optimization problem. For traffic with- gramming are either too low-level or too limited in functionality out bandwidth constraints, Merlin leverages properties of regular to enable effective implementation of rich network-wide policies. expressions and finite automata to efficiently generate forwarding trees that respect the path constraints encoded in the logical topol- Permission to make digital or hard copies of all or part of this work for personal or ogy. Handling these two types of traffic separately allows the com- classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation piler to provide a uniform interface to programmers while reducing on the first page. Copyrights for components of this work owned by others than the the size and number of expensive constraint problems it must solve. author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or The compiler also generates configurations for a variety of network republish, to post on servers or to redistribute to lists, requires prior specific permission elements including switches, middleboxes, and end hosts. and/or a fee. Request permissions from [email protected]. Although the configurations emitted by the Merlin compiler are CoNEXT’14, December 2–5, 2014, Sydney, Australia. static, the system also incorporates mechanisms for handling dy- Copyright is held by the owner/author(s). Publication rights licensed to ACM. ACM 978-1-4503-3279-8/14/12 ...$15.00. namically changing policies. Run-time components called negotia- http://dx.doi.org/10.1145/2674005.2674989. loc 2 Locations program below realizes this specification using a sequence of Mer- t 2 Packet-processing functions lin policy statements, followed by a logical formula. Each state- h 2 Packet headers ment contains a variable that tracks the amount of bandwidth used f 2 Header fields by packets processed with that statement, a predicate on packet v 2 Header field values headers that identifies a set of packets, and a regular expression id 2 Identifiers that describes a set of forwarding paths through the network: n 2 N [ x : (ip.src = 192.168.1.1 and pol ::= [s1; ::: ; sn]; φ Policies ip.dst = 192.168.1.2 and s ::= id : p ! r Statements tcp.dst = 20) -> .* dpi .* ; φ ::= max(e; n) j min(e; n) Presburger Formulas y : (ip.src = 192.168.1.1 and j φ1 and φ2 j φ1 or φ2 j ! φ1 ip.dst = 192.168.1.2 and e ::= n j id j e + e Bandwidth Terms tcp.dst = 21) -> .* ; a ::= : j c j a a j a|a j a∗ j ! a Path Expression [z : (ip.src = 192.168.1.1 and ip.dst = 192.168.1.2 and p ::= p and p j p or p j ! p 1 2 1 2 1 Predicates tcp.dst = 80) -> .* dpi *. nat .* ], j h:f = v j true j false max(x + y,50MB/s) and min(z,100MB/s) c ::= loc j t Path Element The statement on the first line asserts that FTP traffic from the host 192.168.1.1 192.168.1.2 Figure 1: Merlin abstract syntax. at IP address to the host at address must travel along a path that includes a packet-processing function that performs deep-packet inspection (dpi). The next two state- tors communicate among themselves to dynamically adjust band- ments identify and constrain FTP control and HTTP traffic between width allocations and verify that the modifications made by other the same hosts respectively. The statement for FTP control traffic negotiators do not lead to policy violations. Again, the design of does not include any constraints on its forwarding path, while the Merlin’s policy language plays a crucial role. The same core lan- HTTP statement includes both a deep-packet inspection (dpi) and guage constructs used by the compiler for mapping policies into a a network address translation (nat) constraint. The formula on the constraint problem provide a concrete basis for analyzing, process- last line declares a bandwidth cap (max) on the FTP traffic, and a ing, and verifying policies modified dynamically by negotatiors. bandwidth guarantee (min) for the HTTP traffic. We have built a working prototype of Merlin, and used it to im- Note that packet-processing functions may modify packet head- plement a variety of practical policies that demonstrate the expres- ers. In this example policy, the nat function will re-write the packet siveness of the language. These examples demonstrate that Mer- IP addresses. To allow such functions to coexist with predicates on lin supports a wide range of network functionality including sim- packet headers that identify sets of traffic, Merlin uses a tag-based ple forwarding policies, richer packet-processing functions such as routing scheme that will be explained in Section 3.4. The rest of deep-packet inspection that are usually implemented on middle- this section describes the constructs used in this policy in detail. boxes, and policies that include bandwidth constraints. We have 2.1 Syntax and semantics also implemented negotiators that realize max-min fair sharing and additive-increase multiplicative-decrease dynamic adaptation schemes. The syntax of the Merlin policy language is defined by the gram- Our experimental evaluation shows that the Merlin compiler can mar in Figure1. A policy is a set of statements, each of which spec- provision and configure real-world datacenter and enterprise net- ifies the handling of a subset of traffic, together with a logical for- works, and that Merlin can be used to obtain better application per- mula that expresses a global bandwidth constraint. For simplicity, formance for data analytics and replication systems. we require that the statements have disjoint predicates and together Overall, this paper makes the following contributions: match all packets. In our implementation, these requirements are enforced using a simple policy pre-processor.