SUBMITTED TO IEEE COMMUNICATIONS SURVEYS & TUTORIALS 1 Segment Routing: a Comprehensive Survey of Research Activities, Standardization Efforts and Implementation Results Pier Luigi Ventre, Stefano Salsano, Marco Polverini, Antonio Cianfrani, Ahmed Abdelsalam, Clarence Filsfils, Pablo Camarillo, Francois Clad Revision R2 - June 2020 Abstract—Fixed and mobile telecom operators, enterprise net- APIs, Northbound APIs, Open Source, Software Defined Net- work operators and cloud providers strive to face the challenging working, SDN, Service Function Chaining, SFC, Standards demands coming from the evolution of IP networks (e.g. huge bandwidth requirements, integration of billions of devices and millions of services in the cloud). Proposed in the early 2010s, I. INTRODUCTION Segment Routing (SR) architecture helps face these challenging demands, and it is currently being adopted and deployed. SR Egment Routing (SR) is based on the loose Source architecture is based on the concept of source routing and S Routing concept. A node can include an ordered list of has interesting scalability properties, as it dramatically reduces instructions in the packet headers. These instructions steer the the amount of state information to be configured in the core forwarding and the processing of the packet along its path in nodes to support complex services. SR architecture was first implemented with the MPLS dataplane and then, quite recently, the network. with the IPv6 dataplane (SRv6). IPv6 SR architecture (SRv6) The single instructions are called segments, a sequence of has been extended from the simple steering of packets across instructions can be referred to as a segment list or as an SR nodes to a general network programming approach, making it Policy. Each segment can enforce a topological requirement very suitable for use cases such as Service Function Chaining (e.g. pass through a node) or a service requirement (e.g. and Network Function Virtualization. In this paper we present a tutorial and a comprehensive survey on SR technology, an- execute an operation on the packet). The term segment refers alyzing standardization efforts, patents, research activities and to the fact that a network path towards a destination can be implementation results. We start with an introduction on the split into segments by adding intermediate way-points. The motivations for Segment Routing and an overview of its evolution segment list can be included by the original source of the and standardization. Then, we provide a tutorial on Segment packet or by an intermediate node. When the segment list is Routing technology, with a focus on the novel SRv6 solution. We discuss the standardization efforts and the patents providing inserted by an intermediate node, it can be removed by another details on the most important documents and mentioning other node along the path of the packet, supporting the concept of ongoing activities. We then thoroughly analyze research activities tunneling through an SR domain from an ingress node to an according to a taxonomy. We have identified 8 main categories egress node. during our analysis of the current state of play: Monitoring, Traffic Engineering, Failure Recovery, Centrally Controlled Ar- The implementation of the Segment Routing Architecture chitectures, Path Encoding, Network Programming, Performance requires a data plane which is able to carry the segment lists in Evaluation and Miscellaneous. We report the current status of SR the packet headers and to properly process them. Control plane deployments in production networks and of SR implementations operations complement the data plane functionality, allowing (including several open source projects). Finally, we report our arXiv:1904.03471v5 [cs.NI] 3 Jul 2020 the allocation of segments (i.e. associating a segment identifier experience from this survey work and we identify a set of future research directions related to Segment Routing. to a specific instruction in a node) and the distribution of the segment identifiers within an SR domain. Index Terms—Segment Routing, MPLS, IPv6, SR-MPLS, As for the data plane, two instances of SR Architecture have SRv6, Source Routing, Monitoring, Traffic Engineering, Failure Recovery, Path Encoding, Networking Programming, Perfor- been designed and implemented, SR over MPLS (SR-MPLS) mance, Linux, VPP, Data Plane, Control Plane, Southbound and SR over IPv6 (SRv6). With SR-MPLS, no change to the MPLS forwarding plane is required [1]. SRv6 is based on a P.L. Ventre is with with the Department of Electronic Engineer- new type of IPv6 routing header called SR Header (SRH) [2]. ing at the University of Rome Tor Vergata - Rome, Italy, E-mail: Temporally, SR-MPLS has been the first instance of SR [email protected]. S. Salsano is with the Department of Electronic Engineering at the Architecture, while the recent interest and developments are University of Rome Tor Vergata and with the Consorzio Nazionale In- focusing on SRv6. In particular, the IPv6 data plane for SR teruniversitario per le Telecomunicazioni (CNIT) - Rome, Italy, E-mail: is being extended to support the so-called SRv6 Network [email protected]. M. Polverini and A. Cianfrani are with Department of Informa- Programming Model [3]. According to this model, Segment tion Engineering, Electronics and Telecommunications at the Univer- Routing functions can be combined to achieve an end-to-end sity of Rome Sapienza - Rome, Italy, E-mail: {marco.polverini, anto- (or edge-to-edge) networking objective that can be arbitrarily nio.cianfrani}@uniroma1.it. A. Abdelsalam, C. Filsfils, P. Camarillo and F. Clad are with Cisco Systems, complex. This is appealing for implementing complex services E-mail: {ahabdels, cfilsfil, pcamaril, fclad}@cisco.com such as Service Function Chaining. SRv6 can be used as an SUBMITTED TO IEEE COMMUNICATIONS SURVEYS & TUTORIALS 2 overlay tunneling mechanism directly exposed and used by overcome some scalability issues and limitations [6], which servers (similar to VXLAN tunneling) and as an underlay had been identified in the traffic engineered Multi Protocol transport mechanism in network backbones (supporting Traffic Label Switching (MPLS-TE) solutions used for IP backbones. Engineering and Resilience services). In light of this, SRv6 can In particular it was observed that MPLS-TE requires explicit simplify network architectures avoiding the use of different state to be maintained at all hops along an MPLS path; this protocol layers. may lead to scalability problems in the control-plane and in As for the SR Control Plane operations, they can be based the data-plane. Moreover, the per-connection traffic steering on a distributed, centralized or hybrid architecture. In the model of MPLS-TE does not easily exploit the load balancing distributed approach, the routing protocols are used to signal offered by Equal Cost MultiPath (ECMP) routing. On the other the allocation of segments, and the nodes take independent hand, Segment Routing can steer traffic flows along traffic decisions to bind packets to the segment lists. In the centralized engineered paths with no per-flow state in the nodes along the approach, an SR controller allocates the segments, takes the path and exploit ECMP routing within each segment. decision on which packets need to be associated to which In the early 2010s, the IETF started the “Source Packet SR Policy and configures the nodes accordingly. Very often, Routing in Networking” Working Group (SPRING WG) to an hybrid approach which consists of the combination of the deal with Segment Routing. The activity of the SPRING WG previous strategies is used (see for example [4]). has included the identification of Use Cases and Requirements The goal of this paper is to provide a comprehensive survey for Segment Routing (for example, [7], [8] and [9] have be- on Segment Routing technology, including all the achieved come IETF RFCs). In July 2018, the SPRING WG has issued results and the ongoing work. Hereafter (section I-A), we start the “Segment Routing Architecture” document (RFC 8402 with the historical context behind the development of Segment [10]) along with another informational RFC on monitoring Routing. In section II, we provide an introduction to the main use cases [11]. More recently (December 2019) other 3 RFCs concepts of Segment Routing architecture. We consider both have been issued, while many other documents are still under the SR-MPLS and the SRv6 data planes, but we focus more discussion by the WG, as it will be analyzed later in this paper. deeply on SRv6 which is currently attracting a lot of interest. Looking at the scientific bibliography, the seminal paper on In section III, we provide a classification and a discussion the Segment Routing Architecture is [12]. Published in 2014, of the standardization efforts together with an analysis of the it provides an overview of the motivations for SR, describes most relevant patents. We provide a comprehensive review of a set of important use cases and illustrates the architecture. research activities in section IV, covering 88 scientific papers. The basic concepts proposed in [12] have been elaborated and The most relevant implementation results and the status of refined in the RFC 8402 [10]. ongoing deployments are reported in section V. We report in Currently, Segment Routing is receiving a lot of interest section VI lessons learned and our experience. We highlight from operators for its applications in different types of net- future research directions and open issues in section VII. works (transport backbones, access networks, datacenters and 5G networks). The MPLS based data plane (SR-MPLS) relies on the well A. Segment Routing roots and evolution established MPLS technology. SR-MPLS can be seen as an im- The Source Routing approach consists of the inclusion of provement and a simplification of the traditional MPLS control the route of the packet as a list of hops in the packet header. plane, so it is beneficial to operators with an already deployed This has two variants. Strict Source Routing requires the MPLS infrastructure.