HLP: A Next Generation Inter-domain Routing Protocol Lakshminarayanan Subramanian∗ Matthew Caesar∗ Cheng Tien Ee∗ Mark Handley† Morley Mao‡ Scott Shenker∗ Ion Stoica∗ ABSTRACT Designing an inter-domain protocol that satisfies both the It is well-known that BGP, the current inter-domain rout- algorithmic and policy requirements represents a very chal- ing protocol, has many deficiencies. This paper describes a lenging task. There is an inherent conflict between the eco- hybrid link-state and path-vector protocol called HLP as an nomic need for fully-informed and private routing policies alternative to BGP that has vastly better scalability, isolation and the structural need for robust routing algorithms. One and convergence properties. Using current BGP routing in- could consider a spectrum of designs making different trade- formation, we show that HLP, in comparison to BGP, can offs. The Border Gateway Protocol (BGP) takes an extreme reduce the churn-rate of route updates by a factor 400 as position in this design space that all routing policy must be well as isolate the effect of routing events to a region 100 private; no policy information is transmitted in route up- times smaller than that of BGP. For a majority of Internet dates, leaving policy to be implemented entirely by local routes, HLP guarantees worst-case linear-time convergence. filters whose contents are kept secret. As a result, BGP suf- We also describe a prototype implementation of HLP on top fers from inherent algorithmic problems, including poor scal- ability, minimal fault isolation, and slow convergence due to of the XORP router platform. HLP is not intended to be a 1 finished and final proposal for a replacement for BGP, but uninformed path exploration. These problems, while mere is instead offered as a starting point for debates about the nuisances in the Internet’s early days, are becoming signifi- nature of the next-generation inter-domain routing protocol. cantly more serious as expectations and demands placed on the Internet increase. Categories and Subject Descriptors Although BGP does not distribute policy information, in C.2.6 [Communication Networks]: Internetworking practice it is impossible to hide certain policies because the routing protocol must distribute reachability and path infor- General Terms mation. Specifically, most provider-customer relationships Algorithms, Design, Experimentation, Performance. are easily inferable from routing information distributed to Keywords the entire Internet [27, 9]. In addition, even though BGP Inter-domain routing, BGP, scalability, convergence. provides complete path information to all ISPs, the vast ma- jority of implemented policies do not use this information. 1. INTRODUCTION This suggests that the extreme position taken by BGP, keep- ing full privacy and providing full path information, is not Inter-domain routing presents a formidable combination needed, nor perhaps even tenable. of algorithmic and policy challenges. On the one hand, given In this paper, we explore a design point that is less ex- the size and the rapid growth of the Internet, any inter- treme than BGP by proposing and evaluating a hybrid link- domain routing protocol should satisfy basic desirable algo- state path-vector routing protocol, called HLP. The design rithmic properties, such as scalability, robustness, and rapid philosophy of HLP is to expose the common case of policies convergence. On the other hand, for economic reasons inter- and to withhold some path information. This common case domain routing should support policy routing, where ISPs of policies exploits the assumption that a majority of In- have the flexibility to implement a wide variety of private ternet routes (99%) obey the structure of the Autonomous routing policies that ISPs choose not to reveal. Moreover, System (AS) hierarchy as imposed by provider-customer re- the routing protocol should provide sufficient information to lationships. Given that this structure is largely inferable enable ISPs to make informed policy decisions. today [9, 27] and relatively stable (as we show later in this ∗University of California at Berkeley. Email: { lakme,ct-ee, paper), HLP optimizes the routing protocol based on this mccaesar, shenker, istoca }@cs.berkeley.edu structure. By analyzing the evolution of Internet routing †University College London. Email: and the growth of the Internet routing structure, we contend [email protected] that this common case of policies is not merely an artifact of ‡University of Michigan. Email:[email protected] today’s practices but is bound to stay as a common-case be- havior in the future. In essence, HLP leverages the common- case policy behavior that BGP cannot hide and optimizes the protocol design for this common case. For routing poli- Permission to make digital or hard copies of all or part of this work for cies that do not fit the common case behavior, HLP resorts personal or classroom use is granted without fee provided that copies are to mechanisms resembling those of BGP to accommodate not made or distributed for profit or commercial advantage and that copies them. bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific 1 permission and/or a fee. While some problems have been dealt with by modest in- SIGCOMM’05, August 21–26, 2005, Philadelphia, Pennsylvania, USA. cremental modifications [22, 7, 29], we contend that many of Copyright 2005 ACM 1-59593-009-4/05/0008 ...$5.00. the problems are fundamental to BGP’s basic architecture. 13 The central idea used in HLP to optimize for the com- Table 1: Distinctions between HLP and BGP mon case is to use explicit information hiding of unneces- Design issue BGP HLP sary routing updates across provider-customer hierarchies Routing structure Flat Hierarchical and thereby limiting the global visibility and effect of rout- Policy structure Support for Optimize for common ing events. Information hiding is fundamentally required generic policies case of policies to improve the scalability and isolation properties of inter- Granularity of routing Prefix based AS based domain routing. If every routing event is globally visible, Style of routing Path vector Hybrid routing then the network churn grows at least linearly (if not super- linear) with the network size, which is clearly undesirable. HLP uses the provider-customer hierarchy to limit the vis- Convergence and Route Stability: To provide reliable reach- ibility of routing information across hierarchies. Moreover, ability, Internet routes should be relatively stable and, when HLP’s information hiding mechanism naturally fits today’s a change is necessary, they should quickly converge to their routing assumptions and requires minimal modifications for new steady-state. However, BGP is known to suffer from deployment. significant route instabilities, route oscillations and long con- Information hiding on HLP gives substantially improved vergence times. Nearly 25% of BGP prefixes continuously scalability, isolation, convergence and fault diagnosis prop- flap and a large fraction of these have convergence times on erties. For the current Internet topology, the churn rate of the order of hours [5]. The remaining 75% of relatively stable HLP route advertisements is roughly 400 times less than prefixes typically take between 2 − 5 minutes to converge. with BGP. For roughly 50% of inter-AS links, HLP can Isolation: No design can be robust and scalable if local isolate the effects of a fault to a region 100 times smaller faults within a network can have global impact. Unfortu- than that of BGP. For most Internet routes, HLP achieves nately, BGP has very poor fault isolation properties. A linear-time convergence by explicitly constraining the path- simple analysis of Routeviews BGP data [30], shows that exploration process. HLP can support most of BGP’s poli- nearly 20% of the routing events are globally visible and cies and also enables some new ones. HLP also replaces many updates observed at a router are largely a result of BGP’s prefix-deaggregation approach to traffic engineering, events far removed from the router. which can affect route convergence and cause churn, with a cleaner approach based on cost-based traffic engineering and 2.2 Basic Design Issues static prefix deaggregation. HLP also addresses many of the We now contrast BGP’s approach with HLP’s along four security and fault diagnosis problems of BGP, but we do not design issues that face any designer of inter-domain routing discuss these issues in this paper due to space constraints. protocols: routing structure, policy, routing granularity and The rest of the paper is organized as follows. In Sec- routing style. This is not meant to be an exhaustive list, but tion 2, we highlight some of the pressing problems of BGP is limited to the areas where, in our opinion, BGP is in most and elaborate upon the different design issues that confront need of modification. For context, Table 1 summarizes the the designer of any inter-domain routing protocol. In Sec- primary distinctions between HLP and BGP across these tions 3 and 4, we describe the HLP protocol and analyze design issues. its properties. In Sections 5, we discuss traffic engineering issues in HLP and present the router level perspective of 2.2.1 Routing Structure HLP in Section 6. We describe related work in Section 7 In order to support fully general path-based policies, BGP and conclude in Section 8. reveals complete path information. As a result, local rout- ing events can be globally visible [11]. This impairs BGP’s 2. DESIGN RATIONALE scalability, and also makes it fundamentally hard to isolate routing events [15, 11]. Moreover, the resulting interdepen- We start this section by highlighting three specific press- dence between ASs makes the entire Internet vulnerable to ing deficiencies of BGP. We then describe four basic design localized security or configuration problems; a single config- issues and contrast the decisions taken in HLP to those in uration error or compromised router can affect the rest of BGP.