Characterizing the Internet Hierarchy from Multiple Vantage Points Lakshminarayanan Subramanian, Sharad Agarwal, Jennifer Rexford, Randy H. Katz Report No. UCB/CSD-1-1151 August 2001 Computer Science Division (EECS) University of California Berkeley, California 94720 This work was supported by DARPA Contract No. N00014-99- C-0322. 1 Abstract— The delivery of IP traffic through the Internet rest of the Internet. The Internet topology alone does not depends on the complex interactions between thousands of provide enough information to answer these questions. For Autonomous Systems (ASes) that exchange routing informa- example, suppose that AS B connects to two providers, AS tion using the Border Gateway Protocol (BGP). This paper A and AS C. An AS graph would show connectivity from investigates the topological structure of the Internet in terms A to B and from B to C; however, AS B’s routing policies of customer-provider and peer-peer relationships between ASes, as manifested in BGP routing policies. We describe would not permit transit traffic between A and C. a technique for inferring AS relationships by exploiting par- In the absence of a global registry, the AS-level struc- tial views of the AS graph available from different vantage ture of the Internet is typically inferred from analysis of points. Next we apply the technique to a collection of ten routing data. Previous work has focused on constructing a BGP routing tables to infer the relationships between neigh- view of the AS graph from traceroute experiments or in- boring ASes. Based on these results, we analyze the hier- dividual BGP table dumps. Traceroute provides a view of archical structure of the Internet and propose a five-level the path from a source to a destination host at the IP-level. classification of ASes. Our analysis differs from previous characterization studies by focusing on the commercial rela- The traceroute data must be analyzed to infer which inter- tionships between ASes rather than simply the connectivity faces belong to the same router and which routers belong between the nodes. to the same AS [6]. Running experiments between multi- ple source-destination pairs provides a larger collection of paths over time [6], [7], [8]. Other studies have extracted I. INTRODUCTION AS paths directly from BGP routing tables or BGP update Today’s Internet is divided into more than 10,000 Au- messages [9], [10]. The routing table dump from the Uni- tonomous Systems (ASes) that interact to coordinate the versity of Oregon RouteViews server [11], [12] has been delivery of IP traffic. An AS typically falls under the ad- the basis of several studies of basic topological properties, ministrative control of a single institution, such as a univer- such as the distribution of node degrees [13], [14]. With sity, company, or Internet Service Provider (ISP). Neigh- the exception of the work in [10], these studies have fo- boring ASes use the Border Gateway Protocol (BGP) [1], cused on the topological structure without considering the [2] to exchange information about how to reach individual relationship between neighboring ASes. [10] presents a blocks of destination IP addresses (or, prefixes). An AS heuristic for inferring the relationships from a collection of applies local policies to select the best route for each prefix AS paths and evaluates the technique on the RouteViews and to decide whether to propagate this route to neighbor- data. ing ASes, without divulging these policies or the AS’s in- In this paper, we propose a technique for combining data ternal topology to others. In practice, BGP policies reflect from multiple vantage points in the Internet to construct a the commercial relationships between neighboring ASes. more complete view of the topology and the AS relation- AS pairs typically have a customer-provider or peer-peer ships. Each vantage point offers a partial view of the Inter- relationship [3], [4]. A provider provides connectivity to net topology as viewed from the source node. Due to the the Internet as a service to its customers, whereas peers presence of complex routing policies, these partial views provide connectivity between their respective customers. are not necessarily shortest-path trees and may, in fact, in- AS relationships, and the associated routing policies, clude cycles. We generate a directed AS-level graph from have a significant impact on how traffic flows through the each vantage point and assign a rank to each AS based Internet. An understanding of the structure of the Internet on its position. Then, each AS is represented by the vector in terms of these relationships facilitates a wide range of that contains its rank from each of the routing table dumps. important applications. For example, consider a content Finally, we infer the relationship between two ASes by distribution company that has a choice of placing replicas comparing their vectors. The work we describe in this pa- of a Web site in data centers hosted by different ASes. The per is novel in two ways. First, we analyze AS paths seen company can identify the IP prefixes and ASes responsible from multiple locations to form a more complete view of for a large portion of the traffic from the site [5]. With an the graph. Second, rather than simply combining the data accurate view of the connectivity and relationship between from the various vantage points, we propose a methodol- ASes, the company can identify the best locations for its ogy for exploiting the uniqueness of each view to infer the replicas. As another example, consider a new regional relationships between AS pairs. ISP that wants to connect to a small number of upstream We evaluate our technique on a collection of ten BGP providers. An accurate view of the AS topology and rela- routing tables and summarize the characteristics of the AS tionships between ASes can help the ISP determine which relationships. To validate the inferences, we check for ASes would provide the best connectivity to and from the paths that are not consistent with the routing policy as- 2 sumptions underlying customer-provider and peer-peer re- Although ASes typically follow these guidelines, some lationships. We show that these cases account for a small ASes have more complicated relationships in practice. For proportion of the paths and that the most common incon- example, two ASes operated by the same institution may sistencies may stem from misconfiguration or more com- have a sibling relationship where each AS provides tran- plex AS relationships. Then, we analyze the resulting AS sit service for the other [10]. Other AS pairs may have graph to characterize the hierarchical structure of the Inter- backup relationships to provide connectivity in the event net. We present a five-level classification of ASes with a of a failure [15]. Alternatively, two ASes may peer indi- top-most layer that consists of a rich set of peer-peer rela- rectly through a transit AS [16]. Also, an AS pair may tionships between ¾¼ so-called tier-1 providers. This clas- have different relationships for certain blocks of IP ad- sification can aid an institution in selecting or reevaluating dresses; for example, an AS in Europe may be a customer its connections to other ASes in the Internet. of an AS in the United States for some destinations and a peer for others. Router misconfiguration may also cause II. PROBLEM FORMULATION violations in the export rules. For example, a customer may mistakenly export advertisements learned from one In this section, we formulate the problem we are try- provider to another. We initially assume that only a small ing to solve. We first present a brief overview of AS re- fraction of the AS pairs represent exceptions to the tradi- lationships and their implications on BGP export policies. tional provider-customer and peer-peer relationships. Our Then we formally define the Type of Relationship (ToR) inference technique is designed to tolerate occasional ex- problem for finding an assignment of AS relationships that ceptions and, in fact, our algorithm can be used to identify maximizes the number of paths that adhere to the export AS pairs that have unusual relationships. restrictions. B. Type-of-Relationship (ToR) Problem A. AS Relationships and BGP Export Policies BGP export policies have a direct influence on the AS The relationships between ASes arise from contracts paths seen from a particular vantage point in the Inter- that define the pricing model and the exchange of traf- net. If every AS adheres to the customer and provider fic between domains. ASes typically have a provider- export rules, then no path would ever traverse a customer- customer or peer-peer relationship [3], [4]. In a provider- provider edge after traversing a provider-customer or peer- customer relationship, the customer is typically a smaller peer edge, and no path would ever traverse more than one AS that pays a larger AS for access to the rest of the Inter- peer-peer edge [10], [15]. To formulate these properties in net. The provider may, in turn, be a customer of an even mathematical terms, we denote an edge from a customer larger AS. In a peer-to-peer relationship, the two peers are ½ to a provider with a , an edge from one peer to another typically of comparable size and find it mutually advan- with a ¼, and edge from a provider to a customer with a tageous to exchange traffic between their respective cus- ·½. Restating a result from [10] in these terms, we have: tomers. These relationships translate directly into policies for exporting route advertisements via BGP sessions with Theorem 1: If every AS obeys the customer, peer, and neighboring ASes: provider export policies, then every advertised path be- ¼ longs to one of these two types for some Å; Æ : ¯ Exporting to a provider: In exchanging routing infor- ½ ::: ·½ ::: 1.