Designing a Predictable Internet Backbone Network∗
Total Page:16
File Type:pdf, Size:1020Kb
Designing a Predictable Internet Backbone Network¤ Rui Zhang-Shen, Nick McKeown Computer Systems Laboratory Stanford University frzhang, [email protected] ABSTRACT the design process starts over. A well-designed network Designing a backbone network is hard. On one hand, users should support the tra±c matrices presented to it for expect the network to have very high availability, little or as long as possible. In the rest of this section, we will no congestion, and hence little or no queueing delay. On discuss some of the many challenges this presents to the other hand, tra±c conditions are always changing. Over network designers today. time usage patterns evolve, customers come and go, new ap- plications are deployed, and the tra±c matrices of one year 1.1 Obtaining a Traffic Matrix Estimation are quite di®erent from the next. Yet the network operator The key to designing a network is to understand how must design for low congestion over the multiple years that it will be used. The process starts by measuring how the network is in operation. Harder still, the network must the network is used today, and then extrapolating based be designed to work well under a variety of link and router on estimates of how tra±c will grow and change over failures. It is not surprising that most networks today are time. enormously overprovisioned, with typical utilizations around Obtaining a tra±c matrix { a matrix indicating the 10%. In this paper we propose that backbone networks use peak short-term average load between any pair of nodes Valiant Load-balancing over a fully-connected logical mesh. in the backbone { is not an easy task. It is impractical to This is quite a radical departure from the way backbones measure it directly, and the best estimation techniques are built today, and raises as many questions as it answers. (from link load measurements) give errors of 20% or But it leads to a surprisingly simple architecture, with pre- more [1]. Moreover, demand fluctuates over time as a dictable and guaranteed performance, even when tra±c ma- result of special events or even changes external to the trices change and when links and routers fail. It is provably network [2]. the lowest capacity network with these characteristics. In To design a network we need to know what the tra±c addition, it provides fast convergence after failure, making matrix will be in the future { years after the topology is it possible to support real-time applications. ¯rst deployed. Starting with an estimate of the tra±c matrix, and then extrapolating, inevitably leads to a 1. INTRODUCTION crude estimate of future demand. Typically, the tra±c Network design can be formulated as an optimiza- matrix is extrapolated based on historical growth rates, tion problem where total cost is minimized subject to and adjusted according to marketing forecasts and de- topology, demand, and performance constraints. The cisions such as addition and elimination of applications designer chooses where nodes and links are placed, their and services. However, it is impossible to predict future capacities, and how tra±c is routed. On the face of it, growth rates and new applications. Even if total growth this seems like a straightforward problem. We ¯rst de- rate is estimated correctly, the growth rate doesn't take termine the constraints on node and link location, and into account new applications which may change tra±c the expected demand, and then do our best to design patterns. For example, peer-to-peer tra±c has demon- an optimal network. In this paper, we will focus on strated how quickly usage patterns can change in the In- designing a backbone network. ternet. The widespread use of voice-over-IP and video- Once deployed, the expense of the infrastructure dic- on-demand may change usage patterns again over the tates that the backbone topology (i.e., the set of nodes next few years. What's more, the growth rate does not and their inter-connectivity) doesn't change for several take into account large new customers bringing new de- years. As tra±c patterns change, and usage grows, a mand. new topology will eventually need to be deployed, and 1.2 Failures and Routing Convergence ¤This research was funded by NSF under ITR award ANI- 0331653, and by the Lillie Family Stanford Graduate Fel- A network must be designed to continue to operate lowship. when there are failures and service or maintenance in- terruptions in the network. Failures cause two problems: Loss of connectivity, 1 2 which can take several seconds to recover from [3], and is too slow for real-time applications such as voice and N 3 gaming. Second, failures demand extra capacity to carry rerouted tra±c. Today, rerouting leads to large, coarse … 4 flows suddenly traversing a link, requiring large chunks of spare capacity. It is hard enough to design a predictable network when all links and routers are functioning correctly; it is even harder to ensure predictable behavior when there are unpredictable failures. Figure 1: A hierarchical network with N backbone nodes, each serving an access network. The backbone 1.3 Network Design in Practice nodes are connected by a logical full mesh. While network design can be formulated as an opti- mization problem, networks are not designed this way in the most e±cient network design (in the sense that it practice. Juggling the number of constraints, estimates, minimizes total link capacity) that can support an arbi- and unknowns means that network design is more of an trary set of tra±c matrices. Furthermore, the network art than a science, and is dominated by a number of can be designed to support any tra±c matrix under a rules-of-thumb that have been learned over the years. pre-de¯ned number of link and router failures. This explains why each network operator has built a In what follows, we will describe the general approach, very di®erent backbone network. and how Valiant Load-balancing can be applied to back- Ad-hoc design makes it hard or impossible to predict bone network design. We'll ¯rst consider how it works how a network will perform over a wide variety of con- when the network is homogeneous and all nodes have ditions. To o®set the intrinsic inaccuracies in the design the same capacity, and then show how the network can process, it is common to use tra±c engineering in which be designed to work under an arbitrary number of fail- operators monitor link utilization and route flows to re- ures. duce congestion [4], especially during failures [5]. In an accompanying paper [9], the authors indepen- Tra±c engineering only works if the underlying net- dently arrived at the same conclusion and describe a work is able to support the current tra±c matrix. With slightly di®erent scheme. the complicated topologies deployed today, it is not gen- erally possible to determine the set of tra±c matrices that a backbone network can support. 2. VALIANTLOAD-BALANCED NETWORKS The motivation for Valiant Load-balancing is that it 1.4 Problem Statement is much easier to estimate the aggregate tra±c entering In summary, it is extremely hard to design a network. and leaving a node than to estimate the tra±c matrix. First, it is nearly impossible to accurately estimate fu- The approach assumes a full-mesh topology among the ture demand, and how nodes will communicate with nodes, and load-balances tra±c over two-hop paths. each other. Despite inaccurate information, network designers have to ensure the network will operate when 2.1 General Approach links and routers fail unpredictably. Consider a backbone network consisting of multiple We propose a way to design backbone networks that PoPs interconnected by long-haul links. The whole net- are insensitive to the tra±c matrix (i.e., that work equal- work is arranged as a hierarchy, and each PoP connects ly well for all valid tra±c matrices), and continue to an access network to the backbone (see Figure 1). provide guaranteed performance under a user-de¯ned Although tra±c matrices are hard to obtain, it is number of link and router failures. straightforward to measure, or estimate, the total We believe that Valiant Load-balancing is a promising amount of tra±c entering (leaving) a PoP from (to) way to design backbone networks. The approach was its access network. When a new customer joins the ¯rst proposed by Valiant for processor interconnection network, we add its aggregate tra±c rate to the node. networks [6], and has received recent interest for scal- When new locations are planned, the aggregate traf- able routers with performance guarantees [7] [8]. Here ¯c demand for a new node can be estimated from the we apply it to backbone network design. population that the node serves. This is much easier There are several bene¯ts to Valiant Load-balancing. than trying to estimate the tra±c rates from one node First, it can be shown that { given a set of boundary to every other node in the backbone. nodes with a ¯xed maximum capacity { the network Imagine that we establish a full mesh of logical links will support any set of tra±c matrices. It is provably from each node to every other node. They are not nec- essarily physical links; they could be implemented us- What is the optimal interconnection pattern that can ing tunneling or an overlay mechanism. Tra±c enter- guarantee 100% throughput to any valid tra±c matrix ing the backbone is spread equally across all the nodes.