Connections with Multiple Congested Gateways in Packet-Switched Networks Part 1: One-way Traf®c Sally Floyd Lawrence Berkeley Laboratory 1 Cyclotron Road Berkeley, CA 94720 ¯[email protected] Abstract 1 Introduction In this paper we investigate the throughput of connec- In this paper we explore the bias in TCP/IP networks tions in TCP/IP networks with multiple congested gate- against connections with multiple congested gateways. ways. There are two distinct motivations for this inves- We consider the interaction between the bias against tigation. One motivation of this paper is to consider the connections with multiple congested gateways, the bias effect of multiple congested gateways on throughput. of the TCP window modi®cation algorithm against con- The second motivation of this paper is to consider gen- nections with longer roundtrip times, and the bias of eral performance and fairness goals for networks that Drop Tail and Random Drop gateways against bursty include connections with multiple congested gateways. traf®c. Using simulations and a heuristic analysis, we [M90b] reports on a measurement study of a network show that in a network with the window modi®cation al- with multiple congested gateways, comparing the per- gorithm in 4.3 tahoe BSD TCP and with Random Drop formance with Drop Tail and with Random Drop gate- or Drop Tail gateways, a longer connection with mul- ways. In that measurement study, the throughput for the tiple congested gateways can receive unacceptably low longer connection was better for some scenarios with throughput. We show that in a network with no bias Drop Tail gateways, and better in other scenarios with against connections with longer roundtrip times and Random Drop gateways. In this paper we show that with no bias against bursty traf®c, a connection with for our scenario, when the simulations with Drop Tail multiple congested gateways can receive an acceptable gateways are constructed to remove traf®c phase effects level of throughput. [FJ91a], the performance of networks with Drop Tail We discuss the application of several current mea- gateways and with Random Drop gateways are quite sures of fairness to networks with multiple congested comparable. This paper gives a quantitative analysis of gateways, and show that different measures of fair- the throughput of connections with multiple congested ness have quite different implications. One view is that gateways. We use both simulations and a heuristic anal- each connection should receive the same throughput in ysis to investigate performance. bytes/second, regardless of roundtrip times or numbers TCP/IP networks have a bias against connec- of congested gateways. Another view is that each con- tions passing through multiple congested gateways nection should receive the same share of the network's [DKS90] [M90b], a bias against connections with scarce congested resources. In general, we believe that longer roundtrip times [H89] [Z89], and a bias against the fairness criteria for connections with multiple con- bursty traf®c. We consider the interaction of these bi- gested gateways requires further consideration. ases. In a previous paper [FJ91a] we reported brie¯y on investigations of the bias against connections with longer roundtrip times, and we discussed modi®cations to the TCP window-increase algorithm that could cor- rect this bias. In this paper we principally consider the This work was supported by the Director, Of®ce of Energy Re- search, Scienti®c Computing Staff, of the U.S. Department of Energy bias against connections passing through multiple con- under Contract No. DE-AC03-76SF00098. gested gateways. Section 2 discusses previous work discussing the lems of fairness with the DECbit scheme [RJ90] with biases in TCP/IP networks against connections with connections having different roundtrip times. [RCJ87] longer roundtrip times or against connections passing outlines two separate approaches for improving fair- through multiple congested gateways. In section 3 we ness in these conditions, a router-based approach and a describe our simulator, and we de®ne the TCP win- transport-based approach. Examples of a router-based dow modi®cation algorithms and the gateway packet- approach include the selective DECbit scheme [RCJ87] dropping algorithms examined in our simulations. In and Fair Queueing gateways [DKS90]. The transport- section 4 we give the results of simulations. Subsection based approach explored brie¯y in [RCJ87] involves 4.4 discusses the bias against bursty traf®c in simula- modi®cations to the window increase algorithm to en- tions with Drop Tail and Random Drop gateways. Sub- sure an additive increase in the throughput rate rather section 4.5 discusses the implications of the simulations than in the window size. In this paper we combine a for general traf®c with multiple congested gateways. transport-based approach and a router-based approach Section 5 gives a heuristic analysis of the throughput to explore the range of performance possible for TCP/IP for a network with multiple congested gateways. The networks with multiple congested gateways. heuristic analysis and the simulation results are in close agreement. Section 6 considers the con¯icting network goals of maximizing system throughput and maintain- 3 Simulator algorithms ing fairness for networks with multiple congested gate- ways. Section 7 gives conclusions, and discusses re- In this section we brie¯y describe our simulator, and lated work in progress. we describe the different window modi®cation algo- In this paper we restrict our attention to networks rithms investigated in our simulations. These include with one-way traf®c. In a second paper (in progress) the Reduce-to-One and the Reduce-by-Half window- we consider the complications introduced by two-way decrease algorithms and the Increase-by-One and the traf®c. Constant-Rate window-increase algorithms. We also describe several gateway packet-dropping algorithms examined in our simulations. These include Drop Tail, 2 Related work Random Drop, and Random Early Detection (RED) gateways. In [M90b] Mankin presents a measurement study of a network with local and long distance traf®c, with sev- 0 eral congested gateways. The Random Drop and the 0 1a 1b 2a 2b 3a 3b 4a 4b 5a 5b 0 1 2 3 4 5 Drop Tail gateway algorithms are compared. Three 1 1 2 2 3 3 4 4 5 5 topologies are explored, with one, two, and three con- gested gateways respectively. The longer connection's source sink gateway throughput is better with Random Drop gateways for : 1.5Mbps, 50ms delay some topologies, and better with Drop Tail gateways : 10Mbps, 5ms delay for other topologies. (As [FJ91a] explains, we believe that these results should be interpreted keeping traf®c Figure 1: Simulation network with 5 congested gate- phase effects in mind.) Mankin remarks that the longer ways. connection is disproportionately likely to have pack- ets dropped at the gateway, particularly with Drop Tail Figure 1 shows a simulation network with 6 connec- gateways. tions, 10 gateways, and 5 congested gateways. The In [DKS90] several simulations are run for a network congested gateways in Figure 1 are gateways 1a, 2a, 3a, with three congested gateways, with one long connec- 4a, and 5a. The dotted lines show the connection paths; tion and three shorter connections. In the simulations source sends to sink . Each connection has a maxi- with Fair Queueing or with the selective DECbit algo- mum window just large enough so that, even when that rithm, the longer connection receives half of the avail- connection is the only active connection, the network able throughput at each gateway. For the simulations still occasionally drops packets. using the algorithms in 4.3 tahoe BSD TCP and FIFO We use a family of simulation networks similar to Drop Tail gateways, the longer connection receives 29% Figure 1, where the number of congested gateways of the throughput at each gateway. ranges from 1 to 10. Figure 1 only shows the network for Several researchers have discussed the bias in TCP/IP 5 congested gateways. For a simulation network networks against connections with longer roundtrip with congested gateways for 1 there are times [H89] [M90b] [Z89]. [RCJ87] explores prob- 1 connections and 2 gateways. Connection 0 passes through multiple congested gateways, and connections BSD TCP is called the Reduce-by-Half algorithm. The 1 through each pass through one congested gateway. use of the Reduce-by-Half window-decrease algorithm Connection 0 is roughly 2 1 times longer than the also simpli®es the analysis of the behavior of the net- other connections. In these simulations all connection work. paths have the same maximum bandwidth. Therefore a De®nitions: Reduce-by-Half window decreases connection with a longer roundtrip time also has a larger and Selective Acknowledgements. With the Reduce- delay-bandwidth product. by-Half window decrease algorithm, when a packet loss Our simulator is based on the REAL simulator [K88], is detected by the ªfast retransmissionº algorithm the which is built on Columbia's Nest simulation package connection reduces its window by half. The details of [BDSY88]. Our simulator has been extensively rewrit- the Reduce-by-Half window decrease algorithm, which ten by Steven McCanne at LBL. FTP sources always is implemented in our simulator as in 4.3 reno BSD have a packet to send and always send a maximal-sized TCP, are somewhat complex [J90]. For the purposes of packet as soon as the window allows them to do so. A this paper, the important feature of the Reduce-by-Half sink immediately sends an ACK packet when it receives window decrease algorithm is that with the ªfast retrans- a data packet. The gateways use FIFO queueing. Data missionº algorithm, the source retransmits a packet and packets contain 1000 bytes, and ACK packets contain reduces the window by half, rather than reducing its 40 bytes.