Exogenous-Loss Aware Traffic Management in Overlay Networks

ARTICLE IN PRESS

Computer Networks xxx (2005) xxx–xxx www.elsevier.com/locate/comnet

Exogenous-loss aware traﬃc management in overlay networks Toward global fairness q

Mina Guirguis *, Azer Bestavros, Ibrahim Matta

Department of Computer Science, Boston University, 111 Cummington Street, Boston, MA 02215, United States

Received 28 January 2005; received in revised form 27 July 2005; accepted 7 September 2005

Responsible Editor: V. Tsaoussidis

Abstract

For a given TCP flow, exogenous losses are those occurring on links other than the flows bottleneck link. Exogenous losses are typically viewed as introducing undesirable ‘‘noise’’ into TCPs feedback control loop, leading to inefficient network utilization and potentially severe global unfairness. This has prompted much research on mechanisms for hiding such losses from end-points. In this paper, we show that low levels of exogenous losses are surprisingly beneficial in that they improve stability and convergence, without sacrificing efficiency. Based on this, we argue that exogenous-loss awareness should be taken into account in overlay traffic management techniques that aim to achieve global fairness. To that end, we propose an eXogenous-loss aware Queue Management (XQM) approach that actively accounts for and leverages exogenous losses on overlay paths. We envision the incorporation of XQM functionality in Overlay Traffic Managers (OTMs). We use an equation based approach to derive the quiescent loss rate for a connection based on the connections profile and its global fair share. In contrast to other techniques, XQM ensures that a connection sees its quiescent loss rate, not only by complementing already existing exogenous losses, but also by actively hiding exogenous losses, if necessary, to achieve global fairness. We establish the advantages of exogenous-loss-aware OTMs using extensive simulations in which we contrast the performance of XQM to that of a host of traditional exogenous-loss unaware techniques. Ó 2005 Elsevier B.V. All rights reserved.

Keywords: Service overlay networks; Traﬃc management; TCP models; Control theory; Transient analysis; Simulations

1. Introduction neous set of constituents. As such, a network resource is likely to be shared by flows with signif- One of the defining characteristics of the Inter- icantly different characteristics. While some may net is that it caters to an increasingly heteroge- command fairly long RTTs as a result of traversing a satellite link, others may be subject to multiple congestions as they traverse a large number of q This work was supported in part by NSF grants ANI- hops with bursty cross-traffic, and worse yet, oth- 0095988, ANI-9986397, EIA-0202067 and ITR ANI-0205294, ers may be subject to excessive losses as they tra- and by grants from Sprint Labs and Motorola Labs. * Corresponding author. Tel.: +1 617 353 8924. verse noisy wireless channels. To deal with this E-mailaddress: [email protected] (M. Guirguis). heterogeneity, networking research has traditionally

2 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx focused on mitigating the sources of heterogeneity Exogenous losses are problematic as they consti- in a piecemeal approach. Examples of this are tute ‘‘noise’’ with which a transmission controller abound: from wireless TCP research that attempts must reckon. Unchecked, exogenous losses could to contain wireless losses [1,2], to research on new be quite harmful. By preempting a source from rate adaptation mechanisms that are suitable for claiming its fair share of the available bottleneck link high bandwidth-delay product networks [3–5]. capacity, exogenous losses may result in an unfair While dealing with such issues separately leads to allocation of the bandwidth of overloaded links, or simpler ‘‘specialized’’ solutions to different prob- in a decreased utilization of underutilized links. lems (e.g., wireless losses, large bandwidth-delay Moreover, unwarranted reactions to exogenous product flows), it is not clear if such solutions losses may jeopardize stability and convergence may be working at cross purposes from one another. properties. Recent research efforts have started to ad- In this paper we identify one such instance– dress these issues by adding specialized functionality namely, the impact of exogenous losses on TCPs either in the middle or/and at the end-points of the net- performance and the advantages of leveraging such work. For example, through the use of a TCP proxy, losses in an overlay setting to improve stability, losses on a wireless link could be hidden from end- efficiency, and fairness. points [1,2]. Alternatively, the negative impact of exogenous losses could be mitigated by enabling a 1.1. Motivation source to diagnose the cause of packet losses and to react differently to different types of losses [6–9,3], Packet loss (or marking) events are interpreted or by slowing down its reaction to packet losses by end-to-end transmission control mechanisms through the use of ‘‘smoother’’ control rules [10]. (such as TCP) as constituting the ‘‘feedback’’ signal For the purposes of this paper, we focus our from the bottleneck link to which such a mechanism attention on the first of the above-mentioned nega- must adapt its sending rate. As such, packet losses tive implications of exogenous losses—namely their which are not incidental to that link pose a formida- impact on global fairness. The bandwidth allocated ble challenge to an end-to-end transmission control to a flow is globally fair if it reflects the fair share protocols ability to claim its fair share of network of the capacity of the bottleneck link for that flow, resources and/or react effectively to changes in net- in either an absolute or relative sense, e.g., w.r.t. work resource availability. In an overlay setting, the Round Trip Time (RTT). node at the entry point of an overlay link (hence- forth we refer to such node as Overlay Traffic Man- 1.2. Overview and Contributions ager, or OTM for short) would manage the capacity of its overlay link as well as its local buffer. It is very While countering the effects of exogenous losses likely in this case that some losses occur somewhere is a worthy goal, a more important goal is to assess else at any of the underlying physical (underlay) the extent to which these losses actually impact the links that make up the overlay (logical) link to an- behavior of control loops. More to the point, to be other OTM. In this paper, we use the term exo- able to assess the usefulness of the plethora of traffic genous losses to refer to such losses. Exogenous control strategies dealing with effects of exogenous losses can be thought of as occurring in a manner losses, we need a rigorous methodology for the anal- that is independent of the sources short-term ysis of the emergent behaviors that result from the behavior or its long-term fair share of network re- composition of end-to-end protocols (e.g., in- sources. The emergence of exogenous losses could crease–decrease rules), network element behaviors be attributed to two radically different causes: the (e.g., RED/AQM (Active Queue Management) first is simply a consequence of traversing lossy [11]), and new application-level functionalities. To channels (e.g., wireless hops, satellite links), whereas that end, a particularly promising approach is to the second is due to the bursty nature of cross-traffic marshal techniques from control theory and optimi- on non-bottleneck links. The magnitude of the zation theory to the modeling and evaluation of exogenous losses present and observed by a flow complex network transmission control strategies, depends largely on the characteristics of the path as exemplified in a number of recent efforts traversed. One would expect the magnitude to have [12,4,13]. While useful, these efforts were limited a wide variance due to the heterogeneity of the by the fact that they did not explicitly model exo- Internet. genous losses. ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 3

In this paper, we capture the effect of exogenous value corresponding to said globally fair allocation of losses by extending a dynamic fluid model of the the links available bandwidth. widely deployed Transmission Control Protocol Towards a constructive application of our find- (TCP) [14]. As one would expect, we show that high ings, we argue that exogenous-loss awareness levels of exogenous losses lead to inefficient network should be taken into account in overlay traffic man- utilization and potentially severe global unfairness. agement systems. In particular, we propose an Surprisingly though, we also show that low levels eXogenous-loss aware Queue Management (XQM) of exogenous losses introduce convergence to fairness approach that actively accounts for and leverages properties that are both beneficialand desirable! Spe- exogenous losses already introduced by other pro- cifically, we show that if exogenous loss levels do cesses in the network underlay. Three of these not adversely affect global fairness (i.e., they do schemes are demonstrated in [22]. In this paper, not exceed the value necessary for a flow to con- we focus on one instantiation that can be regarded verge to its global fair share dictated by its bottle- as a per-flow (or per-class) implementation of neck link), they tend to improve stability and REM [18] at OTMs, with the exception that its ac- convergence, without sacrificing efficiency. We ela- tion is dictated by the quiescent loss rates necessary borate on this point below. to achieve global fairness among flows in the pres- Since TCP, by its nature, adaptively seeks avail- ence of exogenous losses. Note that this per-flow able bandwidth, exogenous losses in effect impose or per-class state needs to be maintained only for an upper limit on achievable TCP throughput. flows/classes that are active in the overlay net- The extent to which exogenous losses limit achiev- work—we expect their number not to be large. able throughput makes the crucial difference be- Our goal in this paper is not to develop yet an- tween desirable and undesirable exogenous losses. other AQM (albeit used at an overlay node), but In particular, if this limit lies below a connections rather our goalis to introduce the concept of ‘‘exog- long-term fair share, then exogenous losses cripple enous-loss awareness’’ and demonstrate its benefits to that TCP connection. Otherwise, we show that overlay traffic management. We envision the deploy- exogenous losses enable the fast and stable conver- ment of XQM-enabled OTMs at network bound- gence of TCP connections to their long-term fair aries—maintaining a profile for each long-lived shares of network resources. This is because such flow (or flow aggregate) passing through it. This exogenous losses serve as early error notifications profile includes estimates of the current connections to the sources, which, similar to RED (Random throughput. We use an equation based approach to Early Detection) [11], randomize packet drops derive the quiescent packet loss rate to impose based across all connections. This randomness prevents on the connections profile and its fair share (allo- an individual TCP connection from monopolizing cated by the overlay OTM node). In contrast to the bottleneck resource, in addition to preventing other possible OTM queue management techniques, several connections from synchronizing their send- XQM ensures that a connection sees its quiescent ing behavior which may result in high delay vari- loss rate, not only by complementing already existing ance (jitter). Thus, low levels of exogenous losses, exogenous losses, but also by actively hiding exoge- which do not force TCP throughput to dip below nous losses, if necessary, to achieve global fairness. its long-term fair share, can be beneficial in reaching Note that under an exogenous-loss unaware fair- an efficient, stable and fair allocation of resources. queueing/scheduling scheme (reviewed in Section 2), This observation suggests that the common wis- a TCP flow may also fail to reach its allocated rate dom of utterly hiding all exogenous losses may indeed in the presence of (additional) exogenous losses. be counter-productive. Even if such hiding is harm- Although our XQM agent is inspired by AQM tech- less, the overhead of implementing it—for example niques, the extension of fair-queueing/scheduling through local error recovery over wireless access links techniques to also become exogenous-loss aware is using Snoop [2]—may not be justified. This observa- also possible but we do not investigate such FQ tion also suggests that it may be beneficial to ‘‘use’’ extensions in this paper. We note that it is not a mat- exogenous losses for traffic management purposes ter of fine-tuning existing AQM or FQ algorithms—it in overlay settings. Namely, to ensure global fairness, is their unawareness of (additional) exogenous losses Overlay Traffic Managers (OTMs) [15–17] must be that hinders their ability to maintain global fairness. exogenous-loss aware in that they must hide exoge- To illustrate how an XQM agent deployed at an nous losses only when they exceed a certain nominal OTM could leverage exogenous losses, consider a ARTICLE IN PRESS

4 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx

TCP flow for which a quiescent 2% loss rate would denced by a number of results, exemplified by the result in a global fair share. If exogenous losses works in [30,13,26,31,19,32–35]. In that respect, we amount to 1%, then the XQM agent would intro- single out the works in [12,13], which investigated duce additional losses to bring the total loss rate the stability regions for TCP over RED using a dy- to 2%.1 On the other hand, if exogenous losses namic fluid model. Kelly et al. [19] model TCP/ amount to 4% then the XQM agent could leverage AQM as an optimization problem, where the maxi- any number of mechanisms to hide up to half of mization of the aggregate resource utility is sought. these losses to bring the total loss rate down to 2%. These techniques, however, did not explicitly model We establish the advantages of exogenous-loss exogenous losses. Rather, they focused mostly on awareness using extensive simulations in which, we congestion control. contrast the performance of an XQM agent to that of a host of traditional exogenous-loss unaware 2.2. Active Queue Management/scheduling schemes traffic management agents at OTMs. Overlay traffic management is conceptually simi- 1.3. Paper outline lar to (albeit at a higher layer) AQM schemes (e.g., [11,26,35,36]). AQM designs have focused on the The rest of the paper is organized as follows. We management of network congestion, with queue/ motivate this work by presenting relevant related buffer stabilization as the primary goal. Other work in Section 2 and also throughout the paper, schemes—notably FRED [27]—took a more active when appropriate. In Section 3, we present a dy- approach to protect flows that are particularly vul- namic model of TCP that incorporates exogenous nerable (if additional losses are imposed) due to losses. We analytically derive a lower bound on excessive shrinkage in buffer occupancy. As a losses that need to be hidden from TCP sources to byproduct of this special protection of vulnerable ensure efficient operation. In Section 4, we discuss flows, FRED protects flows that are subject to the effects of exogenous losses on the behavior of excessive exogenous losses from further damage as TCP connections. We capitalize on this in Section they go through it. However, it is important to note 5, where we outline and evaluate the performance that FREDs protection of such flows is triggered by of our XQM overlay traffic management approach. inadequate throughput (as opposed to an explicit Section 6 presents XQMs performance evaluation accounting and management of exogenous losses) compared to other buffer management approaches to protect them from excessively poor performance (inspired from the AQM literature) in different set- (e.g., due to the incidence of timeouts). On the other ups. We conclude in Section 7 with a summary of hand, schemes like fair queuing [37] and GPS [38], results. with per-flow state can easily provide local fairness, while our scheme is working towards global fairness. 2. Related work Clearly, the presence of exogenous losses negatively impacts the performance of all these schemes, since The work we present in this paper relates to a they are unaware of (and not equipped to counter- fairly large body of networking literature, targeting act the effects of) such losses. the goal of improving efficiency and fairness of transmission control loops. We exemplify the vari- 2.3. Explicit treatment of exogenous losses ous flavors of this body of work below. Dealing with exogenous losses explicitly was ad- 2.1. Control-theoretic modeling and analysis dressed in projects that considered the impact of wireless communication (and wireless drops in par- Marshaling techniques from control and optimi- ticular) on TCP. A number of studies proposed zation theory has been a fruitful direction as evi- breaking the transmission control loop into two segments [1], thus ‘‘hiding’’ the exogenous, wireless losses from the ‘‘wired’’ segment of the connection 1 Had exogenous losses been hidden through an independent (not to mention ‘‘breaking’’ the end-to-end seman- mechanism elsewhere (e.g., using Snoop), new losses would have had to be introduced. This is a perfect instance of what we tics of TCP). Other schemes (e.g. Snoop [2]) attempt mentioned earlier regarding solutions working at cross purposes to hide all wireless losses using local retransmission from one another. at the wireless access point. Another set of studies ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 5 opted to ‘‘hide’’ exogenous losses by assigning the Xm b_ðtÞ¼ x ðt D ÞC; ð3Þ task of dealing with such losses to end-hosts, where- i sib i¼1 by the sender is empowered with diagnostic functionality that enables it to infer the reason for a which is equal to the input rate xi(Æ) from the m con- packet loss and to react accordingly. Examples of nections minus the output link rate. Notice that the this line of work are given in [6–9]. More recently, input rates are delayed by the propagation delay and in order to avoid the severe implications of from the senders to the bottleneck Dsib. ‘‘overacting’’ to non-congestion-induced (e.g., exog- We assume that the links between the bottleneck enous) losses in high-speed, long-latency networks, and the receivers are subjected to exogenous packet the work in [3] suggests transmission control rules losses, and that all connections see the same level of that use additional predictors (e.g., queuing delays) exogenous losses. It follows that the total packet to moderate the reaction of senders to such losses. loss probability q(t) observed by senders would For both of these approaches (dealing with exo- comprise the congestion-induced loss probability genous losses in the middle or at end-points), exo- pc(t) (due to buffer overflow at the bottleneck) as genous losses are regarded as noise that must be well as the exogenous loss probability pe(t). Thus, completely eradicated (or hidden) from senders. the total loss probability seen by senders is given by None of these techniques advocate that some level qðtÞ¼1 ð1 pcðtÞÞð1 peðtÞÞ of exogenous losses is harmless—let alone beneficial to boosting fairness and stability. And, clearly, none minðpcðtÞþpeðtÞ; 1Þ; ð4Þ of these techniques leverages exogenous losses in where the congestion loss probability pc(t) depends the communication of the feedback signal from on our choice of a queue management implementa- the bottleneck link. tion at the bottleneck node. For DropTail, pc(t) is simply given by 3. Modeling TCP + Exogenous losses 0 bðtÞ < B; pcðtÞ¼ ð5Þ In this section, we extend an analytical fluid 1 bðtÞ¼B; model, similar to that proposed in [13,19,12,20],to where B is the maximum buffer size.2 capture the effect of exogenous losses on closed-loop An overlay node needs to manage its capacity TCP control loops. We present ns-2 [21] simulations and buffer much in the same way as an underlay to validate our observations from the model. router would. If we assume that the bottleneck OTM node manages its local buffer using RED 3.1. Modelderivation [11], the congestion loss probability p (t) is given by3 8 c > 6 We consider a dynamic fluid model of m TCP < 0 vðtÞ Bmin; connections traversing a single bottleneck overlay pcðtÞ¼> rðvðtÞ1Þ Bmin < vðtÞ < Bmax; ð6Þ node whose link capacity is C. The round trip time : 1 vðtÞ P Bmax; ri(t) at time t for connection i is equal to the round- where r and 1 are the RED parameters given by trip propagation delay Di between the sender and P max and Bmin, respectively, and v(t) is the aver- the receiver for connection i, plus the queuing delay BmaxBmin age queue size, which evolves according to the at the bottleneck node. Thus ri(t) can be expressed by equation: v_ðtÞ¼bCðvðtÞbðtÞÞ; 0 < b < 1. ð7Þ bðtÞ r ðtÞ¼D þ ; ð1Þ i i C where b(t) is the backlog buffer size at time t at the bottleneck node. We denote the propagation delay 2 We assume that when operating in a certain regime at time t, e.g., when b(t)

Dsib ¼ aiDi. ð2Þ tions presented later in this section. 3 For simplicity, we follow the same assumptions of other The backlog buﬀer b(t) evolves according to the studies by ignoring the uniformization of packet drops [11]. This equation: assumption is relaxed in our ns-2 simulations. ARTICLE IN PRESS

6 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx

Notice that in the above relationship, we multiply b S1 R1 by C since RED updates the average queue length at Senders© access links every packet arrival, whereas our model is a ﬂuid S R model [13,12]. 2 2 Bottleneck C The throughput of TCP, xi(t) is given by

wiðtÞ xiðtÞ¼ ; ð8Þ riðtÞ Receivers© access links with S R where wi(t) is the size of the TCP congestion window m exogenous losses m for sender i. Fig. 1. Dumbbell topology used in numerical evaluation and According to the TCP Additive-Increase Multi- simulations. plicative-Decrease (AIMD) rule, the dynamics of TCP throughput for each of the m connections can be described by the following differential We refer the reader to [22], where we show the equations: linearization of the system (TCP + exogenous losses) modeled above. In particular, we show how xiðt riðtÞÞ such a system switches between an open-loop x_ iðtÞ¼ 2 ð1 qðt Dbsi ðtÞÞÞ ri ðtÞxiðtÞ control, when exogenous losses are high, and a xiðtÞxiðt riðtÞÞ closed-loop control, otherwise. This switching ðqðt D ðtÞÞÞ 2 bsi between operating regions prevents us from using i ¼ 1; 2; ...; m. ð9Þ traditional transient control analysis. Thus, in the remainder of this section, we solve the above set The first term represents the additive increase of non-linear equations numerically for a careful rule, whereas the second term represents the multi- and continuous tracking of the models behavior plicative decrease rule. Both sides are multiplied through different operating regions. by the rate of the acknowledgments coming back due to the last window of packets xi(t ri(t)). Thus, 3.3. Impact on efficiency for positive acknowledgments that are arriving at a rate of xiðt riðtÞÞð1 qðt Dbsi ðtÞÞÞ, the window 1 Fig. 1 depicts the topology under consideration. wi(t) is increased by and for the negative riðtÞ We set the total number of competing connections acknowledgments that are arriving at a rate of to 20; we set the capacity C to 2000 pkts/s; and we x ðt r ðtÞÞqðt D ðtÞÞ, the window, w (t) is halved. i i bsi i chose the propagation delay of all connections uni- In the above equations, the time delay from the formly at random between 80 and 120 ms. Each con- bottleneck to sender i, passing through the receiver nections fair share of the link is around 100 pkts/s. i, is given by The total buffer size at the bottleneck is chosen to

Dbsi ðtÞ¼riðtÞDsib. ð10Þ be 250 packets. REDs minimum and maximum buffer thresholds are set to 50 and 120 packets, respec- The above analytical model captures the essential tively. The weight parameter b was set to 0.00001 dynamics necessary to gain valuable insights. We and P was set to 0.1. We also chose a in Eq. (2) later validate the model using more detailed simula- max i uniformly at random in the interval [0.25–0.5]. Dur- tion experiments. ing the time period [0,20) we introduce 0% exogenous losses, during [20,40) the rate of exogenous 3.2. Modelapplication losses is increased to 1% and finally during [40,60], exogenous losses are increased further to 5%. Low et al. [12] studied the dynamics of TCP over Fig. 2 shows the throughput and the queue size RED queues through linearization around equilib- obtained using our numerical solution under both rium points.4 While useful, linearization fails to DropTail (top row) and RED (bottom row).5 track the system trajectories across different regions In the first 20 s, i.e., under zero exogenous losses, dictated by the non-linear equations. TCP throughput oscillates between low and high

4 Linearization assumes (and hence requires) that the system 5 We later consider other AQM techniques and investigate their always stays within a certain operating regime. vulnerabilities to exogenous losses. ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 7

200 10 300 12 Throughput Exogenous Losses Queue Size Exogenous Losses 180 9 250 10 160 8 % 140 7 % 200 8 120 6

100 5 150 6 enous Losses Throughput 80 4 Queue Size g 100 4 Exogenous Losses 60 3 Exo

40 2 50 2 20 1

0 0 0 0 0 10 20 30 40 50 60 0 10 20 30 40 50 60 Time Time (a) DropTail:Throughput (b) DropTail:Queue Size

200 10 300 12 Throughput Exogenous Losses Queue Size Exogenous Losses 180 9 250 10 160 8 % 140 7 % 200 8 120 6

100 5 150 6 enous Losses Throughput 80 4 Queue Size g 100 4 Exogenous Losses 60 3 Exo

40 2 50 2 20 1

0 0 0 0 0 10 20 30 40 50 60 0 10 20 30 40 50 60 Time Time (c) RED:Throughput (d) RED:Queue Size

Fig. 2. Impact of exogenous losses (0%, 1%, and 5%) on efficiency of DropTail (top) and RED (bottom) as reflected by throughput (left) and buffering (right). sending rates for DropTail, while RED sustains following two equations should be satisfied for an these oscillations only until the average queue size efficient network utilization. These equations are reaches its steady-state value (around time 10). In obtained by setting the derivatives to zero in Eqs. the next 20 s, when the level of exogenous losses in- (3) and (9). X creases to 1%, TCP throughput converges (perfectly) m ^xi ¼ C; ð11Þ to its fair share under both DropTail and RED. No- i¼1 sffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi tice how the queue size converges to a steady-state 1 1 (non-zero) value, hence the system is well utilized. xî ¼ 2 1 . ð12Þ r^ q^ In the last 20 s, exogenous losses (now increased to i i

5%) result in the convergence of each xi(t), albeit Clearly, the steady-state TCP throughput xî is inver- to a value lower than the fair share and the queue sely proportional to the square root of the total loss size drops to zero, hence the system is under utilized. probability qî, which in turn is directly affected by 6 This observation suggests that low levels of exoge- the exogenous loss rate pê. For a steady-state nous losses (e.g., 1%) do not degrade the throughput behavior, qî must be larger than zero. Having no of TCP. But clearly, when exogenous loss rates are drops removes the upper limit on the rate/window increased significantly (e.g., 5%), TCPs throughput and this, in theory, will cause it to grow indefinitely. suffers and the system becomes under utilized (e.g., below the fair share of 100 pkts/s). 6 Observe that Eq. (12) resembles the so-called TCP-friendly A transmission control loop is said to be efficient equation [23], except that in our model, qî is not necessarily a if the TCP throughput for that loop matches the Bernoulli probability, but depends on queue management bottleneck link capacity. Thus, at steady state, the parameters. ARTICLE IN PRESS

8 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx

As the steady-state value of pê increases, the round-trip time is equal to the propagation delay sending rate would start to decrease, approachingP with zero buffer occupancy. m zero. This could prevent TCP throughput i¼1^xi To validate the above observations, we con- from reachingPC, i.e. Eq. (11) cannot be satisfied. ducted a simple ns-2 simulation on a simple dumb- m The value of i¼1^xi being less than C means that bell topology similar to the one in Fig. 1. The the system is under utilized. Hence TCP is forced bottleneck link capacity is set to 16 Mb and its to operate with no buffering at the bottleneck, and propagation delay is set to 1 ms. A total of 20 no congestion signals going back to senders. When TCP connections are created between the senders this happens, the TCP transmission control loop is and the receivers. Senders and receivers connect to actually broken—it operates as an open-loop control the nodes through access links with propagation de- system with no feedback from nodes. lay chosen uniformly at random between 1 and When exogenous losses are not present, nothing 4 ms. The receivers access links are associated with hinders the increase of TCP throughput so as to error modules that would represent the effect of match its bandwidth share.7 Once the connection exogenous losses. At time 0, we start with no exog- hits its bandwidth share, packets start to accumulate enous losses, at time 100 we set the exogenous losses until b(t) reaches B under DropTail, or the average to 2% across all receivers access links, at time 150, queue size starts building up until it exceeds Bmin exogenous losses are increased to 7% and the exper- under RED. At that time, congestion signals are iment ends at time 200. We use DropTail at the bot- generated and the sender would back off and this tleneck link and we ignore the first 50 s of the cycle repeats (cf. Eq. (9)). simulation experiment. Fig. 3(b) and (c) show the ef- The presence of exogenous losses imposes an fect of exogenous losses on fairness and on queue upper limit on TCPs throughput and it is crucial size, respectively. Fairness is computed across all where this upper limit lies. If this upper limit is close connections every 1-s interval, using Chiu and Jains to the connections long-term fair share, then these Fairness Index [24], which is given by P exogenous losses turn out to improve the connec- m 2 ð x Þ tions convergence to its fair share. This is exactly f ðx ; x ; x ; ...; x Þ¼ iP¼1 i . ð13Þ 1 2 3 m m m x2 what happens in the time period [20,40) in Fig. 2. i¼1 i Without such exogenous losses in [0,20), the con- Notice how the fairness index improved signifi- nections throughput shows large oscillations under cantly when exogenous losses increased to 2% since DropTail. such value in effect helped the connections converge Consider the same setup described above with to their fair share, and also helped the buffer size DropTail, except that all connections have an iden- converge. Increasing exogenous losses to 7% leads tical propagation delay of 100 ms. If the goal is to to a deterioration in the fairness index and leads allocate an equal share of the bandwidth to each to under utilization of the network since the queue TCP connection, then using Eq. (11), each connec- size gets closer to 0. tions (long-term) fair share, xî, would equal Many protocols have been developed for hiding 100 pkts/s. Eq. (12) can be solved for qî for a given all exogenous losses from the sender [2,1]. For round-trip time ^ri, which depends on the steady- example, in Snoop [2] the connection between the state buffer occupancy. Indeed, Fig. 3(a) shows that server and the client is in effect intercepted by an when exogenous losses are in the range of 1–2%, the OTM proxy. This proxy buffers data packets to al- throughput converges to the fair share value. Below low link-layer retransmission when duplicate or above these values, the fair share is not matched acknowledgments, indicating packets lost over the due to either oscillations (left) or under utilization wireless proxy-client link, arrive at the proxy. Snoop (right), respectively.8 Notice that exogenous losses does not allow such duplicate acknowledgments to around 2% represent the case where the steady-state pass back to the sender to prevent it from doing fast retransmit and recovery (i.e., halving its sending rate). While such OTM protocols attempt to im- 7 The connection is still limited by its round-trip time, but prove efficiency by removing the upper limit im- eventually will hit its bandwidth share. We assume that connec- posed on throughput by exogenous losses, they tions are not limited by the advertised receivers window. could be hindering the convergence to fairness! Fur- 8 The oscillations in the fluid model are a by-product of synchronization effects among flows since all of them react to the thermore, hiding further packet losses from connec- same error signal. This effect is less pronounced in simulations. tions that are already getting their fair share would ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 9

250 105 12 Instantaneous Exogenous Losses 100 Average 11 1.0 10 200 95 9 90 0.8 8 150 85 7 0.6 6 80 Queue Size Throughput 5 100

75 Fairness Index 0.4 4 3 70 50 0.2 2 Exogenous Losses % 65 1 60 0 0 0 0 1 2 3 4 5 50 100 150 200 50 100 150 200 a Exogenous Losses % bcTime Time

Fig. 3. Model-predicted effect of exogenous losses on efficiency (left) and validation via simulations of effect on fairness (middle) and queue size (right) under DropTail. not be beneficial, but would only add the overhead marking of packets is conditioned on the local of complete hiding (e.g., the cost of buffering and lo- queue occupancy in order to stabilize the queue. cal retransmission at the Snoop proxy)—not to As evident from the results in Fig. 2(d), RED exhib- mention the fact that another process (in this case its transient inefficiencies when faced with variabil- an AQM) would have to reintroduce packet losses. ity in exogenous loss rates over short time scales. Ideally, we would like to always report a value of Specifically, at time 20, TCPs queue size drops to qî to sender i that corresponds to its fair share, since zero, before converging again to the new steady- this would mean that the network is utilized effi- state value of around 50. This transient anomaly is ciently while, at the same time, connections have less pronounced under DropTail. a fair chance to compete. In the next section, we The undesirable transient behavior exhibited un- address the challenges behind active management der RED is due to REDs unawareness of exoge- of exogenous losses in an overlay setting to achieve nous losses, which is exacerbated by the lag time this goal. necessary for the average queue size (seen by RED) to reflect the ‘‘real’’ conditions. To elaborate 4. Active tuning of exogenous losses on this, consider the case when REDs average queue length is above Bmin. Now consider a situa- As discussed in the previous section, for given tion whereby TCP flows react to a sudden increase bottleneck link and RTT characteristics, there exists in the exogenous loss rate (by backing off), which a desirable value for the loss rate that would pro- in turn would cause the RED queue to drain. Since mote both convergence and efficiency of a TCP con- RED uses the average queue length as an indicator trol loop. We use the term quiescent loss rate to refer of congestion, it would take RED some time to real- to this desirable value. For example, a quiescent loss ize this new ‘‘drained’’ state. As a result, RED rate of 2% yielded both efficiency and convergence would keep on generating congestion signals (by for the experimental setting used in Fig. 3(a). In this dropping or marking packets) according to the stale section we examine the advantages and disadvan- higher value of its average queue length—causing tages of alternative active (AQM-inspired) ap- further degradation in efficiency.9 Obviously, under proaches for relaying such quiescent loss rates such conditions, the congestion-minded design of from an overlay traffic manager to senders. RED is challenged by exogenous losses, since it is no longer true that the sender reduces its rate only 4.1. Exogenous loss unaware signaling (local in response to congestion signals! As soon as the fairness) average queue size catches up with the new value below Bmin, RED ceases to send its feedback signal. At An overlay traffic manager (such as traffic shap- that point, TCP is in fact operating as an open-loop ers) may simply ignore exogenous losses by imposing a loss rate value that improves some local 9 This phenomenon was noted in [26], prompting the need for metric (e.g., stability of the buffer backlog). An decoupling the queue size from the dropping/marking probabil- example of such a technique is RED (or other vari- ity. We later examine the degree of vulnerabilities of more recent ants thereof, e.g., [25]), whereby the dropping or AQM techniques to exogenous losses. ARTICLE IN PRESS

10 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx control system and starts to increase its sending ward error correction techniques, or replication rate. over multiple paths (i.e. dispersity routing [28]) The inability of RED (as a representative of across the overlay network. exogenous-loss unaware traffic management) to In practice, requiring that an XQM be able to ob- cope with exogenous losses is further complicated tain an accurate estimate of exogenous loss rates by issues of heterogeneity in flow characteristics whether explicitly from or through measurement (e.g., the possibly wide range of exogenous loss rates of network underlays is hard to achieve.10 Thus, across flows or aggregates thereof). Clearly, no traf- alternately, an XQM could dynamically adjust its fic management would be able to address issues of behavior (i.e., figure out the exact levels of losses global fairness without some accounting of flow to be introduced or hidden) in response to each con- characteristics. Indeed, if RED were to achieve glo- nections performance, without having to explicitly bal fairness, it would require more than parameter know the exogenous loss rate for such connections. tuning, namely awareness of the presence of these In the above discussion, we have assumed that losses and invoking the right control rules. exogenous loss rates are static. In a real setting, this is likely not to be the case. Thus, it is important to 4.2. Exogenous loss aware signaling (global fairness) assess the impact of such variability, both over long and short time scales. We propose two different The above discussion suggests that, towards glo- methods to massage the exogenous losses observa- bal fairness, it is crucial for a traffic management ble by senders, which we refer to as long-term approach to take into account the presence of exog- adjustment and short-term compensation. Due to enous losses. One possibility is to have overlay traf- space limitation, we refer the reader to [22] for an fic managers use a technique similar to FRED [27]. evaluation of these techniques. Having demon- FRED accounts (albeit implicitly) for exogenous strated the benefits from exogenous aware signaling, losses by protecting fragile flows—flows with small we next turn our attention to how to design queue window sizes. It is important to note that account- management schemes that are exogenous-loss ing for exogenous losses is more of a side effect than aware. a ‘‘by design’’ feature since FRED protects flows with excessively small window sizes, independent 5. eXogenous-loss aware Queue Management of whether flow fragility is due to exogenous losses, or simply a reflection of that flows fair share. In In this section, we discuss and evaluate our pro- contrast, our XQM approach presented later in this posed eXogenous-loss aware Queue Management paper, makes the decision of when to introduce (XQM) approach to traffic management in an over- losses, when to hide losses, and when not to inter- lay setting. An XQM agents main goal is to tune fere, based on the level of exogenous losses present. the exogenous losses that are already present in In effect, an XQM agent in an OTM node utilizes the network to improve fairness of flows going such external losses, toward its own feedback signal. through that agent, without sacrificing efficiency. Thus, it provides the minimum interference and Thus it provides each flow11 with its fair share of re- only when needed. sources. An XQM agent maintains a profile for each Assuming that the exogenous loss rate for a flow active flow it manages. This profile includes the cur- can be relayed to (measured/estimated by) an rent flows throughput xi(t,MP), measured as the XQM-enabled overlay traffic manager, then it is number of packets sent over the past Measurement possible to ensure that the sender will only see the Period (MP) and the current imposed loss rate quiescent end-to-end loss rate by having the XQM qi(t). Note that this per-flow (or per-class) state agent adjust its own control rules accordingly. needs to be maintained only for flows/classes that Namely, if exogenous losses are below the quiescent rate, then it is possible to ‘‘introduce’’ losses to pro- mote efficient convergence to a fair share. This 10 Recent ‘‘measurement from the middle’’ studies suggest that could be done through randomly dropping or mark- such approach may indeed be feasible [29]. 11 ing packets. If exogenous losses are higher than the For the purposes of this discussion, we do not insist on a strict definition of a flow. One can think of a 4-tuple definition (source quiescent rate then it would be necessary to ‘‘hide’’ IP, destination IP, source port #, destination port #), a 2-tuple such losses from the sender. This could be done in definition (source IP, destination IP) or simply aggregates of many ways, including link layer retransmission, for- these. ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 11 are active in the overlay network—we expect their behavior of the XQM agent. We summarize our number not to be large. experience with these parameters next; details can On a packet arrival, the XQM agent in the OTM be found in [22]. The default values used in our node identifies the flow this packet belongs to and simulations were found to be quite robust over a drops the packet with probability pi(t) that is a func- wide range of scenarios. tion of the current quiescent loss rate: First we focus on the interplay between the measurement and control periods. Because the control qiðtÞ period (CP) directly affects the frequency with which piðtÞ¼ ; ð14Þ 1 k qiðtÞ the controller is invoked, we choose a small value (around 10 ms). Having a larger CP will cause the where k is the number of packets queued since the XQM agent to be less responsive, while having a last packet drop/mark. This insures that XQM smaller CP would only add to the overhead of spreads losses uniformly over time [11]. Every Con- invoking the controller. For a correct estimate of trolPeriod (CP), the XQM agent updates the cur- the connections throughput, the measurement peri- rent imposed loss rate q (t) for each active flow. In i od (MP) should be a multiple of the congestion [22] we propose three different schemes for imple- epochs. That is, it should be long enough to capture menting such an update (XQM ON–OFF, XQM multiple packet drops. Having a shorter MP, will PI-T and XQM PI-TB). We only present XQM cause errors in throughput estimation due to win- PI-TBhere; thus for the remainder of this paper, dow fluctuation. However, having a longer MP, will we use XQM and XQM PI-TBinterchangeably. limit the XQM agents ability to capture short-term behaviors.13 5.1. XQM PI-TB: PI throughput and buffer matching We now turn our attention to d and /, the weights in Eq. (15). These weights play an impor- In this implementation, two error signals are ob- tant role in the decision process. They specify the tained by comparing the current throughput of the tradeoff between efficiency and fairness. In particu- flow with its target fair share (allocated by the lar, a higher value of / will tend to improve effi- XQM agent) and the current buffer size with the tar- ciency, while a higher value of d will tend to get buffer size. Maintaining the buffer size at a low improve fairness. There are four possible cases, target ensures less jitter and shorter round-trip time. which we consider next. XQM PI-TBis thus given by: 12 The first two of these cases are straightforward; they correspond to situations in which the two con- qiðt þ CPÞ¼qiðtÞþdðxiðt; MPÞ^xiÞ stituent controllers in Eq. (15) are in agreement as ^ þ /ðbðtÞbÞ; ð15Þ to whether the loss rate imposed on a flow is to go where b(t) is the current buffer size and b^ is the tar- up or down. Namely, these two cases occur when get buffer size. Unless otherwise specified, the allo- the bandwidth (for a flow) and the buffer size are both both cated (fair) share of flow i, ^x , is set by the XQM below their prescribed values, or above i their prescribed values. Clearly, for the former, the agent as simply an equal rate of the bottleneck XQM agent will decrease the loss rate imposed on capacity. As noted later, any weighted allocation the flow, and for the latter, the XQM agent will of rates could also be applied. We also note that ^x i increase the loss rate imposed on the flow. determines the quiescent loss rate dynamically through Eq. (15), and hence there is no need for The other two cases correspond to situations in explicitly computing the quiescent loss rate from which the two constituent controllers in Eq. (15) Eq. (12) (as in XQM ON–OFF [22]) which requires are at odds with one another regarding whether the estimation of RTT from the middle as in [29]. the loss rate imposed on a flow is to go up or down. The explicit estimation of exogenous loss rates is also For example, what if a connections throughput is less than the targeted throughput, but the buffer size not required. Finally the four parameters CP, MP, d is larger than the targeted buffer size? In our exper- and / play a very important role in the general iments (some of which we will present in the next section), we found that having a value of / rela- 12 This version of XQM can be regarded as a per-flow version of REM [18] where fair rates are explicitly allocated to flows so as to overcome the negative effects of exogenous losses. 13 Note that unlike REM, we are decoupling MP and CP. ARTICLE IN PRESS

12 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx tively larger than d is helpful in cases when some congested/bursty path segments a connection tra- connections are unable to get their fair share of verses, the noisier the feedback signal (due to exog- the throughput (e.g., they are source limited). Under enous losses on non-bottleneck links). In this such conditions, we allow other connections to grab section, we show how an XQM agent in effect the available bandwidth by giving a higher ‘‘weight’’ ‘‘adopts’’ packet losses on other underlay network to buffer-size matching. On the other hand, in our links as its own—in effect using them towards the experiments, we found that having a value of / that total packet losses it needs to impose on the flow. is much greater than d tends to hurt connections An XQM agent would decrease the losses it imposes that are already below their fair share (in an attempt as exogenous loss rates increase toward the quies- to whip the buffer into matching its prescribed cent loss rate, moreover it would increase the value value). By tuning the values of d and /, the XQM it hides as exogenous loss rates increase above the agent is able to expose the tradeoffs between quiescent loss rate. However, if the exogenous losses efficiency and fairness. are close to the quiescent loss rate, the XQM agent To summarize, an XQM agent at an OTM node will not interfere. has two key design features. The first is that the Fig. 4 depicts the topology under consideration. XQM agent decouples the measurement period We have three links AB, BC and CD of capacity from the control period. This decoupling allows 100 Mbps, 150 Mbps and 150 Mbps, respectively. the XQM agent to improve fairness (over longer For simplicity, all links have a one-way propagation time scales) without sacrificing efficiency (over delay of 1 ms. A total of 10 FTP connections, with shorter time scales). This is achieved by exercising unlimited data to send, traverse the overlay link from control over short time scales based on throughput A to D. We refer to these as the overlay AD flows, measured over longer time scales. The second key with IDs from 1 to 10. In addition, three groups of feature of the XQM agent is that it exposes the 10 FTP connections, each representing cross-traffic tradeoffs between efficiency (over shorter time with unlimited data to send, traverse exactly one of scales) and fairness (over longer time scales). The the links in the topology (i.e., those underlay links selection of the characteristic time scales for mea- making up the overlay link AD). We refer to these surement and for control, as well as the adjustment as the AB, BC and CD flows. Sources as well as of the tradeoff between efficiency and fairness are receivers of these FTP flows connect to the overlay both possible to manage dynamically based on the traffic managers at A and D, and to the routers at traffic profile. This dynamic tuning of the XQM Band C, through separate ‘‘access’’ links. agents operation (based on traffic profiling) is the In the topology of Fig. 4, the first link (AB) is the subject of a future paper. bottleneck of the 10 overlay AD flows and it uses the traffic management approach being evaluated– 6. Simulation results namely XQM, RED, FREQ, REM, or PI—whereas the second and the third (underlay) links (BC and In this section, we present results from extensive CD) are managed using RED.14 ns-2 [21] simulation experiments we conducted to Unless otherwise stated, in our experiments, we assess the advantages of exogenous-loss awareness. set REDs minimum and maximum buffer thresh- We do so by comparing an OTM that uses XQM olds to 50 and 120 packets, respectively. The weight with that using other exogenous-loss unaware ap- parameter b is set to 0.0001 and Pmax is set to 0.1. proaches similar to those proposed for router-level The buffer size is chosen to be 250 packets at each AQMs—namely RED [11], FRED [27], REM [18] link. All packets are 1000 bytes in size. Also, unless and PI [26]. otherwise stated, in our experiments, we set the parameters of the XQM agent at the OTM node 6.1. Effect of losses due to cross-traffic on A, CP, MP, d and / to be 0.01 ms, 10 s, 0.00001 non-bottleneck links and 0.00004, respectively.

Ideally, for a TCP connection to reach its fair 14 share, it should get its loss signal from the bottle- Note that the bottleneck capacity of the overlay link does not have to be incident to the overlay node, A in this case. In general, neck link only. In practice, due to network dynam- the overlay node assumes its capacity to be the minimum capacity ics, a TCP connection could experience packet among all its underlay links. Several techniques exist to measure losses on (multiple) non-bottleneck links. The more such bottleneck capacity over the overlay link. ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 13

S Sc Rc R 1 Access Links 1 Cross Traffic S R 2 Bottleneck 2 A BCD

S m Cross Traffic Rm S Rc Sc Rc c

Fig. 4. Topology used in ns-2 simulation experiments. A–D represents an overlay link whereby an XQM agent at the OTM node A manages the capacity of the bottleneck.

6.1.1. Experiment 1 maintains the buffer size at the target level of 50 We start with a simple case where all connections packets. have the same round-trip time. To do so, we adjust the propagation delay on the access links so that all 6.1.2. Experiment 2 connections have the same round-trip time, taking In the previous experiment, we fixed the propaga- into account the queuing delay at the (underlay) tion delay so that all connections experience the links BC and CD for overlay connections AD. We same round-trip time, and thus giving us an oppor- present the results across the first link (at the bottle- tunity to observe/study the effect of exogenous neck overlay node A). losses on multiple (non-bottleneck) hops on overlay Fig. 5 compares the performance of the different connections AD. Now, we repeat the same experi- schemes—each shown in plots on separate rows. ment, except that all the access links (for connec- Plots in the first (leftmost) column represent the aver- tions AD as well as for cross traffic connections age throughput achieved by each connection, which AB, BC and CD) have a propagation delay that is is computed over the interval [20–100]. Plots in the uniformly distributed between 5 and 10 ms. Now, second column represent the instantaneous through- connections AD have a longer round-trip time com- put computed every one second interval. We only pared to AB, since they traverse more links. So two present two connections, one that belongs to the questions arise: how much worse would overlay overlay set AD and another one that belongs to the connections AD fare? and how effective is the set AB. Plots in the third column represent the aver- XQM agent in dealing with such scenarios? age losses seen by each flow over the interval [20– Increasing the round-trip time for overlay con- 100]. Overlay connections AD see losses that are also nections AD would only make the situation worse on other underlay links, i.e., ‘‘exogenous losses’’. Fi- (i.e., if they cannot get their fair share with a shorter nally, plots in the last (rightmost) column represent round-trip time, they will certainly not get their fair the instantaneous queue size. In all plots, we ignore share with a longer one, due to the additional bias the first 20 s (for simulation warm-up purposes) of TCP against connections with longer round-trip and we calculate all metrics starting from time 20. times). Indeed, this is the case under RED, FRED, Despite the fact that all connections have the PI and REM; overlay connections AD (with longer same round-trip time, overlay connections AD that RTT and subjected to additional exogenous losses traverse multiple congested links (i.e. connections 1– on underlay links) cannot get their fair share. Due 10) end up with less throughput. Since RED, REM to space limitation, we only present the performance and PI traffic management approaches apply the under RED (as a baseline) and FRED (which had same loss rate across all connections, they do not the best performance among all other exogenous- compensate for any exogenous losses on other links. loss unaware AQMs). FRED, on the other hand, compensates a little bit In our experiments we found that OTMs using as evident by the values of the loss rate for FRED. REM and PI agents behave quite similarly to those FRED applies lower loss rate values to connections using RED, imposing the same loss rate across all AD and higher values to AB. Still, AD (overlay) connections. FRED imposes the minimum losses, connections cannot reach their fair share. however it still taxes AD overlay connections, putt- On the other hand, the XQM PI-TBagent at the ing them at a disadvantage with respect to reaching OTM node A applies just enough losses so that the their global fair share of bandwidth. The XQM total loss rate seen by any connection is the same— agent, on the other hand, does not drop any packets hence the better fairness delivered by XQM. Also, it from AD overlay connections (connections AD are ARTICLE IN PRESS

14 M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx

1000 2500 2.5 250 Connection ID # 1 Bottleneck Losses 900 Connection ID # 20 Exogenous Losses 800 2000 2 200 700 600 1500 1.5 150 500

400 1000 1 Queue Size 100 300 Loss percentage Throughput (pkts/sec) Average Throughput 200 500 0.5 50 100 0 0 0 0 0 5 10 15 20 20 40 60 80 100 0 5 10 15 20 20 40 60 80 100 Connection ID Time Connection ID Time (a) RED

1000 2500 2.5 250 Connection ID # 1 Bottleneck Losses 900 Connection ID # 20 Exogenous Losses 800 2000 2 200 700 600 1500 1.5 150 500

400 1000 1 Queue Size 100

300 Loss percentage Average Throughput

200 Throughput (pkts/sec) 500 0.5 50 100 0 0 0 0 0 5 10 15 20 20 40 60 80 100 0 5 10 15 20 20 40 60 80 100 Connection ID Time Connection ID Time (b) FRED

1000 2500 2.5 250 Connection ID # 1 Bottleneck Losses 900 Connection ID # 20 Exogenous Losses 800 2000 2 200 700 600 1500 1.5 150 500

400 1000 1 Queue Size 100

300 Loss percentage

Average Throughput 200 500 0.5 50 Throughput (pkts/sec) 100 0 0 0 0 0 5 10 15 20 20 40 60 80 100 0 5 10 15 20 20 40 60 80 100 Connection ID Time Connection ID Time (c) REM

1000 2500 2.5 250 Connection ID # 1 Bottleneck Losses 900 Connection ID # 20 Exogenous Losses 800 2000 2 200 700 600 1500 1.5 150 500

Fig. 5. Comparative performance of various AQM approaches, showing long-term throughput for all flows (left), instantaneous throughput for two exemplary flows, average loss rates seen by all flows, and instantaneous buffer size (rightmost). thus only limited by exogenous losses on other and XQM PI-TB, respectively (also, the Fairness In- underlay links). The plots in rows (a), (b) and (c) dex is noted). Since overlay connections AD are still of Fig. 6 show the performance of RED, FRED, limited by exogenous losses, despite the XQM agent ARTICLE IN PRESS

M. Guirguis et al. / Computer Networks xxx (2005) xxx–xxx 15