Combating Bufferbloat in Multi-Bottleneck Networks: Theory and Algorithms

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE/ACM TRANSACTIONS ON NETWORKING 1 Combating Bufferbloat in Multi-Bottleneck Networks: Theory and Algorithms

Jiancheng Ye , Member, IEEE, Ka-Cheong Leung , Senior Member, IEEE, and Steven H. Low , Fellow, IEEE, ACM

Abstract— Bufferbloat is a phenomenon in computer networks “bufferbloat” [2]. Larger buffers and more delay-sensitive where large router buffers are frequently filled up, resulting in applications on the Internet have made bufferbloat a pressing high queueing delay and delay variation. More and more delay- issue. sensitive applications on the Internet have made this phenomenon Active queue management (AQM) has been considered the a pressing issue. Interacting with the Transmission Control key to combating bufferbloat, since it can avoid full buffer Protocol (TCP), active queue management (AQM) algorithms and high queueing delay by dropping or marking packets run on routers play an important role in combating bufferbloat. However, AQM algorithms have not been widely deployed due to probabilistically before a buffer gets full. Random Early complicated manual parameter tuning. Moreover, they are often Detection (RED) [3] proposed in 1993 is the first AQM. designed and analyzed based on network models with a single Numerous AQM algorithms have been proposed since then, bottleneck link, rendering their performance and stability unclear such as Proportional Integral (PI) controller [4], Random in multi-bottleneck networks. In this paper, we propose a general Exponential Marking (REM) [5], and Adaptive Virtual Queue framework to combat bufferbloat in multi-bottleneck networks. (AVQ) [6]. Although research on AQM started almost three We first present an equilibrium analysis for a general multi- decades ago, none of the proposed AQM algorithms has been bottleneck TCP/AQM system and provide sufficient conditions widely deployed because of complicated parameter tuning and for the uniqueness of an equilibrium point in the system. We then sensitivity of algorithm performance to parameter settings. decompose the system into single-bottleneck subsystems and Recently, Controlled Delay (CoDel) [7] has been proposed derive sufficient conditions for the local asymptotic stability of the subsystems. Using our framework, we develop an algorithm as a modern AQM to address bufferbloat by controlling the to compute the equilibrium point of the system. We further queueing delay to a target value. It has been shown via present a case study to analyze the stability of the recently experiments that CoDel performs well across various network proposed Controlled Delay (CoDel) in multi-bottleneck networks settings with a simplified set of default parameters. More and devise Self-Tuning CoDel to improve the system stability. recently, Cisco researchers have proposed Proportional Integral Extensive numerical and packet-level simulation results not only controller Enhanced (PIE) [8], which is a lightweight AQM verify our theoretical studies but also show that our proposed scheme that can effectively control the average queueing delay Self-Tuning CoDel significantly stabilizes queueing delay in multi- to a reference value. bottleneck networks, thereby mitigating bufferbloat. However, the existing AQM algorithms are usually designed Index Terms— Active queue management, bufferbloat, conges- and analyzed based on network models with a single bottle- tion control, equilibrium, stability. neck link. Consequently, their performance and stability are unclear in general networks with multiple bottleneck links, I. INTRODUCTION such as the Internet. We consider a network link that is fully A. Motivation utilized as a bottleneck link in this paper. Stability is an HE Internet is playing an increasingly important role important property for a dynamical system which concerns Tin providing real-time communication and entertain- whether the system with initial state near an equilibrium will ment for people via network applications that have stringent converge to the equilibrium. We find that the recently proposed requirements on end-to-end delay. In recent years, inexpen- CoDel may yield unstable queueing delay in multi-bottleneck sive memory has led to large buffers installed in Internet networks with the default parameters that have been shown routers. Excessive buffering of packets results in high queueing to produce stable queueing delay in various single-bottleneck delay and large delay variation, which is recently termed network scenarios [1]. Thus, it is necessary to investigate the performance and stability of AQM in multi-bottleneck Manuscript received April 25, 2020; revised December 23, 2020; accepted networks. To the best of our knowledge, there is very limited February 14, 2021; approved by IEEE/ACM TRANSACTIONS ON NETWORK- research on AQM in such networks. In [9], the instability of ING Editor C. Joo. This work was supported in part by the Research Grants Council, Hong Kong, China, under Grant 17204614. A preliminary version RED was observed in a network with two bottleneck links of this work was presented in IEEE INFOCOM 2018 [1]. (Corresponding and the root cause for this unstable case was analyzed using author: Ka-Cheong Leung.) control theory. In [10], a mathematical model for TCP/RED Jiancheng Ye is with the Department of Electrical and Electronic Engineer- system with multiple bottleneck links was developed, and the ing, The University of Hong Kong, Hong Kong (e-mail: [email protected]). Ka-Cheong Leung is with the School of Computer Science and Technology, stability of the system was then analyzed based on Lyapunov Harbin Institute of Technology at Shenzhen, Shenzhen 518055, China (e-mail: stability theory. However, [9] only provided a case study for [email protected]). the instability of RED in a two-bottleneck network, whereas Steven H. Low is with the Department of Computing and Mathematical [10] simply assumed that there existed a unique equilibrium Sciences, California Institute of Technology, Pasadena, CA 91125 USA, and also with the Department of Electrical Engineering, California Institute of point in a multi-bottleneck TCP/RED system. In fact, it is Technology, Pasadena, CA 91125 USA (e-mail: [email protected]). possible to have infinitely many equilibria in the system, Digital Object Identifier 10.1109/TNET.2021.3066505 asshowninSectionVI-A.

Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on April 09,2021 at 21:01:56 UTC from IEEE Xplore. Restrictions apply. This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE/ACM TRANSACTIONS ON NETWORKING

On the other hand, the parameters of existing AQM algo- stability analysis of a TCP/AQM system is fundamental to the rithms are usually fixed. However, there is generally no fixed self-tuning of AQM parameters since it usually derives some parameter setting that can guarantee an AQM algorithm to stability conditions for the system, which explicitly relate the perform well in all possible network scenarios. Thus, it is AQM parameters with the network conditions. The stability highly desirable to automate parameter tuning for AQM based of RED and PI has been analyzed in [4]. By applying the on network conditions. In fact, AQM with self-tuning capa- Nyquist stability criterion, proper parameters for RED and PI bility has been recommended by the Internet Engineering are determined to ensure system stability. A sufficient stability Task Force (IETF) in RFC 7567 [11]. In order to automat- condition for the TCP/RED system with a single link and ically tune the parameters of an AQM, one needs to find heterogeneous sources has been derived in [17]. Using control fundamental relationships between the AQM parameters and theory, [18] has derived explicit conditions under which the network conditions. This requires a thorough understanding TCP/RED system is stable and discussed the stability region. of the TCP/AQM congestion control. Congestion control is a It should be noted that the existing AQM algorithms are distributed mechanism that aims to share network resources usually devised and analyzed based on models with a single efficiently among competing flows. It consists of a source bottleneck link. There is very limited research on AQM in algorithm and a link algorithm [12]. On the current Internet, multi-bottleneck networks. The stability of RED in a network the source algorithm is primarily the Transmission Control with two bottleneck links has been analyzed in [9]. In [10], Protocol (TCP). The link algorithm is either a simple drop- the authors have further developed a mathematical model tail algorithm or an AQM algorithm employed by a router. for the TCP/RED system with multiple bottleneck links We realize that AQM is the key to addressing bufferbloat, and analyzed the stability of the system using Lyapunov but there is still a lack of AQM deployment due to complicated stability theory. manual parameter tuning [2]. To facilitate the wide deployment Although this paper focuses on the AQM solution to of AQM, it is necessary to automate its parameter tuning which bufferbloat, we are aware that the approach of reducing the usually involves the stability analysis of the corresponding buffer size itself is also worth further exploration. The series TCP/AQM system. Therefore, we propose a general frame- of [19], [20], and [21] on buffer sizing has provided some work to analyze the equilibrium and stability of general multi- related technologies and theory. bottleneck TCP/AQM systems. The theoretical results obtained from the analysis can then guide us to design distributed C. Contributions self-tuning AQM algorithms, which can be implemented in bottleneck routers, so as to practically combat bufferbloat. In this paper, we systematically address bufferbloat in general multi-bottleneck networks, from both theoretical and B. Related Work practical perspectives. We make the following contributions: Over the last three decades, a variety of AQM algorithms • We present an equilibrium analysis for a general multi- have been proposed in the literature. They can be roughly bottleneck TCP/AQM system and provide sufficient con- categorized into AQM with static parameter settings and self- ditions for the uniqueness of an equilibrium in the system. tuning AQM. We first review representative AQM with static • We conduct stability study for the multi-bottleneck settings. RED [3] as the first AQM drops or marks packets TCP/AQM system with feedback delays and derive suf- with a probability computed by comparing the average queue ficient conditions for the local asymptotic stability of length to the predefined minimum and maximum thresholds. the system. Furthermore, we decompose the system into PI [4] computes the packet dropping probability based on the single-bottleneck subsystems and derive sufficient condi- deviations of the current queue length from a reference value tions for the local asymptotic stability of the subsystems. and a queue length history. REM [5] achieves high utilization • Our equilibrium and stability studies constitute a general and low loss and delay by stabilizing both the input rate around framework for analyzing the equilibrium and stability of the link capacity and the queue length around a small target. loss-based TCP/AQM systems with multiple bottleneck AVQ [6] adjusts the queue size and the capacity of a virtual links. Utilizing the framework, one can investigate the buffer as a function of the arrival rate. CoDel [7] drops a stability of AQM algorithms in multi-bottleneck net- departing packet if the queueing delay exceeds a predefined works, and design novel self-tuning AQM algorithms. target delay for a time period called interval. • Based on our framework, we develop an algorithm to In contrast to AQM with static settings, self-tuning AQM compute the equilibrium point of the system. Moreover, can dynamically adjust parameters based on network condi- we present a case study to analyze the stability of CoDel tions. Adaptive RED [13] has been proposed to increase the in multi-bottleneck networks. We further propose Self- robustness of the original RED by adapting the parameter Tuning CoDel, which dynamically adjusts the interval of the maximum packet dropping probability. Self-Tuning PI parameter of CoDel, so as to stabilize the queueing delays and Self-Tuning RED have been proposed in [14], where at all the bottleneck links, thereby mitigating bufferbloat. their parameters are automatically tuned in response to the • We perform extensive numerical and packet-level simula- online estimations of link capacity and traffic load. In [15], tions, which verify our theoretical studies and show that the authors have proposed a self-tuning feedback controller Self-Tuning CoDel significantly stabilizes queueing delay named NPD-RED, which automatically selects values for the in multi-bottleneck networks. parameters from their determined ranges. Using control theory, The rest of the paper is organized as follows. Section II [16] has proposed two self-tuning controllers for AQM routers presents a general multi-bottleneck TCP/AQM system model. and properly adapted the system to load changes. The recently In Section III, we investigate the existence and uniqueness proposed PIE [8] employs an internal auto-tuning mechanism of an equilibrium point in the system. Section IV presents that can adjust its control parameters based on the computed a stability study for the system. In Section V, we develop packet dropping probability, so that it can make a tradeoff an algorithm to compute the equilibrium point and propose between system stability and response time. On the other hand, the Self-Tuning CoDel algorithm based on our framework.

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 3

TABLE I the packet dropping probability functions of RED [10] and NOTATIONS CoDel [24] are in this general form. Thus, our model can be applied to a class of AQM algorithms that determine the packet dropping probability based on the difference between the queue length and a target value, such as CoDel and RED. On the other hand, we use loss-based TCP Reno as the source algorithm in our model since it is a fundamental and representative TCP algorithm. We note that CUBIC [25] is another popular loss-based TCP algorithm, which employs a different congestion window function. A new fluid model for CUBIC has recently been proposed in [26]. We make use of some preliminary results from [26] to extend our work for supporting CUBIC (see Section V-C). Here, we would like to point out that it would be too complicated if multiple TCP variants and multiple AQM algorithms are taken into consideration in the model simultaneously. Therefore, we fix on a fundamental TCP variant as the source algorithm and mainly focus on the analysis of AQM algorithms in this paper. Our general multi-bottleneck TCP/AQM system model that contains N groups of TCP flows and L bottleneck links is described by (1) and (2): 1 W (t)W (t − R (t)) ˙ i i i Wi(t)= − Ri(t) 2Ri(t − Ri(t)) + · Kl ql(t − Ri(t)) − βl (1) ⎧l∈L(i) N (t)W (t) ⎪ i i − C ,q(t) > 0 ⎪ l l ⎨ Ri(t) Section VI presents our simulation results. Finally, we con- q˙ (t)= i∈F (l) l ⎪ N (t)W (t) + (2) clude the paper with future research directions in Section VII. ⎪ i i − C ,q(t)=0 ⎩⎪ l l Ri(t) i∈F (l) II. SYSTEM MODEL where L(i) denotes the set of bottleneck links that Flow i In this section, we present a general multi-bottleneck traverses, F (l) is the set of groups of flows that traverse TCP/AQM system model. A list of frequently used notations Link l, i =1,...,N,andl =1,...,L (see Table I for other is shown in Table I. notations). In (1), there are N differential equations, each of which A. General Multi-Bottleneck TCP/AQM System Model describes the window size evolution of TCP flows in an i We consider a general TCP/AQM network with N groups individual group. In the subsequent analysis, we use Flow to i W (t) ≥ 0 of TCP flows and L bottleneck links. For simplicity, we do not denote a flow in Group . It should be noted that i , include non-bottleneck links in the model since they generally and that the summation term in (1) denotes the end-to-end i do not affect the flows passing through them. Some flows may packet dropping probability experienced by Flow . Here, have the same set of parameters and traversing path, and we we assume that dropping probabilities at the bottleneck links can categorize them into a group. are generally small, so that they can be added up to approxi- L A well-known fluid model for TCP interacting with mate the end-to-end dropping probability [12]. There are dif- AQM routers was proposed using stochastic differential ferential equations in (2), each of which models the dynamics q (t) ≥ 0 equations in [22]. The fluid model describes the additive of queue length at a bottleneck link. Note that, since l , q (t) q (t)=0 increase/multiplicative decrease (AIMD) behavior of TCP the rate of change of l cannot be negative when l . Reno [23] and the queue dynamics at a router buffer. In [10], B. Network Equilibrium the fluid model was extended to the general AIMD algorithm interacting with RED and multiple bottleneck links. Applying The equilibrium of the multi-bottleneck TCP/AQM system W˙ =0 q˙ =0 i =1,...,N the modeling technique similar to [10], we develop our multi- (1)–(2) is defined by i and l for l =1,...,L N (t) R (t) bottleneck TCP/AQM system model that uses loss-based TCP and . Assume that i and i are fixed on Reno as the source algorithm and employs a general packet their equilibrium values when the system is in equilibrium. N (t)=N R (t)=R i =1,...,N dropping probability function to capture the dynamics of That is, i i and i i for .The a class of AQM algorithms. The general packet dropping equilibrium point of the system is determined by the following equations: probability function p(t) has the following form: p(t)= min K[q(t) − β] +, 1 (a)+ =max{a, 0} p(t) + ,where , is (W ∗)2 · K (q∗ − β ) =2 the packet dropping probability at Time t, q(t) is the queue i l l l (3) length at Time t, β is the target queue length, and K>0 l∈L(i) + is a parameter determined by an AQM. The general form N W ∗ i i indicates that the packet dropping probability increases with − Cl =0 (4) Ri the deviation of the queue length from the target value. In fact, i∈F (l)

4 IEEE/ACM TRANSACTIONS ON NETWORKING

(W ∗,...,W∗ ,q∗,...,q∗ ) where 1 N 1 L represents an equilibrium the uniqueness of an equilibrium point. Properly performing point, i =1,...,N,andl =1,...,L. routing configurations may help satisfy this assumption. W ∗ > 0 W In (3), we have i , which indicates that the equi- Since (8) is linear in only, it is easy to study its solution p∗ = K (q∗ − first. In order to analyze the number of solutions for a system librium window size should be positive. Let l l l + of linear equations, we use the Rouché-Capelli Theorem [27] βl) be the equilibrium packet dropping probability at Link in linear algebra. l. Since we only consider bottleneck links in our model, First, we introduce a lemma, which affirms the existence of we can assume p∗ > 0 for all l and remove the notation ()+. at least one equilibrium point in the system (1)–(2). l N L We rewrite (3) as: Lemma 1: Given and , there exists at least one equi- 2 librium point in the system (1)–(2). p∗ = Proof: See Appendix A for a detailed proof. In addition, l (W ∗)2 (5) l∈L(i) i we notice that [28] presents a method to prove the existence of network equilibrium based on Nash equilibrium. It is also i =1,...,N where . ∗ ∗ feasible to adopt that method to prove Lemma 1. NiWi = C NiWi

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 5

Since W is uniquely determined by (9), E is then fixed. Thus, Define the Lagrangian of (11)–(12) as: P can be uniquely solved from (7) as follows: N L N −1 L(x, p)= N U (x ) − p S N x − C P = S E. (10) i i i l li i i l (15) i=1 l=1 i=1 (W, P) The equilibrium point is uniquely determined by (9) p =[p ,...,p ]T and (10) when N = L. where 1 L denotes the vector of Lagrange Next, we study the case of N>L. In this case, we find that multipliers associated with the equality constraint (12), (STA)=LL x∗ ≥ 0, S N x∗ = C l =1,...,L, system when , we need an additional assumption, li i i l for (16) which is described as follows. i=1 Assumption 2: The utility functions of TCP flows, denoted L ∗ ∗ by Ui(xi), are strictly concave increasing in their rates xi,and N U (x ) − N S p =0 i =1,...,N. i i i i li l for (17) twice continuously differentiable in their domains. l=1 Wi The flow rate xi is defined as xi = ,whereWi Ri Note that complementary slackness and dual feasibility vanish and Ri are the window size and round-trip time of Flow i, respectively. This is a mild assumption. In fact, most proposed from the KKT conditions since the problem does not have TCP algorithms, including TCP Reno, have strictly concave inequality constraints. Using (13), (17) can be rewritten as: increasing utility functions according to [12]. 2 1 L In [29], the determination of the optimal flow rates of ∗ = Slip for i =1,...,N. (18) a network is formulated as a network utility maximization R2 (x∗)2 l i i l=1 (NUM) problem. For the system (1)–(2), the corresponding

NUM problem is defined as: We observe that (16) and (18) coincide∗ with the equilibrium equations (5) and (6) (note that x∗ = Wi ). This means that an i Ri N equilibrium point (W, P) of the system is an optimal solution max NiUi(xi) (11) x≥0 of (11)–(12) and its dual problem, and vice versa. i=1 ∗ Since Ui(xi) are strictly concave, the optimal solution x N of (11)–(12) is unique. Thus, W is unique since W ∗ = x∗R . S N x = C i i i subject to li i i l (12) Next, we study (7). E is now uniquely fixed by W.Since i=1 rank(S)=L,theL × L matrix STS is invertible. From (7), T we can solve for P: where x =[x1,...,xN ] denotes the rate vector, Ni is the i S L × N T −1 T number of flows in Group , li denotes entries of the P =(S S) S E (19) matrix ST, i =1,...,N,andl =1,...,L. The equality constraint (12) corresponds to (6). The objec- which is unique. tive function (11) is a positive weighted sum of the utility Thus, we have proved that there exists a unique equilibrium functions and hence still strictly concave. Next, we prove point (W, P) in the system when N ≥ L. the uniqueness of an equilibrium point by showing that the equilibrium point of the system (1)–(2) is the unique solution IV. SYSTEM STABILITY of (11)–(12). In this section, we conduct a stability study for the system Theorem 2: Suppose that Assumptions 1 and 2 hold. Given (1)–(2). Since the system is nonlinear and has time-varying N L N ≥ L and such that , there exists a unique equilibrium delays, it is difficult to directly analyze its stability. Thus, point in the system (1)–(2). we follow a standard approach used in [10], where a nonlinear N = L Proof: For , we have shown that the equilibrium TCP/RED system was linearized about the equilibrium point point is unique since it is uniquely determined by (9) and (10). and Lyapunov stability theory was then adopted to study the N>L For , the existence of an equilibrium point is proved linearized system. However, we notice that the matrix P in by Lemma 1. Thus, we only need to prove the uniqueness ∗ the Lyapunov function constructed in [10] may not always of the equilibrium point in this case. Let x∗ = Wi be the A i Ri exist since the matrix in their model cannot be proved to be equilibrium rate of Flow i. always stable1 [30]. A similar case also appears in our system Based on (5), we define a utility function Ui(xi) for Flow i (1)–(2). This hinders the subsequent algorithm design and thus such that its derivative equals the right-hand side of (5) [12]: leads us to consider simpler subsystems. 2 1 We name (1)–(2) the global system. In order to facilitate U (x )= . distributed algorithm design, we decompose the global system i i R2 x2 (13) i i into single-bottleneck subsystems. For readability, here we Thus, we have mainly present the stability analysis of subsystems since it motivates our subsequent algorithm design. The stability study 2 1 U (x )=− . of the global system is presented in Appendix B. i i R2 x (14) i i 1A square matrix is called stable matrix if each of its eigenvalues has This utility function is strictly concave increasing in xi > 0. negative real part.

6 IEEE/ACM TRANSACTIONS ON NETWORKING

z (t)=[δW (t),...,δW (t),δq (t)]T When there exists a unique equilibrium point in the global Define l 1 Ml l . The lin- system (1)–(2), we can formulate a single-bottleneck subsys- earized subsystem (22)–(23) can be expressed in the following tem for every bottleneck Link l (l =1,...,L) as follows: form: Ml 1 Wi(t)Wi(t − Ri(t)) ∗ W˙ (t)= − · K (q − β ) z˙l(t)=Glzl(t)+ Hlizl(t − Ri). (26) i j j j Ri(t) 2Ri(t − Ri(t)) i=1 j∈L(i) j=l We now show that all Gl are stable matrices. + G + K q (t − R (t)) − β , i ∈ F (l) Lemma 2: All l defined in (24) are stable matrices. l l i l for Proof: See Appendix C for a detailed proof. G (20) According to Lyapunov stability theory [30], since l ⎧ defined in (24) is stable, there exists a unique positive ⎪ Ni(t)Wi(t) ⎪ − Cl,ql(t) > 0 definite matrix Pl that satisfies the following Lyapunov ⎨ Ri(t) i∈F (l) equation: q˙ (t)= + l ⎪ N (t)W (t) (21) ⎪ i i − C ,q(t)=0 GTP + P G = −I ⎩⎪ l l l l l l (27) Ri(t) i∈F (l) where I is an (Ml +1)× (Ml +1)identity matrix, for every q∗ j l =1,...,L where j denotes the equilibrium queue length at Link ,and . Wi(t) for every i ∈ F (l) and ql(t) for a fixed l are the We then study the stability of the subsystem by finding a variables in the subsystem of Link l. Lyapunov function for it. We choose the following Lyapunov Every subsystem (20)–(21) only considers the evolution of function candidate for every linearized subsystem (26): l q (t) one bottleneck Link and assumes that the queue lengths j V (z )=zTP z (28) at other links j = l that Flow i (i ∈ F (l)) traverses are fixed l l l l q∗ on their unique equilibria j . This is a natural approximation where the positive definite matrix Pl is determined by (27). for the global system. It is feasible because there is a unique The following theorem states the sufficient conditions for q∗ p∗ equilibrium point in the global system, and j (or j ) can be the local asymptotic stability of the subsystems. known to the subsystem, say, by Algorithm 1 in Section V-A. Theorem 3: A single-bottleneck subsystem (20)–(21) is Let Ml = F (l) . The subsystem (20)–(21) thus contains locally asymptotically stable if M l groups of flows and one bottleneck link. We linearize 1 (20)–(21) about the equilibrium point and obtain: 0 1 is a ∗ ∗ constant. where δWi(t)=Wi(t)−W for i ∈ F (l), δql(t)=ql(t)−q . i l Proof: We only need to show that V˙ (z ) < 0 if (29) is In deriving (22)–(23), we assume that Ni(t) and Ri(t) l satisfied. Note that there are delayed terms z (t − R ) in (26). are fixed on the equilibrium values, i.e., Ni(t)=Ni and l i We use the Razumikhin condition [31], which relates the past Ri(t)=Ri. For every subsystem, we locally renumber the Flows i ∈ trajectory and the present state of the system, to bound the F (l) so that i =1,...,M . Define the following matrices for delayed terms. l R =max{R ,...,R } every bottleneck Link l: Let max 1 Ml . Whenever the following Razumikhin condition: Gl1 0 2 Gl = (24) V (zl(τ)) ≤ μ V (zl(t)), for t − Rmax ≤ τ ≤ t Gl2 Gl3 μ>1 V (z ) 2 holds for some constant (note that l defined in where Gl1 = diag − ∗ is an Ml × Ml diagonal Wi Ri (28) is bounded), we have: NM G = N1 ,..., l 1 × M G = 2 T 2 2 matrix, l2 R1 RM is a l matrix, l3 λ (P )z (τ) ≤ z (τ)P z (τ) ≤ μ λ (P )z (t) . l min l l l l l max l l Ml ∗ 1 NiWi − 2 ,and0 is a zero matrix. Cl R Thus, we can obtain i=1 i zl(τ)≤μJzl(t) (30) 0Hli1 Hli = , for i =1,...,Ml (25) 00 where t − Rmax ≤ τ ≤ t. From the definition of Hli in (25), we have T H = (H ) ,...,(H ) ∗ 2 where li1 li1 1 li1 Ml with entries (W ) i ⎧ Hli = Kl. (31) ∗ 2 2R ⎨ (W ) i − i K , j = i (H ) = 2R l if From (26), (27), (28), (30), and (31), we can derive: li1 j ⎩ i 0, otherwise. ˙ V (zl) G H (M +1)× (M +1) = z˙ TP z + zTP z˙ Thus, l and li are l l square matrices. l l l l l l

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 7

Ml = zT(t) GTP +P G z (t)+2 zT(t − R )HTP z (t) Algorithm 1 Computation of the Equilibrium Point l l l l l l l i li l l S N ≥ L i=1 Assume: Full rank routing matrix is known and l (l =1,...,L) Ml 1: Every AQM router at a bottleneck Link ≤−zT(t)Iz (t)+2 zT(t − R )HTP z (t) p l l l i li l l measures its packet dropping probability l periodically. i=1 2: When there is a change in pl or update timer expires, Ml the corresponding router sends an update of pl to a cen- 2 2 ≤−zl(t) +2μJ PlHlizl(t) tralized designated router (DR), which maintains a list of i=1 all pl. Ml (W ∗)2 3: Upon receiving updates of pl, DR updates its list and then = − 1 − 2μJP i K z (t)2. l 2R l l (32) sends the updated list to all AQM routers. =1 i i 4: When the changes of all pl are within a small δ, the system V˙ (z ) < 0 z (t) =0 is assumed to be in equilibrium. With the list of all pl and If (29) is satisfied, then l when l .Thatis, ∗ ∗ 2 V (z ) S, every router can compute all W by W = È , l is a Lyapunov function for (26). Thus, the subsystem i i l∈L(i) pl described by (20) and (21) is locally asymptotically stable. and thus obtains the equilibrium point (W, P). Remark 1: Recall that Kl denotes an algorithm-specific parameter of an AQM employed by Link l. If the parameter value is set within the range specified by (29), the single- bottleneck subsystem is locally asymptotically stable. routers and M is the number of updates occurred before the system reaches the equilibrium. Next, we further give a discussion on the centralized prop- V. A LGORITHMS erty of Algorithm 1 and its possible decentralization: Based on our theoretical studies, here we present an algo- • Centrality for Simplicity: Algorithm 1 assumes that every rithm to compute the equilibrium point of the multi-bottleneck router can know the packet dropping probabilities at TCP/AQM system, and devise Self-Tuning CoDel algorithm all the bottleneck links, which is realizable by using to combat bufferbloat in multi-bottleneck networks. a centralized DR in a single autonomous system (AS) running OSPF protocol. Such a centrality is simple com- A. Algorithm for Computing the Equilibrium Point pared to a potential distributed algorithm that may require much more communication overheads among different According to Theorem 2, there exists a unique equilibrium N ≥ L N ≥ L routers. point in the system when . Note that • Possible Decentralization: For scalability (e.g., across should be quite common in real networks. We now propose multiple ASes), one can follow the OSPF’s mechanism Algorithm 1 to compute the equilibrium point of the system, S that uses an AS boundary router (ASBR) [32] to exchange assuming that the routing matrix is known to all AQM routing information with routers belonging to other ASes. routers. Specifically, we can let the DR router in an AS serve The essential idea of Algorithm 1 is that every AQM router as the ASBR which exchanges packet dropping proba- needs to know all the packet dropping probabilities at the bility information with ASBRs belonging to other ASes. bottleneck links in order to compute the window sizes of Every ASBR updates its corresponding list and sends the the TCP flows from (5). In fact, Algorithm 1 is motivated updated list to all routers in its own AS. Another method by the widely used Open Shortest Path First (OSPF) routing for decentralization is to apply dual decomposition [33] protocol [32]. In OSPF, there is a designated router (DR) for to the NUM problem (11)–(12). This involves an iterative every network segment, which is responsible for gathering and primal algorithm at each TCP source and an iterative dual forwarding link state updates. Thus, we can let a specific AQM algorithm at each link (router). Due to space limitations, router serve as a centralized DR in the system. For example, here we only briefly mention possible methods for decen- an AQM router with the smallest ID is selected as DR. All tralization. AQM routers periodically send packet dropping probability • Influence of Imprecise Estimation: Due to scalability updates to DR. When receiving updates, DR updates its list issues or delayed feedbacks, both the centralized and of packet dropping probabilities and sends the updated list to distributed versions of Algorithm 1 may use imprecise all AQM routers. In this way, every AQM router can compute information about the packet dropping probabilities to the equilibrium point of the system based on (5). Note that the compute the equilibrium point. Fortunately, the equi- routing matrix S can be easily known if all AQM routers use librium window size depends on the end-to-end packet the OSPF routing protocol since OSPF maintains the whole ( p )−1/2 dropping probability in the order of l∈L(i) l . network topology. In Section VI-F, we will show that the performance The computation of the equilibrium point is important since of our proposed Self-Tuning CoDel (Algorithm 2) is this information is usually required when properly tuning the not affected significantly given a reasonable estimation AQM parameters so as to stabilize the system. Note that the error. proposed Algorithm 1 is a general algorithm in the sense that it is not limited to a specific AQM algorithm employed by a bottleneck router. It provides a general method to compute the B. Self-Tuning CoDel Algorithm equilibrium point of the multi-bottleneck TCP/AQM system. The stability of CoDel in single-bottleneck networks has In fact, Algorithm 1 is used by our proposed Self-Tuning been analyzed in [24] and [34]. However, its stability in multi- CoDel algorithm when tuning the parameter, as illustrated in bottleneck networks is still unknown. Thus, here we consider a the next subsection. It is easy to see that the time complexity multi-bottleneck TCP/CoDel system, where all the bottleneck of Algorithm 1 is O(ML),whereL is the number of AQM links employ CoDel as the AQM algorithm.

8 IEEE/ACM TRANSACTIONS ON NETWORKING

In [24], an approximate packet dropping probability func- Algorithm 2 Self-Tuning CoDel tion of CoDel has been derived, and it can be expressed as: Input: Full rank routing matrix S + 1: while update timer expires or condition changes do q (t) − β ∗ l l 2: W N R i ∈ F (l) λ0 pl(t) ≈ min 2 , 1 (33) Estimate i , i, i for every Flow ,and . 4C I (λ0 − C ) l 0 l 3: Construct Gl defined in (24). 4: Solve the Lyapunov equation (27) for Pl based on Gl. where I0 denotes the interval parameter of CoDel, λ0 >C is l λmax(Pl) 5: Compute λmax(Pl) and λmin(Pl), J ← . the arrival rate at the time when the queue length first exceeds λmin (Pl) the target queue length βl,andCl is the link capacity. 6: Compute the 2-norm of Pl: Pl←λmax(Pl). 1 Ml ∗ 2 From (33), we can let K = 2 for CoDel. È (W ) l 4ClI (λ0−Cl) i 0 μJPl 2R i=1 i Substituting this into (29), we can derive: 7: I0 ← + μ>1 >0 2Cl(λ0−Cl) ,where and . 8: CoDel is running with the updated I0. Ml ( ∗)2 μJP Wi 9: end while l 2Ri i=1 I0 > . (34) 2Cl(λ0 − Cl) Algorithm 3 Modified Self-Tuning CoDel for CUBIC If the value of I0 is set such that (34) is satisfied, then every Assume: CUBIC sources can update their slow start thresh- single-bottleneck TCP/CoDel subsystem is locally asymptoti- olds based on the feedbacks from the AQM routers. cally stable according to Theorem 3. W ∗ i ∈ F (l) 1: Initially, estimate i for every Flow . ∗ Note that in order to trigger a packet drop, CoDel requires 2: Send the estimated W to corresponding Flow i,andFlowi a minimum elapsed time since the queue length exceeds β , i l updates its slow start threshold based on this value. which is denoted by I0. Thus, I0 also affects the response 3: Run the original Self-Tuning CoDel (Algorithm 2). time of CoDel. In practice, we can choose a value of I0 that is slightly larger than the lower bound in (34), so that system stability is achieved without sacrificing the response time. From (34), we observe that the range of I0 is determined by C. A Simple Modification of Self-Tuning CoDel for CUBIC several factors and will vary if these factors change over time. A static value of I0 cannot always guarantee the stability of Recall that our system model and stability analysis use TCP the TCP/CoDel system. Thus, we propose Self-Tuning CoDel Reno as the source algorithm. CUBIC [25] is another popular to dynamically adjust I0 at every bottleneck link based on loss-based TCP algorithm, which exhibits a different behavior network conditions, as shown in Algorithm 2. of congestion window evolution. Recently, a new fluid model When the update timer expires, Self-Tuning CoDel firstly for CUBIC has been proposed in [26]. We revisit [26] and estimates the related parameters. The equilibrium window size find that the CUBIC’s fluid model only considers a single- W ∗ i can be estimated by Algorithm 1. Here, it is worth noting bottleneck scenario, and that instead of directly modeling the that this is the only part where Self-Tuning CoDel needs evolution of CUBIC’s congestion window (cwnd), the model global information about the packet dropping probabilities derives the evolutions of the size of the cwnd immediately at related bottleneck links, in order to accurately estimate before loss and the amount of time elapsed since last loss. W ∗ i ∈ F (l) i ( ) for the flows traversing it. We then follow Thus, we cannot directly incorporate the CUBIC’s fluid model the methods proposed in the literature to estimate other net- into our multi-bottleneck TCP/AQM system by using a new work parameters locally. For example, efficient methods for congestion window function for CUBIC. More thorough the- estimating the number of TCP flows and link capacity were oretical analysis for CUBIC interacting with AQM routers in proposed in [14]. In [35], Karn’s algorithm was proposed to multi-bottleneck networks still needs to be conducted, and accurately estimate the round-trip time. To solve the Lyapunov hence we leave it as our future work. equation and compute the eigenvalues, we can call MATLAB However, [26] indeed provides some helpful insights for to perform these computations. Alternatively, there are many stabilizing a CUBIC system, which motivate us to develop a related numerical methods proposed in the literature, such as quick fix for Self-Tuning CoDel such that it can also work [36] for solving Lyapunov equations, and [37] for computing well with CUBIC. As the theoretical results in [26] suggest, eigenvalues. Finally, I0 is updated based on (34). Self-Tuning one can specify the initial slow start threshold of CUBIC to CoDel can be run locally at every bottleneck link, so as to be close to its equilibrium window size, so that the CUBIC stabilize the queueing delays at all the bottleneck links in a system is more likely to settle into its stable state. Motivated distributed manner. by this, we make a simple modification for Self-Tuning CoDel Regarding the time complexity of Self-Tuning CoDel, we do interacting with CUBIC, which is shown in Algorithm 3. not explicitly derive it since it depends on specific algorithms Initially, a router running the Modified Self-Tuning CoDel W ∗ used to solve the Lyapunov equation and compute the eigen- (Algorithm 3) estimates the equilibrium window size i for values. Instead, we point out that after decomposing the global every Flow i ∈ F (l) that traverses it. This can be done by multi-bottleneck system into single-bottleneck subsystems, using Algorithm 1. Note that the equilibrium window sizes of G P 3 the sizes of l and l are much smaller compared to that ∗ 4 cRi W = È the CUBIC flows can be estimated by i ( )3 , of Gg and Pg in the global system (see Appendix B), which b l∈L(i) pl helps facilitate related computations. In fact, we observe via where b =0.2 and c =0.4 according to [26]. The Line 4 of W ∗ extensive simulations that with relatively small sizes (i.e., Algorithm 1 needs to compute i for CUBIC flows using (M +1)× (M +1) G P W ∗ l l )of l and l, the Lyapunov equation this equation instead. The router then sends the estimated i and the eigenvalues in Algorithm 2 can be solved very quickly to the corresponding Flow i,andFlowi will update its slow W ∗ i W ∗ using MATLAB. start threshold based on i .IfFlow receives multiple i

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 9

Fig. 1. Topology for Example 1, where there are inﬁnitely many equilibria.

from different routers (since it traverses multiple bottleneck links), it will select the smallest one. After that, the router Fig. 2. Topology for Example 2, where there is a unique equilibrium. simply runs the original Self-Tuning CoDel. Note that the Lines 1–2 of Algorithm 3 only need to be follows: R1 =0.12 s, R2 =0.07 s, and R3 =0.08 s. Then, performed once during the initial stage. We also investigate the the equilibrium equations (5)–(6) are specified as: case where the source algorithm does not react to the feedback ⎧ 2 ∗ ⎪ ∗ ∗ of W from the router (i.e., the case of CUBIC working ⎪ p1 + p2 = ∗ i ⎪ (W )2 with the original Self-Tuning CoDel), which is presented in ⎪ 2 1 ⎪ p∗ = Section VI-G. ⎪ 1 ∗ 2 ⎨⎪ (W2 ) 2 VI. PERFORMANCE EVA L UAT I O N p∗ = (37) ⎪ 2 (W ∗)2 ⎪ 3 This section presents simulation results obtained from ns-2 ⎪ 50 5 ⎪ W ∗ + W ∗ = 2500 [38] and MATLAB, which evaluate our theoretical studies and ⎪ 0.12 1 0.07 2 proposed algorithms. We use ns-2 for packet-level simulations ⎪ ⎩⎪ 50 ∗ 5 ∗ because CoDel [7] and PIE [8] are also evaluated by it. W + W = 2500. 0.12 1 0.08 3 ∗ A. Numerical Study for the Equilibrium of Multi-Bottleneck We can uniquely solve (37) and obtain: W1 ≈ 4.89, ∗ ∗ ∗ ∗ TCP/AQM System W2 ≈ 6.49, W3 ≈ 7.42, p1 ≈ 0.047,andp2 ≈ 0.036. Hence, Example 2 shows that there is a unique equilibrium point in First, we conduct numerical study for the equilibrium of the multi-bottleneck TCP/AQM system if N ≥ L. the system by MATLAB. We use Example 1 to show that We now further show the convergence of the system to there can be infinitely many equilibrium points in the system, the equilibrium point, and verify the feasibility of decompos- thereby verifying Theorem 1. The topology for Example 1 is ing the global multi-bottleneck system into single-bottleneck shown in Fig. 1, where nodes denote routers. There are two subsystems. Here, the settings of Example 2 are used for groups of TCP flows (N =2), i.e., f1 and f2. Each group illustration. In fact, the system (1)–(2) is a system of delay contains ten flows, i.e., N1 = N2 =10. f1 traverses from differential equations (DDEs) [39]. We can use the dde23 Router 1 to Router 3 via Router 2, whereas f2 goes from solver [40] in MATLAB to solve DDEs with constant delays. Router 2 to Router 4 via Router 3. There are three bottleneck Specifically, assuming Ni(t)=Ni and Ri(t)=Ri for links (L =3) labeled by l1, l2,andl3. The link capacities are i =1,...,N in (1)–(2) and then using the settings of as follows: C1 = 1250 packets/s, C3 = 2500 packets/s, and Example 2, we can solve the corresponding global system C2 = C1 + C3 = 3750 packets/s. Let the round-trip times be (1)–(2) using dde23 and plot the solution evaluated in the R1 =0.08 sandR2 =0.05 s. Then, the equilibrium equation interval [0 s, 50 s] in Fig. 3(a). In order to be consistent (6) is specified as: ⎧ with the previous equilibrium analysis, we use the relationship 10 + ⎪ W ∗ = 1250 p(t)= K[q(t) − β] to replace the queue length by the ⎪ 1 ⎨⎪ 0.08 packet dropping probability in (1)–(2). In addition, we set 10 10 K = K = 1 W ∗ + W ∗ = 3750 1 2 5000 . Let the values of the TCP window ⎪ 1 2 (35) W W W ⎪ 0.08 0.05 sizes in the three groups be 1, 2,and 3, respectively. ⎪ 10 ⎩ W ∗ = 2500. The values of the packet dropping probabilities at the two 0.05 2 bottleneck links are then denoted by p1 and p2, respectively. ∗ ∗ It can be observed in Fig. 3(a) that as time passes, the values We can obtain W1 =10and W2 =12.5, which are unique. ∗ ∗ of W1, W2, W3, p1,andp2 converge to the equilibrium Substituting W and W into (5), we have: 1 2 point (4.89, 6.49, 7.42, 0.047, 0.036). Thus, we can see that ∗ ∗ p1 + p2 =0.02 the global system does converge to the unique equilibrium p∗ + p∗ =0.0128. (36) point computed previously in Example 2. 2 3 Based on the formulation of the subsystem given by It is obvious that (36) has infinitely many solutions. Thus, (20)–(21), we can further solve the corresponding subsystems Example 1 shows that there are infinitely many equilibrium using the dde23 solver. Specifically, for the subsystem at the points in the multi-bottleneck TCP/AQM system if N

10 IEEE/ACM TRANSACTIONS ON NETWORKING

Fig. 4. Packet-level simulation topology: a three-bottleneck network.

version of Reno which improves Reno’s fast recovery phase and still uses the same congestion window function described by (1). Thus, it fits our system model. NewReno is still the default TCP of some Windows and Linux operating systems according to [42] and [43]. CUBIC is more popular nowadays. We suggest a simple modification for Self-Tuning CoDel (Algorithm 3), which enables it to work well with CUBIC. Thus, we will present simulation results using NewReno and CUBIC in the next two subsections, respectively. Several Fig. 3. Stable and unstable solutions to the DDEs describing the global multi- representative AQM algorithms, including CoDel [7], PIE [8], bottleneck system and the single-bottleneck subsystems. (a) Stable global and Adaptive RED [13], are used to compare with Self-Tuning system. (b) Stable subsystem at l1. (c) Stable subsystem at l2.(d)Unstable global system. (e) Unstable subsystem at l1. (f) Unstable subsystem at l2. CoDel. Our simulation topology is shown in Fig. 4, where nodes represent AQM routers. This is a complicated three- bottleneck network, where there are three bottleneck links (l1, the feasibility of system decomposition in the sense that the l2,andl3) and five groups of TCP flows (f1, f2, f3, f4,and equilibrium points of the two single-bottleneck subsystems and f5) traversing in different directions. Specifically, f1 passes the equilibrium point of the global multi-bottleneck system through Routers 1–5. f2 goes from Router 6 to Router 8 via coincide with each other. Routers 2–3, whereas f3 moves from Router 7 to Router 11 via Routers 3–4. f4 traverses Router 9, Router 10, Router 4, and f B. Numerical Study for the Stability of Multi-Bottleneck then Router 5. 5 transits Routers 3–5 and then Router 12. TCP/AQM System In our simulations, the long-lived TCP flows are generated by FTP applications. There are also short-lived HTTP flows and In this subsection, we show numerical results obtained constant bit rate (CBR) flows going through Routers 1–5. Due from MATLAB to illustrate the stability and instability of the to space limitations, here we only present simulation results multi-bottleneck TCP/AQM system. In fact, Fig. 3(a)–(c) have using this three-bottleneck network topology. In fact, we have demonstrated the stability of a multi-bottleneck TCP/AQM conducted extensive simulations and also provided simulation system based on the settings of Example 2, where the values of results using the two-bottleneck topology of Fig. 2 in our K K 1 the two parameters 1 and 2 are set to 5000 . If we increase prior work [1]. Here, we use a more complicated topology the values of K1 and K2, the multi-bottleneck TCP/AQM to further evaluate the proposed Self-Tuning CoDel in terms system will become unstable, as shown in Fig. 3(d). Recall of the stability of queueing delays at the AQM routers and that Theorem 3 and Theorem 4 (in Appendix B) only give the the congestion window (cwnd) evolution of TCP flows. Note sufficient conditions for the local asymptotic stability of the that the stability of queueing delays and the cwnd of TCP subsystems and the global system, respectively. Thus, it may flows are two important metrics for TCP/AQM systems, since not be straightforward to determine the values of K1 and the former measures the performance of the AQM algorithms, K2 that will lead to the instability of the system. However, whereas the latter reflects the throughput of the TCP flows. we observe a trend that sufficiently large values of K1 and K2 The values of the main parameters used in our packet-level will eventually induce the global system and the subsystems simulations are summarized in Table II. The simulation time to become unstable. For example, Fig. 3(d) depicts an unstable is 600 seconds, and the TCP data packet size is 1000 bytes. K = K = 1 global system when 1 2 2700 .Ifwefurther The number of TCP flows varies over time. Specifically, from K K K = 1 increase the values of 1 and 2 such that 1 1600 and 0 sto200 s, N1 = 200, N2 =40, N3 =30, N4 =20, 1 K2 = 1800 , both subsystems become unstable, as illustrated and N5 =60.From200 sto400 s, N4 is increased in Fig. 3(e)–(f). Therefore, the MATLAB numerical results to 40.From400 sto600 s, N1 is then reduced to 140. shown in Fig. 3 indicate that in order to make the system and Note that the equilibrium round-trip time of Flow i can be ∗ K ql subsystems stable, the value of l should not be set too large. expressed as R = T + ,whereT denotes the i pi l∈L(i) Cl pi fixed round-trip propagation delay of Flow i. To significantly C. Packet-Level Simulation Settings differentiate the round-trip times of different flows, we set T =0.240 T =0.130 T =0.160 T =0.100 We now conduct packet-level simulations using ns-2 to eval- p1 s, p2 s, p3 s, p4 s, T =0.210 C =60 C =80 uate our proposed Self-Tuning CoDel. First, we describe our and p5 s. Let 1 Mbps, 2 Mbps, simulation settings as follows. Two representative loss-based and C3 = 100 Mbps when NewReno is used as the source TCP algorithms, namely NewReno [41] and CUBIC [25], are algorithm. When the more advanced CUBIC is employed, adopted as the source algorithms. NewReno is an enhanced we set the link capacities to larger values, as shown in Table II.

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 11

TABLE II PACKET-LEVEL SIMULATION SETTINGS

Fig. 5. Queueing delays at Link l1, when NewReno is the source algorithm. (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED.

To make a fair comparison, we set the target delays of Self- Tuning CoDel, CoDel, PIE, and Adaptive RED to 20 ms at all the three bottleneck links. Other AQM parameters are set to default values. To make a good tradeoff between stability and response time, we limit the tuning of I0 in Self-Tuning CoDel to the range [100 ms, 400 ms]. If the computed lower bound in (34) is less than the default value 100 ms [7], I0 can be set to 100 ms so as to align with CoDel. The update timer Fig. 6. Queueing delays at Link l2, when NewReno is the source algorithm. of Self-Tuning CoDel is set to 10 s. It can be reconﬁgured (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED. whenever needed.

D. Packet-Level Simulation Results Using NewReno In this subsection, we present simulation results using NewReno as the source algorithm, which demonstrate that our proposed Self-Tuning CoDel can effectively improve the system stability and performance. We compare Self-Tuning CoDel with CoDel, PIE, and Adaptive RED in terms of the stability of queueing delays at the bottleneck links and the cwnd of TCP flows. The simulation topology and the values of the main parameters are shown in Fig. 4 and Table II, respectively. In addition, we use PackMime-HTTP [38] to Fig. 7. Queueing delays at Link l3, when NewReno is the source algorithm. simulate short-lived HTTP traffic. There are two new HTTP (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED. connections generated per second and ten CBR flows with the rate of 64 Kbps traversing Routers 1–5. Fig. 7 depicts the queueing delays at Link l3. Due to the Fig. 5 plots the queueing delays yielded by the AQM arrivals of new f4 flows, there is a sudden jump in the queueing algorithms at Link l1, when NewReno is used. It can be seen delay around 200 s for the four AQM algorithms. Self-Tuning in Fig. 5(b) that CoDel exhibits highly fluctuating queueing CoDel and PIE can adapt to the change in the number of delay with some irregular spikes. We find that the default value TCP flows and stabilize the queueing delays, whereas CoDel of I0 in CoDel does not satisfy (34) in this case, such that exhibits irregular spikes in the queueing delay. Adaptive RED the stability of the subsystem at Link l1 is not guaranteed. even shows unstable queueing delay in the beginning. In contrast, Fig. 5(a) shows that our proposed Self-Tuning We now further present the cwnd of the TCP NewReno CoDel can consistently maintain stable queueing delay by flows that are regulated by the AQM algorithms. Due to properly adjusting I0. PIE and Adaptive RED can also stabilize space limitations, here we only show the cwnd evolution the queueing delay at Link l1, as shown in Fig. 5(c)–(d). of f1, f2, f3,andf4 flows (in fact, f5 exhibits stable and Queueing delays at Link l2 are presented in Fig. 6. It can similar evolution of cwnd for all the four AQM algorithms). be observed in Fig. 6(a)–(b) that CoDel also yields unstable Fig. 8 plots the evolution of the cwnd of a f1 flow named queueing delay at Link l2, whereas Self-Tuning CoDel still Flow f1 (note that all flows in the same group have similar stabilizes the queueing delay at Link l2. In addition, Adaptive characteristics for the cwnd). It can be seen in Fig. 8 that the RED produces frequently fluctuating queueing delay, as illus- four AQM algorithms yield similar evolution of cwnd for Flow trated in Fig. 6(d). PIE still maintains stable queueing delay f1.SinceFlowf1 traverses all the three bottleneck links, it has at Link l2, which can be ascribed to its internal auto-tuning a low value of cwnd.Thecwnd evolution of a f2 flow (i.e., mechanism (PIE has auto-tuning for its control parameters so Flow f2) is presented in Fig. 9. The four AQM algorithms as to make a tradeoff between stability and response time). also induce similar evolution of cwnd for Flow f2.

12 IEEE/ACM TRANSACTIONS ON NETWORKING

f Fig. 8. Congestion window (cwnd)ofNewRenoFlow 1. (a) Using Self- Fig. 12. Queueing delays at Link l1, when CUBIC is the source algorithm. Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED.

to the arrivals of new flows, the cwnd of Flow f4 is suddenly reduced around 200 s. As can be seen in Fig. 11(a), Self- Tuning CoDel can maintain the stability of the cwnd for Flow f4. In contrast, CoDel, PIE, and Adaptive RED induce notably larger fluctuations in the cwnd. From Fig. 5 to Fig. 11, we observe that Self-Tuning CoDel stabilizes the queueing delays at the bottleneck links without affecting the cwnd of the TCP flows. Indeed, the cwnd can also be stabilized.

Fig. 9. Congestion window (cwnd)ofNewRenoFlowf2. (a) Using Self- E. Packet-Level Simulation Results Using CUBIC Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. This subsection further presents packet-level simulation results using CUBIC as the source algorithm. The simulation topology and the values of the parameters are almost the same as those used in the previous subsection except the following three changes. First, CUBIC is used as the source algorithm instead of NewReno. Second, to accommodate the more aggressive CUBIC flows, the capacity of each bottleneck link is increased by 20 Mbps such that C1 =80Mbps, C2 = 100 Mbps, and C3 = 120 Mbps. Third, there are 10 new HTTP connections generated per second and 30 CBR flows with the rate of 64 Kbps traversing Routers 1–5 in the network. Note that a simple modification (Algorithm 3) has Fig. 10. Congestion window (cwnd)ofNewRenoFlowf3. (a) Using Self- been incorporated into Self-Tuning CoDel in this subsection. Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. Fig. 12 presents the queueing delays yielded by the AQM algorithms at Link l1, when CUBIC is used. It can be seen that Self-Tuning CoDel shows the smallest fluctuation in the queueing delay at Link l1 among the four AQM algorithms. Moreover, Self-Tuning CoDel also stabilizes the queueing delay at Link l2, whereas the other three algorithms (including PIE) exhibit highly fluctuating queueing delays at Link l2, as illustrated in Fig. 13. Fig. 14 shows that the four algorithms are able to stabilize the queueing delay at Link l3. We also investigate the cwnd of the CUBIC flows that are controlled by the AQM algorithms. Fig. 15 plots the cwnd evolution of a f1 flow (i.e., Flow f1). Recall that since Flow f1 traverses all the three bottleneck links, it maintains a low value of cwnd. The four AQM algorithms induce similar evolution Fig. 11. Congestion window (cwnd)ofNewRenoFlowf4. (a) Using Self- f Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. of cwnd for Flow 1. However, Self-Tuning CoDel yields a slightly higher cwnd for Flow f1 and induces it to go back to the slow start phase less frequently, compared to PIE. The Fig. 10 plots the evolution of the cwnd of a f3 flow (i.e., cwnd evolution of Flow f2 is presented in Fig. 16. It can be Flow f3). It can be observed that Flow f3 exhibits highly seen that Self-Tuning CoDel maintains a stable evolution of fluctuating cwnd when Adaptive RED is used (and it also cwnd for Flow f2 over time. In contrast, CoDel and PIE show frequently goes back to the slow start phase). In addition, some high fluctuations in the cwnd of Flow f2 and sometimes CoDel and PIE produce significantly larger fluctuations in the make the cwnd go back to the slow start phase. Fig. 17 presents cwnd of Flow f3, compared to Self-Tuning CoDel. The cwnd the cwnd evolution of Flow f3. We observe that Self-Tuning evolution of a f4 flow (i.e., Flow f4) is shown in Fig. 11. Due CoDel still yields a stable cwnd for Flow f3, whereas CoDel,

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 13

f2 Fig. 13. Queueing delays at Link l2, when CUBIC is the source algorithm. Fig. 16. Congestion window (cwnd)ofCUBICFlow . (a) Using Self- (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED. Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED.

f Fig. 14. Queueing delays at Link l3, when CUBIC is the source algorithm. Fig. 17. Congestion window (cwnd)ofCUBICFlow 3. (a) Using Self- (a) Self-Tuning CoDel. (b) CoDel. (c) PIE. (d) Adaptive RED. Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED.

Fig. 15. Congestion window (cwnd)ofCUBICFlowf1. (a) Using Self- Fig. 18. Congestion window (cwnd)ofCUBICFlowf4. (a) Using Self- Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. Tuning CoDel. (b) Using CoDel. (c) Using PIE. (d) Using Adaptive RED. PIE, and Adaptive RED exhibit highly fluctuating cwnd for information (due to scalability issues or delayed feedbacks) Flow f3 and also frequently force it to go back to the slow about the packet dropping probabilities at the bottleneck start phase. The cwnd evolution of Flow f4 is shown in Fig. 18. links to compute the equilibrium point, the performance of Self-Tuning CoDel also maintains a more stable cwnd for Self-Tuning CoDel is not affected significantly given a rea- Flow f4, compared to the other three algorithms. The cwnd sonable estimation error. evolution of Flow f5 is similar to that of Flow f1.Dueto We use the same simulation settings mentioned in space limitations, we omit it here. Section VI-D, except that we intentionally inject some “errors” The simulation results using CUBIC further demonstrate into the estimation of packet dropping probabilities such that Self-Tuning CoDel (with a simple modification) also that the computed end-to-end packet dropping probabilities works well with CUBIC and significantly outperforms PIE (i.e., p ) for all the five groups of NewReno flows in terms of the stability of queueing delay and cwnd. l∈L(i) l are underestimated by 20%. This estimation error case can correspond to the scenario where each AQM router at a Link l F. Influence of Imprecise Estimation of Packet Dropping uses smaller end-to-end packet dropping probabilities (e.g., Probability due to unavailable information from other links) to estimate W ∗ i ∈ F (l) In this subsection, we present one set of simulation results to the equilibrium window sizes i of the flows that show that Self-Tuning CoDel can tolerate reasonable estima- traverse Link l based on (5) or (38). tion error in the computation of packet dropping probability, Fig. 19 presents the queueing delay and cwnd produced by which corroborates the feasibility of Algorithms 1 and 2. Self-Tuning CoDel for the above error case. It can be seen that In other words, although the centralized version of Algo- the queueing delay at the bottleneck links and the cwnd of the rithm 1 or a distributed version of it may use imprecise flows can still be stabilized (Flow f2 and Flow f5 are not

14 IEEE/ACM TRANSACTIONS ON NETWORKING

Fig. 19. Queueing delay and cwnd produced by Self-Tuning CoDel, when the Fig. 20. Queueing delay and cwnd yielded by Self-Tuning CoDel, if the end-to-end packet dropping probabilities for all the five groups of NewReno CUBIC sources do not update their initial slow start thresholds. (a) Queueing 20% l1 flows are underestimated by . (a) Queueing delay at Link . (b) Queueing delay at Link l1. (b) Queueing delay at Link l2. (c) Queueing delay at Link l3. l2 l3 f1 delay at Link . (c) Queueing delay at Link .(d)cwnd of Flow .(e)cwnd (d) cwnd of Flow f1.(e)cwnd of Flow f3. (f) cwnd of Flow f4. of Flow f3. (f) cwnd of Flow f4. of solid stability analysis for CUBIC interacting with AQM shown due to space limitations). From the simulation results, 20% routers in multi-bottleneck networks, and hence we consider we observe that a reasonable (e.g., ) underestimate of filling this gap as one of our future research directions. end-to-end packet dropping probabilities only yields a slightly I larger computed lower bound of 0, which does not affect the VII. CONCLUSION stability (but may have a minor effect on the response time). In fact, we have also tried the scenario of 20% overestimate In this paper, we have proposed a general framework of end-to-end packet dropping probabilities (which should be for combating bufferbloat in multi-bottleneck networks. less common compared to the scenario of underestimate). The Specifically, we analyzed the equilibrium of a general results confirm that Self-Tuning CoDel can also tolerate this multi-bottleneck TCP/AQM system and provided the suffi- reasonable overestimation error (they are not shown here due cient conditions for the uniqueness of an equilibrium point to space limitations). in the system. We then conducted stability study for the multi-bottleneck system and the decomposed single-bottleneck subsystems by deriving the sufficient conditions for the local G. CUBIC Interacting With the Original Self-Tuning CoDel asymptotic stability of those systems. Based on the equilibrium In this subsection, we further investigate the case where the and stability studies, we proposed an algorithm for computing W ∗ the equilibrium point of the multi-bottleneck system. We fur- CUBIC sources do not react to the feedbacks of i from the routers in Algorithm 3, which is equivalent to the case of ther presented a case study to analyze the stability of the CUBIC interacting with the original Self-Tuning CoDel. multi-bottleneck TCP/CoDel system and proposed Self-Tuning We use the same simulation settings described in CoDel to improve the system stability and performance. Section VI-E, except that the CUBIC sources do not update By extensive numerical and packet-level simulations, we not their slow start thresholds when receiving the feedbacks from only verified our theoretical studies but also demonstrated that the routers. Fig. 20 plots the queueing delay and cwnd yielded Self-Tuning CoDel effectively stabilizes queueing delay and by Self-Tuning CoDel in this case. It can be observed that improves the cwnd of TCP flows in multi-bottleneck networks, compared to other representative AQM algorithms. the arrivals of new f4 flows around 200 s induce a sudden For future research, we plan to extend our framework jump in the queueing delay at Link l3 and this impact is by considering other popular TCP variants interacting with then propagated to Link l2 and Link l1. After that, it takes representative AQM algorithms in multi-bottleneck networks. much time to bring the queueing delay at Link l2 back to the target value. We also observe that there are some Moreover, we plan to further study the interactions among different AQM algorithms employed by different bottleneck occasional fluctuations in the cwnd of Flow f3, especially links in general networks containing wireless links. when Flow f3 seeks to increase cwnd right after 200 s. Therefore, the original Self-Tuning CoDel has performance degradation when working with CUBIC in multi-bottleneck APPENDIX A networks. In fact, the performances of PIE and CoDel are also PROOF OF LEMMA 1 p∗ > 0 l = degraded significantly if CUBIC is used in this case, as shown Proof: Recall that we have assumed l for all in Section VI-E (note that the auto-tuning of PIE is still based 1,...,Lin the derivation of (5) and (6), since we only consider on the TCP Reno’s fluid model, which can be found in [8]). bottleneck links in the system. To remedy this issue, we propose a quick fix for Self-Tuning From (5), we have: CoDel (i.e., Algorithm 3) based on the theoretical results of 2 [26]. The simulation results in Section VI-E have demonstrated W ∗ = i p∗ (38) the effectiveness of this quick fix. However, there is still a lack l∈L(i) l

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 15

i =1,...,N p∗ > 0 l =1,...,L 0 where . Given any l for all , and is a zero matrix. W ∗ i =1,...,N there exist i for all , which are determined by 0H H = gi1 , i =1,...,N (38). gi 00 for (43) Substituting (38) into (6) gives: where H is an N × L matrix with entries N 2 gi1 i = C ⎧ ∗ l (39) ⎨ (W ∗)2 Ri ∈ ( ) p − i K , j = i k ∈ L(i) i∈F (l) l L i l (H ) = 2R k if and gi1 jk ⎩ i where l =1,...,L. 0, otherwise. P We claim that for the set of all feasible vectors satisfying G H (N + L) × (N + L) (38) and thus determining W, there exists at least one P from Thus, g and gi are square matrices. z(t)=[δW (t),...,δW (t),δq (t),...,δq (t)]T this set, such that it also satisfies (39). To see why this claim Define 1 N 1 L . holds, suppose that there is no such vector P from the above The linearized system (40)–(41) can be expressed in the form: set, which can satisfy (39). That is, there is at least one link, N Ni 2 say Link j, which always has È ∗ i This means that Link cannot be fully utilized but with j 0. It is impossible and contradicts the fact that all the L links According to Lyapunov stability theory [30], there exists ∗ P are bottleneck links with p > 0 for all l =1,...,L.Note a positive definite matrix g that satisfies the following l G that, if Link j is not a bottleneck link, it can be eliminated Lyapunov equation if and only if g defined in (42) is a stable p∗ =0 matrix: with j without affecting the system. Therefore, we can conclude that there exists at least one P T G Pg + PgGg = −I (45) that satisfies (38) and (39). This P determines W by (38) and g hence this (W, P) is an equilibrium point of the system. where I is an (N + L) × (N + L) identity matrix. Note that (42) is much more complicated than (24) when L>1 G APPENDIX B , so that it is unclear if g is always a stable matrix. Since we only seek sufficient conditions for system stability, STABILITY OF THE GLOBAL MULTI-BOTTLENECK we assume that there exists a positive definite matrix P that TCP/AQM SYSTEM g satisfies (45), so as to conduct the subsequent analysis. In this section, we present stability analysis of the global We then choose the following Lyapunov function candidate multi-bottleneck system (1)–(2). Similar to the analysis of for the linearized system (44): subsystems, we first linearize (1)–(2) about the equilibrium T point and obtain: V (z)=z Pgz (46) 2 (W ∗)2 P δW˙ (t)=− δW (t)− i K δq (t−R ) where the positive definite matrix g is determined by (45). i ∗ i l l i (40) Theorem 4: A multi-bottleneck TCP/AQM system descri- W Ri 2Ri i l∈L(i) bed by (1) and (2) is locally asymptotically stable if there Ni exists a positive definite matrix Pg satisfying (45) and δq˙l(t)= δWi(t) Ri N i∈F (l) 1 − 2μJ P H > 0 (47) L 1 N W ∗ g g gi − i i δq (t) i=1 C R2 j (41) =1 j i (P ) j i∈F (l)∩F (j) J = λmax g λ (P ) λ (P ) where g λmin(P ) , max g and min g denote i =1,...,N l =1,...,L g where and . the largest and smallest eigenvalues of Pg, respectively, Pg δq (t) F (l) ∩ F (j) = ∅ In (41), a term j appears only when , denotes the 2-norm of Pg, Hgi is defined in (43), and μ>1 which means that there is at least one Flow i that traverses is a constant. both links l and j. Proof: Similar to the proof of Theorem 3, we use the Define the following matrices: Razumikhin condition [31] to bound the delayed terms and ˙ G 0 show that V (z) < 0 if (47) is satisfied. G = g1 g (42) Let Rmax =max{R1,...,RN }. Whenever the following Gg2 Gg3 Razumikhin condition: 2 where Gg1 = diag − ∗ is an N × N diagonal matrix, 2 Wi Ri V (z(τ)) ≤ μ V (z(t)), for t − Rmax ≤ τ ≤ t Gg2 is a L × N matrix with entries ⎧ holds for some constant μ>1, we can obtain ⎨ N j , if j ∈ F (i) (G ) = R z(τ)≤μJ z(t) (48) g2 ij ⎩ j g 0, otherwise, where t − Rmax ≤ τ ≤ t. Gg3 is a L × L square matrix with entries We then have: ⎧ 1 N W ∗ V˙ (z) ⎪ − k k , i = j ⎨⎪ 2 if T T Cj R = z˙ Pgz + z Pgz˙ (G ) = k∈F (i) k g3 ij ⎪ 1 N W ∗ N ⎪ − k k , otherwise, = zT(t) GTP +P G z(t)+2 zT(t − R )HTP z(t) ⎩ C R2 g g g g i gi g j k k∈F (i)∩F (j) i=1

16 IEEE/ACM TRANSACTIONS ON NETWORKING

N ≤−zT(t)Iz(t)+2 zT(t − R )HTP z(t) [9] D. Bauso, L. Giarre, and G. Neglia, “AQM stability in multiple bottle- i gi g neck networks,” in Proc. IEEE Int. Conf. Commun., vol. 4, Jun. 2004, i=1 pp. 2267–2271. N [10] L. Wang, L. Cai, X. Liu, X. S. Shen, and J. Zhang, “Stability analy- ≤−z(t)2 +2μJ P H z(t)2 sis of multiple-bottleneck networks,” Comput. Netw., vol. 53, no. 3, g g gi pp. 338–352, Feb. 2009. =1 i [11] F. Baker and G. Fairhurst, IETF Recommendations Regarding Active N Queue Management, document RFC 7567, IETF, Jul. 2015. 2 = − 1 − 2μJgPg Hgi z(t) . (49) [12] S. H. Low, “A duality model of TCP and queue management algorithms,” i=1 IEEE/ACM Trans. Netw., vol. 11, no. 4, pp. 525–536, Aug. 2003. [13] S. Floyd, R. Gummadi, and S. Shenker, “Adaptive RED: An algorithm If (47) is satisfied, then V˙ (z) < 0 when z(t) =0. Thus, for increasing the robustness of RED’s active queue management,” V (z) is a Lyapunov function for (44). Consequently, the multi- ACIRI, Tech. Rep., Aug. 2001. [14] H. Zhang, D. Towsley, C. V. Hollot, and V. Misra, “A self-tuning bottleneck system (1)–(2) is locally asymptotically stable. structure for adaptation in TCP/AQM networks,” ACM SIGMETRICS Remark 2: In Theorem 4, the existence of a positive definite Perform. Eval. Rev., vol. 31, no. 1, pp. 302–303, Jun. 2003. matrix Pg satisfying (45) becomes one of the conditions since [15] N. Xiong et al., “A novel self-tuning feedback controller for active Gg cannot be proved to be always stable. Besides, we can queue management supporting TCP flows,” Inf. Sci., vol. 180, no. 11, pp. 2249–2263, Jun. 2010. see that all the AQM parameters Kl are coupled in (47) via H , such that it is difficult to analyze them separately. These [16] Q. Chen and O. W. W. Yang, “On designing self-tuning controllers for gi AQM routers supporting TCP flows based on pole placement,” IEEE J. issues generally hinder the design of distributed algorithms. Sel. Areas Commun., vol. 22, no. 10, pp. 1965–1974, Dec. 2004. [17] S. H. Low, F. Paganini, J. Wang, S. Adlakha, and J. C. Doyle, “Dynamics APPENDIX C of TCP/RED and a scalable control,” in Proc. 21st Annu. Joint Conf. IEEE Comput. Commun. Societies (INFOCOM), vol. 1, Jun. 2002, PROOF OF LEMMA 2 pp. 239–248. Proof: Let λ be the eigenvalues of Gl,and Gl −λI =0, [18] L. Tan, W. Zhang, G. Peng, and G. Chen, “Stability of TCP/RED systems in AQM routers,” IEEE Trans. Autom. Control, vol. 51, no. 8, where I is an (Ml +1)× (Ml +1) identity matrix. Expand G − λI pp. 1393–1398, Aug. 2006. l along the last column using the Laplace expansion [19] D. Wischik and N. McKeown, “Part I: Buffer sizes for core routers,” as follows: ACM SIGCOMM Comput. Commun. Rev., vol. 35, no. 3, pp. 75–78, Jul. 2005. 1 Ml N W ∗ G − λI = − i i − λ [20] G. Raina, D. Towsley, and D. Wischik, “Part II: Control theory for l 2 buffer sizing,” ACM SIGCOMM Comput. Commun. Rev., vol. 35, no. 3, Cl R i=1 i pp. 79–82, Jul. 2005. 2 · − − λ =0. [21] M. Enachescu, Y. Ganjali, A. Goel, N. McKeown, and T. Roughgarden, diag W ∗R “Part III: Routers with very small buffers,” ACM SIGCOMM Comput. i i Commun. Rev., vol. 35, no. 3, pp. 83–90, Jul. 2005. λ = − 2 i =1,...,M λ = [22] V. Misra, W.-B. Gong, and D. Towsley, “Fluid-based analysis of a Thus, we have i ∗ for l,and Ml+1 Wi Ri network of AQM routers supporting TCP flows with an application Ml ∗ to RED,” ACM SIGCOMM Comput. Commun. Rev., vol. 30, no. 4, 1 NiWi − 2 . Since all the eigenvalues of G have negative Cl R l pp. 151–160, Oct. 2000. i=1 i G [23] V. Jacobson, “Congestion avoidance and control,” ACM SIGCOMM real parts, all l are stable matrices. Comput. Commun. Rev., vol. 18, no. 4, pp. 314–329, Aug. 1988. [24] J. Ye and K.-C. Leung, “Adaptive and stable delay control for combating ACKNOWLEDGMENT bufferbloat: Theory and algorithms,” IEEE Syst. J., vol. 14, no. 1, pp. 1285–1296, Mar. 2020. The authors would like to thank the anonymous reviewers [25] S. Ha, I. Rhee, and L. Xu, “CUBIC: A new TCP-friendly high-speed and the associate editor for their useful comments that have TCP variant,” ACM SIGOPS Operating Syst. Rev., vol. 42, no. 5, helped improve this article. pp. 64–74, Jul. 2008. [26] G. Vardoyan, C. V. Hollot, and D. Towsley, “Towards stability analysis of data transport mechanisms: A fluid model and an application,” in Proc. REFERENCES IEEE Conf. Comput. Commun. (INFOCOM), Apr. 2018, pp. 666–674. [27] I. R. Shafarevich and A. O. Remizov, Linear Algebra and Geometry. [1] J. Ye, K.-C. Leung, V. O. K. Li, and S. H. Low, “Combating bufferbloat Berlin, Germany: Springer-Verlag, 2013. in multi-bottleneck networks: Equilibrium, stability, and algorithms,” [28] A. Tang, J. Wang, S. H. Low, and M. Chiang, “Equilibrium of heteroge- in Proc. IEEE Conf. Comput. Commun. (INFOCOM), Apr. 2018, neous congestion control: Existence and uniqueness,” IEEE/ACM Trans. pp. 648–656. Netw., vol. 15, no. 4, pp. 824–837, Aug. 2007. [2] J. Gettys and K. Nichols, “Bufferbloat: Dark buffers in the Internet,” [29] F. P. Kelly, A. K. Maulloo, and D. K. H. Tan, “Rate control for Commun. ACM, vol. 55, no. 1, pp. 57–65, Jan. 2012. communication networks: Shadow prices, proportional fairness and [3] S. Floyd and V. Jacobson, “Random early detection gateways for con- stability,” J. Oper. Res. Soc., vol. 49, no. 3, pp. 237–252, Apr. 1998. gestion avoidance,” IEEE/ACM Trans. Netw., vol. 1, no. 4, pp. 397–413, Aug. 1993. [30] H. K. Khalil, Nonlinear Systems, 3rd ed. Upper Saddle River, NJ, USA: [4] C. V. Hollot, V. Misra, D. Towsley, and W. Gong, “Analysis and design Prentice-Hall, 2002. of controllers for AQM routers supporting TCP flows,” IEEE Trans. [31] K. Gu, V. L. Kharitonov, and J. Chen, Stability of Time-Delay Systems. Autom. Control, vol. 47, no. 6, pp. 945–959, Jun. 2002. Boston, MA, USA: Birkhäuser, 2003. [5] S. Athuraliya, S. H. Low, V. H. Li, and Q. Yin, “REM: Active queue [32] J. Moy, OSPF Version 2, document IETF RFC 2328, Apr. 1998. management,” IEEE Netw., vol. 15, no. 3, pp. 48–53, May 2001. [33] M. Chiang, S. H. Low, A. R. Calderbank, and J. C. Doyle, “Layering [6] S. S. Kunniyur and R. Srikant, “An adaptive virtual queue (AVQ) as optimization decomposition: A mathematical theory of network algorithm for active queue management,” IEEE/ACM Trans. Netw., architectures,” Proc. IEEE, vol. 95, no. 1, pp. 255–312, Jan. 2007. vol. 12, no. 2, pp. 286–299, Apr. 2004. [34] C. Lai, S. H. Low, K.-C. Leung, and V. O. K. Li, “Pricing link by time,” [7] K. Nichols and V. Jacobson, “Controlling queue delay,” ACM Queue, ACM SIGMETRICS Perform. Eval. Rev., vol. 42, no. 1, pp. 421–433, vol. 10, no. 5, pp. 1–15, May 2012. Jun. 2014. [8] R. Pan et al., “PIE: A lightweight control scheme to address the [35] P. Karn and C. Partridge, “Improving round-trip time estimates in bufferbloat problem,” in Proc. IEEE 14th Int. Conf. High Perform. reliable transport protocols,” ACM SIGCOMM Comput. Commun. Rev., Switching Routing (HPSR), Jul. 2013, pp. 148–155. vol. 25, no. 1, pp. 66–74, Jan. 1995.

YE et al.: COMBATING BUFFERBLOAT IN MULTI-BOTTLENECK NETWORKS: THEORY AND ALGORITHMS 17

[36] D. Kressner, “Block variants of Hammarling’s method for solving Ka-Cheong Leung (Senior Member, IEEE) received Lyapunov equations,” ACM Trans. Math. Softw., vol. 34, no. 1, pp. 1–15, the B.Eng. degree in computer science from The Jan. 2008. Hong Kong University of Science and Technol- [37] G. H. Golub and H. A. van der Vorst, “Eigenvalue computation in the ogy, Hong Kong, in 1994, and the M.Sc. degree 20th century,” J. Comput. Appl. Math., vol. 123, nos. 1–2, pp. 35–65, in electrical engineering (computer networks) and Nov. 2000. the Ph.D. degree in computer engineering from the [38] K. Fall and K. Varadhan, “The ns manual (formerly ns notes and University of Southern California, Los Angeles, CA, documentation),” The VINT Project, Nov. 2011. USA, in 1997 and 2000, respectively. He was with [39] R. D. Driver, Ordinary and Delay Differential Equations.NewYork, the Nokia Research Center, Nokia Inc., Irving, TX, NY, USA: Springer-Verlag, 1977. USA, from 2001 to 2002, Texas Tech University, [40] L. F. Shampine and S. Thompson, “Solving DDEs in MATLAB,” Appl. Lubbock, TX, from 2002 to 2005, and The Uni- Numer. Math., vol. 37, no. 4, pp. 441–458, Jun. 2001. versity of Hong Kong, Hong Kong, from 2005 to 2019. He is currently an [41] T. Henderson, S. Floyd, A. Gurtov, and Y. Nishida, The NewReno Associate Professor with the School of Computer Science and Technology, Modiﬁcation to TCP’s Fast Recovery Algorithm, document RFC 6582, Harbin Institute of Technology, Shenzhen, China. His research interests IETF, Apr. 2012. include future Internet, smart city, vehicle-to-grid (V2G), machine learning, [42] P. Yang, J. Shao, W. Luo, L. Xu, J. Deogun, and Y. Lu, “TCP congestion and wireless communications. He has co-guest edited a Special Issue of avoidance algorithm identiﬁcation,” IEEE/ACM Trans. Netw., vol. 22, the IEEE TRANSACTIONS ON NETWORK SCIENCE AND ENGINEERING. no. 4, pp. 1311–1324, Aug. 2014. He is a Subarea-Chair of the IEEE SIG on Intelligent Internet Edge (IIE). [43] The FreeBSD Project. FreeBSD Manual Pages. Accessed: 2021. He is an Associate Editor of the IEEE TRANSACTIONS ON INTELLIGENT [Online]. Available: https://www.freebsd.org/cgi/man.cgi?query=tcp TRANSPORTATION SYSTEMS, IEEE SYSTEMS JOURNAL,andACM/Springer Wireless Networks. Jiancheng Ye (Member, IEEE) received the B.E. degree in network engineering from Sun Yat-sen University, Guangzhou, China, in 2008, the M.Phil. Steven H. Low (Fellow, IEEE) received the B.S. degree in computer science and engineering from degree in electrical engineering from Cornell and the The Hong Kong University of Science and Tech- Ph.D. degree in electrical engineering from Berkeley. nology, Hong Kong, in 2011, and the Ph.D. degree He was with AT&T Bell Laboratories, Murray Hill, in computer networking from The University of NJ, USA, and the University of Melbourne, Aus- Hong Kong, Hong Kong, in 2018. He was a Software tralia. He has held honorary/chaired professorship Engineer with Harmonic Inc., from 2011 to 2014, in Australia, China, and Taiwan. He is currently and a Post-Doctoral Fellow with the Department of the F. J. Gilloon Professor with the Department Electrical and Electronic Engineering, The Univer- of Computing and Mathematical Sciences and the sity of Hong Kong. His research interests include network congestion control, Department of Electrical Engineering at Caltech. He queue management, optimization and protocol design of computer networks, is a Fellow of the ACM. He was a co-recipient of and machine learning. the IEEE Best Paper Awards in networking and power systems.

Authorized licensed use limited to: CALIFORNIA INSTITUTE OF TECHNOLOGY. Downloaded on April 09,2021 at 21:01:56 UTC from IEEE Xplore. Restrictions apply.