“Congestion Control for Transmission Control Protocol (TCP) in Wireless Networks”

Abstract of thesis entitled “Congestion Control for Transmission Control Protocol (TCP) in Wireless Networks”

Submitted by Chengdi LAI for the degree of Master of Philosophy at The University of Hong Kong in August 2011

Congestion control regulates the amount of data traffic injected by end systems into communication networks, preventing persistent network overload. In the Internet, it is typically realized by Transmission Control Protocol (TCP), which backs off transmission of end systems upon detecting packet loss, with the assistance of active queue management (AQM) algorithms, which probabilistically drop packets at the intermediate routers based on the buffer occupancy. The thesis adapts the present Internet congestion control to the wireless environment that is gradually becoming an indispensable component of the Internet. Firstly, we propose a novel TCP variant, known as TCP for non-congestive loss (TCP-NCL), to perform effective congestion control, sequencing control, and loss recovery over wireless networks where reordering, error-prone channels are prevalent. Conventional TCP tends to misinterpret packet reordering and wireless loss as congestive loss, and back off unnecessarily. We develop a smart TCP sender (STS) model that offers more reliable signals of packet loss and network congestion over reordering, error-prone channels. TCP-NCL is devised as a practical approximation of STS. Our simulation results show that TCP-NCL can accurately differentiate among the occurrences of packet reordering, congestive loss, and non-congestive loss, and can thus serve as the aforementioned unified solution. Deployment of TCP-NCL requires sender-side modification only.

Secondly, we have performed a series of simulation experiments to investigate the performance of AQM under wireless losses. Our results report that random early detection (RED), a classical AQM, fails to maintain a stable backlog against time-varying wireless packet error rate (WPER), resulting in poor delay performance and unfairness towards bursty traffic. We apply the internal model principle to devise a family of AQM enhancements, which can automatically adjust packet dropping rate of AQM to compensate for wireless losses, and thus stabilize the backlog. We further devise the integral controller (IC) as an embodiment of the principle. Our simulation studies show that IC is robust against time-varying WPER, even in challenging scenarios like fat pipes and perturbing traffic flows. In summary, our proposed TCP and AQM enhancements can perform effective congestion control over the heterogeneous wired/wireless Internet, and help the latter to operate in the optimal region of low delay with small jitters and high throughput.

(372 words) Congestion Control for Transmission Control Protocol (TCP) in Wireless Networks

Chengdi LAI

B.Eng. (EComE) H.K.

A Thesis Submitted in Partial Fulﬁlment of the Requirements

for the Degree of Master of Philosophy

at The University of Hong Kong

August 2011 Declaration

I declare that the thesis and the research work thereof represents my own work, except where due acknowledgement is made, and that it has not been previously included in a thesis, dissertation or report submitted to this Uni- versity or to any other institution for a degree, diploma or other qualiﬁca- tions.

Signed ...... Chengdi LAI

i Acknowledgements

The thesis marks a ﬁrst milestone for my research journey, which started three years ago when my current advisors, Prof. Victor On-Kwok Li and Dr. Ka-Cheong Leung, introduced to me the prosperous realm of networking research. Over the years, their pursuit of excellence, encouragement of inno- vation, and emphasis on rigorousness have greatly inﬂuenced me. Prof. Li’s great vision and innovative insights have kept me inspired in pursuit of research work of profound impact. Dr. Leung has patiently led me to formulate my research methodology and taste via the numerous email exchanges and conversations. I would like to express my earnest gratitude to them.

I owe my special thanks to Prof. Steven H. Low and Prof. Ricky Y.K. Kwok for serving as my thesis examiners. Their constructive comments assist me to improve the quality of this thesis. I thank Dr. K.P. Chan for chairing my thesis committee.

I have been very fortunate to encounter several wonderful mentors during the past ﬁve years in the University of Hong Kong, spanning my B.Eng. and M.Phil. programs. I appreciate Prof. Kwok, Prof. Lawrence K. Yeung, and Dr. Vincent Tam for their insightful advice on career and life. I would also like to take the opportunity to thank Fr. John Coghlan for providing me three

ii years’ enjoyable residence in Ricci Hall during my undergraduate study. The high level of professionalism of the support staﬀ in the Department of Electrical and Electronic Engineering, the University of Hong Kong, has greatly facilitated my research work. In particular, I would like to thank Ms. Danita Lee, Ms. Julie Hung, and Ms. Lily Lo for their assistance on ad- ministrative issues, and Mr. C.L. Chan and Mr. Andy Fok for their technical support. I am obligated to my colleagues, Dr. Albert Y.S. Lam, Dr. Guanghua Yang,

Dr. Yanhui Geng, Mr. Ze Zhao, Mr. Yile Yang, Mr. Jin Xu, Mr. Xiaoguan Fang, Dr. Qiong Sun, Dr. Jun Hong, Dr. Kuang Xu, and Dr. Jialing Xu, for their support and assistance. Last, but not least, I am deeply indebted to my parents and my girlfriend,

Miss Shuzhen Shen, for their love, and for always being there for me through thick and thin.

iii Contents

Declaration i

Acknowledgements ii

Contents iv

List of Figures vi

List of Tables viii

1 Introduction 1 1.1 ChapterOverview...... 1 1.2 InternetCongestionControl ...... 2 1.2.1 TCP...... 3 1.2.2 AQM...... 8 1.3 WirelessChannels...... 10 1.4 Thesis Statement and Outline of Thesis ...... 12 1.5 ConcludingRemarks ...... 14

2 Related Work 16 2.1 ChapterOverview...... 16 2.2 Wireless TCP Enhancements ...... 16 2.3 DesignandAnalysisofAQM ...... 19 2.4 ConcludingRemarks ...... 21

3 A Serialized-Timer Approach for Enhancing Wireless TCP 23 3.1 ChapterOverview...... 23 3.2 STSModel ...... 25 3.2.1 RDTimer ...... 31 3.2.2 CDTimer ...... 32 3.3 TCP-NCL...... 43 3.3.1 NRUProcess ...... 43

iv 3.3.2 RDandCDTimers...... 46 3.3.3 RTOTimer ...... 50 3.3.4 KernelImplementation ...... 51 3.4 PerformanceEvaluation ...... 54 3.4.1 Non-congestiveloss ...... 56 3.4.2 PacketReordering ...... 61 3.4.3 CongestiveLoss...... 63 3.5 ConcludingRemarks ...... 68

4 Enhancing AQM to Combat Wireless Losses 69 4.1 ChapterOverview...... 69 4.2 AQMunderWirelessLosses ...... 72 4.3 SystemModel...... 77 4.4 Robust AQM based on Internal Model Principle ...... 82 4.4.1 IC ...... 88 4.5 PerformanceEvaluation ...... 94 4.5.1 One Wireless Bottleneck Link ...... 94 4.5.2 Two Wireless Bottleneck Links ...... 103 4.6 ConcludingRemarks ...... 105

5 Conclusions and Future Research 107 5.1 ChapterOverview...... 107 5.2 Contributions ...... 107 5.3 FutureResearch...... 111 5.3.1 A Theoretical Framework for Wireless TCP ...... 111 5.3.2 A Uniﬁed AQM Solution for Wireless Networks . . . . 113 5.4 ConcludingRemarks ...... 115

Bibliography 116

Acronyms 125

v List of Figures

1.1 Fastretransmitandfastrecovery...... 5

3.1 AﬂowchartforSTSmodel...... 26 ∗ 3.2 Evaluation of τCDi ...... 41 3.3 SoftwarearchitectureforTCP...... 52 3.4 The network topologies used for performance comparison ... 53 3.5 Goodput performance against packet error rate over infrastructure- basedwirelessnetwork...... 55 3.6 Goodput performance against delay of the wireless link over infrastructure-based wireless network...... 57 3.7 Goodput performance against bandwidth of the wireless link over infrastructure-based wireless network...... 58 3.8 Goodput performance over multihop ad-hoc wireless network . 62 3.9 Jain’s fairness index (J) over wired network with a dumbbell topology...... 64 3.10 GBR over wired network with a dumbbell topology...... 65

4.1 Awirelessbottlenecklink...... 72 4.2 Queue length dynamics under varying wireless losses in the dumbbell topology with TCP NewReno/RED...... 74 4.3 Queue length dynamics under varying wireless losses in the dumbbell topology with TCP-NCL/RED...... 75 4.4 Control block diagram of the linearized congestion control system...... 78 4.5 Two wireless bottleneck links...... 92 4.6 Comparison of queue length dynamics with same configura- tionsasinFigure4.2...... 97 4.7 Comparison of queue length dynamics under different wireless losses...... 99 4.8 Comparison of queue length dynamics with different bottle- necklinkparameters...... 100

vi 4.9 Comparison of queue length dynamics with the presence of othertraﬃc...... 102 4.10 Comparison of queue length dynamics in the topology of two wireless bottleneck links...... 104

vii List of Tables

3.1 Generalnotations...... 29 3.2 Notationsforevents...... 30 3.3 Network conﬁgurations for the infrastructure-based wireless network ...... 59

4.1 Network conﬁgurations for the single wireless bottleneck network...... 96

viii Chapter 1

Introduction

1.1 Chapter Overview

The thesis adapts the present Internet congestion control to the wireless environment that is gradually becoming an indispensable component of the Internet. In this chapter, we bring the background materials on Internet congestion control and the challenges it encounters in the wireless environment. On this basis, we present the thesis statement and outline the thesis. Section 1.2 succinctly describes the present Internet congestion control, as fulﬁlled by Transmission Control Protocol (TCP) with the assistance of active queue management (AQM) algorithms. Section 1.3 discusses the challenges posed by the wireless environment on the proper functioning of congestion control. Section 1.4 presents our thesis statement, and outlines the thesis. Section 1.5 brieﬂy summarizes the chapter.

1 1.2 Internet Congestion Control

The modern age has witnessed the great success of communication networks in vastly facilitating information exchange and resource sharing among ge- ographically distributed people. Packet-switched networks are among the most common communication networks. In a packet-switched network, end systems are inter-connected via a group of packet switches such as routers and network hubs. An end system groups its transmitted data into separate segments, or packets, before delivering it over the network. Upon receiving a packet from end systems or other packet switches, a packet switch places the packet in its backlog, a queue of packets waiting for forwarding. It for- wards the packet to the next switch towards the destination or directly to the destination when the outgoing link becomes free for transmission. The Internet is a global system of packet-switched network. Using the standardized Transmission Control Protocol/Internet Protocol (TCP/IP) protocol suite, the Internet inter-connects numerous heterogeneous networks and is the largest network of networks. Network congestion or network overload occurs when the amount of data traﬃc injected into the network by end systems exceeds its transmission capacity. At packet switches, packets persistently arrive faster than can be forwarded. The backlog grows in length, elongating the time packets spent in queueing. This queueing delay adds to the end-to-end delay experienced by packets. When the backlog further grows, it can exceed the buﬀer size of the packet switch, thereby discarding arrived and/or backlogged packets. Under persistent network overload, the high levels of transmission delays and losses

2 experienced by packets result in poor data delivery services to end systems and hence users. Historically, severe network congestion, known as congestion collapse [56], occurred to the Internet in the 1980s [30]. Due to poorly regulated transmission of end systems, packets were held in long backlogs at packet switches. End systems deemed these packets as being lost and retransmitted them, exacerbating network congestion and wasting network capacity for transmit- ting duplicate data. Consequently, the network observed up to three orders of magnitude decrease in the rate of successful data delivery. Moreover, the end-to-end delay became excessively long. Congestion control regulates the transmission rate of end systems, preventing persistent network overload. It is therefore crucial to ensuring the proper operation of the network. In the present Internet, the task of congestion control is implemented by TCP running at end systems with the assistance of AQM running at routers in a distributed manner. In the following two subsections, we review TCP and AQM, respectively.

1.2.1 TCP

Transmission Control Protocol (TCP) [4, 30, 60] is the de facto standard for the Internet. Using the connection-less, unreliable data delivery services of Internet Protocol (IP), it provides end-to-end, in-order, reliable data delivery services to various applications like File Transfer Protocol (FTP), Telnet, and Hypertext Transfer Protocol (HTTP). Initially, TCP did not incorporate end-to-end congestion control [60]. The latter was ﬁrst installed in TCP

3 around 1988 in an successful attempt to combat the occurrence of congestion collapse, and has since become an essential element of TCP, central to ensuring the stable, eﬃcient operation of the present Internet.

TCP is connection-oriented. A TCP connection is set up between two end systems when they agree on a few parameter settings and each reserve resources like memory space for the transmission via TCP, after which the transmission begins. The transmission ends when the connection is closed and the resources reserved by end systems will be released.

The TCP connection is duplex. Both end systems can transmit and receive data. Nevertheless, we can conceptually focus on one-way transmission over a TCP connection to avoid unnecessary complications. Moreover, in reality the bulk of data transfer over a TCP connection takes place in one way, say, from a web server to a web client. Correspondingly, the end systems that transmits and receives data are referred to as TCP sender and TCP receiver, respectively. A TCP sender consecutively assigns byte numbers to its transmitted data bytes, and a TCP receiver expects the number of received data bytes to be consecutively ordered. When receiving a data packet, a TCP receiver noti- ﬁes the sender of successfully delivered data via an acknowledgment (ACK) packet that contains acknowledgment number, namely the number of next byte it expects. The time elapsed between when a TCP sender transmits a data packet and when the packet is acknowledged is known as round-trip time (RTT). An example is illustrated in Figure 1.1. The right and left arrows denote data packets and ACK, respectively. The ﬁrst data packet contains bytes numbered 201 to 300, as indicated by the two numbers above its cor-

4 201:300

301:400

401:500 ACK 301 501:600

601:700 ACK 301 Sender Receiver

ACK 301

301:400

Figure 1.1: Fast retransmit and fast recovery. responding arrow. Its arrival at the receiver is acknowledged to the sender via the first ACK containing acknowledgment number 301. These in-order data bytes will be delivered to the applications after some extra processing. The next packet arrived contains bytes numbered 401 to 500 and is thus out of order. In this case, the receiver immediately sends back an ACK with acknowledgment number 301. This duplicate acknowledgement contains the same acknowledgment number as the previous ACK. The out-of-order data packet will be buffered by the receiver until the arrival of all packets before it, when the consecutively ordered data will be further processed and delivered to the application. In-order data delivery is thus fulfilled. We come back to the example shortly.

A TCP sender applies a window-based control to regulate the amount of outstanding data, namely the data that has been transmitted but not yet acknowledged by the receiver. The outstanding data consists of data

5 buffered by the receiver, data packets in transit, and data packets whose corresponding ACKs are in transit. Its amount shall not exceed the minimum of receiver’s advertised window (awnd) and sender’s congestion window (cwnd). awnd is set by the receiver and fed back to the sender via ACK for indicating the available spaces in receiver buffer. It prevents a fast sender from over- whelming a slow receiver. cwnd is determined by the sender, representing the latter’s estimation of the load affordable by the network. The acknowledgment scheme and window-based transmission provides a basis for realizing the important features of end-to-end congestion control and reliable transmission. In fulfilling the former feature, cwnd has to be dynamically adjusted throughout the TCP session in accordance with the ever-changing available network capacity. Signals of network underload and overload are needed for triggering the increase and decrease of cwnd, respectively. In fulfilling the latter feature, signals of packet loss are needed for triggering retransmission of lost data packets to ensure the eventual delivery of every data byte.

A TCP sender uses successful, prompt data delivery as a signal that the network can potentially carry more load than currently oﬀered. The slow start and congestion avoidance algorithms are devised on this premise, and govern the increment of cwnd when it is smaller than and greater than slow start threshold value (ssthresh), respectively. Slow start increases cwnd by one data packet size upon receiving an non-duplicate ACK that acknowl- edges the successfully delivery of new data. Congestion avoidance increases cwnd by one data packet size per RTT if all data packets transmitted in the previously RTT are acknowledged.

6 A TCP sender uses the same set of signals for indicating packet loss and network overload. Two types of signals are used, namely retransmission timeout (RTO) and triple duplicate ACKs. A retransmission timer is started when a data packet is ﬁrst injected into the network, and will timeout if the packet is still not acknowledged when the timer expires. Upon the occurrence of an RTO, all the outstanding packets will be retransmitted. At the same time, the network is deemed severely congested and cwnd will be forced to reopen from one packet size by employing the slow start algorithm.

At the receiver side, each out-of-order packet will trigger a duplicate ACK. At the sender side, when the number of duplicate ACKs reaches a certain threshold value, say, three, the packet expected by the receiver will be deemed as lost due to congestion. Fast retransmit and fast recovery will be activated, retransmitting the packet expected by the receiver and halving cwnd. ssthresh is then set to cwnd so that cwnd will reopen by employing the congestion avoidance algorithm. Therefore, triple duplicate acknowledgment, a direct signal of out-of-order packet events, is further used as an indication of congestive packet loss. In the example illustrated in Figure 1.1, the second, third, and fourth data packets arrived at the receiver are all out of order since they do not contain the byte numbered 301 expected by the receiver, each triggering a duplicate ACK. The sender activates fact retransmit and fast recovery upon receiving the third duplicate ACK.

7 1.2.2 AQM

A queue management algorithm manages the backlog at routers, determining when and how packets should be dropped. Conventionally, a router does not drop packets until the backlog exceeds the buffer size, forcing discard of incoming data packets, namely drop tail, or packets in the backlog. The TCP data flows will keep on increasing cwnd and generate more data packets in transit until a router buffer reaches its limit. Consequently,

1. The end-to-end delay for data delivery is signiﬁcantly increased.

2. Insufficient buffer space incurs a much higher packet loss rate for bursty traffic that tends to fill up the buffer space.

On the other hand, when buffer overflow occurs, the situation is likely to persist for a while due to the delay of TCP in detecting the congestion. Packets belonging to different TCP flows will be dropped simultaneously. All or most TCP flows will be triggered to back off, leaving the buffer largely unoccupied. This TCP synchronization [80] in turn creates another two problems:

1. The backlog ﬂuctuates severely between the full buﬀer size immediately before a TCP synchronization and the low level immediately afterwards. This creates large jitters in the queueing delay and thus

end-to-end delay.

2. The backlog may become empty after a TCP synchronization, leaving the outgoing link idle.

8 Active queue management (AQM) [11] was first proposed in the 1990s to tackle the aforementioned problems with conventional queue management algorithms. AQM probabilistically drops packets before buffer overflow based on a dropping rate determined from the past and/or present queue lengths. The backoff of different TCP flows are therefore desynchronized and spread across time. With the coordination of an effective AQM design, the aggregate transmission of the TCP flows can keep a router buffer occupied with a small, stable backlog. This significantly reduces the end-to-end delay, smooths out delay jitters, leaves sufficient buffer space to absorb bursty traffic, and keeps the bottleneck link backlogged and thus fully utilized. Random early detection (RED) [11, 22] is a most classical AQM algorithm. It maintains an average queue size r, say, by computing the running average of sampled backlog size b with queue weight α:

r (1 α)r + αb (1.1) ← −

The packet dropping probability p is zero if r is less than a preset threshold value minThresh, and one if r is greater than the other preset threshold value maxThresh. When r is between minThresh and maxThresh, p is determined as a linear function of r:

r minThresh p pMax − (1.2) ← maxThresh minThresh −

RED performs probabilistic packet dropping based on p upon each packet arrival.

9 1.3 Wireless Channels

The next generation network is expected to be a heterogeneous network of networks, including both wired and wireless components. Wireless networks may extend beyond access networks to backhaul networks [29], and even backbone networks. The present Internet congestion control is designed based on the premise of conventional wired network, where data are mostly correctly, orderly delivered. It encounters significant challenges in the wireless environment due to the latter’s more unpredictable nature. In particular, non-congestive packet losses and packet reordering, while being rare events in conventional wired networks, are common in wireless networks [44]. As compared with the wired media, the wireless medium provides much more lossy physical links for data transmissions. Signals propagating over wireless channels suffer from degradation, interference, and noise. Packets received may be damaged beyond the recovery capability of error control codes, if any. These packets are thus discarded, leading to the occurrence of non-congestive packet losses. Packet reordering refers to the disruption of the packet order of a TCP flow. Despite conventional beliefs that packet reordering is a transient or pathological network behavior, it is in fact persistently observed over modern networks and can be caused by a myriad of reasons [44, 45]. Due to the high transmission error rates and, in some cases, mobility in wireless networks, packet reordering is further increased substantially when the transmission medium evolves from physical cables to wireless. Specifically, four causes of packet reordering are commonly observed over wireless networks, including:

10 Link-layer retransmission (LLRTX): To combat high transmission error rates of wireless channels, some link layer retransmission mechanisms [7, 28] have been proposed to locally retransmit damaged packets at the link layer.

As a side effect of local retransmission, the packet order of a flow is altered. Path change: Over a mobile ad-hoc networks (MANET), a TCP flow traverses a number of wireless nodes. The transmission path of the flow may be altered when some nodes move. The RTT of the connection may change too. Consequently, it is possible that some packets transmitted after the path change arrive at the receiving end before those packets transmitted prior to the path change. Hand-off: In an infrastructure-based network, the complete transmission coverage of an entire area is achieved cooperatively by a set of base stations.

When a mobile client moves from the radio range of one base station to that of another, a hand-oﬀ between these two base stations occurs, changing the transmission path for the ﬂow. The resulting variation in RTT may lead to the occurrence of packet reordering.

Packet-level multi-path routing: In wireless mesh network (WMN) and many other networks of rich connectivity, multi-path routing have been proposed for load balancing. When it is carried out at the packet level, packets may be reordered due to the diﬀerence in time delays of distinct paths. The present Internet congestion control therefore encounters two-fold challenges in the wireless environment. First, TCP sender triggers fast retransmit and fast recovery upon triple duplicate ACKs. While the latter is dominantly caused by congestive loss over conventional wired networks, it is often triggered by additionally packet reordering and non-congestive loss in

11 the wireless environment. Thus, TCP tends to falsely retransmit packets and reduce cwnd from time to time, injecting duplicated bytes into the network and keeping cwnd unnecessarily small. Consequently, the available network resources are wasted and under-utilized. Second, the extra source of packet losses, namely wireless losses, may interfere with the normal operation of AQM, which communicates implicitly with TCP via active packet drops. This casts doubt on whether AQM can continue to fulﬁll the objective of maintaining a stable, small backlog under wireless losses.

1.4 Thesis Statement and Outline of Thesis

Our thesis statement is to demonstrate that the present Internet congestion control can be systematically adapted to the wireless environment at a minimal disruption and deployment cost.

Presently, the Internet congestion control is generally loss-based. Packet drops are used as a signal of network overload, based on which TCP activate congestion response and AQM probabilistically trigger the congestion responses of diﬀerent TCP connections. We devise our enhancements within the scope of loss-based TCP and AQM. This brings the inherent advantage of minimizing both the deployment cost and the potential disruptions to the existing congestion control deployment. On the TCP side, we have developed a smart TCP sender (STS) model, that uses a serialized-timer structure to oﬀer more reliable signals of packet loss and network congestion over general (reordering), error-prone channels.

12 Timers are serialized in the sense that the second timer is started upon the expiration of the first timer. We have devised a novel TCP variant, known as TCP for non-congestive loss (TCP-NCL), as a practical approximation of the STS. TCP-NCL can accurately differentiate among the occurrences of packet reordering, congestive loss, and non-congestive loss, and can thus serve as a unified solution for effective congestion control, sequencing control, and loss recovery over wireless networks. Deployment of TCP-NCL requires sender-side modification only.

On the AQM side, we have investigated the performance of AQM under wireless losses using simulation experiments. Our results report that RED fail to maintain a stable backlog against time-varying wireless packet error rate (WPER). We have applied the internal model principle to devise a family of AQM enhancements, which can adjust packet dropping rate to compensate for wireless losses without requiring exact knowledge of WPER, and thus stabilize the backlog. We further devise integral controller (IC) as an embodiment of the principle, which performs robustly against time-varying

WPER in various challenging scenarios like fat pipes and perturbing traffic flows. In summary, our proposed TCP and AQM enhancements can perform effective congestion control over the heterogeneous wired/wireless Internet, and help the latter to operate in the optimal region of low delay with small jitters and high throughput. The rest of this thesis is organized as follows. Chapter 2 presents a literature review of existing work on wireless TCP enhancements and on the design and analysis of AQM. It puts our work in

13 perspective. Chapter 3 develops the STS model. TCP-NCL is then devised as a practical approximation of the STS model. Extensive simulation experiments are performed to examine the performance of TCP-NCL against packet reordering, non-congestive packet loss, and congestive loss. Chapter 4 performs simulation-based investigation of AQM under wireless losses. A linear model of TCP/AQM under wireless losses is formulated. On this basis, the internal model principle is applied to devise a family of

AQM enhancements that stabilize the backlog. IC is introduced as an embodiment of the principle, with a set of design rules developed. Extensive simulations are conducted to evaluate IC against time-varying wireless losses under various network scenarios, like fat pipes and perturbing traﬃc ﬂows.

Chapter 5 concludes the thesis. It summarizes our major contributions, and presents an overview of future research directions. The work presented in this thesis has in parts been published in [39, 38, 36, 37, 47].

1.5 Concluding Remarks

In this chapter, we bring the background materials on Internet congestion control and the challenges it encounters in the wireless environment. Con- gestion control is crucial to ensuring the stable, efficient operation of packet- switched networks like the Internet. The present Internet congestion control, as fulfilled by TCP and AQM, performs well for conventional wired networks. However, it encounters significant challenges in wireless networks, in large

14 part due to the frequent occurrences of packet reordering and non-congestive loss. We suggest loss-based TCP and AQM enhancements to adapt the Internet congestion control to the wireless environment at a minimal deployment cost and disruption.

15 Chapter 2

Related Work

2.1 Chapter Overview

In this chapter, we conduct a literature survey of related work performed to date. Section 2.2 studies existing wireless TCP enhancements that adapt TCP to the general (reordering), error-prone channels. Section 2.3 studies existing work on the design and analysis of AQM. Section 2.4 summarizes limitations of these and puts our work in perspective. The work presented in this chapter has in parts been published in [39, 37].

2.2 Wireless TCP Enhancements

In this section, we summarize existing work for adapting TCP to perform congestion control and loss recovery over general error-prone transmission channels in wired/wireless networks. We conﬁne ourselves to the scope of loss-based TCP variants that use congestive loss as signals of network con-

16 gestion. Some TCP variants propose inferring network loading condition based on explicit network feedbacks [32, 75] or RTT [12, 71] for adapting TCP to high-speed networks. Yet, the former requires universal support by all the traversed intermediate routers, which can hardly be attained due to the heterogeneity of the Internet. The latter, known as delay-based TCP variants, may not work well over reordering networks, where RTT can be varied by factors other than network loading. Besides, they may not share the bandwidth fairly with loss-based TCP in widest current usage [55, 67].

AVG, DEL, EWMA, and INC in [9], RR-TCP [81], TCP-DCR [8], and TCP-PR [10] propose to abandon the fixed triple duplicate ACKs as signals of congestive packet loss. Instead, they proactively postpone a packet retransmission until a corresponding timer expires or when the number of duplicate ACKs received reaches an adaptively evolved threshold value. This is considered a more reliable signal of packet loss over reordered channels. However, these variants use the same set of signals for activating congestion response, and are incapable of differentiating between congestive and non- congestive losses. They thus have to rely on LLRTX for making the on-going random packet losses over wireless links transparent to the transport layer. DSACK TCP [23], TCP-DOOR [69], and TCP-Eifel [52] are designed with the premise that triple duplicate ACKs may be unreliable indications of congestive loss. Their approaches differ, however, in that they try to detect spurious retransmission after activating fast retransmission and fast recovery upon the arrival of triple duplicate ACKs1. Upon successful detection, cwnd

1The same design philosophy is also employed by F-RTO [62], which focuses on detecting spurious retransmission triggered by RTO. However, false RTO is not the major reason for performance deterioration over wireless networks. We thus do not include F-RTO as

17 will be recovered immediately or via the slow start process. TCP-DOOR also disables congestion control for a time interval as it assumes out-of-order events are often caused by path change. However, these variants are generally unable to recover cwnd unnecessarily reduced in the event of non-congestive packet losses. Moreover, as shown by our simulation results, TCP-DOOR tends to excessively disable congestion control actions under persistent packet reordering, resulting in substantial deterioration in performance. JTCP [74], TCP-Veno [24], and TCP-Westwood (TCPW) [13] focus on differentiating congestive and non-congestive losses by using the estimated network load upon the arrival of triple duplicate ACKs, which is still treated by them as a credible signal of packet loss for triggering retransmission. One of their major merits is that the binary signal of packet loss no longer dictates the activation of congestion response. Instead, the signal is combined with the estimated network load to decide by how much cwnd should be reduced. Yet, the inherent assumption of nearly in-order channel obviously limit the applicability of these variants over wireless networks and many other modern networks, where packet reordering are common. Particularly, persistent reordering may significantly affect the adaptation of cwnd to the available network bandwidth by frequently activating minor reductions of cwnd and triggering false retransmissions. TCP-Probing [35] applies a different approach for differentiating congestive and non-congestive losses upon inferring packet loss from the arrival of triple duplicate ACKs. After cwnd is halved in response to the inferred packet loss, probing data packets are injected into the network. The inferred a wireless TCP enhancement.

18 packet loss is categorized as non-congestive if the RTT of the first two acknowledged probing packets are smaller than the best RTT, or the minimum of the measured RTTs during the TCP session. Again, TCP-Probing will trigger frequent false fast retransmissions due to persistent packet reordering. Furthermore, the stored best RTT may fail to capture network changes. Specifically, when a path change or handoff occurs within a TCP session and increases the minimum attainable RTT, the stored best RTT will not be updated. Consequently, it may become impossible for the probe packets to be acknowledged within the best RTT even under light network loading. The existing unified solutions require information and/or modifications of the network protocol stack beyond the transport layer. ATCP [48] introduces an ATCP layer between TCP and IP. The new layer switches TCP among various pre-defined states in accordance with the network condition, trying to avoid spurious packet retransmission and congestion response. In summary, there is a lack of a unified solution at the transport layer with reliable signals of network overload and packet loss for performing congestion control and loss recovery over general, error-prone transmission channels. Such unified solution is highly desirable, as it adapts TCP to a wide range of wireless and, in some cases, wired transmission channels at a minimal deployment cost.

2.3 Design and Analysis of AQM

In this section, we summarize the related work on the design and analysis of AQM, with special emphasis on those applying the control-theoretic

19 approaches and/or adapting AQM to wireless networks. First proposed in 1993, RED [22] demonstrates the inherent advantage of AQM in helping the network to operate in the optimal region of high throughput and low delay. However, parameter tunings and new variants of RED invariably adopted the trial-and-error approach due to the diﬃculties in understanding the dynamics of TCP/AQM. The ﬂuid models of TCP/AQM, as presented in, say, [33, 50, 53], provide a basis for systematic design and analysis of AQM. The optimization-based approach [33, 49] interprets TCP/AQM as a distributed algorithm for solving a utility maximization problem, subject to capacity constraints. The major focus is on the optimal solutions attained at the equilibria. However, transient responses of AQM are mostly not captured.

The control-theoretic approach views Internet congestion control as a non- linear control system. The system is ﬁrst linearized around its equilibrium so as to analyze the dynamics of TCP/AQM around the equilibrium. This enables parameter tuning of RED and design of new AQM algorithms based on frequency-domain analysis [26, 27, 51, 79], which is powerful in improving the transient response of AQM and guaranteeing the stability of the linear system. Interested readers can refer to [65] for a survey. The work in this area closest to our study is [26], which designs a proportional-integral (PI) controller for the general network topology. Generally, the IC in our work can be considered a special case of the PI controller with the proportional part being set to zero. However, the PI controller considered in [26] assumes the proportional part to be non-zero, so that it cannot be reduced to the IC. Moreover, we apply the IC to combat wireless losses, whereas [26] focuses on

20 designing a PI controller in wired networks. The global stability and region of attraction of Internet congestion control have been studied in the context of a non-linear system. Please refer to [20,

72, 78] and the references therein. In a nutshell, these are concerned with whether the system will converge to the equilibrium starting from a feasible initial status not necessarily close to the equilibrium [34]. Among them, Fan et al. [20] examine the robustness of the network flow control against disturbances. Yet, the AQM in [20] is static rate-based, which determines the packet dropping rate as a function of the instantaneous incoming data rate. Our work considers buffer-based AQM variants, which are widely used in the Internet to determine the packet dropping rate from the queue length. In adapting AQM to wireless networks, the inter-flow fairness in ad-hoc networks can be improved [76] by virtually aggregating packets queued in those interfering nodes to one queue, and applying AQM to manage the queue.

2.4 Concluding Remarks

In this chapter, we have summarized existing work on wireless TCP enhancements and on the design and analysis of AQM. We ﬁnd that there is a lack of transport layer uniﬁed solution for both packet reordering and non-congestive loss. Such solution is highly desirable for adapting TCP to the reordering, error-prone communication channel prevalent in wireless environment at a minimal deployment cost. The performance of AQM in wireless environment remains to be thor-

21 oughly investigated. In particular, it is largely unexplored in the literature whether AQM can maintain a stable, small backlog under wireless losses. If not, an AQM enhancement for stabilizing the backlog would be highly desirable so as to improve the delay performance and fairness towards bursty traﬃc in wireless networks..

22 Chapter 3

A Serialized-Timer Approach for Enhancing Wireless TCP

3.1 Chapter Overview

In this chapter, we develop a unified solution for performing effective congestion control, sequencing control, and loss recovery over communication networks with general (reordering), error-prone transmission channels like wireless networks. The proposed modifications involved are limited to sender- side TCP only, thereby facilitating possible future wide deployment. As we find in the literature review in Chapter 2, most existing TCP variants only focus on tackling with the presence of either non-congestive packet loss or packet reordering. A few unified solutions for both problems generally require information and modifications beyond the scope of the transport layer, hindering possible future wide deployment. We develop an ideal TCP sender model, known as smart TCP sender

23 (STS), for constructing more reliable signals of packet loss and network congestion over a general error-prone channel for activating packet retransmission and congestion response, respectively. The two signals of packet loss and network congestion are the expirations of two serialized timers. The first timer is started when a packet is first injected into the network, while the second timer is started upon the expiration of the first timer. The timer expiration periods have been optimized for minimizing quantified risks associated with spurious congestion response and excessively delayed congestion response. We further devise a new TCP variant, known as TCP for non-congestive loss (TCP-NCL), as a practical approximation of STS. TCP-NCL realizes the serialized-timer structure economically by using timestamps. Deploy- ment of TCP-NCL requires maintenance of a few unused bits in the existing data structure for TCP and a few extra variables only. The maintenance is performed upon arrival of acknowledgement packet. We have performed extensive simulation experiments to examine the performance of TCP-NCL and compare it with several well-known TCP variants in the literature using Network Simulator (ns) Version 2.29 [19]. Our results report that TCP-NCL attains significant performance improvement over general, error-prone communication networks, thereby demonstrating robustness against packet reordering and non-congestive packet loss. At the same time, multiple competing TCP connections, with some employing TCP-NCL and some employing SACK TCP, the popular standardized TCP variant, are capable of sharing bandwidth fairly over conventional wired networks. TCP- NCL can thus serve as the aforementioned unified solution for performing

24 eﬀective congestion control, sequencing control, and loss recovery over communication networks with general, error-prone transmission channels. Section 3.2 develops the STS model. Section 3.3 presents TCP-NCL based on the analytical results of the STS model. Section 3.4 examines the performance of TCP-NCL and compares it with some popular TCP variants. Section 3.5 concludes and discusses some possible extensions of this work. The work presented in this chapter has in parts been published in [39, 38, 36]. A U.S. patent has been ﬁled for the work [47].

3.2 STS Model

In constructing an aforementioned uniﬁed solution, we start from the following two initial observations: Nature of signals: Over general communication channels, there are two prevalent approaches for inferring the occurrence of packet loss based on: 1) the number of consecutively received duplicate ACKs reaching an adaptively evolved threshold value or, 2) a timer1 expiration. However, one major concern with the former approach is that a TCP sender may not be able to accumulate enough duplicate ACKs to perform loss recovery because of the insuﬃcient number of the outstanding packets. Special measures, like limited retransmit [2], would have to be incorporated in order to alleviate the problem by sending extra data packets upon the arrival of duplicate ACKs. By contrast, a more elegant and robust approach for estimating the probability

1The timer is diﬀerent from the retransmission timer, which triggers a packet retransmission and activates the slow start algorithm when it expires. TCP-PR is an exception since it uses a modiﬁed retransmission timer so that fast retransmit and fast recovery are activated upon the timer expiration.

25 Send Packet Pi & Start Timer RDi

Yes ACKi Received?

No RDi Expires?

Yes

Resend Packet Pi & Start Timer CDi

Yes ACKi Received?

No CD i Expires?

Yes

Activate Congestion Control

RTT Update

(Added by TCP−NCL)

Figure 3.1: A ﬂowchart for STS model.

26 of packet loss is to measure the time elapsed since a packet transmission (or some other similar instances) and refer to the past recent history of RTT. Separation of signals: Over error-prone channels, signals of packet loss alone can only serve as prior signals of network congestion. It is necessary to take additional information into account to confirm whether the inferred loss can be categorized as a congestive loss. In place of the conventional approach of activating fast retransmission and fast recovery simultaneously, we propose to delay the decision on whether to activate a congestion response behind that of a packet retransmission for a short time period. The motivation behind doing so is two-fold. First, information inferred by the occurrences of the events which happen after a packet retransmission can be incorporated to decide whether a congestive loss has occurred. Such information is a real-time reflection of the network loading condition so that this helps to make an appropriate decision. Specifically, if an ACK for a packet lags behind a retransmission of that packet by a short duration, it should be treated as a signal of no network congestion. The ACK packet can either be for the originally transmitted packet or the retransmitted packet. In the former case, false retransmission, which can hardly be fully eliminated in reality, is detected, thereby rejecting the necessity of a congestion response. In the latter case, a fairly short round-trip time and thus a lightly loaded network can be inferred.

Second, it is more “aﬀordable” for TCP to trigger a false fast retransmission than a false congestion response. In the event of a false fast retransmission, there is at most one full-sized packet being retransmitted spuriously. However, there is a signiﬁcant portion of the available bandwidth being left

27 unused in the event of a falsely activated congestion response. For TCP variants which activate fast retransmit and fast recovery simultaneously, they may defer both measures to avoid the penalty of a false congestion response until a congestive loss can be confirmed highly accurately. However, over general error-prone reordering channels, such deferment on fast retransmit may result in an expensive RTO. For example, TCP-DCR postpones fast retransmit by one RTT upon receiving the first duplicate ACK,so that a packet is retransmitted, if needed, about two RTTs after its first transmission. By advancing packet retransmission before the activation of congestion response, the risk of triggering an RTO can be more effectively reduced while avoiding the heavy penalty due to a false congestion response. Nevertheless, while a comparatively higher level of false retransmission can be tolerated with our proposal, it should still be maintained at a minimal level to avoid significant waste of the limited bandwidth. Based on the above observations, we design our STS model, as illustrated in Figure 3.1. A new retransmission decision (RD) timer RDi is started whenever a new packet Pi is injected into the network. If ACKi is received by the TCP sender before RDi expires, RDi will be cancelled. Otherwise,

Pi will be retransmitted and a congestion response decision (CD) timer CDi will be started.

CDi will be cancelled if ACKi arrives before it expires. Otherwise, the congestion control mechanisms will be activated upon the expiration of CDi.

Therefore, the installation of CDi allows the TCP sender extra time after the packet retransmission to decide whether congestion control shall be activated.

ACKi arriving before the expiration of CDi shall be treated as an indication

28 Table 3.1: General notations.

Notation Meaning

CS Number of packets TCP sender misses to send due to SC CT Number of packets TCP sender misses to send due to the extra reduction of TCP sender’s throughput introduced by RTO F (t) Cumulative distribution function of RTT for Packet Pi G Goodput of TCP sender maxRi Maximum attainable RTT for Packet Pi minRi Minimum attainable RTT for Packet Pi o Pi First transmitted Packet Pi r Pi Retransmitted Packet Pi PC Congestive loss rate PL Loss rate PW Non-congestive loss rate T Throughput of TCP sender W Average cwnd over a TCP session

of no network congestion. Thus, this eliminates the need for activating any congestion control measures.

The expiration periods of RDi and CDi, denoted as τRDi τCDi , are evaluated in Sections 3.2.1 and 3.2.2, respectively. To facilitate the subsequent discussion, we deﬁne the notations as shown in Tables 3.1 and 3.2.

The following assumptions are made to simplify the discussion:

1. Data packets are of equal size. Accordingly, we express the size of congestion window and slow start threshold (ssthresh) in terms of the number of packets.

o 2. P (PAi(t) CL ) continuously monotonically increases with time t and | i

29 Table 3.2: Notations for events.

Notation Meaning o o CLi Pi lost due to congestion r r CLi Pi lost due to congestion o o NLi Pi experiencing non-congestive lost r r NLi Pi experiencing non-congestive lost RTXi Fast retransmission of Pi o r PAi(t) Either Pi or Pi acknowledged by time t after RTXi o o PAi (t) Pi acknowledged by time t after retransmission r r PAi (t) Pi acknowledged by time t after retransmission o r PUi(t) Both Pi and Pi unacknowledged by time t after RTXi o o PUi (t) Pi unacknowledged by time t after retransmission r r PUi (t) Pi unacknowledged by time t after retransmission T O Retransmission timeout

o o limt P (PAi(t) CL ) > 0. Given that P L is lost due to congestion, →∞ | i i r the chance of Packet Pi being acknowledged relies on Pi . By allowing

r Pi more time (i.e., increasing t), the chance of Pi being acknowledged does not drop. Furthermore, the chance should be non-zero, given that

P r is allowed suﬃcient time (t ), i → ∞

3. Non-congestive loss is assumed to be random. In other words, we do not consider correlated or burst loss for simplicity. Nevertheless, simulation results will show that our proposed TCP variant still oﬀers competent

o performance under burst loss. Thus, NLi is mutually independent with

r any events occurring with Pi and vice versa. In particular, it follows that:

P (NLo PU r(t)) = P (NLo) P (PU r(t)) (3.1) i ∩ i i · i

30 3.2.1 RD Timer

τRDi determines how long we have to wait before activating packet retrans-

mission. The assignment of τRDi requires careful balancing between the objectives of prompt retransmission for preventing RTO and avoiding spurious packet retransmission.

o Ideally, it should be guaranteed that, when Pi is lost due to congestion,

r Pi should be promptly transmitted so that the remaining time before RTO is suﬃcient for it to be acknowledged. This ensures that cwnd will be halved instead of being reset to one at the onset of network overload. In other words,

r RTO will be incurred if and only if Pi is lost:

o r P (T O CL )= P (P L )= PL (3.2) | i i

On the other hand, an excessively small τRDi will result in spurious packet retransmissions. Let F (τRD )=1 ζ, where 0 ζ 1. A fast retransmit of i − ≤ ≤ packet Pi will be activated upon the expiration of RDi, given Pi is not yet acknowledged. Thus,

o P (RTXi) = P (PUi (0))

= P (PU o(0) P Lo)P (P Lo) i | i i +P (PU o(0) P Lo)P (P Lo) i | i i

= PL + ζ(1 PL) (3.3) −

In addition to fast retransmit, packet retransmission can also be activated by RTO. Yet, with fast retransmit promptly activated, the occurrences of

31 RTO will be maintained at a minimal level. Thus, the diﬀerence between throughput and goodput, without considering the protocol overhead, should mainly be attributable to fast retransmit. Accordingly, we have:

T [1 + P (RTXi)]G (3.4) ≃

Hence,

G −1 [(1 + PL)+ ζ(1 PL)] (3.5) T ≃ −

To improve eﬃciency, we thus have:

ζ 1 (3.6) ≪

Since the retransmission timer only serves as a coarse upper bound for performing loss recovery and congestion control, it is possible for (3.2) and

(3.6) to be simultaneously satisﬁed with τRDi appropriately set. A closed-

form expression for the optimal value of τRDi is left as part of the future work. Nevertheless, we assume that (3.2) and (3.6) are satisﬁed in the following discussion.

3.2.2 CD Timer

The setting of τCDi should guarantee that, if ACKi arrives before the ex-

o piration of CDi, Pi does not experience congestive loss. Analytically, this

32 is:

o P (CL PAi(t))=0 (3.7) i |

for all t [0, τCD ]. In this way, we ensure that the TCP sender will not ∈ i misinterpret congestive loss as non-congestive loss.

o Denote τu , sup t P (PAi(t) CL )=0 , which is well deﬁned by As- t{ | | i } sumption 2. A suﬃcient and necessary condition for (3.7) is given by the following lemma.

o Lemma 3.1 P (CL PAi(t))=0 if and only if t τu. i | ≤

Proof Observe that:

o o o P (CL PAi(t))P (PAi(t)) = P (PAi(t) CL )P (CL ) (3.8) i | | i i and

r P (PAi(t)) P (PA (0)) = (1 PL)(1 ζ) > 0 (3.9) ≥ i − −

o o Thus, P (CL PAi(t)) = 0 if and only if P (PAi(t) CL ) = 0. On the i | | i o other hand, by construction, P (PAi(t) CL ) = 0 if and only if t τu. The | i ≤ result of the lemma therefore follows.

We do not know an exact expression for τu for the general case at the moment. In the simple case of a single flow over a bottleneck link, when a packet is dropped due to buffer overflow, the buffer will be kept full up to when the lost packet is retransmitted. τu is therefore maxR in this case. Yet,

33 this do not apply the the case of multiple flows since upon the occurrence of buffer overflow some flows may back off earlier than others. Nevertheless, a lower bound for τu can be obtained for the general case. Noting that r o P (PAi (minRi)) r P (PAi(minRi) CLi ) o and P (PAi (minRi)) = 0, we have | ≤ P (CLi ) o P (PAi(minRi) CL ) = 0. Thus, a general lower bound for τu is minRi. | i

Now, we shall seek for an optimal solution for τCDi within [0, τu]. Con- sider a time period t (0 t τu) after the retransmission of packet Pi, when ≤ ≤ a TCP sender is facing the decision of whether or not to activate a congestion response. The risk associated with a positive decision is that the network may

o not be congested (i.e. Pi is not dropped due to congestion). Consequently, the spuriously activated congestion response will reduce cwnd unnecessarily.

o On the other hand, if the network is indeed overloaded (i.e. Pi is lost due to congestion) and the activation of congestion response is excessively delayed, the network congestion may be exacerbated and an expensive RTO may be incurred.

To reach a rational decision regarding the optimal τCDi , we need to compare the quantified risks of activating congestion response and delaying congestion response. Before doing that, we first quantify the costs of falsely activating congestion response and excessively delaying congestion response when network is indeed overloaded, respectively, by the following two defini- tions.

Deﬁnition 3.1 The cost of falsely activating congestion response, denoted as CS, is the average number of packets a TCP sender misses to send due to the activation of a congestion response,

34 Deﬁnition 3.2 The cost of excessively delaying congestion response, denoted as CT , is the average number of packets a TCP sender misses to send due to the occurrence of RTO.

Consider the activation of congestion response, when both cwnd and ssthresh are set to 0.5 W . Thus TCP sender is at the congestion avoidance stage to begin. cwnd is incremented by one at every RTT cycle afterwards.

Let nCA be the number of cycles during which the TCP sender is in the congestion avoidance stage with cwnd less than W . Accordingly, we have:

nCA = 0.5 W (3.10) ⌊ ⌋

Therefore, when a congestion response is falsely activated, the number of packets the TCP sender misses to send is:

nCA

CS = W nCA ( 0.5 W + i 1) − ⌈ ⌉ − i=1 X = 0.5 0.5 W ( 0.5W +1) (3.11) ⌊ ⌋ ⌊ ⌋

Consider the occurrence of an RTO, when cwnd is reset to one and ssthresh is set to 0.5 W . For simplicity, we would assume that ACK arrives before the occurrence of another RTO. Thus, after being reset, the TCP sender is in the slow start stage and W will double itself at every RTT round as long as it is less than ssthresh. Once W reaches ssthresh , the TCP ⌈ ⌉ sender leaves the slow start stage and enter the same congestion avoidance stage. Let nSS be the number of cycles during which the TCP sender is in

35 the slow start stage, we have:

nSS = log (0.5 W ) (3.12) ⌈ 2 ⌉

Upon a genuine congestive loss, it is generally desirable for a TCP sender to halve cwnd and directly enter the congestion avoidance stage. Therefore, if an RTO occurs instead and forces cwnd to be reset to one, the TCP sender throughput is penalized when the slow start phase is activated. The number of packets the TCP sender misses to send is:

nSS i−1 CT = W nSS 2 − i=1 X = log (0.5 W ) W 2⌈log2 (0.5 W )⌉ +1 (3.13) ⌈ 2 ⌉· −

We are now ready to quantify the risks associated with activating a congestion response and delaying congestion response. The following two metrics are introduced, respectively, for the purpose.

Deﬁnition 3.3 The expected cost of activating a congestion response, de-

o noted as ECA(t), is the product of the conditional probability of Pi not being lost due to congestion, given that Pi is unacknowledged, and the cost of falsely activating congestion response. In other words,

o ECA(t)= P (CL PUi(t)) CS (3.14) i | ·

Deﬁnition 3.4 The expected cost of delaying a congestion response, denoted as ECD(t), is the product of the conditional probability that a conges-

36 tive loss and an RTO have occurred, given that Pi is unacknowledged, and the cost of excessively delaying congestion response. In other words,

o ECD(t)= P (T O CL PUi(t)) CT (3.15) ∩ i | ·

The two conditional probabilities involved in Deﬁnitions 3.3 and 3.4 are expressed in terms of loss probabilities, namely PC , PW , and PL, and distribution function of RTT, F (t), by Lemmas 3.2 and 3.3, respectively.

Lemma 3.2 For 0 t τu, ≤ ≤

o P (CL PUi(T )) i | PW [1 (1 PL)F (t)] = − − (3.16) PC + PW [1 (1 PL)F (t)] − − (3.17)

Proof We begin by noting that:

o o P (CL ) = P (PUi(t)) P (CL PUi(t)) i · i | o +P (PAi(t)) P (CL PAi(t)) (3.18) · i |

o For 0 t τu, since P (CL PAi(t)) = 0 by Lemma 3.1, this reduces to: ≤ ≤ i |

o o P (CL ) = P (PUi(t)) P (CL PUi(t)) (3.19) i · i |

It follows that:

o o P (CL PUi(t)) = 1 P (CL PUi(t)) i | − i | 37 P (CLo) = 1 i (3.20) − P (PUi(t))

where 0 t τu. ≤ ≤

We now derive P (PUi(t)), which will complete the proof.

P (PUi(t))

= P (PU o(t) PU r(t)) i ∩ i = P (PU o(t) PU r(t) P Lo)P (P Lo) i ∩ i | i i +P (PU o(t) PU r(t) P Lo)P (P Lo) (3.21) i ∩ i | i i

Since:

P (PU o(t) PU r(t) P Lo) P (PU o(t) P Lo) i ∩ i | i ≤ i | i

= 1 F (t + τRD ) − i ζ (3.22) ≤ where ζ 1 by (3.6), we have: ≪

o r o o P (PUi(t)) P (PU (t) PU (t) P L )P (P L ) (3.23) ≈ i ∩ i | i i

Noting that:

P (PU o(t) PU r(t) P Lo) i ∩ i | i = P (PU r(t) P Lo) P (PU r(t) PAo(t) P Lo) i | i − i ∩ i | i (3.24)

38 and

P (PAo(t) PU r(t) P Lo) P (PAo(t) P Lo)=0 (3.25) i ∩ i | i ≤ i | i we further obtain:

P (PU o(t) PU r(t) P Lo) = P (PU r(t) P Lo) (3.26) i ∩ i | i i | i

(3.23) thus simpliﬁes to:

r o P (PUi(t)) P (PU (t) P L ) ≈ i ∩ i = P (PU r(t) CLo)+ P (PU r(t) NLo) i ∩ i i ∩ i = P (CLo)P (PU r(t) CLo) i i | i +P (PU r(t)) P (NLo) (3.27) i · i

The last step above applies (3.1)

o Recall that τu = sup t P (PAi(t) CL )=0 . Thus, for 0 t τu, t{ | | i } ≤ ≤

r o o P (PU (t) CL ) P (PUi(t) CL ) i | i ≥ | i o =1 P (PAi(t) CL )=1 (3.28) − | i

Substituting (3.28) into (3.27) yields:

o r o P (PUi(t)) P (CL )+ P (PU (t)) P (NL ) ≃ i i · i (3.29)

39 r where P (PUi (t)) can be derived as:

P (PU r(t)) = P (PU r(t) P Lr)P (P Lr) i i | i i +P (PU r(t) P Lr)P (P Lr) i | i i

= PL + [1 F (t)](1 PL) (3.30) − −

Substituting (3.30) into (3.29), we obtain:

P (PUi(t)) = PC + [1 (1 PL)F (t)]PW (3.31) − −

where 0 t τu. ≤ ≤

Lemma 3.3 For 0 t τu, ≤ ≤

o PC PL P (T O CLi PUi(t)) = (3.32) ∩ | PC + PW [1 (1 PL)F (t)] − −

Proof Following a similar process to (3.18) - (3.20), we can obtain:

o o o P (T O CLi )P (CLi ) P (T O CLi PUi(t)) = | ∩ | PUi(t)

(0 t τu) (3.33) ≤ ≤

Equ. (3.32) follows by substituting (3.31) and (3.2) into (3.33).

Thus:

PW [1 (1 PL)F (t)] ECA(t) = − − 2 PC + PW [1 (1 PL)F (t)] { − − } 0.5 W ( 0.5W +1) (3.34) ×⌊ ⌋ ⌊ ⌋

40 ECA (t) EC (t) ECD (t) A

ECD (t) ECD (t) ECA (t)

t t t τ τ τ τ u u TH u (a) Case I. (b) Case II. (c) Case III.

∗ Figure 3.2: Evaluation of τCDi .

PC PL ECD(t) = PC + PW [1 (1 PL)F (t)] − − ( log 0.5W W 2⌈log2 0.5W ⌉ +1) (3.35) × ⌈ 2 ⌉ −

Now,

∂ ∂ ECA(t) 0, ECD(t) 0 (3.36) ∂t ≤ ∂t ≥

Thus, when the activation of congestion response is postponed further, ECD(t), which quantiﬁes the risk associated with delaying a congestion response, increases, while ECA(t), which quantiﬁes the risk associated with activating a congestion response, drops. When ECA(t) is greater than ECD(t), it is

advantageous to set τCDi no less than t since the operation cost of activating congestion response is ECA(t), which is larger than that of deferring it,

ECD(t). The cost may drop when t increases. Similarly, when ECD(t) is

greater than ECA(t), it is advantageous to set τCDi no greater than t since the operation cost of deferring congestion response is ECD(t), which is larger than that of activating it, ECA(t). The cost may drop when t decreases.

41 ∗ Therefore, the evaluation of the optimal solution of τCDi , τCDi , subject to the constraint 0 τCD τu, can be divided into three cases, as depicted ≤ i ≤ in Figure 3.2. Case I arises when ECD(t) exceeds ECA(t) for any t 0. ≥ The gap between the two would keep on increasing when t increases. Thus,

∗ the congestion response should be activated at t = 0, or set τCDi to be zero.

Case II arises when ECD(t) fails to catch up with ECA(t) for all t such that

0 t τu. Since we cannot delay a congestion response further according to ≤ ≤ ∗ the prior constraint, we have to set τCDi to be τu. The ﬁnal case arises when

∗ ECD(t) catches up with ECA(t) for some t such that 0 t = τTH τu. τ ≤ ≤ CDi thus corresponds to τTH since ECA(t) is greater than ECD(t) prior to τTH and less than ECD(t) after τTH .

The optimal value of τCDi can thus be determined by the following theorem.

Theorem 3.1 Suppose Assumptions 1-3 hold. The optimal value for τCDi

∗ within [0, τu] in the sense of minimizing the associated expected cost, τCDi , is given by:

CT −1 0, PC > (PL + 1) PL · CS ∗  τ = PL CT −1 CDi  τu, PC < +1 PL  { [1−(1−PL)·F (τu)]·CS } ·  τTH , otherwise.   (3.37) 

−1 1 CT PC PL where τTH = F . 1−PL − CS ·(PL−PC ) · 1−PL Proof The proof follows from the foregoing discussion.

42 When non-congestive loss is negligible, we fall into the ﬁrst condition in Theorem 3.1. Packet retransmission and congestion response should thus be activated simultaneously. When non-congestive loss becomes non-negligible

CT −1 (say, PW > PC ), it generally holds that PC < (PL + 1) PL. It follows CS ∗ that τ [minRi, τu]. CDi ∈

3.3 TCP-NCL

The STS model is a limited, idealized TCP sender. First, it does not provide a closed-form expression for τ . Second, it assumes, among other RD i things, prior knowledge of the RTT distribution F (t), which is impractical. Therefore, we propose TCP-NCL to closely approximate STS. The serialized-timer structure of the STS model is supplemented by TCP- NCL with the NCL-RTT-Update (NRU) process for maintaining statistics of RTT samples so as to estimate F (t). Section 3.3.1 describes the NRU process. Section 3.3.2 explains the implementation of RD and CD timers using timestamps. The pseudocode for these are exhibited in Algorithms 3.3.1 and 3.3.1, respectively. Section 3.3.3 describes the minor modiﬁcations involved in the retransmission timeout (RTO) timers. Section 3.3.3 describes kernel implementation of TCP-NCL and the overhead involved.

3.3.1 NRU Process

The NRU process is invoked if an ACK, say ACK i, arrives before the expiration of its corresponding CD timer, CD i. This can be further divided into two cases: ACK i arrives before or after RD i expires. In the latter

43 o r case, there is an ambiguity regarding whether ACK i is for Pi or Pi . Thus, some additional measures would be needed to make sure that ACK i is for

o Pi before recording the corresponding RTT sample. We identify the time instance for the most recent transmission of Pi as txTime[i]. Upon the arrival of ACKi, if the present time (identiﬁed by now) exceeds txTime[i] by less than β τ (0 <β< 1), this infers that P r cannot be acknowledged within CD i i such a short interval and thus the RTT sample (τ + now txTime[i]), RD i − is inserted into rttRcd. Otherwise, the corresponding RTT sample will be ignored. By updating RTT based on ACKs received in this manner, the RTT sampling process is more robust to changes in the network environment, especially to abrupt increases in RTT caused by path change or handoﬀ.

When the latter occurs, it is likely that ACK i will not be received before

RTi expires. Yet, the corresponding RTT sample can still be accurately recorded as long as it does not exceed (τ + β τ ). RD i CD i The assignment of τ and τ , as will be discussed in Section 3.3.2, RD i CD i only relies on the maximum RTT sample, maxR, and minimum RTT sample, minR. There is thus no need to keep all the recent maximum RTT record length (MRRL) samples in storage. Instead, when a new RTT is greater than maxR/less than minR, it is used to update maxR/minR. However, merely doing so may prevent us to discard some outdated samples that are very large or very small. The distribution of RTT is time- variant over wireless networks, where the network topology and/or loading may change over a TCP session. The time-varying property of F (t) requires prompt discard of outdated samples. In [10], TCP-PR needs to keep track

44 of updated maximum RTT sample as well. A srtt variable is maintained for the purpose. Upon obtaining a new RTT sample, srtt is multiplied by α( 1 ), where α is a system designed parameter in the interval (0, 1). −cwnd The new sample is then used to update srtt if greater than the current value of srtt. The multiplication, which reduces srtt by a factor of α per RTT, ensures that outdated peaks in RTT will eventually be replaced. However, it is generally computationally expensive for the kernel. We thus apply a computationally cheaper strategy for the similar purpose of discarding outdated maximum and minimum RTT samples. We will maintain two new variables maxR2 and minR2 in supplement to the prescribed updating process of maxR and minR. A periodic downward counter, rcdAge, is initialized as MRRL and decremented by one upon obtaining a new RTT sample. Similarly as maxR, an RTT sample is used to update maxR2 if greater than it. Yet, when rcdAge reaches zero, maxR will be set as maxR2 before maxR2 is set as the newly obtained RTT sample without comparison. rcdAge is then set as MRRL again. Thus, every time upon completing the updates on maxR, maxR2, and rcdAge, maxR2 always maintains the maximum value within the most recent (MRRL rcdAge + 1) RTT samples. Regarding maxR: −

1. When rcdAge is MRRL, maxR has just been updated by the old value of maxR2. In addition, it will be set to the new RTT sample if smaller than it. Thus, it will keep the maximum value within the most recent (MRRL + 1) RTT samples.

45 2. Upon obtaining the next n (1 n MRRL 1) samples, rcdAge is ≤ ≤ − decremented by one each time and thus n = MRRL rcdAge. Since − rcdAge is greater than zero, maxR is simply updated by the new RTT

if smaller than it. Thus, it maintains the maximum value within the most recent ((MRRL +1)+ n) RTT samples.

3. Upon obtaining one more sample, rcdAge will be decremented to zero and then set to MRRL. We thus go back to the situation described to 1).

Overall, maxR always maintains the maximum value within the most recent (2MRRL - rcdAge + 1) RTT samples, where 1 rcdAge MRRL. This ≤ ≤ enables discard of outdated samples while ensuring maxR is the maximum among a suﬃciently large number ( MRRL+1) of most recent RTT samples. ≥ The maintenance and usage of minR2 are carried out in an analogous manner. It is therefore ensured that minR maintains the minimum value within the most recent (2MRRL - rcdAge + 1) RTT samples.

3.3.2 RD and CD Timers

τ is set to (maxR + minR). Intuitively, this assignment is an appro- RD i priate choice, keeping spurious retransmission at a minimal level without excessively delaying fast retransmit. As will be shown by our simulation results, TCP-NCL demonstrates a very eﬃcient usage of network bandwidth in that it almost always attains the highest connection goodput among all our compared TCP variants. This performance can hardly be achieved if either spurious retransmission constitutes a signiﬁcant wastage of total through-

46 Algorithm 1 Procedure NRU (Packet Pi) 1: rcdAge rcdAge 1 ← − 2: if Pi is on rcPkts then 3: rtt now txTime[i] ← − 4: else 5: if now txTime[i] <β tauCD then − ∗ 6: rtt now txTime[i]+ tauRD ← − 7: else 8: exit from the process 9: end if 10: end if 11: if rcdAge =0 then 12: minR minR2 ← 13: maxR maxR2 ← 14: minR rtt ← 15: minR rtt ← 16: rcdAge MRRL ← 17: end if 18: maxR2 max(maxR2, rtt) ← 19: maxR max(maxR, rtt) ← 20: minR2 min(minR2, rtt) ← 21: minR min(minR, rtt) ← 22: tauCD maxR + minR ← 23: tauCD K maxR + (1 K) minR ← ∗ − ∗

47 Algorithm 2 Procedure update list (Packet ACK i) 1: for each Packet Pj on cdPkts do 2: if tauCD < now txTime[j] then − 3: remove Pj from cdPkts 4: activate congestion response 5: end if 6: end for 7: Retrieve the Packet Pi acked by ACK i 8: if Pi is in rdP kts or cdP kts then 9: NRU (Pi) 10: remove Pi from rdPkts/cdPkts 11: end if 12: for each Packet Pj on rdPkts do 13: if tauRD < now txTime[j] then − 14: remove Pj from rdPkts 15: add Pj to to-be-rtxed 16: end if 17: end for 18: wnd min(cwnd, awnd) ← 19: while number of outstanding packets wnd do ≤ 20: if there is a Packet Pj on to-be-rtxed then 21: add Pj to cdPkts 22: retransmit Pj 23: txTime[j] now ← 24: else 25: Transmit the next new packet Pk 26: Add Pk to rdPkts 27: txTime[k] now ← 28: end if 29: end while

48 put, or RTO frequently occurs and forces cwnd to be reset to one. Thus, the choice of setting τ as (maxR + minR) seems to be appropriate. RD i ∗ By the discussion at the end of previous section, τ [minRi, τu]. CD i ∈ minRi can be approximated by minR, whereas τu is lower bounded by minR and can be up to maxR. Thus, we assign τCD to be (K maxR + (1 K) i ∗ − ∗ minR), where K [0, 1] is a system-designed parameter. It is worth noting ∈ ∗ that, while STS model specifies τ dependant on PC , we remove such CD i dependence in TCP-NCL. This is because congestive loss are usually rate events in the network, and it is very difficult to measure PC accurately from end systems. With our approximation above, at any time instance, the expiration periods of RD/CD timers for all the RD/CD pending packets would be the same. We thus define two variables, namely, tauRD and tauCD, for keeping the present values of the expiration periods of RD and CD timers, respectively, instead of keeping a unique copy for each packet. To fulfill the effects of the RD and CD timers without incurring the heavy cost of maintaining one timer per outstanding packet, we define rdPkts and cdPkts for keeping the packets whose RD and CD timers are supposedly pending, respectively. The two lists are updated upon arrival of an acknowledgment packet as follows:

1. Activate congestion response measure if there are packets that have stayed on cdPkts for more than tauCD, and remove the timeout’ed packets.

2. Remove the acknowledged packet if it is on rdPkts or cdPkts.

49 3. Retransmit packets that have stayed on rdPkts for more than tauRD, and transfer them from rdPkts to cdPkts with timestamps recording the retransmission time instance.

4. Transmit new packets and add them to rdPkts with timestamps recording the transmission time instance.

When activating the congestion control measures, we adopt an approach similar to that used in TCP-PR, that packet losses within the same burst are considered as a single signal about the onset of network congestion and the reduction of cwnd will be triggered only once. Thus, when cwnd is halved, all the present packets in rdPkts and cdPkts are exempted from causing cwnd to be halved again.

3.3.3 RTO Timer

With standardized TCP design, severe congestion is inferred and go-back-N retransmission is performed by retransmitting all outstanding packets upon the occurrence of RTO. The expiration period of RTO timer is set to the sum of mean RTT and a multiple of RTT deviation. The RTO timer is reset upon packet (re)transmission. Since fast retransmit performs retransmission upon receiving three duplicate acknowledgments, the (fast) retransmitted packet is allowed suﬃcient time to be acknowledged before the occurrence of RTO and the associated the go-back-N retransmission. We keep RTO timer in the design of TCP-NCL so as to perform worst-case recovery. Moreover, this ensures that TCP-NCL can emulate the behavior of standardized TCP design in conventional wired networks, thereby reducing

50 its possible disruptions to the latter. Yet, the occurrence of RTO needs to be well coordinated with the serialized-timer structure that has postponed packet retransmission and congestion response proactively. It is undesirable for RTO timer to fire before the RD and CD timers fires. Moreover, a packet retransmitted upon the expiration of RD timer should be allowed sufficient time to be acknowledged before the go-back-N retransmission is considered necessary and activated. Thus, TCP-NCL impose two modifications to the RTO timer:

1. The expiration period of RTO timer is modiﬁed to be (tauRD +2 ∗ tauCD).

2. A further condition for performing go-back-N retransmission is im- posed. Upon the occurrence of RTO, it checks whether there are

retransmitted packets that are not acknowledged within tauRD, or whether the congestion window is no greater than one. If either is true, rdPkts and cdPkts will be cleared, and all the outstanding packets will be retransmitted at a data rate regulated by the congestion

window.

3.3.4 Kernel Implementation

The diﬃculty in modifying the kernel hinders the implementation of most TCP variants proposed. To facilitate the implementation, we devise a generic framework for developing TCP variants in the Linux/UNIX kernel. We observe that a majority of TCP variants diﬀer in their congestion control and loss recovery operations, which are realized cooperatively by three TCP ma-

51 jor functions2, namely, the TCP input function tcp input, the TCP output function tcp output, and the RTO timer function tcp timer rexmt, as illustrated in Figure 3.3. Thus, a TCP variant can be deployed by: 1) deﬁning its data structures in the TCP control block, 2) maintaining the data structures in the three aforementioned functions, and 3) modifying/replacing the fast retransmit, fast recovery, and/or RTO algorithms.

socket receive system calls buffer tcp_usrreq

tcp_input tcp_output tcp_timer_rxmt

ipintr ip_output softclock

Figure 3.3: Software architecture for TCP.

Following the framework, we have deployed a first version of TCP-NCL . 1) The sk buff structure in TCP control block records all the outstanding packets with timestamps, and each packet has several associated flags that are left unused [63]. The timestamps suffices for the purpose of txTime[i], and the unused flag bits can be set to indicate the membership of a packet in rdPkts and cdPkts. We further define a few extra variables, namely minR, maxR, minR2, maxR2, rcdAge, tauRD, and tauCD in TCP control block. Since each

TCP session maintains one copy of TCP control block, the implementation of TCP-NCL requires storing a small number of extra variables per connection only. 2) In accordance with our algorithms described in Sections 3.3.1-3.3.3,

2The architecture follows that of BSD UNIX [73]. Despite the possible diﬀerences in the naming conventions and function calling sequences, most Linux/UNIX kernels adopt similar structures in their TCP/IP protocol stacks.

52 100 Mbps 1 Mbps S 5 ms BS 50 ms D (a) Infrastructure-based wireless network.

1 Mbps 1 Mbps S 50 ms R1 R3 50 ms D

(b) Multi-hop wireless network.

S1 100 Mbps 100 Mbps D1 5 ms 5 ms 100 Mbps 1Mbps 100 Mbps S2 R1 R2 D2 5 ms 50ms 5 ms ...... 100 Mbps 100 Mbps SN 5 ms 5 ms DN

Figure 3.4: The network topologies used for performance comparison these variables are updated in tcp input based on the incoming acknowledgements, and properly reset in tcp timer rxmt. This requires a constant number of extra maintenance steps per data packet. Again, we stress that the RD and CD timers are approximated via timestamps, and implementing

TCP-NCL does not require maintenance of extra timers. 3) The fast retransmit and fast recovery algorithms are replaced in tcp input by our serialized timer structure. The major change involved is to activate fast recovery upon detecting CD timeout’ed packets (namely, packets that have stayed in cdPkts for more than tauCD) instead of upon activating fast retransmit. The extra overhead involved is minimal.

53 3.4 Performance Evaluation

In this section, we present our simulation results. We have performed extensive simulation experiments to evaluate the performance of TCP-NCL using Network Simulator (ns) 2.29 [19]. We have evaluated several instances of TCP-NCL with diﬀerent settings of K. Recall that K [0, 1] is the system-designed parameter involved in ∈ setting the expiration period of the CD timer (tauCD = (1 K) minR + − · K maxR). A larger value of K generally corresponds to a more aggres- · sive instance of TCP-NCL. In the following, we will focus on two instances of TCP-NCL, TCP-NCL (K=0) and TCP-NCL (K=0.5), with K set to 0 and 0.5, respectively. TCP-NCL (K=0) therefore corresponds to the most conservatively conﬁgured TCP-NCL. We have also included a TCP variant degenerated from TCP-NCL, TCP-

NCL-RD, for comparison. TCP-NCL-RD diﬀers from TCP-NCL in that it disables the CD timer by setting the expiration period of the CD timer to zero. Thus, it proactively postpones packet retransmission until the expiration of the RD timer, when it will assume the packet is lost due to congestion and activates congestion response and packet retransmission simultaneously.

We aim to examine the eﬀect of the serialized-timer structure by comparing TCP-NCL with TCP-NCL-RD. The performance curves of four TCP variants in the literature, namely RR-TCP [81], TCP-DCR [8], TCP-PR [10], and TCPW [13], are replicated from our ongoing study [43] for comparison. The former three variants attain the best goodput performance against packet reordering according to our

54 1 TCP-NCL (K=0.5) TCP-NCL (K=0) TCPW 0.8 TCP-NCL-RD RR-TCP TCP-PR TCP-DCR 0.6

0.4

Connection Goodput (Mbps) 0.2

0 0 0.02 0.04 0.06 0.08 0.1 Packet Error Rate

Figure 3.5: Goodput performance against packet error rate over infrastructure-based wireless network. simulation-based comparison in [77], while TCPW is well known as a solution for wireless packet losses. The simulated network topologies include the infrastructure-based wireless network, the multi-hop wireless network, and the wired network with a dumbbell topology, as illustrated in Figure 3.4. Unless specified otherwise, the queue management algorithm for each link buffer is drop tail. The packet size is 1000 bytes. More detailed network configurations will be described in the following. Sections 3.4.1 and 3.4.2 examine and compare the performance of all the aforementioned TCP variants against non-congestive loss and packet reordering over the infrastructure-based wireless network and the multi-hop wireless network ,respectively. Section 3.4.3 examines the responsiveness of TCP-NCL against congestive loss over the wired network with a dumbbell topology.

55 3.4.1 Non-congestive loss

The infrastructure-based wireless network is illustrated in Figure 3.4 (a). A

TCP sender and a TCP receiver are connected through a wired with bandwidth 100 Mbps and propagation delay 5 ms, and a wireless link with bandwidth bw Mbps and propagation delay d ms. Random packet errors with rate pe are deliberately introduced into the wireless link. Packets experiencing transmission errors are discarded at the receiving end. In each test over the topology, a total of 20 runs, each lasting 2000 seconds and using different seeds for generating the packet error, have been performed to compute an average value and a 95% confidence interval of TCP goodput in Mbps. In order to remove the effect of the transient states, only the statistics in the last 1000 seconds in each run are collected for computing the goodput. Figures 3.5, 3.6, and 3.7 plot the connection goodput of the TCP variants against pe, d, and bw, respectively. In each plot, the legends for TCP variants are sorted descendingly according to their levels of goodput at the rightmost data point. In most cases, the confidence intervals are too narrow to be distinguishable. The configuration settings are summarized in Table 3.3. A hyphen (-) in an entry indicates that the corresponding parameter is being varied to generate the plot. Figure 3.5 exhibits the connection goodputs of the TCP variants against packet error rate pe. The two instances of TCP-NCL attains the best performance. At pe = 1%, TCP-NCL (K=0.5) and TCP-NCL (K=0) attains around 30% and 15% performance improvement over TCP-NCL-RD, respec-

56 1 TCP-NCL (K=0.5) TCPW TCP-NCL (K=0) 0.8 TCP-NCL-RD TCP-DCR TCP-PR RR-TCP 0.6

0.4

Connection Goodput (Mbps) 0.2

0 50 100 150 200 250 300 350 400 Delay of the Wireless Link (ms) (a) pe = 1%.

1 TCP-NCL (K=0.5) TCPW TCP-NCL (K=0) 0.8 TCP-NCL-RD TCP-DCR TCP-PR RR-TCP 0.6

0.4

Connection Goodput (Mbps) 0.2

0 50 100 150 200 250 300 350 400 Delay of the Wireless Link (ms) (b) pe = 5%.

Figure 3.6: Goodput performance against delay of the wireless link over infrastructure-based wireless network. tively. Their performance gains further increase as pe increases, and reach around 200% and 100%, respectively, at pe = 10%. These verify the ef- fectiveness of the installation of the CD timer in combating wireless losses. TCPW attains the third best performance. It performs similarly as TCP-

NCL (K=0) at pe (up to 2%), but performs slightly worse as pe further increases. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates similar drastic decrease in goodput as pe increases. They exclude the pos-

57 7 TCP-NCL (K=0.5) TCPW 6 TCP-NCL (K=0) TCP-PR TCP-DCR 5 TCP-NCL-RD RR-TCP

2 Connection Goodput (Mbps) 1

0 0 1 2 3 4 5 6 7 Bandwidth of the Wireless Link (Mbps) (a) pe = 1%.

7 TCP-NCL (K=0.5) TCPW 6 TCP-NCL (K=0) TCP-PR TCP-DCR 5 TCP-NCL-RD RR-TCP

2 Connection Goodput (Mbps) 1

0 0 1 2 3 4 5 6 7 Bandwidth of the Wireless Link (Mbps) (b) pe = 5%.

Figure 3.7: Goodput performance against bandwidth of the wireless link over infrastructure-based wireless network. sibility of non-congestive packet loss, resulting in severe under-utilization of network resources. Figure 3.6 exhibits the connection goodputs of the TCP variants against propagation delay of the wireless link d, for pe = 1%, 5%. All TCP variants generally perform worse as d increases. TCP-NCL (K=0.5), TCPW, and TCP-NCL (K=0) attain much higher connection goodput than other TCP variants. TCP-NCL-RD, RR-TCP, TCP-DCR, and TCP-PR demonstrates

58 Table 3.3: Network conﬁgurations for the infrastructure-based wireless network

Figure bw d pe Figure 3.5 1 50 - Figure 3.6(a) 1 - 1% Figure 3.6(b) 1 - 5% Figure 3.7(a) - 50 1% Figure 3.7(b) - 50 5%

similar severe performance deterioration as d increases. To fully utilize the wireless link capacity, the size of congestion window (cwnd) has to be at least the bandwidth-delay product, namely the product of bw and the round-trip propagation delay. Such delay increases linearly as d increases. With a larger value of d, it takes a TCP connection longer to resume cwnd to be above the product upon spurious reduction of cwnd due to non-congestive losses. Figure 3.7 exhibits the connection goodputs of the TCP variants against bandwidth of the wireless link bw, for pe = 1%, 5%. TCP-NCL (K=0.5), TCPW and TCP-NCL (K=0) manages to maintain a linear increase in goodput as bw increases up to 4 Mbps, thereby maintaining a constant bandwidth utilization. All other TCP variants essentially maintain constant connection goodput as bw increases, corresponding to a decrease in bandwidth utilization. Similar inferences made from Figure 3.6 can apply here. The bandwidth-delay product increases linearly proportionally to bw. With a larger value of d, it takes a TCP connection longer to recover from spurious cwnd reduction due to non-congestive losses. Up to now, we can observe that the two instances of TCP-NCL attain significant performance gains over TCP-NCL-RD across different configura-

59 tions of bw, d, for non-zero pe, by merely the installation of the CD timers. The serialized-timer structure thus proves to be very eﬀective in shielding the impact of non-congestive losses.

We now draw a closer comparison between TCP-NCL and TCPW. We focus on comparing TCP-NCL (K=0), the most conservatively configured TCP-NCL, with TCPW. Generally, increasing K helps TCP-NCL to avoid spurious reduction of cwnd more effectively. Thus, all inferred advantages of TCP-NCL (K=0) over TCPW in combatting non-congestive losses will safely hold for TCP-NCL configured otherwise. TCP-NCL and TCPW apply distinctly different strategies for performing effective congestion avoidance in the presence of non-congestive losses. TCP-NCL tries to construct more reliable signals of congestive loss by differentiating between congestive loss and non-congestive loss, and halves cwnd when inferring congestive loss. TCPW does not perform the differentiation. Rather, it reduces cwnd to its estimated available network capacity upon each inferred loss. Therefore, we can make three inferences in comparing

TCP-NCL and TCPW:

1. In the presence of non-congestive losses, TCPW tends to trigger spurious reduction of cwnd more frequently than TCP-NCL. This is ex- empliﬁed scenarios with low bandwidth-delay product and high packet

error rates. For example, TCP-NCL (K=0) outperforms TCPW for pe 3%,d = 50, bw = 1 in Figure 3.5. ≥

2. When both perform a spurious reduction of cwnd due to non-congestive losses, TCP-NCL tends to reduce cwnd more abruptly than TCPW.

60 This is exempliﬁed in scenarios with high bandwidth-delay product and low packet error rates. For example, TCPW outperforms TCP- NCL (K=0) for pe = 1%,d 100, bw = 1 in Figure 3.6(a), and for ≥ pe = 1%,d = 50, bw 2 in Figure 3.7(a). ≥

3. The aforementioned two inferences can counteract each other in scenarios with both high bandwidth-delay produce and high packet error rate. For example, TCP-NCL (K=0) performs similarly as TCPW for pe = 5%,d 100, bw = 1 in Figure 3.6(b) and for pe = 5%,d = 50, bw 2 ≥ ≥ in Figure 3.7(b).

3.4.2 Packet Reordering

In the multi-hop wireless network as illustrated in Figure 3.4 (b), a TCP sender is connected to a TCP receiver via six wireless links. Random channel error is introduced into the wireless links with a random packet error rate ranging from 0 to 10%. Packets experiencing transmission errors are discarded at the receiving end of the wireless link. Yet, the setting diﬀers from the infrastructure-based wireless network in that the link layer packets thus discarded are locally retransmitted at the link layer around 110 ms after its most recent (re)transmission. The retransmission period of 110 ms is the sum of two-way propagation delay, (50 ms 2), one-way transmission delay, × (1000 bytes/1 Mbps), and a delay margin of 2 ms. The retransmission will be performed for up to three times. This greatly enhances the chance of successful packet delivery at each hop. Yet, packet reordering will be introduced since the retransmitted packets will be intermixed with later packets.

61 1

0.8

0.6

0.4 TCP-NCL(K=0.5) TCP-PR TCP-NCL(K=0)

Connection Goodput (Mbps) 0.2 TCP-NCL-RD TCPW TCP-DCR RR-TCP 0 0 0.02 0.04 0.06 0.08 0.1 Packet Error Rate (a) Four hop.

0.8

0.6

0.4 TCP-NCL(K=0.5) TCP-PR TCP-NCL(K=0)

Connection Goodput (Mbps) 0.2 TCP-NCL-RD TCPW RR-TCP TCP-DCR 0 0 0.02 0.04 0.06 0.08 0.1 Packet Error Rate (b) Six hop.

Figure 3.8: Goodput performance over multihop ad-hoc wireless network

The intensity of packet reordering will increase as packet error rate increases. Moreover, under high channel error rate, some packets may still fail to be correctly delivered after three link-layer retransmissions. Consequently, TCP will be confronted with both packet reordering and non-congestive packet loss. Figure 3.8 exhibits the connection goodput of the TCP variants against packet error rate over four-hop and six-hop wireless networks. Over the four-hop connection (Figure 3.8(a)), the two instances of TCP-NCL, TCP-

62 PR, and TCP-DCR outperform other variants for pe up to 7%, thereby demonstrating robustness to persistent reordering. TCP-NCL-RD performs slightly worse. It activates congestion response if a packet not on ePkt is not acknowledged tauRD(= maxR + minR) after packet transmission. The time lag is generally shorter than that of TCP-NCL, TCP-PR, and TCP-DCR, leading to a higher chance of premature congestion responses. When pe further increases beyond 7% and non-congestive losses are no longer transparent to the transport layer, TCP-NCL (K=0.5) outperforms

TCP-PR while the performance of TCP-DCR is seriously deteriorated. In such scenario, the installation of the CD timers with a suﬃciently long expiration period helps TCP-NCL (K=0.5) increase the reliability for signals of congestive loss.

Similar inferences can be made for the six-hop connection (Figure 3.8(b)), and TCP-NCL (K=0.5) demonstrates more signiﬁcant advantages over the other variants.

3.4.3 Congestive Loss

In the wired network with a dumbbell topology as illustrated in Figure 3.4(c),

N pairs of TCP senders and receivers are sharing a wired bottleneck link with bandwidth 5 Mbps, propagation delay 50 ms, and buﬀer size 50 packets. For i = 1, 2, ...N, one TCP connection is set up from sender Si to receiver Di. We examine the responsiveness of TCP-NCL against congestive loss over the topology. In general, if a TCP variant, say, TCP-X, is competently responsive against the occurrence of congestive loss, it is expected to fulﬁll the following

63 1.1

1.05

0.95

0.9 SACK TCP TCP-NCL (K=0) TCP-NCL (K=0.5) Jain's Fairness Index 0.85

0.8

0.75 0 10 20 30 40 50 60 70 Number of Connections (a) Drop tail.

1.1

1.05

0.95

0.9 SACK TCP TCP-NCL (K=0) TCP-NCL (K=0.5) Jain's Fairness Index 0.85

0.8

0.75 0 10 20 30 40 50 60 70 Number of Connections (b) RED.

Figure 3.9: Jain’s fairness index (J) over wired network with a dumbbell topology. two objectives:

1. (TCP-friendliness) A connection driven by standardized TCP design (e.g. SACK TCP) can maintain similar goodput when its competing connections are driven by TCP-X as that when its competing connections are driven by standardized TCP design. This ensures that TCP-X

will not grasp bandwidth from standardized TCP in wide current usage.

64 1.2

0.8

0.6

Without TCP-NCL 0.4 With TCP-NCL (K=0) With TCP-NCL (K=0.5)

0.2 Ratio of Mean Goodput to Fair Bandwidth 0 0 10 20 30 40 50 60 70 Number of Connections (a) Drop tail.

1.2

0.8

0.6

Without TCP-NCL 0.4 With TCP-NCL (K=0) With TCP-NCL (K=0.5)

0.2 Ratio of Mean Goodput to Fair Bandwidth 0 0 10 20 30 40 50 60 70 Number of Connections (b) RED.

Figure 3.10: GBR over wired network with a dumbbell topology.

2. (Intra-protocol fairness) Multiple competing connections driven by TCP- X should be able to share links fairly.

For evaluating the TCP-friendliness of TCP-NCL, we conduct three tests with drop tail as the buﬀer management algorithm for the bottleneck link.

In the ﬁrst test, all connections are driven by SACK TCP. In the second and third tests, half of the connections are driven by TCP-NCL (K=0) and TCP-NCL (K=0.5), respectively, with the other half driven by SACK TCP.

65 In each test, suppose connections 1, 2, ..., M are SACK TCP connections and M +1, M +2, ..., N are non-SACK TCP connections (and thus M = N in the ﬁrst test and M = N/2 in the remaining three tests), we compute the goodput to bandwidth ratio (GBR):

1 M xi GBR = M i=1 (3.38) 1 C PN where C denotes the capacity of the bottleneck link and equals 5 Mbps. The GBR is the ratio of average SACK TCP connections to the fair bandwidth for each TCP connection (namely the total link capacity divided by the number of TCP connections), and is comparable across the three tests. Moreover, in the first test, the multiple SACK TCP connections should be able to share the bandwidth efficiently to generate the goodput, thereby attaining a GBR close to one. By examining the GBR obtained in the latter two tests and comparing them with the benchmark GBR obtained in the first case, we can evaluate how the presence of TCP-NCL (K=0)/TCP-NCL (K=0.5) will affect standardized TCP and thus the TCP-friendliness of TCP-NCL. We then repeat the three tests with RED as the buffer management algorithm.

Figure 3.10 plots GBR against N when the buffer management algorithm for the bottleneck link is drop tail and RED. We observe that when competing with TCP-NCL (K=0), the GBR of SACK TCP is generally not affected in most cases. The conservatively configured instance of TCP-NCL demonstrates good TCP-friendliness. On the other hand, when competing with

TCP-NCL (K=0.5), SACK TCP suﬀers up to 10% to 40% decrease in GBR.

66 With a longer expiration period of the CD timer due to large K, some retransmitted packets may be acknowledged before CD timer expires even though the originally packet is lost due to congestion. TCP-NCL (K=0.5) may thus occasionally miss the occurrence of congestive losses, grasping bandwidth yielded by the backoﬀ of SACK TCP and undermining its TCP-friendliness. For measuring intra-protocol fairness, we introduce the Jain’s fairness index [31], J, as the performance metric. For a set of N connections, 1,2,..., N, sharing a bottleneck link with connection goodputs x1, x2, ..., xN , respectively, J is deﬁned as:

( N x )2 J , i=1 i (3.39) N 2 NP i=1 xi P By construction, J is bounded between zero and one, and attains one only when all xi’s are equal. Furthermore, it equals k/N when only k out of the N connections share the bandwidth equally and the remaining (N k) − connections receive a zero bandwidth. Using drop tail as the buﬀer management algorithm for the bottleneck link, we perform three simulation tests with all connections driven by SACK TCP, TCP-NCL (K=0), TCP-NCL (K=0.5), respectively. We then repeat the three tests using RED as the buﬀer management algorithm. Figure 3.9 plots J against N. We observe that all the three variants in comparison attain J close to one. This demonstrates that TCP-NCL is capable of maintaining similar intra-protocol fairness as the standardized TCP design SACK TCP. In particular, even though connections driven by TCP-

NCL (K=0.5) are not very friendly towards connections driven by SACK

67 TCP, they are able to divide the bandwidth fairly among themselves.

3.5 Concluding Remarks

In this chapter, we have proposed a novel TCP variant, known as TCP- NCL, as a unified solution for performing loss recovery, sequencing control, and congestion control over general error-prone channels. In particular, we propose the use of two serialized timers for obtaining more reliable signals for packet loss and network overload separately. The STS model has been constructed based on the concept of expected cost and closed-form expressions are derived as references for setting the timer expiration periods. We note that the timers are mostly determined intuitively in existing work. Our simulation investigations reveal that TCP-NCL is capable of attaining significant performance improvement over general, error-prone channels. It also demonstrates good responsiveness against congestive loss by maintaining effective congestion avoidance and competent TCP-friendliness. There are several possible extensions to our work, some of which are listed below:

1. develop a distribution model for RTT so as to make TCP-NCL a mem-

oryless algorithm, and;

2. implement and examine the performance of TCP-NCL on experimental testbeds.

68 Chapter 4

Enhancing AQM to Combat Wireless Losses

4.1 Chapter Overview

In this chapter, we answer the following basic question that is largely unexplored in the literature: (BQ) Can active queue management (AQM) maintain a stable, small backlog under wireless losses? In wireless networks, TCP tends misinterpret wireless losses as congestive loss and back off spuriously. Nevertheless, when many TCP flows are sharing a bottleneck wireless link, it is unlikely that a large fraction of them will experience wireless losses simultaneously (unless a link is broken). Thus, spurious backoffs due to wireless losses would affect the aggregate transmission of all flows much less severely than when there are only a few flows. The wireless link will still be kept backlogged, as shown by our simulation

69 results.1 In our simulation experiments, we ﬁnd that RED fails to maintain a stable backlog under wireless losses. Apparently, the problem may be made trivial by applying one of the following two approaches:

1. Design a TCP enhancement that can fully diﬀerentiate between congestive and wireless losses. Unfortunately, this is hardly attainable due to the limited network information available to the end systems. Our simulation experiments demonstrate that the performance prob-

lem of RED persists when the end systems are running wireless TCP enhancements like TCP-NCL. As shown by our results in Chapter 3, TCP-NCL is already very successful in diﬀerentiating congestive and wireless losses.

2. Use packet marking instead of packet dropping to signal network congestion, such as [6]. However, we argue that a globally enabled marking scheme is impractical due to heterogeneity of the Internet.2

We therefore opt to tackle the problem through the loss-based AQM enhancements, which make use of packet drops for signalling network congestion. We model the dynamics of the congestion control system around its equilibrium by a linear control system, and study the interaction of wireless losses and AQM in this control-theoretic framework. By employing the queue

1The observation has also been reported in the literature and shaped the design prin- ciples of several enhancements for adapting TCP to high-speed and/or wireless networks. Interested readers can refer to [15, 70] and the references therein. 2For example, explicit congestion notiﬁcation (ECN) [61] is the most widely adopted marking scheme, but is generally disabled by default in several versions of Microsoft Win- dows [17].

70 size as the system output, wireless losses are considered as disturbances to the system. Furthermore, we propose a family of solutions for stabilizing the queue size against wireless losses based on internal model principle [14]. Internal model principle rejects disturbances, or forces the part of the output due to disturbances to asymptotically converge to zero, by including a model of the disturbances, or poles of the disturbances, as part of the feedback control loop. In the case of the congestion control system, it can be realized by simply modifying the formula of the packet dropping rate in AQM. The application of internal model principle in combating wireless losses oﬀers the following advantages:

1. It enables AQM to adjust the packet dropping rate to compensate for the packet drops due to wireless losses without any speciﬁc knowledge about the wireless packet error rate (WPER);

2. It is realized as an independent module in AQM, and thus oﬀers good

extensibility to incorporate other AQM enhancements, such as [76], for improving inter-ﬂow fairness in ad-hoc networks.

By assuming the intensity of wireless losses to be slowly time-varying, we devise the integral controller (IC) as an embodiment of internal model principle. We have developed design rules for the IC. Our simulation results show that IC is capable of maintaining stable queue sizes and performing effective congestion avoidance against wireless losses. Its performance is maintained in the presence of perturbing traffic flows (e.g. HTTP and constant bit rate (CBR) traffic).

71 Section 4.2 discusses the BQ using several simulated examples, and ex- poses the performance problem of RED. Section 4.3 presents our system model, a linear control system obtained by linearizing the non-linear control system of TCP/AQM, as the basis for our analysis. Section 4.4 proposes a family of solutions for enhancing AQM to combat wireless losses based on internal model principle, including the IC. The design rules are developed for the IC. Section 4.5 presents our simulation results. Section 4.6 concludes the paper and discusses some possible extensions to our work. The work presented in this chapter has in parts been published in [37]. Notation conventions: we use bold fonts to denote matrices and vectors, as in “x(t)”, except for the diagonal matrix Λ. When the time index is not speciﬁed, such as “x”, we refer to an equilibrium value. We usex ˜(t) and x˜(t) to represent (x(t) x) and (x(t) x), respectively. λ(.) and σ(.) denotes the − − eigenvalue and singular value of a matrix, respectively.

4.2 AQM under Wireless Losses

In this section, based on a simulation study, we discuss the BQ raised in Section 4.1: Can AQM maintain a stable, small backlog under wireless losses?

S1 100 Mbps 100 Mbps D1 5 ms 1 ms 100 Mbps 100 Mbps S2 5 ms R1 R2 1 ms D2 ...... 100 Mbps 100 Mbps SN 5 ms 1 ms DN

Figure 4.1: A wireless bottleneck link.

72 We use the simulation topology as exhibited in Figure 4.1, similar to [1, 15]. It models the scenario that two wired LANs are connected via a wireless link. A total of N pairs of source and destination are connected via a wireless bottleneck link with capacity 15 Mbps, and propagation delay 50 ms. Within each pair (Si, Di), M long-lived TCP ﬂows are set up from Si to Ni. The size of a link buﬀer is 800 packets. We introduce vary the WPER of the bottleneck link. Time is divided into periods, each of exponential distribution, with a mean of 100 seconds. At the beginning of each period, the WPER of the bottleneck link is determined from a uniform distribution over [0, 4%].3 The error rate is then kept constant during that period, whose duration follows an exponential distribution, and is changed again at the beginning of the next period.

The AQM algorithm simulated is RED. The parameter settings of RED follow the result of frequency domain analysis in [27]. The queue weight and pMax are set to 2.66 10−6 and 0.1, respectively. minThresh and maxThresh · of RED are set to 150 and 700 packets, respectively.

We first consider the following question: Is a wireless link backlogged under wireless losses? Figure 4.2 plots the queueing dynamics when the end systems are running TCP NewReno [21], a standard TCP variant. When there are only 20 flows traversing the bottleneck link as shown in Figure 4.2(a), spurious backoffs of some flows due to wireless losses affect the aggregate transmission of the flows considerably. The aggregate connection goodput is measured to be around 12 Mbps, or 80% of the link capacity, and the back-

3We present simulation results with more exhaustive parameter settings in Section 4.5. Nevertheless, the discussion in the following applies generally.

73 800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (a) N = 20, M = 1.

800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (b) N = 100, M = 1.

800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (c) N = 20, M = 5.

Figure 4.2: Queue length dynamics under varying wireless losses in the dumbbell topology with TCP NewReno/RED.

74 800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (a) N = 20, M = 1.

800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (b) N = 100, M = 1.

800 RED 700 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (c) N = 20, M = 5.

Figure 4.3: Queue length dynamics under varying wireless losses in the dumbbell topology with TCP-NCL/RED.

75 log is zero most of the time. In this case, the major concern is to enhance wireless TCP to improve the link utilization efficiency. When the number of flows increases to 100 by increasing either the number of source-destination pairs, or the number of flows per pair, the aggregate transmission of the flows is able to keep the bottleneck link backlogged and fully utilized, despite the presence of wireless losses. This is clearly exhibited in Figure 4.2(b),(c). A large N naturally corresponds to a large wired LAN. A large M can correspond to multiple simultaneous file transfers, or TCP enhancements that use parallel TCP sockets for better performance over wireless networks, such as [15]. In this case, our major concern shifts from link utilization to the backlog dynamics under wireless losses. We note that the backlog fluctuates severely across time due to the time-varying wireless losses. Thus, the answer to the BQ for RED is negative. Now, we explore some feasible directions for reducing the observed backlog fluctuations. Can TCP be enhanced so as to reduce the backlog fluctuations? Figure 4.3 exhibits the queueing dynamics when the end systems are running TCP-NCL. When there are only 20 flows as shown in Figure 4.3(a), we note that TCP-NCL has improved the link utilization over TCP NewReno. The link is backlogged from the 800th second to the 1300th second, and the aggregate connection goodput is measured to be around 14 Mbps, or 93% of the link capacity.

When the number of flows increases to 100 as shown in Figure 4.3(b),(c), the link is always backlogged. Again, we note that the backlog fluctuates severely, despite the increased accuracy of end systems in differentiating between congestive and wireless loss. This is because it is nearly impossible

76 to fully differentiate between congestive and wireless losses in TCP due to the limited network information available to end systems. Thus, the backlog fluctuation cannot be reduced significantly simply by enhancing TCP only.

The problem should be tackled by enhancing AQM directly. We now take a closer look at the cause of the backlog fluctuations under time-varying wireless losses. In another set of conducted simulations under time-invariant wireless losses, we have observed that different WPER essentially cause the backlog to converge to different levels. Thus, time-varying wireless losses effectively force the backlog to switch between different equilibrium levels from time to time. Moreover, the sluggish transient response of RED in attaining such switching induces further backlog fluctuations. Therefore, the key to maintaining a small, stable backlog under wireless losses is for AQM to direct the backlog to track a fixed reference level despite the variations in the WPER. We now present a systematic approach to achieve this goal.

4.3 System Model

In this section, we present our system model. We start by constructing a non-linear system of TCP/AQM. We then linearize the system around its equilibrium and develop its frequency domain representations. The latter will serve as the basis for further analysis in later sections. We begin by considering a generic communication network with a set of

TCP flows F and a set of bottleneck links L.4 We confine our discussion to 4A non-bottleneck link delays packets for almost a constant amount of time and can thus be modelled as part of the propagation delay experienced by a traffic flow traversing

77 pw ~ + ~ + p b r=0 C(s) + P(s) −

Figure 4.4: Control block diagram of the linearized congestion control system. the case of unique static routing that a Flow f traverses one identiﬁed path

Lf L. ⊆ The non-linear system of TCP/AQM is a time-delay system. There are time lags between when a packet is dropped over Link l, and when the source of Flow f detects the loss, and between when the source of Flow f injects data into the network, and when Link l receives the data. We deﬁne the former as backward delay, ←−dfl(t), and the latter as forward delay, −→dfl(t). Denoting the round-trip time of Flow F as df (t), it can be established that [50]:

−→dfl(t)+ ←−dfl(t)= df (t) (4.1)

We deﬁne the routing matrix incorporated with equilibrium forward delays as:

e−s−→dfl Flow f traverses Link l , [RF (s)]lf  (4.2)  0 otherwise

 and further denote R , RF (0).

For each TCP Flow f, the dynamics of its congestion window Wf (t) evolves in accordance with the aggregate packet dropping probability qf (t). that link.

78 Denote the transmission rate of Flow f as xf (t). By [50], we have:

1 W˙ f (t) = xf (t df )(1 qf (t)) − − Wf (t)

Wf (t) xf (t df )qf (t) (4.3) − − 2

In wireless networks, qf (t) is determined as:

qf (t) Rlf (pl(t ←−dfl)+ pwl(t ←−dfl)) (4.4) ≈ − − l L X∈ where pl(t) and pwl(t) are the packet dropping probabilities over Link l due to AQM and wireless losses, respectively. We note that a packet needs to be stored in the buﬀer managed by AQM, before being transmitted over the link subject to wireless losses. Thus, the delay between packet transmission and packet drop due to wireless losses is slightly greater than the delay between packet transmission and packet drop due to AQM. Nevertheless, we assume the diﬀerence to be negligible for simplicity.

For each link l, the instantaneous queue size bl(t) is determined by the aggregate incoming data rate yl(t) and the link capacity cl, following the system dynamics:

b˙l(t)= yl(t) cl (4.5) −

yl(t) is the sum of the time-delayed source rates of all ﬂows traversing Link

79 l:

Wf (t −→dfl) yl(t) = Rlf xf (t −→dfl) Rlf − (4.6) − ≈ df (t) f F f F X∈ X∈

The approximation above is made based on the assumption that df (t), which is determined by the collected transmission of many flows, varies in a larger time scale than xf (t). Besides data transmission and packet dropping, the dynamics of flows and links are also coupled via the RTTs of the flows. RTT is the sum of queueing delays at links and round-trip propagation delay, τf :

bl(t) df (t) = τf + Rlf (4.7) cl l X

Denote the Laplace transforms of (˜pl(t): l L), (pwl(t): l L), and ∈ ∈ (b˜l(t): l L) as p˜(s), pw(s), and b˜(s), respectively. Following a similar ∈ procedure as [27], we linearize the non-linear system speciﬁed in (4.3), (4.4),

(4.5), and (4.6) around its equilibrium attained with pw(t) 0, and apply ≡ the Laplace transform to the resulting linear system. Deﬁne a plant transfer function matrix, P(s), as the transfer function from (p˜(s)+ pw(s)) to b˜(s), we obtain:

−1 Wf T 1 P(s) = sI + R diag( 2 )R diag( ) − df cl !

−sdf e T RF (s) diag RF ( s) qf Wf 2 · (s + )qf d  − df f   (4.8)

80 P(s) represents the interaction of the TCP dynamics, queueing dynamics, and routing. The stability of this open-loop system can be established by the following lemma.

Lemma 4.1 The system speciﬁed in (4.8) is stable if R is of full row rank.

Wf T 1 Proof It suﬃces to show that all eigenvalues of Ω , R diag( 2 )R diag( ) df cl are negative.

Since eigenvalues are invariant under matrix commutation,

1 Wf T λ(Ω) = λ(diag R diag( 2 )R ) (4.9) {cl } df

By deﬁnition, there exists a unit vector v such that:

1 Wf T λ(Ω)v = diag R diag( 2 )R v (4.10) {cl } df or

T Wf T v R diag( 2 )R v df λ(Ω) = T v diag cl v { } √Wf diag( )RT v 2 k df k2 = 2 0 (4.11) diag √cl v ≥ k { } k2

The equality holds only when RT v = 0, which cannot be attained since RT is full column rank and v = 0. 6

It is time to include AQM into the model. In the time domain, AQM determines p(t) based on b(t). In the frequency domain, it can be represented by a transfer function matrix of dimension L L , C(s), from the queue | |×| | 81 size deviation b˜(s) to the deviation of packet dropping rate by AQM p˜(s). The control loop is therefore closed, as shown in Figure 4.4. The stability of this closed-loop system can be established by the following lemma.

Lemma 4.2 The system in Figure 4.4 is stable if det(I+P(s)C(s)) contains zeros on the open left half plane (OLHP) only.

Proof The closed-loop transfer function from the reference signal, r, to b˜ is given by:

−1 Grb(s)=(I + P(s)C(s)) P(s)C(s) (4.12)

This is stable if the set of poles for Grb(s), which is a subset of the zeros of det(I + P(s)C(s)), are on OLHP.

Finally, the role of wireless losses is represented by pw in Figure 4.4. To- gether with p˜(s), pw(s) determines the system output b˜(s) via P(s). How- ever, while p˜(s) is in turn determined by b˜(s) via C(s) and thus part of the feedback control loop, pw(s) is not in the loop, but serves as an external disturbance to the system.

4.4 Robust AQM based on Internal Model

Principle

In this section, we develop an approach to track the backlog to a reference level despite the presence of wireless losses, based on the frequency domain

82 representation of the TCP/AQM system under wireless losses as shown in Figure 4.4. Per our discussion in Section 4.2, the objective is to maintain a stable, small backlog under wireless losses.

In the time domain, the prescribed objective is met if b˜(t), the backlog deviation from the equilibrium level, converges asymptotically to zero. This means that b˜(s) does not contain any unstable poles in the frequency domain.

We begin by noting the following relationship between b˜ and pw:

−1 b˜(s) = (I + P(s)C(s)) P(s)pw(s) (4.13)

On the other hand, without loss of generality, we can express the disturbance of wireless losses, pw, as:

1 p (s)= k(s) (4.14) w φ(s)

where φ(s) F contains all the unstable poles of pw and k(s) , φ(s)pw(s). ∈ F denotes the set of the rational functions in s. Obviously, all entries in k(s) will not contain any unstable poles. The unstable poles determine the

1 asymptotic behavior of pw(s). Thus, φ(s) is often referred to as a model of the disturbance. It follows that:

1 b˜(s) = (I + P(s)C(s))−1P(s)k(s) (4.15) φ(s)

When the system in Figure 4.4 is stable, the unstable poles of b˜(s), if any, come from the unstable poles contained in φ(s). To attain disturbance rejec-

83 tion of pw, or to prevent the unstable poles contained in φ(s) from becoming poles of b˜(s), the internal model principle proposes to insert the model of

1 disturbance φ(s) inside the feedback loop [14]. The design of AQM determines C(s), whereas the plant transfer function

1 P(s) is invariant across diﬀerent AQM algorithms. Therefore, we put φ(s) inside the feedback loop by selecting C(s) as:

1 C(s)= Λ (s)Λ (s)−1 (4.16) φ(s) N D

|L|·|L| where ΛN (s), ΛD(s) F are coprime diagonal matrices. det(ΛN (s)) does ∈ not share roots with φ(s), and det(ΛD(s)) does not have any roots on closed right half plane. We constrain C(s) to be diagonal so that each intermediate router can execute AQM algorithm distributedly without incurring the need of exchanging the instantaneous queue sizes with each other.

Theorem 4.1 Consider the system in Figure 4.4 with P(s) and C(s) as speciﬁed in (4.8) and (4.16), respectively. Suppose the conditions of Lem- mas 4.1 and 4.2 hold so that both open-loop and closed-loop stabilities are established. Let P(s) = D−1(s)N(s) be a left coprime factorization. The system attains disturbance rejection of (4.14) if det(N(s)) does not contain zeros at the roots of φ(s).

Proof Substituting (4.16) into (4.15) gives:

−1 b˜(s)=ΛD(s)(φ(s)D(s)ΛD(s)+ N(s)ΛN (s)) N(s)k(s) (4.17)

Disturbance rejection is attained if none of the zeros of det(φ(s)D(s)ΛD(s)+

84 N(s)ΛN (s)), which include the poles of b˜(s) as a subset, are on closed right half plane (CRHP). For the roots of φ(s) (which are unstable poles of pw(s) and thus on CRHP), the determinant reduces to:

det(N(s)ΛN (s)) = det(N(s)) det(ΛN (s)) (4.18)

This is non-zero since neither det(ΛN (s)) nor det(ΛN (s)) contains zeros at the roots of φ(s) by assumption. For s s (s) 0 and φ(s) =0 : ∈ { |ℜ ≥ 6 }

det(φ(s)D(s)ΛD(s)+ N(s)ΛN (s))

= det(D(s)) det(ΛD(s))

−1 det(φ(s)I + P(s)ΛN (s)ΛD(s) ) · |L| = φ(s) det(D(s)) det(ΛD(s))

det(I + P(s)C(s)) (4.19) ·

We now consider the three determinants in the ﬁnal expression above one by one. As R is of full row rank, P(s) does not have any unstable poles and thus det(D(s)) does not contain any zeros on CRHP. Neither det(ΛD(s)) nor det(I + P(s)C(s)) contain zeros on CRHP by assumption.

Since CRHP is a subset of s φ(s)=0 s (s) 0 and φ(s) = 0 , { | } { |ℜ ≥ 6 } −1 we conclude that det(φ(s)I + P(s)ΛN (s)ΛD(sS) ) does not contain zeros on CRHP.

Remark 4.1 When det(D(s)) does contain zeros at the roots of φ(s), there will be some cancellations of unstable zero-pole pairs between C(s) and P(s).

85 This violates the condition of total stability, which means that every possible input-output pair of the system is bounded-input-bounded-output (BIBO) stable. It is one of the physical constraints that a practical system should meet.

Interested readers can refer to [14] for details.

Remark 4.2 The attained disturbance rejection is robust against perturbance to P(s), ΛN (s), and ΛD(s), as long as the mild assumptions of Theo- rem 4.1 continue to hold after the perturbance. In practice, TCP AIMD parameters, routing topologies, and network capacities, may change over time, leading to such perturbance. The AQM design speciﬁed in (4.16) can nonethe- less maintain robustness against wireless losses.

Remark 4.3 Indeed, the attained disturbance rejection is not robust against errors in implementing φ(s). Consider when, due to such errors, C(s) becomes

1 −1 C(s)= ΛN (s)ΛD(s) (4.20) φ˜(s) where φ˜(s) is an approximation of φ(s). It follows that:

˜ b1(s) ,

−1 b˜(s) = ΛD(s)(φ˜(s)D(s)ΛD(s)+ N(s)ΛN (s)) N(s)k(s)

z }| b˜2(s) , { ˜ −1 φ(s) φ(s) + ΛD(s)(φ˜(s)D(s)ΛD(s)+ N(s)ΛN (s)) N(s)k(s) − z }| φ(s) { (4.21)

Provided that the assumptions of Theorem 4.1 hold for C(s) speciﬁed in

86 (4.20), b˜1(s) contains stable poles only. b˜2(s) contains the unstable poles of φ(s), and represents the steady state errors in tracking the backlog to a reference size due to the inexact cancellation between φ˜(s) and φ(s).

Nevertheless, it is still very meaningful to apply the internal model principle with inexact implementation of φ(s). This reduces the steady state errors than when the principle is not applied. For example, consider the simple case that φ(s)= s a and φ˜(s)= s a + ǫ, where a and ǫ represent an unstable − − pole and the error involved in implementing φ(s), respectively. b˜2(s) then reduces to:

ǫ b˜ (s)=Λ (s)(φ˜(s)D(s)Λ (s)+ N(s)Λ (s))−1N(s)k(s) (4.22) 2 D D N s a − which is proportional to ǫ. Therefore, we can keep the steady state error minimal with a good approximate implementation of φ(s).

Generally, Theorem 4.1 can be applied in two directions. On the one hand, when φ(s) is known5 , we construct an appropriate C(s) to yield a desired disturbance rejection. For example, wireless losses may vary periodi- cally at a frequency, say, 1 , resulting in φ(s)=1 e−T s. An AQM with C(s) T − containing (1 e−T s) as the denominator can potentially help to maintain − a stable, small backlog under wireless losses with a periodic time-varying packet loss rate. This deserves further study in the future. On the other hand, without any speciﬁc knowledge of φ(s), φ(s) can be designed to serve as a good approximation for the general scenarios, thereby

5We note that φ(s) corresponds to the “pattern” in the variation of wireless losses. Knowledge about φ(s) is a much weaker assumption than knowledge about the exact WPER.

87 attaining good performance in disturbance rejection. We are going to investigate the design of such a controller in the next subsection.

4.4.1 IC

Here, we approximate the disturbance pw as a step input function. This approximation is valid as long as the linear system in consideration converges much faster than the variations in the WPER. It follows that φ(s)= s. From (4.16), we can choose the diagonal element of C(s) as:

gl Cll(s)= , 1 l L (4.23) s ≤ ≤| | which is essentially an integral controller (IC). It converts its input into the integral of that input in the time domain. The integral controller will be part of any other C(s) satisfying (4.23) and φ(s)= s, and is the essential element fulﬁlling the internal model principle. Denote:

−1 Wf Rˆ F ( ) , diag cl RF ( )diag (4.24) · { } · { df } 2 ˆ 2 1 σmin(RF (0)) cl ωg , 0.1 min , , l∈L,f∈F df Wf df Wf ! (4.25)

The following proposition gives the design rules for the integral controller.

Proposition 4.1 Consider the system in Figure 4.4 with P(s) and C(s) as speciﬁed in (4.8) and (4.23), respectively. Suppose that R is of full row

88 g rank. Let gl be chosen as . The system is closed-loop stable and attains cl disturbance rejection of pw as speciﬁed in (4.14) with φ(s)= s if:

2 ˆ 4 σmin(RF (0)) 1 g ωg (4.26) ≤ · Rˆ F (0) (maxf∈F df Wf )maxf∈F Wf k k1

In the case of homogeneous ﬂows sharing a bottleneck link, if maxf Wf 2, ≥ a simpliﬁed condition can be obtained as:

3 0.8(N−) g 5 3 (4.27) ≤ d+c

where N−, d+, and c correspond to the lower bound for the number of ﬂows, the upper bound for RTT, and the link capacity, respectively.

Proof By the generalized Nyquist criteria [18], it suﬃces to prove6 that the eigenloci of the return ratio ( P (jω)C(jω)) does not encircle ( 1, 0) on the − − complex plane for ω = 0. By (4.24), We can rewrite ( P(jω)C(jω)) as: 6 −

P(jω)C(jω) − 1 1 1 T −1 = (jωdiag( )+ Rˆ F (0)diag( )Rˆ F (0) ) Rˆ F (jω) ω cl Wf

π exp(j( 2 ωdf )) T diag − − Rˆ F ( jω) diag(Ilcl) ·  2(jω + qf Wf )  − df   (4.28)

If λ(jω) is an eigenvalue of P(jw)C(jw), there exists a normalized vector

6A complete proof should also cover the eigenloci of P (δe−jω)C(δe−jω) for all ω [ 0.5π, 0.5π] as δ 0. Yet, the proof is trivial. − ∈ − →

89 v such that:

λ(jω)

j(− π −ωd )) ∗ ˆ 2 f ˆ T v RF (jω)diag q W RF ( jω) v 2(jω+ f f ) − df ! = ω ∗ 1 ∗ 1 T (jωv diag( )v + v Rˆ F (0)diag( )Rˆ F (0) v) g cl Wf (4.29)

Note that:

2 T σ (Rˆ F ( jω)) Rˆ F (jω)Rˆ F ( jω) max − ≤ k − k1 T Rˆ F (jω) Rˆ F ( jω) ≤ k k1k − k1

= Rˆ F (0) (4.30) k k1

For ω [0,ωg], we obtain: ∈

λ(jω)

∗ −jdf T v Rˆ F (jω)diag Rˆ F ( jω) v 2qf Wf − ≈ ω ∗ 1 T v Rˆ F (0)diag( )Rˆ F (0) v g Wf (4.31)

With the crossover frequency bounded below by ωg and λ(jω) decreasing | | with ω, it suﬃces to have λ(jωg) < 1, which follows from (4.26). | | In the case of N homogeneous ﬂows sharing a bottleneck link of capacity

90 ˆ ˆ T 1 c, RF (0)RF (0) = N . Thus,

2 1 c 0.2 ωg =0.1 min , , = (4.32) l∈L,f∈F d W d NW max d W f f f f f f and (4.26) reduces to

4 g ωg ≤ · (maxf∈F df Wf )maxf∈F Wf 0.8 = 2 (maxf∈F df Wf ) maxf∈F Wf 0.8N 3 = 3 5 (4.33) c df

The ﬁnal line above follows since all the ﬂows are homogeneous.

3 0.8(N−) The RHS of (4.33) is lower bounded by 5 3 and therefore (4.27) serves d+c as a more stringent lower bound.

Remark 4.4 In [26], the proportional-integral (PI) controller is designed in

kl(s+zl) the form of s , which however cannot reduce to the integral controller as speciﬁed in (4.23).

After discretization via the bilinear transform for an IC, the packet dropping rate for each link l can computed as:

pl[k]= pl[k 1] + Gl(bl[k] bl) (4.34) − −

gl where Gl = . fl is the sampling frequency, i.e., the frequency at which fl the queue size is sampled and (4.34) is used to update the packet dropping rate. pl[k] and bl[k] are the step-wise packet dropping rate and queue size,

... S12 S22 ... S11 S1N S21 S2H D31 S31

12.8 Mbps 9.6 Mbps D32 S32 R1 R2 R3 ... 50ms 50ms

... D3K S3K D11D12 ... D1N D21 D22 ... D2H

Figure 4.5: Two wireless bottleneck links. respectively. The computation can be realized as an independent module in AQM, taking the queue length of a buﬀer as input and computing the packet dropping rate due to the IC as the output.

Example 4.1 Consider the scenario illustrated in Figure 4.1, with the wireless link of capacity 15 Mbps and propagation delay 50 ms. A data packet is 1000 bytes long, and thus cl = 1875 packets/second. A total of N = 100 homogeneous TCP ﬂows share the link. Suppose that the equilibrium queue size is bl = 220 packets. It follows that df is around 0.2 seconds.

Applying (4.27) using d+ = 0.2 seconds, N− = N = 100, c = cl = 1875 Packets,

0.8 1003 g · 0.3792 ≤ 0.25 18753 ≈ ·

Example 4.2 Consider another network scenario illustrated in Figure 4.5, which can arise when three wired LANs are inter-connected to create a WAN via wireless links. Three sets of ﬂows F1, F2, and F3 share a network. There are N1 = 60, N2 = 50, and N3 = 20 ﬂows in F1, F2, and F3, respectively.

Flows within the same set are homogeneous. Each of those flows in Fi (i = 1, 2, 3) traverses from Sij to Dij where 1 j 60 for i =1, 1 j 50 for ≤ ≤ ≤ ≤ 92 i =2, and 1 j 20 for i =3. We thus have two wireless bottleneck links, ≤ ≤ whose dynamics are coupled via flows in F3. Suppose that the equilibrium queue sizes of both wireless links are b1 = b2 = 200 packets. Let di and Wi be the equilibrium RTT and window size for each of the flows in Fi, respectively. It follows that:

c = (1600, 1200) Packets/Second

d = (0.18, 0.22, 0.35)T Seconds

W = (4.2, 4.5, 3.2)T Packets

and thus Rˆ F (0) =0.017. k k1 T Now, the matrix Rˆ F (0)Rˆ F (0) becomes:

0.014 0.00065 ˆ ˆ T RF (0)RF (0)   (4.35) ≈ 0.00065 0.015     2 ˆ which gives σmin(RF (0)) = 0.0136. By (4.25) and (4.26),

2 1 0.0136 1200 ωg 0.1 min( , , · )=0.1786 ≈ 0.35 3.2 0.35 4.5 4 0.0136· 1 g 0.1786 · 0.1134 ≤ · 0.017 · (0.35 3.2) 4.5 ≈ · ·

93 4.5 Performance Evaluation

In this section, we evaluate the performance of IC and compare it with RED using ns-2 simulation. This is necessitated by that the robustness and stability of the IC have been established in the context of the linearized congestion control system, which is nonlinear in reality. Sections 4.5.1 and 4.5.2 present results obtained in simulation topologies illustrated in Figures 4.1 and 4.5, respectively. For IC, the packet dropping rate is updated at a frequency of 10 HZ. Nev- ertheless, we have repeated the experiments with a diﬀerent implementation of IC, which updates packet dropping rate at every packet arrival as RED does, with similar results reported. The parameters of IC and RED follow Proposition 4.1 and references [26, 27], respectively. For RED, pMax and the queue weight are set to 0.1 and

2.6 10−6, respectively. The parameters minT hresh and maxT hresh of × RED, and the gain of IC, Gl, vary and will be speciﬁed in the following.

4.5.1 One Wireless Bottleneck Link

In the topology illustrated in Figure 4.1, a total of N pairs of source and destination are connected via a wireless bottleneck link with capacity c Mbps, and propagation delay τ ms. Within each pair (Si, Di), M long-lived TCP

Reno ﬂows are sent from Si to Di. The size of the link buﬀer and the reference queue size of IC are set to 800 Packets and 220 Packets, respectively. We introduce variable WPER into the bottleneck link. Time is divided into periods, each of exponential distribution, with a mean of lossP seconds. At

94 the beginning of each period, the WPER of the bottleneck is determined randomly from a uniform distribution over [0, mL]. Figures 4.6-4.9 plot the traces of queue sizes under various configurations summarized in Table 4.1. The ”HTTP” and ”CBR” columns in Table 4.1 indicates whether there are HTTP and CBR traffic flows coexisting with the long-lived TCP flows, respectively. The final three columns are parameter settings of IC and RED and computed from the network settings.

95 Table 4.1: Network conﬁgurations for the single wireless bottleneck network.

Figure N M c τ mL lossP HTTP CBR Gl minThresh maxThresh Figure 4.6(a) 100 1 15 50 4% 100 No No 2 10−5 150 700 · Figure 4.6(b) 20 5 15 50 4% 100 No No 2 10−5 150 700 · Figure 4.7(a) 20 5 15 50 8% 100 No No 2 10−5 150 700

96 · Figure 4.7(b) 20 5 15 50 4% 20 No No 2 10−5 150 700 · Figure 4.8(a) 20 5 15 200 4% 100 No No 2 10−7 150 400 Figure 4.8(b) 20 10 15 200 4% 100 No No 1.6· 10−6 150 400 Figure 4.8(c) 20 10 30 50 4% 100 No No 1 ·10−5 150 400 · Figure 4.9(a) 20 5 15 50 4% 100 Yes No 2 10−5 150 700 Figure 4.9(b) 20 5 15 50 4% 100 No Yes 2 · 10−5 150 700 · Figure 4.9(c) 20 5 15 50 4% 100 Yes Yes 2 10−5 150 700 · 800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (a) N = 100, M = 1.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (b) N = 20, M = 5.

Figure 4.6: Comparison of queue length dynamics with same conﬁgurations as in Figure 4.2.

Figure 4.6(a) and Figure 4.6(b) essentially retain the same conﬁgurations as Figure 4.2 and Figure 4.2(b), respectively. We observe that IC helps maintain the queue size stable throughout the simulation period. Despite the variations in WPER, the equilibrium queue size of IC tracks the reference level (220 Packets) very well. We do observe occasional “spikes” in the queueing dynamics with IC. This is because IC needs to converge to a new equilibrium packet dropping rate to maintain the same equilibrium queue size when the

97 WPER changes. Nevertheless, the transition is much smoother than that of RED since the equilibrium queue size is unchanged and IC has better transient response than RED. Moreover, while the diﬀerence between the two equilibrium packet dropping rate should be approximately the change in the WPER, IC is self-adaptive and does not require any knowledge about the WPER to complete the transition. We note that this is an essential property of wireless AQM enhancements developed based on the internal model principle.

In addition, Figure 4.6(a) and Figure 4.6(b) exhibit similar queueing dynamics. Generally, in all our conducted simulation experiments, the queueing dynamics are quite similar across diﬀerent values of M for 1 M 10, as ≤ ≤ long as the total number of ﬂows, NM, is kept constant.

Figure 4.7(a) and Figure 4.7(b) exhibit the queueing dynamics against wireless losses with larger variations and more frequent variations in WPER, respectively. Correspondingly, RED exhibit more severe and more frequent ﬂuctuations, respectively. IC basically maintains a stable queue size around the reference level in both cases, except that occasional spikes due to transition becomes larger in Figure 4.7(a). This is as expected, since more abrupt “jumps” in the WPER requires IC to undergo a greater change in the equilibrium packet dropping rate. Figure 4.8(a)-(c) exhibit the queueing dynamics with “fatter” pipes, i.e., a bottleneck link of larger propagation delay or higher bandwidth. Fig- ure 4.8(a) increases the propagation delay to 200 ms, approximately the two-way propagation delay with a geostationary satellite [3], and maintains the total number of ﬂows to be 100. We observe that queue dynamics tend

98 800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (a) Larger variations in WPER.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (b) More frequent variations in WPER.

Figure 4.7: Comparison of queue length dynamics under diﬀerent wireless losses.

99 800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (a) Fatter pipe due to increased link delay.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (b) Fatter pipe due to increased link delay is shared by more TCP ﬂows.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 200 400 600 800 1000 1200 1400 Time (second) (c) Fatter pipe due to increased bandwidth is shared by more TCP ﬂows.

Figure 4.8: Comparison of queue length dynamics with different bottleneck link parameters. 100 to fluctuate very severely with both IC and RED. In fact, the link is not backlogged for significant portion of time. The congestion window of TCP flows regulate the number of packets in transit. It grows larger with fatter pipes, since more packets in transit are needed to keep the pipe full. When a packet is lost due to wireless losses, the corresponding flow will spuriously reduce its window by half. With fatter pipes, the window reduction in terms of packets becomes larger. This leads to more severe fluctuations in the number of packets in transit, which will possibly fall below the size of the pipe ( 2cτ) and leave the link not ≈ backlogged. We recall that, when a link is not backlogged, it is outside the operational region of wireless AQM enhancements. To mitigate the effect of fatter pipes, we can increase the number of

TCP flows so that the congestion window of a single flow becomes smaller. Figure 4.8(b) plots the queueing dynamics when the total number of flows is increased to 200. In this case, we observe that the link is backlogged throughout the simulation, and IC can effectively maintain a stable queue size. Similarly, Figure 4.8(c) plots the queueing dynamics when 200 flows are sharing a bottleneck link of doubled bandwidth, in which IC still demonstrate a robust tracking of the reference queue size. The backlog fluctuation with RED is not very significant in this case due to the reduction in maxT hresh as computed. maxT hresh is the maximum level of queue size allowed by RED. However, such reduction is generally not recommended in other scenarios due to the stability requirement of RED [27]. Figure 4.9(a)-(c) exhibit the queueing dynamics with the presence of other

101 800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 500 1000 1500 2000 2500 3000 Time (second) (a) HTTP.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 500 1000 1500 2000 2500 3000 Time (second) (b) CBR.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 500 1000 1500 2000 2500 3000 Time (second) (c) HTTP and CBR.

Figure 4.9: Comparison of queue length dynamics with the presence of other traﬃc.

102 traffic. Figure 4.9(a) plots the queueing dynamics when we introduce the HTTP traffic into the dumbbell topology. In doing so, we attach 120 web client nodes to R1 and ten web server nodes to R2. We can see that the traces of the queue size are more “noisy”, since the introduction of the HTTP traffic leads to a higher fluctuation on the network load. Nevertheless, IC demonstrates an effective and robust tracking of the reference queue level against the disruptions from both perturbing traffic and wireless losses. Figure 4.9(b) plots the queueing dynamics when we introduce CBR traf-

fics into the dumbbell topology. In doing so, we add attach 30 UDP sources to R1 to and their corresponding destinations to R2, with each UDP flow driven by long-lived CBR at the constant rate of 0.1 Mbps. The UDP flows represent unresponsive flows that do not back off upon network congestion.

With RED, the backlog generally shifts up due to the increased long-lived traffic loading. On the other hand, the robust tracking by IC of the reference queue level maintains despite the presence of unresponsive flows. Figure 4.9(c) plots the queueing dynamics when both HTTP and CBR traffic are present. We can identify the aforementioned impact of introducing HTTP and CBR traffic. With RED, the backlog trace becomes more “noisy” and shifts up as well. With IC, the backlog becomes more “noisy” but still tracks the reference level.

4.5.2 Two Wireless Bottleneck Links

In the topology illustrated in Figure 4.5, network conﬁguration follow those in Example 4.2. The size of the two link buﬀers are both 600 packets. For

103 800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 500 1000 1500 2000 2500 3000 Time (second) (a) Link 1.

800 RED 700 IC 600 500 400 300 200 100

Queue length (number of packets) 0 0 500 1000 1500 2000 2500 3000 Time (second) (b) Link 2.

Figure 4.10: Comparison of queue length dynamics in the topology of two wireless bottleneck links.

104 both links, minThresh and maxThresh of RED are set to 150 and 500 packets,

−6 −6 respectively. Gl’s are set to be 7 10 and 9.5 10 for Link 1 and Link · · 2, respectively. We introduce time-varying losses into both links, with an average loss period of 100 s and a maximum WPER of 3 %. The simulation results are plotted in Figure 4.10. Again, severe ﬂuctuations can be observed in the queue sizes with RED over both Link 1 and Link 2. On the other hand, the IC is successful in maintaining stable queue sizes throughout the simulation.

4.6 Concluding Remarks

In this chapter, centered around the BQ “Can AQM maintain a small, stable queue under wireless losses?”, we carry out a series of study. We ﬁnd that RED, one of the most representative AQM, fail to maintain a stable queue size against time-varying wireless losses. The key to solving the problem is for the AQM to force the queue size to track a reference queue level despite the presence of wireless losses. Based on the frequency domain representation of TCP/AQM, we apply the internal model principle to reject the disturbance of wireless losses, thereby attaining such tracking. We devise the integral controller (IC) as an embodiment of the internal model principle, and develop a set of design rules for IC. We have conducted a series of experiments to examine the performance of IC and compare it with RED. In particular, we have evaluated the impact of severe variations of wireless losses, fat pipes, and HTTP traﬃcs on the two AQM algorithms. In general, IC demonstrates very robust performance in maintaining a small, stable backlog under wireless

105 losses. There are several possible extensions to our work, including:

1. developing a global stability analysis for the system of TCP/buﬀer-

based AQM perturbed by wireless losses;

2. applying Theorem 4.1 to design an AQM that attains clean disturbance rejection of wireless losses with known φ(s), as discussed in Section 4.4, and;

3. developing a uniﬁed AQM solution for resolving both fairness and stability issues in ad-hoc wireless networks.

106 Chapter 5

Conclusions and Future Research

5.1 Chapter Overview

In this chapter, we conclude the thesis and discusses some future research directions. Section 5.2 discusses main results acquired and summarizes the major contributions of the thesis. Section 5.3 discusses some topics worth further investigation. Section 5.4 concludes the chapter with a discussion.

5.2 Contributions

In Chapter 2, we conduct a literature survey of the work performed to date on wireless TCP enhancements and on design and analysis of AQM. We ﬁnd that there is a lack of transport layer uniﬁed solution for enhancing TCP against packet reordering and non-congestive loss. We identify that the performance

107 of AQM under wireless losses is largely unexplored in the literature. In Chapter 3, we have developed a ideal TCP sender model, known as smart TCP sender (STS), for constructing more reliable signals of packet loss and network congestion over general, error-prone channels. The two signals employed are the expirations of two serialized timers, where the second timer is started upon the expiration of the ﬁrst timer. The timer expiration periods have been optimized for minimizing quantiﬁed risks associated with spurious congestion response and excessively delayed congestion response.

We have devised a novel TCP variant, known as TCP for non-congestive loss (TCP-NCL), as a practical approximation of the STS model. TCP-NCL can thus serve as a uniﬁed solution for eﬀective congestion control, sequencing control, and loss recovery over wireless networks. The deployment of

TCP-NCL requires modiﬁcation to sender-side TCP only, thereby facilitating possible future wide deployment. The deployment of TCP-NCL requires maintenance of a few unused bits in the existing data structure for TCP and a few extra variables only. The maintenance is performed upon arrival of acknowledgement packet. We have performed extensive simulation experiments to examine the performance of TCP-NCL and compare it with several well-known TCP variants in the literature using Network Simulation (ns) 2.29. Our results report that TCP-NCL attains signiﬁcant performance improvement over general, error- prone communication networks, thereby demonstrating robustness against packet reordering and non-congestive packet loss. At the same time, multiple competing TCP connections, with some employing TCP-NCL and the others employing standardized TCP design, are capable of sharing bandwidth fairly

108 over conventional wired networks. TCP-NCL is therefore responsive against congestive loss, and backward compatible with the present TCP Internet. In Chapter 4, we raise the basic question: Can AQM maintain a stable, small backlog under wireless losses? Our simulation results report that, while many TCP flows can keep a wireless link backlogged despite spurious backoffs, RED fails to maintain a stable backlog under time-varying wireless losses. Due to the fundamentally limited information on network loading available to end systems, the backlog stabilization should be best attained via AQM enhancements. We further identify that the key to resolving the problem is to robustly track the backlog to a preset reference level. Based on a linear model of TCP/AQM, we apply the internal model principle to realize such tracking. The application of internal model principle in combating wireless losses enables AQM to adjust the packet dropping rate to compensate for the packet drops due to wireless losses without any specific knowledge about the wireless packet error rate (WPER). The principle can be realized as an independent module in AQM, and thus offers good extensibility to incorporate other AQM enhancements. By assuming the intensity of wireless losses to be slowly time-varying, we devise the integral controller (IC) as an embodiment of internal model principle. We have developed design rules for IC. Our simulation results show that IC is capable of maintaining stable queue sizes and performing effective congestion avoidance against wireless losses. Its performance is maintained in the presence of perturbing traffic flows (e.g. HTTP traffic). The major contributions of the thesis are recapped as follows:

1. A TCP sender model for constructing reliable signals of packet loss and

109 network congestion over general, error-prone communication networks

Based on the novel serialized-timer structure, the STS model constructs reliable signals of packet loss and network congestion. It attains an optimal tradeoﬀ between signal reliability and responsiveness against congestion by minimizing quantiﬁed risks associated with spurious con-

gestion response and excessively delayed congestion response.

2. A uniﬁed solution for eﬀective congestion control, sequencing control, and loss recovery over wireless networks

As a transport layer solution, TCP-NCL enables accurate diﬀerentia- tion among the occurrences of packet reordering, congestive loss, and non-congestive loss. It signiﬁcantly enhances robustness against packet reordering and non-congestive loss while maintaining responsiveness

against congestive loss.

3. A simulation-based investigation of the performance of AQM under wireless losses

The series of simulation experiments on the performance of RED under time-varying losses reveal the irrobustness of RED. It helps to clarify the diﬀerent operational regions of TCP and AQM enhancements. Due

to the fundamentally limited information on network loading available to end systems, the backlog stabilization should be best attained via AQM enhancements. The results further indicate that the key to maintaining a stable backlog is to robustly track the backlog to a preset

reference level.

110 4. A systematic AQM enhancement for backlog stabilization under wireless losses

The application of internal model principle enables AQM to adjust the packet dropping rate to compensate for wireless losses without requiring exact knowledge of the WPER, thereby maintaining a stable backlog

under time-varying wireless losses. This signiﬁcantly improves end-to- end delay performance and fairness towards bursty traﬃc.

5.3 Future Research

Our major results, as summarized in Section 5.2, demonstrate that the In- ternet congestion control fulﬁlled by TCP and AQM can be systematically adapted to the wireless environment at a minimal deployment cost and disruptions, preserving the loss-based nature. However, there remain several research directions worth thorough investigation, such as those enlisted at the end of Chapters 3 and 4. We discuss two topics in greater depth in the following.

5.3.1 A Theoretical Framework for Wireless TCP

The literature on wireless TCP enhancements is vast [44]. Various propos- als generally adapt TCP to the dynamic wireless environment by exploit- ing readily available or self-constructed signals of the occurrences of various events. For example, the network loading conditions can be inferred from

RTT, packet losses, or explicit network feedbacks.

111 However, a theoretical framework is lacking in at least two aspects. First, it would be highly desirable to compare the various signals in terms of quantified metrics and operational regions. For example, RTT provides richer information about network loading than packet losses, thereby enabling finer tuning of end system transmission. The merit is particularly useful in alle- viating the fluctuation of TCP in high-speed networks [71]. On the other hand, RTT can be varied by non-congestive factors in many scenarios such as reordering networks, thereby undermining its reliability as compared with packet loss. A systematic categorization and quantified characterization of these signals would vastly facilitate understanding about the strength and limitations of existing TCP variants, and development of new TCP variants that push towards fundamental limits.

Second, wireless TCP enhancements generally modify TCP at a finer granularity, or in a more general manner, than have been modeled in the literature. For example, many TCP variants opt to proactively postpone packet retransmission and congestion response until congestive loss is prob- able. In the popular control-theoretic formulation [33], the delay would not change the converged equilibrium as long as the stability condition is observed. Yet, the simulation findings reveal that it requires extra effort to maintain TCP-friendliness of such TCP variants [10]. The discrepancy is partly because the sophisticated impact of RTO remains to be taken into account. Consequently, the TCP-friendliness and fairness of wireless TCP enhancements have to be examined via extensive simulation experiments, which cannot exhaust all possible network scenarios in reality. As another example, some recent results on a network with heterogeneous congestion

112 control [67, 68], namely a network with multiple congestion control schemes based on diﬀerent signals, assume each congestion control scheme utilizes one type of signal only. However, TCP variants, such as TCP-NCL proposed in this thesis, tends to exploit information conveyed by the mix of multiple signals to attain better performance. Therefore, it remains to develop theoretical understanding regarding the dynamics and equilibrium of a network shared by TCP variants. This helps to clarify the set of constraints a TCP proposal needs to meet to ensure inter-ﬂow fairness.

The aforementioned two aspects are expected to be complementary. The systematic study of signals available to end systems essentially views the network as a black box and focuses on optimizing a single TCP ﬂow. This hopefully will give rise to a generalized expression of TCP dynamics, and provides a basis for studying the network of TCP variants. A better understanding of the equilibrium and dynamics of such networks, on the other hand, provides further performance metrics for our study of the signals.

5.3.2 A Uniﬁed AQM Solution for Wireless Networks

Wireless networks bring unique challenges to AQM that are not covered by the vast literature on the design and analysis of AQM in wired networks. First, as reported in Chapter 4, different WPERs can cause the backlog to converge to different levels, and time-varying WPER can induce fluctuations in backlog. To the best of our knowledge, our work presented in Chapter 4 is the first investigation on whether and how AQM can maintain a stable backlog under wireless losses.

113 Second, in wireless ad-hoc networks, transmission at adjacent nodes may interfere with each other. The implication on congestion control is two- fold. On the one hand, TCP flows compete for “space”, namely the chance for transmission within an interfering range, in addition to competing for wireless bandwidth. Thus, backlogs at interfering nodes need to be aggre- gately managed by a queue management algorithm to facilitate fair allocation among competing flows [76]. On the other hand, packet losses occur due to channel contention, in addition to router buffer overflow, and both should be interpreted as signals of network congestion. However, unlike losses due to buffer overflow, contention losses may saturate when the offered network load exceeds a threshold value [25]. Consequently, the TCP flows tend to push the network beyond the optimal loaded point, leading to degraded throughput.

AQM should thus be in place to deliberately tune up the packet dropping rate to force earlier backoff of the TCP flows. A unified AQM solution for resolving issues induced by wireless losses, channel contention and other special characteristics of wireless networks is much more than a trivial combination of the solutions proposed for each issue. In particular, to stabilize the backlog under wireless losses we need to tune down packet dropping rate to various levels to compensate for the variation in wireless packet error rate, whereas to force TCP flows to operate at the optimal loading point in ad-hoc networks we need to tune up packet dropping rate to force earlier backoffs in TCP. A possible approach to attain both objectives is to make use of some occurrences of wireless losses to deliberately induce backoffs in TCP to restore the optimal loading point, and at the same time to adjust the packet dropping rate of AQM to compensate for variations

114 of wireless losses. Again, the exact knowledge of wireless packet error rate should not be assumed, or the adaptiveness to the ever-changing wireless environment would be sacriﬁced. The approach can thus be best fulﬁlled on the basis of a valid model of TCP/AQM under wireless losses and channel contentions, which itself is a challenging task to be accomplished.

5.4 Concluding Remarks

In this thesis, we aim to systematically adapt the present Internet congestion control to the wireless environment at a minimal deployment cost and disruption. Our adaptations include TCP and AQM enhancements, which jointly help the heterogeneous wired/wireless Internet to operate in the optimal region of low delay and high throughput. On the TCP side, we have developed the STS model that uses a serialized- timer structure to offer more reliable signals of packet loss and network congestion over general, error-prone channels. We have devised TCP-NCL as a practical approximation of STS, which can serve as a unified solution for effective congestion control, sequencing control, and loss recovery over wireless networks. The deployment of TCP-NCL requires sender-side modification only. On the AQM side, we have investigated the performance of AQM under wireless losses using simulation experiments. Our results report that RED fail to maintain a stable backlog against time-varying WPER. We have applied the internal model principle to stabilize the backlog. We further devise IC as an embodiment of the principle, which performs robustly against time-

115 varying WPER in various simulated challenging scenarios. There are several research directions in wireless Internet congestion control worth future thorough investigation, for which the work completed in this thesis provide a solid basis. These include a theoretical framework for the design and analysis of wireless TCP enhancements, and a uniﬁed AQM solution for wireless networks.

116 Bibliography

[1] I.F. Akyidiz, C. Morabito, and S. Palazzo. TCP-Peach: A New Conges- tion Control Scheme for Satellite IP Networks. IEEE/ACM Transactions on Networking, Vol. 9, No. 3, Jun. 2001. [2] M. Allman, H. Balakrishnan, and S. Floyd. Enhancing TCP’s Loss Re- covery using Limited Transmit. IETF RFC 3042, Jan. 2001. [3] M. Allman, D. Glover, and L. Sanchez. Enhancing TCP Over Satellite Channels using Standard Mechanisms IETF RFC 2488, Jan. 1999. [4] M. Allman, V. Paxson, and E. Blanton. TCP Congestion Control. IETF RFC 5681, Sep. 2009. [5] M. Allman, V. Paxson, and W. Stevens. TCP Congestion Control. IETF RFC 2581, Apr. 1999. [6] S. Athuraliya, S.H. Low, V.H. Li, and Q. Yin. REM: Active Queue Management. IEEE Network, Vol. 15, No. 3, pp. 48-53, May 2001. [7] H. Balakrishnan, V.N. Padmanabhan, S. Seshan, and R.H. Katz. A Comparison of Mechanisms for Improving TCP Performance over Wire- less Links. IEEE/ACM Transactions on Networking, Vol. 5, No. 6, pp. 756-769, Dec. 1997. [8] S. Bhandarkar and A.L.N. Reddy. TCP-DCR: Making TCP Robust to Non-Congestion Events. Lecture Notes in Computer Science, Vol. 3042, pp. 712-724, May 2004. [9] E. Blanton and M. Allman. On Making TCP More Robust to Packet Re- ordering. ACM SIGCOMM Computer Communication Review, Vol. 32, Issue 1, pp. 20-30, Jan. 2002. [10] S. Bohacek, J.P. Hespanha, J. Lee, C. Lim, and K. Obraczka. A New TCP for Persistent Packet Reordering. IEEE/ACM Transactions on Networking, Vol. 14, No. 2, pp. 369-382, Apr. 2006.

117 [11] B. Braden, D. Clark, J. Crowsoft, B. Davie, S. Deering, D. Estrin, S. Floyd, V. Jacobson, G. Minshall, C. Patridge, L. Peterson, K. Ra- makrishnan, S. Shenker, J. Wroclawski, and L. Zhang. Recommenda- tions on Queue Management and Congestion Avoidance in the Internet. IETF RFC 2309, Apr. 1998.

[12] L. Brakmo and L. Peterson. TCP Vegas: End to End Congestion Avoid- ance on a Global Internet. IEEE Journal on Selected Areas in Commu- nication, Vol 13, No. 8, pp. 1465-1480, Oct. 1995.

[13] C. Casetti, M. Gerla, S. Mascolo, M.Y. Sanadidi, and R. Wang. TCP Westwood: End-to-End Congestion Control for Wired/Wireless Net- works. Wireless Networks, Vol. 8, No. 5, pp. 467-479, 2002.

[14] C.-T. Chen. Linear System Theory and Design. New York: Oxford Uni- versity Press, 1999.

[15] M. Chen and A. Zakhor. Flow Control over Wireless Network and Appli- cation Layer Implementation. Proc. of IEEE INFOCOM, pp. 103-113, Mar. 2006.

[16] D.M. Chiu and R. Jain. Analysis of the increase and decrease algorithms for congestion avoidance in computer networks. Computer Networks and ISDN Systems, Vol. 17, No. 1, pp. 1-14, 1989.

[17] J. Davis. New Networking Features in Windows Server 2008 and Windows Vista. http://technet.microsoft.com/en- us/library/bb726965.aspx. Feb. 2008.

[18] C.A. Desoer and Y.T. Wang. On the Generalized Nyquist Stability Cri- terion. IEEE Transactions on Automatic Control, Vol. 25, No. 2, pp. 187- 196, Apr. 1980.

[19] K. Fall and K. Varadhan. The ns Manual (formerly ns Notes and Doc- umentation). The VINT Project, 6 January 2009.

[20] X. Fan, M. Arcak, and J.T. Wen. Robustness of Network Flow Control against Disturbances and Time-Delay. System & Control Letters, Vol. 53, No. 11, pp. 1329, Sep. 2004.

[21] S. Floyd, T. Henderson, and A. Gurtov. The NewReno Modiﬁcation to TCP’s Fast Recovery Algorithm. IETF RFC 3782, Apr. 2004.

118 [22] S. Floyd and V. Jacobson. Random Early Detection Gateways for Con- gestion Avoidance. IEEE Transactions on Networking, Vol. 1, No. 4, Aug. 1993.

[23] S. Floyd, J. Mahdavi, M. Mathis, and M. Podolsky. An Extension to the Selective Acknowledgment (SACK) Option for TCP IETF RFC 2883, July 2000.

[24] C.P. Fu and S.C. Liew. TCP Veno: TCP Enhancement for Transmis- sion over Wireless Access Networks. IEEE Journal on Selected Areas in Communication, Vol. 21, No. 2, pp. 216-228, Feb. 2003.

[25] Z. Fu, P. Zerfos, H. Luo, S. Lu, L. Zhang, and M. Gerla. The Impact of Multihop Wireless Channel on TCP Throughput and Loss. Proc. of IEEE INFOCOM, Mar. 2003.

[26] H. Han, C.V. Hollot, Y. Chait, and V. Misra. TCP Networks Stabilized by Buﬀer-Based AQMs. Proc. of IEEE INOFOCOM, Mar. 2004.

[27] C.V. Hollot, V. Misra, D. Towsley, and W.-B. Gong. Analysis and Design of Controllers for AQM Routers Supporting TCP Flows. IEEE Trans- actions on Automatic Control, Vol. 47, No. 6, June 2002.

[28] F. Hu and N. Sharma. Enhancing Wireless Internet Performance. IEEE Communication Surveys and Tutorials, Vol. 4, No. 1, pp. 2-15, Dec. 2002.

[29] Institute of Electrical and Electronics Engineers. IEEE Standard for Local and Metropolitan Area Networks Part 16: Air Interface for Fixed Broadband Wireless Access Systems. New York: Institute of Electrical and Electronics Engineers, 2009.

[30] V. Jacobson Congestion Avoidance and Control. ACM SIGCOMM Com- pututer Communication Review, Vol. 25, No. 1, pp. 157-187, Jan. 1995.

[31] R. Jain, D.M. Chiu, and W. Hawe. A Quantitative Measurement of Fairness and Discrimination for Resource Allocation in Shared Systems. Technical Report DEC-TR-301, Degital Equipment Corporation, 1984.

[32] D. Katabi, M. Handley, and C. Rohrs. Congestion Control for High Bandwidth-delay Product Networks. ACM SIGCOMM Computer Com- munication Review, Vol. 32, No. 4, Oct. 2002.

119 [33] F. Kelly, A. Maulloo, and D. Tan. Rate Control in Communication Net- works: Shadow Prices, Proportional Fairness and Stability. Journal of the Operational Research Society, Vol. 49, No. 3, pp. 237-252, Mar. 1998. [34] H.K. Khalil. Nonlinear Systems. Upper Saddle River, New Jersey: Prentice-Hall, 2002. [35] A. Lahanas and V. Tsaoussidis. Improving TCP Performance over Networks with Wireless Components using ’Probing Devices’. Interna- tional Journal of Communication Systems, Vol. 15, No. 6, pp. 495-511, July/Aug. 2002. [36] C. Lai, K.-C. Leung, and V.O.K. Li. Demo Abstract: A Testbed for TCP in Heterogeneous Wired/Wireless Networks. Proc. of IEEE INFOCOM 2011, Apr. 2011. [37] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing AQM to Combat Wire- less Losses. Submitted. [38] C. Lai, K.-C. Leung, and V.O.K. Li. Enhancing Wireless TCP: A Serialized-Timer Approach. Proc. of IEEE INFOCOM 2010, Mar. 2010. [39] C. Lai, K.-C. Leung, and V.O.K. Li. TCP-NCL: A Uniﬁed Solution for TCP Packet Reordering and Random Loss. Proc. of IEEE PIMRC 2009, Sep. 2009. [40] T.V. Lakshman and U. Madhow. The The Performance of TCP/IP for Networks with High Bandwidth-Delay Products and Random Loss. IEEE/ACM Transactions on on Networking, Vol. 5, No. 3, June 1997. [41] D. Lapsley and S.H. Low. Random Early Marking for Internet Conges- tion Control. Proc. of IEEE Globecom, pp. 1747-1752, Dec. 1999. [42] G.S. Lee, L.L.H. Andrew, A. Tang, and S.H. Low. WAN-in-Lab: Mo- tivation, Deployment and Experiments. Proc. of PFLDnet, pp. 85-90, Feb. 2007. [43] K.-C. Leung, C. Lai, V.O.K. Li, and D. Yang. A Packet Reordering Solution to Wireless Losses in Transmission Control Protocol (TCP). Submitted. [44] K.-C. Leung and V.O.K. Li. Transmission Control Protocol (TCP) in Wireless Networks: Issues, Approaches, and Challenges. IEEE Commu- nication Surveys and Tutorials, Vol. 8, No. 4, pp. 64-79, Fourth Quar- ter 2006.

120 [45] K.-C. Leung, V.O.K. Li, and D. Yang. An Overview of Packet Reorder- ing in Transmission Control Protocol (TCP): Problems, Solutions, and Challenges. IEEE Transactions on on Parallel and Distributed Systems, Vol. 18, No. 4, pp. 522-535, Apr. 2007

[46] V.O.K. Li, L. Cui, Q. Liu, G.-H. Yang, Z. Zhao, and K.-C. Leung. A Heterogeneous Peer-to-Peer Network Testbed. Proc. of ICUFN, pp. 46- 51, Jun. 2009.

[47] V.O.K. Li, C. Lai, and K.-C. Leung. Diﬀerentiation among Occurrences of Packet Reordering, Congestive Packet Loss, and Random Packet Loss in Communication Networks. United States Regular Application No. 12/879,880, Sep. 2010.

[48] J. Liu and S. Singh. ATCP: TCP for Mobile Ad Hoc Networks. IEEE Journal on Selected Areas in Communication, Vol. 19, No. 7, pp. 1300- 1315, July 2001.

[49] S.H. Low. A Duality Model of TCP and Queue Management Algorithms. IEEE/ACM Transactions on Networking, Vol. 11, No. 4, pp. 525-536, Aug. 2003.

[50] S.H. Low, F. Paganini, and J.C. Doyle. Internet Congestion Control. IEEE Control Systems Magazine, Vol. 22, No. 1, pp. 28-43, Feb. 2002.

[51] S.H. Low, F. Paganini, J. Wang, and J.C. Doyle. Linear Stability of TCP/RED and a Scalable Control. Computer Networks Journal, Vol. 43, No. 5, pp. 633-647, Dec. 2003.

[52] R. Ludwig and R.H. Katz. The Eifel Algorithm: Making TCP Robust Against Spurious Retransmissions. ACM SIGCOMM Computer Com- munication Review, Vol. 30, Issue 1, pp. 30-36, Jan. 2000.

[53] V. Misra, W.-B. Gong, and D. Towsley. Fluid-based Analysis of a Net- work of AQM Routers Supporting TCP Flows with an Application to RED. ACM SIGCOMM Computer Communication Review, Vol. 30, No. 4, pp. 151-160, Oct. 2000.

[54] D. Mitzel. Overview of 2000 IAB Wireless Internetworking Workshop. RFC 3002, Network Working Group, Dec. 2000.

[55] J. Mo, R.J. La, V. Anantharam, and J. Walrand. Analysis and Com- parison of TCP Reno and Vegas. Proc. of IEEE INFOCOM, Mar. 1999.

121 [56] J. Nagle. Congestion Control in IP/TCP Internetworks. RFC 896, Net- work Working Group, Jan. 1984.

[57] R. Pan, B. Prabhakar, and K. Psounis. CHOKe: A Stateless AQM Scheme for Approximating Fair Bandwidth Allocation. Proc. of IEEE INFOCOM, Mar. 2000.

[58] C. Parsa and J. Garcia-Luna-Aceves. Improving TCP Congestion Con- trol over Internets with Heterogeneous Transmission Media. Proc. of IEEE ICNP 1999, pp. 213-221, Oct./Nov. 1999.

[59] L. Peterson, T. Anderson, D. Culler, and T. Roscoe. A blueprint for introducing disruptive technology into the Internet. ACM SIGCOMM Computer Communication Review, Vol. 33, No. 1, pp. 59-64, Jan. 2003.

[60] J. Postel. Transmission Control Protocol. RFC 793, Protocol Speciﬁca- tion, DARPA Internet Program, Sep. 1981.

[61] K.K. Ramakrishnan, S. Floyd, and D. Black. The Addition of Explicit Congestion. Notiﬁcation (ECN) to IP. IETF RFC 3168, Sep. 2001.

[62] P. Sarolahti and M. Kojo. Forward RTO-Recovery (F-RTO): An Algo- rithm for Detecting Spurious Retransmission Timeouts with TCP and the Stream Control Transmission Protocol (SCTP). IETF RFC 4138, Aug. 2005.

[63] S. Seth and M.A. Venkatesulu. TCP/IP Architecture, Design and Im- plementation in Linux. Hoboken, New Jersey: Wiley, 2008.

[64] S. Shenker. Fundamental Design Issues for the Future Internet. IEEE Journal on Selected Areas in Communications, Vol. 13, No. 7, pp. 1176- 1188, Sep. 1995.

[65] R. Srikant. The Mathematics of Internet Congestion Control. Cam- bridge, MA:Birkhauser, 2004.

[66] W. Stevens. TCP Slow Start, Congestion Avoidance, Fast Retransmit, and Fast Recovery Algorithms. IETF RFC 2001, Jan. 1997.

[67] A. Tang, J. Wang, S.H. Low, and M. Chiang. Equilibrium of Hetero- geneous Congestion Control: Existence and Uniqueness. IEEE/ACM Transactions on Networking, Vol. 15, No. 4, pp. 1422-1435, Aug. 2007.

122 [68] A. Tang, D. Wei, S.H. Low and M. Chiang. Equilibrium of Heteroge- neous Congestion Control: Optimality and Stability. IEEE/ACM Tran- sanctions on Networking, Vol. 18, No. 3, pp. 844-857, Jun. 2010.

[69] F. Wang and Y. Zhang. Improving TCP Performance over Mobile Ad- Hoc Networks with Out-Of-Order Detection and Response. Proc. of ACM MOBIHOC 2002, pp. 217-225, June 2002.

[70] J. Wang, J. Wen, J. Zhang, and Y. Han. TCP-FIT: An Improved TCP Congestion Control Algorithm and its Performance. Proc. of IEEE IN- FOCOM, Apr. 2011.

[71] D. Wei, C. Jin, S. Low, and S. Hegde. FAST TCP: Motivation, Architec- ture, Algorithms, Performance. IEEE/ACM Transactions on Network- ing, Vol. 14, No. 6, pp.1246-1259, Dec. 2006.

[72] J.T. Wen and M. Arcak. A Unifying Passivity Framework for Network Flow Control. IEEE Transactions on Automatic Control, Vol. 49, No. 2, pp. 162-174, Feb. 2004.

[73] G. Wright and W. Stevens. TCP/IP Illustrated, Volume 2: The Imple- mentation. Addison-Weasley, 1995.

[74] E. H.-K. Wu and M.-Z. Chen. JTCP: Jitter-Based TCP for Heteroge- neous Wireless Network. IEEE Journal on Selected Areas in Communi- cation, Vol. 22, No. 4, pp. 757-766, May 2004.

[75] Y. Xia, L. Subramanian, I. Stoica, and S. Kalyannaramana. One More Bit is Enough. IEEE/ACM Transactions on Networking, Vol. 16, No. 6, Mar. 2008.

[76] K. Xu, M. Gerla, L. Qi, and Y. Shu. Enhancing TCP Fairness in Ad Hoc Wireless Networks Using Neighborhood RED. Proc. of ACM MO- BICOM, Sep. 2003.

[77] D. Yang, K.-C. Leung, and V.O.K. Li. Simulation-based Comparisons of Solutions for TCP Packet Reordering in Wireless Networks. Proc. of IEEE WCNC, pp. 3240-3245, March 2007.

[78] L. Ying, G.E. Dullerud, and R. Srikant. Global Stability of Internet Congestion Controllers with Heterogeneous Delays. IEEE/ACM Trans- actions on Networking, Vol. 14, No. 3, pp. 579-591, Jun. 2006.

123 [79] H. Zhang, B. Liu, and W. Dou. Design of a Robust Active Queue Management Algorithm Based on Feedback Compensation. ACM SIG- COMM Computer Communication Review, Vol. 33, No. 4, pp. 277-285, Oct. 2003.

[80] L. Zhang and D. Clark. Oscillating Behavior of Network Traﬃc: A Case Study Simulation. Internetworking: Research and Experience, Vol. 1, No. 2, pp. 101-112, 1990.

[81] M. Zhang, B. Karp, S. Floyd, and L. Peterson. RR-TCP: A Reordering- Robust TCP with DSACK. Proc. of IEEE ICNP 2003, pp. 95-106, Nov. 2003.

[82] Z. Zhao, Q. Liu, G.-H. Yang, V.O.K. Li, and L. Cui. A Multi-radio Wireless P2P Network Testbed. Proc. of ACM WiNTECH, pp. 93-94, Sep. 2009.

124 Acronyms

ACK acknowledgment. 4, 17, 18, 25, 27, 28, 35, 43, 44

AQM active queue management. 1, 3, 9, 12–16, 19–22, 69–73, 77–79, 81–84, 86, 90, 97, 100, 104–110, 112–115 awnd advertised window. 6

CBR constant bit rate. 71, 93, 102

CD congestion response decision. 28, 43, 49, 51, 54, 57, 59, 63, 66

CRHP closed right half plane. 85 cwnd congestion window. 6–8, 12, 17, 18, 29, 31, 34–36, 59, 60

ECN explicit congestion notiﬁcation. 70

FTP File Transfer Protocol. 3

GBR goodput to bandwidth ratio. 65, 66

HTTP Hypertext Transfer Protocol. 3, 93, 102, 104, 108

IC integral controller. 13, 14, 20, 71, 72, 87, 90, 92–94, 96, 97, 100, 102, 104, 108, 114

IP Internet Protocol. 3, 19

LLRTX link-layer retransmission. 11, 17

MANET mobile ad-hoc networks. 11

MRRL maximum RTT record length. 44–46

125 NRU NCL-RTT-Update. 43

OLHP open left half plane. 82

PI proportional-integral. 20, 21, 90

RD retransmission decision. 28, 43, 49, 51, 54

RED random early detection. 9, 13, 20, 66, 67, 70, 72, 73, 76, 77, 92–94, 97, 100, 102, 104, 108, 109, 114

RTO retransmission timeout. 7, 17, 31, 43, 49–52, 111

RTT round-trip time. 4, 11, 17, 19, 27–29, 35, 37, 43–46, 50, 68, 80, 88, 91, 110, 111 ssthresh slow start threshold value. 6, 7, 29, 35

STS smart TCP sender. 12, 14, 23–25, 28, 43, 49, 68, 107, 109, 114

TCP Transmission Control Protocol. 1, 3, 8–21, 23–25, 27–30, 33–36, 43, 44, 46, 50–52, 54–56, 58, 59, 61–63, 65–67, 69, 70, 72, 73, 76–78, 81, 83, 91, 93, 100, 104, 106–115

TCP-NCL TCP for non-congestive loss. 13, 14, 24, 25, 43, 46, 49, 51–56, 59, 60, 62–64, 66–68, 70, 76, 107–109, 112, 114

TCP/IP Transmission Control Protocol/Internet Protocol. 2, 53

WMN wireless mesh network. 11

WPER wireless packet error rate. 13, 71, 73, 77, 86, 87, 93, 96, 97, 102, 108, 110, 112, 114, 115

126