A Hybrid Loss-Delay Gradient Congestion Control Algorithm for the Internet Rasool Al-Saadi School of Software and Electrical Engineering Swinburne University of Technology This thesis is submitted for the degree of Doctor of Philosophy Faculty of Science, Engineering and October 2019 Technology I dedicate this thesis to my loving parents, my beloved wife Dalia and my sons Ridha and Yousif. Declaration I declare that this thesis submitted for the degree of Doctor of Philosophy contains no ma- terial that has been accepted for the award to the candidate of any other degree or diploma, except where due reference made a in the text of the examinable outcome. To the best of the candidate’s knowledge contains no material previously published or written by another person except where due reference is made in the text of the examinable outcome. Where the work is based on joint research or publications, discloses the relative contributions of the respective creators or authors. Rasool Al-Saadi October 2019 Acknowledgements I would like to express my thanks and appreciation to all the people and institutions who helped, guided, advised and supported me during my PhD journey. I would like to thank my supervisors Prof. Grenville Armitage, Dr. Jason But and Assoc. Prof. Philip Branch for their insight, patience, friendship, encouragement, continuous support and valuable advice during my PhD candidature. I had the honour of being a student under their wise supervision. Without their support, this work would never have seen the light of day. I am greatly indebted to my family, especially my wife Dalia Al-Zubaidy, my parents, my brother and my sister for their patience, persevering support and advice. Their encouragement allowed me to overcome the trials and tribulations during this long journey. I would like to give special thanks to my sponsor, the Ministry of Higher Education and Scientific Research - Iraq, for giving me the opportunity to complete my PhD candidature and for supporting me during my study. I thank all members of the I4T research group for our interesting discussions. Special thanks to Jonathan Kua for helping me in testing different implementations, for answering my questions and his interesting discussion. Finally, I am immensely grateful to my best friend, Haider Majeed, for following up my financial and administrative matters in Iraq without hesitation or complaint. Abstract Congestion Control (CC) has a significant influence on the performance of Transmission Con- trol Protocol (TCP) connections. Over the last three decades, many researchers have exten- sively studied and proposed a multitude of enhancements to standard TCP CC. However, this topic still inspires both academic and industry research communities due to the change in In- ternet application requirements and the evolution of Internet technologies. The standard TCP CC infers network congestion based on packet loss events which leads to long queuing delay when bottleneck buffer size is large in a phenomenon called Bufferbloat. Bufferbloat harms interactive and latency-sensitive applications and reduces network response speed, leading to a poor quality of experience for end users. A potential end-to-end solution to this problem is to use the delay signal to infer congestion earlier and react to the congestion before the queuing delay reaches a high value. However, delay-based feedback is fraught with difficulties including queuing delay estimation, signal noise and coexistence with conventional TCP. Loss-delay hybrid congestion control approaches aim to remedy the coexistence issue by operating in delay mode to provide low latency transport, and switching to loss mode to im- prove performance when competing with loss-based flows. Using a delay gradient enables the CC protocol to detect congestion without relying on queuing delay estimates. This motivates us to explore the previously published CAIA Delay-Gradient (CDG) CC algorithm. In this thesis, we significantly enhance the CDG algorithm to create Hybrid Loss-Delay Gradient (HLDG), a sender-side host-to-host congestion control algorithm for the Internet. Compared to CDG, HLDG provides higher performance, lower queuing delay and better co- existence when competing with loss-based flows. Like CDG, HLDG uses the delay gradient signal to detect early congestion, preventing a long queue from building up in the bottleneck, and providing low latency transport. It also ex- plicitly switches between delay and loss modes depending on competing traffic characteristics to coexist with loss-based flows. By evaluating CDG in a range of scenarios, we identify weaknesses that impact protocol performance. HLDG enhances CDG throughput by solving the unnecessary back-off issue and by maintaining the number of unacknowledged bytes close to estimated the path bandwidth- x delay product (BDP). HLDG also improves coexistence by switching between loss and delay mode based on a heuristic function. This function infers the presence of loss-based flows based on queuing delay measurements. Finally, HLDG utilises an enhanced slow-start algo- rithm using the delay gradient signal and BDP estimate. HLDG slow-start terminates with a congestion window size close to estimated BDP, achieving high throughput without inducing high queuing delay. HLDG can also be effectively used as a low priority CC to provide scavenger class trans- port for background file transfer without disturbing the foreground traffic. This can be achieved by disabling the HLDG coexistence mechanisms and reducing the back-off factor. We experimentally evaluate HLDG under a range of scenarios and network conditions. This includes a range of link bandwidths and path RTT, with both FIFO Droptail and Active Queue Management (AQM) used at the bottleneck. Our results show that HLDG obtains up to 400% (54% in average) throughput improve- ment over CDG in single flow scenarios without packet loss while maintaining queueing delay under 20ms in the worst case. Our results also show that HLDG is able to achieve up to 3000% (810% in average) better throughput than CDG when competing with TCP CUBIC. HLDG slow-start exits with a congestion window between 96% - 200% of the path BDP compared to 1% - 800% for CDG slow-start. HLDG achieves 94%, 89% and 95% average link utilisation under PIE, FQ-CoDel and FQ- PIE respectively, compared to 89% for CDG under the same conditions. HLDG experiences only 3%, 24% and 4% of packet loss experienced by CDG when run through PIE, FQ-CoDel and FQ-PIE AQMs respectively. Contents Abstract ix Contents xi List of Figures xvii List of Tables xxiii List of Acronyms xxvi 1 Introduction 1 1.1 Researchobjectivesandcontributions . ....... 3 1.2 Summaryofresearchcontributions. ..... 6 1.3 Thesisorganisation .............................. 7 2 Delay-based congestion control 9 2.1 TransmissionControlProtocol–AnOverview. ....... 10 2.1.1 TCPReliabilityandFlowControl . 10 2.1.2 TCPCongestionControl . 12 2.1.3 Controlling the injection of packets into the network ......... 15 2.2 BufferingandQueueManagement . 15 2.2.1 TraditionalBufferingandQueues . 16 2.2.2 ActiveQueueManagement. 17 2.3 TCP Congestion Control Literature -- Signals and Algorithms ........ 17 2.3.1 ImplicitCongestionFeedbackSignals . .... 18 2.3.1.1 Thelosssignal ....................... 18 2.3.1.2 Thedelaysignal . 21 2.3.2 ExplicitCongestionFeedbackSignals . .... 24 2.3.3 TCPCongestionControlAlgorithms . 26 xii Contents 2.3.3.1 SlowStart .......................... 26 2.3.3.2 CongestionAvoidance. 27 2.3.3.3 FastRetransmitandFastRecovery . 28 2.3.4 CongestionControlMetrics . 29 2.4 CCalgorithmsthatutilisethedelaysignal . ........ 31 2.4.1 Delay-basedAlgorithms . 31 2.4.1.1 TCPDUAL ......................... 33 2.4.1.2 TCPVegas.......................... 35 2.4.1.3 TCPVegas-A ........................ 36 2.4.1.4 FASTTCP.......................... 37 2.4.1.5 TCPNICE.......................... 37 2.4.1.6 TCP-LP ........................... 38 2.4.1.7 PERT............................. 39 2.4.1.8 LEDBAT........................... 40 2.4.1.9 TIMELY........................... 41 2.4.1.10 TCPLoLa .......................... 42 2.4.2 DualModeApproaches . 43 2.4.2.1 TCPVegas+ ......................... 43 2.4.2.2 YeAHTCP.......................... 45 2.4.2.3 CAIA-DelayGradient(CDG) . 47 2.4.2.4 Cx-TCP ........................... 48 2.4.2.5 Copa............................. 49 2.4.2.6 Nimbus............................ 50 2.4.3 DualSignalApproaches . 51 2.4.3.1 TCPAfrica.......................... 51 2.4.3.2 CompoundTCP(C-TCP) . 52 2.4.3.3 TCPLibra .......................... 53 2.4.3.4 TCP-Illinois . .. .. .. .. .. .. .. 54 2.4.4 Delay-sensitiveAlgorithms. 55 2.4.4.1 TCPWestwood/Westwood+. 55 2.4.4.2 TCPWestwood-BasedAlgorithms . 57 2.4.4.3 TCP-AP ........................... 59 2.4.4.4 RAPID............................ 59 2.4.4.5 TCPBBR .......................... 60 2.5 TCPslow-startvariations . ... 62 2.6 Challengesfacingthedelaysignal . ..... 63 Contents xiii 2.6.1 Inaccuratedelaymeasurements . 63 2.6.2 Data link layer reliability and channel access method ......... 64 2.6.3 Emergingcommunicationnetworks . 65 2.6.4 EffectsofusingAQMondelaysignal . 66 2.7 CDG:ApromisinghybridCCalgorithm . ... 68 2.7.1 Delaygradientsignal. 69 2.7.2 Probabilisticback-offfunction . .... 71 2.7.3 Non-congestionrelatedlossestolerance . ...... 73 2.7.4 Coexistencewithloss-basedflows . 74 2.7.4.1 Ineffectualback-offandlossmode: . 74 2.7.4.2 Theshadowwindowmechanism . 75 2.7.5 Congestionwindowgrowth . 75 2.7.6 CDGslowstart ............................. 76 2.7.7