Bufferbloat and Beyond
Total Page:16
File Type:pdf, Size:1020Kb
The elephant symbo- lises a big download, The topic of this thesis is the performance of computer networks in commonly referred to general, and the internet in particular. While network performance as an ”elephant flow”, which has generally improved with time, over the last several years we have blocks the link with its bulk so seen examples of performance barriers limiting network performance. everything has to wait behind it. In this work we explore such performance barriers and look for solutions. The mice are smaller Our exploration takes us through three areas where performance barriers flows, such as web are found: The bufferbloat phenomenon of excessive queueing latency, the pages, that take up little space performance anomaly in WiFi networks and related airtime resource sharing but should be able to move problems, and the problem of implementing high-speed programmable freely. Through better queue packet processing in an operating system. In each of these areas we present management, the mice are solutions that significantly advance the state of the art. guided around the elephant, The work in this thesis spans all three aspects of the field of computing, so they don’t have to wait; namely mathematics, engineering and science. We perform mathematical and interactivity is restored. analysis of algorithms, engineer solutions to the problems we explore, The penguin is Tux, and perform scientific studies of the network itself. All our solutions are the Linux mascot, who implemented as open source software, including both contributions to the stands at the bottleneck upstream Linux kernel, as well as the Flent test tool, developed to support and directs traffic onto the the measurements performed in the rest of the thesis. right path. 2018:42 DOCTORAL THESIS Karlstad University Studies, 2018:42 Print & layout ISBN 978-91-7063-878-7 (Print) University Printing Office, Karlstad 2018 ISBN 978-91-7063-973-9 (pdf) Toke Høiland-Jørgensen | Bufferbloat and Beyond Toke Bufferbloat and Beyond The topic of this thesis is the performance of computer networks in general, and the internet in particular. While network performance has generally improved with time, over the last several years we have seen examples of performance barriers limiting network performance. In this work we explore such performance barriers and look for solutions. 2018:42 Bufferbloat and Beyond Our exploration takes us through three areas where performance barriers are Removing Performance Barriers in Real-World Networks found: The bufferbloat phenomenon of excessive queueing latency, the performance anomaly in WiFi networks and related airtime resource sharing problems, and the problem of implementing high-speed programmable packet processing in an operating system. In each of these areas we present solutions that significantly advance the state of the art. Toke Høiland-Jørgensen The work in this thesis spans all three aspects of the field of computing, namely mathematics, engineering and science. We perform mathematical analysis of algorithms, engineer solutions to the problems we explore, and perform scientific studies of the network itself. All our solutions are implemented as open source software, including both contributions to the upstream Linux kernel, as well as the Flent test tool, developed to support the measurements performed in the rest of the thesis. ISBN 978-91-7063-878-7 (Print) ISBN 978-91-7063-973-9 (pdf) Faculty of Health, Science and Technology ISSN 1403-8099 Computer Science DOCTORAL THESIS | Karlstad University Studies | 2018:42 DOCTORAL THESIS | Karlstad University Studies | 2018:42 Bufferbloat and Beyond Removing Performance Barriers in Real-World Networks Toke Høiland-Jørgensen DOCTORAL THESIS | Karlstad University Studies | 2018:42 Bufferbloat and Beyond - Removing Performance Barriers in Real-World Networks Toke Høiland-Jørgensen DOCTORAL THESIS Karlstad University Studies | 2018:42 urn:nbn:se:kau:diva-69416 ISSN 1403-8099 ISBN 978-91-7063-878-7 (Print) ISBN 978-91-7063-973-9 (pdf) © The author Distribution: Karlstad University Faculty of Health, Science and Technology Department of Mathematics and Computer Science SE-651 88 Karlstad, Sweden +46 54 700 10 00 Print: Universitetstryckeriet, Karlstad 2018 WWW.KAU.SE iii Bufferbloat and Beyond: Removing Performance Barriers in Real-World Networks Toke Høiland-Jørgensen Department of Mathematics and Computer Science Karlstad University Abstract The topic of this thesis is the performance of computer networks. While net- work performance has generally improved with time, over the last several years we have seen examples of performance barriers limiting network per- formance. In this work we explore such performance barriers and look for solutions. The problem of excess persistent queueing latency, known as bufferbloat, serves as our starting point; we examine its prevalence in the public internet, and evaluate solutions for better queue management, and explore how to improve on existing solutions to make them easier to deploy. Since an increasing number of clients access the internet through WiFi networks, examining WiFi performance is a natural next step. Here we also look at bufferbloat, as well as the so-called performance anomaly, where stations with poor signal strengths can severely impact the performance of the whole network. We present solutions for both of these issues, and additionally design a mechanism for assigning policies for distributing airtime between devices on a WiFi network. We also analyse the “TCP Small Queues” latency minimisation technique implemented in the Linux TCP stack and optimise its performance over WiFi networks. Finally, we explore how high-speed network processing can be enabled in software, by looking at the eXpress Data Path framework that has been gradu- ally implemented in the Linux kernel as a way to enable high-performance programmable packet processing directly in the operating system’s networking stack. A special focus of this work has been to ensure that the results are carried forward to the implementation stage, which is achieved by releasing imple- mentations as open source software. This includes parts that have been accep- ted into the Linux kernel, as well as a separate open source measurement tool, called Flent, which is used to perform most of the experiments presented in this thesis, and also used widely in the bufferbloat community. Keywords: Bufferbloat, AQM, WiFi, XDP, TSQ, Flent, network measure- ment, performance evaluation, fairness, queueing, programmable packet pro- cessing v Acknowledgements The journey is almost over. It has been challenging, exciting, annoying, incredible and above all interesting. And it would not have been the same without the amazing people who have accompanied and supported me along the way, all of whom deserve a huge thank you. First of all, thank you to the many collaborators I have had these past years. Foremost among them of course my advisors, Anna Brunström and Per Hurtig, who have always been available with their competent feedback, ideas, suggestions and support. A special thank you also to Dave Täht, with whom I have collaborated extensively, on both the papers we have co-authored, and a list of other projects too long to reproduce here. Dave was also, along with Jim Gettys, the people who initially introduced me to the concept of bufferbloat, and they helped set me on the path towards the work represented in this thesis, for which I am exceedingly grateful. Thank you to Jesper Brouer for his collaboration, and for convincing me to continue the work with Linux and open source post-thesis. Many thanks to Fanny Reinholtz for designing the awesome dust jacket for the printed version of this thesis. And finally, thank you to my other co-authors, and all the members of the bufferbloat, make-wifi-fast, Linux and OpenWrt communities that I have collaborated with over the last six years. A journey such as this would not have been possible without the friends who have shared my joys and sorrows along the way. Thank you to my colleagues at the university, my old friends in Denmark, and my new(er) friends in Karlstad. Thank you to Lea, Stefan, Stina and Leonardo for climbing, skiing and everything else. To Martin, Daniel, Alex and Miriam for hanging out and having fun. To Timo, Leo and Ricardo for bringing the band (back) together. To the innebandy team in its various incarnations. To the dwarves, and to Sunni, Morten, Sidsel and Torben for keeping in touch through my five-year Swedish exile. Last, but certainly not least, thank you so much to my wonderful loving family. Thank you mum and dad for helping me get to this point, for always being there for me, and for your love and respect. And thank you Sascha, for coming with me to Sweden and brightening up my days. Karlstad, October 2018 Toke Høiland-Jørgensen vii List of appended papers I. Toke Høiland-Jørgensen, Bengt Ahlgren, Per Hurtig and Anna Brun- strom. Measuring Latency Variation in the Internet. ACM CoNEXT ’16, December 12–15, 2016, Irvine, CA, USA. II. Toke Høiland-Jørgensen, Per Hurtig and Anna Brunstrom. The Good, the Bad and the WiFi: Modern AQMs in a Residential Setting. Computer Networks, vol 89, pg 90–106, October 2015. III. Toke Høiland-Jørgensen. Analysing the Latency of Sparse Flows in the FQ-CoDel Queue Management Algorithm. IEEE Communication Letters, October 2018. IV. Toke Høiland-Jørgensen, Dave Täht and Jonathan Morton. Piece of CAKE: A Comprehensive Queue Management Solution for Home Gateways. IEEE International Symposium on Local and Metropolitan Area Networks (LANMAN 2018), 25–27 June 2018, Washington, DC. V. Toke Høiland-Jørgensen, Michal Kazior, Dave Täht, Per Hurtig and Anna Brunstrom. Ending the Anomaly: Achieving Low Latency and Airtime Fairness in WiFi. 2017 USENIX Annual Technical Conference (USENIX ATC 17), July 12–14 2017, Santa Clara, CA. VI. Toke Høiland-Jørgensen, Per Hurtig and Anna Brunstrom. PoliFi: Air- time Policy Enforcement for WiFi. Under Submission. VII. Carlo Augusto Grazia, Natale Patriciello, Toke Høiland-Jørgensen, Mar- tin Klapez and Maurizio Casoni.