Debugging Modern Web Protocols Doctoral Thesis robin marx Hasselt University, 2016 – 2020 This thesis was made possible due to personal Strategic Basic Research grant #1S02717N from FWO, Research Foundation Flanders. Abstract Today, the Internet plays a key part in our lives. Billions of people visit even more Web pages on a daily basis to consume information from and contribute it to the ever growing Web. The speed with which these pages load can have a profound impact on their users’ experience and subsequent behaviour. Firstly, slow page loads might lead to user frustration, abandoned sessions, less frequent visits and, ultimately, reduced revenue. Secondly, Web performance can be defined as an accessibility and equality issue, as timely access to online educational, medical, navigational, or even legal information can have a large social impact. It is thus no wonder that Web performance is a hot topic, both in industry and academia. The Web page loading process and its efficiency is tightly linked to the network protocols used to transport the page’s resources, such as IP, TCP, QUIC, TLS, and HTTP. Consequently, even today a large amount of time and energy continues to be put into improving the performance of key protocols and into the development of new ones. These upgrades have rarely been trivial however, and have come at the cost of a growing complexity, making the advanced protocols, their features and potential alternatives hard to design, implement, analyze, debug, test, compare, teach, deploy, and use correctly in practice. This text tracks our efforts on tackling these challenges as they occurred for the past five years in the rapid evolution from the older HTTP/1.1 via HTTP/2 to the most recent HTTP/3 and QUIC protocols. During this time we have been actively involved with the Internet Engineering Task Force (IETF), in charge of designing and maintaining these protocols, and much of our work has found real-world impact. In this work, we firstly examine the behaviour and impact of the various performance features and problems of the protocols, including Head-of-Line Blocking, prioritization, 0-RTT connection establishment, and congestion and flow control. We validate both their core designs and a variety of concrete implementations using extensive empiri- cal evaluations. We uncover a wide range of issues and detail their root causes, as well as potential solutions. We find that, contrary to popular belief, HTTP/2 and HTTP/3 do not magically obviate the need for developers to apply the same manual page composi- tion and deployment tweaks that were needed since HTTP/1.1. One example of a key ii feature we explore in depth is that of resource multiplexing and prioritization, which governs in which order page resources are transmitted. We observe that HTTP/2’s approach to this feature is difficult to utilize correctly and that its recommended default scheme, as implemented by several browsers, actually exhibits worst-case performance. Our subsequent exploration of better alternatives has directly contributed to the feature being completely redesigned for its HTTP/3 successor. The second major contribution of this work are the tools and methodologies we have developed to aid in analyzing these protocols. Strangely, despite the complexity and importance of their performance features, we found a general dearth of tools and visualizations aimed at measuring, diagnosing, and improving their low-level behaviours. In support of our need to compare various competing schemes and designs, as well as to debug fledgling implementations, we have created a wide array of such tools ourselves. Main among them is the qlog structured endpoint logging format, which allows encrypted protocols such as QUIC to expose deep protocol and implementation state in a privacy-aware fashion. This information can then be analyzed using our qvis toolsuite, a collection of advanced interactive visualizations including sequencing, congestion control, multiplexing, and packetization diagrams. These projects have found a wide adoption within the QUIC and HTTP/3 community, with over 70% of implementations supporting the format, including those of Facebook and Cloudflare who utilize our work to fine-tune their production deployments of the new protocols. In summary, this text discusses an in-depth exploration of the performance fea- tures and characteristics of both the HTTP/2+TLS+TCP and HTTP/3+QUIC protocol stacks. It details our process in creating and using custom tools and methodologies to perform these analyses, as well as discusses the obtained insights, both of which have helped steer the design and practical implementations of these new networking protocols. Acknowledgements During the past five years, much of my time has been dedicated to two similar and parallel efforts. Firstly, during the working hours, I shaped the text that you see before you. Secondly, in my free time, I shaped myself by practicing longsword fencing and Historical European Martial Arts. The latter was done at our club called Invirtus, which has laid claim to the motto “Aut Viam Inveniam, Aut Faciam”. This phrase, freely translated as “I will find a way or I will make one” is attributed to the warlord Hannibal, prior to his traversal of the Alps. Looking back, this sentence seems highly applicable to both activities. In both endeavours, I have made a similar progression, starting as a wide-eyed student and ending as a teacher who knows his journey is only just beginning. Similarly, while the two undertakings encompass an inherently personal process, they cannot be executed without the help and support of like-minded individuals, friends, and family. Smiling while doing push-ups in the mud on a rainy Sunday afternoon would not have been possible without the camaraderie of the many excellent people I have met during my time at Invirtus. Similarly, my academic efforts have been supported by many who have helped me find and make a way through my own Alps. Firstly, my family. My partner Deborah, who was given many a chance to perfect her eye-rolling skills. My parents, Robert and Carinne, who’s eyes continue to glitter with pride even though they probably have no idea what I’m talking about. My sister Jill, who turns a blind eye to me being a terrible brother by being a great sister. Secondly, the Rhinos. Jens, Pallieter, Jimmy, Boutsen, Steven, Nick, Jan, and Wouter who have provided me with much needed entertainment, advice, support, and friend- ship since starting our Computer Science studies together. Thirdly, my colleagues. Maarten, who has guided me since I annoyed him as an intern all those years ago. Wouter, who always had an interesting out-of-the-box perspective. Joris, who is both a great student and collaborator. Jori, Mariano, Pieter, Hendrik, and others at the EDM who kindly didn’t tell me to bugger off. Fourthly, my bachelor and masterthesis students. Daan, Jonas, Kevin, Tom, Sven, Sander, and Axel all contributed directly and significantly to this work, some even multiple times. It has been my privilege to teach them and learn from them in turn. iv Fifthly, my fellow protocol researchers and my Jury. Törsten and Felix who were excellent company. Oliver and Bala who made me feel knowledgeable. Olivier and Lars who showed that kindness can be found behind a stern appearance. Sixthly, the large group of people in the Web performance and IETF QUIC communi- ties. Lucas, Matt, Marten, Jeremy, Dmitri, Nick, Mirja, Christian, Maxime, Quentin, Ian, Eric, Kazuho, David, Alessandro, Junho, Subodh, Peter, other Peter, Alexis, Tatsuhiro, Kazu, Daniel, James, Rui, Will, Andy, Alex, Brian, Craig, Victor, Jana, Martin, Martin again, Patrick, Andy, Barry, Yoav, and the many others who I did not list. Without you all, this result would not have been possible. Seventhly, my promoters. Peter and Wim, who have given me an enormous amount of opportunities. Thank you for taking me aboard and seeing my potential. Finally, the FWO, Research Foundation Flanders, and all Belgian tax payers. For assigning me project nr. #1S02717N and making sure I didn’t go hungry. It has been a privilege and an honor to go on this journey with you all. To both find and make my way towards this end result. To earn the right to be a Doctor. Aut Viam Inveniam, Aut Faciam Contents Abstract i Acknowledgments iii Contents v I Introduction 1 1 Web Performance 9 1.1 What Is the Importance of Web Performance? ............... 9 1.2 The Web Page Load Process ......................... 13 1.2.1 Resource Semantics .......................... 13 1.2.2 Resource Scheduling ......................... 15 1.2.3 Related Work .............................. 16 1.3 Web Performance Evaluation ......................... 18 1.3.1 Objective Metrics ........................... 18 1.3.2 Correlation of Subjective and Objective Metrics .......... 20 2 About HTTP/2, HTTP/3 and QUIC 23 2.1 The Need for New Protocols ......................... 24 2.1.1 Head-of-Line Blocking ........................ 25 2.1.2 Ossification and Security ....................... 29 2.1.3 Conclusion ............................... 31 2.2 Stream Multiplexing and Prioritization ................... 32 2.2.1 HTTP/2’s Dependency Tree ..................... 33 2.2.2 HTTP/2 Prioritization Challenges .................. 35 2.3 Other Performance Related Features ..................... 37 2.3.1 Binary Framing ............................ 38 2.3.2 QUIC Connection Establishment and 0-RTT ............ 39 2.3.3 QUIC Congestion Control ...................... 41 2.3.4 Flow Control .............................. 42 vi CONTENTS 2.3.5 QUIC Connection Migration and Multipath ............ 43 2.3.6 HTTP Server Push ........................... 44 2.3.7 HTTP Header Compression ..................... 46 3 Focus of this Thesis 49 3.1 Contributions ................................. 49 3.2 Structure of the Text .............................. 51 II HTTP/2 Tools 53 4 Speeder and Oracle 57 4.1 Speeder: Backend and Testrunner ...................... 57 4.1.1 Speeder Details ............................ 58 4.1.2 Limitations and Related Work ...................