Traffic-Engineered Distribution of Multimedia Content by Khaled Diab B.Sc., Cairo University, 2011 Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the School of Computing Science Faculty of Applied Science c Khaled Diab 2019 SIMON FRASER UNIVERSITY Summer 2019 Copyright in this work rests with the author. Please ensure that any reproduction or re-use is done in accordance with the relevant national copyright legislation. Approval Name: Khaled Diab Degree: Doctor of Philosophy Title: Traffic-Engineered Distribution of Multimedia Content Examining Committee: Chair: Steven Bergner University Research Associate Mohamed Hefeeda Senior Supervisor Professor Jiangchuan Liu Supervisor Professor Shervin Shirmohammadi Internal Examiner Adjunct Professor Mostafa Ammar External Examiner Professor School of Computer Science Georgia Institute of Technology Date Defended: 15 August, 2019 ii Abstract The amount of video traffic transmitted over the Internet has been steadily increasing over the past decade. This is due to the recent interest in streaming high-definition (HD), 4K, and immersive videos to many users. Streaming such multimedia content at large scale faces multiple challenges. For example, streaming systems need to handle users heterogeneity and interactivities in varying network conditions. In addition, large-scale video streaming stresses the ISP network that carries the video traffic, because the ISP needs to carefully direct traffic flows through its network to sat- isfy various service level agreements. This increases the complexity of managing the network and system resources. To address these challenges, we first propose a novel client-based rate adapta- tion algorithm for streaming multiview videos. Our algorithm achieves high video quality, smooth playback and efficient bandwidth utilization. For example, it achieves up to 300% improvement in the average quality compared to the algorithm used by YouTube. Then, we propose an algorithm to efficiently solve the resource management problem in the emerging ISP-managed Content Dis- tribution Networks (CDNs). Our solution achieves up to 64% reduction in the inter-domain traffic. To reduce the network load of live streaming sessions, ISPs use multicast to efficiently carry the traffic through their networks. Finally, we propose a label-based multicast forwarding approach to implement traffic-engineered multicast trees in ISP networks. We prove that the proposed approach is scalable as it imposes minimal state and processing overheads on routers. We implemented the proposed multicast approach in a high-speed network testbed, and our results show that it can sup- port thousands of concurrent multicast sessions. Keywords: Multi-view videos; adaptive video streaming; traffic engineering; telco-CDNs; multi- cast forwarding; efficient data planes. iii To Dina, Jana, and my parents and brothers. To my cousin Hazem. iv Acknowledgements I would like to express my sincere gratitude to my senior supervisor, Dr. Mohamed Hefeeda. When I met him in 2012, I only had a good knowledge in computer science and a mere interest in sys- tems and networking. Ever since this moment, Dr. Hefeeda has been pushing my limits to gain the research skills to be a successful scientist. His continuous support, critical reviews and scientific knowledge as well as devotion and enthusiasm to research have contributed greatly to this thesis. I will be always indebted to Dr. Hefeeda. I would like to express my gratitude to Dr. Jiangchuan Liu, my supervisor, for his valuable comments and discussions whenever I needed them. My thanks go to Dr. Shervin Shirmohammadi, my internal examiner, and Dr. Mostafa Ammar, my external examiner, for being on my committee and reviewing the thesis. I would like as well to thank Dr. Steven Bergner for taking the time to chair my thesis defense. I would like to thank my colleagues at the Network Systems Lab who made this journey pleas- ant. Special thanks go to Dr. Joseph Peters, Ahmed Hamza, Kiana Calagari, Saleh Almowuena, Alaa Eldin Abdelaal, Omar Eltobgy, Omar Arafa, Mohammad Amin Arab, and Mohamed Hegazy. In addition, I am grateful for the help of all collaborators since I started my research. I am espe- cially grateful to Mustafa Rafique for his valuable insights and discussions. I also like to thank the administrative and technical staff at the School of Computing Science for their help and support. My special thanks go to Lee Greenough, Johnny Zhang, Dana Brand, Stephen Nix, Chingtai Wong, Allison MacKinnon and Clare McKenna. More importantly, I cannot find the right words to express my heartfelt gratitude to my parents and wife. I would like to thank my wonderful parents for their endless love and support throughout my life. I will be always grateful to my amazing wife, Dina, who has been supporting me to pursue my life and career goals. She has immersed me with her eternal love, support, and encouragement. This thesis would not have been possible without the support of my family. v Bibliographic Notes Publications: 1. Khaled Diab and Mohamed Hefeeda. Joint content distribution and traffic engineering of adaptive videos in telco-CDNs. In Proc. of IEEE INFOCOM’19, Paris, France, April 2019. 2. Khaled Diab and Mohamed Hefeeda. MASH: A rate adaptation algorithm for multiview video streaming over HTTP. In Proc. of IEEE INFOCOM’17, Atlanta, GA, May 2017. 3. Khaled Diab and Mohamed Hefeeda. Efficient multicast forwarding. 2019. (Under submis- sion). 4. Khaled Diab and Mohamed Hefeeda. STEM: stateless traffic-engineered multicast forward- ing. 2019. (Under submission). 5. Mohamed Hegazy, Khaled Diab, Mehdi Saeedi, Boris Ivanovic, Ihab Amer, Yang Liu, Gabor Sines, and Mohamed Hefeeda. Content-aware video encoding for cloud gaming. In Proc. of ACM MMSys’19, Amherst, MA, June 2019. 6. Kiana Calagari, Tarek Elgamal, Khaled Diab, Krzysztof Templin, Piotr Didyk, Wojciech Matusik, and Mohamed Hefeeda. Depth personalization and streaming of stereoscopic sports videos. ACM Trans. on Multimedia Comput. Commun. Appl. (TOMM), vol. 12, no. 3, pp. 41:1–41:23, March 2016. 7. Kiana Calagari, Krzysztof Templin, Tarek Elgamal, Khaled Diab, Piotr Didyk, Wojciech Matusik, and Mohamed Hefeeda. Anahita: A system for 3d video streaming with depth cus- tomization. In Proc. of ACM MM’14, Orlando, Fl, November 2014. 8. Khaled Diab, Tarek Elgamal, Kiana Calagari, and Mohamed Hefeeda. Storage optimization for 3D streaming systems. In Proc. of ACM MMSys’14, Singapore, Singapore, March 2014. 9. Khaled Diab, M. Mustafa Rafique, and Mohamed Hefeeda. Dynamic sharing of GPUs in cloud systems. In Proc. of IEEE IPDPS Workshops’13, Cambridge, MA, May 2013. vi Table of Contents Approval ii Abstract iii Dedication iv Acknowledgements v Bibliographic Notes vi Table of Contents vii List of Tables x List of Figures xi 1 Introduction 1 1.1 Overview . 1 1.2 Thesis Contributions . 3 1.2.1 Adaptive Multiview Video Streaming over HTTP . 3 1.2.2 Joint Content Distribution and Traffic Engineering in Telco-CDNs . 4 1.2.3 Traffic-Engineered Multicast Forwarding . 5 1.3 Thesis Organization . 9 2 Background 10 2.1 Introduction . 10 2.2 Adaptive Video Streaming . 11 2.2.1 History of Video Streaming . 11 2.2.2 HTTP Adaptive Streaming . 11 2.2.3 Immersive Multimedia Content . 12 2.2.4 Quality of Experience (QoE) in HAS Systems . 13 2.3 Rate Adaptation Algorithms . 15 2.3.1 Single-view Videos . 15 2.3.2 Immersive Videos . 19 vii 2.4 Content Distribution Networks (CDNs) . 21 2.4.1 Challenges of CDNs . 22 2.4.2 Collaboration Space in CDNs . 23 2.5 Multicast Systems . 26 2.5.1 Application-layer Multicast . 26 2.5.2 Network-based Multicast . 27 3 MASH: A Rate Adaptation Algorithm for Multiview Video Streaming over HTTP 30 3.1 Introduction . 30 3.2 Related Work . 32 3.3 Overview . 33 3.4 View Switching Models . 34 3.5 Details of MASH . 36 3.6 Analysis and Overheads of MASH . 38 3.7 Evaluation . 39 3.7.1 Experimental Setup . 40 3.7.2 MASH vs. YouTube Multiview Rate Adaptation . 41 3.7.3 Fairness and Comparisons vs. others . 44 3.8 Summary . 46 4 Joint Content Distribution and Traffic Engineering of Adaptive Videos in Telco-CDNs 48 4.1 Introduction . 48 4.2 Related Work . 50 4.3 System Models and Problem Definition . 51 4.3.1 Multimedia Content Model . 51 4.3.2 Telco-CDN Model . 53 4.3.3 Problem Definition . 53 4.4 Overview of CAD . 54 4.5 Details of CAD . 55 4.6 Evaluation . 59 4.6.1 Implementation . 59 4.6.2 Experimental Setup . 60 4.6.3 Results . 63 4.7 Summary . 68 5 Stateless Traffic-Engineered Multicast Forwarding 69 5.1 Introduction . 69 5.2 Related Work . 71 5.3 System Model and Problem Definition . 73 5.4 Proposed STEM System . 74 viii 5.4.1 Overview . 74 5.4.2 Label Types in STEM . 75 5.4.3 Creating STEM Labels at the Controller . 77 5.4.4 Processing STEM Packets in the Data Plane . 80 5.4.5 Efficiency and Generality Analysis of STEM . 82 5.4.6 Illustrative Example . 83 5.5 Evaluation in a Testbed . 84 5.5.1 Testbed Setup . 85 5.5.2 Results . 86 5.6 Evaluation using Simulation . 87 5.6.1 Simulation Setup . 87 5.6.2 Results . 89 5.7 Summary . 93 6 Helix: Efficient Multicast Forwarding 94 6.1 Introduction . 94 6.2 Proposed Helix System . 95 6.2.1 Overview . 95 6.2.2 Creating Labels . 97 6.2.3 Processing Labeled Packets . 103 6.2.4 Illustrative Example . 105 6.3 Evaluation in NetFPGA Testbed . 107 6.3.1 Testbed Setup and Algorithm Implementation . 107 6.3.2 Results . 109 6.4 Evaluation using Simulation . 111 6.4.1 Setup . ..