Improving Peer-to-Peer Video Streaming . Improving Peer-to-Peer Video Streaming Proefschrift ter verkrijging van de graad van doctor aan de Technische Universiteit Delft, op gezag van de Rector Magnificus prof. ir. K. C. A. M. Luyben, voorzitter van het College voor Promoties, in het openbaar te verdedigen op vrijdag 8 april 2016 om 15:00 uur door Riccardo Petrocco ingenieur in de technische informatica geboren te Rome, Italy. Dit proefschrift is goedgekeurd door de promotor: Prof. dr. ir. D. H. J. Epema Copromotor: Dr. ir. J. A. Pouwelse Samenstelling promotiecommissie: Rector Magnificus, voorzitter Prof. dr. ir. D. H. J. Epema Delft University of Technology, promotor Dr. ir. J. A. Pouwelse Delft University of Technology, copromotor Prof. dr. ir. R. L. Lagendijk Delft University of Technology Prof. dr. K. G. Langendoen Delft University of Technology Prof. dr. J. J. Lukkien Eindhoven University of Technology Dr. P. S. César Garcia CWI Amsterdam Dr. A. Legout INRIA, Sophia Antipolis The work described in this thesis was supported by the Future and Emerging Tech- nologies program FP7-COSI-ICT of the European Commission through the P2P- Next project (grant no.:216217) Printed by: CPI-Koninklijke Wöhrmann – Zutphen Front & Back: Alessandro Petrocco © 2016 Riccardo Petrocco ISBN 978-94-6328-031-0 An electronic version of this dissertation is available at http://repository.tudelft.nl/. To my mother, for her love and encouragement. To my father, and his everlasting support. Riccardo . Acknowledgements The work presented in this thesis would not be possible without the help, sup- port,collaboration, and friendship of all the people that have been part of my PhD track. Dick, thank you for all your help, your support, your guidance, and first of all for believing in me. I admire your infinite patience when reviewing my work, your calm personality, and your humor. I’m very grateful for the freedom you gave me. You never imposed your ideas, or tried to push me in another direction, even when I kept running in circles and it seemed like I would never achieve my goal. Thank you for teaching me how to write scientific papers, and the clarity and quality of the work in this thesis is only thanks to your guidance. Johan, thank you for your great energy, enthusiasm, and passion. Thank you for the long discussions and for always keeping an open mind. I always admired how, even when we wouldn’t agree, you kept a smile and never took anything personal. Also, thank you for putting together such a nice group of people, the best colleagues I ever had. All the best for you future and in keeping the research at PDF active. I’m sure you are already looking into the upcoming hot topics. A special thank goes to Arno. Thank you for your support and for being such a good friend over the years. I truly enjoyed working with you, travelling to confer- ences and working as a team on projects such as P2P-Next. I wish you all the best for your new family, and I really hope that our working lives will meet again. To my colleagues, Adele, Boxun, Boudewijn, Dimitra, Elric, Mihai, Niels, Nitin, Lucia, Rahim, and Rameez, thank you for being such great friends. It has been a real pleasure to share this long journey with all of you. I will always keep incredible memories of working at TUDelft, the movie and gaming nights, all the lunches and dinners, the travels, and generally the nice atmosphere that made coming to work such a pleasure. I wanted to thank each and every one of you separately, but then this section would become longer than the introduction! I know that you are all facing great new adventures, and I’m sure our paths will meet again. Dave, Naza, and Tamás, thank you for all the long talks, and for giving me a different view on what it means to do research. Alex, Andreas, Bogdan, Henk, and Otto, even through we never worked together, I really valued all of our chats and it has been great to share the office with you. To all of my friends outside the PDS group, a huge thanks for everything. You are too many to be included in this acknowledgement, and you know that you are the ones that made it extremely difficult to leave The Netherlands, you made my life richer. Finally, my very best thanks go to my family and to partner, Victoria. You made all of this possible, supporting me in my decisions, and being the stable point in a chaotic journey. vii . Contents Acknowledgements vii 1 Introduction 1 1.1 Background . 2 1.2 Research Context . 8 1.3 Problem Statement . 11 1.4 Contributions and Thesis Outline . 12 1.5 Design and Research Methodology . 14 2 A Framework for Distributing Scalable Content over Peer-to- Peer Networks 17 2.1 Related Work . 18 2.2 NextShare Integration . 19 2.3 Producer-Side Architecture . 21 2.4 Consumer-side Architecture . 24 2.5 Evaluation. 29 2.6 Conclusions and Future Work . 35 3 Deftpack: A Robust Piece-Picking Algorithm for Scalable Video Coding in P2P Systems 37 3.1 Problem Statement . 38 3.2 Related Work . 39 3.3 Deftpack. 41 3.4 Experimental Setup. 45 3.5 Experimental Results. 47 3.6 Conclusion . 51 4 The Peer-to-Peer Streaming Peer Protocol (PPSPP) 53 4.1 Overall Operation . 55 4.2 Messages . 56 4.3 Chunk Addressing Schemes . 64 4.4 Content Integrity Protection . 67 4.5 Live Streaming . 73 4.6 Extensibility . 78 5 Libswift: the PPSPP Reference Implementation 79 5.1 Bins and Binmaps for Data Availability . 80 5.2 Libswift’s State Machine . 84 5.3 Pull Mechanism. 87 5.4 Experiments. 90 5.5 Conclusions and Future Work . 95 ix .x 6 Performance Analysis of the Libswift P2P Streaming Protocol 97 6.1 Libswift . 98 6.2 Download Scheduling . 101 6.3 Experimental set-up . 105 6.4 Experimental results . 109 6.5 Conclusions and Future Work . 114 7 Hiding User Content Interest while Preserving P2P Performance 117 7.1 Related Work . 118 7.2 System Design . 119 7.3 Security Analysis . 124 7.4 Experiments. 127 7.5 Conclusion . 131 8 Conclusion 133 8.1 Conclusions . 133 8.2 Reflection on the State of P2P Streaming . 134 8.3 Suggestions for Future Work . 136 Bibliography 139 Summary 151 Samenvatting 153 Curriculum Vitæ 155 Publications of the author 157 . 1 Introduction Since its debut in the late nineteenth century, the motion picture has generated enor- mous interest. Around the same period, the first mechanical devices able to scan and reproduce images appeared [21], but it was not until 1925 that the modern inter- pretation of television was invented [111]. From that moment onwards, television underwent everlasting improvement, both in quality and in delivery mechanisms, making it accessible to a continuously growing audience. In 1936, the BBC started broadcasting the first public television as we know it today, introducing ”the magic of television” 1 to hundreds of thousands of viewers. Television was the only means of delivering video content to peoples’ homes until the arrival of the Internet in the late 20th century. While the first patent for streaming over electrical lines was already granted at the beginning of the century [77], it was not until the late 1990s that the technological advances allowed its deployment. The commercialization of the Internet, and the arrival of the first personal computers, enabled the transmis- sion of video content via this novel medium for the first time. This encouraged the standardization of protocols and procedures, and the coining of new terms such as Video on Demand [94]. Nowadays, the majority of traffic over the Internet represents video content. Following a recent trend analysis [47], all forms of IP video together will constitute around the 80 to 90 percent of total IP traffic by 2018. The content delivery infras- tructure that dominated Internet video distribution over the last two decades has been the client-server architecture [63]. In a client-server architecture, the viewer requests and retrieves the video directly from the content provider, which has di- rect control over the service it provides. While this has worked well in the past, as the content providers were the only ones that could afford the prohibitive costs of producing and distributing video content, this trend is slowly changing. The current generation is moving from being passive viewers, or content consumers, to becoming active content producers. This transition needs to be coupled with 1Mitchell’s introductory words to the first public broadcast performed by the BBC [10]. 1 .2 a change to a content delivery infrastructure that allows users to distribute their content directly from their homes, without having to rely on a centralised author- ity. Peer-to-Peer (P2P) systems are fully distributed networks where its users are both content providers and content consumers. Their widespread adoption [47] has shown how they provide a valuable solution for distributing files to a wide audience, but they require several enhancements, from the original design, in order to qualify for the distribution of time-critical content such as video. In this thesis, we improve P2P streaming systems, presenting the design of ar- chitectures and algorithms that address the distribution of time-critical content in a fully distributed infrastructure. We focus on three major challenges that affect P2P streaming systems. First, we address the diversity of users that access the same content with different devices and connections. We present the architectural description and evaluation of the first open-source P2P system capable of delivering scalable video content, where the quality of the video changes based on the availabil- ity of resources.