University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2015-09-30 NetFlix and Twitch Traffic Characterization Laterman, Michel Laterman, M. (2015). NetFlix and Twitch Traffic Characterization (Unpublished master's thesis). University of Calgary, Calgary, AB. doi:10.11575/PRISM/27074 http://hdl.handle.net/11023/2562 master thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca UNIVERSITY OF CALGARY NetFlix and Twitch Traffic Characterization by Michel Laterman A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE GRADUATE PROGRAM IN COMPUTER SCIENCE CALGARY, ALBERTA SEPTEMBER, 2015 c Michel Laterman 2015 Abstract Streaming video content is the largest contributor to inbound network traffic at the University of Calgary. Over five months, from December 2014 { April 2015, over 2.7 petabytes of traffic on 49 billion connections was observed. This thesis presents traffic characterizations for two large video streaming services, namely NetFlix and Twitch. These two services contribute a significant portion of inbound bytes. NetFlix provides TV series and movies on demand. Twitch offers live streaming of video game play. These services share many characteristics, including asymmetric connections, content delivery mechanisms, and content popularity patterns. This thesis sheds light on the usage of modern video streaming services on an edge network. It's one of only a few studies to utilize long-term network-level data. To the best of our knowledge, it's one of the first studies that uses network-level data for Twitch traffic characterization, and content characterization for NetFlix and Twitch. ii Acknowledgements First, I want to thank my advisors Carey Williamson and Martin Arlitt for their assistance during the writing of the thesis. Their guidance and feedback improved my research and writing abilities. Second, I would like to thank Darcy Grant for taking time out of his very busy schedule to answer my questions and help me configure the machines used for the thesis. Finally, I want to thank my parents. Their contributions to my education throughout my life made this possible. iii Table of Contents Abstract ........................................ ii Acknowledgements .................................. iii Table of Contents . iv List of Tables . vii List of Figures . ix List of Symbols . xi 1 Introduction . .1 1.1 Motivation . .1 1.1.1 Video Content Providers . .3 1.2 Objectives . .4 1.3 Contributions . .5 1.4 Organization . .6 2 Background and Related Work . .7 2.1 TCP/IP Architecture . .7 2.1.1 Physical and Link Layers . .8 2.1.2 Network Layer . .8 2.1.3 Transport Layer . .9 2.1.4 Application Layer . 13 2.2 Media Streaming . 15 2.2.1 Audio Streaming . 16 2.2.2 Video Streaming . 17 2.2.3 Geo-Gating . 17 2.2.4 Content Distribution Networks . 18 2.3 Related Work in Traffic Measurement . 18 2.4 Related Work in Video Traffic Characterization . 22 2.5 NetFlix . 24 2.5.1 History . 25 2.5.2 Related Work . 26 2.6 Twitch . 26 2.6.1 History . 27 2.6.2 Related Work . 27 2.7 Summary . 29 3 Measurement Methodology . 30 3.1 Network Throughput . 30 3.2 Bro . 31 3.3 Other Collection Tools . 32 3.4 Traffic Overview . 32 3.4.1 Outages . 33 3.4.2 TCP . 35 3.5 Summary . 42 4 Video Traffic Analysis . 43 4.1 Video Content . 43 iv 4.1.1 External Video Domains . 44 4.2 Flash Content . 46 4.3 Octet-stream content . 47 4.4 Video Traffic . 48 4.4.1 HTTPS Traffic . 49 4.4.2 YouTube Traffic Volume and Throughput . 49 4.4.3 Video Traffic . 50 4.5 Summary . 59 5 NetFlix Analysis . 60 5.1 Desktop and Mobile Requests . 60 5.2 Movie IDs . 61 5.3 Monthly Breakdown . 62 5.3.1 Response Breakdown . 62 5.3.2 Breakdown of NetFlix video connections . 63 5.3.3 NetFlix Usage Patterns . 65 5.3.4 Popular movieids . 73 5.3.5 Caching . 79 5.4 Weekly Traffic . 82 5.5 Daily Traffic . 84 5.6 Summary . 85 6 Twitch Analysis . 87 6.1 Live-streaming . 87 6.1.1 Hosting . 93 6.2 Monthly Traffic . 94 6.3 Twitch Usage Patterns . 95 6.4 Weekly Traffic . 105 6.5 Daily Traffic . 106 6.6 eSports . 108 6.6.1 Case Study: ESL-One Katowice . 109 6.7 Summary . 112 7 Conclusions . 117 7.1 Thesis Summary . 117 7.2 NetFlix Characterization . 118 7.3 Twitch Characterization . 119 7.4 Conclusions . 120 7.5 Future Work . 121 References . 123 A UDP Traffic . 133 A.1 NTP . 134 A.2 BitTorrent . 134 B HTTP User Agents . 136 B.1 Total HTTP . 136 B.1.1 Operating System . 136 B.1.2 Browser . 137 B.2 Video User-Agents . 138 v B.2.1 Flash User Agents . 139 B.3 NetFlix User Agents . 139 B.4 Twitch User Agents . 141 C NetFlix User Interface . ..