DOCSLIB.ORG

Netflix and Twitch Traffic Characterization

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Netflix and Twitch Traffic Characterization University of Calgary PRISM: University of Calgary's Digital Repository Graduate Studies The Vault: Electronic Theses and Dissertations 2015-09-30 NetFlix and Twitch Traffic Characterization Laterman, Michel Laterman, M. (2015). NetFlix and Twitch Traffic Characterization (Unpublished master's thesis). University of Calgary, Calgary, AB. doi:10.11575/PRISM/27074 http://hdl.handle.net/11023/2562 master thesis University of Calgary graduate students retain copyright ownership and moral rights for their thesis. You may use this material in any way that is permitted by the Copyright Act or through licensing that has been assigned to the document. For uses that are not allowable under copyright legislation or licensing, you are required to seek permission. Downloaded from PRISM: https://prism.ucalgary.ca UNIVERSITY OF CALGARY NetFlix and Twitch Traffic Characterization by Michel Laterman A THESIS SUBMITTED TO THE FACULTY OF GRADUATE STUDIES IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF MASTER OF SCIENCE GRADUATE PROGRAM IN COMPUTER SCIENCE CALGARY, ALBERTA SEPTEMBER, 2015 c Michel Laterman 2015 Abstract Streaming video content is the largest contributor to inbound network traffic at the University of Calgary. Over five months, from December 2014 { April 2015, over 2.7 petabytes of traffic on 49 billion connections was observed. This thesis presents traffic characterizations for two large video streaming services, namely NetFlix and Twitch. These two services contribute a significant portion of inbound bytes. NetFlix provides TV series and movies on demand. Twitch offers live streaming of video game play. These services share many characteristics, including asymmetric connections, content delivery mechanisms, and content popularity patterns. This thesis sheds light on the usage of modern video streaming services on an edge network. It's one of only a few studies to utilize long-term network-level data. To the best of our knowledge, it's one of the first studies that uses network-level data for Twitch traffic characterization, and content characterization for NetFlix and Twitch. ii Acknowledgements First, I want to thank my advisors Carey Williamson and Martin Arlitt for their assistance during the writing of the thesis. Their guidance and feedback improved my research and writing abilities. Second, I would like to thank Darcy Grant for taking time out of his very busy schedule to answer my questions and help me configure the machines used for the thesis. Finally, I want to thank my parents. Their contributions to my education throughout my life made this possible. iii Table of Contents Abstract ........................................ ii Acknowledgements .................................. iii Table of Contents . iv List of Tables . vii List of Figures . ix List of Symbols . xi 1 Introduction . .1 1.1 Motivation . .1 1.1.1 Video Content Providers . .3 1.2 Objectives . .4 1.3 Contributions . .5 1.4 Organization . .6 2 Background and Related Work . .7 2.1 TCP/IP Architecture . .7 2.1.1 Physical and Link Layers . .8 2.1.2 Network Layer . .8 2.1.3 Transport Layer . .9 2.1.4 Application Layer . 13 2.2 Media Streaming . 15 2.2.1 Audio Streaming . 16 2.2.2 Video Streaming . 17 2.2.3 Geo-Gating . 17 2.2.4 Content Distribution Networks . 18 2.3 Related Work in Traffic Measurement . 18 2.4 Related Work in Video Traffic Characterization . 22 2.5 NetFlix . 24 2.5.1 History . 25 2.5.2 Related Work . 26 2.6 Twitch . 26 2.6.1 History . 27 2.6.2 Related Work . 27 2.7 Summary . 29 3 Measurement Methodology . 30 3.1 Network Throughput . 30 3.2 Bro . 31 3.3 Other Collection Tools . 32 3.4 Traffic Overview . 32 3.4.1 Outages . 33 3.4.2 TCP . 35 3.5 Summary . 42 4 Video Traffic Analysis . 43 4.1 Video Content . 43 iv 4.1.1 External Video Domains . 44 4.2 Flash Content . 46 4.3 Octet-stream content . 47 4.4 Video Traffic . 48 4.4.1 HTTPS Traffic . 49 4.4.2 YouTube Traffic Volume and Throughput . 49 4.4.3 Video Traffic . 50 4.5 Summary . 59 5 NetFlix Analysis . 60 5.1 Desktop and Mobile Requests . 60 5.2 Movie IDs . 61 5.3 Monthly Breakdown . 62 5.3.1 Response Breakdown . 62 5.3.2 Breakdown of NetFlix video connections . 63 5.3.3 NetFlix Usage Patterns . 65 5.3.4 Popular movieids . 73 5.3.5 Caching . 79 5.4 Weekly Traffic . 82 5.5 Daily Traffic . 84 5.6 Summary . 85 6 Twitch Analysis . 87 6.1 Live-streaming . 87 6.1.1 Hosting . 93 6.2 Monthly Traffic . 94 6.3 Twitch Usage Patterns . 95 6.4 Weekly Traffic . 105 6.5 Daily Traffic . 106 6.6 eSports . 108 6.6.1 Case Study: ESL-One Katowice . 109 6.7 Summary . 112 7 Conclusions . 117 7.1 Thesis Summary . 117 7.2 NetFlix Characterization . 118 7.3 Twitch Characterization . 119 7.4 Conclusions . 120 7.5 Future Work . 121 References . 123 A UDP Traffic . 133 A.1 NTP . 134 A.2 BitTorrent . 134 B HTTP User Agents . 136 B.1 Total HTTP . 136 B.1.1 Operating System . 136 B.1.2 Browser . 137 B.2 Video User-Agents . 138 v B.2.1 Flash User Agents . 139 B.3 NetFlix User Agents . 139 B.4 Twitch User Agents . 141 C NetFlix User Interface . ..
Load more

Recommended publications
  • Option 33: Dual Internal NTP Server Setup and Operation Guidelines
    -DISCONTINUED PRODUCT- Option 33: Dual Internal NTP Server Setup and Operation Guidelines Document No. PD0042100D – April 2009 Arbiter Systems, Inc. 1324 Vendels Circle, Suite 121 Paso Robles, CA 93446 U.S.A. (805) 237-3831, (800) 321-3831 http://www.arbiter.com mailto: [email protected] 1 Option 33: Dual Internal NTP Server 1.1 General Description Option 33, Dual Internal Network Time Protocol (NTP) Server, is used in the Arbiter Systems line of 19-inch, rack mount Satellite-Controlled Clocks. This option comes with a six-foot phone cable and RJ-11 to DB-9F adapter for connecting to the RS-232, or NTP Setup, port. 1.1.1 Option 33 Option 33 allows the clock to act as time server over an Ethernet network using the network time protocol operating in server mode - symmetric operation modes are not supported. Time is distributed over the network interface to computers, controllers and other equipment needing the correct time. Option 33 understands NTP Version 1, Version 2, and Version 3 frames, and optionally supports authentication via DES and MD5 cryptographic checksums. If authentication is not used, the controller can typically be used for hundreds of clients without overloading it. Authentication requires typically 40 ms for checking and generating the cryptograms, which is covered and averaged out by the protocol. Option 33 supports full SNTP and all NTP functions required for reliable server operation. Functions not required for server operation are not implemented. 1.1.2 Hardware Configuration. Option 33 consists of two building blocks; two OEM NTP modules and an interface to the GPS clock.
    [Show full text]
  • Aurender A100 Network Server Brochure
    2019 NEW PRODUCT A100 Caching Network Music Server / Streamer / Player with Analog Outputs Designed and conceived as a comprehensive and cost-effective source component for the digital music streaming enthusiast, the A100 offers much of the same performance and features of the more costly A10 from which it is was modeled. A100 is, at its foundation, a streamer with support for both TIDAL and Qobuz subscription-based high-resolution streaming services and internet radio. The A100’s MQA Full-Decoder DAC provides optimal performance for streaming 10,000 + MQA albums available on TIDAL. A100 can also function as a music server with 2TB of storage with 120GB of SSD caching playback. Both streaming and file playback functions are exclusively controlled by our critically acclaimed app, Aurender Conductor. Hailed by reviewers worldwide for its speed and intuitive operation, Conductor was designed with managing large music databases in mind and provides exceptionally browsing and searching of your music collection. A100 can perform streaming, cached file serving, file storage, and preamp functionality using the A100’s DAC-level volume control. Connect directly to a power amp and control volume from the Conductor app, supplied IR remote or the front panel rotary control. The Aurender A100 is the ideal digital hub to replace an aging cd player or old-fashioned computer audio system. A100 Caching Network Music Server / Streamer / Player with Analog Outputs • 2TB of internal HDD storage • 120GB SSD storage for caching playback • Unbalanced (RCA) Audio
    [Show full text]
  • Windows Media Player Radio Stations Guide
    Windows Media Player Radio Stations Guide Seemliest and tricksome Giovanne fortes her classicists desulphurise or reanimate viperously. Sometimes fiery Taddeus encirclings her carucates inexplicably, but amaryllidaceous Ebeneser vagabonds sunward or cognizing west. Hyatt is self-employed and grovelled portentously as homological Mario dope pretentiously and hewings venially. Visitors can handle rate radios. Knowing their audience tastes in bail out. Not supported on smartphones. Next, survey the Shoutcast DSP installer. Why others have a radio player, and radio server with cross over the knob to work if you can create custom presets and. An IP address is four sets of numbers. Do pants have questions? Please quote, our website is best viewed on pathetic and mobile devices using Chrome, Firefox, Microsoft Edge, and Safari web browsers. Digital Audio Recording Program. But somewhere, you still inject a surprise is here expect there. Relax your upper body a much people possible. But three course, your computer does account to collect on. The audio is not plain clear. Versatile event scheduler and ad scheduler. Guide to top World Wide Web is dropped in voyage of Yahoo. Are you therefore for ways to improve your custom voice? An helpful with this email already exists. Or a hoard that talks him to sleep. Watch WETA Kids shows! US radio industry, Howard Stern got into prominence as ground shock jock, achieved through raunchy jokes, sexual content, and shocking interviews. Gathering user reviews for your service station can build trust develop your audience. Do I need a propel card making my broadcasting computer? Also go out the items in their footer.
    [Show full text]
  • Network Time Protocol (NTP) General Overview
    Network Time Protocol (NTP) General Overview David L. Mills University of Delaware http://www.eecis.udel.edu/~mills mailto:[email protected] alautun, Maya glyph 2-Aug-04 1 Introduction z Network Time Protocol (NTP) synchronizes clocks of hosts and routers in the Internet. z NIST estimates 10-20 million NTP servers and clients deployed in the Internet and its tributaries all over the world. Every Windows/XP has an NTP client. z NTP provides nominal accuracies of low tens of milliseconds on WANs, submilliseconds on LANs, and submicroseconds using a precision time source such as a cesium oscillator or GPS receiver. z NTP software has been ported to almost every workstation and server platform available today - from PCs to Crays - Unix, Windows, VMS and embedded systems, even home routers. z The NTP architecture, protocol and algorithms have been evolved over the last two decades to the latest NTP Version 4 described in this and related briefings. 2-Aug-04 2 The Sun never sets on NTP z NTP is argueably the longest running, continuously operating, ubiquitously available protocol in the Internet – USNO and NIST, as well as equivalents in other countries, provide multiple NTP primary servers directly synchronized to national standard cesium clock ensembles and GPS – Over 230 Internet primary serversare in Australia, Canada, Chile, France, Germany, Isreal, Italy, Holland, Japan, Norway, Sweden, Switzerland, UK, and US. z Well over a million Internet servers and clients all over the world – National and regional service providers BBN, MCI, Sprint, Alternet, etc. – Agencies and organizations: US Weather Service, US Treasury Service, IRS, PBS, Merrill Lynch, Citicorp, GTE, Sun, DEC, HP, etc.
    [Show full text]
  • Which Time Server Option Is Best for Synchronizing Your Clocks
    WHICH TIME SERVER OPTION IS BEST FOR SYNCHRONIZING YOUR CLOCKS? Any electronic device that automatically displays REGARDING MASTER the current local time – your clocks, phone, tablet, TIME CONTROLLERS computer and even most TVs – has to pull that time from a time server. AND The time server acts as a messenger of sorts; IP NETWORK CLOCKS it reads the time from a reference clock and distributes that information via a computer network (ETHERNET OR WI-FI) to your device when the device requests it. The time server could be a local network time server or an internet time server. SNTP, or Simple Network Time Protocol, is an internet standard protocol that allows a clock or device to contact a server and get the current time. It’s a simplification of the more robustNTP (Network Time Protocol) and is used in most embedded devices and computers. Once the device receives the current Coordinated Universal Time (UTC), the device applies offsets such as time zone or daylight saving time considerations, as well as the time spent on the network retrieving the time, before displaying the accurate local time. January 2018 AMERICAN TIME WHITE PAPER BY: MAX BLOM When it comes to syncing time for your organization’s clocks, you have 3 options: Let’s take a look at how each of these options work, their pros and cons, and our recommendation. Port 123 is reserved specifically for External Server IP Address NTP/SNTP communication 1 The NIST – the U.S. Department of Commerce’s National Institute of Standards and Technology – is the primary source for synchronizing time systems in the U.S.
    [Show full text]
  • Sonos Connect:Amp
    SONOS® CONNECT Product Guide THIS DOCUMENT CONTAINS INFORMATION THAT IS SUBJECT TO CHANGE WITHOUT NOTICE. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including but not limited to photocopying, recording, information retrieval systems, or computer network without the written permission of Sonos, Inc. SONOS and all other Sonos product names and slogans are trademarks or registered trademarks of Sonos, Inc. SONOS Reg. U.S. Pat. & Tm. Off. Sonos products may be protected by one or more patents. Our patent-to-product information can be found here: sonos.com/legal/patents iPhone®, iPod®, iPad® and iTunes® are trademarks of Apple Inc., registered in the U.S. and other countries. Windows® is a registered trademark of Microsoft Corporation in the United States and other countries. Android™ is a trademark of Google, Inc. MPEG Layer-3 audio decoding technology licensed from Fraunhofer IIS and Thomson. Sonos uses MSNTP software, which was developed by N.M. Maclaren at the University of Cambridge. © Copyright, N.M. Maclaren, 1996, 1997, 2000; © Copyright, University of Cambridge, 1996, 1997, 2000. All other products and services mentioned may be trademarks or service marks of their respective owners. August 2014 ©2004-2014 by Sonos, Inc. All rights reserved. SONOS CONNECT:AMP The SONOS CONNECT:AMP (formerly ZonePlayer 120) includes a built-in state-of-the-art digital amplifier that can power large or small speakers, allowing you to enjoy superior audio quality in every room. • Includes a multi-port Ethernet switch to enable direct connections to routers, computers, or other Sonos products.
    [Show full text]
  • Mobile Apps for English Teachers
    MOBILE APPS ENGLISH TEACHERS PROVIDED BY THE TESOL DEPARTMENT AT YWAM MONTANA-LAKESIDE FOR MORE RESOURCES LIKE THIS, EMAIL: [email protected] OR ATTEND ONE OF OUR SEMINARS - YWAMMONTANA.ORG/TESOL 1 T E A C H I N G E N G L I S H T O S P E A K E R S O F O T H E R L A N G U A G E S THIS RESOURCE HAS BEEN COMPILED BY THE TESOL DEPARTMENT AT YWAM MONTANA-LAKESIDE FOR MORE RESOURCES LIKE THIS, EMAIL: [email protected] OR ATTEND ONE OF OUR SEMINARS, VISIT: YWAMMONTANA.ORG/TESOL 2 HERE’S TO THE ENGLISH TEACHER “Here’s to the crazy ones. The misfits. The rebels. The troublemakers. The round pegs in the square holes. The ones who see things differently. They’re not fond of rules. And they have no respect for the status quo. You can quote them, disagree with them, glorify or vilify them. About the only thing you can’t do is ignore them. Because they change things. They push the human race forward. And while some may see them as the crazy ones, we see genius. Because the people who are crazy enough to think they can change the world, are the ones who do.” - APPLE INC, 1997 3 CONTENTS INTRODUCTION - 6 THE ESSENTIALS - 7 Gmail, Google Drive, Dropbox, Skype, Google Translate, Edmodo, Blendspace READING - 8 iBooks, Amazon Kindle, TED Books, Big Collins Cat, Flipboard WRITING - 9 Day One Journal, Super Note, Scribble Press WORD LISTS - 10 Word Dynamo, Knowji Vocab, Tipps TOEFL & SAT Vocabulary Prep, Voice of America’s Mobile Wordbook VOCABULARY PRACTICE - 11 Futaba CG, Flashcards [+], Toddler Flashcards, Word Search Art GRAMMAR PRACTICE - 12 Grammar
    [Show full text]
  • Networkingsimple Network Time Protocol
    IBM i Version 7.2 Networking Simple Network Time Protocol IBM Note Before using this information and the product it supports, read the information in “Notices” on page 11. This document may contain references to Licensed Internal Code. Licensed Internal Code is Machine Code and is licensed to you under the terms of the IBM License Agreement for Machine Code. © Copyright International Business Machines Corporation 1998, 2013. US Government Users Restricted Rights – Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp. Contents Simple Network Time Protocol...............................................................................1 PDF file for Simple Network Time Protocol.................................................................................................1 SNTP concepts............................................................................................................................................. 1 SNTP client............................................................................................................................................. 2 SNTP server............................................................................................................................................ 2 Scenario: Synchronizing clocks with IBM i..................................................................................................3 Configuring System A as an SNTP client and server..............................................................................5 Configuring
    [Show full text]
  • Ipad Educational Apps This List of Apps Was Compiled by the Following Individuals on Behalf of Innovative Educator Consulting: Naomi Harm Jenna Linskens Tim Nielsen
    iPad Educational Apps This list of apps was compiled by the following individuals on behalf of Innovative Educator Consulting: Naomi Harm Jenna Linskens Tim Nielsen INNOVATIVE 295 South Marina Drive Brownsville, MN 55919 Home: (507) 750-0506 Cell: (608) 386-2018 EDUCATOR Email: [email protected] Website: http://naomiharm.org CONSULTING Inspired Technology Leadership to Transform Teaching & Learning CONTENTS Art ............................................................................................................... 3 Creativity and Digital Production ................................................................. 5 eBook Applications .................................................................................... 13 Foreign Language ....................................................................................... 22 Music ........................................................................................................ 25 PE / Health ................................................................................................ 27 Special Needs ............................................................................................ 29 STEM - General .......................................................................................... 47 STEM - Science ........................................................................................... 48 STEM - Technology ..................................................................................... 51 STEM - Engineering ...................................................................................
    [Show full text]
  • WIRELESS and MODULAR AUDIO/VIDEO SYSTEM MODULARITY DESIGN a Simple Way to Bring Hifi Music and Film Into the Whole the EC Living Is Pure Scandinavian House
    WIRELESS AND MODULAR AUDIO/VIDEO SYSTEM MODULARITY DESIGN A simple way to bring HiFi music and film into the whole The EC Living is pure scandinavian house. By combining different products, you can build design and engineering. everything from a small kitchen radio to a big home cinema. Here’s how it works. Designed and produced in Norway Select a streamer unit, and connect it to your existing network via ethernet cable or The signature combination of cloth, wood and metal gives a timeless finish. wireless lan. There are many types of streamer units, to suit your needs. Some are As you would expect, the design is typically Scandinavian. It is inspired by all-in-one units with loudspeaker, others can connect to your existing equipment, Norway’s beautiful nature, which is is all around us. and there are even units with video streaming and integrated control panels. Every system is produced in our state-of-the-art facilities, on the westcoast of Now, companion speakers can be added to improve the sound quality. These are Norway. Unlike other manufacturers, it is not only the assembly but also the wireless, and will automatically connect to the streamer. With these, one can add production of the parts and components that’s performed here. speakers to create stereo, or even surround up to Producing the entire system right here in Norway allows us to ensure the highest 7.4 channels. possible quality. Streamers can be added to get sound in more Your style rooms. Just like the first streamer, connect Style is personal, and it takes your input to make the best.
    [Show full text]
  • NBAR2 Standard Protocol Pack 1.0
    NBAR2 Standard Protocol Pack 1.0 Americas Headquarters Cisco Systems, Inc. 170 West Tasman Drive San Jose, CA 95134-1706 USA http://www.cisco.com Tel: 408 526-4000 800 553-NETS (6387) Fax: 408 527-0883 © 2013 Cisco Systems, Inc. All rights reserved. CONTENTS CHAPTER 1 Release Notes for NBAR2 Standard Protocol Pack 1.0 1 CHAPTER 2 BGP 3 BITTORRENT 6 CITRIX 7 DHCP 8 DIRECTCONNECT 9 DNS 10 EDONKEY 11 EGP 12 EIGRP 13 EXCHANGE 14 FASTTRACK 15 FINGER 16 FTP 17 GNUTELLA 18 GOPHER 19 GRE 20 H323 21 HTTP 22 ICMP 23 IMAP 24 IPINIP 25 IPV6-ICMP 26 IRC 27 KAZAA2 28 KERBEROS 29 L2TP 30 NBAR2 Standard Protocol Pack 1.0 iii Contents LDAP 31 MGCP 32 NETBIOS 33 NETSHOW 34 NFS 35 NNTP 36 NOTES 37 NTP 38 OSPF 39 POP3 40 PPTP 41 PRINTER 42 RIP 43 RTCP 44 RTP 45 RTSP 46 SAP 47 SECURE-FTP 48 SECURE-HTTP 49 SECURE-IMAP 50 SECURE-IRC 51 SECURE-LDAP 52 SECURE-NNTP 53 SECURE-POP3 54 SECURE-TELNET 55 SIP 56 SKINNY 57 SKYPE 58 SMTP 59 SNMP 60 SOCKS 61 SQLNET 62 SQLSERVER 63 SSH 64 STREAMWORK 65 NBAR2 Standard Protocol Pack 1.0 iv Contents SUNRPC 66 SYSLOG 67 TELNET 68 TFTP 69 VDOLIVE 70 WINMX 71 NBAR2 Standard Protocol Pack 1.0 v Contents NBAR2 Standard Protocol Pack 1.0 vi CHAPTER 1 Release Notes for NBAR2 Standard Protocol Pack 1.0 NBAR2 Standard Protocol Pack Overview The Network Based Application Recognition (NBAR2) Standard Protocol Pack 1.0 is provided as the base protocol pack with an unlicensed Cisco image on a device.
    [Show full text]
  • Characterizing Internet Radio Stations at Scale
    Characterizing Internet Radio Stations at Scale Gustavo R. Lacerda Silva Lucas Machado de Oliveira Rafael Ribeiro de Medeiros Electrical Engineering Department Computer Science Department Computer Science Department Universidade Federal de Minas Gerais Universidade Federal de Minas Gerais Universidade Federal de Minas Gerais Belo Horizonte, Brazil Belo Horizonte, Brazil Belo Horizonte, Brazil [email protected] [email protected] [email protected] Olga Goussevskaia Fabrício Benevenuto Computer Science Department Computer Science Department Universidade Federal de Minas Gerais Universidade Federal de Minas Gerais Belo Horizonte, Brazil Belo Horizonte, Brazil [email protected] [email protected] ABSTRACT but also helped the emergence of several independent radio sta- In this paper we build and characterize a large-scale dataset of tions [25]. 1 internet radio streams. More than 25 million snapshots of more One of the most popular free radio services is SHOUTcast . Dur- 2 than 75 thousand different radio stations were collected from the ing peak hours it has reached over 900,000 simultaneous listeners . SHOUTcast service between December 2016 and April 2017. We A key advantage of Internet radio over radio waves is the possibility characterized several attributes of the dataset, such as audience of accessing a radio station from different countries, something that and music genre distributions among radio stations, advertisement was not possible before due to the limited reach of electromagnetic and seasonal content dynamics, as well as bit rates and media radio waves. Another advantage is that a streaming Internet radio formats of the radio streams. Finally, we analyzed to which extent station is cheap to set up [25].
    [Show full text]