Crowdsourced Livecast Systems: Measurement and Enhancement by Cong Zhang M.Sc., Zhengzhou University, 2012 B.Sc., Information Engineering University, 2008 Thesis Submitted in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy in the School of Computing Science Faculty of Applied Sciences c Cong Zhang 2018 SIMON FRASER UNIVERSITY Spring 2018 Copyright in this work rests with the author. Please ensure that any reproduction or re-use is done in accordance with the relevant national copyright legislation. Approval Name: Cong Zhang Degree: Doctor of Philosophy (Computing Science) Title: Crowdsourced Livecast Systems: Measurement and Enhancement Examining Committee: Chair: Dr. Ryan Shea Assistant Professor Dr. Jiangchuan Liu Senior Supervisor Professor Dr. Qianping Gu Supervisor Professor Dr. Kangkang Yin Internal Examiner Associate Professor School of Computing Science Dr. Yonggang Wen External Examiner Associate Professor School of Computer Science and Engineering Nanyang Technological University Date Defended: March 01, 2018 ii Abstract Empowered by today’s rich tools for media generation and collaborative production, mul- timedia service paradigm is shifting from the conventional single source, to multi-source, to many sources, and now towards crowdsource, where the available media sources for the content of interest become highly diverse and scalable. Such crowdsourced livecast systems as Twitch.tv, YouTube Gaming, and Periscope enable a new generation of user-generated livecast systems, attracting an increasing number of viewers all over the world. Yet the sources are managed by unprofessional broadcasters, and often have limited computation capacities and dynamic network conditions. They can even join or leave at will, or crash at any time. In this thesis, we first conduct a systematic study on the existing crowdsourced livecast sys- tems. We outline the inside architecture using both the crawled data and the captured traffic data from local broadcasters/viewers. We then reveal that a significant portion of the un- popular and dynamic broadcasters are consuming considerable system resources. Because cloud computing provides resizable, reliable, and scalable bandwidth and computational resources, which naturally becomes an effective solution to leverage heterogeneous and dyn- amic workloads. Yet, it is a challenge to utilize the resources from the cloud cost-effectively. We thus propose a cloud-assisted design to smartly ingest the sources and cooperatively utilize the resources from dedicated servers and public clouds. In current crowdsourced livecast systems, crowdsourced gamecasting is the most popular ap- plication, in which gamers lively broadcast game playthroughs to fellow viewers using their desktop, laptop, even mobile devices. These gamers’ patterns, which instantly pilot the corresponding gamecastings and viewers’ fixations, have not been explored by previous stu- dies. Since mobile gamers and eSports gamers occupy a large portion of content generators. In this thesis, we target on two typical crowdsourced gamecasting scenarios, i.e., mobile gamecasting and eSports gamecasting, respectively. We investigate the gamers’ patterns to explore their effects on viewers and employ intelligent approaches, e.g., learning-based techniques, to capture the associations between gamers’ patterns and viewers’ experiences. Then, we employ such associations to optimize the streaming transcoding and distribution. Keywords: Crowdsourced livecast; measurement; enhancement iii Dedication To my family ! iv Acknowledgements First and foremost, I would like to express my sincere gratitude to my senior supervisor, Dr. Jiangchuan Liu for his constant support and guidance throughout my PhD studies. His warm encouragement, thoughtful advice, and life wisdom also motivate me to become a good “game player”. I am also grateful to Dr. Qianping Gu, Dr. Kangkang Yin, and Dr. Yonggang Wen for serving in my examining committee. I thank them for their precious time on reviewing my work and for their advices on improving my thesis. I would like to thank Dr. Ryan Shea for charing my PhD thesis defence. I thank a number of colleagues and friends from their help and support during my stay at Simon Fraser University. In particular, I thank Dr. Feng Wang, Dr. Haiyang Wang, Dr. Fei Chen, Dr. Xiaoqiang Ma, Lei Zhang, Xiaoyi Fan, Qiyun He, Jia Zhao, Yifei (Stephen) Zhu and Silvery (Di) Fu, for helping me with the research as well as many other problems during my PhD study. Last but not least, I wish to thank my family for their love and support. This thesis is dedicated to you all ! v Table of Contents Approval ii Abstract iii Dedication iv Acknowledgements v Table of Contents vi List of Tables ix List of Figures x 1 Introduction 1 1.1 OverviewofVideoStreaming . 2 1.2 OverviewofCrowdsourcedLivecast. ..... 3 1.3 Contributions................................... 5 1.4 ThesisOrganization .............................. 6 2 A Twitch.TV-Based Measurement Study 8 2.1 InsidetheTwitchArchitecture . .... 9 2.2 View Statistics and Patterns . 12 2.2.1 PopularityandDuration. 12 2.2.2 Event- and Source-Driven Views . 14 2.3 MessagingandViewLatency . 15 2.4 LiveMessagingLatency . .. .. .. .. .. .. .. .. 16 2.4.1 BroadcastLatency ............................ 16 2.4.2 SourceSwitchingLatency . 17 2.4.3 Impact of Broadcaster’s sources . 17 2.5 Summary ..................................... 18 3 Cloud-assisted Crowdsourced Livecast 19 3.1 RelatedWork................................... 20 vi 3.2 Measurements of Crowdsourced Livecast: Twitch as a Case Study...... 22 3.2.1 Twitch-based Datasets . 22 3.2.2 Characteristics of Crowdsourced Live Broadcasters . ......... 24 3.2.3 Effects of Crowdsourced Live Events . 24 3.2.4 Popularity of Crowdsourced Live Broadcasters . ...... 25 3.2.5 Dynamics of Crowdsourced Live Broadcasters . ..... 27 3.2.6 Challenges of Hosting Unpopular Broadcasters . ...... 28 3.3 CACL:ArchitectureandDesign. 28 3.3.1 EC2-basedmeasurement. 28 3.3.2 Round-tripTime ............................. 29 3.3.3 BroadcastLatency ............................ 30 3.3.4 CACLArchitecture............................ 31 3.3.5 InitialOffloading ............................. 32 3.4 Problem Formulation and Solution . 32 3.4.1 Basic Model with Ingesting Latency . 33 3.4.2 Enhanced Model with Transcoding Latency . 34 3.4.3 Solution.................................. 35 3.5 PerformanceEvaluation . 37 3.5.1 Efficiency of Resource Allocation . 37 3.5.2 Trace-driven Simulation . 38 3.6 Summary ..................................... 40 4 Exploring Viewer Gazing Patterns for Touch-based Mobile Gamecasting 41 4.1 Background .................................... 42 4.2 Motivation .................................... 45 4.3 Interaction-Aware Design . 47 4.3.1 Touch-assisted Prediction Module . 48 4.3.2 Tile-based Optimization Module . 48 4.4 Understanding Game Touch Interactions . ...... 49 4.4.1 Touch Data Collection . 49 4.4.2 Interaction Classification . 50 4.5 Insights into Viewers’ Gazing Patterns . ....... 51 4.5.1 Gazing Data Collection . 51 4.5.2 GazingClassification. 52 4.6 Touch-Gaze Association Learning . ..... 54 4.6.1 Preliminaries ............................... 54 4.6.2 Touch-Gaze Association . 54 4.7 Tile-basedOptimization . 56 4.7.1 ProblemFormulation.. .. .. .. .. .. .. .. 56 vii 4.7.2 Solution.................................. 57 4.8 PerformanceEvaluation . 59 4.8.1 Performance of Touch-assisted Prediction . ...... 59 4.8.2 Trace-driven Simulation . 60 4.8.3 UserStudy ................................ 63 4.9 Summary ..................................... 64 5 Highlight-Aware eSports Gamecasting with Strategy-Based Prediction 65 5.1 Background .................................... 67 5.1.1 Optimization in Crowdsourced Gamecasting Services . ........ 70 5.1.2 Learning-based QoE Improvement in Multimedia Systems...... 70 5.2 MeasurementandObservation . 71 5.2.1 MeasurementSettings . .. .. .. .. .. .. .. 71 5.2.2 MeasurementResults.. .. .. .. .. .. .. .. 72 5.3 StreamingCursor:AnOverview . 72 5.4 Framework Design and Solution . 75 5.4.1 Strategy-based Prediction . 75 5.4.2 Transcoding Task Assignment Optimization . ..... 76 5.5 PerformanceEvaluation . 79 5.5.1 Strategy-based Prediction Results . 79 5.5.2 Highlight-aware Optimization Performance . ...... 81 5.6 Summary ..................................... 82 6 Conclusion 83 6.1 SummaryoftheThesis.............................. 83 6.2 FutureDirections................................ 84 6.2.1 Further Measurements on Crowdsourced Gamecasting . ...... 84 6.2.2 Further Enhancement on the Broadcaster Side . 84 6.2.3 Further Works on Mobile Gamecasing . 85 Bibliography 86 viii List of Tables Table 2.1 Twitch REST APIs used in our crawler . .... 9 Table 2.2 Configuration of broadcasters’ devices . ......... 12 Table 2.3 Configuration of viewers’ devices . ....... 12 Table 4.1 Classification and definition of gamer’s touch interactions . 42 Table 4.2 Classification and definition of viewer’s gazing patterns ........ 43 Table 4.3 Statistics of touch data . 47 Table4.4 Encodingexample . 54 Table 5.1 Configuration of Amazon EC2 instances . ...... 71 Table 5.2 Statistics of a replay file . ..... 75 Table 5.3 Performance of the strategy-based highlight prediction . 79 ix List of Figures Figure 1.1 An illustration of two crowdsourced live streams ........... 4 Figure 2.1 Device distribution of Twitch’s live sources . ........... 10 Figure 2.2 Two broadcasters/game players measurement configuration . 11 Figure 2.3 Live streams rank ordered by