University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School 9-4-2014 An Analysis of (Bad) Behavior in Online Video Games Jeremy Blackburn University of South Florida, [email protected] Follow this and additional works at: https://scholarcommons.usf.edu/etd Part of the Computer Engineering Commons Scholar Commons Citation Blackburn, Jeremy, "An Analysis of (Bad) Behavior in Online Video Games" (2014). Graduate Theses and Dissertations. https://scholarcommons.usf.edu/etd/5412 This Dissertation is brought to you for free and open access by the Graduate School at Scholar Commons. It has been accepted for inclusion in Graduate Theses and Dissertations by an authorized administrator of Scholar Commons. For more information, please contact [email protected]. An Analysis of (Bad) Behavior in Online Video Games by Jeremy Blackburn A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Computer Science and Engineering College of Engineering University of South Florida Major Professor: Adriana Iamnitchi, Ph.D. John Skvoretz, Ph.D. Cristian Borcea, Ph.D. Jay Ligatti, Ph.D. Ken Christensen, Ph.D. Sanjukta Bhanja, Ph.D. Date of Approval: September 4, 2014 Keywords: social network analysis, toxic behavior, cheating, big data, contagion Copyright c 2014, Jeremy Blackburn Dedication To my friend Brandon Lorenz, with whom nearly 20 years ago I forged a friendship around online games giving me first hand experience with much of the behavior we study in this dissertation. To my friend Jon Greene, with whom I spent (too) many hours embedded in the Slaughter House game server we study in this dissertation. To my brother, Dr. Bryan Blackburn, for activating sibling rivalry as a force towards completing this dissertation. To my parents Bob and Janet, for not only believing in me, but unwittingly contribut- ing to this dissertation by letting me spend all those hours gaming with Brandon. To my daughter Ruby, for giving me a whole new perspective on the world. To my wife Lauren, for putting up with me and being my best friend. Acknowledgments First, I thank my lab mates during my time in the Distributed Systems Group at USF: Dr. Nicolas Kourtellis, Xiang Zuo, and Imrul Kayes. Nicolas, in particular, was instrumen- tal in my early work in the lab, as well as a major contributor on the analysis of cheaters in the Steam Community. I've also spent many hours discussing problems with both Xiang and Imrul, and appreciate the thought provoking conversations we've had. I thank two of my collaborators who coauthored some of the work in this dissertation with me: Prof. Matei Ripeanu who worked with me from the beginning on the analysis of cheaters, and Dr. Haewoon Kwak who coauthored the work on toxic behavior with me. I would like to thank my committee members, Professor John Skvoretz, Professor Cristian Borcea, Professor Jay Ligatti, Professor Ken Christensen, Professor Sanjukta Bhanja, and Professor Adriana Iamnitchi. In particular, Prof. Christensen, who oversaw my Masters dissertation and started me down this path by supervising undergraduate research work. Finally, I give my utmost gratitude to Anda, for advising and helping me grow as a scien- tist, for encouraging me to pursue my ideas, and pushing me to submit to the best venues possible. Without Anda's guidance, this dissertation would not have been possible. Table of Contents List of Tables iv List of Figures vi Abstract xi Chapter 1 Introduction1 1.1 Social Network Analysis2 1.2 Video Games: Living Labs for Social Research4 1.3 Bad Behavior in Online Games5 1.4 Dissertation Outline9 Chapter 2 Related Work 11 2.1 Collecting and Analyzing Online Social Network Data 11 2.2 Game Studies 12 2.3 Cheating and Unethical Behavior 17 2.3.1 Cheating in General 17 2.3.2 Cheating in Online Games 18 2.4 Toxic and Anti-social Behavior 24 Chapter 3 ARKS: An Architecture for the Collection and Analysis of Social Net- work Data 26 3.1 Scheduler 29 3.2 Data Collection 30 3.3 Data Storage 32 3.4 Analytics 35 3.4.1 Finding New Users to Crawl 35 3.4.2 Ad-hoc Programs 36 3.4.3 Elaine 37 3.4.3.1 The Coordinator 40 3.4.3.2 Workers 41 3.4.3.3 Postoffice 42 i 3.4.3.4 Vertex Programs 43 3.4.3.5 Performance Analysis 45 3.5 Summary 48 Chapter 4 Datasets 49 4.1 The Steam Community Online Social Network 49 4.1.1 Valve Anti Cheat System 52 4.2 Team Fortress 2 In-game Interactions 57 4.2.1 The Slaughterhouse Server 58 4.3 The League of Legends Tribunal 60 4.3.1 The Tribunal 62 4.4 Summary 65 Chapter 5 Social Network Analysis of Cheaters in Steam Community 67 5.1 Socio-gaming 69 5.2 Cheaters and their Friends 71 5.3 The Social Response to the Cheating Flag 78 5.4 Social Closeness of Cheaters 81 5.4.1 Geographical Proximity 81 5.4.2 Social Proximity 88 5.5 Summary 89 Chapter 6 Analysis of Interactions in Team Fortress 2 91 6.1 General Server Characterization 93 6.2 Declared Relationships 94 6.3 Interactions and Declarations 97 6.4 Summary 99 Chapter 7 Cheating in Online Games as a Contagion 101 7.1 Evidence for Contagion 102 7.1.1 Potential for Influence 102 7.1.2 How Does Cheating Propagate over Time? 104 7.1.3 Hazard Analysis 109 7.1.4 The Diffusion of Cheating Behavior 113 7.2 Data Driven Simulation of Contagious Behavior 119 7.2.1 Augmented Model for Contagion 121 7.2.2 Experiments 123 7.2.2.1 Static Networks 129 7.2.2.2 Dynamic Network 131 7.2.3 Experimental Results 134 7.2.3.1 Static Networks 135 7.2.3.2 Dynamic Network 138 ii Chapter 8 Predicting Crowdsourced Decisions on Toxic Behavior 141 8.1 Features 143 8.1.1 In-game Performance 145 8.1.2 User Reports 147 8.1.3 Chats 147 8.2 Models 151 8.3 Results 152 8.3.1 The Effects of Confidence in Crowdsourced Decisions 153 8.3.2 What Do Tribunal Reviewers Base Their Decisions On? 156 8.3.3 Classifier Portability 162 8.4 Estimating the Real-world Impacts 164 8.5 Limitations and Consequences of Our Approach 166 8.6 Summary 167 Chapter 9 Conclusion 168 9.1 Future Work 170 9.2 Discussion 172 List of References 175 Appendices 189 Appendix A: Statistical Tests 190 Appendix B: Copyright Permissions 192 iii List of Tables Table 4.1 Size of the Static Steam Community dataset. 51 Table 4.2 Details of ban observations dataset. 57 Table 4.3 Details of neighborhood observations dataset. 57 Table 4.4 Details of the SH interaction dataset. 60 Table 4.5 LoL Tribunal data set summary. 64 Table 5.1 Undirected triad census of Steam Community. 77 Table 5.2 Location network properties: the number of nodes, edges, mean dis- tance between users hDuvi, average link length hluvi, average node lo- cality hNLi. 85 Table 7.1 Percentage of cheaters found in top-N% of high degree centrality (DC) and betweenness centrality (BC) users in the Steam Community. 104 Table 7.2 Descriptive statistics for cheater friend distributions of the control group and new cheaters in the ban history dataset. 108 Table 7.3 Cox proportional hazard model analysis for both datasets we studied. 113 Table 7.4 Summary of model parameters. 121 Table 7.5 Parameters to reproduce common contagion processes. 121 Table 7.6 Network statistics for the largest connected components of the static networks. 125 iv Table 8.1 The top 5 ranked features from an information gain evaluator for Tri- bunal decisions. 157 v List of Figures Figure 3.1 A high level view of our architecture. 29 Figure 3.2 A high level overview of the crawler used to collect the static Steam Community data set. 31 Figure 3.3 An overview of the MongoDB cluster for our collection system. 33 Figure 3.4 JSON datastructure for storing neighborhood observations. 34 Figure 3.5 An overview of the Pregel computation model. 38 Figure 3.6 An overview of Elaine. 39 Figure 3.7 A binary tree. 44 Figure 3.8 A Pregel style vertex program for the single source shortest path prob- lem. 44 Figure 3.9 Total running time for the SSSP problem on a binary tree of 1 million vertices executed on varying amount of workers. 46 Figure 3.10 Running time per superstep for the SSSP problem on a binary tree of 1 million vertices executed on varying amount of workers. 47 Figure 3.11 Total running time for SSSP on binary trees of varying size executed on 10 workers. 48 Figure 4.1 Historical VAC ban dates. 54 Figure 4.2 The number of cheaters in our system, per day from June 21 to July 17, 2012. 56 vi Figure 4.3 The League of Legends map, featuring 3 symmetric lanes, separated by a jungle. 61 Figure 4.4 A tribunal case as presented to a reviewer. 64 Figure 4.5 The decision of reviewed cases, available after some time. 65 Figure 5.1 The number of games owned and lifetime hours per genre for cheaters and non-cheaters from the March 2011 crawl, and newly discovered cheaters from the October 2011 crawl. 70 Figure 5.2 Degree distribution for all users and cheaters in Steam Community. 73 Figure 5.3 Degree distribution for users with public, private, friends only, and no profile in Steam Community.