On the Scalability and Security of Distributed Multiplayer Online Games

On The Scalability and Security of Distributed Multiplayer Online Games

Seyed Amir Yahyavi Firouz Abadi

Doctor of Philosophy

School of Computer Science McGill University Montreal, Quebec, Canada

April 2014

A thesis submitted to McGill University in partial fulﬁllment of the requirements of the degree of Doctor of Philosophy

c Seyed Amir Yahyavi Firouz Abadi, 2014 Abstract

Multiplayer Online Games (MOGs) are an extremely popular online technology, one that produces billions of dollars in revenues. Yet, providing scalable, fast, and cheat-resistant games is challenging. In this thesis we propose solutions to some of these challenges.

The underlying architecture plays a crucial role in enabling the games to meet the scalability, response time, and low cost requirements that are of utmost importance in designing a successful massively multiplayer online game. Peer-to-peer architectures where the game runs exclusively on client machines, and hybrid approaches, where some tasks run on the clients while a central server controls other aspects, have, due to their distributed and collaborative nature, low infrastructure costs, and can achieve high scalability. They can also achieve fast response times by creating direct connections between players. However, they introduce their own challenges such as cheating. As one contribution of this thesis, we provide a comprehensive overview of current peer-to-peer and hybrid solutions for massively multiplayer games using a uniform terminology. Most of the solutions we studied fail to meet one of the following main requirements: (1) fast response time, (2) scalability, (3) cheat-resistance. This thesis makes several proposals that address these requirements: Watchmen, mobile security, AntReckoning, and AntAI.

Watchmen is the first distributed scalable protocol designed with cheat detection and prevention in mind that supports fast paced games. It is based on a randomized dynamic proxy scheme for both the dissemination and verification of actions. We show that Watch- men, while scaling to hundreds of players and meeting the tight latency requirements of first person shooter games, is able to significantly reduce opportunities to cheat, even in

i the presence of collusion. In the context of cheating we also look at the fast rising genre of mobile games. We discuss existing and potential new avenues of cheating that can occur in such mobile environment, and suggest several solutions to improve security based on existing and novel services that are or can be offered by carriers, ecosystems, and developers.

AntReckoning is a dead-reckoning algorithm, inspired from ant colonies, which models players’ interests to predict their movements. It helps our architecture by allowing us to send updates at lower frequencies and with more accurate predictions which in turn improves the scalability and the quality of experience of the players. AntReckoning incorporates a player’s interest in specific locations, objects, and avatars in the equations of motion in the form of attraction forces. In practice, these points of interest generate pheromones, which spread and fade in the game world, and are a source of attraction. AntReckoning greatly improves the accuracy of traditional dead reckoning techniques and can decrease the upload bandwidth by up to a third of overall traffic. Interest modeling can also be used to improve the performance of non-player characters in games. In our AntAI project, we detail how movements, interactions, use of items, and relying on static decision-making schemes result in markedly different behaviors from humans in a popular fist-person shooter game. From there, we propose a framework relying on our pheromone maps, which can lead to a more adaptive human-like behavior.

Altogether, this thesis presents components that we believe are key elements for a scalable, cheat-resistant, and fast multiplayer online game architecture.

ii Abrégé

Les jeux multijoueurs en ligne (MOG) constituent à l’heure actuelle une technologie en ligne très populaire qui génère des milliards de dollars en revenus. En dépit de cela, offrir des jeux qui sont à la fois extensibles, rapides et résistants à la tricherie est toujours un déﬁ. Dans cette thèse, nous proposons des solutions à certains de ces déﬁs.

L’architecture sous-jacente joue un rôle crucial lorsque l’on souhaite développer des jeux qui rencontrent des contraintes d’extensibilité, de faible temps de réponse et de faible coût. Ces contraintes sont d’une importance capitale si l’on souhaite que le jeu massivement multijoueurs en ligne ainsi développé devienne un succès. Les architectures pair-à-pair, où le jeu s’exécute uniquement sur les machines des clients (joueurs), ainsi que les approches hybrides, où certaines tâches du jeu s’exécutent sur les machines des clients tandis qu’un serveur central gère d’autres aspects du jeu, permettent, de par leur nature distribuée et collaborative, d’atteindre une haute extensibilité tout en minimisant les coûts d’infrastructure. Ces architectures permettent également d’atteindre des temps de réponse élevés en étab- lissant des connexions directes entre les joueurs. Cependant, elles introduisent également certains déﬁs à résoudre tels que la possibilité de tricher. En tant que contribution à cette thèse, nous exposons une vue d’ensemble exhaustive des solutions pair-à-pair et hybrides courantes pour les jeux massivement multijoueurs en ligne utilisant une technologie uni- forme.

La plupart des solutions que nous avons étudié ne respectent pas l’une des exigences principales suivantes: (1) temps de réponse rapide, (2) extensibilité, (3) résistance à la tricherie. Cette thèse décrit les différents projets sur lesquels nous avons travaillé qui visent

iii à mettre en oeuvre ces exigences: Watchmen, sécurité mobile (mobile security), AntReck- oning et AntAI.

Watchmen est le premier protocole extensible et distribué conçu spécifiquement pour la détection et la prévention de la tricherie qui prend en charge les jeux à rythme rapide (fast-paced games). Ce protocole est basé sur un schéma d’assignation dynamique aléa- toire de noeuds-relais (“proxie”) pour effectuer tant la dissémination que la vérification des actions. Nous démontrons que Watchmen est en mesure de réduire de manière significative les possibilités de tricherie, même en présence de collusion et ce, tout en prenant en charge des centaines de joueurs et en rencontrant les exigences strictes de latence imposées par les jeux de tirs à la première personne. Dans le contexte de la tricherie, nous nous attardons également aux jeux mobiles. Nous discutons des possibilités existantes et des nouvelles avenues potentielles de tricherie qui peuvent survenir dans des environnements mobiles et nous suggérons différentes solutions pour améliorer la sécurité. Ces solutions sont basées sur des services existants ou nouveaux qui sont ou qui pourrait être offerts par les opéra- teurs, écosystèmes et développeurs.

AntReckonning est un algorithme de navigation à l’estime (dead-reckoning) inspiré des colonies de fourmis qui modèle les intérêts des joueurs pour prédire leurs mouvements. Ce modèle est bénéfique pour notre architecture puisqu’il nous permet de transmettre les mises à jour à des fréquences inférieures et avec des prédictions plus précises, ce qui permet d’améliorer l’extensibilité et la qualité de l’expérience des joueurs. AntReckoning in- corpore l’intérêt des joueurs pour des emplacements, objets et avatars spécifiques au sein d’équations de mouvement en tant que forces d’attraction. En pratique, ces points d’intérêt génèrent des phéromones qui se diffusent et s’estompent dans le monde virtuel du jeu et qui sont des sources d’attraction. AntReckoning améliore de manière significative la précision des techniques de navigation à l’estime traditionnelles et peut réduire la bande passante de téléversement (upload) jusqu’à concurrence de un tiers du trafic total.

iv Contributions

The complete list of our contributions can be found in the publications section. Overall, seven papers were published out of which two of the papers complement our earlier publications. I am the ﬁrst author on all of the following works.

• [YK13] A. Yahyavi and B. Kemme. Peer-to-peer architectures for massively multiplayer online games: A survey. ACM Computing Surveys (CSUR), 46(1):9:1–9:51, ACM, 2013. In this paper, we have the most comprehensive look to date at the requirements and different design aspects of the massively multiplayer online games (MMOGs). We present extensive background on the design of these environment and assemble a comprehensive taxonomy of important issues which includes object replication, information dissemination, networking structure, interest management, fault tolerance, consistency control, persistence, security and cheating, industry solutions, incentive mechanisms, and more. We provide an in-depth analysis of each one of these aspects and how they affect performance, quality of experience, cost of design, scalability, and other important issues in the design of such games. This work studies and covers around 200 papers and research projects in this area. This work was done by me, with the guidance of my advisor.

• [YHGS`13] A. Yahyavi, K. Huguenin, J. Gascon–Samson, J. Kienzle, and B. Kemme. Watchmen: Scalable cheat-resistant support for distributed multiplayer online games. In Proceedings of International Conference on Distributed Computing Systems (ICDCS).

v pages 134–144, ACM, 2013. [Our earlier work was published in ACM SIGCOMM NetGames 2011 [HYK11]] In this paper we propose the first scalable and cheat-resistant distributed middleware for first person shooter (FPS) games. Other architectures offered before either completely ignore the problem of cheating, are not scalable enough, or are not able to meet the tight latency requirements or high bandwidth requirements of these games. Our architecture can offer reasonable security by using prevention and cross verification techniques while being able to support hundreds of players when standard broadband bandwidth is available to players. It achieves this even in the presence of hundreds of players in the same area of interest. We have a comprehensive look at the security implications of the design, and provide a detailed evaluation using a popular FPS game instead of simulations. Kévin and I worked on the initial ideas and implementation which was published as a poster paper in NetGames. Later, I completed the work, and Julien helped with debugging and improving the code, and running experiments. The work was done under the guidance of our advisors.

• [YHK12] A. Yahyavi, K. Huguenin, and B. Kemme. Interest modeling in games: The case of dead reckoning. Multimedia Systems (MMSJ), pages 1–16, Springer, 2012. [Our earlier work was published in ACM SIGCOMM NetGames 2011 [YHK11]] While many different generic machine learning and prediction models exist, our approach is the first attempt to model players’ interests inside the game and to use that information in predicting their movement and actions. The significance of this work is that with more and more players joining large scale games, the game engines have to rely on sending updates at different rates to different players and then rely on compensation techniques to improve the quality of experience of players that receive updates at lower rates or experience lag, jitter, or loss. We explore a new simplified and low overhead way to model interesting items in the games and their influence on players’ decisions by using pheromone maps. This practical and low overhead approach is able to improve prediction quality by up to 44% and save network bandwidth usage by up to 32% compared to standard dead-reckoning techniques. This work was done by me with guidance from my advisor. Kévin contributed to writing the paper.

vi • [YPK13] A. Yahyavi, J. Pang, and B. Kemme. Towards providing security for mobile games. In ACM MobiCom International Workshop on Mobility in the Evolving Internet Architecture, MobiArch ’13, pages 47–52. ACM, 2013. In this paper, we compare the threats in mobile game environments with existing console and pc gaming environments. We discuss how these threats are different and what new kinds of threats can emerge. We also discuss new security mechanisms that can help prevent or detect these new threats. Jeff contributed by providing additional information on mobile platforms. The project was done with the guidance of my advisor.

• [YTVK13] A. Yahyavi, J. Tremblay, C. Verbrugge, and B. Kemme. Towards the design of a human-like FPS NPC using pheromone maps. In IEEE International Games Innovation Conference (IGIC), pages 275–282, IEEE, 2013. We develop new metrics to quantitatively measure the differences in human vs NPC players’ behavior in the game. We provide these measurements for the game Quake III as a sample architecture. Furthermore, we propose a platform based on our pheromone maps that can help improve the behavior of the NPCs. Jonathan contributed to writing the paper and related work along with the guidance of our advisors.

vii Acknowledgements

Foremost, I would like to thank my advisor Bettina Kemme for many years of advice and guidance. Her support and contributions have been invaluable to my progress. I thank her for all the effort, enthusiasm, and personal time she spent on my work and for giving me the opportunity to work freely and be creative in my work.

I am grateful as well to Jörg Kienzle and Clark Verbrugge for their insightful comments, advices and for being part of my Ph.D. progress committee. Also, I would like to thank my friends and colleagues Samf, Kasra, Neda, Mitra, Kévin, Julien, and my labmates who it has been an honor knowing them and working with them.

My parents, Shahrzad and Aziz, have been a constant source of moral support and affection throughout my life as well as my sister Noushin and this thesis would certainly not have existed without them.

Last but not least, I want to express my deep appreciation and gratitude to my wife Laleh for her continuous support, patience and encouragement during all phases of this thesis.

viii Contents

1 Introduction 1

I Distributed Architectures for Multiplayer Online Games: A Sur- vey 5

2 Multiplayer Online Games: Background, Design Issues, and Proposed Solu- tions 6 2.1 Motivation ...... 6 2.2 Game Design Principles ...... 9 2.2.1 Object Types ...... 9 2.2.2 Player Interactions ...... 9 2.2.3 Object Replication ...... 10 2.2.4 Game Types and Latency Tolerance ...... 11 2.2.5 Bucket Synchronization & Frame-rate ...... 11 2.2.6 Bandwidth Requirements ...... 12 2.2.7 Interest Management ...... 13 2.2.8 Consistency Control ...... 16 2.3 MOG Architectures ...... 19 2.3.1 Client-Server Architecture ...... 19 2.3.2 Distributed Multi-Server Architecture ...... 20 2.3.3 Peer-to-Peer (P2P) Architecture ...... 21 2.3.4 Characteristics and Comparison ...... 21 2.3.5 Hybrid Architectures ...... 24 2.3.6 Heterogeneous and homogeneous P2P systems ...... 25 2.3.7 Other Functionalities ...... 26

ix 2.4 P2P Architectures ...... 26 2.4.1 Introduction ...... 26 2.4.2 Structured P2P Game Architectures ...... 29 2.4.3 Unstructured P2P Game Architectures ...... 32 2.4.4 Hybrid P2P Architectures ...... 34 2.5 Communications & Multicast ...... 35 2.5.1 Direct Communication ...... 36 2.5.2 Multicast Trees ...... 38 2.5.3 NAT & Firewalls ...... 40 2.6 Interest Management ...... 41 2.6.1 Structured Architectures ...... 41 2.6.2 Unstructured Architectures ...... 42 2.6.3 Challenges ...... 43 2.7 Replication & Consistency Control in P2P Architectures ...... 46 2.7.1 Replication Management ...... 46 2.7.2 Consistency Control ...... 48 2.8 Fault Tolerance & Persistence ...... 50 2.8.1 Fault Tolerance ...... 50 2.8.2 Persistence ...... 52 2.9 Cheating ...... 54 2.9.1 Deﬁnition ...... 55 2.9.2 Cheating Categories ...... 55 2.9.3 Cheating Prevention ...... 59 2.9.4 Cheating Detection ...... 61 2.9.5 Reputation Systems & Penalization ...... 64 2.10 Commercial Adoption ...... 66 2.10.1 Middlewares ...... 66 2.10.2 Industry Solutions ...... 66 2.10.3 Industry Models ...... 68 2.10.4 Client Incentives ...... 69 2.10.5 Other applications ...... 70 2.11 Survey Conclusions ...... 71

x II Cheat-Resistant Support For Multiplayer Online Games 73

3 Watchmen: A Scalable Cheat-Resistant Peer-to-Peer Architecture For Fast- Paced Multiplayer Online Games 74 3.1 Motivation for Watchmen ...... 75 3.2 Watchmen Background ...... 77 3.2.1 Interest Filtering in Quake III ...... 78 3.3 The Watchmen Architecture ...... 79 3.3.1 Subscription Model ...... 81 3.3.2 Proxy architecture ...... 83 3.4 Security Aspects of the Proxy Architecture ...... 84 3.5 Veriﬁcations, reputation and punishment ...... 86 3.5.1 Veriﬁcation ...... 86 3.5.2 Reputation & Punishment ...... 88 3.6 Watchmen Performance Characteristics ...... 89 3.7 Watchmen Evaluation ...... 91 3.8 Watchmen Related Work & Compariosn ...... 98 3.9 Watchmen Conclusions ...... 99

4 Towards Providing Security For Mobile Games 101 4.1 Introduction to Security in Mobile Games ...... 101 4.2 Cheating in the Mobile Environment ...... 102 4.2.1 Existing Cheating Mechanisms ...... 103 4.2.2 Existing Security Mechanisms ...... 105 4.2.3 New Gaming & Cheating Mechanisms ...... 106 4.3 Trust Model & Design Principles in Mobile Games ...... 107 4.3.1 Trust Model ...... 107 4.3.2 Design Principles ...... 108 4.4 Security Architecture for Mobile Games ...... 109 4.4.1 New Security Mechanisms ...... 109 4.4.2 Location cheating ...... 110 4.4.3 Fake Sensor Readings ...... 111

xi 4.4.4 Disruption of Information Flow ...... 113 4.5 Mobile Game Security Conclusions ...... 114

III Interest Modeling Using Pheromone Maps 116

5 Interest Modeling in Games: The Case of Dead Reckoning 117 5.1 Introduction to AntReckoning ...... 118 5.2 Background on Dead-Reckoning ...... 119 5.3 Motivation and Design Rationale for AntReckoning ...... 122 5.4 The AntReckoning Algorithm ...... 126 5.5 Parametrization and Implementation ...... 132 5.5.1 Parametrization ...... 132 5.5.2 Discussion & Implementation ...... 135 5.6 AntReckoning Evaluation ...... 138 5.6.1 Experimental setup ...... 138 5.6.2 Sensitivity analysis ...... 140 5.6.3 Performance evaluation ...... 145 5.7 Discussion ...... 147 5.8 Related Work to AntReckoning ...... 148 5.9 AntReckoning Conclusions ...... 149

6 Towards the Design of a Human-Like FPS NPC using Pheromone Maps 151 6.1 Introduction to AntAI ...... 151 6.2 Existing NPC Technology and Game Adaptivity ...... 153 6.2.1 Decision Making: ...... 154 6.2.2 Game Adaptivity: ...... 155 6.3 NPC Performance Metrics ...... 156 6.4 Quake III Analysis ...... 160 6.4.1 Bot Detection ...... 163 6.5 AntAI Algorithm ...... 164 6.5.1 Decision Making ...... 165

xii 6.6 AntAI Conclusions ...... 167

IV Final Conclusion & Future Work 168

7 Final Conclusion & Future Work 169 7.0.1 Final Conclusion ...... 169 7.0.2 Future work ...... 170

List of Publications 174

Bibliography 175

Acronyms 201

xiii List of Figures

2.1 Multiplayer game components ...... 10 2.2 Different game zoning mechanisms ...... 14 2.3 Different gaming architectures ...... 21 2.4 Routing in Pastry and Mercury ...... 30 2.5 pSense and N-Tree architectures ...... 32 2.6 VON Voronoi, Solipsis(’03) convex hull, and Solipsis(’08) Raynet overlays 43

3.1 Heatmap of player positions in a Quake III ...... 79 3.2 Watchmensubscription-types and corresponding areas ...... 80 3.3 Watchmenproxy architecture ...... 84 3.4 Information exposure ...... 92 3.5 Witness information level ...... 94 3.6 Cheat detection success ...... 95 3.7 Update latency ...... 95 3.8 Number of IS and VS subscriptions ...... 96 3.9 Upload bandwidth projection ...... 97

5.1 Screenshot of a Quake III game environment ...... 121 5.2 The q3dm01 map from Quake III ...... 123 5.3 Presence of players in the q3dm01 map from Quake III ...... 124 5.4 Effect of player interactions on their behavior ...... 125 5.5 Overview of AntReckoning ...... 128 5.6 Illustrative example of the evolution of the concentration of pheromone . . 130 5.7 Prediction correction ...... 131 5.8 Illustration of the metric used in the evaluation of AntReckoning ...... 139

xiv 5.9 Experimental sensitivity analysis of AntReckoning ...... 142 5.10 Effect of visibility and post-processing on AntReckoning ...... 144 5.11 Performance evaluation of AntReckoning ...... 146

6.1 Experimental sensitivity analysis of AntAI ...... 157 6.2 Player movements during ﬁreﬁght ...... 163 6.3 Overview of AntAI ...... 164

xv List of Tables

2.1 Comparison of different architectures ...... 24

2.2 Comparison of representatives of different P2P architectures ...... 28

3.1 Popular cheating mechanisms in distributed multiplayer games ...... 81

4.1 Popular cheating mechanisms in distributed games ...... 104

4.2 Suggested API for securing game services ...... 106

5.1 Important parameters in AntReckoning...... 134

6.1 Differences in weapon usage between Players and NPCs ...... 160

xvi 1 Introduction

Video games have been the fastest-growing form of media over the past few years, with sales rising to $111 billion by 2015 [Gar13]. Video games have traditionally been mostly a social experience. Multiplayer video games are played competitively or cooperatively by using either multiple input devices, or through the network. Tennis for Two, arguably the ﬁrst video game, was a two player game.

Early networked games were generally limited text based adventures or MUDs (Multi- User Dungeon) that were played remotely on a dedicated server. This was the result of both the slow speed connection (300–1200 bit/s), and the high cost. However, due to improvements in speed and latency of connections, the number of players in modern fast-paced multiplayer online games (MOGs) can be 32 or higher, while featuring enhanced graphics with integrated text and/or voice chat. MMOGs (massively multiplayer online games) can support an even higher number of simultaneous players; in 2013, Eve Online hit a record for the maximum number of simultaneous pilots online with 65,303 concurrent accounts logged on to the same server [EVE13].

However, most multiplayer games, in order to support more players, use a distributed architecture. These systems rely on multiple servers, peer-to-peer, or hybrid architectures to provide scalability. But this requires more complicated systems where some responsibilities are distributed among many nodes. As a result it brings up a range of new issues that will need to be addressed such as: object management, concurrency and consistency control, multicast, security and cheating, fault tolerance, availability, and persistence. Distributed

1 Introduction systems require a great deal of coordination and are less secure. Success of massively multiplayer online games, in particular, relies on scalability and fairness of the architecture. Scalability is necessary for the viability of the game while fairness is required for wide adoption by players.

Distributed architectures provide scalability by distributing processing or networking overheads of the game among servers and/or players. This is typically done by offloading the game objects to other servers or players or by relying on them to disseminate network traffic produced by the game. Some tasks such as verification of player actions can be delegated as well. However, this makes the system vulnerable to cheating.

Cheating essentially consists in gaining an unfair advantage and comes mostly in the following forms: disrupting the game state computation and dissemination, performing illegal actions, and gaining access to sensitive information [YR05, WS07]. In centralized games, cheat detection and prevention are achieved by making the server verify the players’ actions, ensure synchronization and reduce the information sent to players to the minimal amount required to render the game world [BLL07]. In decentralized games, however, detection of cheating is more difﬁcult given the natural trade-offs between responsiveness, scalability, veriﬁcation and information disclosure together with the issues of trust and collusion.

The goal of this thesis is to design a distributed, scalable, fast, and cheat resistant architecture for multiplayer online games. To do that, we ﬁrst provide a detailed survey of the existing distributed architectures for massively multiplayer online games (with a focus on peer-to-peer systems) in Chapter 2. A good study of this subject has been lacking, therefore, we decided to do a comprehensive literature review. We detail many of the design issues that these systems face and provide an overview of different solutions offered so far. We categorize these solutions based on their design, and detail and compare their various aspects such as security, replication, consistency control, etc. While Chapter 2 discusses the background necessary for the thesis at the level of a general, broad survey, we further introduce additional background in the other chapters whenever more detailed descriptions are necessary to understand the particular topic of these chapters.

The remainder of the thesis presents new contributions to the ﬁeld of scalable and cheat-

2 Introduction resistant distributed game architectures, and can be roughly split into two main themes. The ﬁrst theme addresses cheating in distributed architectures, and our contributions are presented in Chapters 3 and 4. The second theme is to exploit ant-colony techniques to improve game performance and game play overall, in particular, to lower bandwidth use, increase scalability, and improve the quality of experience of players as well as reduce the amount of inconsistency they perceive. This theme covers Chapters 5 and 6.

More precisely, Chapter 3 proposes a scalable, cheat-resistant peer-to-peer architecture for multiplayer games. Common decentralized approaches to deal with cheating include mutual veriﬁcation and auditing, agreement protocols (e.g., lockstep [BLL07]), and position-based information ﬁltering. However, most of the proposed approaches have one of the following drawbacks: they (1) rely on a central server or trusted third parties [WSL07]; (2) do not deal with collusion; (3) detect cheaters a posteriori [CFFS05]; (4) fail to provide responsiveness and scalability [BLL07].

In order to develop an approach that avoids these pitfalls we analyze the cheating opportunities created by mechanisms commonly used in several types of peer-to-peer multiplayer games, with a special focus on a distributed version of Quake III. From there, we propose Watchmen which uses the following techniques to prevent and detect cheating:

‚ Vision-based filtering and indirect communication are used to reduce the information available at each player close to the minimum amount necessary to render the game world; ‚ Players are assigned a proxy player in charge of update dissemination and action verification. Proxies are chosen at random and dynamically renewed to limit the impact of collusion while allowing on-the-fly mid-term verifications. ‚ Verifications can lead to the emission of blames which can be used to directly take actions against cheaters.

In Chapter 4, we continue to look at cheating but move our attention to mobile games. We ﬁrst discuss how these games are different to standard PC and console multiplayer games. Then, we study how cheating is different in the emerging mobile games makes, and present several new kinds of cheats that can be exploited due to the mobile environment. Based on this analysis, we propose new mechanisms to address these new types of cheating.

The work in Chapter 5 is motivated by the need to control the amount of messages

3 Introduction sent by distributed games and the need to appropriately handle the unavoidable delay in message transmissions. In interactive games, position update messages account for the bulk of the network trafﬁc [KLXH04], As a result, techniques that predict player movements to reduce the update rate while keeping the error on player positions low are utilized. These techniques also help in coping with message loss or delay by extrapolating the new position when the new update is not received in time. Lowering the number of updates sent by the game has a critical impact on the scalability of the system.

We argue that key factors in an avatar’s motion are not only inertia but also the objectives of the game as well as entities in his vicinity that we refer to as points of interest. Following this line of reasoning, we propose AntReckoning. To the best of our knowledge it is the ﬁrst interest-based approach to dead reckoning. The main concepts involved in AntReckoning are as follows:

‚ Each entity is assigned a given attractiveness leading to the generation of pheromones that fade and spread in the world; ‚ Pheromones in the vicinity of an avatar exert attraction on it. Attraction is integrated in the equations of motion, under the form of forces, to estimate its future position.

The main contributions of AntReckoning are: (1) to incorporate players’ interests into the equations of motion used for dead reckoning, and (2) to use pheromones to model such interests, taking temporal and spatial aspects into account. Moreover, pheromones offer a practical solution to the decentralized implementation of interest-based dead reckoning.

Finally, Chapter 6 analyzes how current non-player characters in the games can greatly differ from the human players. Using interest-modeling as introduced in Chapter 5, we propose our AntAI solution, and discuss how the quality of these non-player characters can be improved.

Overall, the contributions that we present in Chapters 3-6 offer scalability, cheat-resistance and overall better performance to multiplayer games.

4 Part I

Distributed Architectures for Multiplayer Online Games: A Survey

5 2 Multiplayer Online Games: Background, Design Issues, and Proposed Solutions

Scalability, fast response time, and low cost are of utmost importance in designing a successful multiplayer online game. The underlying architecture plays an important role in meeting these conditions. Peer-to-peer and hybrid architectures, due to their distributed and collaborative nature, have low infrastructure costs and can achieve high scalability. They can also achieve fast response times by creating direct connections between players. However, these architectures face many challenges. Distributing a game among peers makes maintaining control over the game more complex. Peer-to-peer architectures also tend to be vulnerable to churn and cheating. Moreover, different genres of games have different requirements that should be met by the underlying architecture, rendering the task of designing a general purpose architecture harder. Many peer-to-peer gaming solutions have been proposed that utilize a range of techniques while using somewhat different and con- fusing terminologies. This chapter of thesis presents a comprehensive overview of current peer-to-peer solutions for multiplayer games using a uniform terminology.

2.1 Motivation

Multiplayer Online Games (MOGs) and Massively Multiplayer Online Games (MMOGs) are among popular online technologies that produce billions of dollars in revenues and introduce several new and interesting challenges. One of the main attractions of these games

6 2.1 Motivation lies in the number of players that participate in the game. The more players that are in the game world, the more interactive, complex, and attractive the game environment will become. Successful games such as World of Warcraft with nearly 12 million subscriptions [Bli11] have to provide a truly scalable game world while maintaining responsiveness.

To better understand MOGs, we first need to give a definition. A video game is usually defined as an electronic game that is played by a controller and provides user interactions by generating visual feedbacks. A multiplayer game is a game played by several players. Players can be simply independent opponents or they can play in teams. They can play against each other or can play against the game, i.e., opponents that are controlled using Artificial Intelligence (AI). A MMOG is a game capable of supporting hundreds or thousands of players and is mostly played using the Internet. Many games such as World of Warcraft [Bli11], EVE Online [EVE11], and Final Fantasy XI [FFX11] have shown that MMOGs are a thriving business industry. For example, Star Wars: The Old Republic was able to achieve one million subscribers in three days after launch1. Second Life [Sec11], launched in 2003 by Linden Lab, is the most famous social virtual world with more than 16 million registered users. The emergence of social games (such as Farmville and Mafia Wars 2) with millions of subscribers [Zyn11] as well as mobile games that are played on smartphones, and the popularity of handheld devices such as Sony PSP [PSP11] and Nintendo DS [Nin11], lay the foundation for potential integration of social and mobile environments into massively multiplayer games [ILc10, VV10].

MOGs can produce huge network trafﬁc and processing loads [SM12, CHHL05]. Thus, the main challenges in MOGs are scalability, i.e. providing support for thousands of players simultaneously, consistency, security, and fast response time – usually all at the same time, otherwise customer satisfaction would be reduced. In the next sections we discuss these challenges and different solutions that have been proposed.

Client-server systems, where game execution and game state dissemination is completely controlled by the server, are currently the prevalent game architecture. However, peer-to-peer architectures can be beneﬁcial for gaming infrastructures in several ways. If

1http://www.swtor.com/news/press-release/20111223 2Zynga: 232 Million Monthly Players http://secfilings.com/searchresultswide. aspx?link=1&filingid=8022980

7 2.1 Motivation client nodes communicate directly with each other or perform part of the game state computation, server requirements in terms of computational power and network bandwidth can be signiﬁcantly reduced. Even if the game execution remains completely controlled by servers, peer-to-peer technology can be used to coordinate multiple servers, such as maintaining distributed game state execution and management of server farms and federated servers [CWD`05, IHK04, ASO09]. Cloud-based game streaming services based on content distribution networks [OnL11, Gai11] can also beneﬁt from these architectures [CHJC10]. They can provide 3D streaming services where similar to audio or video media streaming, 3D content is fragmented into pieces at a server, before it is transmitted, reconstructed, and displayed at the clients [WHT09].

Peer-to-peer architectures have received a great deal of research attention in the recent past as they distribute computational and network load among peers, can potentially achieve high scalability, low cost, and good performance as will be discussed further. While most of our discussions are focused on multiplayer games, in particular MOGs, many of these architectures can also be applied to other distributed systems such as distributed simulation environments [Fuj00], virtual worlds such as Second Life [Sec11], and other networked virtual environments [BHML92, KAM04, KS03].

The remainder of this chapter is structured as follows. In Section 2.2 we study common game design principles used in most multiplayer games, and in particular MOGs. In Section 2.3 we study and compare different architectures proposed for MOGs. Next, we present in detail issues related to structure (Section 2.4), update dissemination (Section 2.5), interest management (Section 2.6), replication and consistency (Section 2.7), fault tolerance, availability, and persistence (Section 2.8) in peer-to-peer based MOG solutions. Section 2.9 discusses cheating. We explain how games, in particular P2P architectures, are affected by cheating and discuss some of the security measures proposed for P2P-based games. In Section 2.10 we study different incentives for using P2P architectures by consumers and the industry. Furthermore, we discuss various applications and adoption models of P2P-based gaming architectures. Section 2.11 concludes the chapter.

8 2.2 Game Design Principles

2.2 Game Design Principles

Before getting into peer-to-peer architectures we ﬁrst explain general concepts involved in designing a multiplayer game whether using client-server architectures or P2P. An overview of different components of a sample multiplayer game framework is shown in Figure 2.1(a). Here, we discuss the general concepts and execution patterns used in most multiplayer online games.

2.2.1 Object Types

In modern video games, the game world is usually made up of four types of components [KLXH04]: (1) immutable objects, such as landscape or terrain information, are usually designed and created ofﬂine and never change during the game. These objects are typically installed at the client and are initialized at the start of the game. (2) characters or avatars in the game world that are controlled by the player using an input device. The avatar has a position in the game world and is usually allowed three types of actions: player updates, player-object interaction, and player-player interaction. (3) mutable objects such as food, weapons, and tools that can be modiﬁed. For instance, players can interact with and/or use them in their interaction with other players. (4) Non-Player Characters (NPCs), also called bots, are characters or avatars that are not controlled by a player but are usually controlled using AI. In the rest of this chapter, unless otherwise stated, game objects refer to avatars, mutable objects and NPCs in the game. Object types are shown in Figure 2.1(b).

2.2.2 Player Interactions

Player interactions are typically divided into three categories: Player updates, player-object interactions, and player-player interactions [KLXH04, BPS06]. Player updates are interactions with the game world that only affect the player himself. Position updates and graphical updates to the player’s avatar are examples of player updates. In a simple and unopti- mized implementation, a large proportion of all player interactions can be position updates [KLXH04]. Player-object interactions are the interactions between a player and mutable objects in the game world. For example, picking up a health pack (adding it to the inven- tory) or consuming it are examples of player-object interaction. Player-player interactions

9 2.2 Game Design Principles

Game Logic Manager Replication Game View Manager Client Manager Engine Controller Interaction Replication Player 1 Collision Pathfinding NPC Graphical Client NPC 1 Manager Detection Manager Manager Dead Visibility Consistency Reckoning Manager World Engine & Local IM Controller Immutable Object Voice Partitioning Item Interest Manager Manager Item 2 Strategy Manager Player 2 P1 Monitoring Network Persistence NPC 2 Interaction & Score Manager Engine Manager & Logging Item 1

(a) Multiplayer game components (b) Object types and interactions Figure 2.1: (a) Different components of a multiplayer game, adapted from Mammoth [KVB`09] (adapted, c ACM 2009), a massively multiplayer game research framework, are presented. The components discussed in this section are highlighted. (b) Different game objects and their interactions. Players can interact with each other, objects and NPCs. are interactions between a player and other players in the game world. For example, attacking another player could decrease the other player’s health and increase the experience points for the attacker. Player interactions with NPCs, based on the game design, can be considered either player-object or player-player interactions.

The type of interaction is important when dealing with consistency issues that arise from concurrent and conﬂicting updates to the same object. Also, most security and cheat prevention techniques only apply to certain types of interactions.

2.2.3 Object Replication

When a player joins a game, he receives an instance of the game world (it can be a limited view of the game world) that is made up of various types of game objects. Most game engines follow a primary-copy replication approach. For each object and character there exists an authoritative copy, called primary or master copy. All other copies are secondary copies (also called replicas). Each player has, stored on his computer, copies of game objects which are of interest to the player. Any update to the object has to be ﬁrst performed on the primary copy. How primary and secondary copies are distributed (e.g., primary copies might always reside on the server or might also be held by clients) depends on the game architecture. If a player wants to perform an update on an object for which he does not have the primary-copy, he has to send the update to the primary copy. The holder of the primary

10 2.2 Game Design Principles copy decides whether to accept the update or not, and then sends the updated object to everyone that has a secondary copy, where the changes are applied.

The update dissemination mechanism is quite similar to publish-subscribe systems that have been widely studied [CDKR02]. Every replica becomes a subscriber to the primary copy of the object and receives publications (updates) from the primary copy (the publisher).

2.2.4 Game Types and Latency Tolerance

We refer to the latency as the delay between execution of an update at the primary copy of an object and the replica receiving the object update. This latency is dependent on the architectural design and networking delays as will be discussed.

Various types of multiplayer and massively multiplayer games exist. In Real Time Strat- egy (RTS) and Role Playing Games (RPGs) [SGB`03] the focus is more on game strategy rather than responsiveness. The player tells the avatar to do a possibly complex and long lasting action, e.g., go to a destination, and the avatar performs the requested action. In First Person Shooter (FPS) games, however, the player does short-lived and less complex actions, i.e., the player actually does what he wants the character to do (for example, guides the avatar towards the destination) and as a result, higher responsiveness is required (see [Arm03]) for Quake III latency requirements). Higher latencies than the tolerance threshold of the game have an adverse effect on playability of the game and user satisfaction. Based on the game type and design, games typically can tolerate latencies between 100 to 300 milliseconds (ranging from FPS games to RPGs) [Arm03, BPS06, PW02b], however, games with higher latency tolerance exist (see [SM12]). Latency tolerance requirements have a dramatic effect on the architecture design for games. An architecture would only be feasible if it meets game latency requirements.

2.2.5 Bucket Synchronization & Frame-rate

Bucket synchronization [LGD98] (also called local lag [MVHE04]) is a method used to deal with latency and is used in most multiplayer games. Since network latency for each client is different and it is common for a primary copy to receive various update requests

11 2.2 Game Design Principles concurrently from different clients, most games deliberately lag behind in executing the events. This allows fairness despite latency variations and more control over update dissemination costs. This is in essence similar to Nagle’s algorithm in IP networking. Games implement a discrete event loop (may also be referred to as frame) in which all actions (events) that have been submitted since the last execution of the loop, are received, buffered, and then executed. The updates are then sent at the end of the loop. In addition, most game objects, including NPCs, have a think function which is executed in every game loop and determines the actions of the object in this loop. The game loop is usually executed 10 to 20 times every second ; this frequency is sometimes referred to as frame-rate [BDL`08] (not to be confused with graphical frame-rate). Low frame-rates can degrade the game play experience or even render the game unplayable. Note that based on the game design the number of frames shown to the player, i.e., the graphical frame rate, may be equal to or different from this frame-rate.

The lag should be chosen so that it allows enough time for the updates of different clients to be received. This helps ensure fairness for clients that might have worse connectivity, and control the number of updates that have to be sent per second. At the same time, the lag has to be small enough not to be perceived by the players. In essence, a trade-off has to be found.

In this thesis, we focus on the impact of architectures on networking, processing delays and the resulting frame rate. We do not consider other delays such as those caused by graphical processing.

2.2.6 Bandwidth Requirements

The bandwidth requirement of MOGs can be calculated based on average message size, update rate, and number of recipients (active players). Games with millions of (active) subscribers (e.g., WoW), have high bandwidth requirements due to their dynamic environment (e.g., Second Life ), or high update rates (e.g., Quake) [CHHL05, SM12]. In addition, in order to have an acceptable game play, games should also accommodate occasional bursts in the game trafﬁc that can be many times the average requirements. Such bursts happen due to sudden environment changes or battles. Moreover, even inside the same game different

12 2.2 Game Design Principles gaming activities, e.g, raiding vs. trading in WoW, generate different trafﬁcs [SSM11] that can lead to different network requirements. Game servers typically deal with this by over provisioning.

2.2.7 Interest Management

Interest management (IM) [BF93, Mor96] is an important mechanism used in many games, typically for scalability reasons. The idea is that players of a multiplayer game have only limited movement and vision capabilities. That is, players can only move a small distance in the game world in a given time interval. The players also have limited sensing capabilities, meaning that players can only interact with objects and other players in their vicinity [MGBY99]. As a result, data access in games shows spatial and temporal locality. Utilizing this fact, interest management limits the amount of game state any player can access. That is, the player only receives the game state relevant to it, based on his position and vision in the game world. Interest management is important for scalability, as clients only need replicas of objects that are interesting for them, therefore, keeping update and network overhead low. However, it can also be important as part of game semantics or to address other challenges such as cheating. IM plays an important role in how replication and communication between players are managed as will be discussed in the next sections, but ﬁrst, we explain how it is implemented.

Space-based interest management is based on proximity and follows an aura-nimbus model [BF93]. Aura is the bounding area around the player’s avatar. While nimbus deﬁnes the area around the player that is visible to the player, that is, the player can perceive game objects located in this area. Nimbus is also often referred to as Area-of-Interest (AOI) [KLXH04, SSJ`08], Domain of Interest [Mor96], Aura [GB02], or awareness area [KS03]. We mainly use the term AOI in this thesis. A player can typically only interact with objects and players in his AOI, and therefore, only needs to have copies of these objects. As a result, the necessary computational and network requirements are substantially reduced compared to maintaining copies of all objects.

Zoning The most common mechanism for interest management is zoning where the game world is partitioned into smaller parts, called zones or regions, as depicted in Figure 2.2.

13 Items Player & Vision Range Connections

2.2 Game Design Principles Items Player & Vision Range Connections

Items Player & Vision Range Connections Obstacle

(a) Free format (b) Grid (c) Hexagons (d) Triangles

Figure 2.2: Different game zoning mechanisms ([CWD`05] (adapted, c ACM 2005) and [BKV06] (adapted, c ACM 2006))

Zoning approaches differ in the shape of their zones and how the AOI is mapped onto zones. In the simplest approach, the entire AOI resides in the same zone in which the player is located. In some systems, the player can simply interact with all objects in his zone, i.e., his AOI is the entire zone [BC85, LGD98]. A player’s AOI changes only if he moves from one zone to another. In other approaches, the AOI is a sub-area of the entire zone. Often, it is a ﬁxed-radius circle (or sphere) around the player. When the player moves, his AOI moves accordingly. Therefore interest management has to determine for each game object in the zone whether it falls in the current AOI of the player. Figure 2.2 (a) shows an example of this sub-area approach.

More advanced interest management schemes allow the AOI to cover more than one zone. This is important for continuous worlds. Players at the borders of a zone should be able to see and interact with objects that are just across the zone boundary in a neighboring zone. Figures 2.2(b),(c), and (d) show examples where the AOI can cover several zones.

In regard to zone shapes, the game world can simply be split statically into grid cells as shown in Figure 2.2(b). Since this is a straightforward approach, many solutions use it [CWD`05, IHK04, CRd`04]. Hexagonal zoning (Figure 2.2(c)) has also been widely used for game architectures [YV05, MZP`95, JET03] as well as other systems such as cellular networks. Hexagons have uniform orientation and uniform adjacency, meaning that players will always move to an adjacent zone. In addition, since most AOI mechanisms consider a circle, hexagons provide a good approximation.

14 2.2 Game Design Principles

Using triangulation, as shown in Figure 2.2(d), allows for taking obstacles in the game world into account. This can help in reducing the AOI and thus, the number of objects that have to be considered for interest management, leading to less object replicas to be maintained by the players, and less update messages that need to be sent [BKV06]. Techniques such as Delaunay triangulation have been widely studied. They can be used to triangulate the area inside or around the polygon-shaped obstacle, so that triangles follow the boundaries of obstacles [BKV06, BA08] as shown in the ﬁgure.

Games can also be divided into mini-worlds that have a free format and are connected to each other such as countries in the game world, with portals for moving between them [CWD`05, KLXH04] as in Figure 2.2 (a). This is particularly useful when AOI always resides within a single zone. While mini-worlds do not offer the concept of a continuous world, other approaches do. The zoning is only virtual, done merely for the purpose of interest management, and is not visible to the players.

Determining the right size for zones is challenging. Different games have very different characteristics, and even inside a game, different parts may require different zoning mechanisms. Therefore, an optimal zoning mechanism for all cases does not exist. A very large zone can result in too many objects in a single zone, making interest management less efﬁcient as any player might only be interested in a small set of these objects. On the other hand, too small a zone may be much smaller than the AOI of players, resulting again in complex interest management between multiple zones. This trade-off is addressed by dynamic interest management schemes.

Limits of AOI & dynamic zoning AOI filtering suffers from specific user behaviors, e.g., flocking, which refers to the movement of many players to one area in the game world [PG07, CWD`05]. This can adversely affect replication management (see 2.7). Game hotspots form since some areas become more interesting or more profitable to the players in terms of experience points, treasures, new quests, or invitations by other players. This results in a large number of players coming to the same area while other areas become less populated. Hotspots can change quickly as players move to new interesting areas as they emerge, making it often difficult to prepare for such changes in advance. Populations in real games often follow a power law distribution [PG07], and the increase in the number of

15 2.2 Game Design Principles players in the area can result in a quadratic increase in the network traffic generated due to the increase in the number of interactions between players [BDL`08]. [VFBD11] find that in Second Life the number of objects per region is roughly constant over a one month period and that the active population at any point of time is between 30,000 and 50,000 avatars, i.e., about 0.3% of the registered avatars. About 30% of the regions are never visited in a six day period and less than 1% of the regions have large peak populations. Avatars tend to or- ganize in small groups of 2–10 avatars. Large groups of avatars are very rare and are driven by the presence of events such as concerts and shows. As a result, game designers may have to use artificial means to discourage players from gathering in the same region. This can prevent certain types of interesting game play such as epic battles [Bli11]. A more comprehensive study of player movements in massively multiplayer online role playing games (MMORPG) and session patterns is provided in [VFBD11, SDM09, MC10].

Flocking can be partially addressed with dynamic zones. As more players move into a popular zone, the zone can be split. However, if zones become signiﬁcantly smaller than the typical AOI, splitting does not really help anymore as inter-zone communications become the bottleneck. One possibility then is to shrink the AOI of the players. However, this might lower the game experience for the players. In addition, depending on the game design, AOI can be further broken down into different types [Mor96] but for the purpose of this thesis we use the general term AOI.

2.2.8 Consistency Control

In a distributed architecture like a multiplayer game, concurrent and possibly conflicting updates may be executed at different sites resulting in inconsistent states. Inconsistencies occur due to the execution of parallel and conflicting updates, and consistency mechanisms have to avoid or correct them. For example, if two players shoot a third player, nearly at the same time, all players should see the updates in the same order and only the first shot should be successful at all replicas.

Generally, systems are built such that if all replicas execute all updates in the same order, then all sites will have the same state. However, if messages are received out of order, their execution can result in inconsistent state if the out-of-order messages are causally

16 2.2 Game Design Principles dependent. For example, if a player drinks a healing potion to increase his health points and then, and only based on the increased amount of health, is able to pick up a sword, these actions have to be performed in the same order at all replicas, otherwise inconsistencies might become visible. Other types of inconsistency are caused by loss of updates. Most games, in order to deal with latency issues, use the fast but unreliable UDP messaging protocol where message loss is possible. Solutions are to send some updates with reliable TCP, or to use commit protocols for critical actions, in particular if they change the state of several objects and atomicity is required [MVHE04, BDL`08, BPS06, CF05].

Consistency Definitions Many definitions for consistency exist and consistency control has been studied in other distributed contexts (e.g., shared memory [AG96] or distributed systems [ÖV11]). A very strong form of consistency is achieved if every interaction is treated as a transaction that provides the transactional properties such as isolation and atomicity. However, providing transactional properties is often costly and might not be feasible for all game actions. For instance, running a two-phase commit protocol to guarantee atomicity leads to large latencies which could reduce the interactivity of the game significantly. Eventual consistency [TTP`95] is a weaker form of consistency. It allows individual copies of an object to be temporarily inconsistent but eventually consistent; meaning that if update activity did cease for sufficiently long time all object copies would eventually have the same state. Generally, there exist well-known trade-offs between performance and consistency restrictions, which can also be applied to MOGs. This fact has been well presented in Brewer’s “CAP conjecture” [GL02], indicating that a system can only achieve two out of three properties of consistency, availability, and handling network partitions. As interactivity is of essence to the success of most games, they often sacrifice consistency for other goals, and as a result, games usually provide inconsistency resolution instead of inconsistency prevention, enforcing not more than eventual consistency [CF05, MVHE04].

Different levels of consistency, implemented by various mechanisms, might be considered for different object and interaction types as not all interactions are of the same importance [ZK11]. For example, many virtual game objects are considered valuable and can be sold or traded for real money, making consistency control a critical requirement. In 2009, micro-transactions generated 250 million dollars in U.S. alone, with the most ex-

17 2.2 Game Design Principles pensive item being 635 thousand dollars3, and are projected to hit 13.6 billion dollars by 2014 worldwide4. In contrast, player position updates have much lower consistency requirements, and eventual consistency will be sufﬁcient.

Stale views In the previous sections we have introduced the primary-copy replication model that requires that all updates are first performed at the primary copy. This simpli- fies consistency management as it is not possible to have conflicting updates performed at different copies. Instead all updates are serialized at the primary copy. However, replicas will receive the update changes only some time after they occur at the primary. During this period, replicas are stale. Players observe these state values and might initiate invalid update requests to the primary copy, based on these stale values. Thus, actions submitted by players based on their local state might lead to results that the players did not anticipate. Ideally, the primary copy directly sends updates to all replicas in order to minimize staleness. However, this is expensive and other methods may be employed that may increase the latency and staleness experienced by the replicas. For instance, in Second Life, in a region crowded with virtual objects, avatars have an inconsistent view of their neighbor avatars half of the time, meaning either they do not see them or they see them at a wrong location. Moreover, in 50% of the cases, this inconsistency lasts more than one second [VFBD11].

Consistency Techniques Games use several techniques to hide staleness and inconsistencies due to the latency of update propagation. These techniques can also be used to hide message loss. They typically fall under the two categories of predictive contract mechanisms (PCM) [IEE95] and multi-resolution simulation [HNP97] and are often used together. Dead-reckoning is a common form of PCM [PW02a][GDK99]. It was originally designed in the nautical and aviation domain to calculate the current position of the plane based on the previous position and the motion vector. It is used in games to calculate the position of a player in the upcoming frame based on his previous location and his speed and direction of movement. It can also be used to detect collisions that will happen in the future. Dead-reckoning is used if messages do not arrive in time because of delays or loss.

3Planet Calypso Player Sells Virtual Resort for $635,000 USD http://www.prnewswire.com/news- releases/planet-calypso-player-sells-virtual- resort-for-63500000-usd-107426428.html 4Magid Associates and PlaySpan Release 2nd Annual Survey on Virtual Goods Mar- ket Penetration and Growth in North America. https://developer.playspan.com/developer/pdf/ PlaySpan_Magid_5_27_10_Final.pdf

18 2.3 MOG Architectures

However, it is also possible that all replicas continuously perform dead-reckoning, and the primary copy only sends a state update to the replicas if it detects that the state calculated by dead reckoning has a higher difference from the true state than a certain threshold. Neu- ral networks have been proposed as predictors [MWMD07a] for PCMs as well as a hybrid dead-reckoning/shortest-path predictor [MMWD05]. In Chapter 5, we present AntReck- oning which uses ant colony inspired methods to model players’ interest and improve the accuracy of the predictions based on a recent history of players movements and attraction or repulsion of surrounding game items.

Measuring Consistency The degree of inconsistency can be deﬁned by comparing the (potentially inconsistent) state at each client to a virtual perfect site that receives and executes all interactions with no delay and in the right order [MVHE04]. [CE11] provides an objective evaluation framework for Quality of Experience (QoE) in Games. QoE takes three basic perceptions: responsiveness, precision, and fairness. Responsiveness is the time the system takes to respond to an event, precision is the degree of accuracy required to complete an action successfully, and fairness is the degree of difference among all players’ gaming environments. Similarly, [VFBD11] considers inconsistency, interactivity and discovery latency to measure QoE.

2.3 MOG Architectures

The main game architectures for MOG, as shown in Figure 2.3, are the traditional client- server architecture, multi-server architectures (MS) and peer-to-peer (P2P) architectures. Different architectures try to achieve scalability through various means. Scalability can be achieved either by (1) increasing the resources or by (2) reducing the consumption. The methods discussed above such as AOI ﬁltering and dead reckoning are designed to reduce the resource consumption. Server-based architectures try to increase the amount of resources by adding multiple servers to distribute the load. P2P architectures increase the resources by using the resources available to each client as they join.

19 2.3 MOG Architectures

2.3.1 Client-Server Architecture

The typical client-server architecture is shown in Figure 2.3(a). In this architecture, the server holds the master copies of all mutable objects and avatars and maintains global knowledge of the game world. Clients connect to the server to receive the necessary information about the game world. All player updates and player interactions are sent to the server for execution as well as conﬂict resolution, and the server is responsible for sending object updates to all interested players after the updates have been executed. The main drawback is that even the best provisioned servers can only support a limited number of players. The common solution is to add multiple servers to improve scalability.

2.3.2 Distributed Multi-Server Architecture

The multi-server architecture is shown in Figure 2.3(b). Large game companies usually maintain server farms to provide service for the clients [KDC10, Ter02, But03]. Multi- server architectures can be divided into two categories. In the ﬁrst category, several complete instances of the game world exist, also called shards, and each shard is maintained by a server. Every server is responsible for a different set of clients and has a complete copy of the his own game world. That is, each set of clients and their server follow the traditional client-server architecture. There is usually no need for communication between these servers. Games and maps should be designed in a way that artiﬁcially prohibits players from moving between different games instances. Users are typically assigned to servers based on their geographical location and each server is responsible for a separate region (e.g., North America, Europe, . . . ).

In the second category, only a single game world exists that is divided into several regions. Each region is maintained by a separate server. Players are all in the same game world; however, in most games they are only able to interact with other players in the same region. Players are allowed to move between regions but this requires support for a hand- off mechanism between servers which can be transparent to the player. As the player moves near the border of a region the necessary information by the neighboring region is sent to the player and the player can easily cross the border. The hand-off mechanism also may not be transparent as the player is asked to go through a special portal or gateway in order

20 2.3 MOG Architectures

GW Shard GW Shard Game World

Client Client Client Client Client Client

Server Server Server GW GW Client partitioned partitioned Client Client (a) Client-Server Client (b) Multi-Server Client (c) P2P Client Figure 2.3: Different gaming architectures: (a) In a client-server architecture the server is responsible for maintaining the whole game world. (b) In a multi server architecture either: (1) servers maintain completely separate game worlds (shards) or (2) the game world is divided into different zones maintained by different servers. (c) In a peer-to-peer architecture each peer maintains a part of the game world. to enter a new region. Thus, the concept of regions can be used for interest management, as described in the previous sections, as well as for load-balancing among servers.

2.3.3 Peer-to-Peer (P2P) Architecture

Another option to increase scalability is the use of a peer-to-peer architecture as shown in Figure 2.3(c). Basically every node acts both as a server and a client. In the context of games, this means that each node could become responsible for maintaining master copies of some of the game objects and/or for update dissemination to other nodes. This model can be highly scalable since the load is distributed among all nodes and addition of new nodes also introduces new resources to the system.

2.3.4 Characteristics and Comparison

Here we discuss advantages and disadvantages of each of these architectures.

Server Architecture Centralized architectures provide the highest level of control over the game world. The game state can be the most valuable property of game developers, specially in case the game world is persistent. Game creators can easily change and update the game state and have control over the necessary updates in the software such as software

21 2.3 MOG Architectures patches, character updates, addition of new missions and expansion packs. Manageability and control are two main reasons why game companies typically use this architecture. An- other reason is that programming in server-based architectures is simpler in comparison to P2P architectures. Overall, the client-server architecture is the simplest of these architectures.

Easy consistency management is another reason for the popularity of centralized architectures. Since the server executes all updates and resolves conﬂicts, managing consistency is simpler than in multi-server architectures requiring hand-offs, or P2P architectures that have even higher distribution.

The biggest drawback of a single-server architecture is scalability. Even the best-provisioned servers are not able to handle more than a few thousand players. This number is further reduced for games that have stringent latency requirements. For example, the Quake II game (a popular FPS MOG) uses a client-server architecture and its scalability in the traditional client-server architecture is only a few hundred as measured in [BPS06].

Further problems are cost and fault-tolerance. A single server that is able to handle the massive load will be extremely expensive, and it remains a single point of failure. Its failure will interrupt game play and non-persistent game state will be lost. This problem can be solved by adding backup servers that can take over in case of failure. But maintaining backup servers introduces more complexity and cost, and may result in even further decrease in scalability.

Multi-server architectures Maintaining multiple servers improves scalability and in- herits other beneﬁts of the client-server architectures. Since the game world is divided into instances or regions and each instance is maintained by a separate server, a higher number of players can be supported simultaneously. As servers can serve as backup for other servers, this scheme has also the potential for higher fault-tolerance than a single-server system. But even if no backup mechanism is used, the failure of one server will only affect the players connected to this server and not interrupt execution at other servers. Second Life contains 18,000 regions where each of the regions is managed by a dedicated server [VFBD09].

One of the main drawbacks of multi-server systems is isolation of players if shards are

22 2.3 MOG Architectures used. Players can only interact with other players in the same instance of the game world and are not able to interact with players in other instances. If the game supports movements between regions, a complicated hand-off mechanism [KLXH04] is required that maintains game state consistency. Furthermore, a single region hosted by a server can only support a limited number of players. In case of ﬂocking behavior, even the best provisioned servers might not be able to provide service for an overloaded region. In addition, multi-server systems cannot be scaled without limit if a region-based approach is used, as the game world cannot be divided inﬁnitely into smaller and smaller regions. Most game developers address this issue by expanding or adding new maps as the number of players increases in order to maintain a low population density. However, this approach prevents the players from experiencing a more interactive environment. Another issue with many regions is the inter-server communication due to player movements. Different methods have been proposed in order to balance the load between regions. Most architectures assume that a single server is able to handle the load on a single region and if the load on a region is low, a server may maintain several regions, preferably adjacent to each other. Dynamic load balancing schemes [CWD`05] have been proposed to dynamically allocate overloaded regions to new servers while maintaining region locality in games. A combination of both shards and regions can also be used by game companies.

Finally, a major drawback is the cost. Maintaining server farms is very expensive and can prevent start-up or small game companies from entering the MMOG market. Acquiring servers for 30,000 simultaneous players can be about $800,000 and the bandwidth costs can reach hundreds of thousands of dollars [MPK03]. Maintenance costs such as cooling can match these numbers.

P2P architectures In principle, P2P architectures have the highest potential for scalability as every peer that joins the game, adds new resources to the system. A particular advantage is that these resources are added at no extra cost for the game provider. As the load is managed by the peers the need for expensive servers is greatly reduced if not eliminated. Furthermore, as responsibility is distributed across many nodes, each of them carrying only a bit of the load, the failure of any individual peer should not affect many other players if churn is handled appropriately. Moreover, as will be discussed in more detail in Section 2.5, if direct connections between peers are created properly, low latency can be achieved

23 2.3 MOG Architectures

Table 2.1: Comparison of different architectures Architecture Pros Cons ` Simplicity ´´ Scalability Client-Server ` Easy management ´´ Fault tolerance ` Consistency control ´´ Cost ` Scalability ´ Isolation of players Multi-Server ` Fault tolerance ´ Complexity ´´ Cost `` Scalability ´ Harder to develop Peer-to-Peer `` Cost ´ Consistency control ` Fault tolerance ´ Cheating as there is no need to send the updates to a central server and from the server to other clients. Instead, updates are directly sent to interested peers. For example, P2P Second Life improves user experience, measured in term of consistency, by about 20% compared to a client-server architecture, and avatar interactivity is also ﬁve times faster [VFBD09].

On the other hand, P2P architectures have several drawbacks. One of the major issues with P2P architectures is security. Cheating is easier in a P2P environment [BLL07, KTCB05, GV08]. Another problem is that P2P systems are harder to manage and control given that there is no central server that maintains global knowledge. Since the state is distributed among peers, it would be hard for game companies to have complete control over the game. Providing consistency control in P2P systems is also more difﬁcult since conﬂicting updates might be executed at different sites resulting in inconsistency. Gener- ally, coordination overhead and complexity is likely to rise with the number of nodes in the system.

A summary of the advantages and disadvantages of each architecture is provided in Table 2.1.

2.3.5 Hybrid Architectures

It is possible to combine a P2P architecture with a server-based architecture. These approaches can be divided into several categories according to what is being handled by the P2P system (see Table 2.2):

24 2.3 MOG Architectures

Cooperative Message Dissemination: The game state is maintained by one or multiple servers but update dissemination uses a P2P approach. Typically, players send their interactions directly to the server. After the execution, the server uses a P2P multicasting mechanism to send the updates to the players. This allows for a reduction in bandwidth requirements (and cost savings) at the server side.

State Distribution: The game state is distributed among peers. Peers can hold primary copies of objects and thus, are responsible for execution of player actions. However, all or part of the communication between peers may be managed by the servers. Moreover, the servers are responsible for some centralized operations such as authentication, and keeping track of joins and leaves of players. These architectures achieve scalability by distributing the cost of state execution among clients.

Basic Server Control: Both message dissemination and state distribution are done only through the P2P overlay. The servers’ primary role is to keep highly sensitive data, such as user logins, payment information, and players’ progress and state. They may perform authentication and admission control, meaning the server controls joins and leaves. Servers may also coordinate some types of interactions between the peers, e.g., those, that require the highest consistency. However, they do not maintain games state or perform state dissemination.

P2P systems can be of other uses to game developers as well. In World of Warcraft, for example, P2P systems are used to distribute software updates to clients. These systems can be similar to popular P2P ﬁle distribution systems. In this thesis, however, we focus on P2P systems that are used for maintaining game state or facilitate update dissemination.

2.3.6 Heterogeneous and homogeneous P2P systems

In heterogeneous solutions, some of the peers behave as super-peers (also referred to as coordinators [KLXH04], responsible nodes [YMYI05], and master nodes [YV05]) and some as normal nodes. Super-peers are responsible for most functionalities of the architecture. Super-peers can be chosen among the normal nodes that are given higher responsibilities or can be nodes with higher levels of resources (such as federated servers). Most architectures that divide the game world into regions and have a coordinator for that region deploy

25 2.4 P2P Architectures super-peers.

In homogeneous approaches, all nodes have similar responsibilities. For example, in pSense [SSJ`08], even though there are several types of possible roles for nodes, e.g., sensor nodes and neighbor nodes, all nodes have these responsibilities, resulting in a homogeneous architectures. Most mutual notiﬁcation systems fall under this category.

2.3.7 Other Functionalities

Inter server communication & streaming Multi-server environments can also beneﬁt from many of the techniques presented in P2P architectures. The idea is to deploy peer- to-peer protocols for the communication among servers [KDC10, IHK04, BPS06]. Servers are treated as peers and the same state distribution and update dissemination mechanisms suggested for a peer-to-peer architecture are applied to the server system. Slight modiﬁ- cations will be necessary in order to differentiate between servers (peers in this case) and game clients. This is further discussed in Section 2.10.

2.4 P2P Architectures

Here we discuss different P2P architectures proposed and categorize them according to the underlying P2P structure.

2.4.1 Introduction

Peer-to-peer systems are usually created by ad-hoc addition of nodes and build an application layer overlay network on top of the network layer. A fundamental requirement for very high scalability is that no single peer can know all other peers in the system. Instead it is always only “connected” to a fraction of peers. A prominent use of P2P systems has been in content (file) distribution systems. Instead of having a single server providing content to all clients, peers can download content from other peers, and in turn, offer their content to others. In order for a peer to find the content it is looking for, efficient query routing protocols have to be in place.

Based on how nodes are connected to each other, P2P overlays are generally divided

26 2.4 P2P Architectures into two types: structured and unstructured. In structured P2P systems, a deterministic protocol is used to form a speciﬁc graph structure (by deciding which peers build connections to each other) that ensures that any node can route a message to any other node (or ﬁnd an object) in the network overlay (by exchanging OplogpNqq messages where N is the number of nodes). Key-Based routing mechanisms are used to provide a lookup service similar to a hash table but different in that the (key, value) pairs are distributed throughout the nodes. They are able to add or remove nodes from the overlay with low overhead. Examples of such P2P substrates are Pastry [RD01a], Chord [SMK`01], Tapestry [ZKJ01] and Mer- cury [BAS04]. Distributed Hash Tables (DHTs) can use these substrates as the underlying mechanism for providing a hash table functionality distributed over many nodes.

In unstructured P2P systems, no deterministic algorithm for organization and optimization of network connections between peers exists. Network connections are generally established randomly or using probabilistic mechanisms that aim to converge into a suitable overlay (e.g., by connecting with a higher probability to nodes that are semantically close). Search in such networks is often done in a probabilistic manner, and in order to speed up the search, redundancy mechanisms, such as ﬂooding or content replication are used [SNW08, NWL`06, CMPC03]. Gnutella and Freenet are examples of unstructured P2P systems. The basic idea of many P2P-based games is to distribute the game state among peers and along with it processing, network, and storage tasks. In a primary-copy based replication scheme, this means distributing primary copies of objects among peers. Furthermore, the P2P overlay can be used for update dissemination. Both structured and unstructured P2P-gaming architectures have been proposed. A summary of some of the proposed P2P-based gaming solutions is presented in Table 2.25. This table is not meant to include all the P2P architectures that have been developed for games but to compare examples that exploit different architectures and strategies. We will describe many of these systems in this chapter.

5For an alternative list see: http://vast.sourceforge.net/relatedwork.php. While the categorization and terminology is somewhat different from this thesis, it provides a comprehensive list of related work.

27 2.4 P2P Architectures

Table 2.2: Comparison of representatives of different P2P architectures. (N/A: Not applicable)

Architecture Type Network Overlay Interest Management Replication & Consis- tency Control SimMud Structured DHT: Pastry + AL Static Regions: Rectan- Primary Copy: Region [KLXH04] Multicast: Scribe gular Cells Controller N-Trees Structured DHT: Pastry + AL Nested Regions Primary Copy: Region [GLZ05] Multicast: N-trees (scopes) + Scoped Controller events: Grid Cells Colyseus Structured DHT: Mercury + Di- Dynamic Rectangular Primary Copy: Each [BPS06] rect Cells + Prefetching Peer + Proactive Repli- objects cation + Soft-State Storage P2P Second Life Structured DHT (Kad) Dynamic: Adaptive Primary Copy: Region [VDB09] Cells Controller Badumna Structured DHT Regions + Bounded dy- Primary Copy: Region [KDC10] namic AOI Controller pSense Unstructured Direct(Neighborhood) Dynamic Circular AOI N/A [SSJ`08] + Automatic forwarding ASCEND Unstructured Direct(Neighborhood) Dynamic AOI, Voronoi Primary Copy: Each [HL04] + Connectivity assur- Diagrams Peer ance Solipsis(’03) Unstructured Direct(Neighborhood) Dynamic AOI, Convex Primary Copy: Each [KS03] Hulls Peer Solipsis(’08) Unstructured Direct(Neighborhood) Dynamic: Voronoi Primary Copy: Each [FRP`08] based on RayNet Peer Message Exchange Unstructured Direct(Neighborhood) Dynamic BW depen- Primary Copy: Region Scheme dent AOI, Neighbor- Controller [KAM04] hood Donnybrook Unstructured Direct (All-to-All) + Game world, Interest Primary Copy: Each [BDL`08] Forwarding pools Sets Peers + Doppleganger Amaze Unstructured Direct(All-to-All) Game world Primary Copy: Each [BC85] Peer VSM Unstructured Direct(Neighborhood) Dynamic AOI, Voronoi Primary Copy: Each [HCJ08] diagram Peer MiMaze Hybrid: Server + IP IP Multicast Groups Game World Primary Copy: Each [DG99] Multicast peer FreeMMG Hybrid: Server + Server + Direct Com- Static Regions: Rectan- Primary Copy: Each [CRd`04] Unstructured munication gular Cells Peer Quazal Hybrid: Server + Server + Direct Com- Duplication Spaces Duplication Spaces [Qua11] Unstructured munication Dist. Avatar Mgmt. Hybrid: Server + Direct(Neighborhood) Dynamic: Region + De- N/A [VFBD09] Unstructured launay Triangulation Hydra Hybrid: Server + Region Server + Static Region Primary Copy: Re- [CYB`07] Unstructured Proxies gion(slice) Controller

28 2.4 P2P Architectures

Distributed Event De- Hybrid: Server + Server + Load bal- Static Regions: Rectan- Primary Copy: Region livery System Structured ancing tree gular Cells Controller [YMYI05] IRS Hybrid: Server + Server Message Re- Server Primary Copy: Proxy [GV08] Structured laying + Proxy Exe- peer cution MM-VISA Hybrid: Server + AL Multicast Hexagonal Regions + Primary Copy: Region [ASO09] Structured Player Clusters Controller MOPAR Hybrid: Structured DHT: Pastry + AL Hierarchical AOI + Primary Copy: Region [YV05] + Unstructured Multicast: Scribe Static Regions: Hexag- Controller onal cells VoroGame Hybrid: Unstruc- Direct(Neighborhood) Dynamic: Voronoi Primary Copy: Each [BAC09] tured + Structured + DHT (Convex Polygon) Peer (Random DHT) Mammoth Hybrid: Server + Region Dynamic Regions: Primary Copy: Region [KVB`09] Unstructured Controller(All- Rectangular tiles Controller to-All) Zoned Federation Hybrid: Server + DHT Static Regions: Rectan- Primary Copy: Region [IHK04] Structured gular Cells Controller

2.4.2 Structured P2P Game Architectures

Structured P2P architectures typically use a DHT as the underlying mechanism for game state distribution and update dissemination. In this section, we will describe SimMud [KLXH04] in detail as an example of a DHT-based system. Most other solutions use a conceptually similar approach but optimize various aspects of the architecture.

SimMud uses Pastry [RD01a] to distribute game objects and Scribe [CDKR02] for update dissemination. Here we ﬁrst shortly describe these protocols and then explain Sim- Mud’s use of them.

Figure 2.4(a) shows Pastry’s overlay and message routing. In Pastry, every node has a 128bit NodeID. NodeIDs can be uniformly distributed using a hash code (e.g. SHA-1) of the nodes’ IP addresses or their public keys. In general, in order to distribute objects using Pastry (here object refers to any application object), each object is given an ObjectID (a hash code of the object) and its primary copy is placed on the Pastry node whose NodeID is closest to the ObjectID. An object can later be looked up by routing a message with ObjectID as the key. The message will automatically be routed to the node that has the object. Thus, when a node with a replica wants to update an object, it has to send the update message with the ObjectID as key and it will be automatically received by the node

29 2.4 P2P Architectures

| df dc dac dba df

Routedac dda

af(a)c Pastry Message Routing (b) Attribute Hubs in Mercury

Figure 2.4: (a) Pastry [CDG`02] routing (with permission of Castro et al.). In each jump the message with a given key is routed to a node with a digit more in common with the key. The keys are associated with game objects in SimMud [KLXH04]. (b) Mercury network overlay [BAS04]( c ACM 2004). A different attribute hub is created for each attribute. A node in each hub is responsible for a range of that attribute. Data items are assigned to attribute hubs based on their attribute values. Players in Colyseus [BPS06] use the range queries that specify their AOI for object discovery in games. with the primary copy, which can then perform the update. The design goal in Pastry is the ability to efficiently route a message with a given key to the node with the closest ID to that key. If the number of nodes in the network is N, Pastry routes a message in rlog2b pNqs steps b on average (b is a configuration parameter) using a routing table with rlog2b pNqs ˆ p2 ´ 1q entries. Pastry also provides the means for efficient node addition and deletion and can handle node failures.

Scribe [CDKR02] is a multicast primitive built on top of Pastry. For each multicast group Scribe builds a multicast tree with nodes being the peers of the overlay. Nodes can join and leave multicast groups. Any multicast message to a group is sent to all current members of the group, where it is sent to the root node of the group and is then forwarded along the multicast tree structure. The idea is that the root node is the node whose nodeID is closest to the GroupID. Therefore, to ﬁnd the root node, Scribe uses the query routing protocol of Pastry. Scribe provides best-effort delivery of multicast messages and is fully decentralized.

SimMud follows the approach of traditional multi-server architectures by dividing the

30 2.4 P2P Architectures game world into several regions (similar to Figure 2.2(a) where the region is AOI). In SimMud, game objects always reside in a single region and a player is only able to see game objects of the same region. To exploit Pastry, each region builds a group with a RegionID (a hash of the region’s name), and the peer with the NodeID closest to the RegionID becomes the coordinator of the region. Objects in a region are grouped together and are mapped to the coordinator in the Pastry key space. That is, the coordinator of the region has the primary copies of all objects residing in the region. When a new player enters the region it searches for the coordinator and receives replicas of all objects in the region. All updates on game objects of the region are sent to the coordinator, who serializes them, resolves conﬂicts and sends state updates to interested players.

For update dissemination, Scribe is used. SimMud creates a multicast group for each region such that its GroupID equals the RegionID. This means, that the coordinator of the region becomes the root of the multicast tree and simply sends updates along the multicast tree. All players of the region subscribe to the multicast group and receive the updates.

In SimMud, regions are static and each region is assigned to a coordinator via the Pas- try’s single attribute DHT. The Colyseus game architecture [BPS06] offers a more dynamic mechanism by exploiting Mercury [BAS04], a DHT substrate that supports range queries on several attributes (Figure 2.4(b)) . For every attribute, a number of nodes are joined together in a group called a hub to answer queries about this attribute. The range of an attribute is broken down into several smaller ranges. Every node in the attribute hub becomes responsible for one of these smaller ranges. Similarly, [THS07] uses quadtree index structures to support more complex queries such as range queries on various data types.

The main idea of Colyseus is to have multiple attributes, one for each dimension of the game world. A peer being responsible for a range of the x or y attribute knows about all game objects in this range of the game world. A player interested in game objects within a certain range searches for the peers that are responsible for this range via the Mercury query routing mechanism. These peers then inform the player about all objects that reside within the speciﬁed range. Colyseus uses Mercury only as a mechanism for discovering objects in the area of interest and receives updates afterwards directly from the holder of the primary copy. Objects periodically publish their updated attributes to Mercury to keep it up to date.

31 2.4 P2P Architectures

S S R

Q S S

P S S

S S .

snow_storm.8, .8, . (a) pSense neighbor discovery (b). N-Trees and scopes

Figure 2.5: (a) pSense neighbor discovery and sensor nodes[SSJ`08] (adapted, c IEEE 2008). A neighbor forwards a position update to an incoming node which in turn forwards its own update to the node resulting in a new neighbor discovery. (b) The game world is divided into nested regions represented by a tree where leaf nodes each represent a region and the parent node represents the sum of all its children regions. An event (such as a snow storm) is only sent to peers that are in regions overlapping with the event’s scope and only travels upwards when even scope is bigger than current region [GLZ05] (adapted, c ACM 2005).

Other structured architectures such as [GLZ05] rely on the same principles presented here. However, they might use different DHTs, interest management, or publish-subscribe systems. Similarly, P2P Second Life [VDB09] provides a region-based structured system, however, it uses Kad6 as the underlying substrate.

2.4.3 Unstructured P2P Game Architectures

Here we loosely deﬁne an unstructured architecture as one that lacks a global mechanism such as a DHT that would manage joins and leaves, maintain the overlay or provide global routing protocols.

Most proposed unstructured P2P architectures are based on a mutual notiﬁcation system [CRSZ02]. In these systems it is not necessary to divide the world into regions; instead, players use a mechanism to detect other players in their AOI and directly send their updates to these players. In order to detect players that move into the AOI of a player, players

6http://www.emule.com

32 2.4 P2P Architectures depend on notiﬁcations of their neighboring players. Here, we explain pSense [SSJ`08] as an example of such systems.

pSense is designed to manage position updates of the players and does not take other action types into account. Its design is based on the fact that peers in the AOI of a player should receive updates rapidly while other peers do not need to receive updates at all. Therefore, whenever a player moves, updates are sent to all nodes in the AOI of the player. In order to do this, a list of the nodes in the AOI is maintained, called the neighbor list. Whenever a node leaves the AOI, it is removed from the list.

In order to detect the nodes that enter its AOI, a node p relies on its neighbors, that is players close to this player in the game world (Figure 2.5(a)). If a neighbor q receives an update of p and knows about an approaching node r (which is already in the AOI of q and will soon enter the AOI of p) it forwards the update to r. Thus, r gets to know p and will send its own updates to p in the future. In order to improve on this system, each node p also maintains a set of sensor nodes outside of its AOI as shown in Figure 2.5(a). Node p also sends its updates to its sensor nodes which can then forward them to nodes that are entering p’s AOI. Sensor nodes also help to prevent the overlay from becoming disconnected (see Section 2.6).

In order to determine the sensor nodes around the local node several approaches can be used. One option is to divide the game world around the local player into n equally sized angles. The nearest node in each angle that resides outside the vision range is chosen as the sensor node. In order to maintain the sensor node list, when a sensor node receives a request, it can suggest others nodes that it thinks are better suited. This new node replaces the old sensor node.

Several other mutual notiﬁcation systems [HL04, KAM04, KS03, JMB09] exist that use mechanisms similar to pSense to build an AOI-aware overlay but use different neighbor discovery mechanisms as will be discussed in Section 2.6. While pSense exploits update dissemination to maintain neighbor lists, other mechanism can also use detection mechanisms that regularly exchange neighbor lists, typically through gossip messages.

All-to-all is another form of unstructured P2P architecture where all players maintain the full game state and use point-to-point communication for update dissemination. An

33 2.4 P2P Architectures example is Amaze [BC85], developed in 1985, or Donnybrook [BDL`08]. In such systems, special methods are needed to reduce bandwidth requirements. We will discuss these mechanisms in Section 2.5.

2.4.4 Hybrid P2P Architectures

Several approaches rely on a combination of the architectures discussed so far. Many of the solutions discussed, such as SimMud and Colyseus can be used in combination with a server as well as in a completely server-less setup. As in Section 2.3.5, we refer to solutions that use a combination of different P2P architectures or a combination of servers and P2P architectures as a hybrid architecture.

Hybrid client-server & P2P architectures: In FreeMMG [CRd`04], the server is responsible for maintaining user sensitive data such as user names and passwords. But it also divides the game world into segments and performs interest management. Similar functionalities are provided by the servers in [YMYI05]. In Quazal [Qua11] servers only do user management. In IRS [GV08] the game state execution is distributed between peers but con- trary to many architectures, the server is responsible for message dissemination between peers and running security checks on the updates.

In MM-VISA [AS10, ASO09], the authors propose a fault-tolerance procedure for a hybrid coordinator-based system. They focus on the difference between slow moving players, e.g. a soldier, and a fast-moving player, e.g., a tank, in the game world. A fast-moving player may move to other regions too fast, breaking all the necessary connections between the players before they can be repaired. They propose the use of player clustering, based on the type of their activities, inside each region to deal with fast-moving players. Players in clusters form their overlay according to the cluster they belong to. Therefore, clusters iso- late different types of players in the structure: a leaving player only affects the dependent nodes of that cluster not the whole zone.

In [NK10], the authors propose a hybrid scheme where interest management is performed in the cloud. It relies on P2P multicast trees, e.g. Scribe, to perform pub-sub functionalities.

Hybrid P2P-P2P architectures: In contrast to above systems that combine P2P solu-

34 2.5 Communications & Multicast tions with a server, MOPAR [YV05] is a special form of hybrid system as it combines two types of P2P architectures: structured and unstructured. MOPAR is similar to SimMud in its design and divides the game world into zones controlled by coordinators, and uses Pastry and Scribe for state distribution and update dissemination to the peers. However, coordinators also periodically exchange their neighborhood information. While this is conceptually similar to an unstructured message gossip, MOPAR does this exchange in a hierarchical manner. Coordinators only exchange their neighborhood information with the coordinators of neighboring zones and watch the neighbors for peers that might have subscribed to them. VoroGame [BAC09] uses a combination of structured and unstructured systems as follows: the structured system is based on a DHT and is used to maintain game objects. However, instead of using fixed regions, a mutual notification system based on Voronoi diagrams is used for neighbor discovery, providing more dynamic interest management (see Section 2.6). While these approaches benefit from the advantages of both architectures, one major drawback is the cost of maintaining two separate overlays, in terms of network bandwidth, memory overhead, and processing power.

Note that many of the systems discussed can be run in both hybrid and pure peer-to- peer fashion. Most coordinator-based systems fall in this category. VSM [HCJ08], on the other hand is an example of an unstructured Voronoi-based mutual notiﬁcation system that uses super-peers, be they peers with sufﬁcient resources or servers, for state management. The neighbor discovery in this approach is similar to other unstructured Voronoi-based architectures as discussed in Section 2.6.2. However, in this case the game state is managed by super-peers or arbitrators that can be servers.

2.5 Communications & Multicast

As already discussed, the publish/subscribe paradigm used in primary-copy schemes for update dissemination is typically implemented on top of a multicast mechanism. Thus, the performance of the multicast technique can greatly affect the cost and performance of the game environment. In this section, we look at various mechanisms and optimizations in more detail.

35 2.5 Communications & Multicast

2.5.1 Direct Communication

In its simplest implementation, the primary copy sends all object updates to the nodes that have subscribed to the associated object. The main advantage of this approach is low latency. Updates on a local primary copy can reach the other clients in one virtual hop, therefore, providing a better latency than client-server architectures where updates ﬁrst go to the server for execution and then to the players. The other advantage is relative simplicity. However, this can result in high bandwidth requirements.

In fact, the upload capacity of peers can become a bottleneck as most broadband connections (e.g. DSL) are asymmetric favoring download over upload capacity [MC10, SdDF`11, OSC`11]. The upload requirements can be calculated as the product of the number of clients that should receive the updates (subscribers), the update frequency, the object size and the number of objects. These parameters greatly depend on the game. For example, in Quake II with an average object size of 200 Bytes, 10 updates per second, and between 8 to 64 objects, the upload requirements can be between 1-66Mbps for 8 to 64 players [BPS06](not using delta coding as explained later). The update frequency depends on the type of game. FPS games typically have a higher update frequency than RTS or RPG games. AOI ﬁltering can reduce the number of clients receiving updates, thus reducing the upload requirements.

A particular problem is interest heterogeneity where a player becomes interesting to many other players, such as a player in possession of a valuable game item. These players have to send each update to many subscribers. An example is a “Capture the Flag” game where each team of players has to bring the other team’s flag to their own flag. There, flag carriers are focus points for all players, and the aggregate outbound traffic might surpass the carrier’s upload capacity [BDL`08].

Most unstructured architectures, as introduced in Section 2.4.3, rely on unicasts to all their neighboring nodes, e.g.,[HL04, KAM04, KS03, BDL`08, SSJ`08]. If the number of players in the AOI gets too large, the bandwidth requirements may exceed the available bandwidth. A recent study [MC10] using World of Warcraft logs showed that based on current residential broadband bandwidths available to users in the UK, direct P2P communication would result in saturation and high average latency.

36 2.5 Communications & Multicast

In addition to AOI, there exist several basic mechanisms to reduce bandwidth requirements:

Delta Coding It reduces the message size by only sending the differences (delta) in update messages. For instance, only the attributes of an object that have changed since the last update dissemination are sent. According to [BPS06], the size of an object update message can be reduced from 200 Bytes to 24 Bytes in Quake II using delta coding. As most games use UDP, update messages can be lost and delta coding can lead to incorrect state at secondary copies. Therefore, some games periodically send the complete object in order to correct the errors [BDL`08].

Exploiting resource heterogeneity Simple forwarding mechanisms can be used when individual nodes become overloaded because of interest heterogeneity or because they have generally little upload capacity. In an extreme case, all message forwarding can be performed by a server or a communication hub, and only state computation is distributed across nodes, as described in Section 2.3.5. In a more distributed solution, more powerful peers can serve as forwarders for weaker peers [BLB`04, BDL`08, SSJ`08]. pSense [SSJ`08] uses an ad-hoc approach. If a node does not have enough bandwidth to send an update to all neighboring nodes, it only sends it to a subset. In this case neighboring nodes help: whenever a node sends an update message it includes the list of nodes to which the message is sent. If a neighboring or sensor node receives a message and detects that it was not sent to a peer that might be interested, it will forward the message to this peer if it itself has enough bandwidth. This forwarding mechanism has its roots in probabilistic broadcast mechanisms [EGH`03]. Donnybrook [BDL`08] on the other hand deploys a forwarding pool consisting of nodes with good connectivity properties. Total bandwidth available to the forwarding pool should be able to support aggregate forwarding needs of other nodes. A machine becomes a part of the forwarding pool if its average latency to other nodes is below a certain threshold. In order to advertise their capacity, peers piggy back their available bandwidth on regular update message. A node that does not have enough bandwidth sends its update message to a random node in the forwarding pool which forwards it to other nodes. In each round a new random peer is chosen in order to eliminate the need for coordination and avoid overﬂowing a peer in the forwarding pool.

37 2.5 Communications & Multicast

Message Aggregation Multiple messages can be merged together to form a single message, thus reducing bandwidth requirements. Colyseus [BPS06] aggregates overlapping subscriptions using a local subscription cache, which maintains subscriptions whose TTLs have not yet expired, and an aggregation filter, which takes multiple subscriptions and merges them if they contain sufficient overlap. Multicast trees, as will be explained in the next section, can specially benefit from message aggregation as all messages are sent to the root node of the multicast tree giving it a unique opportunity to merge them together. SimMud [KLXH04], for example, cuts the bandwidth requirements in half by using message aggregation. In many cases an artificial delay can be added before sending messages in order to allow for more messages to arrive and be aggregated together [MC10].

Selective Dissemination The idea is to drop some events in overload situations instead of sending them, in order to maintain interactivity [PFCR06]. This idea is derived from the Random Early Detection (RED) [FJ93] mechanism used by TCP to deal with congestion where overloaded routers randomly choose some packets to drop. In gaming environments, the focus is in avoiding to send events that have become obsolete. For example, a position update becomes obsolete once a new move action has been performed. Methods such as ILA-RED [PFCR06] and obsolescence detection [Fer08b] can be used to drop obsolete events. Gossiping mechanisms [EGH`03] that forward messages from node to node, can be especially prone to obsolete messages. Thus, messages should not be forwarded more than two to three hops. A speciﬁc form of selective dissemination is used by the 2003 version of Solipsis (denoted by Solipsis(’03)) [KS03], an unstructured P2P game, where in case of insufﬁcient bandwidth a player reduces his AOI and drops some of the connections completely to better serve the players that remain in his reduced AOI.

2.5.2 Multicast Trees

Direct communication has its limitations. This is particularly true for coordinator-based architectures such as SimMud where super-peers have more responsibilities: they are in charge of a region and all communication for that region passes through these peers. In such situations, exploiting a P2P overlay for multicast purposes is attractive. A ﬁrst possibility is to exploit IP level multicast. For example, MiMaze [LGD98, DG99], a 3D server- less Pacman game, uses UDP/IP multicast, the Real Time Protocol (RTP) and the MBone

38 2.5 Communications & Multicast to disseminate updates to all players. Similarly, DIVE [FS98] relies on IP multicast and RTP. However, IP multicast requires support from routers which is not currently ubiquitous [CRSZ02]. As a result, End System Multicast (ESM) schemes, implemented at the overlay level, are attractive [CKV01, CDKR02, CRSZ02, MZP`95].

Most common are logical multicast trees where tree nodes are peers in the system. A multicast message is sent to the root of the tree and from there sent by every peer to its children in the tree. The main advantage is lower bandwidth requirements: as each peer in the tree typically has few children it will have sufﬁcient capacity for forwarding all messages, therefore, alleviating the bandwidth problems. For example, Score [LTB04] is a multicast-based communication protocol for large-scale virtual environments. To handle a large number of participants, it supports multiple multicast groups and multiple agents.

One drawback of this approach is higher latency as the number of virtual hops the message has to travel in order to reach the destination node is increased. This number is determined by the length of the path from the root to the destination node. Note that a virtual hop is a hop from one peer in the overlay to another one and may itself be comprised of several actual network hops. Therefore, messages can incur high latencies, and peers that are closer to the root will receive messages earlier than peers that are further away. Many multicast mechanisms try to maintain geographical locality in order to improve latency. As mentioned, games in particular are sensitive to delay, and high latencies render the architecture unusable.

A second problem is that many of the general purpose multicast mechanisms build trees whose inner nodes might simply be forwarders not interested in the messages themselves, that is, they themselves are not subscribers to the updates. In addition to the increased network overhead (i.e., nodes sending and receiving messages they normally do not need to (in other architectures) for rendering the game), games in particular can suffer from the possibility of unauthorized access, delaying the forwarding, or other malicious or selﬁsh behaviors by forwarders (e.g., reading or dropping other players’ messages, see Section 2.9).

Finally, maintaining multicast trees has computational and network overheads. While these issues hold in general for end system multicast mechanisms, they are particularly

39 2.5 Communications & Multicast problematic for games as they are very dynamic. For instance, in SimMud, whenever a player changes his region, he has to unsubscribe from one group and subscribe to a different group, leading to changes in the multicast trees. Similarly, when peers join or leave the game, adjustments have to be done. These changes can be very rapid depending on the speed of the game, and frequent group changes can undermine normal multicast operations. Similarly, since unsuitable multicast group sizes can have adverse effects on the performance, the games might have to adjust region sizes to make groups smaller, therefore, providing a worse game experience.

Several methods have been proposed to address these drawbacks. The N-Tree mechanism [GLZ05] divides the game world into nested regions when a region becomes overloaded. These nested regions are represented by a tree as shown in Figure 2.5(b). In addition, events are scoped. Scoped events are tuples consisting of actions, their location, and the scope of the event’s impact. An event is sent along the tree to other peers in the region or regions according to its scope. Therefore, only nodes that are interested in the update will receive the update message. In [YMYI05], the world is split into regions similar to SimMud; however, a DHT is not used. A distributed event delivery system is proposed that alleviates the load of region servers by building multicast trees with intermediate nodes as it becomes necessary. MM-VISA [AS10], as discussed before, addresses rapid movements in a zone by clustering players together according to their characteristics.

Many ESMs, such as Scribe [CDKR02], have built-in optimization in order to improve scalability and decrease the delay. Scribe employs measures to remove bottlenecks and collapse long path by removing nodes that are not members of groups.

2.5.3 NAT & Firewalls

Clients are typically connected to LANs that are behind NATs (Network Address Transla- tion) and ﬁrewalls. This can make inbound communications difﬁcult. As a result, communication protocols designed for games have to accommodate these limitations. Since most games use UDP for update dissemination, protocols such as hole punching and STUN [RMMW08] can be used for establishing connection mappings between peers [SLY`09]. Players unable to establish a connection between each other may rely on forwarding by

40 2.6 Interest Management other nodes or a server for communication. A study has shown that about 82% of the NATs tested support hole punching for UDP, and about 64% support hole punching for TCP streams [FSK05].

2.6 Interest Management

As mentioned, interest management is tightly coupled with the zoning and architecture used. In keeping with the discussion so far we categorize the interest management techniques according to the architectures.

2.6.1 Structured Architectures

In many structured P2P games [YMYI05, KLXH04, IHK04], the game world is divided into regions that are typically determined at the start of the game, and thus, are static. The AOI is usually deﬁned as the entire region, which allows the coordinator to simply multicast the message to all subscribers of the region. However, players may receive updates for objects that are not really of interest for them as they cannot be seen by the avatar. Alterna- tively, the AOI could be smaller than the entire region [KDC10]. In this case the coordinator would need to do message ﬁltering and a simple multicast cannot be used anymore since dedicated messages have to be sent to each player. In any case, supporting visibility and interaction between players close to borders in different regions is either very hard or expensive. Moreover, a hand-off mechanism is necessary when a peer moves to a new region and changing the coordinator of a region can be complex. While these problems sometimes happen in unstructured networks as well, they are more common in structured systems.

Few structured gaming architectures are ﬂexible in their interest management. One way to make region-based interest management more dynamic is to dynamically change the size of the region in question. N-Trees [GLZ05] and P2P Second Life [VDB09] dynamically divide a region into smaller regions as the region becomes overloaded, i.e., the number of objects in the region is higher than a threshold. The drawback of such an approach is the increased cost of hand-offs to neighboring regions as regions become smaller. Another approach proposed by Badumna [KDC10] is hierarchical interest management: The game world is divided into cells, however, each player has a dynamic AOI, e.g. in the shape

41 2.6 Interest Management of a sphere, inside the cell. This allows for more control and lower interest management overheads. In addition, players close to each other use a gossip protocol between each other to avoid sending too many queries to the coordinator and overloading it. In Coly- seus [BPS06] players use multi-attribute range queries to register their AOI with Mercury [BAS04]. Whenever the AOI of a player changes, he subscribes to the new AOI by sub- mitting a subscription request for the range query corresponding to the new AOI. He also unsubscribes from the old AOI. At subscription time, the player receives replicas of all objects that are in the new AOI if he doesn’t have them yet.

2.6.2 Unstructured Architectures

Most unstructured gaming architectures provide dynamic AOI management. The principal idea of most approaches is that each player keeps a neighbor list of players and objects in his AOI and whenever players move, they send position updates to all neighbors. At the same time, a move changes the AOI of the player to some degree. Therefore, he has to get to know the players in the new area that was not covered by the old AOI. The big difference between these mechanisms is in the shape of the underlying AOI, from a simple circle to more complex shapes, and how these shapes help in ﬁnding new neighbors.

Neighbor Detection We further divide the neighbor discovery methods to two groups:

Tessellation techniques: Many approaches rely on a tessellation technique to determine the area belonging to a player and his neighborhood. For example, in ASCEND [HL04], later VON [HCC06] and VSM [HCJ08], each player computes a Voronoi diagram [BHML92] based on his and other neighboring players’ positions as shown in Figure 2.6(a). In the Voronoi region/cell, all points in the region are closer to the local node than any other node. In these networks nodes in neighboring cell are connected to each other. When players move, they send position updates to all neighboring nodes causing them to recalculate their Voronoi diagrams. If a neighbor detects that a player’s AOI is now overlapping with another neighbor, he sends a notiﬁcation alerting the player about the new approaching player.

In Solipsis(’03) [KS03], each node calculates a convex hull of adjacent entities as shown in (Figure 2.6(b)). Nodes frequently query their neighboring nodes to inquire if

42 2.6 Interest Management

e (a) VON Voronoi diagram (b) Solipsis(’03) convex hull (c) Solipsis(’08) Raynet overlay Figure 2.6: (a) Neighbor discovery and Voronoi tessellation in VON [HCC06]( c IEEE 2006). Different neighbor help in the process of neighbor discovery. (b) Convex hull cal- culation by Solipsis(’03) [KS03] (with permission of Keller and Simon). (c) Solipsis(’08) Raynet (Voronoi based) overlay mapping [FRP`08] (with permission of Frey et al.). they have detected any new entities in their AOI. A newer version of Solipsis in 2008 (denoted by Solipsis(’08)) [FRP`08] relies on Raynet [BKR07], an n-dimensional Voronoi- based overlay, to provide support for multi-participant virtual worlds (Figure 2.6(c)). It uses data-dissemination policies that take advantage of the view-dependent representation of the 3D content. In addition, it distributes the execution of computationally intensive tasks. [VFBD09] uses a Delaunay Network which is an overlay network whose topology is deﬁned through Delaunay triangulation. The coordinates of avatars in the virtual world are used to generate the Delaunay triangulation, and consequently connect neighboring nodes among the respective end-users.

Free format: These methods only rely on exchanging positions between players for neighbor detection. pSense [SSJ`08] simply uses a circle with a given radius around a player as the player’s AOI. In order to detect new neighbors, neighboring peers help out. If they receive a position update and realize that the sender should see a new player he is not yet aware of, they forward the message to the new player which can then initiate the interaction with the sender. In [KAM04], each peer keeps track of a ﬁxed number of nearest nodes. Nodes constantly exchange their neighbor lists via gossiping and calculate and sort the distances (Euclidean distance in game world) between entities. N nearest nodes are chosen as the active entities and the rest as latent entities that have potential for interaction.

43 2.6 Interest Management

2.6.3 Challenges

Structured and unstructured architectures each have several advantages and disadvantages:

Join process for new nodes: In structured networks, join process is typically more streamlined (e.g., O plog nq) but this comes at the higher costs of maintaining different routing tables. Whenever a new node joins the overlay, in addition to ﬁlling its routing tables, neighboring (in the overlay) nodes have to be notiﬁed and update their tables as well.

A problem with most of the unstructured approaches is that when a new node joins the system, it might take a long time to get to know its real neighbors if it only knows peers that are far away from its AOI. Same is true if a player teleports, i.e., instantly moves to a far region.

Connectivity: In structured systems, the global managing system ensures accessibility and connectivity of the peers. However, the node location in the structured overlay may not directly correspond to the player’s location in the game world and his neighboring players. This means the player will have to maintain many connections that are unnecessary from a game rendering point of view.

In unstructured systems, the overlay is more likely to become disconnected. As typically a ﬁnite number of nearest neighbors are maintained by a node, groups of users may lose contact to each other if they are separated by a large distance [KAM04]. In particular, if there are several hot-spots in the game world, different communities might become disconnected. pSense reduces the risk of isolation by requiring each node to connect to a sensor node in each part of area around it even if the sensor node may be far out of its AOI. The use of Voronoi diagrams or convex halls effectively avoids disconnections but requires complex and process intensive calculations. Moreover, the architectures are presented in 2D and are not easily extended to 3D without loss of scalability as these calculations do not scale well in 3D.

Recall: Due to their design, structured overlays provide better recall than most unstructured overlays, i.e., if an object exist system, a request will most likely ﬁnd it.

Overhead: Structured overlays rely on frequent messages, e.g. heartbeat messages,

44 2.6 Interest Management to maintain connectivity and have performance optimization techniques to choose closer neighbor (according to a performance metric) that themselves rely on occasional measurement of networking performance of the neighboring nodes. A drawback of unstructured overlays is the overhead of frequent and explicit exchange of neighborhood lists between peers when gossiping is used [KAM04, KS03]. pSense includes the addresses of all neighboring nodes that are recipients of the update message in the message, resulting in a high overhead for each update message.

Multi resolution interest management In all-to-all P2P game architectures other techniques than AOI have to be used for interest management. For instance, in open world maps or when regions are very big, players have a larger visibility range, and thus, can see a larger number of players. Furthermore, if an avatar is in a popular area many other players will reside in his AOI even if it is small. In all these cases the trafﬁc of this player might exceed his bandwidth capacity. To address this issue multi resolution techniques can be used that rely on sending updates at different rates and with variable details based on how interesting a player is.

MASSIVE [GB02] differentiates between focus, the observer’s allocation of attention, and nimbus, the observed object’s manifestation or observability. The observer’s awareness is a combination of these two factors.

Donnybrook [BDL`08] proposes Interest Sets. The reasoning is that human beings are only able to focus on a very limited number of items at once. Therefore, a player is only interested in the exact actions of few other players and will only notice basic information about other players in the visibility range. This fact can be exploited by creating high- fidelity replicas only for the players that the player has currently high interest in, while maintaining low-fidelity replicas of other players, which Donnybrook calls doppelganger. High-fidelity replicas are maintained by sending frequent updates while low-fidelity replicas receive guidance messages at a slow rate. In order to make doppelgangers look smooth, AI techniques are used to extrapolate realistic behavior between the reception of guidance messages (see Section 2.7). By keeping a fixed number of players in the interest set, the number of high-fidelity replicas does not depend on the size of the AOI or the number of players in the AOI but can be chosen to easily handle the bandwidth requirements. The fre-

45 2.7 Replication & Consistency Control in P2P Architectures quency in which low-ﬁdelity replicas are updated can be adjusted depending on the number of replicas in the AOI. Similarly, the use of different Level of Details (LoD) [CHJC10] has been proposed in the context of 3D content streaming as well as games hosted in the cloud [NK10].

A question is how the game engine can determine which objects actually get the player’s attention and need to be high-fidelity replicas. In order to estimate attention, Donnybrook uses the weighted sum of three parameters. Proximity reflects the fact that players tend to pay more attention to other players in their very close vicinity. Aim refers to first-person- shooter games, where players closely observe objects they are aiming at. It is determined by the angle between the player’s view and the other players. Finally, interaction recency takes temporal locality into account as players who interacted recently are more likely to pay attention to each other.

2.7 Replication & Consistency Control in P2P Architec- tures

Here we present a brief discussion on where the primary copies of players, NPCs, and other items are stored and how their replicas are managed.

2.7.1 Replication Management

As mentioned, most multiplayer games use a primary-copy scheme. Different architectures propose different locations for keeping the primary copy and how to send updates to replicas. Also, they provide different methods for keeping the game state as a whole consistent.

Location of primary copies In systems where super-peers are coordinators for regions [KLXH04, YMYI05, YV05] the primary copies of all objects in the region reside on the coordinator. This includes players, non-player characters and other types of objects. The coordinator receives all update requests for these objects, executes them and disseminates updates typically via multicast. In these systems, load balancing can be achieved by making regions smaller for the overloaded nodes [GLZ05], thus, reducing the number of objects maintained by the coordinator. A particular problem can be NPCs, as they have to execute a

46 2.7 Replication & Consistency Control in P2P Architectures think function to determine their next actions. Given that AI calculations can be processing intensive, this may lead to overloading the coordinator. The coordinator may decide to delegate this task to other players in the region [IHK04].

On the other hand, typically in mutual notification and all-to-all systems, each player is the holder of his avatar’s primary copy and can perform updates locally. The update dissemination is done directly to other peers. Many of the unstructured systems, e.g. pSense or distributed avatar management [VFBD09], ignore the problem of state management for objects and NPCs. A possible solution, e.g., in Voronoi-based architectures like Solipsis(’08), is a state management scheme that puts primary copies of game objects and NPCs inside the (Voronoi) region onto the peer responsible for that region. In addition, Solipsis(’08) distributes the heavy physics computations to allow for better scalability. One problem with such an approach is that primary copies of objects migrate too often between players as the player moves in the game world and their Voronoi regions change. Furthermore, NPCs can also change their positions frequently, leading to additional migrations. However, the fact that players in the neighboring regions already would have a replica of the objects and NPCs, can help lower the object migration costs. VSM [HCJ08] provides an state management mechanism for Voronoi based mutual notification systems. However, it relies on super-peers, called arbitrators or aggregators, for this task. Aggregators are responsible for multiple cells. An underload and an overload parameter is defined and the aggregator merges the load of underloaded peers or splits overloaded cells. The radius of an aggregator is determined according to its resources and shrinks as it becomes overloaded.

Fast replica management In general, a player only maintains replicas of objects in his current AOI. When the AOI changes, the peer might have to receive new replicas. This can be time consuming and lead to lower game quality. To circumvent this problem, objects can be prefetched. Often, the direction a player is moving to can be predicted (see Chapter 5). Thus, replicas of relevant objects can be transferred before they actually enter the AOI [BPS06]. One can also simply have an area of subscription that is larger than the AOI. Whenever an object is in this area of subscription, the player subscribes to it and receives a replica. Similarly, object replicas are only deleted when they are no more in this AOI for some time. This prevents thrashing behavior where a player continuously creates and deletes replicas due to small movements.

47 2.7 Replication & Consistency Control in P2P Architectures

Standard object discovery might not be fast enough for short-lived objects in the game such as missiles in Quake II. These objects are created during the game and live only for short times. Therefore, prefetching does not work. Colyseus solves this through proactive replication. It attaches each short-lived object, e.g., a rocket, to the long-lived object, e.g., player, that creates it. Players that are interested in the creator will automatically receive replicas of attachments when they are created. The same mechanism can be used for NPCs that are attached to player avatars such as companions or pets that accompany the player’s avatar in the game. Since these companions are geographically close to the player their primary copies are typically assigned to the same node as the main avatar.

2.7.2 Consistency Control

As discussed in Section 2.2.8, various levels of consistency can be deﬁned. Games can often tolerate relaxed forms of consistency control. The primary-copy model provides strong consistency control over an object (meaning concurrent and conﬂicting updates are not sent to replicas), however, replicas might have stale information.

Hiding staleness In order to hide the staleness of data, several mechanisms can be used. In Section 2.2.8 we mentioned dead-reckoning and multi-resolution mechanisms. These can also be applied in P2P architectures. Dead-reckoning can be used for players that do not send their updates frequently. However, even though many approaches have been proposed to increase the accuracy, e.g. AntReckoning, that we discuss in Chapter 5, dead-reckoning is generally only able to tolerate delays of a few hundred milliseconds and is restricted to player movements. Donnybrook, on the other hand, uses a special replica called doppelganger. A doppelganger is an AI controlled program aiming to be a realistic approximation of the remote player. For that the remote player sends infrequent guidance messages which include statistics about the player and the possible future moves. The AI tries to never de- viate substantially from the guidance messages, and thus, the character appears natural to players that are not paying close attention.

Multi-level consistency control A particular challenge arises if an action updates more than one object as this requires coordination between different primary copies. Ideally such execution is transactional, providing atomicity and isolation. However, well-known proto-

48 2.7 Replication & Consistency Control in P2P Architectures cols such as locking or quorum techniques are expensive, and most games accept weaker consistency levels.

In [ZK11] the authors analyze various types of game actions and categorize them according to their consistency requirements. Different interactions have different levels of sensitivity and can be categorized accordingly. For each category, a consistency algorithm is provided. The two lowest levels are for actions on a single object. In the no-consistency level, replicas are only updated through a best-effort approach. Graphical effects could be actions for which this level is sufﬁcient. The Low-consistency level provides a bound on the staleness of the data. Progressive movements of players can fall in this category. Dead reckoning for example can be considered a low consistency mechanism. The three next levels are for multi-object actions. They differ in the way in which they provide the transactional property isolation. Medium consistency level does not provide isolation, in particular it allows stale reads, thus, actions might lead to results that are not anticipated by the player (e.g., picking up an empty bottle that the player thought to be full). High consistency level provides isolation only in regards to some speciﬁcally marked attributes such as the price of an object. Finally, exact consistency level provides complete isolation avoiding any form of stale reads. Clearly, the higher the consistency level is, the more costly the mechanism that achieves it.

Not all games use a classic primary-copy approach. Some allow updates to be submitted to any copy (e.g., for better responsiveness for the client). In this case, an inconsistency handling protocol is needed.

Optimistic consistency control In many games, updates are performed optimistically on replicas at the time the update is submitted locally. This helps to hide the latency experienced by sending the updates to the primary. If the action leads to different changes at the replica and the primary, an inconsistency resolution protocol is necessary. One simple resolution mechanism, used for example in mobile games [CF05], could be to choose one of the states as the authoritative state, discarding all other inconsistent states. In Colyseus if there are concurrent updates, the last received update wins without any conﬂict resolution. An alternative form of conﬂict resolution is to roll back to older, consistent states. Timewarp [Jef85] is a rollback-based mechanism proposed for distributed simulation and

49 2.8 Fault Tolerance & Persistence databases, where all copies are allowed to execute updates optimistically. They keep track of old states, and roll back when inconsistencies occur. Such inconsistencies can occur, e.g., if nodes receive out of order updates. Timewarp also periodically calculates Global Virtual Time (GVT) which is a time point in the system where it is guaranteed that no participant will roll back to a time before it. States with a time point before GVT can be removed as they will never be used. As rollbacks can be dissatisfying to players, several optimizations have been proposed over Timewarp to improve its performance in a game environment [Fer08b]. Hydra [CYB`07] proposes a new programming model to provide consistency guarantees when nodes fail in a P2P gaming architecture. This is achieved through a system of checkpointing and restoration of application game state as well as imposing conditions on message processing and message delivery.

2.8 Fault Tolerance & Persistence

2.8.1 Fault Tolerance

Generally, peers in a P2P system are much more prone to failures or unscheduled disconnections than servers in corporate data centers. This can result in the loss of the game state maintained by the failed peers or messages that are forwarded within a damaged multicast tree. In addition, it may result in loss of connectivity and isolation of peers.

Note that some of the general failure assumptions for P2P systems can be weakened in case of games, as discussed in [KLXH04]: (1) As players in MOGs are typically geographically diverse, node failures will be independent and it is unlikely that a large number of nodes fail simultaneously. (2) In short-lived games players typically stay until the end of the gaming sessions and do not suddenly leave. In continuously running games ungraceful exit from the game might be penalized as it can be considered as escaping. Thus, players are likely to leave in a controlled way allowing for a correct overlay and multicast tree reconﬁguration. (3) Games usually provide only weak consistency guarantees even if no node failures occur. For instance, sporadic message loss can be handled, inconsistent replicas are possible, and reconciliation mechanisms exist. Thus, failure handling becomes less critical.

50 2.8 Fault Tolerance & Persistence

In addition, many general purpose DHTs and multicast trees have built-in mechanisms that repair the network overlay and the multicast trees in case of node failures. They typically rely on heartbeat messages sent frequently by nodes to each other in order to detect unresponsive (failed) nodes. MOGs that are built over these existing P2P systems can take advantage of these mechanisms.

Primary copy & connection loss One main issue in case of node failures is the loss of the primary copies that resided on the failed node. In case of structured P2P games with region coordinators, a primary/backup mechanism can be used where the state at the coordinator is replicated on one or more backups that will take over in case of failure [CDKR02]. As DHTs such as Pastry automatically send all messages that were directed to a failed node to the node with the closest NodeID, this node can be chosen as the backup of the coordinator, and failover occurs transparently. In [AS10, ASO09] a hybrid approach is presented where player clusters are used to minimize the loss of connections and messages due to fast-moving players. In addition, two approaches are proposed to reduce the loss of connections. First, players are required to maintain redundant connections to their siblings and grandparents in the multicast tree. Second, a standby node is used to forward game states when needed.

In unstructured architectures peers often only hold the primary copies of their own game objects. Therefore, in case of failure only little game state is lost and the other peers can continue the game. In case nodes can hold primary copies of general game objects, e.g., of objects in their Voronoi region, VSM [HCJ08] suggests to choose a random backup node for each node N in the overlay, that is in charge of keeping replicas of all primary copies that node N holds. In case of failure, neighboring nodes will have to initiate primary copies for these general game objects and can receive the state by querying the corresponding replicas on the backup node.

Hybrid architectures might be able to rely on servers that are less likely to fail. Servers could maintain replicas of all objects which can then be used to reinitialize failed primary copies.

Availability of the system is tightly coupled with fault tolerance. It depends on how well the failure of nodes are handled and the service is restored by other nodes taking over. Many

51 2.8 Fault Tolerance & Persistence

P2P architectures have built in mechanisms to determine availability of nodes, e.g., heart beat messages. In addition, several availability monitoring and management systems has been proposed for both structured and unstructured P2P systems (not specific to games). Total Recall [BTC`04] manages availability by adapting the degree of redundancy and frequency of repairs to the distribution of failures and can guarantee user-specified levels of availability. It performs three main tasks: (1) Availability prediction: the system monitors the current state of the system and predicts future availabilities. (2) Redundancy Manage- ment: based on predictions redundancy requirements are calculated. (3) Dynamic Repair: current availability monitoring information along with the predictions are used for dynamically repairing the system. Total Recall is built on top of a DHT substrate making it more compatible for use with the structured P2P architectures we have studied. Similarly, AV- MON [MG09] provides a protocol for availability monitoring of the overlay in a scalable and efficient manner, that works with any arbitrary monitor selection scheme that is consistent and verifiable.

2.8.2 Persistence

Apart from replication, making game state persistent is another way to survive node failures. Even if replication can be leveraged for fault-tolerance and quick fail-over, persistence remains an important building block to allow for game maintenance, to be able to recover from severe failures, and for very critical information for which maintenance within a transactional database system is necessary. Generally, persistence in P2P based MOGs has not received a lot of attention [GE11].

What to store: Depending on the type of the game, the information that needs to be persistently stored is different. In a game like World of Warcraft, as avatars progress, their appearance, level, items and location are typically made persistent as they are essential to the game. For games that are more short lived, it is often game summary information, such as reputation of players, and their wins and losses, that are kept in persistent storage as these are the information that should survive the termination of the game session. In general, statistics, e.g., how successful players have been in different actions, might be recorded on stable storage, but not every detail about the actions. [Zha10] categorizes actions into persistence levels, similar to the consistency levels discussed in Section 2.7.

52 2.8 Fault Tolerance & Persistence

Certain action types might never need to be written to stable storage, for others it might be sufﬁcient to write them asynchronously, i.e., some time after they have occurred, which allows for aggregation, batch writes, and lower response times. Finally, for the most important actions it will be necessary to make them transactional, forcing persistence before the action concludes.

Persistence mechanism: The persistence mechanism is also dependent on the architecture used. In its simplest form, hybrid architectures can rely on the server component to provide persistence. If the server component is anyways responsible for game state, i.e., peers are only responsible for dissemination, this is straightforward. However, when game state is maintained by peers, i.e., they have the primary copies, the server would need to keep replicas of objects in order to receive the updates, e.g. by subscribing to the multicast groups, and then store them to permanent storage. The way the server subscribes to these updates, e.g. frequency and level of detail, should reﬂect the persistence categories of the items and actions in question.

Similarly, in super-peer approaches where coordinators are responsible for a region, the coordinator can be in charge for the persistence of its region. However, in pure P2P systems, game engines need to resort to a distributed permanent storage system. There exist many solutions [BTC`04, RD01b, NWL`06, CDH`06] in structured or unstructured systems that provide these functionalities. Most approaches achieve availability and durability through redundancy: by keeping several copies of each data file so that it is not lost due to churn. For instance, Total Recall [BTC`04] constantly monitors nodes in order to decide on the required replication degree and to dynamically repair lost copies. Other distributed storage systems take advantage of the particularities of structured DHTs, for example in order to decide where to store which files. Architectures like [HBH06] can use PAST [RD01b] to provide a persistent storage. PAST uses Pastry as the underlying content location and routing mechanism. It assigns a hash code as the file IDs to each file in order to distribute files. Caching and load balancing techniques are also used to ensure efficient use of resources and the availability of the system.

Unstructured P2P systems typically use locality to maintain replicas for game items, therefore, they cannot use a typical distributed ﬁle management system. However, they can rely on typical unstructured replica and ﬁle distribution techniques to create a distributed

53 2.9 Cheating storage system [NWL`06]. Carbonite replication algorithm [CDH`06] provides efﬁcient replica maintenance for a distributed storage system. This is achieved by a range of techniques such as separating durability guarantees from availability, given that availability requirements are generally more expensive.

[GE11] provide an overview of persistence mechanisms in P2P architectures and compare different approaches for their suitability for P2P based MOGs.

2.9 Cheating

Cheating is generally one of the main concerns in the design of a game architecture and one of the reasons game publishers have avoided the use of P2P architectures. Cheating is used by players to gain an unfair advantage over other players and to speed up their progress in the game. This results in non-cheating players (also called legit players) becoming frus- trated and either becoming a cheater themselves or leaving the game [GV08]. Even though game companies are reluctant to publish data on cheating and security breaches, there is compelling evidence, such as users being banned from playing78, that security problems and cheating are a serious issue. Frequent patches by game publishers, containing security ﬁxes to prevent cheating, and disciplinary actions taken against cheaters, show that cheating is a pervasive threat. For example, over 35% of players in Diablo admitted to cheating at least one time. In addition, many companies have experienced losses in customer base due to high rates of cheating, i.e., legit players leave the game due to frustration caused by players who cheat [GV08].

P2P architectures, as a result of their design, are more prone to attacks and cheating than centralized systems [CM04a, MKY`07]. For instance, if the P2P overlay is used for update dissemination, players can cheat by withholding or accessing update messages giving them an unfair advantage. If the P2P architecture provides distributed execution of updates and allows players to hold primary copies of game objects, cheaters can manipulate their object repositories and perform game state changes that violate game rules.

7Bungie resets 15,000 Halo Reach players: http://www.bungie.net/Forums/posts.aspx? postID=49997802 8Blizzard bans 5,000 StarCraft 2 cheaters http://us.battle.net/sc2/en/blog/882508

54 2.9 Cheating

In this section, we present a deﬁnition of what constitutes cheating, different categories of cheating, and why P2P architectures are more vulnerable to cheating. We do not aim to provide a list of every single form of cheating but we focus on architecturally more important cheats. From there, we discuss various cheat detection and prevention techniques that encompass different categories of cheating and architectures. This section only aims to give an overview of this important topic. Given the importance and range of the subject a detailed study of the subject seems appropriate and can be a survey paper in its own right [WS07, HM07].

2.9.1 Deﬁnition

In this section, we use the deﬁnition of Goodman [Goo08] for cheating: “Cheating occurs when a player causes updates to game state that defy game rules and result in unfair advantage”. Cheating is possible as games are big and complex software systems and tend to have bugs and errors that can be exploited by users to give them an unfair advantage (as opposed to the developers’ intents). A game can be considered fair if the game state perceived by every player in the game is consistent with their expectations [BLL07]. In addition, cheating mechanisms can be based on collusion where several players collabo- rate for cheating purposes [Goo08]. This collaboration can take place inside or outside the game world. For example, in a game of bridge, players might inform others of their hand.

Note that defying game rules alone may not be used as a complete measure since rules may not be clear to the players resulting in unintended behavior. Furthermore, there are emergent behaviors where players are attracted to unintended aspects of the game, but this is typically not considered cheating. For example, in a racing game players may prefer crashing into each other rather than racing. Another concern is social misconduct which occurs when players do not break any game code rules but their behavior is nonetheless considered inappropriate. Abuse of other clients and other anti-social behaviors such as verbal abuse are examples of social misconduct. A well known type of social misconduct in gaming is grieﬁng 9 where a player deliberately tries to ruin other players’ game experience, for example by stalking. As unintended and inappropriate behaviors are often very game speciﬁc, we will not discuss them further.

9http://www.wowwiki.com/Griefing

55 2.9 Cheating

2.9.2 Cheating Categories

Cheating techniques can be categorized along various dimensions such as the layer in which cheating occurs (e.g., [WS07, YR05]), or along the architectural nature of cheating (e.g., Chapter 3,[Goo08, BLL07, KTCB05, WS07]). We chose the latter, given our interest in the impact of P2P architectures on the vulnerability of multiplayer games. While many of these cheating techniques can occur even in client-server systems, we argue that in most cases P2P systems are more vulnerable to them. The taxonomy is fairly similar to the parts listed above.

Interrupting information dissemination. By delaying, dropping, corrupting, or changing the rate of the updates or sending wrong or inaccurate information, a player can blind or confuse other players about his current state. As a result, the chances of his avatar being targeted is decreased or he gains an unfair advantage in his attacks. Given that in P2P architectures nodes are in charge of disseminating their own updates as well as those of others, they are more vulnerable to these types of cheating. Speciﬁcally, nodes that are part of a forwarding pool or multicast tree can easily interrupt information dissemination. Several cheat types fall under this category:

‚ Escaping: The player intentionally terminates his connection in order to escape imminent loss in the game. Given higher failure rates of peers and relative lack of peer performance statistics in comparison to client-server systems, detection of escapes may be difficult. Detection techniques in this case may have overlap with availability monitoring techniques discussed in Section 2.8 [YR05]. ‚ Time cheating (look ahead cheating): The player deliberately delays the updates he sends to base his actions on those he receives from others. Another form of this cheating uses fake timestamps in the past, where a player, after receiving updates from others, sends his own update with an old timestamps evading detection. This is only possible in a P2P system. In a client-server system the server sends state updates to all players after executing all the updates from the previous frame [BLL07]. ‚ Network flooding: A player overflows the game server in order to create lags and disrupt game play. These attacks fall under Denial of Service (DoS) and Distributed Denial of Service (DDoS) attacks and are widely studied. Many firewalls provide methods for de-

56 2.9 Cheating

tection and dealing with DoS attacks. Given relative lack of security measures at peers in comparison to a server as well as limited resources available to peers, they are more vulnerable to network ﬂooding. ‚ Fast rate cheat: The cheater mimics a rate of game event generation that is faster than the real one, giving the player an unfair advantage in planning activities. Similar to escaping and look ahead cheat, it is harder for peers to prevent and detect this behavior [Fer08a]. ‚ Suppress-correct cheat: The cheater purposefully drops a number of consecutive updates and then sends an incorrect update that provides him with some advantage before being considered a dead peer. The incorrect update is harder to verify and dispute due to lack of recent information about the player. Given global knowledge of a server, it is in a better position to run an accurate validity test on delayed updates compared to normal peers [BLL07]. ‚ Replay cheat: A cheater resends signed and encrypted updates of a different player that he has previously received [CDST06]. For example, replaying a reduction in health can eventually lead to the death of the other player. In P2P systems, given that packets are forwarded by different peers, keeping track of the messages may become hard. In addition, forwarders are in a unique position to easily replay messages they forward. ‚ Blind Opponent: A cheater drops some updates to opponents, blinding them about the cheater’s actions, while still receiving updates from opponents [WSL07]. This is only applicable to P2P systems as servers are typically trusted to send correct state updates at correct intervals. Similarly, by dropping update and guidance packets a player can make replicas of other players being guided by the AI (as in doppelgangers) act dumb which can possibly give the player an advantage (Chapter 3).

Illegal game actions. A player can circumvent the game physical laws (e.g., limited velocity) and unduly change his state (e.g., increase his health or ammunitions) by tampering with the game code. A cheater can also unduly increase his score: in distributed FPS games, the players’ scores are generally updated by having the players claim they have killed some other player, which constitutes a trivial opportunity to cheat, without proper veriﬁcations. Players may also use third party software to perform game tasks such as aiming.

‚ Client-side code tampering: The player modiﬁes the client-side code to get an unfair

57 2.9 Cheating

advantage. This is considered abuse of authority, and P2P systems are known to be vulnerable to these tactics. An example can be speed-hacks where the player tampers with the code to increase his avatars’ movement speed. A number of such techniques are discussed in [FKS08]. ‚ Aimbots: The player uses an intelligent program to provide him with automatic weapon aiming. Aimbots are mostly used in FPS games. Both P2P and client-server are vulnerable to this attack, however, lack of global knowledge and statistics about players can make detection harder in P2P systems. ‚ Spooﬁng: The cheater sends messages pretending to be a different player. Given limited authentication capabilities of P2P systems, they are more vulnerable than client-server systems. ‚ Consistency Cheat: The cheater sends different updates to different players. This cheat can be used by a player or a group of players, and is only applicable to P2P systems.

Unauthorized information access. A player can exploit information available but not supposed to be disclosed (e.g., position of players behind walls) to increase his chances to kill other players or to foresee danger, thus, helping him evade. This can occur as players might receive state updates or guidance messages about players that are not currently in their vision range. For example, players receive excess information to be able to quickly access the information necessary about other players should they enter their vision range, as in prefetching (see Section 2.7), therefore, On Demand Loading (ODL) [LDMW04] techniques are rendered inefﬁcient. In addition, players may manage subscriptions to their avatars and, therefore, be aware of avatars chasing them or aiming at them even if they are behind their backs. While untampered code might not show the information on the screen, such information can often be easily extracted. Moreover, the expected future position contained in dead reckoning messages can be exploited by means of aim aides to increase shooting accuracy. One has to be aware that cheaters have access to information their colluding partners hold.

‚ Snifﬁng: Many tools, e.g. ShowEQ [Sho11], exist that allow players to log and access all kinds of information sent by the game across the network. This is a serious problem especially if the communication is not done over a secure (encrypted) connection. Given that P2P systems receive extra information for neighbor discovery, forwarding, etc. they

58 2.9 Cheating

are more prone to these types of attacks than client-server systems. ‚ Maphack: The player, by tampering the client code and libraries, is able to see through walls and obstacles, gaining an advantage. Many P2P systems, e.g. mutual notification systems, receive excess information about other nodes to perform neighbor discovery, and are more vulnerable. ‚ Rate analysis: Multi resolution techniques, where updates are sent to interested players more often than the ones that are not interested, are prone to cheating by using traffic analysis. By analyzing the update packets being sent and the frequency of the updates, the player can find other players that are paying attention to him and try to escape [HYK11].

In general, many of the scalability and consistency techniques that we describe in this section are susceptible to cheating. Dead-reckoning, for example, may be abused by users in suppress-correct cheat. Sending an update, after a number of dropped updates, causes the dead-reckoning to apply the new update directly and makes it harder to detect cheating, giving the cheater unfair advantage.

2.9.3 Cheating Prevention

Cheating in P2P architectures has received some research attention. However, typically techniques proposed address a limited number of cheat types that can occur in speciﬁc architectures and scenarios. It is partly due to the fact that different architecture are more prone to different types of cheating. Some techniques try to prevent cheating from happening, i.e., proactive mechanisms, while others detect cheating, i.e., reactive approaches, and punish cheaters.

Prevention techniques pro-actively reduce or eliminate the possibility of cheating by players. These techniques fall under cheat prevention and proactive approaches:

Cryptographic measures: Cryptographic measures, such as message encryption, signatures and checksums, are effective in eliminating message sniffing and illegal message modifications [MGM06, EKP`07]. Encryption ensures that forwarders do not have access to private data and cannot modify the messages. In order to protect against replay cheats, messages have to be uniquely identifiable. For example [PG11] use cryptographically secure hashes of cards to secure trading card games (TCGs) eliminating the need for a referee

59 2.9 Cheating

(later explained). Cryptography mechanisms as well as message digests have been used to address replay and consistency attacks as well as spooﬁng [CDST06, CHJ08].

Commitment & agreement protocols: Commitment & agreement protocols are another mechanism to prevent cheating, addressing a range of attacks. For instance, time cheating is addressed by the lockstep protocol [BLL07]. It requires all players to ﬁrst submit a hash code of their next actions and only after every player’s hash code has been received, players send their actions to each other. By comparing the hash code with the action, players can make sure that other players have not changed their actions after receiving input from others. This approach is quite similar to a two-phase commit protocol. Lockstep incurs very high latencies since players have to wait and receive all other players’ hash codes before committing their own actions. This delay might not be tolerable by some games. Asynchronous synchronization [BLL07] tries to alleviate this problem by only requiring players within the area of interest to wait for one another. New Event Ordering (NEO) [GZLM04] improves this further by limiting the maximum latency and using a quorum of players for deciding whether an update from a certain player has been received in time or not. This causes players who have slow connections to most players to not be able to play. The Secure Event Agreement (SEA) protocol [CDST06] improves NEO to address replay and consistency attacks as well as spooﬁng using changes in the cryptography mechanisms. EASES [CHJ08] improves SEA by computing a message digest before signing the message.

Many other infrastructures, with roots in agreement protocols, have also been devised. [CFFS05] addresses maphacks. In every frame, players exchange information about their viewable area. If they detect that they are in each others viewable range they send their updates in plain text, otherwise, they only send a hash of their update. That is, updates are only sent once a player has veriﬁed that he is in the other player’s view.

Other techniques: Several other techniques have been proposed to address a range of attacks. Here we present a few that address common cheating concerns such as aimbots. [GD05, SGJ07] propose using CAPTCHA texts to deal with aiming bots. CAPTCHA texts are an example of a Turing test. However distinguishing aiming bots from competent players based on the quality of their game play remains an open challenge. Hack-Proof Syn- chronization [Fun06] reduces the opportunity of cheating by using speed-hack in dead

60 2.9 Cheating reckoning. Instead of directly including and sending the position and angular velocity of the player, other information is included in the message that can be used for dead reckoning. [HYK11] prevents rate analysis cheats by using a proxy architecture where proxies forward messages and manage subscriptions of the interest sets of players. Proxies also hide dead reckoning information from players unless they are in the vision range. Proxies are frequently changed in a random but veriﬁable fashion in order to decrease the chances of collusion and other types of cheating. Another popular method to prevent cheating is by choosing a super-peer for an area in which the super-peer itself is not playing and therefore is not interested in cheating. However, this method does not address collusion. CPP/FPS [CM04b] uses a pulser as an active authorized unit to broadcast pulse messages with the frame-rate to synchronize the game pace.

2.9.4 Cheating Detection

Detection techniques are reactive and rely on the fact that if cheaters know that they will get caught and be punished, they will have less incentive to cheat. Punishment typically constitutes being banned from playing for a certain duration or indeﬁnitely.

Verification A common technique is that all actions are audited and verified for security breaches. For instance, a node that receives update messages from other nodes is able to verify them and check whether they are feasible according to the game physics or not and in essence act as a referee. Similarly, comparing hash messages of future updates with the actual updates is a form of verification that can detect cheaters [BLL07, GZLM04, CDST06]. Such techniques have been used in other areas of distributed computing as well. For instance, accountable virtual machines (AVMs) [HARD10a] execute binary software images in a virtualized copy of a computer system and can record non-repudiable information that allows auditors to check whether the software has behaved as intended or not.

Referee selection: The question now arises who can serve as referee. In coordinator- based architectures, since a centralized authority for each region exists, i.e., the super-peer, most security checks can be performed by this node (similar to a server), assuming that this node has a high trust level [IYM`06]. Some hybrid systems such as [LL08, CRd`04], use players as referees that record and compute state updates. Cheaters are then reported

61 2.9 Cheating to a trusted server. Referees can be chosen randomly to reduce the chances of collusion. In [WST09], the authors propose a secure referee selection scheme. Referee Anti-Cheat Scheme (RACS) [WSL07] is another hybrid approach that uses a secure server to validate all client updates to detect cheating.

Veriﬁcation of computation: In coordinator-based games, the peers taking on coordinator tasks might not be trustworthy. Mutual checking schemes [KTCB05] can be used by super-peers to ensure coordinators, given the same updates, reach the same state. Nodes periodically compare their states to detect cheaters. It can also be used to detect collusion between super-peers and normal peers. In hybrid systems such as DaCAP and FreeMMG [LL08, CRd`04] players are used as referees that record and compute state updates. Upon detection of cheating, the cheater is reported to a trusted server. In DaCAP monitors are chosen randomly to reduce the chances of collusion. RACS uses a secure server to receive, simulate and validate all client updates to detect cheating. Clients are allowed to communicate directly with each other, however they send an update to the server as the referee. Players that are detected as cheaters are only allowed to communicate through the referee. In [WST09], the authors propose mechanisms to secure referee selection schemes. Another hybrid approach is Invigilation Reputation Security (IRS) [GV08] in which communication is handled by the server and update execution is managed by peers. In this scheme the server assigns a proxy peer to each peer. Proxy peers are chosen randomly and are regularly re- assigned. This proxy peer is responsible for executing the peers’ updates and returning the results. Message relaying between peer and its proxy is done by the server. The server runs a quick auditing test to see if the returned result is possible according to the game rules or not. Conﬂicting updates and a percentage of updates randomly chosen are then re-executed by a selected monitor peer to verify the results and detect cheating. In most of these system, additional overhead is needed as update messages have to be sent to the chosen referees, in addition to the original target group.

When to verify: Another important question is when the veriﬁcations are performed. One possibility is to do so at the end of the gaming session. In this case little overhead is incurred during the game. However, this approach does not work in real time and cannot detect and ban cheaters during the gaming session. Real-time veriﬁcations are more effective but add overhead and latency to the game.

62 2.9 Cheating

Verification process: The verification process itself can be very game and task-specific: verifying whether movement speeds are accurate and actions are allowed given the characteristics and locations of players depends on the game itself. [Fer08a] proposes a statistical method for detecting fast rate cheating by measuring the latencies between packets. [LPBC07] relies on behavioral monitoring to detect cheaters without any knowledge of the vulnerabilities in the game. By collecting information and statistics about the player’s behavior through the game engine, anomalous behavior can be detected. In addition, [CPC08] proposes AI Bot detection by using a manifold learning approach. Two general classifica- tion techniques, kNN and support vector machines (SVM), are used to differentiate AI and human behavior.

Other detection mechanisms Apart from veriﬁcation through referees, other detection mechanisms are possible. For instance, AC/DC [FR06] proposes a method to detect looka- head cheating in which a player that is suspicious of another player delays sending his action for a certain amount of time to see whether the suspected player waits for his action or not. In addition, several tools exist to detect tampering with the game code or to detect cheating processes running in the memory. PunkBuster [Pun11] and Valve Anti-Cheat (VAC) [VAC13] are examples of such systems. Closed and monitored environments such as XBox Live [XBo11] and PlayStation Network [PSN11] manage code tampering with regular updates to the gaming consoles. However, these tools can only detect a limited set of hacks and cheating tools, and always need to be updated to address a new cheating tool. In addition, simple methods such as occasional random aiming when using aimbots can result in not detecting a cheater [WS07]. Randomly Created Checksum Algorithms (RC- CAs) [MGM06] are used as an enforcement algorithm, by making its results necessary to gain access to the game-data. RCCAs are deterministic checksum algorithms that are randomly created. Moreover, Mobile Guards are used to ensure the integrity of the protection mechanism. Mobile Guards are downloaded codes that contain the protection mechanisms and are not statically embedded into the game-client. [FKS08] proposes use of hardware- based, stealth measurements such as Intel’s Active Management Technology (AMT)10 that has been used in rootkit detection in operating systems. In this approach, a tamper-resistant

10Intel vPro Technology http://www.intel.com/content/www/us/en/architecture-and-technology/vpro/vpro -technology-general.html

63 2.9 Cheating processor resident on a client and isolated from the system’s primary processor is used to perform randomly-timed measurements underneath the host’s software stack to detect cheats. Since the measurements are hardware based, they are more difﬁcult to circumvent than if the checks were done by a software-based system.

The design of many cheat detection techniques is heavily dependent on the underlying P2P architecture and none of the techniques studied can address all possible cheating types. Often, a combination of these techniques has to be used.

Finally, once cheaters are detected and banned, one has to assure that they cannot re- enter the game under a new alias. This is a question of authentication and persistence. It can be best handled, if users have to join a game by going through a central authentication service.

2.9.5 Reputation Systems & Penalization

While prevention techniques are self sufﬁcient to address cheating, detection techniques rely on punishments to deter cheaters. As mentioned, thousands of players’ accounts have been banned by the game developers such as Blizzard after running cheat detection algorithms [Goo08, EST08]. Unless the detection technique is deterministic, a reputation mechanism is required to ﬁnd cheaters over time.

Reputation Systems Particularly in architectures where peers are in charge of the veriﬁ- cation and cheat detection, some form of a reputation system is required. Given that a single case of cheat detection may not always be conclusive evidence that a player is cheating, i.e. it could be a false positive, it can not be a basis for penalization. For example, a player may truly experience network problems and only send an update before being considered dead, similar to a suppress-correct cheat. In such a case, one needs to keep record of players over time to be able to differentiate cheaters from honest nodes facing connectivity issues. Only after the number of detections passes a threshold it can be said with high probability that a player is a cheater. This threshold can be calculated based on how likely it is for normal behavior to be mistaken as cheating.

Several general purpose distributed reputation systems have been proposed. These sys-

64 2.9 Cheating tems are in essence similar to the eBay [eBa11] rating system where buyers and sellers are able to rate each other. The overall reputation of a participant is the overall sum of these rating in the last six months. EigenTrust [KSGM03] is an example of a popular distributed mutual rating mechanisms designed for file sharing systems. Similar gaming reputation systems also exist. PlayerRating [KF09] and REPS [HHJ08a, HHJ08b] are distributed game rating management systems. These reputation systems use the same basic rules: after each interaction, participants in the interaction rate each other and store the rating. Queries sent to friends and certain groups of players as well as some random peers allow these rating to be collected and aggregated for different players, giving them an overall reputation. Reputation systems themselves are vulnerable to several attacks such the Sybil attacks [Dou02] where the attackers, by creating a large number of pseudonymous entities, gain a disproportionately large influence in the ratings. Centralized system deal with this by requiring verifications through phone numbers, credit cards, or other forms of identification. This can possibly be used by trusted peers as well. Badmouthing and collusion between players can also subvert the reputation system which has been studied [SZA08, SAZ10].

Other approaches such as proactive reputation [SZA08] and [SAZ10] provide reputation systems that do not rely on third party ratings and provide measures to evaluate reputation of peers directly. In proactive reputation, transactions are anonymized and can not be distinguished from normal requests. This prevents peers from trying to boost their reputation. In [WK07] authors propose a trust management system without the need for a reputation system given trusted super-peers. In addition, commercial third party player ranking solutions, e.g. GameRank [Gam11], have been proposed that maintain a score for players by analyzing their activities in the game. Hybrid architectures, e.g. [LL08, CRd`04, Qua11], can use the server for authentications and the subsequent punishments of the players.

Penalization Penalization typically happens in the form of expelling players from their current games and/or banning them from accessing future sessions. In super-peer based architectures, the coordinator, after detection of cheating by itself or other nodes, can ban the player from the game by stopping update dissemination to this node and ignoring its updates. However, measures have to be taken to ensure that a coordinator does not abuse this

65 2.10 Commercial Adoption power [IYM`06, KTCB05]. In mutual notification systems neighboring nodes have to all participate in banning a player from the game. In addition, cheaters have to be blacklisted (e.g., by distributing and storing their identifying information in the persistent storage) by peers to prevent them from reentering the game. However, given the relative ease of changing aliases, this can be difficult. In hybrid systems, players may have to go through the central server for authentication and can then join games. As a result, cheaters can be easily kicked out of a game or be banned by the server for a certain duration or indefinitely. Given a server’s better authentication and persistence capabilities, it’s harder for cheaters to leave and join the game under a new alias.

2.10 Commercial Adoption

Generally, P2P approaches are not widely adopted in commercial systems. In this section, we discuss some existing engines that use P2P solutions or that we believe can be extended for P2P support. Then we discuss why industry has not yet widely adopted the P2P model and what would be possible options for a better market penetration. This is followed by a discussion of how users could be given incentives to contribute their resources to P2P systems. Finally, we discuss how the P2P solutions for MOGs presented here could be applied to other applications.

2.10.1 Middlewares

In most cases, game functionalities such as replication management, interest management, and update dissemination are implemented in a game engine that presents a middleware [Big11, Hen04, DKDV08] to the programmer. It hides the complexities of the underlying architecture while providing an adequate programming interface to the game designers. Many different games can be implemented using the same middleware.

2.10.2 Industry Solutions

Many network engines or SDKs that include a network engine, have been proposed for multiplayer gaming (mostly centralized) which provide easy to use APIs that have built-in support for many of the common functionalities required for gaming. These functional-

66 2.10 Commercial Adoption ities include: reliable UDP streams, delta coding, NAT hole punching, dead reckoning, object synchronization, and others. Examples of such network engines are: Zoidcom11, ClanLib12, NLNet13, RakNet14, OpenTNL15, and OpenSkies16. As another example, Big- World 17 provides a middleware that supports shards as well as zoning of worlds utilizing a dynamic real-time load-balancing system to remove zone limitations. In addition, similar to the multi-resolution schemes mentioned before, see Section 2.6, it provides Level of De- tail (LOD) and data prioritization that is applied to all object property changes and events to make sure that only the most important data is sent to the client.

While most of these network engines are designed and optimized for the client-server model, many of the functionalities, such as object synchronization and reliable UDP streams, can be used in a P2P engine as well. NAT hole punching, in particular, is of utmost importance in P2P networks since it is essential to be able to directly connect players behind NATs to each other. General-purpose APIs, e.g., to create sessions, can be used in P2P networks but they lack necessary optimizations to address P2P issues and do not beneﬁt from P2P network strengths.

RakNet with additional plug-ins provides support to connect all peers automatically. ReplicaNet18 allows for creating network sessions and session migrations for both client- server and P2P network topologies. ludiloom19 uses a hybrid P2P architecture for efﬁ- cient game content distribution and high availability. OpenSkies implements an MMO networking and simulation architecture to improve the real time massively multiplayer capabilities of a High Level Architecture (HLA), the IEEE standard [IEE00] framework that supports modeling and simulation. It provides both a P2P implementation for small-scale multiplayer networking as well as a peer-to-lobby manager/router network implementation (lobby is explained further below). Furthermore, it allows clients to communicate with each other via external servers (federation hosts). Quazal Net-Z and Rendez-Vous [Qua11]

11zoidcom.com 12clanlib.org 13opennel.org 14raknet.com 15opentnl.org 16openskies.net 17bigworldtech.com 18replicanet.com 19ludiloom.com

67 2.10 Commercial Adoption provide an optimized routing system for P2P connectivity as well as a game lobby service to provide matchmaking services between players. Game companies can use the lobby to ensure players pay to gain access to games and to authenticate players. The lobby provides access to different game sessions for players. The game sessions themselves are managed by P2P communication between clients, minimizing costs for the game companies. Bad- umna Network Suite20 provides the necessary APIs for a scalable service for game state synchronization and object replication by using a decentralized network according to the designs in [KDC10].

2.10.3 Industry Models

As mentioned, industry adoption of P2P networks has been limited. More than the technical difﬁculties as discussed so far in this section, the challenge remains in the problem of maintaining control over the game and dealing with cheating. Therefore, hybrid architectures are attractive since they allow to maintain control and can exploit the beneﬁts of P2P architectures. We discuss some of the more attractive options to companies:

‚ Game lobby model: In this model the game company provides a matchmaking service that allows players to meet and play together. By enforcing all players to go through the lobby, the game lobby can ensure that the players meet the conditions of the game company, have paid subscription fees, and are authenticated and authorized to access the game sessions. In addition, it provides a means for punishing cheaters and preventing them from gaining access to games. This model minimizes the costs as providing a game lobby is not expensive and the games are mostly managed by the clients. Quazal provides a sample architecture for such a system. This model is currently mostly used for relatively short-lived games. ‚ Federated servers model: Many of the architectures discussed in this section use a hybrid approach with super-peers where some high level tasks can be delegated to the federated servers, while other tasks are performed by peers. These servers may belong to the same company or not. The distributed design may also be exploited by publishers to beneﬁt from third party federated servers. For example, the Badumna network can provide such a solution. Similarly, content distribution networks (CDN) such as Aka-

20scalify.com

68 2.10 Commercial Adoption

mai and Limelight and hybrid cloud and P2P based games can also beneﬁt from these architectures [CCR10]. Similarly, edge servers [GDN`03] (where inter-server communication is managed in a P2P fashion) can be used by CDNs and live game streaming companies such as Gaikai and Onlive [Gai11, OnL11] to maintain game state close to the geographically distributed clients to reduce the latency or to provide 3D streaming services [CHJC10, HJC10, WHT09]. Moreover, in-browser gaming that typically relies on HTML5, web players, is becoming more popular [Big11] making it suitable for 3D streaming. Similarly, using these technologies, the game can be executed in the cloud and players can use their PC, console, or browser as a terminal connected to the cloud [NK10]. ‚ Free model: Players are in charge of maintaining the game state and playing is free of charge for players. It can be used by the open-source community to create free games. It can also be attractive to companies to provide low cost free-to-play (F2P) games, relying on selling in-game items for proﬁt that has proven to be a very successful model21 22. Given the low cost of maintenance it can reduce the risk of investment. In addition, existing P2P architectures that already support apps, e.g. Skype23 and µTorrent24, can use games to make their platforms more attractive. They can use their already existing and powerful P2P network to support games, however, they need to adapt the game architecture to the network.

2.10.4 Client Incentives

Here we present several reasons why the P2P architectures can be attractive to consumers and to incentivize them to contribute resources to the system. Clients have several incentives to use a P2P game environment:

‚ Better game environments: P2P architectures have the potential to create more engaging game environments by removing many of the limiting factors in the game design. For

21e 1000 Virtual Item For Game Raises e 2 Million In 4 Days For Bigpoint: http://techcrunch.com/2011/11/23/e1000-virtual-item-for-game-raises-e2-million-in-4-days-for-bigpoint/ 224 in 10 playing freemium games make in-game payment to extend or enhance the game https://www.npd.com/wps/portal/npd/us/news/pressreleases/pr_120423a 23skype.com 24utorrent.com

69 2.10 Commercial Adoption

example, higher number of players can be allowed in a single region of the game leading to more engaging gameplay. ‚ Lower fees: Reducing costs can allow game developers to reduce the subscription fees, therefore, reaching a wider costumer base. In addition, schemes that adjust the fees according to the resources players can contribute to the game are possible. This is in particular attractive for free-to-play MOGs that are supported by advertising and purchases of in-game items.

In order to ensure that players contribute resources to the game distributed accounting mechanisms can be used. Many Tit-for-Tat (TfT) mechanisms, first proposed for file- sharing systems [Coh03], have been proposed for P2P systems that force nodes to contribute to the system in exchange for using the system [LCM`08]. Free rider detection mechanisms, e.g. [GHK`10], can detect nodes that are mostly consumers and do not contribute to the system. Unlike file sharing systems, creating incentives and punishments are easier in games. Distributed accounting systems such as [GJA03] exist that use a credit/debit system that tracks nodes’ activities. Nodes are billed according to their use of resources, and are credited based on the time and the quality of resources they have provided. By implementing such systems, free riders can be punished through mechanisms such as limited access to some parts of the game world, limiting the gameplay duration, and limited access to game items. Contributors, or players who actually pay for credits, can receive better reputation, experience point, or gain access to exclusive items. However, using these methods, one must ensure the gameplay is not too unfair to players with limited resources available by finding a suitable balance based on the game.

2.10.5 Other applications

In addition to games, other applications may also beneﬁt from the P2P techniques presented in this section. Note that the discussions also apply to 3D virtual environments other than games25. We ignore other applications of P2P networks such as delivering game software updates.

‚ Mobile Multiplayer Games. The emergence of mobile platforms such as smart phones,

25For a list see: http://www.virtualworldsreview.com

70 2.10 Commercial Adoption

Sony PSP, and Nintendo DS that integrate various wireless network capabilities, enables distributed multiplayer gaming on these platforms. However, the limitations in connectivity, latency, memory and processing power make it unlikely that mobile platforms will become powerful peers in a massively multiplayer game. In addition, special mechanisms are needed to handle higher fault and increase latency tolerance. In particular, inconsistencies due to message delays and loss should be hidden and eventual consistency must be achieved without requiring large computational costs at the mobile platform [CF05, KK07, WKV`09, HHLH10]. Peer-to-peer mechanisms can be attractive to avoid costly communication among the mobile unit and an external powerful server when players are physically close. Mobile devices can use IEEE 802.15 or similar low power communication interfaces and create ad-hoc networks to perform some of the communication locally between players in a P2P fashion. ‚ Social Games. A new emerging area of games are social games. These games are typically built on social network platforms. Many of them are multiplayer in nature and as millions of players simultaneously play these games, they can be considered a form of massively multiplayer online game. However, currently these games typically lack a graphically intensive and immersive game play and are different in nature than the games studied in this section. These games may be asynchronous (turn-based) and typically rely on social interactions between players, such as exchange of gifts and virtual goods, as one of the main game functionalities. These games rely on the social platforms for their operations. Architectures have been proposed to enhance the social networks built around MOGs. CAMEO [ILc10] provides an architecture for continuous analysis for MOGs using cloud resources, e.g., Amazon EC2, to increase the revenue from advertisements or selling virtual goods and services in the social network communities. Second Life can be considered the largest immersive social network environment. Re- search has shown that avatars in Second Life tend to interact similarly to human beings in real life. They meet, spend time together and make friends. This behavior suggests that avatars construct an online social network that represents the social relationships existing among human beings [VV10]. As demonstrated in [VDB09] P2P systems can be used in such systems. In addition, in such systems closeness is not only measured as the physical distance in the virtual world but also considers how close two players are in the social graph, or how many interactions they had in the past. P2P architectures can exploit this

71 2.11 Survey Conclusions

closeness measure as a factor for task distribution and cheat detection and prevention. Integration of social games and traditional MOGs [VDB09] can further make the use of P2P architectures possible.

2.11 Survey Conclusions

Massively multiplayer online games are a thriving business industry and they introduce a range of interesting challenges. Most MOGs follow a client-server architecture but there has been increasing interest in designing P2P or hybrid architectures for games. P2P architectures reduce the cost of maintaining expensive servers and improve the scalability of games.

P2P architectures are used in state distribution as well as update dissemination to peers. These architectures can be divided into three categories: structured, unstructured, and hybrid. While P2P architectures can provide higher scalability and lower cost they have to address many challenges such as limited bandwidth, or vulnerability to cheating. Many techniques have been proposed to solve these challenges, but many topics such as availability, persistence, and cheating remain to be fully addressed. In addition, while some of these techniques can be applied to different architectures, many are dependent on the underlying architecture. In this chapter, we do not present all available architectures, however, we believe each type of architecture is studied with sufﬁcient related work to provide a comprehensive study of different techniques using a uniform terminology.

72 Part II

Cheat-Resistant Support For Multiplayer Online Games

73 3 Watchmen: A Scalable Cheat-Resistant Peer-to-Peer Architecture For Fast-Paced Multiplayer Online Games

As we discussed in the previous chapter, multiplayer online games are inherently distributed applications, and a wide range of distributed architectures have been proposed. However, only few successful commercial systems follow such approaches, even given their beneﬁts, due to one main hurdle: the easiness with which cheaters can disrupt the game state computation and dissemination, perform illegal actions, or unduly gain access to sensitive information. The challenge is that any measures used to address cheating must meet the heavy scalability and tight latency requirements of fast paced games.

We propose Watchmen, the first distributed scalable protocol designed with cheat detection and prevention in mind that supports fast paced games. It is based on a randomized dynamic proxy scheme for both the dissemination and verification of actions. Furthermore, Watchmen reduces the information exposed to players close to the minimum required to render the game. We build our proof-of-concept prototype on top of Quake III. We show that Watchmen, while scaling to hundreds of players and meeting the tight latency requirements of first person shooter games, is able to significantly reduce opportunities to cheat, even in the presence of collusion.

74 3.1 Motivation for Watchmen

3.1 Motivation for Watchmen

While massively multiplayer games such as World of Warcraft are the best known genre of large-scale gaming environments, fast-paced multiplayer games, mostly First-Person Shooter (FPS) games such as Quake and Halo, have been popular for decades. Given that we already covered the necessary background in the previous chapters, here we brieﬂy repeat, summarize and detail those parts of the background that are needed to understand the remainder of this chapter and motivate the design choices we make.

In the traditional setting of fast-paced multiplayer games, a small number of players (e.g, up to 16) play relatively short instances of the game on a local area network following a client/server architecture, where one of the player machines hosts the server, which controls all game actions and communication. In such a case, the players are typically friends and thus, cheating is not a critical issue. However, this genre of games has evolved quickly over the years. Popular game networks (e.g., XBox Live, PSN) and the concept of game lobbies [Qua11, uLo12] allow players across the world to connect and participate together in such games. Thus, there is no more a homogeneous local area network that would connect the players, and the trust and solidarity that used to exist among players has vanished. Furthermore, as it has become so easy to ﬁnd a large set of players that want to play together at the same time, the desire arises to develop a new generation of this genre where hundreds of players can play within the same game instance. Therefore, hosting the server on one of the client machines has become infeasible: such a client would have many powerful options for cheating and the client machines are unlikely to handle the game server load in terms of processing power and upload bandwidth.

One approach to support such games would be to transfer the game server into the cloud. This is cheaper for the gaming company than hosting large and expensive custom built server farms themselves, because the system can scale dynamically with the number of isolated concurrent game instances. However, several problems arise: (1) the costs are still non-negligible, (2) individual cloud instances may not scale to hundreds of players in the same game world, thus, requiring distributed solutions that incur higher costs and complexity, (3) these solutions do not support spontaneous, short-lived, and locally run gaming sessions particularly by third parties or gamenets. Thus, the question arises why

75 3.1 Motivation for Watchmen not keep the game execution with the clients and develop a decentralized solution.

Over the last years, several decentralized solutions have been developed (e.g. [VDB09, BPS06, BDL`08]). They distribute the load among the players, and gain scalability by introducing new resources with every joining player. However, as discussed in the previous chapter, many of them ignore cheating, which is a serious threat as players have access to and can manipulate sensitive game data [FTT10]. The decentralized approaches that address cheating have typically been developed for strategy games (e.g., lockstep [BLL07]) and are usually not efﬁcient enough for fast-paced games.

As outlined in the previous chapter, cheating essentially consists of gaining an unfair advantage and comes mostly in the following three forms: disrupting the game state computation and dissemination, performing illegal actions, and gaining access to sensitive information [WS07]. In server-based game engines, cheat detection and prevention can be achieved by making the server verify the players’ actions, ensure synchronization, and reduce the information sent to players to the minimal amount required to render the game world [BLL07]. Nevertheless, even server-based systems are vulnerable to cheats, particularly when a player becomes the server. In decentralized games, detecting cheating is even more difﬁcult and natural trade-offs between responsiveness, scalability, veriﬁcation and information disclosure have to be made. One has to be aware that while security is important in games, it often comes second to performance. While millions of players currently play games that provide no or low security measures1,2, only a few would play a very secure but low-performance game.

Motivated by these developments, this chapter presents Watchmen, a distributed, scalable, and cheat-resistant architecture designed for FPS games. Our goal is to design a platform that is reasonably secure and able to handle fast paced FPS games. Speciﬁcally, we aim to (1) provide a low-latency distributed infrastructure; (2) not rely on a central server or trusted third parties but be able to take advantage of them should they be present; (3) limit the opportunities for cheating even in the presence of collusion; (4) detect cheaters during game play. To achieve this Watchmen uses a combination of both novel and well-

1Bungie resets credits of 15,000 Halo Reach players: http://www.bungie.net/Forums/ posts.aspx?postID=49997802 2Blizzard bans 5,000 cheaters: http://us.battle.net/sc2/en/blog/882508

76 3.2 Watchmen Background established techniques:

‚ Randomization: player connections are randomized and frequently changed to avoid unfairness, collusion, and other forms of cheating. ‚ Cross veriﬁcation: requests and updates are cross veriﬁed by players to ensure validity and correctness. ‚ Information hiding: information available to players is kept close to the minimum required to render the game. ‚ Multi resolution: players request and receive updates at different rates depending on the current situation.

A major challenge addressed by Watchmen is finding the right balance between the following contradictory goals: meet the tight scalability and latency constraints, provide enough information to players about each other to allow efficient mutual verifications, and at the same time limit this information to the minimum to limit cheating opportunities.

We chose the popular open-source FPS game Quake III to implement and evaluate our system, and have extended the platform to support up to 48 players in a decentralized setting. The reason for using Quake III is the game’s popularity and the extensive past research conducted on it, which makes it easier to determine the base game requirements and to compare our results. We show that Watchmen scales far beyond the standard number of players supported by Quake. We also show that opportunities to cheat are signiﬁcantly reduced, even in the presence of collusion (often ignored), while keeping good performance with respect to scalability and responsiveness. Although the thesis focuses on ﬁrst person shooter games, we believe that the concepts and mechanisms used in Watchmen can be applied to a broad range of games including role playing and real-time strategy games, as they have lower networking requirements [BPS06].

3.2 Watchmen Background

Here we brieﬂy go over the necessary background that motivates our design choices:

Latency Requirements: FPS games have tight latency requirements, and lag has an adverse effect on the game-play. In Quake III and similar games, latencies and lags of up

77 3.2 Watchmen Background to 150 ms are tolerable [Arm03, BCL`04, AJZ06, LABP08]. To meet these requirements, games typically limit the geographic location of players to the same country or continent [Cla08, uLo12]. In addition, games rely on UDP for faster communication.

Decentralization: In a centralized game, players send all their updates to the server that controls all game entities. The server can verify each update and sends it, given its global knowledge, to only those players who need it (if designed well). In a decentralized system, players typically control their own avatars. Bots and game items can be distributed across the players as well or be controlled by the company’s servers. In small to medium sized games (e.g., FPS games such as Quake III), players are usually aware of all entities of the game. In large-scale RPGs (e.g., World of Warcraft) decentralization is achieved by dividing the game world into zones as discussed in the previous chapter.

Bandwidth Requirements: Most broadband connections are asymmetric, with upload bandwidth being the limitation. In a distributed scenario where players all send updates to each other, the growth in overall bandwidth requirements is quadratic. Average bandwidth requirements in centralized Quake III is 12n kbps where n is the number of players [BDL`08].

3.2.1 Interest Filtering in Quake III

Interest filtering is designed to limit the updates a player sends; for example, only to entities inside a fixed-radius sphere around the avatar. In Quake III, this is done via potentially visible sets (PVS) that determine which players are visible and hence should receive an update. However, using a fixed geographical area as AOI does not necessarily bound the actual number of players inside the area. As we show in Figure 3.1, players have an exponential presence in some area of the game, due to their strategic location or presence of important game items, rendering AOI filtering unusable (color intensity is normalized logarithmic values of presence in each region). NPCs, i.e., bots, further worsen the situation as they tend to use predetermined paths and locations.

Variable object ﬁdelity: To address this issue, as discussed, games can use multi-resolution schemes where updates are sent at different rates to players. Donnybrook [BDL`08], designed for FPS games, instead uses the set of the top 5 avatars with respect to an attention

78 3.3 The Watchmen Architecture

Health packs Ammunitions Weapons Armors Respawn spots

(a) Human movements (b) NPC Movements Figure 3.1: Heatmap of player positions in a Quake III deathmatch game in the q3dm17 map. Darker colors show higher presence in a region. metric based on proximity, aim and interaction recency, called interest set (IS). A player typically receives frequent updates only about avatars in his IS and infrequent so-called dead-reckoning updates (see below) about other avatars which contain information to simulate the expected motion of the avatar for several frames.

Use of Dead reckoning: Players that do not send frequent position updates rely on infrequent dead-reckoning messages containing the avatar’s expected next position and aim (computed locally) and its current position, aim, rate of ﬁre, etc. We will discuss how dead reckoning can be greatly enhanced using our AntReckoning technique. AntReckoning is discussed in Chapter 5. For the purpose of Watchmen, however, the particular details of the dead-reckoning mechanism are not relevant, and various dead-reckoning mechanisms could be used.

3.3 The Watchmen Architecture

Watchmen aims to provide security measures while being able to handle large-scale fast- paced games. The goal is to substantially reduce the potential for cheating. In general, cheating can be roughly divided into three categories: (1) Disruption of information ﬂow

79 3.3 The Watchmen Architecture covers actions that stop or change the normal pace of information flow (e.g., dropping messages), (2) Invalid updates cover actions that are invalid according to game rules (e.g., too fast moves), repetitions, or spoofing, (3) Unauthorized access includes any action (e.g., sniffing) that enables access to unauthorized information. In Table 3.1, we describe a range of cheating types within each category (for details see Chapter 2).

As we discuss our architecture, we describe how Watchmen detects or prevents these cheat types. Note that there is a range of cheat detection and prevention techniques that can be applied to any game architecture including Watchmen (e.g., code tampering detection software or hardware, aimbot detection tools, CAPTCHAs, tamper-resistant logging). In our discussion, we focus on security mechanisms that are based on the speciﬁc architecture of Watchmen. Moreover, while our focus is on gaming, techniques presented can be used in fault detection and dealing with selﬁsh and malicious nodes in other similar distributed systems.

Vision – Spherical cone Vision ﬁeld Interest – Top-5 in vision Other Vision ﬁeld (ext.)

Figure 3.2: Subscription-types and corresponding areas: Vision set (VS) is composed of avatars inside a ﬁxed-radius (˘60 degrees) and angle spherical cone directed along the player’s aim; Interest set (IS) is composed of the 5 avatars inside VS which catch the player’s attention the most.

Watchmen achieves its goal through three main techniques (we justify our choices in the security analysis): (1) with vision-based information ﬁltering (Figure 3.2), players receive updates at different rates and containing different information depending on the players’ location in the game and his surroundings. To do this, players subscribe only to the information they should receive. (2) Proxy-based indirect communication (Figure 3.3), where each player is periodically assigned a new random proxy responsible for a player’s up-

80 3.3 The Watchmen Architecture date dissemination and subscription messages. (3) Mutual veriﬁcation verifying players’ updates, actions, and subscriptions.

Table 3.1: Popular cheating mechanisms in distributed multiplayer games Type Name Description Watchmen Escaping Terminating the connection in order to es- Detected by proxy and others cape imminent loss Disruption of Time cheating (look Delaying the updates to base one’s actions Detected by proxy and others information ahead) on those received from others flow Network flooding Overflowing the game server to create lags Prevented through distribution and disrupt game play Fast rate cheat Mimicking a rate of game event generation Detected by proxy and others that is faster than the real one Suppress-correct Dropping consecutive updates, then send- Detected by proxy and others cheat ing an invalid update afterwards Replay cheat Resend signed & encrypted updates of a Prevented/Detected by proxy and others different player Blind opponent Dropping updates to opponents, blinding Detected by proxy and others them about the cheater’s actions Client-side code tam- Modifying the client-side code to get an un- Detected by sanity checks & action repeti- Invalid pering fair advantage tion updates Aimbots Using an intelligent program to provide it Detection by proxy (statistical analysis) with automatic weapon aiming Spoofing Sending messages, pretending to be a dif- Detected by players ferent player Consistency cheat Sending different updates to different play- Prevented by proxy and others ers Sniffing Logging & accessing different information Prevented by minimizing information expo- Unauthorized sent across the network sure access Maphack Hacking to see through walls and obstacles Prevented by minimizing information exposure Rate Analysis Analyzing the updates rates to detect play- Prevented by proxy and subscription model ers attention and escape

3.3.1 Subscription Model

Watchmen uses the notion of interest. Players receive updates at different rates and containing different information according to the details required to render the game (their interests). This concept has been successfully used in game engines and has been shown to greatly increase scalability [BDL`08] while maintaining game experience. However, in our context it helps us to reduce the chances for unauthorized access to sensitive information as we limit the information available to a player as much as possible.

The technique works as follows: each player partitions the other players in the game into 3 different sets as shown in Figure 3.2. The player sends subscription requests to the other players according to what set they belong to and receive 3 types of updates according

81 3.3 The Watchmen Architecture to their subscription:

Vision Set (VS): this set contains all the avatars visible to the player. In FPS games, it corresponds to a spherical cone, defined by a given radius and angle (e.g., ˘60 degrees in Quake III) and centered on the avatar. In practice, the cone is made slightly larger than the actual avatar’s vision field (see Figure 3.2) in order to handle fast avatar movement (typically rapid spin). Furthermore, beyond the avatar’s vision field, the vision set takes into account the features of the game world, known by all players. For instance, the avatars that are in a player’s vision range, but behind a wall do not appear in his vision set. This greatly reduces information available to players for map-hacks and sniffing.

Information Dissemination: The player receives infrequent (one per second in our implementation) dead reckoning messages (guidance messages) from players in its VS, which contain information such as the current velocity, future position predictions, and AI guidance instructions that enable the player to simulate the avatar’s near-future actions. We have described how accuracy of the predictions can be greatly improved using AntReckoning in Chapter 5. This allows for lower frequency updates as well as better quality of experience for the players.

Interest Set (IS): it is composed of visible avatars that catch the player’s attention the most (measured by a combination of proximity, aim and interaction recency). These are the avatars the player is most likely to interact with, therefore, requiring detailed information. Given the limited attention span of human players [BDL`08], the size of the IS can be ﬁxed (e.g., 5). Only avatars in a player’s vision set are considered as candidates, thus preventing the player to obtain frequent and accurate information about avatars he cannot see.

Information Dissemination: the player receives frequent (i.e., every 50 ms) state updates from players in the IS, including the avatars position, aim, ammunition, weapons, health, etc. Avatars in a player’s interest set are automatically removed from its vision set to prevent him from receiving information about future actions he could exploit to cheat, as he is likely to interact with these avatars in a near-future.

Others: any player outside the VS and IS belongs to the others set. While players do not actually need information about the players outside their vision sets for rendering the game world, they still need a minimum amount of information to determine whether a given

82 3.3 The Watchmen Architecture player does or does not enter their vision sets.

Information Dissemination: to allow players to decide on the type of subscription they need, they receive infrequent (i.e., typically every second) partial state updates containing only the position of the avatars, sufﬁcient to determine the subscription type. This subscription type is assigned by default and thus it does not require explicit subscription.

3.3.2 Proxy architecture

Typical to most games, our solution assumes that the game runs in a discrete event-loop, dividing time into ﬁxed-length frames. At any frame, a player has a single designated proxy (another player), responsible for managing subscriptions, forwarding messages, and for verifying the player’s actions. As such, the proxy has tasks similar to a server.

Proxy assignment is done in a random, but veriﬁable way in order to prevent collusion from happening. To achieve this, Watchmen utilizes pseudo-random number generators: each player maintains a pseudo-random number generator for each player, including himself, initialized with the player’s id and a common seed. This means each player can determine both its own proxy and the other players’ proxies, in any given frame, without the need for communication. All players act as proxies for other players (unless performance or the security mechanism by design favor some nodes, see Section 3.6). Moreover, there is no need for explicit subscription by the players to their proxies as both parties are aware of their assignment to each other. Proxies are renewed every couple of seconds. At the time of this renewal, old and new proxies go through a hand-off mechanism that exchanges player state information for veriﬁcation purposes.

Figure 3.3 shows how messages ﬂow through the proxy architecture. Basically, all messages sent by the player are sent to his proxy: subscriptions, guidance messages, frequent state updates and infrequent position updates. A player’s proxy relays subscriptions to the proxy of the target player to which the player wants to subscribe. That proxy has a list of all players that are interested in its player, and the player itself does not know who it interested in him: a subscription ﬂows from the player to his proxy and from there to the proxy of the target player. In contrast, the proxy sends update messages directly to the players that have subscribed, no redirection through their proxy is needed. That is, an update message

83 3.4 Security Aspects of the Proxy Architecture

p1’s proxy veriﬁes and forwards p1’s subscriptions Vision (VS) VS-sub. p1 ···

state up., pos. up., dead reckoning Other dead reckoning VS-sub. from p pos. up. p2 ··· IS-sub. from player p ’s proxy state up. publishes updates manages subscriptions p 3 Interest (IS) through its proxy to p veriﬁes and forwards IS-sub. p3 ··· p’s updates p 3’s proxy players veriﬁes and forwards subscribe through p3’s subscriptions their proxies

Figure 3.3: Proxies (1) manage incoming subscriptions and (2) verify and forward outgoing subscriptions and updates. Players p1 and p3 respectively IS-subscribe and VS-subscribe to p by sending a subscription message to p’s proxy through their own proxies. By default, p2 receives infrequent position updates.

ﬂows from the player to his proxy and then copies are sent to the players in the proxy’s subscription list

3.4 Security Aspects of the Proxy Architecture

While it is quite straightforward to see how the subscription model enables the prevention of unauthorized access, it is less clear how the proxy architecture contributes to cheat detection and prevention. In fact, our proxy architecture has been built from ground up with the goals of (1) preventing as many security flaws as possible, (2) provide sufficient means for verification of player’s information and action, and (3) meeting performance requirements. Here we list some of the main architectural security advantages of the proxy architecture:

Verification Potential: The proxy is in a unique position to verify subscriptions and updates. It enables the detection of flow cheats such as escaping, look ahead, fast-rate, suppress- correct, replay, and blind opponents cheats, since proxies know exactly the required updates and the correct rate. Verification is discussed in more detail in the next section.

84 3.4 Security Aspects of the Proxy Architecture

Consistency Cheat Avoidance: As a player only sends updates to the proxy who relays them, players cannot perform consistency cheats, i.e., send different updates to different players. If a player sends direct updates, it is detected.

Secured Subscriptions: Multi-resolution schemes are necessary for achieving the desired scalability. However, these approaches create an easy and yet very important opportunity for cheating: by sending subscriptions for high frequency updates, a cheating player is directly informed of interested players that are likely to target him. This information can be easily extracted, even without tampering with the game code, by using a simple network rate analysis tool during the game play. Watchmen’s proxy scheme ensures that players are not informed about subscriptions to them.

Random, Verifiable, & Dynamic Proxies: In all distributed gaming architectures, part or all of the players’ traffic goes through other players. This can be easily exploited by cheaters. In Watchmen, proxies are random meaning have no control over who they are a proxy for or who is their proxy. They are verifiable, meaning all players in the game can verify each other’s proxy and automatically send to the correct proxy. And finally, they are dynamic, meaning the proxies are rearranged after a predetermined period of time (40 frames in our implementation). This is done by each player using the same random number generator to determine its own and other players new proxies. Thus, a cheating proxy can only disrupt a single other player’s updates, only for a very limited period, with no direct benefit to his own avatar, and with high possibility of detection, therefore, greatly reducing his motives to do so. Furthermore, the dynamism of our approach makes any form of collusion very difficult.

Collusion Resistance & Veriﬁability: While most gaming architectures and even security mechanisms completely ignore collusion between players due to the difﬁculties in dealing with it, it is a present problem. In order to address this, Watchmen greatly reduces the information available to colluding cheaters. Proxy renewal is designed to limit the cheating and collusion opportunities available to cheating or malicious proxies. The proxy period is chosen long enough to be able to cross-check updates, but not long enough for colluding cheaters to cooperate in their actions.

Encryption & Signatures: To prevent proxies from tampering with the messages they for-

85 3.5 Verifications, reputation and punishment ward – namely updates, subscriptions and handoff messages – Watchmen uses lightweight (i.e., „100 bits while state update messages are 700 bits on average) digital signatures [EKP`07], and each player verifies the digital signature of the messages it receives. This also prevents replaying and spoofing.

Handoff: is performed between a player’s successive proxies to allow longer-term follow- up: before a player’s proxy is renewed, it sends a summary of the player’s state to the player’s next proxy, i.e., its own successor. In addition, to limit the impact of player-proxy collusion, a proxy also embeds the summary it has received from its predecessor (follow up on two previous proxies).

3.5 Veriﬁcations, reputation and punishment

Not all cheating types can be prevented, therefore, most architectures rely on detection to deter players from cheating. For successful cheat detection we need to make sure: (1) enough information is available to perform efficient verification, (2) verification methods are successful in finding cheaters, and (3) detection results are useful for a blame/punishment system.

3.5.1 Veriﬁcation

Each player can perform veriﬁcations of each other player. The types of veriﬁcations and their accuracy depend on whether he is the other player’s proxy and/or whether he has the other player in his IS or VS.

The veriﬁcations done by the players are as follows:

Subscriptions: The proxy of a player p can verify whether a subscription of p to player q is justiﬁed based on p’s position, aim, and movement history. This information is available at the proxy. A VS subscription is only valid if q is in p’s vision cone. For incorrect VS subscriptions, the distance between the q and p’s vision cone is used as a metric of the deviation. For IS-subscriptions, a proxy computes interest with sufﬁcient accuracy based on the attention metric.

Position Changes: of a player p can be veriﬁed depending on the information available

86 3.5 Veriﬁcations, reputation and punishment by the verifying player. As the proxy and players that have p in their IS receive the exact frequent position updates, they can easily compare successive updates and control whether the movements follow game physics (e.g., gravity, limited velocity, angular speed, permit- ted position). Proxies and players that have p in their VS range can a posteriori verify if the actual movements of a player are consistent with the predictions included in dead reckoning messages. We use the area between the simulated and the actual trajectory of the avatar as a metric of the deviation. Players that do not have p in their IS or VS can also do veriﬁcations as they receive infrequent update messages, but the accuracy is obviously reduced.

Actions: Similar to position changes, the proxy and the players that have p in their IS are able to verify that p’s actions follow game rules (e.g., decreasing health after a fall) comparing successive state updates.

Dissemination Frequency: Proxies can control whether a player sends timely updates and other players verify that proxies forward them.

Interactions: such as hit and kill-claims are verified by proxies and by players acting as witnesses. The verification consists of checking that, e.g., a rocket was effectively fired and the distance between the position of the rocket and that of the target is used as a metric of the deviation.

Players are in charge of the short-lived objects they create, in addition to their avatars. Hence, such objects are checked by proxies and other players as well.

For efﬁciency reasons, we perform sanity checks [GV08] to detect cheating. How- ever, action repetition checks (e.g., tamper-resistant logging mechanisms [HARD10b]) that would provide more accuracy but incur higher costs are also possible.

As most of these sanity checks are not exact (some information may be slightly outdated or be an estimation/prediction), each action is rated from 10 to 1 with regards to cheating probability (10 most likely cheating, 1 most likely normal). To determine the cheating rating, the verifying player checks whether the observed behavior (movement, dissemination frequency, subscription, etc.) falls within the expected behavior. If yes, the cheating rating is set to one. Otherwise, the higher the deviation from a expected behavior, the higher the cheating score for the action. What it means for an action to fall within the acceptable range

87 3.5 Veriﬁcations, reputation and punishment depends on the action types. A simple example is: when a player receives two position updates, it can calculate the speed and determine the movement to be acceptable if the speed is lower than the maximum allowed velocity. As another example, when comparing previous dead reckoning messages with a current position update, the action is acceptable if the area a between the predicted trajectory and the actual trajectory of an avatar is smaller than the average value a¯ observed for honest players plus a certain tolerance (typically the observed standard deviation σa to keep the false positive rate acceptable). That is pa´pa¯ ` σaqq ă 0 indicates a valid dead reckoning message–position update pair and everything above is suspected.

These ratings are further modulated by a confidence factor, taking into account the confidence of the player in his rating. Confidence depends on the accuracy of the information available at the player: proxies are assigned high confidence cP , players that have the con- cerned avatar in their IS or VS have medium high cIS and medium low confidence cVS respectively, and other players have a low confidence cO (cP ă cIS ă cVS ă cO). In addition, it takes into account the staleness of updates: discrepancy of a new update with a very old guidance message is assigned a very low confidence.

3.5.2 Reputation & Punishment

Because the detection system has false positives (due to e.g., message loss), a single detection of cheating does not result in banning of players. Instead, each player tags the interactions he has with other players as successful (if no cheat was detected) or as failed, and this information is fed to a reputation system. Such information can be collected by either (1) a centralized game lobby that manages access and logins and can thus ban the players or (2) a distributed reputation system. In its simplest form, a reputation system decides to ban a node, if the proportion of acceptable interactions of a player drops below a given threshold. This threshold is set, based on the success and false positive rates of the detection system. More elaborate reputation systems (e.g., [AF12]) incorporate the notions of conﬁdence and credibility to modulate the nodes’ reports, prevent bad mouthing, and deal with collusion by analyzing relationships between nodes (e.g., the game’s social network), resulting in an improved robustness. The Watchmen detection algorithm can be plugged into any reputation system. The design of such systems is out of the scope of this thesis.

88 3.6 Watchmen Performance Characteristics

3.6 Watchmen Performance Characteristics

The performance of the architecture is of utmost importance. Measuring Quality of Expe- rience (QoE) [CE11], quantitatively, has always been hard. However, factors that directly impact the user experience have been well documented, speciﬁcally in a well-studied game such as Quake III (and similar games) [Arm03]. The main factors are latency and bandwidth requirements which in turn, combined with network loss, translate to loss of timely updates. We will show our architecture meets and exceeds these requirements and com- pares favorably with similar distributed architectures.

Latency: Most distributed architectures proposed for games rely on some form of multicast trees [KLXH04] or routing hubs [BPS06] who need several hops to get a message to the destination. The best case scenario is represented in a direct subscription systems [BDL`08] that relies on forwarding pools. These forwarders relay trafﬁc for nodes with worse connectivity, resulting in two hops (from publisher to forwarder to subscriber). This is in essence similar to our model in which trafﬁc is forwarded by players’ proxies (we’ll discuss low bandwidth support further below). Furthermore, we use the following optimizations to even further improve latency:

(1) In each frame players calculate their subscriptions for the coming frame and send the subscriptions ahead of time. In our experience (similar to Chapter 5), given small differences between frames, using current angular and physical momentum, subscriptions can be calculated. (2) Subscriber retention, in which subscriptions are maintained for several frames, eliminates subscription latency after the initial subscription. To decide on the retention period, one must calculate the average change frequency in subscriptions. In our experiments, nearly 50% of the players in the IS change after 4 frames, less than 10% last more than 30 frames. While this value can be slightly different for different maps, we found it to be fairly accurate for most gaming sessions. This value can be used in a timeout mechanism for subscriptions: subscriptions are kept for a predetermined number of frames. Only new subscriptions are sent out explicitly. In addition, note that it normally („ 83% in our analysis) takes at least one or two frames to become the center of attention after entering the IS. (3) In extreme cases, one can relax the ﬁrst hop requirement, if bandwidth allows it, and remove forwarding proxy requirement at the cost of lower security. Note that the

89 3.6 Watchmen Performance Characteristics subscriptions will remain anonymous as they are handled by target’s proxy (the receiver’s proxy also requires concurrent but less veriﬁable copies to be sent to it).

Upload capacity & Fairness: Not all peers in a distributed system may have similar bandwidth capacities available to them. This is usually solved by using forwarding through more powerful nodes. However, if only low bandwidth players use forwarding, they will be at a disadvantage as they send and receive updates later than others. In our system all players’ traffic is processed through proxies, therefore, it is fair to the players. In addition, the selection process can be refined, if necessary, to take into account resource heterogeneity. This means that using the same verifiable random generator players’ with low resources are removed from the proxy pool and more powerful can become proxies for more than one player. In such a case a player with low resources needs to send a single copy of updates to its proxy. The disadvantage, however, is that this will increase proxies access to information and should be avoided unless necessary. Similar to most current systems a feasibility test can be run at the beginning of gaming session to determine if players meet the minimum requirements. Changes in the available bandwidth can be advertised (infrequently) piggybacked on the updates sent.

Churn & NAT: Most architectures have to deal with churn. In our case, updates sent between players also act as a heartbeat mechanism that easily identiﬁes the players that have been disconnected or left. These nodes are removed in the next round, through an agreement protocol, from the proxy pool. For NAT support, Internet Gateway Device Protocol (using the MiniUPnP [Min] library) is used to add translation rules at the router. If the protocol is not supported by the router (or disabled), NAT traversal through “hole punching” is employed using the STUN(T) library [RMMW08].

Hybrid architecture: One of the advantages of Watchmen is that if game servers exist they can be easily incorporated by providing the game lobby, extra bandwidth, and becoming the proxy for some or all players. Tasks can be delegated to players as soon as they are determined to be trustworthy.

90 3.7 Watchmen Evaluation

3.7 Watchmen Evaluation

No game research framework can realistically emulate commercial multiplayer games, which take years and millions of dollars to develop. To best showcase the abilities of our architecture, we have therefore chosen to run experiments with the popular Quake III game. It is currently the best studied open-source game, and its requirements have been well documented, and therefore can be used to validate the experimental results. Our evaluations are based on the following: (1) Our enhanced Quake III (Written in C) can support up to a maximum of 48 players (instead of 8-16). Given inherent Quake III limitations, the game cannot be scaled to more players. We compute the different sets (IS, VS, Others) inside the game. In addition, a tracing module has been added to the game that records in a trace file all important game information, e.g., different sets, players’ position, aim, weapons, ammo, health, and speed, as well as items location, item pickups, shootings, and killing of players. (2) A replay engine (in Python) has been built that can replay game traces and generate the same network traffic repeatedly and under different networking and proxy architectures to measure different aspects of the performance (e.g., latency). It helps with repeatable experiments by replaying the same game session over the network, exactly as Quake III would, using the collected traces (with various number of machines up to the maximum players) and is remotely deployable (e.g., over PlanetLab). Given availability of traces to all machines, each one can accurately measure which messages should have ar- rived and at what frame. (3) A trace-based simulation environment (in Python), that aims to measure the information exposure, scalability of the game, and the quality of verifications performed, using multiple traces and under different configuration settings.

The goal of our experiments is to show that Watchmen: (1) limits the effects of collusion and the information available to cheaters; (2) can detect cheaters using different types of veriﬁcations; (3) meets the latency and loss requirements already established for Quake III and similar games while keeping the scalability properties obtained by the distributed multi-resolution scheme3. We use the following for comparison: Donnybrook as a sample of a good multi-resolution system, where players receive frequent updates for players in their interest set and dead reckoning messages for others; and an optimal Client-

3We do not evaluate distributed reputation schemes, since well established methods that cope with wrong- ful blames and collusion exist [AF12].

91 3.7 Watchmen Evaluation

Client / Server Donnybrook Watchmen

1 1 1 Complete Freq. up. + Dead reck. 0.8 0.8 0.8 Freq. up. Dead reck. 0.6 0.6 0.6 Infreq. up. Nothing 0.4 0.4 0.4

0.2 0.2 0.2

proportion of honest players 0 0 0 12345678 12345678 12345678 # of colluding players # of colluding players # of colluding players

Figure 3.4: Information about players available to players (potentially cheaters). 48-player game in q3dm17 map.

Server case where players receive frequent updates for avatars in their PVS and nothing for the rest; and Watchmen as described in Section 3.3.

Information Disclosure & Collusion: We evaluate how well Watchmen minimizes information disclosure through messages, even in the presence of collusion: Players can obtain several types of information about other players: complete information (i.e., proxies about the players they are in charge of), frequent state update (about avatars in the interest set), guidance (dead reckoning) messages (about avatars in the vision set), and infrequent position update. The ﬁrst type is the most informative, the second and third complement each other, even though frequent updates are more detailed they are not directly comparable (as guidance includes extra player statistics and predictions for game rendering), and the fourth is the least informative. We measured the joint information obtained by a coalition of colluding cheaters about other players using a 48-player trace from a Quake III game in the q3dm17 map. This is a worst case scenario as we assume all colluding players work together and any information available to one cheating player is immediately available to all colluding partners. We consider three infrastructures: Client-Server gives the minimum necessary information and thus serves as a baseline, Donnybrook as a sample multi resolution system (our implementation of interest sets according to Donnybrook, since the code was not available), and Watchmen.

We compiled the results under the forms of stacked histograms, shown in Figure 3.4, as follows. Consider a simple example of a coalition of two cheaters for a game with eight players: player 1 with IS t4, 5u, VS t2, 6u and other t3, 7, 8u, who acts as a proxy for

92 3.7 Watchmen Evaluation player t3u; player 2 with IS t1, 6u, VS t7u and other t3, 4, 5, 8u, who acts as a proxy for player t1u. The coalition therefore has: complete information about 1 honest player (t3u), frequent update and dead reckoning for 1 honest player (t6u), frequent update only for 2 honest players (t4, 5u), dead reckoning only for 1 honest player (t7u) and infrequent update for 1 honest player ({8}). It can be observed in Figure 3.4 that despite the information leakage caused by proxies, Watchmen signiﬁcantly reduces the information disclosed when compared to Donnybrook. Our trace-driven simulations shows that Watchmen, a coalition of four cheaters has minimum information (i.e., infrequent position update only) for about 31% of the honest players and partial information (i.e., dead reckoning or frequent state update) for about 48% of them. In Donnybrook, the same coalition has dead reckoning information only for about 65% of the honest players and precise information (i.e., dead reckoning and frequent state update) for the rest. Since in Donnybrook players receive dead reckoning messages about all the players not in their interest set, the proportion of honest players about whom a coalition has frequent state updates alone is very low (ă1%). In practice, Donnybrook makes use of forwarders (i.e., clients with high bandwidth multicasting updates for of clients with low bandwidth), which constitute a large and additional source of information exposure. Therefore, the results presented above are a lower bound of Donnybrook’s information exposure. Note that these beneﬁts are in addition to the subscription hiding gained by the architecture.

Effectiveness of Verifications: To showcase Watchmen’s ability for verifications (1) we measure useful information about a player that is available to players (and in particular to proxies) that can be exploited to effectively implement most forms of verifications suggested in the literature. Furthermore, (2) we measure how often colluding cheaters are assigned to honest players, and (3) we measure effectiveness of several of these verifications in the form of sanity checks. Note that more complicated and rigorous action repetition verifications would only improve the results presented here.

In Watchmen, verifications rely on proxies and witnesses (i.e., players having sufficient information to assess the validity of actions). To evaluate the potential for effectiveness, we measure, for a given cheater, the average number of honest players that: act as proxy for him, have him in their IS, or have him in their VS. The results, shown in Figure 3.5, show great potential for verifications: even when a player colludes with 3 other cheaters (out of

93 3.7 Watchmen Evaluation

48 players), he is assigned an honest proxy in 94% of the cases (1´3{47) and 10 players on average witness his actions (4 through frequent state updates and 6 through dead reckoning messages).

Watchmen 14 Complete 12 Freq. Up. Dead reck. 10 8 6 4 2 avg # of honest players 0 12345678 # of colluding players

Figure 3.5: Levels of information about cheaters available to witnesses.

To further show the effectiveness of the information available, we set up an experiment where a cheater sends up to 10% invalid cheat messages. We measure the overall success ratio (high confidence detection by one of the honest players) of different verifications, where false positives (honest messages wrongly identified as cheats) are limited to a maximum of „ 5%. Note that this is regulated by the confidence factor of the players and how much deviation the update has from expected behavior and is very system and cheat specific. In practice, we manually and through experiments configured these values for different cheat types. However, these values can be properly obtained and configured by the reputation system used. The encouraging results are shown in Figure 3.6 for the following verifications performed by an honest player (other verifications would also be possible): Position Updates: Updates are verified to adhere to the game rules. For example, a player cannot move faster than allowed in the game. In our system, cheaters move randomly at 1.5–3 times the acceptable speed; Kills: One trivial opportunity to cheat in FPS is to unduly claim kills. We address this by verifying the type of weapon, the distance, the visibility, and how long the attacker had the target in his IS (Typically 4„10% of the kills had their target in the IS for less than 2 out of 5 frames, depending on the map);

Guidance: Guidance messages are compared against future frequent updates by the

94 3.7 Watchmen Evaluation

100 80 60 40 20

Detection success (%) 0 Position Kill Guidance IS-sub VS-sub Cheat verification type

Figure 3.6: Success rates of the different veriﬁcation mechanisms executed. proxies as well as dead reckoning computed by proxies; IS and VS Subscriptions: Sub- scriptions are checked from the accurate information available to proxies.

70 King Latency Set 60 PW Latency Set 50 ) %

( 40 30 PDF 20 10 0 0 1 2 3 4 5 6 Age of updates in frames

Figure 3.7: Distribution of the age of received updates (all three types) from the frame they should have been received.

Responsiveness: To ensure that our architecture meets the ofﬁcial latency requirements of Quake III, we performed experiments on a LAN as well as preliminary tests on Planet- Lab. Furthermore, we simulated latency in our networking module using latencies available from the King [GSG02] and PeerWise [LBSB09] datasets, ﬁltered using a Geo-IP location dataset that limits the locations of IP addresses to the United States (with mean latencies of 62 and 68 ms respectively). Our latencies are consistent with Halo’s [LABP08]. Message loss is simulated with a rate of 1%. Note that FPS games that receive updates within 150

95 3.7 Watchmen Evaluation ms latency with loss of under 10% deliver a good gameplay to users [BCL`04].

The performance results for Watchmen are shown in Figure 3.7. Given that most P2P architectures use forwarding or multicast trees, Watchmen provides relatively low latencies and meets the requirements of FPS games. Quake tolerates up to 150 ms latency, therefore, only the messages that are 3 frames old or more of Figure 3.7 are counted as loss. One

25 IS Subscriptions 20 VS Subscriptions )

% 15 ( 10 PDF 5 0 0 5 10 15 20 25 Number of Subscriptions

Figure 3.8: PDF of the number of IS and VS subscriptions. source of delay is when an avatar, neither in IS nor in VF, enters IS or VF. Indeed, the movement of the avatar should be displayed smoothly while only a possibly outdated position update is available. This phenomenon is however rather infrequent: in a frame, on average 88% of the players in IS were already in IS in the previous frame, 8.5% were in VS and only 3.5% would suffer from a slight delay. This can be dealt with by slightly widening the vision range (˘60 degrees in our experimentation), thus increasing the responsiveness and the effectiveness of veriﬁcations, but also the amount of information disclosed.

Bandwidth: The upload bandwidth needed by our system is dependent on the IS and VS subscriber sets, i.e., on average how many players have subscribed to a given player. The distribution of these subscriptions is shown in Figure 3.8. This distribution does not significantly change as the number of players grows, thanks to the limited visibility and IS sizes. Even though our goal is not to further decrease bandwidth over existing multi-resolution schemes, Watchmen decreases the bandwidth usage for updates sent because of the employed information ﬁltering. On the other hand, bandwidth is increased because of proxies, cross-veriﬁcations, and encryption. Upload bandwidth use is highlighted (per type of up-

96 3.7 Watchmen Evaluation

5000 Freq. Dead Reck. Infreq. up. 4000 IS sub. VS sub.

3000

2000 Upload bandwidth (kbit/s) 1000

0 28 50 100 300 500 700 900 28 50 100 300 500 700 900 28 50 100 300 500 700 900 Quake 3 P2P Donnybrooke Watchmen # of players

Figure 3.9: Upload bandwidth use projected for Quake III P2P, Donnybrooke, Watchmen including message signatures. date sent) in Figure 3.9. Although the proxy architecture may incur some computational and bandwidth overhead, it does not hurt the scalability as, due to its decentralized nature, a player is in charge of only one other player on average, regardless of the total number of players. As a simpliﬁed example with 5 players in IS, average frequent update size of 32-36 bytes, and 20 updates per second as well as 895 players receiving 1 update per second with average size of 55 bytes, the average networking requirements would be less than 0.5Mbps. Considering subscriptions and the encryption overheads (the larger XBox live header is around 44 bytes), the average bandwidth requirement is still around 0.8 Mbps. Even if delta coding is not used, the results should be manageable by less than 2Mbps. Therefore, we estimate Watchmen to have network requirements similar to that of Donny- brook and be able to scale up to 900 players (with trafﬁc distribution of mean < 1 Mbps and a range of 256 Kbps to 5 Mbps, with a 6 Mbps upload bandwidth cap). Note that average bandwidth requirement is not the only determining factor and a more comprehensive study of meeting the necessary requirements is done in Donnybrook which we do not repeat here.

97 3.8 Watchmen Related Work & Compariosn

3.8 Watchmen Related Work & Compariosn

This section summarizes the work that is speciﬁcally related to Watchmen. Some work has already been mentioned in the previous chapter but will now be discussed again to better put it in comparison with Watchmen.

Distribution: Many distributed game architecture exist (e.g., [KLXH04]), however, few are designed for fast paced games like FPS. Use of DHTs [BPS06] or multicast trees is too slow for these games. In addition, network overlays such as trees cannot follow fast paced changes in the players interest. Closest to our work is Donnybrook [BDL`08], which we use for comparison, a multi-resolution scheme for update dissemination. All of these architectures ignore cheating, i.e., they do not provide any cheat prevention and detection mechanisms, and even introduce additional cheating possibilities because messages between players are routed through other player nodes and at different rates. Note that there exist complementary tools that can be used by most architectures to deal with some types of cheating, e.g., secured client side veriﬁcations such as PunkBuster [Pun11] and Valve Anti Cheat (VAC) [VAC13], prevent the players from tampering with the game code. Pseudo- random number generators have been successfully used in P2P systems [LCW`06].

Mutual verification: A range of cheat detection approaches for distributed games, including mutual verification and log auditing, are described in [KTCB05]. Following these guidelines, Watchmen uses mutual verification, but only short-term auditing for improved responsiveness. It also mitigates unnecessary information disclosure. RACS [WSL07] relies on trusted referees who simulate the game world locally to assess the validity of the players’ action. IRS [GV08] considers the case where a server delegates tasks, including sporadic audit of other players, to one or several players, namely, proxies. One of the few parts that consider collusion is [WST09], where a referee selection algorithm optimizing both responsiveness and the probability of player-referee collusion is proposed for the case where (untrusted) players can be referees.

AVM [HARD10b] runs software binaries inside a virtual machine and uses tamper- resistant logging which allows for exact replaying of events and detection of cheaters. Technically, AVM can be used on our platform as others for added security. However,

98 3.9 Watchmen Conclusions

AVMs impose a non-negligible overhead on the game and log recording (particularly on now ubiquitous multi-core platforms) and exchanging them for veriﬁcations is non-trivial. Also, such platforms require that messages eventually be delivered, which is impractical in games, and do not protect against some cheats such as rate analysis and collusion. More- over, deployment of VMs on game consoles for current-generation games is currently not available.

Unnecessary information exposure: In [PG11], a set of basic secure primitives are proposed to build cheat-proof P2P trading card games. A cheat-proof peer-to-peer implementation of each primitive is proposed, using cryptography.

Based on the fact that knowing additional information impacts the way players behave (e.g., a player obtaining information about players behind walls is likely to stare at walls), a statistical cheat detection approach has been proposed [LPBC07] where uncommon patterns in players’ behaviors (e.g., movement) are identified. This is also possible in Watch- men through verifications run by proxies to further improve performance. Another singular approach to cheat detection [FR06] is to deliberately delay messages and analyze players reaction thus detecting typical look-ahead cheats, i.e., waiting for other players to declare their actions before committing one’s own to take informed decisions (e.g., avoiding a rocket fired by another avatar). Other protocols (e.g.,[BLL07]) make use of commitment schemes to prevent timing and look-ahead attacks, at the price of responsiveness.

SpotCheck [MDF11] limits the information disclosed to the client by having the clients submit view requests to obtain information about the area that is visible to the player every time they move. The server probabilistically checks the validity of the view request based on position and visibility information Moreover, the server runs basic checks (e.g., size of the requested area) for all requests.

3.9 Watchmen Conclusions

This chapter proposed Watchmen, the first distributed protocol designed for FPS games that aims in detecting and preventing cheating and collusion. This is done through cross verifications and vision-based filtering facilitated by a dynamic proxy scheme. Experimentations on Quake III show a great reduction in the information exposed to cheaters even in the pres-

99 3.9 Watchmen Conclusions ence of collusion. Furthermore, it shows great potential for effective veriﬁcation of player actions by other players. Watchmen is able to scale well and can meet well-established latency requirement for fast-paced games.

100 4 Towards Providing Security For Mobile Games

Mobile game industry has been experiencing exponential growth over the past few years. However, Mobile security for games is often ignored when discussing architectural design of mobile systems. Substantial increase in the power and networking capabilities of smartphones has led to the emergence of mobile games, and mobile gamers now outnum- ber "core" gamers. These games include many genres of gaming with different and new requirements. Here we discuss existing and potential new avenues of cheating in mobile games and suggest several solutions based on how different existing and novel services can be provided by carriers, ecosystems, and developers to improve the security.

4.1 Introduction to Security in Mobile Games

Mobile games, their requirements, and effects are often treated as an afterthought in architectural design of mobile systems. Nearly 136 million people in the U.S. now own a smartphone (58% of mobile subscribers) with numbers increasing fast. Around 34% of all mobile subscribers play mobile games [Com13] with the gaming rate being much higher in the rising smartphone user group. Many of these smartphones incorporate several sensors such as a GPS chip, an accelerometer, a digital compass, a proximity sensor, an ambient light sensor, a microphone, and a camera. This variety of sensors has given rise to a huge number of innovative mobile applications including games: The sensors, along with multi-

101 4.2 Cheating in the Mobile Environment touch support, turn the smartphone into an effective gaming controller. Increase in the processing power of smartphones further transforms them into full ﬂedged gaming platforms. In fact, mobile games already provide 15% of the overall gaming revenues (10 Billion) and are fast on the rise [NPD13]. They also provide higher proﬁts, in terms of man years in- vested, than console and desktop games. As a result, architectural support for mobile game security becomes important [HLR11].

While we discussed many different cheating types in previous chapters, mobile ecosystems open the door for new types of cheating and existing cheating opportunities need to be judged differently. While mobile ecosystems can be fairly controlled, given the preva- lence and legality of modifying the operating systems, e.g., jailbreaking and custom ROMs, cheaters can easily gain access and tamper with game codes. Furthermore, mobile devices create new opportunities for cheating such as faking various sensor readings. Currently, it is fairly easy to fabricate or lie about readings from the sensors. These fake readings can include modifying the pictures taken by the cameras or fake locations by fabricating readings from GPS sensors [SW10].

Given the new challenges that arise in mobile environments, it is crucial to understand the implications of mobility on cheating and the options to avoid or detect it. Therefore, this chapter proposes an analysis of how existing cheating mechanisms change in mobile environments, how new opportunities arise, and how they all impact mobile game play. From there, we outline how a framework of stakeholders, consisting of game developers, game engines, operating system ecosystems and carriers can help in securing mobile services for games.

4.2 Cheating in the Mobile Environment

Based on our discussions in the previous parts, we divide cheating types into two categories of existing and new mechanisms. Existing cheating mechanisms are those used in traditional PC and console gaming environments that also exist in mobile environments while the new ones exploit mobile environment characteristics. In addition, some forms of cheating, e.g., time-travel hack where a cheater changes the date and time of the handset, are used in other types of gaming as well and do not change due to the mobile environ-

102 4.2 Cheating in the Mobile Environment ments. However, they become more prominent as a result of some types of games being more popular in mobile systems, e.g., in freemium models where some actions can only performed a limited number of times in a given time period unless players pay.

Note that here we focus on cheating mechanisms that exploit vulnerabilities in the game design or the mobile environment and do not consider other forms of abuse such as social misconduct and collusion outside the game environment (e.g., exchanging information through chatting).

4.2.1 Existing Cheating Mechanisms

Recall the summary Table 3.1 we provided of the commonly used cheats. Here we highlight the threats that are different in mobile environments. Some cheating mechanisms get easier as a result of characteristics of mobile environments, while some become harder to detect. One of the advantages of mobile environments, however, is that due to the limited capabilities of the smartphones the cheating softwares can, in some cases, be less complicated. On the other hand, if validations are to be run on the smartphones, they too will have limited resources available and therefore tests need to be less complicated than their desktop counterparts. The main causes for worse security are:

‚ Exacerbated vulnerabilities: Wireless networks make it easier to perform some types of cheating. For example, snifﬁng WiFi network packets can be easier, specially if players are relatively close to each other. ‚ Harder Detection & prevention: Higher latencies, loss, and jitter makes detection of cheaters harder. For example, higher latency makes it harder to detect cheaters who with- hold their updates and/or delay them.

Recall that we divided cheating types into the following three categories (similar to Chapter 3) and then discuss how they are affected as a result of architectural differences in mobile systems: (1) Interrupting information dissemination (2) Illegal game actions (3) Unauthorized information access.

Interrupting information dissemination: Includes dropping, corrupting, delaying, changing the rate of the updates, or sending wrong or inaccurate information. This helps a

103 4.2 Cheating in the Mobile Environment

Table 4.1: Popular cheating mechanisms in distributed games Type Name Description Mobile Games Escaping Terminating the connection in order to escape imminent loss Worse Time Cheating (look ahead) Delaying the updates to base one’s actions on those received from others Worse Disruption Of Network Flooding Overflowing the game server to create lags and disrupt game play Better Information Fast Rate Cheat Mimicking a rate of game event generation that is faster than the real one Worse Flow Suppress-Correct Cheat Dropping consecutive updates, then sending an invalid update afterwards Worse Replay Cheat Resend signed & encrypted updates of a different player Worse Blind Opponent Dropping updates to opponents, blinding them about the cheater’s actions Worse Client-side Code Tampering Modifying the client-side code to get an unfair advantage Same Invalid Up- Aimbots Using an intelligent program to provide it with automatic weapon aiming Same dates Spoofing Sending messages, pretending to be a different player Worse Consistency Cheat Sending different updates to different players Same Sniffing Logging & accessing different information sent across the network Worse Unauthorized Maphack Hacking to see through walls and obstacles Same Access Rate Analysis Analyzing the updates rates to detect players attention and escape Worse player blind or confuse others about his current state. As a result, the chances of his avatar being targeted is decreased or he gains an unfair advantage in his attacks.

‚ Disconnections & Loss: Given higher disconnection and loss rates in mobile environments, detection of cheats such as escaping where the player terminates the connection to escape imminent loss, or blinding opponents where updates are not sent to some players, is harder. ‚ Latency: Higher latency in mobile environments makes it harder for different time cheating mechanisms to be detected. For example, in look ahead cheating, cheater deliberately delays his updates to base his actions on those he receives from others or sends messages with fake old time stamps. Similarly, fast rate cheat where higher rate of actions is submitted, is harder to detect. ‚ Retransmissions: Given easier snifﬁng in wireless environments (see below), replay attacks are easier to perform. In such a case the cheater replays the messages sent by others, e.g., a health reduction, to damage others.

Illegal game actions: A cheater can circumvent the game physical laws (e.g., limited velocity) and unduly change his state (e.g., increase health or ammo) by tampering with the game code. Players may also use third party software to perform game tasks such as aiming. In such a case, the player uses an intelligent program to provide automatic weapon aiming which is mostly used in FPS games.

In closed mobile environments, it may be harder to perform modiﬁcations to the game, e.g., app code tampering. However, modiﬁed apps can be used through jailbreaking or third

104 4.2 Cheating in the Mobile Environment party channels. In addition, open source operating systems such as Android can be more vulnerable through use of custom ROMs. Given limited memory of these devices, memory injection tools can be used more effectively to modify critical game values.

Unauthorized information access: Available information in the game that is not supposed to be disclosed to the players (e.g., position of players behind walls) can be exploited to increase the chances to kill other players or to foresee danger and evade. Players typically receive excess information to be able to quickly render view changes in the game, however, this information is not meant to be displayed to the player until the view changes. While untampered code might not show the information on the screen, such information can often be easily extracted by cheaters. Furthermore, due to the nature of wireless medium, it’s easier to sniff the packets sent and this information can be used for cheating. Most new cellular communication technologies provide reasonable encryption mechanisms and snifﬁng tools are not widely available, however, snifﬁng is fairly easy in WiFi networks. Similarly, players’ communications can be more easily jammed due to the nature of wireless communications, especially by players in close physical proximity. This can be used simply as a DoS attack or to gain unfair advantage in the game. These advantages include preventing other players from receiving necessary game information or performing their actions. In addition, a simple rate analysis (see Chapter 3) can help cheaters in gaining unauthorized information in the game (e.g., in multi-resolution games where updates are sent at different rates to different players).

4.2.2 Existing Security Mechanisms

As we have discussed, many cheat prevention (proactive) and detection (reactive) techniques have been proposed to address cheating in networked games [WS07]. Most of these techniques can be used in mobile environments: Cryptographic measures, signatures and checksums, are effective in eliminating message sniffing and illegal modifications. These techniques can often be refined for mobile environments as we will see later in the chapter.

Dealing with delayed and dropped messages is harder in mobile environments due to higher latency and loss. The state of the player in the game (i.e., winning, losing) along with statistical analysis (frequency of disconnects when player is losing) can be used in

105 4.2 Cheating in the Mobile Environment

Table 4.2: Suggested API for securing game services Category Method Description Validate Inf. Flow VerifyNetStats(Netstats *nets, Loc *loc) Verifies the network connectivity stats such as loss and latency ValidateLocWiFi(LocInfo *Loc, WifiInfo *SSIDs) Validates location against WiFi info e.g., list of nearby SSIDs Validate Location ValidateLocCell(LocInfo *Loc, CellInfo cell) Validates location against the cell tower information VisualLocValidation(GeoTaggedPic pic) Validates the player’s location through visual verification ValidatePath(Path *path) Validates accessibility of player’s path and location PhysicalVerification(ActionList *alist, SensorReadings *sr ) Validates last n actions using sensor readings for duration t Validate VerifyLocQuestion(QuestionHash qh, Answer ans) Verification by questions about local information Readings PhysicsCheck(SensorReading *sr) Validate state updates or predictions according to physical rules TuringTest(SensorReading *sr, Credentials *cr) Novel Turing tests to ensure human player TPMSign(TPMSign s, Message m) TPM verifies running specific hardware and/or software FaceRecognition(UserPic pic) Authentication and Bot prevention Validate Access ValidateTransaction(Transaction *Tr) Validate micro-transactions improving the detection of escaping cheats. One approach to detect these cheaters is by delaying the updates and determining who waits for the updates (e.g., AC/DC [FR06]). However, in a mobile environment signal strength as well as location information can be used to verify whether high loss and latency is expected or not.

Similarly, many of the cheat detection and prevention techniques mentioned in the previous chapters such as CAPTCHA texts, as an example of Turing tests [GD05, SGJ07], learning approaches to detect aimbots [CPC08], code tamper detection software [Pun11, VAC13], and hardware-based solutions [FKS08, SW10] can be used to address cheating with variable success.

4.2.3 New Gaming & Cheating Mechanisms

In addition to existing cheating opportunities, mobile platforms are vulnerable to new cheating types as well. This can be a result of new vulnerabilities introduced by the mobile environment or due to new gaming environments such as augmented reality (AR) 1 games. These games feature live view of a physical real-world environment whose elements are augmented by computer-generated sensory input such as sound, video, graphics that constitute the game items. As a result new cheating opportunities emerge:

‚ Faking sensor readings: it is relatively easy for malicious applications to fabricate or lie about readings from these sensors as most systems currently lack software or hardware checks [SW10]. By lying about the gyroscope, accelerometer, and other sensor readings,

1http://www.google.com/glass/

106 4.3 Trust Model & Design Principles in Mobile Games

players can gain unauthorized access to information or gain an unfair advantage. Fur- thermore, sensor readings are used as controller input for games and by faking sensor readings the cheater can use cheating programs such as aimbots. ‚ Faking Location: One main difference between smartphones and current game consoles or PCs is in providing real world location services. This provides the potential for location based games (e.g., geo-caching) as well as augmented reality games. In such cases, faking the location can result in access to information such as virtual items in the game that are only available to players in close proximity to the item. Similarly, social check-in based games suffer as well. ‚ Exploiting Consistency Techniques: Many techniques have been proposed or optimized to deal with higher loss and latency and lower bandwidth in mobile networks. These techniques can be exploited by the cheaters. For example, dead reckoning used to deal with high latency and loss can be abused in suppress-correct cheats. ‚ Micro-Transactions: one of the most popular forms of gaming is freemium models where the game is offered for free but in game items and enhancements are offered for a pur- chase. Cheating in collecting or purchasing these items is one the most common forms of cheating.

4.3 Trust Model & Design Principles in Mobile Games

4.3.1 Trust Model

We identify ﬁve main components involved in securing mobile games: (1) Game app, (2) Game server, (3) Carrier, (4) OS, (5) Phone hardware. Since we are focusing on the mobility aspects of providing security and do not discuss back-end security, we assume that the game server and carriers can be trusted. Given jailbreaking and custom ROMs in which security features can be turned off, we assume the operating system cannot be trusted. Tampering with game code is commonplace in cheating in desktop and mobile games, therefore, apps cannot be trusted. In addition, phone hardware readings can be easily faked [SW10] and unless trusted platform modules (TPM) exist the phone hardware cannot be trusted.

107 4.3 Trust Model & Design Principles in Mobile Games

4.3.2 Design Principles

Several design principles have to be taken into account in designing an efﬁcient anti- cheating mechanism.

Simplicity: One of the main advantages of mobile games is their rate of return on invest- ments, i.e., mobile games are generally cheaper to make and return a higher proﬁt. Complex cheat detection and prevention technique make the cost of development and maintenance much higher.

Low Overhead: Excessive network use can cost the customer money in addition to draining the battery. Similarly, high computational costs degrade the game performance and drain the battery.

Privacy Concerns: Sending the necessary information for veriﬁcation purposes can raise privacy concerns for the players as players may need to submit additional location information, pictures, list of nearby wireless networks, cell information, etc. Privacy and security present a trade off. However, it should be noted that much of the information necessary is already collected by the carriers, operating systems, and gaming environments. There are several protocols that provide some level of privacy while collecting data that rely on aggregating the data to provide privacy or by using negative surveys [HLN`07].

API Abuse: Security techniques can often themselves be abused by cheaters. In addition, they may introduce new vulnerabilities for the environment. For example, an API providing location veriﬁcation using publicly available information about the player can be abused by other players to locate players they are not supposed to know about.

To devise a comprehensive security solution we will ﬁrst explain how different components involved in mobile gaming can help in prevention and detection of cheating. Here, we mostly focus on the new or exacerbated threats in mobile environments, however, in some cases solutions may apply to desktop versions of the games as well. These components each can play a role in improving the security:

‚ Game Apps: Can best perform veriﬁcations to ensure that the platform (e.g., OS, hardware) is not compromised. Good authentication, authorization, and cross-checking ac-

108 4.4 Security Architecture for Mobile Games

tions all rely on a well designed game app. Apps may be required by the APIs to submit additional information to enable server and carrier side verifications. ‚ Game Servers: In addition to performing authentication and maintaining players’ records, servers can perform verifications by demanding additional information when players request certain data. More complicated detections techniques can also be delegated to servers. ‚ Operating Systems: Other than the general security mechanisms such as memory isolation, the OS can ensure the correctness of sensor readings as well as higher security for financial transactions. ‚ Carriers: Are in the best place to verify, independently from smartphone readings, the physical location of the phone and perform authentication for the user. They can also provide statistics on network quality.

4.4 Security Architecture for Mobile Games

We propose a security architecture in which different stakeholders provide support for prevention and detection of location cheating, fake sensor readings, and disruption of information ﬂow. The security architecture is exposed as a set of APIs where each method tackles a speciﬁc issue. Note that while the apps may be tampered with, the API aims to enable server and carrier validation of actions.

4.4.1 New Security Mechanisms

A summary of the proposed APIs is presented in Table 4.2. We provide cheating scenarios that explain how cheating is performed and discuss how these APIs can address these issues. In addition to discussing these new methods, we further elaborate on how they can be implemented using the currently available APIs as well as suggesting new necessary architectural support from OS and carriers to help further improve their accuracy. As an example we focus on the APIs currently provided by two large ecosystems: AT&T2 and Android operating system3, however, other popular operating systems and carriers provide

2http://developer.att.com/developer 3http://developer.android.com/reference/

109 4.4 Security Architecture for Mobile Games similar functionalities.

4.4.2 Location cheating

Consider a player that has to be in a certain location to ﬁnd an item, e.g., a geo-cache, unlock an achievement or to see a virtual item in an AR game [HLR11]. In this case players have incentive to fake their location and tools to fake location are readily available in app stores or through third party channels. To deal with this the game server can require submission of additional location information about the player along with the requests for local information. In such a case, when the game server suspects a player is a cheater or it is simply performing a random audit, additional information is used for veriﬁcation. Cur- rently there are three methods available for determining the physical location of a player: (1) Cell ID (cell tower), (2) WiFi, (3) GPS, GLONASS, or other future platforms. Any combination of these sensor readings can be used to cross check the location reported:

‚ WiFi veriﬁcation: available wireless network information can help identify the location of the player (Wi-Fi-based positioning system (WPS)). Sending a list of nearby available wireless SSIDs and their strength is currently extensively used in helping increase the accuracy and speed of the location detection (e.g., Google’s WPS database). This can be used to also verify the location of the user where player are required to occasion- ally or upon request send in a list of nearby SSIDs that will be matched against existing databases. Android currently provides WiFi location services through its LocationManager API which provides a 200 meter or better accuracy, as does AT&T, as well as any other WiFi location service. Proposed ValidateLocWiFi provides such an interface. ‚ Cell veriﬁcation: connection to a cell tower determines that the player is located within the range of the tower. In addition, its signal strength can help approximate the player’s location and cell triangulation can be performed by carriers. While this information is useful, given long range of some cell towers it might not be accurate enough. Further- more, this information may not be directly accessible by game companies. Android’s LocationManager and AT&T API both provide Network-based location services. The accuracy depends on the cell density in an area and can be up to several thousand meters. Proposed ValidateLocCell provides an interface for validating reported location information against cell tower information available which is provided by the op-

110 4.4 Security Architecture for Mobile Games

erating system, carrier, or other openly available databases. Listing 4.1 provides a sample code on how these two APIs can be used. ‚ Question answering: the player can be asked a question about his current location in which case he should be able to answer if he’s in the right spot. Many map services provide "local places information" APIs which can be used. An example would be: "name the pizzeria that you’re in front of?". Google Places API (PlacesService) provides nearby location information as does AT&T. Other similar ecosystems exist. VerifyLocQuestion provides such an interface. Using unlisted items and places for question answering tests can improve the security of the questions answering schemes. ‚ Accessibility: information about accessibility of the location sent by the player can help determine cheaters. For example, unreachable positions (e.g., middle of a lake) reported as well as the path between movements (e.g., crossing a river with no bridge) can be verified using accessibility information. ValidatePath provides an interface for such validation. Android provides Google maps API, as do many other ecosystems. Third party open solutions, e.g., OpenStreet Maps, also exist. ‚ Visual Verification: the smartphones’ camera can be used in visual verification of players’ locations. This will become more usable with use of AR devices. Visual verifications can also be done against available databases of location pictures (e.g., Google Street View and goggles or Microsoft Synth). VisualLocValidation provides such an interface. Google goggles and project glass currently does not yet provide an API, however, future API is expected. Google also maintains a large street view database that can be used.

4.4.3 Fake Sensor Readings

Smartphones can act as a game controller, therefore, faking different sensor readings can give the players an advantage. This can be achieved using an emulator (e.g., Android emulator) that hosts the operating system and generates readings. It can also be done using a compromised build of the operating system that allows for such readings to be generated by a bot. In such a scenario a cheater uses aiming bots and similar tools that can generate the necessary sensor readings to control a player’s avatar. The game has to verify the hardware and OS it is being run on in order to ensure they are valid and not compromised.

111 4.4 Security Architecture for Mobile Games

Listing 4.1: Basic Sample Location Request Veriﬁcation

1 # Game 2 move() 3 server.getLocalItems(gps, cellid, ssidlist) 4

5 # Game Server 6 def getLocalItems(gps, cellid, ssidlist): 7 if randomAudit == True or trustRating < threshold: 8 validLoc = validateLocation(gps, cellid, ssidlist) 9 if validLoc: 10 return items 11 else: 12 //player.lowerTrust() or player.penalize() 13 14 def validateLocation(gps, cellid, ssidlist): 15 cellValid = carrierAPI.ValidateLocation(gps, cellID) 16 ssidValid = WiFiLocAPI.ValidateLocation(gps, ssidlist) 17 if cellValid == Valid and ssidValid == Valid: 18 return Valid 19 return Invalid 20

21 # Carrier 22 def ValidateLocation(gps, cellID): 23 cell = carrierCells.get(cellID) 24 if distance(cell.loc, gps) < cell.range: 25 return Valid 26 return Invalid

‚ Turing Tests: are used to verify that a human is indeed playing and not a bot. Given the range of sensors available in smartphones, new forms of Turing tests can be developed. For example, a novel CAPTCHA test can be designed that asks a player to lift and move the phone in a certain pattern or touch a certain point on the screen. Accurate sensor readings (e.g., Android SensorManager) can be easily used to verify the movement pattern. TuringTest provides a general interface for this veriﬁcation which can be provided by the OS. ‚ Physics veriﬁcation: during gameplay, where the smartphone is used as a controller, detection of micro movements as the result of interaction with the phone, and matching them against the actions inside the game can be used to detect aimbots and similar cheating software. These movements have already been shown to provide detailed interac-

112 4.4 Security Architecture for Mobile Games

tion information such as detecting typing[XBZ12]. Furthermore, different sensors on the smartphone can be used to verify each other’s readings, e.g., location changes should be reflected in the accelerometer and gyroscope readings that can be used to verify direction and acceleration of the player. Most operating systems provide detailed sensor readings (e.g., using the Hardware & SensorManager APIs in Android). However, to verify the reported readings to the game (e.g., by an aimbot), the OS needs to maintain a short time history of readings. This has to be securely managed as otherwise it can be exploited itself. PhysicsCheck provides such checks which can be provided by the OS. ‚ Hardware based security: mechanisms to secure sensor readings through hardware and software checks have been proposed [SW10]. Trusted Platform Modules (TPM) can verify that a specific software has been booted on a specific hardware. Similar techniques to the ones suggested for hardware based security measures in desktop games can be used: [FKS08] proposes use of hardware-based, stealth measurements such as Intel’s Active Management Technology (AMT)4 that has been used in rootkit detection in operating systems. In this approach, a tamper-resistant processor resident on a client and isolated from the system’s primary processor is used to perform randomly-timed measurements underneath the host’s software stack to detect cheats. TPMSign provides such an interface requiring hardware support. ‚ Visual Identifications: ubiquitous presence of cameras in smartphones makes it possible to use them to visually confirm the fact that a human is present as well as his identity. Face unlocking technologies, that already exist in the smartphones, can be provided to 3rd party apps via APIs. FaceRecognition provides an interface for authentication and authorization of the players. This can be provided by the operating system, or third party providers. Google currently provides face unlocking services but it is not yet presented as an API to the developers. Third party solutions also exist.

4.4.4 Disruption of Information Flow

Cheaters can disable their communications (e.g., by simply turning of WiFi or mobile data) enabling them to perform escaping or suppress-correct cheats cheats. Determining whether

4Intel vPro Technology http://www.intel.com/content/www/us/en/ architecture-and-technology/vpro/vpro-technology-general.html

113 4.5 Mobile Game Security Conclusions the network statistics are acceptable, heavily depends on the network conditions that can be veriﬁed by the carriers. Only carriers have enough information about signal strength, network speed, average and burst loss rates, and other network statistics in certain locations. In such a scenario the game server or other players can use the carrier API to verify their recorded network statistics of the player (e.g., update loss they have experienced) and decide whether it falls within accepted threshold (extracted using honest players) or not. VerifyNetStats provides such an interface using available network statistics which can be best provided by carriers. AT&T currently provides basic APIs that can provide information about device capabilities, however, it doesn’t provide network information to apps. Android provides considerable network information available to apps. For example, TelephonyManager and NetworkInfo provides detailed information about network and its type, speed, state, etc. and ConnectivityManager provides information about connectivity type (e.g., wireless, cellular, etc.). TrafficStats provides network statistics such as number of sent and received packets and similar detailed statistics. These information can be required by a game server when performing veriﬁcations.

Transaction security: Invalid in-app transactions are one of the most popular forms of cheating. However, most transaction security problems can be solved by correct implementation of billing and authentication services. ValidateTransaction provides an interface for further validation of in-game transactions. This can be provided by the app-store ecosystem (usually by the operating system provider, e.g., Google Play), or third party eco-systems (e.g., Amazon). Carriers, the game ecosystem, or other third party services (e.g., PayPal) can be used for verifying credentials as well. Google Play store provides InAppBillingService that is used for in app billing. AT&T provides Notary Management API that can be used for in-app payments. It also enables signatures and encryption for secure communication. Visual identiﬁcation (e.g., Face Unlocking in Android) can further help with authentication.

4.5 Mobile Game Security Conclusions

Mobile games are a fast growing industry. We have done a preliminary study of how cheating is different in the mobile environments and have proposed some solutions that might be able to address some of these security concerns. Addressing security concerns in these

114 4.5 Mobile Game Security Conclusions environment requires more attention from all stakeholders, i.e., carriers, platforms, game developers, and hardware designers and requires support from all them.

115 Part III

Interest Modeling Using Pheromone Maps

116 5 Interest Modeling in Games: The Case of Dead Reckoning

This chapter is the ﬁrst of two that uses ant-colony techniques to improve game performance and game design. AntReckoning, presented this chapter, deals with improving the accuracy of dead-reckoning, and AntAI, presented in the next chapter, deals with analyzing and improving NPC performance.

In games, the goals and interests of players are key factors in their behavior. However, techniques used by networked games to cope with infrequent updates and message loss, such as dead reckoning, estimate a player’s movements based mainly on previous observations. The estimations are typically made by using dynamics of motion, taking only inertia and some external factors (e.g., gravity, wind) into account while completely ignoring the player’s goals (e.g., chasing other players or collecting objects).

This chapter proposes AntReckoning: a dead reckoning algorithm, inspired from ant colonies, which models the players’ interests to predict their movements. AntReckoning incorporates a player’s interest in speciﬁc locations, objects, and avatars in the equations of motion in the form of attraction forces. In practice, these points of interest generate pheromones, which spread and fade in the game world, and are a source of attraction.

To motivate and validate our approach we collected traces from Quake III. We conducted speciﬁc experiments that demonstrate the effect of game-related goals, map fea-

117 5.1 Introduction to AntReckoning tures, objects, and other players on the mobility of avatars. Our simulations using traces from Quake III and World of Warcraft show that AntReckoning improves the accuracy by up to 44% over traditional dead reckoning techniques and can decrease the upload bandwidth by up to 32%.

5.1 Introduction to AntReckoning

We have already discussed in this thesis that position update messages account for the largest portion of the network trafﬁc in multiplayer online games, raising signiﬁcant bandwidth issues and hindering scalability. As a result, major online multiplayer gaming platforms are forced to develop a range of techniques to increase their scalability. One of these techniques is to accurately predict player movements in order to reduce the update rate, while keeping the error on player position low.

Traditionally, the current position of an avatar is estimated from previous positions. Only when the error is higher than a threshold a new position update is sent, thus reducing the update rate [Fuj00, ABK`04]. Upon reception of a new update, a convergence step is performed to hide the estimation errors from the player in rendering the motion [Mur11]. Such techniques also help cope with message loss by extrapolating the new position when the new update is not received.

Dead reckoning estimates the position of an object from the equations of motion, based on previous observations. It has been successfully used in a number of areas including distributed simulations [Fuj00, MWMD07b], games [ABK`04, Mur11] and aviation. Al- though the performance of dead reckoning, in its current form, is good enough for vehi- cles moving smoothly [BP98], it may degrade to an unacceptable degree in games where players, driven by their short-term environment-related goals, make frequent and sudden changes to their movements. A typical example of this is a wounded player, moving in a given direction, with both an attacker shooting at him and a health pack in his vision ﬁeld: he tends to maintain the same motion (because of inertia), while trying to move towards the health pack and evade from the attacker. As games generally have relaxed physical rules, sudden drastic changes in movements (e.g., U-turn) can occur. These unpredictable changes dramatically reduce the performance of dead reckoning as it only takes mechanics

118 5.1 Introduction to AntReckoning into account.

Inspired by this example, we argue that key factors in an avatar’s motion are not only inertia but also the player’s interests, speciﬁcally the objectives of the game, as well as entities in his vicinity that we call points of interest. Following this line of reasoning, we propose AntReckoning. To the best of our knowledge it is the ﬁrst approach to use interest modeling for dead reckoning. The main concepts involved in AntReckoning are as follows:

‚ Each entity is assigned a given attractiveness leading to the generation of pheromones that spread in the game world and fade over time; ‚ Pheromones in the vicinity of an avatar exert attraction on it. Attraction is integrated in the equations of motion, under the form of forces, to estimate the avatar’s future position.

The main contributions of AntReckoning are (1) to incorporate player interest into the equation of motion used for dead reckoning, and (2) to provide a framework for interest modeling and to use pheromones which take temporal and spatial aspects of players’ interest into account. In addition, pheromones offer a practical solution to the decentralized implementation of interest based dead reckoning. We motivate by showing the usefulness of our interest based dead reckoning by using traces from Quake III. We demonstrate that game objects and map features have a measurable impact on the player mobility patterns and, not surprisingly, so do player interactions.

In addition, we evaluate our AntReckoning algorithm by using traces of World of War- craft and Quake III, providing a basis to fine tune AntReckoning’s game and player-specific parameters. Our simulation results show that AntReckoning, if properly configured for the game, consistently outperforms dead reckoning, and improves the average accuracy of the estimation by up to 44% over traditional dead reckoning. As AntReckoning involves several parameters, we identify the key ones and perform a thorough sensitivity analysis to evaluate their respective effect on the accuracy. We also discuss solutions to set game-related parameters, such as the attractiveness of objects, and practical implementation aspects.

119 5.2 Background on Dead-Reckoning

5.2 Background on Dead-Reckoning

As discussed, dead reckoning exploits information contained in the last state updates to extrapolate the time-dependent future state of entities. The applications of dead reckoning can typically be divided into two categories: (1) enabling less frequent exchange of state updates by issuing an update only when the prediction error is higher than an acceptable threshold (this is called threshold-based dead reckoning); (2) helping cope with loss or jitter when frequent update messages are sent at a ﬁxed rate. Therefore, a typical dead reckoning problem is the estimation of the position of a moving entity, which is required for rendering the virtual world at the clients, between two successive updates. In this situation, the key variables are the kinematic variables: the entity’s last position ~xt, its velocity ~vt “ ~x9 t, and possibly its acceleration ~at “ ~x:t (as deﬁned by the IEEE Standard for Distributed Inter- active Simulation [IEE96]), where t represents time. Extra information, that helps estimate the forces the entity is subjected to, can also be included. For objects extended in three- dimensional space, the kinetic state includes orientation and angular velocity as well (and possibly the angular acceleration).

The study of the trajectory of objects relies on the dynamic equations of motion, and more speciﬁcally, on Newton’s second law, which links the acceleration ~at of an object, its mass m and the forces f~ it is subject to:

1 ~a “ f~ . (5.1) t m ÿ When a closed form expression of the (sum of the) forces is known, the ordinary differential equation characterizing the trajectory of the object can be obtained from this relation, and formally or numerically solved. When the mass is constant and in the case where the forces are determined by the entity’s position and external factors such as wind and gravity, the future position of the object is fully determined by its initial position and velocity

In practice, a polynomial approximation derived from the Taylor series expansion of the position, as a function of time, is used to predict the position in a near future. For instance,

120 5.3 Motivation and Design Rationale for AntReckoning the second-order polynomial predictor is given by

1 ~x “ ~x ` ~v δt ` ~a δt2 (5.2) t`δt t t 2 t

Note that such predictors are accurate only for small values of δt (compared to the speed at which players move and change their direction). It has been shown that using derivatives of orders higher than two usually results in a negligible improvement in the prediction [SZ99, PW02a]. As a result, the use of ﬁrst and second order derivatives is usually preferred, and estimating the velocity and acceleration and sending them with the current position is sufﬁcient for short-term dead reckoning.

Estimating velocity and acceleration is commonly done from previous observations using an exponential moving average (EMA). In short, EMA estimates the velocity by a weighted sum of its current instantaneous value, speciﬁcally the difference between the current position ~xt and the last position ~xt´δt divided by the time interval δt, and the last estimation:

~x ´ ~x ~v “ α t t´δt ` p1 ´ α q ~v (5.3) t v δt v t´δt ~v ´ ~v ~a “ α t t´δt ` p1 ´ α q ~a (5.4) t a δt a t´δt

In other terms, the estimate of the velocity is a weighted sum of the current and previous values of the instantaneous velocity. The weights of the previous values decrease expo- nentially with time. Such an approach has proved to have a beneﬁcial smoothing effect [Bro04].

5.3 Motivation and Design Rationale for AntReckoning

When using a ﬁrst order predictor, the velocity of the avatars is usually estimated from the short-term history of their states, thus, taking into account only their inertia in the prediction. To increase the accuracy of dead reckoning, one needs to estimate the forces the avatar is and will be subject to and incorporate them in the second derivative (i.e., the acceleration in the equations of motion).

121 5.3 Motivation and Design Rationale for AntReckoning

strategic spot b other avatar health pack

weapon

ammunitions health

Figure 5.1: Screenshots of a Quake III game in the q3dm01 map. Objects, avatars and map features in the vicinity of an avatar play an important role in the decision made by the player and thus affect the way he moves.

Estimating the forces an avatar is subject to is a difficult task since it depends on the player’s decisions: For instance, a player can suddenly accelerate to have his avatar in the game world chase another avatar. The intuition behind this reasoning is that a player is more likely to follow another player or to go to pick up a game item (e.g., a weapon) rather than just continuing on its current path. The key here is the fact that players’ moves are usually driven by specific goals and interests that are themselves related to features of the virtual world (see Figure 5.1). Indeed, in games such as World of Warcraft, players are interested in specific locations, certain objects, and other avatars, namely points of interest (POI).

To support this claim a number of experiments were done using Quake III traces in q3dm01 map. The q3dm01 map is spread over a single level and it is composed of two main rooms connected by two crossing corridors. A third corridor on the south of the southernmost room leads to a cave (dead-end). Objects are disseminated on the map, including a powerful weapon in the center of each room and a body armor in the cave. All objects disappear when picked up and reappear at the same location a few seconds later. When killed, players respawn at one of the so-called respawn spots. Figure 5.2 depicts the map with the different objects it contains and the respawn spots.

We conducted two sets of experiments. The ﬁrst set of experiments examines the effect of game items (weapons, ammunitions, etc.) and map features (walls, corridors, corners,

122 5.3 Motivation and Design Rationale for AntReckoning

Health packs Ammunitions Weapons Armors Respawn spots

Figure 5.2: The q3dm01 map from Quake III. The map is spread over a single level and it is composed of two main rooms connected by two crossing corridors. A third corridor leads to a cave (dead-end). etc.) on the players’ behavior, whereas the latter examines the effects of interactions between players on their behavior. Both are crucial to model player’s interest and should be taken into account in a behavior prediction algorithm.

Game Items and Map Features: In order to demonstrate the effect of players’ interest in game items, two experiments were conducted in which the locations of interesting items (e.g., weapons, health packs, ammunitions) was changed. Figure 5.3 depicts the concentration of player movements in the game world, where darker regions are regions more populated by the players during the gaming session. Based on Figure 5.3, a number of observations can be made:

‚ Some places and paths on the map are popular due to their strategic advantage in the game, regardless of the location of the game items. These spots are advantageous because they provide a better cover, vantage points, or shortcuts in the game due to the map design. ‚ Changing the location of popular game items results in dramatic changes in the popularity of different regions in the game world. In Figure 5.3, players’ presence in lower map parts (i.e., two lower corners of the southernmost room and the corridor leading to the cave) is almost diminished after removing interesting items (i.e., ammunitions, health packs and the body armor). This shows that player’s interest, and by extension their behavior and

123 5.3 Motivation and Design Rationale for AntReckoning

Health packs Ammunitions Weapons Armors Respawn spots

Figure 5.3: Presence of players in the q3dm01 map from Quake III with different entity positions (logarithmic grayscale colormap). The presence data was extracted from a 16- player game trace of 10 minutes.

their mobility, is affected by game items: when game items’ locations are changed, new hotspots emerge and some regions become less populated. ‚ Popular player paths change as the game item locations are changed. This is due to the fact that players tend to choose the shortest and safest paths to items of interest. As a result, when item locations change, players’ movement patterns also change. Examples of such path changes can be observed in different regions in Figure 5.3: The lower right and left corridors are equally attractive when the left corner of the southernmost room contains ammunitions; However when all the items in the second room are gathered in the centers, the right corridor is preferred.

Player Interactions: Player interactions also affect players in the game world. In order to evaluate their effect, the following measurements were performed: When an interaction, in which player P shoots player Q, occurs at time t, we record the position of both players, as well as the time elapsed since Q last shot P , referred to interaction recency. We then look at the difference between the distance between P and Q’s positions at time t and the distance between P ’s position at time t ` δt and Q’s position at time t. If this difference is positive, it means that P tried to get closer to Q while shooting at him. We aggregate these points by time bins of one second and compute, for each bin, the probability that P tries

124 5.3 Motivation and Design Rationale for AntReckoning

50%

40%

30%

20%

10%

prob. that the attacker gets closer 0% to the target in next seconds (in %) 0 < t < 1 1 < t < 2 2 < t < 3 3 < t < 4 t > 4 time t elapsed since target shot the attacker (in seconds)

Figure 5.4: Effect of player interactions on their behavior. When engaged in a fight, players tend to get away from each other (escaping or shooting while moving backward). However, when not being shot at back, players moves towards their target more often. to get closer to Q. Results are depicted in Figure 5.4 as a function of time elapsed since the interaction happened. We observe that players in a firefight usually get away from each other as they want to get out of each others line of fire. However, they are less likely to do so when the player they are shooting at is not shooting back at them. The rationale is that by getting close to a target that is not responding they are able to better target them. However, when being shot back by the other player, players tend to move to a more defensive stance. This is typical of most first-person shooter death match games. In other types of games, however, player interactions might have different effects. For example, friends playing in teams completing a mission together will not run away from each other and may move together to carry on different game tasks.

To sum up, a successful interest-modeling algorithm should handle the following situations: (1) Game items attract players. This is especially true if they are valuable or when the player is in urgent need of them. For example, if a player comes across a powerful weapon, he will most likely move towards it and pick it up. Similarly, players running out of ammunitions or who are wounded would pick up ammunitions or health packs. Attrac- tion therefore depends on both the objects of interest and the state of the moving avatar. (2) Players are attracted by the avatars they are chasing or they want to trade with, and

125 5.4 The AntReckoning Algorithm repulsed by the avatars that are chasing them. These can be determined using interaction history between players, their teams, their social network (i.e. friendship relations) and the nature of the game they are playing. (3) Interesting and popular locations (e.g., top of a hill, corners, etc.) in the game, namely hotspots, are sources of attraction. Such attraction points can be inferred from the history of the movements of all players and/or the map design. (4) Players are repulsed by some game locations and map items, e.g. locations where players are endangered, or become under attack by map items such as automatic game-controlled towers.

In order to use the traditional framework of dynamics while considering game strategy for predicting avatar movement, we incorporate player interest in the second order predictor in the form of attraction/repulsion forces. The intensity of the forces exerted by POIs on the avatars depends on their attractiveness and can be determined or learned.

5.4 The AntReckoning Algorithm

AntReckoning implements a scalable algorithm based on pheromones to model players’ interests in a lightweight and efﬁcient way. This model is used to improve the accuracy of position predictions made by the dead reckoning algorithm.

In AntReckoning, points of interest are treated as ants that generate pheromones modeling their relative attractiveness. Pheromones are chemicals (which concentration is coded by a ﬂoating point number) that exert attraction forces on players, integrated in the second order predictor. They spread in the game world, and fade over time, therefore capturing the geometrical and temporal aspects of interest. Throughout this chapter, we use the example depicted in Figure 5.5 to illustrate the different mechanisms involved in AntReckoning. Table 5.1 (located on page 134 at the end of this chapter) summarizes AntReckoning’s parameters together with a brief description and the values used in the evaluation. We discuss how to tune these parameters in Section 5.5.

Model Consider a game evolving in discrete event loops called frames. In each frame each player needs to know the positions of other avatars, which he receives through position updates. Consider player Q who seeks to estimate the position ~xt`δt of the avatar of player

P in frame t ` δt while the last update received contains the position ~xt (and possibly the

126 5.4 The AntReckoning Algorithm

estimated velocity ~vt and acceleration ~at) of P in frame t.

Dead Reckoning AntReckoning makes use of a second order predictor where the second order term is a weighted sum of the acceleration of the avatar and the attraction forces. The estimated position therefore writes:

1 1 2 ~xt δt “ ~xt ` ~vt δt ` α F~t ` p1 ´ αq ~at δt , (5.5) ` 2 ˆ m ˙

where δt is the number of frames elapsed since the last position update, F~t is the sum of the

attraction forces exerted by pheromones on P and other forces (e.g., gravity), and ~vt (resp.

~at) is the estimated velocity (resp. acceleration) of P . In AntReckoning, the estimation of velocity and acceleration is performed from previous observations by using EMA, as

described in Section 5.2, with parameters αv and αa.

Figure 5.5 illustrates the estimation of the position of P for the next frame using the

current instantaneous velocity alone (i.e., αv “ 1 in Equation 5.3) and attraction forces alone (i.e., α “ 1 in Equation 5.5). For a player P and for each cell in its attraction region with non-zero pheromone values, attraction forces on the player are computed. Each attraction force (i.e., dashed vectors), corresponds to the direction of the cell, the amount of pheromone in that cell, and the distance to the cell. These attraction vectors are then summed up and added to other physical forces the player is subject to. The ﬁnal force vector then creates acceleration that is added to the current speed of the player: ~v.

Pheromones As common to most games, AntReckoning assumes a game world divided into non-overlapping cells, e.g., Delaunay triangulation, Voronoi tessellation, binary space partitioning, or regular grids (e.g., square grid in Figure 5.5) typically used for tasks such as path ﬁnding, collision detection, or graphical rendering. We denote by C the size of a cell in game world unit. The management of pheromones and the computation of attraction forces exerted by them is performed at the granularity of a cell: for each avatar P for which Q performs dead reckoning (P “ Q is also possible as will be described later), Q computes the concentration of pheromone (represented in grayscale in Figure 5.5) in each cell and computes and sums the corresponding attraction forces. For the sake of scalability, only the cells in a limited region around P , called the attraction region and denoted by R,

127 5.4 The AntReckoning Algorithm

dead reckoned avatar (P ) other avatar object of interest v 1 Σf P 2m predicted position past trajectory attraction forces

Figure 5.5: Overview of AntReckoning: the game world is divided into cells by using a regular square grid. Each cell contains a certain amount of pheromone, represented here in grayscale. To estimate the current position of P , one adds (1) his velocity, estimated from his past trajectory, and (2) the sum of the attraction/repulsion forces (divided by his mass) generated by cells inside a square region around him (called attraction region), to the position of P in the last frame. Attraction forces are directed towards the attracting cells and their intensity is proportional to the amount of pheromone they contain. are considered, e.g., a fixed-size square represented with dashed lines in Figure 5.5 (Note, however, that other shapes may be considered for the attraction region, such as a cone reflecting the player’s vision field as discussed in Section 5.5.) As pheromones spread, even points of interest outside R are taken into account.

For each P for which player Q has to perform dead reckoning, the concentration of pheromone inside a cell that is part of the attraction region R of P is calculated as follows:

‚ Generation: In each frame, each point of interest within a cell, be it an avatar or an object, generates a given amount of pheromone related to its attractiveness to P . Attrac- tiveness is a function of the characteristics of the object and possibly the current state of the considered avatar (as in the wounded player example). This amount is added to the

concentration of the cell. The maximum concentration of a cell can be capped (phmax) to limit the attractiveness of any single cell at a given frame. ‚ Evaporation: In order to limit in time the attraction of previous positions of points of interest, pheromones fade in time, meaning that their concentration is decreased at the beginning of each frame. Exponential decays, i.e., removing a ﬁxed percentage of the

128 5.4 The AntReckoning Algorithm

old pheromones at the beginning of each frame, have been successfully used in previous work on ant colonies (e.g., Max-Min ant colonies [SH00]). Beyond its simplicity and its effectiveness, such an evaporation model ensures that the total amount of pheromones in the game world does not grow to inﬁnity over time. Using an evaporation factor of 1 (i.e., pheromones do not fade) gives a pheromone map similar to the presence map depicted in Figure 5.3, which captures popular locations and paths but disregards the dynamic environment of the game. In case objects always respawn on the same location, such a map would also capture object attraction. However, it would do so even if the object has not respawn yet. Using an evaporation factor of 0 (i.e., pheromones entirely fade in one frame), on the other hand, captures only the interest in surrounding objects and players at time t. ‚ Dissemination: As pheromones spread, the concentration of pheromone in neighboring cells are mutually dependent. At the beginning of each frame (after the evaporation step), a given amount of pheromone is simultaneously removed from each cell and evenly dispatched to its neighboring cells. The size and shape of this neighborhood depend respectively on the predetermined speed of pheromones dissemination and the game world topology (e.g., wall, hills, etc.). For example, obstacles may block the dissemination of pheromones to avoid attraction to unreachable areas.

These phenomena are captured by the following recursive expression of the concentration of pheromone in a cell, for a given player P , at frame t:

evaporation generation

ph pcellq “ ε ph pcellq ` attractivenesspentity,P q ` t hkkkkkkikkkkkkjt´δt hkkkkkkkkkkkkkkkkikkkkkkkkkkkkkkkkj entityÿPcell ε ¨ γ ph pcq ´ε ¨ γ ph pcellq , (5.6) |N (c)| t´δt t´δt cPNÿ(cell)

incoming dissemination outgoing dissemination loooooooooooomoooooooooooon loooooooomoooooooon where ε is the evaporation factor (percentage of pheromones that remain after evaporation), γ is the dissemination factor (percentage of pheromones that spread in the neighboring cells), and N p¨q is the set of a cell’s neighboring cells. The attractiveness of a player to itself is set to zero. These phenomena can be observed in Figure 5.5 around the trajectory of

129 5.4 The AntReckoning Algorithm a moving avatar: some pheromones remain and some spread around its previous positions; all pheromones fade.

To better understand the evolution of the concentration of pheromone described by Equation (5.6), consider the central cell c in the simpliﬁed example depicted in Figure 5.6. Cell c contains an object (depicted with a triangle), its neighborhood N is composed of the four adjacent cells, and its current concentration of pheromone is 32. Assuming an evaporation factor ε of 0.5 the concentration is ﬁrst reduced to 16. Considering a dissemination factor γ of 0.5, another 8 pheromones are then removed and evenly dispatched to the four neighboring cells (i.e., 2 pheromones each). As a result of pheromone dissemination from the neighboring cells, cell c receives a total of 1 ` 4 ` 2 ` 4 “ 11 incoming pheromones. Finally the pheromones generated by the object in c, say 5, are added to its concentration yielding a total of 16 ´ 8 ` 11 ` 5 “ 24 pheromones in cell c at the next frame.

32 u object of interest 2 u 64 32 64 dissemination of pheromones 4 dissemination neighborhood 16

Figure 5.6: Illustrative example of the evolution of the concentration of pheromone inside a cell with an evaporation factor of ε “ 0.5 and a dissemination factor of γ “ 0.5: half of the pheromones are removed due to evaporation and half of the remaining pheromones are evenly dispatched to the four neighboring cells. In addition, the object of interest lying in the cell generates pheromones and incoming pheromones disseminate from the neighboring cells.

To further improve AntReckoning’s performance, concentrations of pheromone lower than a given threshold are ignored, as their attraction power is negligible.

The recursive equation 5.6 is linear. One can compute the equation separately and then sum up the pheromone maps corresponding to each static point of interest. More speciﬁ- cally, for static objects, the dissemination/evaporation pattern converges in time. Its limit depends only on ε and γ and can be determined formally or estimated by simulations. The

130 5.4 The AntReckoning Algorithm results can be incorporated into the game or the map to simplify pheromone generation. Interestingly enough, the limit has a spatially limited support: Dissemination remains localized, which limits the complexity of the computation. The pheromone map corresponding to a set of static points of interest is the sum of the respective dissemination patterns centered on the points of interest and multiplied by their attractiveness.

Attraction In physics, attraction forces between two bodies are generally directed along the line connecting them, and their intensity is a decreasing function of the distance between them. In the case of spring attraction the intensity of the force is inversely proportional to the distance between the two bodies. However, in gravitational and electromagnetic attractions the force is inversely proportional to the square of the distance. In AntReckoning, the attraction force exerted by a cell on an avatar is directed along the line that connects the position of the avatar, i.e., ~xt, to the center of the cell. The intensity of the attraction force is proportional to the concentration of pheromone in the cell divided by the distance raised to a certain power: ~ phtpcellq }ftpcell, ~xtq} “ k , (5.7) d pcell, ~xtq where k is a parameter of the system. Attraction forces of various intensities originating from P and directed towards cells containing pheromones can be observed in Figure 5.5.

Throughout this chapter, we considered solely players attracted by objects and other players. However, repulsion of players by one another, as described in the motivation Chap- ter (Section 5.3, Figure 5.4), can easily be incorporated into the force model of AntReck- oning: making repulsive objects (e.g., time bomb) or avatars (e.g., an attacker) generate pheromones with negative values would result in repulsive forces moving P away from them in the predictions.

Post-Processing In order to improve the predictions and make them consistent with the game physiques, a number of post-processing checks are performed. In these checks the predicted position is corrected to take into account the game physics and the game map as well as other limiting factors: As the speed of avatars is bounded, the predicted position should remain within a distance vmax ¨ δt of the last known position.

The current speed of the player can also be taken into account to ensure that the attrac-

131 5.5 Parametrization and Implementation

+ + vmax ·δt × wall past trajectory 1 Σf 2m r dead reckoned avatar (P ) r v r P P × + predicted position × predicted position (corrected) path finding (a) (b) Figure 5.7: Prediction correction: The predicted position of player P , based on his position and on external forces, violates the game physics, here (a) the speed limitation and (b) the map reachability. It is corrected as follows: for (a), the predicted direction is preserved but the distance is adjusted to the maximum possible value vmax ¨ δt. For (b), a path-finding algorithm is used to determine the path from the current position of the avatar to the predicted position. If the predicted path is longer than vmax ¨δt, the predicted position is placed at a distance vmax ¨ δt to the current position of the avatar, along the path. tion forces do not cause a fast change in a player movement when the player is standing still. Similarly, a reachability map is used to correct the final prediction for the player’s position to take into account the topology of the map (i.e., a player cannot move beyond a wall but may be attracted to a player that has just moved there). Figure 5.7a illustrates a basic correction technique to ensure that the predicted position is consistent with the speed limitations (we used this technique in our experiments): the circle represents the acceptable range of a player’s movement. If the predicted position falls outside the area of possible movement or if the predicted path crosses an obstacle, then the predicted direction is preserved but the predicted position is changed to be the intersection between the path and the boundaries or the obstacle, if any. Another possible approach is to use a path finding algorithm to move around the obstacles towards the destination as illustrated in Figure 5.7b. In case a player is standing still we ignore attraction forces completely or greatly reduce their influence. If the attraction forces produce a very large vector while the speed vector is relatively small, we limit the size of the final vector to a factor of the speed vector size.

5.5 Parametrization and Implementation

We discuss practical considerations about AntReckoning, more speciﬁcally the tuning of its parameters and its implementation in a decentralized setting with partial knowledge.

132 5.5 Parametrization and Implementation

5.5.1 Parametrization

The deﬁning parameters of AntReckoning are as follows:

Attractiveness of points of interest can be divided into three categories: (1) game objects (2) players and (3) locations.

‚ Game objects: Most games are able to define the attractiveness of the objects based on their value or power. In addition, the attractiveness of objects can be defined as a function of key factors in the player’s current state, estimated from the analysis of game play traces. The amount of pheromones generated by an item then depends on its attractiveness. For instance, in our experiments we defined the attractiveness of health packs in Quake III as a function of the avatar’s health. We did so in our implementation by experimentally estimating the probability that a player with a given health level, having a health pack in his sight, picks it up within the next δt seconds. ‚ Players: Attractiveness of players for one another can be based on their recent interactions, e.g., trading or fighting, and built into the equations as follows: The attractiveness

of a player Q to a player P is a function of the time tP,P 1 elapsed since their last interaction. In Quake III, given the results from Figure 5.4 a linearly decreasing factor takes the time of the interaction and the pace of the game into account. This means the player generates less negative pheromone for their targets if they have not shot back. The type of interaction (e.g., shooting or being shot at, trading, chatting, etc.) as well as their team can determine the sign of attractiveness, i.e., attraction or repulsion. Other factors such as the items a player is carrying, e.g., ﬂag, can be incorporated into the attraction as well. In our experiments, we consider the interaction history between players to determine the sign and the amount of pheromone generated by players. ‚ Locations: Some locations in the game offer an advantageous position for players. These regions are not only popular because of the game items that exist in them but are attractive themselves. Examples of such locations are the top of a hill, which provides a good vantage point, or behind a wall that provides good cover. In order to improve the accuracy of predictions, these locations are assigned a pheromone generator that attracts, or in case of unpopular locations, repulses players by generating pheromones. Hotspots can be determined by either the game designers or through a trace analysis and heat maps, for ex-

133 5.5 Parametrization and Implementation

ample by looking at the number of players passing through a given location (as in Figure 5.3) or the number of kills/deaths that occurred at a given location to determine popular and unpopular regions. These measurements can be limited to the past few minutes of the games to capture the dynamic nature of hotspots.

Table 5.1: Important parameters in AntReckoning. Parameter Description Value δ # of frames since last position update variable α weight coef. acceleration v.s. forces 0.5 αv weight coef. in EMA of velocity 0.8 αa weight coef. in EMA of acceleration 0.8 R region for attraction variable ε evaporation fact. (% remaining) variable γ dissemination fact. (% disseminating) variable k decreasing power of attraction forces 2 attractiveness(¨) attractiveness of avatars and objects variable ρ base amount pheromone generated 40 phmax maximum pheromone in a cell 100 C cell size variable η prediction threshold variable

Mass modulates the effect of attraction forces on avatars. Avatars with higher masses are less subject to attraction than others. Different mass levels can be used to capture the relative attraction of avatars by objects: For instance, heroes may move only to achieve important goals—and should therefore be assigned a high mass—, whereas regular units that move to achieve secondary goals (e.g., collect resources) should be assigned small masses. In addition, slow characters are assigned a higher mass whereas faster characters receive a lower mass to take into account their movement capabilities in the way attraction and repulsion forces affect their speed. Note that the scale of mass is proportional to that of attractiveness: doubling the mass of all avatars is equivalent to halving the attractiveness of all objects. Given that in Quake (and other similar games) most players are equal in their capabilities, we assign the same mass to all players in our implementation. However, in games like Warcraft players are assigned levels and experience points and different units have different moving and ﬁghting capabilities, thus helping to assign different masses to them.

134 5.5 Parametrization and Implementation

Region Sizes: The size of the attraction region, in game world distance units, and the size of each cell affects the accuracy of predictions. Larger attraction regions take into account farther objects, and smaller cells compute the direction of attraction forces at a ﬁner granularity, thus providing better accuracy. This, however, comes at the cost of computational and memory overheads. Note that as attraction decreases rapidly with the distance, increasing the size of the attraction region beyond a certain point may bring only a negligible improvement. Therefore, in case of a large area of interest, the size of the attraction region can be limited based on the phmax that can be left on a single cell, and how pheromones force is reduced as a function distance which is modeled by the parameter k. We experiment with different region sizes in our evaluation and k is set to 2 (see Table 5.1).

Vision field: In games where players see the world through the eyes of their avatars, the players’ vision field should be taken into account in the attraction region. The rationale is that players would be more attracted by objects they can actually see, which depends on their vision field and on the world map (e.g., walls). The vision field of players’ depends on the viewing vector (i.e., a spherical cone with angle 45o around the player’s viewing vector in Quake III) and on the visibility information typically stored in the game map files (bsp files in Quake III). The region of attraction can be extended beyond the vision field to cope with players or objects suddenly entering a player’s vision field. Our implementation of AntReckoning takes the players’ vision fields into account and we experiment with this parameter.

Finally, taking into account the game map for dissemination and attraction prevents avatars from being attracted by objects they cannot reach, e.g., behind an obstacle.

5.5.2 Discussion & Implementation

In order to produce pheromone maps, a player needs to be aware of the players and objects in its vicinity. This is addressed by interest management.

Interest management handles which game objects (and their corresponding updates) should be received by the player, based on his location in the game world. This information is necessary to render the game world and are received from the server, other players, or extracted from the map. In some games (e.g., Quake III) the game world is relatively small and the

135 5.5 Parametrization and Implementation area of interest can be the whole or all the visible portions of the game world. The player receives updates about all (potentially) visible players and objects in the game. In such a case, AntReckoning can easily calculate the necessary pheromone maps for the game. In games like World of Warcraft where the player receives new game objects as he explores the game world, a pheromone map for the player’s area of interest is created. Items are taken into account and start generating pheromones as soon as they enter the area of interest. Over time, evaporation and dissemination remove the effect of the items no longer present in the area of interest. As a result, maintaining pheromone maps in both types of games comes at no additional network cost and is possible with the locally stored information.

Distributed implementation As discussed in Section 5.2, we divide the use of dead reckoning into two categories and discuss the implementation details of each form: (1) akin to normal dead reckoning, all players perform dead reckoning for all other players in their area of interest. In this case, all players, including the owner of the avatar, run the exact same algorithm for calculating pheromones with the exact same information. Through this, every player can detect when other players’ prediction errors become higher than the acceptable threshold and send an update. (2) players perform dead reckoning only for themselves and include their prediction (a prediction vector) in the update sent to other players. In this case, other players do not have to perform dead reckoning and the player sends an update when he detects that the prediction he sent now has an error above the threshold. Note that AntReckoning can work both in centralized and distributed (Peer-to-Peer such as Donny- brook [BDL`08]) settings. In a centralized architecture, the server is a primary copy holder for all players’ avatars and performs all the required task. The rest of algorithm is the same as in a distributed architecture.

There exist three possible implementations of AntReckoning. Two implementations are for the ﬁrst type of AntReckoning in which players all perform dead reckoning. One of the two uses a higher number of pheromone maps and yields better accuracy than the other. The second type of AntReckoning has an implementation which yields better accuracy than both implementations of the ﬁrst type.

1. Two possible implementations exist for the ﬁrst type of dead reckoning: (1) maintain a different pheromone map for each player in the game world taking their interaction

136 5.5 Parametrization and Implementation

history, vision field, status, etc. into account. This scheme provides accurate predictions. However, it requires access to the interaction history of other players and information about their area of interest typical of games where updates are sent to everyone or area of interest is large enough (e.g., AOI is at least twice the size of attraction region). It also has a high overhead as a different pheromone map has to be maintained for each player. (2) maintain a single pheromone map for the area of interest. In this case, interaction history and similar information are ignored: only general information about points of interest is used to generate pheromone maps. Thus, the cost of generating pheromone maps is greatly reduced as only one map is maintained. But this comes at the price of lower accuracy in the predictions. 2. The second type of dead reckoning yields better accuracy and lower overhead in comparison to both implementations of the first type. In this scheme the player only maintains one pheromone map for itself, taking the interaction history, vision field, mass, and its status into account. Since the player has access to much more information about himself than other players, the pheromone map can be more accurately calculated. In addition, as only one pheromone map is maintained, the computational and memory overheads are minimal. The player will include his prediction vector in the updates sent to others.

Memory Overhead: The only memory overhead of AntReckoning is the maintenance of the pheromone maps. However, it can effectively be reduced to maintaining a single pheromone map and only for the player’s area of interest, meaning the size of the pheromone map would be small. In addition, it is possible to use the built in tessellation mechanisms in games to maintain the pheromone map. In this case, the only overhead is for maintaining a single ﬂoating point number for each already existing cell. As the size of the pheromone map is a function of the size of area of interest and the size of each individual cell (|R|{C), it does not grow as the size of the game world becomes larger, and only requires a ﬁxed amount of memory independent from the number of players and from time.

Computational Overhead: The computational overhead of AntReckoning is limited to pheromone processing and computing the attraction forces. During the generation of pheromones

137 5.6 AntReckoning Evaluation one needs to go through all the points of interest inside the area of interest. However, most games already perform such a loop of the items in each frame and execute the think function for each item. The think function is used for updating an item’s status in the game [BDL`08]. Therefore, the only overhead is the generation of the corresponding pheromone amount for the item during the execution of its think function. Calculating the attraction forces is a function of number of cells inside the attraction region and does not grow with the number of items inside the area of interest or the size of the map.

Note that an alternative interest-based dead reckoning algorithm in which the precise current and past positions of all items are stored and taken into account for calculating the attraction forces, is also possible. However, such an approach would result in substantial memory and computational overhead. One of the main beneﬁts of using pheromone-based interest modeling is that it enables us to keep a history of movements and interactions inside the game world in a low overhead manner.

Setting the Parameters: Finding the optimal parameter values would require some experimentation as shown in the next chapter. However, we believe that it would be fairly simple to set parameters in a way that, while not optimal, would still offer a better performance compared to simple dead reckoning. For example, the parameters can be adjusted based on the type of game, e.g., for a fast-paced FPS game a relatively higher evaporation rate can be used. For better results, a game developer can easily run a few experiments and include the parameter information inside the game. Furthermore, for the optimal values which may depend on the map, the map ﬁle itself can contain the optimal values for that particular map when it is shipped to players.

5.6 AntReckoning Evaluation

The goal of the evaluation is two-fold: (1) Compare AntReckoning to traditional dead reckoning in order to validate our approach and estimate the gains it conveys. We compare with respect to the accuracy of the extrapolation when updates are sent at a ﬁxed rate and with respect to the bandwidth usage when position updates are sent only when the prediction error grows above a given threshold; (2) Conduct a sensitivity analysis of AntReckoning to identify its key parameters and evaluate their individual effects on the performance.

138 5.6 AntReckoning Evaluation

xt+δt

xAR past trajectory actual positions predicted positions xDR xt xt−δt

Figure 5.8: Illustration of the metric used in the evaluation of AntReckoning: the relative improvement over traditional dead reckoning is deﬁned as the ratio of the prediction errors dpxAR, xt`δtq{dpxDR, xt`δtq, where d denotes the Euclidean distance.

5.6.1 Experimental setup

We evaluated AntReckoning by using traces collected from Quake III and World of War- craft. The first consists of 16 players involved in a 10-minute death match in the q3dm01 map. All the 16 players remained connected during the entire time of the game. The trace includes players’ positions, items in their possession (e.g., weapons and ammunitions), their state (e.g., health, armor, speed, viewing angle), and their interactions (e.g., shooting, killing) in each frame. In addition, it contains information about items available in the game world, in each frame, and the players’ item pickups. Map information (e.g., walls) is extracted from the map file (i.e., the bsp file). The second trace, from World of War- craft, contains sparse position information about more than 200 players in the Wintergrasp region, and was obtained from [MC10].

The methodology used for the evaluation is as follows. We divide time in frames and set the players’ positions for each frame as their last position update in the corresponding time interval. We predict the position of players δt frames ahead, with both traditional second-order dead reckoning, i.e., based on both the estimated velocity and acceleration, and AntReckoning. First, we compare their performance with respect to the relative error, knowing the actual position of the players at this frame. More speciﬁcally, we compute the Euclidean distance between the prediction and the actual position, i.e., the error, and look at the ratio between the prediction error of AntReckoning and that of traditional dead reckoning (as illustrated in Figure 5.8). A value of 0.8 for this metric means that AntReck- oning decreases the prediction error by 20% over traditional dead reckoning. That is, values smaller than 1 denote an improvement in accuracy. Next, we implement a threshold-based

139 5.6 AntReckoning Evaluation dead reckoning algorithm, where players send position updates only when the prediction error (i.e., the Euclidean distance between the predicted position and the actual position) grows beyond a given threshold. We compute and compare the respective upload bandwidth consumptions of dead reckoning and AntReckoning.

The implementation of AntReckoning for Quake III is as follows. We focus on predicting the movements of a single player inside the game. The pheromone map is generated based on the game world as observed by this player. Players generate the same initial amount (ρ) of negative pheromones for other players inside the game and game items generate the same initial amount of positive pheromones for other players. The amounts of pheromone generated is then modulated by the players interaction recency as discussed in Section 5.3.

For World of Warcraft, we use a basic version of AntReckoning since no information about the objects and interactions were available in the trace: only players generate pheromones and all players generate the same amount ρ of positive pheromone regardless of the state of the avatar for which dead reckoning is performed. Consequently a single pheromone map can be used for all avatars. We use positive pheromones because players in World of Warcraft tend to get closer to each other in most cases, be they ﬁghting or trading. We assign the same mass value to all players in both games: In Quake III we do so because each player controls a single avatar and all avatars have roughly the same characteristics; In World of Warcraft we do so because of the lack of information on the avatars’ characteristics.

5.6.2 Sensitivity analysis

We conduct our sensitivity analysis along the following parameters: (1) the diameter of the region of attraction, denoted by |R|; (2) the reference attractiveness of players and objects, denoted by λ; (3) the evaporation factor, denoted by ε, note that this is the percentage of pheromone left after evaporation in each frame; (4) the duration of the prediction step, denoted by δt. (5) the pheromone dissemination factor, denoted by γ, which captures the percentage of pheromone that spread to neighboring cells; (6) size of the cells, denoted by C (For comparison, in Quake III, the entire q3dm1 map is of size „1400ˆ2700 game units

140 5.6 AntReckoning Evaluation and an avatar occupies a floor area of 30ˆ30 game units. Translated into real-world length units, assuming that a person occupies a floor area of 0.5ˆ0.5 square meters, a cell of size 50ˆ50 corresponds to „0.8ˆ0.8 square meters). Other parameters were fixed to the values specified in Table 5.1 (page 134). The experimental results for the evaluation of Quake III and World of Warcraft are compiled in Figure 5.9, with respect to the prediction accuracy, when position updates are sent at a fixed rate. The results in this figure are with post processing and visibility turned off to better highlight the effects of different parameters. The graphs show only the relative error: AntReckoning v.s. traditional dead reckoning. Another interesting number is the absolute error of traditional dead reckoning normalized by the distance covered by the avatar between times t and t ` δt, i.e., dpxDR, xt`δtq{dpxt, xt`δtq, using the notations from Figure 5.8. This number only depends on the prediction steps δ: it increases with δ and is generally around 46-110% in our experiments, where an error above 100% means worse prediction than standing still. Therefore, it is both significant and relevant to improve the performance by up to 44% over traditional dead reckoning.

Before studying the graphs, the differences between these two games should be highlighted. In Quake III player movements and interactions happen at a very fast pace. Players’ are mostly interested in and affected by other players in their close vicinity. The game world is relatively small and players mostly have short time goals and their interactions are mostly repulsive (e.g., shooting at each other). In World of Warcraft, players have more attractive interactions (e.g., trade, chat, etc.), the game world is very large, and can be very sparse. Players are interested in a larger region and have long term goals. The pace of the game is relatively slow.

The effect of the size of the attraction region is shown in Figure 5.9a. In World of Warcraft, by increasing the size of the attraction region, a larger number of attraction forces from farther regions are taken into account, therefore, improving the quality of predictions. However, increasing the size beyond a certain point results in negligible performance gains, conﬁrming our intuition. In Quake III however, players are interested only in POIs in their close vicinity and taking distant cells into account slightly decreases the performance. The limited effect of distant cells enables efﬁcient interest-based dead reckoning: by limiting the attraction region to the area of interest (or a part of it), pheromone maps can be computed at no network cost (i.e., with limited information available locally at each player) and with

141 5.6 AntReckoning Evaluation

1.15 Q3 AR/DR Q3 AR/DR Q3Q3 AR/DRAR/DR 1.15 WoW AR/DR WoW AR/DR WoWWoW AR/DRAR/DR 1.05 1.05 0.95 0.95 0.85 0.85 0.75 0.75 relative error relative error 0.65 0.65 0.55 0.55 50 150 250 350 450 0.1 0.5 0.9 1.3 1.7 10 0.2 20 30 0.4 40 0.650 60 0.8 70 size of attraction region (|R|) attractiveness (λ) evaporationcell size factor(C) (ε) (a) (b) (c)

1.15 Q3 AR/DR Q3 AR/DR Q3 AR/DR 1.15 WoW AR/DR WoW AR/DR WoW AR/DR 1.05 1.05 0.95 0.95 0.85 0.85 0.75 0.75 relative error relative error 0.65 0.65 0.55 0.55 20 40 60 80 100 0.2 0.4 0.6 0.8 10 20 30 40 50 60 70 prediction step (δ) dissemination factor (γ) cell size (C) (f) (g) (h) Figure 5.9: Experimental sensitivity analysis of AntReckoning using a 16-player trace from Quake III in the q3dm01 map and a movement trace from World of Warcraft in the Winter- grasp region. A frame corresponds to 50 ms in Quake III and 300 ms in World of Warcraft. The analysis was performed around the point |R| “ 500, λ “ 0.9, ε “ 0.5, δ “ 80, γ “ 0.7, C “ 50 in Quake III and |R| “ 500, λ “ 0.9, ε “ 0.8, δ “ 70, γ “ 0.1, C “ 50 in World of Warcraft.

limited computational and memory overhead.

Figure 5.9b demonstrates the effect of attractiveness: although taking attraction forces into account improves the accuracy, over-estimating their inﬂuence and disregarding inertia decreases the accuracy. This demonstrates that the current speed and movement of the player should indeed be taken into account along with the attraction forces. This is specially acute for short-term predictions as players are mostly inﬂuenced by their inertia, as

142 5.6 AntReckoning Evaluation explained below.

Figure 5.9c shows the effect of evaporation: given the fast pace of Quake III and short term goals of players, leaving a higher percentage of pheromone at each frame decreases the quality of predictions. The evaporation factor controls how much history of the game is taken into account and should reﬂect the pace of the game: lower percentage of pheromone should be kept for fast paced games. As expected, in World of Warcraft, where the pace of the game is slower and players are more motivated by long term goals, evaporation has a different impact: larger values of ε, which characterize a slower evaporation, increase the performance.

Human interactions are an order of magnitude slower than game events. Therefore, in very short periods of time (e.g., 50 ms), an avatar’s movements are mostly a function of his inertia. However, in longer periods (e.g., one second) they follow players’ interests. This fact is illustrated in Figure 5.9f where AntReckoning performs better when prediction is done further into the future. Given that many protocols (e.g., Donnybrook) rely on dead reckoning to increase the delay between sending updates to up to seconds, AntReckoning will be of use. In fast paced games, these improvements might slightly decrease for very long prediction periods as players movements become more affected by interactions, given the rate of interactions, in the future that are not taken into account in the current attraction forces. In slower games (i.e., World of Warcraft) predicting further into the future is possible.

Figure 5.9g demonstrates the effect of the dissemination factor on the quality of predictions. In World of Warcraft given the size of the game world, cells and sparsity of the players, using larger dissemination factors result in less accurate attraction forces being calculated (i.e., attraction vectors not directly pointing towards the point of interest) and lower quality of predictions. In Quake III given the large choice of attraction region and cell size, and the length of the prediction, higher dissemination factors can slightly help in predicting further into the future.

The effect of the cell size is demonstrated in Figure 5.9h. In general, smaller grid cells improve the accuracy of the predictions since they provide a higher accuracy in calculating the direction of the attraction forces. In Quake III, where players interact in smaller game

143 5.6 AntReckoning Evaluation

1.35 1.25 V-P AR/DR V-NP AR/DR 1.15 NV-P AR/DR 1.05 NV-NP AR/DR 0.95 0.85 0.75 relative error 0.65 0.55

20 40 60 80 100 prediction step (δ)

Figure 5.10: Effect of visibility and post-processing on AntReckoning’s performance. The tag V (resp. P) in the legend of the graph means that visibility (resp. post-processing) is used and NV (resp. NP) means it is ignored. The analysis was performed around the non- optimal point |R| “ 500, λ “ 1.1, ε “ 0.6, γ “ 0.6, C “ 50 in Quake III to better show the effects of post-processing and visibility. worlds, and are interested in a small region around them, small cells can greatly improve the quality of predictions. In World of Warcraft, however, where the game world is large and players are sparse, the use of smaller cells conveys only small improvements. Note that |R| using smaller cells comes at the price of higher computational overheads (i.e., C ). The performance gain must therefore be balanced with the overheads.

It should be noted that the optimal sets of parameter values were quite different for Quake III and World of Warcraft showing the importance of ﬁne tuning AntReckoning parameters based on the game characteristics. In World of Warcraft, players are motivated by longer term objectives and interactions happen at a lower pace, highlighting the effects of the evaporation, and dissemination factors, and the length of the prediction step. The game world is large, therefore, highlighting the effects of the attraction region while undermining the effects of the cell size. In Quake III, on the other hand, due to the smaller size of the world, cell size can play an important role while a larger attraction region provides negligible gains. Evaporation and dissemination factors should also be ﬁne tuned according to the fast pace of the game and the size of game world. In both cases the attraction factor should

144 5.6 AntReckoning Evaluation

Input: prediction error threshold: η last time a position was sent: t0 actual position (at time t): ~xt t0 Ð ´8; at each frame (time t) do ˆ ~xt Ð predictionp~xt0 , tq; ˆ if ~xt ´ ~xt ą η then › › › xsendp~›tq; › › t0 Ð t; end end ALGORITHM 1: Threshold-based dead reckoning: a player predicts the position of his avatar for the current frame, based on the last sent position update, sending an update only if the prediction error is larger than a threshold η. be ﬁne tuned to take the prediction length into account: shorter prediction lengths require smaller attraction factors. In addition, we suspect in games such as world of Warcraft, use of different mass values based on the character type can further help improve the accuracy of predictions.

Post processing can further improve quality of prediction in some cases by up to 15- 20% and help stabilize the performance of AntReckoning. Similarly, vision ﬁltering can increase the quality by 10-15%. This is shown in Figure 5.10 where the effect of taking visibility and post-processing into account is shown as a function of the prediction duration (δ). We chose a non-optimal set of parameters to show that post-processing can correct the prediction errors, for example, when the attraction forces are too large due to bad parameter selection. It also improves the performance further for longer periods of prediction as error correction becomes more important for longer periods of time. Similarly, taking visibility into account is more important for capturing longer term goals.

5.6.3 Performance evaluation

We now look at the bandwidth savings provided by AntReckoning in a threshold-based approach. In short, the threshold-based approach operates as follows (see Algorithm 1 for

145 5.6 AntReckoning Evaluation

40 Q3 Improvement 35 30 25 20 15 10 5 Bandwidth savings (%) 0 20 60 100 140 180 Prediction error threshold

Figure 5.11: Performance evaluation of AntReckoning with respect to upload bandwidth consumption in a threshold-based approach. Units are in game world distance. a pseudo-code version): each player predicts the position of his avatar and that of the other avatars. Since players run the same algorithm, they can detect when the distance between their predicted position and the actual position of their avatar is higher than the threshold, and therefore, send a position update message. The bandwidth savings are calculated based on the number of updates sent, for a given threshold, by traditional dead reckoning and by AntReckoning, in several sample Quake III gaming sessions. Results are depicted in Figure 5.11 (the prediction error threshold η is given in game world distance units). It can be observed that higher thresholds are typically considered when a higher prediction length is chosen, and in this situation, AntReckoning conveys signiﬁcant bandwidth savings; up to 32%.

Note that while for lower thresholds the percentage of improvements is smaller, each percentage improvement translates into a higher number of updates not being sent given the fact that at lower thresholds (and prediction lengths) updates are sent at a much higher rate.

While we do not provide a user study of how better predictions improve the quality of game play for players, the effects of these prediction errors, as the inconsistencies perceived by players, have been well studied [CE11, RAM`08, CLC99, PW02a]. Therefore, AntReckoning can be used to both improve the network requirements of games as well as the quality of game play. In addition, it is demonstrated that different types of games can beneﬁt from AntReckoning: even in games such as Quake III where sudden changes in

146 5.7 Discussion movements are common, large improvements can be gained. While we achieved improvements up to 20% for World of Warcraft with information currently available, showing that even a simple implementation with a single pheromone map for all players can provide improvements over dead reckoning, we suspect that given enough information about the game world, games similar to World of Warcraft can beneﬁt more from AntReckoning in comparison to fast paced games due to their characteristics and higher predictability of players actions.

5.7 Discussion

Choosing the threshold: The threshold is a parameter of the algorithm and is used as follows: as soon as the error grows beyond the threshold, an update is sent. In practice, the prediction error threshold is chosen as a trade-off between bandwidth and accuracy of prediction: it should be small enough so that the motion of avatars remains smooth and that the games-play remains fair (e.g., missing the target because the avatar is not displayed at its actual position). Such subjective aspects can only be evaluated through experiments coupled with user studies.

A number of techniques improving the performance of dead reckoning by choosing better thresholds have been proposed. In [CLC99] the threshold above which a new update is sent is adapted to the precision requirements, determined by the relative position of the entities. In [RAM`08], the metric used to evaluate the prediction error takes temporal aspects into account in order to optimize the perceived inconsistency. Chen and Zarki [CE11] provide an objective evaluation framework for Quality of Experience (QoE) in Games. QoE takes not only the precision but three basic perceptions into account: responsiveness, precision, and fairness. These techniques are orthogonal to our approach and could be used together with AntReckoning to increase the performance of dead reckoning with respect to the quality of experience.

Cheating: Multiplayer online games are vulnerable to cheating. Speciﬁc techniques, including dead reckoning, have been successfully exploited for cheating. In particular, with dead reckoning a player can (1) actively drop position updates or predictions and send incorrect predictions to confuse/blind other players and (2) passively exploit the prediction

147 5.7 Discussion to gain an unfair advantage.

With suppress-correct cheat a player purposefully drops a number of consecutive updates and then sends an invalid update before being considered a dead peer. Similarly, players can implement a consistency cheat where they send different predictions to different players. Many methods have been proposed to address these general types of cheats [WS07, KTCB05]. For instance, one can verify updates based on the maximum speed of players and the game world physics [GV08], make all players send hash codes of their updates, namely commitments, before sending their actual updates [BLL07], and compare a posteriori predictions against the actual positions using (trusted) servers [GV08], referees [WSL07], or dynamic proxies-players (Chapter 3). Detected misbehavior can then feed reputation systems [KF09].

Position predictions can also be used to unduly assist the players in the game. For instance, aim bots can automatically adjust the players’ aiming directions, based on other avatars predicted positions, to increase the chances of hitting the target. Similarly, predictions can be used to implement a map hack attack, i.e., displaying objects/avatars the player is not supposed to see (e.g., behind walls). Such aids can be detected using player behavior analysis tools [LPBC07]. Note that such cheats apply to normal dead reckoning as well. In fact, in most cases AntReckoning does not create new opportunities to cheat. However, in AntReckoning, the fact that predictions implicitly give an insight on the players’ interests provides an additional opportunity to cheat. For example, a prediction which indicates that an avatar will move towards a health pack in the next few seconds can possibly show that this avatar has a low health. This opportunity is however relatively difﬁcult to exploit as the ﬁnal prediction is the summation of many interest-based attractions forces, e.g., a weapon behind the health pack or another player shooting at the player.

Finally, the calculations involved in AntReckoning can be secured by tools such as PunkBuster [Pun11] and Valve Anti Cheat (VAC) [VAC13], which prevent the players from tampering with the game code, and accountable systems [HARD10a].

148 5.8 Related Work to AntReckoning

5.8 Related Work to AntReckoning

A number of techniques improving the performance of dead reckoning have been proposed. In [CLC99] the threshold above which a new update is sent is adapted to the precision requirements, determined by the relative position of the entities. In [RAM`08], the metric used to evaluate the prediction error takes temporal aspects into account in order to optimize the perceived inconsistency. These techniques are orthogonal to our approach and could be used together with AntReckoning to increase the performance of dead reckoning.

In [MWMD07b] neural networks are trained and used, instead of instantaneous estimates, to predict changes of the entity velocity. ARIVU [ATL`10] predicts the next player actions in mobile games, using historical data, to determine if the wireless interface can be put into sleep mode, thus saving energy. These techniques could be incorporated in AntReckoning to help us estimate velocity and acceleration, and player actions respectively.

Ant colony optimization has been successfully used to solve a wide range of problems, e.g., the traveling salesman problem [DMC96, SH00]. AntReckoning is inspired from these techniques, e.g., the concepts of evaporation, spreading, etc. However, to the best of our knowledge, this is the ﬁrst application of pheromones to interest modeling and dead reckoning.

Beyond predicting avatar position, the interest model of AntReckoning, e.g., attractive pheromones, can be used to make convergence, path-finding (e.g., A*), and artificial intelligence [Mur11, BSYL09] algorithms exhibit human-like behavior. The information contained in pheromone maps can be used to choose between popular and unpopular paths in the path-finding algorithm for players. The AI players can use pheromone maps to direct them towards popular parts of the map and to choose paths typically used by players.

While most dead reckoning algorithms rely on linear movements between predicted positions, pheromone maps in AntReckoning can be used to generate non linear and hence more realistic movements between positions. In [BDL`08] guidable AIs are used to move between position, hence providing non linear movements. This approach can further beneﬁt from AntReckoning by guiding the AI to follow pheromone paths in order to move between

149 5.9 AntReckoning Conclusions two consecutive position updates.

Close to our work, a significant body of work has been devoted to the analysis of human mobility (e.g., [LHK`09, HFB09]) in real life, with numerous applications to urban planning and delay tolerant networking. In this context, several studies have demonstrated a strong influence of the locations of points of interest (e.g., specific businesses) on human mobility [ZZXM09]. These results have been further leveraged for pedestrian mobility predictions [CDR10]. A similar approach to AntReckoning can be exploited to predict pedestrian movements, where other people, buildings, and shops are treated as points of interest that generate pheromone.

5.9 AntReckoning Conclusions

This chapter proposed AntReckoning, an interest-based algorithm to improve dead reckoning. AntReckoning models a player’s interest in other players and game objects in the form of attraction forces exerted by pheromones at a low computational and memory overhead. Our results on Quake III and World of Warcraft have shown improvements over traditional dead reckoning: up to 44% in accuracy for a ﬁxed update rate and up to 32% with respect to bandwidth for a bounded prediction error, even in basic settings, demonstrating a great potential. In addition, while the focus of this chapter was on its applications in dead-reckoning in multiplayer games, the proposed algorithm can be applied to many other areas such as such as pedestrian movement prediction. For example, in the next chapter we will explain how it can be used to improve the quality of NPCs.

150 6 Towards the Design of a Human-Like FPS NPC using Pheromone Maps

Non-player characters (NPCs) in video games tend to be easily recognized by human players, reducing the sense of immersion and limiting the complexity of character interactions. In this chapter, we study various aspects of the NPCs performance and how it differs from human players. We provide categorization and metrics for quantifying some aspects of the NPCs performance and provide an in-depth analysis of the behavior of NPCs. We detail how movements, interactions, use of items, and relying on static decision-making schemes result in markedly different behaviors from humans in the popular FPS Quake III. In addition, we propose a framework relying on our pheromone maps, similar to the ones introduced in the previous chapter, which can lead to a more adaptive human-like behavior. These maps can efﬁciently give a summary of the events in the game world, be adaptive in nature, and be effectively used in the decision making process of NPCs.

6.1 Introduction to AntAI

As discussed before, NPCs can play alongside or against human players or be used to simply populate the game, ensuring a rich and interactive game world. They should be able to play in teams and handle different goals and strategies. Ideally, they should also show human-like movements and interactions, giving players the sense of an active and well populated virtual world.

151 6.1 Introduction to AntAI

Within the first person shooter (FPS) genre, many rely on variations of the area awareness system (AAS) that provides navigation information for the NPCs [Wav01]. These systems give a simplified 3D map of the world where different areas are calculated as simplified volumes so navigation within an area is simply point-to-point movement, and reachability can be easily determined. While navigation information is very useful for tasks such as path optimizations, the navigation itself relies on decision-making algorithms often in the shape of a simple decision tree and fuzzy logic. Combined together they determine the long-term and short-term goals of the AI in the game [MF09]. For example, in Quake III, the NPC selects, using predetermined preferences, a target item such as a weapon. It also tries to pickup any useful items along the way to his chosen target.

Although this produces functional NPCs, a drawback to this approach is that static decision-making tends to result in overdetermined and unrealistic behavior. In terms of movement, some areas end up being overused by the NPCs while some areas are never explored. With respect to use of weapons and other game items, more important differences become visible, clearly distinguishing NPCs from humans. For example, while reaction speed means that human players will usually avoid use of slower acting distance weapons such as sniper riﬂes in close or rapid combat, NPCs do not necessarily suffer from the same limitations. These constraints are not a result of limitations of the weapons or the game world in any obvious and ahead-of-time sense, but are due to limitations of the human players themselves, and so cannot always be easily extracted from the game mechanics.

Inspired by these examples and backed by traces collected from several gaming sessions in Quake III, we discuss a number of metrics that can be used to measure the performance of the AI in comparison to a human player (we refer to the characteristics that the AI exhibits as its performance). Furthermore, we provide an in-depth analysis of how these decision- making algorithms can lead to substantial differences in behavior of NPCs and humans. As a potential solution, we discuss how special kinds of inﬂuence maps, pheromone maps, similar to the ones introduced in the previous chapter, can be used to adaptively change the behavior of NPCs, making them more human-like. In this model each interesting entity or event is assigned a given attractiveness leading to the generation of pheromones that spread in the game world and fade over time. This attraction is then integrated into the process of decision making of NPCs, giving the AI a useful summary of the dynamic game state. Our

152 6.2 Existing NPC Technology and Game Adaptivity main contributions are:

1. We discuss several categories of metrics that can be used to measure NPC performance and how it differs from human players. 2. We provide in-depth analysis of differences between human and typical AI players based on a popular FPS game AI. Detailed Quake III game traces are used to compare various aspects of gameplay. Our results give quantitative insight into how NPC and player behaviors differ. 3. We describe a lightweight, adaptive framework that takes temporal and spatial aspects of the game into account in the decision making process. Pheromone maps are designed to allow for a dynamic and adaptive AI gameplay.

6.2 Existing NPC Technology and Game Adaptivity

As a proxy for real opponents and team-mates, players expect NPCs to be reactive to player actions, the game environment, and be adaptive in their behavior according to the player’s preferences and performance [Hun05]. In fact, in many cases, it is desirable that the NPC shows a behavior very similar to that of a human player.

To better understand why NPCs act the way they do, we consider the following underlying mechanisms: (1) The basic underlying mechanisms, such as decision trees, that are shared by many of the NPCs and are used to implement the AI. (2) The way the game is able to adapt to the capabilities of the player.

Essentially, a game is an state of conflict that is typically resolved by the player achieving certain goals [SZ03]. This conflict closely affects the design of any Non Player Char- acter (NPC). NPCs are expected to offer a challenge to make the game more interesting, and this challenge is supposed to be related to the skill of the players. In many games such as Quake III, the player chooses himself the gameplay difficulty at the beginning of the game, e.g., I Can Win, Bring It On, Hurt Me Plenty, Hardcore and Nightmare in Quake III. NPCs have to be reactive to the player’s actions and the environment and have to adapt to the player’s preferences and performance.

153 6.2 Existing NPC Technology and Game Adaptivity

6.2.1 Decision Making:

There are several mechanisms that are used to model the decision process of a NPC such as decision trees, hierarchical state machines, behavior trees, fuzzy logic, etc., each with their advantages and disadvantages [MF09]. For example, decision trees in which every branch node is a condition and leaf node is an action, are simple to represent and use but does not scale well and is hard to modify.

In Quake III, as a typical game AI implementation, the agents use multiple decision trees embedded inside a state machine [Wav01]. The state machine represents states an agent can reach in the game such as Battle Fight or Seek Long Term Goal. At every think frame, the agent goes through the network of states and ﬁnds the most appropriate one, best suiting the agent’s current situation. In each state, there exist a structure of if-then-else, rep- resenting the decision making process of that agent in that particular state. Moreover, inside the structure new states can be reached. For example, in the state Seek Long Term Goal, using the agent’s interests coming from fuzzy variables, the agent will select a goal, such as picking up a weapon. Human-like, opportunistic behavior is encoded by temporarily switching to a Seek Short Term Goal state if the agent notices a useful item along the way. Once the short term goal is completed it goes back to the Seek Long Term Task state.

Another development is to use influence maps to guide AI behavior. Influence maps can be used to determine who is in control of a certain part of a map [ML06]. Conceptually, influence maps have some similarity with potential fields used in robotics to define motion control. It refers to a desired position b in the world where the robot wants to reach. In this case, the robot at position a is treated as a particle and gets under the influence of an attractive field U from position a to b [DJ10]. This field represents the free-space of the environment. Obstacles are modeled as negative charges and they repulse the particle. Thus, the robot is exposed to forces that can be formulated as Upqq “ Ubpqq ´ Uobstaclepqq. ř This concept has been applied to games as well, with some authors arguing that a potential field approach can be a viable and cheaper option to replace more conventional, A˚ path finding [HJ08]. In [WG08], an algorithm is proposed that determines where is the best place for Pac-Man to move, based on attractiveness of potential fields, taking into consid- eration the moving ghosts and dots to eat. In [ML06], the authors build influence maps into

154 6.2 Existing NPC Technology and Game Adaptivity a tree form, where every leaf is a representation of the game state in form of an influence map. From there a genetic algorithm is used to investigate the best strategy. In [DST`08], the authors use flocking behavior common in real-time strategy games (RTS), and employ information from an influence map to increase the safety of units.

Influence maps have rarely been used in FPS games, mostly focusing on RTS games [Cha11]. In addition, most uses of these maps in decision making have been limited, utilizing a simple distance based approach to determine areas under own or enemy’s influence, for tasks such as building a new building. We argue that they can efficiently be applied to FPS games as well, and be used in different tasks and levels of decision making by NPCs.

6.2.2 Game Adaptivity:

Game adaptivity has received a considerable amount of research attention from the community [LB11]. In general, to provide adaptivity, the game ﬁrst monitors the actions of the players, computes an abstract value of their performance/preferences, and makes changes to the game environment according to those values. These parameters can include player’s rank, accuracy, and time taken for task completions. The game can adapt to these parameters, for example, by making it harder for an experienced player by improving the AI performance.

The simplest form of adaptivity is given by offering a player various difﬁculty levels, controlled by the player, to play the game so that he can adapt the game according to his skills [Che07]. Most games, e.g., Quake III, provide such controls where the capacities of the NPCs depend on the chosen difﬁculty level. A more dynamic approach, known to players as the rubber band, can be described as a negative feedback loop [SZ03]. This loop tends to keep its environment stable by dampening the differences; e.g., by allowing computer-controlled opponents to catch up no matter how far behind they are. An example is the Mario Kart racing game series by Nintendo, which handicaps players in proportion to their relative position. A better form of adaptivity is found in the Left 4 Dead series of games, which uses a dynamic balancing system that models the game intensity as experienced by the player, in form of a metric based on particular actions within the game. When a threshold is reached, the game’s pace slows down by removing opponents, increasing the

155 6.3 NPC Performance Metrics interval of their appearance, and decreasing their accuracy [Boo09].

While gameplay can be improved using such techniques, they highlight the differences between NPCs and humans, for example, by unusually increasing the accuracy. Our interest in this work is to use more natural methods of adapting NPC behavior, e.g., using pheromone maps, to reduce the glaring differences between human players and NPCS.

6.3 NPC Performance Metrics

Many metrics for measuring the performance of the game NPCs can be developed. One aspect that is often hard to deﬁne is how natural the NPCs reactions are in response to various game events. This happens as a result of the player imagining how he himself or other human players would react to the same situation and how it is different from what the NPC has shown. Given that most players differ from each other in their playing style, as long as these differences fall inside an acceptable margin of the human players, they are deemed acceptable. Note that here we are focusing on NPCs that take human roles in the game and are expected to behave like one and does not include other kinds of NPCs. In addition, extreme cases of NPCs, e.g. NPCs that are used as a near perfect player with perfect aim, mobility, etc., and are speciﬁcally designed for their non-human characteristics are not considered.

To be able to quantify these differences we divide them into the following categories: (1) Mobility, (2) Item Pickups, (3) Fighting Strategy, (4) Use of Items, (5) Difﬁculty Level, and discuss how they can determine the performance of the game NPC. This categorization is based on the design of the different modules in Quake III AI [Wav01] and we believe is representative of many games in the FPS genre.

Note that here we are ignoring teamwork as a part of the NPCs task. Many NPCs are used as teammates for the players and are expected to support the player in the game or perform various other tasks (e.g., defend a certain location). This, however, is quite game speciﬁc and given fairly basic support from Quake III, we ignore it here and instead focus on death match type games where each player is independent from others and its aim is to simply win the game by getting points through eliminating other players.

156 6.3 NPC Performance Metrics

Health packs Ammunitions Weapons Armors Respawn spots

(a) NPC Move- (b) Human move- (c) Difference: Hu- (d) Difference be- (e) Low Health (f) Kill Difference: (g) Death ments ments man vs NPCs tween NPCs in ses- Difference: Human Human vs NPCs Difference: Human sions vs NPCs vs NPCs

(i) NPC Movements (j) Human movements (k) Difference: Human (l) Difference between (m) Low Health Dif- (n) Kill Difference: (o) Death Difference: vs NPCs NPCs in sessions ference: Human vs NPCs Human vs NPCs Human vs NPCs

Figure 6.1: Experimental sensitivity analysis of AntAI using a 12-player trace in a deathmatch game from Quake III in the q3dm01 (ﬁrst row) and q3dm17 maps (second row). Difference maps are normalized logarithmic representation of differences between gaming sessions. Intense colors, blue for players and red for NPCs, shows a considerable difference in behavior, while pale color shows lower differences or no event in that area of the map.

Here, we introduce a number of performance metrics in order to quantitatively compare the performance of human and NPC players. Note that the metrics we propose are designed to be useful for comparing different AI strategies and are not necessarily indicative of a better approach in general as game developers may choose to differentiate NPC behavior from human players by choice depending on the game and playing scenario. We later, in Section 6.4, analyze these measurements in Quake III.

Movements: One basic but defining task of the players is movements. While providing basic movements e.g., randomly, is easy, it affects all other behaviors of the NPC as it define what enemies it will face, what items it can pickup and in what strategic locations it will end up. One main drawback of use of AAS files and optimized path finding algorithms such as A˚ is that it makes NPC movements repetitive and fairly predictable. While long term strategic goals of the NPCs determines their final destination, the path they take to

157 6.3 NPC Performance Metrics get to their destination is fairly predictable. In addition, even though game events, such as becoming under ﬁre and escaping, and short term goals do make the behavior appear more complex it is usually not enough. Moreover, it prevents the NPCs from exploring certain parts of the maps which can be particularly useful depending on different game styles and skills of other players.

Metrics: (1) To quantitatively, measure the differences between human and NPC movements we divide the game world into cells and measure how often they are used by NPCs and humans. Each cell maintains a count for humans and NPCs which is increased every time a player passes through the cell depending on its type. In a binary space partitioning (bsp) based approach the original bsp maps can be used as cells. This can be used as a tool for developers to highlight the different movement patterns. In addition, a difference map that calculates the difference between the two counts on each cell can be used as a quantiﬁable metric.

(2) The previous metric provides an overview, of how on average players move and does not highlight individual cases. One of the main differences in movements that is jarring to players is during their close interactions. How human players and NPCs move when in close combat can act as another measure for comparison. This can be calculated as a function of whether the players is under attack or not and as a function of the time passed since the interaction.

Item pickups: What game items the players pickup affects their gameplay. These items may include different weapons, ammo, health packs, armor, etc. For example, it affects the path the players choose (e.g., deciding to pick up a certain item). Moreover, a player with good armor and health can afford to be more aggressive, and the type of weapon (e.g., long range, short range, melee) dramatically affects player’s behavior.

Metrics: An indicator is how players act when they are in a particular state, e.g., in low health. Players are expected to take more defensive instances when they are in low health and are vulnerable or go to location where health pack, armor, or similar items exist. Therefore, a difference map as discussed before but taking into account a particular state (e.g., health ă threshold) can act as a measurement of their performance. Similarly, the frequency with which players pick up items and the percentage of picking up different

158 6.3 NPC Performance Metrics items, can also serve as a measure of performance.

Battles: Players are expected to be able to take advantage of strategic locations. These locations typically provide a good vantage point, access to powerful weapons and items, or a good cover or a hiding place from attacks. In addition, players are expected to avoid hazardous locations that makes them open to attacks and unable to defend. One typical problem with NPCs is that players ﬁnd a weak spot in the game and exploit repetitive behavior of NPCs (e.g., passing through the same path) to gain easy points by repetitively attacking the NPC in the same spot. Note that these spots change during the game and depending on other players behavior, weapons, and locations. For example, a path that is not necessarily very dangerous can become one if another player takes cover in a good vantage point and uses an sniping weapon. A human player would learn after one or two interactions to avoid such a place until the threat is eliminated but most NPCs lack such a long term strategic decision making process.

Metrics: Strategic decision making of bots is fairly hard to measure without considering the speciﬁc situation and since a single best solution rarely exists. However, we can measure how on average NPCs performs. We do this by comparing locations that human and NPC are typically killed or are able to score a point. These places highlight the vantage points good for attacking others as well as vulnerable locations.

Use of items: How players use certain items, greatly affects how they play. For example, a sniping weapon, given high accuracy at a distance and long reload times, requires finding a good vantage point and hiding from other weapons that are less accurate but faster at a close range. In addition, most human players would find use of sniper gun at a close range very difficult while NPCs would not have that problem and it can lead to a tangible different in their behavior. Another important factor is that most players have certain weapons of choice. This affects their choice of paths (to find this weapon or ammo) and where they are most likely to go. They also show great differences in their accuracy when using their favorite weapon in comparison to others. NPCs however do not have such behaviors, unless it is explicitly programmed, which often is not.

Metrics: To measure the use of game items we consider one of the most important category of items, i.e., weapons. How often players use different weapons (percentage of

159 6.3 NPC Performance Metrics

Table 6.1: Differences in usage, distance, and accuracy of important weapons between Players and NPCs NPC P% NPC % NPC % P Avg. P% NPC % N-NPC Map Weapon P % Kill Avg. Shot Shot Kill Dist. Acc. Acc. % Acc. Dist. Machine 60.4% 43.4% 32.2% 20.1% 324.2 149.1 5.0% 3.8% 5.4% Gun q3dm01 Shotgun 5.28% 34.5% 5.1% 28.0% 253.9 203.8 9.0% 6.6% 8.6% Rocket 9.4% 11.5% 30.8% 37.6% 273.7 185.4 30.4% 27.0% 32.2% Launcher Plasma 24.7% 10.4% 31.8% 14.1% 256.3 178.4 12.0% 11.1% 12.4% Gun Machine 49.6% 60.7% 15.7% 21.8% 451.9 343.6 4.1% 3.3% 5.0% Gun q3dm17 Shotgun 20.8% 19.5% 10.7% 12.7% 226.5 374.6 6.7% 6.0% 8.0% Rocket 26.5% 19.0% 60.3% 61.2% 259.7 270.9 29.5% 30.1% 37.6% Launcher Rail 2.9% 0.65% 12.6% 4.2% 782.9 488.6 55.9% 60.8% 66.7% Gun

each weapon’s use) as well as how often they are successful at targeting people can serve as an indicator of their performance.

Difﬁculty level: To provide different levels of challenge to players with different skills, the performance of the NPCs needs to be adaptable. This, however, is mostly accomplished by improving the accuracy of NPCs in targeting players. While this is acceptable to a certain level, e.g. expert human players have very good accuracy, it has its limits and after a certain point this performance becomes too unrealistic as NPCs can have near-perfect aim, given that NPCs can use trajectories based on game physics, even for weapons such as rocket launchers that are often hard to aim as they take a longer time to reach a moving target. Another way to improve this performance can be through improving the strategic thinking of the NPCs and their resulting movements.

Metrics: Difficulty can be measured as the efficiency of players in achieving their goals. In a deathmatch game that goal is to score points by eliminating other players. When NPC difficulty is increased most games resort to improving the accuracy of their weapon use which can be measurable. Furthermore, strategic decision making can be measured as it affects movement and use of strategic locations by using various difference maps as discussed before.

160 6.4 Quake III Analysis

6.4 Quake III Analysis

In this chapter,we provide a thorough analysis of NPC behavior based on decision-trees and state machines as used in many of today’s games. In particular we show that while it yields somewhat complex behaviors, this behavior is considerably different from the behavior of human players, and often less versatile. In addition, the NPC’s behavior might seem predictable, erratic, or irrational to a human player.

For the purpose of the analysis, we have selected Quake III, a popular FPS game, given its popularity and availability of its source code. While newer games use more complex decision making processes, many rely on the same techniques used in Quake III (e.g., Left 4 Dead 2009 [Boo09]), therefore, we believe the analysis can be representative of the FPS genre of games. We have used two popular maps in the game, and played the game in a total of ten gaming sessions using players of various levels of skill. P represents players in ﬁgures and tables, NPC represents NPC of difﬁculty level Hurt me plenty, and N-NPC is NPC with Nightmare level. Around ten gaming sessions are performed where each session lasts for around 20-30 minutes and includes 6-12 players of types: players, NPCs, and both. This is a typical case for most Quake III gaming sessions and while the study is not exhaustive, it can be fairly representative of the general gaming behavior in Quake III.

In our results, we emphasize the differences between NPCs and human players in the categories discussed:

Movements: Figures 6.1a and 6.1i depict the movements of NPCs in several gaming sessions. Darker areas represent more activity in that part of the map. Note that the figures use logarithmic presence in the map essentially showing a power law distribution in the use of some regions. As shown, NPCs tend to use specific paths and locations in the map as indicated by the AAS files. This is a result of the decision making process and not a limitation of the accessible areas as the NPCs can use other paths and locations but often don’t. Figures 6.1b and 6.1j show the same map for human players. As one can observe, players use the maps more uniformly. While popular locations still exist, as Figure 6.1j particularly shows for the larger, more complex q3dm17 map, they are significantly different from the paths chosen by NPCs. Figures 6.1c and 6.1k highlight these differences in player and NPC movements in several gaming sessions. Blue regions are mostly used by

161 6.4 Quake III Analysis players, while red regions are mostly used by NPCs with the intensity showing the amount of difference. White shows the same amount of activity (or lack of it) in the region. The ﬁgures clearly indicate how players and NPCs use different regions in the game.

Figures 6.1d and 6.1l show the differences in AI behavior in different gaming sessions. Even in different gaming sessions and playing against human players with various skills, NPCs show minimal or no difference in their behavior, lacking adaptability to the players’ skills and preferences.

Item pickups: Figures 6.1e and 6.1m further highlight the differences in movements between players and NPCs when the entity has low health (30 from 100). Players are much more likely to move towards items such as healthpacks and armors. Particularly, NPCs on average where 10% less likely to pick up an item during the game in comparison to humans.

Battles: Players are able to exploit strategic locations, giving them a good vantage point, access to powerful weapons, or take cover from attacks. This difference, and lack of adaptivity is further used by players as shown in Figures 6.1f and 6.1n highlighting differences in locations where players and NPCs score by killing opponents. Players tend to go to locations providing powerful items, e.g., rocket launchers, and with a good vantage point, e.g., top level of the map, to perform attacks targeting those in lower levels. This is also shown in death locations in Figures 6.1g and 6.1o. NPCs are usually targeted in lower levels and smaller corridors in the game. In gaming sessions in which both players and NPCs were playing, players were able to take advantage of predictability and lack of adaptivity of NPCs, and attack NPCs in bottlenecks while maintaining strategic locations near powerful weapons and with a good vantage point.

This difference is further shown in the aggressiveness of the players. Human players are more likely to follow their target even if the target is shooting back. This behavior increases as a function of the time elapsed since last attack by target. In addition, while players die on average in a game at the same rate, 4.34 vs. 4.50 deaths for a NPC per minute in q3dm1 map, they achieve 1.15 vs. 0.82 kills per death.

Use of items: Another significant difference between human players and NPCs is in their use of items. Some items, due to their nature, are hard to use by human players in specific situations. For example, a sniper rifle or its equivalent in Quake III, the Rail Gun, is hard

162 6.4 Quake III Analysis

75% P 70% NPC 65% 60% 55% 50% 45% 40% 35%

the target in next seconds (in %) 30% prob. that the attacker gets farther from 0 < t < 1 1 < t < 2 2 < t < 3 3 < t < 4 t > 4 time t elapsed since target shot the attacker (in seconds)

Figure 6.2: When engaged in a ﬁght, players tend to get away from each other. However, when attackers are not being shot back, they move towards their target more often. to use in close quarters combats (CQB). Players, as shown in Table 6.1, use this weapon at substantially higher distances than NPCs do. These values also depend on the map. While the use of RocketLauncher in map q3dm17, which features large open areas, is very similar to the players, it is different in map q3dm1, where the world consists of rooms and corridors and there is higher risk of damaging oneself when using the weapon.

Difficulty level: Lack of adaptability is further shown, when different NPC difficulty levels are used. While, as shown in Table 6.1, the accuracy of attacks are increased along with the difficulty, our difference maps showed no significant difference between movements of NPCs of different levels, similar to the case in Figures 6.1d and 6.1l. Similarly, most of the other game statistics remained the same. This shows that NPCs are not using a better decision making process when the difficulty is increased.

6.4.1 Bot Detection

While the main focus of this chapter is on improving the performance of the game AI, this analysis can also be employed in detecting when a bot is playing the game instead of a human player. One of common forms of cheating in the games is by having an NPC play the game while the player is away, in order to easily gain experience points or improve one’s ranking (see Chapters 2 and 3). A range of techniques have been used to detect these

163 6.5 AntAI Algorithm

b r NPC (P ) b other avatar u P death location r q item of interest past trajectory preferred direction

Figure 6.3: Overview of AntAI: the game world is divided into cells by using a regular square grid. Each cell contains a certain amount of pheromone, represented grayscale (crosshatched for negative values). Attraction forces are directed towards the attracting cells and their intensity is proportional to the pheromone they contain. cheat types [GD05, SGJ07, CPC08]. Our analysis can be used to extract the necessary features (e.g., movements, interactions, etc) to be used in statistical analysis and machine learning techniques that can classify the players into human or (cheating) NPC players.

6.5 AntAI Algorithm

Here we propose AntAI, our vision of how a more human-like NPCs can be designed using pheromone maps as discussed in the previous chapter. Pheromone maps, inspired by ant colony systems and a form of inﬂuence maps, are used to model online events in the game world in a lightweight and efﬁcient manner. We argue that this model can be then used to improve the behavior of the NPC, allowing it to make more strategic and human-like decisions.

Similar to AntReckoning, in AntAI, points of interest (e.g., weapons, items, other players, events, and specific locations) are treated as ants that generate pheromones, modeling their relative influence. The NPC computes the concentration of pheromone (represented in grayscale in Figure 6.3) in each cell and computes and sums the corresponding influences.

Attraction: The amount of pheromone generated for items of interest, denoted by the attraction function in equation, is dependent on the NPC’s state and goals. For example,

164 6.5 AntAI Algorithm when the health of the player is low, the value generated for a health pack is higher than other items attracting the player to it’s location. However, this is substantially reduced after the health pack is picked up. The effect of this increase is quickly faded through evaporation, allowing new interests, such as picking up a powerful weapon for attack, to be taken into account.

Number of maps & Overhead: While a single pheromone map can provide a summary of the game world, multiple maps can also be used taking only specific items or events into account. In the former, a single map exists for each NPC, where the current state of the NPC at each frame is taken into account in deciding how much pheromone is generated for each item or event inside that map. However, in the latter, the state of the player is not taken into account when the pheromones are generated for each map. In this case, to take individual state of NPCs into account, each map receives a coefficient according to each NPCs current status, e.g., the healthpack map receiving a higher coefficient for an injured NPC. Many different maps may be generated for different interests such as death locations, kill locations (using different weapons), successful item pickups (e.g., healthpack), strategic item pickups (e.g., flag in capture the flag), and player movements.

The number of these maps is a design decision: higher number of maps gives more detailed information about the game state at higher overhead costs. One big advantage of using different maps, without taking individual state into account, is that information can be shared between all NPCs and is suitable for server systems. On the other hand, single maps generated per NPC can allow the map to be much smaller, e.g., size of AOI, and be distributed among players. A combination of the two approaches may also be used: some maps are shared and each NPCs has one or more exclusive maps. In general pheromone maps are easily implemented and have relatively low overhead. Furthermore, vectorization of the matrix can be used to allow hardware accelerated technologies to be used.

Here, we explain our vision on how to use one or more such maps in the decision making process and how it can improve the performance of the NPCs.

165 6.5 AntAI Algorithm

6.5.1 Decision Making

Pheromone maps can be used to improve different aspects of decision making by the NPCs. We list most common tasks performed during the decision making process of a FPS NPC and how the pheromone maps can be applied. They are easily integrated in the current decision making process of the NPCs and are not meant to replace them.

Long term goals: Long term goals of NPCs in a FPS game, can be goals such as attacking an area in the game, defending the base, providing support for teammates, or maintaining strategic locations. The pheromone map can provide a summary of locations under attack as well as vulnerable positions (with regards to visibility and game mechanics) to help the NPC choose the target destinations. For example, when the player is in attack state it can help choose goals that are more strategically vulnerable. Note that while many options may exist on the map, their attractiveness is dependent on the amount of pheromone for that location as well as their Euclidean distance from the player as shown in equation 6.1, where k is a configuration parameter depending on the game mechanics. These forces are then summed up as shown in Figure 6.3 (when multiple maps exist, the values are first multiplied by map coefficient) and used to decide on a target.

phtpcellq }Attractionpcell, NPCpos,tq} “ k , (6.1) dpcell, NPCpos,tq

Short term goals: An example of short term goals would be picking up an item while pursuing a long term goal. These decisions in particular are related to the current state of the player. For example, a player with low health tries to pick up a health pack on it’s way to attack the enemy base. The pheromone map provides necessary information for choosing between items and selecting a suitable path to pick up the healthpack while minimizing the threats.

Battles: Pheromone maps can be used to improve the battle performance of the NPCs. A pheromone map highlighting death and kill locations can help choose better locations for ﬁghting and defense: locations with higher death score are more dangerous while kill locations provide a good vantage point. Furthermore, this is calculated according to whether the

166 6.6 AntAI Conclusions

NPC is in attack or defense mode. Similarly, this can take the weapon being used into account, highlighting where kills and death occur when players are using a particular weapon. This also helps with making the NPC’s usage of weapons follow the normal game pattern among players. Pheromone trails left by the players help with chasing the enemies or supporting teammates. When the player is wounded and in retreat the map highlighting health pickups would direct the player, while avoiding death locations.

Path ﬁnding: As shown in the previous chapter, current NPC players choose predictable paths not taking the current state of the game into account. The pheromone maps highlight the paths being taken by other players, allowing the NPC to choose more human-like paths. In addition, it allows the NPCs to avoid bottlenecks and strategically vulnerable locations, reducing their predictability.

Difficulty level: Pheromone maps enable the NPC to make more strategic decisions in the game. Therefore, it allows the NPC to act smarter when the difficulty level is increased by the player instead of only increasing the accuracy of the NPCs’ attacks. It allows easier control of the difficulty level: the amount of information and details, e.g., number and size of maps, used in the process of decision making can determine the difficulty level of the NPCs.

State transitions: So far we have discussed how to better perform goals and tasks decided by the already existing decision making process. In most games offensive and defensive roles of NPCs do not adapt to the current state of the game but are assigned at the beginning of the session. Pheromone maps can help determine whether the NPC should act in an offensive or defensive state, act as a supporting companion of the leader of the team, chase, or retreat given the current state of the game. For example, a large amount of negative pheromone left by attack of the enemies would trigger a state transition of the NPC to a defensive role if he’s the closest to that location and the values are above a certain threshold.

6.6 AntAI Conclusions

In this chapter, we analyzed how FPS NPCs’ reliance on traditional decision-making approaches can result in showing signiﬁcant differences from human players. We categorized

167 6.6 AntAI Conclusions different types of NPC behavior and how it can differ from human players. Furthermore, we proposed measurable metrics for NPC performance and perform such analysis on popular FPS game Quake III. In addition, we proposed a framework using pheromone maps, a special kind of inﬂuence map, to improve various aspects of NPCs’ behavior.

168 Part IV

Final Conclusion & Future Work

169 7 Final Conclusion & Future Work

7.0.1 Final Conclusion

Multiplayer online games are one of the biggest entertainment industries in the world and they introduce some of the most challenging requirements. Many distributed architectures such as C/S, P2P, or hybrid have been proposed for MOGs where the main beneﬁt of P2P and hybrid architectures is to reduce the cost of maintaining expensive servers and improve the scalability and fault tolerance of games. Each of the architectures studied offer a different trade-off for speed, scalability, fault tolerance, etc and this thesis presents a comprehensive overview and categorization of existing solutions. Many techniques have been proposed to solve these challenges, but many topics, such as availability, persistence, and cheating remain to be fully addressed. For example, one of the most important aspects in these games, security, has been mostly an afterthought. In addition, while some of these techniques can be applied to different architectures, many are dependent on the underlying structure.

Motivated by our survey of work in this area, we have proposed Watchmen, AntReck- oning, and AntAI that complement each other in creating a more secure and scalable game framework. We believe Watchmen is the ﬁrst fast and scalable distributed protocol designed for FPS games that aims at detecting and preventing cheating, even in the presence of collusion. Watchmen is able to scale well and can meet well-established latency requirement for fast-paced games. Cheat resistance is achieved through cross veriﬁcations and vision-

170 Final Conclusion & Future Work based ﬁltering facilitated by a dynamic proxy scheme. Our experiments on Quake III show a great reduction in the information exposure to cheaters even in the presence of collusion. Furthermore, it shows great potential for effective veriﬁcation of player actions by other players. Moreover, in the same line we discuss security in mobile games. Mobile games are one of the fastest growing categories of mobile apps. These games introduce new and interesting forms of security concerns that need to be addressed. Addressing security concerns in these environments requires more attention from all stakeholders, i.e., carriers, platforms, game developers, and hard designers, and requires support from all them.

AntReckoning, which is used in our platform to increase the game tolerance to latency and to lower bandwidth requirements, proposes an interest-based algorithm to improve dead reckoning. AntReckoning is designed to model a player’s interest in other players and game objects, in the form of attraction forces exerted by pheromones achieving low computational and memory overheads. Using traces from Quake III and World of Warcraft, we show improvements over traditional dead reckoning: up to 44% in accuracy for a ﬁxed update rate and up to 32% with respect to bandwidth for a bounded prediction error, even in basic settings, demonstrating a great potential. Furthermore, while the focus of AntReck- oning is on its applications in multiplayer games, the proposed algorithm can be applied to many other areas such as pedestrian movement prediction. As an example, we analyze how FPS NPCs’ reliance on traditional decision-making approaches can result in showing signiﬁcant differences from human players. We categorize different types of NPC behavior and how it can differ from human players. In addition, we propose measurable metrics for NPC performance and perform such an analysis on Quake III as well as a framework for using pheromone maps to improve various aspects of NPCs’ behavior.

Overall, we believe that the proposed solutions are important cornerstones of a scalable, cheat resistant, deployable architecture for large-scale multiplayer online games, in particular fast-paced games where the requirements are more stringent.

7.0.2 Future work

Many of the research questions in games have to deal with many different areas from networking and systems to graphics and AI. As a result, solutions offered have to take those

171 Final Conclusion & Future Work requirements into account in order to be practical which is sometimes ignored in the literature. On the other hand, one of the main obstacles in deploying the research approaches in games, is the lack of the gaming industry’s interest in working with academia as well as industry’s risk aversion rather than the infeasibility of the solutions.

The aim of this thesis was to design a scalable cheat-resistant and fast platform for multiplayer online games. To that goal we surveyed the literature and discussed Watchmen, mobile security, AntReckoning, and AntAI. These can be enhanced by further development, testing, and evaluation which would require more time and resources than a single thesis.

With the rapidly increasing amount of processing and networking resources available to the players many of the problems facing P2P and hybrid systems will be assuaged. While the academic research interest in P2P and hybrid systems has recently become lower, many commercial and popular AAA games still use P2P systems for their networking support. This is mainly due to the many benefits that these systems offer. The main problem preventing wider deployment of P2P systems is cheating and maintaining control over the game environment while maintaining a good performance. Our Watchmen and AntReck- oning architectures are aimed at addressing that. Based on our research, we believe that maintaining a good level of security and performance while benefiting from P2P and hybrid systems is feasible. Furthermore, many of the solutions proposed here can be applied to other research areas as well. In addition, other techniques, e.g., cloud computing, can be added to our solution to be able to benefit from advantages that each system has to offer and improve the system.

An interesting future work would be the implementation of Watchmen under the form of a generic middleware for MOGs that can be used by many games, e.g., by providing an API for determining different interest sets. While the current system can be easily used in a hybrid setting further implementation can help demonstrate this. Cloud computing platforms, where proxy nodes are added and removed dynamically as the need arises, can be efﬁciently used to increase the scale of the system or the overall security of the architecture. These nodes can take over in case additional bandwidth is needed or in case further audits are required by the architecture to ensure the correctness of certain game proxies’ operations. In addition, further experimentation can be done once real-world cheat logs have

172 Final Conclusion & Future Work become available. Companies have been very reluctant to share any cheating data which makes running experiments that can prove the real world effectiveness of the system very hard.

An emerging area of research is augmented and virtual reality platforms. Augmented reality application combine many of the characteristics of gaming environments with mobile platforms and provide a range of new and challenging research questions. For example, they require almost real-time communication and similarly have to rely on multi- resolution/ﬁdelity schemes to provide information only for interesting items and people. In addition, virtual reality applications share most of their characteristics with fast-paced games and can greatly beneﬁt from our research in this area.

Another avenue of research would be experimenting with a trust factor where some players or servers are trusted with more critical tasks. This can be done by auditing their performance over time, where a satisfactory performance history can determine the trust- worthiness of the players. Further study of reputation systems can prove beneﬁcial to the subject.

Furthermore, techniques to parametrize AntReckoning with offline trace-driven analysis and online learning can be of interest. In AntReckoning, depending on game physics and environment, optimal values for different parameters would be different. As a result, the game developers will need to fine-tune these parameters based on their game. Tech- niques can be developed where, by using traces of different game sessions, the parameters can be configured automatically as necessary.

Another possibility is to use pheromone maps for prediction of the pedestrian mobility. In such a scenario, pedestrians, similar to players, move according to their attraction to points of interests such as their workplace, coffee shops, restaurants, etc. and therefore, their movements can be predicted as such. These predictions can have many applications such as mobile hand-off predictions.

One of the reasons we studied cheating in mobile games was that this subject was greatly lacking in the research literature. We believe our proposed API can provide a foundation for building security systems that can enhance existing mobile security mechanisms. Similar to PC and console games, currently it is very hard to get cheating data from mobile

173 Final Conclusion & Future Work game developers, making it hard to develop a comprehensive threat model and run detailed experiments. An implementation of the current API can help measure the effectiveness of different methods proposed and the cost associated with the veriﬁcations.

Even though studying the differences between human and NPC performance was outside the focus of this thesis on scalability, security and performance of multiplayer games, we explored the differences between human and NPC behavior in games, and proposed use of pheromone maps to improve them. However, a detailed implementation and evaluation of the system remains to be done. Furthermore, these differences may not be of equal importance, when measuring the quality of experience of players. Therefore, a study on what kind of NPC behaviors would be more jarring to human players would be interesting.

174 List of Publications

Published:

‚ [YK13] A. Yahyavi and B. Kemme. Peer-to-peer architectures for massively multiplayer online games: A survey. ACM Computing Surveys (CSUR), 46(1):9:1–9:51, ACM, 2013. ‚ [YHGS`13] A. Yahyavi, K. Huguenin, J. Gascon–Samson, J. Kienzle, and B. Kemme. Watchmen: Scalable cheat-resistant support for distributed multiplayer online games. In Proceedings of International Conference on Distributed Computing Systems (ICDCS). pages 134–144, ACM, 2013. ‚ [YHK12] A. Yahyavi, K. Huguenin, and B. Kemme. Interest modeling in games: The case of dead reckoning. Multimedia Systems (MMSJ), pages 1–16, Springer, 2012. ‚ [YTVK13] A. Yahyavi, J. Tremblay, C. Verbrugge, and B. Kemme. Towards the design of a human-like FPS NPC using pheromone maps. In IEEE International Games Innovation Conference (IGIC), pages 275–282, IEEE, 2013. ‚ [YPK13] A. Yahyavi, J. Pang, and B. Kemme. Towards providing security for mobile games. In ACM MobiCom International Workshop on Mobility in the Evolving Internet Architecture, MobiArch ’13, pages 47–52. ACM, 2013. ‚ [YHK11] A. Yahyavi, K. Huguenin, and B. Kemme. AntReckoning: A pheromone-based dead reckoning algorithm for games. In Proceedings of International ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES). pages 1–6, ACM, 2011. ‚ [HYK11] K. Huguenin, A. Yahyavi, and B. Kemme. Cheat detection and prevention in P2P MOGs. In Proceedings of International ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–2. ACM, 2011.

175 Bibliography

[ABK`04] S. Aggarwal, H. Banavar, A. Khandelwal, S. Mukherjee, and S. Rangarajan. Accuracy in dead-reckoning based distributed multiplayer games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 161–165. ACM, 2004.

[AF12] Erman Ayday and Faramarz Fekri. Iterative Trust and Reputation Manage- ment Using Belief Propagation. In Proc. of IEEE Transactions on Depend- able and Secure Computing TDSC, 9:375–386, 2012.

[AG96] S. V. Adve and K. Gharachorloo. Shared memory consistency models: A tutorial. IEEE Computer, 29(12):66–76, 1996.

[AJZ06] Grenville Armitage, Carl Javier, and Sebastian Zander. Post-Game Esti- mation of Game Client RTT and Hop Count Distributions. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), 2006.

[Arm03] G Armitage. An experimental estimation of latency sensitivity in multiplayer Quake 3. In Proc. of IEEE Int. Conf. on Networks (ICON), pages 137–141. IEEE, 2003.

[AS10] D.T. Ahmed and S. Shirmohammadi. A fault tolerance procedure for P2P online games. In Proc. of Int. Conf. on Information Sciences Signal Pro- cessing & their Applications (ISSPA), pages 614–617, 2010.

[ASO09] D T Ahmed, S Shirmohammadi, and J C. Oliveira. A hybrid P2P communications architecture for zonal MMOGs. Multimedia Tools and Applications, 45:313–345, October 2009.

176 BIBLIOGRAPHY

[ATL`10] B. Anand, K. Thirugnanam, L. Long, D. Pham, A. Ananda, R. Balan, and M. Chan. ARIVU: Power-aware middleware for multiplayer mobile games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2010.

[BA08] E. Buyukkaya and M. Abdallah. Efﬁcient triangulation for P2P networked virtual environments. In Proc. of Int. ACM SIGCOMM Workshop on Net- work & System Support for Games (NETGAMES), pages 34–39. ACM, 2008.

[BAC09] E. Buyukkaya, M. Abdallah, and R. Cavagna. VoroGame: A hybrid P2P architecture for massively multiplayer games. In Proc. of Int. IEEE Conf. on Consumer Communications & Networking (CCNC). IEEE, 2009.

[BAS04] A. Bharambe, M. Agrawal, and S. Seshan. Mercury: supporting scalable multi-attribute range queries. In Proc. of ACM SIGCOMM Conference, pages 353–366. ACM, 2004.

[BC85] E. J. Berglund and D. R. Cheriton. Amaze: A multiplayer computer game. IEEE Software, 2(3):30–39, 1985.

[BCL`04] Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, and Mark Claypool. The Effects of Loss and Latency on User Perfor- mance in Unreal Tournament. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 144–151, 2004.

[BDL`08] A. Bharambe, J. Douceur, J. Lorch, Thomas Moscibroda, J. Pang, S. Se- shan, and X. Zhuang. Donnybrook: enabling large-scale, high-speed, peer- to-peer games. In Proc. of ACM SIGCOMM Conference, pages 389–400. ACM, 2008.

[BF93] S. Benford and L. Fahlén. A spatial model of interaction in large virtual environments. In Proc. of Int. Euro. Conf. on Computer-Supported Coop- erative Work (ECSCW), pages 109–124. Kluwer, 1993.

177 BIBLIOGRAPHY

[BHML92] B. Blau, C. E. Hughes, M. J. Moshell, and C. Lisle. Networked virtual environments. In Proc. of Intl. Symp. on Interactive 3D graphics, pages 157–160. ACM, 1992.

[Big11] BigWorld technology. http://indie.bigworldtech.com/, 2011.

[BKR07] O Beaumont, A M Kermarrec, and E Rivière. Peer to peer multidimen- sional overlays: approximating complex structures. In Proc. of Int. Conf. on Principles of Distributed Systems (OPODIS), pages 315–328. Springer, 2007.

[BKV06] J S Boulanger, J Kienzle, and C Verbrugge. Comparing interest management algorithms for massively multiplayer games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2006.

[BLB`04] S. Banerjee, S. Lee, R. Braud, B. Bhattacharjee, and A. Srinivasan. Scal- able resilient media streaming. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 4–9. ACM, 2004.

[Bli11] Blizzard Entertainment. http://www.worldofwarcraft.com/ pvp/battlegrounds, 2011.

[BLL07] N. Baughman, M. Liberatore, and B. Levine. Cheat-proof playout for centralized and peer-to-peer gaming. IEEE Trans. on Networking (TON), 15:1– 13, 2007.

[Boo09] M. Booth. The AI systems of Left 4 Dead. In AI and Interactive Digital Entertainment (AIIDE), 2009.

[BP98] Z Berman and J D Powell. The role of dead reckoning and inertial sensors in future general aviation navigation. In Proc. of Int. Symp. on Position Location & Navigation (PLANS), pages 510–517. IEEE, 1998.

178 BIBLIOGRAPHY

[BPS06] A. Bharambe, J. Pang, and S. Seshan. Colyseus: A distributed architecture for online multiplayer games. In Proc. of Int. Conf. on Networked Systems Design & Implementation (NSDI), pages 12–12. USENIX, 2006.

[Bro04] R. Brown. Smoothing, Forecasting, and Prediction of Discrete Time Series. Dover Publications, 2004.

[BSYL09] J. Bai, D. Seah, J. Yong, and B. Leong. Ofﬂoading ai for peer-to-peer games with dead reckoning. In Proc. of Int. Workshop on Peer-to-Peer Systems (IPTPS), page 3. USENIX, 2009.

[BTC`04] R Bhagwan, K Tati, Y C Cheng, S Savage, and G M. Voelker. Total Recall: system support for automated availability management. In Proc. of Int. Conf. on Networked Systems Design & Implementation (NSDI), pages 25– 25. USENIX, 2004.

[But03] Butterﬂy.net. The butterﬂy grid: A distributed platform for online games, later emergent game technologies. http://www.gamebryo.com/, 2003.

[CCR10] E Carlini, M Coppola, and L Ricci. Integration of P2P and clouds to support massively multiuser virtual environments. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 17:1–17:6. ACM, 2010.

[CDG`02] Miguel Castro, Peter Druschel, Ayalvadi Ganesh, Antony Rowstron, and Dan S. Wallach. Secure routing for structured peer-to-peer overlay networks. Operating Systems Review (SIGOPS), 36(SI):299–314, December 2002.

[CDH`06] B G Chun, F Dabek, A Haeberlen, E Sit, H Weatherspoon, M. F Kaashoek, J Kubiatowicz, and R Morris. Efﬁcient replica maintenance for distributed storage systems. In Proc. of Int. Conf. on Networked Systems Design & Implementation (NSDI), pages 4–4. USENIX, 2006.

179 BIBLIOGRAPHY

[CDKR02] M Castro, P Druschel, A M Kermarrec, and A I T Rowstron. SCRIBE: a large-scale and decentralized application-level multicast infrastructure. IEEE Journal on Selected Areas in Communications, 20(8):1489–1499, 2002.

[CDR10] F Calabrese, G Di Lorenzo, and C Ratti. Human mobility prediction based on individual and collective geographical preferences. In Proc. of Intl. Conf.on Intelligent Transportation Systems (ITSC), pages 312–317. IEEE, 2010.

[CDST06] A.B. Corman, S. Douglas, P. Schachte, and V. Teague. A secure event agreement (SEA) protocol for peer-to-peer games. In Proc. of Int. Conf. on Availability, Reliability & Security (ARES), page 8 pp. IEEE, 2006.

[CE11] P. Chen and M. El Zarki. Perceptual view inconsistency: an objective evaluation framework for online game quality of experience (QoE). In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2011.

[CF05] A. Chandler and J. Finney. On the effects of loose causal consistency in mobile multiplayer games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–11. ACM, 2005.

[CFFS05] Chris Chambers, Wu-Chang Feng, Wu-Chi Feng, and Debanjan Saha. Mit- igating information exposure to cheaters in real-time strategy games. In Proc. of Int. Workshop on Network & Operating Systems Support for Digi- tal audio & Video (NOSSDAV), pages 7–12. ACM, 2005.

[Cha11] Alex J. Champandard. The Mechanics of Inﬂuence Mapping: Represen- tation, Algorithm & Parameters. http://aigamedev.com/open/ tutorial/influence-map-mechanics/, 2011.

[Che07] Jenova Chen. Flow in Games (and Everything Else). ACM Communication, 50(4):31–34, April 2007.

180 BIBLIOGRAPHY

[CHHL05] Kuan-Ta Chen, Polly Huang, Chun-Ying Huang, and Chin-Laung Lei. Game trafﬁc analysis: an MMORPG perspective. In Proc. of Int. Work- shop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 19–24. ACM, 2005.

[CHJ08] M C Chan, S Y Hu, and J R Jiang. An efﬁcient and secure event signature (EASES) protocol for peer-to-peer massively multiplayer online games. Computer Networks, 52:1838–1845, June 2008.

[CHJC10] C H Chien, S Y Hu, J R Jiang, and C W Cheng. Bandwidth-aware peer-to- peer 3D streaming. Proc. of Int. Journal of Adv. Media & Communication (IJAMC), 4(4):324–342, 2010.

[CKV01] G. V. Chockler, I. Keidar, and R. Vitenberg. Group communication spec- iﬁcations: a comprehensive study. ACM Computing Surveys (CSUR), 33(4):427–469, 2001.

[Cla08] Mark Claypool. Network Characteristics for Server Selection in Online Games. In In Proc. of ACM/SPIE Multimedia Computing and Networking (MMCN), 2008.

[CLC99] W. Cai, F. Lee, and L. Chen. An Auto-Adaptive Dead Reckoning Algorithm for Distributed Interactive Simulation. In Proc. of Int. Workshop on Prin- ciples of Advanced & Distributed Simulation (PADS), pages 82–89. ACM, 1999.

[CM04a] B. Chen and M. Maheswaran. A cheat controlled protocol for centralized online multiplayer games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 139–143. ACM, 2004.

[CM04b] Bei Di Chen and M. Maheswaran. A fair synchronization protocol with cheat prooﬁng for decentralized online multiplayer games. In Proc. of Int. Symp. on Network Computing & Applications (NCA), pages 372–375. IEEE, 2004.

181 BIBLIOGRAPHY

[CMPC03] P Costa, M Migliavacca, G P Picco, and G Cugola. Introducing reliability in content-based publish-subscribe through epidemic algorithms. In Proc. of Int. Workshop on Distributed Event-Based Systems (DEBS), pages 1–8. ACM, 2003.

[Coh03] B. Cohen. Incentives build robustness in BitTorrent. In Workshop on Eco- nomics of Peer-to-Peer systems, volume 6, pages 68–72, 2003.

[Com13] Comscore smartphone report. http://www.comscore.com/ Insights/Press_Releases/2013/5/comScore_Reports_ March_2013_U.S._Smartphone_Subscriber_Market_ Share, 2013.

[CPC08] K. Chen, H. K. Pao, and H. C. Chang. Game bot identiﬁcation based on manifold learning. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 21–26. ACM, 2008.

[CRd`04] F R Cecin, R Real, R de Oliveira Jannone, C F Resin Geyer, M G Martins, and J L Victoria Barbosa. FreeMMG: A scalable and cheat-resistant distribution model for internet games. In Int. Symp. on Distributed Simulation and Real-Time Applications DS-RT, pages 83–90. IEEE, 2004.

[CRSZ02] Y. Chu, S G Rao, S Seshan, and H. Zhang. A case for end system multicast. IEEE Journal on Selected Areas in Communications, 20(8):1456– 1471, 2002.

[CWD`05] J. Chen, B. Wu, M. Delap, B. Knutsson, H. Lu, and C. Amza. Local- ity aware dynamic load management for massively multiplayer games. In ACM SIGPLAN Symp. on Principles and Practice of Parallel Programming (PPoPP), pages 289–300. ACM, 2005.

[CYB`07] Luther Chan, James Yong, Jiaqiang Bai, Ben Leong, and Raymond Tan. Hydra: a massively-multiplayer peer-to-peer architecture for the game developer. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 37–42. ACM, 2007.

182 BIBLIOGRAPHY

[DG99] C Diot and L Gautier. A distributed architecture for multiplayer interactive applications on the internet. IEEE Networks magazine, 13(4):6–15, 1999.

[DJ10] Gregory Dudek and Michael Jenkin. Computational Principles of Mobile Robotics. Cambridge University Press, 2nd edition, 2010.

[DKDV08] A Denault, J Kienzle, C Dionne, and C Verbrugge. Object-Oriented network middleware for massively multiplayer online games. Technical report, SOCS-TR-2008.5 McGill University, Montreal, Canada, 2008.

[DMC96] M Dorigo, V Maniezzo, and A Colorni. Ant system: Optimization by a colony of cooperating agents. IEEE Trans. on Systems, Man, and Cyber- netics (SMC), 26:29–41, 1996.

[Dou02] John R. Douceur. The Sybil attack. In Proc. of Int. Workshop on Peer-to- Peer Systems (IPTPS), pages 251–260. USENIX, 2002.

[DST`08] H. Danielsiek, R. Stuer, A. Thom, N. Beume, B. Naujoks, and M. Preuss. Intelligent Moving of Groups in RTS Games. In IEEE Sym. on Comput. Intell. and Games (CIG), pages 71 –78, dec. 2008.

[eBa11] eBay. http://www.ebay.com/, 2011.

[EGH`03] P Th. Eugster, R Guerraoui, S B Handurukande, P Kouznetsov, and A.-M. Kermarrec. Lightweight probabilistic broadcast. ACM Trans. on Computer Systems (TOCS), 21(4):341–374, 2003.

[EKP`07] T. Eisenbarth, S. Kumar, C. Paar, A. Poschmann, and L. Uhsadel. A survey of lightweight-cryptography implementations. IEEE Design & Test, 24:522–533, 2007.

[EST08] S Egenfeldt-Nielson, J H Smith, and S P Tosca. Understanding Video Games: The Essential Introduction. Routledge, 2008.

[EVE11] EVE Online. http://www.eveonline.com, 2011.

183 BIBLIOGRAPHY

[EVE13] EVE. A new record for the maximum number of simultaneous pilots online. https://forums.eveonline.com/default.aspx?g= posts&m=2979625#post2979625, 2013.

[Fer08a] S Ferretti. Cheating detection through game time modeling: A better way to avoid time cheats in P2P MOGs. Multimedia Tools and Applications, 37(3):339–363, 2008.

[Fer08b] S. Ferretti. A synchronization protocol for supporting peer-to-peer multiplayer online games in overlay networks. In Proc. of Int. Workshop on Distributed Event-Based Systems (DEBS), pages 83–94. ACM, 2008.

[FFX11] Final Fantasy XI. http://www.playonline.com/ff11us, 2011.

[FJ93] S Floyd and V Jacobson. Random early detection gateways for congestion avoidance. IEEE Trans. on Networking (TON), 1(4):397–413, 1993.

[FKS08] W. Feng, E. Kaiser, and T. Schluessler. Stealth measurements for cheat detection in online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 15–20. ACM, 2008.

[FR06] S. Ferretti and M. Roccetti. AC/DC: an algorithm for cheating detection by cheating. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 23:1–23:6. ACM, 2006.

[FRP`08] D Frey, J Royan, R Piegay, A M Kermarrec, E Anceaume, and F Le Fessant. Solipsis: A decentralized architecture for virtual environments. In Proc. of Int. Workshop on Massively Multiuser Virtual Environments (MMVE). IEEE, 2008.

[FS98] E Frécon and M Stenius. DIVE: a scalable network architecture for distributed virtual environments. Distributed Systems Engineering Journal, 5(3):91–100, 1998.

184 BIBLIOGRAPHY

[FSK05] Bryan Ford, Pyda Srisuresh, and Dan Kegel. Peer-to-peer communication across network address translators. In Proc. of the USENIX Annual Techni- cal Conf. (ATC), pages 179–192. USENIX, 2005.

[FTT10] L. Fan, P. Trinder, and H. Taylor. Design issues for peer-to-peer massively multiplayer online games. Proc. of Int. Journal of Adv. Media & Commu- nication (IJAMC), 4:108–125, 2010.

[Fuj00] Richard Fujimoto. Parallel and Distributed Simulation Systems. Wiley Interscience, 2000.

[Fun06] Yeung Siu Fung. Hack-proof synchronization protocol for multiplayer online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 305–331. ACM, 2006.

[Gai11] Gaikai Video Game Network. http://www.gaikai.com, 2011.

[Gam11] GameRank. http://us.playstation.com/psn/, 2011.

[Gar13] Gartner. Worldwide video game market to total $93 billion in 2013. http: //www.gartner.com/newsroom/id/2614915, 2013.

[GB02] C Greenhalgh and S Benford. MASSIVE: a distributed virtual reality system incorporating spatial trading. In Proc. of Int. Conf. on Distributed Com- puting Systems (ICDCS), pages 27–34. IEEE, 2002.

[GD05] P Golle and N Ducheneaut. Preventing bots from playing online games. Computers in Entertainment, 3(3):3–3, 2005.

[GDK99] L Gautier, C Diot, and J Kurose. End-to-end transmission control mechanisms for multiparty interactive applications on the internet. In Proc. of IEEE Int. Conf. on Computer Communications (INFOCOM), volume 3, pages 1470–1479 vol.3. IEEE, 1999.

[GDN`03] L Gao, M Dahlin, A Nayate, J Zheng, and A Iyengar. Application speciﬁc data replication for edge services. In Proc. of Int. World Wide Web Confer- ences (WWW), pages 449–460. ACM, 2003.

185 BIBLIOGRAPHY

[GE11] J.S. Gilmore and H.A. Engelbrecht. A survey of state persistency in peer- to-peer massively multiplayer online games. IEEE Trans. on Parallel & Distributed Systems (TPDS), PP(99):818–834, 2011.

[GHK`10] Rachid Guerraoui, Kevin Huguenin, Anne-Marie Kermarrec, Maxime Monod, and Swagatika Prusty. LiFTinG: Lightweight protocol for freerider- tracking in gossip. In Proc. of Int. Middleware Conf. (Middleware), pages 313–333. ACM/IFIP/USENIX, 2010.

[GJA03] Minaxi Gupta, Paul Judge, and Mostafa Ammar. A reputation system for peer-to-peer networks. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 144–152. ACM, 2003.

[GL02] S Gilbert and N Lynch. Brewer’s conjecture and the feasibility of consistent, available, partition-tolerant web services. ACM SIGACT News, 33(2):51– 59, 2002.

[GLZ05] C GauthierDickey, V Lo, and D Zappala. Using n-trees for scalable event ordering in peer-to-peer games. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 87–92. ACM, 2005.

[Goo08] J Goodman. A hybrid design for cheat detection in massively multiplayer online games. Master’s thesis, McGill University, 2008.

[GSG02] K.P. Gummadi, S. Saroiu, and S.D. Gribble. King: Estimating Latency Be- tween Arbitrary Internet end Hosts. In In Proc. of the 2nd ACM SIGCOMM Workshop on Internet measurment (IMW), pages 5–18, 2002.

[GV08] J Goodman and C Verbrugge. A peer auditing scheme for cheat elimination in MMOGs. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 9–14. ACM, 2008.

[GZLM04] C GauthierDickey, D Zappala, V Lo, and J Marr. Low latency and cheat- proof event ordering for peer-to-peer games. In Proc. of Int. Workshop on

186 BIBLIOGRAPHY

Network & Operating Systems Support for Digital audio & Video (NOSS- DAV), pages 134–139. ACM, 2004.

[HARD10a] A Haeberlen, P Aditya, R Rodrigues, and P Druschel. Accountable virtual machines. In Proc. of Int. Conf. on Operating Systems Design & Implemen- tation (OSDI), pages 1–16. USENIX, 2010.

[HARD10b] Andreas Haeberlen, Paarijaat Aditya, Rodrigo Rodrigues, and Peter Dr- uschel. Accountable Virtual Machines. In Proc. of Int. Conf. on Operating Systems Design & Implementation (OSDI), pages 1–16, 2010.

[HBH06] Thorsten Hampel, Thomas Bopp, and Robert Hinn. A peer-to-peer architecture for massive multiplayer online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES). ACM, 2006.

[HCC06] S Y Hu, J F Chen, and T H Chen. VON: a scalable peer-to-peer network for virtual environments. IEEE Network, 20(4):22–31, 2006.

[HCJ08] S Y Hu, S C Chang, and J R Jiang. Voronoi state management for peer- to-peer massively multiplayer online games. In Proc. of Int. IEEE Conf. on Consumer Communications & Networking (CCNC), pages 1134 – 1138. IEEE, 2008.

[Hen04] M Henning. Massively multiplayer middleware. Queue Magazine, 1(10):38–45, 2004.

[HFB09] J Harri, F Filali, and C Bonnet. Mobility models for vehicular ad hoc networks: a survey and taxonomy. IEEE Communications Surveys Tutorials, 11:19–41, 2009.

[HHJ08a] G Y Huang, S Y Hu, and J R Jiang. Scalable reputation management for p2p MMOGs. In Proc. of Int. Workshop on Massively Multiuser Virtual Environments (MMVE), pages 1–5. IEEE, 2008.

187 BIBLIOGRAPHY

[HHJ08b] G Y Huang, S Y Hu, and J R Jiang. Scalable reputation management with trustworthy user selection for p2p MMOGs. Proc. of Int. Journal of Adv. Media & Communication (IJAMC), 2:380–401, 2008.

[HHLH10] R C Harvey, A Hamza, C Ly, and M Hefeeda. Energy-efﬁcient gaming on mobile devices using dead reckoning-based power management. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2010.

[HJ08] Johan Hagelbäck and Stefan J. Johansson. Using Multi-Agent Potential Fields in Real-Time Strategy Games. In Proc. of the Int. Joint Conf. on Autonomous agents and multiagent systems, pages 631–638, 2008.

[HJC10] S Y Hu, J. R. Jiang, and B Y Chen. Peer-to-peer 3D streaming. IEEE Internet Computing, 14(2):54–61, 2010.

[HL04] S Y Hu and G M Liao. Scalable peer-to-peer networked virtual environment. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 129–133. ACM, 2004.

[HLN`07] Wenbo He, Xue Liu, Hoang Nguyen, K. Nahrstedt, and T.T. Abdelzaher. Pda: Privacy-preserving data aggregation in wireless sensor networks. In Proc. of IEEE Int. Conf. on Computer Communications (INFOCOM), pages 2045–2053, may 2007.

[HLR11] Wenbo He, Xue Liu, and Mai Ren. Location cheating: A security challenge to location-based social network services. In Proc. of Int. Conf. on Distributed Computing Systems (ICDCS), pages 740–749, 2011.

[HM07] Greg Hoglund and Gary McGraw. Exploiting Online Games: Cheating Massively Distributed Systems. Addison-Wesley, 1 edition, July 2007.

[HNP97] J A Hamilton, D A Nash, and U W Pooch. Distributed simulation. CRC, 1997.

188 BIBLIOGRAPHY

[Hun05] Robin Hunicke. The Case for Dynamic Difﬁculty Adjustment in Games. In Proc. of the ACM SIGCHI Int. Conf. on Advances in computer entertainment technology ACE, pages 429–433, 2005.

[HYK11] K. Huguenin, A. Yahyavi, and B. Kemme. Cheat detection and prevention in P2P MOGs. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–2. ACM, 2011.

[IEE95] IEEE. IEEE standard for distributed interactive simulation application protocols, IEEE standard 1278.1-1995. Technical report, IEEE, September 1995.

[IEE96] IEEE Standard for Distributed Interactive Simulation – Application Proto- cols, 1996. 10.1109/IEEESTD.1996. 80831.

[IEE00] IEEE. IEEE standard for modelling and simulation high level architecture (HLA). Technical report, IEEE, September 2000.

[IHK04] T Iimura, H Hazeyama, and Y Kadobayashi. Zoned federation of game servers: a peer-to-peer approach to scalable multiplayer online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 116–120. ACM, 2004.

[ILc10] A. Iosup, A. Lasc˘ ateu,˘ and N. ¸Tapu¸s.˘ CAMEO: enabling social networks for massively multiplayer online games through continuous analytics and cloud computing. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2010.

[IYM`06] T Izaiku, S Yamamoto, Y Murata, N Shibata, K Yasumoto, and M Ito. Cheat detection for MMORPG on P2P environments. In Proc. of Int. ACM SIG- COMM Workshop on Network & System Support for Games (NETGAMES). ACM, 2006.

[Jef85] D R Jefferson. Virtual time. ACM Trans. on Programming Language Sys- tems, 7(3):404–425, 1985.

189 BIBLIOGRAPHY

[JET03] J E Jaramillo, L Escobar, and H Trefftz. Area of interest management by grid-based discrete aura approximations for distributed virtual environments. In Proc. of Int. Symp. on Virtual Reality (SVR), pages 342–353, 2003.

[JMB09] M Jelasity, A Montresor, and O Babaoglu. T-Man: gossip-based fast overlay topology construction. Computer Networks, 53:2321–2339, August 2009.

[KAM04] Y Kawahara, T Aoyama, and H Morikawa. A peer-to-peer message exchange scheme for large-scale networked virtual environments. Telecom- munication Systems, 25(3):353–370, March 2004.

[KDC10] Santosh Kulkarni, Scott Douglas, and David Churchill. Badumna: A de- centralised network engine for virtual environments. Computer Networks, 54(12):1953 – 1967, 2010.

[KF09] E Kaiser and W C Feng. PlayerRating: a reputation system for multiplayer online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 8:1–8:6. ACM, 2009.

[KK07] H Korhonen and E M I Koivisto. Playability heuristics for mobile multiplayer games. In Proc. of Int. Conf. on Digital Interactive Media in Enter- tainment & Arts (DIMEA), pages 28–35. ACM, 2007.

[KLXH04] B Knutsson, H Lu, W Xu, and B Hopkins. Peer-to-peer support for massively multiplayer games. In Proc. of IEEE Int. Conf. on Computer Com- munications (INFOCOM), pages 97–107. IEEE, 2004.

[KS03] J Keller and G Simon. Solipsis: A massively multi-participant virtual world. In Proc. of Int. Conf. on Parallel and Distributed Processing Techniques and Applications (PDPTA), 2003.

[KSGM03] S Kamvar, M Schlosser, and H Garcia-Molina. The EigenTrust algorithm for reputation management in P2P networks. In Proc. of Int. World Wide Web Conferences (WWW), pages 640–651. ACM, 2003.

190 BIBLIOGRAPHY

[KTCB05] P Kabus, W Terpstra, M Cilia, and A Buchmann. Addressing cheating in distributed MMOGs. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2005.

[KVB`09] J Kienzle, C Verbrugge, Kemme B, A Denault, and M Hawker. Mammoth: a massively multiplayer game research framework. In Proc. of Int. Conf. on Foundations of Digital Games (FDG), pages 308–315. ACM, 2009.

[LABP08] Youngki Lee, Sharad Agarwal, Chris Butcher, and Jitu Padhye. Measure- ment and Estimation of Network QoS Among Peer Xbox 360 Game Play- ers. In In Proc. of Passive and Active Measurement Conference PAM, pages 41–50, 2008.

[LB11] Ricardo Lopes and Rafael Bidarra. Adaptivity Challenges in Games and Simulations: A Survey. IEEE Trans. Comput. Intellig. and AI in Games, 3(2):85–99, 2011.

[LBSB09] Cristian Lumezanu, Randy Baden, Neil Spring, and Bobby Bhattacharjee. Triangle Inequality Variations in the Internet. In In Proc. of Internet Mea- surement Conference IMC, pages 177–183, 2009.

[LCM`08] H C. Li, A Clement, M Marchetti, M Kapritsos, L Robison, L Alvisi, and M Dahlin. FlightPath: Obedience vs. choice in cooperative services. In Proc. of Int. Conf. on Operating Systems Design & Implementation (OSDI), pages 355–368. USENIX, 2008.

[LCW`06] Harry C. Li, Allen Clement, Edmund L. Wong, Jeff Napper, Indrajit Roy, Lorenzo Alvisi, and Michael Dahlin. BAR Gossip. In Proc. of Int. Conf. on Operating Systems Design & Implementation (OSDI), 2006.

[LDMW04] K Li, S Ding, D McCreary, and S Webb. Analysis of state exposure control to prevent cheating in online games. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 140–145. ACM, 2004.

191 BIBLIOGRAPHY

[LGD98] E Lety, L Gautier, and C Diot. MiMaze, a 3d multiplayer game on the internet. In Proc. of Int. Conf. on Virtual Systems & Multimedia, pages 84–9. IEEE, 1998.

[LHK`09] K Lee, S Hong, S J Kim, I Rhee, and S Chong. Slaw: A mobility model for human walks. In Proc. of IEEE Int. Conf. on Computer Communications (INFOCOM), pages 855–863. IEEE, 2009.

[LL08] H I Liu and Y T Lo. DaCAP - a distributed anti-cheating peer to peer architecture for massive multiplayer on-line role playing game. In Proc. of Int. Symp. on Cluster Computing & the Grid (CCGRID), pages 584 – 589. IEEE/ACM, 2008.

[LPBC07] P Laurens, R Paige, P Brooke, and H Chivers. A novel approach to the detection of cheating in multiplayer online games. In Proc. of Int. Conf. on Engineering of Complex Computer Systems (ICECCS), pages 97–106. IEEE, 2007.

[LTB04] E. Lety, T. Turletti, and F. Baccelli. SCORE: a scalable communication protocol for large-scale virtual environments. IEEE Trans. on Networking (TON), 12(2):247–260, 2004.

[MC10] J L Miller and J Crowcroft. The near-term feasibility of P2P MMOGs. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2010.

[MDF11] S Moffat, A Dua, and W C Feng. SpotCheck: An efﬁcient defense against information exposure cheats. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 8:1–8:6. ACM, 2011.

[MF09] Ian Millington and John Funge. Artiﬁcial Intelligence for Games. Morgan Kaufmann, second edition, 2009.

192 BIBLIOGRAPHY

[MG09] R Morales and I Gupta. AVMON: Optimal and scalable discovery of consistent availability monitoring overlays for distributed systems. IEEE Trans. on Parallel & Distributed Systems (TPDS), 20:446–459, 2009.

[MGBY99] Y Makbily, C Gotsman, and R Bar-Yehuda. Geometric algorithms for message ﬁltering in decentralized virtual environments. In ACM Interactive 3D graphics (I3D), pages 39–46. ACM, 1999.

[MGM06] C Mönch, G Grimen, and R Midtstraum. Protecting online games against cheating. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES). ACM, 2006.

[Min] MiniUPnP. http://miniupnp.free.fr.

[MKY`07] S Mogaki, M Kamada, T Yonekura, S Okamoto, Y Ohtaki, and M Reaz. Time-stamp service makes real-time gaming cheat-free. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES). ACM, 2007.

[ML06] C. Miles and S.J. Louis. Towards the Co-Evolution of Inﬂuence Map Tree Based Strategy Game Players. In IEEE Sym. on Comput. Intell. and Games (CIG), pages 75–82, may 2006.

[MMWD05] A McCoy, S McLoone, T Ward, and D Delaney. Dynamic hybrid strategy models for networked mulitplayer games. In Proc. of Int. Euro. Conf. on Modelling & Simulation (ECMS), pages 727–732, 2005.

[Mor96] K L Morse. Interest management in large-scale distributed simulations. Technical Report ICS-TR-96-27, 1996.

[MPK03] J Mulligan, B Patrovsky, and R Koster. Developing online games: An in- sider’s guide. Pearson Education, 2003.

[Mur11] Curtiss Murphy. Game Engine Gems 2, chapter Believable Dead Reckoning for Networked Games, pages 308–326. AK Peters/CRC Press, 2011.

193 BIBLIOGRAPHY

[MVHE04] M Mauve, J Vogel, V Hilt, and W Effelsberg. Local-lag and timewarp: providing consistency for replicated continuous applications. IEEE Trans. on Multimedia, 6:47–57, 2004.

[MWMD07a] A McCoy, T Ward, S McLoone, and D Delaney. Multistep-ahead neural- network predictors for network trafﬁc reduction in distributed interactive applications. ACM Trans. on Modeling & Computer Simulation (TOMACS), 17:1–30, 2007.

[MWMD07b] Aaron McCoy, Tomas Ward, Seamus McLoone, and Declan Delaney. Multistep-Ahead Neural-Network Predictors for Network Trafﬁc Reduc- tion in Distributed Interactive Applications. ACM Trans. on Modeling & Computer Simulation (TOMACS), 17:1–30, 2007.

[MZP`95] M R Macedonia, M J Zyda, D R Pratt, D P Brutzman, and P Barham. Ex- ploiting reality with multicast groups. IEEE Computer Graphics and Ap- plications, 15(5):38–45, 1995.

[Nin11] DS at Nintendo. http://www.nintendo.com/ds, 2011.

[NK10] M.T. Najaran and C. Krasic. Scaling online games with adaptive interest management in the cloud. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6, 2010.

[NPD13] NPD, Gfk, IDG, MSFT Estimates. http://news.xbox.com/2013/ 05/x360-aaron-greenberg-industry-growth, 2013.

[NWL`06] W Nagel, W Walter, W Lehner, E Leontiadis, V Dimakopoulos, and E Pi- toura. Creating and maintaining replicas in unstructured peer-to-peer systems. In Proc. of Int. Euro. Conf. on Parallel Processing (EuroPar), volume 4128, pages 1015–1025. Springer, 2006.

[OnL11] OnLive. http://www.onlive.com, 2011.

[OSC`11] J S Otto, M A Sánchez, D R Choffnes, F Bustamante, and G Siganos. On blind mice and the elephant: understanding the network impact of a large

194 BIBLIOGRAPHY

distributed system. In Proc. of ACM SIGCOMM Conference, pages 110– 121. ACM, 2011.

[ÖV11] M. Tamer Özsu and P Valduriez. Principles of Distributed Database Sys- tems. Springer US, 2011.

[PFCR06] C E Palazzi, S Ferretti, S Cacciaguerra, and M Roccetti. Interactivity-loss avoidance in event delivery synchronization for mirrored game architectures. IEEE Trans. on Multimedia, 8(4):874–879, 2006.

[PG07] D Pittman and C GauthierDickey. A measurement study of virtual populations in massively multiplayer online games. In Proc. of Int. ACM SIG- COMM Workshop on Network & System Support for Games (NETGAMES), pages 25–30. ACM, 2007.

[PG11] D Pittman and C GauthierDickey. Cheat-proof peer-to-peer trading card games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2011.

[PSN11] Playstation network PSN. http://us.playstation.com/psn/, 2011.

[PSP11] PlayStation Portable. http://us.playstation.com/psp, 2011.

[Pun11] PunkBuster: the original anti-cheat system for online multiplayer games. http://www.evenbalance.com/, 2011.

[PW02a] L Pantel and L Wolf. On the suitability of dead reckoning schemes for games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 79–84. ACM, 2002.

[PW02b] L Pantel and L C Wolf. On the impact of delay on real-time multiplayer games. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 23–29. ACM, 2002.

[Qua11] Quazal. http://www.quazal.com/en/products/net-z, 2011.

195 BIBLIOGRAPHY

[RAM`08] D Roberts, R Aspin, D Marshall, S Mcloone, D Delaney, and T Ward. Bounding inconsistency using a novel threshold metric for dead reckoning update packet generation. Simulation, 84(5):239–256, 2008.

[RD01a] A Rowstron and P Druschel. Pastry: Scalable, decentralized object location, and routing for large-scale peer-to-peer systems. In Proc. of Int. Middleware Conf. (Middleware), pages 329–350. ACM/IFIP/USENIX, 2001.

[RD01b] A. Rowstron and P. Druschel. Storage management and caching in PAST, a large-scale, persistent peer-to-peer storage utility. Operating System Review (SIGOPS), 35:188–201, 2001.

[RMMW08] J. Rosenberg, R. Mahy, P. Matthews, and D. Wing. Session Traversal Utili- ties for NATs (STUN). Technical Report RFC 5389, IETF, 2008.

[SAZ10] G Swamynathan, K C. Almeroth, and B Y. Zhao. The design of a reliable reputation system. Electronic Commerce Research, 10:239–270, December 2010.

[SdDF`11] S Sundaresan, W de Donato, N Feamster, R Teixeira, S Crawford, and A Pescapè. Broadband internet performance: a view from the gateway. In Proc. of ACM SIGCOMM Conference, pages 134–145. ACM, 2011.

[SDM09] M Suznjevic, O Dobrijevic, and M Matijasevic. MMORPG player actions: Network performance, session patterns and latency requirements analysis. Multimedia Tools and applications, 45(1):191–214, 2009.

[Sec11] Second Life: Virtual Worlds, Avatars, free 3D chat, online meetings. http://secondlife.com, 2011.

[SGB`03] N Sheldon, E Girard, S Borg, M Claypool, and E Agu. The effect of latency on user performance in Warcraft III. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 3–14. ACM, 2003.

196 BIBLIOGRAPHY

[SGJ07] T Schluessler, S Goglin, and E Johnson. Is a bot at the controls?: Detecting input data attacks. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2007.

[SH00] T Stützle and H Hoos. Max-min ant system. Future Generation Computer Systems, 16:889–914, 2000.

[Sho11] ShowEQ. http://www.showeq.net, 2011.

[SLY`09] Daryl Seah, Wai Kay Leong, Qingwei Yang, Ben Leong, and Ali Razeen. Peer NAT proxies for peer-to-peer games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 6:1–6. ACM, 2009.

[SM12] Mirko Suznjevic and Maja Matijasevic. Player behavior and trafﬁc char- acterization for MMORPGs: a survey. Multimedia Systems, pages 1–22, 2012.

[SMK`01] I Stoica, R Morris, D Karger, M F Kaashoek, and H Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. In Proc. of ACM SIGCOMM Conference, pages 149–160. ACM, 2001.

[SNW08] M Sozio, T Neumann, and G Weikum. Near-optimal dynamic replication in unstructured peer-to-peer networks. In Proc. of the ACM Int. Symp,. on Principles of Database Systems (PODS), pages 281–290. ACM, 2008.

[SSJ`08] A Schmieg, M Stieler, S Jeckel, P Kabus, B Kemme, and A Buchmann. pSense - maintaining a dynamic localized peer-to-peer structure for position based multicast in games. In Proc. of Int. Conf. on Peer-to-Peer Computing (P2P), pages 247–256. IEEE, 2008.

[SSM11] Mirko Suznjevic, Ivana Stupar, and Maja Matijasevic. Trafﬁc modeling of player action categories in a MMORPG. In Proc. of International ICST Conf. on Simulation Tools and Techniques(SIMUTools), pages 280–287. ICST, 2011.

197 BIBLIOGRAPHY

[SW10] Stefan Saroiu and Alec Wolman. I am a sensor, and i approve this message. In ACM workshop on Mobile Computing Systems and Applications HOTMOBILE, pages 37–42. ACM, 2010.

[SZ99] S. Singhal and M. Zyda. Networked Virtual Environments: Design and Implementation. Addison-Wesley, 1999.

[SZ03] Katie Salen and Eric Zimmerman. Rules of Play: Game Design Fundamen- tals. The MIT Press, 2003.

[SZA08] G Swamynathan, B Y. Zhao, and K C. Almeroth. Exploring the feasibility of proactive reputations. Concurrency and Computation: Practice & Experience, 20:155–166, 2008.

[Ter02] Terazona. Terazona: Zona application frame work white paper. http: //www.zona.net/whitepaper/Zonawhitepaper.pdf, 2002.

[THS07] E Tanin, A Harwood, and H Samet. Using a distributed quadtree index in peer-to-peer networks. Int. Conf. on Very Large Data Bases (VLDB), 16:165–178, 2007.

[TTP`95] D. B. Terry, M. M. Theimer, Karin Petersen, A. J. Demers, M. J. Spreitzer, and C. H. Hauser. Managing update conﬂicts in bayou, a weakly connected replicated storage system. In Proc. of ACM Symp. on Operating System Principles (SOSP), pages 172–182. ACM, 1995.

[uLo12] uLobby. http://muchdifferent.com/?page= game-unitypark-products-ulobby, 2012.

[VAC13] VAC: valve anti cheat. https://support.steampowered.com/ kb_article.php?p_faqid=370, 2013.

[VDB09] M. Varvello, C. Diout, and E. Biersack. P2P Second Life: Experimental validation using Kad. In Proc. of IEEE Int. Conf. on Computer Communi- cations (INFOCOM), pages 1161–1169. IEEE, 2009.

198 BIBLIOGRAPHY

[VFBD09] M Varvello, S Ferrari, E Biersack, and C Diot. Distributed avatar management for Second Life. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 5:1–5:6. IEEE, 2009.

[VFBD11] M. Varvello, S. Ferrari, E. Biersack, and C. Diot. Exploring Second Life. IEEE Trans. on Networking (TON), 19(1):80–91, February 2011.

[VV10] M Varvello and G M Voelker. Second Life: a social network of humans and bots. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 9–14. ACM, 2010.

[Wav01] J. M. P. Van Waveren. The Quake III Arena Bot. Master’s thesis, University of Technology Delft, Netherlands, 2001.

[WG08] N. Wirth and M. Gallagher. An Inﬂuence Map Model for Playing Ms. Pac- Man. In IEEE Sym. on Comput. Intell. and Games (CIG), pages 228 –233, 2008.

[WHT09] C H Wu, S Y Hu, and L M Tseng. Discovery of physical neighbors for p2p 3d streaming. In Proc. of Int. Conf. on Ultra Modern Telecommunications Workshops (ICUMT), pages 1–6. IEEE, 2009.

[WK07] A Wierzbicki and T Kaszuba. Practical trust management without reputation in peer-to-peer games. Multiagent Grid Systems, 3:411–428, December 2007.

[WKV`09] X Wang, H Kim, A V Vasilakos, T T Kwon, Y Choi, S Choi, and H Jang. Measurement and analysis of world of warcraft in mobile WiMax networks. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2009.

[WS07] S D Webb and S Soh. Cheating in networked computer games: a review. In Proc. of Int. Conf. on Digital Interactive Media in Entertainment & Arts (DIMEA), pages 105–112. ACM, 2007.

199 BIBLIOGRAPHY

[WSL07] S Webb, S Soh, and W Lau. RACS: A referee anti-cheat scheme for P2P gaming. In Proc. of Int. Workshop on Network & Operating Systems Sup- port for Digital audio & Video (NOSSDAV). ACM, 2007.

[WST09] S Daniel Webb, S Soh, and J L Trahan. Secure referee selection for fair and responsive peer-to-peer gaming. Simulation, 85:608–618, 2009.

[XBo11] XBox Live. http://www.xbox.com/live, 2011.

[XBZ12] Zhi Xu, Kun Bai, and Sencun Zhu. Taplogger: inferring user inputs on smartphone touchscreens using on-board motion sensors. In ACM Conf. on Security and Privacy in Wireless and Mobile Networks WISEC, pages 113–124. ACM, 2012.

[YHGS`13] Amir Yahyavi, KÃl’vin Huguenin, Julien Gascon-Samson, Jorg Kienzle, and Bettina Kemme. Watchmen: Scalable cheat-resistant support for distributed multiplayer online games. In Proc. of Int. Conf. on Distributed Computing Systems (ICDCS), pages 134–144. ACM, 2013.

[YHK11] A Yahyavi, K Huguenin, and B Kemme. AntReckoning: A pheromone- based dead reckoning algorithm for games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–6. ACM, 2011.

[YHK12] A. Yahyavi, K. Huguenin, and B. Kemme. Interest modeling in games: The case of dead reckoning. Multimedia Systems (MMSJ), pages 1–16, July 2012.

[YK13] Amir Yahyavi and Bettina Kemme. Peer-to-peer architectures for massively multiplayer online games: A survey. ACM Computing Surveys (CSUR), 46(1):9:1–9:51, July 2013.

[YMYI05] S Yamamoto, Y Murata, K Yasumoto, and M Ito. A distributed event delivery method with load balancing for MMORPG. In Proc. of Int. ACM SIG- COMM Workshop on Network & System Support for Games (NETGAMES), pages 1–8. ACM, 2005.

200 BIBLIOGRAPHY

[YPK13] Amir Yahyavi, Jeffrey Pang, and Bettina Kemme. Towards providing security for mobile games. In ACM MobiCom International Workshop on Mobility in the Evolving Internet Architecture, MobiArch ’13, pages 47– 52. ACM, 2013.

[YR05] J Yan and B Randell. A systematic classiﬁcation of cheating in online games. In Proc. of Int. ACM SIGCOMM Workshop on Network & System Support for Games (NETGAMES), pages 1–9. ACM, 2005.

[YTVK13] A. Yahyavi, J. Tremblay, C. Verbrugge, and B. Kemme. Towards the design of a human-like FPS NPC using pheromone maps. In IEEE International Games Innovation Conference (IGIC), pages 275–282. IEEE, Sept 2013.

[YV05] A Yu and S T Vuong. MOPAR: a mobile peer-to-peer overlay architecture for interest management of massively multiplayer online games. In Proc. of Int. Workshop on Network & Operating Systems Support for Digital audio & Video (NOSSDAV), pages 99–104. ACM, 2005.

[Zha10] K Zhang. Persistent transaction models for massively multiplayer online games. Master’s thesis, McGill University, 2010.

[ZK11] Kaiwen Zhang and Bettina Kemme. Transaction models for massively multiplayer online games. In Proc. of the Int. Symp. on Reliable Distributed Systems (SRDS), pages 31–40. IEEE, 2011.

[ZKJ01] B Y Zhao, J D Kubiatowicz, and A D Joseph. Tapestry: An infrastructure for fault-tolerant wide-area location and. Technical report, UC Berkeley, 2001.

[Zyn11] Zynga - Connecting the world through games. http://www.zynga. com/games, 2011.

[ZZXM09] Y Zheng, L Zhang, X Xie, and W Y Ma. Mining interesting locations and travel sequences from GPS trajectories. In Proc. of Int. World Wide Web Conferences (WWW). ACM, 2009.

201 Acronyms

MMOG Massively Multiplayer Online Games MOG Multiplayer Online Games P2P Peer-to-Peer NPC Non-Player Character AI Artiﬁcal Intelligence NAT Network Address Translation RTS Real Time Strategy RPG Role Playing Game FPS Fist Person Shooter QIII Quake III AOI Area-of-Interest SDK Software Development Kit CAP Consistency, Availability, Partition tolerance PCM Predictive Contract Mechanism QOE Quality of Experience CS Client-Server LOD Level-of-Detail DHT Distributed Hash-Table DOS Denial of Service IS Interest Set VS Vision Set VF Vision Field API Application Programming Interface ESM End System Multicast TFT Tit-for-Tat AAS Area Awareness System

202