Cascading Impact of Lag on User Experience in Multiplayer Games

Eben Howard†, Clint Cooper†, Mike P. Wittie†, Stephen Swinford*, Qing Yang† †Department of Computer Science, *Department of Sociology and Anthropology Montana State University {ehoward, clint.cooper, mwittie, qing.yang}@cs.montana.edu, [email protected]

1. INTRODUCTION 3. METHODOLOGY Playing cooperative multiplayer games should be fun for We measured QoE metrics during sessions of Mass Ef- everyone involved and part of having fun in games is being fect 3, a popular cooperative online game by Bioware [2]. able to perform well, be immersed, and stay engaged [13, 17]. Four factor sets were used to emulate lag for one of the These indicators of enjoyment are part of a user’s Quality group members. The factor sets include representative net- of Experience (QoE), a measure which further includes ad- work conditions of a user located in the same city, state, ditional metrics such as attention levels and ability to suc- country, and continent as the other group members [12]. ceed. Players stop playing the game when it ceases to pro- QoE was measured during each game session using both vide a high enough QoE, especially in cooperative and social subjective and objective metrics. A questionnaire given af- games. [8, 18, 19]. ter each match was used to record Enjoyment, Immersion, Industry application development and current research and Engagement [21]. Game Score was recorded for each both operate with the assumption that for any given in- individual after each match. A wearable EEG device was dividual in a group, that individual’s QoE is affected only used to measure player Attention level during the match [1]. by their own network condition and not the network condi- tions of the other group members [4, 7, 8]. We show that QoE Metric Correlation (r2) p-value this assumption is incorrect. Enjoyment ≈ 0.56 < 0.002 Our research shows that the QoE of all group members Immersion ≈ 0.56 < 0.002 is negatively affected by a single member’s lag (communica- Engagement ≈ 0.38 < 0.015 tion delay, or loss caused by poor network conditions). Un- Score ≈ 0.62 < 0.001 derstanding a user’s QoE as a function that includes other Attention ≈ 0.52 < 0.003 users’ network conditions has the potential to improve lag mitigation strategies for multiplayer games and other group Table 1: QoE Metric Correlations applications.

2. RELATED WORK 4. RESULTS Prior studies evaluate the effect of lag on a single user’s The collected data show that the QoE of a group member QoE [3, 4, 5, 8, 9] and show that increasing a single user’s lag is negatively impacted by the decreased network conditions through higher latency, jitter, or loss decreases that user’s of other group members. This is counter to previous as- QoE in multiplayer games. sumptions that a group member’s QoE is only impacted by QoE for interactions of users in group applications has their own network conditions [4, 7, 8]. been studied with simulations using artificial intelligence (AI) Table 1 shows each QoE metric’s level and strength of players, observational studies of users in real-world applica- correlation between the lagged individual’s degradation of tions, and as by-products of studies focused on single indi- QoE due to lag and that of the other members of the group. viduals’ QoE [7, 10, 11, 14, 15, 16, 18, 22]. These previous All correlations have very strong p-values and significant studies considered QoE of group members with lag, but did correlation levels. not examine the QoE of group members not experiencing Our results indicate that reducing lag of of one user will lag. improve their QoE and the QoE of the entire group. Having Our work explores a different aspect of user experience in a group member lag decreases the enjoyment for everyone, that we look at QoE of all group members within a popular but understanding that lag has a cascading impact opens real world collaborative application. Unlike work of Park many new areas of systems research and application devel- and Kenyon and of Beznosyk et al., we consider interactions opment. For example, prioritizing game requests of only the of more than two human participants [6, 16]. We examine lagged users in data centers using mechanisms such as D3 the cascading impact of lag by adjusting the level of network can improve application ausability for all users [20]. performance on the path to the game server for only a single member of the group while taking QoE measurements for all group members. 5. REFERENCES tial Internet access. In Passive and Active Measurement, January 2012. [1] Neurosky’s esense meters and detection of mental state. [13] R. Hirota and S. Kuribayash. Evaluation of fairness in multi- white paper, September 2009. player network games. In Pacific Rim Conference on Com- [2] Bioware accolades. http://masseffect.bioware.com/ munications, Computers and Signal Processing (PacRim). about/accolades/, February 2013. August, 2011. [3] G. Armitage. An experimental estimation of latency sensi- [14] A. Kaiser, D. Maggiorini, N. Achir, and K. Boussetta. On tivity in multiplayer Quake 3. In International Conference the objective evaluation of real-time networked games. In on Networks, 2003. Global Telecommunications Conference, November 2009. [4] G. Armitage, M. Claypool, P. Branch, J. Wiley, G. Armitage, [15] W. Palant, C. Griwodz, and P. Halvorsen. Consistency re- and M. Claypool. Networking and Online Games - Under- quirements in multiplayer online games. In SIGCOMM work- standing and Engineering Multiplayer Internet Games. John shop on Network and system support for games, October Wiley & Sons Ltd, June 2006. 2006. [5] T. Beigbeder, R. Coughlan, C. Lusher, J. Plunkett, E. Agu, [16] K. S. Park and R. V. Kenyon. Effects of network character- and M. Claypool. The effects of loss and latency on user istics on human performance in a collaborative virtual envi- performance in unreal tournament 2003 R . In SIGCOMM ronmnet. In IEEE Virtual Reality, March 1999. workshop on Network and system support for games, August [17] M. Seif El-Nasr, B. Aghabeigi, D. Milam, M. Erfani, 2004. B. Lameman, H. Maygoli, and S. Mah. Understanding and [6] A. Beznosyk, P. Quax, K. Coninx, and W. Lamotte. In- evaluating cooperative games. In International conference fluence of network delay and jitter on cooperation in multi- on Human factors in computing systems, April 2010. player games. In Virtual Reality Continuum and Its Appli- [18] M. Suznjevic, O. Dobrijevic, and M. Matijasevic. MMORPG cations in Industry (VRCAI), December 2011. player actions: Network performance, session patterns and [7] M. Bredel and M. Fidler. A measurement study regarding latency requirements analysis. Multimedia Tools and Appli- quality of service and its impact on multiplayer online games. cations, 45:191–214, October 2009. In Workshop on Network and Systems Support for Games, [19] P. Tarng, K. Chen, and P. Huang. On prophesying on- November 2010. line gamer departure. In Network and Systems Support for [8] K.-T. Chen, P. Huang, and C.-L. Lei. Effect of network qual- Games (NetGames), November 2009. ity on player departure behavior in online games. Parallel [20] C. Wilson, H. Ballani, T. Karagiannis, and A. Rowtron. Distributed Systems, 20:593–606, May 2009. Better never than late: meeting deadlines in datacenter [9] M. Claypool and K. Claypool. Latency and player actions networks. In SIGCOMM Computer Communication Review, in online games. Communications of the ACM, 49:40–45, August 2011. November 2006. [21] W. Wu, A. Arefin, R. Rivas, K. Nahrstedt, R. Sheppard, [10] M. Dick, O. Wellnitz, and L. Wolf. Analysis of factors af- and Z. Yang. Quality of experience in distributed interactive fecting players’ performance and perception in multiplayer games. In SIGCOMM workshop on Network and system multimedia environments: toward a theoretical framework. support for games, October 2005. In International conference on Multimedia, October 2009. [11] T. Fritsch, H. Ritter, and J. Schiller. The effect of latency [22] T. Yasui, Y. Ishibashi, and T. Ikedo. Influences of network and network limitations on MMORPGs: a field study of Ev- latency and packet loss on consistency in networked racing erquest 2. In SIGCOMM workshop on Network and system games. In SIGCOMM workshop on Network and system support for games, October 2005. support for games, October 2005. [12] O. Goga and R. Teixeira. Speed measurements of residen-