By Kuan-Ta Chen, Polly Huang, and Chin-Laung Lei HOW SENSITIVE ARE ONLINE GAMERS TO NETWORK QUALITY? Game-playing time is Network quality is critical to real-time, interactive, online game play. Yet even though today’s best-effort strongly related to Internet does not provide quality of service (QoS) network QoS, helping guarantees, the popularity of online gaming is only determine user increasing. This popularity may be because QoS is satisfaction and deliver not really a must-have characteristic or because play- ers are accustomed to struggling with and playing better service quality to through unfavorable network conditions. online gamers. Ironically, despite the popularity of online games, complaints about high “ping times” and “lags” con- tinue to surge in game-player forums (such as www.gamedev.net/community/forums/). It would appear that most players view network latency and loss as a major hindrance to an enjoyable gaming experience. Here, we investigate whether players are really as sensi- tive to network quality as they claim. If they are, we would like to answer whether they quit games earlier than they might otherwise due to unsatisfactory network conditions. A number of studies have sought to evalu- outweigh their QoS-intolerance and affect ate the effect of network quality on online their decisions. Besides, it is very costly to gamers [1–3, 5, 7–9, 11]. In most of them, a scale up such studies by including more series of games is played in a controlled net- human subjects. Meanwhile, objective evalu- work environment in which the gaming ations are usually based on user performance experience of subjects is graded either subjec- in a specific context (such as the number of tively or objectively. Subjective tests require kills in a shooting game, the time needed to participants to report their feelings about complete each lap in a racing game, and the playing games with different levels of network capital accumulated in a strategy game). QoS. While this approach might capture However, game scores are highly dependent human perceptions in a controlled situation, on player skills, game design, and content, so it cannot measure how players react to poor results are not comparable and generalizable game quality in real life, as other factors may across different games. 34 November 2006/Vol. 49, No. 11 COMMUNICATIONS OF THE ACM E ONLINE GAMERS TO ? Psychologically, the pleasing sensation players expe- UserJoy Technology in Taiwan (www.ewsoft.com. rience in online games is analogous to being in the tw). MMORPGs (such as EverQuest and World of flow state after taking a mood-changing substance like Warcraft) are networked computer role-playing marijuana. One study [6] found that the most signif- games in which a large number (perhaps 10,000) of icant factors related to the flow experience in online players interact with one another in a virtual world. gaming are skill, challenge, and involvement. The While many game genres are round- or stage-based, game-playing experience can be described as so plea- a design that allows players to regain consciousness surable that gamers are unaware of the time that of the real world, the adventure in role-playing passes while they’re playing the game. According to games is continuous; no explicit mechanism forces the theory laid out in [6], if the feeling of involvement players to take a break. in the virtual world is diminished by network lags, An uninterrupted sense of immersion in a virtual users become more conscious of the real world, miti- environment is one of the main attractions of gating their sense of time distortion. Some players MMORPGs because it entices players into a flow may simply decide to quit a game as soon as they experience; other attractions are elaborate reward detect unacceptable lags. For these reasons, we con- cycles and evolving social networks [10]. The impor- jecture that game-playing time is affected (to some tance of the sense of immersion implies that network extent) by the network quality players experience. quality, a key factor in maintaining a perfect sense of There are certainly exogenous reasons other than immersion, strongly influences the flow experience of the quality of network conditions that might affect MMORPG players; therefore, game-playing time gamers’ decisions to continue with (or leave) a game. should be shorter if the network conditions that users For example, they might leave a game due to their experience are unsatisfactory. schedule constraints or mental state. On the other Another reason for investigating MMORPGs is hand, they may be tied by social bonds, no matter that they are relatively slow-paced compared to other how unstable the game play might be. Although the popular genres (such as first-person shooter) that usu- behavior of individual players is highly variable and ally require players to make split-second decisions. In unpredictable, we show here that, overall, there is a addition to the pace of a game, there is a great deal of consistent and strong link between player departure variety in how players control their virtual characters. decisions and network QoS. In fast-action genres (such as shooters) players must instruct characters “what” actions to take and “how” NETWORK QUALITY VS. GAME-PLAYING TIME to perform them; for example, to move a character to Our conjecture is validated by real-life traces taken a new location, a player must control each step it from the commercial massively multiplayer online needs to take (such as three steps west followed by five role-playing game Shen Zhou Online developed by steps north). On the other hand, in slow-action gen- COMMUNICATIONS OF THE ACM November 2006/Vol. 49, No. 11 35 res (such as MMORPGs Korean MMORPGs and war strategy games) ) played in Japan [4], r 4 u players instruct characters o showing that the average h ( 3 only “what” to do; that is, e session time ranges from m i t they point out only the 80 to 120 minutes. How- y 2 a location the character l ever, the difference in p should move to; it then e individual game-playing m 1 a automatically heads G times is quite large; for toward it via a route that is 150 200 250 300 example, the shortest either predetermined by a Network latency (msec) 20% of sessions span less game designer or com- ) 6 than 40 minutes, but the r u puted on-the-fly. o top 20% of players spend h ( By any standard, e 4 more than eight continu- m i t MMORPGs are slow- ous hours in a game. y a action games with less-strin- l We use graphical plots p 2 gent service requirements e to illustrate the difference m a than fast-action games. G 0 in the game-playing time Therefore, they could be 52515 35 45 55 of sessions experiencing viewed as a baseline of real- Network delay variation (msec) different levels of net- ) time interactive online r work quality (see Figure u o games such that if network h 1). The figure depicts the ( 6 QoS frustrates MMORPG e association of game-play- m i t players, it should also frus- 4 ing time with network y a l trate players of other game p latency, network delay e 2 genres. m variation, and network a To play Shen Zhou G 0 loss rate, respectively. All Online, thousands of play- 10-10 10-8 10-6 10-4 10-2 three plots indicate that ers pay a monthly sub- Network loss rate the more serious the net- scription fee at a work impairment players convenience store or experienced, the sooner Figure 1. Relationship online. As is typical of between game session time and they were likely to leave the game. The changes in MMORPGs, players can network QoS factors. game-playing time are surprisingly significant. For engage in fights with ran- instance, gamers who experienced 150msec latency dom creatures, train them- played four hours on average, but those experiencing selves to acquire particular skills, partakeChen in fig250msec 1 (11/06) latency played for only one hour on aver- commerce, or take on a quest. With the help of the age—a ratio of 4:1. Moreover, variations in network Shen Zhou Online staff, we recorded all delay and network loss induce more variation in game- inbound/outbound game traffic of the game servers playing time (note the span of the y-axis of the located in Taipai, Taiwan, a total of 15 140-game ses- graphs). sions over two days. Having demonstrated that players are not only sen- We extracted the network performance for each sitive but also reactive to network quality, we still must session based on the sequence number and flags in the define “good quality” and “bad quality.” For example, TCP packet header. We computed network latency given two QoS configurations—“low latency with based on the difference between the acknowledge- moderate variation” and “moderate latency with low ment and the delivery time of nonretransmitted game variation”—which one is better? To answer, we pro- packets and inferred the network loss rate based on pose a model that describes the changes in game-play- the receipt (or lack of receipt) of TCP acknowledge- ing time due to network QoS. This allows us to grade ments. The observed players were spread over 13 the overall quality of game playing based on a set of countries, including China, India, Hong Kong, and network performance metrics (such as latency and Malaysia, and hundreds of autonomous systems, loss) in terms of user satisfaction. manifesting the heterogeneity of network-path char- acteristics—therefore the generality of this trace.