The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 R

The Effects of Loss and Latency on User Performance in Unreal Tournament 2003 r Tom Beigbeder, Rory Coughlan, Corey Lusher, John Plunkett, Emmanuel Agu, Mark Claypool Computer Science Department at Worcester Polytechnic Institute 100 Institute Road, Worcester, MA 01609, USA {emmanuel—claypool}@cs.wpi.edu ABSTRACT ful personal computers has increased their acquisition rates The growth in the popularity of interactive network games and created a growing user base for computer games. The has increased the importance of a better understanding of increase in residential broadband Internet connections with the effects of packet loss and latency on user performance. high capacities and low latencies have encouraged more and While previous work on network games has studied user more game developers to incorporate multi-player features tolerance for high latencies and has studied the effects of into their products. An understanding of how network re- latency on user performance in real-time strategy games, lated issues, such as latency and packet loss, affect the us- to the best of our knowledge, there has been no systematic ability of games can be of great use to the companies that study of the effects of loss and latency on user performance. make these games, network software and equipment man- In this paper we study user performance for Unreal Tourna- ufacturers, Internet Service Providers (ISPs), and the re- ment 2003 (UT2003), a popular first person shooter game, search community at large. In particular, once established under varying amounts of packet loss and latency. First, we latency requirements and any associated trade-offs are known, deduced typical real world values of packet loss and latency ISPs can establish tariffs based on customers’ indicated max- experienced on the Internet by monitoring numerous oper- imum delays, requested Quality of Service (QoS), and the ational UT2003 game servers. We then used these deduced ISP’s ability to meet these demands. Moreover, experimen- values of loss and latency in a controlled networked environ- tal study of network games can provide the data required for ment that emulated various conditions of loss and latency, accurate simulations, a typical tool for evaluating network allowing us to monitor UT2003 at the network, application research, as well as insight for network architectures and and user levels. We designed maps that isolated the funda- designs that more effectively accommodate network game mental first person shooter interaction components of move- traffic turbulence. ment and shooting, and conducted numerous user studies While there has been research qualitatively characterizing under controlled network conditions. We find that typical the effects of latency for car racing [11], custom games [12], ranges of packet loss have no impact on user performance or and real-time strategy games [13] as well as a general aware- on the quality of game play. The levels of latency typical for ness of latency issues [3, 4, 8, 10], work on the effects of most UT2003 Internet servers, while sometimes unpleasant, latency in popular First Person Shooter (FPS) games [1, do not significantly affect the outcome of the game. Since 6] has not quantified its impact on player performance. In most first person shooter games typically consist of generic concentrating on the effects of latency on FPS games, the player actions similar to those that we tested, we believe possibility that packet loss may be the bottleneck in perfor- that these results have broader implications. mance for some network conditions may be overlooked. To the best of our knowledge, there have been no systematic Categories and Subject Descriptors: C.2 Computer Com- munication Networks: Distributed Systems studies of packet loss on user performance in FPS games. The study of loss on network games is increasingly impor- General Terms: Performance, Experimentation tant as wireless channels, more prone to packet loss than Keywords: Network Games, Loss, Latency traditional wire-line environment, become widely adopted. In general, the most popular FPS games have descended from two game lineages, using either a Quake or Unreal- 1. INTRODUCTION based game engine [5]. As previous research has concen- In recent years, the declining costs of increasingly power- trated on FPS games derived from Quake, we used Epic Game’s award winning1 Unreal Tournament 20032 (UT2003) in our experiments. UT2003 is very popular, with approx- imately 1700 servers and 4400 players online at any given Permission to make digital or hard copies of all or part of this work for time.3 personal or classroom use is granted without fee provided that copies are First, we deduced typical real world values of packet loss not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to 1Winner of “Best of Show” at the Electronic Entertainment republish, to post on servers or to redistribute to lists, requires prior specific Expo, Los Angeles California, May 2002. permission and/or a fee. 2http://www.unrealtournament.com/ut2003/ SIGCOMM’04 Workshops Aug. 30+Sept. 3, 2004, Portland, Oregon, USA. 3 Copyright 2004 ACM 1-58113-942-X/04/0008 ...$5.00. Gamespy, October 2003 and latency experienced on the Internet by monitoring op- features a futuristic football style match with teams pass- erational UT 2003 game servers. We then used these values ing and running to either cross a goal for 7 points, or shoot as guidelines for induced loss and latency values in a con- the ball into the goal for 3 points. Since the most popular trolled emulated environment we designed, allowing us to modes of play are Deathmatch and Capture the Flag, we monitor UT2003 at the network, application and user lev- used Deathmatch and Capture the Flag maps for all our els. We divided user interaction in UT2003 into fundamen- tests. However, despite the slight differences in game play tal FPS interaction components in order to isolate particular modes we believe our results would pertain to other modes facets of game-play. These interaction components include as well. movement, precision shooting, general shooting, and mov- Like many FPS games, UT2003 includes a large assort- ing and shooting simultaneously. We designed experiments ment of weapons. Examples include the Mini-gun which with game maps that allowed us to isolate each game com- is capable of firing high volumes of bullets in a very short ponent. Using our testbed, we ran numerous user studies time, the Flak Cannon, which can scatter shards of metal in during which we systematically changed the loss and latency the general vicinity of opponents, and the Rocket Launcher, and measured the impact on performance. which can load and launch up to three rockets at a time. We find that the levels of packet loss and latency typi- While all these weapons destroy opponents, we hypothesize cally encountered on the Internet, while sometimes unpleas- they differ in their timing requirements based on the preci- ant, do not drastically impact user performance in UT2003. sion required in aiming. Table 1 shows a table of weapons Loss, in particular, goes unnoticed and does not measurably broken into four categories based on our subjective view of affect user interaction. Latencies as low as 100 ms, on the the amount of precision that is required to use them. We other hand, can significantly degrade performance for shoot- hypothesize that higher precision weapons tend to be more ing with precision weapons both in terms of accuracy and difficult to use effectively when network packets are either game responsiveness. Although the effects of latency on user lost or delayed, while weapons that require less precision are performance in full UT2003 games with all components is less affected by lost or delayed packets. less noticeable, there is still a clear user performance degra- dation trend as latency increases. Moreover, UT2003 feels Precision sluggish when latencies are 150 ms and higher. Required Example Weapons The rest of this paper is organized as follows: Section 2 High Shock Rifle, Link Gun, Lightning Gun presents background information on UT2003; Section 3 de- Medium Assault Rifle, Mini-gun, Bio Rifle scribes our approach to measure the effects of latency and Low Flak Cannon, Rocket Launcher, loss on user performance in UT2003; Section 4 analyzes Redeemer, Ion Painter the application, network and user results from our experi- Other Shield Gun, Translocator, Ball Launcher ments; Section 5 summarizes our conclusions; and Section 6 presents possible future work. Table 1: UT2003 Weapon Precision 2. BACKGROUND The “high precision” weapons require timely response in First Person Shooters (FPS) are games in which a user order to accurately determine an opponent’s location while interacts with the game world through the eyes of a vir- aiming and may be the most impacted by lost or delayed tual character (the “first person”), collects weapons and packets. attempts to destroy other players (the “shooter”). Unreal The “medium precision” weapons are less accurate and Tournament 2003 (UT2003) is an online FPS in which up so do not require as accurate an opponents’ location while to 32 players can compete simultaneously on a single server aiming and may be less impacted by lost or delayed packets. over the Internet. Players navigate pre-defined obstacle cour- The “low precision” weapons require merely aiming in the ses or maps while playing the game. There are over 35 in- general direction of the target in order to hit without needing door and outdoor maps that come with UT2003, while many precise location information, and therefore may be the least more user-created maps can be acquired from Web sites or impacted by lost or delayed packets. simply by joining a server running a custom map. The “other” weapons are not commonly used, or serve There five multi-player modes UT2003 users can compete special purposes within particular types of games and so we in: Deathmatch, Team Deathmatch, Capture the Flag, Dou- do not consider them further.

Load more