EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

VISUALIZATION TECHNIQUES IN VIDEO GAMES

Veronica Zammitto School of Interactive Arts and Technology Simon Fraser University 250 -13450 102 Avenue Surrey, British Columbia, V3T 0A3 Canada [email protected] / [email protected] http://www.sfu.ca/~vzammitt

Abstract - Video games rely extensively on visual information displayed to the players. Current digital games show enormous amounts of data, and generally in complex 3D virtual worlds. During a game, there is new data added constantly, but it’s also important to have access to information already collected. Menus, minimaps, aiming cursor, character’s health and capabilities are some of the data categories displayed in games. This paper aims to identify how video games present useful visual information to the player, if principles of visualization are applied, and highlight techniques that would be addressed to games for improving their data visualization.

VIDEOGAMES AND VISUALIZATION Videogames relies extensively on the information displayed on the screen. It is given to the user in many ways and is necessary for continuing and advancing through the game. If the information is not correctly visualized and the user misses it, it might jeopardize her performance; this would transform what is meant to be a pleasant activity into a frustrating one. Playing a game is an immersive experience, the interface should not be a reminder that this is all about a game, specialist game interface designers agree that a good interface is the one that is not noticed [3, 6]. It is considered that aesthetic is an important factor, and it should be consistent with the game world. It is a common practice that throughout the game the core interface does not change; this might be one of the reasons why in advertisement screenshots the interface is not shown, otherwise screenshots would be alike and not transmit a dynamic impression of the game. The vast majority of the literature that deals with visualization in games belongs to the field of Game Interface Design, a branch of game development. The other areas that contribute to game visualization are Information Visualization and the younger game studies discipline [4]. In the literature, there is no general agreement regarding the terminology for referring to the different elements of what is being displayed in the games. It might change according to the author’s background and the genre of the game [6]. Saunders and Novak (2007) mention that: “In the context of game development, the term ‘interface’ refers to anything that helps the player interact with the game. It includes hardware such as console controllers, computer keyboards, and television monitors. Game components such as health bars, menus, and targeting reticles are also parts of the interface. Even the game characters can be considered interface elements!” This wide definition of interface is the top of the iceberg of a need for a better terminology and classification of the information visualized in digital games. To contribute to this task, in this paper a successful title of three game genre are analyzed

267 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

in order to identify what the critical information is, how it is displayed, what visualization techniques are employed and what could be improved.

Heads-Up Display (HUD) This is the term employed for referring to a method for game interface. Its main characteristic is that it uses transparences to avoid obstructing what is happening. It takes its name from HUDs used in real life, such us in aircraft (figure 1 A). This term is used indiscriminately for referring to the permanent type of information given to the player, for example: health status, available weapons, armor, resources, among others. It varies radically depending on the game genre. For getting a better immersion, HUDs are designed to match the game world context, like in Stars Wars: Republic Commando, the HUD is presented as the avatar helmet (figure 1 B). Nowadays, there is a trend in the game industry to have minimalist HUDs, with as little information as possible [6, 12].

Figure 1 A. On the left, view using a HUD in a Boeing 727. Figure 1 B. On the right, HUD helmet-like in Star Wars.

Videogames have changed substantially since they appeared, and have evolved into different genres. Each genre provides certain goals that will require prototypical tasks to achieve them. For this paper, I will follow the game genre taxonomy presented by Ernest Adams and Andrew Rollings [5]. They divided all games into nine genres: ‘Action’, ‘Strategy’, ‘Role-Playing’, ‘Sports’, ‘Vehicles-Simulations’, ‘Construction- and-Management-Simulations’, ‘Adventure’, ‘Online-Games’ and ‘Artificial-Life- Puzzle-games-and-others’. Going over all different genres would exceed the scope of this paper. I selected three popular genres and a main game example for each: First- Person Shooter subgenre from Action Games (example: Half Life 2 by Valve), Real- Time Strategy subgenre from Strategy Games (example: Command & Conquer 3 by Electronic Arts), and Massively-Multiplayer Online subgenre from Online Games (example: World of WarCraft by Blizzard).

GAME GENRES In this section, I will present for each selected genre: what tasks the gamer has to do for achieving the given goals, what information is given to the player, and what visualization techniques are employed.

First- Person Shooter (FPS): It belongs to the Action Genre. Playing an FPS game requires a reaction-based skill, this means a fast reaction time for hand-eye coordination. Violence is a game mechanism that we can find in most shooter games. The player’s character, her avatar, has to move around a 3D world to get to a certain location, on her way enemies will attack her, but she can also attack with a range of weapons. In Half Life 2 (HL2) for the

268 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

different game elements (such as, health, ammunition, armor) the quantitative display of data – interval scale type - is the dominant one. This means that numbers will be primary shown to provide information [2]. Examples will be presented for each of these different game elements. Another characteristic of data display in Half Life 2 is that information is visualized only if the element, besides health, is being used or changed. For instance, if the avatar is not using a weapon there will not be information displayed about that element. This dynamic update of the HUD visualization is considered an active interface [6]. The point of view (POV) is the avatar’s one: an egocentric perspective, its characteristic is that we see through the eyes of the avatar. It could be argued that this POV is the most immersive one because character and player share the same view [6]. Out of the different navigation metaphors, ‘walking’ is the one that best suits this POV since there is gravity and there are 6 degrees of freedom, the player can move the avatar up-down, left-right, tilt down, tilt sides, and turn sides. However, from this POV it is difficult to identify from where one is being attacked; it is necessary to provide such information to the player in an effective way. Red vertical bars appear on the screen side from where the player is being attacked. Health is critical information for the player, it determines how much room for mistakes is left. If the player runs out of health, the game is over, and she has to start again. Knowing the health status will help her to decide a strategy, for instance to keep moving but in a stealthy way, or looking for some medical kit to increase health. These decisions radically change the game experience. Changes in health status have to be shown all the time. In this game full health is represented by 100 (see figure 2 A). When health decreases and is dangerously low, color-code changes from its regular color into red. This color-coded technique corresponds to established conventions, meaning red danger for western cultures [11]. See figure 2 B.

Figure 2 A (Left). Health, and armor (suit) are displayed on the bottom left of the screen Figure 2 B (Right). Player’s health is very low. She might die if hit.

In HF2 the player uses weapons: .357 Magnum, rifle, shotgun, rocket launcher, and a “gravity gun” among others. Ammunitions and reloading them are required to use these weapons. When the player is about to run out of ammo, a silhouette of the bullet is shown in red. This is critical information because when reloading the player is not able to fire. When the player completely runs out of ammunition, the number zero in red is displayed. Two techniques are identified here: silhouette, and color coding. Silhouette is a very powerful tool for perceiving the structure of objects. In HL2 bullet silhouettes are shown from a canonical view, this allows an easier recognition [11]. See figure 3 A. Color-coding is used for turning the regular color number (yellow) into red. This scheme also responds to conventions as above mentioned. See figure 3 B.

269 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

Figure 3 A. Left, HL2. Bullet silhouette for indicating that reloading is necessary. Figure 3 B. Right, Player has run out of ammunition.

The player has to move in a 3D world (“City 17”) and in this game there is no mini- map provided. Hence, constrained paths and landmarks are used for easing navigation. For example, the Combine Citadel is easily spotted, that tower is so high that reaches the clouds; this giant structure helps users get their bearings (see figure 4). Since there is no map to read, wayfinding is the option given to players to construct a representation of this 3D environment [8, 11]. There are three steps: first, ‘declarative knowledge’ is to provide a landmark that will be recognized: Combine Citadel. Then, a ‘procedural knowledge’ will be developed by the player who will construct routes and learn how to get to different locations using landmarks as key information. Finally, a ‘cognitive spatial map’ will be built.

Figure 4. Combine citadel. Recounting the tasks to be performed by the player, there are: to move a single avatar; to get weapons/armor; to keep health; to get ammunition; to kill enemies. Summary for HL 2, it has a simple interface following the minimalism trend of the game industry, using just one main color for the HUD, one color-coded consistent for ammunition, avatar and NPC health. The quantitative display of data is the type most used for elements. Silhouette technique is applied for fast recognition, in this fast-paced game. Wayfinding is the technique developed for navigating in this 3D virtual world.

Real-Time Strategy games (RTS) Strategy games have complex sets of rules. Managing is the game mechanism in RTS games. Time within the game passes continuously, and players can perform tasks without waiting a turn. This type of game involves managing resources, commanding several units simultaneously, having as main goals conquering, or exploring worlds. Like in most RTSs, in Command and Conquer 3 (C&C3) the battlefield and map are not completely revealed at the beginning of the game. There are areas shown black until they are at view range of the player’s units. This “blackness” is called “fog of war” (See figure 5). There is another “unveiling” mechanism in the game that is called the “tech-tree”, its characteristic is that there is a pre-established sequence for developing science (buildings, and units) in game; this means that certain structures and units have to be developed first in order to get others, and there might be branches to choose one kind of technology instead of other.

270 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

Figure 5. “Fog of War”. It is not possible to see unless it is at units’ sight.

The classical perspective used in RTS games is isometric, also known as 2.5D, because it is a 2D representation of objects but with some resemblance to 3D. The degrees of freedom are four: left, right, top, bottom. A fifth liberty is not really utilized, although there are flying and ground units, vertical direction is not allowed. The camera is fixed, but it is possible to zoom in and out. The navigation metaphor in this case is world-in-hand with 4 degrees of freedom. Action button is the industry term for the button that causes an event, for example producing a new unit. This kind of button is depicted by an icon that represents its function, and when the cursor is over the button it will display a description of the event, such as specifying how much of a resource the new unit will cost, and what specifications this unit has. These buttons are not always able to be used, sometimes a tech-tree has to be developed in order unlock action buttons, or more resources are needed. This “momentary” disabled state of the button is related to pre-attentive processing, a theoretical mechanism that allows users to identify symbols in a glimpse. By using colored and grayed-out buttons it is faster to “see” the available options offered. Colored buttons “pop out”, while grayed-out ones remain as background. (see figure 6)

Figure 6. Pre-attentive processing. Colored buttons stand-out, grayed-out do not.

When the action button is able and the player clicks on it, the event will not happen until a certain time has passed. This is the case when producing several units, it would break the time dimension if units appear automatically; so, it is necessary to queue their production. There is a visualization for indicating this passing time. Action buttons are whited-out, this whiting layer is swept clockwise. (see figure 6) Mini-maps are representations of the battlefield. It is necessary to overview different locations at the same time, for example the player’s base, the opponent’s one, and where units are. In C&C3, mini-maps have color-coding, shading, and profile image techniques for displaying the terrain more effectively (see figure 7)

271 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

Figure 7. Command & Conquer 3 mini-maps Figure 7 Right. “Ping” on a mini-map. A blinking yellow circle shows where a new event has happened.

RTSs, like FPSs, require quick reactions. Collecting information at a glance is highly valuable. Another way of working with the previously described pre-attentive processing is by designing a distinctive symbol set for the mini-map where objects are easily recognized. Since there is no labeling, this set has to be extremely simple. In C&C3 the symbol set is oversimplified: player’s units and buildings are shown as blue dots; opponent’s units and buildings are shown as red dots; resources appeared as green dots. There is an issue with the color-coding: color blindness, almost 15% of Caucasian males have it [1]. This is not a minor fact, since there is still a majority of male players. The C&C3 set is not color blindness proof, red and green can be undistinguishable by color-blind players. Opponent’s units that are not moving might be mistaken with resources. For solving this matter, a color-blindness friendly palette should be offered, like the software patch provided for another RTS called Alpha Centauri. There is a visual grammar of map elements in this mini-map: a closed contour of trapezoid shape for delimiting the region of the mini-map displayed in the main part of the screen. See figure 7. The word “ping” is used in the game industry for referring to the visual clue that points in the mini-map where an event has just happened. The RTS player constantly performs different tasks, her attention may be on constructing a building, developing units and selecting troops, meanwhile an event that demands attention happens on another part of the map. It is hard to attract attention to the visual data outside the fovea area, changing the color of an icon will not work because peripheral vision is color- blind, so the change will not be noticed by the player. That is why it is necessary to use techniques such as blinking or motion, in C&C3 a growing, blinking circle will appear on the mini-map where an event just happened (see figure 7 Right). Technique for levels of detail of the information visualized has three levels [9]. In C&C3 these are: one includes all the units of interest, another level shows information about a subset of unit, and a third level gives details of single units. This distinction of level is pretty useful in RTSs, because it allows managing in a meta-level, but at the same time to keep track of the different units. For example, the player wants to know what kind of unit she has selected: Missile Squad unit (first level); how many units are available: eight units (second level); what health level these units have: low and medium health status (third level). The possibility of having this information at once allows fast decision making. (See figure 8)

Figure 8. Levels of detail Bottom right, first level. Upper right, second level. Upper left, third level.

272 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

In the main menu of C&C3, the Edge Enhancement technique is applied. Like artists such as Seurat have used it [11], its purposes are to display objects clearly, and enhance a certain part, in this case the menu where the options are, by an edge produced through shading out the two sides. See figure 9.

Figure 9. Edge Enhancement technique is applied in the main menu of C&C 3

Recounting the tasks to be performed by the player, there are: to establish buildings; to explore the terrain; to collect/manage resources; to create units; to conquer new locations; to attack/defeat opponents. Summary for RTS, several information visualization techniques are implemented: Pre-attentively processing, use of color for popping-out information. In maps: color- coding, shading, and contour image techniques. Use of edge enhancement through an artistic implementation for stressing a part of the screen. Transparency technique to overlap data. A very simple symbol set for map. Use of levels of detail.

Massively Multiplayer Online (MMO) World of WarCraft (WoW) is ambient in a Tolkienesque world. This kind of world is called a “persistent world” because the environment is always available and events happen continuously. Online games allow a great opportunity for social interaction. However, the main method for communication among players is through in-game chat, this is by typing text. Chatting in WoW can be organized in 5 types, they differ in who will be the recipient of the communication: 1) Location based: Includes “Say” (the recipients are other players that are nearby), “Yell” (all players in the map are addressed) and “General” (to those where are in the same map area); 2) Groups: Can be Guild, Party, Raid; to communicate with all the players that belong to that group; 3) Whisper: message to a specific player, regardless where she is; 4) Interests: “Trade” and “Looking for group”; 5) System information: text that describes what happens. These different kinds of chatting are color-coded but it changes (see figure 10)

Figure 10. Different chatting channels

WoW has role-playing mechanisms, hence a good part of the game also consists in evolving a character. Players develop their avatars by doing quests by their own or in group. Quests have specific goals to achieve, by taking missions to accomplish in return to objects and experience. When enough experience has been earned, the avatar upgrades one level. The visualization of this information is through the progress bar. Experience is counted as interval type data, it is represented in an experience bar located

273 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

on the bottom of the screen. It is always visible, so player can calculate how much experience is needed for leveling. (see figure 11)

Figure 11. Experience Progress Bar.

In WoW, health bar status does not use color-coding as others games do. Instead bar length clue is applied. The color of the bar is always the same, green; but, when avatar is hit, the length of the bar decreases. If one method of conveying health information qualitatively is used, it is preferably applying length instead of only color; because changes of shades of green to red might not be perceived by color-blind people. See figure 12.

Figure 12. Health bar

MMO game worlds are crowded ones. There are loads of objects, and gamers should be able to move without inconveniences. The point of view (POV) is an exocentric perspective; player can see his avatar from all sides, the most common camera angle being from behind and above the avatar. Moving around this 3D virtual world is connected with the “walking” navigation metaphor; the computer keyboard and mouse are the input devices. For navigating smoothly in 3D virtual worlds, it is necessary to provide data for an accurate space perception. Two techniques are: occlusion, and cast shadows [11]. Occlusion happens when an object overlaps another, the first one is perceived closer than the last one. However, it is not possible to specify the distance between the two objects. Cast shadows are helpful for providing information about the height of objects. See figure 13.

Figure 13. Space perception. Occlusion and shadows

WoW’s interface is heavily icon-driven, and can be fairly customized by dragging windows to a new location, sizes of the UI and there are several popular add-ons created by the WoW users using Lua scripting language . This kind of concession to manipulate the HUD is related to the concept of “active interface” where some level of control is given to the players. Windows are closed rectangular contours that powerfully divide the visual field; it is ideal for grouping consistent information, and retrieving it fast. Pre-attentive processing is also addressed in WoW like in C&C3, action buttons that are momentarily unavailable are grayed-out, and perceived as background; meanwhile accessible ones remain colored for standing out. See figure 14.

274 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

Figure 14. Customized Action Bar and Add-ons

In WoW the mini-map is by default north-up oriented. There are techniques develop by Saito and Takahashi [11] used in this mini-map: shaded image, and profile image. Applying such techniques helps to show the terrain effectively. In this mini-map it is allowed to zoom in, and out. Arrowed-dots, a visual grammar code, are used in the mini-map to point to cities and party members. Our location is always in the middle of the mini-map, represented by an arrow heading to where we are facing; this last feature is one of the ways to enhance visualization with maps suggested by Ware. See figure 15.

Figure 15. Mini-maps in WoW

Recounting the tasks to be performed by the player, there are: to develop a character; to chat with other players; to manage inventory; to attack enemies. Summary of techniques for MMO: changeable color-coding in chatting channels; techniques applied for space perception: occlusion and cast shadows; Bar length; Pre- attentive processing.

CONCLUSION This literature review shows that information visualization in digital games is still not very developed. The main body work on this topic comes from the game developer industry but the lack of an established terminology for the field makes it difficult to work towards its foundations, looking into the established information visualization field would help in this regard. It seems that information analysis by genre is a consistent method, because each game genre emphasizes certain tasks that require particular information. Three subgenres were selected: First-Person Shooter, Real-Time Strategy, and Massively-Multiplayer Online. FPSs have violence as a main game mechanism. It is about kill or be killed. HL2 has a minimalism interface where information will be displayed if it is being used, the only exemption is health. The information shown is quantitative rather than qualitative. Health and ammunition are very important information, armor could be considered as part of health. Color-coding is a technique applied homogenously to the HUD. Silhouettes help to easily recognize the need of ammo reload. Moving in a 3D world is essential, there is no map but wayfinding is successfully applied. The main game mechanism for RTS is managing. It is about conquering and exploring. “Fog of War” avoids revealing all the information about the map at once. C&C3 has an isometric view of the world. Simultaneous and multiple tasks have to be done, for this reason all techniques that support pre-attentive processing are useful in

275 EVA 2008 London Conference ~ 22-24 July Veronica Zammitto ______

RTSs, like grayed-out temporally unavailable action buttons. Color-coding in the C&C3 mini-map is oversimplified, and it should offer color-blindness friendly palettes. Information provided in levels of details allows fast decision making. MMOs are socially oriented. It is about communicating with other players, and developing an avatar, these tasks generally take place around quests. When just one method for visualization health information is used, it is better to use bar length rather that just color due to color shadings not distinguishable by color-blind people and outside the fovea area. Pre-attentive processing is also addressed in WoW by grayed- out unavailable buttons. Due to the differences in tasks and information needed, certain visualization techniques work better in different game genres. For example, health is the most important game elements in FPS, so quantitative (very precise data) and color-coding techniques are used, whereas in MMO a one color length bar is enough. Further research should be done for contributing to this area by analyzing in depth one genre through different games, and comparing what visualization techniques are used for the same kind of tasks and information.

ACKNOWLEDGMENTS Thanks are due to: Electronic Arts for Command & Conquer 3 screenshots, Blizzard for World of WarCraft screenshots, Valve Corporation for Half Life 2 screenshots.

References [1] DUVALL, H: It's all in your mind. Visual psychology and perception in game design.Gamasutra http://www.gamasutra.com/features/20010309/duvall_01.htm March 9, 2001. [2] FEW, S: Data visualization: Quantitative vs categorical data: A difference worth knowing. DM Review April, 2005. [3] FOX, B: Game Interface Design. Boston, Thomson Course Technology, 2004. [4] NITSCHE, M: Games, montage, and the first-person point of view. DiGRA 2005 proceedings. Available: http://www.digra.org/dl/db/06276.11074.pdf [5] ROLLINGS, A and Adams, E: Andrew Rollings and Ernest Adams on game design / Andrew Rollings, Ernest Adams. New Riders Games. 2003. [6] SAUNDERS, K and Novak, J: Game development essentials. Game interface design / Kevin Saunders, Jeannie Novak. NY, Thomson Delmar Learning, 2007. [7] SEIF EL-NASR, M and Yan, S : Visual attention in 3D video games" in ACE 06 proceedings, 2006. [8] STTIT, C and Fiorito, J: Lessons in color theory for spyro the dragon. Gamasutra, http://www.gamasutra.com/features/20000502/spyro_01.htm 2000. [9] TAN, K and DAUT, D: A review on level of detail" in International Conference on Computer Graphics, Imaging and Visualization, 2004. CGIV 2004. Proceedings, pp. 70 - 75. 2004. [10] WAGNER, I, Kompast M and Lainer, R: Visualization Strategies for the Design of Interactive Navigable 3-D Worlds" Interactions, vol. 9, pp. 25-34, 2002. [11] WARE, C: Information Visualization. Perception for Design. 2nd ed. San Francisco, CA: Morgan Kaufman, 2004. [12] WILSON, G: Off with their HUDs!: rethinking the heads-up display in console game design. Gamasutra http://www.gamasutra.com/features/20060203/wilson_01.shtml 2006.

276