IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 1 Toward Visualization for Games: Theory, Design Space, and Patterns Brian Bowman, Niklas Elmqvist, Member, IEEE, and T.J. Jankun-Kelly, Member, IEEE

Abstract—Electronic games are starting to incorporate in-game telemetry that collects data about player, team, and community performance on a massive scale, and as data begins to accumulate, so does the demand for effectively analyzing this data. In this paper, we use examples from both old and new games of different genres to explore the theory and design space of visualization for games. Drawing on these examples, we define a design space for this novel research topic and use it to formulate design patterns for how to best apply visualization technology to games. We then discuss the implications that this new framework will potentially have on the design and development of game and visualization technology in the future.

Index Terms—Computer games, video games, interactive entertainment, entertainment, visualization, game analytics. !

1 INTRODUCTION and use it to mark special achievements and unlock new content. For example, the multiplayer version IDEO and computer games have a long history of the recent Call of Duty: Black Ops [57] first-person of using visual representations of data to convey V shooter collects vast amounts of statistics about player information to the player; for example, colored bars performance, and not only uses this to award special are routinely used to present vitals—health, energy, titles, emblems, and equipment to players as they play and magical power—of the player character and are the game, but also to aggregate all statistics (such as accordingly often called “healthbars” (Figure 1(a)), the number of shots fired worldwide, the number of and many games use overhead maps to show the cumulative feet fallen by all players, or the number locations of entities in the game world. Some games of vehicles exploded) and feed it back to the entire have already gone beyond such simple visual repre- player community as a social motivator. sentations and are employing statistical data graphics We believe that the extraction, management, and to present more data to the player. For example, the visualization of data in games is an exciting new first version of the classic SimCity [56] game from branch of research with huge potential impact—video 1989 used chloropleth maps to convey metrics such games sales in the U.S. alone reached $10.5 billion in as crime rate in a particular city (Figure 1(b)). 2009 [1], and the games industry estimates the number In general, interactive games can be seen as real- of game consoles in North America to surpass 60 time simulations—just like SimCity—and there are million units (not counting personal computers, which many situations where feeding back the state of the clearly also can be used for gaming) [2]. Furthermore, simulation to the player or the game developer can gamers tend to be highly motivated and often have a be beneficial, regardless of game genre. For example, a high degree of spatial and visual literacy [3]. For these player of a first-person shooter can use session replays reasons, we think that we are on the cusp of a major to improve their movement around a map, or a mili- adoption of casual visualization [4] technology in mil- tary strategy game player can use statistics about map lions of living rooms worldwide. The only question is influence and army strength over time to optimize how to best harness visualization technology for the their use of resources. The ongoing revolution in challenges inherent to this exciting new domain. gaming on social media sites (such as on Facebook) In this paper, we build on the existing academic also showcases the power of sharing a player’s per- research on this topic— [5], [6], [7]—to take the initial formance and experience as a social indicator. In ad- steps toward a theoretical framework and methodol- dition, data collection and feedback can also be used ogy for the use of visualization in games. We begin as a motivational factor; a recent trend in video and by examining existing evidence of visualization and computer game design—previously mostly restricted data collection in games, starting from SimCity in to sports games—is to increasingly instrument games 1989 and moving to the Black Ops and Halo: Reach to collect large amounts of data during gameplay of present day. We then derive a design space of data and visualization types in the context of games. • B. Bowman and N. Elmqvist are with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN. Based on this design space and on the evidence E-mail: [email protected], [email protected] collected, we propose a set of design patterns for how • T.J. Jankun-Kelly is with Mississippi State University, Starkville, MS. to best approach games as a visualization researcher E-mail: [email protected] or practitioner. Finally, we derive implications that IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 2

(a) Health (green) and mana bars (blue) for a player (b) Chloropleth map of (c) Hit heatmap in (d) Death heatmap in Halo: character in World of Warcraft. crime rate in SimCity. Black Ops. Reach (level Anchor 9).

Fig. 1. Examples of visualizations in genres ranging from role-playing games (World of Warcraft), construction and management simulations (SimCity), and first-person shooters (Call of Duty: Black Ops and Halo: Reach). this work will have on both game and visualiza- Furthermore, we should emphasize that although tion research. These implications show how social virtually all computer games today incorporate ad- motivators often drive the introduction of visualiza- vanced and often near-photorealistic graphics, we re- tions in games: player communities for competitive gard such aspects to fall within the scope of computer multiplayer games use statistics and visualizations graphics (and applied domains such as Virtual Real- to communicate with each other, share game experi- ity) and not visualization. In fact, games are one of the ences, and improve their performance. Furthermore, driving forces for consumer-level computer graphics, we discuss the benefits of studying visualization for and, by extension, visualization [8], [9]. However, games, both in terms of benefits to games—such as while the 3D rendering of a fictional world within a better gameplay, easier balancing and debugging, and computer game strictly speaking is a visual represen- more enjoyable spectating—as well as in terms of tation of data (the 3D world), we delimit this work benefits to visualization research—such as massive to focus on the use of visualization to represent data adoption, a dedicated user base, and a controlled and with no intrinsic spatial form (such as health, points easily instrumented data source (the game world). over time, political influence, etc). In other words, our treatment here is basically performed from an information visualization (InfoVis) viewpoint [10]. 2 BACKGROUND A computer or video game is an electronic game where 2.1 HCI and Games the player interacts with a visual display using some form of input device. Stated in this very generic Games and human-computer interaction (HCI) have form, it is clear that video games have many com- a long and colorful history [11]. Already in 1983, Ben monalities with general interactive systems, such as Shneiderman used games as illustrative examples in word processors, web browsers, and spreadsheets. his landmark paper on direct manipulation [12], laud- The difference lies primarily in the goal-driven nature ing them for smooth, effortless, and direct interaction. of the game, and sometimes in specialized hardware, Galloway’s classification of action in gaming partic- including dedicated game controllers—such as joy- ularly identify interface elements as key nondiegetic sticks, gamepads, steering wheels—or even dedicated (i.e. outside of the game world) aspects of games [13]. video game consoles—such as the current seventh- As a result, considerable research has been directed generation consoles like the Sony Playstation 3, the toward applying HCI methods to games [14], [15], Microsoft Xbox 360, or the Nintendo Wii. [16], [17], but also toward applying techniques from Although we use the terms computer game and video games into HCI [18], [19], [20], [21]. game interchangeably, these terms traditionally refer to A key difference between general HCI and games whether the game is targeted at a personal computer is that for the former, the goal is to help users to (PC or Mac) or a video game console. This paper is become experts as quickly and effortlessly as possi- about both of these types of platforms and makes little ble, whereas games are generally designed using the distinction between them—discrepancies arise only motto “easy to learn but difficult to master” [20]. This for visualization components outside a game (such is an important point; as with HCI, the usability of as on a website), which generally are targeted for a visualization should never be a hindrance against computers, regardless of the actual game platform. players reaching their goals (intrinsic difficulties in the gameplay serve this purpose already). We delimit our treatment in this paper to games, and not real-time interactive simulations for training purposes outside of the entertainment domain. While 2.2 Applied Visualization these share many of the same characteristics, we lack Visualization for games is still in its infancy, but there the space to survey this domain as well. is a number of existing visualization technologies that IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 3 may be relevant to this novel domain. The target process that can complicate source code and cause ex- audience of a visualization for a computer game is tra overhead. To ease this process, some research and one of the first issues to consider. If the visualization commercial frameworks exist for game telemetry and is targeted primarily at the gamer community, these analytics. For Flash-based games, the key frameworks visualizations will likely fall under the topic of casual are Mochibot [32], Nonoba [33], and Playtomic [34] information visualization [4]. Such visualizations will where developers add hooks inside their games to need to target attributes such as being “useful” and send player events to external servers. Kontagent [35] “pleasing” rather than “utilitarian.” If, on the other is targeted at tracking browser-based games running hand, the target audience is the development team, on social media platforms such as Facebook and Mys- then techniques from software visualization [22]— pace. The Microsoft Game Studios TRUE (Tracking such as for log traces, program structure, and run- Real-Time User Experience) [36] system is used to time behavior—may be more appropriate. Bioware’s instrument source code and can then be utilized to Skynet [23] is an example of such a developer-oriented automatically collect rich sequences of user events, telemetry and visualization framework that we will similar to observational ethnographic studies. The discuss in depth later on in this paper. RAD Telemetry Profiling System [37] is targeted at Just as for normal visualization, the data governs programmers, and collects real-time application per- which technique to use. Because many games tend formance for the purpose of optimizing game code. to be spatial in nature, spatiotemporal visualization In the absence of standardized frameworks, differ- techniques are often useful; see Andrienko and An- ent games and developers have built custom solutions drienko [24] for an exhaustive survey. Other data may for game telemetry and analytics. The MMORPG be abstract or purely temporal in nature, and for these Pirates of the Burning Sea [58] uses advanced logging we can apply temporal visualization techniques [25] technology to support the design and development as well as traditional statistical data graphics [26]. process [38]. Similarly, Bioware’s role-playing games Furthermore, multidimensional, graph, and text vi- Dragon Age: Origins [59] and Mass Effect 2 [60] in- sualization techniques—currently very uncommon in corporate a developer-oriented telemetry framework games—may become more prevalent in the future. called SkyNet [23]. SkyNet collects data from several Finally, a closely related, yet equally (if not more) running instances of a game on the network and unexplored, research topic is the use of visualization presents the data in a web portal. Also, the Maxis for sports. Sports share many of the characteristics game Spore [61] provides a gamer-oriented telemetry of games in the types of data, the target audience, framework through a public API [39], [40] that allows and the questions that the audience is likely to ask players to view, analyze, and visualize the data freely (not to mention the fact that sports games form an and without being constrained by the game itself. extremely popular share of the market). Medler [27] draws connections between the collection of statistics in sports and games; however, visualization is equally 2.4 Visualization and Games underutilized in both domains. TennisViewer [28], Although not nearly as intimately intertwined with which visualizes tennis matches, is one of only a select games as HCI, visualization has had its eye on games few examples. More recently, Cox and Stasko [29] use for some time. However, the primary line of re- information visualization to discover hidden mean- search has been on how to leverage and steer the ing in baseball season data using bar displays and rapid advances in computer graphics spurred by the treemaps. Healey parses play-by-play summaries of video games industry for (mainly scientific) visualiza- NFL football games and combines temporal and spa- tion [8], [9]. While this is certainly an important topic tial representations to visualize them graphically [30]. for visualization research, our focus in this paper goes beyond computer graphics to study the adoption of 2.3 Game Telemetry and Analytics actual visualization techniques in video games. Game telemetry (telemetry is defined as the use of Surprisingly little work has been conducted in this various instruments and sensors to remotely collect domain. Halper and Masuch [41] focus on extracting real-time measurements) has lately become the catch- and summarizing highlights from real-time game- phrase for all kinds of techniques that collect real-time play footage to aid spectators in first-person shoot- data on game sessions, and game analytics [27], [31] is ers. Hoobler et al. [42] use local visualizations—such the practice of analyzing this data. Applications for as player glyphs, paths, and tracers—in combina- this analysis are numerous, and include both the de- tion with global spatial visualizations to represent veloper side (usability, game balancing, profiling, etc) team-based competitive gameplay in a multiplayer as well as the gamer side (achievements, maps, log first-person shooter (Wolfenstein: Enemy Territory [62]). analysis, movies, etc). As we shall see in this paper, These ideas sparked additional work on spatializ- one approach to game analytics is to use visualization. ing movement in 3D environments using visualiza- Collecting data inside a game requires the devel- tion [43], [44], [45]. In recent work, Medler et al. [5] opers to instrument the source code appropriately, a present the Data Cracker for the multiplayer version IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 4 of the first-person shooter Dead Space 2 [63] that uses in novel ways. For example, the Steam [65] online web-based interactive graphs to display information games distribution and multiplayer gaming platform about gameplay behavior. incorporates rudimentary statistical graphics showing Unfortunately, the above examples are only point the usage of the platform and various games. For designs for specific games, and it is only recently that the Xbox 360, playing games will earn the player so- visualization research has begun to take a broader called “achievements”, whereas Playstation 3 players view of games as an exciting new domain. Joslin et earn “trophies”, both of which can be shown on the al. [46] propose a gameplay logging and visualization player’s profile screen and used in rankings. A recent manifesto, using World of Warcraft [64] as an example tutorial [49] by the gaming middleware company of how to visualize data from a role-playing game. GameSpy argued in favor of more advanced visual However, their examples are all interactive mock-ups representations for aggregated game statistics using outside of the actual game. Medler [47] studies how streamgraphs [50] as an example. player dossiers are starting to be used as containers for In general, game developers are beginning to ex- many visualization components in games. tend the reach of the gaming world outside the games Zammitto [7] was among the first to make the con- themselves onto websites accessible using an online nection between visualization and the visual informa- account that is connected to the player’s copy of the tion presented in games that go beyond the 2D or 3D game. Examples for specific games (all described in spatial representations of the game world. Her paper detail below) include Bungie.net for Halo: Reach [66], reviews and surveys both the academic literature and FIFA Earth for FIFA 10 [67], and the online stats for the video game markets to find evidence for such Call of Duty: World at War [68] (Figure 2). Because Xbox usage of information visualization. While we build on achievement and PS3 trophy data can be accessed her initial review in this paper, our work focuses more from outside their respective communities, aggregator on the visualization technology aspects of this topic. sites such as GiantBomb.com are starting to combine In a very recent paper, Medler and Magerko [6] and visualize this data online, as well as comparing a study the use of information visualization in a large player’s performance with the whole community. number of existing games on the market today. Start- ing from the concept of casual information visualiza- tion [4], they introduce the notion of playful InfoVis as the use of information visualization to support and promote play. We have adopted many of the examples and classifications in Medler’s and Magerko’s work in this paper, but our work instead focuses on the design space, design patterns, and implications for the future design of visualization technology in games. Finally, another very recent paper, this one by Diakopoulos et al. [48], take the opposite approach Fig. 2. Example player dossier [47] in Call of Duty: compared to ours by incorporating game elements World at War [68] (http://www.callofduty.com/stats/). into visualization (referred to as infographics in the paper) instead of vice versa. Their two designs and Community-created websites are also growing as evaluation show the potential of using playable data game telemetry data becomes increasingly accessible to influence and motivate exploration. outside of games. We have already mentioned the god game Spore [61] where the developers are providing 3 EVIDENCEOF VISUALIZATION IN GAMES an actual API for hobby programmers to access in- The purpose of this section is to review existing game data, but for games where such mechanisms are games that incorporate some form of visualization less (or not at all) developed, motivated players are components. For the sake of simplicity, we have resorting to manual (and tedious) data collection that categorized our review by game genre, informally is then aggregated and visualized on various websites. drawn from online gaming community sites such as IGN (http://www.ign.com/) and GameSpot (http:// 3.2 CMS, 4X, and God Games www.gamespot.com/). In addition to examples, each A popular set of game genres that are particularly subsection also includes a description of that genre. amenable to the use of visualization are construc- tion and management simulations (CMS), Explore, 3.1 Game Middlewares and Lobbies Expand, Exploit and Exterminate (4X) games, and While not strictly games per se, there is a growing god games. Common among all of these is that they trend of game middlewares, gaming communities, put the player in control of a business, empire, or and unified gaming lobbies that are beginning to even an entire world from a macro-scale leadership utilize visual representations to convey abstract data perspective (although micro-scale decisions are often IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 5

Fig. 3. City screen in the 4X game Civilization [69]. Fig. 4. Game interface in the god game Populous [71]. necessary). Because of this high-level perspective, vi- One common feature of most first-person shooters sual representations are easier to integrate into this has long been the use of heads-up displays (HUDs) (see kind of game while maintaining immersion than into Section 5.1) that overlay game status such as player games with more direct and literal gameplay. health, ammunition, and energy levels on top of the Accordingly, some of the first evidence of visualiza- 3D world view. However, as discussed later in this tion in games is found in these genres. SimCity [56], paper, developers are now starting to decrease the size the canonical CMS game that puts the player in charge and prominence of HUDs in favor of more immersive of urban planning and administration of an entire mechanisms, such as showing blood spatter, weapon, city, was released in 1989 and incorporates numerous and ammunition in the game world itself. advanced visual representations such as the chloro- Given the focus on immersive first-person game- pleth map for spatial data shown in Figure 1(b). Other play, it is perhaps somewhat surprising that visualiza- CMS games, in the Sim series and beyond, have since tion has recently become so prominent in FPS games. carried on this tradition by continuing to incorporate However, the explanation may be in the multiplayer visual representations for this type of data. and social aspects of these games; multiplayer sup- Another innovator in this area is the Civilization port is common in FPS games, which often spawns series of 4X games designed by Sid Meier, where an online community of players that are interested the player leads an empire spanning through thou- in furthering their own play, communicating with sands of years and entire worlds. The original Civi- each other, and comparing achievements. Accord- lization [69] came out in 1991, and incorporated novel ingly, there has been much innovation in support- visual representations such as glyph-based bars (Fig- ing these communities with visualization: Hoobler et ure 3) as well as a summary replay of a game session al. [42] (discussed above) visualize data from Enemy (Figure 5(a)). Later games in the series have continued Territory [62], Call of Duty: World at War [68] provides this tradition, with Civilization III [70] in particular a statistics web portal (Figure 2), Halo: Reach [66] introducing several new representations (Figure 5). sports Bungie.net that tracks and visualizes both ag- God games take the concept a step further by gregated and individual statistics (Figure 1(d)), and turning the player into a deity with (semi-) abso- Black Ops [57] maintains an in-game combat record lute control over a world. Populous [71] from 1989 that shows spatial and temporal statistics (Figure 1(c)). was first among such games, and incorporated some rudimentary visual representations, as did Populous 3.4 Real-Time Strategy II [72], its successor (Figure 4). Later offerings, such as Real-time strategy (RTS) games are a relatively re- the aforementioned Spore [61], also often use abstract cent type of strategy game, often military- or science visual representations for conveying game state. fiction-themed, where the player becomes the general of an army and is asked to defeat an enemy. Dune 3.3 First-Person Shooters II [75] was arguably the first RTS, and introduced First-person shooters (FPS) are action games where the notion of a real-time, continuously updating game the players view and interact with the 3D world world at a time when its strategy brethren were all through the eyes of the protagonist. With id Soft- turn-based. Similar to FPS games, modern RTS games ware’s Wolfenstein 3D [73] from 1992 as the genre’s often include an online multiplayer mode (largely spiritual ancestor, FPSes are today one of the most influenced by Warcraft [76] (1994) and StarCraft [77] popular game genres on the market, particularly for (1998), the latter which is played professionally). competitive multiplayer gaming (a subgenre that was Perhaps for this reason, statistics and visualization arguably sparked by Doom [74] in 1993, also by id). figure heavily in the RTS player communities. Heroes IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 6

(a) Summary replay. (b) Diplomacy (undirected graph). (c) Technology tree (directed graph).

Fig. 5. Different visualizations for replay, foreign relations, and technological inventions in Civilization III [70].

analytics and visualization portal. However, none of these visualization systems are intended for players. On the player side, many RPGs also sport community-maintained websites with data collec- tions. This is particularly prominent for the subgenre known as MMORPGs, or massively multiplayer on- line role-playing games, which are large-scale and Fig. 6. The “game DNA” of a competitive game in persistent game worlds for hundreds or thousands StarCraft II [78] from SC2Replayed.com showing the of simultaneous players. Again we see the impact of build order of installations for different players on the community: MMORPGs such as World of Warcraft [64], opposing teams (left vs. right columns). Dark Age of Camelot [80], Warhammer Online: Age of Reckoning [81], and EVE Online [82] have extensive online repositories, statistics databases, and map col- of Newerth [79] has a web interface that collects player lections that have all been painstakingly collected, statistics, but little visualization support. Replays are classified, and chronicled by enthusiasts worldwide. very popular for StarCraft II [78], with some im- portant matches even being narrated by professional 3.6 Sports Games game commentators. The site SC2Replayed.com col- Statistics has long been an intrinsic component in pro- lects many replays and also analyzes them to show fessional sports, and so it follows that sports games by “DNA” screens of players and games (Figure 6). extension often also incorporate numerous statistics. Just as for FPS games, HUDs are used prominently However, while sports visualization is growing (see in most RTS games. However, because of the fact Section 2.2), most games tend to leave the statistics that RTSes are typically not designed with immersion in numerical format and use few in-game visual rep- foremost in mind, RTS games have still not seen the resentations (although in-game tactics and playbooks diminishing HUD phenomenon that FPS games have. arguably are a form of visual representation). An alternative explanation is that HUD components Websites outside of the sports games themselves are such as abstract markers, information readouts, and starting to incorporate visual representations, how- user interface elements fit better with the bird’s eye ever. The New York Times published a recent info- perspective used by most RTS games than they fit the graphic [51] showing how the ups and downs of real first-person perspective of shooter games. NFL teams during the season influenced the teams and matchups people were playing in the football 3.5 Role-Playing Games game Madden NFL 11 [83]. FIFA Earth for the FIFA 10 [67] soccer game is a web-based game portal Computer role-playing games (RPGs), like their that aggregates statistics from multiplayer matches paper-based counterparts, give the player control of to show the macro-scale performance of different one or several characters that grow in ability as the countries using visual representations. game progresses. These games are often complex to develop and require careful design to reach the cor- rect balance of challenge and difficulty, making them 3.7 Driving Games excellent candidates for visualization in support of Driving games put the player behind the wheel of developers. Accordingly, the aforementioned Pirates different types of vehicles, ranging from sports cars, of the Burning Sea [58] has a built-in logging and motorcycles, racing cars, etc. Immersion and realism analytics system [38], and Bioware’s Mass Effect 2 [60] is often an important aspect of such games, so visual- (a science fiction RPG) and Dragon Age: Origins [59] izations are generally rare. However, some visual rep- (a fantasy RPG) were designed using the SkyNet [23] resentations can be integrated even while maintaining IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 7 immersion: ghost cars, or ludophasmas [27], that show (Figure 1(a)), ammo counters, and hit direction in- the performance of other players (or the player herself dicators. Visual representations are often chosen in another session) as an intangible shape on the in lieu of a simple number (like a percentage for road have been used ever since the heydays of the the character’s health) because the game design- old Amiga and Atari computers, and many driving ers feel that the visualization is more immersive games (starting from the original Formula One Grand and easier to read quickly, or even preattentively. Prix [84] from 1992) render the optimal racing line on • Training: Some visualizations are designed to the tarmac to help novices improve their turn-taking. help players—either the current player, or other Social aspects of multiplayer gaming have again players—to improve their gameplay. We classify had an impact on modern driving games. Need this as training, and these visualizations often for Speed: Hot Pursuit [85] has an in-game compo- use a reflective mode, such as replay theaters (in nent called Autolog that collects racing statistics for Starcraft II [78], Halo: Reach [66], or Call of Duty: each player and visually compares these against the Black Ops [57]), but continuous approaches such player’s friends within the game world. as a visual overlay showing the optimal line for taking a curve in a driving game are similar. 4 FRAMEWORK:VISUALIZATION IN GAMES • Progression: Many times players are faced with different choices for how to advance in the game, Drawing from the above examples, we now define the and designers are increasingly using visualization design space of visualization for games. Unlike tra- in these situations to show these options. For ditional visualization taxonomies [52], we focus only example, many games use visual skill, technology on components that are specific to the intersection or even dialogue trees to show the player how between games and visualization technology. For this to progress in the game, and the supply and reason, our design space does not incorporate data demand bars in SimCity [56] tell the user what type, utility, and visual mappings; such categories zones to build in order to grow their city. are specific to visualization techniques and not their • Communication: Another purpose for a visual- application to games. ization is for communicating the game or the Our design framework consists of the following five player’s state to others. This practice has become categories that can be used to classify any specific increasingly popular with the rise of social games visualization technique found in a computer game on sites such as Facebook and Myspace, where (described in detail in subsequent sections): status messages posted to one’s profile may in- • Primary Purpose: Intended use; corporate visual representations that serve as so- • Target Audience: Intended audience; cial indicators of progress. Another recent phe- • Temporal Usage: Timing of use; nomenon is games as spectating entertainment, • Visual Complexity: Level of visual sophistication where replay theaters and movies are perused by of the visualization in the game; and spectators watching professional players compete • Immersion/Integration: Whether the visualiza- against each other (for example, this practice is tion is in spirit with the game and whether it is widespread in StarCraft II [78], where commenta- inside the game or not. tors typically talk over game footage). Note that a single game could contain several • Debugging and balancing: Our purposes so far have different visualization techniques, each having to be been mostly focused on the player side of the classified separately using these dimensions. Further- fence, but as evidenced by our literature survey, more, these visualizations can also be classified using developers are also avid consumers of visualiza- traditional visualization taxonomies. tions for game analytics. Such visualizations are commonly designed for allowing the developer to debug, balance, and compare player perfor- 4.1 Primary Purpose mance between and within different sessions; Visualization techniques are added to games for a they are therefore often not integrated within the reason. The Primary Purpose dimension captures the game, they aggregate several game sessions, and main purpose that the visualization was created (al- they may also rely on a higher visual literacy on though some visualizations can arguably have mul- behalf of the users. Examples include Flying Lab tiple purposes). However, classifying visualizations Software’s logging technology for Pirates of the using this dimension often requires reconstructing Burning Sea [58] or Bioware’s Skynet [23] portal the design decisions that went into the game design, that aggregates large amounts of gameplay data which can be difficult and error-prone. to enable in-depth analysis. We use the following levels for Primary Purpose: • Status: Status visualizations are designed for con- 4.2 Target Audience veying important information to the player, typi- We use Target Audience to model the intended users cally continuously—examples include healthbars of a visualization in a computer game. Our division IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 8 into levels here is very rough; Medler and Magerko [6] • Intermittent: Some visualizations are designed to use Brown’s classification of player types [3] to dis- not always be visible and instead for being called cuss the appropriateness of analytical interactions for up intermittently during gameplay, often because different audiences. We recommend their approach they are large and potentially distracting. Ex- for delving into player personalities. Our framework amples of such representations include in-game has the virtue of being simpler and also including maps for first-person games, inventory screens non-gamers, one goal of this work: that show the character’s current equipment, or • Player: Most visualizations are designed for play- heads-down views in flight simulators. ers, and may either be of a precise nature (such • Retrospective: The newer breed of visual repre- as skill and tech trees), or instances of casual in- sentations in games are designed for retrospec- formation visualization [4] (or even what Medler tive use, often outside of the game or the main and Magerko [6] calls “playful information visu- gameplay, and allow for the user to study their alization”). The purpose of this visualization type performance in a post-mortem fashion. The pur- is not necessarily optimal performance (although pose of this kind of visualization is often to help it may be for competitive multiplayer games), the player improve their skills, for example using and conforming to a player often also means replay theaters in games such as Call of Duty: that the visualization is in spirit with the game’s Black Ops [57] and Halo: Reach [66], but other uses atmosphere and integrated within the game. are just for revealing all of the game state to the • Developer: Visualization has lately become a pop- user after the game, like in summary replays for ular tool among developers for game analytics the Civilization [69] series of games. of telemetry data for debugging, balancing, and • Prospective: Finally, some visual representations playtesting the game. An early example is the are designed to help the user make predictions logging and analytics system used in Pirates of the about the future. A simple example is the dialog Burning Sea [58]; a particularly impressive one is system in Dragon Age II [87] that uses small Bioware’s SkyNet [23] tool that integrates a web- glyphs next to each dialog option to indicate its based analytics portal with the game. emotional tone, or the practice of color-coding the • Observer: A growing trend in the gaming commu- names of monsters (i.e., what is shown when the nity is not only playing games, but also watching user selects a monster to potentially attack) in other people play them. Competitive multiplayer many role-playing games to indicate the risk of gaming is now a spectator sport, with first-person failure if the user would try to kill the monster. shooters such as Quake [86] and real-time strategy games such as StarCraft [77] leading the way, and the needs of an observer are very different 4.4 Visual Complexity from those of a player. Visualization can help; The Visual Complexity captures the level of visual for example, Halper and Masuch [41] use visual sophistication of the visualization that is embedded techniques to automatically create a highlight reel in the game. It can be argued that this is a feature of from an FPS session. the visualization itself and not its application to com- puter and video games, but we think it is important because it gauges the level of visual literacy that the 4.3 Temporal Usage game designers are attributing to the players. Another The first visual depictions of game state were proba- issue that may be interesting to study is whether the bly score and healthbars, and these were designed to level of visual complexity is increasing with time, always be visible in order to give the player continu- with newer games including more advanced visual ous feedback about their character’s state. However, representations than older ones. as visualization in games matures, we are starting to • Basic: Very simple visual representations, with see other timing aspects for its use. We capture these healthbars, powerbars, and ammo counters being aspects in the Temporal Usage dimension, and we the canonical example. This level of visual com- define it as having the following values: plexity has basically existed within games since • Continuous: The most common level is continuous the inception of digital games. representation, where the visualization is (more • Intermediate: This level represents visual represen- or less) constantly updating and provides live tations that use more advanced mappings that go feedback about the game state. The canonical beyond the standard visual variables of length, example of a continuous visual representation is size, color, position, etc [53]. We count most stan- the ubiquitous healthbar and its variations (am- dard statistical data graphics (such as the score mobar, powerbar, energybar, etc); see Figure 1(a). and kill statistics in Call of Duty: Black Ops [57]), Several such continuous representations are often illustrative animations (such as the summary integrated into heads-up displays [7]. replay in Civilization [69]), and in-game sprites IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 9

for guidance or recorded gameplay (known as through them, and “radar” tools reveal enemy ludophasmas [27]) as belonging to this category. positions on a map (note that some games use • Advanced: Finally, we also survey techniques that radars as immersive components of gameplay, so use an advanced visual language, thus assuming classification depends on the game). a high degree of visual literacy on behalf of • Immersive/Separate: Although being separate from the users. While the choice between intermedi- the game itself would seem to suggest that these ate and advanced complexity sometimes can be visual representations cannot be immersive, there debatable, we would classify advanced represen- still exists such examples in the market. One such tations to be those that casual users would not is World of Warcraft Remote [64] (http://us.battle. often come across in their daily lives. Given this net/wow/en/services/wow-remote/), which al- guideline, we classify the chloropleth maps in lows players to chat in-game and participate in SimCity [56], diplomacy graphs showing relations in-game item auctions using their smartphone. between factions (commonly used in the Civiliza- • Informative/Separate: Finally, we also include vi- tion [69] series and other 4X games), and basic sual representations that are implemented out- hierarchical representations for character progres- side of the game. This case has received little dis- sion (such as skill trees in Diablo II [88] or talent cussion so far in this paper, but represents a sig- trees in World of Warcraft [64]) as part of this class. nificant trend on the Internet as game developers open up their data APIs so that hobby gamers can analyze (and visualize) them. Examples include 4.5 Immersion/Integration annotated monster maps for games such as World Finally, we believe it to be useful to capture the of Warcraft [64] and Dark Age of Camelot [80], the visualization’s immersion factor in the game, as well data API [40] for Spore [61], and the player-drawn as whether or not it is part of the game application political maps in EVE Online [82]. itself or is found outside the game proper. These are clearly not orthogonal factors—a visualization sepa- rate from the game itself is naturally less immersive 5 DESIGN PATTERNS than one inside the game—but we think it is useful In this section we show how to use our new frame- to keep this distinction to capture the full spectrum work by aggregating common uses of visual repre- of visualization in games and their application. sentations in games into design patterns [54] and then • Immersive/Integrated: These visualizations are im- classifying them using the taxonomy—see Table 1. We plemented inside the game itself, and they are focus on five high-level categories of patterns in this in spirit with the game’s atmosphere. The latter text, each grouping together several actual patterns. part naturally depends on the game genre—for example, line charts and pie diagrams are fully 5.1 Heads-Up Display appropriate for a business simulation game, but not necessarily so for a first-person shooter—and A heads-up display [7] (HUD) is a common term for must therefore be judged on the visualization’s all in-game visual representations that are integrated application inside the game rather than on the with the live gameplay; in particular, the HUD often visualization itself. One example of an interest- consists of healthbars, magic bars, and ammunition ing immersive/integrated visual representation counters as well as interactive maps, toolbars, and is the hit direction indicator found in almost all other interface components. All of these visual rep- modern first-person shooters; it shows the rough resentations are overlaid on top of the 2D or 3D view direction the player is taking fire from, but we of the game world using transparency. still regarded it as immersive because it mimics Despite their widespread use, it may be argued information that the character would have but that HUDs are not optimal designs because they exist which is difficult to convey to the player. outside of diegetic space (i.e., the space of the story). • Informative/Integrated: Other visual representa- For this reason, an increasing number of games are tions are added to a game for the purpose of reducing the size of their HUDs, or removing them providing additional information to the player. altogether; for example, RPGs such as Dragon Age They are still implemented inside the game ap- II [87] and Mass Effect 2 [60] only show healthbars plication, but they often reveal information about when in combat, and only then when characters have game state to the player that the player’s char- taken damage, and many first-person shooters such as acter would have no access to inside the game Call of Duty: Modern Warfare 2 [89] and Battlefield: Bad world. Examples include replay theaters and per- Company 2 [90] are replacing healthbars with making formance charts over time in an FPS. It should be the player’s screen flash red, darken at the edges, and noted that many cheats fall inside this category; grow blurry as the player takes damage (Figure 7). for example, so-called “wallhacks” in FPS remove Some of the common low-level design patterns for walls in levels so that players can see opponents heads-up displays include the following: IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 10

Primary Target Temporal Visual Immersion/ Pattern Group Purpose Audience Usage Complexity Integration Examples Heads-Up Display status player continuous basic imm/int common Replay Theater training/comm player/observer retrospective basic inf/– [57], [78] Progression Tree progression player prospective advanced inf/int [69], [70], [88] Data in Space train/comm/status player/observer retrospective inter/advanced inf/– [56], [57], [66] Data in Time train/comm/status player/observer retrospective inter/advanced inf/– [66], [69], [70] TABLE 1 Common design pattern groups for visualization in games. Fields marked as ’–’ change between designs.

observers that are spectating on a match. Developers may also want to use replays to catch bugs and game design issues (it is integrated into the SkyNet [23] sys- tem). Gaming, particularly multiplayer gaming, has a long history of recording game sessions as videos (a practice more generally known as machinima [55]), but these restrict the viewer to just watching a non- Fig. 7. Immersive heads-up display in Call of Duty: interactive animation. Replay theaters take this a step Modern Warfare 2 [89] where the blood on the screen further by recording telemetry for all entities in a indicates that the player has taken damage, and the game session, not just the screen of one player. red fan shape (inset) is the hit direction indicator. • Time controls: The addition of time controls relax the temporal constraints of the live gameplay and allows the viewer to navigate the recorded • Information readout: Basic visual representations session in time, as well as freeze time at will. of game state, such as score, healthbars, power- • Space controls: Like time controls, the space con- bars, compass heading, current target, etc. trols pattern for a replay theater allows the user to • Crosshairs: Common in first-person shooters, freely navigate in the game environment to study crosshairs augment the visual display with an the gameplay action from different viewpoints. indication of where exactly the player character’s • Event timeline: Timeline visualizations can be in- weapon is aimed. This is different from using the tegrated with the replay theater to aid navigation actual weapon sights for this purpose, in which in the replay data; for example, the theater mode case the crosshairs are part of the game world in the recent Call of Duty: Black Ops [57] includes and not a HUD component. a timeline visualization that shows player deaths • Hit indicator: A direction indicator that is shown and kills over the course of the session (Figure 8). on the screen whenever the player is being hit by gunfire to indicate roughly from which direction the enemy fire is coming (Figure 7). There also exist immersive (i.e., that are integrated in the game world) HUD design patterns: • Cockpit: Some genres (particularly simulations) lend themselves to using the actual cockpit or Fig. 8. Playback control interface for the replay theater dashboard of the in-game vehicle to communicate in Call of Duty: Black Ops [57]. Note the timeline on top information, for example using cockpit gauges that visualizes the player’s kills and deaths as green and meters to show speed, heading, and fuel. and red lines at different positions in time. • World annotation: These add visual representa- tions to the world itself, such as the optimal rac- ing line in driving games, and directional arrows 5.3 Progression Tree and highlighted objects in 3D environments. Progression trees are directed graphs (or trees, in the • Ludophasmas: In-game sprites that guide the special case) modeling dependencies between con- player or communicate previously recorded cepts to allow for player progression in a game. These gameplay inside the world, but which still are graphs are often visualized in games to make these narrative elements (see Medler [27] for details). dependencies visible to players and allow them to plan ahead. Since the graphs are generally fixed and 5.2 Replay Theater do not change dynamically, graph layout algorithms Replay theaters are mechanisms for post-mortem anal- are typically not needed in the actual game. ysis of a game session by playing back the session, • Skill tree: Many games that are focused on either for players to see their own performance or for character progression, such as RPGs, use skill IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 11

trees to let the player customize their character’s 5.5 Data in Time abilities. The skill tree thus captures the prerequi- This group of design patterns convey information sites needed to learn a particular skill. Examples about game state as a function of the temporal di- include the skill system in Blizzard’s Diablo II [88] mension, and for this reason these patterns are often and the talent system in World of Warcraft [64], retrospective and informative in nature: • Technology tree: This pattern is used to capture • Event timeline: We discussed event timelines as progression of technology in 4X games; for ex- part of the replay theater, but these have a more Civilization ample, in the game [69], the technolo- general use for conveying discrete event sequen- gies Horseback Riding and Feudalism are both cies over time. For example, SC2Replayed.com needed in order to research Chivalry. Figure 5(c) renders build order over time (Figure 6) for Star- Civilization III shows part of the tech tree in [70]. Craft II [78] multiplayer matches. • Time-series chart: This pattern models the visual representation of one or several statistical vari- 5.4 Data in Space ables over the temporal domain. Business simu- Patterns in the data in space group are visual repre- lation games regularly incorporate sales diagrams sentations of spatial game data. This is not a new and other forms of data in time, and exercise phenomenon in games, but it is increasingly gaining games such as Wii Fit [91] often visualize player traction outside of the CMS and 4X genres. performance as a function of time. Call of Duty: Black Ops • Annotated map: Mapmaking of game worlds [6] [57] has a player dossier [47] system has always been common for RPGs, and this containing several time-series charts (Figure 9). practice is even more widespread in communities for MMORPGs such as EVE Online [82], Dark Age of Camelot [80], and World of Warcraft [64]. • Influence map: This pattern creates color-coded political maps of space to show the influence and territory of different factions in a game. For example, the Civilization [69] franchise includes so-called “histographs” (Figure 5(a)) that show the respective influences of different empires over continents as a space-filling visualization. • Heatmap: These thematic maps convey quantity Fig. 9. Wager earnings over time in the Combat of a statistical variable over spatial units, such Record for competitive multiplayer performance of a as the chloropleth maps in SimCity [56]. Perhaps player in Call of Duty: Black Ops [57]. the most exciting development here are the spa- tial visualizations used in FPS games, including Halo: Reach Black Bungie’s [66] and Treyarch’s 5.6 Additional Patterns Ops [57], that aggregate millions of multiplayer games to visualize where players tend to die and Our above list of design patterns is far from exhaus- to kill other players in different game levels. Fig- tive and merely represents the most common such ure 10 shows death and kill heatmaps for seven patterns in a first attempt to standardize a simple levels (provided by Bungie.net), and then show pattern catalogue. Several additional design patterns the difference map that we have constructed exist: for player dossiers [47], which aggregate metrics which indicates areas on each map where players about a player’s performance; for mission planners, tend to be at an advantage (i.e., more players kill which allow players (and/or game designers) to plan other players here than are killed). scenarios, e.g., for a military simulation involving • Fog of War: Modern strategy games often employ many units; or for status reports, which show the the concept of fog of war where the player is current status of the game as a series of reports, such only made aware of enemy movement in areas as the economy, technology, and diplomacy advisor to where one of their units has direct line-of- screens (Figure 5) in the Civilization [69] franchise. sight, and the visual representations of this con- cept is an instance of a data in space pattern. 6 IMPLICATIONS In practice, this is often implemented by only There are many implications to be drawn from our showing actively viewed areas of a game world work, both for improving games using visualization in full detail, and blacking out the unexplored as well as for improving visualization using games. areas and greying out the explored but currently Below we review some of these implications and ben- not actively viewed areas (this is the case in the efits. However, we start with the effect of community real-time strategy game Warcraft [76]). on visualization for games. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 12

Anchor 9 Boardwalk Boneyard Forge World Condemned Breakpoint Countdown Fig. 10. Global heatmaps for deaths (top row), kills (middle row), and their difference (bottom row) for seven multiplayer maps in Halo: Reach [66]. The difference map (our contribution) represents areas in the map where, statistically speaking, you are at an advantage to kill other players but not be killed yourself.

6.1 Social Motivators 6.3 Potential Benefits to Visualization In surveying the repertoire of games with visualiza- One obvious question in all this is what the benefit of tion components, one trend is obvious: games sup- applying visualization to games is to us as visualiza- porting multiplayer are disproportionally represented tion researchers. While it can be argued that games in this list. The reason for this is that multiplayer have a long way to go until they start incorporating games create a community of players interacting with advanced visualization techniques, let alone begin each other inside the game, and this drives a need to influence research in visualization as a field, the for communicating with each other, sharing game benefit of this approach lies more in adoption and experiences, and comparing oneself to other players. impact than scientific progress. With more than 60 Having a strong community also causes a player’s million video game consoles in North America alone, investment of time and effort in a game to increase, the video and computer game industry represents thereby giving an incentive to players to create their a huge base of potential casual users scattered in own visualization components using live telemetry living rooms worldwide. As mentioned early on in data provided by the game developers. The benefits this paper, we believe that we are on the cusp of a are clearly major, and we hope that more game devel- major adoption of visualization on a large scale, and opers in the future follow the example of games such visualization researchers should be part of this trend as Spore [61] and Halo: Reach [66] in providing public to help shape and guide it in the right direction. APIs for accessing game statistics and telemetry. Gamers are an interesting population because they are often very passionate and willing to spend time 6.2 Potential Benefits to Games on improving their own gameplay, or sometimes just watch other people play. In addition, they often have A large portion of this paper has been about show- a high degree of visual literacy and spatial ability [3]. casing the benefit of applying visualization in games: Integrating visualization components in games rep- • Players can use visualization to improve their resents a unique opportunity for tapping into this skills, be more aware of the game world and its population and exposing it to our technologies. state, and communicate their achievements and As stated earlier in this paper, one potential synergy progress to fellow players; for games is the equally unexploited sports domain, • Developers can use visualization to detect bugs, which also shares many of the same challenges, au- find game balance issues, and draw inspiration dience, and opportunities—not to mention data types from player behavior; (primarily spatiotemporal data and simple statistics). • Observers can use visualization to draw more One benefit may thus be that advances in one domain enjoyment from watching others play through could also be applied to the other domain. better access to game data. Finally, while visual representations in games cur- We look forward to seeing how future work in rently admittedly are simplistic, this is not to say this design space can extend and improve games in that researchers cannot make significant progress by directions not covered in this paper. conducting visualization research related to games; in IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 13 fact, many of the visualization problems encountered [5] B. Medler, M. John, and J. Lane, “Data Cracker: Developing in games are reflections of similar problems encoun- a visual game analytic tool for analyzing online gameplay,” in Proceedings of the ACM Conference on Human Factors in tered in the real world. The difference may be that Computing Systems, 2011, pp. 2365–2374. the game world can be more easily controlled and [6] B. Medler and B. Magerko, “Analytics of play: Using infor- instrumented than the real world. For this reason, mation visualization and gameplay practices for visualizing video game data,” Parsons Journal for Information Mapping, vol. we certainly believe there is ample opportunity for III, no. 1, pp. 1–12, 2011. deriving new visualization techniques for game data [7] V. Zammitto, “Visualization techniques in video games,” in that may become more broadly applicable to real- Proceedings of Electronic Information, the Visual Arts, and Beyond, 2008, pp. 267–276. world data and real-world problems as well. [8] T.-M. Rhyne, “Computer games’ influence on scientific and information visualization,” IEEE Computer, vol. 33, no. 12, pp. 6.4 Limitations and Improvements 154–156, 2000. [9] ——, “Computer games and scientific visualization,” Commu- The framework proposed in this paper is consistent nications of the ACM, vol. 45, no. 7, pp. 40–44, Jul. 2002. [10] S. K. Card, J. D. Mackinlay, and B. Shneiderman, Eds., Read- with earlier work by Zammitto [7] as well as Medler ings in Information Visualization: Using Vision to Think. San and Magerko [6], but extends these works in signifi- Francisco: Morgan Kaufmann Publishers, 1999. cant new ways with categorical dimensions, practical [11] Truna aka J. Turner and D. Browning, “Workshop on HCI design patterns, and future implications. Furthermore, and game interfaces: A long romance,” in Proceedings of the Austalian HCI Conference, 2010. we have successfully used the framework to classify [12] B. Shneiderman, “Direct manipulation: A step beyond pro- and categorize five design patterns that are commonly gramming languages,” Computer, vol. 16, no. 8, pp. 57–69, recurring in game design. While no design space 1983. [13] A. R. Galloway, Gaming: Essays On Algorithmic Culture. Min- framework is ever truly complete, these are still good neapolis: University of Minnesota Press, 2006. indications that our framework has sufficient expres- [14] P. Barr, J. Noble, and R. Biddle, “Video game values: Human- sive power for capturing these design patterns. computer interaction and games,” Interacting with Computers, vol. 19, no. 2, pp. 180–195, 2007. Having said that, there is certainly room for im- [15] R. Bernhaupt, Evaluating User Experience in Games: Concepts and provements and refinements in the framework. For Methods. London: Springer, 2010. example, the only real concession to the nature of [16] D. Pinelle, N. Wong, and T. Stach, “Heuristic evaluation for games: Usability principles for video game design,” in Pro- visual representation is the “Visual Complexity” cat- ceedings of the ACM Conference on Human Factors in Computing egory; many more visual features could be modeled Systems, 2008, pp. 1453–1462. here. On the other hand, there is a value to keeping [17] P. Zaphiris and C. S. Ang, “HCI issues in computer games,” Interacting with Computers, vol. 19, no. 2, pp. 135–139, 2007. the framework orthogonal to general visualization [18] J. Dyck, D. Pinelle, B. Brown, and C. Gutwin, “Learning from taxonomies [52] to minimize complexity, which future games: Hci design innovations in entertainment software,” in improvements should try to maintain. Proceedings of Graphics Interface, 2003, pp. 237–246. [19] C. Johnson, “Using cognitive models to transfer the strengths of computer games into human computer interfaces,” in Pro- 7 CONCLUSIONSAND FUTURE WORK ceedings of the Workshop on Fun and Human-Computer Interaction, 1998. Drawing on a wide variety of examples from the last [20] T. W. Malone, “Heuristics for designing enjoyable user inter- two decades of video game history, we formulate the face: Lessons from computer games,” in Proceedings of the ACM design space of visualization for games as an exciting Conference on Human Factors in Computer Systems, 1982, pp. 63– 68. new branch of casual information visualization with [21] R. Pausch, R. Gold, T. Skelly, and D. Thiel, “What HCI the potential for broad impact in millions of living designers can learn from video game designers,” in Proceedings rooms worldwide. We proceed to use our framework of the ACM Conference on Human Factors in Computing Systems, 1994, pp. 177–178. to identify common design patterns for how to em- [22] J. T. Stasko, J. Domingue, M. H. Brown, and B. A. Price, ploy visualization in computer and video games. Eds., Software Visualization — Programming as a Multimedia It is clear that both the visualization and the game Experience. The MIT Press, 1998. [23] G. Zoeller, “Development telemetry in video games projects,” development communities so far only have scratched in Proceedings of the Game Developers Conference, 2010. the surface of this exciting new problem domain. Our [24] N. Andrienko and G. Andrienko, Exploratory Analysis of Spatial future work will continue mapping out interesting and Temporal Data: A Systematic Approach. Springer-Verlag, 2006. examples that will help progress in both fields. [25] B. Shneiderman, “The eyes have it: A task by data type taxonomy for information visualizations,” in Proceedings of the REFERENCES IEEE Symposium on Visual Languages, 1996, pp. 336–343. [26] W. S. Cleveland, Visualizing Data. Summit, NJ, USA: Hobart [1] S. E. Siwek, “Video games in the 21st Century: The 2010 Press, 1993. report,” Entertainment Software Association, Tech. Rep., 2010. [27] B. Medler, “Generations of game analytics, achievements and [2] Ipsos MediaCT, “Essential facts about the computer and video high scores,” Journal for Computer Game Culture, vol. 3, no. 2, game industry: 2010 sales, demographic and usage data,” pp. 177–194, 2009. Entertainment Software Association, Tech. Rep., 2010. [28] L. Jin and D. C. Banks, “TennisViewer: A browser for com- [3] S. Brown and C. Vaughan, Play: How It Shapes the Brain, Opens petition trees,” IEEE Computer Graphics & Applications, vol. 17, the Imagination, and Invigorates the Soul. Avery, 2009. no. 4, Jul.–Aug. 1997. [4] Z. Pousman, J. T. Stasko, and M. Mateas, “Casual information [29] A. Cox and J. Stasko, “SportVis: Discovering meaning in sports visualization: Depictions of data in everyday life,” IEEE Trans- statistics through information visualization,” in Conference actions on Visualization and Computer Graphics, vol. 13, no. 6, pp. Compendium of the IEEE Symposium on Information Visualization, 1145–1152, 2007. 2006, pp. 115–116. IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 14

[30] C. G. Healey, “Visualizing NFL football games,” http://www. [60] ——, Mass Effect 2. Electronic Arts, 2010. csc.ncsu.edu/faculty/healey/NFL viz/, 2008. [61] Maxis, Spore. Electronic Arts, 2008. [31] A. Drachen and A. Canossa, “Towards gameplay analysis via [62] Splash Damage, Wolfenstein: Enemy Territory. Activision, 2003. gameplay metrics,” in Proceedings of the International MindTrek [63] Visceral Games, Dead Space 2. Electronic Arts, 2011. Conference, 2009, pp. 202–209. [64] Blizzard, World of Warcraft. Blizzard, 2004. [32] “Mochibot,” http://www.mochibot.com/. [65] Valve Corporation, Steam. Valve Corporation, 2003. [33] “Nonoba: Statistics & tracking,” http://nonoba.com/ [66] Bungie, Inc., Halo: Reach. Microsoft Game Studios, 2010. developers/statistics. [67] EA Canada, FIFA 10. EA Sports, 2009. [34] “Playtomic,” http://playtomic.com/. [68] Treyarch, Call of Duty: World at War. Activision, 2008. [35] “Kontagent: Social analytics,” http://www.kontagent.com/. [69] MicroProse, Civilization. MicroProse, 1991. [36] J. H. Kim, D. V. Gunn, E. Schuh, B. Phillips, R. J. Pagulayan, [70] Firaxis Games, Civilization III. Infogrames, 2001. and D. R. Wixon, “Tracking real-time user experience (TRUE): [71] Bullfrog, Populous. Electronic Arts, 1989. a comprehensive instrumentation solution for complex sys- [72] ——, Populous II: Trials of the Olympian Gods. EA, 1991. tems,” in Proceedings of the ACM Conference on Human Factors [73] id Software, Wolfenstein 3D. Apogee Software, 1992. in Computing Systems, 2008, pp. 443–452. [74] ——, Doom. id Software, 1993. [37] RAD Game Tools, “Telemetry profiling system,” 2010, http: [75] Westwood Studios, Dune II: Battle for Arrakis. Virgin, 1992. //www.radgametools.com/telemetry.htm. [76] Blizzard, Warcraft: Orcs & Humans. Blizzard, 1994. [38] J. Ludwig, “Flogging: Data collection on the high seas,” in [77] ——, StarCraft. Blizzard, 1998. Proceedings of the Game Developers Conference, 2007. [78] ——, StarCraft II: Wings of Liberty. Blizzard, 2010. [39] D. Moskowitz, S. Shodhan, and M. Twardos, “Spore API: ac- [79] S2 Games, Heroes of Newerth. S2 Games, 2010. cessing a unique database of player creativity,” in SIGGRAPH [80] Mythic, Dark Age of Camelot. Mythic Entertainment, 2001. Talks, 2009. [81] ——, Warhammer Online: Age of Reckoning. EA Games, 2008. [40] M. Twardos, “Spore data,” http://thevioletpiece.com/spore/. [82] CCP Games, EVE Online. CCP Games, 2003. [41] N. Halper and M. Masuch, “Action summary for computer [83] EA Tiburon, Madden NFL 11. EA Sports, 2010. games: Extracting action for spectator modes and summaries,” [84] MicroProse, Formula One Grand Prix. MicroProse, 1992. in Proceedings of the International Conference on Application and [85] Criterion Games, Need for Speed: Hot Pursuit. EA Games, 2010. Development of Computer Games, 2003, pp. 124–132. [86] id Software, Quake. GT Interactive, 1996. [42] N. Hoobler, G. Humphreys, and M. Agrawala, “Visualizing [87] Bioware, Dragon Age II. Electronic Arts, 2011. competitive behaviors in multi-user virtual environments,” in [88] Blizzard North, Diablo II. Blizzard, 2001. Proceedings of the IEEE Conference on Visualization, 2004, pp. [89] Infinity Ward, Call of Duty: Modern Warfare 2. Activision, 2009. 163–170. [90] DICE, Battlefield: Bad Company 2. Electronic Arts, 2010. [43] L. Chittaro, R. Ranon, and L. Ieronutti, “VU-Flow: A visu- [91] Nintendo, Wii Fit. Nintendo, 2007. alization tool for analyzing navigation in virtual environ- ments,” IEEE Transactions on Visualization and Computer Graph- ics, vol. 12, no. 6, pp. 1475–1485, Nov./Dec. 2006. [44] R. Thawonmas and K. Iizuka, “Visualization of online-game players based on their action behaviors,” International Journal Brian Bowman is a senior undergraduate of Computer Games Technology, 2008. student in Computer Engineering at Purdue [45] R. Thawonmas, M. Kurashige, and K.-T. Chen, “Detection of University in West Lafayette, IN, USA. He will landmarks for clustering of online-game players,” International be graduating from Purdue in Spring 2012. Journal of Virtual Reality, vol. 6, no. 3, pp. 11–16, 2007. He is a member of IEEE-HKN. He probably [46] S. S. Joslin, R. A. Brown, and P. Drennan, “The gameplay visu- plays too many video games. alization manifesto: a framework for logging and visualization of online gameplay data,” Computers in Entertainment, vol. 5, 2007. [47] B. Medler, “Player dossiers: Analyzing gameplay data as a re- ward,” International Journal of Computer Game Research, vol. 11, no. 1, Feb. 2011. [48] N. Diakopoulos, F. Kivran-Swaine, and M. Naaman, “Playable data: Characterizing the design space of game-y infographics,” in Proceedings of the ACM Conference on Human Factors in Computing Systems, 2011. Niklas Elmqvist is an assistant professor [49] T. Northcutt, “Leaderboards can suck it! five better ideas for in the School of Electrical and Computer visualizing game data,” in Proceedings of the Game Developers Engineering at Purdue University in West Conference, 2010. Lafayette, IN, USA. He was previously a [50] L. Byron and M. Wattenberg, “Stacked graphs - geometry postdoctoral researcher at INRIA in Paris, & aesthetics,” IEEE Transactions on Visualization and Computer France. He received his Ph.D. in 2006 Graphics, vol. 14, no. 6, pp. 1245–1252, 2008. from Chalmers University of Technology in [51] K. Quealy, “Gamers mimic the season’s ups and downs,” The Goteborg,¨ Sweden. He is a member of the New York Times, Feb. 2011. IEEE and the Computer Society. In another [52] S. K. Card and J. Mackinlay, “The structure of the information life, he is a wizard, assassin, or knight in visualization design space,” in Proceedings of the IEEE Sympo- whatever game currently takes his fancy. sium on Information Visualization, 1997, pp. 92–99. [53] J. Bertin, S´emiologiegraphique: Les diagrammes - Les r´eseaux- Les cartes, Paris, France, 1967. [54] C. Alexander, S. Ishakawa, and M. Silverstein, A Pattern Language: Towns, Buildings, Construction. New York: Oxford T.J. Jankun-Kelly is an associate Professor University Press, 1977. of computer science and engineering within [55] Machinima.com, http://www.machinima.com/. the James Worth Bagley College of En- gineering, Mississippi State University, MS, USA. His research lies at at the intersec- GAME EXAMPLES tion of scientific and information visualization. He received his PhD from the University of [56] Maxis Software, SimCity. Brøderbund, 1989. California, Davis, in 2003 and is a member [57] Treyarch, Call of Duty: Black Ops. Activision, 2010. of IEEE and the Computer Society. He, too, [58] Flying Lab Software, Pirates of the Burning Sea. SOE, 2008. probably plays too many video games. [59] Bioware, Dragon Age: Origins. Electronic Arts, 2009.