IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 1 Toward Visualization for Games: Theory, Design Space, and Patterns Brian Bowman, Niklas Elmqvist, Member, IEEE, and T.J. Jankun-Kelly, Member, IEEE Abstract—Electronic games are starting to incorporate in-game telemetry that collects data about player, team, and community performance on a massive scale, and as data begins to accumulate, so does the demand for effectively analyzing this data. In this paper, we use examples from both old and new games of different genres to explore the theory and design space of visualization for games. Drawing on these examples, we define a design space for this novel research topic and use it to formulate design patterns for how to best apply visualization technology to games. We then discuss the implications that this new framework will potentially have on the design and development of game and visualization technology in the future. Index Terms—Computer games, video games, interactive entertainment, entertainment, visualization, game analytics. F 1 INTRODUCTION and use it to mark special achievements and unlock new content. For example, the multiplayer version IDEO and computer games have a long history of the recent Call of Duty: Black Ops [57] first-person of using visual representations of data to convey V shooter collects vast amounts of statistics about player information to the player; for example, colored bars performance, and not only uses this to award special are routinely used to present vitals—health, energy, titles, emblems, and equipment to players as they play and magical power—of the player character and are the game, but also to aggregate all statistics (such as accordingly often called “healthbars” (Figure 1(a)), the number of shots fired worldwide, the number of and many games use overhead maps to show the cumulative feet fallen by all players, or the number locations of entities in the game world. Some games of vehicles exploded) and feed it back to the entire have already gone beyond such simple visual repre- player community as a social motivator. sentations and are employing statistical data graphics We believe that the extraction, management, and to present more data to the player. For example, the visualization of data in games is an exciting new first version of the classic SimCity [56] game from branch of research with huge potential impact—video 1989 used chloropleth maps to convey metrics such games sales in the U.S. alone reached $10.5 billion in as crime rate in a particular city (Figure 1(b)). 2009 [1], and the games industry estimates the number In general, interactive games can be seen as real- of game consoles in North America to surpass 60 time simulations—just like SimCity—and there are million units (not counting personal computers, which many situations where feeding back the state of the clearly also can be used for gaming) [2]. Furthermore, simulation to the player or the game developer can gamers tend to be highly motivated and often have a be beneficial, regardless of game genre. For example, a high degree of spatial and visual literacy [3]. For these player of a first-person shooter can use session replays reasons, we think that we are on the cusp of a major to improve their movement around a map, or a mili- adoption of casual visualization [4] technology in mil- tary strategy game player can use statistics about map lions of living rooms worldwide. The only question is influence and army strength over time to optimize how to best harness visualization technology for the their use of resources. The ongoing revolution in challenges inherent to this exciting new domain. gaming on social media sites (such as on Facebook) In this paper, we build on the existing academic also showcases the power of sharing a player’s per- research on this topic— [5], [6], [7]—to take the initial formance and experience as a social indicator. In ad- steps toward a theoretical framework and methodol- dition, data collection and feedback can also be used ogy for the use of visualization in games. We begin as a motivational factor; a recent trend in video and by examining existing evidence of visualization and computer game design—previously mostly restricted data collection in games, starting from SimCity in to sports games—is to increasingly instrument games 1989 and moving to the Black Ops and Halo: Reach to collect large amounts of data during gameplay of present day. We then derive a design space of data and visualization types in the context of games. • B. Bowman and N. Elmqvist are with the School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN. Based on this design space and on the evidence E-mail: [email protected], [email protected] collected, we propose a set of design patterns for how • T.J. Jankun-Kelly is with Mississippi State University, Starkville, MS. to best approach games as a visualization researcher E-mail: [email protected] or practitioner. Finally, we derive implications that IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 2 (a) Health (green) and mana bars (blue) for a player (b) Chloropleth map of (c) Hit heatmap in (d) Death heatmap in Halo: character in World of Warcraft. crime rate in SimCity. Black Ops. Reach (level Anchor 9). Fig. 1. Examples of visualizations in genres ranging from role-playing games (World of Warcraft), construction and management simulations (SimCity), and first-person shooters (Call of Duty: Black Ops and Halo: Reach). this work will have on both game and visualiza- Furthermore, we should emphasize that although tion research. These implications show how social virtually all computer games today incorporate ad- motivators often drive the introduction of visualiza- vanced and often near-photorealistic graphics, we re- tions in games: player communities for competitive gard such aspects to fall within the scope of computer multiplayer games use statistics and visualizations graphics (and applied domains such as Virtual Real- to communicate with each other, share game experi- ity) and not visualization. In fact, games are one of the ences, and improve their performance. Furthermore, driving forces for consumer-level computer graphics, we discuss the benefits of studying visualization for and, by extension, visualization [8], [9]. However, games, both in terms of benefits to games—such as while the 3D rendering of a fictional world within a better gameplay, easier balancing and debugging, and computer game strictly speaking is a visual represen- more enjoyable spectating—as well as in terms of tation of data (the 3D world), we delimit this work benefits to visualization research—such as massive to focus on the use of visualization to represent data adoption, a dedicated user base, and a controlled and with no intrinsic spatial form (such as health, points easily instrumented data source (the game world). over time, political influence, etc). In other words, our treatment here is basically performed from an information visualization (InfoVis) viewpoint [10]. 2 BACKGROUND A computer or video game is an electronic game where 2.1 HCI and Games the player interacts with a visual display using some form of input device. Stated in this very generic Games and human-computer interaction (HCI) have form, it is clear that video games have many com- a long and colorful history [11]. Already in 1983, Ben monalities with general interactive systems, such as Shneiderman used games as illustrative examples in word processors, web browsers, and spreadsheets. his landmark paper on direct manipulation [12], laud- The difference lies primarily in the goal-driven nature ing them for smooth, effortless, and direct interaction. of the game, and sometimes in specialized hardware, Galloway’s classification of action in gaming partic- including dedicated game controllers—such as joy- ularly identify interface elements as key nondiegetic sticks, gamepads, steering wheels—or even dedicated (i.e. outside of the game world) aspects of games [13]. video game consoles—such as the current seventh- As a result, considerable research has been directed generation consoles like the Sony Playstation 3, the toward applying HCI methods to games [14], [15], Microsoft Xbox 360, or the Nintendo Wii. [16], [17], but also toward applying techniques from Although we use the terms computer game and video games into HCI [18], [19], [20], [21]. game interchangeably, these terms traditionally refer to A key difference between general HCI and games whether the game is targeted at a personal computer is that for the former, the goal is to help users to (PC or Mac) or a video game console. This paper is become experts as quickly and effortlessly as possi- about both of these types of platforms and makes little ble, whereas games are generally designed using the distinction between them—discrepancies arise only motto “easy to learn but difficult to master” [20]. This for visualization components outside a game (such is an important point; as with HCI, the usability of as on a website), which generally are targeted for a visualization should never be a hindrance against computers, regardless of the actual game platform. players reaching their goals (intrinsic difficulties in the gameplay serve this purpose already). We delimit our treatment in this paper to games, and not real-time interactive simulations for training purposes outside of the entertainment domain. While 2.2 Applied Visualization these share many of the same characteristics, we lack Visualization for games is still in its infancy, but there the space to survey this domain as well. is a number of existing visualization technologies that IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, VOL. XX, NO. Y, MONTH 2012 3 may be relevant to this novel domain.