Characterizing and Understanding Game Reviews José P. Zagal Amanda Ladd, Terris Johnson College of Computing and Digital Media College of Computing DePaul University Georgia Institute of Technology 243 South Wabash Avenue 801 Atlantic Drive Chicago, IL 60604, USA Atlanta, GA 30332, USA [email protected] {amanda.ladd,terris}@gatech.edu ABSTRACT Game reviews often overshadow other forms of journalistic Game reviews are one of the primary forms of videogame discourse surrounding games such as news, investigative journalism and are also one of the prevalent forms of discourse reporting, and commentary (Thomas 2007). Reviews have also about games. How can we characterize them and better been maligned for a variety of reasons such as being inconsistent understand the role they might play in helping create and shape in writing and style (Thomas, Orland et al. 2007), or being “rife our understanding of games? This article reports on the results of with grammatical errors, historical inaccuracies, plagiarism, run a study that analyzed and deconstructed a selection of game on sentences, [and] clichés” (Buffa 2006b). They have also been reviews from popular online websites. Our findings show that accused of failing to take a critical and analytical viewpoint game reviews are rich and varied in terms of themes and topics (Costikyan 2008). Another accusation is that reviews don’t covered and that considering them purely as “shopping guides” explain what playing a particular game feels like and that they ignores the broader role they play in the discourse of games. We don’t analyze what a game means outside the game itself found that game reviews often include game design suggestions, (Klostermann 2006). hypothesize about the intentions and goals of game creators, and Despite the controversy surrounding their quality or importance1, offer advice to readers on how to approach and best enjoy game reviews are undoubtedly an influence on the ways that particular games. More generally, game reviews can also help people view, understand and talk about games. In a prior study, preserve videogame history by contextualizing the links and Zagal and Bruckman found that students taking videogame- historical connections that exist between games. related classes at the university level had difficulties expressing ideas about gameplay or articulating their experience with games Categories and Subject Descriptors (2007). They argue that a possible explanation for this lies in the fact that most of what students read about games are videogame K.8.0 [Personal Computing]: General - Games reviews, and that they are thus generally lacking in models of what in-depth analysis or critique about games look like. Keywords However, what are game reviews like? How can we characterize Videogames, game reviews, understanding games, reviews them? How do game reviews talk about games and what things do they talk about? Is the professional game designer Ernest 1. INTRODUCTION Adams essentially correct when he says that “reviews only Game reviews are one of the primary forms of videogame compare games to other games; they don’t analyze games in their journalism. Andy McNamara, editor-in-chief of Game Informer larger cultural context” (Friedl 2002)? These are some of the Magazine, recently commented at the 2008 Game Developer’s questions we explored by analyzing and deconstructing a large Conference that the purpose of game reviews is to help consumers number of game reviews. Our goal was not to determine whether by leading them to games that make them happy (2008). His game reviews are “good” or “bad”, comment on the quality of words echo the purpose, or role, that game reviews attempt to their writing, or consider whether or not they are effective for fulfill, namely inform the reader so that he or she may make a guiding purchasing decisions. Rather, we aimed to get a sense of decision of whether or not to purchase or play the game reviewed. the kinds of things that appear, or are routinely discussed, in From this perspective, game reviews can be considered as games reviews in order to better understand how these may shape consumer advice. In fact, Stuart argues that most game magazines our understanding of games and the discourse surrounding their are structured in such a way to guide the reader through to use. purchasing a game (2005). The reason for this is that presumably “most readers seem to see magazines as buying guides and little 2. METHODS AND DATA COLLECTION else. They want to know what they’ll be able to buy in the future, In order to characterize videogame reviews we sampled and and what’s worth buying now” (Stuart 2005). analyzed a random selection of game reviews from the most popular specialized media sources on the world wide web. In Permission to make digital or hard copies of all or part of this work for order to determine popularity (and thus, indirectly, reach) we used 2 personal or classroom use is granted without fee provided that copies are the Alexa web ranking system. The only game review sites to not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, 1 For a summary refer to (McCrea 2007). requires prior specific permission and/or a fee. 2 GameFaqs, ranked #30, was not considered because although it ICFDG 2009, April 26–30, 2009, Orlando, FL, USA. Copyright 2009 ACM 978-1-60558-437-9…$5.00. does have game reviews, these are written and submitted by site 215 appear on the 100 most popular sites on the web were IGN reviews). While some of these characteristics are predictable, (www.ign.com, ranked #43) and Gamespot (www.gamespot.com, there are others that are perhaps particular to games (further ranked #48).3 We considered Alexa’s ranking of the most popular research would be required to explore this in detail). sites by users in the United States only. From our analysis we were able to identify nine common themes: For each of the sites, we created a list of all of the reviews posted description, personal experience, reader advice, design during the year 2006 and then, as needed, randomly selected suggestions, media context, game context, technology, design reviews from the list for our analysis. We decided to sample from hypotheses, and industry. Each theme wasn't necessarily present an entire year rather than shorter period such as a few months in in all of the reviews. However, each review had more than one order to minimize the effects of the lack of uniformity in games theme in evidence. For instance, the first few passages of a review releases over an entire year. might relate the personal experience of the reviewer. The Analysis of each of the reviews was conducted in an iterative following sentences might be descriptions interspersed with process in which data from one review confirmed or contradicted reader advice before wrapping up with some sentences about the data from others in order to refine theoretical categories, media context of the game being reviewed. propositions, and conclusions as they emerged from the data We will now briefly describe each theme and show excerpts from (Glaser and Strauss 1967). Essentially, we used open coding to game reviews that exemplify the theme described. In order to bring themes to the surface from deep inside the data (Neuman provide sufficient context the excerpts may include sentences that 2000). We assigned codes or labels to each sentence (or, in some were coded differently from the theme being described. We also cases a few sentences) in a review. Initially, these codes or labels note that the themes are not presented in order of prevalence or often overlapped, and individual sentences often had more than importance. one code or label assigned. As we analyzed more reviews, new codes emerged and existing ones were modified. The goal of this 3.1 Description process was to identify consistencies between codes (codes with Perhaps the most evident or obvious theme for a game review are similar meanings or pointing to the same basic idea) that would those passages dedicated to describing the game being reviewed. begin to reveal themes. This process continued until no further This theme essentially covers the description of the features, codes emerged. By the end of this process more than 120 different modes and/or characteristics, and gameplay of a particular game. 4 reviews were analyzed. "The gameplay in Emergency 3 is divided between a 20- mission single-player campaign and a freeplay mode. In 3. FINDINGS freeplay, you can complete a series of challenges or As described earlier, we were not interested in evaluating the address an endless stream of emergency situations quality or effectiveness of game reviews. Rather, our goal was to spread out over a large town. Regardless of which mode get a sense of the kinds of things that appear, or are routinely you choose, the basic gameplay remains the same. discussed, in games reviews. Do game reviews have particular You're given specific objectives to complete and you characteristics that define them? For example, are game reviews must use a variety of rescue units to do so." – Emergency focused on the technical aspects of games such as the quality and 3 (PC) resolution of the graphics? When reviewing games, what sort of Additionally, it is common for the description to include a brief context and knowledge do reviews assume the reader has? Also, discussion of what the player needs to do to play this game. are there different styles of reviews and how do they vary? Sometimes the descriptive parts of a review may include specific Finally, how could we explain or understand these characteristics instructions such as what buttons need to be pressed on the and what might they tell us about the medium of videogames in controller to achieve certain results.