Heuristics for Video Games Evaluation: How Players Rate Their Relevance for Different Game Genres According to Their Experience Florentin Rodio, Christian Bastien
Total Page:16
File Type:pdf, Size:1020Kb
Heuristics for Video Games Evaluation: How Players Rate Their Relevance for Different Game Genres According to Their Experience Florentin Rodio, Christian Bastien To cite this version: Florentin Rodio, Christian Bastien. Heuristics for Video Games Evaluation: How Players Rate Their Relevance for Different Game Genres According to Their Experience. 25ème conférence francophone sur l’Interaction Homme-Machine, IHM’13, AFIHM, Nov 2013, Bordeaux, France. 10.1145/2534903.2534915. hal-00877287v1 HAL Id: hal-00877287 https://hal.inria.fr/hal-00877287v1 Submitted on 28 Oct 2013 (v1), last revised 6 Nov 2013 (v2) HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Heuristics for Video Games Evaluation: How Players Rate Their Relevance for Different Game Genres According to Their Experience Florentin Rodio J. M. Christian Bastien University of Lorraine University of Lorraine Laboratoire PErSEUs (EA 7312) Laboratoire PErSEUs (EA 7312) 57006 Metz (France) 57006 Metz (France) [email protected] [email protected] Alcatel-Lucent Bell Labs France, Route de Villejust, 91620 Nozay, France [email protected] ABSTRACT player’s expectations, and to avoid wasting their time on This paper reports a study evaluating the relative issues that may not be important. In the next section, we relevance of evaluation heuristics for different game present current inspection methods for video games, as genres and players. 120 players (amateur or e-Sport well as the most common game and player taxonomies. players) were invited to assess the relevance of 47 heuristics for different game genre (Real Time Strategy, Inspection methods for Video Games Massively Multi-player Online Role-Playing or First- There are two main approaches for the evaluation of Person Perspective Shooting). Results show that the games: the empirical and the analytical approach. relevance of heuristics varies according to game genres Empirical evaluation methods are based on user’s and player types. These results have implication for both performances or opinions while the analytical evaluation the design and evaluation processes of games. methods are based on the examination of a product using Implications of these results and future research a set of theories or models [16]. The latter have the directions are discussed. advantage of being performed early in the development process and at a low cost [13]. In the field of video Author Keywords games, these analytical assessment methods are usually Video Games; Methodology; Evaluation; Game genres; performed by the designers themselves or during beta Player typologies; Heuristics. testing. These evaluations however are lacking in rigor ACM Classification Keywords and tend to be accompanied by more structured H.5.2 [Information Interfaces and Presentation]: User inspection methods such as heuristic evaluation [13] or Interfaces – graphical user interfaces, the use of ergonomic criteria [16]. Basically, a set of evaluation/methodology, theory and methods. evaluators inspects the interface with respect to a small set of fairly broad principles, referred to as the INTRODUCTION “heuristics”. In recent years, several sets of heuristics for Increased attention has been dedicated to the scientific video games design and evaluation have been proposed. study of video games given the economic appeal of the They are based on dimensions such as usability [8,14], field. One of these concerns has been the improvement playability [6,8,10] or player experience [7,17]. of evaluation methods, such as heuristics evaluation, a flexible, quick and suitable method for most Game genres development studios. This paper reports on a study To categorize video games, we often refer to the notion aimed at assessing the relevance of heuristics as judged of genre. A genre is a category of games with similar by different users’ types and for different game genres. type of interaction (or “gameplay”). One of the most famous classifications is given by Callois [3], who By supplying evidences of heuristics relative relevancies, classifies play into four categories: Agon (competition), we believe that it will help designers focus on important Alea (chance), Mimicry (role playing), ilinx (vertigo). In aspects of the game, according to their game’s genres or this study, we used three subgenres that belong to three widely accepted genres in the video game industry: “strategy game”, “action game” and “role-playing game” (Figure 1). Strategy games involve managing various resources and units to accomplish a given objective and require high decision-making skills. The goals of the strategy games could be either to build a city or to destroy an enemy base. It can be in real time (RTS) or Figure 1. Examples of RTS, FPS and MMORPG game genres, respectively. turn based. Action games are mostly based on players’ according to different player types. For this reason, the dexterity and reflexes. It includes first-person aim of this study was to assess whether heuristics have perspective shooting games (FPS). Role playing games different level of relevance according to (1) game genres are focused on character development and its progression and (2) player types. through a scenario. « Massively Multi-player Online This study differs significantly from classical heuristics Role-Playing Games » (MMORPG) are part of it, and studies on three main points. First, this study does not enable the player to operate in a persistent world. focus on the validation of a particular set of heuristics. As Cockton et al. [4] have stated, we think that Typologies of players Typologies of players are usually based on skill level, fundamental knowledge on inspection methods must be personality or motivation. To differentiate the players in acquired before via controlled experimentation. It is terms of level, we generally refer to casual gamers and especially true in video game inspection, since hardcore gamers. They are usually differentiated in contextual variables such as game genres and player terms of play time, « meta-gaming » involvement or profiles seem to play a key role in game evaluation. motivation. Some hardcore gamers are also called pro- Secondly, this study focuses solely on heuristics gamers or e-sportsmen. These players enjoy competition “relevance”. Usually, to decide which issues should and play video games as high-performance sports. Other receive developer attention first, severity rating is used. player models are based on personality or play However, other criteria have been considered in video motivation. Bateman and Boon [2] identifies four game inspection to refine the prioritization of issues such personality types: Killer, Achiever, Explorer and as genre ratings [12]. Relevance indicator follows this Socializer. Lazarro [11] outlines four main motivational trend but also benefit of a broader use: it can be seen as a factors of players: Hard Fun (personal triumph over generic tool to prioritize heuristics according to external adversity), Easy Fun (Curiosity), Altered State criterion. Finally, this study is based on player judgment (Relaxation / excitement) and People Factor (social instead of usability expert. As other researcher [6,9], we Interaction). Finally, Yee [18] identifies 10 online think that taking into account real gamer experiences gaming motivations that fall into 3 higher-level would improve the validity of game studies. Indeed, categories related to achievement, social, and immersion because video game heuristics are based more on motivations. playability and player experience, confronting target users to those heuristics would be very informative. The RESEARCH QUESTION only likely bias could be to underestimate the relevance When approaching a field as rich as video games, it is of some usability heuristics because players do not fully essential to consider that heuristics’ relevance can differ understand their scopes. according to game genres and player types. Indeed, heuristics guide the design of game elements in a METHODOLOGY specific way, and, it is this particular balancing that Material establishes a game play proper to a specific game genre. To test the aforementioned assumption, a representative From the perspective of existing inspection models, set of heuristics was chosen, as well as a given taxonomy many authors are aware that game genres are an of game genre and player type. The taxonomy used was important factor guiding the choice and the weighting of kept intentionally simple to optimize the statistical power evaluation criteria [10,12,15]. However, none of them of our tests and we did not intend to represent the whole have provided strong and formal evidences that diversity of existing genre or player types. heuristics relevancy differs according to game genres. Similarly, only Desurvire and Wiberg [5] have developed a list of principles specifically for a given player type, the casual player. But, similarly, no author has provided evidence that heuristics relevancy differs Category Heuristics A1. The players