Abstract

Procedural Content Generation means the algorithmic creation of game content with limited or indirect user input. This technique is currently widespread in the game industry. However, its effects when applied to elements that do not engage directly with the player, also known as Game Bits, require more research. This paper focuses on how players experience a game when these Game Bits are procedurally generated, and how this alters their will to continue playing the game. By developing and using a 2D Roguelike game to perform a qualitative study with eight participants, this dissertation shows an indication that procedurally generating Game Bits does not alter how the players experience a game or their desire to replay it.

Keywords: Procedural content generation, PCG, Game Bits, Game User Experience, Roguelike.

Table of Contents

1. Introduction 1

2. Background 2

2.1 Procedural Content Generation 2

2.2 PCG Methods 3

2.3 Layers of Game Content 4

2.4 PCG in games 5

2.5 Roguelikes 6

2.6 User Experience (UX) and Game User Experience 7

2.7 Color and emotion 10

2.8 Related studies 10

3. Problem 12

3.1 Method 13

3.2 Ethical considerations 15

4. The game prototype 16

4.1 Core mechanics and User Interface 16

4.2 Environment 18

4.3 Enemies 19

4.4 Environmental objects 22

5. Results and analysis 23

5.1 Participants and their background 23

5.2 General impression of the game 24

5.3 Enjoyment 25

5.4 Frustration 27

5.5 Replayability 28

5.6 Environment and aesthetic value 30

5.7 Analysis 32

6. Conclusions 34

6.1 Summary 34

6.3 Discussion 37

6.3 Future Work 39

References 40

Appendix A I

Appendix B III

1. Introduction

Procedural Content Generation (PCG) is the creation of game content by algorithmic means, independently of the degree of input introduced by the designer. Its benefits and possible applications are many. PCG can decrease the amount of work needed to produce a vast number of assets, thus decreasing time and cost expenses when developing a game. In addition, it also allows developers to experiment with randomization and produce unexpected results, opening new possibilities in the game industry. Interactive narrative and dungeon generation are examples of that.

This technology can be applied to several aspects of any game. To be able to analyze the possible effects PCG has on these elements, this dissertation uses the hierarchy of layers of game content proposed by Hendrikx et al (2013). Two of these categories are of most importance to this study: Game Bits and Game Space. Game Bits is the layer that contains all the elements of a game that do not have any active effect on gameplay, like textures and sounds, and Game Space represents the different virtual spaces in which any game takes place. The reason why these layers are focused on is that they are the most common target for PCG, especially in some game genres.

One of those genres is Roguelikes. These types of games are characterized by their lack of progression, resetting the game completely after every run, regardless of the player either succeeding or failing. To be able to offer some replayability, these games feature implementation of PCG in their Game Space layer, meaning their levels, usually represented by dungeon floors, are completely generated from scratch by an algorithm every run. In modern adaptations of this genre, the application of PCG has also extended to the Game Bits layer, offering some variance in the background or environmental elements.

Since this technology is becoming a common practice within the game industry, it is interesting to study how its implementation might affect game user experience. This dissertation develops a 2D roguelike game from scratch in order to properly test and analyze the effects PCG has on the players’ levels of enjoyment when implemented in the Game Bits layer, and how it might also affect the replayability of the game.

The outcome of the experiment shows that, when focusing on the Game Bits layer, PCG has no major impact on players’ enjoyment and desire to replay a game. After analyzing the results, possible factors that might have affected the outcome of the experiment are isolated, and a set of suggestions to further improve future implementations of PCG in the Game Bits layer is suggested.

1

2. Background

2.1 Procedural Content Generation Shaker, Togelius and Nelson (2016) define Procedural Content Generation (PCG from now on) as “the algorithmic creation of game content with limited or indirect user input” (p. 1). This implies that a program run by a computer is able to generate something on its own, or with little human input. From their perspective, they limit the content that can be procedurally generated in games to the different elements that compose them; from the different levels and maps to the many textures and objects that can be found in every game. However, they decided to exclude the game engine itself and the different Artificial Intelligences that are often present in games. In a similar way, Summerville et al. (2018) consider PCG to be “the creation of game content through algorithmic means”, which leaves some space to interpret to what degree the algorithm should create the content on its own, or what kind of content it should create. Hendrikx et al. (2013) describe PCG as “the application of computers to generate game content, distinguish interesting instances among the ones generated, and select entertaining instances on behalf of the players” (p. 2). This definition, however, seems to be oriented towards specific methods of PCG that will be described in chapter 2.2: search-based and solver-based generation. Even so, this adds another challenge to PCG when using these methods: the software has to be able to not only generate content but also judge the instances created using pre-established values and choosing the most suitable ones.

Without a method to supervise the possible outcome, the results of PCG can be unpredictable and not always as well perceived as authored content might be. So why risk the quality of the resulting product to have a program generate content for the designers? There are several reasons for that. For instance, the economic benefit of not requiring an artist to produce ​ every piece or component that is featured in the game. Whether the algorithm is producing most of the graphical and design content or only part of it, this can reduce costs considerably. Directly linked to this effect, PCG also helps to reduce the development ​ time. On the other side of the spectrum, PCG could also be used as a tool to increase ​ ​ artists’ creativity, allowing even smaller development studios to generate a large amount ​ of content by skipping a part of the design process. PCG is not only limited to generating graphical assets. When paired with level and environment generation it can potentially boost a game’s replayability. It also solves possible memory constraints, as the ​ ​ ​ different assets that would occupy that memory space could be created on the fly. Furthermore, PCG opens new possibilities as games could generate content during run ​ time that adapts and tailors itself around the player’s tastes and behavior, by using player modeling and neural networks (Khaled, Nelson and Barr, 2013; Togelius et al., 2013; Shaker, Togelius, and Nelson, 2016).

Naturally, PCG not only brings advantages to the development process and the product. Implementing it can carry a set of challenges and disadvantages that must be considered before continuing with the process. One possible disadvantage is the evaluation of the PCG process. In order to make the generation system more accessible and malleable, additional control systems can be implemented in the tool. However, this can make the interface ​

2

overly complicated and hard to work with, augmenting the development time and ​ undoing part of what was considered an advantage in the paragraph above. One way to counter this negative effect is by applying generate-and-test methods (Frade, de Vega, and Cotta, 2012). In a similar way, when both authored and procedurally generated content have to be implemented in the same space there are some issues that have to be solved, such as properly balancing user control and automatic model consistency maintenance and integrating both authored content and PCG in the same iterative process (Smelik et al., 2010).

2.2 PCG Methods This chapter will briefly describe the main general procedure to implement procedurally generated content in a game. Since it is not the objective of this dissertation to analyze or categorize the different methods that can be used and especially considering the vast number of options that are available, this description will be kept concise and general, and focus on the two basic ways to integrate PCG from which many other methods derive.

Constructive generation methods are a family of algorithms that produce just a single instance of the chosen content each time they are executed. Due to their relative simplicity, constructive generation methods are usually very fast processes, some of them being able to produce content at runtime. On the other side of the coin, without proper fine-tuning these methods offer little control over the overall process, limiting the designer’s input and generating inconsistent content, i.e. an unplayable level (Shaker et al., 2016). Procedural generation has the risk of producing content that does not meet the expected quality or even creating broken content. Constructive generation methods produce content once and do not test the output so, in order to avoid producing defective content, these methods usually limit themselves to using only systems of operations that are guaranteed to succeed.

Generate and test methods, like the ones discussed below, add a final layer to the ​ process by testing the generated output and discarding it partially or entirely, repeating the whole process if it does not meet the previously established requirements (Togelius et al., 2011).

Search-based methods are a special case of the generate-and-test family for one specific reason: the testing process does not operate on the absolutes of valid or not valid, but grades the output through an evaluation function by following a preset vector. This means that the results with the lowest rating will be discarded and the ones with the highest value will be used to generate new candidates with even higher scores. (Togelius et al., 2011). These approaches allow the system to break away or limit the need for previously designed rules, thus reducing development costs and allowing the creation of new content that does not need to adhere to a strict set of rules, unlike purely constructive methods (Risi et al., 2015).

3

“In search-based procedural content generation, an evolutionary algorithm or some ​ ​ other stochastic search/optimisation algorithm is used to search for content with the desired qualities.”

(Shaker et al., 2016, p. 17).

Solver-based methods are another generate-and-test system that uses machine learning models (i.e. neural networks, probabilistic models, decision trees, etc.) in order to evaluate the output of the PCG. However, the content is still generated the same way as with search-based methods. This method can produce a wide range of content such as levels, maps, characters, and events, among others (Summerville et al., 2018). Answer Set Programming (ASP for short) is a solver-based method that uses the AnsProlog language to generate and validate the content that it produces. AnsProlog programs return the variables that do not contradict the predetermined validity requirements, meaning that, in PCG, every answer given by ASP can be considered an artifact (Togelius, Justinussen, and Hartzen, 2012).

2.3 Layers of Game Content To fully understand how PCG can potentially transform a game, it is important to dissect and categorize the different parts that make the building blocks of any game. In their paper, Hendrikx et al. (2013) introduce a detailed taxonomy of the different components that any game is composed of and how PCG can alter them. They define these main components in six categories and classify different aspects under them. These areas are game bits, game space, game systems, game scenarios, game design, and derived content (see figure 1).

Figure 1 - Layers of game content. Based on Hendrikx et al. (2013)

4

Game Bits are the lowest layer of this six-tiered hierarchy. Following the framework proposed by Hendrikx et al. (2013), they are any basic component featured in a game, whether it is concrete or abstract, that does not engage directly with the player. In this group are included textures, sounds, and various environmental objects, as well as their behavior when the player acts upon them. Any graphical variance of any object in the game would be classified as a texture and thus fall in this category.

The Game Space is where the game takes place and where the player, as well as other game ​ bits, are placed and interact. This layer includes any virtual space where the player can interact with other objects, whether it is a dungeon, a platforming level, or an overworld map. It is important to note that only the design of these spaces is included in this stratum, as any visual representation falls within the Game Bits layer.

Game Systems are complex algorithms that control underlying procedures in the game such as ecosystems, traffic simulations, NPC behavior on a large scale, etc. Not every game includes this layer as not every game needs such complex systems.

Game Scenarios include more abstract concepts like the game’s narrative, its different puzzles, and the level concepts. This layer contains all the context for what takes place in the game and, even if a game does not feature a strong narrative, it is present in every game to some degree.

Following Hendrikx et al. (2013) hierarchy, Game Design includes all the rules and goals ​ that build the basic design of the game. The design of all the elements classified under other categories is also included.

Every side-product of the game world falls under the Derived Content category, such as ​ leaderboards, broadcasts, and others. Similarly to the Game Systems layer, this last level of the hierarchy is not always present in every game.

2.4 PCG in games Although integrating PCG in a game was uncommon in the past, since the start of this millennium, games have been featuring this technology increasingly often in different manners. The most common way to implement PCG in a game is through the “Game Space” level, by procedurally generating partially or completely the maps in which the game takes place (Hendrikx et al., 2013). The Diablo franchise (2000-2012, Blizzard Entertainment) is an action-focused role-playing game that uses PCG to generate random dungeon layouts, enemy spawn points, and loot while keeping the narrative and the rest of the world outside of these completely authored. In the third game of the saga, there were some chunks of the overworld that were randomly generated to fit and adapt to the rest of the handcrafted map. Another example of PCG being implemented in this layer is the extremely successful Minecraft (2011, Mojang Studios). This sandbox indie game completely generates the world the action takes place in and the placement of every element at run time, following a random seed number that determines the generation settings.

5

Another common approach to PCG in games is to generate or randomize the appearance and in-game effects of smaller components, such as weapons or pieces of terrain, thus altering the Game Bits level (Hendrikx et al., 2013). Borderlands (2009-2019, Gearbox Software), is a first-person shooter with RPG elements that uses procedural content generation to create a vast array of weapons with randomized components, effects, and stats, that still adapt to the context of the level in order to avoid feeling overpowered. Not a game on its own, but used in some popular games such as Grand Theft Auto IV (2008, RockStar North) and Batman: Arkham Asylum (2009, Rocksteady Studios), among others, SpeedTree (2002, Interactive Data Visualization, Inc) is a middleware used to quickly generate trees and reducing the workload for artists and designers (Van Der Linden, Lopes, and Bidarra, 2013).

Concerning the narrative aspect of games, there are modern approaches that offer an alternative to traditional linear storytelling, usually denominated Interactive Storytelling (IS) or Interactive Digital Narrative (IDN), and implement procedural generation to adapt and modify the story to the player’s actions (Koenitz, 2015). Considering the content structure of Hendrikx et al. (2013), this kind of game applies PCG on the “Game Scenario” level. Façade (2005, Procedural Arts) is a narrative-driven indie game in which players influence the events that occur by talking, typing directly on the keyboard, to the two NPCs that have the leading role in the game. In this case, PCG is implemented to alter the plot and adapt it to the player’s choices and the current tension of the narrative.

IDN is not the only example of a genre being originated by the use of PCG. Rogue (1980, Artificial Intelligence Design) is a turn-based dungeon crawler with RPG elements and a permanent death system that uses PCG to generate every level of the dungeon the adventure takes place in. The combination of procedurally generated dungeons with the lack of player progression made every new adventure different, potentially boosting the game’s replayability, and its success originated a new genre of video games.

2.5 Roguelikes While Rogue (1980) was not the first game to include procedurally generated levels, the addition of RPG elements such as weapons and magic scrolls, combined with the by then new permanent death system (nowadays known as “permadeath”), spawned a series of games inspired by said mechanics. This created the “roguelike” genre, which name obviously references the aforementioned video game. Whereas there have been countless adaptations that have put a twist on the traditional formula, the basic mechanics that define the genre are the same that made the original game popular: procedural environment and level generation, and a permadeath system that resets all the progress at the end of a play session (Ho, Tomitsch, and Bednarz, 2016). Later on, a sub-genre called “rogue-lite” appeared, in which games tried to lessen the drastic effects of permadeath with a progression system that made the adventure potentially easier as the player continued to replay the game.

Within this genre, many games have mimicked the original formula by having dungeon-like procedurally generated levels, fantasy elements, and turn-based combat. The very long list of examples feature games such as Moria (1983, Koeneke), Pokémon: Mystery Dungeon

6

(2005-2020, Spike Chunsoft), and Dwarf Fortress (2006, Bay 12 Games). Additionally, there is also an extensive list of hybrid games that have tried to put a twist on the old formula by altering the main mechanics while keeping those that are key for the genre (PCG and permadeath). Spelunky (2008, Mossmouth) combined the action platforming genre with roguelike mechanics to make an extremely challenging but widely popular game. This game featured procedurally generated maps, and item and enemy locations, as well as some light PCG in the Game Bits layer. Slay the Spire (2017, MegaCrit) integrated permadeath and a procedurally generated map with a deck-building game. The events and encounters that happen during each run are also procedurally generated. Many other games have adapted the traditional formula to cater more modern audiences, changing from turn-based combat to an action-oriented one, using other settings besides fantasy (i.e. sci-fi or mythical) and adding a large amount of extra content such as alternate characters, items (weapons, trinkets, skills, etc), and secret optional levels in order to make the experience feel even more diverse (i.e. The Binding of Isaac (2011, McMillen), Enter the Gungeon (2016, Dodge Roll), and Hades (2018, Supergiant Games)).

The booming popularity of the Roguelike genre in the indie industry is causing new developers to try and reinvent the formula by adding new takes on the original recipe. Also, as technology advances and game engines and general knowledge improve, new and more impressive implementations of PCG can be seen in the industry. For example, Noita (2019, Nolla Games) not only completely generates its dungeons as every other Roguelike does, but also integrates this with the physical simulation of every pixel, generating and simulating flowing liquids and other effects like fire and smoke at runtime, allowing players to interact with these elements. As new technologies and ways to implement PCG are invented, it becomes increasingly important to determine how it can possibly affect the player’s experience.

2.6 User Experience (UX) and Game User Experience User experience (UX from now on) can be defined in many different ways, depending on the source. This lack of a standard concept for UX is due to many reasons, such as the broad and often abstract fields it explores, ranging from emotional to aesthetic variables, some of them sometimes left out and omitted depending on the researcher’s interests and points of view. Its multidisciplinary aspect also complicates matters, since different academic areas attribute different notions to its many facets (Law et al., 2009). The ISO 9241, focused on Ergonomics of human-system interaction, offers a standardized definition for UX describing it as the “user’s perceptions and responses that result from the use and/or anticipated use of a system, product or service”, where the user’s perceptions and responses are further explained as “the users’ emotions, beliefs, preferences, perceptions, comfort, behaviors, and accomplishments that occur before, during and after use” (ISO 9241-11:2018, 3.2.3). Even with these definitions, UX is still a heterogeneous field where different disciplines apply their own concepts and interpretations, hence the importance of using a specific framework in order to be able to work with focus on the subject.

7

Schønau-Fog and Bjørner (2012) propose a method with the objective to classify the experience of engagement in video games. Engagement is a vague concept that includes many different aspects of UX, like flow, immersion, and fun. Within engagement, Schønau-Fog et al. (2012) place their focus on the player’s inclination to continue playing.

“By evaluating the desire to continue aspect of the player experience, we can determine the most fundamental requirement of a successful player experience, as it is can be argued that it is not possible to immerse or involve oneself, experience the flow state, feel incorporated in the game world or have fun without wanting to keep playing.”

(Schønau-Fog et al., 2012, pp. 405-406)

They divide this experience into six engagement clauses that are not necessarily independent from each other: intellectual, physical, sensory, social, narrative, and emotional.

When analyzing the core elements of the gaming experience, Calvillo-Gámez, Cairns, and Cox (2010) separate them into two categories: the game itself, and what they define as the “puppetry”. The game is needed in order to have the experience at all, and it is perceived through two different elements: gameplay and environment. The gameplay represents the core of the game, its rules and main mechanics, and the environment is the aesthetic part, the graphics and sounds the game outputs to the player. In the hierarchy proposed by Hendrikx et al. (2013), the environment would represent the bottom layer, Game Bits, while gameplay would be represented by the rest of the levels. The puppetry is explained by Calvillo-Gámez et al. (2010) as the interaction of the player with the game, and it is altered by three conditions: control, ownership, and facilitators. The first of these three conditions represents the players’ ability to use the options they are given and manipulate the game as they will. Ownership is the product of the player taking responsibility for their actions, as they are an output of their own conscious will. Lastly, facilitators are external factors that alter the player’s experience, such as their previous experiences with other games or their own aesthetic bias. Both control and facilitators influence the feeling of ownership that, in conjunction with the video-game, generates the feeling of enjoyment.

From this categorization of the different elements that compose the gaming experience, they propose a model to analyze player experience, the CEGE model (see Appendix A) and accompany it with a questionnaire based on said theory. This model relies on observable and latent variables in order to measure how the player perceives the game and the “puppetry” presented by it (see figure 2).

8

Figure 2 - The CEGE model. The figure depicts the relationships among observable (text next to brackets) and latent (circles) variables. Based on Calvillo-Gámez et al. (2010)

The CEGE questionnaire consists of 38 items, all of them are 7-point Likert scales. These items are focused on measuring the observable variables, i.e. the graphics and aesthetic values, in order to understand the latent constructs, such as the environment and the facilitators (see figure 2 and table 1). The questionnaire has to be filled after the player has finished playing the game.

Table 1 - Variable focus by item. Observable variables are considered as items in the questionnaire context and latent variables as scales. Based on Calvillo-Gámez et al. (2010)

Items Scale 1 Scale 2 1, 4, 5 Enjoyment - 2, 3 Frustration - 6-38 CEGE - 6-12, 38 Control 13-18 Facilitators Puppetry 19-24 Ownership 25 Control/Ownership 26-31 Environment Video Game 32-37 Gameplay

9

This framework has been used in multiple pieces of research as a means to measure player enjoyment and perception of the core elements of a game. Schönauer et al. (2011), besides designing their own system to capture body motion and measure muscular activity, developed and tested a serious game aimed at aiding with the rehabilitation of patients with chronic pain of the lower back and neck. In their evaluation, they used the CEGE questionnaire to measure the patients’ enjoyment and frustration levels. Iacovides, I. et al. (2015) used the CEGE model to test how much removing the HUD in a shooter game would affect the player’s experience, and whether that turned the game unplayable or not.

2.7 Color and emotion It is widely accepted that video games, like any other media or art form, are capable of generating or triggering emotions in their consumers. There are many factors that facilitate this reaction, such as narrative elements and powerful visual moments. Color is another factor to consider as it also has an impact on emotions. Joosten, Lankveld, and Spronck (2010) researched color’s effect on the players’ emotional responses and concluded that the use of red in a relatively complex video game evoked a highly-aroused, negative emotional response, and yellow provoked a positive emotional response. Gao et al. (2007) found that the meaning of a color is influenced in its majority by the brightness and chroma, and to a lesser degree by its hue. Geslin, Jégou, and Beaudoin (2016) found an existing link between emotions such as joy or sadness and environment color properties like brightness, value, saturation, chroma, and lightness. Brighter, more saturated, and colorful imagery, as they found, increased this perception of joy.

2.8 Related studies In their paper, Korn et al (2017) performed a similar study to the one proposed in this dissertation. Their aim was to show the potential effects of PCG when applied to specific graphic props in an explorer game. In their comparative study, they tested two different approaches to the creation of a coral reef prop in the game. One version would be entirely crafted by an artist, and the other would use PCG to generate the reef. In the game, players had to maneuver around islands and coral reefs in order to reach an advantageous position. They specifically targeted the coral reefs because their high visual diversity made them a good PCG candidate, while still being a central part of the gameplay. For the PCG version, they procedurally generated the reefs by using a Perlin noise method, which would fall into the generate-and-test family of PCG. Their methodology proceeded as follows: they started by gathering personal and background data from their participants, followed by having them play both versions of the game. The version with which they started the play session was randomized. After they played both versions, they used a questionnaire to measure the players’ experience and to determine which version was favored by the participants. Their results showed that users perceived the PCG version as both more realistic and more visually

10

pleasing, as well as favoring the flexibility of the changing environment when implemented with PCG as opposed to the limited amount of options the traditionally designed version had.

These results are very interesting, considering the possible implications they might carry for the game development industry. Were they the general case, it would mean PCG would not only be beneficial from a cost and time investment perspective but also from a purely aesthetical point of view, being the preferred method to implement graphic material. That being said, there are other studies outside of the gaming area of focus that have also compared computer-generated against manmade art. Such is the case of the study proposed by Chamberlain et al. (2018), in which they observed the perception people had towards visual art created either by humans or artificial intelligence. Their results deferred from the ones of the previous research above, as they showed observers were influenced not only by the degree a piece of artwork looked “too perfect” or mechanical but also pre-conceived negative notions about computer-generated content. This dichotomy between different cases displays the need for more research focused on this subject.

11

3. Problem

As seen in chapter 2.4, PCG is a widespread technique in modern video games. It can be applied to almost every layer of game content (Hendrikx et al., 2013), to the point of being almost a requisite in some video game genres, like Roguelikes (chapter 2.5).

It is also important to note that there are many reasons to implement PCG to some degree in the development process (chapter 2.1), the main reason being it accelerates the creative process, cheapening the development process, and sometimes enables for new possibilities. In the case of the Roguelike genre, for example, the procedural generation of the dungeons allows the game to offer an, ideally, endless gameplay loop, boosting the game’s replayability and player engagement. On the other hand, a badly implemented PCG system can negatively affect the development process, making the interface overly complicated or presenting generation problems that take time to fix. In a similar fashion, badly implemented PCG can have a negative effect on the players’ experience. These unwanted side-effects might vary depending on the layer in which the PCG is implemented. Bad PCG in the Game Space layer might result in confusing map layouts or even produce broken or unsolvable scenes, while in the Game Bits layer it might result in strange and alien visuals, glitches in sound or unenjoyable music, although this layer is much more susceptible to subjectivity than the rest. Striking a balance between efficiency and efficacy, by choosing a proper method, and properly testing are, in consequence, utterly important when using PCG to develop a game.

Although PCG is a very popular research area, most of the papers place their attention on the varied methods to implement such technology (i.e. the study of various methods to generate dungeons by Van Der Linden et al. (2013), or the proposal of a data driven method aimed at - generating educational content by Hooshyar et al. (2018)). The research of the possible effects this technique has on the player’s experience is rarer. The study performed by Korn et al. (2017) shows that PCG has a positive impact on players’ experience. However, in order to validate this perspective, more studies with a similar approach need to be conducted. Furthermore, Korn et al’s (2017) study focuses on changing a single element of a game. Whether their results can be extrapolated to more aggressive or different implementations of PCG in the Game Bits layer of a game is uncertain.

In roguelikes, PCG is used in order to potentially enhance the player experience and boost the replayability aspect by generating every level and randomizing the placement of enemy spawns and the different power-ups the player is going to need in order to advance. Most of these efforts focus on the “Game Space” layer, as the generation of the dungeons is the flagship of this genre. Representation of PCG in the Game Bits layer in these cases is commonly reduced to increasing the visual variety of the different elements that populate the dungeons, usually decorative elements. Considering the aforementioned benefits of PCG and how it could affect the player’s perception of the game, the author of this paper proposes the next research questions:

➔ How do procedurally generated game bits affect the player’s experience and desire to continue playing? ➔ In what way does procedurally generating game bits alter a player’s engagement?

12

Considering the increasing popularity of Roguelikes and the widespread use of PCG in other genres, and the results shown in Korn et al’s (2017) research, the main hypothesis is that the presence of PCG in the Game Bits layer should increase player’s enjoyment and, thus, also increase their desire to replay the game.

3.1 Method This study analyzes how implementing PCG in the Game Bits layer (the aesthetic part of a game) affects the player’s engagement, and how it might affect the game’s potential replayability with the use of a qualitative study. However, in order to gather the data, mixed methods were implemented.

To properly evaluate the differences in player experience, a game that could toggle PCG implementation in its Game Bits layer was necessary. Since no such game could be found or accessed by the time the study was designed, a new one was developed for this specific purpose. Due to heavy time restrictions and also to cater to the team’s strengths, the game was designed to be a 2D action Roguelike. Developing the game in 2D would simplify the process and shorten its development time, but it would likewise reduce the available methods to apply PCG to it, especially considering PCG was to be applied to every possible visual aspect within the limitations of the Game Bits layer. Sound and music are also included in this layer. Nevertheless, the complexity of procedurally generating sound and music accompanied by the lack of experience in this matter of the members of the team made applying PCG to these elements an impossibility. Despite that, since the different parts of the study were to be performed online through voice chat and the players were expected to talk over the sound of the game, this aspect (sound) was considered less crucial than the visual aspects. The Roguelike genre was chosen because of its intrinsic relationship with PCG. As described in chapter 2.5, Roguelike games already feature PCG in the Game Space layer, meaning its dungeons or levels are entirely procedurally generated. In addition, modern representations of this type of game also feature PCG in some elements of the Game Bits layer. Choosing this genre would presumably allow this study to experiment with the different Game Bits elements, without making the resulting product too off-putting for the experienced players. In a similar fashion, the game was designed to mimic as much as possible the qualities of a commercial game in order to appeal to players that were already familiar with Roguelikes.

Finally, the game would have two playable versions:

➔ One without any variance due to PCG in the Game Bits layer. ➔ An alternative version with PCG applied to the Game Bits layer, specifically all textures and models but the player character. Chapter 4 further explains how this prototype was designed and how PCG was implemented in it.

Subjects were chosen through a mix of convenience and snowballing sampling. Three candidates were contacted directly by the researchers and then instructed to invite any acquaintances of theirs that might be interested in participating. In total, eight subjects participated in the experiment. They were all within the age range of young adults, with

13

different backgrounds and nationalities. Four of them were used to playing video games on a daily basis and were familiar to some degree with the Roguelike genre, while the rest of them were more casual gamers or non-gamers.

Volunteers gave their informed consent (see chapter 3.2) and were informed about the procedure before starting the experiment. In order to be able to have the experiment online, participants were asked to share their screen with the researchers through Discord (2015, Discord Inc.) and they were reminded once more that their voice and screen would be recorded for the purpose of the experiment. During the testing session, the procedure was explained once more to make sure they understood how to operate. The test worked as follows:

➔ Participants were instructed to start with the tutorial section, where they would learn the controls and how to advance in the game. ➔ After that, they continued with one of the versions of the game. Half of the participants started by playing version A, which did not have any PCG implemented in the Game Bits layer, while the other half started with version B, which had relatively heavy PCG in the Game Bits layer. ➔ They were asked to share their thoughts and opinions as much as possible while they were playing and reminded that they were free to stop playing at any moment or continue until a set time of ten minutes was over. ​ ​ ➔ Following, they were asked to complete the CEGE questionnaire (Calvillo-Gámez et al. , 2010)(see Appendix A for reference). ➔ Then they were instructed to repeat the same process with the other version of the game and complete the same questionnaire. ➔ Both questionnaires were used to facilitate pinpointing players’ experiences in case they were not communicative enough during the playthrough, so related questions could be asked during the post-game interview. ➔ Finally, the researchers held a semi-structured interview with the participants, where they were questioned about matters referring to what was observed during the playtime and in their questionnaire answers, besides other predetermined questions aimed at answering the research questions (see the interview question guide in Appendix B). ➔ The whole procedure took an average of 60 minutes.

During the testing session, multiple types of data were gathered:

➔ The whole playing session was observed by both researchers who took notes regarding players’ behavior and remarks during the game. ➔ Screen and voice were recorded for later reference. ➔ After playing each version, players were asked to complete the CEGE questionnaire (see chapter 2.6 and Appendix A for reference). This data was used as an alternative way of finding interesting differences in the players’ experience, to complement the notes and recordings. Special focus was put on the items concerning enjoyment, frustration, visual environment, and aesthetic values, as these were the items that were expected to present a wider range of variation.

14

➔ Finally, a semi-structured interview was held and participants were asked about their gaming background (i.e. weekly game time average, familiarity with roguelikes, etc), subjects concerning their general impressions of the game, their desire to continue playing, preference of one version, and other data related to the notes gathered during their play session and their questionnaire results (see Appendix B for reference).

3.2 Ethical considerations Each volunteer gave their informed consent after a brief explanation about how the experiment would work. This explanation included a brief description of the game, a detailed description of every step of the procedure, and what would be requested of them during the test, such as recording their screens and voices, completing questionnaires, and having a post-game interview. Before consenting, they were also informed about how their data would be handled. Their anonymity would be respected at all times and no personal information would be requested besides what was strictly necessary for the test, which was exclusively their gaming background information. Data gathered for this dissertation would be safely stored to avoid violating the volunteers’ anonymity. The data would be stored until it is no longer needed for analysis purposes and will be destroyed afterward.

The game presented depictions of cartoony violence. The violence was kept to a bare minimum and no blood or gore was represented in the game. The player character carries a knife and uses it to defend itself from the attacking enemies, that use bottles, planks, and blunderbusses to attack. This minimum level of violence was considered necessary as it is a common trait in any modern combat-centered Roguelike, as producing a game that felt as close to a commercial product as possible was the objective.

Because of circumstances due to the COVID-19 global pandemic, the safety of volunteers and researchers alike was guaranteed by performing the tests online.

15

4. The game prototype

This chapter explains in detail how the game artifact used in this paper was designed and how PCG was implemented into its different components.

The prototype is an action roguelike game specifically designed for this thesis, with real-time gameplay and procedurally generated dungeons. Players that are already used to playing roguelike games will presumably be familiar with some degree of PCG since it is deeply ingrained in this genre’s typical mechanics.

To compare the results, two versions of the said game were produced, one presenting a heavy use of PCG on the “Game Bits” layer, and one without any PCG on that level. The PCG method used for this layer would fall within the constructive method family (chapter 2.2). The implementation of PCG on the “Game Space” level (the layout of the dungeon) is constant on both games since this is a staple of the Roguelike genre.

The game features a cartoony graphic style that makes its environment and the actions taking place more eye-friendly. It still depicts cartoon violence since the game’s core mechanic is based on combat with weapons, but there is no blood or unnecessary gore in the totality of the game.

4.1 Core mechanics and User Interface The prototype features a set of simple game mechanics and an equally straightforward scenario. The player is represented by a pirate that is stuck on an island, and their objective is to progress through a maze-like dungeon in order to reach the end. Rooms that have enemies in them will lockdown until the player clears the room. To move between levels, players have to find a specific room of the dungeon where they are presented with a final challenge in the shape of a difficult fight. After completing the challenge, players automatically proceed to the next level.

In order to succeed, players have a limited set of actions available to them. They can either move, attack, or block incoming melee attacks. While blocking, players can move slowlier than usual, but will only successfully block incoming attacks when facing the attacking enemy. These actions can be performed by using either the keyboard or a gamepad controller (see table 2 for specific controls).

Table 2 - Game controls

Action Keyboard input Controller input

Move W, A, S, D keys Left stick

Attack Spacebar A button

Block Left shift X button

16

The game features a simple user interface that plainly shows the player how many hit points (HP) they have left, as well as a minimap that shows the explored layout of the current level, as well as unexplored rooms that connect to explored areas, represented with a question mark (see figure 3 for visual reference). A simple tutorial level was also created in order to teach players the basic controls without showing too much of the actual game (figure 4), as well as an easy-to-navigate main menu (figure 5) and a loading screen. The main menu provided an easy way to navigate between game modes such as the tutorial and both game versions.

Figure 3 - The minimap presented in the prototype. Explored areas were represented by green squares and unexplored connecting rooms with a question mark.

Figure 4 - Tutorial level. The game controls were taught through animated images in an attempt to cause boredom to players by having them read the instructions.

17

Figure 5 - Main menu. From here players can either leave the game, access the tutorial section, or any of the game versions.

4.2 Environment Two different environments were designed for this prototype. The objective was to add more variance to the Game Bits layer in order to trigger player reactions more easily. Both environments worked fundamentally the same, with the only difference among them being their visual appearance and accompanying background sound. Each environment featured ten different types of rooms. Obstacle items such as rocks and vegetation were kept constant between stages. Visual-wise, the first environment is represented by a jungle, covered with grass and with dense palm tree groves as walls. The doors connecting each room are wooden stockades that block the player’s path once a combat room is activated (see figure 6 for visual reference). The second environment is depicted as a beach area, surrounded by sea. Non-accessible areas are shown as pitfall traps, in an attempt to continue with the vague tribal vibe of the previous stage (see figure 7 for visual reference). Doors connecting different rooms are stepping stones that connect different islands and sink once a combat room is triggered.

Figures 6 and 7 -Default appearance of the jungle and the beach level, respectively.

18

When applying PCG to the environment, only its color palette was changed. An array of alternative color combinations were designed to potentially substitute the look of the standard version of each environment. Considering what was reviewed in chapter 2.7, the color variation might elicit different kinds of emotions on the players, thus potentially altering their game experience. These colors were chosen to be as varied as possible, in order to trigger these reactions on the testers (see figure 8 for visual comparison). Other than that, no other element of the background was changed.

Figure 8 - Visual variations of the background’s colors. Not all variations are included in this figure.

4.3 Enemies For the version of the prototype used in this paper, there are three different kinds of enemies. Each enemy type presented a distinctive appearance and a different behavioral pattern:

● “Matey”: Represented by a drunk pirate stumbling towards the player (see figure 9 ​ for visual reference). These enemies simply move in a straight line towards the player and, once they get close enough to the player, they attack with an empty bottle.

Figure 9 - Default look of the “matey”.

19

● “Blundercroc”: Depicted by an anthropomorphic crocodile wearing either tribal or pirate clothing and armed with a blunderbuss. This enemy AI is the most complex of the group since it presents two different stages. If the enemy’s weapon is loaded, the crocodile will move towards the player until they are in shooting range and then proceed to shoot the player. The weapon will then be unloaded, which will trigger the enemy to move away from the player until it is at a safe distance where it can reload the blunderbuss (see figures 10 and 11 for visual reference).

Figure 10 (top) - The “blundercroc” in its default look, aiming at the player. Figure 11 (bottom) - The “blundercroc” is running away to a safe spot to reload.

● “Planker”: This enemy is represented by a large man carrying a plank (see figure 12 ​ for visual reference). Its behavior is very similar to the one of the “matey” type of enemy, with the only exception that the “planker” is more resilient and does not get interrupted by attacks until it receives enough damage.

Figure 12 - Default look of the “planker”, ready to hit the player.

20

All the default enemy sprites featured a shade of red in their clothing, as it is a common practice to give enemies a bright color in order for players to identify them more easily.

Based on their intended challenge level, ordered from easier to more difficult in this section, “mateys” are the most predominant enemy in both environments, while “blundercrocs” are rarer in the first level and appear more in the second one. “Plankers” are spawned very rarely and only on the second level. This was designed in order to create a smoother learning curve for the players.

Each one of these types of enemies was modularly designed and built, meaning that their different body parts had alternative designs that could be swapped with the originals. When integrating PCG in the enemy sprites, these modular parts are randomized when generating the dungeon, producing a vast array of slightly differently looking enemies. Similarly to the environment, enemies also have a set of predetermined color palettes that are randomized at the start of each run. However, in order to avoid creating confusion in the player with different looking enemies that have also very different color palettes, the generated color palette was shared among the same enemy type on the same floor (see figure 13).

Figure 13 - Some of the possible variations of the previously described enemies.

21

4.4 Environmental objects Plants and rocks were also added to the game to further increase the number of elements in the Game Bits layer that were susceptible to PCG. These elements were designed in a likewise fashion to the enemies, being composed of three different modular layers. In their standard non-PCG version, three different looking plants and two different rocks were chosen from the possible combinations (see figure 14).

Figure 14 - The standard hand-picked look for the environmental objects that were shown in the non-PCG version.

When applying PCG to these elements, their modular parts were randomized, as well as their color palette swapped (see figure 15).

Figure 15 - Some of the possible variations that could be generated in the PCG versions.

22

5. Results and analysis

In this chapter, the data gathered during the tests is compiled, summarized, and analyzed. For clarity’s sake, each following sub-chapter goes over a specific chunk of data, analyzing each participant’s input on a specific matter before synthesizing all of their answers into a comprehensible summary. The charts at the end of chapters 5.2, 5.3, 5.4, and 5.5 are simplified representations of the data gathered through the CEGE questionnaire. These charts are just a graphical representation to help clarify the results. For the analysis, only the real values, and not the average, were considered.

5.1 Participants and their background As stated in chapter 3.1, the test was performed with eight participants. The order in which they played the different versions of the prototype was swapped for every other player, to discard any dependencies between the effects experienced in each version. At the end of the semi-structured interview, a quick background check was performed to measure players’ experience with video games in general, and with the Roguelike genre in particular. The results of this background check can be seen in table 3.

Table 3 - Participants’ background.

Weekly Familiarity Experience Participant average with with playtime Roguelikes Roguelikes #1 5 h/week Yes Limited Very #2 10-15 h/week Yes experienced #3 < 1 h/week No None #4 < 1 h/week No None #5 12 h/week Yes None #6 < 2 h/week No None #7 5 h/week Yes Limited #8 < 1 h/week No None

23

5.2 General impression of the game In the post-game interview, participants were asked about their thoughts on the game as a whole before going in-depth later with more specific questions.

Participant #1: Liked the game, although would have liked it more if it had another ​ gameplay element besides combat, like collecting and building. The combination of art style, how the combat worked and the camera perspective reminded him of the game Don’t Starve (Klei entertainment, 2013).

Participant #2: Generally liked the game and its feel. They thought the game was fun and ​ liked how everything fit well together with comments like “The graphics and music were nice, they really fit well together with the game”.

Participant #3: Thought the game was simple and fun and not too complicated even for ​ someone that does not play often.

Participant #4: Liked the game. With statements like “It was quite intuitive what you had ​ to do, just survive and kill the bad guys” they expressed how simple and easy to follow they found the game. However, they felt that there was not any direction or end game objective, for example by saying “There was little story background, in that sense”.

Participant #5: They do not usually enjoy that type of gameplay loop. They liked how the ​ game controls worked and described them as “fluid” and “fast”. They also enjoyed the pacing and difficulty progression. However, they also felt the game needed a story and more content and stated “To me, it felt like it was half a game”. They did not think the game needed a tutorial.

Participant #6: They described the game as “nice” and “entertaining”. They also felt like ​ the game had a retro feel to it.

“The game was like [...] one of these back-in-the-days games, I would say”.

Participant #7: They liked the style of the game and its theme, and commented: “They ​ (Roguelikes) don’t usually have that pirate theme with this happy pirate-y style, so that I liked”. However, they felt the difficulty curve was too steep and that made the experience somewhat frustrating.

“The challenge was increased considerably from the first level to the second one [...] so that made it a little frustrating, maybe”.

Participant #8: They thought the game was generally fun, even if they are not used to ​ playing this kind of game.

“It was fun playing it. I mean, I’m not a gamer but I think [...] it was amusing”.

The general impression seems to be a positive one. People generally liked the game and its core mechanics, although some found it lacking in other departments, like missing a story or an explanation that gave some context to the game. Regarding its difficulty curve, most

24

seemed to agree that it was too steep, although some exceptions thought otherwise (i.e. participants #2 and #5). This difference in opinion might be related to the players’ experience with similar types of games.

5.3 Enjoyment To test the participants’ enjoyment of the game, notes were taken during their playtime which was also recorded for later reference, if necessary. Participants were asked about their enjoyment with the game as a whole and regarding particular versions, and if they had any preference of one over the other. The questions were also motivated by items 1, 4, and 5 of the CEGE questionnaire (see Appendix A).

Participant #1: In the questionnaire, they rated the PCG version lower when compared to ​ the non-PCG version. When asked, they retracted and explained they felt like they did not clearly prefer a version over the other.

Participant #2: They enjoyed the non-PCG version better, but not because of the lack of ​ procedural content. Although they felt the layout of the rooms was more confusing, they also felt this version moved faster, which made them enjoy it more. They did not really notice the changes in the Game Bits layer.

Participant #3: Showed a clear enjoyment of the game in the questionnaire, giving the ​ highest scores in both versions. When asked, they explained they did not have a clear favorite. They liked one of the alternative aesthetic versions for the jungle environment but it was only because the predominant color was their favorite, and they did not feel that affected the gameplay or their enjoyment in general.

Participant #4: During the test, they found it surprising when colors changed in the PCG ​ version. They did not like a specific color variation of the jungle environment but did not feel like that ruined their immersion or enjoyment of the game. They found this change in aesthetics somewhat confusing, so they would prefer the non-PCG version because of its consistency.

Participant #5: Gave both versions a medium score in the enjoyment-related items. They ​ enjoyed the PCG version more because it felt faster. They did not value the changes in the Game Bits layer as a huge factor, but the variety they introduced was appreciated.

Participant #6: During their playtime, they showed signs of enjoyment. In the ​ questionnaire, both versions got a high score in the related items.

Participant #7: Following the questionnaire, they felt less bored in the PCG version but ​ still felt like they enjoyed the non-PCG version more. When asked about this discrepancy during the interview, they attributed it to their level of engagement rising since it was the second version they tried, and to there being more progression in the environment. However, they still preferred the non-PCG version because the environments felt more natural and fitting.

25

Participant #8: When asked about their enjoyment, they gave both versions medium to ​ high scores in the questionnaires rating the non-PCG version noticeably below the other one. In the interview, they stated they liked the non-PCG version less because one of the layouts they came across was confusing and the game felt much more difficult and unfair.

It seems the general enjoyment of the game is relatively high. However, there was not a clear preference between both versions, and when a player favored one it was mostly for reasons not related to the procedural generation of the elements in the Game Bits layer. When it was, it was due to a lack of consistency or some of the alternative environment versions looking unnatural. It is important to note that some of the participants did not really notice the differences between the versions and that might be related to the order in which they played said versions (the three participants that did not notice the differences, or barely noticed them, started by playing the PCG version). Although it is not the focus of this study, it is interesting to note that some players felt the game was moving faster when changing versions. This might be due to the random layouts of the dungeon casually favoring a faster playstyle, but it also might be related to a visual effect or mental trick caused by the color variation of the background.

Figure 16 - Average scale value of enjoyment-related items in the CEGE questionnaire. A higher scale would mean the player perceived more enjoyment.

26

5.4 Frustration Frustration was also a factor taken into consideration. When the elements of a game change unexpectedly, this can lead to confusion and frustration. This factor was measured through the participants’ playtime with notes and recordings, as well as reflected on items 2 and 3 in the CEGE questionnaire (see Appendix A). During the interview, all players were asked about their frustration with the prototype in general and version-specific.

Participant #1: They rated their level of frustration low overall in the questionnaire, but it ​ also showed an increase in frustration the second time they played, which was the PCG version. When asked about it in the interview, they explained the source of this increased frustration was related to the difficulty curve of the game.

“I was starting to get a little bit frustrated that I was just wasting lives and I knew that I had to have them for the end”.

Participant #2: Did not show any variance in their frustration levels between versions, and ​ they were relatively low overall. During their playtime, they did not show any clear signs of frustration, either.

Participant #3: While playing the game they showed some frustration, mostly related to ​ the difficulty level. In both questionnaires, they got an average-to-low score on the frustration items, showing a slight decrease the second time they played, which was the PCG version. They felt their second time playing the game was more difficult, but only because they progressed further in the level.

Participant #4: They showed some minor frustration while playing the game. In the ​ questionnaire, they rated their level of frustration relatively low and it decreased on the second time they played, which was the non-PCG version. When asked about it, they attributed it to them getting used to the mechanics of the game.

Participant #5: Did not show obvious signs of frustration during their playtime and rated the frustration-related items in the questionnaire very low.

Participant #6: Little to no frustration was expressed during the test and in the ​ questionnaires. In the interview, they stated they got slightly frustrated in the non-PCG version because they felt it was more difficult and the dungeon layout was more confusing.

Participant #7: This participant showed signs of frustration during their playtime and that was reflected in the questionnaires, as well. In the interview, they stated this was due to bugs in the prototype and the mastering curve being too steep. Also, they felt the experience was not rewarding enough.

“I didn’t get any rewards for finishing the room or anything, so I didn’t get even one health”.

Participant #8: Showed clear signs of frustration during the test and rated accordingly in the questionnaires with a significant difference between both versions, rating the PCG

27

version, the first they played, considerably lower. They felt annoyed at some bugs that happened during the test and felt the game was unfair and frustrating in consequence.

Some level of frustration was experienced by most of the participants. This was caused by faults in the prototype design like the difficulty curve being too steep for some participants or players experiencing bugs that put them at a disadvantage. These factors caused little to no effect on some participants, which might be related to their experience with similar games. The presence of PCG was never pointed as the cause of such frustration during the post-game interview.

Figure 17 - Average scale value of frustration-related items in the CEGE questionnaire. A higher scale would mean the participant perceived more frustration.

5.5 Replayability Like Schønau-Fog and Bjørner (2012) stated in their research, the will to continue playing is the most fundamental indicator of successful playing experience. In order to measure this factor, the players’ predisposition to continue playing during the test was noted and it is reflected in items 5 and 16 of the CEGE questionnaire. The participants were also asked about their will to continue playing and which version would they choose, if so.

Participant #1: In the CEGE questionnaire they showed a high desire to continue playing ​ the game, especially the non-PCG version. When asked, they showed interest in replaying the game because they wanted to complete the levels. However, in order to dedicate more time to the game, they indicated that some kind of progression system would be desirable. When

28

asked which version they would continue replaying, the answer was the non-PCG version due to its colors, mentioning brightness and chroma (colorfulness, or saturation). This might correlate to Geslin et al. (2016) study of color and emotion, mentioned in chapter 2.7, with brighter and more saturated colors causing positive emotions such as joy.

Participant #2: Did not show a clear desire to continue playing any of the versions in the ​ respective questionnaires. When asked during the interview, the player stated that they would replay the non-PCG version because it felt faster for them.

Participant #3: They showed interest in continuing replaying the game for both versions in ​ the questionnaires. When asked in the interview they reconsidered their answers and expressed some interest in replaying the PCG version but not in a decisive way.

“I don’t feel any difference in the color [...] Maybe I would’ve chosen version B to see if there were more colors”.

Participant #4: Rated both versions of the game very high in all items regarding the game ​ replayability. During the interview, they confirmed their desire to continue playing the game but they did not show a prevalence of one version over the other.

Participant #5: Did not have any desire to replay any of the versions. They stated this style of game was not of their liking, but if more mechanics and a story were added they would give it another try.

Participant #6: In the related questions of the CEGE questionnaire, they showed the desire ​ to continue playing both versions, equally. They corroborated this in the interview and stated that they would choose the non-PCG version because they felt exploration played a bigger role since the rooms were bigger.

Participant #7: Expressed the desire to continue playing during the test, and that was reflected in the questionnaire as well. They expressed a preference for the non-PCG version because the changes in the environment in the other version made them feel like they lost the chance to complete the previous version, even after they acknowledged the dungeon layouts were always changing.

“I would like to play version 1 (non-PCG version) more because I didn’t like that when I died I got to play a new world [...] because I really wanted to finish the first world before I moved on to the next”.

Participant #8: In the questionnaire, they rated the items concerning replayability high on the scale for both versions. In the interview, they stated that they felt like the non-PCG version (the second they tried) was harder and preferred the PCG version because of that.

Similarly to the frustration levels, it seems that some of the reasons why the participants showed interest in continuing playing were not related to the procedural generation of the Game Bits elements. In some cases, it was due to the random generation of the dungeon, which are changes affecting the Game Space layer and not the Game Bits one, making the game feel faster, more difficult, or more open. In others, players’ tastes took part, as they preferred one version over the other simply due to the environment having a more pleasing

29

color for them. An interesting point, however, were the cases where participants preferred the non-PCG version because they felt the changes in the environment needed a context or explanation, or it made them feel like the game was unforgiving for not giving them the chance to beat a certain environment and moving on to the next.

Figure 18 - Average scale value of replayability-related items in the CEGE questionnaire. A higher scale would mean the player perceived a higher desire to continue playing the game.

5.6 Environment and aesthetic value Finally, since all the changes in the Game Bits layer were kept to the visual aspect of it, leaving sound and secondary behavior untouched, it is important to also study how players experienced the visual environment and their aesthetic value of both versions to see how it might have affected the rest of the factors above. Similarly to the rest of the factors, the appreciation of the visual environment and aesthetic value of the game was measured through observation during the test, as well as through items 13, 14, 26, 29, and 30 in the CEGE questionnaire. Players were also asked about this matter during the post-game interview.

Participant #1: The participant enjoyed the environment and the graphic style of the game ​ overall and did not show any prevalence for the environment of a specific version in the questionnaires. In the interview, they explained how the non-PCG version looked more bright and colorful.

30

Participant #2: They rated the aesthetic value of the game and its environment relatively ​ high in both questionnaires and corroborated it during the interview. They did not notice any graphic differences between both versions.

“The visuals were nice and they were fitting to the game. There was nothing that really stood out.”

Participant #3: They enjoyed the visual aspect of the game, as well as the environment. ​ They did not prefer one version over the other at this level.

Participant #4: The player liked the game’s aesthetics and environment and gave a high ​ score to the related items in both questionnaires. They found it surprising when the colors changed, but did not consider that as a negative point.

“In terms of style, or graphics, or music, I enjoyed it both times”.

Participant #5: In the questionnaire, the participant rated the aesthetic value and ​ environment a medium-high score overall. They thought both versions presented an equally appropriate visual aspect and did not have an obvious favorite. The player felt like the PCG version had a lot more attention to detail, especially the “blundercroc” type of enemy.

Participant #6: They gave a medium to high score to the items related to graphic pleasantness and environment, with the non-PCG version rated slightly higher. In the interview, they stated they only slightly noticed the differences between both versions, and they had no clear favorite between them.

Participant #7: During their playtime, the participant expressed their liking for the visual aspect of the game. Found some of the environment variants more unnerving or somewhat disturbing. They stated the non-PCG version looked more natural and they preferred it because of that.

“It looked like the environment could actually have that color (in the non-PCG version). If being set up with a narrative [...] then I would buy it (the PCG version).”

Participant #8: In the questionnaire, they gave a medium to high score to both versions when aesthetic value and environment were related. They did not notice a visual difference between versions, therefore they did not have a clear favorite.

The graphic style of the game was generally liked in both versions. When asked about their favorite version, it seems most players decided based on the environment color alone and the variations in the rest of the elements (such as enemies, vegetation, and rocks) did not hold as much weight in the matter, if any. The non-PCG version was liked by some of the participants because of its colorfulness and brightness, but that might have been caused because the random alternative versions they experienced during their playtime with the PCG version were actually darker, so their opinion might have been different if they had had the chance of experiencing other alternative color palettes. One common observation from the players was that the environment in the non-PCG version seemed more natural and appropriate when compared to the other version, so the choice of making the differences as

31

obvious as possible might have affected the outcome of this study. On the other hand, some participants did not notice any difference between both versions, so more subdued variances might have caused the same effect on more players.

Figure 19 - Average scale value of the visual environment and aesthetic preference related items in the CEGE questionnaire. A higher scale would mean the player perceived the visual aspects of the game to be more pleasant or fitting. For this reason, item 14 (“The graphics of the game were plain”) was not added to the average scale value of this chart since it was not necessarily a negative or positive value.

5.7 Analysis The results show that the implementation of PCG within the Game Bits layer did not change the players’ experience in any major way. Most players enjoyed both versions of the game equally, and when they did not it was not due to the implementation of PCG in the Game Bits layer but the randomness of the dungeon generation or technical and design issues in the game. When asked about their favorite version, some players chose the non-PCG version. While some of these cases were caused by spikes in the difficulty level and other non-related factors, others liked the PCG version less due to the more radical changes in colors in the background. While this is connected to the generation of the background in the Game Bits layer, it did not seem to affect their enjoyment overall, and a more subdued array of alternative color palettes might have caused the opposite effect. When discussing the Game Bits elements, most players seemed to focus on the background and its different color palettes, giving much less weight in the matter to enemy variation and almost no importance

32

to environmental items. This must have been caused by the level of importance of such elements, as something as a palette change in the background is immediately noticeable while other smaller elements are harder to notice. Environmental objects did not have any function in the game but to serve as decoration and obstacles, so players might have paid less attention to them as they were not an important part of the gameplay loop. Keeping this in mind, the following observations can be made about the role that PCG in Game Bits has in game user experience:

➔ PCG in smaller, less interactive elements has a lesser impact. Repetitive ​ elements that blend with the background and are not the focus of players, such as vegetation or other decorative elements, can easily be algorithmically generated without the results causing a negative effect on the player’s experience. These elements are already expected to present some variety, so they tend to be less scrutinized by the players. ➔ Subtle variations give better results. Most players in the experiment ​ acknowledged and appreciated the variety in the PCG version. When noticed, the changes in enemy aesthetics were always liked. Although the different modules that comprised each enemy type changed considerably, especially their heads and clothing, giving the same color palette to all instantiations of the same enemy type for that run made them look more consistent and easier to identify when playing, thus avoiding generating confusion and frustration. However, the extreme color changes in the background caused the opposite effect. While it was not enough to hinder players’ enjoyment of the game, some were confused by these sudden changes and preferred the standard color variation chosen for the non-PCG version, which was closer to realism. This problem can be easily fixed with more subdued variations. ➔ Bigger changes require context. If having a wide range of variation is a design choice, then players require a reason to suspend their disbelief. Players that did not like the more abstract-looking background color variations expressed the need for a narrative context. ➔ When applying PCG to interactive elements, tying their changes to their behavior might boost players’ enjoyment. This is related to the previous point. ​ Some players expected a link between visual variations and gameplay. When presented with an unexplained change in the background color, participant #4 commented on how they expected the game to get more difficult. Participant #7 also commented on how some background variations felt more disturbing and oppressive. Although, in this case, it was not enough to burden the players’ enjoyment of the game, not delivering what players expect can be the cause of disappointment and frustration.

33

6. Conclusions

6.1 Summary Procedural Content Generation (PCG) is a technology that has been around for decades. However, it is becoming progressively more popular as the virtual worlds are getting bigger and more varied. Creating every environmental object by hand would suppose a massive increase in costs, especially in bigger types of games like open-world games, so developers recur to either repetition or, in order to add some variety, PCG. Besides reducing development time and costs, PCG can help boost the artists’ and designers’ creativity, opening new possibilities with combinations of authored and computer-generated content. It can also carry negative consequences to the development process, but if implemented correctly, the advantages usually outweigh the negatives. While this technique is most commonly used in environmental elements (i.e. vegetation like trees, or different kinds of terrain like mountains) it can also be applied to almost any layer of games. Whole new genres have emerged with the use of PCG. For example, Interactive Digital Narrative games emerged when procedurally generating a narrative and adapting it to the player’s actions, and Roguelikes appeared when a game with the same name used PCG to generate a new dungeon every time the player started a new game.

With increased popularity, it is only natural that there is no lack of research based on this subject. However, most of it focuses on categorizing or testing new or existing methods to apply PCG. The effects this technology has on the players’ experience are still not very well researched, especially when used to introduce as much variety as possible in the building blocks of a game. On one hand, more variety makes a game presumably less repetitive and thus more enjoyable and replayable since the game will keep surprising the player for an extended period of time. On the other, unpredictability can lead to confusion and frustration, scarring the player’s experience.

In order to study the possible effects of PCG, it is important to be able to differentiate the different applications it can have in a game. By dividing the structure of games into layers, it becomes easier to categorize and analyze the consequences of integrating PCG in each of these areas. Following the categorization of game layers that Hendrixk (2015) proposes, PCG is most commonly applied to the bottom two layers: Game Space and Game Bits. Game Space comprises the structure of the levels the games take place in, and several genres, especially Roguelikes, make use of this to provide almost endless replayability to players. It is very interesting, however, to test the effects when PCG is applied in the Game Bits layer, as they are not as obvious. The Game Bits layer is composed of all the elements that do not affect the gameplay actively, mainly the visual aspect of the game, as well as the sound and other effects. Usually, PCG is used sparingly on these elements to a point where players can barely notice what was authored and what was created with an algorithm. However, if we try to introduce as much variation as possible in these building blocks, the presence of this technology might become more evident and affect the player’s experience in an unexpected manner. Results indicate that focusing PCG only on this layer does not seem to alter the players’ experience in a significant way, leaving the choice of implementing it to a development matter (i.e. implementing it in order to save costs and time).

34

This dissertation tries to cast some light on this subject as it uses a game developed for this specific purpose to compare the players’ experience and desire to continue playing the game when PCG is applied to its Game Bits and when it is not.

The artifact created for the purposes of this paper was a simple 2D action Roguelike game divided into two different but mechanically identical versions: one exempt of any PCG in the Game Bits layer, and another one with relatively heavy implementation of PCG in this area. The reason why this genre was chosen over any other is that PCG is already being actively used in many of the commercial products released within this type of game, always applied in the Game Space layer, but also sometimes present in the Game Bits area. Such is the case of Spelunky (2008, Mossmouth) and Noita (2019, Nolla Games). The prototype was intended to resemble other commercial modern adaptations of the Roguelike genre and presents 2D cartoony graphics, themed around pirates, with a top-down perspective. Players have to traverse a maze of rooms while facing any enemy they encounter and clear the final area to move on to the next stage. Enemies and environmental objects were created modularly, meaning their appearance was separated into different parts which would be randomized and changed color in the PCG version, while in the other one these pieces would be hand-picked by the artist. The backgrounds of each stage would only change in color between versions, presenting only one standard color in the non-PCG version of the game, and an array of different color palettes in the other one.

For the experiment, a sum of eight subjects participated with very diverse backgrounds and game experience. Multiple sources of data were registered. Players’ screens were shared in order to observe their playtime, and their video and voice were recorded for later reference. Participants played both versions of the game, with the order being swapped for every other player. They had a limit of ten minutes to play each version and they were instructed to share their opinions and thoughts out loud as much as possible. They were also notified that they could stop playing a version whenever they wanted to. After playing each version, they were asked to complete the CEGE questionnaire. To end the experiment, every player was interviewed about what was observed both during their playtime and their answers in the questionnaires, as well as other questions related to the research questions presented. The whole procedure took an average of sixty minutes.

Following the CEGE model (chapter 2.6 and figure 2), since the changes introduced in the game were purely visual, the latent variables of this model that were expected to present some variation were the environment, which is within the video game area, and the facilitators relating to aesthetic values, which are within the puppetry section. Since in the model both variables have influence over the perception of enjoyment, this was expected to change as well, along with the frustration levels if the experience resulted to be negative. Participants noticed the background color changes above anything else, with some of them not noticing the finer differences in enemies and environmental objects and others even not noticing any difference at all between both versions. The game experience was similar between players, with a generally high level of enjoyment and will to continue playing, but also with a wide range of levels of frustration across the sample, all of them caused by faults in the game design such as bugs and some balancing issues in the difficulty curve.

35

As for the effects that applying PCG in the Game Bits layer has on the player’s experience and their desire to continue playing, it did not seem to be an important factor for most participants. Some acknowledged and welcomed the increase in the variety of the elements, but stated in the post-game interview that this had no major effect on their experience.

Overall, it seems applying PCG in the Game Bits layer in 2D games does not have a strong impact on the players’ experience. This would also suggest that, regarding the CEGE model, the observable variables that were altered and studied in this paper (graphics and aesthetics) do not have enough weight on their own to affect the latent variables they belong to (environment and facilitators, respectively) or the rest of the latent variables those are connected to and, ultimately, they do not have enough weight either to majorly affect the enjoyment of the game. Table 4 summarizes the results obtained.

Table 4 - Summary of the results obtained and discussed in chapters 5 and 6.

➔ The game was liked overall. ➔ Some players found some aspects lacking, mostly due to the early stage of General game the game. impressions ➔ The difficulty curve was hit-and-miss.

➔ There was little to no difference between versions. ➔ Game Space and Game Design layers were more influential than the Enjoyment Game Bits layer.

➔ There was a wide range in the frustration levels between participants in the questionnaire. ➔ Bugs and an unbalanced difficulty curve were the main cause of Frustration frustration, accompanied by the randomness of the dungeon generation. ➔ PCG in the Game Bits layer was not a factor.

➔ Mostly not related to PCG in the Game Bits layer. ➔ Artificial factors, like the perception of speed or difficulty differences between versions, were a cause for the increase or decrease of the desire to replay a version. Replayability ➔ A lack of context for the bigger changes in the Game Bits layer, the color variations in the background environment, was a minor negative factor for some participants.

➔ Most players focused on the background differences, ignoring or giving Visual little importance to variations in enemies or environmental objects. environment ➔ “Extreme” variations of the background colors were generally disliked and aesthetic but did not seem to affect the enjoyment factor in any major way. value

36

6.3 Discussion Testing the game online might have had a repercussion on the outcome of the experiment. Although very few technical issues arose from this method, there were still some cases where the quality of the connection would fluctuate, making it more difficult to communicate or decreasing the quality of the recordings, or lower-end computers would take longer to load new levels, increasing the level of boredom in the participants. This could have been solved by inviting the participants to a prepared room with all the needed materials already set in. Also, players might have felt keener on sharing their thoughts and opinions during their playtime if they were sharing the same space with another person, instead of “talking alone”. In opposition, players might have experienced increased levels of nervousness and social anxiety due to the test taking place in an unfamiliar location with the researchers being physically present. This was not a factor when using an online method, since participants played in a familiar environment, presumably their own homes. The method of this experiment was, however, not an option. Due to the global pandemic taking place during the time of this dissertation, having the experiment performed online was the only viable choice.

The testing sample had a mixed background, with only half of the participants being familiar with the Roguelike genre, and only one of them being a habitual player of the said family of games. Although the results were relatively consistent for all participants, their experience with the genre (or lack thereof) might have affected the manner in which each of them perceived the game. It is extremely difficult to speculate how the results might have differed if all the participants had identical backgrounds, as it might influence the outcome either way. More experienced Roguelike players might disregard or overlook PCG as they are more used to it being present, or they might be more critical with it since they have previous experiences to compare it with. Whatever the case, in this research paper the results were alike between very different backgrounds, so a similar outcome could be extrapolated to any other person with a different playing history. Also relating to the testing sample, only eight people could be tested for this experiment so the results, even if they are constant enough to be indicative of what may be a general player experience, are inconclusive until this is tested again in a larger sample.

The design choices taken when creating the artifact that was used for this paper carried a set of consequences for the PCG method chosen. Since the game was hand-drawn and traditionally animated, with each animated element being composed of a sequence of hand-drawn frames, the possible methods to apply PCG were severely limited. The method chosen was picked because of its simplicity to integrate into the game, with all the different elements being authored by the artist and then ensembled together by the algorithm, creating a wide range of possible combinations from a relatively limited amount of work. A different method might provide different results, as could have happened with the ones Korn et al (2017) produced. This difference in results, when compared to the ones shown in this paper, might not only be caused due to different methods of PCG being used but might be also related to the elements that this computer generation was applied to. In their study, Korn et al (2017) chose a single element in the game to test. Although this element, the coral reefs, was crucial for the mechanics of their game, it also blended in with the rest of the environment while still being noticeable. This can be easily achieved when PCG is applied to

37

small and repetitive environmental objects, which was also the case in this dissertation’s game with the vegetation and rocks as they seemed to blend in with the environment and they were not the focus of any player when asked about their enjoyment or opinion about the aesthetic values of the game, although they might have taken part in it in a smaller scale. However, when PCG is applied to bigger or more obvious elements, like the whole background or the enemies, new and different results can and did appear.

As stated in the summary section (chapter 6.1) other players preferred the non-PCG version due to the colors in the background, as they felt the greener version was more fitting and realistic for a jungle environment, despite the game not striving for realism. Another possible explanation for these preferences is related to the theories discussed in chapter 2.7. It is a fact that differences in the color of a game can elicit emotions such as joy or sadness, so brighter more saturated versions of the game could result in the player experiencing more joy, thus increasing their overall enjoyment of the game. Some participants also expected some kind of gameplay change linked to the new colors and were confused or disappointed when this did not happen. When furtherly inquired about it, some of these players stated that they would have had a different impression if these changes were linked to some narrative aspect or their context was explained. This is a factor to keep in mind when implementing big noticeable changes in a game through PCG, as this study shows that without a context they might mildly affect the player experience.

Another important factor to note is that some other players had difficulty noticing the changes caused by the procedural generation. This could be caused by the random bits shown to them being very similar or identical to the other version (which was the case for most of the background colors participant #6 experienced, for example). Another possibility is that the allotted time to play each version was too short and participants did not have enough time to experience the PCG at its fullest, so longer exposition to the game might have produced different results in those cases.

The dichotomy between high levels of enjoyment and the wide range of levels of frustration can be explained in two ways. The first one being players enjoyed the game overall, including its aesthetics and gameplay loop, but still were occasionally frustrated due to an unbalanced difficulty curve or technical issues (bugs) that hindered their control over the game. A different explanation could have been that some of the participants presented response bias and answered both questionnaires and interview questions inaccurately, giving the answer that they thought would please the researchers the most when asked about their enjoyment and desire to continue playing. Although this is still a possibility, it is very unlikely as participants were also observed during their playtime and most of them expressed clear signs of enjoyment, such as moments of laughter or entering a state of flow.

Finally, the results indicate that when PCG is applied to the Game Bits layer it does not affect the users’ experience in a major way. This should not be enough to suppose a deterrent to using PCG in this layer. Even if integrating procedurally generated content in the visual aspect of a game does not affect the player’s experience positively, it is still an advantageous technology that helps with the development process by cutting its costs and the time required to produce all the different building blocks that comprise this important layer.

38

6.3 Future Work This dissertation shows an indicative that implementing PCG in the Game Bits layer does not affect the player’s experience or their will to continue playing in any major way. These results contrast the ones provided by Korn et al (2017) as they found that players preferred to play with PCG integrated into a specific Game Bits element. Future research could try to discern the reason behind this difference or try to further experiment on the effects of PCG in other types of games or game layers. After analyzing the possible variables that may have affected this outcome, the following suggestions are made for future research:

➔ Aim for a specific target audience or background: In this paper, the sampling ​ was made with a mix of convenience and “snowball” sampling. While this caused an interesting and balanced mix of backgrounds, focusing the target audience to a specific player background might provide more specific results. ➔ Use a bigger sample: The sample used in this experiment was relatively small. ​ Although it is not necessary to make use of huge samples to conduct a qualitative study, a bigger sample might provide more distinct data and interesting cases. ➔ Other types of games: The game used in this dissertation fell into the 2D ​ Roguelike genre. While the results were consistent and the wide range of backgrounds could make it possible to extrapolate this case to other genres, it might be of interest to investigate the same research questions within a different genre or a 3D game. The reason for this is that the array of PCG methods available in each case would be completely different and thus might produce different results.

39

References

Artificial Intelligence Design (1980) Rogue [Computer game]. PC. Epyx. ​ ​ Bay 12 Games (2006) Dwarf Fortress [Computer Game]. PC. Bay 12 Games. ​ ​ Blizzard Entertainment (1997-2012) Diablo Franchise [Computer Games]. PC. Blizzard ​ Entertainment.

Calvillo-Gámez, E.H., Cairns, P. and Cox, A.L., (2010) Assessing the core elements of the gaming experience. In Game user experience evaluation (pp. 37-62). Springer, Cham. ​ ​ Chamberlain, R., Mullin, C., Scheerlinck, B. and Wagemans, J. (2018) Putting the art in artificial: Aesthetic responses to computer-generated art. Psychology of Aesthetics, ​ Creativity, and the Arts, 12(2), p.177. ​ Discord inc. (2015) Discord [Computer Software]. Discord inc.

Dodge Roll (2016) Enter the Gungeon [Computer Game]. PC. Devolver Digital. ​ ​ Frade, M., de Vega, F.F. and Cotta, C., (2012) Automatic evolution of programs for procedural generation of terrains for video games. Soft Computing, 16(11), pp.1893-1914. ​ ​ Gao, X.P., Xin, J.H., Sato, T., Hansuebsai, A., Scalzo, M., Kajiwara, K., Guan, S.S., Valldeperas, J., Lis, M.J. and Billger, M., (2007) Analysis of cross cultural color emotion. - Color Research & Application: Endorsed by Inter Society Color Council, The Colour Group - (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur, 32(3), pp.223-229. ​ Gearbox Software (2009-2019) Borderlands Franchise [Computer Games]. PC. 2K Games. ​ ​ Geslin, E., Jégou, L. and Beaudoin, D., (2016) How color properties can be used to elicit emotions in video games. International Journal of Computer Games Technology, 2016. ​ ​ Hendrikx, M., Meijer, S., Van Der Velden, J. and Iosup, A., (2013) Procedural content generation for games: A survey. ACM Transactions on Multimedia Computing, ​ Communications, and Applications (TOMM), 9(1), pp.1-22. ​ Ho, X., Tomitsch, M. and Bednarz, T., (2016) Finding design influence within roguelike games. In International Academic Conference on Meaningful Play (pp. 1-27). ​ ​ Hooshyar, D., Yousefi, M., Wang, M. and Lim, H., (2018) A data driven - procedural content generation approach for educational games. Journal of Computer - - ​ Assisted Learning, 34(6), pp.731-739. ​ Iacovides, I., Cox, A., Kennedy, R., Cairns, P. and Jennett, C., (2015) Removing the HUD: the impact of non-diegetic game elements and expertise on player involvement. In Proceedings ​ of the 2015 Annual Symposium on Computer-Human Interaction in Play (pp. 13-22). ​

40

Interactive Data Visualization, Inc. (2002) SpeedTree [Computer Software]. Interactive Data ​ Visualization, Inc.

International Organization of Standardization (2018) ISO 9241-11:2018, 3.2.3 Ergonomics of human-system interaction — Part 11: Usability: Definitions and concepts. User Experience. Geneva: ISO.

Joosten, E., Lankveld, G.V. and Spronck, P. (2010) Colors and emotions in video games. In ​ 11th International Conference on Intelligent Games and Simulation GAME-ON (pp. 61-65). ​ Khaled, R., Nelson, M.J. and Barr, P., (2013) Design metaphors for procedural content generation in games. In Proceedings of the SIGCHI conference on human factors in ​ computing systems (pp. 1509-1518). ​ Klei Entertainment (2013) Don’t Starve [Computer Game]. PC. 505 Games. ​ ​ Koeneke (1983) Moria [Computer Game]. PC. Koeneke. ​ ​ Koenitz, H., (2015) Towards a Specic Theory of Interactive Digital Narrative. In Interactive ​ Digital Narrative (pp. 107-121). Routledge. ​ Korn, O., Blatz, M., Rees, A., Schaal, J., Schwind, V. and Görlich, D., (2017) Procedural content generation for game props? a study on the effects on user experience. Computers in ​ Entertainment (CIE), 15(2), pp.1-15. ​ Law, E.L.C., Roto, V., Hassenzahl, M., Vermeeren, A.P. and Kort, J., (2009) Understanding, scoping and defining user experience: a survey approach. In Proceedings of the SIGCHI ​ conference on human factors in computing systems (pp. 719-728). ​ McMillen, E. and Himsl, F. (2011) The Binding of Isaac [Computer Game]. PC. McMillen, E. ​ ​ MegaCrit (2017) Slay the Spire [Computer Game]. PC. Humble Bundle. ​ ​ Mojang Studios (2009) Minecraft [Computer Game]. PC. Mojang Studios and Microsoft ​ ​ Studios.

Mossmouth (2008) Spelunky [Computer Game]. PC. Mossmouth. ​ ​ Nolla Games (2019) Noita [Computer Game]. PC. Nolla Games. ​ ​ Procedural Arts (2005) Façade [Computer Game]. PC. Procedural Arts. ​ ​ Risi, S., Lehman, J., D'Ambrosio, D.B., Hall, R. and Stanley, K.O., (2015) Petalz: Search-based procedural content generation for the casual gamer. IEEE Transactions on ​ Computational Intelligence and AI in Games, 8(3), pp.244-255. ​ Rockstar North (2013) Grand Theft Auto V [Computer Game]. PC. Rockstar Games. ​ ​ Rocksteady Studios (2009) Batman: Arkham Asylum [Computer Game]. PC. Eidos ​ Interactive.

41

Schønau-Fog, H. and Bjørner, T., (2012) “Sure, I Would Like to Continue” a method for mapping the experience of engagement in video games. Bulletin of Science, Technology & ​ Society, 32(5), pp.405-412. ​ ​ ​ Schönauer, C., Pintaric, T., Kaufmann, H., Jansen-Kosterink, S. and Vollenbroek-Hutten, M., (2011) Chronic pain rehabilitation with a serious game using multimodal input. In 2011 ​ International Conference on Virtual Rehabilitation (pp. 1-8). IEEE. ​ Shaker, N., Liapis, A., Togelius, J., Lopes, R. and Bidarra, R., (2016) Constructive generation methods for dungeons and levels. In Procedural Content Generation in Games (pp. 31-55). ​ Springer, Cham.

Spike Chunsoft (2005-2020) Pokémon: Mystery Dungeon Franchise [Computer Games]. ​ Various Nintendo platforms. Nintendo.

Shaker, N., Togelius, J. and Nelson, M.J., (2016) Procedural content generation in games. ​ ​ Switzerland: Springer International Publishing.

Smelik, R., Tutenel, T., de Kraker, K.J. and Bidarra, R., (2010). Integrating procedural generation and manual editing of virtual worlds. In Proceedings of the 2010 Workshop on ​ Procedural Content Generation in Games (pp. 1-8). ​ Summerville, A., Snodgrass, S., Guzdial, M., Holmgård, C., Hoover, A.K., Isaksen, A., Nealen, A. and Togelius, J., (2018) Procedural content generation via machine learning (pcgml). IEEE Transactions on Games, 10(3), pp.257-270. ​ Supergiant Games (2018) Hades [Computer Game]. PC. Supergiant Games. ​ ​ Togelius, J., Champandard, A.J., Lanzi, P.L., Mateas, M., Paiva, A., Preuss, M. and Stanley, K.O., (2013) Procedural content generation: Goals, challenges and actionable steps. Schloss ​ Dagstuhl-Leibniz-Zentrum fuer Informatik

Togelius, J., Justinussen, T. and Hartzen, A., (2012) Compositional procedural content generation. In Proceedings of the The third workshop on Procedural Content Generation in ​ Games (pp. 1-4). ​ Togelius, J., Yannakakis, G.N., Stanley, K.O. and Browne, C., (2011) Search-based procedural content generation: A taxonomy and survey. IEEE Transactions on Computational ​ Intelligence and AI in Games, 3(3), pp.172-186. ​ Van Der Linden, R., Lopes, R. and Bidarra, R., (2013) Procedural generation of dungeons. IEEE Transactions on Computational Intelligence and AI in Games, 6(1), pp.78-89. ​

42

Appendix A

CEGE Questionnaire

1. I enjoyed playing the game

2. I was frustrated at the end of the game

3. I was frustrated whilst playing the game

4. I liked the game

5. I would play this game again

6. I was in control of the game

7. The controllers responded as I expected

8. I remember the actions the controllers performed

9. I was able to see in the screen everything I needed during the game

10. * The point of view of the game that I had spoiled my gaming

11. I knew what I was supposed to do to win the game

12. * There was time when I was doing nothing in the game

13. I liked the way the game look

14. The graphics of the game were plain

15. * I do not like this type of game

16. I like to spend a lot of time playing this game

17. * I got bored playing this time

18. * I usually do not choose this type of game

19. * I did not have a strategy to win the game

20. The game kept constantly motivating me to keep playing

21. I felt what was happening in the game was my own doing

22. I challenged myself even if the game did not require it

23. I played with my own rules

I

24. * I felt guilty for the actions in the game

25. I knew how to manipulate the game to move forward

26. The graphics were appropriate for the type of game

27. The sound effects of the game were appropriate

28. * I did not like the music of the game

29. The graphics of the game were related to the scenario

30. The graphics and sound effects of the game were related

31. The sound of the game affected the way I was playing

32. * The game was unfair

33. I understood the rules of the game

34. The game was challenging

35. The game was difficult

36. The scenario of the game was interesting

37. * I did not like the scenario of the game

38. I knew all the actions that could be performed in the game

* Denotes items that are negatively worded.

II

Appendix B

Interview Guide

General impressions and mechanics:

The post-game interview starts with some warm-up questions about general impressions and general game mechanics.

1. What did you think of the game and your playing experience? 2. Were the controls understandable and easy to use? 3. Was it clear what you had to do in the game?

Game user experience:

The focus of this section is to get more input about enjoyment, frustration, replayability, and visual environment perception and aesthetic values. This part is designed to be complemented by the CEGE questionnaires. If noticeable differences are observed in them, ask about the reason behind those differences.

4. Did you enjoy the game overall? 5. Would you say you enjoyed one version over the other? Why? 6. Did you feel frustrated while playing the game? What caused this frustration? 7. Did any of the versions cause more frustration or seem more unfair than the other one? Why? 8. Would you continue replaying any of the versions? Why? 9. Which version would you prefer to continue playing? 10. What is your opinion about the visual aspect of the game? 11. Did you prefer the visual aspects of one version over the other? Why?

Difference between both versions of the game:

This section is aimed at unveiling to what degree the player noticed the changes that happened during the PCG version.

12. What differences did you notice when playing both versions?

Previous experience:

The interview is finished with some background questions focused on previous experience and familiarity with the Roguelike genre.

13. How many hours do you spend playing games on average every week? 14. How familiar are you with the genre roguelike? 15. How much do you play roguelikes on average every week? 16. How long is your average play session when playing roguelikes? 17. (If not familiar or doesn’t play roguelikes) What kind of game do you usually enjoy?

III