Abstract

This dissertation presents a study that explores the idea of implementing PCG in game bits. Procedural Content Generation (PCG) refers to content in games that is created by an algorithm rather than a human. Game bits is the part of game content that relates to graphics, audio and other elements that don’t directly affect gameplay. The goal was to find out what a PCG implementation in game bits needs to affect player behaviour. Qualitative play sessions with interviews were performed to examine potential ways this could occur. Results show that no noticeable behavioural differences appeared due to PCG in game bits, but three properties are set up detailing how implementations would increase the odds of affecting player behaviour. These properties are: PCG implementation changing visuals drastically, different visual elements matching and game space generation matching game bits repeatedly.

Keywords: PCG, game bits, player behaviour, roguelike ​

Table of Contents

1. Introduction……………………………………………………………………………....1 ​ 2. Background…………………….…………………….…………………………………..2 ​ 2.1 Procedural Content Generation…………………….……………………………………….. 2 ​ ​ 2.1.1 Reasons to use PCG…………………….…………………….………………………………. 2 ​ 2.2 Different methods for PCG…………………….…………………….……………………….. 3 ​ ​ 2.2.1 Search-based approach…………………….…………………….………………………….... 3 ​ 2.2.2 Agent-based dungeon growing…………………….…………………….…………………….4 ​ 2.3 Games that apply PCG…………………….…………………….………………………….... 4 ​ ​ 2.4 Layers of Game Content…………………….…………………….…………………………. 7 ​ ​ 2.5 Roguelike…………………….…………………….…………………….…………………….. 9 ​ ​ 2.5.1 Roguelikes Over the Years…………………….…………………….………………………. 10 ​ 2.5.2 Roguelikes and Layers of Game Content…………………….……………………………..11 ​ 2.5.3 Game Bits in Roguelikes…………………….…………………….…………………………. 12 ​ 2.6 Player behaviour…………………….…………………….…………………………………. 15 ​ ​ 3. Problem…………………….…………………….……………………………………...16 ​ 3.1 Method…………………….…………………….…………………….……………………....16 ​ ​ 3.1.1 Artifact…………………….…………………….…………………….………………………... 17 ​ 3.1.2 Test procedure…………………….…………………….……………………………………..18 ​ 3.1.3 Data gathering and analysing…………………….…………………….…………………….18 ​ 4. Implementation…………………….…………………….……………………………. 21 ​ 4.1 Menu…………………….…………………….…………………….………………………... 21 ​ ​ 4.2 Controls and HUD…………………….…………………….……………………………….. 22 ​ ​ ​ 4.3 Progression and content…………………….…………………….………………………... 23 ​ ​ ​ 4.4 Parts that use PCG…………………….…………………….……………………………… 24 ​ ​ 5. Evaluation…………………….…………………….………………………………….. 26 ​ 5.1 Results…………………….…………………….…………………….……………………....26 ​ ​ 5.1.1 General thoughts of the game…………………….…………………………………………. 26 ​ 5.1.2 Perceived differences…………………….…………………….…………………………….. 27 ​ 5.1.3 Behavioural differences…………………….…………………….…………………………...27 ​ 5.2 Analysis…………………….…………………….…………………….……………………...27 ​ ​ 5.2.1 Commercial game sense…………………….…………………….………………………….27 ​ 5.2.2 Reasons for specific behaviour…………………….………………………………………... 28 ​ 5.2.3 Attention focus…………………….…………………….…………………………………….. 28 ​ 5.2.4 Connection with previous experiences…………………….……………………………….. 30 ​ 5.3 Conclusions…………………….…………………….………………………………………. 30 ​ ​ 6. Concluding Discussions…………………….…………………………………….. 32 ​ 6.1 Summary…………………….…………………….…………………………………………. 32 ​ ​ 6.2 Ethical Considerations…………………….…………………….…………………………...33 ​ ​ 6.3 Discussion…………………….…………………….………………………………………... 34 ​ ​ 6.3 Future Work…………………….…………………….……………………………………….35 ​ ​ References…………………….…………………….……………………………………..37

1. Introduction

Procedural Content Generation (PCG) is content in games that is created by an algorithm, rather than a developer or player. Content itself can refer to many things, but often means graphics, audio, complete game components or levels. These algorithms often create content much faster than a human ever could, making it an efficient way to add a lot of content to a game. This can also be exploited to have games create unique content for every player while the game is running. A main reason to use PCG is then development efficiency, but it can also be used to increase replay value or create novel mechanics. PCG implementations can either be constructive or generate-and-test. As constructive methods only generate content once, it can be used during runtime. Generate-and-test methods instead repeatedly generate and evolve these results until something desirable comes up, making it more fitting to create content before play. While PCG can be implemented in most types of games, one genre that is closely connected to it is roguelike.

Roguelikes were originally games very inspired by the game Rogue (1980, A.I. Design) and has since then become a way to distinguish games that use a specific combination of core mechanics. These mechanics are procedurally generated levels and permadeath. Permadeath refers to the player having to restart the entire game when they reach a failstate. This, combined with the fact that levels are generated when starting the game, means that every attempt in the game contains unique levels that are lost as soon as the player loses or wins. Roguelikes then becomes a very fitting genre to explore PCG further with.

Game content can be divided into several layers. Two of these are game space and game bits. While game space refers to content that the player directly interacts with (ground, walls, enemies), game bits are the, mostly, aesthetical parts of content that don’t directly relate to gameplay, like environmental graphics, audio and effects. Roguelikes always apply PCG to game space, as generating levels is part of the genre. PCG is also occasionally applied in game bits, both in roguelikes and other genres. When it comes to applying PCG, not much is known on what effects it has when used in game bits, mostly because generating game space receives the most focus. Knowing how certain elements, mechanics or implementations affect how players behave is important since it allows developers to control the intended player behaviour and choose the most efficient methods, knowing it doesn’t disrupt these intentions. By letting players play through two different versions of a roguelike, one where PCG is applied to game bits and one where it isn’t, then following that with qualitative interviews, conclusions on how it potentially affects player behaviour and to what degree are drawn. The question is the following:

Which properties of game bits using PCG changes player behaviour?

Results show that no noticeable behavioural differences appeared due to PCG in game bits, but conditions are set up detailing how some ways of implementing it would increase the odds of affecting player behaviour. These conditions generally come down to how drastic the PCG implementation changes visuals and if certain results match up with other aesthetical or gameplay related elements to create false player assumptions.

1 2. Background

2.1 Procedural Content Generation Procedural Content Generation (PCG), as it is used in games, can refer to “the creation of game content automatically using algorithms” (Togelius, Kastbjerg, Schedl & Yannakakis, 2011, p. 1). Other definitions often point to the exact same meaning (Summerville et al., 2018). PCG is specific for digital games, as it requires the algorithm to be run by software. The content that is created is then what the player of the game experiences and interacts with. While creating content for a game manually is a more traditional and common method, using PCG will allow content to be made without developer input. Content created by players is, however, not included in this definition. Content itself can be many different things, and is likely defined differently between areas of game development, so what it means to apply PCG is very loosely defined. A variation of the definition can be found from Hendrikx et al. (2013): “the application of computers to generate game content, distinguish interesting instances among the ones generated, and select entertaining instances on behalf of the players” (p. 2). This definition implies a specific purpose with using PCG: to create entertaining content. There are, however, other possible motivations behind using the method.

2.1.1 Reasons to use PCG This section will go over common reasons to apply PCG in games.

Time efficiency. PCG can be used to help level designers. (Smith & Bryson, 2014). A level ​ designer can be tasked with creating content in the form of interactable environment geometry, placement of items or general structure of the area used for gameplay. PCG can then be used in this area for efficiency, as having an algorithm create this kind of content is generally much faster than making it manually. If the goal of the game is to provide a large amount of levels, then applying PCG can greatly benefit that game. It can even be that a development team doesn't need a level designer since they already have algorithms to make that content, making this a potentially cost efficient solution.

Replay value. Another reason can be to increase a game’s replay value (Summerville et al. ​ 2018). Replay value generally refers to how sparse the user experience is (Roth, Vermeulen, Vorderer & Klimmt, 2012). A game having a high replay value means that there is an incentive to replay the game, be it a complete restart or with some parameters carrying over. PCG helps in this regard as it can generate new content every time the player plays the game. In that way, the player always experiences new content which creates a reason to keep playing and/or keep replaying.

Memory. All content in a game needs to be stored in a memory of some form. With ​ manually created content, everything needs to have allocated storage before play. If content is instead created during play, then only the content needed at the moment can be stored in memory after being generated, and then discarded, thereby saving space (Khaled, Nelson & Barr, 2013).

Dynamic difficulty. PCG can also be used during play to give the game dynamic difficulty ​ which alters how difficult the game is while it is being played (Shaker, Togelius & Nelson,

2 2016). This can tailor the game experience in an attempt to provide content that fits the current player best. For example, if the player, through gameplay, appears to struggle against a certain enemy, PCG can lower the rate at which that enemy appears.

With a broad range of reasons to use PCG, it has become an increasingly popular technique for creating game content. While reasons such as replay value or dynamic difficulty are only relevant for some games, other motivations like time efficiency and memory can be applied to all games and can thereby benefit all games when it comes to quality, quantity, development or distribution.

2.2 Different methods for PCG Alongside the loose definition of content when describing PCG and the varying motives behind using it, there are also many different ways of implementing it. Based on an existing taxonomy (Togelius, Yannakakis, Stanley and Browne, 2011), sets of properties that PCG methods can have can be summarized. All methods or solutions using PCG lie somewhere between the two ends of each set.

Necessary and optional. There are cases where the content generated is required for ​ gameplay to progress. This can be the case if the entire area players can move around in is using PCG to create its geometry. PCG can also be used to create optional content that only relates to aesthetic aspects or a part of a game that isn’t required for the player to go through.

Generic and adaptive. While generic content generation doesn’t take player actions and behaviour into account, adaptive content generation is to some degree based on what the player does. Adaptive variants have received more attention over time as it can be used to alter content towards a specific player experience. Generic content generation remains the most common method.

Deterministic and stochastic. Trying to recreate content that a stochastic PCG ​ implementation has generated previously is normally impossible, as it's created to always produce different results no matter what. Deterministic implementations can use specific parameters or starting points to always produce the same outcome. One way to do this is by applying a seed. Seeds are codes that specify exactly how the content will be generated.

Constructive and generate-and-test. PCG methods that use generate-and-test generate ​ their content in multiple instances, before evaluating them and repeating the process to end up with an instance with the best possible evaluation score. Constructive methods only generate a single instance and apply it without testing. Generally, constructive methods are more common in commercial games.

Any individual implementation of PCG can combine properties from these sets in different ways. Below are two specific examples of PCG methods.

2.2.1 Search-based approach Search-based methods are generate-and-test implementations used in a specific way. An algorithm, for example an evolutionary type, is used to search through the generated content samples to find ones that score high when evaluating. (Togelius and Shaker, 2016). Rather

3 than picking the first generation that satisfies a specific need, all generations are graded. The results with the lowest score are discarded while the highest ones are used in the algorithm to create better results. Rules that may need to apply to constructive methods can be limited or completely discarded as every generation isn’t required to function properly. This can result in lower development costs (Risi et al., 2015).

2.2.2 Agent-based dungeon growing Agent-based methods are constructive, often necessary ways of generating environments in games that take the form of dungeons. A dungeon in games is a restricted labyrinthine environment that can contain various gameplay objects and goals (Shaker, Togelius and Nelson, 2016). Agent-based methods create these dungeons by using often a single agent to create rooms and tunnels sequentially. The agent digs these spaces as it moves in the available space. Agents can have different levels of stochasticity, where a high level will result in a more chaotic dungeon and a lower level will try to avoid intersecting corridors and rooms. Letting the agent dig in an unpredictable manner can be desirable, as the dungeon’s layout may become more organic or realistic in its structure (Shaker et al., 2016). Figure 1 shows an example of how an agent (red square) can generate a dungeon by creating rooms one by one.

Figure 1 Dungeon created by agent-based PCG.

2.3 Games that apply PCG PCG can be applied to any type of game, since it only specifies a certain way of creating content. This section goes over common types of games and how implementations of PCG can differ between them.

Roguelikes. A genre of games that is closely related to PCG is roguelike. The name references the game Rogue (1980, A.I. Design) since the game itself invented the genre through innovative gameplay mechanics (Ho, Tomitsch and Bednarz, 2016). Games within this genre mostly focus on using PCG for level design, specifically to create the floors and dungeons the player explores. These are among the first to create content procedurally. Roguelikes traditionally have two dimensional gameplay.

4

Figure 2 A procedurally generated dungeon in Rogue.

First person games. Games that use a first person perspective occasionally apply PCG to ​ create variation in aesthetic details. This is done in the Borderlands franchise (2009-2019, Gearbox Software) as well as in Elder Scrolls IV: Oblivion (Bethesda Game Studios, 2006). In these cases, visual elements like vegetation growth and placement are procedurally generated. This is mostly to save time, as placing out assets of vegetation can be time consuming. This type of functionality can also exist within game engines. Such is the case for Unreal Engine 4 (Epic Games, 2014). Using this functionality means that games created within that engine can apply PCG to vegetation without developers having to create their own algorithms.

Third person games. A common way of incorporating PCG in third person games is through level design. This can be done with any set or player controller camera. Some games that do this are Spelunky (Mossmouth, 2013), Bloodborne (From Software, 2015) as well as the Diablo franchise (1997-2012, Blizzard Entertainment). Out of these examples, Spelunky uses PCG for its level creation most prominently. Only the game’s hub and tutorial are not generated procedurally, while all of the levels do apply PCG. The Diablo franchise applies PCG to its dungeons, which are levels that are accessed from a manually created overworld. As such, only some of the gameplay takes place within procedurally generated content. Bloodborne applies PCG in a specific type of area called Chalice Dungeons. These areas are optional and are small in size relative to the rest of the game. As such, using PCG for level design can be done successfully on very different scales and with different importance to the gameplay.

5

Figure 3 A Chalice Dungeon in Bloodborne.

Narrative driven games. There have been cases where PCG is applied in narrative driven games, for narrative purposes. An example of this is Façade (2005, Procedural Arts). As the player types text to interact with the non-playable characters, the algorithm generates the most fitting available response.

Implementations of PCG are in no way limited to game type. One way of using PCG is by creating entire worlds with it. Rather than generating separate levels, Minecraft (2011, Mojang) creates its entire, endless, singular play space using PCG. This includes all terrain and other objects. Minecraft can be played both in first person and third person mode. While different camera perspectives and focal points (gameplay, story or aesthetics) change how PCG can be implemented in meaningful ways, these types of games often overlap. Because of this, the best ways to apply PCG differs on a case by case basis. An exception to this is roguelikes, as the genre is defined by a specific way of using PCG. Roguelikes are covered in depth in chapter 2.5.

Figure 4 Generated terrain in Minecraft.

6 2.4 Layers of Game Content In order to more closely examine game content with reference to PCG, a taxonomy created by Hendrikx et al. (2013) can be used. This taxonomy divides game content into six layers, all of which can use PCG.

Figure 5 Layers of game content that can be procedurally generated.

These layers are structured in a pyramid shape where those higher up can be built using elements of layers below them. While most layers refer to content directly within games, derived content refers to other types of content that is derived from the in-game content (Hendrikx et al., 2013).

Game bits consist of parts of content that generally do not directly affect gameplay. As such, they are mostly related to aesthetics. They can be divided into abstract and concrete bits, where concrete bits are parts of what makes objects in a game world, such as texture and sound, and concrete bits are the resulting objects that are part of the game. Textures are often made by hand, but can be generated when representing general materials, such as stone, cloth or dirt. A study was made (Korn el al., 2017) that tested the effect on user experience when applying this to the visuals of reef in a top-down ship game. Users tested one version with PCG and one without, and results showed that users consistently perceived a more aesthetically pleasing and realistic experience with PCG.

Sound is generally not generated, as it is difficult to have fitting sound effects or enjoyable songs made procedurally (Hendrikx et al., 2013). Beyond that, musicians and the general public alike have shown resistance to music created this way (Edwards, 2011). Vegetation and buildings can be used both for looks and to influence gameplay as these objects can lead the player’s vision and attention (Hendrikx et al., 2013). How objects behave is also part of game bits, as their interactions when colliding and receiving forces can contribute to creating a complete game world. Flowing water or moving clouds are types of objects that have more recently been applied using procedural generation, as it avoids repetition (Ebert, Musgrave, Peachy, Perlin and Worley, 2002).

Game space is generally the area and environment where gameplay takes place (Hendrikx et al., 2013). The geometry and locations of interactable terrain and other spaces the player can move in define levels and worlds in games. While game spaces indoor can take the form of dungeons, houses or grottos, outdoor spaces are often larger and consist of entrances to indoor spaces. Some game spaces have constricted movement. This can be achieved with tiles in a top-down game like Dungeons of Dredmor (Gaslamp Games, Inc., 2011) or by only

7 allowing specific positions and rotations in a first-person game like Myst Masterpiece Edition (Cyan Worlds, 1999). Other games have free movement in a two dimensional or three dimensional space, like Risk of Rain 2 (Hopoo Games, 2019). Generating game space can be done in all of these cases, but it is more common with games that have restricted movement.

Figure 6 A pre-set position in Myst Masterpiece Edition.

Game systems refers mostly to environmental systems in place that are used to populate spaces with objects (Hendrikx et al., 2013). Animals and plants can rely on ecosystems that decide how they evolve over time, while Clusters of buildings and roads can create the basic structures and contents of outdoor maps. The behaviour of self-acting objects that, for example, represent other humans or animals are also part of this layer. Procedurally generating parts of game systems become more relevant as the game worlds become larger, since it will save more time. Generating only behaviours can lead to interesting gameplay interactions and world building. This was done in Rain World (Videocult, 2017) where the creature Scavenger would develop its behaviour towards the player over the course of the game through procedural generation based on how the player acted.

Game scenarios are how and when events within a game occur (Hendrikx et al., 2013). Events can, for example, be a cutscene playing, unlocking a level or completing a task. Progressing through puzzles and stories is all thanks to how the game scenarios are laid out. One way of procedurally generating game scenarios is by randomly creating quests that the player receives. Although fairly uncommon, games like Tales of Maj’el (Netcore Games, 2012) can generate stories and other elements of game scenarios through normal gameplay.

Game design as a layer covers rules and goals of a game. It determines the rules that game scenarios and everything below it in the pyramid of layers must follow (Hendrikx et al., 2013). There are graphical components to the game design layer as well. For example, art style can decide which types of models, textures and effects should exist. The art style itself is part of game design. When playing a game, the player is running an instance of a game design, either a pre-made one or a procedurally generated one.

8 Derived content is referring to content related to a game that does not exist within the game itself. This type of content can exist on websites or other media and can take the form of news, leaderboards or other databases with information about the game or its players (Hendrikx et al., 2013). Another form of derived content is broadcasting games, which has been popular in Asia for several years (Rossignol, 2008).

While derived content, game design, game scenarios and game systems are more or less uncommon layers for PCG to appear in, game space and game bits appear more often (Hendrikx et al., 2013). Game space is often receiving focus as it is very clearly and directly connected to what the player experiences in the game. This study instead focuses on the game bits layer, which is explained in further detail in chapter 2.5.3.

2.5 Roguelike After Rogue (1980, A.I. Design) grew popular with its then innovative mechanics based on PCG, other games were created that used the same concept as a base and built upon it. Most notable were Moria (Koeneke, Todd Jr. and Wilson, 1983), and Hack (Fenlason, Woodland, Thome and Payne). The clear inspiration from Rogue created the term and genre roguelike (Ho, Tomitsch and Bednarz, 2016).

Figure 7 Left: Rogue. Middle: Hack. Right: Moria.

This genre further developed with both Moria and Hack getting sequels and derivative works, such as NetHack (The NetHack DevTeam, 1987), UMoria (Wilson, 1988), Angband (Angband Development Team, 1990) and ADOM (Thomas Biskup et al., 1994). Aside from using PCG to create randomized dungeons, there were other core characteristics of the genre:

Permadeath is a way to treat failure states in games and is a portmanteau of permanent ​ death. As the player character dies or loses in any way, there is no way to continue on from a

9 previous state as they have to start from the very beginning again (Parker, 2017). This is often done through deleting the save file immediately. As this mechanic was present in Rogue and other roguelikes, it became a core part of the genre (Ho et al., 2016).

Turn based gameplay is also an attribute that all of these games have in common. When a ​ game is turn based, events occur based on turns, rather than on real time (Björk and Holopainen, 2005). In the roguelikes mentioned above, players can typically choose to move their character a single tile (a specific length) cardinally or perform a combat action every turn. This means that both exploration and combat is turn based. Dungeon items and enemies occupy other tiles and take their turns after the player. Turn based gameplay was partly present in these games due to technical limitations, but became a standard in the genre.

Top-down view is the camera perspective all of these games use, and is correlated with the genre. This perspective meant that the camera had no angles that could be changed, as it was always looking directly down on the player character. Although this makes the gameplay fully two dimensional, graphics of later roguelikes may represent characters and the environment from various angles. Even so, the games are still considered top-down. Just like with turn based gameplay, this decision was mostly due to technical limitations. Even so, the perspective gave a clear view of the surrounding area in all directions, which worked well with the tactical decision making of the turn based gameplay.

2.5.1 Roguelikes Over the Years As the genre eventually migrated to home consoles, it was largely due to a single franchise: Mystery Dungeon, which initiated with Torneko no Daibōken: Fushigi no Dungeon (Chunsoft, 1993). Although most games in the Mystery Dungeon franchise were developed by Chunsoft, some entries had different developers and publishers while using the name Mystery Dungeon with permission and applying the same style of roguelike gameplay. Generally, Mystery Dungeon games are less complex than personal computer games of the roguelike genre. Another differentiation is that the gameplay loop often consists of choosing a dungeon that has its PCG elements restricted to specific items, enemies or dungeon lengths set for that dungeon, rather than always generating with the same dungeon parameters.

Further development of the genre can be seen when games started to apply the core principles of roguelikes to action gameplay rather than turn based. An early successful example of this is Diablo (1997, Blizzard Entertainment). As the definitions of a genre can be loose and stretched, it can be difficult to know exactly which games should be classified for the genre and when it becomes something else. This became a clear problem when indie games started to apply PCG-based level design and permadeath in completely different game styles. An early successful example of an indie game using roguelike mechanics is Spelunky (2008, Mossmouth) which applies them to a side-view action platformer rather than a turn based top town game. As this indie developed action oriented take on the genre gained popularity, a trend of letting some progress persist over failure started to appear, with a clear example being Rogue Legacy (2013, Cellar Door Games). In Rogue Legacy, the player can get stronger by using resources that they keep after failed attempts, to the point where completing the game in the first try is unintended and highly unlikely. To differentiate games where progress can be made between attempts, the term roguelite was created, as the

10 permadeath system is exchanged with something more forgiving. In an attempt to drop the connection to Rogue, the term procedural death labyrinth was also created, indicating the core concepts without direct mention of the original inspiration.

Figure 8 Using resources after a failed attempt in Rogue Legacy.

2.5.2 Roguelikes and Layers of Game Content Since generating levels or dungeon structure is part of the roguelike genre, all roguelikes apply PCG in the game space layer. Although game space can refer to many different types of spaces, as long as the main gameplay space of a game is procedurally generated, it can be considered a roguelike (assuming the other typical characteristics of roguelikes are also there). Previous research on the effects of PCG from roguelikes or other genres often revolves around this layer (Guckelsberger, Salge, Gow & Cairns, 2017).

Game systems is a layer that is more rarely procedurally generated. Although some games use it in this layer, like Dwarf Fortress (Tarn Adams, 2006) and Elite (David Braben & Ian Bell, 1984), no roguelikes doing this have been found (Hendrikx et al. 2013). Game scenarios is a fairly common layer for roguelikes to apply PCG in. This happens in Rogue (1980, A.I. Design) and Spelunky (2008, Mossmouth). This is because game scenarios often change naturally when entire dungeons have different layouts.

Game bits is a somewhat common layer to procedurally generate within roguelikes. Applying PCG to this layer can create more variety in aesthetic aspects, and roguelikes that have a lot of visual details sometimes incorporate this. Spelunky does this, as does the game Rogue Legacy (2013, Cellar Door Games) and The Binding of Isaac (Edmund McMillen & Florian Himsl, 2011).

11

Figure 9 The Binding of Isaac with visual details using PCG.

No roguelikes, or games in general, that apply PCG to game designs or derived content has been found (Hendrikx et al. 2013). There is however some focus put into expanding PCG to be able to generate these wider parts of games (Togelius et al., 2013)

2.5.3 Game Bits in Roguelikes This section will further detail what the game bits layer contains and how it is generally applied in roguelikes.

Game bits refer to part of games that do not engage the player by themselves (Hendrikx et al., 2013). As such, they can be seen as supplementary content that indirectly affects gameplay. Game bits can be concrete if they are complete objects in the game world or abstract if they are only a part of it. Parts that belong to game bits, according to Hendrikx et al., are here categorized into the following sections:

Graphical assets. Abstract components that make up all the visual elements of objects and ​ effects in games can be considered graphical assets. These components are including, but not limited to:

● Textures. A texture is “an image that is used to provide surface covering for a 3D ​ model” (Hosseini, Peters & Shirmohammadi, 2013) and is used for most games, regardless of genre, that use 3D environments and models.

● Models. Creating 3D models is the use of computers to create images and graphics ​ that look to have three dimensions. These make up the shapes of objects in a 3D environment and are often covered with textures. Textured models can be complete objects in the game world by themselves or only a part of an object.

● Colors. Which colors objects in games have can affect which feelings that are evoked ​ in players, especially among inexperienced players (Joosten, Lankveld, & Spronck, 2010). Furthermore, certain colors often mean the same thing within genres.

12 ● Sprites. Sprites are bitmap images used to represent anything graphically in games ​ (Pile Jr, 2013). They are typically used for 2D objects and environments and have existed in games since the first arcade games (Bandeira & Walter, 2009).

● Particle effects. Small visual fragments can be used in large numbers in particle ​ systems to create particle effects. The particle system models objects with particles before simulating their movements and renders the result (Yang, Yip, & Xu, 2009). These effects can for example be used to render smoke, fire and clouds (Wenzel, 2006).

● Animation. There are several ways to implement how objects move in a game world, ​ but some procedural methods exist (Togelius, Champandard, Lanzi, Mateas, Paiva, Preuss, & Stanley, 2013). PCG can be used to blend animations of different parts of an object in realistic ways or to base an entire object’s animation completely on external factors and randomness.

All of these components make up most of what is seen by the player during gameplay. This is true for all genres of games, including roguelikes. PCG can be safely used for particle effects like rain, clouds and fog, but is rarely used for the other graphical assets, as it can lead to unexpected visuals that might hinder the gameplay in different ways. If implemented correctly, PCG could let this part of game development utilize the benefits of applying PCG.

Auditory assets. Any audio that is being played in the game belongs to this category. This ​ can include music, sound effects, ambience or voice overs. Procedural generation can be used for most types of sound (Farnell, 2017). This can be done using rule based systems where rules, that make sure the audio fulfills its purpose and doesn’t sway into random noise, are defined beforehand. The type of audio that most often applies to PCG is ambience, as these sounds can be defined to stay within the range of sounding like what they’re supposed to be while randomizing parts. This can for example be done with wind or rain. In roguelikes, audio is typically pre defined, though generating ambience can occur.

Environmental objects. When the separate graphical assets are decided upon (generated ​ or pre-made), they create objects that are placed in the game world. Objects that are environmental in nature and do not directly influence gameplay are part of the game bits layer. These are, for example, trees, other vegetation and buildings (Hendrikx et al., 2013). How these objects are put together with their graphical assets or how they’re positioned can be decided with PCG. This works better for some genres than others, depending on utilization and importance of variation of environmental objects. Roguelikes become a good fit for this, as every playthrough is supposed to be unique. One roguelike that does apply PCG for environmental objects is Spelunky (2008, Mossmouth).

13

Figure 10 Examples of Spelunky’s environment that apply PCG: rocks (1) and treasure (2).

Figure 10 shows a part of Spelunky’s environment where several graphical layers are applying PCG to create the walls and all objects on top. First, it is decided which of a list of predefined rooms that consist of tiles arranged in an 8x10 matrix that is used in a section of the level (Shaker, Togelius & Nelson, 2016). All solid tiles then recieve sprites based on PCG for their basic look. After that, additional decorative sprites are placed on top as a new layer, which can be simple rocks (1). Finally, treasure is also placed and drawn on various tiles as another layer (2). This multilevel approach to rendering graphics with PCG provides ample variation and enhances the experience of a unique environment every playthrough for the player. Note that none of the individual graphical assets of each sprite are randomized, so only how they combine for the complete tile and their arrangement with other tiles use PCG.

Behaviour. Behaviour of game objects define how and when objects move, change state or ​ interact with the environment. It is determined by both the characteristics of the object and its surroundings (Hendrikx et al., 2013). Global systems such as physics and ecosystems, along with player behaviour, are often what results in any specific object behaviour. Behaviour encompasses everything that can happen to an object, and some behaviours can be procedurally generated (Hidalgo, Camahort, Abad & Vicent, 2008). Procedurally generated behaviour can for example be used for explosions. When an object explodes, the resulting fragments’ sizes, shapes and velocity can be determined by an algorithm, thus procedurally generating that behaviour.

14 2.6 Player behaviour When discussing how players act within a game, specific terms can be used to properly separate different concepts. Below is a small vocabulary for these terms as presented by Lankoski and Björk (2015).

Components. Entities that exist within game worlds can be called several different things, ​ one of which is components. Components have values and can be manipulated in some way, be it by a player or game system (Järvinen, 2008). Objects such as walls, platforms or other boundaries also count as components, as the player or other objects can collide with them.

Actions. When values belonging to components change in some way, an action is ​ performed. These actions can, for example, result in components changing velocity, changing state, disappear or appear. Actions can be divided into player actions and component actions. Player actions are the direct results of input performed by the player, and usually affects a player controlled component. Component actions are perceived as coming from the components themselves, and are not the direct result of player input. This often applies to non-player characters and environmental objects.

Goals. Goals describe a state of individual components, a set of components or the entire ​ game that the player is trying to achieve. They can be divided into short term goals and long term goals. Short term goals might relate to components the player is currently interacting with, and can be defeating an enemy or reaching a certain position with their character. Long term goals are often milestones in the game and can be to complete a level, reach a high status with their character or simply finish the game. To reach a long term goal, a series of short term goals are typically needed to be completed first. While some goals are obligatory to complete a game, others are not, making it up to the player which short term and long term goals to take on. There can also be cases where completing the game is not a goal for the player.

Using these terms, a lot of what happens in games can easily be divided and categorized. For example, if a player character kicks a ball, we can say that a player action was performed (kicking), which resulted in a specific component (the ball) performing an action that changed its velocity (being kicked). Where the ball lands may determine the player’s short term goals. As actions are both everything the player does and everything that happens in the game world, they encompass player behaviour as a whole very well.

Knowing how players behave in a game, how they choose their actions and when their goals change, is a valuable resource for developers. This data can be used to improve decision-making when finishing or updating a game. It can, for example, be used to understand player movement (Moura, El-Nasr & Shaw, 2011), get a sense of how balanced game systems are (Kim et al., 2008) or simply to find bugs (Zoeller, 2010). Because of this, knowing which game elements or implementations that can alter player behaviour is important when deciding both what should be included in a game and how it should be created. Applying a novel mechanic in an otherwise established gameplay loop may disrupt expected player behaviour. This unexpected behaviour may greatly alter design aspects of a game, such as level design (Lankoski & Björk 2015).

15 3. Problem

A lot of research has been done in the area of PCG, but not all parts of it have received equal focus. Some choose to focus on expanding the limits of what is possible to accomplish with PCG (Togelius et al., 2013); (Smith, Othenin-Girard, Whitehead, & Wardrip-Fruin, 2012) while others only look at certain parts of existing PCG, most commonly within the game space layer (Guckelsberger, Salge, Gow & Cairns, 2017). The game bits layer, on the other hand, seems to be underresearched in this regard.

Data suggests that using PCG on a specific object that resides within the game bits can create a more aesthetically pleasing and realistic experience (Korn et al., 2017) but there is a lack of research done on if it actually affects player behaviour. Additionally, Korn et al. suggests that more similar studies are required in order to validate their findings, pointing to a desire for more knowledge in the area. As no studies have been found testing its effect on player behaviour, the next logical step would be to see if applying PCG in elements of game bits has an effect in this area. Since the chance of this effect occurring might increase with the number of parts of game bits that use PCG, this should be tested on multiple parts for a more conclusive answer. If it turns out that this doesn’t affect player behaviour, then developers can safely reap the benefits of PCG without worrying that it might alter how their players act in negative ways. If there are specific ways of implementing it that leads to change in player behaviour, then that would also help developers when creating a game where altered behaviour is the goal.

Some games apply PCG to game bits while others don’t. Deciding where to use PCG can be crucial as it can greatly alter the development time, and it can cause the player to play and think differently. Having the decision of whether or not to apply PCG in game bits be more educated would greatly benefit development of future games. This leads to the research question:

Which properties of game bits using PCG changes player behaviour?

By answering this question, it should become clear how to implement PCG in game bits no matter the desired results in regards to player behaviour.

3.1 Method This section goes over the full method of the study. In short, a game in the roguelike genre was created in two versions: one with PCG in game bits and one without. This game was used in qualitative studies where participants first played the game and then had an interview which includes stimulated recall. Mixed methods were used to collect data, before triangulation helped reach conclusions on how PCG in game bits affect player behaviour. An alternative method would have been to perform a quantitative study where participants play through the game and only fill in a survey detailing their behaviour. This would however minimize the chance of follow-up questions and limit topics to what was put in the survey beforehand. Additionally, stimulated recall would not be feasible with quantitative methods, making it difficult to conclude why behavioural differences occurred. Therefore, qualitative interviews remain the best fit. Participants were partly chosen from a comfort selection and

16 partly by letting these participants spread the study further, while making sure that different levels of previous experience with games and roguelikes were included. Principles regarding the interviewing subjects can effectively be gathered by focusing questions on the how of decision making. (Bogner, Littig & Menz, 2009). As such, the interview and stimulated recall focused on motivations behind behaviours and what it was that led to various decisions.

3.1.1 Artifact In order to test whether PCG in game bits affects player behaviour or not, a game was created and behaviours of players measured. The game had to be possible to create within the scope of this project while providing gameplay where PCG in game bits is very prevalent and makes sense. Typical gameplay of games in the roguelike genre consists of exploring PCG-based environments. This makes this genre closely connected with PCG as a whole. Top-down view is also a typical feature, which makes it relatively quick to develop as the gameplay is fully two dimensional. Because of this, the game for this project was decided to be a roguelike. Specifically, a top-down roguelike with action gameplay. This is because many recent games in the genre use this combination, for example The Binding of Isaac: Rebirth (Edmund McMillen & Florian Himsl, 2014), Wizard of Legend (Contingent99, 2018), Nuclear Throne (Vlambeer, 2015) and Dungeon Souls (Lamina Studios & Mike Studios, 2016). In order to properly capture player behaviour that would occur when playing commercial games, one goal of this game was to have a presentation similar to commercial games, rather than bare bones aesthetics and gameplay only suited for a specific study. Common aspects of games like music, sound effects, animations and a decent amount of content were therefore included. The game was developed using Unity 2019.2.0f1 (Unity Technologies, 2019), Microsoft Visual Studio Community 2017 15.9.14 (Microsoft, 2017) and version control was handled using Github Desktop 2.4.3 (Github, 2020).

Two versions of the game were created, one with PCG in game bits, “PCG version”, and one without, “normal version”. These versions existed as a single executable. By comparing how players behaved between both versions, data on how behaviour was affected was gathered. Which version participants played first was selected so that there were an equal amount of both variants. Since the game is a roguelike, PCG is applied in game space for both versions. This was done using agent-based dungeon generation to create rooms to explore. Specifically, one with a high level of stochasticity but with the limitation that the distance between all rooms are always the same and that all rooms occupy a spot on a grid. This way, dungeons get varying shapes and sizes while ensuring that nothing strange like having rooms merge can happen. This type of implementation is typically used for this purpose (creating dungeons), which makes it a better fit than generate-and-test methods or other constructive variants.

The version that includes PCG in game bits applies it to graphical assets and environmental objects. Behaviour and auditory assets were not included since development time was limited, since visual assets can by themselves adjust a large part of the aesthetic experience and since there was a lack of experience when it comes to creating audio. This means that PCG wasn’t utilised to the fullest in game bits in this case. However, since many other games only implement PCG in a few parts of game bits, for example Spelunky (Mossmouth, 2013), Rogue Legacy (2013, Cellar Door Games) and The Binding of Isaac (Edmund McMillen & Florian Himsl, 2011), this makes the game closer to commercial games in its

17 implementation. The parts of game bits that apply PCG do so by selecting elements at random from set lists. This type of implementation was chosen as it was relatively quick and simple to develop. Which exact parts of the game that this affected is explained in 4.4.

3.1.2 Test procedure The qualitative tests included semi-structured interviewing and multiple sources of data. They were also performed online. The procedure also collected data for a different study, meaning that some parts of it were not relevant to this study.

The full test procedure was as follows:

1. Invite and prepare. Invitations were sent out in the form of a letter of consent. ​ This informed them that a game would be played, that interviewing and recordings occur and that they could opt out at any point during the procedure. After accepting the invitations to participate in the study, participants were sent the game itself and a call was set up using Discord (Discord inc., 2015). In this call, the participant shared their screen where the game was played. That screen share, along with all call audio, was recorded throughout the procedure. 2. Play through the tutorial. Participants were first told to enter the tutorial of the game. This consists of a short segment where the player can learn and test the controls of the game. This part contains no PCG as it's only meant to get the player used to how the game works. 3. Play through the first version. When the tutorial was finished, participants were told to begin playing a specific version of the game, without knowing what that entails. They may play the game for up to 10 minutes, or until they feel like stopping. Winning or losing in the game does not affect this. Participants are encouraged to think-aloud when playing. 4. Answer questionnaire 1. After having played the first version, participants fill an online survey which is used for a different study. 5. Play through the second version. The other version of the game is played with the exact same limitations as the first. 6. Answer questionnaire 2. Another questionnaire not related to this study is filled in. 7. Post-interview. Finally, an interview takes place that is divided into several ​ sections.

Parts 4 and 6, as well as one section of the post-interview do not relate to this study, but still takes up time in the procedure. As such, they might affect how participants go through the rest of the procedure. This risk is minimized by having the questionnaires be relatively short and not relate very closely to what is being asked in the post-interview.

3.1.3 Data gathering and analysing Throughout the procedure, data was gathered in several different ways. As such, this can be considered a use of mixed methods. Mixed methods is a technique where a new, different line of sight towards a game, concept or other object of inquiry is drawn by integrating multiple data streams (Lieberoth & Roepstorff, 2015). In total, data is gathered from gameplay recording, voice recording, observations during play, saved seed and

18 post-interview. A previous successful example of using mixed methods is by Iacovides et al. (2011, 2013) who combined observations and biometric measurements with surveys, post play-interviews and three weeks of play diaries. This was done with eight participants playing a game in a lab environment. Using these multiple streams of data allowed the researchers to understand their results from multiple angles, creating wider conclusions. As some of those data streams are the same as the ones used in this study, their success suggests that this variant of mixed methods should also work.

When considering the types of data that can be collected from gameplay sessions, they can be divided into different levels of observation, for example “in-game”, “in-room” and “in-world” (Stevens, Satwicz & McCarthy, 2008) with “in-body” being a possible extra level (Spapé, et al., 2013). In-game refers to what is happening in the application, including what it is being shown on the screen and the audio that is played. In-room is what happens to the participant during play, their movements, reactions and what they say. In-world is any element of real life that is somehow related to the play session. This could be previous experiences or consequential actions outside of the game. In-body would be properties of the body that cannot be measured by merely observing, but needs tools, such as brain activity and pulse.

Using multiple levels of observation is a large part of using mixed methods (Lieberoth & Roepstorff, 2015). Out of the different levels, in-game is most represented in this study, mostly because it is performed online and with no follow up. All visuals and audio from the game, as well as the seed used to generate the specific dungeon, is saved as data. Although in-room is impossible to fully capture without having local sessions, it is to some degree captured with voice. In-world is also gathered by having questions about previous experiences in the post-interview. In-body is not captured in this study.

The post-interview at the end of the procedure collected data in multiple ways. The goal of the interview was to find motivations behind observed behaviours, find out intended behaviour and gather data on previous experiences. Since the data is qualitative in nature, interviews were fitting, as they can give in-depth information about the topic of interest (Morgan, 1997) Firstly, questions are asked about the participant’s general experience with the game. Specifically what they thought of the game and their playing experience, if the controls were understandable and if the long term goal of the game was clear. After that, questions regarding their short term goals, their strategies to reach those goals and differences between the versions were asked. This included how their goals differed between versions, which versions they preferred and what they thought the differences between the versions were. These questions aim to gather general thoughts on their experience and the different versions before discussing specific interactions. This helped conclude how successful the game was in being experienced like a commercial game rather than an artifact.

Next, stimulated recall was used. Stimulated recall is a procedure where participants are invited to recall their thinking during an event while a video sequence of that event is shown, thus allowing cognitive processes to be investigated (Lyle, 2003). In this case, the video sequence was the recording of their gameplay, and specific events that were chosen during their play session are brought up for discussion. They were asked what their thought process was and why they think they did what they did. This was to more closely examine reasoning

19 behind behaviours. Events are selected to be included for this section if they meet one of the following criteria:

● The participant performed actions in situations differently from how they were expected to. How they are expected to act is determined by their first ​ couple of interactions with every type of object in the game. For example, if the participant interacts with a specific enemy the first three times by running straight into it and attacking, then that behaviour becomes the expected behaviour. ● The participant reacted strongly to something. For example if the participant ​ expressed audible confusion or surprise.

Finally, questions about the participant’s previous experiences were asked. These specifically asked how many hours per day they spend playing games, how familiar they are with the roguelike genre and how much they play roguelikes in terms of a weekly basis and a session by session basis. This section was included for the potential that there could be a correlation between previous experiences with games and their behavioural changes due to PCG in game bits.

In order to properly analyse the data for each case, triangulation was used. Triangulation is a way of handling mixed methods where multiple types of data are combined to form a new theoretical or other construct (Bazeley & Kemp, 2011). The metaphor emerged from the concept of locating the position of points using trigonometry. With this, the different types of data are known corners and the final construct is the last, unknown corner that requires information from the other corners to be calculated. While the term triangulation may refer to a triangle, this is also applied when more than two methods are used. In games, this has been used in various different cases (Hoffman & Nadelson, 2009); (Anderson, et al., 2012).

For this study, triangulation is applied as data is gathered in multiple forms with each test case. The interviews are combined with participant observations (in the form of game- and audio recordings) while stimulated recall may also be considered its own method. This way, what is being said by participants in the interviews can be compared to how they actually played. Using triangulation this way helps to form conclusions on exactly what behaviour that was influenced by PCG in game bits.

20 4. Implementation

This section explains how the finished artifact works, from running the application to completing the game.

4.1 Menu The first thing participants are presented with when starting the game is the main menu. From here, they can quit the game, start a tutorial, play “Version A” (normal version) or play “Version B” (PCG version). All of these options are chosen by pressing buttons.

Figure 11 Main menu.

A pilot study was performed where the game had no tutorial. To teach the controls of the game, the participant was simply told how things work. A tutorial inside the game makes for a more natural learning experience that is more similar to commercial games and then carries over when playing both versions. Therefore, a tutorial was added to the game that shows all controls in pre defined rooms. Participants are instructed to first enter the tutorial. After completing it, they are sent back to the menu and then play the normal or PCG version, followed by the other version. Every other participant is instructed to play the normal version first, while the others play PCG version first. This ensures that both versions get played first an equal number of times. After having played both versions, the participant is told to exit the game to continue the procedure with stimulated recall and post-interview.

21 4.2 Controls and HUD During gameplay, the following actions are available for the corresponding input:

Keyboard Xbox Controller

Move WASD or Arrow keys Left stick or DPad

Attack Space A

Block Left shift Left button Table 1 Controls.

The game can be played with either a keyboard or a controller. This was made to ensure participants were able to play the game even if they had no controller available. When playing, a heads-up display (HUD) gives certain information to the player while the screen follows the player character.

Figure 12 In-game.

At all times, the player can see their health (1) and where they are in the current floor (minimap) (2). The player character moves and turns to the direction of the currently held move input, and attacks are always performed in the currently turned direction. While moving can be done in any angle if the player uses a stick, attacking can only be performed in the four cardinal directions. This is partly to create familiar gameplay as it is similar to several entries in the Legend of Zelda franchise (1986-2019, Nintendo), but also for efficiency, as it only requires drawn sprites in four directions. While attacking, the player must wait until the attack is finished before performing another action. Blocking is done by holding down the corresponding button for however long the player wants to block. Like attacking, blocking can only be done in a cardinal direction. While blocking, the player can move slowly using the move input, during which their blocked direction cannot change.

22 4.3 Progression and content The player starts out in an empty room with one or more open paths along the edges. The generated floor contains rooms that are all connected through these paths. Agent-based dungeon growing is used according to the specifications in 3.1.1. By moving along these paths, the player can navigate the floor, and the minimap updates to show where all previously visited rooms are in relation to the player.

Some rooms contain enemies. These must be defeated for all paths in the room to become available. The player character attacks enemies repeatedly to deplete their health points and defeat them, and enemies can attack the player character, lowering the current health. If the player character is blocking towards an enemy attack, no damage is received. When the player character dies, the player is presented with a choice to restart the game or return to the main menu. When all enemies in a specific room are defeated, the next floor is generated and presented to the player. Rooms can also contain small stationary obstacles in the form of flowers and rocks. Each floor has a unique environment, and when the player completes the final floor, the game is complete and they are once again asked to either restart the game or return to the main menu.

One major goal for the game was to be experienced similarly to a commercial game by participants. This was to increase the odds that participants acted the same way they would when playing a game normally. To that end, multiple enemies were created to all work differently to create interesting gameplay. Additionally, objects in the game were visually detailed. Audio and animations for all actions were added to include most common elements of commercial games. When attacks land on enemies, visual effects accentuate the action. This provides important feedback to the player along with a typical satisfaction often connected to attacking in games. There are three enemies that the player can come across when exploring floors:

Matey. A basic enemy that’s about as big as the player and has five health points. Matey ​ always walks towards the player and attacks when close. The attack has a small range, requiring the player to be very close to get hit. As soon as the player hits a Matey, they get hit stunned and cannot attack for a moment, making them easy to defeat with little risk.

Blundercroc. Blundercrocs fire a single projectile at a range before running away to reload. ​ They have seven health points and also get hit stunned. When running away, they are almost as fast as the player, making it difficult to deal with them when grouped up with other enemies. The projectile travels straight to where the player was when shooting.

Planker. A stronger version of Matey. Plankers move slowly but have ten health points and ​ a big range on their attack. They only get hit stunned after being attacked multiple times and their attack deals three damage to the player, as opposed to one damage from Matey and Blundercroc. This makes timing blocks while getting attacks in the best strategy against Plankers.

23

Figure 13 Enemies. From left to right: Matey, Blundercroc, Planker.

These enemies are combined in various ways depending on which rooms the players enter. To further create the illusion of a commercial game, the environment changes drastically to provide new experiences when clearing the first floor. This, along with new rooms and having the Planker enemy only appear here, creates a proper sense of progression when playing.

4.4 Parts that use PCG Several parts of game bits use PCG in the PCG version. These parts are instead consistent and always the same in the normal version. The parts that use PCG in PCG version are the following:

Sprites for enemies, rocks and flowers. All of these objects are made up of several ​ smaller sprites. For example, the Matey enemy has separate sprites for its limbs and flowers have their stems, sepals and petals separated. These smaller sprites are all selected randomly from a predefined list for each individual object, thereby applying PCG to environmental objects and enemies.

Colors for enemies, rocks and flowers. A color is also selected at random for each ​ ​ sprite these objects are built from. These colors are not completely randomized, as it selects from a list of specific color schemes.

Figure 14 Enemies, flowers and rocks. PCG is applied on the right.

24 Colors for the room tiles. The visuals that make up the rooms (walls and floors) are made ​ out of tiles. A tile is “a stored digital image of the texture which is then replicated, in perspective, over a surface in the fashion of tiles” (Dungan Jr, Stenger & Sutty, 1978). Although a room may contain many tiles in total, the number of unique tiles is limited due to this replication. When generating a floor, one color scheme is randomly selected from a predefined list, which is then applied on every tile on that floor.

Figure 15 A small room. PCG is applied on the right.

Together, these elements make up most of what the player sees when playing. Only the player character, HUD and animated effects are not affected by PCG. This makes for a clear and immediate difference between the versions for players, even without using PCG for audio or behaviours. This level of usage of PCG in game bits is more than in other games like Spelunky (Mossmouth, 2013).

25 5. Evaluation

The test procedure was performed with eight participants, hereby known as “p1”-“p8”. Which version each participant started playing first along with how much they spend playing games in general and their familiarity with roguelikes is noted in Table 2.

Participant First version Time spent playing Aware of Frequency of played games every week roguelikes playing roguelikes (hours)

p1 Normal 5 Yes Never

p2 PCG 10-15 Yes Often

p3 Normal 0,25 No Never

p4 PCG Less than 1 No Occasionally

p5 Normal 12 Yes Never

p6 PCG Less than 2 No Never

p7 Normal 5 Yes Sometimes

p8 PCG 5 No Never Table 2 Participants background data.

5.1 Results Results from all methods used (gameplay recordings, interviews and stimulated recall) are here presented together while being categorised based on subject.

5.1.1 General thoughts of the game When discussing their experience with the game in general, all participants expressed a positive reaction to the visuals, while also having generally positive gameplay experiences. As the controls and objectives of the game were relatively simple, participants with little previous experience with games could still properly navigate and progress. P3 and p8 noted that the game was fun despite them considering themselves not being gamers. At the same time, some of the more experienced participants felt that the game was lacking a bit in complexity or depth. P1 would have liked some other system in place like crafting or gathering while p5 expressed a lack of secrets and p7 thought the controls were a bit too few and simple. Even so, p7 still noted that the game felt like “an early stage commercial game.” As p8 wasn’t used to the modern keyboard grip that this game applies, they weren’t able to utilize blocking and p5 found the tutorial to be excessive and would have preferred to start the game properly immediately. Some noted the difficulty of the game going up too fast, but were still able to enjoy the game as a whole.

26 5.1.2 Perceived differences When discussing actual differences between the normal and PCG versions of the game, perceptions varied a lot. While some participants concluded that differences were purely visual, others thought they were only different in ways related to gameplay and some figured both graphics and gameplay were different. P1, p3, p4, p5 and p7 all noted some graphical differences. While all of them noticed the colors of the environment differ, only some noticed that enemies also consisted of different sprites. Additionally, p4 thought enemies moved more slowly in the normal version while p5 similarly thought PCG version in general was faster. P2, p6 and p8 concluded that there were only gameplay related differences, with p2 saying “the only thing I could notice was the layouts and speeds”, referring to the normal version being faster. Both p6 and p8 thought the normal version being more difficult was the only difference.

5.1.3 Behavioural differences In general, participants showed consistent behaviour between versions when traversing the rooms. This was especially the case when it comes to approaching enemies, as strategies stayed the same at all times with a few exceptions. P1, p4 and p7 tried to attack flowers in the first version they played (normal, PCG and normal respectively) but not in the second. When recalling that behaviour, p1 said that they attacked in the first version to check for the presence of any type of gathering or crafting system while p7 felt that the flower looked different enough to probably be interactable in some way. P4 similarly expected to gather something, saying “maybe I was going to get a bottle of Rum or something to refill my HP.” When asked about why they didn’t repeat this behaviour in the second version they played, the answer for all participants was that the object still looked similar enough to be assumed to not work differently. One of the few times where approach to enemies differed between versions was with p1. In the normal version, the first version played, p1 consistently performed one attack before backing away against all enemies at times when the player character was low on health. When moving on to the PCG version however, their action sequences in equivalent situations changed as they attacked continually until enemies were defeated. During recall, they claimed this was only due to experience and having realised that continuous attacking also was the most effective defensive maneuver by the time they had started the PCG version. The perceived differences in gameplay that related to speed and difficulty among some participants still did not change how they decided to perform their player actions in common situations.

5.2 Analysis Triangulation was applied to the mixed methods, combining all individual methods to reach conclusions.

5.2.1 Commercial game sense Before conclusions are drawn regarding how participants’ behaviour differed, it needs to be analysed how well this data would transfer to commercial games. Based on how participants expressed their general reactions to the game, it would seem that the game succeeded at least somewhat in emulating the experience of a commercial game. Graphics were universally praised, while gameplay and degree of content received mostly positive feedback. Although it

27 wasn’t part of the interview, the ambience (audio) that plays during the beach environment received praise from p8 who noted that it added a lot to giving that environment a different feel compared to the forest. Having the tutorial and all information related to how to progress in the game within the game itself further made it more similar to commercial games. Since the combat was considered simple by some participants, their behavioural differences when it comes to approaching enemies might not translate fully to commercial games. At the same time, there was never a lack of feedback when engaging in combat as participants quickly adapted to the system and showed no signs of confusion.

5.2.2 Reasons for specific behaviour Behaviours were very consistent between the two versions when it comes to encountering enemies and mostly consistent when it comes to environment interaction. While some participants didn’t notice enemies looking different between versions, those that did often mentioned it during play while still acting the same. For example, p1 said “bandana does nothing, they're the same” when noticing that enemies wearing bandanas in the PCG version worked the same as those not wearing them. When recalling this reaction, they claimed that there were never any actual expectations that there would be differences, but they merely concluded that their assumption was right. This corresponds well to their actual behaviour, as no actions deviated from the expected behaviour during this encounter. Similarly, p3 said “is something supposed to be different aside from the colors?” when beginning their second attempt at the PCG version, noting that the colors of the environment had changed. During recall, they clarified that they immediately figured that only the colors had changed, while also saying that they did not adjust their strategy for traversing the area. This, again, matches their actual behaviour.

P7, having displayed similar reactions, added that there might have been some expectations in behavioral differences from enemies if their visuals matched the altered environment in the PCG version thematically. Other participants noted that objects (enemies and flowers) simply looked too similar between both versions to be assumed to work differently. This potential expectation of objects acting differently would then lead to players acting differently as a response. This means that since visuals weren’t drastically different or themed in a specific way, behaviours were indeed consistent between the version that didn’t use PCG in game bits and the version that did. However, this also points towards these specific properties being ones that would need to be included for PCG in game bits to alter behaviour.

5.2.3 Attention focus There were some perceived differences between the versions that didn’t exist, for example p4 and p5 perceiving that one version was faster than the other. These all relate to gameplay. At the same time, some didn’t notice the graphical changes. How a player perceives things to be different appears to partly depend on what they focus their attention on. For example, p2, p6 and p8 all seemed to keep focus on gameplay and progress, while not reacting to or interacting with any aesthetical elements. This could be for different reasons, as p2 was interested in the details of the combat system, while p6 and p8 reacted very little to anything not directly happening to their player character, perhaps due to relatively low experience with games. None of those three participants noticed any graphical changes between

28 versions, which suggests that any potential behavioural differences could not have been due to applying PCG in game bits. Though it could be possible that the actual differences subconsciously altered behaviour. The other participants, who either kept an open focus on the whole experience or a more narrow focus in aesthetics, didn’t seem to actually alter their behaviour from noticing differences, due to what is explained in 5.2.2.

Although their behaviours didn’t change directly due to PCG, it might be possible that the incorrect perceived differences could have appeared from the actual differences which would lead to different behaviour in the long term. However, these perceived differences seem to mostly stem from the seed they got. In other words, the specific rooms that happen to be created in their dungeons between versions created these false assumptions. For example, p8 thought the normal version (second version they played) was more difficult and explained that it felt like it had more enemy groups. Figure 16 shows the layout of rooms that p8 got when playing both versions.

Figure 16 Dungeon layout that p8 received (PCG version left, normal right)

Each square in Figure 16 represents a room. The number on each square represents which rooms they are. Each unique room has a specific shape and placement of enemies and environmental objects. Room 0 is the room where the player character starts and room 10 is the final room they need to clear to win. As can be seen on the layout of the normal version that p8 received, the number 7 appears frequently. This room has a larger group of enemies, while rooms 2, 5 and 6 have fewer enemies. Since room 7 appeared multiple times in the normal version and only rooms with few enemies in the PCG version, it can be concluded that content in the game space layer is the reason for p8 gaining their perceived differences. This false connection drawn by the player indicates that assumptions can be made when presented with repeated specific combinations. As such, this would constitute a property that could alter player behaviour, when included in game space generation alongside game bits generation. It cannot be concluded why some participants felt their second version (normal once and PCG once) was faster. It could relate to experience from the previous version, that they got smaller rooms or perhaps the changes to game bits.

29 5.2.4 Connection with previous experiences As mentioned in 5.2.3, previous experience could potentially have affected participants’ focus when playing. Previous experiences with games could also have affected their general view, as those that experienced a lack of content related to gameplay (p1, p5 and p7) were all relatively used to games and familiar with roguelikes. Since there was no significant behavioural changes between versions for any participant, no conclusions can be drawn regarding connections between it and previous experience. It can only be speculated that having at least a decent amount of previous experience along with an open focus when playing raises the chance that behaviour would change due to PCG in game bits given the right circumstances. For example, as p7 noted, if the PCG happens to generate very thematically fitting visuals, it could potentially give the expectation that components would be different, thus altering player behaviour.

5.3 Conclusions The game developed for this study consisted of two versions: one with PCG in game bits (PCG version) and one without (normal version). The game is a top-down action roguelike with 2D visuals and many components typically found in commercial games. The use of PCG in game bits did not cause any noticeable difference in player behaviour, as participants acted similarly in both versions with a few exceptions. Changes in behaviour that did occur did so because they either had experienced the game more or got vastly different dungeon layouts. There are, however, some factors to take into consideration in this, as interview data points to the possibility of altered behaviour if certain conditions are met. Fulfilling the following conditions makes it probable that behaviour would alter and thereby raising the risk of players changing their strategies and approaches along with changes in PCG in game bits:

● Changes being visually drastically different. As several participants noted, the changes present between versions were not drastic enough to expect objects to behave differently. A “Matey” (specific enemy in the game) is still expected to be a Matey even if they suddenly wear different clothes with different colors, thus resulting in similar player behaviour. If the shape or general presentation of Matey was instead clearly different, then it is probable that it would lead to different player behaviour. This does not only apply to enemies, as environmental objects might change from being known to be non-interactable to assumed to have some kind of interaction with different PCG results. ● Different visual elements matching up. The game used in this study altered color schemes and other elements randomly, with no consideration taken for how well they fit together. The color and style of enemies generally wouldn’t match thematically with what the environment happened to look like. If there was a clear match, however, this could risk players thinking there is a reason for it and expecting something new or different. ● Changes to game space matching visuals. In this case, the game generated dungeons filled with randomly selected rooms in order to emulate a typical roguelike experience. Some dungeons may end up being more difficult than others, or shaped in peculiar ways. If any of these noticeable generation results comes with easily

30 recognisable and distinct game bits generations, the player might connect the two. If this combination happens multiple times, the player might even start to expect the resulting game bits to always come with this peculiar game space generation. This would then alter the player behaviour to match these specific expectations.

The research question is the following:

Which properties of game bits using PCG changes player behaviour?

Based on the results in this study, fulfilling the above conditions would give implementations the properties that change behaviour. To summarize, the properties of game bits using PCG that changes player behaviour are the following:

● Drastic visual variation. ● Matching visuals from different elements. ● Repeated specific combinations of game bits generation and game space generation.

When deciding whether or not to include PCG in game bits for development of future games, how many of these properties that would be contained in a potential implementation needs to be considered. If one or more properties are very likely to come up due to how the game is structured, it might end up unexpectedly changing player behaviour and make for a worse experience. Thus, these properties should be avoided if PCG in game bits is not supposed to be a factor in player behaviour for a game. However, if a goal for a game is to have players react to and adjust their behaviour from different generations in game bits, then their implementation of PCG should contain these three properties to maximize the desired effect.

As this study was done on a two dimensional top down action roguelike, games that are more similar to this genre combination are more likely to be affected by these specific properties. Games with little similarity to this one could require vastly different properties. That said, for a top down action roguelike, the PCG implementation in game bits causing drastically different visuals, generating very distinct themes across different elements or having a chance for specific combinations of game space generation and game bits generation occurring repeatedly is what determines how likely it is for player behaviour to be affected.

31 6. Concluding Discussions

6.1 Summary This dissertation analyses how applying PCG in game bits can alter player behaviour. This was done by creating a game in the roguelike genre and performing playtests with interviews. The question becomes:

Which properties of game bits using PCG changes player behaviour?

Procedural content generation (PCG) can be a very efficient way of creating content for games, as algorithms can generate content faster than a human can manually make it. This has been utilized in various ways, but is very prevalent in roguelikes. Games within the roguelike genre always apply PCG to their level creation, making them good candidates to do research regarding PCG with. PCG can be applied to different parts of game content, and is occasionally used for aesthetical elements, like tree generation or fire effects. Roguelikes also apply PCG in these areas other than levels from time to time.

Separating content in games into layers makes it simple to categorize different elements. As game space is the layer containing everything directly connected to gameplay, that becomes the layer that roguelikes always use PCG in. Game bits, on the other hand, relates to graphics, audio, effects and other parts that do not directly relate to gameplay. Analysing what effects using PCG in game bits has is of interest since that layer is unexplored compared to game space. Getting to know the effects of using PCG in game bits would better allow developers to make more informed decisions. If a game has a goal to encourage a specific or consistent behaviour, then knowing which types of implementations that can negatively impact this goal is crucial.

This study consisted of creating a game in the roguelike genre and having participants play through it and be interviewed, partly using stimulated recall. The game had two versions: one with PCG in game bits and one without. It was an action roguelike in the vein of The Binding of Isaac (Edmund McMillen & Florian Himsl, 2011) with separated rooms that are placed in randomized layouts to form dungeons. Graphics were 2D and the game had a constant top down view. The version that applied PCG in game bits did so in the areas of graphical assets and environmental objects. Enemies and stationary objects had their sprites and colors randomized while the environment also had a random color scheme. All of these elements were consistent in the other version.

Data was collected in the form of audio recordings of the interview, recordings of their entire play session and by saving seeds used to examine afterwards. Participants were asked several questions regarding their general experience with the game, the differences between versions, specific interactions during their play session (stimulated recall) and background experience with games and roguelikes. How they behaved between versions was compared to see if there were any noticeable differences that occurred due to PCG in game bits.

A total of eight players participated in the study. Total procedure time became around one hour and their background with games varied greatly. Some participants didn’t notice any graphical differences between versions, while some perceived there to be other, gameplay

32 related differences. Behaviours were generally very consistent between versions, with only some participants displaying noticeable differences in their approach to objects or environments between the two versions. In all of these cases, the different behaviour was concluded to be due to experience or game space generation, rather than changes in game bits. Using the interview data, conditions for increasing the chance of affecting player behaviour could be brought forth. Having visual differences be drastic enough to make objects not clearly recognizable was one condition. Another was to have generated visual elements of individual objects match up thematically with other elements such as the environment. Finally, the third condition was to have specific combinations of generated, distinct game bits and game space elements occur repeatedly.

Fulfilling these conditions gives an implementation of PCG in game bits the properties needed to affect player behaviour. In other words, the properties of game bits using PCG that changes player behaviour are:

● Drastic visual variation. ● Matching visuals from different elements. ● Repeated specific combinations of game bits generation and game space generation.

By avoiding implementations with these properties, developers can ensure that player behaviour won’t be affected by implementing PCG in game bits, and can safely do so without risk of unpredictable or inconsistent experiences. Alternatively, developers can strive to fulfill these conditions if unpredictable behaviours are desirable for their game. Either way, being aware of these properties can help make decisions related to development.

6.2 Ethical Considerations Personal data like age, gender and occupation was not gathered from participants. The only background data that was collected related to experience playing games and familiarity with roguelikes. This was mainly because only background data directly connected to the study would be most likely to provide meaningful data, but also because the procedure was fairly long, making it untempting to add more questions than necessary. Since the interview was semi-structured in nature, additional relevant background data could be gathered to complement other data if needed. This did however remove the possibility of finding a connection between this ungathered personal data and other collected data. Not collecting this data, along with having the procedure be online with no way of seeing each other made for a high level of anonymity. This might have led participants to accept taking part when they wouldn’t have otherwise, and could have led to them feeling more safe and comfortable during the procedure.

Participants were instructed of all conditions before starting the procedure. This included that they would play a game, answer questionnaires and be interviewed, that everything would be recorded for the sake of the studies, that they would be anonymous, that the procedure would take around an hour and that they could opt out at any time with no reason given. These conditions were presented first through text when scheduling times, and then through conversation. Only when the participant responded that they accepted these terms did the procedure begin. All participants were adults.

33 The game contains a level of cartoon violence. Although there is no blood, severed limbs or gore of any kind, attacks are performed by the player character and enemies in attempts to defeat the other. This is done through having the player character stab with a knife and enemies slam a bottle, shoot a cannon ball through a blunderbuss and swing a plank. This was mainly included as it would greatly increase the familiarity of the gameplay for participants. Most, if not all, top down action roguelikes contain at least this level of violence, including The Binding of Isaac: Rebirth (Edmund McMillen & Florian Himsl, 2014), Enter the Gungeon (Dodge Roll, 2016), Nuclear Throne (Vlambeer, 2015), Wizard of Legend (Contingent99, 2018) and Dungeon Souls (Lamina Studios & Mike Studios, 2016). Other similar games, such as the top down entries from the Legend of Zelda franchise (1986-2019, Nintendo) also contain this level of violence. Since this level of violence is so common and often expected, including it in this game reduced the risk of confusion or unfamiliarity.

6.3 Discussion This game had a specific way of implementing PCG in game bits, as well as in game space. Although simple, this method of generating environmental objects and their graphical assets may not give the same results as other methods. Though it wasn’t feasible to test multiple implementations of PCG in game bits for this study, it is a factor to consider when determining how well these results carry over to other games.

Considering the entire procedure was performed online, very few technical issues arose. p6 had to delay their scheduled time by a day due to having to use a different computer, but the procedure itself worked like it was supposed to. P8 had a relatively poor internet connection. This caused the screen sharing to run at a low and inconsistent framerate, which made it slightly more difficult to determine the exact behaviour that was displayed. The ensuing interview was then used to clarify any uncertainties. In general though, the specification that participants needed to use their own computer from home worked well.

Depending on how PCG is implemented, it might give results that are very similar to not using PCG. This partly happened with p6, who happened to get environments in the PCG version that were visually very similar to the ones in the normal version. This might have been a factor in the participant not noticing any graphical differences between versions. If the generation was identical to the normal version, then individual behaviour data from this participant would not be useful, as differences in behaviour could not have been due to a nonexistent difference in game bits. That said, the result of getting a generation identical to the normal version by itself would have been interesting. In this case, there were enough differences to include the data from this participant normally.

This game is relatively simple when it comes to controls and mechanics. There are not a lot of actions to remember and the game generally doesn’t require the player to think hard or react quickly. It could be that the differences between the two versions that are noticed by players don’t lead to different behaviour because of this. As it is easy to keep full control of your character even when attention is drawn to other things like visual changes, this might hide potential different behaviours that would occur in complex, fast paced games. This game was

34 still decided to be simple in nature to make it accessible to participants with little previous experience with similar games.

Half of the participants recognized the term roguelike, but only one of them claimed to play them fairly often. This general low familiarity with roguelikes might have affected the results. Perhaps the chance is larger that behavioural differences due to PCG in game bits occur when players are all fans of the genre. It could also be the other way around, since being very familiar with typical elements of roguelikes might make players more easily aware of what exactly is happening, thereby quickly noticing what is different and what isn’t. Different levels of experience can make for vastly different expectations when trying a game for the first time.

The degree that PCG is used in game bits for this game is more than other roguelikes like Spelunky (Mossmouth, 2013). It could be that a more typical implementation of PCG in game bits is nowhere near to have an effect on player behaviour and contain generations that are rarely noticed by players. Even in this case, almost half of participants didn’t notice the graphical changes between versions, making it likely that most players don’t notice other implementations of PCG in game bits, for example in Spelunky which is detailed at 2.5.3. Of course, if a goal for a game is to have players notice PCG in game bits and adjust their behaviour accordingly, this could discourage applying PCG in game bits at all. The properties that do change player behaviour should stay the same, but the conditions might need to be fulfilled to larger degrees. However, in cases where player behaviour isn’t supposed to change from game bits, this is a very positive thing since the potential benefits of development efficiency and added replay value gained from utilizing PCG are great.

6.3 Future Work In this study, no significant change in player behaviour occurred due to applying PCG in game bits, but conditions raising the change of that happening have been established. With the resulting properties, quantitative studies can be performed using the properties as variables to gather generalised data. This would make it clear which types of PCG implementations for game bits that potentially do consistently alter player behaviour. Using that data, developers would be fairly certain if a planned implementation for a future game will alter behaviour, thus making informed decisions early in the development cycle and avoiding unforeseen consequences. Participant types and implementations could also be adjusted for future studies for broader results. This could be beneficial in the following areas:

● High experience with roguelikes. Participants for this study were mostly fairly inexperienced with roguelikes, even if some of them knew what it was. By performing future studies specifically with players well versed with the genre, better conclusions can be drawn regarding connections between experience and altered behaviour. It might be worthwhile to also perform a version of this study that only targets players who are completely unfamiliar with roguelikes. ● Complex PCG implementations. As this game’s PCG implementation was simple and straightforward, these results might only represent cases with similarly simple solutions. Complex implementations could, for example, be used to generate how grass grows alongside different terrain and in different environmental conditions.

35 Studies could be made testing these more advanced solutions to see if there are differences in results. ● Complete games. The game used in this study was developed specifically for it, while trying to have a presentation similar to a commercial game. Even so, participants did notice a lack of features or content that would be needed for it to contend with actual commercial games. Performing studies on completely finished, commercial games would give more reliable results. That wasn’t a possibility in this case, but could prove to be worthwhile future work, given enough time and resources. ● Other genres. PCG can essentially be used in game bits in any genre. Seeing how these results carry over to completely different genres would be helpful for a wider range of developers. An example of a different genre would be first person shooter. This genre has a different camera perspective and often has a big focus on shooting, making it fairly different from a typical roguelike. This could be tested on narrative focused games, to see if this complete shift in focus causes results to change.

36 References

A.I. Design (1980) Rogue [Computer Game]. PC. Epyx. ​ ​ Anderson, C.A., Gentile, D.A. and Dill, K.E. (2012) Prosocial, antisocial, and other effects of recreational video games. In: D. G. Singer and J. L. Singer, eds., Handbook of children and ​ the media. 2nd ed. Thousand Oaks: Sage, pp.249–272. ​ Angband Development Team (1990) Angband [Computer Game]. PC. Angband Development Team.

Bandeira, D. and Walter, M. (2009) Automatic sprite shading. In 2009 VIII Brazilian ​ Symposium on Games and Digital Entertainment (pp. 27-31). IEEE. ​ Bazeley, P. and Kemp, L. (2011) Mosaics, triangles, and DNA: Metaphors for integrated analysis in mixed methods research. Journal of Mixed Methods Research, 6(1), pp.55–72. ​ ​ Bethesda Game Studios (2006) Elder Scrolls IV: Oblivion [Computer Game]. PC. Bethesda ​ Softworks.

Biskup, T., Terstiege, J., Rogue, Z., Dycha, K and Dieguez, L. (1994) ADOM [Computer Game]. PC. Biskup, T.

Björk, S. and Holopainen, J. (2005) Patterns in game design. Hingham: Charles River ​ ​ Media.

Blizzard Entertainment (1997-2012) Diablo Franchise [Computer Games]. PC. Blizzard Entertainment.

Bogner, A., Littig, B. and Menz, W. (2009) Interviewing Experts. Macmillan Distribution ​ Ltd. ​ Braben, D. and Bell, I. (1984) Elite [Computer Game]. BBC Micro. Acornsoft.

Cellar Door Games (2013) Rogue Legacy [Computer Game]. PC. Cellar Door Games.

Chunsoft (1993) Torneko no Daibōken: Fushigi no Dungeon [Computer Game]. Super Famicom. Enix.

Contingent99 (2018) Wizard of Legend [Computer Game]. PC. Humble Bundle.

Cyan Worlds (1999) Myst Masterpiece Edition [Computer Game]. PC. Cyan Worlds.

Denisova, A. and Cairns, P. (2015) Adaptation in digital games: the effect of challenge adjustment on player performance and experience. In Proceedings of the 2015 Annual ​ Symposium on Computer-Human Interaction in Play (pp. 97-101). ​ Discord inc. (2015) Discord [Computer Software]. Discord inc.

Dodge Roll (2016) Enter the Gungeon [Computer Game]. PC. Devolver Digital.

37 Dungan Jr, W., Stenger, A. and Sutty, G. (1978) Texture tile considerations for raster graphics. In Proceedings of the 5th annual conference on Computer graphics and ​ interactive techniques (pp. 130-134). ​ Ebert, D. S., Musgrave, F. K., Peachy, D., Perlin, K. and Worley, S. (2002) Texturing and ​ Modeling: A Procedural Approach 3rd Ed. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Edwards, M. (2011) Algorithmic composition: computational thinking in music. Commun. ​ ACM 54, 58–67. ​ Epic Games (2014) Unreal Engine 4 [Computer Game]. Epic Games. ​ ​ Farnell, A. (2007) An introduction to procedural audio and its application in computer games. In Audio mostly conference (Vol. 23). ​ ​ Fenlason, J., Woodland, K., Thome, M. and Payne, J. (1982) Hack [Computer Game]. PC. ​ ​ From Software (2015) Bloodborne [Computer Game]. Sony Computer Entertainment. ​ ​ Gaslamp Games, Inc. (2011) Dungeons of Dredmor [Computer Game]. PC. Gaslamp Games, Inc.

Gearbox Software (2009-2019) Borderlands Franchise [Computer Games]. PC. 2K Games. ​ ​ Github (2020) Github Desktop 2.4.3 [Computer Software]. Github.

Guckelsberger, C., Salge, C., Gow, J. and Cairns, P., 2017, October. Predicting Player Experience without the Player. An Exploratory Study. In Proceedings of the Annual ​ Symposium on Computer-Human Interaction in Play (pp. 305-315). ​ Heeter, C., Lee, Y.H., Medler, B. and Magerko, B. (2011) August. Beyond player types: gaming achievement goal. In Proceedings of the 2011 ACM SIGGRAPH Symposium on Video ​ Games (pp. 43-48). ​ Hendrikx, M., Meijer, S., Van Der Velden, J. and Iosup, A. (2013) Procedural content generation for games: A survey. ACM Transactions on Multimedia Computing, ​ Communications, and Applications (TOMM), 9(1), pp.1-22. ​ Hidalgo, J.L., Camahort, E., Abad, F. and Vicent, M.J. (2008) Procedural graphics model and behavior generation. In International Conference on Computational Science (pp. ​ 106-115). Springer, Berlin, Heidelberg.

Ho, X., Tomitsch, M. and Bednarz, T. (2016) Finding design influence within roguelike games. In International Academic Conference on Meaningful Play (pp. 1-27). ​ ​ Hoffman, B. and Nadelson, L. (2009) Motivational engagement and video gaming: a mixed methods study. Educational Technology Research and Development, 58(3), pp.245–270. ​ ​ Hopoo Games (2019) Risk of Rain 2 [Computer Game]. PC. Gearbox Publishing.

38 Hosseini, M., Peters, J. and Shirmohammadi, S. (2013) Energy-budget-compliant adaptive 3D texture streaming in mobile games. In Proceedings of the 4th ACM Multimedia Systems ​ Conference (pp. 1-11). ​ Iacovides, I., Aczel, J., Scanlon, E. and Woods, W. (2011) What can breakdowns and breakthroughs tell us about learning and involvement experienced during game-play? In: 5th ​ European conference on games based learning, Athens. ​ Iacovides, I., Aczel, J., Scanlon, E. and Woods, W. (2013) Making sense of game-play: How can we examine learning and involvement? Transactions of the Digital Games Research ​ Association, 1(1), pp.1–17. ​ Järvinen, A. (2008) Games without frontiers: Theories and methods for game studies and ​ design. PhD. University of Tampere. ​ Joosten, E., Lankveld, G.V. and Spronck, P. (2010) Colors and emotions in video games. In 11th International Conference on Intelligent Games and Simulation GAME-ON (pp. 61-65). ​ Khaled, R., Nelson, M.J. and Barr, P. (2013) April. Design metaphors for procedural content generation in games. In Proceedings of the SIGCHI conference on human factors in ​ computing systems (pp. 1509-1518). ​ Kim, J.H., Gunn, D.V., Schuh, E., Phillips, B., Pagulayan, R.J. and Wixon, D. (2008) Tracking real-time user experience (TRUE): A comprehensive instrumentation solution for complex systems. In: SIGCHI conference on human factors in computing systems. Florence. ​ ​ Koeneke, R, Tood Jr., J and Willson, J. (1983) Moria [Computer Game]. PC. Koeneke, R, ​ ​ Tood Jr., J and Willson, J.

Korn, O., Blatz, M., Rees, A., Schaal, J., Schwind, V. and Görlich, D. (2017) Procedural content generation for game props? a study on the effects on user experience. Computers in ​ Entertainment (CIE), 15(2), pp.1-15. ​ Lamina Studios and Mike Studios (2016) Dungeon Souls [Computer Game]. PC. Black Shell Media.

Lankoski, P. and Björk, S. (2015) Formal analysis of gameplay. In Game research methods: ​ An overview. (pp. 23-35). ​ Lankoski, P. and Björk, S. (2015) Game research methods: An overview. Lulu. com. ​ ​ Lieberoth, A. and Roepstorff, A. (2015) Mixed methods in game research. In Game research ​ methods: An overview. (pp. 271-289). ​ Lyle, J. (2003) Stimulated recall: A report on its use in naturalistic research. British ​ Educational Research Journal, 29(6), pp.861–878. ​ McMillen, E. and Himsl, F. (2011) The Binding of Isaac [Computer Game]. PC. McMillen, E.

39 Microsoft (2017) Microsoft Visual Studio Community 2017 15.9.14 [Computer Software]. Microsoft.

Mojang (2011) Minecraft [Computer Game]. PC. Mojang. ​ ​ Morgan, D.L. (1997) Focus groups as qualitative research, 2nd ed. Thousand Oaks: Sage. ​ ​ Mossmouth (2013) Spelunky [Computer Game]. PC. Mossmouth. ​ ​ Moura, D., El-Nasr, M.S. and Shaw, C.D. (2011) Visualizing and understanding players’ behavior in video games: discovering patterns and supporting aggregation and comparison. In: ACM SIGGRAPH symposium on video games. New York: ACM. ​ ​ Netcore Games (2012) Tales of Maj’el [Computer Game]. PC. Netcore Games.

Nicalis, Inc. (2014) The Binding of Isaac: Rebirth [Computer Game]. PC. Nicalis, Inc.

Nintendo (1986-2019) The Legend of Zelda Franchise [Computer Games]. Various platforms. Nintendo.

Parker, R. (2017) The culture of permadeath: Roguelikes and terror management theory. Journal of Gaming & Virtual Worlds, 9(2), pp.123-141. ​ Pile Jr, J. (2013) 2D graphics programming for games. CRC Press. ​ ​ Procedural Arts (2005) Façade [Computer Game]. PC. Procedural Arts. ​ ​ Risi, S., Lehman, J., D'Ambrosio, D.B., Hall, R. and Stanley, K.O. (2015) Petalz: Search-based procedural content generation for the casual gamer. IEEE Transactions on Computational ​ Intelligence and AI in Games, 8(3), pp.244-255. ​ Rossignol, J. (2008) This Gaming Life: Travels in Three Cities. U. Michigan Press. ​ ​ Roth, C., Vermeulen, I., Vorderer, P. and Klimmt, C. (2012) Exploring replay value: shifts and continuities in user experiences between first and second exposure to an interactive story. Cyberpsychology, Behavior, and Social Networking, 15(7), pp.378-381. ​ ​ Shaker, N., Togelius, J. and Nelson, M.J., (2016) Procedural content generation in games. ​ ​ Switzerland: Springer International Publishing.

Smith, A.J. and Bryson, J.J. (2014) A logical approach to building dungeons: Answer set programming for hierarchical procedural content generation in roguelike games. In Proceedings of the 50th Anniversary Convention of the AISB. ​ Smith, G., Othenin-Girard, A., Whitehead, J. and Wardrip-Fruin, N. (2012) PCG-based game design: creating Endless Web. In Proceedings of the International Conference on the ​ Foundations of Digital Games (pp. 188-195). ​

40 Spapé, M.M., Kivikangas, J.M., Järvelä, S., Kosunen, I., Jacucci, G. and Ravaja, N. (2013) Keep your opponents close: social context affects EEG and fEMG linkage in a turn-based computer game. PloS One, 8(11). ​ ​ Stevens, R., Satwicz, T. and McCarthy, L. (2008) In-game, In-room, In-world: Reconnecting video gameplay to the rest of kids’ lives. In: Salen, K., ed.,The ecology of games. Cambridge: ​ MIT Press, pp.41–66 ​ Summerville, A., Snodgrass, S., Guzdial, M., Holmgård, C., Hoover, A.K., Isaksen, A., Nealen, A. and Togelius, J. (2018) Procedural content generation via machine learning (pcgml). IEEE ​ Transactions on Games, 10(3), pp.257-270. ​ Summerville, A., Snodgrass, S., Guzdial, M., Holmgård, C., Hoover, A.K., Isaksen, A., Nealen, A. and Togelius, J. (2019) Procedural content generation via machine learning (pcgml). IEEE ​ Transactions on Games, 10(3), pp.257-270. ​ Tarn Adams (2006) Dwarf Fortress [Computer Game]. PC. Bay 12 Games.

The NetHack DevTeam (1987) NetHack [Computer Game]. PC. The NetHack DevTeam.

Togelius, J., Champandard, A.J., Lanzi, P.L., Mateas, M., Paiva, A., Preuss, M. and Stanley, K.O. (2013) Procedural content generation: Goals, challenges and actionable steps. Schloss ​ Dagstuhl-Leibniz-Zentrum fuer Informatik. ​ Togelius, J., Kastbjerg, E., Schedl, D. and Yannakakis, G.N. (2011) What is procedural content generation? Mario on the borderline. In Proceedings of the 2nd international ​ workshop on procedural content generation in games (pp. 1-6). ​ Togelius, J., Yannakakis, G.N., Stanley, K.O. and Browne, C. (2011) Search-based procedural content generation: A taxonomy and survey. IEEE Transactions on Computational ​ Intelligence and AI in Games, 3(3), pp.172-186. ​ Togelius, J. and Shaker, N. (2016) The search-based approach. In Procedural Content ​ Generation in Games (pp. 17-30). Springer, Cham. ​ Tremblay, J., Torres, P.A. and Verbrugge, C. (2014) Measuring risk in stealth games. In FDG. ​ Unity Technologies (2019) Unity 2019.2.0f1 [Computer Software]. Unity Technologies. van den Hoogen, W., Braad, E. and Ijsselsteijn, W. (2014) Pressure at play: measuring player approach and avoidance behaviour through the keyboard. In 8th International Digital ​ Games Research Association (DiGRA) Conference. ​ Videocult (2017) Rain World [Computer Game]. PC. Adult Swim Publishing.

Vlambeer (2015) Nuclear Throne [Computer Game]. PC. Vlambeer.

Wenzel, C. (2006) Real-time atmospheric effects in games. In ACM SIGGRAPH 2006 ​ Courses (pp. 113-128). ​

41 Wenzel, C. (2006) Real-time atmospheric effects in games. In ACM SIGGRAPH 2006 ​ Courses (pp. 113-128). ​ Wilson, J. (1988) UMoria [Computer Game]. PC.

Yang, X., Yip, M. and Xu, X. (2009) Visual effects in computer games. Computer, 42(7), ​ ​ pp.48-56.

Zoeller, G. (2010) Development telemetry in video games projects. In: Game Developer ​ Conference 2010. ​

42