sign in sign up
<<
Home , Metroid, Metroidvania, Open world, Super Monkey Ball 2, Video game

Backtracking: An Ecological Investigation to Contextualize Rewards in

Thesis Presented

by

Jack Hart

to

The College of Arts, Media and Design

In Partial Fulfillment of the Requirements for the Degree of Master of

Science in Science and Design

Northeastern University

Boston, Massachussetts

May, 2019

1 Contents

1 Introduction 7

2 Background 10 2.1 Rewards ...... 10 2.1.1 Rewards in Games ...... 11 2.1.2 A Note On Gamification ...... 14 2.1.3 From The Designers ...... 15 2.2 Biometrics ...... 18 2.2.1 Biometrics & Rewards ...... 18 2.3 in Academia ...... 19 2.4 Review of Metroidvanias ...... 20

3 Game Development: Is your as Good as your Research De- sign? 22 3.1 initial design ...... 22 3.2 Blockvania ...... 26 3.3 Reward Design Methodology ...... 26 3.3.1 Access ...... 27 3.3.2 Facility ...... 28 3.3.3 Sustenance ...... 28 3.3.4 Glory ...... 29 3.3.5 Sensory ...... 29 3.3.6 Positive Feedback ...... 30 3.3.7 ...... 30

4 Methods 31

2 4.1 Participants ...... 31 4.2 Procedure ...... 31

5 Measures 32 5.1 Physiological ...... 32 5.2 Subjective ...... 33

6 Study Results 33

7 Study Discussion 34

8 Design Discussion 37 8.1 The Participant Experience ...... 37 8.2 Games for Research ...... 39 8.3 To What End? ...... 40

9 Study Conclusion 43

10 Appendix 50 10.1 Surveys ...... 50

3 Abstract

Rewards have always been an important part of games to motivate and increase engagement in players. In recent years, work to taxonomies and understand rewards has been made. However this work often lacks extrinsic validity due to the lack of environmental and player experience considerations. In this work I argue for a more ecological design of games to understand different aspects of rewards. I built Block- vania, a style game aimed at understanding how rewards affect player behavior through Galvanic Response (GSR), and Qualitative analysis.

Finally, I retrospectively analyze my thesis as a case study to understand where shortcomings in player experience and games user research exist. I argue for a more a holistic approach to researching games as well as caution when developing games for research. A distinction should be made between the “participant experience” and the

“player experience” and games researchers should do more work to make their studies replicable.

4 Acknowledgements

First and foremost I’d like to thank my advisor Christoffer Holmgard who not only dealt with my insanity but managed to help me mold it into a work that I’m actually happy with. I’d also like thank Alisa who is genuinely one of the greatest people I’ve ever met and also let me stand in her office and yell a lot. I’d also like to thank my cat Jeremy. Jeremy, sorry I was late feeding you sometimes. Thanks to my mom and dad who’ve supported for my entire and allowed me to do what ever I wanted which has always been great. Finally thank you to Allison who has supported me in ways that I will always find amazing and am completely in awe of.

5 For Allison, another dogshit day in suck city. Figure 1: Samus using her “morph-ball” ability

1 Introduction

It is 1994, you are playing Super for the first time. You see a a path that is much too small for Samus, the (PC), to fit in. You poke and prod to no avail. It must be something wrong with the game. Later on you upgrade Samus’s suit and are able to turn into a morph-ball, allowing for Samus to shrink significantly in size (figure: 1). This is when it dawns on you, as the player: you are able to go back to the path that was too small, use your newfound ability and access a previously impossible to reach part of the game. This is the metroidvania experience in a nutshell; hit an impossible task, upgrade the player character and backtrack to overcome an obstacle. These upgrades are some of the most direct reward experiences in games. They change the game itself leaving a lasting impact on the way the player interacts with the environment. For this reason, the metroidvania genre is ripe with potential for games researchers who are interested in developing games to investigate how rewards affect players. Rewards are a fundamental part of the metroidvania genre, and allow for an organic player experience where the amount of rewards can be rich, and the types of rewards are diverse. Rewards have been and will continue to be an essential part of the player experience in games. Rewards can be intrinsic or extrinsic motivators that reinforce desired behav-

7 ior [8]. Early examples of rewards in games are high scores in arcades or secret fighting characters in . In years since, rewards in games have evolved con- siderably including achievements, a popular form of rewards that exist outside of the game itself, or loot boxes; virtual slot with in-game items. Understanding how rewards motivate players, and the way players perceive rewards is important for academics, and recreational game designers. As Amon Rapp has critiqued in previous works, definitions of rewards are often created through expert knowledge, and not the player experience [35]. It leaves a hole in the literature where we do not know if the way we discuss rewards is indicative of the player experience. If these two parties are diametric in their thinking we as researchers need to reassess how we discuss rewards. As game designers seek to understand how to get more engagement and enjoyment from players, it is important to understand the effects of rewards in games. Rewards are not often compared to each other. Previous work in rewards research has not investigated the relationship between qualitative and physiological data. Some work such as [31] used qual- itative measures like the Intrinsic Motivation Inventory (IMI) combined with physiological work, however these questionnaires only assess overall motivation and do not specifically investigate the rewards. Efforts have been made to classify different types of rewards [35], [36], [13], [32]. To date there is little research that has investigated how these rewards affect players differently. Johnson et al. investigated the impact of the number of rewards given for a specific action [17]. Philips et al. investigated how different rewards affect overall player motivation [31]. The works mentioned above both investigated the compounding effect of multiple rewards affect participants, however they do not study if there is a measurable difference between reward types. Understanding the individual difference of rewards is important for several reasons. One, if meaningful differences can be found in individual reward types, it suggests different rewards

8 could be implemented to elicit a different player experiences. Second, if there is no meaningful different to the player when comparing difference rewards, the current rewards literature should be reassessed to create strong methodologies for creating reward frameworks. To this end, I developed a metroidvania, Blockvania, to find out if there is a measurable difference in Philips and Colleagues Revised Videogame Reward Taxonomy [32]. To assess the impact of each reward type players were asked to rank the rewards from favorite to least favorite [48]. Their survey data was compared to their galvanic skin response (GSR, sweat response) from each time they received a reward. This method was conducted to also see if the physiological response from players has a relationship with their subjective feelings towards the game. Johnson et al. has studied the physiological response to rewards with the same taxonomy [17], however they do not assess the difference between individual reward types. Investigating the measurable difference between rewards is important for several reasons. One, The Video Reward Taxonomy categorizes rewards based on how the reward impacts the player, if these rewards are not measurably different from a player perspective we should further investigate the language we use to discuss rewards in games. Second, if rewards are measurably different for players, understanding why and how they are ranked is also important. Third, I discuss the lack of reward delivery method in the literature and I aim to understand if there is a relationship between rewards and the mechanic players do to get said reward. Research Question 1: Is there a meaningful difference in player response to different rewards? Research Question 2: Is there a correlation with the physiological response to rewards and a qualitative ranking of said rewards? Previous research in this vain has often focused on either the physiological response or how rewards impact the overall motivation of a game. Research Question 3: Do specific rewards pair better with specific reward mechanisms

9 then others? After, I conclude the results and discussion of the study, I also reflect on the project itself and call into question whether this style of player experience research has any merit regardless of results. In retrospect, collecting data from participants in a usability lab seems averse to the way people recreational games. Additionally, I call researchers to be more reflective of their study methodology, the design of the games they create for research, and the generalizability of their findings.

2 Background

In this section I discuss the literary findings that I base much of my work on. I addi- tionally discuss how game designers who are not academics discuss games to get an outside perspective on the work being done here. Third, I review a collection of Metroidvania games to gain an understanding of what makes up the genre due to the lack of a formalized under- standing of the genre.

2.1 Rewards

All rewards are extrinsic motivators, that is they exist outside of the player. As early as the 1950’s we’ve recognized that extrinsic motivators can be an effective way to motivate people [43]. The majority of games reward research is interested in using rewards for gamification, “serious games” [17] or free-to-play/gacha games (F2P). Gamification, the act of using “game design elements in non-game contexts” [9] in an attempt to create a more engaging experience for the end user. A popular example of this is the use of achievements or badges to reward users’ use of the system [14]. This work is not not specifically related to, or trying to comment on free to play (F2P)

10 or gacha games. The primary reason is that the focus for these genres is for players to get rewards from spending money similarly to a slot . Therefore, the reward mechanics are not aimed at affecting player behavior (the in-game behavior) but are aimed at creating habit forming behavior that borders on addiction where players spend real money on virtual goods. With this in , I believe including games in this nature is out of scope for this work and that researchers should work to further divide “rewards for profit” and “rewards for engagement”. In recent years, attempts to understand, classify, and discuss rewards in recreational games has increased. For the sake of simplicity, I am only going to discuss rewards in the context of what Philips and colleagues refer to as “recreational games” [32] that is research that is not in serious games, educational games, or research in gamification. I am interested in how rewards work as part of a larger game system. Work in gamification and serious games is often interested in bolting-on rewards to increase engagement. This background section provides an understanding of how others have defined rewards in games and, if available, how those reward definitions have been implemented into game design.

2.1.1 Rewards in Games

In 2001 Hallford & Hallford developed a Reward Taxonomy for role-playing games (RPG’s) [13]. These rewards included access, facility, sustenance, and glory [13]. Access, is the re- ward of unlocking game content , facility includes enhancements such as upgradeable weapons or abilities, sustenance is defined as burden mitigation such as health upgrades, Glory do not affect itself, but are experiences the player takes from the game [13] [32]. In 2013 Philips and colleagues reassessed the above taxonomy. In their review, they conclude that Glory was “relatively broader than that of the other types of rewards”, and therefore revised the taxonomy [32]. Rewards of Access, Facility, and Sustenance were all re-

11 Reward Type Characterized By Access Unlocked game content Facility Avatar enhancements Sustenance Burden Mitigation Glory Non-gameplay rewards

Table 1: Hallford & Hallford Videogame Reward Types [13]

Reward Type Characterized By Taxonomy Access Unlocked game content Retained Facility Avatar enhancements Retained Sustenance Burden Mitigation Retained Glory Score systems Revised Sensory Feedback Affective visual/aural/tactile feedback Emergent Positive Feedback Flatter and praise Emergent

Table 2: Emergent, retained and Revised Reward Types [32] tained, rewards of Glory was “revised”, and two new reward types include; rewards of Sensory Feedback and Positive Feedback. Sensory Feedback includes “affective visual/aural/tactile feedback” and Positive Feedback is characterised by “Flattery and praise” [32]. Philips and colleagues then reviewed sixty recreational games all rated eighty or higher in to find if these rewards exist across genres and styles of games. They concluded that of the sixty games, all included rewards of access, facility, glory, and sensory feedback however not all games included rewards of sustenance or praise [33]. In 2018 Johnson and colleagues developed an endless runner to measure how the number of rewards received impacted the player experience [17]. They concluded that a high amount of rewards increases presence, enjoyment, and effort via subjective measurement. However, physiological measures (heart rate and galvanic skin response) resulted in insignificant re- sults. Philips, who also based their work in The Revised Videogame Reward Taxonomy found that player motivation increased when multiple rewards were received versus a single reward [31]. In contrast to The Videogame Reward Taxonomy, Wang & Sun developed eight reward

12 Figure 2: Reward Usage Classification systems based on survey analyses [47]. These included Score Systems, Developable Avatars, Granting, Resources, Achievements, Feedback Messages, Plot and Pictures, Unlocking mechanisms. Unlike iterations of the Video Game Reward Taxonomy, Wang & Sun note that their collection of rewards is incomplete, and instead generalize their findings into a Reward Usage Classification system. Rapp conducted an ethnographic study on World of (WoW) and concluded that not only is the type of reward important, but also the delivery method [36]. While, their findings is not directly related to this work, his methods and reasoning is of special interest. Rapp posits that an issue with much rewards research is that “In these works the gaze is that of the game designer, failing to address the subjective experience of the player that is receiving the reward.” He concluded with 3 different delivery styles of rewards deliveries; Flexible, Enabling, and Exchanging.

13 Reward Ascribed Values Experiential ef- Examples Game situations Type fects that make re- wards visible Enabling Agency, progres- Performance Experience Drops from sion, power and motivation points, repu- the completion enhancement, tation points, of quests, in- Possible addic- Gear, Glyphs, stances, and tion Abilities, Social raids. Access to gratifications, and group Add-ons that play, Social expose players’ recognition of performances to player’s skills the group during group play Exchanging Instrumental Increased reten- Gold, Consum- Farming, Craft- tion, Possible able (like hers, ing, Social: com- automation of minerals), Pro- mitment to the performances fessions, Dupli- guilds’ roles cate objects Flexible Appearance, so- Behavior Tier gear, Completion ciality, empow- change, Pos- legendary equip- of long erment sible escapism ment pets, chains, Battle Extremely against world difficult achieve- bosses, Comple- ments tion of mythic raids “Being in the spotlight” in a

Table 3: Classification of WoW’s Rewards [32]

2.1.2 A Note On Gamification

Originally, this work sought to divide literature of gamification and games research as separate works. However, this become increasingly messy as one understands the narrative arc of much of the literature in game rewards. Take for example of the work described on The Videogame Reward Taxonomy. This work by itself is not specifically aimed at gamification in that it is a recreational game built for research, however, they also argue that results could be used for gamification.

14 The simple albeit rarely asked question I have is why? Why are reward researchers interested in applying these frameworks to gamification? There are several answers I have hypothesized. One, and arguably the most likely, is that the researchers have an invested interest in gamification. This could be due to their research leaning closer to psychology and an interest in motivation or funding stemming from interest in work in increasing motivation in not game applications. Second, gamification offers a much more measurable source of data. Games are complex systems with an incredible amount of moving parts. It is therefore difficult to measure single phenomenons in games as well as draw meaningful conclusions from them. This second line of thinking is line with other critics of Player Experience Research and wider psychological research [30], [3] arguing for caution against operationalism and the fallacy that exists in reporting numbers because they’re significant. The fallacy lies in the idea that statistical errors stem from limitations in tools for measurement. However, given the complexity of the human psychology it would be naive to think that human behavior would be consistent across time from a single person let alone multiple people. The divide of gamification and games research is unclear and should be made more distinct because the academic aims are divergent.

2.1.3 From The Designers

It is also important to review how game designers (non academics) discuss rewards in games to gain an outside perspective. Where game researchers create games to further understand the medium or other scientific aims, designers create games for as a recreational activity for customers. Game designers are not inherently interested in the “why are games engaging” but the “what makes games engaging”. A designer may investigate the “why” but the end product is to make the “what” better. As pointed out by other authors ([35], [32]) there is a lack of literature on rewards of recreational games. Due to this lack of research I include the work of several game designers who have spoken on their expertise of rewards in

15 games. Additionally, it is common for other forms of design (architecture, product, graphic etc.) to include work of practitioners. This style of review has been documented by Sch¨on’s work in The Reflective Practitioner. While doing ethnographic research like Schon’s is out of scope of this work, it is still important to find and document how practitioners who create recreational games for a living think and design rewards into games. Travis Day, a game designer at Blizzard (creators of WoW) states that there are four important parts of creating good in-game rewards [7]; Maintaining established rewards, bal- ance deterministic and random, make everything feel powered, and be generous. Day cites the Overjustification Effect, the effect of rewards negatively motivating people [2], [45], as a common problem in creating rewards systems in games. To overcome this, he suggests build- ing systems where players can always have methods of progression in the game. If a player is engaging with your game there should always be a reward. Day’s second note is to balance deterministic and random rewards. He cites several examples from where players felt deterministic rewards were structured, while random rewards meant players felt they were being let-down after completing a hard challenge. There is importance in knowing both the up-side and down-side to both style of rewards and knowing how the player expe- rience can be adjusted. Third using rewards that are “overpowered” (i.e. a better reward than the player expects) allows players to not focus on the nitty-gritty math of “which one of these weapons has more attack” but allows the player to feel powerful when given new items. Making the distinction between items obvious create a deeper sense of reward. Last, “be generous”. Day argues that it is easier to solve a player having everything they want, then solving player frustration. Game designer Michail Katkoff lays out how games are designed for “Mid-Core” players, players who are considered to be heavily in gaming but may not have significant time to commit to playing games [19]. In his series on successful mid-core games he argues for multiple core loops in games. Games are oftentimes discussed in terms of gameplay-loops.

16 Figure 3: Core loop diagram from [16]

These loops are a collection of mechanics that are repeated multiple times throughout the game. For example, Pac-Man has a core-loop of collecting Pac-Dots while being chased by ghosts until they eat a Power-Pellet, allowing them to defeat the ghosts. These loop of being chased, and then becoming the chaser can be seen as the gameplay-loop of Pac-Man. Katkoff argues that for mobile games where players may not be able to spend long play sessions should offer multiple loops inside of the main gameplay loop. These mini-loops allow for either several small play-sessions or one longer one without disrupting the flow of the game for either play-styles. Each loop should always conclude with a reward equal to the size of the loop to motivate the player. For example, if a core-loop is made out of three smaller loops each small loop should give a small reward and then a large reward should be given to the player for the larger core-loop Jo Kim, a game designer on Rock Band, , and Online outlines effective methods for creating achievement/badges in games to create an engaging metagame. They argues for a loop where players earn points to earn rewards, and then use those rewards as

17 “viral outreach”, the 3rd part is especially important for social games [16]. Rigby,a researcher at Immersyve, a games consultancy company, defines four types of reward strategies: Verbal Rewards, Unexpected and Task-Noncontingent Rewards, Talk-Contingent, and Performance Contigent Rewards [39].

2.2 Biometrics

Biometrics are of interest for games researchers for several reasons including alternative controls [46], [44], environmental manipulation [23], [24], and user research [48], [11], [17]. Biometrics can be used as a method of measuring player emotions including engagement, motivation, and frustration (CITE). Different biometrics including eye tracking, heart rate, galvanic skin responses (GSR, ie. sweat), and electrocardiagram (EKG) are all popular methods of collecting biometric data. Galvanic Skin Response is commonly used as a measure of arousal [22], and has been shown to have a linear relationship with arousal [21]. Several researchers have used GSR to measure these emotions in games. Biometrics, in particular GSR, offers noninvasive ways to gain insightful data, not able to be gained through traditional user research methods such as observation or surveys [26].

2.2.1 Biometrics & Rewards

Physiological responses is a popular method of measuring engagement as heart rate, and galvanic skin response spikes (GSR) positively correlate with arousal. Arousal is the body’s physical response to external stimuli including an increase in heart rate, sweat, etc [38]. Rise in arousal has shown to be related with changes in emotions [38]. An underlying question in rewards literature has been how do rewards cause arousal and by what magnitutde? Duarte & Carri¸coanalyzed Average Heartbeat Rate (HBR) and Heart Rate Variability (HRV) of participants playing a whack-a-mole style game [10]. They found that their in-game rewards

18 that aimed at relaxing the player, slowed down HBR and HRV. These findings contrast other works from authors such as [37] and [17] who found the opposite (physiological responses rise as participants receive rewards), however the style of rewards largely differ. Regardless, it is worth noting that in all three works above, rewards have a significant affect on physiological responses in games.

2.3 Metroidvanias in Academia

As described in the intro, Metroidvanias are a popular genre of games typically defined as 2D, open-world games where a blend of Rewards of Access and Reward of Facility are the main motivation for players to engage with the game. The name Metroidvania, is a portmanteau of “Metroid” and “” two series that have defined the genre. This genre has been of little interest in game design research, but has been a focus of artificial intelligence (AI), and Procedural Content Generation (PCG). While the work in AI and PCG is not directly related to this work, it is important to note how academics have framed Metroidvanias as a style or genre of game. There is very little work on what sets it apart from traditional 2D Action Games, however it is clear that authors such as Cook believe there to be a difference because these games are genrified as metroidvanias. Rodriguez et al. developed an assist system for designing metroidvania games. They divide metroidvanias by their game elements concluding that they games are made out of Mechanics, Rules, elements, Objects, Characters, and Levels [40]. The main work in Metroidvanias is the work Cook, Colton, and Gow. Cook and colleagues developed AN- GELINA, an engine aimed at specifically creating metroidvanias with the help of automated platform design. Their interest in the genre was specifically the exploration component, citing it as a “ element” [5]. While no authoritative definition of the genre exists, the best definition found was made by Nielsen in their thesis on narrative games in PCG [27]:

19 Typically experienced from a 2D perspective, akin to platform games such as Super Bros (1985, ) and (1991, ), these games feature large explorable non-linear layouts. At the same time however, they are typically presented with a completely linear narrative structure.

2.4 Review of Metroidvanias

What makes metroidvania games unique from other 2D Action Platformers is the use of exploration, lack of linerarity and the use of player upgrades to enhance exploration. A classic example of this is when players come to a place where they must jump over a gap. However, upon trying they realize that they cannot jump high enough and must go a different way. Later the player may unlock the ability to have a higher jump, and therefore are able to jump over the aforementioned gap. There is no formal definition or framework for what makes up a metroidvania, even though the games industry at large makes a clear distinction between these and other 2D games. This work is not aimed at creating a comprehensive understanding of the genre, but instead points out and critiques the lack of formal process to understand the genre, despite academic work being done with metroidvanias as described above. To create a metroidvania to analyze rewards in games, I had to first understand what makes up a metroidvania. To do this I analyzed 5 metroidvanias. The first two were Castle- vania: Symphony of The Night and Super Metroid. These two were included due to the fact that they are considered the two most important games with regards to defining the genre. The other 3 games analyzed were Story, Hallow Knight“, and Ori and The Blind For- est. These games were chosen because they were the highest rated Metroidvanias according to metacritic, discounting other games in the metroid or castlvania series. Each game was played for one hour. Every time a reward was given, the reward of documented alongside the game actions that gave the rewards. These actions were then put into five emergent chal-

20 Reward Type Characterized By From Access Unlocked game content [13] Facility Avatar enhancements [13] Sustenance Burden Mitigation [13] Glory Score systems [13] , [32] Sensory Feedback Affective visual/aural/tactile feedback [32] Positive Feedback Flatter and praise [32] Narrative Story or Flavor Text Game Review Table 4: Hart Revised Videogame Reward Type Taxonomy lenge categories; Platforming Motor, Platforming Puzzle, Exploration, Enemies Multiple, and Enemies . Each of these challenges require a different type of skill the player must have to receive the reward. For Platforming Motor, the player must use fine motor function to move from platform to platform. This is juxtaposed against platforming puzzle, where moving in the environment is not difficult, instead solving the puzzle (e.g. pulling a series of switches to unlock a door) is the challenge. Enemies multiple is defined by having more then 1 enemy on screen who would be easy to defeat by themselves, however the multitude of enemies creates a more challenging experience. An Enemies Boss challenge is defined by one or two enemies who are large in stature and require multiple hits to kill. Their routines are often more complex then the enemies in Enemies Multiple. An example of this is in Super Metroid. In addition to analyzing reward mechanisms rewards were also noted. An emergent property of this review was an additional reward type not mentioned before in Hallford & Hallford or the work done by Philip and colleagues [13], [32], [33]. This reward was narrative. Specifically defined as a reward where story is given to the player. In past work narrative has been categorized as ”access“ However, this definition of story as just discounts many ways games deliver narrative. Something about Bioshock audio dialogues or letters being found in

21 Challenge Type Required Skill Platforming Motor Fine motor function Platforming Puzzle Puzzle Solving Exploration Curiosity Enemies Multiple Quick movement, avoiding death Enemies Boss Pattern recognition, quickmovement, avoiding death Table 5: Emergent Challenges in Metroidvanias also an example from the metroidvania review. For this reason I propose that Philips et al. revised taxonomy of rewards should include narrative (see Table 4).

3 Game Development: Is your Game Design as Good

as your Research Design?

To assess the legitmatcy of the Revised Videogame Reward Taxonomy developed through literature, I created Blockvania, a metroidvania style game created to test reward types in games. In this section I describe the design of the game, how rewards are distributed to players, and the reasoning a metroidvania design was implemented for this project.

3.1 initial design

When designing Blockvania one main design decision rose; how should the in-game events be ordered for the player? With five game element types and seven rewards types, there would at the very least have to be at least thirty-five events where players were given rewards if players were to experience every combination of reward and game element type. This style of design was the case because to make the comparison between GSR and a qualitative ranking, a within subject method was required. A traditional method of based in psychological research would have the participant play through sets of levels where they experience one type of reward for every type of game element (See figure 4. The participant would repeat

22 this process with different rewards until the participant has played all 35 rooms. While this style of study would have a high internal validity, it lacks a strong external validity, a crucial part of applied sciences particularly in games. This ”traditional“ psycho- logical design method is flawed on the basis that it is not how games are played in natural environments. By clearly manipulating the reward type players receive it’s highly possible that a performance bias would occur. Second, if the game and by extension the study is not created with design norms of traditional games, is the data reflective of normal play? Find- ing this balance between game and research was incredibly important so that the outcomes could be applied to recreational games. Beside the argument that metroidvanias could be powerful research tools, being transpar- ent in how games for research as designed has its own merit. There is a lack of transparency currently in the community games research in regards to making games easily downloadable or source code availability.

23 Figure 4: A Traditional Game Study Setup

Figure 5: A Metroidvania approach to Game Study

To thwart these common problems in game design research a more ecological research design was created, in which players would play a game where the rewards existed in the game naturally. To limit order bias as much as possible the game had to be open-world, this would allow for participants to engage with different sections of the game at their leisure and not be forced to interact with the game in a set order. This design choice, combined with the need for rewards to be meaningful meant that the metroidvania genre would be perfect for this game. As stated above the methods section rewards as an integral part of the design of metroidvanias.

24 Figure 6: World Map of Blockvania

25 Figure 7: Platforming Motor Room of Blockvania

3.2 Blockvania

Above, 6 is the map of Blockvania, the game developed for this research. Blockvania is a 2D metroidvania using an abstract inspired by Thomas Was Alone. Players control a small pink, neon square moving and traversing the world through lateral movement and jumping. They also come across different enemy types they must defeat through different attacks. In all, Blockvania has thirty-five rooms the player must complete to finish the game. At the core design of each room there is a challenge the player must overcome, and the reward that the player receives for each challenge. This full list of combinations can be seen in figure 6.

3.3 Reward Design Methodology

How players received rewards in-game is important for this research. A lot of the details on reward research isn’t yet clear. In particular should rewards be instantaneous for the action like sensory rewards in , a game famous for its ”juicy-feedback“ [18], or Reward items in Hallow Knight which are more focused around character abilities? While the differ- ence of these delivery styles is out of scope of this research, it is important to note that all rewards in Blockvania are delivered to the player as physical items. Blockvania was designed this way for two reasons. One, making all rewards a physical item gives the player a clear

26 Figure 8: Reward of Access motivation. Second, the player will always be aware with whether or not they’ve received the reward and therefore have a regular response. Mechanically, players enter a room and see the challenge (enemies multiple, platforming etc.) and the reward which is inaccessible without finishing the challenge (an example of this can be seen in figure 7). After players complete the challenge the player can get the reward. Each type of reward was visually distinct to communicate the difference to players. For example, the health was a plus sign, while Rewards of Facility were up arrows. In the rest of this chapter I will explain in detail the different specific rewards players were given for each item.

3.3.1 Access

Rewards of Access are defined by allowing players to access areas of the game that prior to the reward was inaccessible. For these rewards an image of a skeleton key was used to communicate what type of reward it was through the game. All Access rewards revolved around allowing the player ”access“ to parts of the map they were not able to get to. Access reward were designed in several ways including removing walls or allowing players to traverse platforms that prior to the reward would kill the player character. Areas of access could be in the same room as the reward, but also are sometimes in a different room then where the reward is. This decision is based off of the play review of metroidvania games.

27 Figure 9: Reward of Facility

Figure 10: Rewards of Sustenance

3.3.2 Facility

Rewards of Facility are a purple up arrow. Rewards of Facility gave the player additional abilities in the game. Five different abilities were given to the player: double-jump, air-dash, shrink ability, beam attack, and projectile attack. All of these rewards were inspired by abilities from the game review. The first three instances of a Reward of Facility all aided the player with traversing the environment, and the last two gave the player different styles of attacking enemies. Unlike traditional metroidvanias it should be noted that Rewards of Facility were not mixed with Rewards of Access. For example, it is traditional for a reward of Facility such as double jump, to open up new sections of the game. This choice goes against design norms of metroidvanis, however this decision was made due to the need to isolate rewards as single entities.

3.3.3 Sustenance

In Blockvania rewards of Sustenance only included health. These rewards were pink plus signs and would increase the player’s max health by one, while also giving them back any health they had lost in the game previously. Unfortunately, this does make Sustenance one

28 Figure 11: Rewards of Glory

Figure 12: Reward of Sensory of the most repetitive rewards, along with sensory feedback.

3.3.4 Glory

Rewards of Glory are often characterized by a Score System, or achievements. Systems that players can use as a metric to assess their skill at the game. Rewards of Glory were visualized as gold coins. When the player collected one of these gold coins it would fill in a UI element of the same shape. This also gave the player a small measure of how far through the game they had progressed which was helpful for usability.

3.3.5 Sensory Feedback

Sensory feedback includes visual, aural, and tactile feedback for actions. A particle system was included in Blockvania for this reason. When a player got a Reward of Sensory Feedback a particles would fly out of the player similar to a firework while a special audio

29 Figure 13: Reward of Positive Feedback

Figure 14: Reward of Narrative cue would play. The color of the particles was random as to vary the reward as much as possible.

3.3.6 Positive Feedback

Rewards of Positive Feedback is the use of flattery and praise to motivate a player. When a challenge with Positive Feedback was completed, text would appear on the screen acknowledging the specific challenge overcome (include picture of ”boss defeated!“

3.3.7 Narrative

The last reward in Blockvania was Narrative. As defined earlier in the paper Narrative rewards include ”Story or flavor text“ that gives the player more information about either the player character or the in-game world. In Blockvania, narrative rewards gave players a small narrative from the point of view of the player character. They note that this is a new

30 place to them and that they are lost.

4 Methods

Blockvania was built to study rewards in games and measure the difference between rewards. The game was built side by side with the study development so that the two would compliment each other. The study uses a mixed methods approach combining qualitative subjective ranking of rewards alongside biometric data to find the relationship between subjective experience and physiological response to games.

4.1 Participants

Fifteen participants took part in the study. All participants were over eighteen and their participation was completely voluntary. Participants were recruited through word of mouth and a event page on the universities website.

4.2 Procedure

To investigate the relationship between subjective and objective reward experience, a game, Blockvania, was developed. The entire study time ranged from forty-five to ninety minutes. Participants played the game for up to an hour during which game and physiological data was recorded. Afterwards they would answer a short survey on the . For the study, a within subjects procedure was used so that physiological data could be compared to participant survey results. Participants signed up on an online scheduling website and met the researcher at Northeastern’s Games User Research lab. Once there, they were greeted and the task was explained. Participants were asked to play the game to the best of their ability, for as long as they’d like for up to an hour. If at any point they wanted to stop, the study would end and the participant would be allowed to leave.

31 Before playing the game, straps with galvanic skin response and heart-rate sensors were attached to the participants right index and middle finger. Once the biometric sensors were set up participants played through Blockvania, exploring the game at their leisure. After finishing the game, participants answered two surveys. The first survey, asked participants to rank the in-game rewards from their favorite to least favorite [48]. To limit bias and confusion between the name of the rewards and their in-game correlation they were shown images of the rewards they received and not given textual titles for each, as this is now they experienced the rewards in game. Afterwards participants answered questions from Salmon et al.’s survey on video game player preferences in adults [41]. All surveys can be viewed in the appendix.Once participants finished answering the survey, the biometric device was removed from their hand, participants were thanked for their time and they were allowed to leave.

5 Measures

To gain meaningful information from the study setup a combination of biometric and survey results were used. In this section I go into detail to explain specifically what these measures are and how data was collected from them during the study.

5.1 Physiological

To measure the physiological response from participants, Galvanic Skin Response (GSR) is measured from sensors placed on the bottom of the index and middle finger which has shown to be an effective placement for GSR[46]. Data was collected through a Shimmer device using iMotions [15], an application that is used to collect multiple styles of physiological data in one application. Data was then exported and cleaned in R[34] a programming language built for statistical analyses. The data was then analyzed using a

32 continuous decomposition analysis (CDA) in LedaLab [1] with an event time window of 1- 5 seconds. After the data was analyzed using LedaLab, an analysis of variance would be applied to check significance in CDA GSR difference across reward types.

5.2 Subjective

Participants responded to a set of surveys to gather demographic and subjective data from their experience. The first survey, participants were asked to rank order images of the rewards they saw from favorite to least favorite in-game item. No language is used in the game to describe the different rewards therefore I forwent textual information in the survey as well. Pending significance from the CDA anova, a Kendall rank correlation would then be applied to find if there was a linear relationship between CDA and the subjective ranking results[25]. After the ranking of the surveys players answered a general assessment of game prefer- ences survey [41] this was used to determine if style of games played or amount of games played influenced the participants results.

6 Study Results

Of the fifteen participants who took part in the study thirteen finished a meaningful portion of the game and therefore had meaningful biometrics and survey data. However, due to nonresponders [29] [42], and GSR data corruption only eight participants were able to be analyzed via Continous Decomposition Analysis. Of the 188 total events (rewards received) across all eight participants, 152 (80.8%) had a significant increase in GSR (Figure 16). An Analysis of Variance on normalized GSR data showed the effect of reward type on the physiological response was insignificant (Figure 17), F(6,181) = 1.186, p = 0.316. Despite the majority of individual physiological response to

33 Figure 15: Rank Sum Data rewards having a significant signal, the difference in those signals is negligible at best. Due to the lack of significance in difference of GSR events, a statistical analysis could not be applied to compare the GSR results to those of the questionnaire. However a rank sum was calculated to see if there was a meaningful rank order across participants (Figure 15). However, this difference was also insignificant.

7 Study Discussion

Based on the above results we can conclude that there is not a meaningful linear relation- ship between a physiological response to rewards and one’s subjective ranking. Additionally, responses to rewards are not significant across participants, that is, reward preference is highly personal. While it is likely that a participant will have a rise in GSR when receiving a reward, the difference across reward types is insignificant. This could be for several reasons. One, biometric data may not be reflective of nuanced human responses.

34 Figure 16: GSR Across All Participants

Figure 17: ANOVA Results CDA SCR Reward Comparison

Johnson and colleagues found that heart rate [17], increases with a greater amount of rewards. Their SCR data was insignificant, however ”the pattern of results for EDA fits with the idea that greater rewards led to increased arousal“. However, this data is not compared with a subjective measure. While the authors measure participant motivation with the Intrinsic Motivation Inventory (IMI) and The Player Experience of Need Satisfaction Scale (PENS), there is an argument to be made that this style of questionnaire can be problematic. Both of these questionnaires are interested in an overall motivation/engagement for the entire game experience. There are simply too many confounding variables during a play session to consider the survey results reflective of changes in rewards design. There is also issue with their underlying design. Participants were either given low rewards (rewards of access), medium (access of access, facility and sustenance), and high (rewards of access, facility, sustenance, glory, praise, and sensory feedback). One could argue that this study did not

35 investigate ”diversity“ of rewards, but studied the difference between the specific collection of rewards implemented that are described above. No effort was made to counter-balance the type of rewards given to low, medium, and high conditions. Due to this, comparing results with the ones in this paper is difficult, if not impossible. The study design described above has little to none extrinsic motivation, based on the design of their study applying the conclusions to rewards that were not categorized as the above would be impossible. Additionally, it is not clear how one would compare the results of a metroidvania to an endless runner [31], [17], a whack a mole game [10], or 2 [37] all of which vary in player motivation, perspective, reward types, play time, and analysis methods. This observation calls into question how best to move forward as player researchers and how to generalize our findings. It is also worth pointing out that the framework used in [31] and [17] is the same, and the taxonomy used in this thesis is continued work from the origi- nal Videogame Reward Taxonomy. The lack of significant differences in rewards does line up with Philip and colleagues findings [31] of no change in motivation and responsiveness across reward types. Based on these similar results I recommend an investigation into if the Videogame Reward Taxonomy as an effective means of defining different rewards. If the player behavior does not change based on different rewards how do we know if players perceive there to be a difference at all? Rewards are not only made up of their inner nature as described by Hallford and Hallford [13], but also their delivery method [35], as well as when their received and how they’re received as desribed in this paper. Last, it is possible that there were not enough rewards in the game to get an accurate response from each par- ticipant. This however, raises the question of what would be enough responses versus what would be realistic for a participant to play in a given time? To account for this Blockvania would have to grow significantly and would take several hours to complete most likely over several play sessions. While I do argue that Metroidvanias are an interesting domain to re- search games, this study suggests that they are not an effective tool for physiological studies

36 given the slow pacing and learning the game requires of participants. Ancedotedly, several participants struggled with issues related to design rooted in the language of metroidvanias including opening doors with attacks, backtracking through environment, and exploring in non traditional side directions (left-to-right).

8 Design Discussion

Through this study, several questions have risen regarding player experience research and the importance of reflection on study and design methodology. In this section I describe the shortcomings not only of my own work but also previous player experience research. I argue that many player experience studies have focused too much on the statistical results while overlooking the method and design of the game and study. I believe this style of reflection and argumentation is important because it allows us as researchers to reflect on how we can do better and push the practice of games research forward.

8.1 The Participant Experience

The Theory of Affordance (specifically to design), is the notion that the actions a person can make depends on the environment they are in [12]. For example the action that one would give to a baseball bat in a dark alleyway is different then the action given to it in a baseball field. Norman defined affordances in design as

...the possible interactions between people and the environment. Some affor- dances are perceivable, others are not [28].

We as player experience researchers rarely, if ever, focus on the environmental affordances of our designs. Norman in his book The Design of Everyday Things (originally titled The Psychology of Everyday Things argues that this idea of affordance exists in all objects, that

37 objects communicate their use through their make-up [28]. This not only obviously pertains to games, but the design of scientific studies interested in human research Take for example the environment I had participants enter and ”play“ in during my study. They were sat down and explained how to play the game and the controls they would use. Then they wore two sensors on their fingers, one on the index, one on the middle finger. A small box, about the size of a large face, was then strapped to their wrist (this is where the leads from the GSR sensors go). They then sat in an office chair, with little mobility, an inability to stretch their legs due to lack of room under the desk, and they played in a cubicle while I and another researcher observed in the same room. Is this a player experience worth reporting on? The notion that we can collect data in a lab with a one-way mirror from a ”player“ who is a ”participant“ is, in retrospect, worth questioning. I as a researcher was communicating in that moment was that this was science and they were a subject that I needed to extract data from. This is far from the ideal way to play a game. This study aimed to find out the relationship between a player and rewards in the environment of a videogame, with the hopes of reporting those findings and generalizing them across the field of games research. There is not significant evidence to assume that the behavior participants elicit in a research study is indicative of how they would act in a more relaxed environment. The experience they have is not a ”player experience“ but a ”participant experience“. Even if we assume that human behavior data is meaningful and replicable, if we are measuring is not ”play“ then what are our findings applicable to? There is no check for external validity towards games, and if we, as player experience researchers, are not trying to influence how games are designed then what are we doing? This is also in part with the lack of design reflection in games research. Some of the prior research explains the style of genre of the game [31], [17], [37]. However, they do not make the games easily available to play, or make source code easily available for replication. Rarely, if ever do games researchers reflect on

38 the design of the game they’re building or even make game publicly available let alone the source code. When did games research become so different from traditional science that we threw replication away?

8.2 Games for Research

If we hypothetically are able to collect data in recreational environments where the par- ticipant is honestly playing there is still the issue of the environmental affordances of the game itself. With the lens of environmental affordances one can see how tricky designing games for research can be. The game serves a dual purpose. On one hand, it must elicit a response that the researcher can measure for the research they are conducting. On the other hand, the game to something that a participant can play and see as a full game. For example, a participant playing Blockvania reported that they were confused and did not understand the game. After the study they offhandedly mentioned this was not the type of game they would play in their off-time. While this may not be indicative of a larger population is important to gauge who your participants are. In game design, designers have a ”core audience“ a population that they assume will want to play their game. For player experience researchers who is our core audience? Does it matter, and if so when? This is the crux of the player experience issue. The balance of what is a videogame and what is scientific study is not clear cut and also not something discussed in the literature. Designing and studying games for research often times feels like it is in the middle in respects to games studies. Findings on one game are not as generalizable as theory driven work, but they also are not examples of application of game theory or research. Take Csikszentmihalyi’s theoretical work on Flow [6] the concept of focusing on a task so strongly one loses track of time. Csikszentmihalyi’s work defines a phenomenon, describes what it is and creates a theoretical model for it. On the opposite end is Chen’s game (and supporting thesis work) Flow; a game where the aim is to elicit Csikszentmihalyi’s described theory [4].

39 Both of these works are meaningful in different ways, however the takeaway from either work is simpler to apply than that of a user research creating a study around measuring flow. Let us say hypothetically I want to design a study to see if different rewards elicit different amounts of flow during the player experience. First I need to have a measure for Flow, a phenomenon that is well documented but difficult to measure (cite). I then have players play my game with different reward types, apply the measures I have decided upon, and analyze the results. The issue here is that my results are not generalizable across other games. The genre, mechanics, gameplay loops, etc. all vary from any different game and therefore give players different experiences. One would have to apply the same study design over multiple games and have meaningful results across all of them. This example is an attempt to highlight the methodological crisis player experience re- search currently finds itself in. The academic merit of measuring a phenomenon in one example is unclear at best. From an Experimental Psychological lens it is understandable that if one wants to measure a phenomenon they create a scenario (e.g. the experiment) to measure the phenomenon. There is however, no significant data or theoretical argument that I have found that argues for this style of behavioral research applies to games in a meaning- ful way. Humans, and by extension their behaviors, are constantly influx from internal and external influences, assuming their response to stimuli is a stable measure of phenomenon compounded by the complexity and wild variation in games leads to research with systematic flaws.

8.3 To What End?

This raises the question to what end? If player experience research is not replicable across games due to too many confounding variables then what is the role of games for research? Player experience research in this vein can be used in industry to improve games, but what is the academic merit in this? As stated above, one could replicate the same results

40 Figure 18: GSR Sensor on Hand

Figure 19: GSR Sensory on Hand

41 Figure 20: Desk where Participants Played Blockvania across enough games to theorize the phenomenon will happen in most games - but this is time consuming and costly. My critique of player experience research is not to argue for a nihilistic approach in which we hang up our academic hats and all join large corporations to increase the profit of free-to-play game company shareholders. Instead, this is an attempt to open up a dialogue of what player experience research is, and what we can do to make it better. Some of the practices outlined in this chapter, such as the participant experience, should be scrutinized and reevaluated. Possibly naive, but I hope that this can open up a discourse to discuss the role of games academia and what we want out of our work. Our work is not to increase engagement/profits like our corporate counter-parts, but also current player experience research is rarely, if ever framed as knowledge for knowledge’s sake; grants are not easily won this way. Investigating the fundamental role of our work and who it benefits should not be viewed as a pejorative action. Creating work that benefits marginalized people and not institutions is important [20]. In this regard my work, and the ones before it have failed. In retrospect I am not sure if there is much merit in the work of understanding the difference in rewards or by extension

42 their taxonomy.

9 Study Conclusion

This study aimed to find if there was a meaningful difference in player’s physiological response to rewards, and if there was a relationship between that and their post-task ranking of said rewards. While the results was inconclusive they do raise several important points in regards to study design, biometric methodology, and game design for research. Originally I sought out to prove that the Metroidvania genre would be an effective method to create an ecological game study. While I believe this to be the case I also found there to be several flaws surrounding the size of the game and methodologies commonly applied in player experience research. In an effort to create a more well rounded study, I believe I have shined light on where player experience researchers can improve in their methods and design. Researchers should be more reflective of the games they’re creating and the environment that game/play studies take place in. Additionally, if a researcher creates a game for research I believe they can have an ethical responsibility to at the very least, make their game playable or downloadable online, and also make their source code easily available. What should player experience research look like? In this paper I critiqued the current field of player experience research so how does one go about making it better? First, I believe we need to reevaluate what player experience research is and how the research is conducted. There is a lack of contemporary research that is fundamental ground work for games research. Additionally, by having a clear definition of what the field is, we then can argue for the importance of ”games research for games“ in which the importance lies not in how we our work influences outside domains such as Psychology, Sociology, Artificial Intelligence etc. We are first and most important, games researchers. Second, an analysis

43 of how one conducts and reports on research. Player experience research is built on the fundamentals of Experimental Psychology, which has several drawbacks particularly the lack of design analysis. The format of this thesis should double as an example of how to include design descriptions and analysis in future work. If we make games, we should talk about games. Third, replicability has become an afterthought. As described in the prior chapter it is difficult if not impossible obtain source code or even play many games researchers have created for their research. This lack of replication while a large problem can be easily solved either through individual efforts or institutional requirements. Last and most radically, we should ask ourselves who benefits from our research? Academics are not politically agnostic and neither is our work. Creating work that is inclusive and not focused on strengthening powers of systematic oppression would allow for a more diverse field of researchers and by extension more diverse work.

44 References

[1] Benedek, M., and Kaernbach, C. A continuous measure of phasic electrodermal activity. Journal of neuroscience methods 190, 1 (2010), 80–91.

[2] Boggiano, A. K., and Ruble, D. N. Competence and the overjustification effect: A developmental study. Journal of Personality and Social Psychology 37, 9 (1979), 1462.

[3] Borsboom, D. Measuring the mind: Conceptual issues in contemporary psychometrics. Cambridge University Press, 2005.

[4] Chen, J. Flow in games (and everything else). of the ACM 50, 4 (2007), 31–34.

[5] Cook, M., Colton, S., and Gow, J. Initial results from co-operative co-evolution for automated platformer design. In European Conference on the Applications of Evo- lutionary Computation (2012), Springer, pp. 194–203.

[6] Csikszentmihalyi, M., and Graef, R. Flow and the quality of experience in everyday life. Unpublished manuscript, University of Chicago (1979).

[7] Day, T. Rewards in video games, 2017.

[8] Deci, E. L., Koestner, R., and Ryan, R. M. A meta-analytic review of experi- ments examining the effects of extrinsic rewards on intrinsic motivation. Psychological bulletin 125, 6 (1999), 627.

[9] Deterding, S., Dixon, D., Khaled, R., and Nacke, L. Du game design au gamefulness: d´efinirla gamification. Sciences du jeu, 2 (2014).

[10] Duarte, L., and Carric¸o, L. Power me up!: an interactive and physiological perspective on videogames’ temporary bonus rewards. In Proceedings of the 4th Inter- national Conference on Fun and Games (2012), ACM, pp. 55–63.

45 [11] El-Nasr, M. S., and Yan, S. Visual attention in 3d video games. In Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertain- ment (2006), ACM, p. 22.

[12] Gibson, J. J. The theory of affordances. Hilldale, USA 1 (1977), 2.

[13] Hallford, N., and Hallford, J. Swords and Circuitry: A Designer’s Guide to Computer Role-Playing Games. Premier Press, 2001.

[14] Harms, J., Seitz, D., Wimmer, C., Kappel, K., and Grechenig, T. Low- cost gamification of online surveys: Improving the user experience through achievement badges. In Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play (2015), ACM, pp. 109–113.

[15] iMotions. imotions, 2019.

[16] Jo Kim, A. Meta-game design: Reward systems that drive engagement, 2010.

[17] Johnson, D., Klarkowski, M., Vella, K., Phillips, C., McEwan, M., and Watling, C. Greater rewards in videogames lead to more presence, enjoyment and effort. in Human Behavior (2018).

[18] Juul, J., and Begy, J. S. Good feedback for bad players? a preliminary study of ‘juicy’interface feedback.

[19] Katkoff, M. Mid-core part 1: Core loops, Oct 2013.

[20] Keyes, O., Hoy, J., and Drouhard, M. Human-computer insurrection.

[21] Lang, P. J. The emotion probe: studies of motivation and attention. American psychologist 50, 5 (1995), 372.

46 [22] Lang, P. J., Greenwald, M. K., Bradley, M. M., and Hamm, A. O. Looking at pictures: Affective, facial, visceral, and behavioral reactions. Psychophysiology 30, 3 (1993), 261–273.

[23] Liu, C., Agrawal, P., Sarkar, N., and Chen, S. Dynamic difficulty adjustment in computer games through real-time anxiety-based affective feedback. International Journal of Human-Computer Interaction 25, 6 (2009), 506–529.

[24] Lobel, A., Gotsis, M., Reynolds, E., Annetta, M., Engels, R. C., and Granic, I. Designing and utilizing biofeedback games for emotion regulation: The case of nevermind. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (2016), ACM, pp. 1945–1951.

[25] Martinez, H. P., Yannakakis, G. N., and Hallam, J. Don’t classify ratings of affect; rank them! IEEE transactions on affective computing 5, 3 (2014), 314–326.

[26] Mirza-Babaei, P., Long, S., Foley, E., and McAllister, G. Understanding the contribution of biometrics to games user research. In DiGRA conference (2011), Citeseer.

[27] Nielsen, B. H. Authored in procedurally generated worlds. Master’s thesis, n.d.

[28] Norman, D. The design of everyday things: Revised and expanded edition. Basic books, 2013.

[29] Ohman,¨ A. Electrodermal activity and vulnerability to schizophrenia: A review. Bio- logical Psychology 12, 2-3 (1981), 87–145.

47 [30] Pedersen, C. H., and Yannakakis, G. N. Epistemological challenges for opera- tionalism in player experience research. In Proceedings of the 7th International Confer- ence of Foundations of Digital Games (2012).

[31] Phillips, C., Johnson, D., Klarkowski, M., White, M. J., and Hides, L. The impact of rewards and trait reward responsiveness on player motivation. In Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (2018), ACM, pp. 393–404.

[32] Phillips, C., Johnson, D., and Wyeth, P. Videogame reward types. In Proceed- ings of the First International Conference on Gameful Design, Research, and Applica- tions (2013), ACM, pp. 103–106.

[33] Phillips, C., Johnson, D., Wyeth, P., Hides, L., and Klarkowski, M. Re- defining videogame reward types. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (2015), ACM, pp. 83–91.

[34] R Development Core Team. R: A Language and Environment for Statistical Com- puting. R Foundation for Statistical Computing, Vienna, Austria, 2008. ISBN 3-900051- 07-0.

[35] Rapp, A. The value of rewards: Exploring world of warcraft for gamification design. In Proceedings of the 2016 Annual Symposium on Computer-Human Interaction in Play Companion Extended Abstracts (2016), ACM, pp. 253–259.

[36] Rapp, A. From games to gamification: A classification of rewards in world of warcraft for the design of gamified systems. & Gaming 48, 3 (2017), 381–401.

[37] Ravaja, N., Saari, T., Salminen, M., Laarni, J., and Kallinen, K. Phasic emotional reactions to video game events: A psychophysiological investigation. Media Psychology 8, 4 (2006), 343–367.

48 [38] Rickard, N. S. Intense emotional responses to music: a test of the physiological arousal hypothesis. Psychology of music 32, 4 (2004), 371–388.

[39] Rigby, S. Finding the right rewards to sustain player engagement: A close-up look at the rpg, 2009.

[40] Rodrıguez, A. G., Cotta, C., and Leiva, A. J. F. An evolutionary approach to metroidvania videogame design.

[41] Salmon, J. P., Dolan, S. M., Drake, R. S., Wilson, G. C., Klein, R. M., and Eskes, G. A. A survey of video game preferences in adults: building better games for older adults. computing 21 (2017), 45–64.

[42] Simons, R. F., Losito, B. D., Rose, S. C., and MacMillan III, F. W. Electro- dermal nonresponding among college undergraduates: Temporal stability, situational specificity, and relationship rate change. Psychophysiology 20, 5 (1983), 498– 506.

[43] Skinner, B. F. Science and human behavior. No. 92904. Simon and Schuster, 1953.

[44] Smith, J. D., and Graham, T. Use of eye movements for video game control. In Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology (2006), ACM, p. 20.

[45] Tang, S.-H., and Hall, V. C. The overjustification effect: A meta-analysis. Applied Cognitive Psychology 9, 5 (1995), 365–404.

[46] van Dooren, M., Janssen, J. H., et al. Emotional sweating across the body: Comparing 16 different skin conductance measurement locations. Physiology & behavior 106, 2 (2012), 298–304.

49 [47] Wang, H., and Sun, C.-T. Game reward systems: Gaming experiences and social meanings. In DiGRA Conference (2011).

[48] Yannakakis, G. N., and Hallam, J. Entertainment modeling through physiology in physical play. International Journal of Human-Computer Studies 66, 10 (2008), 741– 755.

10 Appendix

10.1 Surveys

50 51 52 53