Impact of Textual Feedback on Player Retention in Endless Runner Mobile Games A thesis presented to the academic faculty in partial fulfillment of the requirement for the Degree Masters of Science in Game Science and Design in the College of Arts, Media and Design by Riddhi Padte Thesis Committee Chair: Dr. Casper Harteveld Thesis Advisor: Dr. Giovanni Troiano Northeastern University Boston, Massachusetts December 2019 1 2 Impact of Textual Feedback on Player Retention in Endless Runner Mobile Games by Riddhi Padte Abstract Player retention is key to free-to-play mobile games as their success relies heavily on players actively engaged in the game at any instant. Free-to-play games are widely played on mobile devices and endless runner mobile games, a type of game genre has been in the game industry since 2009. Endless runner games allow players to play endlessly while the game mechanic progressively becomes difficult. While such free-to-play business models aim to incentivize player retention, to date retaining players in a game for a long time remains challenging. This study investigates the use of textual feedback as a possible enhancer of player retention. I study textual feedback in free-to-play games under three different conditions, namely high-positive, ​ ​ low-positive, and no-textual-feedback. A study with 20 participants revealed that high-positive ​ ​ ​ textual feedback impact player retention in free-to-play endless runner games, while low-positive has no impact. I conclude by discussing what the results mean to player retention and provide design recommendations for designers of free-to-play mobile games. Keywords: Endless Runner Games, Free-to-play, Sentiment Analysis, Survey, Player Retention ​ Northeastern University Boston, Massachusetts December 2019 3 TABLE OF CONTENTS Acknowledgements 7 1. Introduction 8 2. Background 9 3. Methodology 15 4. Results 19 5. Discussion 25 6. Conclusion 28 References 29 Appendix A 35 4 LIST OF TABLES Table 3.1: List of words used in both versions of the endless runner game. 17 Table 4.1: Average session duration and number of players for each version ​ ​ 22 Table 4.2: Wilcoxon rank sum test results for each combination 23 5 LIST OF FIGURES Figure 2.1: Screenshots of endless runner games: Temple Run, JetPack Joyride 11 Figure 2.1: Screenshot of game-play with textual feedback (Game: Candy Crush by King) 13 Figure 3.1: Screenshots of game-play with higher positive score words 16 Figure 4.1: Bar chart of the responses to the questions “What part of free-to-play mobile games do you enjoy?” 19 Figure 4.2: Bar chart of the responses to the questions “Why do you play free-to-play mobile games?” 20 Figure 4.3: Bar chart of the responses to the questions “Do you like playing free-to-play mobile games?” 21 Figure 4.4: Bar chart of the responses to the questions “How often do you play free-to-play mobile games” 22 6 ACKNOWLEDGMENTS This thesis would not have been possible without the constant guidance from my committee members. Thank you to Giovanni Troiano for supporting me and suggesting me better ways to write my thesis. Thank you to Casper Harteveld for making sure I meet my deadlines on time and keep up with the thesis. Thank you Jennifer Gradecki for helping me shape the idea of this thesis from the very start. I would not have been able to finish my thesis without the regular feedback and guidance from all of you. Lastly, thank you Tejashree Sawadkar for being a constant encouragement throughout this process. A big thank you to my friends and family for being supportive and making sure I do this efficiently. 7 1. INTRODUCTION ​ Player Retention (PR) is key to successful video games and is being increasingly investigated in various gaming contexts. Previous work investigated how game design influences PR in first-person shooters (FPS) (Allart et al., 2016), how the aggressive attitude of gamers can ​ ​ compromise PR of other gamers in online video games (Shores et al. 2014), and how music, ​ ​ sound effects, and animations affect PR in online casual games (Andersen et al. 2011). Recently, ​ ​ Koski (Koski 2019) showed that PR is important in free-to-play mobile games, and that these have higher chances to retain their players if they make them wait a while before resuming for in-game rewards. Free-to-play games do not charge players money upfront before downloading the game and hence, the players that engage in micro-transactions heavily contribute to generate revenue. More number of active players increase the chances of making profit for the game studio and hence, retaining players becomes important for such a business model. However, to date, finding in-game elements that may increase PR remains challenging. Feedback has been an effective way to benefit learning (Azevedo and Bernard 1995). Textual ​ ​ instructions (or feedback) is often used in games to communicate status and help gamers progress through the gameplay (Green et al., 2018), and the benefits of text feedback in serious games have been demonstrated (for a review, see Johnson et al., 2015). As such, I argue that textual feedback may benefit PR in free-to-play mobile games too. Yet, the use of textual 8 feedback in free-to-play mobile games has received little attention and remains mostly under-investigated. My approach to explore the effect of textual feedback (i.e., words like “great” or “well-done” popping-up on screen during gameplay, to provide players with feedback on their gaming performances) PR involves an endless runner free-to-play mobile game. The textual feedback that I investigate in this thesis are words meant to encourage players while they are playing the game. In other words, the feedback aims to promote players to continue playing the game. The game will have three different versions with different words. The first two versions will have words with different semantic scores (e.g., “delicious” food = 5- higher positive score, “good” taste = 3- lower positive score). The third version will have no words. Using Semantic Orientation Calculator (SOCal) I categorize words into two lists: (1) higher positive score words ​ (e.g., excellent ) and (2) lower positive score words (e.g., sweet) (Taboada et al., 2016). Then, I ​ ​ ​ compare the session durations of each version and identify which version retains players the longest, to show the effect of different words on session-level retention. This thesis will benefit game companies that develop free-to-play games. Designers will also get insight into how textual feedback can be used to enhance engagement and player-retention in free-to-play games. As I have three versions of textual feedback, I use multivariate testing. Finally, I discuss how the results can be generalized to game design at-large and ethical implications to consider when designing for player retention. 2. BACKGROUND 9 This thesis investigates the use of textual feedback and its influence on PR in free-to-play mobile games. I investigate the above using sentiment analysis. Hence, I position my work in the areas of PR in games, particularly free-to-play mobile games, and sentiment analysis for textual feedback. Next, we provide background for previous work in those areas. Prominence of Games Games are widely accessible media and represent one of the most popular forms of entertainment. Jesse Schell defines games as problem-solving with a playful approach (Schell ​ 2019). Games are often used as educational tools for a variety of topics, including the learning ​ of social skills (Ducheneaut and Moore 2005), or engaging and training people in specific tasks ​ ​ like developing new knowledge or learning new skills (Breuer and Bente 2010). Through games, ​ ​ users can participate in immersive experiences and train their decision-making for real life situations. (Breuer and Bente 2010). For instance, “That Dragon Cancer” is a game that makes ​ ​ the player go through the experience of losing a family member to a terminal illness, and help users understand how to cope with their loss. Games can sometimes have lasting impact on players like in the case of “That Dragon Cancer”. In the same way, there are fleeting experiences that make a player feel good for the time its engaged in any casual game due to the game mechanics or graphics. There are a variety of games that need inference through data analytics to enhance game design or answer other game-play questions. In my thesis, I collect game data (i.e., score, time spent playing the game), to analyse player retention in endless runner free-to-play mobile games. “Endless runner game is a genre of platform game in which the ​ player character is continuously moving forward through an endless procedurally generated 10 world.” (D. Medeiros and D. Medeiros 2014) Examples of some endless runner games include ​ Canabalt (2009), Temple Run (2011) and JetPack Joyride (2011). Figure 2.1: Screenshots of endless runner games: Temple Run, JetPack Joyride Further, we will learn why Player Retention is important in free-to-play games and how it has been measured. Player Retention in Games Free-to-play games are gaining more and more popularity in the industry. The game Fortnite by Epic Games, for instance, has single-handedly generated $2.4 billion in revenue for the company (Forbes). As such, understanding and enhancing player retention is becoming key to design successful free-to-play mobile games. The success of Free-to-Play mobile games heavily rely on how many players are actively playing the game (Drachen et al., 2015 ). The more time players ​ ​ spend playing a game, the higher the chances that players will generate revenue for game 11 companies.