Difficulty influence on motivation over time in video games using survival analysis Thibault Allart, Guillaume Levieux, Michel Pierfitte, Agathe Guilloux, Stéphane Natkin
To cite this version:
Thibault Allart, Guillaume Levieux, Michel Pierfitte, Agathe Guilloux, Stéphane Natkin. Difficulty influence on motivation over time in video games using survival analysis. FDG - Foundation ofDigital Games, Aug 2017, Hyannis, United States. 10.1145/3102071.3102085. hal-02436676
HAL Id: hal-02436676 https://hal.archives-ouvertes.fr/hal-02436676 Submitted on 13 Jan 2020
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. Diiculty Influence on Motivation over Time in Video Games using Survival Analysis
ibault Allart Guillaume Levieux Michel Piere Ubiso CNAM / CEDRIC Ubiso 126 rue de Lagny 292 rue Saint Martin 126 rue de Lagny Montreuil 93100, France Paris 75003, France Montreuil 93100, France [email protected] [email protected] michel.piere@ubiso.com Agathe Guilloux Stephane Natkin UEVE-PSaclay/LaMME CNAM / CEDRIC 23 Boulevard de France 292 rue Saint Martin Evry 91037, France Paris 75003, France [email protected] [email protected]
ABSTRACT complex subjective notions, and their relationship with motivation In this paper, we study the link between diculty and player’s moti- is not obvious. In this paper, our aim is to investigate this rela- vation in two games developed by Ubiso®: Rayman®Legends and tionship, and thus help the design of more compelling interactive Tom Clancy’s e Division®. We describe a method to estimate play- experiences. ers’ diculty over time and link it’s time varying eect with players To study the motivational aspects of video games, many authors retention. Results conrm ow and self-ecacy theory. Also, for observe a recruited sample of players during a controlled experi- the rst hours of playtime, results dier between the two games. ment, sometimes even using games specically designed for the We explain that discrepancy with regard to aribution theory : study. is type of study allows A/B testing and ne-grained ob- in Rayman Legends, failure can be mainly aributed to the player servation of the players’ behavior. However, such an experimental skills, while in Tom Clancy’s e Division, avatar’s strength plays a seing has a certain cost, and cannot be maintained for a long fundamental role and can always be relatively quickly improved. period of time. However, when it comes to challenge and diculty, time seems CCS CONCEPTS to be an important parameter. Of course, the player’s knowledge and skills are changing over time and thus the absolute diculty •Mathematics of computing → Survival analysis; •Applied needs to rise with them. But even from a relative point of view, the computing → Computer games; players may not appreciate the same level of relative diculty at KEYWORDS the beginning of the game, when they totally discover the gameplay mechanics and at the end, when they combine the basic mechanics Game Analytics, Diculty, Motivation, Survival Analysis, Video to discover more advanced gameplay dynamics. e problem is that, Games from an experimental point of view, the impact of diculty over ACM Reference format: time on motivation might be very hard to observe in a controlled ibault Allart, Guillaume Levieux, Michel Piere, Agathe Guilloux, and Stephane experiment, as the players’ progression may require many hours Natkin. 2017. Diculty Inuence on Motivation over Time in Video Games of playtime. using Survival Analysis. In Proceedings of FDG’17, Hyannis, MA, USA, August Hopefully, out of laboratories, player monitoring is becoming 14-17, 2017, 6 pages. standard. Gaming hardwares are today almost always connected DOI: 10.1145/3102071.3102085 to the Internet. Game sessions can be recorded, and gathered by game companies. ese data allow game developers to get a more 1 INTRODUCTION detailed feedback on their work, and even to update their game When it comes to foster and maintain a video game player’s motiva- aer release, according to the players behavior. ese recordings tion, diculty is one of the most important parameter to carefully allow us to study players that have not been recruited, and that adjust. Authors have tried to explain the inherent appeal of video play polished gameplays for long period of times. games, and many of them consider challenge as one of its most is gives us a chance to see how motivational models behave fundamental characteristics. However, challenge or diculty are in a real-world gaming context, over large periods of time. In the rst sections of the paper, we describe how we compute Publication rights licensed to ACM. ACM acknowledges that this contribution was diculty for both games, while explaining in detail the survival authored or co-authored by an employee, contractor or aliate of a national govern- ment. As such, the Government retains a nonexclusive, royalty-free right to publish analysis model we developed for this study. en, we analyze the or reproduce this article, or to allow others to do so, for Government purposes only. link between diculty and player’s retention measured in both FDG’17, Hyannis, MA, USA games. Finally, we explain how these results may conrm motiva- © 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM. 978-1-4503-5319-9/17/08...$15.00 tional theories in the context of video games. DOI: 10.1145/3102071.3102085 FDG’17, August 14-17, 2017, Hyannis, MA, USA T. Allart et al.
2 DIFFICULTY IN VIDEO GAMES Many authors consider challenge as one of the most fundamental aspect of video games’ inherent appeal. Malone has studied young children’s motivation to play video games. He proposes three fea- tures that make computer games so captivating: challenge, curiosity and fantasy [15]. In his model, challenge is directly related to the game’s diculty and corresponds to the uncertainty for the player to reach the game’s goals. Lazzaro proposes a four factor model, where Hard Fun is related to the feeling of overcoming dicult tasks [13]. Sweetser et al see challenge as one of the most impor- tant part of their Game Flow framework [18]. e work of Sweetser et al stems from Mihaly Csikszentmihalyi’s eory of Flow [7]. Csikszentmihalyi has been trying to gure out the properties of ac- Figure 1: Estimated diculty of one Rayman Legends player tivities showing a strong, intrinsic ability to motivate. His research states that these activities provide perceived challenges, or opportu- nities for action, that stretch (neither overmatching nor underusing) this paper, we are particularly interested in the impact over time existing skills [7]. Ryan et al also studied intrinsic motivation and of diculty on motivation. First, it is to note that diculty varies apply their Self-Determination eory to video games, showing with time in a video game. e player gets beer at playing, but how enjoyment is related to the feeling of competence, which relies the game also constantly increases the required performance level. on an optimal level of challenge, and thus, to the game’s diculty Moreover, from a motivational perspective, diculty has not the [17]. Jesper Juul provided insight on how failure, and thus diculty, same impact on motivation at the beginning or at the end of the is one of the core aspects of video game enjoyment and learning game. Klimmt et al realized an experiment on a shooter game progression [10]. where player enjoyed the lowest level of diculty. ey suggest However, while everyone seems to agree on the fact that dif- that at the beginning of a game, the players’ representation of culty is a central aspect of video games, it is still very hard to the game’s diculty is not calibrated, and that they tend to like accurately describe the relationship between diculty and motiva- high amounts of positive feedback and do not realize that diculty tion in video games. First, because diculty, as well as motivation, is low. Having a low diculty at the start of the game is also are complex notions that can’t be directly measured. We can use coherent with self-ecacy theory: Albert Bandura explains that questionnaires to assess the subjective aspects of diculty and failure lowers mastery expectations, especially in the early course motivation but each measure will interrupt the game and thus this of events. If failure happens aer a strong ecacy expectation has assessment cannot be made very oen. is is even more problem- been developed, then failure should have a smaller negative impact atic if we want to assess the impact of diculty on motivation over on motivation, or even have a positive one if the players know that time, because we hardly can rely on a post experiment question- failure can be overcome with sustained eort [2]. naire to accurately evaluate the evolution of diculty throughout e inuence of diculty over time can be hard to capture in the game. It is possible to integrate such an assessment to the a controlled experiment, primarily because of the complexity and gameplay, as Constant et al did for the subjective diculty [5] cost of following players over long period of times. Hopefully, video But this kind of assessment is restricted to turn based games and games remote monitoring gives us access to the whole history of necessitates a gameplay modication. players’ behavior. In this paper, we describe a method to analyze en, we can try to estimate diculty and motivation from the such longitudinal data, and apply this method to two video games player’s actual behavior in the game. First, the player’s perfor- where players were remotely monitored during their game sessions. mance is related to the game’s diculty. If the game is harder, the player will lose more oen, and this failure ratio can be monitored. 2.1 Estimating diculty However, as detailed by Levieux et al, this measure of diculty is We estimate the diculty of a challenge from the players’ failures also linked to the player’s motivation [14]. Indeed, the player’s per- and successes for that challenge, as Levieux et al dened it [14]. Let formance can decrease both because the game is more demanding X being game variables that represent both player and challenges or because the player’s motivation has dropped, and he invests less characteristics, then we estimate the diculty as the probability to aentional resources in the game. We thus have to make rough fail knowing those characteristics. assumptions to study a game’s diculty using the player’s perfor- P(fail|X) mance: that while he is playing, he always does his best to win the game, and that if his motivation drops below a certain level, he will At this point, any statistical model can be used to model diculty. just stop playing. With this assumption, we can consider that the However, to model the diculty of a challenge, we need variables player’s performance allows us to estimate the objective diculty describing both the challenge and the player skills. Telemetric data of the game, and that the link between diculty and motivation is oen describe challenge completion, but are rarely accurate enough similar to the link between diculty and retention. to describe the player’s mastery of the game mechanics, i.e., the Many research point the importance to understand the temporal player skills. us, we need to take into account skills variability aspect of the game [12] and propose a way to model it [21]. In among players, that is, the fact that a challenge will not have the same diculty for each player. We make the assumption that Diiculty Influence on Motivation over Time in Video Games using Survival Analysis FDG’17, August 14-17, 2017, Hyannis, MA, USA skills variability is randomly distributed among players, following Various approaches have been proposed to deal with time vary- a normal distribution. Consequently, we can model a challenge ing variables [24] , time varying coecients [25] or both [16]. How- diculty and take into account this unobserved random eect by ever, none of them include both time varying variables and time using a mixed eect logistic regression. Challenge characteristics varying coecients and can also be scaled up to deal with the very are modeled as xed eects and a random intercept is to take into large amount of data available in the video game industry. each player’s skills. Once we get a model of player’s diculty associated to a given challenge, player’s characteristics and player’s 3.2 Cox model parameter estimation skill, we can compute the estimated diculty. As for many statistical methods, we search for parameters values Our diculty model allows us to estimate the game’s diculty that maximize the probability to have seen the events we observed, over time. Indeed, for each value of playtime, we know the value i.e., the model’s likelihood. Cox proportional hazard log-likelihood of game variables associated to the current challenge. us, we can is shown in equation(4)2. 1 compute the estimated diculty over playtime for each player e log-likelihood can be decomposed in two parts. e rst as illustrated on gure 1. A diculty of 0.3 means that player integral corresponds to the probability given by the model to see an probability to fail the challenge knowing his characteristics is of event occur, when this event actually occurred, while the second 30%. integral corresponds to the probability not to have seen this event before 3 SURVIVAL ANALYSIS n p 3.1 Retention 1 Õ n ¹ τ Õ `n(β) = Xi, j (t)βj (t)dNi (t) Motivation is a complex psychological construct that we cannot n i=1 0 j=0 directly observe. In this research, we estimate players’ motiva- ¹ τ p tion through retention. Indeed, we suppose that the lowest the Õ o − Yi (t) exp Xi, j (t)βj (t) dt (4) motivation, the more chances there are the player stops playing. 0 j=0 Many authors have studied the impact of design variables on retention [1, 3, 8, 9, 11, 22]. None of this research has however 3.3 Time varying variables and coecients focused on the impact of diculty on retention. Our aim here is In video games, variables are changing over time: avatar’s char- not to predict retention but rather to be able to beer describe the acteristics change as the player equips new weapons, diculty link between diculty and retention, and thus, player’s motivation. goes slowly up and sometimes peaks up or down. In equation (4) To do so, we rely on survival analysis. For an overview of survival we thus use X(t). Our model is then more accurate as we do not analysis applied to playtime measurement see [20]. Let T be a summarize X over time by using a mean for instance, but rather random variable associated to player’s total playtime. Retention take into account a more timely accurate value of X. is dened as the amount of players remaining in the game aer t Variables inuence also vary with time. For instance, self ecacy hours of playtime and is noted S(t). theory states that high levels of diculty can be more harmful at the beginning of the game. We can model this time varying link S(t) = P(T > t) (1) between diculty and motivation with time varying coecients. In our research, we estimate the link between diculty and In order to allow coecient to vary with time without increasing retention using a regression model that relies on a slightly dierent too much the computational complexity, we introduce a piecewise way to look at retention: the quit rate. it rate, noted λ(t), is constant estimator. e time interval is discretized in multiple sets, the probability that a player stops playing and never plays again with each set having its own specic coecient. ( ) [ ] aerwards, in a small interval of time, knowing that he was playing Let Il l ∈{0,L} be a partition of 0,τ , then just before. Among many authors, it was used by Chen et al to L Õ model the impact of network quality on player departure [4]. ( ) 1 ( ). βj t = βj,l (Il ) t l=1 P (t < T ≤ t + h|T > t) λ(t) = lim (2) To avoid the curse of dimensionality, we introduce a group total h→0 h variation penalty (5), forcing the model to use as few variations in It is well know that we can compute quit rate from retention and the parameters as possible. Figure 2 illustrates the eect of total vice-versa. A natural way to model the inuence of game variables variation penalty. e model without penalty (orange lines) has 9 on quit rate is to use the Cox proportional hazard model [6]. parameters. e penalty value associated with these coecients is Equation (3) shows an extend version of Cox proportional hazard shown in green. It corresponds to the additional cost related to the that includes time varying variables and time varying coecients. combination of successive coecients. Finally, using a model with total variation (in red) we get a model with only 3 parameters. p is penalty can be combined with a Lasso penalty [19] in order Õ ( ) ( ) © ( ) ( )ª λ t = λ0 t exp Xi, j t βj t ® (3) to reduce the number of game variables, excluding those that are j=1 less predictive of player retention. « ¬ 2 Ni (t) is the counting process associated to an event (player departure) and Yi (t) 1Playtime unit is hours for every gures equal 1 before the player stop playing and 0 aer FDG’17, August 14-17, 2017, Hyannis, MA, USA T. Allart et al.
estimation is thus a function of the mission id, the diculty selected by the player for this mission, and various avatar characteristics 3. In both games, the sequence of levels is not totally imposed to the player. Several levels are unlocked at specic moments, and the player may always freely choose between dierent levels, or even to play again a level he already nished. 5 RESULTS 5.1 Estimated diculty For both games we plot estimated diculty over time using a Figure 2: Illustration of Total Variation penalty. Coecient weighted kernel regression (g. 3). e area under the ROC curve without penalty (orange) and penalize (red). Penalty value (AUC) for logistic mixed regression is 80 for Rayman Legend and (green). 81 for e division. Estimated diculty in Rayman Legends starts around 15 percent and continuously increases to reach nearly 30 percent aer 30 hours of playtime. e diculty smoothly increases as the players level up. On the other hand, Tom Clancy’s e Division’s diculty quickly p L ! increases during the rst game hours, and reaches a plateau of Õ Õ kβ kTV = |βj,1| + |βj,l − βj,l−1| (5) nearly 30 percent. We can notice that diculty variation among j=1 l=2 players is higher in Rayman Legends than in Tom Clancy’s e Division, as shown by quantiles lines. As a rst step we estimate the parameters that maximize the log-likelihood while minimizing the penalty term. e choice of extra parameter λ is set by cross-validation.
TV βˆ ∈ argmin {−`n(β) + λkβ kTV} , λ > 0 (6) β ∈RL
Finally, once we get the coecients estimated with the penalty, we compute the coecients support. It consists in gathering to- gether close intervals that have similar values. en we estimate the nal coecients with this new discretization, without penalty.
4 DATA AND GAMES In this study, we use the previous model on Ubiso internal data to Figure 3: Estimated diculty on Rayman Legends and Tom estimate the link between diculty and motivation on two games Clancy’s The Division: Median, quantiles 0.25 and 0.75 for which we have access to the players’ longitudinal data. (dashes) and 0.05 and 0.95 (dots) e rst game is Rayman Legends, a platform game developed by Ubiso and released in 2013. In this game, the players has to jump between platforms to reach the end of a level, by collecting e dierences between the diculty curves can be explained as much as possible of dierent items and avoiding to be killed by in two ways. First, in Tom Clancy’s e Division, the player can dierent traps. very easily reduce the diculty of the game while he may not do In Rayman Legends, we consider that the player fails every times so in Rayman Legends. In Tom Clancy’s e Division, the diculty he quits a level without nishing it. When the players dies in the of a mission depends on the player skills, but also a lot on the level, he spawns to a previous checkpoint, but the death or spawn characteristics of his avatar. e point is that the player is always events are not tracked individually. We thus compute the diculty able to accomplish easier side missions to buy beer weapons, have as the probability that the player quits the current level. a more powerful avatar and thus lower the diculty of the next Tom Clancy’s e Division is an open world third-person shooter missions. In Rayman Legends, the player may chose between many with Role Playing Games mechanics, also developed by Ubiso and available levels, but he may not change a level’s diculty. If every released in March 2016. In this game, the player has to accomplish open level is hard, his only choice is to beat them. several missions for his avatar to get stronger, to unlock new regions Moreover, in Tom Clancy’s e Division, failure is much more of the open world and to learn about the game’s unfolding story. punitive than in Rayman Legends. When the player aempts a mis- In Tom Clancy’s e Division, we compute the diculty by con- sion, he has to walk from a shelter to the mission site, and missions sidering a failure each time the player is not able to nish a mission, are oen longer than a Rayman Level. Tom Clancy’s e Division’s either because his avatar died or because he did not manage to 3total health, damages per second for primary weapon, secondary weapon and side accomplish a specic mission goal and had to try again. Diculty arms, stamina, skillpower, electronics and rearms Diiculty Influence on Motivation over Time in Video Games using Survival Analysis FDG’17, August 14-17, 2017, Hyannis, MA, USA
Figure 4: Time-varying coecient for Rayman Legend Figure 5: Time-varying coecient for The Division slower pace may encourages players to lower their chances of fail- on the other hand, they may not like to have the diculty change ure and be beer prepared for each mission, in order to avoid being very quickly from easy to hard, for instance. e right curve on frustrated by the time lost. On the contrary, in Rayman Legends, 4 describes the eect of diculty variation on quit rate. Here, a the player can restart a level very quickly. diculty variation has to be understood as the diculty derivative over time. 5.2 Eect on retention e results show that in Rayman Legends, a high diculty vari- 5.2.1 Rayman Legends. First, we studied the eect of diculty ation in the rst hours of the game will have a negative impact on quit rate, to estimate what level of diculty fosters the high- on player retention. Once again if causality holds and given the est level of motivation in players. Results from the time varying previous results, a design recommendation could be to smoothly Cox proportional hazard model are shown on gure 4 for Rayman increase the diculty in the rst hours of the game and increase it Legends 4. much more as playtime goes up. In Rayman Legends, we can see that diculty has a negative link 5.2.2 Tom Clancy’s The Division. Figure 5 show the same analy- with quit rate. is is equivalent to saying that when they play sis on Tom Clancy’s e Division. Here also we notice than a higher harder levels, players tend to quit less oen. is link between diculty is correlated with an higher retention. However the eect diculty and retention is growing over time, being close to -0.1 at is strong even during the rst hours of the game. In the rst hours, the beginning and close to -0.8 at the end. a player with a diculty of 30% has 21% (exp(−0.2 ∗ (−0.95))) more More precisely, the diculty curve can be interpreted as follows. chance to keep playing than a player with a diculty of 10%. is Say we pick two players A and B that start their 12th hour of eect goes up to 27% aer 8 hours of playtime. playtime. Depending on their skills and the level they choose, In Tom Clancy’s e Division, diculty variation has no inuence diculty may be at 20% for player A and 40% for player B. e on player retention. Even an abrupt increase of diculty seems to diculty gap between them is 20%. On the le curve from 4 we can have no impact on player departure rate. read that the coecient value associated to diculty at 12 hours Both these results are coherent with motivational aspects of of playtime is around −0.7. en player A has seen his chances to Video Games that we described in the rst sections. First, the stop playing multiplied by exp(−0.2 ∗ (−0.7)) = 1.15 relatively to ow theory states that players tend to prefer an optimal level of player B. Reversely player B has 15% more chances to stay in the challenge. However, we do not know the sweet spot of such a game than player A. diculty level. We may postulate that diculty is balanced when If causality holds, this results mean that by increasing diculty, the player has 0.5 chances to win. If we look at gure 3, we can see especially in the late hours of the game, we can increase player that the diculty is largely distributed under 0.5 for both games. retention. Moreover, we may give an estimate of the retention gain us, it seems logical that player tend to prefer higher level of induced by this change. However, as we are not in a controlled diculty in both games. Having between 70% and 80% of chances experiment where players are forced to play at a given diculty to reach a goal might be considered as lower than what a challenging level, therefore we can not conclude to causality. Furthermore many level should be. As Malone puts it, challenge is motivating because unobserved information can aect both diculty and retention. it is a source of uncertainty, but it may not be so if the player wins is experiment shows only interesting correlations that have to be that oen. further investigated in controlled experiments to derive a causality en, results in Rayman Legends are coherent with the self ef- link. cacy theory and Klimmt’s experiment. In the beginning of the en, we study the eect of diculty variation on quit rate. game, players tend to prefer lower level of diculty, and do not Indeed, players may tend to prefer a higher level of diculty but like the strongest variation of the game’s diculty. Indeed, we can 4Note that we did not plot condence interval on gures 4 and 5 because they are postulate that players need safe practice conditions at the begin- small and negatively aect readability ning of the game to create a strong belief that they will be able to FDG’17, August 14-17, 2017, Hyannis, MA, USA T. Allart et al. beat the game’s challenges. us, failure by itself, or unpredictable beginning of Tom Clancy’s e Division. Indeed, in Tom Clancy’s e failure due to a quickly heightened level of diculty at the early Division, causes of failure can be aributed to the avatar strength, stages of the game may be harmful. But aer few hours of playtime, which is can be quickly modied. In Rayman Legends, failure is players may realize that the game is fair and believe in their own mainly due to lack of skills, and thus is much more harmful for the capacity to beat it, and be ready to appreciate more challenging player’s motivation during the rst hours of the game. and variables levels of diculty. What may be surprising at rst glance is that we really do not REFERENCES get the same results with Tom Clancy’s e Division. In this game, [1] ibault Allart, Guillaume Levieux, Michel Piere, Agathe Guilloux, and Stephane Natkin. 2016. Design Inuence on Player Retention: A Method Based players still tend to link harder levels of diculty, but this impact of on Time Varying Survival Analysis. In Proc. IEEE Computational Intelligence and diculty on retention seems to vary much less over time. Also, it Games Conf., IEEE. seems that variation of diculty over time has no impact on player [2] Albert Bandura. 1977. Self-ecacy: toward a unifying theory of behavioral change. Psychological review 84, 2 (1977), 191. retention. However, we may explain this result through Weiner’s [3] Z. Borbora, J. Srivastava, Kuo-Wei Hsu, and D. Williams. 2011. Churn Prediction eory of Aribution [23]. In Rayman Legends, the main cause of in MMORPGs Using Player Motivation eories and an Ensemble Approach. failure is the player’s skills. is cause of failure can be controlled, In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE ird Inernational Conference on Social Computing (SocialCom). but it has an internal locus of control, and does not change very [4] Kuan-Ta Chen, Polly Huang, and Chin-Laung Lei. 2009. Eect of network quality quickly over time. In Tom Clancy’s e Division, when the player on player departure behavior in online games. Parallel and Distributed Systems, IEEE Transactions on 20, 5 (2009), 593–606. fails, it is also because of his lack of skills but also because his avatar [5] omas Constant, Guillaume Levieux, Axel Buendia, and Stephane Natkin. 2017. was not strong enough. Hopefully, this later cause is external (the From Objective to Subjective Diculty Evaluation in Video Games. In 16th IFIP avatar), and can quickly be controlled by doing some side missions Conference on Human-Computer Interaction (INTERACT). Springer International Publishing, 107–127. or even more quickly by buying beer weapons. is may explain [6] D. R. Cox. 1972. Regression Models and Life-Tables. Journal of the Royal Statistical why failure seems to have a much more negative motivational Society. Series B (Methodological) 34, 2 (1972), pp. 187–220. impact on Rayman Legends than is has on Tom Clancy’s e Division. [7] Mihaly Csikszentmihalyi and Mihaly Csikzentmihaly. 1991. Flow: e psychology of optimal experience. Vol. 41. HarperPerennial New York. One last interesting thing to note in Tom Clancy’s e Division [8] omas Debeauvais, Cristina V Lopes, Nick Yee, and Nicolas Ducheneaut. 2014. is that, even though players can manipulate the game diculty Retention and progression: Seven months in World of Warcra.. In FDG. [9] Fabian Hadiji, Rafet Sifa, Anders Drachen, Christian urau, Kristian Kerst- explicitly by changing the mission’s level, or by increasing their ing, and Christian Bauckhage. 2014. Predicting player churn in the wild. In avatar level, results shows that players would like the game to be Computational intelligence and games (CIG), 2014 IEEE conference on. IEEE, 1–8. harder. is may be explained by the fact that avatar progression is [10] Jesper Juul. 2013. e art of failure: An essay on the pain of playing video games. Mit Press. not only to be interpreted in term of diculty : player may prefer [11] Jaya Kawale, Aditya Pal, and Jaideep Srivastava. 2009. Churn prediction in to have stronger avatar per se, and regret it later when the games MMORPGs: A social inuence based approach. In Computational Science and is too easy. Also, as we said in section 5.1, failure is much more Engineering, 2009. CSE’09. International Conference on, Vol. 4. IEEE, 423–428. [12] Jun H Kim, Daniel V Gunn, Eric Schuh, Bruce Phillips, Randy J Pagulayan, and punitive in Tom Clancy’s e Division, and thus, player may be Dennis Wixon. 2008. Tracking real-time user experience (TRUE): a comprehen- much more cautious when playing to avoid the frustration failure, sive instrumentation solution for complex systems. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 443–452. at the expense of a more balanced level of diculty. [13] Nicole Lazzaro. 2004. Why We Play Games: Four Keys to More Emotion Without Story. In Game Developers Conference. [14] Guillaume Levieux. 2011. Mesure de la diculte des jeux video. Ph.D. Dissertation. 6 CONCLUSION Conservatoire national des arts et metiers-CNAM. [15] omas W. Malone. 1982. Heuristics for designing enjoyable user interfaces: In this paper, we study the link between diculty and motivation Lessons from computer games. In Proceedings of the 1982 conference on Human in video games, using telemetric data from two games developed factors in computing systems. ACM, New York, NY, USA, 63–68. [16] Torben Martinussen and omas H Scheike. 2007. Dynamic regression models by Ubiso: Rayman Legends and Tom Clancy’s e Division. We for survival data. Springer Science & Business Media. dene diculty as a probability of failure, that we estimate using a [17] Richard M Ryan, C Sco Rigby, and Andrew Przybylski. 2006. e motivational pull of video games: A self-determination theory approach. Motivation and mixed eect logistic regression. We estimate the player’s motivation emotion 30, 4 (2006), 344–360. by considering that when his motivation is too low, he will stop [18] Penelope Sweetser and Peta Wyeth. 2005. GameFlow: a model for evaluating playing. We then use survival analysis with time varying variables player enjoyment in games. Computers in Entertainment (CIE) 3, 3 (2005), 3–3. [19] Robert Tibshirani. 1996. Regression shrinkage and selection via the lasso. Journal and coecients to estimate the link between diculty and the of the Royal Statistical Society. Series B (Methodological) (1996), 267–288. player’s motivation. is is a model we developed and that is fully [20] Markus Viljanen, Ani Airola, Jukka Heikkonen, and Tapio Pahikkala. 2017. explained in the rst sections. Playtime Measurement with Survival Analysis. arXiv preprint arXiv:1701.02359 (2017). Our results tend to conrm theories about player motivation. [21] Guenter Wallner. 2015. Sequential analysis of player behavior. In Proceedings First diculty is by itself an explanatory variable of player retention. of the 2015 Annual Symposium on Computer-Human Interaction in Play. ACM, 349–358. en, overall, players tend to prefer higher level of diculty. e [22] Ben George Weber, Michael John, Michael Mateas, and Arnav Jhala. 2011. Mod- ow theory does not state what the optimal level of diculty is, but eling Player Retention in Madden NFL 11.. In IAAI. in both games, probability of failure is almost always under what [23] Bernard Weiner. 2005. Motivation from an aribution perspective and the social psychology of perceived competence. Handbook of competence and motivation we may call a balanced diculty at 50% chances of failure. More (2005), 73–84. precisely, in Rayman Legends player do not want rapid changes of [24] Angela Winne and Peter Sasieni. 2003. Iterated residuals and time-varying diculty as well as a higher diculty for the rst few hours. is is covariate eects in Cox regression. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65, 2 (2003), 473–488. coherent with self ecacy theory : failure, at the beginning of the [25] Jun Yan and Jian Huang. 2012. Model Selection for Cox Models with Time- game, can harm the belief in their capacity to success. is point is Varying Coecients. Biometrics 68, 2 (2012), 419–428. central to the fact that we do not observe the same results at the