MSc Thesis Faculty of Science Department of Information and Computing Sciences Modeling Race Track Difficulty in Racing Games Robin van der Ploeg ICA-3019934 July 8, 2015 Supervisors: dr. M. Wand M. van de Hoef, MSc Contents Abstract 1 1 Introduction 3 2 Related Work 7 3 Test Environment 11 3.1 Requirements . 11 3.2 Considered environments . 12 3.2.1 TORCS . 12 3.2.2 Speed Dreams, VDrift and other free / open source projects . 13 3.2.3 Unity / Custom-made racing game . 13 3.3 Unity implementation . 14 3.3.1 Terrain generation . 14 3.3.2 Track generation . 14 3.3.3 Racing game . 16 4 Data driven model 19 4.1 Hypothesis . 19 4.2 Model . 19 4.3 Method . 20 4.3.1 Flagging records as off-road . 21 4.3.2 Invariant position / rotation . 21 4.3.3 Cropping the data to exclude the ends . 22 4.3.4 Feeding the data into SVM . 24 4.4 Test Results . 24 4.4.1 Results . 24 4.4.2 Analysis . 27 4.4.3 Conclusion . 27 5 Analytical model 29 5.1 Hypothesis . 29 5.2 Model . 30 5.3 Implementation . 33 5.4 Test results . 35 5.4.1 Results . 35 5.4.2 Analysis . 38 i Contents Contents 5.5 Derivative of maximum velocity . 40 5.6 Subjective analysis . 40 6 Comparison of models 43 6.1 Comparison of the results . 43 6.2 Discussion and limitations of models . 44 6.2.1 Discussion of differences . 44 6.2.2 Limitations . 45 7 Conclusion 47 7.1 Future work . 48 Acknowledgments 51 Bibliography 53 ii Abstract Recent years have shown a rising popularity of procedurally generated content, such as automated level design. To ensure the player enjoys the content, game developers need to make sure it is suitably difficult. This is challenging if at all possible when the content is generated after the product has shipped. The designers need to make sure the game can automatically alter the variables that control the difficulty, depending on the performance of the player. To determine which variables used in level generation control difficulty, a difficulty model is required. We attempt to find such a model for the racing game genre. To identify what parts of the track most define the difficulty, we use two approaches. First, a data driven model, which uses machine learning to recognize difficult sections on the track. Second, an analytical model that attempts to predict where cars are most likely to lose traction, following the rules of physics. Using a custom-made racing game, our methods are tested empirically through player testing on various procedurally generated racetracks. Results show that while we can not perfectly predict all difficult sections of a race- track, crashes can indeed be predicted with above-average accuracy (over 60%) using simple algorithms, with relatively sparse data. The varying level of player perfor- mance is identified as one of the most influential reasons why accurate predictions are very hard to achieve. Further analysis of the data suggests some increased accuracy may potentially be achieved with slightly altered approaches. Our exploratory work helps game developers identify at least the most problematic sections of tracks. We also believe it can be used as a foundation upon which further work can be based. 1 1 Introduction Procedural Content Generation Interest in Procedural Content Generation (PCG) has risen with recent developments in the game industry [23]. Cited as influential changes are improving hardware and rapidly increasing costs for big productions.1 Content generation algorithms, such as those for generating levels, can keep devel- opment costs lower by reducing workload [12]. Many varying methods and implementations of PCG can be found, both in literature and in practice. Some areas of game development have received relatively little attention, such as system and world design, or story. Others, including smaller bits of games (such as vegetation or textures) and terrain have been adopted quite rapidly in recent years [10]. Indie games are most frequently produced by small studios with very few level designers, who use these techniques at runtime to increase replayability. After the introduction of Minecraft in 2009, many “sandbox” and exploration games featuring procedurally generated worlds have been developed, such as Terraria, Cube World and Starbound. Most of these games provide gameplay mechanics that can overcome any issues or unexpected side effects of the algorithm. The terrain might contain mountains that can not be crossed, but the game then allows players to build over or tunnel through the mountain instead. Bigger game studios may also use similar techniques, but are more likely to employ more advanced graphics. This leads to more complicated algorithms in order to avoid game-breaking corner cases, which take more time to develop. Racing games One specific game genre which has received little attention in terms of procedural generation is racing games. While racing games have been around for decades, only a select few contain any obvious form of PCG. Gran Turismo 5 contains a feature where the game generates a track based on a few settings. While Fuel contains an open world which is generated through procedural techniques, it is not “randomly” generated and will be the same every time. This means it is mostly used as a compression technique, rather than to add replayability. There are also some (but not many) indie games that feature racing in procedurally generated worlds, such as Race The Sun and Infinity Random Race (IRR). Both of these 1C Chapple, "Can the game industry keep a lid on rising development costs?", in Develop. May 14th 2014, viewed on 11 June 2015, http://www.develop-online.net/analysis/can-the-game- industry-keep-a-lid-on-rising-development-costs/0192815 3 Chapter 1 Introduction games include game mechanics that allow the player to get around any unforeseen issues generated by the algorithms: Race The Sun is a game that is not limited to a track, and IRR allows the player to transform the car into a robot that can fly over obstacles. We believe that procedurally generated tracks can add a lot of value to racing games, but that it is not trivial to do so correctly. An obvious limitation is that racing games are less forgiving when it comes to terrain smoothness than other types of games. You usually can not easily change the world or go around obstacles while you’re playing, unless special features to specifically do so are added (as previously mentioned). But even when taking only racing tracks that are possible to finish into account, difficulty is still an important factor. To assure our work towards generating tracks and analysing the difficulty of these tracks is useful, we are interested in exactly how difficulty affects the game expe- rience. Research shows that players like to be challenged, but not overwhelmed [15, 17]. One of the most essential terms in this field is “Flow”, a theory developed by M. Csíkszentmihályi [19]. It has been used to better understand how players react to certain situations. It can be summarized as follows: when players are not challenged appropriate to their skill level, they either get bored (challenge level too low) or frustrated (challenge level too high). As neither of these will improve the player’s experience, we want to be able to adapt the game’s difficulty level to be more appropriate for the player. Measuring difficulty Adapting the difficulty to the player requires two things. First, we need to be able to detect how well the player is doing. This is not the focus of this work, but could in theory be implemented for racing games by looking at how often the player crashes, how much progress they make along the track in a certain amount of time, et cetera. Second, we need to be able to control the difficulty of the environment around the player. In racing games, the most obvious way to do so is to change the tracks to make them easier or more difficult to drive on (at high speeds). But in order to change the difficulty, we first need to determine what variables control the difficulty of a race track. For traditionally designed tracks this can be done by play testing, but as procedurally generated tracks are different every time, this approach is not an option. This means it needs to be done through computational methods, implemented by the game developers beforehand. To assess the difficulty of a race track computationally, we will need a difficulty model, capable of analyzing the difficulty of a given section of a race track. To summarize: our main goal is to find a method of predicting difficulty of race tracks, that can be applied in racing games that use procedural generation to gen- erate race tracks. Preferably, we want a method that can easily be applied, without the need of expensive equipment or long term testing, so that it can also be used by smaller game studios. 4 Introduction Structure of this thesis After going through some of the most relevant related work in Chapter 2, we will explore the environment in which we implemented our methods in Chapter 3. We first discuss the requirements we set in Section 3.1, then the potential candidates for test environments we explored in Section 3.2, and fi- nally our eventual Unity implementation in Section 3.3. We continue with our first approach, the data driven model, in Chapter 4, followed by the analytical model in Chapter 5. Both of these sections are subdivided into subsections for the hypothesis, the model, our implementation method and the test results.