Agent-Based Adaptive Level Generation for Dynamic Difficulty Adjustment in Angry Birds Matthew Stephenson,1,2 Jochen Renz,1 1Research School of Computer Science, Australian National University, Canberra, A.C.T. 0200, Australia 2Department of Data Science and Knowledge Engineering (DKE), Maastricht University, Netherlands Abstract constraints compared to most traditional video games. Suc- cessfully generating levels for Angry Birds that are equally This paper presents an adaptive level generation algorithm for as challenging as human-designed levels is a difficult task, the physics-based puzzle game Angry Birds. The proposed but will likely be necessary for Angry Birds agents to im- algorithm is based on a pre-existing level generator for this prove beyond their current capabilities. Previous level gener- game, but where the difficulty of the generated levels can be adjusted based on the player’s performance. This allows ation methods for Angry Birds used either a heuristic calcu- for the creation of personalised levels tailored specifically to lation based on level properties or the performance of several the player’s own abilities. The effectiveness of our proposed agents to help set the difficulty of a level. However, as differ- method is evaluated using several agents with differing strate- ent players often possess varying levels of ability, many peo- gies and AI techniques. By using these agents as models / ple would likely find these levels too hard or easy to solve. representations of real human player’s characteristics, we can This is also a problem for training and evaluating agents, as optimise level properties efficiently over a large number of levels where most agents either can or cannot solve them generations. As a secondary investigation, we also demon- yield very little discriminatory information (Stephenson et strate that by combining the performance of several agents al. 2018a). We therefore suggest an agent-based adaptive together it is possible to generate levels that are especially generation method for dynamic difficulty adjustment, where challenging for certain players but not others. the generator adjusts the difficulty of its levels depending on the player’s performance. This method can also be used Introduction to generate levels that are difficult for one player whilst be- ing easy for another, exploiting the player’s own strengths or Procedural level generation (PLG), where levels for a game weaknesses. are created automatically without the need for human de- The remainder of this paper is organised as follows. Sec- signers, is a key area of investigation for video game re- tion 2 describes the large amount of background and related search (Hendrikx et al. 2013; Togelius et al. 2011). PLG work, both for Angry Birds and adaptive level generation in can be extremely useful for increasing a game’s length and general. Section 3 presents our proposed adaptive generation replayability, as it allows a large number of levels to be cre- method. Section 4 describes our conducted experiments and ated in a relatively short time. It is also possible to tailor the results. Sections 5 discusses what these results could mean generated levels towards specific user’s playstyles, known as for both human players and agents, Section 6 concludes this adaptive level generation, which allows for a unique and per- work and outlines future possibilities. sonalised gameplay experience (Yannakakis and Togelius 2011). Dynamic difficulty adjustment (DDA) is a form of adaptive level generation, where the difficulty of generated Background arXiv:1902.02518v1 [cs.AI] 7 Feb 2019 levels is modified to better suit the player’s current skill Adaptive Level Generation based on their performance (Hunicke 2005). This is accom- plished by modifying certain generator parameters that con- While most games that contain some form of PLG typi- trol different level features, so that the end result is more cally use generic generation techniques that are not influ- likely to achieve the desired amount of challenge for the enced by the player’s behaviour, adaptive level generation, player. also referred to as experience-driven, personalised or player- This paper presents an adaptive level generator for the centred level generation, takes the player’s behaviour into physics-based puzzle game Angry Birds. This game has account (Shaker et al. 2016). Examples of game or level been used substantially in AI research over the past few characteristics that could be adjusted for specific players in- years, primarily for developing agents and level generators, clude qualities such as difficulty, engagement, frustration, as the game’s environment presents more realistic physical enjoyment, complexity, learning potential, etc. These prop- erties are indirectly controlled by adjusting certain parame- AAAI-2019 Workshop on Games and Simulations for Artificial In- ters of the generator. Different players will likely behave or telligence. perform differently even when playing the same game. If an accurate model of the player can be determined, then this can be used to tailor the properties of the generated content towards their individual preferences. Constructing a model of the player is a difficult and im- precise task, but is essential for adaptive level generation to be effective. Methods for determining player behaviour in- clude analysing their performance across several “test” lev- els, or using a questionnaire for measuring more intangi- ble qualities. This information can then be used to directly evaluate generated content in the future, allowing us to esti- mate whether it will be suitable for the player. Another ap- proach, and the one that we will be using in this paper, is to Figure 1: Screenshot of a level from the Angry Birds game. use AI agents to estimate the quality of levels (i.e. agent / simulation-based evaluation functions). Using a collection of agents as representations of different playing styles or person, allowing us to evaluate a larger number of levels in a abilities allows us to generate levels that are suited to a par- much shorter time (requiring volunteers to playtest hundreds ticular player, or a collection of levels that require several of levels is not very practical). Agents can also typically give different techniques to solve them. more accurate estimations of certain level properties (espe- Examples of genre’s where adaptive level generation has cially difficulty) than by just analysing the level’s features. been used to improve the player experience include board Human players are also likely to improve the longer they games (Marks and Hom 2007), racing games (Togelius et play, making repeated performances inconsistent between al. 2007), action-RPG (Heijne and Bakkes 2017), rogue-like different experiments. The downside of this method is that (Stammer et al. 2015), tower defence (Sutoyo et al. 2015), it naturally requires a large and diverse range of agents to and platformers (Shaker et al. 2012; Shaker et al. 2010). already exist, which Angry Birds thankfully has (agents de- Dynamic Difficulty Adjustment Dynamic difficulty ad- scribed in more detail later). justment (DDA) is often considered to be one of the simplest and most common forms of game adaption, where the dif- Angry Birds ficulty of a game increases or decreases if the player is per- Angry Birds is a popular physics-based puzzle game where forming too well or poorly respectively (Wheat et al. 2015). the player’s objective for each level is to kill all pigs using a Because a player’s performance in a game can typically be set number of birds. A typical Angry Birds level like the one evaluated without the need for questionnaires or overly com- shown in Figure 1, requires the player to shoot the birds they plex estimations, DDA can usually be implemented in most have from a slingshot at structures made of blocks that are games without significant issue. Nearly all games feature protecting the pigs. All objects within the level obey simpli- some form of increasing difficulty as the skill of the player fied physics principles defined by the game’s engine. Blocks increases, but this element is often lost or overly simplified can come in several different shapes and materials, and birds with most PLG approaches. However, just because estimat- can also be one of several different types (all with differ- ing the difficulty of a game for a specific player is relatively ing properties). Pigs and blocks can be killed / destroyed by simple compared to other more complex behavioural charac- hitting them with either a bird or another object. Points are teristics, this certainly doesn’t make the task trivial. The dif- awarded to the player once the level is solved (all pigs in the ficulty of a game can often be multi-dimensional in nature, level have been killed) based on the number of birds remain- where the same level could be considered hard or easy for ing and the total amount of damage caused. The source code various different reasons (Jennings-Teats et al. 2010). Play- for the official Angry Birds game is not currently available, ers can often have unbalanced skill sets, where they are adept so a modified version of the Unity-based clone known as at overcoming certain tasks or challenges more than others. Science Birds, originally created by Lucas Ferreira (Ferreira One player may be very good at forward planning, another and Toledo 2014), was used instead. at making precise actions, another with fast response times, and so on. A successful DDA system should therefore be Level Generation Several level generators have been pre- able to adjust the difficulty of its generated levels in many sented for Angry Birds in recent years, some of which different ways, and also detect which of these most influ- have attempted to adapt the generated content based on the ence the player’s performance. player’s experience. Previous work by (Kaidan et al.