The AIIDE-17 Workshop on Experimental AI in Games WS-17-19 Leveraging Multi-Layer Level Representations for Puzzle-Platformer Level Generation Sam Snodgrass, Santiago Ontan˜on´ Drexel University 3141 Chestnut St Philadelphia, PA 19104 [email protected], [email protected] Abstract The remainder of the paper is organized as follows. We first formulate the specific problem we are trying to ad- Procedural content generation via machine learning dress. We then give background on recent PCGML tech- (PCGML) has been growing in recent years. However, many niques. Next, we discuss our multi-layer representation, PCGML approaches are only explored in the context of linear multi-dimensional Markov chain approach, and our playa- platforming games, and focused on modeling structural level information. Previously, we developed a multi-layer level bility constraint for non-linear paths. We then describe our representation, where each layer is designed to capture experimental set-up and present our results. We close by specific level information. In this paper, we apply our multi- drawing our conclusions and suggesting future work. layer approach to Lode Runner, a game with non-linear paths and complex actions. We test our approach by generating levels for Lode Runner with a constrained multi-dimensional Problem Statement Markov chain (MdMC) approach that ensures playability and a standard MdMC sampling approach. We compare In this paper we address the problem of generating playable the levels sampled when using multi-layer representation levels in domains that require complex paths. We do this against those sampled using the single-layer representation; through the use of a multi-layer level representation and we compare using both the constrained sampling algorithm modeling approach and a constrained sampling algorithm. and the standard sampling algorithm. Specifically, we model the game Lode Runner using a struc- tural layer, a player path layer, and a section layer in order to more accurately capture the intricacies of the domain. Introduction Procedural content generation (PCG) studies the algorith- mic creation of content (e.g., levels, textures, items, etc.), Related Work often for video-games. PCG via machine learning (PCGML) Procedural content generation via machine learning (Summerville et al. 2017) is the use of machine learning (PCGML) is the algorithmic creation of content using techniques to create a model from which to sample con- machine learning models trained on some form of training tent. Recently there has been increased interest in PCGML data (Summerville et al. 2017). This section discusses for video game levels (Guzdial and Riedl 2016; Dahlskog, PCGML approaches. Togelius, and Nelson 2014; Summerville and Mateas 2016; Snodgrass and Ontan˜on´ 2016b). However, in most cases There are several PCGML approaches for level gener- PCGML techniques have been used to generate linear plat- ation; we discuss a few recent approaches below. Sum- former levels (e.g., Super Mario Bros. levels), and typically merville and Mateas (2016) employed long short-term mem- only capture structural level information. We previously de- ory recurrent neural networks to generate Super Mario Bros. veloped a multi-layer representation in order to capture more levels accounting for player paths via level annotations. varied information from the training levels (Snodgrass and Guzdial and Reidl (2016) used Bayesian networks to gener- Ontan˜on´ 2017). We tested this approach in the domain of ate levels, also for Super Mario Bros. implicitly accounting linear platformer level generation, but we believe this ap- for player interactions through model parameters approxi- proach can be used to model and generate levels in more mated by observing player movements in videos. Notice, complex domains, because of its ability to more deeply rep- that though these methods model more than just the struc- resent domains. Therefore, we explore the use of this multi- tural information of the level, the way the information is layer technique in Lode Runner, a puzzle-platformer game included is not easily extensible to other types of informa- that requires complex paths to complete levels, and allows tion. There are other PCGML level generation approaches for level destruction to open up more paths. (Dahlskog, Togelius, and Nelson 2014; Hoover, Togelius, and Yannakis 2015; Shaker and Abou-Zleikha 2014), but no- Copyright c 2017, Association for the Advancement of Artificial tice that the majority of approaches have only been tested in Intelligence (www.aaai.org). All rights reserved. the domain of linear platformer level generation. 125 Figure 1: This figure shows a Lode Runner level (top-left) represented using a structural layer (top-right), a player path layer (bottom-left), and a section layer (bottom-right). Color is added for clarity. Methods tile types, Ti, with meanings defined by the layer. In addition to the approaches above, there are a few ap- Figure 1 shows a Lode Runner level (top-left) represented proaches that leverage multiple layers of representation. We using three layers: a structural layer (top-right), represent- previously explored a hierarchical approach that used clus- ing the placement of objects in the level; a player path tering to find common structures in levels (Snodgrass and layer (bottom-left), representing the path a player may take Ontanon 2015). The domain was then modeled at multiple through the level; and a section layer (bottom-right), repre- levels, and new levels were sampled at increasing resolution. senting different sections of the level;. We refer to the struc- Notably, we applied this approach to Lode Runner with little tural layer as the main layer, because it is the layer that pro- success (i.e. sampling very few usable levels). Summerville vides the tile types we will use during level sampling. and Matteas (2015) explored a hierarchical approach to dun- Though this is a straightforward concept, it opens up geon generation, by modeling the layout of the dungeon and the possibility for more faithful and deep level represen- the structure of the rooms separately. Notice, however, that tation than is possible with previous representations which both of these approaches focus on only the structural ele- only capture structural information (Guzdial and Riedl 2016; ments of the levels. Snodgrass and Ontan˜on´ 2016b) and occasionally player path To more faithfully and deeply represent and model levels, information (Summerville et al. 2016a). For example, in this we recently developed a multi-layer approach that is able to paper in addition to a structural and a player path layer, we capture structural information, and other level information use a section layer which signifies different sections of the (such as player paths) in a general format (Snodgrass and level. Notice that these are only two possible additional lay- Ontan˜on´ 2017). We previously used this approach to gener- ers. Others can include a difficulty layer, which could cap- ate linear platformer levels, but in this paper we show that it ture the progression of difficulty as you progress through it, is usable in more complex domains, such as Lode Runner. or an enemy path layer which captures the behavior of the enemies. In this section we discuss our multi-layer representation ˜ ´ introduced in (Snodgrass and Ontanon 2017), explain how Markov Chain-based Level Generation we train and sample from an MdMC with a multi-layer rep- resentations, and introduce a playability constraint. We now explain how we train and sample from our model. We first introduce multi-dimensional Markov chains, before Multi-layer Level Representation discussing our training and sampling approaches. We represent a level using a set of layers, L = Markov Chains Markov chains (Markov 1971) model {l1,l2, ..., ln}, where each li is a two-dimensional matrix of stochastic transitions between states over time. A Markov tiles with dimensions h × w. Each layer has a separate set of chain is defined as a set of states, S = {s1,s2, ..., sn}, and 126 the conditional probability distribution (CPD), P (Sx|Sx−1), R Q S ∈ S S S 2,2 2,2 representing the probability of transitioning to a state x ns0 : 1,1 ns1 : 1,1 2,1 S given the previous state, Sx−1 ∈ S. The set of previous P(Sx,y) P(Sx,y | Sx-1,y) nsl : S S states that influence the CPD are the network structure. 3 1,2 2,2 S S ns2 : S1,2S2,2 ns3 : 1,2 2,2 Multi-dimensional Markov chains (MdMCs) are an ex- S SS tension of higher-order Markov chains (Ching et al. 2013) 1,1 2,1 S S S2,1 1,1 2,1 that allow any surrounding state in a multi-dimensional P(Sx,y ||S Sx-1,y , Sx,y-1, Sx-1,y-1, P(S | S , S ) P(S | S , S , S ) graph to be considered a previous state. In our case, x,y x-1,y x,y-1 x,y x-1,y x,y-1 x-1,y-1 Rx,y, Qx,y) the multi-dimensional graph includes previous tiles in the current layer as well as from other layers. For exam- Figure 2: This figure shows the network structures used by ple, the CPD defined by nsl 3 in Figure 2 (right) can be the single-layer MdMC (left), and a network structure used written as P (Sx,y|Sx−1,y,Sx,y−1,Sx−1,y−1,Qx,y,Rx,y), by the multi-layer MdMC. Note, that each nsl i follows the where S, Q, and R the set of states in the various layers. No- same pattern as the corresponding nsi, but the multi-layer tice, that the set of states are also the set of tile types for each network structures retain the dependencies from the other of the layers. Further, in this case S is the set of states (and layers in each network structure.