The AIIDE-17 Workshop on Experimental AI in Games WS-17-19 Towards General RPG Playing Joseph C. Osborn Ben Samuel Computational Media Computer Science University of California, Santa Cruz University of New Orleans 1156 High St 2000 Lakeshore Dr Santa Cruz, California 95064 New Orleans, Louisiana 70148 Adam Summerville, Michael Mateas Computational Media University of California, Santa Cruz 1156 High St Santa Cruz, California 95064 Abstract techniques unless they are split into smaller episodes, but this splitting is generally ad hoc and game-specific; more- General videogame playing has come a long way in a short over it has not been shown for games with complex inter- period of time, but remains at the level of solving relatively nal structure. Agents can do well in situations like a DOOM short games made up of distinct and isolated episodes. Even simple console role-playing games (RPGs) are far beyond the deathmatch because each individual encounter with an en- reach of current techniques, requiring the synthesis of cul- emy is more or less independent, and the high-level struc- tural knowledge with compositional reasoning over several ture of moving between firefights and looking for powerups interconnected sub-games. We explore how the challenges of is extremely simple. Moreover, both types of play share the playing these games could spark new advances in composi- same basic input modalities, short-term rewards, and so on. tional analysis of games and common-sense reasoning. Successful game-playing agents generally learn policies General RPG playing can leverage advances in episodic gen- for maximizing their probability of victory, but rarely re- eral game playing and in areas like text understanding, image cover knowledge about a game’s (implicit) rules or high- classification, and automated game design learning. It has di- level design features. There are some exceptions in the do- rect applications in design support and AI-based game design, and the techniques used to enable it could generalize to other mains of general game playing (Clune 2007) and general families of games such as adventure, open-world, and simu- videogame playing (Perez-Liebana et al. 2016b). The auto- lation games. In this paper, we describe the motivation behind mated play of specific, individual games can in theory be general RPG playing in a sub-domain of Nintendo Entertain- achieved using approaches from expert systems or deep Q- ment System (NES) RPGs, some promising approaches to learning, but these might fail to generalize to new games or some of its fundamental issues, and immediate next steps; we even to variations on the same game; classical search-based conclude by describing a few concrete benchmark problems approaches often fail to scale to longer games without a pro- on the path towards automated play of these complex games. hibitive amount of domain knowledge. On the other hand, recent research in schema networks seems like a promising Introduction approach which reinforces the notion that we must learn de- sign information explicitly in order to transfer knowledge General (video)game playing is an established research area and skills across games (Kansky et al. 2017). Automated with several active threads; as a natural stepping-stone to game design learning has been recognized as an impor- artificial general intelligence it is an attractive target for aca- tant problem with many application areas including general demic and industrial research labs alike (Genesereth, Love, game playing (Osborn, Summerville, and Mateas 2017). and Pell 2005; Perez-Liebana et al. 2016a; Bellemare et al. 2015). Human-like (or at least strong) general game-playing Real-time strategy games comprise an active area of re- AI also seems necessary for many approaches to automated search for game-playing agents generally optimized for spe- game design support and game generation (Smith 2013; cific games (Ontanon´ et al. 2013). Given the successes of Barros et al. 2015). Unfortunately, these systems are limited RTS AI, we suggest exploring the general play of a different to episodic games for which planning can succeed only over genre: console role-playing games (RPGs) like Dragon War- relatively short time horizons (minutes at most, or a few hun- rior or Final Fantasy. These games are typified by highly dred turns in discrete games), even with considerable com- compositional design relying extensively on both cultural putational resources. Longer games are inaccessible to these knowledge and numeric reasoning over probabilistic com- bat models, exercising different aspects of commonsense Copyright c 2017, Association for the Advancement of Artificial reasoning than existing RTS or arcade game players. By Intelligence (www.aaai.org). All rights reserved. compositional design, we mean a game design consisting of 92 many distinct game systems with significant interactions be- HP for Health/Hit Points) that when depleted puts them in a tween systems (e.g., exploration, combat, item economies, knocked out state; if the entire party is knocked out, the game and plot progression). These games have many iterations, typically ends and the player must restart from a set location sequels, hacks, and even randomizers, yielding a substan- or a saved game. Most characters also have MP or Magic tial corpus whose members share some similarities but also Points that determine how many times they can use special introduce game-specific concepts, systems, and user inter- abilities, though the nature of this quantity is game-specific. faces. We can usefully investigate problems of composi- For instance, in Final Fantasy, a character has eight MP val- tional game understanding and transfer in the special case ues, where each value signifies the number of times they can of general RPG playing. cast a spell of the corresponding rank; in Dragon Warrior, MP is a single quantity that decreases by a set amount based General RPG Playing on the cost of the spell cast, with more powerful spells cost- To address compositional game understanding, it is impor- ing more MP. Tools from numeric transition systems might tant to identify the sub-components (or sub-systems) that, be appropriate to apply here, for example by synthesizing when combined, form what a human player would likely integer programs approximating the combat simulation; we identify as a role playing game. Breaking down RPGs into might also be able to learn the effects of specific combat ac- these constituent parts is an important first step in determin- tions by Bayesian inference. ing an effective knowledge representation. Moreover, this There are also attributes like attack power and magic process provides a clear (if somewhat naive) blueprint for power which affect a character’s combat prowess, that can be accomplishing our goal of developing a general RPG player, bolstered through purchased and looted equipment. Though as this goal can be broken down into creating game players typically higher numbers are better, some games involve for each of the game’s sub-systems. These sub-systems do characters that are meant to specialize in particular at- not operate in isolation, but inform and communicate with tributes. Besides permanent equipment, there are often con- each other; we hope to leverage previous work in RTS play- sumable items that adjust resources and states (e.g., items ing where sub-component managers cooperate to achieve a that restore HP or MP, restore a party member with the shared goal (Weber, Mateas, and Jhala 2010), lifting this to knocked out status to fighting condition, etc.). To compli- the level of learned goals and systems. cate things further, most games have non-combat items or At a high level, many NES-era RPGs take the form of abilities that indirectly affect the sub-game of resource man- sweeping epics where players control an individual or a agement, such as the Dragon Warrior series’ Wing of the small party to overcome impossible odds and defeat the Wyvern item which teleports the player to a previously- forces of evil. RPGs typically involve exploring the world, visited town and its available shops and amenities. conversing with Non-Player Character (NPC) townsfolk and Menus: Though menus are an important aspect of combat, royalty, battling countless monsters or named villains in both many other systems rely on menu navigation as well. Most random and scripted combat encounters, and applying the RPGs include shops where equipment can be purchased with spoils of war towards the acquisition of equipment that as- coin accrued through combat; these shops are almost always sists in subsequent fights. Though each game has its own id- menu-based (some exceptions have the player bump into the iosyncrasies, several components seem to be shared by most item they wish to purchase). games in the genre: Navigating menus is often a prerequisite to playing the Movement: RPG game worlds are typically represented as game. Some games involve choosing a save slot (e.g., a series of tile-based maps. Each tile has a variety of prop- Dragon Warrior), while others do not (e.g., Final Fantasy). erties, including: their appearance; whether the player can Many involve naming the character(s) that comprise the walk through them; whether they might trigger a fight (the party. Some games ask the player to determine the make-up tile’s safety); or if they are a link to another map. Some tiles, of the party, which has significant ramifications for future often grasslands or deserts, are passable by default, while gameplay decisions. In Final Fantasy, the party makeup is others can only be traversed when the player owns a certain determined at the beginning of the game, while in Dragon item or is riding a vehicle (e.g., Final Fantasy’s oceans can Warrior 3 the number and type of characters can be changed be traversed by ship, and rivers by canoe), and still others during play at a specific location, and in Final Fantasy 3 are completely impassable.