The AIIDE-17 Workshop on Experimental AI in Games WS-17-19 Towards Positively Surprising Non-Player Characters in Video Games Vadim Bulitko Shelby Carleton Delia Cormier Computing Science English and Film Studies Art and Design University of Alberta University of Alberta University of Alberta Edmonton, AB Edmonton, AB Edmonton, AB [email protected] [email protected] [email protected] Devon Sigurdson John Simpson Computing Science Compute Canada University of Alberta University of Alberta Edmonton, AB Edmonton, AB [email protected] [email protected] Abstract game such as acting as sources of equipment or fighting for or against players. Regardless, the standard for a good aNPC Video games often populate their in-game world with numer- in a video game remains its ability to perform exactly as pro- ous ambient non-playable characters. Manually crafting in- grammed, going where told, attacking the correct enemies, teresting behaviors for such characters can be prohibitively expensive. As scripted AI gets re-used across multiple char- and responding with scripted dialogue to player choices, all acters, they can appear overly similar, shallow and generally without disrupting the immersive experience. aNPCs that act uninteresting for the player to interact with. In this paper we outside of these boundaries risk earning the ire of the gaming propose to evolve interesting behaviors in a simulated evo- community by doing things such as glitching through ob- lutionary environment. Since only some evolution runs may jects, getting caught in corners, repeatedly needing rescue, give rise to such behaviors, we propose to train deep neural or simply not advancing the plot as intended. The number networks to detect such behaviors. The paper presents work of YouTube rants and blogs offering top 10/50/100 “Worst in progress in this direction. NPCs in Video Games” lists is a testament to the widespread nature of these phenomena (WatchMojo 2014). 1 Introduction Perhaps even more unfortunate than outright errors are aNPCs meant to be interesting/surprising but that ultimately Can ambient non-player characters in video games sur- leave the player disappointed because they may be boring. prise players in positive ways without being explicitly pro- Consider the PC Gamer review of No Man’s Sky, which of- gramed? The prevailing approach to the development of fers as a verdict, “64/100. Relaxing exploration and some video games suggests that doing so may not be a desirable lovely scenery coupled with repetitive systems, frustrating risk to take. We believe that the answer in the future will be menus, and a lack of real discovery” (Livingston 2016). The “Yes, and the benefits outweigh the risks” but that this is still full review makes it clear that lack of discovery is the in- a way off because techniques still need to be developed and surmountable problem with the game as it failed to produce prevailing opinions changed. However, a significant step to- anything “fascinating” and was instead filled with uninter- wards this goal can be taken now by focusing attention on esting predictable behavior and little meaningful player in- a subclass of non-player characters (NPCs) in video games teraction during exploration. For a game that is first and fore- that do not typically require the highly sophisticated behav- most about discovery, about surprising players with crea- iors related to language and culture. The NPCs that we have tures that no one else has ever seen—the designers made in mind are ambient NPCs (aNPCs) which are characters extensive use of procedural generation in an attempt to offer that exist predominantly in the background of games with- players new experiences throughout gameplay—such criti- out any direct tie to a quest or mission. Most animals and cism carries a heavy weight. similar non-language using characters would be of this type. When it comes to developing aNPCs game designers aim Rather than accept that the solution to producing aNPCs to enhance the atmosphere of the game without distracting that are neither boring or broken is expertly produced be- the player from the main storyline or side quests. While aN- havior trees of ever increasing complexity, we suggest that PCs predominantly wander around to produce atmosphere an alternative methodology be explored: evolved artificial through their simply being there or providing small interac- life. By producing simple models of the aNPCs to make tive opportunities they can also play larger roles within the up the game world and allowing them to reproduce and die with evolutionary mechanisms in place, behaviors that Copyright c 2017, Association for the Advancement of Artificial actually worked for the specified environment can be pro- Intelligence (www.aaai.org). All rights reserved. duced, reducing the likelihood of producing broken NPCs. 34 Further, with the appropriate agent complexity this approach range of background conditions in order to produce behav- can possibly generate surprising behaviors that a human de- iors of the sort described above. This large number of runs— signer might never have considered. As there is no guaran- possibly in the hundred thousands to millions—is necessary tee that an interesting behavior would emerge on every run in order to understand the robustness of produced behaviors of such a simulated evolution, many evolutions may have to to the various sorts of possible environments that could be be run, making it intractable for a human game developer produced during game play. Producing such a large num- to detect interesting behavior manually. Thus, we propose ber of simulations is not the interesting problem here, find- building an automated detector of interesting behavior. Such ing the surprising behaviors amongst what will surely be an a program would sift through numerous evolution runs, flag- overwhelming field of much more mundane behaviors is. ging possibly interesting evolved behavior for game devel- The detection of surprising behaviors problem can be opers to examine and subsequently include into a game. framed as a problem of detecting behaviors that stand out This paper is organized as follows. Section 2 formally in- from the rest of the crowd no matter what the behavior is, troduces what makes behavior positively surprising in the that is, such behaviors can be seen as anomalies. context of a video game, and discusses how to detect such behavior. We then discuss related work in Section 3. Sec- 3 Related Work tion 4 details our approach to the problem, where we set up an evolutionary model to exhibit interesting behaviors, and Improving non-player characters has been an ongoing pro- train a deep neural network to visualize frames and recog- cess since early in the history of video games with an array nize differences as a first step toward anomaly detection. of ever sophisticated methods being employed. For the pur- Section 5 details our current challenges and future work, poses of locating our specific problem and anticipating a so- where we discuss the limitations of our research and the pos- lution, in this section we group relevant and related works sible steps we can take to progress our anomaly detector, into four categories—AI, anomaly detection, agent-based finally followed by Section 6, our conclusion. evolutionary models, and science fiction provocations— briefly summarizing key works in each. 2 Problem Formulation 3.1 Artificial Intelligence What counts as a surprising aNPC behavior within a game will vary between designers, players, critics, and observers, Interesting NPC behaviors have already begun to emerge as well as across games and time. Acknowledging this vari- within the video game industry. F.E.A.R., a first person ability we are seeking to constrain possible interpretations of shooter, utilizes an A*-based automated planner to let its what counts as surprising behaviors by focusing on assess- NPCs reduce goals to actions. As a result more com- ments made by players of aNPC behaviors that are positive plex, squad-like and sometimes surprising behaviors such as and sustainable through repeated and varied game play, for flanking emerge without explicit hand-coding (Orkin 2006). the following reasons. In a similar vein, the Forza franchise uses machine learn- First, players are the group to focus on because they are ing on data collected from real-world players to create dri- the ones who most directly experience the game and it is vatars: complex and more realistic NPC drivers that exhibit their reaction that is the ultimate test of a games general suc- realistic driving strategies used by real people around the cess. Our intent is to produce behaviors that are at least unex- world, rather than preprogrammed opponent racers (Xbox pected (i.e., surprising). This could be due to either a single Wire Staff 2014). While their methods of producing NPC behavior or how sets of behaviors interact within the game behaviors are different from our proposal in this paper, their environment. Second, by positive aNPC behaviors we intend intent matches ours. to capture those behaviors that enhance game play rather Social physics is also being introduced into the construc- than detract from it. Such detraction can happen by either tion of AI systems in games. The Comme Il Faut, AI system removing the player from the immersive experience (e.g., within Prom Week, allows for rich emergent story lines and an aNPC who violates the accepted physics of the world NPCs by extracting and encoding exaggerated social logic by clipping through walls) or by maintaining immersion but from pre-existing media. The system used gives a plethora creating undue frustration (e.g., an aNPC who is essential to of outcomes compared to traditional game stories, but uses a quest but too difficult to interact with to allow the quest to predetermined rules of social norms and behaviors that fa- be completed).