Discovering Combos in Fighting Games with Evolutionary Algorithms Gianlucca L. Zuin* Yuri P. A. Macedo* Luiz Chaimowicz Departamento de Ciência da Departamento de Ciência da Departamento de Ciência da Computação Computação Computação UFMG UFMG UFMG [email protected] [email protected] [email protected] Gisele L. Pappa Departamento de Ciência da Computação UFMG [email protected] ABSTRACT difficult task. Sometimes these unexpected behaviors can In fighting games, players can perform many different ac- be embraced by designers, creating features that can be ex- tions at each instant of time, leading to an exponential num- panded and designed upon. But they may also break player's ber of possible sequences of actions. Some of these combi- immersion, and maybe even go as far as to ruin the game nations can lead to unexpected behaviors, which can com- for that player. This is commonly observed in some game promise the game design. One example of these unexpected genres, such as the fighting games. behaviors is the occurrence of long or infinite combos, a long Fighting games are very popular and have been around for many years. The largest fighting game competition, Evolu- sequence of actions that does not allow any reactions from 1 the opponent. Finding these sequences is essential to en- tion Championship Series (EVO) , formerly Battle by the sure fairness in fighting games, but evaluating all possible Bay, was founded in 1996 and, in 2015, had over 19 million sequences is a time consuming task. In this paper, we pro- views on the streaming website twitch.tv [15]. Due to their pose the use of an evolutionary algorithm to find combos competitive nature, they require a careful balancing between on a fighting game. The main idea is to use a genetic al- the characters, such that the gameplay can be perceived gorithm to evolve a population composed of sequences of as fair by the both players. Although these characters are inputs and, using an adequate fitness function, select the overall designed to be equally efficient, sometimes they may ones that are more suitable to be considered combos. We feature some unexpected behaviors due to the large num- performed a series of experiments and the results show that ber of play combinations. One example of these unexpected the proposed approach was not only successful in finding behaviors is the occurrence of long or infinite combos. combos, managing to find unexpected sequences, but also In fighting games, whenever a player strikes a blow to an- superior to previous methods. other, the player taking the hit becomes temporarily stunned and unable to take actions. Should the attacking player per- form another attack that is fast enough to hit an opponent 1. INTRODUCTION before he is able to recover, this sequence of attacks becomes One of the tasks in game design is to define an environ- a combo. More specifically, a combo [18] is a sequence of at- ment with a predetermined set of rules that dictate how the tacks by a player that, while damaging an opponent, does player should interact with the game and evolve its state. In not allow it to take any actions. While this sequence of video games, programming these rules is a non trivial task attacks should eventually end allowing the opponent to re- and is subject to design flaws and bugs. Moreover, testing taliate, infinite combos, as the name suggests, do not end as can be very time consuming in game development, and var- long as the attacking player does not miss his correct inputs. ious titles end up being delivered with some design errors. These types of combos are usually the result of unexpected In many games that rely on physics simulations, even interactions between a character's actions, where a sequence though the designed environment follows a set of prede- of attacks loops with itself indefinitely. termined rules, predicting all emergent behaviors can be a Despite how negative infinite or long combos can be to a player's experience, it is not correct to say that a combo is an undesired feature in fighting games. Combos are an inte- Permission to make digital or hard copies of all or part of this work for personal or gral part of many action games, rewarding precise execution classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full cita- of commands and encouraging players to keep on practic- tion on the first page. Copyrights for components of this work owned by others than ing. Competitive fighting game players strive to perform ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re- long combos that take the most out of any of their oppo- publish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. nents' vulnerable moments. As fighting games are also an GECCO ’16, July 20-24, 2016, Denver, CO, USA electronic sport that is commonly watched live or through c 2016 ACM. ISBN 978-1-4503-4206-3/16/07. $15.00 1 DOI: http://dx.doi.org/10.1145/2908812.2908908 http://evo.shoryuken.com/ their goal. Another example is [14], which illustrates an at- tempt to use computational intelligence to generate a game. Through an evolutionary algorithm, the game controllers and rules are improved. When it comes to fighting games, it is difficult to find any academic research that does not focus on improving the game AI. The authors of [11], for example, focus on creating a bot capable of mimicking a human player. The bot uses a Naive Bayes classifier over all possible actions and a finite state machine to choose a set of action that would better mimic a player. The different sets attempt to reproduce dif- ferent kinds of human behavior during a match, like playing Figure 1: An infinite combo from the game Ultimate Marvel more aggressively when ahead or conservatively if overpow- Vs Capcom 3. Combos in this game are longer than in most ered. The bot developed by Yamamoto et al. [16], in turn, games, but even then it suffers from combos that have made attempts to predict its adversary course of actions through some characters be considered much superior to others. the K-Nearest algorithm. With that information and the knowledge that some actions are advantageous against an- other, it deploys the appropriate countermeasures. Graepel, online streaming, combos can also can be very entertaining Herbrich and Gold [4] explore the use of reinforced learning for viewers of competitive matches. with different reward functions trying to learn relevant data An important task for adequately balancing fighting games by inference in a Markov decision process. Their objective is is to discover these combos. The biggest problem, however, to develop an AI with good policies in a commercial game. lies on the exponential number of combinations of attacks, In previous work [18], we tackled the problem of finding which makes the task of predicting all of their outcomes, and large combos in fighting games using Hidden Markov Models consequently detecting combos, a very hard one. [10]. Both supervised and unsupervised learning algorithms In this paper, we propose the use of an evolutionary algo- were used but due to the excess of noise and particularities rithm to successfully find combos on an open source fighting of the implemented model, we were unable to successfully game, by discovering which sequences of player inputs are predict combos. Changing the minimal discrete time inter- likely to generate combos. The main idea is to use a genetic val to a player action, rather than a game frame, we were algorithm to evolve a population composed of sequences of able to identify small combos lasting up to 112 frames. In inputs and, using an adequate fitness function, select the this paper, we propose a different methodology, and use an ones that are more suitable to be considered combos. evolutionary method to discover combos. Evolutionary algorithms allow us to trim the exponen- tially large search space of attack combinations, making the 3. PROBLEM DEFINITION search for longer combos much easier. Also, by automating A common pattern in game programming is the defini- this task, we are able to attempt combinations of actions tion of a \game loop" [9] where, at each turn, inputs are that normally would be counter-intuitive for manual testing processed, the game state is updated, and the game is ren- standards, allowing the discovery of new combos. dered. This loop executes once for every frame, which is the The remainder of this paper is organized as follows: Sec- shortest discrete time interval in a game. tion 2 discusses some works regarding Artificial Intelligence Most fighting games display a fully deterministic behavior, techniques applied to fighting games and non-traditional ar- which means that, under the same circumstances, whenever eas of game design. Section 3 presents the problem defini- the same action is performed, the outcome is the same. It tion while section 4 describes our proposed methodology. In also means that no event relevant to the gameplay relies section 5, we present our experiments and discuss the ob- on any sort of random timer or choice. For fighting games tained results. Finally, Section 6 brings the conclusion and that follow this behavior, any state within the game can be directions for future work. described by the sequence inputs, one at each frame, that led to that point. Therefore, by giving a specific set of inputs 2.