A Review of Computational Intelligence in RTS Games
Total Page:16
File Type:pdf, Size:1020Kb
A review of computational intelligence in RTS games Raul´ Lara-Cabrera, Carlos Cotta and Antonio J. Fernandez-Leiva´ Abstract—Real-time strategy games offer a wide variety of content generation can be used to create maps, units and fundamental AI research challenges. Most of these challenges technologies for RTS games. Traditionally, academic game have applications outside the game domain. This paper provides AI was mainly linked to non player character (NPC) behavior a review on computational intelligence in real-time strategy games (RTS). It starts with challenges in real-time strategy and pathfinding. However, there are new research areas that games, then it reviews different tasks to overcome this chal- have recently provided innovative solutions for a number lenges. Later, it describes the techniques used to solve this of game development challenges, like player experience challenges and it makes a relationship between techniques and modeling (PEM), procedural content generation (PCG) and tasks. Finally, it presents a set of different frameworks used as large scale game data mining [86]. test-beds for the techniques employed. This paper is intended to be a starting point for future researchers on this topic. Real-time strategy and turn-based games (RTS and TBS Index Terms—Computational intelligence, real-time strategy respectively) are sub-genres of strategy games. They have a games, review. lot of aspects in common. Many of the proposed challenges and tasks could be applicable to RTS and TBS without I. INTRODUCTION distinction. However, aspects related to real-time (i.e. adver- Commercial video-games are a rising value in the en- sarial real-time planning) are only applicable to RTS. tertainment industry. The total spent in the video-game Computational intelligence in RTS video games is quite a industry in 2010 was 25.1 billion dollars [15]. Video-game new field of research (although there are old papers on the use budgets are high and their development teams are composed of evolutionary algorithms for real time adversarial gaming of many people. Traditionally, game developers have over- [6], [19]), as opposed to the computational intelligence in looked their non-player characters’ artificial intelligence, fo- TBS and board games like Chess and Go. Since this is a cusing to other aspects of the game, such as graphic engines recent topic, there are no previous reviews. This is the main and 3-D modeling. This situation leads to a poor gaming motivation behind this paper. experience, since human players are able to win the game without much effort. Computational intelligence is growing II. CHALLENGES IN REAL-TIME STRATEGY GAMES in importance and video-game players are demanding good As we briefly noted before, RTS games offer a large vari- artificial intelligence (AI) which makes these games more ety of fundamental AI research problems [7]. In this section, interesting and harder to beat. we describe these research challenges and its relationships RTS games are a genre of video-games which require with RTS games. managing different kind of units and resources in real-time. In a RTS game the participants position and maneuver units A. Adversarial real-time planning and structures under their control to secure areas of the map As its name suggests, RTS game’s actions are made in real- and/or destroy their opponents’ assets. In a typical RTS, it is time, so players have to make their decisions under severe possible to create additional units and structures during the time constraints and they need to be able to execute multiple course of a game, but this is generally limited by the number orders simultaneously. In addition to this, games take place in of accumulated resources, which are gathered by controlling dynamic and hostile environments that contains adversaries special points on the map and/or possessing certain types of who modify the game state asynchronously. These charac- units and structures devoted to this purpose. The typical game teristics denote the need to investigate adversarial real-time of the RTS genre features resource gathering, base building, planning approaches. Turn-based games’ (like Chess) agents in-game technological development and indirect control of have to deal with dynamic and hostile environments as well, units. They are usually played by two or more players but the fact of the game not happening in real time makes this (human or not) that have to deal with incomplete information challenge less difficult to overcome than the same challenge during the game (the map is covered by fog of war, the in a real-time environment. technology developed by a player is unknown by every other player, ...). These features make RTS games a great tool B. Decision making under uncertainty for computational intelligence research, since a RTS game In most commercial RTS games, NPCs have all the in- player needs to master many challenging problems such formation on the game state, including the location of the as resource allocation, spatial reasoning, strategy planning human player units and buildings. This situation should be and opponent’s strategy prediction. In addition, procedural considered as cheating, because players lack this information. To prevent this unbalanced situation, it is necessary to impose The three authors are with Department “Lenguajes y Ciencias de la Computacion”,´ ETSI Informatica,´ University of Malaga,´ Spain (emails: partial observability onto RTS games, ensuring that all play- fraul,ccottap,[email protected]) ers play the game on equal terms. This partial observability, usually named “fog of war”, represents another challenge to All these aspects greatly complicates the problem of finding the design of game agents, because players are not aware suitable paths. of the enemies base locations and intentions. This challenge is not present when designing an agent for a turn-based or H. Content Generation board game like Chess and Go, whose player usually have Game content refers to all aspects of the game that affect all the information on the game state, including the location game-play other than non-player character (NPC) behavior of other players units (there are, however, turn-based games and the game engine itself. When it comes to RTS games, whose state is partially known by the players). there is assorted content such as maps, units, buildings and weapons that can be generated in a procedural manner. If new C. Opponent modeling content can be generated with enough variety then it may Human players have an ability to analyze the enemy’s become possible to create endless games, with new maps, actions and spot their weaknesses, exploiting them in future units and buildings on every new game. In addition to this, games. Artificial players need to be qualified to analyze their the generated content can adapt itself to specific criteria, such opponents’ actions and predict their behaviors in basis of as the playing style of a particular player [76]. previous observations. This challenge is not exclusive of RTS III. TASKS IN REAL-TIME STRATEGY GAMES games, because in any other game, modeling your opponent will be always useful. Due to their characteristics, RTS games give us many challenges to deal with. The work done in computational D. Spatial and temporal reasoning intelligence in RTS games can be classified by the problem Maps in RTS games are elements with high influence tackled. In the following section we will describe the tasks during the course of the game (i.e. different types of terrain associated with these challenges and what work has been produce different types of resources, elevated positions give done in relation to each of these tasks. advantages to a unit’s attack, ...), so it becomes necessary to A. Planning make good terrain analysis. This way, agents can develop bet- ter offensive and defensive plans and more efficient resources Humans and adversaries can use any available action to gathering as well. Another advantage (which can be seen as form their game strategy, which is a plan. Planning is the another challenge) of human players against artificial ones is process of determining action sequences that when executed their ability to understand temporal relations of actions. accomplish a given goal. The presence of adversaries in addition to real-time and hidden information constraints E. Resource management greatly complicates the planning process. RTS games usually include resources which are used to Aha, Molineaux and Ponsen introduced in [1] a plan create or upgrade units and buildings, and develop new retrieval algorithm which uses three key sources of domain technologies as well. These resources can be distributed knowledge and removes the assumption of a static opponent. through the map and then gathered by units or they can This algorithm was called Case-based Tactician (CaT). This be obtained from buildings placed on certain places on the same algorithm was used in [51], where the authors focused map. A proper resource management strategy is therefore an on defeating a selected opponent while training up others, in- essential part of any successful strategy. stead of defeating randomly selected opponents. The authors in [52] introduced an integrated RL/CBR algorithm that uses F. Collaboration continuous models instead of discrete approximation of these During a RTS game, each player generate many units for models. The algorithm was called the Continuous Action and her army. This army can be considered as a multi-agent State Space Learner (CASSL), and is an improvement of the system, so it is necessary to develop coordination methods results obtained in [1], including the ability to learn and rea- that lead to good team tactics and strategies. There is not son with continuous action spaces. An improvement for plan only collaboration at unit level, in team matches there are retrieval can be found in [50].