Multi-Agent Potential Field Based Architectures for Real-Time Strategy Game Bots Johan Hagelbäck c 2012-01-11 Johan Hagelbäck School of Computing Publisher: Blekinge Institute of Technology Printed by Printfabriken, Karlskrona, Sweden 2012-01-11 ISBN 978-91-7295-223-2 Blekinge Institute of Technology Doctoral Dissertation Series No. 2012:02 ISSN 1653-2090 ISBN 978-91-7295-223-2 Multi-Agent Potential Field Based Architectures for Real-Time Strategy Game Bots Johan Hagelbäck School of Computing Blekinge Institute of Technology Sweden Contact information: Johan Hagelbäck School of Computing Blekinge Institute of Technology 371 79 Karlskrona SWEDEN email: [email protected] “The weather outside is hostile with a slight chance of fog-of-war.” – Medivacs in StarCraft 2 ABSTRACT Real-Time Strategy (RTS) is a sub-genre of strategy games which is running in real-time, typically in a war setting. The player uses workers to gather resources, which in turn are used for creating new buildings, training combat units, build upgrades and do research. The game is won when all buildings of the opponent(s) have been destroyed. The numerous tasks that need to be handled in real-time can be very demanding for a player. Computer players (bots) for RTS games face the same challenges, and also have to navigate units in highly dynamic game worlds and deal with other low-level tasks such as attacking enemy units within fire range. This thesis is a compilation grouped into three parts. The first part deals with navigation in dynamic game worlds which can be a complex and resource demanding task. Typically it is solved by using pathfinding algorithms. We investigate an alternative approach based on Artificial Potential Fields and show how an APF based navigation system can be used without any need of pathfinding algorithms. In RTS games players usually have a limited visibility of the game world, known as Fog of War. Bots on the other hand often have complete visibility to aid the AI in making better decisions. We show that a Multi-Agent PF based bot with limited visibility can match and even surpass bots with complete visibility in some RTS scenarios. We also show how the bot can be extended and used in a full RTS scenario with base building and unit construction. In the next section we propose a flexible and expandable RTS game architecture that can be modified at several levels of abstraction to test different techniques and ideas. The proposed architecture is implemented in the famous RTS game StarCraft, and we show how the high-level architecture goals of flexibility and expandability can be achieved. In the last section we present two studies related to gameplay experience in RTS games. In games players usually have to select a static difficulty level when playing against computer oppo- nents. In the first study we use a bot that during runtime can adapt the difficulty level depending on the skills of the opponent, and study how it affects the perceived enjoyment and variation in playing against the bot. To create bots that are interesting and challenging for human players a goal is often to create bots that play more human-like. In the second study we asked participants to watch replays of recorded RTS games between bots and human players. The participants were asked to guess and motivate if a player was controlled by a human or a bot. This information was then used to identify human-like and bot-like characteristics for RTS game players. i ACKNOWLEDGMENTS First, I would like to thank my main supervisor Dr. Stefan J. Johansson for invaluable support and guidance throughout the work. I would also like to thank Professor Paul Davidsson who was my secondary supervisor in my early work, and Professor Craig Lindley for taking over his role. In addition I would like to thank the community at AIGameDev.com and the users of BTHAI for comments and ideas on the project. Last but not least this thesis wouldn’t have been much without the support of my family, Maria and Mio, and our dog for forcing me to take refreshing walks every day. Karlskrona, December 2011 Johan Hagelbäck iii PREFACE This thesis is a compilation of nine papers. The papers are listed below and will be referenced to in the text by the associated Roman numerals. The previously published papers have been reformatted to suit the thesis template. I. J. Hagelbäck and S. J. Johansson (2008). Using Multi-agent Potential Fields in Real- time Strategy Games. In L. Padgham and D. Parkes editors, Proceedings of the Seventh International Conference on Autonomous Agents and Multi-agent Systems (AAMAS). II. J. Hagelbäck and S. J. Johansson (2008). Demonstration of Multi-agent Potential Fields in Real-time Strategy Games. Demo Paper on the Seventh International Conference on Autonomous Agents and Multi-agent Systems (AAMAS). III. J. Hagelbäck and S. J. Johansson (2008). The Rise of Potential Fields in Real Time Strat- egy Bots. In Proceedings of Artificial Intelligence and Interactive Digital Entertainment (AIIDE). IV. J. Hagelbäck and S. J. Johansson (2008). Dealing with Fog of War in a Real Time Strat- egy Game Environment. In Proceedings of 2008 IEEE Symposium on Computational Intelligence and Games (CIG). V. J. Hagelbäck and S. J. Johansson. A Multi-agent Potential Field based bot for a Full RTS Game Scenario. In Proceedings of Artificial Intelligence and Interactive Digital Entertainment (AIIDE), 2009. VI. J. Hagelbäck and S. J. Johansson. A Multiagent Potential Field-Based Bot for Real- Time Strategy Games. International Journal of Computer Games Technology, vol. 2009, Article ID 910819, 10 pages. doi:10.1155/2009/910819 VII. J. Hagelbäck. An expandable multi-agent based architecture for StarCraft bots. Submit- ted for publication. VIII. Johan Hagelbäck and Stefan J. Johansson. Measuring player experience on runtime dy- namic difficulty scaling in an RTS game. In Proceedings of 2009 IEEE Symposium on Computational Intelligence and Games (CIG), 2009. IX. Johan Hagelbäck and Stefan J. Johansson. A Study on Human like Characteristics in Real Time Strategy Games. In Proceedings of 2010 IEEE Conference on Computational Intelligence and Games (CIG), 2010. The author of the thesis is the main contributor to all of these papers. v CONTENTS Abstract i Acknowledgments iii Preface v 1 Introduction 1 1.1 Background and Related Work . .4 1.1.1 Design and Architectures . .4 1.1.2 Navigation . .4 1.1.3 Gameplay experience . .8 1.2 Research Questions . 10 1.3 Research Methods . 11 1.4 Contributions . 12 1.4.1 RQ1: How does a MAPF based bot perform compared to traditional solu- tions? . 12 1.4.2 RQ2: To what degree is a MAPF based bot able to handle incomplete information of a game world? . 13 1.4.3 RQ3: How can a MAPF based RTS bot architecture be designed to support flexibility and expandability? . 13 1.4.4 RQ4: What effects does runtime difficulty scaling have on player experi- ence in RTS games? . 14 1.4.5 RQ5: What are important characteristics for human-like gameplay in RTS games? . 14 1.5 Discussion and Conclusions . 15 1.5.1 Potential Fields . 15 1.5.2 RTS game architectures . 17 1.5.3 Gameplay experience in RTS games . 18 1.6 Future Work . 18 vii 2 Paper I 21 2.1 Introduction . 21 2.2 A Methodology for Multi-agent Potential Fields . 23 2.3 ORTS . 24 2.4 MAPF in ORTS . 25 2.4.1 Identifying objects . 25 2.4.2 Identifying fields . 25 2.4.3 Assigning charges . 25 2.4.4 On the granularity . 31 2.4.5 The unit agent(s) . 32 2.4.6 The MAS architecture . 32 2.5 Experiments . 36 2.5.1 Opponent Descriptions . 36 2.6 Discussion . 37 2.6.1 The use of PF in games . 38 2.6.2 The Experiments . 39 2.6.3 On the Methodology . 39 2.7 Conclusions and Future Work . 40 3 Paper II 41 3.1 The ORTS environment . 41 3.2 The used technology . 42 3.3 The involved multi-agent techniques . 45 3.4 The innovation of the system . 45 3.5 The interactive aspects . 45 3.6 Conclusions . 46 4 Paper III 47 4.1 Introduction . 47 4.2 ORTS . 48 4.2.1 The Tankbattle competition of 2007 . 49 4.2.2 Opponent descriptions . 49 4.3 MAPF in ORTS,V.1................................ 49 4.3.1 Identifying objects . 50 4.3.2 Identifying fields . 50 4.3.3 Assigning charges . 50 4.3.4 Granularity . 52 4.3.5 Agentifying and the construction of the MAS ............... 52 4.4 Weaknesses and counter-strategies . 52 4.4.1 Increasing the granularity, V.2 . 53 4.4.2 Adding a defensive potential field, V.3 . 55 4.4.3 Adding charged pheromones, V.4 . 56 4.4.4 Using maximum potentials, V.5 . 56 4.5 Discussion . 57 4.5.1 Using full resolution . 57 4.5.2 Avoiding the obstacles . 57 4.5.3 Avoiding opponent fire . 57 4.5.4 Staying at maximum shooting distance . 58 4.5.5 On the methodology . 58 4.6 Conclusions and Future Work . 59 5 Paper IV 61 5.1 Introduction . 61 5.1.1 Research Question and Methodology . 62 5.1.2 Outline . 62 5.2 ORTS . 62 5.2.1 Descriptions of Opponents . 63 5.3 Multi-agent Potential Fields . 63 5.4 MAPF in ORTS . 64 5.4.1 Identifying objects . 64 5.4.2 Identifying fields . 64 5.4.3 Assigning charges . 65 5.4.4 Finding the right granularity . 66 5.4.5 Agentifying the objects .