Making and Acting on Predictions in StarCraft: Brood War Bachelor of Science Thesis in Computer Science and Engineering Henrik Alburg Filip Brynfors Florian Minges Bjorn¨ Persson Mattsson Jakob Svensson Department of Computer Science and Engineering Chalmers University of Technology University of Gothenburg Gothenburg, Sweden, June 2014 The Authors grant to Chalmers University of Technology and University of Gothenburg the non-exclusive right to publish the Work electronically and in a non-commercial purpose make it accessible on the Internet. The Authors warrant that they are the authors to the Work, and warrant that the Work does not contain text, pictures or other material that violates copyright law. The Authors shall, when transferring the rights of the Work to a third party (for example a publisher or a company), acknowledge the third party about this agreement. If the Authors have signed a copyright agreement with a third party regarding the Work, the Authors warrant hereby that they have obtained any necessary permission from this third party to let Chalmers University of Technology and University of Gothenburg store the Work electronically and make it accessible on the Internet. Making and Acting on Predictions in StarCraft: Brood War Henrik Alburg Filip Brynfors Florian Minges Bjorn¨ Persson Mattsson Jakob Svensson c Henrik Alburg, June 2014 c Filip Brynfors, June 2014 c Florian Minges, June 2014 c Bjorn¨ Persson Mattsson, June 2014 c Jakob Svensson, June 2014 Examiner: Jan Skansholm Chalmers University of Technology University of Gothenburg Department of Computer Science and Engineering SE-412 96 Gothenburg Sweden Telephone + 46 (0)31-772 1000 Department of Computer Science and Engineering Gothenburg, Sweden June 2014 Abstract Making predictions and strategic decisions is not just a problem in real life, but also in the com- plex environments of most real-time strategy games. Due to the lack of complete information, such games can be used as testbeds for AI research in this area. This thesis presents how an AI agent, in the game StarCraft, can make use of a Bayesian network to predict the state of the opposing player's technology tree. It is also shown how the predictions can be used to assist the AI agent in making strategic decisions. Bayesian networks are used for the predictions, and the agent's army composition is generated dynamically based on the predictions and the observed opponent units. The agent is tested against StarCraft's built-in AI. The results show that it is possible to accurately predict the state of the opponent's technology tree, and that the predic- tions have a positive effect on the AI agent's win rate. Sammandrag Att g¨ora f¨oruts¨agelser och fatta strategiska beslut ¨ar inte bara ett problem i vardagliga situ- ationer, utan ocks˚ai de komplexa milj¨oer som finns i m˚angarealtidsstrategispel. P˚agrund av aspekter som exempelvis inkomplett information kan s˚adana spel anv¨andas som testmilj¨o f¨or AI-forskning inom komplexa system. Denna kandidatrapport presenterar hur en AI-agent i spelet StarCraft kan anv¨anda Bayesiska n¨atverk f¨or att f¨oruts¨aga motst˚andarens teknologitr¨ad. Rapporten redog¨or ocks˚af¨or hur f¨oruts¨agelser kan anv¨andas som underlag f¨or beslutsfattande. Bayesiska n¨atverk anv¨ands f¨or f¨oruts¨agelser, och AI-agentens arm´eanpassas dynamiskt baserat p˚af¨oruts¨agelserna och de observationer som g¨ors av motst˚andaren. Agenten testas mot den inbyggda AI som finns i StarCraft. Resultaten visar att det ¨ar m¨ojligt att noggrannt f¨oruts¨aga motst˚andarens teknologitr¨ad och att dessa f¨oruts¨agelser positivt p˚averkar matchernas utg˚angn¨ar de anv¨ands i beslutsfattandet. Acknowledgements \There are only two types of people, those who can extrapolate from incomplete data." - Unknown We would like to thank our supervisor Andreas Abel for his support throughout the project. We would also like to thank the department of Language and Communication at Chalmers Uni- versity of Technology for their continuous effort to improve our writing, as well as Astrid Hyl´en and P¨ar Persson Mattsson for their valuable feedback on the report. The Authors, Gothenburg 2014-06-05 Contents 1 Introduction 1 1.1 Background.......................................1 1.2 Purpose.........................................3 1.3 Scope of the project..................................3 2 Fundamentals of StarCraft4 2.1 Real-time strategy games...............................4 2.2 StarCraft........................................4 3 AI development in StarCraft9 3.1 Bayesian networks...................................9 3.2 Case-based reasoning..................................9 3.3 AI research in RTS games............................... 10 3.4 Existing StarCraft AI agents............................. 11 3.5 The design of UAlbertaBot.............................. 13 4 Method 16 4.1 Modifications of UAlbertaBot............................. 16 4.2 Implementing predictions............................... 18 4.3 Implementing strategic decision-making....................... 20 4.4 Build Order Bot.................................... 21 5 Tests 23 5.1 Predictions....................................... 23 5.2 Opening strategies................................... 26 5.3 Army composition generator.............................. 29 5.4 The attack-defend state uncertainty factor...................... 29 6 Results and discussion 31 6.1 Predictions....................................... 31 6.2 Opening strategies................................... 31 6.3 Army composition generator.............................. 32 6.4 The attack-defend state uncertainty factor...................... 33 6.5 Possible sources of errors................................ 33 6.6 Problems during implementation........................... 34 6.7 Contributions...................................... 35 6.8 Future work....................................... 35 7 Conclusion 37 References 38 Appendices 41 A Tools 41 B StarCraft maps used for testing 43 C Unit counters 44 D Opening build orders 46 E Prediction results 48 1 Introduction Artificial Intelligence (AI) has reached the level where computers are able to beat top human players in games such as Chess. In 1997, the supercomputer Deep Blue beat the reigning world champion Garry Kasparov in a six-game match of Chess [1]. In order to win, Deep Blue made use of a problem solving technique called brute-force which systematically checks all possible solutions and picks the best one. However, solving problems by using brute-force is often impos- sible for intelligent agents in real-life situations due to the complex environments in which they act. The number of solutions to evaluate is simply too large even for most modern computers, and with increasingly complex environments the number grows even larger [2]. As a result, the situation might have changed before the AI is able to find the optimal, but no longer valid, solution. Further, complete information is not available in many real-life situations. A chess player will know for certain where the pieces are on the board, but the driver of a car can never be sure if there is an obstacle behind a house corner or not. An AI designed to tackle more complex problems, such as those encountered in real life, must be able to handle incomplete knowledge. At the same time it must also use techniques that quickly generate functioning solutions before the situation has changed. In order to handle these requirements, many AI agents strive to find a near-optimal solution, one that is good enough for its purpose, instead of trying to find the absolute optimal solution. When investigating different algorithms and methods for letting AI agents interact in complex environments, it is useful to have a platform where it is possible to test the different techniques. As an alternative to developing specialised simulators, some researchers look at modern games as a possible arena for testing the AI agents [3]. Games put players in situations where they need to make advanced decisions, often quickly and while dealing with limited resources. One genre of game that focuses on long-term strategic planning while still forcing the player to constantly make short-term decisions is Real-Time Strategy (RTS). In RTS games, events happen continuously, as opposed to most board games which are turn-based. One such game, StarCraft: Brood War, is used in this project. The RTS genre and StarCraft is explained more thoroughly in section 2.1 and 2.2. 1.1 Background The behaviours of AI agents are heavily affected by the architecture with which they are built. Some agents make use of scripts; lists of commands that are executed sequentially. Scripting is a method that is commonly used in the game industry since scripts are understandable, predictable, and easy to implement and extend [4]. However, scripts are static and are thus often easy for players to exploit. Other agents are modelled by finite state machines, where the behaviour is defined by predefined states which the agent can transition between (see Figure1). Finite state machines are practical to use in cases where the behaviour is well-defined [5,6]. However, they are also static in the sense that a finite state machine is only transitioning between predefined states. Yet another method is machine learning, the science of getting computers to act with- out being explicitly programmed. By applying machine learning, developers try to give the AI agent an instrument to learn from experience. One example is case-based reasoning, a technique which will be discussed more in detail in section 3.2. Many previous works have explored the implementation of case-based reasoning in RTS games, but very few have managed to create a successful AI agent executing the behaviour [7,8]. 1 Figure 1. Example of a finite state machine. Graphics adapted from Martinez [9]. One problem in many RTS games is also that the AI agents cheat in one way or another. The goal of most game companies is to earn money by selling games, and it is often both easier and cheaper to tune the difficulty of the AI by letting it cheat in a controlled manner, rather than to spend development time on creating an AI that plays the game on the same terms as a human.