Developing a 5v5 Framework for DotA 2 Bot Competition
Dennis Nilsson Kalle Lindqvist Malmö University Malmö University [email protected] [email protected]
Faculty of Science Department of Computer Science Master Thesis in Computer Science, 15 credits 2020-05-18 Supervisors: José Maria Font Fernandez & Alberto Enrique Alvarez Uribe Examiner: Johan Holmgren CONTENTS
I Introduction 3 I-A Motivation ...... 3 I-B Goals ...... 4 I-C Research Questions ...... 4
II Literature Review 4 II-A Data collection ...... 4 II-A1 Game Conferences ...... 4 II-B Systematic search ...... 4 II-B1 Keywords, search string and restrictions ...... 5 II-B2 Databases ...... 5 II-B3 Selection of papers ...... 5 II-C Discussion and results ...... 5 II-C1 Applying artificial intelligence algorithms in MOBA games ...... 5 II-C2 Evolving a Designer-Balanced Neural Network for Ms PacMan ...... 6 II-C3 An Analysis of Artificial Intelligence Techniques in Multiplayer Online Battle Arena Game Environments ...... 6 II-C4 MOBA: a New Arena for Game AI ...... 6 II-C5 MOBA games: A literature review ...... 7 II-C6 On The Development of Intelligent Agents For MOBA ...... 7 II-C7 What Contributes to Success in MOBA games? An Empirical Study Of Defense of the Ancients 2 ...... 7
III Research Method 7 III-A Design Science ...... 7 III-B Experiment ...... 8 III-C Threats to validity ...... 8
IV Frameworks 8 IV-A 1v1 framework ...... 8 IV-B 5v5 framework ...... 10 IV-B1 DotA 2 Addon ...... 10 IV-B2 Web API ...... 10 IV-B3 Client ...... 10
V Experiments 10 V-A Experiment 1 - Hero movement ...... 11 V-B Experiment 2 - Level up hero abilities ...... 11 V-C Experiment 3 - Hero attack ...... 11 V-D Experiment 4 - Buy items ...... 11 V-E Experiment 5 - Sell items ...... 11 V-F Experiment 6 - Use items ...... 11 V-G Experiment 7 - Target Types ...... 11 V-G1 Experiment 7.1 - No Target ...... 11 V-G2 Experiment 7.2 - Toggle ...... 11 V-G3 Experiment 7.3 - Target Point ...... 11 V-G4 Experiment 7.4 - Target Area ...... 11 V-G5 Experiment 7.5 - Target Unit ...... 11 V-G6 Experiment 7.6 - Vector Targeting ...... 12 V-G7 Experiment 7.7 - Combination of Target Point or Target Unit ...... 12 V-G8 Experiment 7.8 - Target Unit with Area of Effect ...... 12 V-H Experiment 8 - Validating usage of all heroes ...... 12 V-I Experiment 9 - Full length match ...... 12
1 VI Experiment results 12 VI-A Experiment 1 - Hero movement ...... 12 VI-B Experiment 2 - Level up hero abilities ...... 12 VI-C Experiment 3 - Hero attack ...... 13 VI-D Experiment 4 - Buy items ...... 13 VI-E Experiment 5 - Sell items ...... 13 VI-F Experiment 6 - Use items ...... 13 VI-G Experiment 7 - Target types ...... 13 VI-H Experiment 7.1 - No Target ...... 13 VI-I Experiment 7.2 - Toggle ...... 13 VI-J Experiment 7.3 - Target Point ...... 14 VI-K Experiment 7.4 - Target Area ...... 14 VI-L Experiment 7.5 - Target Unit ...... 14 VI-M Experiment 7.6 - Vector Targeting ...... 14 VI-N Experiment 7.7 - Combination of Target Point and Target Unit ...... 14 VI-O Experiment 7.8 - Combination of Target Unit and Target Area ...... 14 VI-P Experiment 8 - Full length match ...... 15 VI-Q Experiment 9 - Validating usage of all heroes ...... 15
VII Discussion 15 VII-A Future work ...... 15
VIII Conclusion 15
References 16
2 Abstract—Multiplayer online battle arena (MOBA) games have paths are referred to as lanes. The three lanes are called bot properties that make them suitable for research in artificial and lane, mid lane, top lane. Figure 1 gives a graphic representation computational intelligence. MOBA games are generally played of the DotA 2 map, presenting the meaning of lanes and team vs team meaning planning and organizing the team is one of the most important aspects. This makes it suitable for research towers. Mid lane is usually occupied by 1 player from each in artificial intelligence agent cooperation. team. Bot and top lane is usually occupied by 2 players from This paper presents a literature review performed in the area each team. On top of this there is also a jungle in the map of artificial and computational intelligence regarding MOBA where players can kill neutral creeps for more experience games. The findings are that there is little or no research made and gold. Players can make their own modifications to the in the area. The research found concerning MOBA games is in the context of player behaviour and toxic behaviour in the game in the programming language Lua [2]. The Lua sandbox, MOBA game league of legends. What is found is encouragement included in the workshop tools that is downloadable content, on developing frameworks supporting 5v5 matches in MOBA provides a comprehensive API to control units and view the games which this paper also presents, a 5v5 framework for bot- game state [3]. matchmaking in DotA 2 making it possible for users to develop bots to be run versus the in-game AI. Keywords—ai agent framework, DotA 2, moba, agent cooper- ation
I.INTRODUCTION The topic for this thesis is research within computational intelligence and artificial intelligence. Artificial intelligence has been used to play games for a long time. As mentioned by Julian Togelius in [1], Alan Turing re-invented the Minimax algorithm to play Chess. Furthermore it sheds light on the suitability games provide for artificial intelligence research. This is due to their ability of challenging players and training the cognitive capabilities of players [1]. Different games provide different challenges. Some games challenge one player and some games challenge a team of players. Therefore the research that can be done could differ depending on which game is chosen. In this thesis a framework has been developed to provide a platform for research of artificial intelligence teamwork. The chosen game for this framework is DotA 2. Conveniently Fig. 1. Overview of the DotA 2 map. Radiant team is marked in green and a framework exists for DotA 2 but it can only handle 1v1 Dire team in red. The circles in green and red represent the teams defending matches which takes away the ability to research teamwork of towers [4] artificial intelligent agents. DotA 2 is a MOBA game where the ultimate goal is A. Motivation to destroy the opponent’s base. The opponent’s base, called Ancient, is positioned at the opposite end of the map and Ms Pac-Man [5] is a framework for competition in Ms outputs creeps that pushes the lanes during the game. Creeps Pac-Man. This is partly used for competition because of are in-game controlled AI units that are either neutral or lane the properties of the game. Research within computational creeps. Neutral creeps are found in the jungle in small camps and artificial intelligence fields is made by the help of this and provide the possibility for players to farm experience and framework. The complexity of the game is fitting to research gold. The lane creeps, spawning every 30 seconds, pushes all due to its challenges in real-time decision-making and hetero- lanes automatically towards the enemy’s Ancient. A match geneous player types. There are other AI-agent frameworks traditionally consists of two teams each having their own base used in competitions around the world such as the TAC SCM to protect and an opponent’s base to attack. Each team consists framework [6]. The idea behind these competitions is to use of five players. Each player controls their own hero i.e. a them as a testbeds for new techniques and algorithms that playable character with multiple abilities and different play can be applied, not just to game AI field, but rather the field styles. Each player is assigned a courier, used to buy and of AI research [7]. As an example, in 2017, researchers at transport items to the player, enabling the player to stay in it’s Microsoft developed an AI-agent for Ms Pac-Man that scored lane without going back to base to buy items. A hero levels the highest score ever. Rahul Mehrotra [8] who is a product up during the match, increasing it’s strength and abilities. manager and AI-researcher at Microsoft later stated that the Furthermore the heroes can be equipped with items that are algorithm developed to beat Ms. Pac-Man could be used to bought with earned gold. The gold is earned by farming, help a company’s sales organization make precise predictions destroying enemy towers and/or heroes. Each team shares about which potential customers to target at a particular time three paths leading to the enemy’s Ancient structure. These or a particular day [9].
3 A framework supporting 1v1 matches currently exists for II.LITERATURE REVIEW DotA 2, made by Tobias Mahlmann [10]. This is a good starting point but since it’s only possible to use in 1v1 it lacks A. Data collection the ability of teamwork and therefore also the possibility of The data has been collected from game conferences and by testing algorithms that involve multiple agent cooperation and conducting a systematic literature review. team decision making. Teamwork is an essential part of DotA 1) Game Conferences: Competitions held at IEEE COG 2 in which matches usually aren’t played in 1v1. To be able to 2019 [14], one competition that was held is not mentioned evaluate computational and artificial intelligence concerning here due to its irrelevance. teamwork in DotA 2 a framework supporting 5v5 matches must therefore be developed. • 4th Angry Birds Level Generation Competition [15] • Bot Bowl I [16] Another important aspect is that the prize pool in DotA 2 • Fighting Game AI Competition [17] [11] is very large and increases every year. Every year the • First TextWorld Problems: A Reinforcement and Lan- base prize pool is set at $1,600,000. The year 2019, the prize guage Learning Challenge [18] pool went up to $34,330,068 compared to 2018’s prize pool of • Geometry Friends Game AI Competition [19] $25,532,177. The importance of this aspect is that it implies • General Video Game AI Competitions [20] the popularity of the game. There is a project, OpenAI Five, • Hanabi Competition [21] being used for developing AI systems using DotA 2 as a • Hearthstone AI competition [22] research platform. On April 13th 2019, OpenAI Five defeated • microRTS AI Competition [23] the current world champions in a best-of-three match. This was • StarCraft AI Competition [24] the first time an AI system won against world champions in an • Strategy Card Game AI Competition [25] esports game. Furthermore it won over 99.4% of encountered human players in a showcase that lasted a couple of days. The Competitions held at FDG 2019 [26] name OpenAI Five is somewhat confusing due to the fact that • The Codenames AI Competition [27] it is unavailable to the public which is easily misinterpreted • Fifth Aeon Collectible Card Game [28] due to its name. This prevents others from performing research • GDMC / Chronicle Challenge [29] using the OpenAI Five framework which is one of the strong Coveted competition papers sought for at TCIAIG 2019 motivations for developing our 5v5 framework [12]. [30].
• Angry Birds Level Generation B. Goals • Computer Game Olympiads (including Chess, Amazons, The main goal of this thesis is to upgrade the framework Backgammon, Bridge, Chinese Chess, Dots and Boxes, that was developed by Tobias Mahlmann [10] to be able Draughts, Go, LOA, Shogi, ...) to handle 5v5 matches with bot communication in DotA 2. • DotA 2 Bot The framework [13] in its current state uses a client/server • Fighting Game AI implementation that communicates via HTTP calls. The client • Game Data Mining hosts a web service that the server sends information to in • General Video Game AI JSON format using HTTP POST calls and get responses/ac- • Geometry Friends Cooperative Game AI tions using HTTP GET calls. There is an example Client • microRTS AI developed in Java by the author of the framework but the client • Ms. Pac-Man VS Ghost Team code can be written in any language as long as it implements • Showdown AI the functionality required to handle the communication with • StarCraft AI the server. The subgoals are making sure that all components • Text-based Adventure AI function as intended. This includes that our artifact allows all • Visual Doom AI heroes to execute all of their functionalities i.e. their individual As evident by our findings in these game conferences, it abilities and basic actions. Furthermore validating that the lacks a 5v5 DotA 2 bot competition which is due to the fact communication works as intended between the bots within the that a 5v5 framework does not exist for public use. TCIAIG framework. called for DotA 2 bots submissions in 2019 but did not receive any submissions. This is possibly because a 5v5 framework C. Research Questions was not available. A 1v1 framework was available at the time [10]. 1) How can the 1v1 DotA 2 AI Framework project be extended to support 5v5? B. Systematic search 2) How can the intelligent agents communicate with each other in the extended framework? In addition to the data collected from the game conferences, 3) How could a multiple agent strategy be implemented in a systematic literature review was conducted to find additional the 5v5 framework? research related to our topic.
4 TABLE I TABLE II LISTOFKEYWORDS THEINITIALSEARCHRESULTS
Keyword Synonyms Database Results Defence of the ancients 2 DotA 2 ACM 16 Heroes of Newerth HoN IEEE Xplore 8 Ms Pacman Google Scholar 50k+ Minecraft Total 25 + 50k+ League of legends LOL Framework MOBA In the first stage the titles and abstracts of the articles found Intelligent agents were read during the initial search and a decision was made to determine if there was a chance the article could contribute 1) Keywords, search string and restrictions: The keywords anything to our research. The ones that seemed promising used to compile the search string are the following and were moved on to the second stage. extracted from the research questions and motivation: In the second stage each of the remaining articles were read. The search string was compiled using the keywords listed If an article contained information that were relevant to our in table I and using Boolean operators to make it easy for the research it was included in our paper. reader to understand what was searched for as recommended by Neiva and Silva [31]. The way it’s used in this literature TABLE III RESULTS AFTER THE FIRST AND SECOND STAGE OF THE SELECTION OF review is the same way an SQL query is executed. The PAPERS following search string was used: Source Initial results First stage Second stage • ((Defence of the ancients 2 OR DotA 2) OR (Heroes of ACM 16 3 1 Newerth OR HON) OR (Ms Pacman) OR (Minecraft) IEEE Xplore 8 4 2 OR (League of legends OR LOL) OR MOBA) and Google Scholar 50k+ 7 4 Total 24 + 50k+ 14 7 (Framework OR Intelligence agents) The search string was adapted to the mechanics used to As can be seen in table III the search in Google Scholar search each of the different databases. The search string was yielded a massive amount of articles and since it’s not feasible used to match abstracts, titles and keywords of potential to read them all, we read the results in sequence until the papers. results were no longer relevant. In addition to the search string the following restrictions were applied when searching the sources for articles: TABLE IV THE FINAL RESULTS AND THE SOURCES THEY WERE FOUND IN • Only articles written in English • Only articles referenced in at least one other confer- Source Reference ence(s) or journal(s) ACM [35] IEEE Xplore [36] The reason for the first restriction is that English is the IEEE Xplore [37] language this thesis is written in and most works in the field Google Scholar [38] Google Scholar [39] of computer science are published in English. The second Google Scholar [40] restriction is used as a simple quality assurance, if a paper is Google Scholar [41] referenced by external source(s) we can at least be reasonably sure that it is reviewed by one other party. C. Discussion and results 2) Databases: The following three databases were used in As evident by the findings in the game conferences and this literature review: literature review a framework supporting 5v5 in DotA 2 1) Google Scholar [32] could not be found. Submissions for a competition in a 1v1 2) ACM [33] match in DotA 2 was sought for by TCIAIG 2019 [30] 3) IEEE Xplore [34] but none were received. This possibly depends on DotA We selected ACM and IEEE Xplore since they, to our 2 being a game that relies on teamwork and at the time knowledge, contain the largest amount of published literature the framework only supported 1v1 matches. For those who in the field of computer science. Additionally, to cover the wanted to compete with their AI in 1v1 matches there are most ground possible, Google Scholar was used which aggre- possibly more interesting competitions to join such as the gates articles from multiple sources. Starcraft AI competition. 3) Selection of papers: The gathered results were subjected to a two stage analysis that ultimately determined if they would 1) Applying artificial intelligence algorithms in MOBA be included in this paper. The goal was to select at least 5 games: In their paper on AI algorithms in MOBA games articles containing the most relevant information related to our [38], authors Niewiadomski et al. highlight the fact that most research. players in MOBA games prefer to play against bots rather than
5 human competition due to the widespread cyberbullying in To realise this they suggest a system that predicts the op- online MOBA games. This further strengthens our claim that posing team’s strategy i.e. which lanes their heroes would a 5v5 bot framework is needed. Related to our specific topic, occupy. On top of this they suggest a counter strategy in Niewiadomski et al. also highlight the need for a framework case the initial guess is not correct, to minimize the risk to have fast response times and provide quick updates on the of losing the match. state of the game so the agent can make the needed sub- • Laning Phase, the challenge here is to determine when to second decisions needed in the heat of the battle. Furthermore buy new items, when to fight enemy heroes and where to the paper concludes, by experiments, that the most efficient position the hero. Furthermore to be aware of the team algorithm in terms of winning the game for map navigation heroes status, for instance if they are in a fight and in out of Bruce Force, Directional, Split, Monte Carlo and GA, need of assistance. the Split algorithm was the most efficient. The authors describe • Mid Game, when mid game is reached, the laning phase the algorithm in the following way: is traditionally over. The challenges of this phase can “The Split algorithm starts from computing single steps differ in many ways because it depends on several factors. using all available directions. Then, every obtained point is However they state that the most important aspect is the explored further, but only in two, new directions. An angle cooperation of the team and if they are in sync. Regarding formed between the new directions is the same as the angles this they present that it is a difficult task due to the between the subsequent directions of the first step” [38]. complexity of actions that can be made by the players, 2) Evolving a Designer-Balanced Neural Network for Ms both opposing and friendly. PacMan: In their paper [36] Morsan et al. presents a tool that • Late Game, last phase of the match. The challenges found combines several 3rd party tools made by researchers into a in this phase is fast decision making. The complexity common interface. This tool can be used to generate AI agents of the combats increases due to the capacity of the of different levels of difficulty for any game. The tool requires heroes. In late game, the heroes are stronger and able to a lot of user input and validation to work but the authors claim fight in more complex ways. Due to the complexity, the that it’s a great way to develop AI agents quickly. The authors challenges of the AI agents would be to make decisions validate their tool with experiments generating agents for the and predict enemy strategies as fast as a professional game Ms Pac-Man. human player would. 3) An Analysis of Artificial Intelligence Techniques in Mul- • Side Challenges, the challenges that are inherent in every tiplayer Online Battle Arena Game Environments: Waltham match due to the nature of the game. The challenges they et al. tried to incorporate an emotional element in their paper point out is the combat outcome of team fights or solo about AI techniques in MOBA games [35]. The authors fights, if a set of actions could kill an enemy, enemy hypothesis was that by creating an agent that behaves more like team strategy and winner of the phase. These challenges a human with emotions instead of performing as efficiently as are related to prediction and combat analysis. To deal possible. It would perhaps offer a more interesting experience with these challenges they suggest techniques such as for the human player(s) who play with or against it. This Ant Colony Optimization and Genetic Algorithms. technique seems promising because more and more players Furthermore they present a model to use to break down are playing against bots instead of human competition due to the gameplay of MOBA into components so that an AI agent toxic behavior in online games against human opponents [38]. could make decisions on micro and macro levels. An agent that mimics human behavior might offer a more The model they provide is formed by the following compo- engaging experience for those players. nents. 4) MOBA: a New Arena for Game AI: This paper investi- gates properties that MOBA games can provide to Game AI • Item building, to optimize this they suggest that the AI research. They mention the following phase challenges. agent can take the enemy team’s status and strategy in • Pick and Ban Phase, character picks are important be- account and decide which items to buy. cause they can possibly decide the outcome of the match. • Laning, to optimize this phase a system that can simulate The challenge of the AI agent is to select a composition micromanagement skills is proposed. This system should of heroes being able to cooperate with each other, be be able to teach the AI agent to control the lane, push effective on their own and be chosen in regard to the the lane, execute last hits for farm, taking multiple enemy opposite team’s chosen heroes. heroes into account and denying enemy hero farm. • • Opening Phase, the first items the hero buys determines Team fights, they suggest that the team fight model should its success on it’s lane. The optimal situation for a hero be structured as a one lane map and allow the AI agents when buying the first items would be to know which to communicate with each other. This model could then hero it would meet on it’s lane. Through experience a be used to simulate scenarios of team fights which could human player can guess which hero it would meet on it’s possibly enable the AI agents in a match to perform lane. The challenges, for an AI system, in this context efficient. is selecting the first items to buy, selecting five heroes This is valuable to our thesis because we have to develop bots and determining which lanes the heroes should occupy. in order to confirm that our 5v5 framework is fully functioning.
6 5) MOBA games: A literature review: The authors of this paper performed a literature review and presents a couple of insights valuable to this thesis. The player base and ac- cessibility of MOBA games is massive, meaning that it’s a good environment to perform research in. They state that improvements to the API of MOBA games could help future research or to provide data sets that are relevant to the topic. Furthermore they found that MOBA games, in general, are not investigated which indirectly provides a gap i.e. an opportunity for research. At the time the literature review was written they found that the research being made in the context of MOBA games primarly deals with player behaviour and toxic behaviour in league of legends. They also state that league of legends is the MOBA game that has received most attention in research. This strengthens the claim that a 5v5 framework is needed in the area. 6) On The Development of Intelligent Agents For MOBA: Fig. 2. Overview of the Information Systems Research Framework [42] The authors of this paper [37] have identified that using In- fluence Maps is a common approach used for tactical analysis for agents in other game genres but not yet commonly used The problems identified by the research is defined in the in MOBA. It’s a method to map out desirable and undesirable environment. This column contains persons, organizations and locations on a map. The authors try this approach together the existing or planned technology which in our case is the with what they call a “two-layered agent architecture” and 1v1 version of the DotA 2 framework developed by Tobias perform experiments to measure it’s effectiveness. The results Mahlmann [13] and also our supervisors José Font and Alberto are however inconclusive and until further research is done Enrique Alvarez Uribe. The knowledge base contains the on influence maps and “two-layered agent architecture” in a foundations upon which we conduct our research, which in our MOBA context we believe this approach should be avoided case is the knowledge gathered in section II. The knowledge when developing AI agents. base also contain our methodologies which is the basis for 7) What Contributes to Success in MOBA games? An how we will conduct and validate our research. We discuss Empirical Study Of Defense of the Ancients 2: In their paper our method in length in section III-B. The environment and [41] the author examines what contributes to victory in DotA knowledge base are used to develop theories and artifacts to 2 matches. Among the results presented by the author it is justify and evaluate them. This is an iterative process that mentioned that kills made by multiple players and the usage continues until we have completed our research. Each iteration of wards contributes heavily on whether the outcome is a win expands our knowledge base and adds to our environment, this or defeat. This information could possibly be used to optimize is illustrated in figure 2. the efficiency of a bot developed for our framework.
III.RESEARCH METHOD
In this section a discussion regarding the chosen research methodology Design science, research approach and data collection is presented.
A. Design Science The methodology used in this thesis is design science [42]. The reason design science is elected is that it fits this thesis. It is motivated by the process of creating an artifact and evaluating it in iterations which according to March and Smith Fig. 3. Three cycle view of design science research [44] [43] are the two basic activities of design science. Design science originates from the natural sciences and was Each iteration described above can also be divided into three situated in information systems research by Hevner et al. [42] sub-cycles [44] as shown in figure 3. The rigor cycle provide in a framework called Information Systems Research Frame- scientific grounding and additions to the knowledge base. The work. When research is conducted in the field of information relevance cycle makes sure the research is relevant in regard science Hevner et al. writes that the process is best approached to requirements and by doing field tests. The design cycle is using conceptional frameworks. where we build and evaluate the artifacts.
7 Writing this thesis consisted of doing the following four the latest version to be able to run the game, we can not major iterations: be certain that our framework will be compatible with future • Performing a systematic literature review to identify the versions of DotA 2. We know for a fact that an update to gap in existing research regarding a DotA 2 5v5 bot- DotA 2 has broken the 1v1 framework in the past due to matchmaking framework. The results of the systematic renaming of API functions that the framework was using to literature review is presented in section II. control the game. We hope we can mitigate this by releasing • Selecting an appropriate methodology for conducting our our framework as open-source under a permissive licence once research which is explained in section III-A. we have finished developing it. Hopefully a third party will be • Developing and testing the artifact. This iteration consists interested in the functionality of the framework and keep it in of many smaller iterations where we confirm that the new sync with eventual future changes to the game API or maybe functionality meets the requirements from the environ- even improve it, like we did with the 1v1 framework. ment, identifying potential problems and bugs and finally Since we only evaluate the functionality of our framework adding everything we learned to our knowledge base so and not it’s usability, we can’t know how easy or difficult it that we can develop it further in future iterations. is for other researchers and other interested parties to actually • Evaluating the functionality of our artifact with the exper- develop agents for it. There is no graphical user interface for iments defined in section III-B of which the results will our framework, the only way to understand and use it is to be used to answer our research questions and conclude read the code of our implementation and the provided example our research. agent. This might not be a problem, since other popular frameworks like Ms Pac-Man [5] and TAC SCM framework B. Experiment [6] rely on the user of the framework to read the code of the The chosen method for this thesis is experiment. The suit- example agents to implement their own. ability of this method for our thesis is that an artifact is devel- oped, a DotA 2 framework supporting 5v5 bot-matchmaking, IV. FRAMEWORKS and to be able to confirm that the artifact work as intended a During a 10-week period the 1v1 framework has been number of experiments are performed. The quantitative data extended to a 5v5 framework for bot-matchmaking in DotA 2. from the experiments are used to confirm functionality of We developed the framework in several iterations. During the the artifact and to answer the research questions. The data first iteration we expanded the framework to support 5v5 as it collected from the experiments are Boolean values. For every was the primary goal. During the second iteration we converted action executed by a bot the result is either true or false i.e. the Java web API and client framework to Python to give that the action has been executed successfully. The actions are future bot developers access to machine learning libraries that defined in the experiments since different experiments evalu- are available in the Python ecosystem. During our third and ates different actions available to the bots. The experiments final iteration we tested and debugged our framework to make are deemed successful when every action of every hero gives it ready for release. During each iteration we gained further the Boolean result true. knowledge of what could be applied to the next iteration Other quantitative methods, such as case studies or surveys, and discussed what should be done in the next iteration as are not as suitable for our artifact due to a number of reasons. described in section III-A and described in detail by Hevner A survey would possibly not contribute in confirming or deny- et al. [42]. ing that the artifact is functional in an efficient way. A possible scenario could be that, after the artifact is developed, a survey A. 1v1 framework is performed with participants in a made up competition. This would enable us to question the participants of their A 1v1 framework for bot-matchmaking has been developed experience of the artifact. Which could possibly lead to being by J. M. Font and T. Mahlmann and the description of overwhelmed with feedback about various functionalities that the framework is visualized in figure 5 [10]. DotA 2 sends are not working properly, previously untested due to running information about the current game state to the Lua sandbox no experiments, in a late stage of the artifact development. which is then sent using HTTP post calls, in JSON files, On the other hand, by running experiments, the answers to a web API. The web API reads the information given in to our research questions are answered without involving JSON files. This information is taken into account by the bot third parties giving us direct feedback. This will allow us implemented by its user which gives them the ability to choose to efficiently iterate the development of the artifact to ensure what actions the bot would execute. The actions are chosen that it functions as intended. Another important motivation and are converted to JSON and sent back to the Lua sandbox for using experiments, as opposed to a case study, is that an which then forwards it to the game. experiment is easily reproduced. An issue with their web API is that it is written in Java which demands a number of libraries to be able to compile. C. Threats to validity The usability aspect is affected by this since it requires the Due to the rapid development of DotA 2 leading to frequent user to set up the environment in the integrated development updates and the fact that all clients have to be updated to environment and installing libraries.
8 The original 1v1 framework had support for using items stored in the bot’s inventory. There was no support to actually see how many or which items were in the inventory. This made it near impossible to use items and there also was no functionality to target areas or other units with items, you could only use them on the bot which had them in it’s inventory. All this has been fixed in our adaptation of the framework. The one thing that we could not implement in regards to items is for the bots to use a courier to buy items. The documentation about couriers and their API is completely Fig. 5. A description of the architecture of the 1v1 framework [10] missing, or the functionality itself is not available within the Lua sandbox. This has prompted us to disable the couriers in our implementation as their existence in the game would provide an unfair advantage to the AI-bots of the enemy team, as the official bots are able to use them, since they are implemented in C++ and have access to the core functionality of the game. In practice this means that for a bot to buy items, it has to be inside the home base or in the vicinity of the shops scattered around the map. Another issue with the implementation of the Web API is the non-standard use of HTTP verbs. The best practices for developing web API’s is to use HTTP-verbs to describe the functionality of the different routes [45]. A description of the During the development of our experiments we also found best practices is shown in figure 4. and corrected numerous issues and bugs in the Lua code. The most prominent were:
• The system that was determining what kind of spell was being cast only had support to cast three different types of abilities, abilities you cast on yourself, abilities you cast on a specific area and abilities you cast on enemies. The functionality to cast toggle abilities was missing completely. • As mentioned in the section above the ability to see what is in each heroes inventory was missing. During testing Fig. 4. Description of the HTTP verbs and the actions they are associated with [46] we also found that the logic to use testing was just a method that redirected to the use ability logic, which did not work on items. • When serializing the world into JSON, the code that The 1v1 framework is using the POST verb for every single gathered information on buildings did not check if they route it has which adds unnecessary confusion as it implies had been destroyed which resulted in nil values in the that something is created and returned which is not always building array which in Lua signals that the array ends. the case as they use it in the route that updates the world. This meant that once buildings gets destroyed, a lot of This is connected with the issue that the 1v1 framework only buildings that are still standing are missing from the accounts for 1 bot being able to send its commands in time serialized game state. i.e. each game tick. In the extended framework this is taken • Toweraggro, which is when a hero gets attacked by a into account and for each HTTP POST all bot actions are tower, is information that is vital for the bots survival. computed locally and gathered into a single JSON file that The towers deal massive amount of damage and would is then sent back to the Lua sandbox. If the architecture had kill a bot if it does not flee or otherwise mitigate the been kept from the 1v1 that would resolve in 5 HTTP POST situation. There are facilities to detect this in the Lua calls for the bots. game API but this was never serialized in the game state.
9 B. 5v5 framework and what is visible to each individual bot into account. This information is serialized and sent to the web API where it is processed. The final action during each game tick is to gather eventual commands from the web API and translate it into actions to be performed by the in-game bots. All actions are individual to each bot, which means that to make multiple bots move as a single unit you would have to implement that in the Python implementation of the bot. We have implemented the following seven actions, available to each hero in team good guys: • Buy item Buy an item from the store available to each team. • Sell item Sell an item in the above mentioned store. • Use item Use an item from the heroes inventory. • Move to Move the hero to specific coordinates on the game map. • Level up Level up the hero by increasing the ability points invested in a specific ability. • Attack Make the hero use it’s standard attack on an enemy hero or creep. • Use ability Use a hero ability. Available targets can be itself, friendly units and enemy units depending on what type of ability is being used. 2) Web API: The web API makes use of Bottle [49], a Python library distributed as a single module. Bottle is a web Fig. 6. A visualisation of the architecture of our framework framework that offers the use of the HTTP calls POST, GET, PUT, DELETE. The routes of the web API is listed in the Our implementation of the framework can be divided into following bullet list. four subsystems. In this section we provide a detailed descrip- • The web API receives JSON data from the Lua sandbox tion of the implementation of each subsystem. In figure 6 an and sends it to the client overview of the framework is presented. • The client decodes the JSON data and provides the We rewrote most parts of the 1v1 framework as it was information to the bot scripts not developed with future extension to support more bots. • The bot scripts decides which actions to take This gave us the opportunity to decouple the subsystems more • The actions are chosen and JSON data containing the loosely to allow for future developers to implement their own information is created versions. However, when we submit our framework for AI • The JSON data is sent back to the Lua sandbox game developer competitions our intentions is that our full • The Lua sandbox provides the game with the information implementation will be used by the competitors. Our addon 3) Client: The functionality of the client is that it uses [47] and framework [48] are hosted on Github and can be one DotA 2 client to connect to the game. When this client visited by following the links in the reference section. connects to the game, the in-game AI bots fill up the server. 1) DotA 2 Addon: Addons for DotA 2 are written in the The bots in our team, called the good guys, are kicked. The programming language Lua and are loaded with the workshop in-game AI is either enabled or disabled for both teams. The tools which is downloadable content. The game itself is written reason for kicking the bots in our team is to be able to keep in C++ which is a statically typed compiled language and Lua the in-game AI enabled for the bots in the opposing team. which is a dynamically typed scripting language. The Lua API The connecting DotA 2 client from our framework creates 4 for DotA 2 is extensive and seem to cover most, if not all game fake clients that connects to the server. These 5 bot clients are functionality [3]. controlled by the scripts written by its user. Our addon creates a 5v5 bot match and takes complete control of one team, called the good guys. With each game V. EXPERIMENTS tick it gathers information about the world (game state) as The following experiments were performed to evaluate the it is known to the good guys, that means taking fog of war framework.
10 A. Experiment 1 - Hero movement These experiments validates that the bots can utilize their Confirm that heroes can be moved to specific locations abilities using the corresponding target types. The passive abilities functionality are handled by the game and therefore • Start a 5v5 match not included in these experiments. • Move heroes 1) Experiment 7.1 - No Target: This experiment validates B. Experiment 2 - Level up hero abilities functionality of the target type “No Target”. This target type Confirm that heroes can level up abilities does not target a point or unit and is cast when the command is given i.e. pushing the corresponding button for the ability. • Start a 5v5 match In this experiment the heroes Mirana, Slardar, Alchemist, • Level up heroes abilities Brewmaster and Clinkz are used. C. Experiment 3 - Hero attack • Mirana makes use of ability “Starstorm” Confirm that heroes can auto-attack • Slardar makes use of ability “Slithereen Crush” • Brewmaster makes use of ability “Thunder Clap” • Start a 5v5 match • Alchemist makes use of ability “Unstable Concoction” • Push lanes with all bots to validate attack functionality • Clinkz makes use of ability “Skeleton Walk” D. Experiment 4 - Buy items 2) Experiment 7.2 - Toggle: This experiment validates Confirm that bots can buy items functionality of the target type “Toggle”. Target type toggle does not target units or points. They provide a momentarily • Start a 5v5 match effect, while toggled on, while draining either mana or health. • Buy one specific item with each hero If an ability is toggled on which drains mana, its toggled • Confirm that the bought items are in the inventory of the heroes off if not sufficient mana remains to keep it enabled. In this experiment the heroes Medusa, Morphling, Witch Doctor, E. Experiment 5 - Sell items Pudge and Mars are used. Confirm that bots can sell items • Medusa makes use of ability “Mana Shield” • Start a 5v5 match • Morphling makes use of abiliy “Attribute Shift (Agility • Buy one specific item with each hero Gain)” • Confirm that the bought items are in the inventory of the • Witch Doctor makes use of ability “Voodoo Restoration” heroes • Pudge makes use of ability “Rot” • Sell each of the bought items. • Mars makes use of ability “Bulwark” • Confirm that all of the items are sold and no longer in 3) Experiment 7.3 - Target Point: This experiment validates the heroes inventories. functionality of the target type “Target Point”. Target point type abilities requires the caster to target an area or a point. F. Experiment 6 - Use items In this experiment the heroes Techie, Spectre, Morphling, Lina Confirm that bots can use items and Mars are used. • Start a 5v5 match • Techie makes use of ability “Techie Suicide” • Let bots buy items that are actively used • Spectre makes use of ability “Spectral Dagger” • Use items • Morphling makes use of ability “Waveform”. • Lina makes use of ability “Light Strike Array” G. Experiment 7 - Target Types • Mars makes use of ability “Spear of Mars” Heroes in DotA 2 have abilities. These abilities are either 4) Experiment 7.4 - Target Area: This experiment validates active, passive or auto cast. Active abilities are applied when functionality of the target type “Target Area”. Target area used. Passive attacks are applied permanently or when the type requires the caster to target an area. In this experiment requirements for the ability are met. Auto casting abilities are the heroes Crystal Maiden, Alchemist, Enigma, Invoker and applied when manually cast or when toggled on. There are Brewmaster are used. different targeting types that these experiments validates the • Crystal Maiden makes use of ability “Crystal Nova” functionality of. • Alchemist makes use of ability “Acid Spray” • No Target • Enigma makes use of ability “Midnight Pulse” • Toggle • Disruptor makes use of ability “Kinetic Field” • Target Point • Brewmaster makes use of ability “Cinder Brew” • Target Area 5) Experiment 7.5 - Target Unit: This experiment validates • Vector Targeting the functionality of the target type “Target Unit”. Target unit • Target Unit type requires the caster to target a unit. In this experiment the Furthermore there are combinations of target types. heroes Lina, Vengeful Spirit, Omniknight, Chen and Clinkz • Target Point or Unit are used. • Target Unit with Target Area • Lina makes use of ability “Dragon Slave”
11 • Vengeful Spirit makes use of ability “Magic Missile” I. Experiment 9 - Full length match • Omniknight makes use of ability “Heavenly Grace” Confirm that the framework supports full length matches. • Chen makes use of ability “Penitence” • Connect 5 bots to the framework • Clinkz makes use of ability “Death Pact” • Select heroes randomly 6) Experiment 7.6 - Vector Targeting: This experiment • The bots should perform basic actions, such as farming validates functionality of the target type “Vector Targeting”. and attacking enemy heroes/towers, during the match Vector targeting abilities requires two locations, a starting • Complete match from start to end location and a direction location. In this experiment the heroes Grimstroke, Pangolier, Dark Seer, Mars and Void Spirit are VI.EXPERIMENT RESULTS used. To eliminate threats to validity experiments has been con- • Grimstroke makes use of ability “Stroke of Fate” ducted. The experiments are simple bots written to perform • Pangolier makes use of ability “Swashbuckle” certain actions, as described in each experiment. After the • Dark Seer makes use of ability “Vacuum” commands to perform the actions are called, the game state • Void Spirit makes use of ability “Aether Remnant” is validated to confirm that the expected effects of the com- • Mars makes use of ability “God’s Rebuke” mands has occurred which means that the functionality being 7) Experiment 7.7 - Combination of Target Point or Target tested works. After an experiment is finished, the results are Unit: This experiment validates the functionality of the com- printed to STDOUT of the terminal in which the experiment bination of the target types “Target Point” and “Target Unit”. was started. The results is a summary of what part of the There are abilities that supports targeting a point and or a unit. experiment passed and what failed. If one part of an exper- That means that the caster can target the unit directly or use iment failed then the experiment is considered a failure. All a target point to execute the ability. In this experiment the experiments are located in the src/bots folder of our framework heroes Sand King, Nature Prophet, Lina, Lion and Zeus are and can be tested by anyone who has DotA 2 with our addon used. To test the functionality the abilities are executed using running on their computer. The framework and experiments both target types. can be found on Github [48]. To run a specific experiment • Sand King makes use of ability “Burrowstrike”. you start our framework with the following command: python • Nature Prophet makes use of ability “Sprout” framework.py –bot
12 C. Experiment 3 - Hero attack F. Experiment 6 - Use items
Fig. 9. Screenshot of the result from experiment 3
Fig. 12. Screenshot of the result from experiment 6
D. Experiment 4 - Buy items G. Experiment 7 - Target types H. Experiment 7.1 - No Target
Fig. 13. Screenshot of the result from experiment 7.1
Fig. 10. Screenshot of the result from experiment 4 I. Experiment 7.2 - Toggle
E. Experiment 5 - Sell items
Fig. 11. Screenshot of the result from experiment 5 Fig. 14. Screenshot of the result from experiment 7.2
13 J. Experiment 7.3 - Target Point M. Experiment 7.6 - Vector Targeting
Fig. 18. Screenshot of the result from experiment 7.6
Fig. 15. Screenshot of the result from experiment 7.3 N. Experiment 7.7 - Combination of Target Point and Target Unit K. Experiment 7.4 - Target Area
Fig. 19. Screenshot of the result from experiment 7.7
Fig. 16. Screenshot of the result from experiment 7.4 O. Experiment 7.8 - Combination of Target Unit and Target Area L. Experiment 7.5 - Target Unit
Fig. 17. Screenshot of the result from experiment 7.5 Fig. 20. Screenshot of the result from experiment 7.8
14 P. Experiment 8 - Full length match A. Future work An important part of any DotA 2 match that is missing in our framework is a unit called a courier. In a normal DotA 2 match each player gets assigned a courier that can be used to buy items from the various in-game shops and deliver them to the player. This is a valuable game mechanic that buys each player a lot of time since they won’t have to go back to the shop to buy items. When we developed the 5v5-framework we were not aware of this functionality as it was disabled in the 1v1 framework and neither of us who Fig. 21. Screenshot of the result from experiment 8 developed the 5v5-framework had any previous experience playing the game. Once we learned about this functionality Q. Experiment 9 - Validating usage of all heroes we attempted to implement it but were not able to get it in working condition and decided to disable the couriers for both teams. It is not clear from the official documentation [3] how to use the couriers from the Lua sandbox and we discovered this too late in the development process for us to spend any significant amount of time solving it. We even tried to get in contact with the game developer Valve but sadly we never received a response. Implementing the functionality to be able to use couriers in the framework is therefore one of our suggestions of future work that can be done in regards to Fig. 22. Screenshot of the result from experiment 9 the 5v5-framework. Another interesting functionality that could be supported VII.DISCUSSION by our framework in the future is the ability to allow both teams to be controlled by custom AI-agents. Currently we only We believe that by having developed a working 5v5 frame- support controlling one of the two teams with our framework work for DotA 2 we have made a valuable contribution to and the bots in the opposing team are controlled by Valves in- our field of research, the field of artificial intelligence. As game AI. This feature is interesting because it would allow described in our motivation I-A and shown in our literature for developers to test their agents against each others and review II, there was a need for a 5v5 AI-agent framework arrange tournaments like those being held for the TAC SCM in the MOBA genre. Our framework fills that void and we framework [6] and Ms Pac-Man [5]. This feature was not in prove it’s functionality by performing a series of experiments, our scope when we developed our framework but we believe described in section V and the results of which are declared that implementing it would be straight forward and would in section VI. increase the usability of our framework. We opted to implement our framework in the Python The core of our framework is the Lua addon for DotA 2. It is language for it’s simplicity, rich standard library and platform loosely coupled with a web-API/client written in Python. It is independent nature [50]. We think that our framework would the client that loads the code of the agent that is to control the be fitting to use in university courses about AI, to let students bots in DotA 2. It would be relatively easy for a programmer to implement and test their algorithms in. By using a game, in implement a bot client in a different language than Python as this case DotA 2, the algorithms implemented in the agents are we have defined a simple set of API routes and commands that visualised in real-time in-game. This possibly provides a better are used for the addon and the bot framework to communicate understanding of the behaviour of the algorithm the developer with each-other. In the future it may be interesting to develop has implemented. As a bonus, DotA 2 is a popular game which a framework in another language with support for different possibly increases the amount of students that are familiar machine learning libraries than Python. The implementation with the game and therefore understands the core mechanics of our client can be found at Github [48]. and finds the assignments appealing. To further increase the potential usefulness of our framework as an educational tool VIII.CONCLUSION we have created a detailed documentation. It contains a tutorial The conclusion is that the framework is able to handle 5v5 that is meant to help users setup the framework and implement bot-matchmaking allowing it to be a plausible framework to a basic bot. It also contains a detailed description of the be used in competitions such as the conference on games commands available for the developer, both the commands [14] and foundations of digital games conference [26]. The supported by the Lua addon and the API routes needed to final product is a composition of the DotA 2 addon in Lua, be implemented to create an entire new client. An example a web API and client written in Python. The addon handles bot and a number of experiments are provided to give further the communication between DotA 2 and the web API. The insight into the possibilities of the framework. communication consists of data regarding the world and its
15 entities serialized in JSON language. The data is forwarded ized Python class. The division of intelligence is up to to the client using JSON commands which the client uses to the implementer of the agents to decide. The power of determine a set of actions. The client responds with the actions the Python programming language sets the limitations. which is then executed by the addon. The developer of a bot could for example implement Below we present the answers to our research questions. a socket server where clients controlling each separate bot could connect to and use the TCP protocol to send 1) How can the 1v1 DotA 2 AI Framework project be messages to each other. Another implementation could extended to support 5v5? use different algorithms for each bot in the team by For every game tick DotA 2 streams data to Lua. Lua implementing threads running for them and then use is then able to forward this data using by sending com- a thread safe global variable for the bot communica- mands to a web API. In the 1v1 framework one client tion. The point is that our architecture allows multiple was responsible for handling the data and returning the implementations of the AI agents and the way they actions that the user-controlled bot should perform. This communicate. could have been extended so that five clients would do 3) How could a multiple agent strategy be implemented in the same thing but this could have possibly resulted in the 5v5 framework? loss of actions per game tick. The reasoning is that if We have developed our own example bot as an answer to the 1v1 architecture was kept, five JSON commands this question. It is available on Github in the framework would be sent from Lua every game tick to a chosen repository [48] in the “src/bots/BotExample.py” file. It web API and the only information that would differ works roughly in the following way.
would be the information regarding the bots actions. • Fetching the names of the heroes to identify them At any given game tick the world looks identical for • Assigning a lane to each bot in which they will do every bot involved in the match. Therefore running one battle client that would be responsible for locally handling • The bots can switch lanes with each other all bot actions, by receiving one JSON command per • If one of the bots is struggling to maintain its game tick, was more appealing because this would in lane, it communicates to its teammates that it needs theory be five times faster. The in-game AI is either assistance and the teammates will come to assist turned on or off for both teams in DotA 2 which brought • If the enemy maintains pressure in the lane, the issues when developing the framework. The solution was bots switches lane so that the strongest friendly hero to kick the bots, except the connecting client, of our stays on the lane own team. The user-controlled bots then connected to • For each game tick calculating the best possible the game through the connected client. By doing this, course of action. For example, if the hero receives the possibility was enabled to control the bots of our damage from a tower it retreats to a safe distance own team while keeping the in-game AI enabled in the away from it opposing team. The setback of this was that the couriers • If the bot is alone it looks for a group of friendly for our own team was lost. To solve this issue attempts creeps to follow and attacks enemies or buildings were done such as kicking the bots in our team after the alongside them couriers had been loaded in the game. Unfortunately this • If the bot has enough gold, it buys one or more led to the in-game AI buying items for some of the, to useful items to the hero be, user-controlled heroes. This unwanted behaviour was disregarded at first and attempts were made to use the There are other courses of action the hero can take and couriers for our user-controlled heroes. But it seemed it’s always based on the current game state. that the connections between heroes and courier were no longer valid. This is possibly because the heroes that REFERENCES they were assigned to by the game were kicked by us. The lack of information regarding couriers in the official [1] J. Togelius, “Ai researchers, video games are your friends!” in 2015 7th documentation [3] made it unfeasible for us to solve International Joint Conference on Computational Intelligence (IJCCI), this issue during our given time frame. A more detailed vol. 2, Nov 2015, pp. 5–5, Accessed 2019-12-16. [2] “The programming language lua,” accessed online 2020-03-19. [Online]. explanation of our 5v5 framework is presented in section Available: http://www.lua.org/ IV and the improvements we made to the 1v1 framework [3] “Api - valve developer community,” accessed online 2020-01-12. extending it to a 5v5 framework. [Online]. Available: https://developer.valvesoftware.com/wiki/Dota_2_ 2) How can the intelligent agents communicate with each Workshop_Tools/Scripting/API [4] M. Wu, S. Xiong, and H. Iida, “Fairness mechanism in multiplayer other in the extended framework? online battle arena games,” in The 2016 3rd International Conference The architecture of our framework supports communi- on Systems and Informatics (ICSAI 2016), At Shanghai, China, 11 2016. cation between the agents in any way possible in the [5] P. Rohlfshagen, J. Liu, D. Perez-Liebana, and S. M. Lucas, “Pac-man conquers academia: Two decades of research using a classic arcade Python programming language. The commands received game,” IEEE Transactions on Games, vol. 10, no. 3, pp. 233–256, Sep. and forwarded to the Lua addon is handled in a central- 2018.
16 [6] I. Kontogounis, K. Chatzidimitriou, A. Symeonidis, and P. Mitkas, [37] V. d. N. Silva and L. Chaimowicz, “On the development of intelligent “A robust agent design for dynamic scm environments,” in SETN’06: agents for moba games,” in 2015 14th Brazilian Symposium on Com- Proceedings of the 4th Helenic conference on Advances in Artificial puter Games and Digital Entertainment (SBGames), Nov 2015, pp. 142– Intelligence, vol. 3955, 05 2006, pp. 127–136. 151. [7] M. Emilio, M. Moises, R. Gustavo, and S. Yago, “Pac-mant: Optimiza- [38] M. Wisniewski´ and A. Niewiadomski, “Applying artificial intelligence tion based on ant colonies applied to developing an agent for ms. pac- algorithms in moba games,” Studia Informatica : systems and informa- man,” in Proceedings of the 2010 IEEE Conference on Computational tion technology, vol. Vol. 1-2(20), pp. 53–64, 2016, Accessed 2019-12- Intelligence and Games, Aug 2010, pp. 458–464. 16. [8] “Rahul mehrotra: Product manager,” Accessed 2019-12-16. [Online]. [39] V. do Nascimento Silva and L. Chaimowicz, “MOBA: a new arena for Available: https://www.microsoft.com/en-us/research/people/ramehrot/ game AI,” CoRR, vol. abs/1705.10443, 2017, Accessed 2019-12-16. [9] A. Linn, “Divide and conquer: How microsoft researchers [Online]. Available: http://arxiv.org/abs/1705.10443 used ai to master ms. pac-man,” May 2018, Accessed [40] M. Mora-Cantallops and M.-Á. Sicilia, “MOBA games: A literature 2019-12-16. [Online]. Available: https://blogs.microsoft.com/ai/ review,” Entertainment Computing, vol. 26, pp. 128–138, May 2018, divide-conquer-microsoft-researchers-used-ai-master-ms-pac-man/ Accessed 2019-12-16. [Online]. Available: https://doi.org/10.1016/j. [10] J. M. Font and T. Mahlmann, “Dota 2 bot competition,” IEEE Transac- entcom.2018.02.005 tions on Games, vol. 11, no. 3, pp. 285–289, Sep. 2019. [41] B. Xia, H. Wang, and R. Zhou, “What contributes to success in moba [11] “Dota 2 Prize Pool Tracker,” Accessed 2019-12-16. [Online]. Available: games? an empirical study of defense of the ancients 2,” Games and https://dota2.prizetrac.kr/ Culture, vol. 14, p. 155541201771059, 05 2017, Accessed 2019-12-16. [12] OpenAI, “OpenAI Five,” Dec 2019, Accessed 2019-12-16. [Online]. [42] A. R. Hevner, S. T. March, J. Park, and S. Ram, “Design science in Available: https://openai.com/projects/five/ information systems research,” MIS quarterly, pp. 75–105, 2004. [13] Lightbringer, “lightbringer/dota2ai,” Accessed 2019-12-16. [Online]. [43] S. T. March and G. F. Smith, “Design and natural science Available: https://github.com/lightbringer/dota2ai research on information technology,” Decision Support Systems, [14] “IEEE COG 2019 Competitions,” Accessed 2019-12-16. [Online]. vol. 15, no. 4, pp. 251 – 266, 1995. [Online]. Available: Available: http://ieee-cog.org/2019/competitions_conference/ http://www.sciencedirect.com/science/article/pii/0167923694000412 [15] “Angry Birds AI Competition,” Accessed 2019-12-16. [Online]. [44] A. Hevner, “A three cycle view of design science research,” Scandina- Available: https://aibirds.org/ vian Journal of Information Systems, vol. 19, 01 2007. [16] Njustesen, “Bot Bowl AI Competition,” Sep 2019, Accessed 2019-12- [45] M. Biehl, RESTful API design: APIs your consumers will love. API- 16. [Online]. Available: https://bot-bowl.com/ University Press, 2016. [17] “Fighting Game AI Competition,” Accessed 2019-12-16. [Online]. [46] “What is rest api?” accessed online 2020-06-06. [Online]. Available: Available: http://www.ice.ci.ritsumei.ac.jp/~ftgaic/ https://www.visual-paradigm.com/guide/development/what-is-rest-api/ [18] “CodaLab Competition,” Accessed 2019-12-16. [Online]. Available: [47] K. Lindqvist and D. Nilsson, “5v5dota2ai-addon,” May 2020, http://aka.ms/textworld-challenge accessed online 2020-05-12. [Online]. Available: https://github.com/ [19] “GFGAI Competition,” Accessed 2019-12-16. [Online]. Available: ellakk/5v5dota2ai-addon https://geometryfriends.gaips.inesc-id.pt/ [48] ——, “5v5dota2ai-framework,” May 2020, accessed online 2020-05-12. [20] “General Video Game AI Competitions,” Accessed 2019-12-16. [Online]. Available: https://github.com/ellakk/5v5dota2ai-framework [Online]. Available: http://www.aingames.cn/ [49] “Bottle - python web framework,” accessed online 2020-03-30. [21] “Fireworks Agent Competition,” Accessed 2019-12-16. [Online]. [Online]. Available: http://www.bottlepy.org/docs/dev/ Available: http://hanabi.fosslab.uk/ [50] J. V. Guttag, Introduction to Computation and Programming Using [22] “Hearthstone AI Competition,” Accessed 2019-12-16. [Online]. Python: With Application to Understanding Data (The MIT Press). The Available: http://www.is.ovgu.de/Research/HearthstoneAI.html MIT Press, 2016. [23] “Micro RTS AI Competition,” Accessed 2019-12-16. [Online]. Available: https://sites.google.com/site/micrortsaicompetition/home [24] “Starcraft AI Competition,” Accessed 2019-12-16. [Online]. Available: https://cilab.gist.ac.kr/sc_competition/ [25] “Strategy Card Game AI Competition COG 2019.” [Online]. Available: https://jakubkowalski.tech/Projects/LOCM/COG19/ [26] “FDG 2019 Competitions: San Luis Obispo, California, USA,” Accessed 2019-12-16. [Online]. Available: http://fdg2019.org/competitions.html [27] “The Codenames AI competition,” Accessed 2019-12-16. [Online]. Available: https://sites.google.com/view/the-codenames-ai-competition [28] “Fifth Aeon Collectible Card Game,” Accessed 2019-12-16. [Online]. Available: https://fifthaeon.com/tournament [29] M. C. Green, “Generative Design in Minecraft,” Accessed 2019-12-16. [Online]. Available: http://gendesignmc.engineering.nyu.edu/ [30] “TCIAIG Special Issues,” Accessed 2019-12-16. [Online]. Available: https://cis.ieee.org/publications/t-games/tciaig-special-issues [31] F. Weidt and R. Silva, “Systematic literature review in computer science- a practical guide,” Relatórios Técnicos do DCC/UFJF, vol. 1, 2016, Accessed 2019-12-16. [32] “Google Scholar,” Accessed 2019-12-16. [Online]. Available: https: //scholar.google.com/ [33] “Acm Digital Library,” Accessed 2019-12-16. [Online]. Available: https://dl.acm.org/ [34] “IEEE Xplore Digital Library,” Accessed 2019-12-16. [Online]. Available: https://ieeexplore.ieee.org/ [35] M. Waltham and D. Moodley, “An analysis of artificial intelligence techniques in multiplayer online battle arena game environments,” in Proceedings of the Annual Conference of the South African Institute of Computer Scientists and Information Technologists, ser. SAICSIT ’16. New York, NY, USA: ACM, 2016, pp. 45:1–45:7, Accessed 2019-12-16. [Online]. Available: http://doi.acm.org/10.1145/2987491.2987513 [36] M. Morosan and R. Poli, “Evolving a designer-balanced neural network for ms pacman,” in 2017 9th Computer Science and Electronic Engi- neering (CEEC), Sep. 2017, pp. 100–105.
17