Team Sports for Game AI Benchmarking Revisited

Hindawi International Journal of Computer Games Technology Volume 2021, Article ID 5521877, 9 pages https://doi.org/10.1155/2021/5521877

Research Article Team Sports for Game AI Benchmarking Revisited

Maxim Mozgovoy ,1 Mike Preuss ,2 and Rafael Bidarra 3

1The University of Aizu, Japan 2LIACS, Universiteit Leiden, Netherlands 3Computer Graphics and Visualization Group, Delft University of Technology, Netherlands

Correspondence should be addressed to Rafael Bidarra; [email protected]

Received 16 January 2021; Accepted 29 April 2021; Published 31 May 2021

Academic Editor: Michael J. Katchabaw

Copyright © 2021 Maxim Mozgovoy et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Sport games are among the oldest and best established genres of computer games. Sport-inspired environments, such as RoboCup, have been used for AI benchmarking for years. We argue that, in spite of the rise of increasingly more sophisticated game genres, team sport games will remain an important testbed for AI benchmarking due to two primary factors. First, there are several genre- speciﬁc challenges for AI systems that are neither present nor emphasized in other types of games, such as team AI and frequent replanning. Second, there are unmistakable nonskill-related goals of AI systems, contributing to player enjoyment, that are most easily observed and addressed within a context of a team sport, such as showing creative and emotional traits. We analyze these factors in detail and outline promising directions for future research for game AI benchmarking, within a team sport context.

1. Introduction teams playing against each other, each composed of a set of players with their individual roles and abilities” [2]. If further Historically, artificial intelligence (AI) researchers have often specification is desired on what makes up a team in contrast relied on popular games as testbeds for evaluating new algo- to a group, we may go with the definition given by Thompson rithms and approaches. Discussing a famous adage “chess is a (from a management perspective) [3]: “a team is a group of Drosophila of AI”, J. McCarthy wrote: “To computer scien- people who are interdependent with respect to information, tists in general I have only one wish to express: let there be resources, and skills and who seek to combine their efforts to more of these Drosophila-like experiments; let us create some achieve a common goal”. What is not explicitly said here but more specific examples!” [1]. clear from the context is that there must be ways of communi- Several factors may contribute to the success of a partic- cation between the team members. If we add communication ular testbed. For example, in the case of chess, one may note to the team sport definition above, we may get a more com- that the game is easy to set up, it has a wide appeal both to plete definition: team sports are games that involve at least general public and to researchers themselves, it poses prob- two teams playing against each other, each composed of a set lems that are perceived as generalizable beyond the game of players with their individual roles and abilities who cooperate world, and even allows independent study of isolated aspects by means of communication in order to win the game. of the game such as endings. These features make team sports an interesting chal- We can say that most team sports have similar features: lenge for a game AI system, and we, therefore, propose to they are popular among the general public, they are easy to consider them as a promising testbed for AI benchmarking. set up, and they require the participating team members to In the remainder of this paper, we outline our vision on the exhibit both athletic abilities and a certain level of tactical topics involved. and strategic thinking. Although various definitions of team Before suggesting to focus AI benchmarking more on the sport can be given, we propose to abide here by that put for- team sport area, let us first have a look at how the competi- ward in a recent survey: “a game that typically involves two tion landscape currently evolves. Without doubt, game- 2 International Journal of Computer Games Technology related benchmark environments have recently propelled AI 2. Real-World and Virtual Team Sports progress and led to a huge impact in the media, thereby con- fi “ ” tributing to the current AI hype. There is no universal de nition of team sports. According “ The game of Go was already an unofficial AI benchmark to Collins Dictionary, a team sport is a sport in which teams ” when chess lost much of its appeal for AI researchers as a play against each other [11]. The current Wikipedia article fi “ result of DeepBlue’s victory over Kasparov in 1997. Whereas expands this de nition as follows: A team sport includes Silver et al. [4] already provided an approach that was able to any sport where individuals are organized into opposing beat professional human Go players, the old Atari console teams which compete to win. Team members act together environment was used to invent Deep Reinforcement Learn- towards a shared objective. This can be done in a number ” ing (DRL) [5, 6]. It soon turned out that most of the games of ways such as outscoring the opposing team [12]. of the so-called Atari learning environment (ALE) are rela- For some sports, one may wonder to which extent they fl tively easy to deal with by this new technique; even playing should be considered full- edged team sports. For example, in many games with superhuman performance was quickly rowing, relay racing, or swimming, there are competing teams achieved. In search for new challenges, research turned to of athletes, but little communication is required among team real-time strategy (RTS) games. members and virtually no contact with the opposing team. StarCraft was established as a benchmark and competi- More interesting cases are sports in which teams take on ff ff tion environment in 2010 [7] and was still considered very distinct roles and follow di erent objectives in di erent fi challenging when DeepMind and Blizzard teamed up to pro- phases of the game, like elding team and batting team in vide the SC2LE environment for StarCraft 2 in 2017 [8]. Fol- baseball, or when each team consists of several groups with lowing the reports of the AlphaStar team of DeepMind [9], it immutable roles as in quidditch (https://en.wikipedia.org/ seems that RTS games are not yet completely done, but get- wiki/Quidditch_(sport)), where two largely independent ting in reach. This is surprising on one hand, because most but somehow interacting games (chaser and beater) take fi researchers had expected it to take years to get that far. On place on the same eld, extended by a third one (seeker) the other hand, there is higher emphasis on generalizability during the end phase of the game. now: we are not satisfied with an AI system that can be used Having noted these scenarios, let us narrow down the for a single game and in previously seen environments any- subsequent discussion to the types of sports in which both more; we expect the AI shall be able to transfer knowledge teams have the same structure and follow the same goal as, between situations, environments, and eventually games. for example, in ice hockey, basketball, soccer, and volleyball. An interesting development in this direction is the Obsta- Sport games are also a genre staple of the video game cle Tower Challenge [10] that features almost endless gener- industry. Since Tennis for Two (1958), virtual renditions of ated mazes that gradually get more difficult. Generalization sport games spread literally to every gaming platform avail- in game mechanisms and visuals is key here, but there is only able. Obviously, it is not possible to capture all aspects of a ff a single avatar that is completely autonomous; thus, interac- physical athletic activity on a computer, and di erent game ff tion happens only with more or less static objects. projects emphasize di erent elements of real-world sport Another aspect, the team play between different AI sys- events. As extreme examples, departing from a somewhat tems, is emphasized in the cooperating OpenAI Five standard formula of a sport video game, we can mention Sub- (http://openai.com/blog/openai-five-benchmark-results/) buteo (1990), representing soccer as a turn-based billiard-like Multiplayer online battle arena (MOBA) game bots. The strategy, Captain Tsubasa (1988), emphasizing role-playing number of bots in a team is limited to five, but they clearly elements of soccer and player relationships, and a number need to work together well in order to win a game—a capabil- of games like Super Mario Strikers (2005) or Nintendo World “ ” ity that is also fundamental for team sport AI. We, therefore, Cup (1990), with unrealistic fun elements such as powerful ’ also compare the properties of MOBA, RTS, and team sport supershots, players special abilities, and extreme weather games and the resulting challenges for AI later on. conditions. Even within the same franchise, developers “ ” With this work, we highlight the need to define good bench- sometimes create games that emphasize simulation or “ ” ’ mark environments that integrate team AI, here understood as fun aspects, such as in case of EA s NBA Live vs. NBA multiple game AIs that interact among themselves and with Jam or NHL vs. NHL Slapshot. Therefore, the discussion “ ” fi humans in order to reach a common goal. For this, team sport around team sports as a game AI testbed requires rst games provide an excellent context with numerous advantages. deciding which aspects of team sports should be investigated We start this endeavour with reviewing the differences and what are the ultimate goals. between real-world and virtual team sports (Section 2), before discussing the case of RoboCup (Section 3), a famous 3. The Case of RoboCup team sport AI environment that was actually not conceived as a game. As said before, user enjoyment is an important Fortunately, we have a great reference point: the event series aspect also for AI competitions, and it is addressed in Section organized by the RoboCup Federation [13]. One may wonder 4. After discussing the interplay of strong and fun game AI in why we are bringing the topic of team sports AI benchmarks Section 5, we also look at the tactics and strategy perspective in spite of the existence of an established RoboCup frame- (Section 6), before more generally collecting the challenges of work. The main reason for it is the very specific agenda of sports AI, also in comparison with MOBA and RTS (Section the RoboCup Federation, which ultimate goal is stated as fol- 7) and ending with conclusions. lows: “by the middle of the 21st century, a team of fully International Journal of Computer Games Technology 3 autonomous humanoid robot soccer players shall win a soc- (v) Adjust its strategy if some player is removed from cer game, complying with the official rules of FIFA, against the field by a referee the winner of the most recent World Cup” [14]. (vi) Be good at playing set pieces (kick-ins, free-, pen- On the way to the accomplishment of this grand vision, a alty-, and corner-kicks) number of smaller goals have to be achieved. Thus, RoboCup competitions consist of several independent events, each While these diverse abilities are relevant to the field of focusing on a few relatively isolated subproblems, such as game AI, it is difficult to develop all of them within a small hardware challenges, computer vision, and team-based behav- group having expertise in just one specific subfield of AI “ ” ior strategies. In practice, it means separation into hardware research. A public release of the award-winning Helios “ ” ff and software leagues and organization of di erent tourna- team’s agend2d codebase [16] motivated around 80% of ff ments for di erent types of robots (including separate compe- other teams to abandon their own developments and switch titions for 2D and 3D virtual software-simulated robots). to agent2d [17]. This process is seen by different authors as These tournaments are deliberately designed to support either “greatly beneficial” [18] or “detrimental” and leading the grand vision of RoboCup organizers and to ensure to lower diversity of teams [19]. Furthermore, RoboCup smooth knowledge transfer between independent RoboCup was never designed as a game world; instead, it was conceived events. For example, software robots must deal with the same as downgraded (software-only) robot training environment. type of constraints as their hardware counterparts, such as This opens up an additional perspective for our discussion. limited visibility, noisy sensory input, imprecise locomotion, and low-level actions (for instance, an agent may not dribble 4. The Role of Fun in Game AI Competitions the ball: it has to program each ball kick independently). Such setup is perfectly reasonable given the ultimate goal Which game environments are good for AI competitions? of RoboCup competitions. However, RoboCup environ- Discussing a related topic of organizing such contests, Toge- ments, such as 2D Simulation League [15], were not designed lius [20] gives the following advices: “choosing a fun game” to resemble computer games. While one can connect mouse (“it also helps if the game is famous”, he adds) and “making and keyboard to RoboCup software and take control of one it really easy to get started”. Applying this to our domain, of the 22 players, this experience will be significantly different we might add as well that it definitely helps if the respective from a typical sport video game session. real-world sport is popular worldwide. Let us focus on the 2D Simulation League, representing the RoboCup apparently relies on another formula. Analyz- most video game-like environment among RoboCup events. It ing a 20-year long story of RoboCup, Ferrein and Steinbauer is striking that the pool of successful 2D Simulation League [21] note “the atmosphere of some three thousand robot participants is very stable. During the last 5 years, only 7 differ- enthusiasts” and “fascinating outreach to the general public” ent teams managed to reach a top 3 position (WrightEagle, and emphasize the interdisciplinary nature of participants’ Helios, Gliders (Fractals), Cyrus, FRA-United, Oxsy, and work, community building and team building efforts, and a MT. See https://ssim.robocup.org/soccer-simulation-2d/2d- chance for the participants to solve real robotics problems. awards/) (the total number of participants in each of these In other words, it seems that RoboCup is backed first and events ranged from 13 to 19 teams). All these teams were foremost with people’s interest to physical robots, while established at least 6 years prior to their prize-winning seasons. “game-like” aspects of this event (especially in case of simu- In addition, we also notice a general decline in participa- lation leagues) play a somewhat secondary role. tion: 24 teams played in each of five consecutive seasons from The notion of what constitutes fun is largely subjective, 2000 to 2004, and since that period, no competition could but we can at least remark that, while it is possible to play attract more than 20 teams. RoboCup 2D Simulation League matches between human- We believe these observations can be at least partially controlled teams, people do not really do it, and, in any case, explained by the sheer amount of work required to establish the gameplay of such a match would radically differ from any a decent team: high entrance barriers work against aspiring successful commercial soccer game. It is, therefore, highly contestants. In general, a skillful RoboCup 2D Simulation likely that (i) RoboCup is not a very fun video game, and League AI system must be able to (ii) there is a niche for sport games in game AI competitions, which is not filled by RoboCup by design. One of the candi- (i) Efficiently and reliably translate high-level decisions dates for filling this niche could be the game Rocket League (such as “run with a ball for 10 meters”) into a series [22] which combines ball game and car racing fun elements of low-level kick and dash actions and is already played by different bots in teams of 3. The role of fun in a game is clearly broader than just to be (ii) Reason on the basis of noisy and limited audiovisual an instrument for keeping the public interested in a certain sensory data AI competition. But it is one of the factors, shaping the game ff (iii) Handle teammate-teammate and teammate-coach world, directly a ecting the design goals of a game AI system, messages using limited communication channels and the principles of its subsequent evaluation. Long- standing popularity of a certain game genre proves that the (iv) Efficiently distribute roles between the teammates core game mechanics is both fun (game-wise) and challeng- according to their skill profiles generated by the ing enough to continue fueling interest of both spectators RoboCup server at the beginning of a match and participants. 4 International Journal of Computer Games Technology

For team sports, the battle of team strategies has always Team sports are a good testbed for investigating such been a major part of spectator enjoyment. Sport fans appre- nonskill-related traits of AI systems as well. There is exten- ciate teams showing both spectacular and efficient play, and sive literature on factors making team sports exciting for both the balance between these goals is not easy to achieve. In athletes and spectators, for which there are many good exam- addition, professional regulating bodies monitor actual ples [32, 33]. Likewise, there is a general understanding of developments of common game tactical patterns and adjust what constitutes fun in the context of an AI system for a sport the rules to keep competitions appealing to the audience video game. In particular, we often observe that people prefer and fight against degenerate strategies [23]. Given the long playing against other people, because people behave in a cer- history of most popular team sports, one can assume that tain “human-like” way that is perceived as inherently enjoy- current rules are well balanced and encourage inventive, able [34]. In conclusion, striving for a human-like behavior nontrivial team play. For example, the early offside rule of can be a legitimate goal for a sport game AI system, as impor- soccer, introduced to prevent “goal hanging” (a degenerate tant and challenging as a highly skilled behavior. yet efficient tactical pattern) [24], underwent significant revi- sions in 1925 and 1990 and was last refined in 2005—every 6. Tactics and Strategy in Sports time to encourage attacking play and to limit the abuse of defensive “offside traps” [25, 26]. Similarly, a back-pass rule, An obvious problem with the choice of team sports as an AI preventing the goalkeeper from handling the ball received benchmark lies in the fact that they require certain physical from a teammate, was introduced in 1992 and extended in abilities and athletic skills in addition to tactical and strategic 1997 as an antitime-wasting measure [27]. decision-making. Since these physical aspects cannot be cap- These considerations make us believe that team sport tured within the game framework, it is necessary to investi- games are an appealing choice as a game AI testbed: they gate the extent to which sport team behaviors pose a are fun, spectacular, well-balanced, reasonably complex, easy significant challenge for AI. This, in turn, can be further split to set up, rely on simple rules, and enjoy a wide fan base. into questions as follows: (1) what is the role of tactics and strategy in real-world team sports? (2) What is the complexity of actual decision-making in real-world team sports? (3) 5. Strong Game AI and Fun Game AI How well can these experiences translate into a video game setting (see Figure 1)? While games can be used and are actually used to benchmark Obviously, the contribution of tactical/strategic factors general AI technologies, there are important features, charac- highly depends on a particular sport. However, there is a terizing game AI systems. It is generally presumed that the plethora of works dedicated specifically to these game ele- main point of a game is to win, and thus, the best AI system ments in a large variety of team sports [35–38]. In turn, good in a typical AI competition is the system that wins. game strategies are not easy to construct. For instance, soccer However, the purpose of game AI is not necessarily to be history is characterized by a series of paradigm shifts in build- strong. According to Dill [28], “The one thing that is univer- ing up team strategies, as Wilson remarks in his book solely sally true is that games are about creating a particular experi- dedicated to the history of soccer tactics [25]: “football is not ence for the player — whatever that experience may be. The about players, or at least not just about players; it is about purpose of Game AI (...) is to support that experience.” Thus, shape and about space, about the intelligent deployment of depending on a particular game, the goal of a good AI system players, and their movement within that deployment.” might well involve being strong, predictable, erratic, friendly, The differences in player abilities impose additional con- hostile, and so on [29]. In other words, a good game AI straints on coach decisions: one has to devise an efficient tac- testbed should support, at least, theoretically, various possi- tical scheme, applicable in the given context, in a match ble goals for AI-controlled characters. between two specific teams consisting of specific players. The goal of strong AI development for games like chess We can observe this phenomenon even in relatively uncom- or Go can be considered achieved, since computers are able plicated games, such as beach volleyball, where each team to defeat even the best human players. We can expect that consists of two players, and there is no physical contact with more game genres will be added to this list in the nearest the opponents. Koch and Tilp [39] show how differences in future. For example, a recent work by Oh et al. [30] discusses physical characteristics between male and female athletes the development of an AI system, able to defeat professional (among other factors) encourage them to prefer distinct human players in a modern fighting game. techniques in order to win. It is difficult to say how good modern AI methods are in Speaking of computer game renditions of sports, it seems playing sport games. Michael and Obst [31] observe that AI that the goal of the mainstream AAA projects, such as EA’s teams of RoboCup 2D Simulation League play better than FIFA, is “to make the games even closer to the actual game, human teams; they remark, however, that RoboCup is not that is, to make the computer game converge with the sport,” designed to be played by people, which supposedly affects as observed by Sicart [40]. Interestingly, in his analysis of the their performance. In any case, there are independent goals differences between the real soccer and FIFA’12, the most of designing a strong AI system and an AI system that con- salient feature was FIFA’s AI system, controlling the players tributes to the overall user enjoyment. Arguably, the latter and the referee. Sicart is generally satisfied with simulation task is even more important for the needs of practical game of physical aspects of the game, but believes that AI falls short development. in its understanding of soccer. While real soccer rules leave International Journal of Computer Games Technology 5

Physical contacts Ice hockey American football Basketball Soccer

Tennis (doubles) Team cooperation: Volleyball Low beach volley | Average | High

Number of players

Figure 1: Features of particular team sports in 3 dimensions: number of players, physical contacts, and importance of team cooperation. Sports written in boldface require the highest team cooperation levels. Here, American football has the highest levels in all three dimensions. enough room for referee’s interpretation, AI referees of still unclear which parts of the setup can be responsi- FIFA’12 are scripted in a deterministic way, leaving no space ble for these differences. We also do not know who, in for ambiguity. Similarly, AI-controlled players behave in a principle, plays computer soccer better—skillful logical yet predictable manner, and even superstar players human players or AI teams. We can compare AI- honored for their creativity and tactical vision (like Messi) controlled characters in games like FIFA with their do not exhibit such abilities in the computer game. real-world prototypes, but there has been no compar- These limitations actually have deep implications for the ison between the best real soccer teams and AI- players. The rigid, predictable nature of AI decision-making controlled computer soccer teams. It is possible that encourages people to exploit this knowledge and build their AI systems can greatly benefit from the arrival of dig- strategy around it. As Sicart puts it, “FIFA players (...) need itized real soccer team behavior data, and it is also to learn how to think procedurally, how to decode the techni- possible that real-world teams could benefit from cal implementation of a known set of rules, tactics, and player studying team tactics, exhibited by the best RoboCup characteristics, and apply this way of thinking to ways of teams playing the game.” This observation reinforces our proposi- (2) Which facets does a good team sport AI integrate? tion that sport game AI is still far from satisfactory and Sport games almost inevitably have to rely on some requires further research. type of AI technology. A computer soccer match without AI would require simultaneous presence of 7. The Challenges of Sports AI exactly 22 participants plus referees. Thus, the role of AI in this setup is to be both an opponent and a One of the goals of AI competitions is to develop AI methods, teammate and to provide smooth gaming experience applicable to a wider domain or at least to a wider set of sub- and “suspension of disbelief” for human players. “ tasks in the given domain. As expressed by Togelius [20], we Environments like RoboCup set a straightforward should try to counteract the tendency of competition partic- goal for an AI system: it has to win as many matches fi ipants to over t their solution to particular problems and as possible. However, as AI methods mature and problem parameters, so as to make the results of competi- become more skillful, like in chess or fighting games, ” tions more generally valid. This is a relatively recent we will need another benchmark, emphasizing the approach for setting up competitions: it comes with the rise user enjoyment facet, crucial for a game system. fi of arti cial general intelligence (AGI), which requires a much Player appeal is much harder to evaluate—the goal higher level of autonomy than most AI approaches currently is elusive, and the benchmarks are subjective, but have. A recent example of a step into this direction is the one cannot just ignore the existence and importance Obstacle Tower Challenge [10] with its procedurally gener- of this factor ated, diverse levels that require high generalization skills. Likewise, team sport games can serve as a good vehicle (3) What is emergent and stable team behavior? While for researching a diverse set of AI-related challenges, includ- dealing with team sports, one has to clarify what ing the following: constitutes team behavior, and what are the characteristics of successful teams. It is possible that team (1) What can coaches and athletes learn from AI and vice behavior can be defined in terms of goal-driven deci- versa? One can easily observe that the strategies of the sion-making, where team goals (e.g., scoring) take top RoboCup 2D Simulation League teams are sub- precedence over individual goals (e.g., showing off). stantially different from the strategies of teams in real However, real soccer teams possess other important soccer. Despite the specific goals of RoboCup and traits, such as adaptability to opponent counter- their implications for the game (see Section 3), it is actions, efficient repetition of the same successful 6 International Journal of Computer Games Technology

Table 1: Control property. Are there multiple autonomous agents?

Team sports AI Mostly yes; for RoboCup, 12 per team (11 players+the coach agent directing the overall strategy). MOBA AI Yes, but mostly at the end of the game—the roles are quite ﬁxed at the beginning, and there is not much coordination. RTS AI Central control is possible (but not enforced) for lower-level decisions; strategic-level decisions are centrally controlled.

Table 2: Unit diversity property. Does the AI have to ﬁnd suitable ways to combine diﬀerent units or to counter the enemy strategy?

Team sports AI Average; units can be quite diﬀerent, but, in principle, most units are able to replace other units.

MOBA AI High; units are deﬁnitively diﬀerent and need to be combined in suitable ways; only a small subset of possible unit types is used in each game. RTS AI High, but there is less diversity in abilities (special powers) than in MOBA.

Table 3: Agent role property. Does the game impose speciﬁc roles the agents have to fulﬁll?

Team sports AI Yes, but there is a variety of “play systems” with slightly diﬀerent roles (in general, there are oﬀensive and defensive roles, and often dedicated goalkeepers).

MOBA AI Quite diverse roles and very diverse agent templates (heroes); the choice of heroes alone can have a signiﬁcant impact on team performance.

RTS AI Roles are not obvious in RTS, except in cases of clear specialization (workers/army units/air units); units have little individuality and mostly count as a part of their squad.

patterns, or adjusting strategies on the go. Profes- common in real game sessions, and thanks to a carefully con- sional players are able to predict the actions of their structed design of game economy and team structure. teammates and opponents (and quickly adapt their Team sport games have grown naturally around the con- behavior if these predictions turn out to be wrong), cept of team-vs.-team competition, and their rules are a prod- as well as perform clearly identifiable elements of uct of long evolution, aimed at keeping the game interesting team strategy. Teams can quickly regroup after an for both participants and spectators. While in each sport there unsuccessful attack and readjust formations in case is a number of superstar athletes showing exceptional abilities, of player injury or removal. Thus, team behavior is generally, sport teams are comprised of people possessing a complex interplay of individual and group tactics, comparable skills (in any case, there is no such “unit diversity” which makes it an interesting challenge for an AI sys- as in RTS), and game elements like “resource mining” or tem with high potential impact “research and construction” are absent. Thus, one of the ways to make the game complex enough to keep people’s interest is 8. Discussion to make room for diverse and nontrivial team strategies. This might mean that some team members have indeed to perform Intelligent team behavior is an interesting problem of AI activities that can be perceived as “less exciting,” like goal- research, having great practical significance. Currently, there keepers in soccer, who normally have no chances to score a are no established testbeds for benchmarking team AI behav- goal. However, their somewhat auxiliary team role is compen- ior, and we propose that sport games similar to soccer, ice sated by the possibility to exhibit the personal mastery hockey, or basketball have a potential to become such a required to perform their role well. testbed. Arguably, alternative options could include game Therefore, we believe that sport games may be at least as genres like RTS and MOBA. According to our comparison challenging as RTS/MOBA games in terms of developing effi- (see Tables 1–6), MOBA games are especially close to sport cient team strategies. Moreover, it is easier to start an AI games in several aspects. However, there are notable differ- research project with something relatively simple like two- ences, too. player-team beach volleyball, continue with five-a-side foot- While successful strategies in RTS and especially MOBA ball or futsal, and then proceed to a more complicated game games require team coordination, arguably, “team behavior” like soccer or American football. The problem of AI creation is not the most complex skill to master in these genres. RTS is also limited in this case with a relatively uniform set of and MOBA are specifically designed to be enjoyable for all tasks, dealing with various aspects of team behavior, while participants. Thus, every player typically controls a “hero,” in case of RTS/MOBA, a successful AI system has to address being in charge of their own group of subordinate units. Com- diverse issues like resource management, unit choosing, and plex team-based strategies might require some participants to complex hierarchical goal planning. sacrifice their squad for the benefit of the team and/or concen- In terms of the sheer number of game elements, sport trate on unpleasant menial tasks to support the frontline actu- games are much simpler than RTS/MOBA, and thus, their ally fighting the enemy. However, such patterns are not very game engines should be easier to develop and easier to International Journal of Computer Games Technology 7

Table 4: Action space property. Does the agent have many microlevel actions available? ﬀ — Team sports AI Yes, several: movement, di erent ball actions (shooting, dribbling, header most of them are parameterized); also speciﬁc actions for interaction with other agents (e.g., fouls and tackles).

MOBA AI Yes, very often the action space is growing during the game as new actions become available; it is slightly higher than for team sports at the start, with a low number of agent-speciﬁc actions.

RTS AI Action space is dynamically growing, but this is less emphasized and not available for all unit types; movement action space is quite limited, and certain actions such as shooting are often done automatically.

Table 5: Movements/state space property. How free is the AI in controlling movement? What additional data is needed to make up the state space? ﬁ Team sports AI In principle, agents can move over the whole eld, but roles strongly restrict movement space in most situations; additional restrictions are imposed by other factors, such as physical conditions and penalties of the players. In principle, agents move freely, but in most situations (especially at the beginning), the movements are strongly MOBA AI restricted; possible movements are also aﬀected by many additional properties, such as physical state and obtained items.

RTS AI Free movement, but rarely used in practice (e.g., for scouting), since units have to stay ogether for successful combat; reasoning is mostly done on the level of squads rather than individual units.

Table 6: Rhythm/independent episodes property. Is it possible to change the strategy easily (replan requirement)?

Team sports AI Most sport games have a restart behavior: game state is relatively independent of what happened in the last episode.

MOBA AI Frequent restarts in the beginning, and towers have similar meaning as goals in soccer; he more successful party (with more kills) gains advantage over time. RTS AI No restart behavior: once one team has strategic dominance, it is hard to overturn the situation. communicate with. A considerable challenge of modern tracking data can provide valuable insights for reaching sport game development lies in the need to implement these goals. smooth and complex animated sequences for a diverse set We can also note that the world of a typical sport game of onscreen actions. However, this task is not relevant for can hardly be characterized as “rich”—there are no fantasy- AI research, and thus can be greatly simplified. If we look themed landscapes, intricate dungeons, or exciting storylines. at existing team sport-like systems actually used for AI pro- As there is not plenty of “decorative” game elements to “dis- jects, such as RoboCup 2D Simulation platform, WeBots tract from” poor AI technology, it has to perform on par with (https://www.cyberbotics.com/) (used in recent AI World other core elements, such as animations and physics. Maybe Cup competitions), or MuJoCo Soccer Environment even more than in other game genres, some AI individuality (https://deepmind.com/research/open-source/mujoco- is expected, e.g., in a football video game, where Messi and soccer-environment) (serving as a platform for DeepMind’s Ronaldo shall not only visually look like their real-world experiments in team AI), we observe that all of them down- counterparts, but also somehow behave similar to them. play animation and player contact and instead focus on Another very interesting aspect of AI for team sport movement and passing behavior. games is related to the diversity of possible goals: take a role We must recognize, however, that there is no agreement of a teammate/opponent/referee; be a superstar forward on what constitutes “core” game elements that must be some- player or a supportive defender; be strong and efficient, or how represented in a computer simulation. For example, in be fun and inventive. Since the rise of really strong AI sys- soccer-like games, we see environments that implement or tems for a variety of games, it can be expected that these ignore elements like physical contact/tackling, overhead aspects of AI development would get more attention. As a passes, and the offside rule. Thus, it is difficult to assert which firmly established activity, sport games can be good environ- particular elements are crucial for “interesting” team behav- ments for investigating such goals, as we already know much ior (though, the rules of actual sports should serve as a about motivation of spectators and athletes, principles of reasonable approximation, and, probably, should not be good refereeing, and understand the general psychological ignored without good reasons). environment surrounding sports competitions. Since sport games represent real sports, serious players have more elaborate expectations about AI, especially as 9. Conclusion other game aspects reach higher levels of realism. The AI-controlled opponents have to be believable, and the This paper has argued that team sports occupy a special, pos- teammates have to be reasonably creative and supportive. sibly unique, niche in the AI development landscape, and thus It is likely that the growing availability of actual player deserve attention on their own, rather than within the context 8 International Journal of Computer Games Technology of other multiagent environments, such as RTS or MOBA [4] D. Silver, A. Huang, C. J. Maddison et al., “Mastering the game games. Sport video games are virtual representations of real- of Go with deep neural networks and tree search,” Nature, world competitions, and thus hold a borderline position vol. 529, no. 7587, pp. 484–489, 2016. between the real world and the virtual world. On the one hand, [5] V. Mnih, K. Kavukcuoglu, D. Silver et al., Playing Atari with they have to capture the essence of real-life events and repre- deep reinforcement learning, 2013, CoRR, abs/1312.5602. sent them faithfully to be appealing for sports fans. On the [6] V. Mnih, K. Kavukcuoglu, D. Silver et al., “Human-level con- other hand, they have to be fun to play and accessible for trol through deep reinforcement learning,” Nature, vol. 518, gamers, who want to be immersed in a make-believe world, no. 7540, pp. 529–533, 2015. where they can play the roles of successful top-class athletes. [7] S. Ontañón, G. Synnaeve, A. Uriarte, F. Richoux, D. Churchill, A general trend in the sport game genre is convergence and M. Preuss, “A survey of real-time strategy game AI between the game and the reality [40], which in practice research and competition in StarCraft,” IEEE Transactions on Computational Intelligence in AI and Games, vol. 5, no. 4, means more realistic graphics, environments, and AI sys- – tems. Good AI is one of the cornerstones of a high-quality pp. 293 311, 2013. team sport game; it is hard to imagine, for example, a [8] O. Vinyals, T. Ewalds, S. Bartunov et al., Starcraft II: a new soccer-like game set up in a people-only online multiplayer challenge for reinforcement learning, 2017, CoRR, mode and played without AI. abs/1708.04782. [9] O. Vinyals, I. Babuschkin, J. Chung et al., “AlphaStar: master- Currently, the advancements in AI technologies extend ” the list of games considered “solved” in the sense that AI ing the real-time strategy game StarCraft II, 2019, https:// deepmind.com/blog/alphastar-mastering-real-time-strategy- beats professional human players. In practice, this means that game-starcraft-ii/. the focus of game AI research can shift to secondary chal- [10] A. Juliani, A. Khalifa, V.-P. Berges et al., Obstacle tower: a gen- lenges, aimed at maximizing entertainment value of game eralization challenge in vision, control, and planning, 2019, products. Sport games are especially good for studying such arXiv preprint arXiv:1902.01378. emotional-driven goals of AI: the pool of popular sport [11] Collins, Collins English Dictionary Complete and Unabridged events is stable, and we know much about motivation and (13th ed.), 2018. enjoyment of both participants and fans. [12] Wikipedia contributors, Team sport — Wikipedia, the free One may argue that the issues related to subjective user encyclopedia, Wikimedia Foundation, Inc., 2019. enjoyment are hard to deal with: there are no easy ways [13] RoboCup, RoboCup Federation, RoboCup Federation, 2021. known to identify AI-related sources of user enjoyment, to implement AI that maximizes fun, nor to evaluate obtained [14] RoboCup, RoboCup federation: objective. results. However, dealing with sport game AI systems, it is [15] RoboCup, RoboCup soccer simulation. impossible to ignore these factors, especially in the light of [16] H. Akiyama and T. Nakashima, “Helios Base: An Open their growing importance. It is, therefore, both reasonable Source Package for the RoboCup Soccer 2D Simulation,” in Robot World Cup XVII, S. Behnke, M. Veloso, A. Visser, and necessary to anticipate this trend, by focusing more – research effort towards defining an appropriate and long- and R. Xiong, Eds., pp. 528 535, Springer, Berlin Heidel- berg, 2013. lasting AI benchmarking environment. [17] M. Prokopenko and P. Wang, Disruptive innovations in Robo- Cup 2D soccer simulation league: from cyberoos’98 to gliders, Data Availability 2016, CoRR, abs/1612.00947. The only external data used are the references in the manu- [18] M. Prokopenko, P. Wang, S. Marian, A. Bai, X. Li, and “ script, all available online. X. Chen, RoboCup 2D Soccer Simulation League: Evaluation Challenges,” Robot World Cup XXI [Nagoya, Japan, July 27-31, – Conflicts of Interest 2017], pp. 325 337, 2017. [19] P. MacAlpine and P. Stone, “UT Austin Villa RoboCup 3D ” The authors declare that there is no conflict of interest Simulation Base Code Release, in Robot Soccer World Cup fi regarding the publication of this paper. XX, Lecture Notes in Arti cial Intelligence, S. Behnke, D. D. Lee, S. Sariel, and R. Sheh, Eds., pp. 135–143, Springer Verlag, Acknowledgments Berlin, 2016. [20] J. Togelius, “How to run a successful game-based AI competi- ” The authors thank Tomoharu Nakashima and Tomohiro tion, IEEE Transactions on Computational Intelligence and AI – Harada for preliminary discussions on this article’s topic. in Games, vol. 8, no. 1, pp. 95 100, 2016. [21] A. Ferrein and G. Steinbauer, “20 years of RoboCup,” KI- References Künstliche Intelligenz, vol. 30, no. 3-4, pp. 225–232, 2016. [22] Y. Verhoeven and M. Preuss, “On the potential of Rocket Lea- [1] J. McCarthy, “Chess as the Drosophila of AI,” in Computers, gue for driving team AI development,” in Proceedings of IEEE chess, and cognition, pp. 227–237, Springer, 1990. SSCI 2020, pp. 2335–2342, 2020. [2] R. Beal, T. J. Norman, and S. D. Ramchurn, “Artificial intelli- [23] G. Kendall and L. J. A. Lenten, “When sports rules go awry,” gence for team sports: a survey,” The Knowledge Engineering European Journal of Operational Research, vol. 257, no. 2, Review, vol. 34, article e28, 2019. pp. 377–394, 2017. [3] L. Thompson, Making the Team: A Guide for Managers, Pear- [24] G. Bradley and C. Toye, Playing Soccer the Professional Way, son Education, 5th edition, 2014. Harper & Row, 1973. International Journal of Computer Games Technology 9

[25] J. Wilson, Inverting the Pyramid: The History of Football Tac- tics, Orion, 2010. [26] J. Wilson, Why Is the Modern Offside Law a Work of Genius?, The Guardian, 2010. [27] FIFA, Goalkeepers are not above the law, FIFA, 1997. [28] K. Dill, “What Is Game AI?,” in Game AI pro, S. Rabin, Ed., pp. 3–10, A K Peters/CRC Press, 2013. [29] M. Swiechowski, “Game AI competitions: motivation for the imitation game-playing competition,” in 2020 15th Conference on Computer Science and Information Systems (FedCSIS), pp. 155–160, Sofia, Bulgaria, 2020. [30] I. Oh, S. Rho, S. Moon, S. Son, H. Lee, and J. Chung, Creating pro-level AI for real-time fighting game with deep reinforcement learning, 2019, arXiv preprint arXiv:1904.03821. [31] Olivia Michael and Oliver Obst, Betarun soccer simulation league team: variety, complexity, and learning, 2017, arXiv preprint arXiv:1703.04115. [32] A. Correia and S. Esteves, “An exploratory study of spectators’ motivation in football,” International Journal of Sport Man- agement and Marketing, vol. 2, no. 5/6, pp. 572–590, 2007. [33] T. K. Scanlan, P. J. Carpenter, M. Lobel, and J. P. Simons, “Sources of enjoyment for youth sport athletes,” Pediatric Exercise Science, vol. 5, no. 3, pp. 275–285, 1993. [34] P. Sweetser, D. Johnson, J. Sweetser, and J. Wiles, “Creating engaging artificial characters for games,” in Proceedings of International Conference on Entertainment Computing, pp. 1–8, 2003. [35] G. Gandolfi, NBA Coaches Playbook: Techniques, Tactics, and Teaching Points, Human Kinetics, 2008. [36] M. Johnston and R. Walter, Hockey Plays and Strategies, 2nd Ed, Human Kinetics, 2018. [37] T. F. Tahtouh, Volleyball: Techniques and Tactics, Lulu Pub- lishing Services, 2017. [38] A. Zauli, Soccer: Modern Tactics, Reedswain, 2011. [39] C. Koch and M. Tilp, “Beach volleyball techniques and tactics: a comparison of male and female playing characteristics,” Kinesiology, vol. 41, pp. 52–59, 2009. [40] M. Sicart, “A Tale of Two Games: Football and FIFA 12,” in Sports Videogames, pp. 40–57, Routledge, 2013.