The Many AI Challenges of Hearthstone
Total Page：16
File Type：pdf, Size：1020Kb
The Many AI Challenges of Hearthstone Amy K. Hoover · Julian Togelius · Scott Lee · Fernando de Mesentier Silva Abstract Games have benchmarked AI methods since of a single game, discovering a few new variations on the inception of the field, with classic board games such existing research topics. as Chess and Go recently leaving room for video games Keywords Artificial Intelligence · Games · Hearth- with related yet different sets of challenges. The set of stone · Deckbuilding · Gameplaying · Player Modeling AI problems associated with video games has in recent decades expanded from simply playing games to win, to playing games in particular styles, generating game con- 1 Introduction tent, modeling players etc. Different games pose very different challenges for AI systems, and several differ- For decades classic board games such as Chess, Check- ent AI challenges can typically be posed by the same ers, and Go have dominated the landscape of AI and game. In this article we analyze the popular collectible games research. Often called the \drosophila of AI" in card game Hearthstone (Blizzard 2014) and describe reference to the drosophila fly’s significance in biolog- a varied set of interesting AI challenges posed by this ical research, Chess in particular has been the subject game. Collectible card games are relatively understud- of hundreds of academic papers and decades of research ied in the AI community, despite their popularity and [18]. At the core of many of these approaches is design- the interesting challenges they pose. Analyzing a single ing algorithms to beat top human players. However, game in-depth in the manner we do here allows us to despite IBM's Deep Blue defeating Garry Kasparov in see the entire field of AI and Games through the lens the 1997 World Chess Championships and DeepMind's AlphaGo defeating Lee Sedol in the 2016 Google Deep- Mind Challenge Match [47], such programs have yet Amy K. Hoover to exhibit the general intelligence hoped for when such New Jersey Institute of Technology benchmarks were originally proposed. 323 Dr Martin Luther King Jr Blvd Newark, NJ 07102 Since these victories, the AI and games commu- E-mail: [email protected] nity has gradually shifted focus to challenges in digi- arXiv:1907.06562v1 [cs.AI] 15 Jul 2019 Julian Togelius tal games. AI agents can outperform human players in New York University simulated arcade games, sometimes beating them when 6 MetroTech Center a win condition is possible [3, 39]. Agents and can also Brooklyn, NY 11201 outperform humans in real time strategy games (RTSs) E-mail: [email protected] like Starcraft II [56] and multiplayer online battle arena Scott Lee games (MOBAs) like Defense of the Ancients 2 [42]. Independent Irvine, CA 90066 While there is value in designing algorithms to win (e.g. E-mail: [email protected] the popularity of minimax and alpha-beta pruning al- Fernando de Mesentier Silva gorithms [46, 54] and Monte Carlo Tree Search (MCTS) Independent [9, 47]), like the drosphila fly necessarily shapes biolog- Rio de Janeiro, Brazil ical research it is possible that such focus limits the E-mail: [email protected] types of problems that can be solved in AI. Generally 2 Amy K. Hoover et al. such agents only play the particular game they were any given game. Each card costs the player a certain built or trained to play. amount of a resource called mana, and has other at- However, both digital and analog games pose a vari- tributes like attack, health, or spells it can cast. ety challenges for which a variety of AI-based methods While often combined by players in the way they have been developed in response [6, 8, 32, 36, 59] Re- discuss the game, at its core Hearthstone can be di- search field of AI and games focuses not only on play- vided into two related challenges: 1) playing the game ing to win, but on modeling player behavior, modeling and 2) selecting decks to be played in matches. When player experience, generating content and many other describing their overall approach to others, players of- challenges. There are many reasons people play games ten refer to their deck archetype and hero, which is beyond winning, and consequently many AI challenges inherently packaged with heuristics to effectively play present in any suitably rich game environment [60]. against other players. For example, the Odd Paladin Rather than the typical approach of isolating par- is at the time of writing a powerful type of Paladin ticular challenges in artificial intelligence and solving deck built around a particular card called Baku the them through exploration of a digital game, this paper Mooneater, which gets more powerful if all of the other instead discusses the multiple AI challenges posed by cards in the deck cost an odd amount of mana to play. the popular collectible card game called Hearthstone. The deck favors an an aggressive gameplay strategy While there are other digitial collectible cards games where the player focuses on destroying the enemy hero like Gwent [], The Elder Scrolls:Legends [], and Clash as quickly as possible rather than controlling the board of Clans [], Hearthstone is selected for its popularity or relying on clever card combinations. Each new ex- with over 100 million players. While it is perhaps most pansion of Hearthstone changes the set of possible cards common to consider such a game from a particular an- from which a player can build decks, that can in turn gle (e.g. playing to win it or modeling its players), this result in new sometimes more nuanced ways of play. paper presents a kaleidoscopic view of the AI challenges So even when exploring playing to win, as the game is the game presents and some of the current approaches updated, so are the decks and strategies. to addressing them. We do not intend this to be an ex- The following sections enumerate and outline AI haustive survey of AI Challenges in Hearthstone, but to challenges in Hearthstone, including playing in Section be representative; the AI challenges covered are likely to 3, building decks in Section 4, helping human players be closely related to some challenge already described learn to play and build decks in Section 5, and help- for other games. ing designers build the game in Section 6. These ideas Hearthstone is a game rich in AI challenges and are then combined to explore how the field of AI can relatively unexplored, perhaps more than most games, be taught by approaches addressing these challenges in given the many facets of the game. We also think that Hearthstone. But first we will outline the characteris- most of the research challenges identified here carry tics of Hearthstone that delineate the challenges that over to other collectible card games, and to some extent the game poses. games that include elements of these games, such as deck building. Still, we believe that many other games 2.1 Characteristics of Hearthstone have a rich diversity of AI challenges, far more than are usually considered, if you just look. This paper can As proposed by Elias et al. [17], analyzing a game based therefore also serve as a paradigm for papers elucidating on its characteristics is an important way to understand and cataloguing AI challenges in other types of games. the challenges it poses. Some of the most salient charac- teristics of Hearthstone from the perspective of game- 2 The Challenges of Hearthstone playing agents are the following: the game has discrete inputs and outputs, meaning that the state observa- Like many traditional board games, Hearthstone [5] is a tion is discrete and so is the action (i.e. what card to two-player, turn-taking, adversarial game. However, un- play and potentially its target). The observable game like these games, it contains a large amount of stochas- state has a natural and simple structured form (cards ticity in play and partial information. The goal of this on the table and in hand), meaning that playing from digital collectible card game is through playing differ- pixels is unnecessary as it gives no new information ent types of cards, decrease the opponents health from and only adds an arbitrary computer vision problem. thirty to zero. Players initially choose one of nine dif- The branching factor is variable depending on game ferent heroes, which will determine the types of cards state, but generally high if one considers all the actions that the player can access. They then build decks of a player can take in a turn (five cards that can all be thirty cards from over 1900 which will be available in played in one turn, where each card can have one of five The Many AI Challenges of Hearthstone 3 targets, lead to a branching factor of 375000; but it is ulator with at least fourteen contributors. The develop- not uncommon with game states where the player can ers support research initiatives through their software choose only whether to play a single card or do nothing and subreddit 3. Other simulators like Metastone 4 and at all). Spellsource 5 have a fully functioning GUI for human The game is considerably impacted by hidden infor- players to play games. mation (partial observability), both in that the player does not know the opponent's hand and which cards will be drawn next from the deck. While some infor- 3 Playing the Game mation can be observed or predicted (e.g. how long a particular card has been in an opponent's hand), only Different games pose different challenges for playing. guesses can be made based on a priori knowledge of Some games are about long-term planning, while others the deck or current known successful strategies.