A Pattern Mining Approach to Study Strategy Balance in RTS Games
The MIT Faculty has made this article openly available. Please share how this access benefits you. Your story matters.
Citation Bosc, Guillaume; Tan, Philip; Boulicaut, Jean-Francois; Raissi, Chedy and Kaytoue, Mehdi. “A Pattern Mining Approach to Study Strategy Balance in RTS Games.” IEEE Transactions on Computational Intelligence and AI in Games 9, no. 2 (June 2017): 123–132 © 2017 Institute of Electrical and Electronics Engineers (IEEE)
As Published https://doi.org/10.1109/TCIAIG.2015.2511819
Publisher Institute of Electrical and Electronics Engineers (IEEE)
Version Author's final manuscript
Citable link http://hdl.handle.net/1721.1/110366
Terms of Use Article is made available in accordance with the publisher's policy and may be subject to US copyright law. Please refer to the publisher's site for terms of use. 1 A Pattern Mining Approach to Study Strategy Balance in RTS Games
Guillaume Bosc1, Philip Tan2, Jean-Franc¸ois Boulicaut1, Chedy Ra¨ıssi3, and Mehdi Kaytoue1 1Universite´ de Lyon, CNRS, INSA-Lyon, LIRIS, UMR5205, F-69621, France 2MIT Game Lab, Cambridge (MA), USA 3LORIA (CNRS - Inria NGE - Universite´ de Lorraine), Vandoeuvre-les-Nancy,` France
Abstract—Whereas purest strategic games such as Go and This context roots the motivation of our work: behavioral Chess seem timeless, the lifetime of a video game is short, data can help to study the balance of a game, that is, to influenced by popular culture, trends, boredom and technolog- adjust the rules over time while still enabling novel rules ical innovations. Even the important budget and developments allocated by editors cannot guarantee a timeless success. Instead, to counter boredom. This is especially important for games novelties and corrections are proposed to extend an inevitably played as an electronic sport, and also for game lifetime bounded lifetime. Novelties can unexpectedly break the balance extension in general. We will focus on the concept of balance of a game, as players can discover unbalanced strategies that which is a core concept in competitive game design: it consists developers did not take into account. In the new context of of defining and tuning the basic rules that prevent extreme electronic sports, an important challenge is to be able to detect game balance issues. In this article, we consider real time strategy situations, thus balancing fairness and competitive aspects. games (RTS) and present an efficient pattern mining algorithm as In this article, we define and mine patterns in game his- a basic tool for game balance designers that enables one to search torical data for a better understanding of balance issues in for unbalanced strategies in historical data through a Knowledge RTS games. The intuition of the balanced sequential pattern Discovery in Databases process (KDD). We experiment with our discovery problem is the following. Consider a set of games, algorithm on StarCraft II historical data, played professionally as an electronic sport. each of them represented by a sequence of actions of two Index Terms—Mining methods and analysis, video game players (thus entailing the player strategy). The problem is to find patterns as sequence generalizations that frequently occur in the historical data and whose balance is given by I. INTRODUCTION proportions of their wins and losses. Our goal is twofold: (i) we give the basic algorithmic tools that enable an efficient The recent and fast development of the video game in- pattern mining, (ii) we show that the extracted patterns reveal dustry has been catalyzed by technological innovations, a interesting knowledge: democratized access to connected electronic devices, new (i) We revisit the problem of strategy elicitation from economic models (free games where users may pay for extra • contents), and recently with competitive gaming (esports) and two player RTS games by differentiating two cases: when video game live streaming platforms [1]. People not only both players have access to (a) different game actions enjoy playing, but also enjoy learning from watching others and (b) the same game actions. In the first case, we performing, as a daily leisure activity [2], [3]. show how existing pattern mining methods enable with Producing a video game is an expensive process, thus, keys slight modifications to discover frequent strategies and to a massive, immediate and durable success are sought. Prag- compute a balance measure. In the second case, the most matically however, one attempts to extend the game lifetime general, existing approaches fail: we propose an original after the release, by correcting bugs, introducing new features algorithm, BALANCESPAN. (ii) We show through experiments that BALANCESPAN is and considering user feedbacks. Whereas it could be easily • argued that bugs are not acceptable after a release, it is hard scalable and able to discover patterns of interest in a large to predict the results of human creativity in presence of rich StarCraft II dataset that can help detecting balance issues. environments that are video games. For that matter, we anchor our algorithm in a Knowledge Discovery in Databases process [8]. Pattern mining is one Hopefully, companies realized in the current big data at- of the many steps of this interactive and iterative process mosphere that the tremendous amounts of game behavioral guided by an expert of the data domain who selects and data they store are valuable to face many new challenges interprets the patterns [9], [10]. such as: detecting unexpected situations and bugs [4] cheaters [5], designing artificial agents [6], improving match making The paper is organized as follows. Section II recalls the systems and adjusting game difficulty [7]. Analyzing these basics of sequential pattern mining before the introduction of massive sets of historical data by means of visualization, our mining problem in Section III. Our method is developed machine learning and data mining techniques is at the heart in sections IV and V. Algorithms are designed (Section VI) of video game analytics for enhancing user experience and and experimented with E-Sport data (Section VII) before over- extending game lifetime [4]. viewing related work and concluding. 2
II. PRELIMINARIES III. THE PROBLEM OF STRATEGY ELICITATION We recall the basic definitions of frequent sequential pat- terns [11] and emerging patterns [12] useful in the sequel. Let A zero-sum game, or competitive interaction, can be mod- be a finite set of items. Any non-empty subset X eled as a sequence of actions performed by two players where I ✓ I is called an itemset.Asequence s = X ,...,X is an exactly one player wins (no ties). Such a sequential game can h 1 li ordered list of l>0 itemsets. l is the length of the sequence, be represented as a sequence of sets of actions, each action whereas l X is its size. Considering as a set of events performed by one of the two players. When both players play i=1 | i| I (or actions), an itemset denotes simultaneous events while in real-time, one can describe these interactions as sequences the orderP between two itemsets indicates a strict preceding of itemsets. An itemset is then a set of simultaneous actions, relation. A sequence database is a set of sequences or within an insignificant interval of time. D |D| over . Sequences may have different lengths and sizes and Definition (Interaction (sequence) database). Given a set of I are uniquely identified, see Table I (omitting the third column). players Players and a set of actions Actions, a sequence database is called an interaction database. Each sequence R Definition (subsequence). A sequence s = X1,...,Xl is a denotes one single game, i.e., an interaction between two h s i subsequence of a sequence s0 = X10 ,...,X0 , denoted s players, and is defined over the set of items = Actions h ls0 i v I ⇥ s , if there exists 1 j IV. BALANCED PATTERNS IN NON-MIRROR DATABASES TABLE III A NON MIRROR INTERACTION DATABASE In this section, we consider an interaction database, called a non-mirror database, where the set of actions is different for id Interaction sequence Winner each of the players in a single interaction. It means that we s1 ab c 1 h{ }{ }i I only have two types of players in each sequence and in the s2 ab a c 1 h{ }{ }{ }i I whole database (e.g., Protoss and Zerg factions in the RTS s3 abc c 2 h{ }{ }i I s4 ac a c 1 game StarCraft II), and these types are determined by the h{ }{ }{ }i I s5 c b ac 2 actions they can do. As such, the type can also be used as a h{ }{ }{ }i I winning class label. To characterize balanced patterns in such databases, we consider a simple transformation of the original TABLE IV interaction database by dropping the player associated to A SIGNED INTERACTION DATABASE. R each action, and labeling each sequence by the type of the id s winner. This enables to express a balance measure as a growth- 2 S + + rate measure in this new data representation. The transformed s1 a b c h +{ + }i+ s2 a b c c database is then formally defined as follows. h +{ + + } i s3 d b c d h { + +} i s4 a b c Definition (Transformed interaction database). A sequence h { }i database defined over the set of items (actions) T I1 [ I2 such as 1 2 = and class : 1, 2 is called a I \ I ; T ! {I I } V. BALANCED PATTERNS IN MIRROR DATABASES transformed interaction database. In this section, we consider interaction sequence databases Consider an interaction s where the winner is char- where the players have access to the same set of actions. 2 T acterized by the actions : we have class(s)= that Consequently, the latter cannot be partitioned in two sets I1 I1 gives the winner of the interaction. This brings back the and the previous approach can not apply. We propose a problem of finding frequent balanced interaction patterns to new interaction database representation, signed interaction well-known the emerging patterns settings. Indeed, consider databases. It enables to define frequent balanced patterns from an arbitrary pattern s over : its support in the whole an arbitrary interaction database (either mirror or non-mirror). I1 [ I2 database sup(s, ) tells us its frequency, while sup(s, I1 ) T T and sup(s, I2 ) enable to define a balance measure as a Definition (Signed interaction database). Recall that T growth-rate. Actions is a finite set of actions shared by both players. We introduce = Actions +, denoting actions associated Is ⇥{ } Problem (Mining balanced patterns from non-mirrors either to a positive or negative class. A signed database is S interactions). Let be a transformed database obtained from built from an interaction database as follows: Each action T R a non-mirror interaction database . is defined over of an interaction sequence is signed + if it is performed by the R T I1 [I2 where represents the type k of player (k 1, 2 ) and winner and signed if performed by the loser (both players Ik 2 { } class : , assigns to any sequence its winner and class labels are dropped). T ! {I1 I2} type. is a minimum frequency threshold. The problem is to extract the set of so-called frequent balanced patterns t such Definition (Dual of an item, itemset and sequence). Let 1 F2 as for any s t, freq(s, I ) and freq(s, I ) s = Actions +, be the set of signed items, or actions. 2 F T T I ⇥{ } c (implying freq(s, ) ) and the balance measure is For any (a, c) s, also written a , we define its dual as T 2 I computed and given by: c +, c dual(a, c)=dual(a )=(a, +, c)=a{ }\ sup(s, k) { }\ balance(s, k)= | T | T sup(s, 1) + sup(s, 2) Informally, it means that the dual of a signed action is the | T | | T | same action where the class c has changed. This definition is simply propagated for itemsets and sequences of itemsets, for Remark. The balance measure is a normalized version of the any X and any s = X ,X ,... a sequence over : growth rate given in previous section such that balance(.) ✓ Is h 1 2 i Is 2 ]0, 1] and balance(s, 1)+balance(s, 2)=1which entails dual(X)= dual(x), x X T T { 8 2 } a zero-sum game property. dual(s)= dual(X ),dual(X ),... h 1 2 i Example. Table III gives a transformed interaction database , obtained from a non-mirror interaction database , with Example. In Table IV, we have s = a, b, c, d +, , T R + I { + } ⇥ { } = a, b and = c being the sets of actions of each dual(a )=a and dual(s1)= a b c . I1 { } I2 { } h { }i player type. With =0.2, s = ab c is a frequent bal- 1 h{ 2}{ }i 2 1 anced pattern since freq(s, I )= 3 and freq(s, I )= 2 . These definitions enable now to naturally introduce a bal- T 2 T 1 Moreover, balance(s, I1 )= and balance(s, I2 )= . It ance measure that would, for a sequential pattern s give the T 3 T 3 means that s wins two times more for the type 1 of player proportion of its support among the support of both itself and than for the type 2. its dual. 4 Definition (Balance measure). Let s be a frequent sequential A. The PREFIXSPAN algorithm pattern in a database . The balance measure of s is Given a sequence database over items and a minimum S D I sup(s, ) frequency threshold ,PREFIXSPAN outputs all frequent se- balance(s)= | S | (1) quential patterns and only them [11]. Firstly, the database is sup(s, ) + sup(dual(s), ) D | S | | S | scanned once to find all the frequent items from , called size- This intuitive definition however does not hold. Since ac- I 1 sequential patterns. Secondly, each of these prefixes is used tions are shared by the two players, both a sub-sequence and to divide the search space: for one prefix, say a (and a ), its dual may occur in a single sequence s . Consider h i 2 I + +2 S + one retains only sequences of containing a and only keeps the following example: = a b a b , a b D 1 S {h{ }{+ }i h{ }i} for each of these sequences the longest suffix starting by a. with = . The sequence s = a b is a frequent 2 h{ }i The set of the prefixes sequences of the remaining sequences sequential pattern, and sup(s, ) =1. We have also | S | + is called a projected database w.r.t. to prefix a , written a . sup(dual(s), ) = sup(dual( a b ), ) =2. Hence, h i D|h i | S | | h{ }i S | Thirdly, this projected database is scanned to generate the balance(s)= 1 = 1 . However, since s and dual(s) both 1+2 3 size-2 sequential patterns having a as prefix. The process appear in the first sequence, it should not be counted two h i is recursively applied leading to a tree structure where each times. This leads us to the definition of the balance measure node represents a frequent sequential pattern (associated with a in the general case in which we ignore sequences where both projected database of a least sequences) and an edge a pattern and its dual appear. d ⇥|D|e to an extension: the item added to a size-k sequential pattern to generate a size-(k+1) sequential pattern. For a prefix s and Definition (Generalized balance measure). For a sequential an item a, two kinds of extensions are considered: appending pattern s, the generalized balance measure is given by a as a new suffix itemset of s, noted s a, and appending s sup(s, ) sup(dual(s), ) a within the last itemset of s, written s i a ( denotes an balancegen(s)= | S \ S | (2) sup(s, ) sup(dual(s), ) extension in general). At the end, the pattern tree structure is | S t S | where denotes the exclusive union A B =(A B) (A B). explored and each node outputs a pattern. t t [ \ \ In the following, balance will always refer to the general version. We have that balance(s) [0, 1] and balance(s)+ Example. We briefly illustrate PREFIXSPAN on the toy 2 balance(dual(s)) = 1 which expresses a zero-sum game example of the Table I with =0.5. A larger example is property. available in the original publication of PREFIXSPAN [11]. The first step of PREFIXSPAN consists of finding frequent item from : a ( sup( a , ) =4), b ( sup( b , ) =4), Problem (Mining balanced patterns from signed interac- D h i | h i D | h i | h i D | c ( sup( c , ) =4), d ( sup( d , ) =3), e tions). Let be a signed interaction database defined over s S I h i | h i D | h i | h i D | h i generated from an interaction database , and a minimum ( sup( e , ) =3) and f ( sup( f , ) =3). For each R | h i D | h i | h i D | frequency threshold. The problem is to extract the set of so- of these previous frequent sequential patterns s of size 1, PREFIXSPAN projects into a projected database with called frequent balanced patterns s such as for s s, D F 2 F prefix s. Thus, the a -projected database a is composed freq(s, ) , freq(dual(s), ) and the balance h i D|h i S S of 4 sequences: abc ac d cf , d c bc ae , measure is computed and given by (2). Furthermore, the fact h{ }{ } { }i h{ } { }{ }i b df cb and f cbc . Then, PREFIXSPAN that both s and dual(s) have to be frequent leads to redundant h{ }{ } i h{ } i searches for frequent sequential patterns of size 2: aa information: it is enough to keep s along with its support, h i ( sup( aa , a ) =2), ab ( sup( ab , a ) =4), (ab) balance measure and intersection of support common(s)= | h i D|h i | h i | h i D|h i | h i ( sup( (ab) , a ) =2), ac ( sup( ac , a ) =4), ad sup(s, ) sup(dual(s, )) to know the measures of its | h i D|h i | h i | h i D|h i | h i | S \ S | ( sup( ad , a ) =2) and af ( sup( af , a ) =2). dual. As such, the problem is also to compute a non redundant | h i D|h i | h i | h i D|h i | Then for each of these sequential patterns of size 2 collection of patterns s where, if s s then dual(s) s. F 2 F 62 F PREFIXSPAN creates the projected databases and extracts Example. Table IV gives a signed interaction database ob- frequent patterns of size 3, and so on. 1 + S tained from an arbitrary . With = 4 , s = a c appears R h i 2 two times, its dual appears only once, hence balance(s)= 3 . B. Mining Balanced Patterns with EMERGSPAN Given a sequential database over an arbitrary , where I Remark. Any interaction database, mirror or non-mirror, can each sequence is labeled by a class (among two classes) and be represented as a signed interaction database. For the non- a minimum frequency threshold, the general problem here mirror case, one can easily prove that for any balanced pattern is to find all frequent patterns, each one provided with its s, we have common(s)= and thus Formula (1) holds. ; growth-rate. In our settings, this involves mining a non-mirror interaction database as a transformed database where R T VI. ALGORITHMS sequences are composed of actions of both players and the We present several algorithms to extract frequent balanced class label gives the winner’s type. The balance measure is patterns from interaction databases. We introduce first a well- then exactly the growth rate as defined in the general settings known framework for mining frequent sequential patterns, and can be computed in negligible time as a post-processing called PATTERN-GROWTH and its associated algorithm PRE- [14]. Based on PREFIXSPAN,EMERGSPAN works as follows. FIXSPAN [11]. For each sequence of the database , its class is appended T 5 ALGORITHM:PREFIXSPAN C. Mining Balanced Patterns with PREFIXSPANNAIVE Now we consider a signed interaction database over s Require: A sequence s, the s-projected DB s, and a threshold S I D| and a minimal frequency threshold . The problem is to Ensure: The frequent sequential pattern set F 1: F s ; extract the set of frequent sequential patterns and compute { } 2: scan s once, find every frequent item a such that their balanced measure. We first use the original PREFIXSPAN (a)D|s can be extended to (s a), or i algorithm to build the frequent pattern tree structure. We need (b) s can be extended to (s s a) 3: if no valid a available then for each pattern to (i) compute its balance, and (ii) to ensure 4: return F ; that for a pattern and its dual only one of them is outputted. We 5: end if then propose to store for each node (equivalently sequential 6: for all valid a do 7: (a) F F PrefixSpan(s i a, s a, ) or pattern) a pointer to its dual pattern. At the first level of the [ D| i 8: (b) F F PrefixSpan(s s a, s a, ) tree, a node represents a single frequent item, and there is a [ D| s 9: end for pointer towards its dual (if both are frequent). Recursively, 10: return F ; when an item is used to expand a sequential pattern p to obtain a pattern q, we compute the pointer towards dual(q) by searching among the children of dual(p). If dual(q) exists, the ALGORITHM:EMERGSPAN pattern q is outputted, dual(q) is flagged as already outputted (redundant pattern) and the process recursively continues. Oth- Require: A sequence s, the s-projected transformed DB s, and a minimum threshold T| erwise, the algorithm backtracks. In this way, q and dual(q) Ensure: The frequent sequential pattern set F are never outputted together. Computing the balance for non 1: F ; ; mirror databases is straightforward since for each node/pattern 2: if s ends with the class then 3: Compute balance(s) thanks to sup(parent(s)) we have access to its dual support. For mirror databases, 4: F s we need however to know common(q) which is stored for { } 5: return F ; each node. The proof of the completeness and correctness 6: end if REFIX PAN AIVE 7: scan s once, find every frequent item a such that of P S N for extracting all balanced patterns T| (a) s can be extended to (s i a), or without redundancy is direct: first, PREFIXSPAN extracts all (b) s can be extended to (s s a) frequent patterns, thus any pattern s and its dual dual(s) are 8: if no valid a available then 9: return F ; nodes of the pattern tree and none can be missed; second, 10: end if as we visit in the tree traversal both a pattern s and its dual 11: for all valid a do dual(s) (if frequent), we ensure no redundant patterns. 12: (a) F F EmergSpan(s i a, s a, ) or [ T| i 13: (b) F F EmergSpan(s s a, s a, ) [ T| s 14: end for Example. PREFIXSPANNAIVE requires a signed intereaction 15: return F ; database. Let us consider that of Table IV with =1/4 (i.e., frequent items are items that only appear once). PRE- FIXSPANNAIVE starts with an empty sequence s = hi (dual( )= ) and the entire database. First, it searches as a new itemset at its end. It ensures that frequent patterns hi hi + for frequent items: e.g., both a and a are frequent. Thus, containing a class label are leaves of the pattern tree, and that + PREFIXSPANNAIVE flags that a is the dual of a , and two sequential patterns that differ only by their classes have + h i h i calls P refixSpanNaive( a , a+ , ). Then, PREFIXS- the same direct parent, allowing a direct computation of the h i S|h i PANNAIVE searches for frequent items in a+ : e.g., c is balance measure. S|h i frequent. So, it will search in the children of a if there is + h i c , but at this step, the node of a has not been explored yet. Example. To illustrate how EMERGSPAN works, let us con- h i Thus, when a is explored, PREFIXSPANNAIVE extracts sider the previous example on the transformed interaction h i + frequent items in a : e.g., c is frequent. So it will database of Table III. Remind that this database is obtained S|h i + T search among the children of dual( a ) if dual(c ) exists: from a non-mirror interaction database , with 1 = a, b + h i R I { } a c is a frequent sequential pattern that could be outputted. and = c being the sets of actions of each player type. h i I2 { } Finally, PREFIXSPANNAIVE will proceed iteratively until none Thus, we transformed each of these sequences by appending sequential patterns could be extended. the item (resp. ) at the end if the player with actions in I1 I2 (resp. ) won. For example, s becomes ab ac . The I1 I2 2 h{ } I1i exploration of EMERGSPAN works the same way as PREFIXS- D. Mining Balanced Patterns with BALANCESPAN PAN excepted that it only keeps patterns that end up with either The problem of PREFIXSPANNAIVE is that it generates or . Thus, computing balance of pattern X ...X both a pattern and its dual as different nodes in the pattern I1 I2 h 1 lIji only requires to backtrack to the parent (i.e., X ...X ) to tree. Furthermore, it also generates nodes for patterns that are h 1 li get its support to evaluate the balance. So, when EMERGSPAN frequent but whose dual is not frequent. Consequently, and deals with the sequential pattern s = ab c , since the last this is shown in the experiments (Section VII), an important h{ } I1i item of s is , it will output this pattern, and to compute amount of nodes are useless. To solve that problem, and to I1 its balance measure, it uses the support of its parent node be sure only nodes corresponding to balanced patterns are sup(s, ) s0 = ab c : balance(s)= |sup(s ,T )| =0.67. generated (and only them, i.e. correct and complete), we h{ } i | 0 T | 6 ALGORITHM:PREFIXSPANNAIVE ALGORITHM:BALANCESPAN Require: A sequence s, the s-projected signed DB s, and a minimum Require: A sequence s, its dual sequence s0 = dual(s), the s-projected S| threshold signed DB s, the s0-projected signed DB s , and a minimum S| S| 0 Ensure: The frequent sequential pattern set F threshold 1: Compute balance(s) thanks to dual(s) Ensure: The frequent balanced pattern set F 2: F s ; 1: Compute balance(s) { } 3: scan s once, find every frequent item a such that 2: F (s, s0) ; S| { } (a) s can be extended to (s i a), or 3: scan s once, find every frequent item a such that S| (b) s can be extended to (s s a) (a) s can be extended to (s a), or i 4: if no valid a available then (b) s can be extended to (s s a) 5: return F ; 4: scan s once, find every frequent item b such that S| 0 6: end if (a) s can be extended to (s b), or 0 0 i 7: for all valid a do (b) s can be extended to (s s b) 0 0 8: (a) Search for dual(s i a) among children of dual(s) 5: if no valid a or b available then 9: if dual(s i a) exists then 6: return F ; 10: Link dual(s i a) to s i a 7: end if 11: F F PrefixSpanNaive(s i a, s a, ) 8: for all valid a and b such that a = dual(b) do [ S| i 12: end if 9: (a) F F BalanceSpan(s i a, s0 i b, s a, s b, ) [ S| i S| 0 i or, 10: (b) F F BalanceSpan(s s a, s0 s b, s a, s b, ) [ S| s S| 0 s 13: (b) Search for dual(s s a) among children of dual(s) 11: end for 14: if dual(s s a) exists then 12: return F ; 15: Link dual(s s a) to s s a 16: F F PrefixSpanNaive(s i a, s a, ) [ S| s 17: end if 18: end for successor of StarCraft: Brood War, test bed for many research 19: return F ; in AI [6]. A game involves two players each choosing a faction among Zerg (Z), Protoss (P) and Terran (T): there are 6 different possible match-ups with different strategies of propose the BALANCESPAN approach. The general idea is the following: instead of considering each item a as an game. During a game, two players are battling on a map (aerial 2 Is extension on sequence s leading to a new projected database view), controlling buildings and units to gather supply, build an army with the final goal of winning by destroying the s a and consequently a new node in the pattern tree, S| we consider simultaneously an item and its dual, hence two opponent’s forces. Such actions (training, building, moving, attacking) are done in real-time. Each faction (Z, P, T) allows projected databases s a and dual(s) dual(a) are related S| S| to a single node (this is done for both kinds of extensions different units and buildings with distinctive weaknesses and and ). Thus, it ensures that no redundant patterns are strengths following a rock-paper-scissors principle. As such, i s generated, since both a sequence s and dual(s) are generated there are mirror match-ups (TvsT, PvsP, ZvsZ) and non-mirror at the same node, and it allows to compute balance(s) (or match-ups (TvsP, TvsZ, PvsZ). A strategy is hidden in large replays balance(dual(s)) directly if and only if both s and dual(s) sequences of actions generated by players and called . Played as an electronic sport, StarCraft II is regularly are frequent. It follows that BALANCESPAN produces a correct and complete collection of frequent balanced patterns. patched: basic rules of the games are adjusted (properties of units, building times, ...), new rules are introduced through expansion sets (heart of the swarm and legacy of the void). Example. BALANCESPAN also requires a signed interaction The balance design team of StarCraft II, often needs to study database, but contrary to PREFIXSPANNAIVE, it proceeds to a double projection at a time. Let us still consider the historical data, care about player feedback on Web forums, and toy dataset of Table IV with =1/4. Starting with the finally to justify their choices. After quantitative experiments empty sequence and the entire dataset, the first step consists of our algorithms, we will discuss the usefulness of our of finding frequent items: e.g., a+ is frequent. Contrary to approach to help studying balance issues in RTS games. PREFIXSPANNAIVE,BALANCESPAN directly generates the dual of a+ to proceed to the double projection. Thus, it B. Datasets h i + checks if a is frequent and then projects on both a and h i StarCraft II replays are files that store any action performed a at a time: i.e., it creates a new node in the pattern tree h i + by all players during a game, and are easily accessible on the that is related to the couple ( a , a ). The next step calls h i h i Web 1. We retained 91, 503 games with a total of 3.19 years of BalanceSpan on this new node. So it searches for frequent game time. The average length of a game is about 20 minutes. items in the projected database a+ : e.g., c . Thus it checks S|h i A game is selected if it involves a high level players (in the if dual(c ) is frequent in a : the node containing the + + S|h i highest leagues and playing at least 200 actions per minute), couple ( a c , a c ) is created and explored in the next h i h i since casual (by opposition to professional) players are not step. able to follow specific strategies. We divided the 91, 503 VII. EXPERIMENTS replays into six different sequence datasets, one for every match up. Buildings are one of the key elements of a strategy, A. StarCraft II in a nutshell since they enable different kinds of units production: from We study one of the most competitive real-time strategy games (RTS), StarCraft II (Blizzard Entertainment, 2010), 1http://wiki.teamliquid.net/starcraft2/Replay Websites 7 Run Time (log-scale (sec)) # Patterns 2.5e+06 Run Time (log-scale (sec)) # Patterns Run Time (log-scale (sec)) # Patterns 10000 BalanceSpan 1.6e+06 450000 10000 BalanceSpan 10000 BalanceSpan PrefixSpanNaive 2e+06 1.4e+06 400000 1000 PrefixSpan PrefixSpanNaive PrefixSpanNaive 1000 PrefixSpan 1.2e+06 1000 PrefixSpan 350000 300000 100 1.5e+06 1e+06 100 100 250000 800000 10 1e+06 200000 10 600000 10 150000 500000 400000 1 1 1 100000 200000 50000 0.1 0 0.1 0 0.1 0 10075 50 30 20 10 5 3 2 100 75 50 30 20 10 5 3 2 10075 50 30 20 10 5 3 2 1 0.50.1 MinSupp % MinSupp % MinSupp % (a) PvP (b) TvT (c) ZvZ. Run Time (log-scale (sec)) # Patterns Run Time (log-scale (sec)) # Patterns Run Time (log-scale (sec)) # Patterns 300000 1.8e+06 3e+06 10000 10000 10000 PrefixSpanNaive PrefixSpanNaive 1.6e+06 PrefixSpanNaive BalanceSpan 250000 BalanceSpan BalanceSpan 2.5e+06 1.4e+06 1000 EmergSpan 1000 EmergSpan 1000 EmergSpan PrefixSpan 200000 PrefixSpan 1.2e+06 PrefixSpan 2e+06 100 100 1e+06 100 150000 1.5e+06 800000 10 10 10 100000 600000 1e+06 400000 1 50000 1 1 500000 200000 0.1 0 0.1 0 0.1 0 100 75 50 30 20 10 100 75 50 30 20 10 5 3 2 10075 50 30 20 10 5 3 2 1 MinSupp % MinSupp % MinSupp % (d) PvT. (e) PvZ (f) TvZ Fig. 1. Run time and number of patterns. Percentage of used nodes TABLE V 100 DATASETS: SEQUENCE AND ITEM COUNTS; MAX. AND AVG. SEQUENCE BalanceSpan SIZES (smax, savg ); MAX. AND AVG. ITEMSETS SIZES (imax, iavg ). PrefixSpanNaive 80 EmergSpan Dataset s s i i 60 |D| |I| max avg max avg - PvP 6,823 26 42 21.7 6 1.8 S 40 - PvT 19,270 62 42 25.9 8 1.8 S - PvZ 23,491 52 41 23.1 7 1.7 S 20 - TvT 7,598 36 38 23.8 7 1.8 S - TvZ 24,459 62 37 21.2 8 1.6 S - ZvZ 9,922 26 28 10.4 7 1.4 0 S 100 75 50 30 20 10 5 3 2 - PvT 19,270 34 43 26.9 8 1.8 T - PvZ 23,491 29 42 24.1 7 1.7 MinSupp % T - TvZ 24,459 34 38 22.2 8 1.6 T Fig. 2. Percentage of used nodes in the PvZ dataset. (since signed databases correspond to the general case whereas each replay, we derive a sequence where the items represent transformed databases are specific to non-mirror databases). the buildings the players chose to produce in real time, and For different minimum frequency thresholds , we present the itemsets denote time windows of 30 seconds. We consider runtime of PREFIXSPAN on as a rough baseline (since it only the 10 first minutes of each game (the strategical impact S does not compute the balance of a pattern), and the runtimes of a building is less important after 10 minutes). Table V of the three other algorithms on their respective data represen- summarizes all characteristics of the datasets. tation. It follows that our general algorithm BALANCESPAN is the only one able to be executed with lowest (Figure 1 (d), C. Run-time analysis and memory usage (e) and (f)). We report the same results for the general case with mirror datasets (i.e., only for PvP, TvT and ZvZ) in We implemented our algorithms over the original C++ S Figure 1 (a), (b) and (c). BALANCESPAN clearly outperforms version of PREFIXSPAN [11], and experimented on a 1.8 GHz PREFIXSPANNAIVE, its only competitor (remembering that Intel Core i5 with 8 GB machine. Note that we released PREFIXSPAN is given as baseline since it does not compute both the source-code and the datasets used in the following the balances, and EMERGSPAN does not apply for mirror experiments [15]2. We discuss running times of the proposed databases). Indeed, even if sometimes PREFIXSPANNAIVE algorithms. Firstly, we consider the non-mirror databases given seems to have the same run time as BALANCESPAN with high by transformed interaction databases and signed databases T S value of , it cannot reach lowest frequency thresholds . Note 2Datasets, source-code and scripts are available on https://github.com/ that on the figures, missing points correspond to unterminated guillaume-bosc/BalanceSpan runs when available memory is insufficient. Figure 3 shows 8 # Patterns # Patterns 300000 450000 400000 250000 350000 200000 300000 250000 150000 200000 100000 150000 100000 50000 50000 0 0 1 5 10 15 1 5 10 15 Length of pattern Length of pattern Fig. 3. Cumulative distribution of the length of the patterns (number of itemsets) for the PvT (left) and the ZvZ (right) datasets. Fig. 4. Patterns support and balance for the dataset ZvZ the distribution of the length of the extracted patterns for the PvT and ZvZ datasets. Exploring a large collection of patterns can be done in The quantity of memory used is a very important aspect of many ways. First, as illustrated hereafter, the expert can filter our algorithms. Indeed, since the number of outputted patterns the collection with specific constraints such as a minimum grows exponentially, see Figure 1, the memory usage becomes number of itemsets, or specific items using regular expressions, more and more important. Thus, the quantity of memory etc. Second, the expert can introduce preferences as measures needed by our algorithms should be the as low as possible. In on the patterns (size, length, support, balance, etc.) that he fact, the percentage of used nodes which are created in the tree wishes to minimize or maximize given his goals (the so- structure are sensibly different for the algorithms. Each of the called skypatterns [16]). Indeed, one expert may favor highly proposed algorithms builds a pattern tree in which each node balanced patterns with high support (probably the standard represents a frequent sequential pattern, but not necessarily a strategies), while another could be interested in maximizing frequent balanced pattern from (or ). BALANCESPAN the support while favoring patterns whose balance is closest Ft Fs is the only algorithm that creates a node, and only one, for to 0.5 (giving hints to possible game design problems). Finally, each pattern to be outputted, see Figure 2: in the best cases the discovery can be done interactively, through an interactive PREFIXSPANNAIVE have only the half of useful nodes (by algorithm (not only the full KDD process [17]). The basic definition). This number drops to 10% of useful nodes for very assumption is that the expert does not really know what he low supports on some datasets. For EMERGSPAN, it is worst, is looking for in the data, and guides the pattern discovery at as only the direct predecessors of the leaves of the prefix tree each step of the algorithm (such as sequence expansion in our are balanced patterns by definition. case). In all cases, the pattern language must be clearly defined and efficient algorithms proposed to compute the measure of interest, our main contribution. D. Exploration of the extracted patterns We now provide a basic example of exploration by querying It is interesting to visualize the distribution of both the the pattern set. Out of the 43, 610 patterns for ZvZ with support and the balance of the patterns. Figure 4 gives such =0.001, we can keep the patterns involving two players distribution for dataset ZvZ that enables very fast computations (containing both + and ), which returns 40, 674 patterns. with low (less than 5 seconds for =0.001). There, both a Then we restricted the set of patterns to those involving pattern and its dual are presented, which allows interestingly two specific items (RoachWarren and Spire) to get only + + to observe that the equation y =0.5 (where y is the vertical 290 patterns. SpawPool , SpawnPool ,SpiCrawler , h+ axis) gives almost a symmetry axis. Indeed, both a pattern and RoachW arren ,Spire denotes for example games where i its dual do not necessarily have the same support. One can one of the players go to air units and the second to ground notice that empirically, there are high chances for a pattern units, two known openings, with balance 0.47 and support 68. with high frequency to have a fair balance around 0.5. This behavior applies for the other dataset and is what we could The pattern mining task can be adapted in various ways, expect given the definition of the balance measure. It is possible to query the set of extracted patterns in various ways. Indeed, the pattern mining task is related to the Knowledge Discovery in Databases (KDD) process that aims at extracting knowledge from data [8]. Data mining approaches and more precisely pattern mining approaches are a step of the KDD process that results in patterns from transformed data [9]. Our work is a pattern mining approach to study strategy balance that is applied to RTS games. Thus, one gets a set of patterns that are a generalization of the local strategies within the data. BALANCESPAN still requires an analysis once the patterns are outputted. Fig. 5. Game openings support and balance for the dataset ZvZ 9 depending on which and how the basic actions are encoded in 3) On the diversity in mirror match-ups: It is more delicate the sequence. Let us now sketch different scenarios. to speak about balance when both players belong to the same 1) Discovering strategy openings: Openings are the most faction (mirror match-ups): both players have access to the well-known strategies and executed during the first five to same strategies (same building, hence a symmetrical game). ten minutes of a StarCraft II game. It is expected that Let us observe for example the dataset PvP with a new vocab- (building, sign, ithwindow, jbuilding) openings are balanced to make the game enjoyable for ulary: items are tuples the casual player, competitive for the professional, but also where the last element records how many of this building interesting to watch for the spectators [2]. We build our were already made at the moment of the action (cumulative). =0.05 3418 sequence databases with a set of items composed of tu- Setting a minimal support , we obtain patterns. 4 Gates ples (building, sign, ithwindow), with fixed windows of We can find here the so-called strategy through th the pattern s = (Gateway, +, 3, 1) (Assimilator, +, 3, 1) 30 seconds by default, i.e., the i window contains the h{ } th (Cyb.Core,+, 4, 1) (Gateway, +, 7, 2) (Gateway, +, 7, 3) items performed between the ((i 1) 30) second and the { }{ th ⇥ (Gateway, +, 7, 4) with balance(s)=0.59. Such a high ((i 30) 1) second. We expect that openings are found }i ⇥ as the more frequent patterns and being also balanced: Figure value may be surprising, but it reflects the effectiveness of this 5 shows the complete set of patterns for the ZvZ dataset strategy, and consequently the poor diversity of the strategies which differs with Figure 4 by its skewness. We explored used in the PvP match-up. It is an easy and non-risky strategy the resulting patterns with a game expert. When considering to apply: according to the expert, a player has better chances another dataset (PvZ), we obtain only 591 patterns with = to win with this strategy against riskier strategies. After several 0.05. Top frequent patterns represent all well-known strategies: recurrent complaints, nothing changed in the game until a new s = (Nexus,+, 5) (Gateway, +, 6)(PhotonCannon,+6 major update of the games (with new units and buildings). h{ }{ Since then strategies used in the PvP match up are more ) represents a popular Protoss strategy, no matter the strategy }i 4 gates of the opponent is. It is balanced (balance(s)=0.52). diversified and the strategy is rarely used. 2) Discovering possible balance issues (hypotheses elicita- VIII. RELATED WORK tion): The rules of the game are set by the editors and devel- Discovering patterns that highly distinguish a dataset from opers. However, such rules are not always fair and balanced, others (e.g., win labeled objects versus lose labeled objects) is and such weaknesses can only be discovered after weeks. We an important task in machine learning and data mining [13]. asked an expert to highlight a well known imbalanced strategy. One of the main reason is that such patterns enable the The so called bunker rush was used a lot by Terran players building of comprehensible classifiers [18]. In the general against their Zerg opponents. It consists of building in the settings, we are given a set of objects of different classes early stage of the game a bunker near the opponent’s base to that take descriptions, generally from a partially ordered set put his economy and development in difficulty. After several (itemsets, graphs, intervals, etc.) [19]. The goal is to find complaints from the StarCraft II community, the rules changed good description generalizations that mostly appear in one on 10th May 2012: a Zerg counter unit (the queen) has been class of objects and not in the others. In different fields of slightly improved. Since then, this strategy stopped to be used AI and applied mathematics, such descriptions have different for some time. Our approach should be able to reflect/discover names [13] like version spaces [20], contrast sets [21] and that fact: we proceed as follows. We split dataset TvZ into subgroups discovery [22] in machine learning, emerging pat- two parts: the first one, called before, contains replays that terns [12] in data-mining; or no counter-example hypothesis S happened strictly before 10th May 2012 (17, 171 replays), and in formal concept analysis [19]. Our contribution in this field the second one, called after, contains the replays that hap- is to consider and compute efficiently a balance measure that S pened strictly after this change (6, 698 replays). The mining of existing methods can partially or non efficiently compute. the dataset (respectively ) with a low =0.05 Sbefore Safter StarCraft II and other real time strategy games (RTS) in returns 8, 138 patterns (resp. 7, 735). According to the experts, general, face several research challenges in artificial intelli- the bunker should be built during the sixth window of time gence as deeply discussed in a recent survey [6]. Our work is (between 2’30” and 3’ of the game). There are 20 (resp. 12) related to the challenge that the authors of the survey qualify as patterns that involve the item (Bunker,c,6) with c +, . 2 { } prior learning, that is, techniques that can “exploit available With (resp. ), the average value of the balance Sbefore Safter data such as existing replays [...] for learning appropriate is 0.58 (resp. 0.51) with a standard deviation equals to 0.5 strategies beforehand”. Strategies in RTS game are complex (resp. 1.6). This is clear that since the patch was released, this and divided in several tasks, each bringing difficult problems. strategy has become balanced. Moreover, we can remark that Several case-based reasoning approaches have been proposed, this strategy is no longer used by players: in fact the number mainly to retrieve and adapt strategies (especially build orders) of extracted patterns related to this strategy decreases by 40% to be used then by an automated agent [23], [24]. Other whereas the number of extracted patterns only decreases by kinds of approaches are also used for several prediction tasks. 5% from to . Thus, BALANCESPAN enables to Sbefore Safter Predicting the opponent’s production was studied with answer see the impact of the release of patch, by analyzing the period set programming [25], while learning transition probabilities before and after this key date. within build orders was achieved with hidden Markov models [26]. Weber et al. described any past game by a vector of 10 building and upgrade timings: such features allow an accurate [6] S. Ontan˜on,´ G. Synnaeve, A. Uriarte, F. Richoux, D. Churchill, and strategy prediction [27]. This comes with a limit: game logs are M. Preuss, “A survey of real-time strategy game AI research and competition in starcraft,” IEEE Trans. Comput. Intellig. and AI in a priori labeled with a strategy using rules based on “manual” Games, vol. 5, no. 4, pp. 293–311, 2013. expert analysis. The same applies for opening prediction [28]. [7] O. Missura and T. Gartner,¨ “Predicting dynamic difficulty,” in Advances To discover strategies in large volumes of replays, avoiding in Neural Information Processing Systems 24: 25th Annual Conference to manually label game logs, knowledge discovery in database on Neural Information Processing Systems 2011, 2011, pp. 2007–2015. [8] U. M. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “The KDD process (KDD) methods are required, and especially pattern mining for extracting useful knowledge from volumes of data,” Commun. ACM, techniques. This was highlighted in the open problems cate- vol. 39, no. 11, pp. 27–34, 1996. gory Domain knowledge of the recent survey mentioned before [9] C. C. Aggarwal, Data Mining - The Textbook. Springer, 2015. [10] A. Giacometti, D. H. Li, P. Marcel, and A. Soulet, “20 years of pattern [6]: “Is it possible to devise techniques that can automatically mining: a bibliometric survey,” SIGKDD Explorations, vol. 15, no. 1, mine existing collections [...] and incorporate it into the bot?”. pp. 41–50, 2013. We did not study in this article the second step (incorporation), [11] J. Pei, J. Han, B. Mortazavi-Asl, H. Pinto, Q. Chen, U. Dayal, and M. Hsu, “Prefixspan: Mining sequential patterns by prefix-projected but presented a way to extract efficiently such patterns and growth,” in Proceedings of the 17th International Conference on Data focused onbalance issues for helping game designers. A long Engineering, 2001, pp. 215–224. road remains to be able to select the right patterns to be used [12] G. Dong and J. Li, “Efficient mining of emerging patterns: Discovering by artificial agents as discussed recently in [6], [29]. trends and differences,” in Proceedings of the Fifth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1999, pp. 43–52. IX. CONCLUSION [13] P. K. Novak, N. Lavrac, and G. I. Webb, “Supervised descriptive rule discovery: A unifying survey of contrast set, emerging pattern and We tackled the problem of mining frequent sequential subgroup mining,” Journal of Machine Learning Research, vol. 10, pp. patterns in real time strategy games whose balance measures 377–403, 2009. provide meaningful insights on the strategies played and [14] M. Plantevit and B. Cremilleux,´ “Condensed representation of sequential patterns according to frequency-based measures,” in Advances in Intel- their ability of being in equilibrium or not. For that matter, ligent Data Analysis VIII, 8th International Symposium on Intelligent we revisited the well known notions of discriminant pattern Data Analysis, IDA 2009, 2009, pp. 155–166. mining to provide efficient algorithms for the elicitation of [15] G. Bosc, M. Kaytoue, C. Ra¨ıssi, J. Boulicaut, and P. Tan, “Balancespan,” 2015. [Online]. Available: https://github.com/ balance hypotheses from the data. guillaume-bosc/BalanceSpan From that, we presented several algorithms that enable [16] A. Soulet, C. Ra¨ıssi, M. Plantevit, and B. Cremilleux,´ “Mining dominant (partially or not) dealing with interaction databases, and we patterns in the sky,” in 11th IEEE International Conference on Data Mining, ICDM 2011, Vancouver, BC, Canada, December 11-14, 2011, showed that only BALANCESPAN enables to deal with all 2011, pp. 655–664. datasets efficiently. [17] M. van Leeuwen, “Interactive data exploration using pattern mining,” We empirically validated that the balance measure is able in Interactive Knowledge Discovery and Data Mining in Biomedical Informatics - State-of-the-Art and Future Challenges, 2014, pp. 169– to distinguish balanced and imbalanced strategies. We believe 182. that our approach can become a basic tool for balance design- [18] M. Garc´ıa-Borroto, J. F. M. Trinidad, and J. A. Carrasco-Ochoa, “A ers when analyzing a subset of historical data of a game in beta survey of emerging patterns for supervised classification,” Artif. Intell. phase, or even after its release, through an exploratory process Rev., vol. 42, no. 4, pp. 705–721, 2014. [19] S. O. Kuznetsov, “Galois connections in data analysis: Contributions (KDD and interactive mining). A major difficulty remains to from the soviet era and modern russian research,” in Formal Concept select and construct features of interest from the game logs. Analysis, Foundations and Applications, 2005, pp. 196–225. [20] T. M. Mitchell, Machine learning, ser. McGraw Hill series in computer science. McGraw-Hill, 1997. ACKNOWLEDGMENTS [21] S. D. Bay and M. J. Pazzani, “Detecting group differences: Mining This work was partially supported by the MI CNRS contrast sets,” Data Min. Knowl. Discov., vol. 5, no. 3, pp. 213–246, 2001. Mastodons program that allowed the two last authors to visit [22] F. Herrera, C. J. Carmona, P. Gonzalez,´ and M. J. del Jesus,´ “An the MIT Game Lab in July 2014. overview on subgroup discovery: foundations and applications,” Knowl. Inf. Syst., vol. 29, no. 3, pp. 495–525, 2011. [23] D. W. Aha, M. Molineaux, and M. J. V. Ponsen, “Learning to win: REFERENCES Case-based plan selection in a real-time strategy game,” in Case-Based [1] T. L. Taylor, Raising the Stakes : E-Sports and the Professionalization Reasoning, Research and Development, 6th International Conference, of Computer Gaming. MIT Press, 2012. on Case-Based Reasoning, ICCBR 2005, 2005, pp. 5–20. [2] G. Cheung and J. Huang, “Starcraft from the stands: understanding [24] B. G. Weber and M. Mateas, “Case-based reasoning for build order in the game spectator,” in Proceedings of the International Conference on real-time strategy games,” in Proceedings of the Fifth AAAI Conference Human Factors in Computing Systems, CHI 2011, 2011, pp. 763–772. on Artificial Intelligence and Interactive Digital Entertainment, AIIDE [3] M. Kaytoue, A. Silva, L. Cerf, W. M. Jr., and C. Ra¨ıssi, “Watch me 2009. The AAAI Press, 2009, pp. 106–111. playing, i am a professional: a first study on video game live streaming,” [25] M. Stanescu and M. Certicky, “Predicting opponent’s production in in Proceedings of the 21st World Wide Web Conference, WWW 2012, real-time strategy games with answer set programming,” Computational 2012, pp. 1181–1188. Intelligence and AI in Games, IEEE Transactions on, vol. PP, no. 99, [4] A. Von Eschen, “Machine learning and data mining in call of duty,” in pp. 1–1, 2014. European Conference on Machine Learning and Knowledge Discovery [26] E. W. Dereszynski, J. Hostetler, A. Fern, T. G. Dietterich, T. Hoang, and in Databases (ECML-PKDD), ser. LNCS 8724. Springer, 2014, an M. Udarbe, “Learning probabilistic behavior models in real-time strategy Industrial Invited Talk. games,” in Proceedings of the Seventh AAAI Conference on Artificial [5] M. A. Ahmad, B. Keegan, J. Srivastava, D. Williams, and N. S. Intelligence and Interactive Digital Entertainment, AIIDE 2011. The Contractor, “Mining for gold farmers: Automatic detection of deviant AAAI Press, 2011, pp. 20–25. players in mmogs,” in Proceedings of the 12th IEEE International [27] B. G. Weber and M. Mateas, “A data mining approach to strategy pre- Conference on Computational Science and Engineering, CSE 2009, diction,” in Proceedings of the 2009 IEEE Symposium on Computational 2009, pp. 340–345. Intelligence and Games, CIG 2009, 2009, pp. 140–147. 11 [28] G. Synnaeve and P. Bessiere,` “A bayesian model for opening prediction in RTS games with application to starcraft,” in 2011 IEEE Conference on Computational Intelligence and Games, CIG 2011, 2011, pp. 281–288. [29] M. Leece and A. Jhala, “Sequential pattern mining in starcraft: Brood war for short and long-term goals,” in Proceedings of the Tenth AAAI Conference on Artificial Intelligence and Interactive Digital Entertain- ment, AIIDE 2014. The AAAI Press, 2014, pp. 281–288.