ARTIFICIAL INTELLIGENCE 77 Game Tree Searching by Min / Max Approximation* Ronald L. Rivest Laboratory for Computer Science, MIT, Cambridge, MA 02139, U.S.A. Recommended by Hans Berliner ABSTRACT We present an iterative method for searching minlmax game trees based on the idea of approximating the “min” and “max” operators by generalized mean-valued operators. This approximation is useci to guide the selection of the next leaf node to expand, since the approximations allow one to select ejficiently that leaf node upon whose value the (approximate) value at the root most highly depends. Experimental results from almost 1,000 games of Connect-Four’ suggest that our scheme is superior to minimax search with alpha-beta pruning, for the same number of calls to the move routine. However, our scheme has higher overhead, so that further work is needed before it becomes competitive when CPU time per turn is the limiting resource. 1. Introduction This paper introduces a new technique for searching in game trees, based on the idea of approximating the min and max operators with generalized mean-value operators. Game playing by computer has a long history, and many brilliant ideas have led us to the point where high-quality play for many games can be obtained with pocket-sized computers. (See [3] for an exposition of previous work in this area, and Pearl’s book [13] for an excellent introduction to the mathematical analysis of game-playing programs.) However, further improvement is certainly possible, and this area of re- search is still an active one. The combinatorial explosion of possibilities in a game such as chess tax our most powerful computers, and even special-purpose hardware soon reaches its limits. Clearly, the most careful organization and allocation of computational resources is needed to obtain expert-level play. ” This research was supported by NSF grants DCR-8006938 and DCR-8607494. ’ Connect-Four is a trademark of the Milton-Bradley company. Artificial Intelligence 34 (1988) 77-96 0004-3702/B/$3.50 0 1988, Elsevier Science Publishers B.V. (North-Holland) 78 R.L. RIVEST Techniques such as alpha-beta pruning and its successors [6, 2] have been essential in reducing the computational burden of exploring a game tree. Still, new techniques are needed. Nau et al. [10], after much expermentation with existing methods, assert that "A method is needed which will always expand the node that is expected to have the largest effect on the value." This paper suggests such a method. Our method, "min/max approximation," attempts to focus the computer's attention on the important lines of play. The key idea is to approximate the "min" and "max" operators with generalized mean-value operators. These are good approximations to the rain/max operators, but have continuous deriva- tives with respect to all arguments. This allows us to define the "expandable tip upon whose value the backed-up value at the root most heavily depends" in a nontrivial manner. This tip is the next one to be expanded, using our heuristic. In Section 2 of this paper I present the essential results about generalized mean values that underly the new method. Then, in Section 3, these ideas are applied to the problem of searching game trees. In Section 4, I give some thoughts regarding implementation details. Section 5 describes our preliminary experimental results. Some final thoughts are presented in Section 6 . 2. Generalized Mean Values Let a = (a 1.... , an) be a vector of n positive real numbers, and let p be a nonzero real number. Then we can define the generalized p-mean of a, Mp(a), by Mp(a)= n aPi (1) i=l Of course, M l(a) is the ordinary arithmetic mean. We can extend our notation to the case p =0 by io(a ) = lim ip(a) = (al. a,) 1/" , (2) p~O so that Mo(a ) is the geometric mean. We begin with the fact that p < q ~ Mp(a) <~ Mq(a), (3) where equality only holds on the right if all the a i are equal. (See [5] for proofs and other facts about generalized mean values.) For our purposes, we are most interested in the following two facts: lira Mp(a) = max(a I ..... a,,), (4) lim Mp(a) = min(a I ..... a,,). (5) GAME TREE SEARCHING 79 TABLE l. Mp(a) for a = (10, 21, 29, 32) p -32 -16 -8 -4 -2 -1 0 1 2 4 8 16 32 Mp(a) 10.4 10.9 11.9 13.9 16.6 18.7 21.0 23.0 24.5 26.5 28.3 29.7 30.7 To illustrate the above facts, consider Table 1 where various values of Mp(a) are given for a = (10, 21, 29, 29, 32) and various p. For large positive or negative values of p, Mp(a) is a good approximation to max i (ai) or min i (ai), respectively. However, Mp(a), unlike max or min, has continuous derivatives with respect to each variable a i. The partial derivative of Mp(a) with respect to ai is OMp(a) 1 ( a i ~p-1 Oa~-~ - n \ Mp(a) / " (6) The major reason that the generalized means are of interest to us here is that they are more suitable for a "sensitivity analysis" than the min or max functions. We propose that OMp(a)/Oa i (for large p) is a more useful quantity than Omax(a)/Oa i, since the latter is zero unless a i is the maximum, in which case it is one. This discontinuous behavior is awkward to work with, whereas the derivative (6) is continuous. We also note that the derivative (6) ranges in value from 0to n l/p~l forp>>lnn). By way of example, with a = (a l, a2, a3, a4) = (10, 21, 29, 32), and p - 32, we have VMp(a)~(2 x 10 16, 2 x 10 6, 0.04, 0.90). The a i values near the maximum have much more effect on Mp(a) here than do smaller values. Although we shall not use them in this paper, other forms of generalized mean values exist. For example, if f is any continuous monotone increasing function (such as the exponential function), we can consider mean values of the form f 1 (1 ~ f(ai)) " (7) i=1 Using f--exp(. ) would yield a good approximation to max ( • ), and f = In(-) would yield a good approximation to min(.). One of the ideas of this paper is that by using the generalized mean values to approximate the min and max functions, we can identify in an interesting way that leaf in a game tree upon whose value the value at the root depends most strongly. This is done by taking derivatives of the generalized mean value functions at each node and using the chain rule. This leaf will be the one to expand next. These ideas will be made more precise in the next sections. 80 R.L. RIVEST 3. Game Tree Searching 3.1. Game trees Consider a two-person zero-sum perfect information game between players Min and Max which begins in a starting configuration s with Max to move, after which they alternate turns. The game defines a finite tree C of configura- tions with root s. We split C into subsets Min and Max depending on whose turn it is to play. For each c E C we let S(c) denote the set of c's successors (or children). To move from configuration c a player selects some d C S(c); his opponent must then move from configuration d. Configurations with no successors are called terminal configurations; T(C) will denote the set of terminal configurations--these form the leaves of the tree. The game stops when a terminal configuration is reached. We assume that the relation S is acyclic, so the game always stops. The actual play traces out a path in the tree from the root s to leaf representing a terminal configuration t. Each leaf tE T(C) has an associated value or score v(t); this is the value of the terminal position from Max's point of view. By induction on the structure of the tree we may determine the value v(c) of any configuration c E C by "backing up" or "minimaxing" the values at the leaves: v(c), if c E T(C) , v(c)- ~I max v(d), if c E Maxk T(C) , -- } dcS(c) (8) ! min v(d), if c E MinX T( C) . I deS(c} FIG. 1. A small game tree. GAME TREE SEARCHING 81 Given v(c) for all c E C, optimal moves are easy to determine: if c E Max, then player Max should select any configuration d E S(c) with maximum value v(d). Similarly, if c E Min, then player Min should select any configuration dES(c) with minimum value v(d). Figure 1 shows a small game tree. Configurations in Max are shown as squares, those in Min are shown as circles. The value of each configuration is shown inside the square or circle. The value of the game is 16; optimal play is indicated with the heavy arrows. 3.2. Searching a game tree When C is small, the tree can be explored completely, so optimal play is possible. On slightly larger trees minimax search with alpha-beta pruning [6] may produce optimal play even though only a small fraction of the game tree is explored--the portions of the tree that are "pruned" (not explored) are known not to be relevant.