AI Magazine Volume 22 Number 3 (2001) (© AAAI) Articles A Gamut of Games Jonathan Schaeffer I In 1950, Claude Shannon published his seminal make this amazing feat possible. Often over- work on how to program a computer to play chess. looked, however, is that this result was also a Since then, developing game-playing programs testament to human abilities. Considering the that can compete with (and even exceed) the abil- formidable computing power that DEEP BLUE ities of the human world champions has been a used in its 1997 exhibition match against long-sought-after goal of the AI research commu- world chess champion Garry Kasparov nity. In Shannon’s time, it would have seemed (machine: 200,000,000 chess positions a sec- unlikely that only a scant 50 years would be need- ond; man: 2 a second), one can only admire ed to develop programs that play world-class backgammon, checkers, chess, Othello, and Scrab- the human champions for withstanding the ble. These remarkable achievements are the result technological onslaught for so long. of a better understanding of the problems being Computer game research was started by solved, major algorithmic insights, and tremen- some of the luminaries in computing science dous advances in hardware technology. Computer history. In 1950, Claude Shannon published games research is one of the important success sto- his seminal paper that laid out the framework ries of AI. This article reviews the past successes, for building high-performance game-playing current projects, and future research directions for programs (Shannon 1950). In 1951, Alan Tur- AI using computer games as a research test bed. ing (1953) did a hand simulation of his com- puter chess algorithm (a lack of resources pre- vented him from actually programming it); the algorithm lost to a weak human player. ames are ideal domains for exploring Around this time, Arthur Samuel began work the capabilities of computational intel- on his famous checkers-playing program, the Gligence. The rules are fixed, the scope of first program to achieve notable success the problem is constrained, and the interac- against human opposition (Samuel 1967, tions of the players are well defined. Contrast 1959). By 1958, Alan Newell and Herb Simon the game world with the real world—the game had begun their investigations into chess, of life—where the rules often change, the which eventually led to fundamental results scope of the problem is almost limitless, and for AI and cognitive science (Newell, Shaw, and the participants interact in an infinite number Simon 1958). An impressive lineup to say the of ways. Games can be a microcosm of the real least! world (for example, the role of game theory in In the half century since Shannon’s paper, economics, social interaction, and animal enormous progress has been made in con- behavior), and successfully achieving high structing high-performance game-playing pro- computer performance in a nontrivial game grams. In Shannon’s time, it would have can be a stepping stone toward solving more seemed unlikely that within a scant 50 years challenging real-world problems. checkers (8 8 draughts), Othello, and Scrab- Historically, games have been a popular ble programs would exist that exceed the abil- choice for demonstrating new research ideas in ities of the best human players,1,2 and back- AI. Indeed, one of the early goals of AI was to gammon and chess programs could play at a build a program capable of defeating the level comparable to the human world champi- human world chess champion in a match. This on. These remarkable accomplishments are the challenge proved to be more difficult than was result of a better understanding of the anticipated; the AI literature is replete with problems being solved, major algorithmic optimistic predictions. It eventually took insights, and tremendous advances in hard- almost 50 years to complete the task—a ware technology. The work on computer remarkably short time when one considers the games has been one of the most successful and software and hardware advances needed to visible results of AI research. For some games, Copyright © 2001, American Association for Artificial Intelligence. All rights reserved. 0738-4602-2000 / $2.00 FALL 2001 29 Articles one could argue that the Turing test has been these enhancements include iterative deepen- passed (Krol 1999). ing, caching previously seen subtree results This article discusses the progress made in (transposition tables), successor reordering, developing programs for the classic board and search extensions and reductions, probabilistic card games. For a number of games, a short his- cutoffs, and parallel search. The results are tru- tory of the progress in building a world-class ly amazing. Even though there is an exponen- program for the game is given, along with a tial difference between the best case and the brief description of the strongest program. In worst case for an alpha-beta search, most high- each case, a single feature of the program that performance game-playing programs are Considering is a major contributor to the program’s searching within a small constant of the best strength is highlighted. The histories are neces- case (Plaat et al. 1996). the sarily brief. I apologize in advance to the many Sadly, the community of researchers in- formidable hard-working researchers and hobbyists whose volved in this work has done a relatively poor computing work is not mentioned here. job of selling the technology, resulting in many Enabling Technologies briefly summarizes of the ideas being reinvented for other power that some of the major advances in technology that domains. For example, many search tech- DEEP BLUE facilitated the construction of world-champi- niques pioneered with alpha-beta have become onship–caliber programs. Success in Computer standard in other search domains, with few used in its Games reports the past successes where comput- realizing the lineage of the ideas. 1997 ers have met or exceeded the best human play- At the heart of many game-playing programs ers (backgammon, checkers, chess, Othello, and is an evaluation function. Early on, game exhibition Scrabble). Current Research Efforts highlights developers quickly encountered the knowl- match against games of current academic interest (bridge, go, edge-acquisition bottleneck and traded quality world chess and poker). The Future of Computer Games dis- of knowledge for speed of the program. Simple cusses some of the future challenges of using evaluation functions, linear combinations of champion games as a research test bed for AI. easily identifiable features, were the mainstay Garry Although this article emphasizes the AI of computer game programs for many decades. viewpoint, one should not underestimate the Alternative approaches, such as modeling Kasparov engineering effort required to build these pro- human cognitive processes, turned out to be (machine: grams. One need only look at the recent suc- much harder to do than initially expected and cess of the DEEP BLUE chess machine to appreci- generally resulted in poor performance. Game 200,000,000 ate the effort required. This project spanned 8 programmers quickly learned that a little chess years (12 if one includes the pre-IBM time) and heuristic knowledge, when combined with positions a included several full-time people, extensive deep search, can produce amazing perfor- computing resources, chip design, and grand mance results. Indeed, one could argue that the second; man: master consultation. Some of the case studies viability of brute-force search, once a term with 2 a second), hint at the amount of work required to con- negative connotations in the AI community, is struct these systems. In all cases, the successes one of the main research results from games- one can only reported in this article are the result of consis- related research (Ginsberg 1996b). admire the tent progress over many years. In the last decade, new techniques have human moved to the forefront of games research. Two Enabling Technologies in particular are mentioned here because they champions for are likely to play a more prominent role in the withstanding The biggest advances in computer game play- near future: (1) Monte Carlo simulation and (2) ing have come as a result of work done on the temporal-difference learning. the alpha-beta search algorithm. This algorithm Monte Carlo simulation has successfully technological received the most attention because of the been applied to games with imperfect or non- research community’s preoccupation with deterministic information. In these games, it is onslaught for chess. With the DEEP BLUE victory over world too expensive to search all possible outcomes. so long. chess champion Garry Kasparov, interest in Instead, only a representative sample is chosen methods suitable for chess has waned and been to give a statistical profile of the outcome. This replaced by activity in other games. One could technique has been successful in bridge, poker, argue that the chess victory removed a ball and and Scrabble. shackle that was stifling the creativity of Temporal-difference learning is the direct researchers who were building high-perfor- descendent of Samuel’s machine learning mance game-playing systems. research (Sutton 1988). Here, a database of The alpha-beta research led to a plethora of games (possibly generated by computer self- search enhancements, which significantly play) can be used to bootstrap a program to improved the efficiency of the search. Some of find a good combination of knowledge fea- 30 AI MAGAZINE Articles tures. The algorithm has successfully been ences” (Berliner 1980a, p. 215). Thus, assessing applied to backgammon and has recently the results of a five-game match is difficult. shown promise in chess and checkers (Schaef- Afterwards, Berliner (1980b, p. 71) analyzed fer, Hlynka, and Jussila 2001). the program’s play and concluded that The most obvious hardware advance is sim- There was no doubt that BKG9.8 played ply speed.