Machine Learning, Game Play, and Go
Total Page:16
File Type:pdf, Size:1020Kb
Machine Learning, Game Play, and Go David Stoutamire Abstract The game of go is an ideal problem domain for exploring machine learning: it is easy to define and there are many human experts, yet existing programs have failed to emulate their level of play to date. Existing literature on go playing programs and applications of machine learning to games are surveyed. An error function based on a database of master games is defined which is used to formulate the learning of go as an optimization problem. A classification technique called pattern preference is presented which is able to automatically derive patterns representative of good moves; a hashing technique allows pattern preference to run efficiently on conventional hardware with graceful degradation as memory size decreases. 1 Contents 1 Machine Learning 6 1.1 What is machine learning? X X XXX XXX XX XXX XXX XXX XXX XXX X 6 1.2 Learning as optimization X XX XXX XXX XX XXX XXX XXX XXX XXX X 7 1.3 The Bias of Generalization XX XXX XXX XX XXX XXX XXX XXX XXX X 10 1.4 Two graphical examples X XXX XXX XXX XX XXX XXX XXX XXX XXX X 11 2 Computer Gamesmanship and Go 14 2.1 Computers playing games X XX XXX XXX XX XXX XXX XXX XXX XXX X 14 2.1.1 Simple state games X X XXX XXX XX XXX XXX XXX XXX XXX X 14 2.1.2 Computer chess X XXX XXX XXX XX XXX XXX XXX XXX XXX X 17 2.2 The game of go X X XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 18 2.3 Go programs X XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 21 2.3.1 David Fotland X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 22 2.3.2 Ken Chen X XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 29 2.3.3 Bruce Wilcox XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 30 2.3.4 Kiyoshi Shiryanagi X X XXX XXX XX XXX XXX XXX XXX XXX X 30 2.3.5 Elwyn Berlekamp X XX XXX XXX XX XXX XXX XXX XXX XXX X 30 2.3.6 Dragon II X XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 31 2.3.7 Swiss Explorer X XXX XXX XXX XX XXX XXX XXX XXX XXX X 31 2.3.8 Star of Poland XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 32 2.3.9 Goliath X XX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 32 2.3.10 Observations X XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 33 3 Machines Learning Games 34 3.1 Samuel’s checkers player XXX XXX XXX XX XXX XXX XXX XXX XXX X 34 3.1.1 Learning by rote X XXX XXX XXX XX XXX XXX XXX XXX XXX X 34 3.1.2 Learning linear combinations X XX XX XXX XXX XXX XXX XXX X 34 3.1.3 Learning by signature tables X XX XX XXX XXX XXX XXX XXX X 35 3.2 Tesauro’s backgammon player X XXX XXX XX XXX XXX XXX XXX XXX X 37 3.2.1 Backgammon X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 37 3.2.2 Neural Nets XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 37 3.2.3 Features XX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 38 3.3 Generalities X XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 38 3.3.1 Signature tables vs. Neural Nets X XX XXX XXX XXX XXX XXX X 38 2 3.3.2 Importance of representation X XX XX XXX XXX XXX XXX XXX X 39 4 A Go Metric 40 4.1 Defining an error function X XX XXX XXX XX XXX XXX XXX XXX XXX X 40 4.1.1 Expert game data X XX XXX XXX XX XXX XXX XXX XXX XXX X 40 4.1.2 Simple rank statistic X X XXX XXX XX XXX XXX XXX XXX XXX X 41 4.1.3 Normalized rank statistic X X XXX XX XXX XXX XXX XXX XXX X 42 4.2 Plotting X X XX XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 43 4.3 Evaluation of performance X XX XXX XXX XX XXX XXX XXX XXX XXX X 45 5 Pattern Preference 46 5.1 Optimization X XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 46 5.2 Simple methods of categorization X X XXX XX XXX XXX XXX XXX XXX X 48 5.3 Hashing X XX XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 55 5.4 Pattern cache XXXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 58 5.5 Improvements X XX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 59 6 Example Game 61 6.1 Evaluation on a master game X XXX XXX XX XXX XXX XXX XXX XXX X 61 6.2 Discussion X X XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 71 7 Conclusion 75 7.1 Summary X XX XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 75 7.2 Future work X XXX XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 76 A Resources 78 B Methods 80 C Game record 81 C.1 Moves from section 6.1 X XXX XXX XXX XX XXX XXX XXX XXX XXX X 81 D The Code 85 3 List of Figures 1.1 Simple two dimensional error function X X XX XXX XXX XXX XXX XXX X 8 1.2 Complex two dimensional error function X XX XXX XXX XXX XXX XXX X 8 1.3 Effect of the bias of generalization on learning X XX XXX XXX XXX XXX X 12 1.4 Simple function to extrapolate X XXX XXX XX XXX XXX XXX XXX XXX X 13 2.1 Part of the game graph for tic-tac-toe. XXXX XXX XXX XXX XXX XXX X 15 2.2 Examples used in the text X XX XXX XXX XX XXX XXX XXX XXX XXX X 19 2.3 Important data structures in Cosmos X XX XX XXX XXX XXX XXX XXX X 23 2.4 Group classification used by Cosmos X XX XX XXX XXX XXX XXX XXX X 25 2.5 Move suggestion in Cosmos X X XXX XXX XX XXX XXX XXX XXX XXX X 26 2.6 Sample move tree from pattern X XX XXX XX XXX XXX XXX XXX XXX X 28 3.1 Samuel’s signature table hierarchy X XXX XX XXX XXX XXX XXX XXX X 36 4.1 A sample NRM plot X XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 43 4.2 Random NRM plot X XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 44 4.3 Example study plot X XX XXX XXX XXX XX XXX XXX XXX XXX XXX X 44 5.1 Windows used for pattern extraction X XX XX XXX XXX XXX XXX XXX X 50 5.2 NRM plots for 3x3 and radius 1 diamond windows X XX XXX XXX XXX X 51 5.3 Comparison of NRM plots for diamond window X X XXX XXX XXX XXX X 51 5.4 Study graph for plots in previous figure X X XX XXX XXX XXX XXX XXX X 52 5.5 Comparison of NRM plots for square window X XX XXX XXX XXX XXX X 52 5.6 Study plot of previous figure X XXX XXX XX XXX XXX XXX XXX XXX X 53 5.7 Comparison of NRM plots for square window X XX XXX XXX XXX XXX X 53 5.8 Study plot for graph-based window X XXX XX XXX XXX XXX XXX XXX X 54 5.9 Comparison of hash and map stategies X X XX XXX XXX XXX XXX XXX X 56 5.10 Effect of collisions X XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 57 5.11 Study plots with liberty encoding X X XXX XX XXX XXX XXX XXX XXX X 60 5.12 NRM plot for best classification X XX XXX XX XXX XXX XXX XXX XXX X 60 6.1 Move 2, master game X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 62 6.2 Move 5, master game X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 63 6.3 Move 6, master game X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 63 6.4 Move 9, master game X X XXX XXX XXX XX XXX XXX XXX XXX XXX X 64 6.5 Move 11, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 65 4 6.6 Move 15, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 65 6.7 Move 20, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 66 6.8 Move 21, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 67 6.9 Move 22, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 67 6.10 Move 24, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 68 6.11 Move 28, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 68 6.12 Move 43, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 69 6.13 Move 47, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 70 6.14 Move 79, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 70 6.15 Move 92, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 71 6.16 Move 138, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 72 6.17 Move 145, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 72 6.18 Move 230, master game X XXX XXX XXX XX XXX XXX XXX XXX XXX X 73 D.1 Classes used in iku XXX XXX XXX XXX XX XXX XXX XXX XXX XXX X 86 5 Chapter 1 Machine Learning 1.1 What is machine learning? ML is an only slightly less nebulous idea than Artificial Intelligence (AI).