Scalable Search in Computer Chess Computational Intelligence Edited by Wolfgang Hillel and Rudolf Kruse

Ernst A. Heinz

Scalable Search in Computer Chess Computational Intelligence edited by Wolfgang Hillel and Rudolf Kruse

The books In .this series conthDute to the long-" r~nge goal pfumlerstanrung and rea li zing intelligent behaviour in some envI ronment. 'Thus t!;ley' covertopics from the disCiplines ofArtificial1ntetl,genceandCogni!:tve Science, combined also caJled In- ' tellectics, as well as from fi erds interdJsciplinarilyieJ:at'ed with these. Computati(}' nallnteUigence cOmprises ba ~ic k~Owl edge as'wellas applications. Das rechnende Gehlrn by Patricia S. Church land 'lnd Terrence:r. Sejnows~ i Neuronale Netze und Fuzzy..systeme by DetlefNauck,Frank Klawonn and Rudolf Kruse Fuzzy-Clusteranalyse by Frank Hoppner,. Frank KlaW\Jnn.and Rudol f Kruse Elnfiihrung In Evolutloniire,Algorlthmen by Volker Nissen Neuronale Netze by Andreas Scher.er Sehen und die Verarbeltung~vl.ueller Informatlonen by Hanspeter A. Mallo! Betriebswlrtscbaftllche.Anwendungen des Soft Computing by Biethabn et al. (Ed.) Fuzzy Theorle und Stochastlk by Rudolf Seising (Ed.) MultiobJecflve Heuristic Search by Pa llab Dasgupta, P. P. Chakrabarti and S. C. DeSarkar The Efflclence ofTheorem Proving Strategies by David A. Plaisted an~ Yunshan Zhu Scalable Search in Computer Chess by Ernst A. He inz Among others the followi ng books were published in the series of ArtiRciallntelligence Automated Theorem Proving by Wolfgang Bi bel (out of print) Fuzzy Sets and Fuzzy Logic Foundation of Application - from a Mathematical Point of View by Siegfried Gottwald . . Fuzzy Systems In Computer Science edited by Rudolf Kruse, Jorg G'ebhard arid {{ainerPalm Automatlsche Spracherkennung by Ernst Gunter Sthukat-Talamazzinl ; Deduktive Datenbanken by Armin B. Cremers, UJrike Grielhahn and RaJf Hinze

Wissensreprasentation und Inferenz l by Wolfgang Bibel, Steffen Holldobler and Torsten ~Chaub . Ernst A. Heinz

Scalable Search in Computer Chess

Algorithmic Enhancements and Experiments at High Search Dephts

IIvleweg Die Deutsche Bibliothek - CIP-Cataloguing-in-Publication-Data A catalogue record for this publication is available from Die Deutsche Bibliothek (http://www.ddb.de )

ISBN 978-3-528-05732-9 ISBN 978-3-322-90178-1 (eBook) DOI 10.1007/978-3-322-90 I 78- I Authors adress: [email protected]

1sl Edition 2000

No part of this publication may be reproduced, stored in a retrieval system or transmitted, mechanical, photocopying or otherwise without prior permission of the copyright holder.

Printed on aCid-free paper v

Preface

This book presents the results of our past two-and-a-half years of research aimed at increasing the scalability and performance of game-tree search in computer chess. We elaborate on our respective works in the areas of (I) selective forward pruning, (II) the efficient application of game-theoretical knowledge, and (III) the behaviour of the search at increasing depths. The broad range of topics covered by the three distinct parts of the book seek to provide interesting material for everybody interested in the field of "Compu tational Intelligence", regardless of their individual focus (researcher, student, or other). The text does not require readers to know about chess and computer game-playing beforehand. The initial chapter entitled "Computer-Chess Primer" introduces all the necessary basics and fundamentals thereof. The remaining chapters, however, go far beyond those topics. They show how to make sophisticated game-tree searchers still more scalable at ever higher depths. Throughout the whole book, our high-speed and master-strength chess program DARKTHOUGHT serves as a realistic test vehicle to conduct numerous experiments at unprecedented search depths. The extensive experimental evalu ations provide convincing empirical evidence for the practical usefulness of the techniques presented by us. These results will certainly be of special interest to researchers and programmers of computer strategy-games alike (chess, checkers, Go, and Othello in particular). Last but not least, I like to mention that I am most grateful to the series editors for offering me the opportunity to publish my book under their auspices.

Ernst A. Heinz - September 1999 VII

Thanks

• Prof. Dr. Walter F. Tichy remained committed to serve as my supervisor and provided continued support even after I switched topics from parallel com puting to computer chess in late 1996. From him I learned how to contribute to science and how to embrace the scientific community.

• Prof. Dr. T. Anthony Marsland agreed to serve as my co-supervisor although he hardly knew me at the time of my request. In the end, he even shortened his personal holidays for the sake of letting me schedule my thesis defence smoothly.

• Prof. Dr. Jaap van den Herik superbly edited my articles in the ICCA Jour nal. Our most pleasant and constructive cooperation markedly improved the quality of my writing.

• Markus M. Gille dedicated much of his spare time to improve the implemen tation of our chess program DARKTHOUGHT.

• Pawel Lukowicz and I first met in October 1987, shortly before our first term in Computer Science began at the University of Karlsruhe. We then lived together in the same flat for 6 years, earned our M.Sc. degrees together in early 1992, worked together in the same bureau for 7 years, and finally finished our Ph.D. studies together in mid-1999. We always had fun together and our fruitful discussions mostly clarified topics to the very last detail.

• My parents Jutta Heinz and Prof. Dr. Ernst Heinz shielded me against hardship throughout my whole life.

• Marianne J. S. Hardt makes it all worthwhile. She cheered me up when I was down although she had to endure my countless days of late-night work on computer chess during the past years. VIn

Acknowledgements

• A.s.E. GmbH loaned additional RAM for free during the 13th World Micro computer-Chess Championship (October 1995) and a 433 MHz DEC Alpha- 21164a PC-l64 workstation for free during the 14th World Microcomputer Chess Championship (October 1996).

• DELTA Computer Products GmbH loaned a 500 MHz Compaq Alpha-21264 XP-lOOO workstation with 1 GB SDRAM for free during the 9th World Computer-Chess Championship (June 1999) and the 16th World Micro computer-Chess Championship (June 1999) thanks to the courtesy of Heinz Dieter Schutt, then Chief Marketing Officer of DELTA.

• Digital Equipment Corp. (DEC) paid for two 500 MHz DEC Alpha-21164a PC-l64 workstations in March 1997 under external technology research con tract DE-0032-97 entitled "Optimization and Fine-Tuning of a Sophisticated Tree-Searching Application for 64-bit Alpha CPUs" as negotiated by the courtesy of Dr. Frank Severin, then Head of Digital's External Technology Group Europe.

• Digital Equipment Corp. 's Workstation Division (in close collaboration with KryoTech Inc., see below) shipped a cooled 767 MHz Alpha-21164a Digital Personal Workstation prototype with 512 MB SDRAM from the USA to France and back for use by DARKTHOUGHT during the 15th World Micro computer-Chess Championship (October 1997). Together with the prototype they sent Jeff Forsythe over to Paris to attend the week-long championship where he provided superb service to anybody who needed help. Furthermore, Digital Equipment Corp. shipped an identically equipped 600 MHz Alpha- 21164a workstation from the USA to Germany for the final preparations of our team. Digital Equipment Corp. did all of the above for free thanks to the courtesy of Ron Locklin, then Vice President of Digital's World-Wide Workstation Marketing Division.

• KryoTech Inc. (in close collaboration with Digital Equipment Corp., see above) provided a cooled 767 MHz Alpha-21164a Digital Personal Worksta tion prototype for use by DARKTHOUGHT during the 15th World Micro computer-Chess Championship (October 1997) thanks to the courtesy and greatly appreciated personal efforts of Rob diGiacomo, then Executive Vice President and Chief Marketing Officer of KryoTech Inc. Acknow ledgements IX

• Sun Microsystems Inc. donated a 167 MHz UltraSparc Creator workstation for program development and 64-bit Sparc V9 benchmarking in February 1996 due to the courtesy of Eckhard Schaumann, then Head of Sun's German University Cooperation Unit.

• The Gunther & Ingrid Schroff Foundation (administered by the Chancellor of the University of Karlsruhe), the Karlsruher Hochschulvereinigung, and the Institute for Program Structures and Data Organization (IPD, Chair Prof. Tichy) awarded several generous travel grants on the occasions of tour nament participations from the 8th World Computer-Chess Championship (May 1995) onwards.

• The Institute for Program Structures and Data Organization (IPD, Chair Prof. Tichy) at the University of Karlsruhe provided abundant computing resources for our exclusive use throughout the whole work. Without these plentiful resources our extensive experiments with DARKTHOUGHT would have been impossible to undertake.

• The International Computer-Chess Association (ICCA) paid for our air fares from Germany to Indonesia on the occasion of the 14th World Micro computer-Chess Championship (October 1996).

• The editorial staff of the ICCA Journal provided outstanding cooperation and superb editing of my published material.

• Peter W. Gillgasch, a former co-founder of IPD's computer chess group, belonged to the DARKTHOUGHT team until April 1996. His various con tributions are highly appreciated until today. As described in Section A.2 "Implementation History", DARKTHOUGHT still contains some source code written by Peter.

• Andreas Junghanns from the University of Alberta (Canada) provided the detailed self-play results of the game-playing programs CHINOOK, KEYANO, PHOENIX, and THE TURK in electronic form. XI

Contents

Summary and Contributions 1

0 Computer-Chess Primer 7 0.1 The Game of Chess. . · . 7 0.2 Basic Search Techniques · . 11 0.2.1 Minimax and N egamax 11 0.2.2 Alpha-Beta · .. 13 0.2.3 Minimal-Window Search . 16 0.2.4 Quiescence Search 17 0.3 Advanced Search Techniques 20 0.3.1 Search Extensions 20 0.3.2 Transposition Tables 22 0.3.3 Move Ordering 22 0.3.4 Iterative Deepening 23 0.3.5 Aspiration Search 24 0.3.6 Forward Pruning . 24

I Forward Pruning without Tears

1 Adaptive Null-Move Pruning 29 1.1 Introduction. · . · . 29 1.2 Related Work . · . · . 31 1.3 Standard Null-Move Pruning 32 1.4 Recursively Adaptive Null-Move Pruning 34 1.4.1 Theory 35 1.4.2 Practice · . 36 1.5 Conclusion · . · . · . 39 1.6 Appendix - Experimental Setup 40

2 Extended Futility Pruning 41 2.1 Introduction. . . · . 41 2.2 Normal Futility Pruning 42 2.2.1 Theory · . · . 42 2.2.2 Practice · . · . 44 2.3 Futility Pruning at Pre-Frontier Nodes 44 2.3.1 Theory 44 2.3.2 Practice 45 XII Contents

2.4 Limited Razoring at Pre-Pre-Frontier Nodes. 47 2.4.1 Theory 47 2.4.2 Practice...... 49 2.5 Conclusion ...... 50 2.6 Appendix - Experimental Setup 50

3 AEL Pruning 53 3.1 Introduction...... 53 3.2 Combined AEL Pruning 54 3.2.1 Theory 54 3.2.2 Practice. 56 3.3 Test Games . . . 56 3.3.1 Self-Play. 57 3.3.2 Nunn Matches 58 3.4 Conclusion ...... 60 3.5 Appendix - Experimental Setup 60

II Integration of Perfect Knowledge

4 Efficient Interior-Node Recognition 65 4.1 Introduction...... 65 4.2 Fundamentals of Interior-Node Recognition 66 4.3 Recognizers and Transposition Tables 68 4.3.1 Recognizer Results .... . 68 4.3.2 Recognizer Scores ...... 69 4.4 Efficient Recognizer Detection and Selection . 71 4.4.1 Material Signatures .... 72 4.4.2 Further Empirical Refinements 73 4.5 Recognizer Functions . . 76 4.5.1 Implementation Example 78 4.6 Discussion and Conclusion. 78

5 Index Schemes of Endgame Databases 83 5.1 Introduction...... 83 5.2 Related Work ...... 85 5.3 Indexing Endgame Databases without Pawns 86 5.4 Indexing Endgame Databases with Pawns 88 5.4.1 The Two Kings. 88 5.4.2 Directly Rammed Pawns. . . 89 5.4.3 En-Passant Captures. .. 89 5.5 Further General Indexing Improvements 90 5.5.1 Equal Locations 91 5.5.2 Equal Pieces .. 92 XIII

5.5.3 Equal Material ...... 94 5.6 Discussion and Conclusion ...... 95 5.7 Appendix - Thompson's Endgame Databases 97 5.8 Appendix - Edwards' Tablebases 98 5.9 Appendix - Nalimov's Tablebases ...... 98

6 Knowledgeable Endgame Databases 99 6.1 Introduction ...... 99 6.2 Knowledgeable Encoding. 101 6.3 Knowledgeable Probing . 105 6.4 Knowledgeable Scoring .. 106 6.5 Knowledgeable Querying. 110 6.6 Knowledgeable Databases in Practice. 112 6.7 Related Work ...... 114 6.7.1 Infallible Rule-Based Endgame Play in Chess · 117 6.8 Discussion and Conclusion...... · 119

III Search Behaviour at Increasing Depths

7 DarkThought Goes Deep 123 7.1 Introduction...... 123 7.2 Search Depth vs. Strength of Chess Programs . 125 7.3 Newborn's Original Hypothesis Revisited . 128 7.4 Corrected Test Positions ...... 129 7.5 Experimental Results...... 130 7.5.1 "Best Change" Rates for All Test Positions . 131 7.5.2 Experimental Results for All Test Positions 133 7.5.3 Experimental Results for the Opening Positions. . 135 7.5.4 Experimental Results for the Middlegame Positions . 136 7.5.5 Experimental Results for the Remaining Positions . 138 7.6 Conclusion ...... 140 7.7 Appendix - Experimental Setup...... 141 7.8 Appendix - Bounds on the "Best Change" Probabilities 141 7.9 Appendix - Published Results of Crafty 1997 ...... 142

8 Modeling the "Go Deep" Behaviour 145 8.1 Introduction...... 145 8.2 General Considerations...... 147 8.3 Modeling the Behaviour of Crafty. 148 8.3.1 Exponential Model ..... 148 8.3.2 Piece-Wise Linear Model . 149 8.3.3 Piece-Wise Constant / Linear Model 149 8.3.4 Comparative Evaluation of the Models . · 150 XIV Contents

8.4 Modeling the Behaviour of DarkThought . · 150 8.4.1 Exponential Model ...... · 151 8.4.2 Piece-Wise Linear Models . . . . . · 152 8.4.3 Piece-Wise Constant I Linear Models . · 153 8.4.4 Comparative Evaluation of the Models . · 154 8.5 Discussion and Conclusion ...... · 155

9 Self-Play Experiments Revisited 157 9.1 Introduction ...... · 157 9.2 Statistical Analysis of Self-Play Experiments · 158 9.3 Self-Play Experiments in Computer Chess .. · 162 9.3.1 1982 - Belle (Thompson) ...... · 162 9.3.2 1983 - Belle (Condon and Thompson) · 163 9.3.3 1988 - TechMate (Szabo and Szabo) . · 164 9.3.4 1990 - Hitech and Lotech (Berliner et al.) · 166 9.3.5 1994 - Zugzwang (Mysliwietz) .. . · 171 9.3.6 1996 - Phoenix (Schaeffer) ...... · 172 9.3.7 1997 - The Turk (Junghanns et al.) . · 173 9.4 Self-Play Experiments in Computer Checkers · 174 9.4.1 1993 - Chinook (Schaeffer et al.) .. · 174 9.4.2 1995 - Chinook (Schaeffer et al.) .. · 174 9.5 Self-Play Experiments in Computer Othello · 175 9.5.1 1990 - Bill (Lee and Mahajan) · 175 9.5.2 1997 - Keyano (Brockington) · 176 9.6 Conclusion ...... · 178

Perspectives on Future Work 181

IV Appendices

A How DarkThought Plays Chess 185 A.1 Introduction ...... · 185 A.2 Implementation History . . . . · 186 A.3 Bitboard Engine ...... · 187 A.3.1 Bitboard Infrastructure · 187 A.3.2 Rotated Bitboards · 189 A.4 Search Engine ...... · 191 A.4.1 Node Expansion .. · 192 A.4.2 Extension Heuristics · 193 A.4.3 Search Parameterization . · 194 A.5 Evaluation Engine ...... · 195 A.5.1 Programmable Evaluation Function · 195 A.5.2 Evaluation Machines ...... · 197 xv

A.6 Future Work ...... 198

B Tournament History of DarkThought 199 B.1 World Championships ...... 199 B.2 AEGON Man vs. Machine Tournaments . 200 B.3 Public Exhibition Matches...... 200

C DarkThought and Test Suites 201 C.1 Solution Times for BS-2830 . 201 C.2 Solution Times for BT-2630 . 201 C.3 Solution Times for LCT-II . . 201 C.4 Measured Peak Speed . 202 C.5 Test Configuration ...... 202

D DarkThought at Test Games 203 D.1 Test Games vs. Strong PC Chess Programs ...... 203 D.l.1 Games Played from Nunn Position #2 (ECO B89) . 205 D.1.2 Games Played from Nunn Position #3 (ECO C19) . 210 D.1.3 Games Played from Nunn Position #4 (ECO C97) . 214 D.1.4 Games Played from Nunn Position #5 (ECO D36) . 219 D.l.5 Games Played from Nunn Position #7 (ECO E15) . 224 D.1.6 Games Played from Nunn Position #8 (ECO E98) . 229 D.l.7 Games Played from Nunn Position #9 (ECO A25) . 234 D.2 Selected Self-Play Games ...... 238

Bibliography 243

Index 259 XVI

List of Figures

0.1 Empty Chess Board ...... 7 0.2 Initial Starting Position . . . . . 8 0.3 Unconstrained Negamax Search. 12 0.4 N ode-Centric View of Alpha-Beta Search 13 0.5 Depth-Limited Alpha-Beta Search 15 0.6 Principal-Variation Search ...... 17 0.7 Capture-Only Quiescence Search . . . . . 18 0.8 Search with Probing of Transposition Tables. 21 0.9 Selective Search with Standard Null-Move Pruning 26

1.1 Selective Search with Adaptive Null-Move Pruning 37

2.1 Selective Search with Extd. Futil. Pruning and Limited Razoring 48

3.1 Selective Search with Combined AEL Pruning . 55

4.1 Slate's Interior-Node Score Bounds ...... 67 4.2 Search with Result Handling of Successful Recognitions 70 4.3 Search with Detection and Selection of Recognizers . . . 75 4.4 Initialization of Lookup Tables for Flags and Recognizers 77 4.5 Recognizer Function for KB[B]K Endgame Positions ... 79 4.6 Position #23 of Test Suite BT-2630 (Solution Move "g4") 80

6.1 Knowledgeable Query Function for KXKY Endgame Databases · 111 6.2 Endgame Test Position #1 (Solution Move "g4") . · 113 6.3 Endgame Test Position #2 (Solution Move "Bd3") · 114 6.4 Endgame Test Position #3 (Solution Move "Na5") · 115 6.5 Endgame Test Position #4 (Solution Move "h4") . · 116 6.6 Endgame Test Position #5 (Solution Move "Kc5") · 117

8.1 Overall "Best Change" Rates of CRAFTY and DARKTHOUGHT · 146 8.2 Exponential Fit for the "Best Change" Rates of CRAFTY ... · 148 8.3 Linear Fits for the "Best Change" Rates of CRAFTY ...... · 149 8.4 Exponential Fit for the "Best Change" Rates of DARKTHOUGHT · 151 8.5 Linear Fits for the "Best Change" Rates of DARKTHOUGHT (I) . · 152 8.6 Linear Fits for the "Best Change" Rates of DARKTHOUGHT (II) · 153 8.7 Linear Fits for the "Best Change" Rates of DARKTHOUGHT (III) · 154 XVII

List of Tables

1.1 Performance of Null-Move Depth Reduction R = 1, 2, 3 34 1.2 Performance of Null-Move Depth Reduction R = 2 vs. R = 362 38 1.3 Performance of Null-Move Depth Reduction R = 362 vs. R = 3 39

2.1 Performance of Normal vs. Extended Futility Pruning . . . . . 46 2.2 Performance of Extended Futility Pruning vs. Added Razoring 49 2.3 Performance of Normal vs. Extd. Futil. Pruning + Ltd. Razoring 50

3.1 Performance of Normal vs. AEL-Pruning DARKTHOUGHT . . .. 57 3.2 Test-Suite Results of DARKTHOUGHT with AEL Pruning .... 57 3.3 DARKTHOUGHT with AEL Pruning vs. Normal DARKTHOUGHT 58 3.4 Strong PC Chess Programs vs. AEL-Pruning DARKTHOUGHT . 59

5.1 Index Schemes for 3-Piece, 4-Piece, and 5-Piece Databases. . . 96

6.1 Knowledgeable Encoding of 3-Piece / 4-Piece Endgame Databases. 104 6.2 Knowledgeable Probing of 3-Piece / 4-Piece Endgame Databases .. 107 6.3 Knowledgeable Scoring of 4-Piece Endgames ...... 109 6.4 Knowledgeable Endgame Databases in Tests with Deep Solutions. 114 6.5 Possible Knowledgeable Encoding of 5-Piece Endgame Databases . 119

7.1 "Best Change" Rates of BELLE, CRAFTY, and DARKTHOUGHT ... 132 7.2 Results of DARKTHOUGHT for All 343 Corrected Test Positions . . 133 7.3 Results of CRAFTY for All 343 Test Positions ...... 134 7.4 Results of DARKTHOUGHT for the 106 Opening Positions .. . 135 7.5 Results of CRAFTY for the 106 Opening Positions...... 136 7.6 Results of DARKTHOUGHT for the 107 Middlegame Positions . 137 7.7 Results of CRAFTY for the 107 Middlegame Positions. . . . . 137 7.8 Results of DARKTHOUGHT for the 130 Remaining Positions . 138 7.9 Results of CRAFTY for the 130 Remaining Positions . . 139 7.10 Confident Bounds on the "Best Change" Probabilities . . . . 142 7.11 "Best Change" Rates as Published in 1997...... 143 7.12 Calculated Results of CRAFTY for the 107 Middlegame Positions . 143

8.1 Interpolation Errors of the 3 Models for CRAFTY ...... 150 8.2 Interpolation Errors of the 7 Models for DARKTHOUGHT . . 155 8.3 Extrapolation Errors of the 7 Models for DARKTHOUGHT . 156

9.1 Direct Self-Play with 1000 Games per Match . 161 9.2 Direct Self-Play with 400 Games per Match . . 161 XVIII List of Tables

9.3 Direct Self-Play with 300 Games per Match . 162 9.4 Direct Self-Play of BELLE (1982) . . . . . 163 9.5 Direct Self-Play of BELLE (1983) ...... 164 9.6 Round-Robin Self-Play of BELLE (1983) . . . 164 9.7 Direct Self-Play of TECHMATE (1988, Time Doubling) . 165 9.8 Direct Self-Play of TECHMATE (1988, Time Quadrupling) . 165 9.9 Direct Self-Play of HITECH (1990) ...... 166 9.10 Round-Robin Self-Play of HITECH (1990) ...... 167 9.11 Round-Robin Play of HITECH vs. LOTECH (1990) . . 168 9.12 Total Round-Robin Play of HITECH (1990) . 168 9.13 Direct Self-Play of LOTECH (1990) ...... 169 9.14 Round-Robin Self-Play of LOTECH (1990) ...... 170 9.15 Round-Robin Play of LOTECH vs. HITECH (1990) . . 170 9.16 Total Round-Robin Play of LOTECH (1990) . . . . . 171 9.17 Direct Self-Play of ZUGZWANG (1994, Speed Doubling) . . 172 9.18 Direct Self-Play of PHOENIX (1996) ...... 173 9.19 Direct Self-Play of THE TURK (1997) ...... 173 9.20 Direct Self-Play of CHINOOK (1993, 1). Depth = 2) . 174 9.21 Direct Self-Play of CHINOOK (1995, 1). Depth = 2) . 175 9.22 Direct Self-Play of BILL (1990, 1). Depth = 1) . . . 176 9.23 Direct Self-Play of BILL (1990, 1). Depth = 2) . . . 176 9.24 Direct Self-Play of KEYANO (1997, 1). Depth = 1) . 177 9.25 Direct Self-Play of KEYANO (1997, 1). Depth = 2) . 178

D.1 Strong PC Chess Programs vs. AEL-Pruning DARKTHOUGHT. . 205