ICSEA 2019 : The Fourteenth International Conference on Software Engineering Advances Computer Analysis of World Chess Championship Players Oscar Romero Lorena Parra Universitat Politecnica de Valencia Universitat Politecnica de Valencia Valencia, Spain Valencia, Spain E-mail: [email protected] E-mail: [email protected] Jose Fernando Cuenca Jaime Lloret Chess24 Universitat Politecnica de Valencia Madrid, España Valencia, Spain E-mail: [email protected] E-mail: [email protected] Abstract— In some sports, it is difficult to know who has been The best players can foresee the development of a game 10 the best winner of the world championship. In athletism, it is to 15 moves to decide the best strategy [4]. The current not so difficult because world records clearly state which is the world champion (since 2013), Magnus Carlsen, has an Elo best mark. Nevertheless, in the case of chess, it is challenging to rating of 2882 [5]. On the other hand, the current versions of know who has been the best world chess championship player. the best chess programs have more than 3400 [6]. Nowadays, it is well known that many chess engines can beat In 2006, Guid and Bratko used CRAFTY, a chess the best chess players in the world, so we can use it for program, to evaluate the quality of chess players regardless comparison purposes. In this paper, we use one of the best of the game score [7]. They evaluate players of the World chess engines, Stockfish 10, in order to know which world Chess Championships (WCCs). CRAFTY calculates the best chess championship player is the best of all time. We have move for each given position and compare the move that did compared their moves during the world championship with the ones suggested by the chess engine in each game. Results show the players with the best move and assign an average error to how good each one of them was, compared with Stockfish 10, each player. Their results were strongly criticized because which player obtained the greatest percentage of best moves some of the best players as Fischer were placed as weaker during their games, how the quality of their moves evolved than players who only won the WCC one year. On the other during the games and the average percentage of best moves hand, their results were disputed as the engine used to throughout the games. calculate the best moves was considered weaker than most of the analyzed players. Keywords-Chess; Computer Analysis; World Chess In this paper, we are going to analyze the performance of Championship. all chess players in the WCC, like the work presented by M. Guid and I. Bratko in 2006, but using a stronger engine. Our I. INTRODUCTION hypothesis is that the results obtained in the past were Chess is a strategy board game that involves two players. skewed by the used engine. Nowadays, computers are more It is one of the most popular strategy games played across powerful and probably the best move calculated in the past the globe. Modern chess is based on the rules adopted in will be something different than the best move calculated in Spain in the 15th century [1]. It is played on a checkered this paper. Therefore, the average error for each player may board with 64 squares and includes 6 different types of be different. We use all the games of the WCC, from 1886 to pieces. Each player starts with 16 pieces and the player who 2018, and the chess engine is configured with a depth of 28. has the white pieces starts the game by moving first. The A total of 20 computers were used to calculate the average number of game states that can be reached through a legal error of each player and some other parameters. Then, the play was estimated to be around 1046 [2]. Because of its results of the average error or each player are compared. complexity, chess has been used as a testbed for most of the Finally, our results will be compared with the results Artificial Intelligence (AI) systems. obtained in the past to evaluate the ranking of best chess It 1947, Alan Turing designed a program to play chess players according to their average error change. for the first time in history. Since 1950, different programs The remaining of this paper is structured as follows. have been developed to play chess. Different strategies have Section 2 describes the related work. Section 3 details the been applied to improve their results. While in 1960 chess material and methods utilized. The results are shown in programs only can beat amateur players, in 1990 those Section 4. Finally, Section 5 presents the conclusion and programs have become powerful and can win chess masters future work. [3]. In the last decades, chess players have evolved and II. RELATED WORK improved using chess programs to practice and learn. The In this section, we are going to summarize the papers Elo is a method used to calculate the skills of a chess player. which deal with the topic of chess, algorithms and AI. Copyright (c) IARIA, 2019. ISBN: 978-1-61208-752-8 200 ICSEA 2019 : The Fourteenth International Conference on Software Engineering Advances Guid and Bratko compared the quality of different chess CRS4EAs is comparable with NHST, but it is easier to use players of the WCC [7]. They used the CRAFTY program. and it is less sensitive to outliers. CRAFTY evaluated the individual move realized by each In recent years, machine learning techniques have been player. They used the games played in the WCC from 1886 applied to strategy games, and then, trying to play better than to 2006. CRAFTY used a depth of 12 moves in the analyses. static algorithms. Alphago Zero was developed to play go, a The parameters used to compare the players were the simple game, with simple rules, but with many possible average error, % of blunders, complexity expected error, % moves. It is an artificial intelligence, based on deep learning of best moves and difference between best moves. Their and neuronal networks. After a deep training, it was able to results showed that Capablanca was the best player and win the best human player on year 2016. In chess, several Fischer was the one with the highest difference between best artificial intelligences have been developed. One of the most moves. important is Leela Chess Zero (LCZero), also based on Ribeiro et al. [4] in 2013 used CRAFTY (Elo rating of neuronal networks. After an intense training, it became the 2950) to evaluate the white player advantage move-by-move. They used 73,444 high-level chess matches available in champion in Top Chess Engine Champion (TCEC) season Portable Game Notation (PGN) Mentor. CRAFTY 15, in May 2019, where Stockfish, one of the best static calculated de advantage in terms of the number of remaining algorithms, obtained the second position. Recently, Stockfish pieces and its placement. A positive value indicated that the has been improved and it is the current TCEC champion, white player has an advantage, while the negative value season 16, celebrated on October 2019. This time, LCZero indicated that is the black player who has the advantage. was in second position. Probably both of them will be Their results included the advantage, mean of advantage and improved again, and perhaps they will get an ELO above of variance of advantage along with the game in each match. 4000 very soon. They compare the data from different periods, 1857-1918, 1919-1949 and 1950-2011 to evaluate the changes in the chess players. Their results suggested that the opening stage III. TEST BENCH AND METHOD of a match is becoming longer and pointed out that this In this work, we have evaluated all players in all world might be related to a collective learning process. chess championships, from 1886 to 2018. Today, there are On the other hand, many authors used chess to test and chess engines that play clearly better than the best human train AI systems. One example is the work proposed by players. Thus, using one of those chess engines we can Vázquez-Fernández et al. in 2011 [8]. They proposed a evaluate a human player. For this analysis, we get the score method for tuning the weights of the evaluation function of of each move and compare it with the best move obtained by chess based on evolutionary programming. They used 10 the engine to get the human player error. From the different players and 6 training games. They used as information provided by the chess engine, we can extract “theoretical” values: 100 (pawn (fixed value)), 300 (knight), additional information such as if the human selected the first, 330 (bishop), 500 (rook) and 900 (queen). As mobility, a second, etc. best move. weight of 10, and bounds of [0,300], was used. After 50 generations, the weight changed to 100 (pawn), 310.89 A. Test Bench (knight), 325.32 (bishop), 514.92 (rook), 841.61 (queen) and The chess engine selected for this study is Stockfish 64 5.62 (mobility). Finally, their engine was tested playing 10 bits Version 10, one of the strongest engines in early 2019. games with a human player. It is important to note that, in We have created a program, using the Universal Chess this paper, the Elo of the used engine was 1463. The Elo of Interface (UCI) protocol to communicate with the chess the human who played with the engine was 1737. engine. The feature of the computers used to perform the test One year later, Vázquez-Fernández et al.