http://lib.ulg.ac.be http://matheo.ulg.ac.be Creation of a video game and learning of an intelligent agent Auteur : Lorent, Nicolas Promoteur(s) : Ernst, Damien Faculté : Faculté des Sciences appliquées Diplôme : Master en sciences informatiques, à finalité approfondie Année académique : 2015-2016 URI/URL : http://hdl.handle.net/2268.2/1665 Avertissement à l'attention des usagers : Tous les documents placés en accès ouvert sur le site le site MatheO sont protégés par le droit d'auteur. Conformément aux principes énoncés par la "Budapest Open Access Initiative"(BOAI, 2002), l'utilisateur du site peut lire, télécharger, copier, transmettre, imprimer, chercher ou faire un lien vers le texte intégral de ces documents, les disséquer pour les indexer, s'en servir de données pour un logiciel, ou s'en servir à toute autre fin légale (ou prévue par la réglementation relative au droit d'auteur). Toute utilisation du document à des fins commerciales est strictement interdite. Par ailleurs, l'utilisateur s'engage à respecter les droits moraux de l'auteur, principalement le droit à l'intégrité de l'oeuvre et le droit de paternité et ce dans toute utilisation que l'utilisateur entreprend. Ainsi, à titre d'exemple, lorsqu'il reproduira un document par extrait ou dans son intégralité, l'utilisateur citera de manière complète les sources telles que mentionnées ci-dessus. Toute utilisation non explicitement autorisée ci-avant (telle que par exemple, la modification du document ou son résumé) nécessite l'autorisation préalable et expresse des auteurs ou de leurs ayants droit. University of Liège Faculty of Applied Sciences Montefiore Institute Development of a Video Game and Creation of an Artificial Player Master Thesis Submitted for the Degree of MSc in Computer Science Author: Supervisor: Nicolas LORENT Prof. Damien ERNST Academic year 2015 ­ 2016 2 Acknowledgements First, I am grateful to Prof. Damien Ernst for approving the subject of my thesis, and allowing me to combine my passion with my studies. I also wish to express my sincere thanks to David Taralla and Aaron Qiu, for their dedication and invaluable advices. Their help and support considerably improved my work, and I am thankful for the time that they dedicated to reviewing this thesis. I was really pleased to meet these friendly fellows who share the same passion for card games, and I honestly could not wished for better co­supervisors. You guys rock! 3 Purpose The purpose of this work is twofold. In a first part, we develop a real­time strategic card game from scratch. It includes the design of game rules, a user interface, cards, decks, and the implementation of the game in java with neither a game engine nor an external library. The game is playable online or offline, against other human players or artificial players, and with pre­built or custom decks. In a second part, we consider the problem of creating an artificial intelligence to play the card game. Two artificial players are proposed. Firstly, a heuristic player that evaluates moves by quantifying the amount of resources it gains (or loses) versus the amount of resources its opponent gains (or loses). Secondly, a more advanced player that simulates the long­term consequences of a move until the end of the game to predict its outcome. Lastly, the performances of both artificial players are assessed by comparing their win rates. 4 Table of Contents Purpose 4 ​ ​ Part 1: Development of a Video Game 8 ​ ​ 1. Small Tour in the CCG World 9 ​ ​ 1.1 Hearthstone . 9 ​ ​ ​ ​ 1.2 Magic: The Gathering . 13 ​ ​ ​ ​ 1.3 Pokémon . 17 ​ ​ ​ ​ 1.4 Yu­Gi­Oh! . 19 ​ ​ ​ ​ 1.5 Discussion . 21 ​ ​ ​ ​ ​ 2. Game Design 22 ​ ​ 2.1 Questioning the Basics . 22 ​ ​ ​ 2.1.1 Turn­based Advantages . 22 ​ ​ ​ 2.1.2 Turn­based Drawbacks . 22 ​ ​ ​ 2.1.3 Proposed Solution . 23 ​ ​ ​ 2.2 Resources . 24 ​ ​ ​ ​ 2.2.1 Resources Production . 24 ​ ​ ​ ​ 2.2.2 Resources Types . 24 ​ ​ ​ 2.3 Deck Building . 25 ​ ​ ​ ​ 2.3.1 Deck Segmentation . 25 ​ ​ ​ ​ 2.3.2 Deck Size . 25 ​ ​ ​ 2.4 Hero . 26 ​ ​ ​ ​ 2.5 Protection & Positioning . 27 ​ ​ ​ ​ 2.6 Simplicity . 28 ​ ​ ​ ​ 2.7 Randomness . 28 ​ ​ ​ ​ 2.8 Game Art . 29 ​ ​ ​ 2.8.1 Cards . 29 ​ ​ ​ 2.8.2 Board . 30 ​ ​ ​ 31 3. Implementation ​ 3.1 The Network Layer . 31 ​ ​ ​ ​ 3.1.1 Multiplayer Mode . 31 ​ ​ ​ ​ 3.1.2 Single­player Mode. 34 ​ ​ ​ ​ 3.2 The Control Layer . 36 ​ ​ ​ ​ ​ 3.2.1 Mouse Inputs . 36 ​ ​ ​ 3.2.2 Time Steps . 37 ​ ​ ​ ​ 3.2.3 Game Events . 37 ​ ​ ​ ​ 3.2.4 Audio . 38 ​ ​ ​ ​ 5 3.3 The Logical Layer . 39 ​ ​ ​ 3.3.1 Effects . 39 ​ ​ ​ 3.3.2 Parameters . 41 ​ ​ ​ ​ 3.3.3 Targets . 42 ​ ​ ​ ​ ​ 3.3.4 Filters . 44 ​ ​ 3.3.5 Conditional Effects . 46 ​ ​ ​ 3.3.6 Triggered Effects . 47 ​ ​ ​ ​ ​ ​ 3.3.7 Ongoing Effects . 49 ​ ​ ​ 3.3.8 Temporal Effects . 50 ​ ​ ​ ​ ​ 3.4 The Model Layer . 52 ​ ​ ​ ​ 3.4.1 Cards . 52 ​ ​ 3.4.2 Creatures . 53 ​ ​ 3.4.3 Heroes . 53 ​ ​ ​ 3.4.4 Spells . 53 ​ ​ ​ 3.4.5 Events . 54 ​ ​ ​ 3.4.6 Decks . 54 ​ ​ ​ 3.5 The Graphical User Interface Layer . 56 ​ ​ ​ ​ ​ ​ 3.5.1 Cards . 56 ​ ​ ​ 3.5.2 Deck Selection Screen . 57 ​ ​ ​ 3.5.3 Board Screen . 58 ​ ​ ​ Part 2: Creation of an Artificial Player 62 1. Artificial Intelligence in Video Games . 63 ​ ​ ​ ​ ​ ​ 1.1 Tree Representation . 63 ​ ​ ​ ​ 1.2 Two­player Games . 63 ​ ​ ​ ​ ​ 1.3 Pruning . 64 ​ ​ ​ ​ 1.4 Randomness . 65 ​ ​ ​ ​ 1.5 Tree Search . 66 ​ ​ ​ ​ 1.6 Monte Carlo Tree Search . 67 ​ ​ ​ ​ 2. Problem Statement . 69 ​ ​ ​ ​ 2.1 Game Properties . 69 ​ ​ ​ ​ 2.1.1 Two­player . 69 ​ ​ ​ 2.1.2 Partially Observable . 69 ​ ​ ​ 2.1.3 Non­deterministic . 69 ​ ​ ​ ​ 2.1.4 Real­time . 70 ​ ​ ​ 2.2 Game Decomposition . 70 ​ ​ ​ 2.2.1 Attacking . 70 ​ ​ ​ ​ 2.2.2 Positioning . 72 ​ ​ 2.2.3 Playing Cards . 73 ​ ​ ​ ​ 3. Solutions . 75 ​ ​ ​ ​ 6 3.1 The Heuristic Player . 75 ​ ​ ​ ​ ​ 3.1.1 Attacking . 75 ​ ​ ​ ​ 3.1.2 Positioning . 77 ​ ​ ​ ​ 3.1.3 Playing Cards . 78 ​ ​ ​ ​ 3.2 The Simulating Player . 78 ​ ​ ​ 3.2.1 Representing Real­time . 79 ​ ​ ​ 3.2.2 Simulating Real­time . 79 ​ ​ 3.2.3 Choosing a Move . ..