AN ARTIFICIAL INTELL IGENCE AGENT FOR TEXAS HOLD’EM PO KER PATRICK MCCURLEY – 0 62 4 91 7 90 2 An Artificial Intelligence Agent for Texas Hold’em Poker I declare that this document represents my own work except where otherwise stated. Signed …………………………………………………………………………. 08/05/ 2009 Patrick McCurley – 062491790 Introduction 3 TABLE OF CONTENTS 1. Introduction ................................................................................................................................................................ 7 1.1 Problem Description...................................................................................................................................... 7 1.2 Aims and Objectives....................................................................................................................................... 7 1.3 Dissertation Outline ....................................................................................................................................... 8 1.4 Ethics .................................................................................................................................................................... 8 2 Background................................................................................................................................................................10 2.1 Artificial Intelligence and Poker .............................................................................................................10 2.1.1 Problem Domain Realization .........................................................................................................10 2.1.2 Hand Evaluation Algorithms ..........................................................................................................11 2.1.3 Using Hand Evaluation and Opponent Predictions to Determine Value.....................12 2.1.4 The Nash Equilibrium........................................................................................................................12 2.2 Opponent Modelling ....................................................................................................................................14 2.2.1 Pre-flop Opponent Modelling.........................................................................................................14 2.2.2 Artificial Neural Networks ..............................................................................................................15 2.2.3 Bayesian Approach .............................................................................................................................15 2.2.4 Particle Filtering ..................................................................................................................................16 2.3 Strategy Implementation and Performance Measurement ........................................................17 2.3.1 DIVAT Tool .............................................................................................................................................17 2.3.2 Limitations of DIVAT .........................................................................................................................18 2.4 Data Analysis...................................................................................................................................................19 2.4.1 Data Mining ............................................................................................................................................19 2.4.2 Important Statistics ............................................................................................................................19 3 Design...........................................................................................................................................................................21 3.1 Approach...........................................................................................................................................................21 3.2 Requirements .................................................................................................................................................22 3.3 Technologies ...................................................................................................................................................23 3.4 Resources..........................................................................................................................................................23 3.5 Architecture.....................................................................................................................................................26 3.5.1 Phase One................................................................................................................................................26 3.5.2 Phase Two...............................................................................................................................................29 4. Implementation .......................................................................................................................................................38 4.1 Phase One Components ..............................................................................................................................38 4.1.1 Scraping Manager................................................................................................................................38 4.1.2 Rules Manager ......................................................................................................................................40 4.1.3 Hand Evaluation...................................................................................................................................41 4.2 Phase Two Components .............................................................................................................................44 4.2.1 Data Clustering Manager..................................................................................................................44 4 An Artificial Intelligence Agent for Texas Hold’em Poker 4.2.2 Opponent Modelling Manager .......................................................................................................45 4.2.3 Game Tree Simulator .........................................................................................................................55 5. Results..........................................................................................................................................................................61 5.1 Data Clustering Results ..............................................................................................................................61 5.2 Neural Network Results .............................................................................................................................63 5.3 Phase One Agent Results............................................................................................................................64 5.4 Phase Two Agent Results...........................................................................................................................65 6. Evaluation...................................................................................................................................................................66 6.1 Results Evaluation ........................................................................................................................................66 6.1.1 Data Clustering Results.....................................................................................................................66 6.1.2 Neural Network Results ...................................................................................................................66 6.1.3 Rule-Based Agent Results ................................................................................................................67 6.1.4 AI Agent Results ...................................................................................................................................67 6.2 Project Evaluation.........................................................................................................................................68 6.2.1 Architectural Implementation .......................................................................................................68 6.2.2 Calculation Performance ..................................................................................................................68 6.2.3 Final Implementation ........................................................................................................................69 7. Conclusion ..................................................................................................................................................................71 7.1 Objectives .........................................................................................................................................................71 7.2 Project reflection ...........................................................................................................................................72 7.3 Further Work ..................................................................................................................................................73 Acknowledgements..........................................................................................................................................................74