DOCSLIB.ORG

An Artificial Intelligence Agent for Texas Hold'em Poker

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
An Artificial Intelligence Agent for Texas Hold'em Poker AN ARTIFICIAL INTELL IGENCE AGENT FOR TEXAS HOLD’EM PO KER PATRICK MCCURLEY – 0 62 4 91 7 90 2 An Artificial Intelligence Agent for Texas Hold’em Poker I declare that this document represents my own work except where otherwise stated. Signed …………………………………………………………………………. 08/05/ 2009 Patrick McCurley – 062491790 Introduction 3 TABLE OF CONTENTS 1. Introduction ................................................................................................................................................................ 7 1.1 Problem Description...................................................................................................................................... 7 1.2 Aims and Objectives....................................................................................................................................... 7 1.3 Dissertation Outline ....................................................................................................................................... 8 1.4 Ethics .................................................................................................................................................................... 8 2 Background................................................................................................................................................................10 2.1 Artificial Intelligence and Poker .............................................................................................................10 2.1.1 Problem Domain Realization .........................................................................................................10 2.1.2 Hand Evaluation Algorithms ..........................................................................................................11 2.1.3 Using Hand Evaluation and Opponent Predictions to Determine Value.....................12 2.1.4 The Nash Equilibrium........................................................................................................................12 2.2 Opponent Modelling ....................................................................................................................................14 2.2.1 Pre-flop Opponent Modelling.........................................................................................................14 2.2.2 Artificial Neural Networks ..............................................................................................................15 2.2.3 Bayesian Approach .............................................................................................................................15 2.2.4 Particle Filtering ..................................................................................................................................16 2.3 Strategy Implementation and Performance Measurement ........................................................17 2.3.1 DIVAT Tool .............................................................................................................................................17 2.3.2 Limitations of DIVAT .........................................................................................................................18 2.4 Data Analysis...................................................................................................................................................19 2.4.1 Data Mining ............................................................................................................................................19 2.4.2 Important Statistics ............................................................................................................................19 3 Design...........................................................................................................................................................................21 3.1 Approach...........................................................................................................................................................21 3.2 Requirements .................................................................................................................................................22 3.3 Technologies ...................................................................................................................................................23 3.4 Resources..........................................................................................................................................................23 3.5 Architecture.....................................................................................................................................................26 3.5.1 Phase One................................................................................................................................................26 3.5.2 Phase Two...............................................................................................................................................29 4. Implementation .......................................................................................................................................................38 4.1 Phase One Components ..............................................................................................................................38 4.1.1 Scraping Manager................................................................................................................................38 4.1.2 Rules Manager ......................................................................................................................................40 4.1.3 Hand Evaluation...................................................................................................................................41 4.2 Phase Two Components .............................................................................................................................44 4.2.1 Data Clustering Manager..................................................................................................................44 4 An Artificial Intelligence Agent for Texas Hold’em Poker 4.2.2 Opponent Modelling Manager .......................................................................................................45 4.2.3 Game Tree Simulator .........................................................................................................................55 5. Results..........................................................................................................................................................................61 5.1 Data Clustering Results ..............................................................................................................................61 5.2 Neural Network Results .............................................................................................................................63 5.3 Phase One Agent Results............................................................................................................................64 5.4 Phase Two Agent Results...........................................................................................................................65 6. Evaluation...................................................................................................................................................................66 6.1 Results Evaluation ........................................................................................................................................66 6.1.1 Data Clustering Results.....................................................................................................................66 6.1.2 Neural Network Results ...................................................................................................................66 6.1.3 Rule-Based Agent Results ................................................................................................................67 6.1.4 AI Agent Results ...................................................................................................................................67 6.2 Project Evaluation.........................................................................................................................................68 6.2.1 Architectural Implementation .......................................................................................................68 6.2.2 Calculation Performance ..................................................................................................................68 6.2.3 Final Implementation ........................................................................................................................69 7. Conclusion ..................................................................................................................................................................71 7.1 Objectives .........................................................................................................................................................71 7.2 Project reflection ...........................................................................................................................................72 7.3 Further Work ..................................................................................................................................................73 Acknowledgements..........................................................................................................................................................74
Load more

Recommended publications
  • Most Important Fundamental Rule of Poker Strategy
    The Thirty-Third International FLAIRS Conference (FLAIRS-33) Most Important Fundamental Rule of Poker Strategy Sam Ganzfried,1 Max Chiswick 1Ganzfried Research Abstract start playing very quickly, even the best experts spend many years (for some an entire lifetime) learning and improving. Poker is a large complex game of imperfect information, Humans learn and improve in a variety of ways. The most which has been singled out as a major AI challenge prob- obvious are reading books, hiring coaches, poker forums and lem. Recently there has been a series of breakthroughs culmi- nating in agents that have successfully defeated the strongest discussions, and simply playing a lot to practice. Recently human players in two-player no-limit Texas hold ’em. The several popular software tools have been developed, most 1 strongest agents are based on algorithms for approximating notably PioSolver, where players solve for certain situa- Nash equilibrium strategies, which are stored in massive bi- tions, given assumptions for the hands the player and oppo- nary files and unintelligible to humans. A recent line of re- nent can have. This is based on the new concept of endgame search has explored approaches for extrapolating knowledge solving (Ganzfried and Sandholm 2015), where strategies from strong game-theoretic strategies that can be understood are computed for the latter portion of the game given fixed by humans. This would be useful when humans are the ulti- strategies for the trunk (which are input by the user). While mate decision maker and allow humans to make better deci- it has been pointed out that theoretically this approach is sions from massive algorithmically-generated strategies.
    [Show full text]
  • “Algoritmos Para Um Jogador Inteligente De Poker” Autor: Vinícius
    Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Florianópolis/SC, 2008 Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Orientador: Mauro Roisenberg Banca: Benjamin Luiz Franklin Banca: João Rosaldo Vollertt Junior Banca: Ricardo Azambuja Silveira Florianópolis/SC, 2008 AGRADECIMENTOS Agradeço a todos que me ajudaram no desenvolvimento deste trabalho, em especial ao professor Mauro Roisenberg e aos colegas de trabalho João Vollertt e Benjamin Luiz Franklin e ao Ricardo Azambuja Silveira pela participação na banca avaliadora. Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 4 RESUMO Poker é um jogo simples de cartas que envolve aposta, blefe e probabilidade de vencer. O objetivo foi inventar e procurar algoritmos para jogadores artificiais evolutivos e estáticos que jogassem bem poker. O jogador evolutivo criado utiliza aprendizado por reforço, onde o jogo é dividido em uma grande matriz de estados com dados de decisões e recompensas. O jogador toma a decisão que obteve a melhor recompensa média em cada estado. Para comparar a eficiência do jogador, várias disputas com jogadores que tomam decisões baseados em fórmulas simples foram feitas. Diversas disputas foram feitas para comparar a eficiência de cada algoritmo e os resultados estão demonstrados graficamente no trabalho. Palavras-Chave: Aprendizado por Reforço, Inteligência Artificial, Poker; Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 5 SUMÁRIO 1. Introdução........................................................................................................12 2. Fundamentação Teórica..................................................................................15 2.1. Regras do Poker Texas Hold'em................................................................16 2.1.1.
    [Show full text]
  • Texas Hold'em Poker
    11/5/2018 Rules of Card Games: Texas Hold'em Poker Pagat Poker Poker Rules Poker Variants Home Page > Poker > Variations > Texas Hold'em DE EN Choose your language Texas Hold'em Introduction Players and Cards The Deal and Betting The Showdown Strategy Variations Pineapple ‑ Crazy Pineapple ‑ Crazy Pineapple Hi‑Lo Irish Casino Versions Introduction Texas Hold'em is a shared card poker game. Each player is dealt two private cards and there are five face up shared (or "community") cards on the table that can be used by anyone. In the showdown the winner is the player who can make the best five‑card poker hand from the seven cards available. Since the 1990's, Texas Hold'em has become one of the most popular poker games worldwide. Its spread has been helped firstly by a number of well publicised televised tournaments such as the World Series of Poker and secondly by its success as an online game. For many people nowadays, poker has become synonymous with Texas Hold'em. This page assumes some familiarity with the general rules and terminology of poker. See the poker rules page for an introduction to these, and the poker betting and poker hand ranking pages for further details. Players and Cards From two to ten players can take part. In theory more could play, but the game would become unwieldy. A standard international 52‑card pack is used. The Deal and Betting Texas Hold'em is usually played with no ante, but with blinds. When there are more than two players, the player to dealer's left places a small blind, and the next player to the left a big blind.
    [Show full text]
  • Abiding Chance: Online Poker and the Software of Self-Discipline
    ESSAYS Abiding Chance: Online Poker and the Software of Self- Discipline Natasha Dow Schüll A man sits before a large desktop monitor station, the double screen divided into twenty- four rectangles of equal size, each containing the green oval of a poker table with positions for nine players. The man is virtu- ally “seated” at all twenty- four tables, along with other players from around the world. He quickly navigates his mouse across the screen, settling for moments at a time on flashing windows where his input is needed to advance play at a given table. His rapid- fire esponsesr are enabled by boxed panels of colored numbers and letters that float above opponents’ names; the letters are acronyms for behavioral tendencies relevant to poker play, and the numbers are statistical scores identifying where each player falls in a range for those tendencies. Taken together, the letters and numbers supply the man with enough information to act strategically at a rate of hundreds of hands per hour. Postsession, the man opens his play- tracking database to make sure the software has successfully imported the few thousand hands he has just played. After quickly scrolling through to ensure that they are all there, he recalls some particularly challenging hands he would like to review and checks a number Thanks to Paul Rabinow and Limor Samimian- Darash, for prompting me to gather this material for a different article, and to Richard Fadok, Paul Gardner, Lauren Kapsalakis, and the students in my 2013 Self as Data graduate seminar at the Massachusetts Institute of Technology, for helping me to think through that material.
    [Show full text]
  • Pokerface: Emotion Based Game-Play Techniques for Computer Poker Players
    University of Kentucky UKnowledge University of Kentucky Master's Theses Graduate School 2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Lucas Cockerham University of Kentucky, [email protected] Right click to open a feedback form in a new tab to let us know how this document benefits ou.y Recommended Citation Cockerham, Lucas, "POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS" (2004). University of Kentucky Master's Theses. 224. https://uknowledge.uky.edu/gradschool_theses/224 This Thesis is brought to you for free and open access by the Graduate School at UKnowledge. It has been accepted for inclusion in University of Kentucky Master's Theses by an authorized administrator of UKnowledge. For more information, please contact [email protected]. ABSTRACT OF THESIS POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Numerous algorithms/methods exist for creating computer poker players. This thesis compares and contrasts them. A set of poker agents for the system PokerFace are then introduced. A survey of the problem of facial expression recognition is included in the hopes it may be used to build a better computer poker player. KEYWORDS: Poker, Neural Networks, Arti¯cial Intelligence, Emotion Recognition, Facial Action Coding System Lucas Cockerham 7/30/2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS By Lucas Cockerham Dr. Judy Goldsmith Dr. Grzegorz Wasilkowski 7/30/2004 RULES FOR THE USE OF THESES Unpublished theses submitted for the Master's degree and deposited in the University of Kentucky Library are as a rule open for inspection, but are to be used only with due regard to the rights of the authors.
    [Show full text]
  • Van Fleet Climbs to No
    www.CardPlayer.com Vol. 34/No. 5 February 24, 2021 World Poker Tour Sells For $78 Million Q&A With Six-Time WSOP Circuit Winner Max Young Poker Strategy: The Wrong Time To Aggressively Play A Flush Draw JONATHAN ‘APESTYLES’ VAN FLEET CLIMBS TO NO. 3 ON ONLINE POKER’S ALL-TIME TOURNAMENT EARNINGS LIST PLAYER_35_5B_Cover.indd 1 2/4/21 9:43 AM PLAYER_05_GlobalPoker_DT.indd 2 2/2/21 10:24 AM PLAYER_05_GlobalPoker_DT.indd 3 2/2/21 10:24 AM Masthead - Card Player Vol. 34/No. 5 PUBLISHERS Barry Shulman | Jeff Shulman Editorial Corporate Office EDITORIAL DIRECTOR Julio Rodriguez 6940 O’Bannon Drive TOURNAMENT CONTENT MANAGER Erik Fast Las Vegas, Nevada 89117 ONLINE CONTENT MANAGER Steve Schult (702) 871-1720 Art [email protected] ART DIRECTOR Wendy McIntosh Subscriptions/Renewals 1-866-LVPOKER Website And Internet Services (1-866-587-6537) CHIEF TECHNOLOGY OFFICER Jaran Hardman PO Box 434 DATA COORDINATOR Morgan Young Congers, NY 10920-0434 Sales [email protected] ADVERTISING MANAGER Mary Hurbi Advertising Information NATIONAL SALES MANAGER Barbara Rogers [email protected] LAS VEGAS AND COLORADO SALES REPRESENTATIVE (702) 856-2206 Rich Korbin Distribution Information cardplayer Media LLC [email protected] CHAIRMAN AND CEO Barry Shulman PRESIDENT AND COO Jeff Shulman Results GENERAL COUNSEL Allyn Jaffrey Shulman [email protected] VP INTL. BUSINESS DEVELOPMENT Dominik Karelus CONTROLLER Mary Hurbi Schedules FACILITIES MANAGER Jody Ivener [email protected] Follow us www.facebook.com/cardplayer @CardPlayerMedia Card Player (ISSN 1089-2044) is published biweekly by Card Player Media LLC, 6940 O’Bannon Drive, Las Vegas, NV 89117.
    [Show full text]
  • Early Round Bluffing in Poker Author(S): California Jack Cassidy Source: the American Mathematical Monthly, Vol
    Early Round Bluffing in Poker Author(s): California Jack Cassidy Source: The American Mathematical Monthly, Vol. 122, No. 8 (October 2015), pp. 726-744 Published by: Mathematical Association of America Stable URL: http://www.jstor.org/stable/10.4169/amer.math.monthly.122.8.726 Accessed: 23-12-2015 19:20 UTC Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/ info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Mathematical Association of America is collaborating with JSTOR to digitize, preserve and extend access to The American Mathematical Monthly. http://www.jstor.org This content downloaded from 128.32.135.128 on Wed, 23 Dec 2015 19:20:53 UTC All use subject to JSTOR Terms and Conditions Early Round Bluffing in Poker California Jack Cassidy Abstract. Using a simplified form of the Von Neumann and Morgenstern poker calculations, the author explores the effect of hand volatility on bluffing strategy, and shows that one should never bluff in the first round of Texas Hold’Em. 1. INTRODUCTION. The phrase “the mathematics of bluffing” often brings a puzzled response from nonmathematicians. “Isn’t that an oxymoron? Bluffing is psy- chological,” they might say, or, “Bluffing doesn’t work in online poker.
    [Show full text]
  • Abiding Chance: Online Poker and the Software of Self-Discipline
    ESSAYS Abiding Chance: Online Poker and the Software of Self- Discipline Natasha Dow Schüll A man sits before a large desktop monitor station, the double screen divided into twenty- four rectangles of equal size, each containing the green oval of a poker table with positions for nine players. The man is virtu- ally “seated” at all twenty- four tables, along with other players from around the world. He quickly navigates his mouse across the screen, settling for moments at a time on flashing windows where his input is needed to advance play at a given table. His rapid- fire esponsesr are enabled by boxed panels of colored numbers and letters that float above opponents’ names; the letters are acronyms for behavioral tendencies relevant to poker play, and the numbers are statistical scores identifying where each player falls in a range for those tendencies. Taken together, the letters and numbers supply the man with enough information to act strategically at a rate of hundreds of hands per hour. Postsession, the man opens his play- tracking database to make sure the software has successfully imported the few thousand hands he has just played. After quickly scrolling through to ensure that they are all there, he recalls some particularly challenging hands he would like to review and checks a number Thanks to Paul Rabinow and Limor Samimian- Darash, for prompting me to gather this material for a different article, and to Richard Fadok, Paul Gardner, Lauren Kapsalakis, and the students in my 2013 Self as Data graduate seminar at the Massachusetts Institute of Technology, for helping me to think through that material.
    [Show full text]
  • Building a Poker Playing Agent Based on Game Logs Using Supervised Learning
    FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO Building a Poker Playing Agent based on Game Logs using Supervised Learning Luís Filipe Guimarães Teófilo Mestrado Integrado em Engenharia Informática e Computação Orientador: Professor Doutor Luís Paulo Reis 26 de Julho de 2010 Building a Poker Playing Agent based on Game Logs using Supervised Learning Luís Filipe Guimarães Teófilo Mestrado Integrado em Engenharia Informática e Computação Aprovado em provas públicas pelo Júri: Presidente: Professor Doutor António Augusto de Sousa Vogal Externo: Professor Doutor José Torres Orientador: Professor Doutor Luís Paulo Reis ____________________________________________________ 26 de Julho de 2010 iv Resumo O desenvolvimento de agentes artificiais que jogam jogos de estratégia provou ser um domínio relevante de investigação, sendo que investigadores importantes na área das ciências de computadores dedicaram o seu tempo a estudar jogos como o Xadrez e as Damas, obtendo resultados notáveis onde o jogador artificial venceu os melhores jogadores humanos. No entanto, os jogos estocásticos com informação incompleta trazem novos desafios. Neste tipo de jogos, o agente tem de lidar com problemas como a gestão de risco ou o tratamento de informação não fiável, o que torna essencial modelar adversários, para conseguir obter bons resultados. Nos últimos anos, o Poker tornou-se um fenómeno de massas, sendo que a sua popularidade continua a aumentar. Na Web, o número de jogadores aumentou bastante, assim como o número de casinos online, tornando o Poker numa indústria bastante rentável. Além disso, devido à sua natureza estocástica de informação imperfeita, o Poker provou ser um problema desafiante para a inteligência artificial. Várias abordagens foram seguidas para criar um jogador artificial perfeito, sendo que já foram feitos progressos nesse sentido, como o melhoramento das técnicas de modelação de oponentes.
    [Show full text]
  • Mike Caro Caro's Book of Poker Tells
    1 2 MIKE CARO CARO’S BOOK OF POKER TELLS THE PSICHOLOGY AND BODY LANGUAGE OF POKER 3 FREE GAMING NEWSLETTERS!!! www.cardozapub.com When you sign up for Avery Cardoza's online newsletters, you receive three for free: Avery Cardoza's Poker Newsletter, Avery Cardoza’s Gambling Newsletter and Avery Cardoza's Online Gaming Newsletter. Our newsletters are packed with tips, expert strategies, discounts, free money sign-up bonuses on online sites, pre-publication discounts, poker and gaming tournament news, schedules, and words of wisdom from the world's top experts, authorities, and World Champions. And what's best — it's FREE! Sign up now! www.cardozapub.com Mike Caro University of Poker Gaming and Life Strategy (MCU) merges Caro's own research with the collective wisdom of other great poker minds to form a one-of-a-kind learning center teaching poker strategy and psychology. In addition to instructional courses, MCU serves as a central force for popularizing poker, maintaining its integrity, and standardizing rules. Contact MCU's website at: UniversityofPoker.com. To join the mailing list: [email protected]. To contact the author: [email protected]. Cardoza Publishing is the foremost gaming and gambling publisher in the world with a library of more than 100 up-to-date and easy-to-read books and strategies. These authoritative works are written by the lop experts in their fields and with more than 7,500,000 books in print, represent the best-selling and most popular gaming books anywhere. FIRST CARDOZA EDITION First Printing February 2003 Second Printing July 2003 Third Printing October 2003 Fourth Printing March 2004 Fifth Printing October 2004 Library of Congress Catalog No: 2002106098 ISBN: 1-58042-082-6 This Cardoza edition is printed by special arrangement with Mike Caro University of Poker Press.
    [Show full text]
  • Building a Champion Level Computer Poker Player
    University of Alberta Library Release Form Name of Author: Michael Bradley Johanson Title of Thesis: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player Degree: Master of Science Year this Degree Granted: 2007 Permission is hereby granted to the University of Alberta Library to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s prior written permission. Michael Bradley Johanson Date: Too much chaos, nothing gets finished. Too much order, nothing gets started. — Hexar’s Corollary University of Alberta ROBUST STRATEGIES AND COUNTER-STRATEGIES: BUILDING A CHAMPION LEVEL COMPUTER POKER PLAYER by Michael Bradley Johanson A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements for the degree of Master of Science. Department of Computing Science Edmonton, Alberta Fall 2007 University of Alberta Faculty of Graduate Studies and Research The undersigned certify that they have read, and recommend to the Faculty of Graduate Studies and Research for acceptance, a thesis entitled Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player submitted by Michael Bradley Johanson in partial fulfillment of the requirements for the degree of Master of Science. Michael Bowling Supervisor Duane Szafron Michael Carbonaro External Examiner Date: To my family: my parents Brad and Sue Johanson, and my brother, Jeff Johanson.
    [Show full text]
  • Top Poker Software
    Top poker software If you understand only one thing about poker strategy software, understand that knowing how to use these specialized programs gives you an enormous edge. Get better at poker with the top five best poker strategy apps, tools and software programs: PokerStove, ICM calculators, Hold'em Manager. We have collected the very best online poker tools. The software then reads the database and makes use of a HUD to display information of. Data mined hand histories that can be used and uploaded into analysis programs like Poker Tracker or Hold' Em Manager. This is not software. Recommendations on the best poker tools such as Poker Tracker and Poker Stove. Information on how best to use them to improve your poker play. There are currently only two stable and complete online poker tracking software solutions for analyzing HUD statistics - Holdem Manager and Poker Tracker. Stay up to date with poker software used in the modern game, and maximize your PokerTracker and Holdem Manager are by far the most popular poker tools. The ultimate online poker software suite of tools, created for players just like you. is the best game tracking choice for both Texas Holdem and Omaha players. There are several different types of poker tools and software used by online poker players these days. None of them are a magic wand that will. We created a list of the best poker tools, poker software & poker accessories! They may not make you a better player, but you'll certainly be fully. The Best Poker Tools And Calculators Here Will Give You The Edge That You rules about which poker software tools can and can't be used on their sites.
    [Show full text]