DOCSLIB.ORG

“Texas Hold 'Em”

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
“Texas Hold 'Em” University of Nottingham Supervisor - Graham Kendall gp-gxk2 2nd Year Software Engineering Project - “Texas Hold ‘Em” Group Members: wxa00u Wafa Al-Ajmi wlc00u Wai Leuk Chu kxg00u Keeran Gohil mjn00u Micheal James Nolan vxp00u Viral Patel txw00u Timothy Wood ‘Final Group Report’ May 2002 Gp-gxk2 Group Project May 2002 Contents Page page 2 1. Abstract page 5 2. Introduction page 6 2.1. Initial Problem page 6 2.2. Brief Introduction page 6 3. Research page 8 3.1. Brief History of Poker page 8 3.2. Poker Variations page 8 3.3. Rules of the Game page 9 3.4. Betting Strategy page 9 3.5. AI Computer Modelling Methods page 9 3.6. GUI - Design Options page 10 4. The Back-End page 11 4.1. The Need for a Back-End page 11 4.2. Research page 11 4.3. An Example Back-End page 11 4.3.1. A working Model page 12 4.3.2. Perfect Information page 13 4.3.3. Obvious Limitations page 13 4.4. Developing the Model page 13 4.4.1 Stripping the Back-End page 13 4.4.2 Modifications to Accommodate a GUI page 14 4.4.2.1. Getting a Card page 14 4.4.2.2. Getting a Decision page15 4.4.3. Putting it all together – the Back-End Class page16 4.4.4. Putting it all together – the Poker Class page16 5. Building a Computer Poker Player page18 Poker Theory page19 Implementations page20 6. Front-End Design - GUI page21 Initial Ideas page21 Mock Up Design page21 7. Creating the GUI page22 7.1. Initial Design page22 8. Developing the GUI page24 8.1. Limitations of the Interim GUI page24 Page 2 Gp-gxk2 Group Project May 2002 8.2. Developing a Front-End page24 8.2.1. GUI layout and Features page24 8.2.2. Interaction With The User page25 8.2.3. Establishing Initial Game Data page25 8.2.4. Representing Dealt Cards page26 8.2.5. Establishing User Decisions page27 8.2.6. Board and Hold Cards page 28 8.3. Implementation page28 8.3.1. Implementation details page28 8.3.2. Back-End & Front-End Communication page29 9. Back-End & Front-End Communication page30 9.1. Background to Message Passing page30 9.2. Research page30 9.3. Control page31 9.3.1. Back-End to Front-End Message Passing page31 9.3.2. An Intermediate Class page32 9.4. Message Area page33 9.4.1. Initial Requirements page33 9.4.2. Naïve Implementation page33 9.4.3. Producer-Consumer Analogy page34 9.4.4. Building the Message Area page35 9.4.5. The Implementation page36 9.4.6. The Messages page37 9.4.6.1. Message Syntax page37 9.4.6.2. Implemented Messages page37 9.4.6.2.1. Messages to the Front-End page37 9.4.6.2.2. Messages to the Back-End page38 10. Division of Labour page39 10.1. Meetings page39 10.2. Web Site page39 10.3. Time plan page39 11. Evaluation page40 11.1. Limitations of the Software page40 11.1.1. Limitations of the GUI page40 11.1.2. Limitations of the Message Area page40 11.1.3. Limitations of the AI page41 11.2. Overall Successes of the Project page42 Page 3 Gp-gxk2 Group Project May 2002 12. Bibliography page43 Appendix page45 Appendix Ai page45 Appendix Aii page45 Appendix Bi page46 Appendix Bii page46 Appendix Biii page47 Appendix Biv page47 Appendix C page49 Page 4 Gp-gxk2 Group Project May 2002 1. Abstract Games are an interesting and challenging domain for Computer Science Research. However, developing a program to play a strategic game involves converting application of theoretical concepts to practical situations. Recently Poker has received a lot of attention from Computer Scientists and game researchers. The game is logistically simple yet strategically complex. Most importantly, Poker is a non-deterministic game with imperfect information. Due to these factors Poker provides an excellent domain for investigating problems of decision making under conditions of uncertainty. In this report we will discuss how we went about creating a high class automated Poker playing program that plays a legal game of Poker. Each section will describe how the separate points were addressed and how we sorted out any encountered problems. Also, some sections are divided into subsections giving detail of all the steps that were followed to create our high performance Poker player. Page 5 Gp-gxk2 Group Project May 2002 2. Introduction Initial Problem In this section of the report we will describe the initial task that was set for our Group project at the beginning of the academic year. Texas Hold 'Em (gp-gxk2) This year Dr Kendall is supervising two projects but both groups will be given the same project and will have to produce a piece of software that competes against the other group and, potentially, other players as well. The competition will take place at some time convenient to ourselves (possibly the open day). The group that wins this competition will be awarded a prize of £75. Your task is to develop an automated poker player that plays "Texas Hold 'Em". This is the variation of poker used in the World Series of Poker played in Las Vegas each year. At the end of the project I will arrange a competition so that the two groups can play against each other (and possibly other players) to see which group can win most (or lose less!!) chips. The group that wins this competition will be awarded a prize of £75. The competition will take place at www.paradisepoker.com, on one of their Texas Hold 'Em play tables (so you should use this as a test bed). When we run the competition we will choose a table to play at (which could have other players as well), both players will start with the same number of chips (which I will announce just before we start) and we will play for a given time period (which I will announce just before we start). Whilst the competition is underway you are not allowed to change the way the automated player plays. You are only allowed to enter commands that keeps it informed of the game status and allows it to decide what to do. As well as playing a legal (and hopefully good!!) game of poker your project should also be graphically pleasing. The graphical display should also, of course, represent the current state of the game accurately. The method you use to play poker is up to you (e.g. rule based, mathematical, heuristics, evolutionary etc.). At the start of the project I will give both teams an identical set of notes that you can use to start your research. Note - Any changes that were made to this problem description will be declared later in the report. Brief Introduction Past research efforts of the Artificial Intelligence community have concentrated on building high performance computer game programs where conditions of certain information, performance and progress can easily be measured. This was first achieved by the ‘Deep Blue Victory’ over the World Chess Champion ‘Garry Kasparov’. The success that was achieved by this program is due to the ‘Brute Force Search algorithm’. As the correlation of search speed to program performance was faster this allowed the program to succeed [ref 1]. However, in recent years, a number of interesting games have attracted the attention of the AI researchers such as ‘Poker’ and ‘Bridge’. These games have been avoided in the past as they have imperfect information (they have hidden information, unknown to opponents). Therefore, the ‘Brute Force Search’ was not successful in this area as it searches game trees hence it works for perfect information games (no hidden factors). However, Poker has a number of attributes that make it an interesting domain. These attributes include: Page 6 Gp-gxk2 Group Project May 2002 • Multiple competing agents (more than two players), • Imperfect knowledge as the opponent hold hidden cards, • Risk management that involve different betting strategies and consequences, • Agent modelling, allowing the player to identify patterns in the opponent strategy and exploring them, • Deception, which could be bluffing and dealing with unreliable information. All of these attributes are challenging dimensions to a difficult problem that attracted the computing community [ref 2]. Research started on this area few years ago, but up until now, programmers could not build a program that can play a world championship game. This is due to two reasons, firstly, the limitation of the program and the complexity of the game activities and secondly most of the research has been done on a simple type of game that includes only two players only and one round of betting. However, research continues in the field of creating a high performance Poker playing program. The remainder of this report will show the work done by our group to complete the task of building an automated Poker Player. Page 7 Gp-gxk2 Group Project May 2002 3. Research We as a group felt that this section was a very important part of the project, as it is vital that we all the facts about poker before we get stuck in to the project. Various areas of research needed to be carried out, including everything from the history of Poker to the AI modelling of Poker. Also, many members of our group had never played poker before this project was started, meaning we had to learn how to play Poker before we got started on the project to gain a better grasp of the task in hand.
Load more

Recommended publications
  • “Algoritmos Para Um Jogador Inteligente De Poker” Autor: Vinícius
    Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Florianópolis/SC, 2008 Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Orientador: Mauro Roisenberg Banca: Benjamin Luiz Franklin Banca: João Rosaldo Vollertt Junior Banca: Ricardo Azambuja Silveira Florianópolis/SC, 2008 AGRADECIMENTOS Agradeço a todos que me ajudaram no desenvolvimento deste trabalho, em especial ao professor Mauro Roisenberg e aos colegas de trabalho João Vollertt e Benjamin Luiz Franklin e ao Ricardo Azambuja Silveira pela participação na banca avaliadora. Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 4 RESUMO Poker é um jogo simples de cartas que envolve aposta, blefe e probabilidade de vencer. O objetivo foi inventar e procurar algoritmos para jogadores artificiais evolutivos e estáticos que jogassem bem poker. O jogador evolutivo criado utiliza aprendizado por reforço, onde o jogo é dividido em uma grande matriz de estados com dados de decisões e recompensas. O jogador toma a decisão que obteve a melhor recompensa média em cada estado. Para comparar a eficiência do jogador, várias disputas com jogadores que tomam decisões baseados em fórmulas simples foram feitas. Diversas disputas foram feitas para comparar a eficiência de cada algoritmo e os resultados estão demonstrados graficamente no trabalho. Palavras-Chave: Aprendizado por Reforço, Inteligência Artificial, Poker; Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 5 SUMÁRIO 1. Introdução........................................................................................................12 2. Fundamentação Teórica..................................................................................15 2.1. Regras do Poker Texas Hold'em................................................................16 2.1.1.
    [Show full text]
  • Pokerface: Emotion Based Game-Play Techniques for Computer Poker Players
    University of Kentucky UKnowledge University of Kentucky Master's Theses Graduate School 2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Lucas Cockerham University of Kentucky, [email protected] Right click to open a feedback form in a new tab to let us know how this document benefits ou.y Recommended Citation Cockerham, Lucas, "POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS" (2004). University of Kentucky Master's Theses. 224. https://uknowledge.uky.edu/gradschool_theses/224 This Thesis is brought to you for free and open access by the Graduate School at UKnowledge. It has been accepted for inclusion in University of Kentucky Master's Theses by an authorized administrator of UKnowledge. For more information, please contact [email protected]. ABSTRACT OF THESIS POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Numerous algorithms/methods exist for creating computer poker players. This thesis compares and contrasts them. A set of poker agents for the system PokerFace are then introduced. A survey of the problem of facial expression recognition is included in the hopes it may be used to build a better computer poker player. KEYWORDS: Poker, Neural Networks, Arti¯cial Intelligence, Emotion Recognition, Facial Action Coding System Lucas Cockerham 7/30/2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS By Lucas Cockerham Dr. Judy Goldsmith Dr. Grzegorz Wasilkowski 7/30/2004 RULES FOR THE USE OF THESES Unpublished theses submitted for the Master's degree and deposited in the University of Kentucky Library are as a rule open for inspection, but are to be used only with due regard to the rights of the authors.
    [Show full text]
  • An Artificial Intelligence Agent for Texas Hold'em Poker
    AN ARTIFICIAL INTELL IGENCE AGENT FOR TEXAS HOLD’EM PO KER PATRICK MCCURLEY – 0 62 4 91 7 90 2 An Artificial Intelligence Agent for Texas Hold’em Poker I declare that this document represents my own work except where otherwise stated. Signed …………………………………………………………………………. 08/05/ 2009 Patrick McCurley – 062491790 Introduction 3 TABLE OF CONTENTS 1. Introduction ................................................................................................................................................................ 7 1.1 Problem Description...................................................................................................................................... 7 1.2 Aims and Objectives....................................................................................................................................... 7 1.3 Dissertation Outline ....................................................................................................................................... 8 1.4 Ethics .................................................................................................................................................................... 8 2 Background................................................................................................................................................................10 2.1 Artificial Intelligence and Poker .............................................................................................................10 2.1.1 Problem Domain Realization .........................................................................................................10
    [Show full text]
  • Building a Champion Level Computer Poker Player
    University of Alberta Library Release Form Name of Author: Michael Bradley Johanson Title of Thesis: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player Degree: Master of Science Year this Degree Granted: 2007 Permission is hereby granted to the University of Alberta Library to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s prior written permission. Michael Bradley Johanson Date: Too much chaos, nothing gets finished. Too much order, nothing gets started. — Hexar’s Corollary University of Alberta ROBUST STRATEGIES AND COUNTER-STRATEGIES: BUILDING A CHAMPION LEVEL COMPUTER POKER PLAYER by Michael Bradley Johanson A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements for the degree of Master of Science. Department of Computing Science Edmonton, Alberta Fall 2007 University of Alberta Faculty of Graduate Studies and Research The undersigned certify that they have read, and recommend to the Faculty of Graduate Studies and Research for acceptance, a thesis entitled Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player submitted by Michael Bradley Johanson in partial fulfillment of the requirements for the degree of Master of Science. Michael Bowling Supervisor Duane Szafron Michael Carbonaro External Examiner Date: To my family: my parents Brad and Sue Johanson, and my brother, Jeff Johanson.
    [Show full text]
  • Author's Personal Copy Artificial Intelligence Computer Poker: a Review
    Author's personal copy Artificial Intelligence 175 (2011) 958–987 Contents lists available at ScienceDirect Artificial Intelligence www.elsevier.com/locate/artint Computer poker: A review Jonathan Rubin, Ian Watson ∗ Department of Computer Science, University of Auckland, New Zealand article info abstract Article history: The game of poker has been identified as a beneficial domain for current AI research Received 19 February 2010 because of the properties it possesses such as the need to deal with hidden information Accepted 21 December 2010 and stochasticity. The identification of poker as a useful research domain has inevitably Available online 21 January 2011 resulted in increased attention from academic researchers who have pursued many separate avenues of research in the area of computer poker. The poker domain has Keywords: Computer poker often featured in previous review papers that focus on games in general, however a Imperfect information games comprehensive review paper with a specific focus on computer poker has so far been Nash equilibrium lacking in the literature. In this paper, we present a review of recent algorithms and Computational game theory approaches in the area of computer poker, along with a survey of the autonomous Opponent modelling poker agents that have resulted from this research. We begin with the first serious attempts to create strong computerised poker players by constructing knowledge-based and simulation-based systems. This is followed by the use of computational game theory to construct robust poker agents and the advances that have been made in this area. Approaches to constructing exploitive agents are reviewed and the challenging problems of creating accurate and dynamic opponent models are addressed.
    [Show full text]
  • On the Construction, Maintenance and Analysis of Case-Based Strategies in Computer Poker
    On the Construction, Maintenance and Analysis of Case-Based Strategies in Computer Poker Jonathan Rubin A thesis submitted in fulfilment of the requirements for the degree of Doctor of Philosophy in Computer Science, The University of Auckland, 2013. Abstract The state-of-the-art within Artificial Intelligence has directly benefited from research con- ducted within the computer poker domain. One such success has been the the advancement of bottom up equilibrium finding algorithms via computational game theory. On the other hand, alternative top down approaches, that attempt to generalise decisions observed within a col- lection of data, have not received as much attention. In this thesis we examine top down ap- proaches that use Case-Based Reasoning in order to construct strategies within the domain of computer poker. Our analysis begins with the development of frameworks to produce static strategies that do not change during game play. We trace the evolution of our case-based ar- chitecture and evaluate the effect that modifications have on strategy performance. The end result of our experimentation is a coherent framework for producing strong case-based strate- gies based on the observation and generalisation of expert decisions. Next, we introduce three augmentation procedures that extend the initial frameworks in order to produce case-based strategies that are able to adapt to changing game conditions and exploit weaknesses of their opponents. Two of the augmentation procedures introduce different forms of opponent mod- elling into the case-based strategies produced. A further extension investigates the use of trans- fer learning in order to leverage information between separate poker sub-domains.
    [Show full text]
  • A Demonstration of the Polaris Poker System
    A Demonstration of the Polaris Poker System Michael Bowling, Nicholas Abou Risk, Nolan Bard, Darse Billings, Neil Burch, Joshua Davidson, John Hawkin, Robert Holte, Michael Johanson, Morgan Kan, Bryce Paradis, Jonathan Schaeffer, David Schnizlein, Duane Szafron, Kevin Waugh, Martin Zinkevich Department of Computing Science University of Alberta Edmonton, AB, Canada T6G 2E8 poker.cs.ualberta.ca ABSTRACT depends on the type of game being played. Two common types Poker games provide a useful testbed for modern Artificial Intel- of betting rounds are limit and no-limit. After the betting round is ligence techniques. Unlike many classical game domains such as finished, three community cards are placed face up for everyone to chess and checkers, poker includes elements of imperfect informa- use to make a hand. This round is called the flop. The flop betting tion, stochastic events, and one or more adversarial agents to inter- round starts from the small blind. Two more rounds follow, the act with. Furthermore, in poker it is possible to win or lose by vary- turn and the river, with one community card each. After the river ing degrees. Therefore, it can be advantageous to adapt ones’ strat- betting round finishes, the player with the best five card poker hand egy to exploit a weak opponent. A poker agent must address these wins the pot. Texas Hold’em refers to the described game when challenges, acting in uncertain environments and exploiting other limit betting is used. agents, in order to be highly successful. Arguably, poker games Over the last three years, the AAAI Computer Poker Compe- more closely resemble many real world problems than games with tition[5] has been used as the measuring stick for progress in the perfect information.
    [Show full text]
  • Automated Lazy Metalearning in Introspective Reasoning Systems Doctor
    Doctoral theses at NTNU, 2020:193 al thesis T or Gunnar Høst Houeland Tor Gunnar Høst Houeland Automated lazy metalearning in introspective reasoning systems Doctor ISBN 978-82-326-4734-7 (printed ver.) ISBN 978-82-326-4735-4 (electronic ver.) 1503-8181 Doctor NTNU al theses at NTNU, 2020:193 al theses at NTNU, echnology Engineering Philosophiae Doctor Thesis for the Degree of Thesis for the Degree Department of Computer Science Faculty of Information Technology and Electrical Faculty of Information Technology Norwegian University of Science and T ,.1++.6/0,1#)+" &#/'/$,.0&##%.##,$&'),/,-&'#,!0,. .,+"&#'*1+# ,.3#%'++'2#./'04,$!'#+!#+"#!&+,),%4 !1)04,$7+$,.*0',+#!&+,),%4+")#!0.'!)+%'+##.'+% #-.0*#+0,$,*-10#.!'#+!# ,.3#%'++'2#./'04,$!'#+!#+"#!&+,),%4 &#/'/$,.0&##%.##,$&'),/,-&'#,!0,. !1)04,$7+$,.*0',+#!&+,),%4+")#!0.'!)+%'+##.'+% #-.0*#+0,$,*-10#.!'#+!# 5,.1++.6/0,1#)+" 7 -.'+0#"2#. 7 #)#!0.,+'!2#. 7 ,!0,.)0&#/#/0 .'+0#" 4.$'/(/#+0#. Abstract Machine learning systems are becoming increasingly widespread and important, and these days machine learning is used in some form in most industries. However, the application of machine learning technology still has a relatively high barrier to entry, requiring both machine learning expertise and domain knowledge. In this thesis, we present a metareasoning approach to multi-method machine learning that allows the system to adapt and optimize learning for a given domain automatically, without requiring human expert judgment. In contrast to popular deep learning methods for similar situations, the approach presented here does not require specialized hardware nor large data sets. Multiple machine learning components are continuously evaluated at run-time while solving problems, using a framework to analyze overall system performance based on observed prediction performance and time spent.
    [Show full text]
  • An Investigation Into Tournament Poker Strategy Using Evolutionary Algorithms
    An Investigation into Tournament Poker Strategy using Evolutionary Algorithms Richard G. Carter I V N E R U S E I T H Y T O H F G E R D I N B U Doctor of Philosophy Centre for Intelligent Systems and their Applications School of Informatics University of Edinburgh 2007 Abstract Poker has become the subject of an increasing amount of study in the computational in- telligence community. The element of imperfect information presents new and greater challenges than those previously posed by games such as checkers and chess. Ad- vances in computer poker have great potential, since reasoning under conditions of uncertainty is typical of many real world problems. To date the focus of computer poker research has centred on the development of ring game players for limit Texas hold’em. For a computer to compete in the most prestigious poker events, however, it will be required to play in a tournament setting with a no-limit betting structure. This thesis is the first academic attempt to investigate the underlying dynamics of successful no-limit tournament poker play. Professional players have proffered advice in the non-academic poker literature on correct strategies for tournament poker play. This study seeks to empirically validate their suggestions on a simplified no-limit Texas hold’em tournament framework. Starting by using exhaustive simulations, we first assess the hypothesis that a strat- egy including information related to game-specific factors performs better than one founded on hand strength knowledge alone. Specifically, we demonstrate that the use of information pertaining to one’s seating position, the opponents’ prior actions, the stage of the tournament, and one’s chip stack size all contribute towards a statistically significant improvement in the number of tournaments won.
    [Show full text]
  • A Learning AI Algorithm for Poker with Embedded Opponent Modeling
    International Journal of Fuzzy Logic and Intelligent Systems, vol. 10, no. 3, September 2010, pp.170-177 DOI : 10.5391/IJFIS.2010.10.3.170 A Learning AI Algorithm for Poker with Embedded Opponent Modeling Seong-gon Kim* and Yong-Gi Kim** *CISE department, University of Florida, USA **Dept of Computer Science, Gyeongsang National University, Korea Corresponding author: [email protected] Abstract Poker is a game of imperfect information where competing players must deal with multiple risk factors stemming from unknown information while making the best decision to win, and this makes it an interesting test-bed for artificial intelligence research. This paper introduces a new learning AI algorithm with embedded opponent modeling that can be used for these types of situations and we use this AI and apply it to a poker program. The new AI will be based on several graphs with each of its nodes representing inputs, and the algorithm will learn the optimal decision to make by updating the weight of the edges connecting these nodes and returning a probability for each action the graphs represent. Key Word : Learning AI Algorithm, Poker, Opponent Modeling 1. Introduction 2. Texas Hold’em Artificial intelligence has long been applied to games such There are several important aspects to consider when as Chess[1], Checkers[2][3], Go[4], Othello[5][6] and other playing Texas Hold’em. The first is the Pre-Flop Evaluation. various board games with positive results in beating human Pre-flop play in Hold’em has been extensively studied in players and has gained much public attention in the process.
    [Show full text]
  • Monte Carlo Tree Search and Opponent Modeling Through Player Clustering in No-Limit Texas Hold’Em Poker
    Monte Carlo Tree Search and Opponent Modeling through Player Clustering in no-limit Texas Hold'em Poker A.A.J. van der Kleij August 2010 Master thesis Artificial Intelligence University of Groningen, The Netherlands Internal supervisors: Dr. M.A. Wiering (Artificial Intelligence, University of Groningen) Prof. Dr. L. Schomaker (Artificial Intelligence, University of Groningen) Abstract Texas Hold'em Poker is a challenging card game in which players have to deal with uncertainty, imperfect information, deception and multiple competing agents. Predicting your opponents' moves and guessing their hidden cards are key elements. A computer Poker player thus requires an opponent modeling component that performs these tasks. We propose a clustering algorithm called "K-models clustering" that clusters players based on playing style. The algorithm does not use features that describe the players, but models that predict players' moves instead. The result is a partitioning of players into clusters and models for each of these clusters. Learning to reason about the hidden cards of a player is complicated by the fact that in over 90% of the games, a player's cards are not revealed. We in- troduce an expectation-maximization based algorithm that iteratively improves beliefs over the hidden cards of players whose cards were not revealed. After convergence, we obtain models that predict players' hidden cards. We built a computer Poker player using these algorithms and Monte Carlo Tree Search: an anytime algorithm that estimates the expected values of moves using simulations. The resulting program can hold its own in 2 player games against both novices and experienced human opponents.
    [Show full text]
  • Building Poker Agent Using Reinforcement Learning with Neural Networks
    Building Poker Agent Using Reinforcement Learning with Neural Networks Annija Rupeneite Faculty of Computing, University of Latvia, 19 Raina blvd., LV-1586 Riga, Latvia Keywords: Poker Game, Reinforcement Learning, Neural Networks. Abstract: Poker is a game with incomplete and imperfect information. The ability to estimate opponent and interpret its actions makes a player as a world class player. Finding optimal game strategy is not enough to win poker game. As in real life as in online poker game, the most time of it consists of opponent analysis. This paper illustrates a development of poker agent using reinforcement learning with neural networks. 1 STAGE OF THE RESEARCH adapt opponent play style and change its strategy during the game. Poker is a game with incomplete and imperfect information. The ability to estimate opponent and interpret its actions makes a player as a world class 3 RESEARCH PROBLEM player. Finding optimal game strategy is not enough to win poker game. As in real life as in online poker Games are to AI as grand prix racing is to game, the most time of it consists of opponent automobile design. analysis. Author is working on development of poker Poker game has become a field of interest for agent that would find optimal game strategy using artificial intelligence technologies. There are some reinforcement learning (RL) in combination with concealed cards in the game that makes impossible artificial neural network (ANN) for value function to calculate the final outcome of the hand. approximation. Therefore, artificial intelligence approach is used more often to develop online poker agents.
    [Show full text]