DOCSLIB.ORG

Computers Can Now Bluff Like a Poker Champ. Better, Actually

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Computers Can Now Bluff Like a Poker Champ. Better, Actually FRIDAY, JULY 12, 2019 © 2019 Dow Jones & Company, Inc. All Rights Reserved. Computers Can Now Bluff Like a Poker Champ. Better, Actually. A new artificial intelligence program is so advanced at a key human skill — deception — that it wiped out five human poker players with one lousy hand. By Daniela Hernandez A super-powered computer performed trillions of calculations to hone an elusive skill: bluffing. Then it proceeded to wipe out five human poker champions with one lousy hand. “It [felt] very hopeless. You don’t feel like there’s anything you can do to win,” said Jason Les, who’s played poker professionally for 15 years and fell repeatedly to the poker bot’s bluffs. Researchers at Facebook Inc. and Carnegie Mellon University have built the most effective non-human bluffer to date. The bot, called Pluribus, is a leap forward in the capabilities of artificial intelligence — and also a cunning player of Texas Hold ’em. Artificial intelligence systems Carnegie Mellon University Professor of Computer Science Tuomas Sandholm, in partner- ship with researchers at Facebook Inc., developed Pluribus. The artificial intelligence developed at academic and corpo- system reasoned and bluffed in real time to best five human poker champions at their own rate labs have a disturbingly good game. PHOTO: CARNEGIE MELLON UNIVERSITY track record of beating humans at their own games. Pluribus, described two chips. By comparison, Libratus Brown, a Facebook AI research scien- in a paper published in the journal used 100 chips in its one-on-one tist and one of Pluribus’s creators. Science, follows in the digital foot- poker matches. AlphaGo, developed Multiplayer poker is considered less steps of superhuman AIs that have by Alphabet Inc.’s Google DeepMind, a game than an artform that requires vanquished humans at games like used 1,920 chips against a human a multitude of skills, especially the checkers, chess, “Jeopardy!”, Dota-2 Go player. International Business ability to read human interactions and and Go. Two years ago, another AI Machines Corp.’s Deep Blue used leverage that knowledge to exploit system developed at Carnegie Mellon, 480 custom-designed chips against mistakes and weaknesses. called Libratus, even bested a poker top chess champ Garry Kasparov, Pluribus developed its winning star. according to the paper. poker strategy and superb bluffing But Libratus beat just one human at DeepMind declined to comment. skills by playing trillions of hands a time in a two-player game. Pluribus IBM did not immediately respond to a against five other clones of itself, said beat five opponents at once — without request for comment. Dr. Brown. breaking a sweat. Pluribus couldn’t try to predict After each round, it analyzed its According to its creators, the new the endgame; playing poker against decisions. If these resulted in wins, bot uses less than 128 gigabytes of multiple opponents meant it had to be the bot would be more likely to opt for memory while playing, and ran on able to reason in real time, said Noam such moves in the future. (over please) Pluribus’s digital brain realized it of getting a flush. It still had a slim Sandholm said, but it’s much harder could win by making a bet when it chance at a straight if it got lucky with to bluff successfully when juggling had a weak hand by forcing its oppo- the final card. It chose to bluff with multiple opponents. nent to fold, which also taught it a $2,400 bet, and the last standing Poker allows researchers to test that it should bluff in future plays, human, Linus Loeliger, called. The algorithmic strategies for dealing said Dr. Brown. It then used those final community card was an eight with the unknown and to build the lessons to make real-time decisions of spades. The AI’s ace was most foundations for software that can when battling top human players, likely a losing hand, but it went all sleuth out fraud and deception in all of whom had earned more than in, betting all its chips, worth $6,550. real-world settings. Dr. Sandholm is $1 million playing professionally, Mr. Loeliger folded. He had a ten of involved in two startups, Strategy according to the paper. diamonds and a king of diamonds, a Robot Inc. and Optimized Markets “People have this notion that strong hand. He would have won. Inc., that are leveraging technology [bluffing] is a very human ability — Mr. Loeliger could not be reached similar to that baked into Pluribus that it’s about looking into the other for comment. for applications in defense, financial Unpredictability is what made services, gaming and health care. playing Pluribus difficult, profes- Facebook says it doesn’t have imme- sional poker players said. It’s also at diate plans to commercialize the tech- the core of the advance, experts said. nology. It exploits the very essence of poker, “A good AI has a ridiculously unfair uncertainty, by using math. advantage against humans: They Mr. Les, the other poker pro who don’t get tired. They don’t get hungry. lost to Pluribus and was also beaten They don’t deal with emotions,” said by its predecessor, Libratus, said the Michael “Gags” Gagliano, a profes- new bot’s moves were aggressive. AI WinnerBut can they takes learn to it love? all sional poker player with 11 years of development “is moving more rapidly experience who also lost to Pluribus. than people realize.” person’s eyes,” Dr. Brown said. “It’s Scientists are interested in building Playing successfully hinges on skill, really about math, and this is what’s AIs that can play games like poker and but also on how players deal with going on here. We can create an AI StarCraft where uncertainty abounds fatigue and stress, Mr. Gagliano said. algorithm that can bluff better than because they are microcosms of Exploiting their lack of mental and any human.” the real world, which is unpredict- physical stamina to force them to In one game against five human able. Traditionally, AI has struggled make a mistake is a big part of the players, Pluribus was dealt an ace of in uncertain situations, limiting the game. clubs and a two of clubs, not a great range of applications to which it can “The algorithm is not doing that. hand. It started by raising $250, a be applied, according to AI experts. It’s just sitting and waiting,” he said. standard move. Two humans called, In poker, “there’s hidden informa- “Any time you slip up ... it’s going to the other two folded, for a total pot tion, and to make matters worse, collect money in that situation. It’s size of $800. Three community cards, your adversary know things you don’t really difficult.” a jack of spades, a five of diamonds, know,” said Tuomas Sandholm, a Mr. Gagliano said the experience and a king of clubs, were dealt. (In Carnegie Mellon University professor made him more cognizant that poker, Texas Hold ’em, players share a set of and a Pluribus developer. “You have to like the rest of our lives, is becoming cards.) One of the humans checked. think about whether your adversary is increasingly more about the data. Pluribus bluffed and bet $800. trying to deceive you.” “It made me think about the theory Another player folded, while the other The more adversaries, the more behind poker, and the math of each called. hidden information an AI needs to situation,” said Mr. Gagliano. “Instead The next card was a three of contend with. Previous poker-playing of playing the player, you’re sticking hearts, which killed the bot’s chances bots also had the ability to bluff, Dr. to the stats.” THE PUBLISHER’S SALE OF THIS REPRINT DOES Not CONSTITUTE OR IMPLY ANY ENdoRSEMENT OR SpoNSORSHIP OF ANY PRODUct, SERVICE, COMPANY OR ORGANIZATION. Custom Reprints 800.843.0008 www.djreprints.com DO NOT EDIT OR ALTER REPRINT/REPRODUCTIONS NOT PERMITTED 56714.
Load more

Recommended publications
  • Most Important Fundamental Rule of Poker Strategy
    The Thirty-Third International FLAIRS Conference (FLAIRS-33) Most Important Fundamental Rule of Poker Strategy Sam Ganzfried,1 Max Chiswick 1Ganzfried Research Abstract start playing very quickly, even the best experts spend many years (for some an entire lifetime) learning and improving. Poker is a large complex game of imperfect information, Humans learn and improve in a variety of ways. The most which has been singled out as a major AI challenge prob- obvious are reading books, hiring coaches, poker forums and lem. Recently there has been a series of breakthroughs culmi- nating in agents that have successfully defeated the strongest discussions, and simply playing a lot to practice. Recently human players in two-player no-limit Texas hold ’em. The several popular software tools have been developed, most 1 strongest agents are based on algorithms for approximating notably PioSolver, where players solve for certain situa- Nash equilibrium strategies, which are stored in massive bi- tions, given assumptions for the hands the player and oppo- nary files and unintelligible to humans. A recent line of re- nent can have. This is based on the new concept of endgame search has explored approaches for extrapolating knowledge solving (Ganzfried and Sandholm 2015), where strategies from strong game-theoretic strategies that can be understood are computed for the latter portion of the game given fixed by humans. This would be useful when humans are the ulti- strategies for the trunk (which are input by the user). While mate decision maker and allow humans to make better deci- it has been pointed out that theoretically this approach is sions from massive algorithmically-generated strategies.
    [Show full text]
  • Texas Hold'em Poker
    11/5/2018 Rules of Card Games: Texas Hold'em Poker Pagat Poker Poker Rules Poker Variants Home Page > Poker > Variations > Texas Hold'em DE EN Choose your language Texas Hold'em Introduction Players and Cards The Deal and Betting The Showdown Strategy Variations Pineapple ‑ Crazy Pineapple ‑ Crazy Pineapple Hi‑Lo Irish Casino Versions Introduction Texas Hold'em is a shared card poker game. Each player is dealt two private cards and there are five face up shared (or "community") cards on the table that can be used by anyone. In the showdown the winner is the player who can make the best five‑card poker hand from the seven cards available. Since the 1990's, Texas Hold'em has become one of the most popular poker games worldwide. Its spread has been helped firstly by a number of well publicised televised tournaments such as the World Series of Poker and secondly by its success as an online game. For many people nowadays, poker has become synonymous with Texas Hold'em. This page assumes some familiarity with the general rules and terminology of poker. See the poker rules page for an introduction to these, and the poker betting and poker hand ranking pages for further details. Players and Cards From two to ten players can take part. In theory more could play, but the game would become unwieldy. A standard international 52‑card pack is used. The Deal and Betting Texas Hold'em is usually played with no ante, but with blinds. When there are more than two players, the player to dealer's left places a small blind, and the next player to the left a big blind.
    [Show full text]
  • Early Round Bluffing in Poker Author(S): California Jack Cassidy Source: the American Mathematical Monthly, Vol
    Early Round Bluffing in Poker Author(s): California Jack Cassidy Source: The American Mathematical Monthly, Vol. 122, No. 8 (October 2015), pp. 726-744 Published by: Mathematical Association of America Stable URL: http://www.jstor.org/stable/10.4169/amer.math.monthly.122.8.726 Accessed: 23-12-2015 19:20 UTC Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at http://www.jstor.org/page/ info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact [email protected]. Mathematical Association of America is collaborating with JSTOR to digitize, preserve and extend access to The American Mathematical Monthly. http://www.jstor.org This content downloaded from 128.32.135.128 on Wed, 23 Dec 2015 19:20:53 UTC All use subject to JSTOR Terms and Conditions Early Round Bluffing in Poker California Jack Cassidy Abstract. Using a simplified form of the Von Neumann and Morgenstern poker calculations, the author explores the effect of hand volatility on bluffing strategy, and shows that one should never bluff in the first round of Texas Hold’Em. 1. INTRODUCTION. The phrase “the mathematics of bluffing” often brings a puzzled response from nonmathematicians. “Isn’t that an oxymoron? Bluffing is psy- chological,” they might say, or, “Bluffing doesn’t work in online poker.
    [Show full text]
  • Abiding Chance: Online Poker and the Software of Self-Discipline
    ESSAYS Abiding Chance: Online Poker and the Software of Self- Discipline Natasha Dow Schüll A man sits before a large desktop monitor station, the double screen divided into twenty- four rectangles of equal size, each containing the green oval of a poker table with positions for nine players. The man is virtu- ally “seated” at all twenty- four tables, along with other players from around the world. He quickly navigates his mouse across the screen, settling for moments at a time on flashing windows where his input is needed to advance play at a given table. His rapid- fire esponsesr are enabled by boxed panels of colored numbers and letters that float above opponents’ names; the letters are acronyms for behavioral tendencies relevant to poker play, and the numbers are statistical scores identifying where each player falls in a range for those tendencies. Taken together, the letters and numbers supply the man with enough information to act strategically at a rate of hundreds of hands per hour. Postsession, the man opens his play- tracking database to make sure the software has successfully imported the few thousand hands he has just played. After quickly scrolling through to ensure that they are all there, he recalls some particularly challenging hands he would like to review and checks a number Thanks to Paul Rabinow and Limor Samimian- Darash, for prompting me to gather this material for a different article, and to Richard Fadok, Paul Gardner, Lauren Kapsalakis, and the students in my 2013 Self as Data graduate seminar at the Massachusetts Institute of Technology, for helping me to think through that material.
    [Show full text]
  • An Artificial Intelligence Agent for Texas Hold'em Poker
    AN ARTIFICIAL INTELL IGENCE AGENT FOR TEXAS HOLD’EM PO KER PATRICK MCCURLEY – 0 62 4 91 7 90 2 An Artificial Intelligence Agent for Texas Hold’em Poker I declare that this document represents my own work except where otherwise stated. Signed …………………………………………………………………………. 08/05/ 2009 Patrick McCurley – 062491790 Introduction 3 TABLE OF CONTENTS 1. Introduction ................................................................................................................................................................ 7 1.1 Problem Description...................................................................................................................................... 7 1.2 Aims and Objectives....................................................................................................................................... 7 1.3 Dissertation Outline ....................................................................................................................................... 8 1.4 Ethics .................................................................................................................................................................... 8 2 Background................................................................................................................................................................10 2.1 Artificial Intelligence and Poker .............................................................................................................10 2.1.1 Problem Domain Realization .........................................................................................................10
    [Show full text]
  • Building a Poker Playing Agent Based on Game Logs Using Supervised Learning
    FACULDADE DE ENGENHARIA DA UNIVERSIDADE DO PORTO Building a Poker Playing Agent based on Game Logs using Supervised Learning Luís Filipe Guimarães Teófilo Mestrado Integrado em Engenharia Informática e Computação Orientador: Professor Doutor Luís Paulo Reis 26 de Julho de 2010 Building a Poker Playing Agent based on Game Logs using Supervised Learning Luís Filipe Guimarães Teófilo Mestrado Integrado em Engenharia Informática e Computação Aprovado em provas públicas pelo Júri: Presidente: Professor Doutor António Augusto de Sousa Vogal Externo: Professor Doutor José Torres Orientador: Professor Doutor Luís Paulo Reis ____________________________________________________ 26 de Julho de 2010 iv Resumo O desenvolvimento de agentes artificiais que jogam jogos de estratégia provou ser um domínio relevante de investigação, sendo que investigadores importantes na área das ciências de computadores dedicaram o seu tempo a estudar jogos como o Xadrez e as Damas, obtendo resultados notáveis onde o jogador artificial venceu os melhores jogadores humanos. No entanto, os jogos estocásticos com informação incompleta trazem novos desafios. Neste tipo de jogos, o agente tem de lidar com problemas como a gestão de risco ou o tratamento de informação não fiável, o que torna essencial modelar adversários, para conseguir obter bons resultados. Nos últimos anos, o Poker tornou-se um fenómeno de massas, sendo que a sua popularidade continua a aumentar. Na Web, o número de jogadores aumentou bastante, assim como o número de casinos online, tornando o Poker numa indústria bastante rentável. Além disso, devido à sua natureza estocástica de informação imperfeita, o Poker provou ser um problema desafiante para a inteligência artificial. Várias abordagens foram seguidas para criar um jogador artificial perfeito, sendo que já foram feitos progressos nesse sentido, como o melhoramento das técnicas de modelação de oponentes.
    [Show full text]
  • Optimal Strategy Hand-Rank Table for Jacks Or Better, Double Bonus, and Joker Wild Video Poker
    Optimal Strategy Hand-rank Table for Jacks or Better, Double Bonus, and Joker Wild Video Poker By John Jungtae Kim A Thesis Submitted to the School of Graduate Studies in Partial Fulfilment of the Requirements for the Degree Master of Science McMaster University c Copyright by John Jungtae Kim, January 2012 MASTER OF SCIENCE (2012) McMaster University (Statistics) Hamilton, Ontario TITLE: The Optimal Strategy Hand-rank Table for Jacks or Better, Double Bonus, and Joker Wild Video Poker AUTHOR: John Jungtae Kim (McMaster University, Canada) SUPERVISOR: Dr. Fred M. Hoppe NUMBER OF PAGES: viii, 76 i Abstract Video poker is a casino game based on five-card draw poker played on a computerized console. Video poker allows players an opportunity for some control of the random events that determine whether they win or lose. This means that making the right play can increase a player's return in the long run. For that reason, optimal strategy hand-rank tables for various types of video poker games have been recently published and established to help players improve their return (Ethier, 2010). Ethier posed a number of open problems in his recent book, The Doctrine of Chances: Probabilitistic Aspects of Gambling among which were some in video poker. In this thesis we consider the most popular video poker games: Jacks or Better, Double Bonus, and Joker Wild. Ethier produced an optimal strategy hand-rank table for Jacks or Better. We expand on his method to produce optimal hand-rank tables for Double Bonus,and Joker Wild. The method involves enumerating all possible discards, computing the expected returns, and then finding a way to rank them according to optimal discard based on the payoffs.
    [Show full text]
  • Building a Champion Level Computer Poker Player
    University of Alberta Library Release Form Name of Author: Michael Bradley Johanson Title of Thesis: Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player Degree: Master of Science Year this Degree Granted: 2007 Permission is hereby granted to the University of Alberta Library to reproduce single copies of this thesis and to lend or sell such copies for private, scholarly or scientific research purposes only. The author reserves all other publication and other rights in association with the copyright in the thesis, and except as herein before provided, neither the thesis nor any substantial portion thereof may be printed or otherwise reproduced in any material form whatever without the author’s prior written permission. Michael Bradley Johanson Date: Too much chaos, nothing gets finished. Too much order, nothing gets started. — Hexar’s Corollary University of Alberta ROBUST STRATEGIES AND COUNTER-STRATEGIES: BUILDING A CHAMPION LEVEL COMPUTER POKER PLAYER by Michael Bradley Johanson A thesis submitted to the Faculty of Graduate Studies and Research in partial fulfillment of the requirements for the degree of Master of Science. Department of Computing Science Edmonton, Alberta Fall 2007 University of Alberta Faculty of Graduate Studies and Research The undersigned certify that they have read, and recommend to the Faculty of Graduate Studies and Research for acceptance, a thesis entitled Robust Strategies and Counter-Strategies: Building a Champion Level Computer Poker Player submitted by Michael Bradley Johanson in partial fulfillment of the requirements for the degree of Master of Science. Michael Bowling Supervisor Duane Szafron Michael Carbonaro External Examiner Date: To my family: my parents Brad and Sue Johanson, and my brother, Jeff Johanson.
    [Show full text]
  • Online Poker Statistics Guide a Comprehensive Guide for Online Poker Players
    Online Poker Statistics Guide A comprehensive guide for online poker players This guide was created by a winning poker player on behalf of Poker Copilot. ​ ​ Poker Copilot is a tool that automatically records your opponents' poker statistics, ​ and shows them onscreen in a HUD (head-up display) while you play. Version 2. Updated April, 2017 Table of Contents Online Poker Statistics Guide 5 Chapter 1: VPIP and PFR 5 Chapter 2: Unopened Preflop Raise (UOPFR) 5 Chapter 3: Blind Stealing 5 Chapter 4: 3-betting and 4-betting 6 Chapter 5: Donk Bets 6 Chapter 6: Continuation Bets (cbets) 6 Chapter 7: Check-Raising 7 Chapter 8: Squeeze Bet 7 Chapter 9: Big Blinds Remaining 7 Chapter 10: Float Bets 7 Chapter 1: VPIP and PFR 8 What are VPIP and PFR and how do they affect your game? 8 VPIP: Voluntarily Put In Pot 8 PFR: Preflop Raise 8 The relationship between VPIP and PFR 8 Identifying player types using VPIP/PFR 9 VPIP and PFR for Six-Max vs. Full Ring 10 Chapter 2: Unopened Preflop Raise (UOPFR) 12 What is the Unopened Preflop Raise poker statistic? 12 What is a hand range? 12 What is a good UOPFR for beginners from each position? 12 How to use Equilab hand charts 13 What about the small and big blinds? 16 When can you widen your UOPFR range? 16 Flat calling using UOPFR 16 Flat calling with implied odds 18 Active players to your left reduce your implied odds 19 Chapter 3: Blind Stealing 20 What is a blind steal? 20 Why is the blind-stealing poker statistic important? 20 Choosing a bet size for a blind steal 20 How to respond to a blind steal
    [Show full text]
  • Check/Raise Flush Draws in the Middle
    P G RULES FOR 20PLAYING FLUSH DRAWS IN 2018 You are about to read some of the secrets Ryan Fee and I (Doug Polk) have used to separate us from your average poker player. RYAN FEE DOUG POLK We, like many players, used to aimlessly bet the flop every time we had a flush draw without much of a plan for the turn and river and with little consideration for the impact it had on the rest of our range. (Sound familiar…?) After spending years refining and optimizing our games we have deduced a methodology to playing flush draws that is balanced, sneaky, and let’s us fight for pots where other players aren’t even looking. By following these rules you will make more money in two ways: 1 More often, you will make better hands fold when bluffing, worse hands call when value betting, and put in the minimum when we are behind. 2 Most players are still behind the curve and play most of, if not all of their flush draws the same on the flop. You will make chips by having bluffs and value bets in spots your opponents do not expectP and are not prepared for. G 20 RULES FOR PLAYING FLUSH DRAWS 1 RULE #1 Ask yourself “If my hand wasn’t a flush draw, how would I play it?” Chances are you should play the flush draw the same way. Example: j t 2 If you T 9 on would normally X check t 9 you should also check XX RULE #2 Check the nut flush draw most of the time, except in instances where it is a very strong hand and you are borderline value-betting.
    [Show full text]
  • Doyle Brunson's Super System
    Doyle Brunson's Super System Chapter One - General Poker Strategy • Introduction • Tells • ESP • Superstitions • Honor • Being Competitive • Important Tips • Money Management • Courage and Other Important Qualities • Vacations • Being Versatile • Tournament Play Chapter Two • Hold'em - Limit and No-Limit: Similarities and Differences Chapter Three - No Limit Hold'em • Introduction o Part 1 o Part 2 o Part 3 • Ante and Other Considerations • Big Pairs (AA/KK) • Ace King • Queens • Small Pairs (Jacks down to Deuces) • Small Connecting Cards • Borderline Hands • Trash Hands • Short Handed Play • Insurance • A Final Word Chapter One: General Poker Strategy - Introduction Poker is a game of people. That's the most important lesson you should learn from my book. I'll be teaching you guidelines and concepts you'll be able to use with great success, and you'll quickly become a very good player. But, if your ambition is to become a great player, a top flight Pro, a superstar ... you'll need to really understand your opponents. You'll need to get inside your opponent's head and be able to estimate with a high degree of certainty what his check, bet or raise really means ... and what hand he's likely to be playing. Being able to do that accurately isn't easy. But you can do it if you're alert, observant, disciplined and if you concentrate whenever you play (whether or not you're involved in the pot). Using my advice and the advice of my expert collaborators, you'll find that the "task" of unmasking your Poker faced opponents will become easier and easier.
    [Show full text]
  • Identifying Player's Strategies in No Limit Texas Hold'em Poker Through
    EPIA'2011 ISBN: 978-989-95618-4-7 Identifying Player’s Strategies in No Limit Texas Hold’em Poker through the Analysis of Individual Moves Luís Filipe Teófilo and Luís Paulo Reis Departamento de Engenharia Informática, Faculdade de Engenharia da Universidade do Porto, Portugal Laboratório de Inteligência Artificial e de Ciência de Computadores, Universidade do Porto, Portugal [email protected], [email protected] Abstract. The development of competitive artificial Poker playing agents has proven to be a challenge, because agents must deal with unreliable information and deception which make it essential to model the opponents in order to achieve good results. This paper presents a methodology to develop opponent modeling techniques for Poker agents. The approach is based on applying clustering algorithms to a Poker game database in order to identify player types based on their actions. First, common game moves were identified by clustering all players’ moves. Then, player types were defined by calculating the frequency with which the players perform each type of movement. With the given dataset, 7 different types of players were identified with each one having at least one tactic that characterizes him. The identification of player types may improve the overall performance of Poker agents, because it helps the agents to predict the opponent’s moves, by associating each opponent to a distinct cluster. Keywords: Poker, Clustering, Opponent Modeling, Expectation-maximization 1 Introduction Poker is a game that is became a field of interest for the AI research community on the last decade. This game presents a radically different challenge when compared to other games like chess or checkers.
    [Show full text]