DOCSLIB.ORG

Computing Strong Game-Theoretic Strategies and Exploiting Suboptimal Opponents in Large Games Sam Ganzfried CMU-CS-15-104 May 2015

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Computing Strong Game-Theoretic Strategies and Exploiting Suboptimal Opponents in Large Games Sam Ganzfried CMU-CS-15-104 May 2015 Computing Strong Game-Theoretic Strategies and Exploiting Suboptimal Opponents in Large Games Sam Ganzfried CMU-CS-15-104 May 2015 School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213 Thesis Committee: Tuomas Sandholm, Chair Avrim Blum Geoffrey Gordon Michael Bowling, University of Alberta Vincent Conitzer, Duke University Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy. Copyright c 2015 Sam Ganzfried This research was sponsored by the National Science Foundation under grant numbers IIS-0427858, IIS-0905390, IIS-0964579, CCF-1101668, CCR-0122581, and IIS-1320620, as well as XSEDE computing resources provided by the Pittsburgh Supercomputing Center. I also acknowledge Intel Corporation and IBM for their machine gifts. The views and conclusions contained in this document are those of the author and should not be interpreted as representing the official policies, either expressed or implied, of any sponsoring institution, the U.S. government, or any other entity. Keywords: artificial intelligence, game theory, multiagent systems, game solving, game abstraction, equilibrium computation, qualitative models, opponent modeling. Abstract Designing successful agents for multiagent strategic settings is a challenging problem for several reasons. First, many games of interest are far too large to be solved (for a relevant game-theoretic solution concept) by the best current algo- rithms. For example, no-limit Texas hold ’em has approximately 10165 nodes in its game tree, while the best algorithms for computing a Nash equilibrium only scale to games with around 1017 states. A second challenge is that it is not even clear that our goal should be computing a Nash equilibrium in the first place. In games with more than two players (or two-player games that are not zero sum (competitive)), playing a Nash equilibrium has no performance guarantee. Furthermore, even in two-player zero-sum games, we can often obtain significantly higher payoffs by learning to ex- ploit mistakes of a suboptimal opponent than by playing a Nash equilibrium. The leading paradigm for addressing the first challenge is to first approximate the full game with a strategically similar but significantly smaller game, and then to solve this smaller abstract game. All of this computation is done offline in advance, and the strategies are then looked up in a table for actual game play. We have de- veloped new algorithms for improving each step of this paradigm. Specifically, I present new algorithms for performing game abstraction and for computing equilib- ria in several game classes, as well as new approaches for addressing the problem of interpreting actions for the opponent that have been removed from the abstraction and further post-processing techniques that achieve robustness against limitations of the abstraction and equilibrium-finding phases. I then describe two new game-solving paradigms: in the first, relevant portions of the game are solved in real time to a better degree of accuracy than the abstract game, which is solved offline according to the leading paradigm, and in the second, qualitative representations of the structure of equilibrium strategies are leveraged to improve the speed of equilibrium finding. The latter paradigm can be utilized to ob- tain human-understandable knowledge from strategies, which are often represented as massive binary files, thereby enabling improved human decision-making. In the final portion of the thesis, I address the second challenge by presenting new algorithms for effectively learning to exploit unknown static opponents in large imperfect-information games after only a small number of interactions. Further- more, I present new algorithms for exploiting weak opponents that are able to guar- antee a good worst-case performance even against strong dynamic opponents. The approaches are domain independent and apply to any games within very broad classes, which include many important real-world situations. While most of the approaches are presented in the context of two-player zero-sum games, they also apply to games with more than two agents, though in some cases this results in a modification of theoretical guarantees. One application domain that was considered was two-player no-limit Texas hold ’em. Several of the approaches were utilized to create an agent that came in first place in the most recent (2014) AAAI Annual Com- puter Poker Competition, beating each opposing agent with statistical significance. iv Acknowledgments I would like to thank my advisor Tuomas Sandholm for allowing me to pur- sue an extremely ambitious research agenda; a more conservative one would have been significantly less fulfilling. I am glad that we ended up agreeing on Claudico as the name of our man-machine competition agent as opposed to some of Profes- sor Sandholm’s earlier suggestions, which included Carnegietastic, Finntastic, and Sharknegie—I don’t think those would have had quite the same ring! I would like to thank Dong Kim, Jason Les, Bjorn Li, and Doug Polk for taking it easy on Claudico so that our loss was slightly below the value needed to be statistically significant at the 95% confidence level! Claudico may not have been possible without the groundwork that was paved by Andrew Gilpin, my predecessor on the poker project. The “workhorse” file is still called “DoylesGame.java,” even though the no-limit competition has not been 500 BBs since 2008! Claudico also certainly benefited from having Noam Brown on the team, and I am confident that I am leaving the code in extremely capable hands at my departure. Rest assured Troels Sørenson your contributions have not been forgotten, and your code has been pivotal in producing some of the finest betting abstractions in the world to this day. I also enjoyed our conversations on various equilibrium re- finements, including trembling-hand perfect equilibrium, quasi-perfect equilibrium, and proper equilibrium (both the normal-form and extensive-form variants!) The same goes for Branislav Bosansky, Jir˘´ı Cerm˘ ak,´ and Nicola Gatti. I would like to thank the other members of my thesis committee—Avrim Blum, Michael Bowling, Vincent Conitzer, and Geoffrey Gordon, as well as Kevin Leyton- Brown who was on my thesis proposal committee. I appreciate the time, effort, and thoughtfulness you each put into providing valuable feedback. I hope that Geoff was not too devastated to learn that the equilibrium strategies he had computed for one-card poker that had been on his website for many years were actually (weakly) dominated! It was a close call, but Vince wins the award for longest distance tra- versed to attend—both for the proposal and the defense! I would also like to thank Michael Wellman for insightful discussions—it’s too bad that purification has not yet proven to be as helpful for trading agents as it has for poker agents. I enjoyed working with Ariel Procaccia on a project for optimizing weights for exam questions to minimize the average squared error difference from the homework average, which we used as a proxy for the students’ “true ability.” It is too bad that much of the improvements of the techniques we considered disappeared when all the scores were normalized by the average, but I am optimistic that there is a bright future ahead for this line of inquiry. I would like to thank everyone who has been a part of the poker competition and workshop over the years (am I the only person who has attended all four events?). I’d particularly like to thank Michael Johanson, who was extremely helpful in pro- viding details on the information abstraction algorithms used by Alberta’s poker agents. Without his numerous detailed emails, it would have taken me eons longer to comprehend the differences between perfect and imperfect-recall abstraction, not to mention distribution-aware abstraction, the merits of earth-mover’s distance over L2, and “OCHS.” It was a pleasure co-organizing the workshop with Eric Jackson in 2014, and I am very appreciative of Nolan Bard, Neil Burch, Eric Jackson, Jonathan Rubin, and Kevin Waugh for their effort running the computer poker competition, as well as Adrian Koy, Jon Parker, and Tim Reiff for many enjoyable conversations. I would also like to thank Abe Othman (aka “othasaurus”), Daniel Miller (aka “millertie”), Winslow Strong (aka “WINSLOW STRONG”), and Raja “fitbit” Sam- basivan for entertaining all my wacky “startup” ideas. I just know that “SHPRATZ” will go bigtime. I will never forget the dentist of Italian descent who rejoined at just the proper time, shrieking “Oh No!” after ripping backhands that exceeded the base- line by not more than two inches. Thank you HH for informing me that it probably isn’t the greatest idea to wear a t-shirt in my main website/profile photo—they don’t teach these things in grad school! Andrew “azntracker” Li, now that you are retired, I feel ok telling you that you call wayyy too wide on the bubble, jeez. Deb Cavlovich and Charlotte Yano have done far more than necessary to ensure the logistical aspects of my time at CMU proceeded impeccably. And finally, a big thank you to Sally, “Big J,” and of course “the cowboy,” among other things, for all the couscous. vi Contents I Introduction 1 1 Overview 3 2 Game Theory Background 5 2.1 Extensive-form games . 7 2.2 Repeated games . 9 2.3 Other game representations . 9 3 Poker 11 4 Leading Game-Solving Paradigm 13 II New Approaches for Game Solving within the Leading Paradigm 17 5 Potential-Aware Imperfect-Recall Abstraction with Earth Mover’s Distance in Imperfect-Information Games 19 5.1 Potential-aware abstraction . 21 5.1.1 Domain-independent example . 22 5.1.2 Poker example . 23 5.2 Algorithm for potential-aware imperfect-recall abstraction, with EMD .
Load more

Recommended publications
  • “Algoritmos Para Um Jogador Inteligente De Poker” Autor: Vinícius
    Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Florianópolis/SC, 2008 Universidade Federal De Santa Catarina Centro Tecnológico Bacharelado em Ciências da Computação “Algoritmos para um jogador inteligente de Poker” Autor: Vinícius Sousa Fazio Orientador: Mauro Roisenberg Banca: Benjamin Luiz Franklin Banca: João Rosaldo Vollertt Junior Banca: Ricardo Azambuja Silveira Florianópolis/SC, 2008 AGRADECIMENTOS Agradeço a todos que me ajudaram no desenvolvimento deste trabalho, em especial ao professor Mauro Roisenberg e aos colegas de trabalho João Vollertt e Benjamin Luiz Franklin e ao Ricardo Azambuja Silveira pela participação na banca avaliadora. Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 4 RESUMO Poker é um jogo simples de cartas que envolve aposta, blefe e probabilidade de vencer. O objetivo foi inventar e procurar algoritmos para jogadores artificiais evolutivos e estáticos que jogassem bem poker. O jogador evolutivo criado utiliza aprendizado por reforço, onde o jogo é dividido em uma grande matriz de estados com dados de decisões e recompensas. O jogador toma a decisão que obteve a melhor recompensa média em cada estado. Para comparar a eficiência do jogador, várias disputas com jogadores que tomam decisões baseados em fórmulas simples foram feitas. Diversas disputas foram feitas para comparar a eficiência de cada algoritmo e os resultados estão demonstrados graficamente no trabalho. Palavras-Chave: Aprendizado por Reforço, Inteligência Artificial, Poker; Algoritmos para um jogador inteligente de Poker – Vinícius Sousa Fazio 5 SUMÁRIO 1. Introdução........................................................................................................12 2. Fundamentação Teórica..................................................................................15 2.1. Regras do Poker Texas Hold'em................................................................16 2.1.1.
    [Show full text]
  • Going Beyond Dual Execution: MPC for Functions with Efficient Verification
    Going Beyond Dual Execution: MPC for Functions with Efficient Verification Carmit Hazay abhi shelat ∗ Bar-Ilan University Northeastern Universityy Muthuramakrishnan Venkitasubramaniamz University of Rochester Abstract The dual execution paradigm of Mohassel and Franklin (PKC’06) and Huang, Katz and Evans (IEEE ’12) shows how to achieve the notion of 1-bit leakage security at roughly twice the cost of semi-honest security for the special case of two-party secure computation. To date, there are no multi-party compu- tation (MPC) protocols that offer such a strong trade-off between security and semi-honest performance. Our main result is to address this shortcoming by designing 1-bit leakage protocols for the multi- party setting, albeit for a special class of functions. We say that function f (x, y) is efficiently verifiable by g if the running time of g is always smaller than f and g(x, y, z) = 1 if and only if f (x, y) = z. In the two-party setting, we first improve dual execution by observing that the “second execution” can be an evaluation of g instead of f , and that by definition, the evaluation of g is asymptotically more efficient. Our main MPC result is to construct a 1-bit leakage protocol for such functions from any passive protocol for f that is secure up to additive errors and any active protocol for g. An important result by Genkin et al. (STOC ’14) shows how the classic protocols by Goldreich et al. (STOC ’87) and Ben-Or et al. (STOC ’88) naturally support this property, which allows to instantiate our compiler with two-party and multi-party protocols.
    [Show full text]
  • Equilibrium Refinements
    Equilibrium Refinements Mihai Manea MIT Sequential Equilibrium I In many games information is imperfect and the only subgame is the original game. subgame perfect equilibrium = Nash equilibrium I Play starting at an information set can be analyzed as a separate subgame if we specify players’ beliefs about at which node they are. I Based on the beliefs, we can test whether continuation strategies form a Nash equilibrium. I Sequential equilibrium (Kreps and Wilson 1982): way to derive plausible beliefs at every information set. Mihai Manea (MIT) Equilibrium Refinements April 13, 2016 2 / 38 An Example with Incomplete Information Spence’s (1973) job market signaling game I The worker knows her ability (productivity) and chooses a level of education. I Education is more costly for low ability types. I Firm observes the worker’s education, but not her ability. I The firm decides what wage to offer her. In the spirit of subgame perfection, the optimal wage should depend on the firm’s beliefs about the worker’s ability given the observed education. An equilibrium needs to specify contingent actions and beliefs. Beliefs should follow Bayes’ rule on the equilibrium path. What about off-path beliefs? Mihai Manea (MIT) Equilibrium Refinements April 13, 2016 3 / 38 An Example with Imperfect Information Courtesy of The MIT Press. Used with permission. Figure: (L; A) is a subgame perfect equilibrium. Is it plausible that 2 plays A? Mihai Manea (MIT) Equilibrium Refinements April 13, 2016 4 / 38 Assessments and Sequential Rationality Focus on extensive-form games of perfect recall with finitely many nodes. An assessment is a pair (σ; µ) I σ: (behavior) strategy profile I µ = (µ(h) 2 ∆(h))h2H: system of beliefs ui(σjh; µ(h)): i’s payoff when play begins at a node in h randomly selected according to µ(h), and subsequent play specified by σ.
    [Show full text]
  • László Lovász Avi Wigderson De L’Université Eötvös Loránd À De L’Institute for Advanced Study De Budapest, En Hongrie Et À Princeton, Aux États-Unis
    2021 L’Académie des sciences et des lettres de Norvège a décidé de décerner le prix Abel 2021 à László Lovász Avi Wigderson de l’université Eötvös Loránd à de l’Institute for Advanced Study de Budapest, en Hongrie et à Princeton, aux États-Unis, « pour leurs contributions fondamentales à l’informatique théorique et aux mathématiques discrètes, et pour leur rôle de premier plan dans leur transformation en domaines centraux des mathématiques contemporaines ». L’informatique théorique est l’étude de la puissance croissante sur plusieurs autres sciences, ce et des limites du calcul. Elle trouve son origine qui permet de faire de nouvelles découvertes dans les travaux fondateurs de Kurt Gödel, Alonzo en « chaussant des lunettes d’informaticien ». Church, Alan Turing et John von Neumann, qui ont Les structures discrètes telles que les graphes, conduit au développement de véritables ordinateurs les chaînes de caractères et les permutations physiques. L’informatique théorique comprend deux sont au cœur de l’informatique théorique, et les sous-disciplines complémentaires : l’algorithmique mathématiques discrètes et l’informatique théorique qui développe des méthodes efficaces pour une ont naturellement été des domaines étroitement multitude de problèmes de calcul ; et la complexité, liés. Certes, ces deux domaines ont énormément qui montre les limites inhérentes à l’efficacité des bénéficié des champs de recherche plus traditionnels algorithmes. La notion d’algorithmes en temps des mathématiques, mais on constate également polynomial mise en avant dans les années 1960 par une influence croissante dans le sens inverse. Alan Cobham, Jack Edmonds et d’autres, ainsi que Les applications, concepts et techniques de la célèbre conjecture P≠NP de Stephen Cook, Leonid l’informatique théorique ont généré de nouveaux Levin et Richard Karp ont eu un fort impact sur le défis, ouvert de nouvelles directions de recherche domaine et sur les travaux de Lovász et Wigderson.
    [Show full text]
  • Pokerface: Emotion Based Game-Play Techniques for Computer Poker Players
    University of Kentucky UKnowledge University of Kentucky Master's Theses Graduate School 2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Lucas Cockerham University of Kentucky, [email protected] Right click to open a feedback form in a new tab to let us know how this document benefits ou.y Recommended Citation Cockerham, Lucas, "POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS" (2004). University of Kentucky Master's Theses. 224. https://uknowledge.uky.edu/gradschool_theses/224 This Thesis is brought to you for free and open access by the Graduate School at UKnowledge. It has been accepted for inclusion in University of Kentucky Master's Theses by an authorized administrator of UKnowledge. For more information, please contact [email protected]. ABSTRACT OF THESIS POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS Numerous algorithms/methods exist for creating computer poker players. This thesis compares and contrasts them. A set of poker agents for the system PokerFace are then introduced. A survey of the problem of facial expression recognition is included in the hopes it may be used to build a better computer poker player. KEYWORDS: Poker, Neural Networks, Arti¯cial Intelligence, Emotion Recognition, Facial Action Coding System Lucas Cockerham 7/30/2004 POKERFACE: EMOTION BASED GAME-PLAY TECHNIQUES FOR COMPUTER POKER PLAYERS By Lucas Cockerham Dr. Judy Goldsmith Dr. Grzegorz Wasilkowski 7/30/2004 RULES FOR THE USE OF THESES Unpublished theses submitted for the Master's degree and deposited in the University of Kentucky Library are as a rule open for inspection, but are to be used only with due regard to the rights of the authors.
    [Show full text]
  • The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space
    The Flajolet-Martin Sketch Itself Preserves Differential Privacy: Private Counting with Minimal Space Adam Smith Shuang Song Abhradeep Thakurta Boston University Google Research, Brain Team Google Research, Brain Team [email protected] [email protected] [email protected] Abstract We revisit the problem of counting the number of distinct elements F0(D) in a data stream D, over a domain [u]. We propose an ("; δ)-differentially private algorithm that approximates F0(D) within a factor of (1 ± γ), and with additive error of p O( ln(1/δ)="), using space O(ln(ln(u)/γ)/γ2). We improve on the prior work at least quadratically and up to exponentially, in terms of both space and additive p error. Our additive error guarantee is optimal up to a factor of O( ln(1/δ)), n ln(u) 1 o and the space bound is optimal up to a factor of O min ln γ ; γ2 . We assume the existence of an ideal uniform random hash function, and ignore the space required to store it. We later relax this requirement by assuming pseudo- random functions and appealing to a computational variant of differential privacy, SIM-CDP. Our algorithm is built on top of the celebrated Flajolet-Martin (FM) sketch. We show that FM-sketch is differentially private as is, as long as there are p ≈ ln(1/δ)=(εγ) distinct elements in the data set. Along the way, we prove a structural result showing that the maximum of k i.i.d. random variables is statisti- cally close (in the sense of "-differential privacy) to the maximum of (k + 1) i.i.d.
    [Show full text]
  • 6.5 X 11.5 Doublelines.P65
    Cambridge University Press 978-0-521-87282-9 - Algorithmic Game Theory Edited by Noam Nisan, Tim Roughgarden, Eva Tardos and Vijay V. Vazirani Copyright Information More information Algorithmic Game Theory Edited by Noam Nisan Hebrew University of Jerusalem Tim Roughgarden Stanford University Eva´ Tardos Cornell University Vijay V. Vazirani Georgia Institute of Technology © Cambridge University Press www.cambridge.org Cambridge University Press 978-0-521-87282-9 - Algorithmic Game Theory Edited by Noam Nisan, Tim Roughgarden, Eva Tardos and Vijay V. Vazirani Copyright Information More information cambridge university press Cambridge, New York,Melbourne, Madrid, Cape Town, Singapore, Sao˜ Paulo, Delhi Cambridge University Press 32 Avenue of the Americas, New York, NY 10013-2473, USA www.cambridge.org Information on this title: www.cambridge.org/9780521872829 C Noam Nisan, Tim Roughgarden, Eva´ Tardos, Vijay V. Vazirani 2007 This publication is in copyright. Subject to statutory exception and to the provisions of relevant collective licensing agreements, no reproduction of any part may take place without the written permission of Cambridge University Press. First published 2007 Printed in the United States of America A catalog record for this book is available from the British Library. Library of Congress Cataloging in Publication Data Algorithmic game theory / edited by Noam Nisan ...[et al.]; foreword by Christos Papadimitriou. p. cm. Includes index. ISBN-13: 978-0-521-87282-9 (hardback) ISBN-10: 0-521-87282-0 (hardback) 1. Game theory. 2. Algorithms. I. Nisan, Noam. II. Title. QA269.A43 2007 519.3–dc22 2007014231 ISBN 978-0-521-87282-9 hardback Cambridge University Press has no responsibility for the persistence or accuracy of URLS for external or third-party Internet Web sites referred to in this publication and does not guarantee that any content on such Web sites is, or will remain, accurate or appropriate.
    [Show full text]
  • Algorithmic Mechanism Design
    Algorithmic Mechanism Design Noam Nisan Institute of Computer Science Hebrew University of Jerusalem Givat Ram Israel and Scho ol of Computer Science IDC Herzliya Emailnoamcshujiacil y Amir Ronen Institute of Computer Science Hebrew University of Jerusalem Givat Ram Israel Emailamirycshujiacil This researchwas supp orted by grants from the Israeli ministry of Science and the Israeli academy of sciences y This researchwas supp orted by grants from the Israeli ministry of Science and the Israeli academy of sciences Algorithmic Mechanism Design contact Amir Ronen Institute of Computer Science Hebrew University of Jerusalem Givat Ram Israel Emailamirycshujiacil Abstract We consider algorithmic problems in a distributed setting where the participants cannot b e assumed to follow the algorithm but rather their own selfinterest As such participants termed agents are capable of manipulating the algorithm the algorithm designer should ensure in advance that the agents interests are b est served by b ehaving correctly Following notions from the eld of mechanism design we suggest a framework for studying such algorithms In this mo del the algorithmic solution is adorned with payments to the partic ipants and is termed a mechanism The payments should b e carefully chosen as to motivate all participants to act as the algorithm designer wishes We apply the standard to ols of mechanism design to algorithmic problems and in particular to the shortest path problem Our main technical contribution concerns the study of a representative problem task
    [Show full text]
  • How Proper Is the Dominance-Solvable Outcome? Yukio Koriyama, Matias Nunez
    How proper is the dominance-solvable outcome? Yukio Koriyama, Matias Nunez To cite this version: Yukio Koriyama, Matias Nunez. How proper is the dominance-solvable outcome?. 2014. hal- 01074178 HAL Id: hal-01074178 https://hal.archives-ouvertes.fr/hal-01074178 Preprint submitted on 13 Oct 2014 HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non, lished or not. The documents may come from émanant des établissements d’enseignement et de teaching and research institutions in France or recherche français ou étrangers, des laboratoires abroad, or from public or private research centers. publics ou privés. ECOLE POLYTECHNIQUE CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE HOW PROPER IS THE DOMINANCE-SOLVABLE OUTCOME? Yukio KORIYAMA Matías NÚÑEZ September 2014 Cahier n° 2014-24 DEPARTEMENT D'ECONOMIE Route de Saclay 91128 PALAISEAU CEDEX (33) 1 69333033 http://www.economie.polytechnique.edu/ mailto:[email protected] How proper is the dominance-solvable outcome?∗ Yukio Koriyamay and Mat´ıas Nu´ nez˜ z September 2014 Abstract We examine the conditions under which iterative elimination of weakly dominated strategies refines the set of proper outcomes of a normal-form game. We say that the proper inclusion holds in terms of outcome if the set of outcomes of all proper equilibria in the reduced game is included in the set of all proper outcomes of the original game. We show by examples that neither dominance solvability nor the transference of decision-maker indiffer- ence condition (TDI∗ of Marx and Swinkels [1997]) implies proper inclusion.
    [Show full text]
  • An Artificial Intelligence Agent for Texas Hold'em Poker
    AN ARTIFICIAL INTELL IGENCE AGENT FOR TEXAS HOLD’EM PO KER PATRICK MCCURLEY – 0 62 4 91 7 90 2 An Artificial Intelligence Agent for Texas Hold’em Poker I declare that this document represents my own work except where otherwise stated. Signed …………………………………………………………………………. 08/05/ 2009 Patrick McCurley – 062491790 Introduction 3 TABLE OF CONTENTS 1. Introduction ................................................................................................................................................................ 7 1.1 Problem Description...................................................................................................................................... 7 1.2 Aims and Objectives....................................................................................................................................... 7 1.3 Dissertation Outline ....................................................................................................................................... 8 1.4 Ethics .................................................................................................................................................................... 8 2 Background................................................................................................................................................................10 2.1 Artificial Intelligence and Poker .............................................................................................................10 2.1.1 Problem Domain Realization .........................................................................................................10
    [Show full text]
  • Pseudorandomness and Combinatorial Constructions
    Pseudorandomness and Combinatorial Constructions Luca Trevisan∗ September 20, 2018 Abstract In combinatorics, the probabilistic method is a very powerful tool to prove the existence of combinatorial objects with interesting and useful properties. Explicit constructions of objects with such properties are often very difficult, or unknown. In computer science, probabilistic al- gorithms are sometimes simpler and more efficient than the best known deterministic algorithms for the same problem. Despite this evidence for the power of random choices, the computational theory of pseu- dorandomness shows that, under certain complexity-theoretic assumptions, every probabilistic algorithm has an efficient deterministic simulation and a large class of applications of the the probabilistic method can be converted into explicit constructions. In this survey paper we describe connections between the conditional “derandomization” re- sults of the computational theory of pseudorandomness and unconditional explicit constructions of certain combinatorial objects such as error-correcting codes and “randomness extractors.” 1 Introduction 1.1 The Probabilistic Method in Combinatorics In extremal combinatorics, the probabilistic method is the following approach to proving existence of objects with certain properties: prove that a random object has the property with positive probability. This simple idea has beem amazingly successful, and it gives the best known bounds for arXiv:cs/0601100v1 [cs.CC] 23 Jan 2006 most problems in extremal combinatorics. The idea was introduced (and, later, greatly developed) by Paul Erd˝os [18], who originally applied it to the following question: define R(k, k) to be the minimum value n such that every graph on n vertices has either an independent set of size at least k or a clique of size at least k;1 It was known that R(k, k) is finite and that it is at most 4k, and the question was to prove a lower bound.
    [Show full text]
  • 2.- Game Theory Overview Game Theory Studies Situations in Which
    2.- Game Theory Overview Game Theory studies situations in which the outcome of the decisions of an indi- vidual depends on the decisions taken by other individuals as well as on his own - Strategic Interdependence - 2.- Game Theory Overview Plan 2.1 Introduction 2.2 Static Games of Complete Information: Nash Equilibrium 2.3 Dynamic Games of Complete Information: Backwards Induction and Subgame Perfection 2.4 Static Games of Incomplete Information:Bayesian Equilibrium 2.5 Dynamic Games of Incomplete Information: Perfect Bayesian Equilibrium and Sequential Equilibrium 2.- Game Theory Overview 2.1 Introduction Any (non cooperative) game has the following elements 1. The Players Who is to take decisions 2 The Rules Who moves first, who moves next, what can be done and what can not, ... 3 The Strategies What actions are available to the players 4 The Outcomes What is the consequence of each combination of actions by the players 5 The Payoffs What each player obtains in each possible outcome 2.- Game Theory Overview 2.1 Introduction Example: Rock - Scissors - Paper 1. The Players Player 1 and Player 2 2 The Rules Simultaneously, both players show either rocks, scissors or paper 3 The Strategies Each player can choose among showing rocks, scissors or paper 4 The Outcomes Rock beats scissors, which beats paper, which beats rock 5 The Payoffs The winning player receives 1 from the loser (thus getting -1) 2.- Game Theory Overview 2.1 Introduction Game Scenarios Complete Information Incomplete Information Static Players have all relevant information Not all players have all the information and move simultaneously and move simultaneously Dynamic Players have all relevant information Not all players have all the information and move in turns and move in turns Examples Complete Information Incomplete Information Static Rock, Scissors, Paper Sealed-bid Auction Dynamic Chess English Auction 2.- Game Theory Overview 2.2 Static Games of Complete Information Players take decisions without knowing the actions taken by other players Basic Assumptions 1.
    [Show full text]