Game Theory, Machine Learning and Reasoning under Uncertainty This workshop explores the benefits that may result from carrying out research at the in- terface between machine learning and game theory. While classical game theory makes limited provision for dealing with uncertainty and noise, research in machine learning, and particularly probabilistic inference, has resulted in a remarkable array of powerful algo- rithms for performing statistical inference from difficult real world data. Recent research work at this interface has suggested computationally tractable algorithms for analysing games that consist of a large number of players, whilst insights from game theory have also inspired new work on strategic learning behaviour in probabilistic infer- ence and are suggesting new algorithms to perform intelligent sampling in Markov Monte Carlo methods. The goal of this workshop is to explore the significant advantagesthat game theory and ma- chine learning seem to offer to each other, to explore the correspondences and differences between these two fields and to identify interesting and exciting areas of future work. Schedule Morning Session 07:30 Introduction and Aims, Iead Rezek, University of Oxford 07:35 Invited Talk: Machine Learning: Principles, Probabilities and Perspectives, Stephen J. Roberts, Oxford University 08:05 Invited Talk: Learning Topics in Game-Theoretic Decision Making, Michael Littman, Rutgers 08:35 Invited Talk: Predictive Game Theory, David Wolpert, NASA Ames Research Center 09:05 Break 09:15 Model-based Reinforcement Learning for Partially Observable Games with Sampling-based State Estimation, Hajime Fujita, Nara Institute of Science and Technology 09:40 Effective negotiation proposals using models of preference and risk behavior, Angelo Restificar, Oregon State 10:05 Mechanism Design via Machine Learning, Yishay Mansour, Tel-Aviv University 10:30 Ski Break Afternoon Session 16:00 N-Body Games, Albert Xin Jiang, University of British Columbia 16:25 Probabilistic inference for computing optimal policies in MDPs, Marc Toussaint, University of Edinburgh 16:50 Graphical Models, Evolutionary Game Theory, and the Power of Randomization Siddharth Suri, University of Pennsylvania 17:15 Probability Collectives for Adaptive Distributed Control, David H. Wolpert, NASA Ames Research Center 17:40 Break 17:50 Probability Collectives: Examples and Applications, Dev Rajnarayan, Stanford University 18:15 A Formalization of Game Balance Principles, Jeff Long, University of Saskatchewan 18:40 A stochastic optimal control formulation of distributed decision making Bert Kappen, Radboud University, Nijmegen 19:05 Discussion, Wrapping Up 19:30 End Organizers I. Rezek University of Oxford, Oxford, UK. [email protected] www.robots.ox.ac.uk/∼irezek A. Rogers University of Southampton, Southampton, UK. [email protected] www.ecs.soton.ac.uk/people/∼acr David Wolpert NASA Ames Research Center, California, USA. [email protected] ti.arc.nasa.gov/people/dhw Abstracts Learning Topics in Game-Theoretic Decision Making, M. LITTMAN, Rutgers, NJ This presentation will review some topics of recent interest in AI and economics concern- ing design making in a computational game-theory framework. It will highlight areas in which machine learning has played a role and could play a greater role in the future. Cov- ered areas include recent representational and algorithmic advances, stochastic games and reinforcement learning, no regret algorithms, and the role of various equilibrium concepts. Machine Learning: Principles, Probabilities and Perspectives,S.J. ROBERTS, University of Oxford, UK This talk will offer an overview of some of the key principles in machine learning. It will discuss how uncertainty is involved, from data to models; how learning may be defined and how we may evaluate the value of information. Based on simple principles, strategies may be seen in the light of maximizing expected information. The differences (and similarities) between machine learning and game theory will be considered. Predictive Game Theory, DAVID H. WOLPERT, NASA Ames Research Center Abstract: Conventional noncooperative game theory hypothesizes that the joint strategy of a set of reasoning players in a game will necessarily satisfy an ”equilibrium concept”. All other joint strategies are considered impossible. Under this hypothesis the only issue is what equilibrium concept is ”correct”. This hypothesis violates the first-principles arguments underlying probability theory. In- deed, probability theory renders moot the controversy over what equilibrium concept is cor- rect - every joint strategy can arise with non-zero probability. Rather than a first-principles derivation of an equilibrium concept, game theory requires a first-principles derivation of a distribution over joint (mixed) strategies. If you wish to distill such a distribution down to the prediction of a single joint strategy, that prediction should be set by decision theory, using your (!) loss function. Accordingly, for any fixed game, the predicted joint strategy - one’s ”equilibrium concept” - will vary with the loss function of the external scientist mak- ing the prediction. Game theory based on such considerations is called Predictive Game Theory (PGT). This talk shows how informationtheory can providesuch a distribution over joint strategies. The connection of this distribution to the quantal response equilibrium is elaborated. It is also shown that in many games, having a probability distribution with support restricted to Nash equilibria - as stipulated by conventional game theory - is impossible. PGT is also used to: i) Derive an information-theoretic quantification of the degree of rationality; ii) Derive bounded rationality as a cost of computation; iii) Elaborate the close formal relationship between game theory and statistical physics; iv) Use this relationship to extend game theory to allow stochastically varying numbers of players. Model-based Reinforcement Learning for Partially Observable Games with Sampling-based State Estimation, HAJIME FUJITA AND SHIN ISHII, Graduate School of Information Science Nara Institute of Science and Technology, Ikoma, JP We present a model-based reinforcement learning (RL) scheme for large scale multi-agent problems with partial observability, and apply it to a card game, Hearts. This game is a well-defined example of an imperfect information game. To reduce the computational cost, we use a sampling technique based on Markov chain Monte Carlo (MCMC) in which the heavy integration required for the estimation and prediction can be approximated by a plausible number of samples. Computer simulation results show that our RL agent can perform learning of an appropriate strategy and exhibit a comparable performance to an expert-level human player in this partially observable multi-agent problem. Effective negotiation proposals using models of preference and risk be- havior,ANGELO RESTIFICAR, Oregon State University, Corvallis, OR, and PETER HADDAWY, Asian Institute of Technology Pathumthani, Thailand In previous work, we infer implicit preferences and attitude toward risk by interpreting offer/counter-offer exchanges in negotiation as a choice between a certain offer and a gam- ble [A. Restificar et.al. 2004]. Supervised learning can then be used to construct models of preference and risk behavior by generating training instances from such implicit informa- tion. In this paper, we introduce a procedure that uses these learned models to find effective negotiation proposals. Experiments were performed using this procedure via repeated ne- gotiations between a buyer and a seller agent. The results of our experiments suggest that the use of learned opponent models leads to a significant increase in the number of agree- ments and a remarkable reduction in the number of negotiation exchanges. Mechanism Design via Machine Learning,YISHAY MANSOUR, Tel-Aviv University, IL We use techniques from sample-complexity in machine learning to reduce problems of incentive-compatible mechanism design to standard algorithmic questions for a wide vari- ety of revenue-maximizing pricing problems. Our reductions imply that given an optimal (or beta-approximation) algorithm for the standard algorithmic problem, we can convert it into a (1+ epsilon)-approximation (or beta(1 + ǫ)-approximation) for the problem of designing a revenue-maximizing incentive-compatible mechanism, so long as the number of bidders is sufficiently large as a function of an appropriate measure of complexity of the comparison class of solutions. We apply these results to the problem of auctioning a digital good, the ”attribute auction” problem, and to the problem of item-pricing in unlimited- supply combinatorial auctions. From a learning perspective, these settings present unique challenges: in particular, the loss function is discontinuous and asymmetric, and the range of bidders’ valuations may be large. This is a joint work with Maria-Florina Balcan, Avrim Blum, Jason D. Hartline N-Body Games, ALBERT XIN JIANG, KEVIN LEYTON-BROWN AND NANDO DE FREITAS, University of British Columbia, CA This paper introduces n-body games, a new compact game-theoretic representation which permits a wide variety of game-theoretic quantities to be efficiently computed both approx- imately and exactly. This representation is useful for games which consist of choosing actions from a metric space (e.g., points in space) and in which payoffs are computed as a function of the distances between players’ action choices. Probabilistic