Proceedings of the Twenty-Sixth AAAI Conference on Artificial Intelligence Computing Game-Theoretic Solutions and Applications to Security∗ Vincent Conitzer Departments of Computer Science and Economics Duke University Durham, NC 27708, USA [email protected] Abstract theory in the design of their agents, enabling them to reason strategically. For this to work, it is essential to design effi- The multiagent systems community has adopted game cient algorithms for computing game-theoretic solutions— theory as a framework for the design of systems of mul- algorithms that take a game as input, and produce a way to tiple self-interested agents. For this to be effective, effi- cient algorithms must be designed to compute the solu- play as output. tions that game theory prescribes. In this paper, I sum- The rest of this paper is organized as follows. We first dis- marize some of the state of the art on this topic, focus- cuss various ways to represent games. We then discuss vari- ing particularly on how this line of work has contributed ous solution concepts—notions of what it means to solve the to several highly visible deployed security applications, game—and whether and how these solutions can be com- developed at the University of Southern California. puted efficiently, with a focus on two-player normal-form games. Finally, we discuss exciting recent developments in the deployment of these techniques in high-value security Introduction applications. How should an agent choose its actions optimally? The stan- dard decision-theoretic framework is to calculate a utility for Representing Games every possible outcome, as well as, for each possible course of action, a conditional probability distribution over the pos- We first discuss how to represent games. This is important sible outcomes. Then, the agent should choose an action that first of all from a conceptual viewpoint: we cannot have gen- maximizes expected utility. However, this framework is not eral tools for analyzing games precisely without making the straightforward to apply when there are other rational agents games themselves precise. There are several standard repre- in the environment, who have potentially different objec- sentation schemes from the game theory literature that al- tives. In this case, the agent needs to determine a probability low us to think clearly about games. Additionally, the rep- distribution over the other agents’ actions. To do so, it is resentation scheme of course has a major impact on scala- natural to assume that those agents are themselves trying to bility. From this perspective, a good representation scheme optimize their actions for their own utility functions. But this allows us to compactly represent natural games, in the same quickly leads one into circularities: if the optimal action for sense that Bayesian networks allow us to compactly rep- agent 1 depends on agent 2’s action, but the optimal action resent many natural probability distributions; and it allows for agent 2 depends on agent 1’s action, how can we ever for efficient algorithms. While the standard representation come to a conclusion? This is precisely the type of prob- schemes from game theory are sufficient in some cases, in lem that game theory aims to address. It allows agents to other cases the representation size blows up exponentially. form beliefs over each other’s actions and to act rationally In this section, we review standard representation on those beliefs in a consistent manner. The word “game” schemes from game theory, as well as, briefly, representa- in game theory is used to refer to any strategic situation, in- tion schemes that have been introduced in the AI commu- cluding games in the common sense of the word, such as nity to represent families of natural games more compactly. board and card games, but also important domains that are Discussion of compact representation schemes for security not so playful, such as the security games that we will dis- games is postponed towards the end of the paper. cuss towards the end of this paper. Social scientists and even biologists use game theory to Normal-form games model behaviors observed in the world. In contrast, AI re- Perhaps the best-known representation scheme for games is searchers tend to be interested in constructively using game the normal form (also known as the strategic form, or, in the ∗This paper was invited as a “What’s Hot” paper to the case of two players, the bimatrix form). Here, each player i AAAI’12 Sub-Area Spotlights track. has a set of available pure strategies Si, of which she must Copyright c 2012, Association for the Advancement of Artificial choose one. Each player i also has a utility function ui : Intelligence (www.aaai.org). All rights reserved. S1 × ::: × Sn ! R which maps outcomes (consisting of a 2106 pure strategy for each player) to utilities. Two-player games are often written down as follows: R P S R 0,0 -1,1 1,-1 P 1,-1 0,0 -1,1 S -1,1 1,-1 0,0 This is the familiar game of rock-paper-scissors. Player 1 is the “row” player and selects a row, and player two is the “column” player and selects a column. The numbers in each entry are the utilities of the row and column players, respec- tively, for the corresponding outcome. Rock-paper-scissors is an example of a zero-sum game: in each entry, the num- Figure 1: An extremely simple poker game. bers sum to zero, so whatever one player gains, the other loses. (taking the expectation over Nature’s randomness). This re- sults in the following normal-form game: Extensive-form games CC CF FC FF Extensive-form games allow us to directly model the tempo- BB 0,0 0,0 1,-1 1,-1 ral and informational structure of a game. They are a gener- BC .5,-.5 1.5,-1.5 0,0 1,-1 alization of the game trees familiar to most AI researchers. CB -.5,.5 -.5,.5 1,-1 1,-1 They allow chance nodes (often called “moves by Nature” in CC 0,0 1,-1 0,0 1,-1 game theory), non-zero-sum outcomes, as well as imperfect information. In a perfect-information game, such as chess, While any extensive-form game can be converted to nor- checkers, or tic-tac-toe, nothing about the current state of mal form in this manner, the representation size can blow up the game is ever hidden from a player. In contrast, in (for exponentially; we will return to this problem later. example) most card games, no single player has full knowl- edge of the state of the game, because she does not know the Bayesian games cards of the other players. This is represented using infor- In recreational games, it is generally common knowledge mation sets: an information set is a set of nodes in the game how much each player values each outcome, because this is tree that all have the same acting player, such that the player specified by the rules of the game. In contrast, when mod- cannot distinguish these nodes from each other. eling real-world strategic situations, generally each player For example, consider the following simple (perhaps the has some uncertainty over how much the other players value simplest possible interesting) poker game. We have a two(!)- the different outcomes. This type of uncertainty is natu- card deck, with a King and a Jack. The former is a winning rally modeled by Bayesian games. In a Bayesian game, each card, the latter a losing card. Both players put $1 into the pot player draws a type from a distribution before playing the at the outset. Player 1 draws a card; player 2 does not get game. This type determines how much the player values a card. Player 1 must then choose to bet (put an additional each outcome.1 Each player knows her own type, but not $1 into the pot) or check (do nothing). Player 2 must then those of the other players. Again, Bayesian games can be choose to call (match whatever additional amount player 2 converted to normal form, at the cost of an exponential in- put into the pot, possibly $0), or fold (in which case player 1 crease in size. wins the pot). If player 2 calls, player 1 must show her card; if it is a King, she wins the pot, and if it is a Jack, player 2 Stochastic games wins the pot. Stochastic games are simply a multiplayer generalization of Figure 1 shows the extensive form of this game. The Markov decision processes. In each state, all players choose dashed lines between nodes for player 2 connect nodes in- an action, and the profile of actions selected determines the side the same information set, i.e., between which player 2 immediate rewards to all players as well as the transition cannot tell the difference (because he does not know which probabilities to other states. A stochastic game with only one card player 1 has). state is a repeated game (where the players play the same It is possible to convert extensive-form games to normal- normal-form game over and over). form games, as follows. A pure strategy in an extensive-form game consists of a plan that tells the player which action Compact representation schemes to take at every information set that she has in the game. One downside of the normal form is that its size is ex- (A player must choose the same action at every node inside ponential in the number of players. Several representation the same information set, because after all she cannot dis- schemes have been proposed to be able to represent “natu- tinguish them.) One pure strategy for player 1 is “Bet on a ral” games with many players compactly.