Game Theory 6. Extensive Games Albert-Ludwigs-Universität Freiburg Bernhard Nebel and Robert Mattmüller May 24, 2017 Motivation Definitions Solution Concepts One- Deviation Motivation Property Kuhn’s Theorem Two Extensions Summary May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 2 / 68 Motivation So far: All players move simultaneously, and then the Motivation outcome is determined. Definitions Solution Often in practice: Several moves in sequence (e. g. in Concepts One- chess). Deviation Property cannot be directly reflected by strategic games. Kuhn’s Extensive games (with perfect information) reflect such Theorem situations by modeling games as game trees. Two Extensions Idea: Players have several decision points where they can Summary decide how to play. Strategies: Mappings from decision points in the game tree to actions to be played. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 4 / 68 Motivation Definitions Solution Concepts One- Deviation Definitions Property Kuhn’s Theorem Two Extensions Summary May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 5 / 68 Extensive Games Definition (Extensive game with perfect information) Motivation An extensive game with perfect information is a tuple Definitions Γ = hN;H;P;(ui ) i that consists of: i2N Solution Concepts A finite non-empty set N of players. One- H Deviation A set of (finite or infinite) sequences, called histories, Property such that Kuhn’s the empty sequence hi 2 H, Theorem H is closed under prefixes: if ha1;:::;ak i 2 H for some Two Extensions k 2 [ f¥g, and l < k, then also ha1;:::;al i 2 H, and N Summary H is closed under limits: if for some infinite sequence i ¥ i k i ¥ ha ii=1, we have ha ii=1 2 H for all k 2 N, then ha ii=1 2 H. i k All infinite histories and all histories ha ii=1 2 H, for which k+1 i k+1 there is no a such that ha ii=1 2 H are called terminal histories Z. Components of a history are called actions. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 7 / 68 Extensive Games Definition (Extensive game with perfect information, ctd.) Motivation A player function P : H n Z ! N that determines which Definitions Solution player’s turn it is to move after a given nonterminal history. Concepts For each player i 2 N, a utility function (or payoff function) One- Deviation ui : Z ! R defined on the set of terminal histories. Property Kuhn’s The game is called finite, if H is finite. It has a finite horizon, if Theorem the lenght of histories is bounded from above. Two Extensions Summary Assumption: All ingredients of Γ are common knowledge amongst the players of the game. Terminology: In the following, we will simply write extensive games instead of extensive games with perfect information. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 8 / 68 Extensive Games Example (Division game) Two identical objects should be divided among two Motivation Definitions players. Solution Player 1 proposes an allocation. Concepts One- Player 2 agrees or rejects. Deviation Property On agreement: Allocation as proposed. Kuhn’s On rejection: Nobody gets anything. Theorem Two P(hi) = 1 Extensions Summary (2;0) (0;2) (1;1) P(h(2;0)i) = 2 P(h(1;1)i) = 2 P(h(0;2)i) = 2 y n y n y n (2;0) (0;0) (1;1) (0;0) (0;2) (0;0) May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 9 / 68 Extensive Games Example (Division game, formally) Motivation P(hi) = 1 Definitions (2;0) (0;2) Solution (1;1) Concepts P(h(2;0)i) = 2 P(h(1;1)i) = 2 P(h(0;2)i) = 2 One- Deviation y n y n y n Property Kuhn’s Theorem (2;0) (0;0) (1;1) (0;0) (0;2) (0;0) Two Extensions Summary N = f1;2g H = fhi;h(2;0)i;h(1;1)i;h(0;2)i;h(2;0);yi;h(2;0);ni;:::g P(hi) = 1, P(h) = 2 for all h 2 H n Z with h =6 hi u1(h(2;0);yi) = 2, u2(h(2;0);yi) = 0, etc. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 10 / 68 Extensive Games Motivation Notation: Definitions h ha1;:::;aki a Solution Let = be a history, and an action. Concepts 1 k Then (h;a) is the history ha ;:::;a ;ai. One- Deviation If h0 = hb1;:::;b`i, then (h;h0) is the history Property 1 k 1 ` Kuhn’s ha ;:::;a ;b ;:::;b i. Theorem The set of actions from which player P(h) can choose Two Extensions after a history h 2 H n Z is written as Summary A(h) = faj(h;a) 2 Hg: May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 11 / 68 Strategies Definition (Strategy in an extensive game) Motivation Definitions A strategy of a player i in an extensive game Solution Γ = hN;H;P;(ui )i2Ni is a function si that assigns to each Concepts nonterminal history h 2 H n Z with P(h) = i an action a 2 A(h). One- Deviation The set of strategies of player i is denoted as Si . Property Kuhn’s Theorem Remark: Strategies require us to assign actions to histories h, Two even if it is clear that they will never be played (e. g., because h Extensions will never be reached because of some earlier action). Summary Notation (for finite games): A strategy for a player is written as a string of actions at decision nodes as visited in a breadth-first order. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 12 / 68 Strategies Example (Strategies in an extensive game) Motivation Definitions P(hi) = 1 B Solution A Concepts One- P(hAi) = 2 Deviation D Property C Kuhn’s P(hA;Ci) = 1 Theorem F E Two Extensions Summary Strategies for player 1: AE, AF, BE and BF Strategies for player 2: C and D. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 13 / 68 Outcome Definition (Outcome) Motivation The outcome O(s) of a strategy profile s = (si )i2N is the Definitions (possibly infinite) terminal history h = hai ik , with k 2 [ f¥g, i=1 N Solution such that for all ` 2 N with 0 ≤ ` < k, Concepts One- 1 ` `+1 Deviation sP(ha1;:::;a`i)(ha ;:::;a i) = a : Property Kuhn’s Example (Outcome) Theorem Two Extensions P(hi) = 1 Summary B A P(hAi) = 2 D O(AF;C) = hA;C;Fi C P(hA;Ci) = 1 O(AE;D) = hA;Di: F E May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 14 / 68 Motivation Definitions Solution Concepts One- Deviation Solution Concepts Property Kuhn’s Theorem Two Extensions Summary May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 15 / 68 Nash Equilibria Motivation Definitions Solution Definition (Nash equilibrium in an extensive game) Concepts One- A Nash equilibrium in an extensive game Γ = hN;H;P;(ui )i2Ni Deviation Property is a strategy profile s∗ such that for every player i 2 N and for Kuhn’s all strategies si 2 Si , Theorem Two ∗ ∗ ∗ Extensions ui (O(s−i ;si )) ≥ ui (O(s−i ;si )): Summary May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 17 / 68 Induced Strategic Game Definition (Induced strategic game) Motivation Definitions The strategic game G induced by an extensive game Solution 0 0 Concepts Γ = hN;H;P;(ui )i2Ni is defined by G = hN;(Ai )i2N;(ui )i2Ni, One- where Deviation Property A0 S i 2 N i = i for all , and Kuhn’s 0 Theorem ui (a) = ui (O(a)) for all i 2 N. Two Extensions Summary Proposition The Nash equilibria of an extensive game Γ are exactly the Nash equilibria of the induced strategic game G of Γ. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 18 / 68 Induced Strategic Game Motivation Definitions Solution Remarks: Concepts One- Each extensive game can be transformed into a strategic Deviation Property game, but the resulting game can be exponentially larger. Kuhn’s The other direction does not work, because in extensive Theorem Two games, we do not have simultaneous actions. Extensions Summary May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 19 / 68 Empty Threats Example (Empty threat) Motivation Extensive game: Strategic form: Definitions LR Solution P(hi) = 1 Concepts T 0;0 2;1 One- T B Deviation Property P(hTi) = 2 B 1;2 1;2 Kuhn’s (1;2) Theorem L R Nash equilibria: (B;L) and (T;R). Two Extensions However, (B;L) is not realistic: (0;0) (2;1) Summary Player 1 plays B, “fearing” Strategies: response L to T. Player 1: T and B But player 2 would never play L in the extensive game. Player 2: L and R (B;L) involves “empty threat”. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 20 / 68 Subgames Idea: Exclude empty threats. Motivation Definitions How? Demand that a strategy profile is not only a Nash Solution equilibrium in the strategic form, but also in every subgame. Concepts One- Deviation Definition (Subgame) Property Kuhn’s A subgame of an extensive game Γ = hN;H;P;(ui )i2Ni, starting Theorem after history h, is the game Γ(h) = hN;Hjh;Pjh;(ui jh)i2Ni, where Two Extensions 0 0 Hjh = fh j(h;h ) 2 Hg, Summary 0 0 0 Pjh(h ) = P(h;h ) for all h 2 Hjh, and 0 0 0 ui jh(h ) = ui (h;h ) for all h 2 Hjh. May 24, 2017 B.