Game Theory 6. Extensive Games

Albert-Ludwigs-Universität Freiburg

Bernhard Nebel and Robert Mattmüller May 24, 2017 Motivation

Definitions

Solution Concepts

One- Deviation Motivation Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 2 / 68 Motivation

So far: All players move simultaneously, and then the Motivation outcome is determined. Definitions Solution Often in practice: Several moves in sequence (e. g. in Concepts One- chess). Deviation cannot be directly reflected by strategic games. Property Kuhn’s Extensive games (with perfect information) reflect such Theorem situations by modeling games as game trees. Two Extensions

Idea: Players have several decision points where they can Summary decide how to play. Strategies: Mappings from decision points in the game tree to actions to be played.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 4 / 68 Motivation

Definitions

Solution Concepts

One- Deviation Definitions Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 5 / 68 Extensive Games

Definition (Extensive game with perfect information) Motivation An extensive game with perfect information is a tuple Definitions Γ = hN,H,P,(ui ) i that consists of: i∈N Solution Concepts A finite non-empty set N of players. One- H Deviation A set of (finite or infinite) sequences, called histories, Property

such that Kuhn’s the empty sequence hi ∈ H, Theorem H is closed under prefixes: if ha1,...,ak i ∈ H for some Two Extensions k ∈ ∪ {∞}, and l < k, then also ha1,...,al i ∈ H, and N Summary H is closed under limits: if for some infinite sequence i ∞ i k i ∞ ha ii=1, we have ha ii=1 ∈ H for all k ∈ N, then ha ii=1 ∈ H. i k All infinite histories and all histories ha ii=1 ∈ H, for which k+1 i k+1 there is no a such that ha ii=1 ∈ H are called terminal histories Z. Components of a history are called actions.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 7 / 68 Extensive Games

Definition (Extensive game with perfect information, ctd.) Motivation A player function P : H \ Z → N that determines which Definitions Solution player’s turn it is to move after a given nonterminal history. Concepts For each player i ∈ N, a utility function (or payoff function) One- Deviation ui : Z → R defined on the set of terminal histories. Property Kuhn’s The game is called finite, if H is finite. It has a finite horizon, if Theorem the lenght of histories is bounded from above. Two Extensions

Summary Assumption: All ingredients of Γ are common knowledge amongst the players of the game. Terminology: In the following, we will simply write extensive games instead of extensive games with perfect information.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 8 / 68 Extensive Games

Example (Division game) Two identical objects should be divided among two Motivation Definitions players. Solution Player 1 proposes an allocation. Concepts One- Player 2 agrees or rejects. Deviation Property On agreement: Allocation as proposed. Kuhn’s On rejection: Nobody gets anything. Theorem

Two P(hi) = 1 Extensions Summary (2,0) (0,2) (1,1) P(h(2,0)i) = 2 P(h(1,1)i) = 2 P(h(0,2)i) = 2 y n y n y n

(2,0) (0,0) (1,1) (0,0) (0,2) (0,0) May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 9 / 68 Extensive Games

Example (Division game, formally) Motivation P(hi) = 1 Definitions

(2,0) (0,2) Solution (1,1) Concepts P(h(2,0)i) = 2 P(h(1,1)i) = 2 P(h(0,2)i) = 2 One- Deviation y n y n y n Property Kuhn’s Theorem (2,0) (0,0) (1,1) (0,0) (0,2) (0,0) Two Extensions

Summary N = {1,2} H = {hi,h(2,0)i,h(1,1)i,h(0,2)i,h(2,0),yi,h(2,0),ni,...} P(hi) = 1, P(h) = 2 for all h ∈ H \ Z with h =6 hi

u1(h(2,0),yi) = 2, u2(h(2,0),yi) = 0, etc. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 10 / 68 Extensive Games

Motivation

Notation: Definitions h ha1,...,aki a Solution Let = be a history, and an action. Concepts 1 k Then (h,a) is the history ha ,...,a ,ai. One- Deviation If h0 = hb1,...,b`i, then (h,h0) is the history Property 1 k 1 ` Kuhn’s ha ,...,a ,b ,...,b i. Theorem The set of actions from which player P(h) can choose Two Extensions after a history h ∈ H \ Z is written as Summary

A(h) = {a|(h,a) ∈ H}.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 11 / 68 Strategies

Definition (Strategy in an extensive game) Motivation Definitions A strategy of a player i in an extensive game Solution Γ = hN,H,P,(ui )i∈Ni is a function si that assigns to each Concepts nonterminal history h ∈ H \ Z with P(h) = i an action a ∈ A(h). One- Deviation The set of strategies of player i is denoted as Si . Property Kuhn’s Theorem

Remark: Strategies require us to assign actions to histories h, Two even if it is clear that they will never be played (e. g., because h Extensions will never be reached because of some earlier action). Summary Notation (for finite games): A strategy for a player is written as a string of actions at decision nodes as visited in a breadth-first order.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 12 / 68 Strategies

Example (Strategies in an extensive game) Motivation Definitions P(hi) = 1 B Solution A Concepts One- P(hAi) = 2 Deviation D Property C Kuhn’s P(hA,Ci) = 1 Theorem F E Two Extensions

Summary

Strategies for player 1: AE, AF, BE and BF Strategies for player 2: C and D.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 13 / 68 Outcome

Definition (Outcome) Motivation The outcome O(s) of a strategy profile s = (si )i∈N is the Definitions (possibly infinite) terminal history h = hai ik , with k ∈ ∪ {∞}, i=1 N Solution such that for all ` ∈ N with 0 ≤ ` < k, Concepts One- 1 ` `+1 Deviation sP(ha1,...,a`i)(ha ,...,a i) = a . Property Kuhn’s Example (Outcome) Theorem Two Extensions P(hi) = 1 Summary B A P(hAi) = 2 D O(AF,C) = hA,C,Fi C P(hA,Ci) = 1 O(AE,D) = hA,Di. F E

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 14 / 68 Motivation

Definitions

Solution Concepts

One- Deviation Solution Concepts Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 15 / 68 Nash Equilibria

Motivation

Definitions

Solution Definition (Nash equilibrium in an extensive game) Concepts One- A Nash equilibrium in an extensive game Γ = hN,H,P,(ui )i∈Ni Deviation Property is a strategy profile s∗ such that for every player i ∈ N and for Kuhn’s all strategies si ∈ Si , Theorem Two ∗ ∗ ∗ Extensions ui (O(s−i ,si )) ≥ ui (O(s−i ,si )). Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 17 / 68 Induced Strategic Game

Definition (Induced strategic game) Motivation Definitions

The strategic game G induced by an extensive game Solution 0 0 Concepts Γ = hN,H,P,(ui )i∈Ni is defined by G = hN,(Ai )i∈N,(ui )i∈Ni, One- where Deviation Property A0 S i ∈ N i = i for all , and Kuhn’s 0 Theorem ui (a) = ui (O(a)) for all i ∈ N. Two Extensions

Summary Proposition The Nash equilibria of an extensive game Γ are exactly the Nash equilibria of the induced strategic game G of Γ.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 18 / 68 Induced Strategic Game

Motivation

Definitions

Solution Remarks: Concepts One- Each extensive game can be transformed into a strategic Deviation Property game, but the resulting game can be exponentially larger. Kuhn’s The other direction does not work, because in extensive Theorem Two games, we do not have simultaneous actions. Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 19 / 68 Empty Threats

Example (Empty threat) Motivation Extensive game: Strategic form: Definitions LR Solution P(hi) = 1 Concepts T 0,0 2,1 One- T B Deviation Property P(hTi) = 2 B 1,2 1,2 Kuhn’s (1,2) Theorem L R Nash equilibria: (B,L) and (T,R). Two Extensions However, (B,L) is not realistic: (0,0) (2,1) Summary Player 1 plays B, “fearing” Strategies: response L to T. Player 1: T and B But player 2 would never play L in the extensive game. L R Player 2: and (B,L) involves “empty threat”. May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 20 / 68 Subgames

Idea: Exclude empty threats. Motivation Definitions How? Demand that a strategy profile is not only a Nash Solution equilibrium in the strategic form, but also in every subgame. Concepts One- Deviation Definition (Subgame) Property Kuhn’s A subgame of an extensive game Γ = hN,H,P,(ui )i∈Ni, starting Theorem after history h, is the game Γ(h) = hN,H|h,P|h,(ui |h)i∈Ni, where Two Extensions 0 0 H|h = {h |(h,h ) ∈ H}, Summary 0 0 0 P|h(h ) = P(h,h ) for all h ∈ H|h, and 0 0 0 ui |h(h ) = ui (h,h ) for all h ∈ H|h.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 21 / 68 Subgames

Motivation Definition (Strategy in a subgame) Definitions Solution Let Γ be an extensive game and Γ(h) a subgame of Γ starting Concepts One- after some history h. Deviation Property For each strategy si of Γ, let si |h be the strategy induced by si 0 Kuhn’s for Γ(h). Formally, for all h ∈ H|h, Theorem Two 0 0 Extensions si |h(h ) = si (h,h ). Summary

The outcome function of Γ(h) is denoted by Oh.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 22 / 68 Subgame-Perfect Equilibria

Motivation

Definitions Definition (Subgame-perfect equilibrium) Solution Concepts ∗ A strategy profile s in an extensive game Γ = hN,H,P,(ui )i∈Ni One- Deviation is a subgame-perfect equilibrium if and only if for every player Property i ∈ N and every nonterminal history h ∈ H \ Z with P(h) = i, Kuhn’s Theorem

∗ ∗ ∗ Two ui |h(Oh(s−i |h,si |h)) ≥ ui |h(Oh(s−i |h,si )) Extensions

Summary for every strategy si ∈ Si in subgame Γ(h).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 23 / 68 Subgame-Perfect Equilibria

Two Nash equilibria: Motivation

(T,R): subgame-perfect, Definitions

because: Solution In history h = hTi: Concepts P(hi) = 1 One- subgame-perfect. Deviation T B In history h = hi: player 1 Property h i obtains utility 1 when Kuhn’s P( T ) = 2 Theorem (1,2) choosing B and utility of 2 L R Two when choosing T. Extensions (B,L): not subgame-perfect, Summary (0,0) (2,1) since L does not maximize the utility of player 2 in history h = hTi.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 24 / 68 Subgame-Perfect Equilibria

Example (Subgame-perfect equilibria in division game) Motivation Equilibria in subgames: Definitions 1 in Γ(h(2,0)i): y and n Solution (2,0) (0,2) Concepts (1,1) in Γ(h(1,1)i): only y One- 2 2 2 in Γ(h(0,2)i): only y Deviation y n y n y n Property in Γ(hi): ((2,0),yyy) Kuhn’s Theorem (2,0) (0,0) (1,1) (0,0) (0,2) (0,0) and ((1,1),nyy) Two Extensions

Nash equilibria (red: empty threat): Summary ((2,0),yyy), ((2,0),yyn), ((2,0),yny), ((2,0),ynn), ((2,0),nny), ((2,0),nnn), ((1,1),nyy), ((1,1),nyn), ((0,2),nny), ((0,2),nnn).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 25 / 68 Motivation

Definitions

Solution Concepts

One- Deviation One-Deviation Property Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 26 / 68 Motivation

Motivation

Definitions

Existence: Solution Does every extensive game have a subgame-perfect Concepts equilibrium? One- Deviation If not, which extensive games do have a subgame-perfect Property equilibrium? Kuhn’s Theorem Computation: Two If a subgame-perfect equilibrium exists, how to compute Extensions it? Summary How complex is that computation?

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 28 / 68 Motivation

Motivation

Definitions

Solution Positive case (a subgame-perfect equilibrium exists): Concepts

One- Step 1: Show that is suffices to consider local deviations Deviation from strategies (for finite-horizon games). Property Kuhn’s Step 2: Show how to systematically explore such local Theorem deviations to find a subgame-perfect equilibrium (for finite Two Extensions games). Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 29 / 68 Step 1: One-Deviation Property

Motivation

Definitions

Solution Concepts

One- Definition Deviation Property ` Let Γ be a finite-horizon extensive game. Then (Γ) denotes Kuhn’s the length of the longest history of Γ. Theorem Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 30 / 68 Step 1: One-Deviation Property

Definition (One-deviation property) Motivation

∗ Definitions A strategy profile s in an extensive game Γ = hN,H,P,(ui )i∈Ni Solution satisfies the one-deviation property if and only if for every Concepts player i ∈ N and every nonterminal history h ∈ H \ Z with One- Deviation P(h) = i, Property

Kuhn’s ∗ ∗ ∗ Theorem ui |h(Oh(s−i |h,si |h)) ≥ ui |h(Oh(s−i |h,si )) Two ∗ Extensions for every strategy si ∈ Si in subgame Γ(h) that differs from s | i h Summary only in the action it prescribes after the initial history of Γ(h).

Note: Without the highlighted parts, this is just the definition of subgame-perfect equilibria!

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 31 / 68 Step 1: One-Deviation Property

Lemma Motivation

Let Γ = hN,H,P,(ui )i∈Ni be a finite-horizon extensive game. Definitions ∗ Solution Then a strategy profile s is a subgame-perfect equilibrium of Concepts

Γ if and only if it satisfies the one-deviation property. One- Deviation Property

Proof Kuhn’s Theorem (⇒) Clear. Two (⇐) By contradiction: Extensions Summary Suppose that s∗ is not a subgame-perfect equilibrium.

Then there is a history h and a player i such that si is a profitable deviation for player i in subgame Γ(h). ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 32 / 68 Step 1: One-Deviation Property

Lemma Motivation

Let Γ = hN,H,P,(ui )i∈Ni be a finite-horizon extensive game. Definitions ∗ Solution Then a strategy profile s is a subgame-perfect equilibrium of Concepts

Γ if and only if it satisfies the one-deviation property. One- Deviation Property

Proof Kuhn’s Theorem (⇒) Clear. Two (⇐) By contradiction: Extensions Summary Suppose that s∗ is not a subgame-perfect equilibrium.

Then there is a history h and a player i such that si is a profitable deviation for player i in subgame Γ(h). ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 32 / 68 Step 1: One-Deviation Property

Lemma Motivation

Let Γ = hN,H,P,(ui )i∈Ni be a finite-horizon extensive game. Definitions ∗ Solution Then a strategy profile s is a subgame-perfect equilibrium of Concepts

Γ if and only if it satisfies the one-deviation property. One- Deviation Property

Proof Kuhn’s Theorem (⇒) Clear. Two (⇐) By contradiction: Extensions Summary Suppose that s∗ is not a subgame-perfect equilibrium.

Then there is a history h and a player i such that si is a profitable deviation for player i in subgame Γ(h). ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 32 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐) . . . WLOG, the number of histories h0 with 0 ∗ 0 Definitions si (h ) =6 s |h(h ) is at most `(Γ(h)) and hence finite (finite i Solution horizon assumption!), since deviations not on resulting Concepts outcome path are irrelevant. One- Deviation ∗ ∗ Property Illustration: strategies s1|h = AGILN and s2|h = CF red: Kuhn’s P(h) = 1 Theorem Two A B Extensions Summary 2 2 C D E F 1 1 1 1 G H I K L M N O

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 33 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐) . . . WLOG, the number of histories h0 with 0 ∗ 0 Definitions si (h ) =6 s |h(h ) is at most `(Γ(h)) and hence finite (finite i Solution horizon assumption!), since deviations not on resulting Concepts outcome path are irrelevant. One- Deviation ∗ ∗ Property Illustration: strategies s1|h = AGILN and s2|h = CF red: Kuhn’s P(h) = 1 Theorem Two A B Extensions Summary 2 2 C D E F 1 1 1 1 G H I K L M N O

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 33 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐) . . . Illustration for WLOG assumption: Assume Definitions Solution s1 = BHKMO (blue) profitable deviation: Concepts P(h) = 1 One- Deviation Property A B Kuhn’s Theorem 2 2 Two C D E F Extensions 1 1 1 1 Summary G H I K L M N O

Then only B and O really matter.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 34 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐) . . . Illustration for WLOG assumption: And hence Definitions Solution s˜1 = BGILO (blue) also profitable deviation: Concepts P(h) = 1 One- Deviation Property A B Kuhn’s Theorem 2 2 Two C D E F Extensions 1 1 1 1 Summary G H I K L M N O

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 35 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐)... Definitions Solution Choose profitable deviation si in Γ(h) with minimal Concepts

number of deviation points (such si must exist). One- ∗ ∗ ∗ ∗ Deviation Let h be the longest history in Γ(h) with si (h ) =6 si |h(h ), Property i.e., “deepest” deviation point for si . Kuhn’s Theorem ∗ ∗ Then in Γ(h,h ), si |h∗ differs from si |(h,h∗) only in the initial Two history. Extensions Summary ∗ Moreover, si |h∗ is a profitable deviation in Γ(h,h ), since ∗ ∗ ∗ ∗ h is the longest history in Γ(h) with si (h ) =6 si |h(h ). So, Γ(h,h∗) is the desired subgame where a one-step deviation is sufficient to improve utility.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 36 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐)... Definitions Solution Choose profitable deviation si in Γ(h) with minimal Concepts

number of deviation points (such si must exist). One- ∗ ∗ ∗ ∗ Deviation Let h be the longest history in Γ(h) with si (h ) =6 si |h(h ), Property i.e., “deepest” deviation point for si . Kuhn’s Theorem ∗ ∗ Then in Γ(h,h ), si |h∗ differs from si |(h,h∗) only in the initial Two history. Extensions Summary ∗ Moreover, si |h∗ is a profitable deviation in Γ(h,h ), since ∗ ∗ ∗ ∗ h is the longest history in Γ(h) with si (h ) =6 si |h(h ). So, Γ(h,h∗) is the desired subgame where a one-step deviation is sufficient to improve utility.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 36 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐)... Definitions Solution Choose profitable deviation si in Γ(h) with minimal Concepts

number of deviation points (such si must exist). One- ∗ ∗ ∗ ∗ Deviation Let h be the longest history in Γ(h) with si (h ) =6 si |h(h ), Property i.e., “deepest” deviation point for si . Kuhn’s Theorem ∗ ∗ Then in Γ(h,h ), si |h∗ differs from si |(h,h∗) only in the initial Two history. Extensions Summary ∗ Moreover, si |h∗ is a profitable deviation in Γ(h,h ), since ∗ ∗ ∗ ∗ h is the longest history in Γ(h) with si (h ) =6 si |h(h ). So, Γ(h,h∗) is the desired subgame where a one-step deviation is sufficient to improve utility.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 36 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐)... Definitions Solution Choose profitable deviation si in Γ(h) with minimal Concepts

number of deviation points (such si must exist). One- ∗ ∗ ∗ ∗ Deviation Let h be the longest history in Γ(h) with si (h ) =6 si |h(h ), Property i.e., “deepest” deviation point for si . Kuhn’s Theorem ∗ ∗ Then in Γ(h,h ), si |h∗ differs from si |(h,h∗) only in the initial Two history. Extensions Summary ∗ Moreover, si |h∗ is a profitable deviation in Γ(h,h ), since ∗ ∗ ∗ ∗ h is the longest history in Γ(h) with si (h ) =6 si |h(h ). So, Γ(h,h∗) is the desired subgame where a one-step deviation is sufficient to improve utility.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 36 / 68 Step 1: One-Deviation Property

Proof (ctd.) Motivation (⇐)... Definitions Solution Choose profitable deviation si in Γ(h) with minimal Concepts

number of deviation points (such si must exist). One- ∗ ∗ ∗ ∗ Deviation Let h be the longest history in Γ(h) with si (h ) =6 si |h(h ), Property i.e., “deepest” deviation point for si . Kuhn’s Theorem ∗ ∗ Then in Γ(h,h ), si |h∗ differs from si |(h,h∗) only in the initial Two history. Extensions Summary ∗ Moreover, si |h∗ is a profitable deviation in Γ(h,h ), since ∗ ∗ ∗ ∗ h is the longest history in Γ(h) with si (h ) =6 si |h(h ). So, Γ(h,h∗) is the desired subgame where a one-step deviation is sufficient to improve utility.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 36 / 68 Step 1: One-Deviation Property Example

A B 1 Motivation 2 2 C D E F Definitions 1 1 Solution G H I K Concepts One- Deviation To show that (AHI,CE) is a subgame-perfect equilibrium, it Property Kuhn’s suffices to check these deviating strategies: Theorem

Two Player 1: Player 2: Extensions Summary G in subgame Γ(hA,Ci) D in subgame Γ(hAi) K in subgame Γ(hB,Fi) F in subgame Γ(hBi) BHI in Γ In particular, e.g., no need to check if strategy BGK of player 1 is profitable in Γ.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 37 / 68 Step 1: One-Deviation Property Remark on Infinite-Horizon Games

The corresponding proposition for infinite-horizon games does not hold. Motivation Counterexample (one-player case): Definitions Solution Concepts C C C C C C,... One- 1 Deviation Property

S S S S S S Kuhn’s Theorem 0 0 0 0 0 0 Two Extensions

Summary

Strategy si with si (h) = S for all h ∈ H \ Z satisfies one deviation property, but is not a subgame-perfect equilibrium, since it is ∗ ∗ dominated by si with si (h) = C for all h ∈ H \ Z.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 38 / 68 Motivation

Definitions

Solution Concepts

One- Deviation Kuhn’s Theorem Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 39 / 68 Step 2: Kuhn’s Theorem

Motivation

Theorem (Kuhn) Definitions

Solution Every finite extensive game has a subgame-perfect Concepts equilibrium. One- Deviation Property

Proof idea: Kuhn’s Theorem Proof is constructive and builds a subgame-perfect Two equilibrium bottom-up (aka backward induction). Extensions For those familiar with the Foundations of AI lecture: Summary generalization of Minimax algorithm to general-sum games with possibly more than two players.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 41 / 68 (1,5)

(1,5) (0,8)

s2(hAi) = C t1(hAi) = 1 t2(hAi) = 5

s2(hBi) = F t1(hBi) = 0 t2(hBi) = 8

s1(hi) = A t1(hi) = 1 t2(hi) = 5

Step 2: Kuhn’s Theorem

Example Motivation

1 Definitions A B Solution Concepts 2 2 One- Deviation Property C D E F Kuhn’s Theorem (1,5) (2,3) (3,4) (0,8) Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 42 / 68 (1,5)

(0,8)

s2(hBi) = F t1(hBi) = 0 t2(hBi) = 8

s1(hi) = A t1(hi) = 1 t2(hi) = 5

Step 2: Kuhn’s Theorem

Example Motivation

1 Definitions A B Solution Concepts 2 (1,5) 2 One- Deviation Property C D E F Kuhn’s Theorem (1,5) (2,3) (3,4) (0,8) Two Extensions

Summary

s2(hAi) = C t1(hAi) = 1 t2(hAi) = 5

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 42 / 68 (1,5)

s1(hi) = A t1(hi) = 1 t2(hi) = 5

Step 2: Kuhn’s Theorem

Example Motivation

1 Definitions A B Solution Concepts 2 (1,5) 2 (0,8) One- Deviation Property C D E F Kuhn’s Theorem (1,5) (2,3) (3,4) (0,8) Two Extensions

Summary

s2(hAi) = C t1(hAi) = 1 t2(hAi) = 5

s2(hBi) = F t1(hBi) = 0 t2(hBi) = 8

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 42 / 68 Step 2: Kuhn’s Theorem

Example Motivation , 1 (1 5) Definitions A B Solution Concepts 2 (1,5) 2 (0,8) One- Deviation Property C D E F Kuhn’s Theorem (1,5) (2,3) (3,4) (0,8) Two Extensions

Summary

s2(hAi) = C t1(hAi) = 1 t2(hAi) = 5

s2(hBi) = F t1(hBi) = 0 t2(hBi) = 8

s1(hi) = A t1(hi) = 1 t2(hi) = 5

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 42 / 68 Step 2: Kuhn’s Theorem

Motivation

A bit more formally: Definitions

Solution Proof Concepts Let Γ = hN,H,P,(u ) i be a finite extensive game. One- i i∈N Deviation Construct a subgame-perfect equilibrium by induction on Property Kuhn’s `(Γ(h)) for all subgames Γ(h). In parallel, construct functions Theorem ti : H → R for all players i ∈ N s. t. ti (h) is the payoff for player i Two Extensions in a subgame-perfect equilibrium in subgame Γ(h). Summary Base case: If `(Γ(h)) = 0, then ti (h) = ui (h) for all i ∈ N. ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 43 / 68 Step 2: Kuhn’s Theorem

Motivation

A bit more formally: Definitions

Solution Proof Concepts Let Γ = hN,H,P,(u ) i be a finite extensive game. One- i i∈N Deviation Construct a subgame-perfect equilibrium by induction on Property Kuhn’s `(Γ(h)) for all subgames Γ(h). In parallel, construct functions Theorem ti : H → R for all players i ∈ N s. t. ti (h) is the payoff for player i Two Extensions in a subgame-perfect equilibrium in subgame Γ(h). Summary Base case: If `(Γ(h)) = 0, then ti (h) = ui (h) for all i ∈ N. ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 43 / 68 Step 2: Kuhn’s Theorem

Motivation

A bit more formally: Definitions

Solution Proof Concepts Let Γ = hN,H,P,(u ) i be a finite extensive game. One- i i∈N Deviation Construct a subgame-perfect equilibrium by induction on Property Kuhn’s `(Γ(h)) for all subgames Γ(h). In parallel, construct functions Theorem ti : H → R for all players i ∈ N s. t. ti (h) is the payoff for player i Two Extensions in a subgame-perfect equilibrium in subgame Γ(h). Summary Base case: If `(Γ(h)) = 0, then ti (h) = ui (h) for all i ∈ N. ...

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 43 / 68 Step 2: Kuhn’s Theorem

Proof (ctd.) Motivation Inductive case: If ti (h) already defined for all h ∈ H with Definitions ∗ ∗ ∗ `(Γ(h)) ≤ k, consider h ∈ H with `(Γ(h )) = k + 1 and P(h ) = i. Solution Concepts ∗ ∗ For all a ∈ A(h ), `(Γ(h ,a)) ≤ k, let One- Deviation Property ∗ ∗ si (h ) := argmaxti (h ,a) and Kuhn’s ∗ a∈A(h ) Theorem

∗ ∗ ∗ Two tj (h ) := tj (h ,si (h )) for all players j ∈ N. Extensions

Summary Inductively, we obtain a strategy profile s that satisfies the one-deviation property. With the one-deviation property lemma it follows that s is a subgame-perfect equilibrium.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 44 / 68 Step 2: Kuhn’s Theorem

Proof (ctd.) Motivation Inductive case: If ti (h) already defined for all h ∈ H with Definitions ∗ ∗ ∗ `(Γ(h)) ≤ k, consider h ∈ H with `(Γ(h )) = k + 1 and P(h ) = i. Solution Concepts ∗ ∗ For all a ∈ A(h ), `(Γ(h ,a)) ≤ k, let One- Deviation Property ∗ ∗ si (h ) := argmaxti (h ,a) and Kuhn’s ∗ a∈A(h ) Theorem

∗ ∗ ∗ Two tj (h ) := tj (h ,si (h )) for all players j ∈ N. Extensions

Summary Inductively, we obtain a strategy profile s that satisfies the one-deviation property. With the one-deviation property lemma it follows that s is a subgame-perfect equilibrium.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 44 / 68 Step 2: Kuhn’s Theorem

Proof (ctd.) Motivation Inductive case: If ti (h) already defined for all h ∈ H with Definitions ∗ ∗ ∗ `(Γ(h)) ≤ k, consider h ∈ H with `(Γ(h )) = k + 1 and P(h ) = i. Solution Concepts ∗ ∗ For all a ∈ A(h ), `(Γ(h ,a)) ≤ k, let One- Deviation Property ∗ ∗ si (h ) := argmaxti (h ,a) and Kuhn’s ∗ a∈A(h ) Theorem

∗ ∗ ∗ Two tj (h ) := tj (h ,si (h )) for all players j ∈ N. Extensions

Summary Inductively, we obtain a strategy profile s that satisfies the one-deviation property. With the one-deviation property lemma it follows that s is a subgame-perfect equilibrium.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 44 / 68 Step 2: Kuhn’s Theorem

Motivation

Definitions

Solution In principle: sample subgame-perfect equilibrium Concepts

effectively computable using the technique from the One- Deviation above proof. Property In practice: often game trees not enumerated in advance, Kuhn’s Theorem hence unavailable for backward induction. Two E.g., for branching factor b and depth m, procedure needs Extensions time O(bm). Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 45 / 68 Step 2: Kuhn’s Theorem Remark on Infinite Games

Motivation

Definitions

Corresponding proposition for infinite games does not hold. Solution Concepts

One- Counterexamples (both for one-player case): Deviation Property

Kuhn’s A) finite horizon, infinite branching factor: Theorem a ∈ A , u hai a Two Infinitely many actions = [0 1) with payoffs 1( ) = for Extensions all a ∈ A. Summary There exists no subgame-perfect equilibrium in this game.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 46 / 68 Step 2: Kuhn’s Theorem Remark on Infinite Games

Motivation

B) infinite horizon, finite branching factor: Definitions

Solution C C C C C C,... Concepts 0 One- Deviation S S S S S S Property Kuhn’s 1 2 3 4 5 6 Theorem Two Extensions u (CCC ...) = 0 and u (CC ...C S) = n + 1. Summary 1 1 | {z } n No subgame-perfect equilibrium.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 47 / 68 Step 2: Kuhn’s Theorem

Motivation

Definitions

Solution Concepts Uniqueness: One- Deviation Kuhn’s theorem tells us nothing about uniqueness of Property subgame-perfect equilibria. However, if no two histories get the Kuhn’s same evaluation by any player, then the subgame-perfect Theorem Two equilibrium is unique. Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 48 / 68 Extended Example: Pirate Game

1 There are 5 rational pirates, A,B,C,D and E. They find 100 gold coins. They must decide how to distribute them. Motivation Definitions 2 The pirates have a strict order of seniority: A is senior to B, Solution who is senior to C, who is senior to D, who is senior to E. Concepts One- 3 The pirate world’s rules of distribution say that the most Deviation senior pirate first proposes a distribution of coins. The Property Kuhn’s pirates, including the proposer, then vote on whether to Theorem

accept this distribution (in order from most junior to Two senior). In case of a tie vote, the proposer has the casting Extensions Summary vote. If the distribution is accepted, the coins are disbursed and the game ends. If not, the proposer is thrown overboard from the pirate ship and dies, and the next most senior pirate makes a new proposal to apply the method again.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 49 / 68 Extended Example: Pirate Game

1 There are 5 rational pirates, A,B,C,D and E. They find 100 gold coins. They must decide how to distribute them. Motivation Definitions 2 The pirates have a strict order of seniority: A is senior to B, Solution who is senior to C, who is senior to D, who is senior to E. Concepts One- 3 The pirate world’s rules of distribution say that the most Deviation senior pirate first proposes a distribution of coins. The Property Kuhn’s pirates, including the proposer, then vote on whether to Theorem

accept this distribution (in order from most junior to Two senior). In case of a tie vote, the proposer has the casting Extensions Summary vote. If the distribution is accepted, the coins are disbursed and the game ends. If not, the proposer is thrown overboard from the pirate ship and dies, and the next most senior pirate makes a new proposal to apply the method again.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 49 / 68 Extended Example: Pirate Game

1 There are 5 rational pirates, A,B,C,D and E. They find 100 gold coins. They must decide how to distribute them. Motivation Definitions 2 The pirates have a strict order of seniority: A is senior to B, Solution who is senior to C, who is senior to D, who is senior to E. Concepts One- 3 The pirate world’s rules of distribution say that the most Deviation senior pirate first proposes a distribution of coins. The Property Kuhn’s pirates, including the proposer, then vote on whether to Theorem

accept this distribution (in order from most junior to Two senior). In case of a tie vote, the proposer has the casting Extensions Summary vote. If the distribution is accepted, the coins are disbursed and the game ends. If not, the proposer is thrown overboard from the pirate ship and dies, and the next most senior pirate makes a new proposal to apply the method again.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 49 / 68 Pirates: General Setting & Utility

Motivation 4 The pirates do not trust each other, and will neither make Definitions nor honor any promises between pirates apart from a Solution Concepts proposed distribution plan that gives a whole number of One- gold coins to each pirate. Deviation Property

5 Pirates base their decisions on three factors. First of all, Kuhn’s each pirate wants to survive. Second, everything being Theorem Two equal, each pirate wants to maximize the number of gold Extensions coins each receives. Third, each pirate would prefer to Summary throw another overboard, if all other results would otherwise be equal.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 50 / 68 Pirates: General Setting & Utility

Motivation 4 The pirates do not trust each other, and will neither make Definitions nor honor any promises between pirates apart from a Solution Concepts proposed distribution plan that gives a whole number of One- gold coins to each pirate. Deviation Property

5 Pirates base their decisions on three factors. First of all, Kuhn’s each pirate wants to survive. Second, everything being Theorem Two equal, each pirate wants to maximize the number of gold Extensions coins each receives. Third, each pirate would prefer to Summary throw another overboard, if all other results would otherwise be equal.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 50 / 68 Pirates: Formalization

Players N = {A,B,C,D,E}; Motivation actions are: Definitions

proposals by a pirate: hA : xA,B : xb,C : xB,D : xD,E : xE i, Solution Concepts with ∑i∈{A,B,C,D,E} xi = 100; One- votings: y for accepting, n for rejecting; Deviation Property

histories are sequences of a proposal, followed by votings Kuhn’s of the alive pirates; Theorem Two utilities: Extensions for pirates who are alive: utilities are according to the Summary accepted proposal plus x/100, x being the number of dead pirates; for dead pirates: -100. Remark: Very large game tree!

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 51 / 68 Pirates: Analysis by Backward Induction

Motivation 1 Assume only D and E are still alive. D can propose hA : 0,B : 0,C : 0,D : 100,E : 0i, because D has the Definitions Solution casting vote! Concepts One- 2 Assume C, D, and E are alive. For C it is enough to offer Deviation 1 coin to E to get his vote: hA : 0,B : 0,C : 99,D : 0,E : 1i. Property Kuhn’s 3 Assume B, C, D, and E are alive. B offering D one coin is Theorem enough because of the casting vote: Two Extensions hA ,B ,C ,D ,E i : 0 : 99 : 0 : 1 : 0 . Summary 4 Assume A, B, C, D, and E are alive. A offering C and E each one coin is enough: hA : 98,B : 0,C : 1,D : 0,E : 1i (note that giving 1 to D instaed to E does not help).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 52 / 68 Pirates: Analysis by Backward Induction

Motivation 1 Assume only D and E are still alive. D can propose hA : 0,B : 0,C : 0,D : 100,E : 0i, because D has the Definitions Solution casting vote! Concepts One- 2 Assume C, D, and E are alive. For C it is enough to offer Deviation 1 coin to E to get his vote: hA : 0,B : 0,C : 99,D : 0,E : 1i. Property Kuhn’s 3 Assume B, C, D, and E are alive. B offering D one coin is Theorem enough because of the casting vote: Two Extensions hA ,B ,C ,D ,E i : 0 : 99 : 0 : 1 : 0 . Summary 4 Assume A, B, C, D, and E are alive. A offering C and E each one coin is enough: hA : 98,B : 0,C : 1,D : 0,E : 1i (note that giving 1 to D instaed to E does not help).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 52 / 68 Pirates: Analysis by Backward Induction

Motivation 1 Assume only D and E are still alive. D can propose hA : 0,B : 0,C : 0,D : 100,E : 0i, because D has the Definitions Solution casting vote! Concepts One- 2 Assume C, D, and E are alive. For C it is enough to offer Deviation 1 coin to E to get his vote: hA : 0,B : 0,C : 99,D : 0,E : 1i. Property Kuhn’s 3 Assume B, C, D, and E are alive. B offering D one coin is Theorem enough because of the casting vote: Two Extensions hA ,B ,C ,D ,E i : 0 : 99 : 0 : 1 : 0 . Summary 4 Assume A, B, C, D, and E are alive. A offering C and E each one coin is enough: hA : 98,B : 0,C : 1,D : 0,E : 1i (note that giving 1 to D instaed to E does not help).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 52 / 68 Pirates: Analysis by Backward Induction

Motivation 1 Assume only D and E are still alive. D can propose hA : 0,B : 0,C : 0,D : 100,E : 0i, because D has the Definitions Solution casting vote! Concepts One- 2 Assume C, D, and E are alive. For C it is enough to offer Deviation 1 coin to E to get his vote: hA : 0,B : 0,C : 99,D : 0,E : 1i. Property Kuhn’s 3 Assume B, C, D, and E are alive. B offering D one coin is Theorem enough because of the casting vote: Two Extensions hA ,B ,C ,D ,E i : 0 : 99 : 0 : 1 : 0 . Summary 4 Assume A, B, C, D, and E are alive. A offering C and E each one coin is enough: hA : 98,B : 0,C : 1,D : 0,E : 1i (note that giving 1 to D instaed to E does not help).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 52 / 68 Motivation

Definitions

Solution Concepts

One- Deviation Two Extensions Property Kuhn’s Theorem

Two Extensions Simultaneous Moves Chance

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 53 / 68 Simultaneous Moves

Motivation Definition Definitions Solution An extensive game with simultaneous moves is a tuple Concepts

Γ = hN,H,P,(ui )i∈Ni, where One- Deviation N, H, P and (ui ) are defined as before, and Property N Kuhn’s P : H → 2 assigns to each nonterminal history a set of Theorem

players to move; for all h ∈ H \ Z, there exists a family Two Extensions (Ai (h))i∈P(h) such that Simultaneous Moves Chance A(h) = {a|(h,a) ∈ H} = ∏ Ai (h). Summary i∈P(h)

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 55 / 68 Simultaneous Moves

Motivation

Definitions Intended meaning of simultaneous moves: All players Solution Concepts from P(h) move simultaneously. One- Deviation Strategies: Functions si : h 7→ ai with ai ∈ Ai (h). Property Histories: Sequences of vectors of actions. Kuhn’s Theorem

Outcome: Terminal history reached when tracing strategy Two Extensions profile. Simultaneous Moves Payoffs: Utilities at outcome history. Chance Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 56 / 68 Simultaneous Moves One-Deviation Property and Kuhn’s Theorem

Remark: The one-deviation property still holds for extensive game Motivation with perfect information and simultaneous moves. Definitions Solution Kuhn’s theorem does not hold for extensive game with Concepts simultaneous moves. One- Deviation Example: Matching Pennies can be viewed as extensive Property game with simultaneous moves. No Nash Kuhn’s Theorem

equilibrium/subgame-perfect equilibrium. Two Extensions player 2 Simultaneous Moves HT Chance H 1,−1 −1, 1 Summary player 1 T −1, 1 1,−1 Need more sophisticated solution concepts (cf. mixed strategies). Not covered in this lecture.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 57 / 68 Simultaneous Moves Example: Three-Person Cake Splitting Game

Motivation

Setting: Definitions

Solution Three players have to split a cake fairly. Concepts

Player 1 suggest split: shares x1,x2,x3 ∈ [0,1] s.t. One- Deviation x1 + x2 + x3 = 1. Property Kuhn’s Then players 2 and 3 simultaneously and independently Theorem

decide whether to accept (“y”) or reject (“n”) the Two Extensions suggested splitting. Simultaneous Moves If both accept, each player i gets his allotted share (utility Chance Summary xi ). Otherwise, no player gets anything (utility 0).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 58 / 68 Simultaneous Moves Example: Three-Person Cake Splitting Game

Formally: Motivation

Definitions

Solution N = {1,2,3} Concepts One- 3 Deviation X = {(x1,x2,x3) ∈ [0,1] |x1 + x2 + x3 = 1} Property H = {hi} ∪ {hxi|x ∈ X} ∪ {hx,zi|x ∈ X,z ∈ {y,n} × {y,n}} Kuhn’s Theorem

P(hi) = {1} Two Extensions P(hxi) = {2,3} for all x ∈ X Simultaneous Moves ( Chance z ∈ { , , , , , } 0 if (y n) (n y) (n n) Summary ui (hx,zi) = for all i ∈ N xi if z = (y,y).

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 59 / 68 Simultaneous Moves Example: Three-Person Cake Splitting Game

Motivation

Definitions

Solution Subgame-perfect equilibria: Concepts One- Subgames after legal split (x1,x2,x3) by player 1: Deviation Property NE (y,y) (both accept) Kuhn’s NE (n,n) (neither accepts) Theorem

If x2 = 0, NE (n,y) (only player 3 accepts) Two Extensions If x3 = 0, NE (y,n) (only player 2 accepts) Simultaneous Moves Chance

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 60 / 68 Simultaneous Moves Example: Three-Person Cake Splitting Game

Subgame-perfect equilibria (ctd.): Entire game: Motivation Definitions Let s2 and s3 be any two strategies of players 2 and 3 Solution such that for all splits x ∈ X the profile (s2(hxi),s3(hxi)) is Concepts

one of the NEs from above. One- Let X = {x ∈ X |s (hxi) = s (hxi) = y} be the set of splits Deviation y 2 3 Property accepted under s and s . Distinguish three cases: 2 3 Kuhn’s Xy = /0 or x1 = 0 for all x ∈ Xy. Then (s1,s2,s3) is a Theorem subgame-perfect equilibrium for any possible s1. Two Extensions Xy =6 /0 and there are splits xmax = (x1,x2,x3) ∈ Xy that Simultaneous Moves maximize x1 > 0. Then (s1,s2,s3) is a subgame-perfect Chance

equilibrium if and only if s1(hi) is such a split xmax. Summary Xy =6 /0 and there are no splits (x1,x2,x3) ∈ Xy that maximize x1. Then there is no subgame-perfect equilibrium, in which player 2 follows strategy s2 and player 3 follows strategy s3.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 61 / 68 Chance Moves

Motivation Definition Definitions Solution An extensive game with chance moves is a tuple Concepts Γ = hN,H,P,fc,(ui )i∈Ni, where One- Deviation N, A, H and ui are defined as before, Property Kuhn’s the player function P : H \ Z → N ∪ {c} can also take the Theorem valuec for a chance node, and Two Extensions Simultaneous for each h ∈ H \ Z with P(h) = c, the function fc(·|h) is a Moves probability distribution on A(h) such that the probability Chance distributions for all h ∈ H are independent of each other. Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 62 / 68 Chance Moves

Motivation

Definitions

Intended meaning of chance moves: In chance node, an Solution applicable action is chosen randomly with probability Concepts One- according to fc. Deviation Property Strategies: Defined as before. Kuhn’s Outcome: For a given strategy profile, the outcome is a Theorem Two probability distribution on the set of terminal histories. Extensions Simultaneous Payoffs: For player i, Ui is the expected payoff (with Moves Chance

weights according to outcome probabilities). Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 63 / 68 Chance Moves

Example Motivation

Definitions P(hi) = 1 (2,3) Solution A B Concepts One- Deviation P(hAi) = c (1,4) (2,3) P(hBi) = c Property |h i 1 Kuhn’s fc(E A ) = 2 |h i 1 |h i 1 |h i 2 Theorem fc(D A ) = 2 D E fc(F B ) = 3 F G fc(G B ) = 3 Two Extensions P(hB,Fi) = 2 (0,3) (3,3) P(hB,Gi) = 2 Simultaneous (0,6) (2,2) Moves Chance H I K L Summary

(0,3) (2,2) (4,1) (3,3)

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 64 / 68 Chance Moves One-Deviation Property and Kuhn’s Theorem

Motivation

Definitions

Solution Concepts Remark: One- Deviation The one-deviation property and Kuhn’s theorem still hold in the Property presence of chance moves. When proving Kuhn’s theorem, Kuhn’s Theorem expected utilities have to be used. Two Extensions Simultaneous Moves Chance

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 65 / 68 Motivation

Definitions

Solution Concepts

One- Deviation Summary Property Kuhn’s Theorem

Two Extensions

Summary

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 66 / 68 Summary

For finite-horizon extensive games, it suffices to consider local deviations when looking for better strategies. Motivation Definitions

For infinite-horizon games, this is not true in general. Solution Concepts Every finite extensive game has a subgame-perfect One- equilibrium. Deviation Property

This does not generally hold for infinite games, no matter Kuhn’s is game is infinite due to infinite branching factor or Theorem Two infinitely long histories (or both). Extensions

Summary With chance moves, one deviation property and Kuhn’s theorem still hold. With simultaneous moves, Kuhn’s theorem no longer holds.

May 24, 2017 B. Nebel, R. Mattmüller – Game Theory 68 / 68