Extensive-Form Games with Imperfect Information

Jeffrey Ely

May 6, 2015

Jeffrey Ely Extensive-Form Games with Imperfect Information Example

¨¨ 2, 2 ¨¨ A¨ r ¨¨ 3, 3 ¨¨ C ¨ ¨ ¨ r ¨ ¨¨ Player 1 H Player¨¨ 1 H H ¨ H b H Up ¨ r H H ¨ p D H H ¨ p HH B HH ¨ p 0, 0 H ¨ p PlayerH 2 ¨ 1 p r HH p H p rr H p ¨ 0, 0 H p C ¨¨ Down H p ¨ r H p ¨ HHp¨ Player 1 pH p H prr H D HH H 3, 3 r

Jeffrey Ely Extensive-Form Games with Imperfect Information Extensive-Form Games With Imperfect Information

Finite No simultaneous moves: each node belongs to a single player (possibly “chance.”) Information Sets

I The nodes h at which i moves are partitioned into sets H. I The interpretation is that player i knows that it is his move and some node in the set has been reached, he does not know which one. (Behavioral) Strategies Subgames

I Always begin with singleton information sets I Can’t break information sets I Nash and Subgame Perfect Nash Equilibria

Jeffrey Ely Extensive-Form Games with Imperfect Information The Weakness of SPNE 1 ¨¨ HH ¨ b H Stay out ¨ H ¨¨ HHEnter, Red ¨ Enter, Green H ¨¨ HH ¨ H ¨¨ 2 HH ¡A ¡A (0,r 0) rpppppppppppppppppppp r rr Fight ¡ A Fight ¡ A ¡ AAccom. ¡ AAccom. ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (−1,r−2) (1, −r 1) (−1,r−2) (1, −r 1)

This example has no proper subgames so SPNE is just NE. So we need to extend the concept of “sequential rationality”

Jeffrey Ely Extensive-Form Games with Imperfect Information Beliefs

Does the player’s continuation strategy maximize his continuation payoff? We need a way to pose the same question at a non-singleton information set. But what is the continuation payoff at a non-singleton information set? We need to specify the player’s beliefs over the nodes in the information set.

Definition

A system of beliefs is a collection µ = (µ1, µ2, ... , µN ) where µi (·|H) ∈ ∆H specifies a probability distribution over the nodes in any information set H belonging to player i. We interpret µ(·|H) ∈ ∆H as i’s belief about which node in H has been reached.

Jeffrey Ely Extensive-Form Games with Imperfect Information Continuation Payoffs

Given a behavioral strategy profile σ At any node h belonging to i we can compute the probability distribution over terminal nodes. And thus we can compute the expected payoff to i conditional on having arrived at h, ui (σ|h) Now to compute the continuation payoff at a non-singleton information set H, we take expectations with respect to the belief µ(·|H): ui (σ|H) = ∑ µ(h|H)ui (σ|h) h∈H

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Rationality

Definition A behavioral strategy profile σ is sequentially rational relative to a system of beliefs µ if for each i and for each information set H belonging to i,

0 ui (σ|H) ≥ ui (σi , σ−i |H)

0 for every alternative strategy σi .

Jeffrey Ely Extensive-Form Games with Imperfect Information Assessments

In extensive form games with imperfect information a solution is described by two objects The strategy profile The system of beliefs A pair (σ, µ) consisting of the strategy profile σ and the system of beliefs µ is called an assessment. Solution concepts will impose requirements on the assessment: Sequential rationality Some condition on the beliefs.

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Rationality

1 ¨¨ HH ¨ b H Stay out ¨¨ HH Enter, Red ¨Enter, Green H ¨¨ HH ¨¨ HH ¨¨ 2 HH ¡A ¡A (0,r 0) rpppppppppppppppppppp r rr Fight ¡ A Fight ¡ A ¡ AAccom. ¡ AAccom. ¡ A ¡ A ¡ A ¡ A ¡ A ¡ A (−1,r−2) (1, −r 1) (−1,r−2) (1, −r 1)

Sequential rationality alone restores the intuitive solution of this game.

Jeffrey Ely Extensive-Form Games with Imperfect Information Beliefs

¨¨ 1, −1 H ¨ ¨¨ r ¨ Player¨ 1 H ¨¨ H ¨ r H H¨ p T H ¨ p HH ¨ p −1, 1 ¨¨ p Player 2 H 1 p r H p H p b H p ¨ −1, 1 H p H ¨¨ T H p ¨ r H p ¨ HHp¨ Player 1 pH p H prr H T HH H 1, −1 r

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Rationality Alone Is Not Enough

Sequential rationality alone is not enough for this game. Consider the assessment in which both players play T but 2’s belief assigns probability greater than 1/2 to the top node. It is sequentially rational. But it isn’t even a Nash equilibrium. (This is Matching Pennies)

Jeffrey Ely Extensive-Form Games with Imperfect Information Bayes’ Rule on the Path

Given a behavioral strategy profile, we say that a system of beliefs satisfies Bayes’ rule on the path if at every information set H which is arrived at with positive probability, the belief µi (·|H) is the conditional probability distribution.

Jeffrey Ely Extensive-Form Games with Imperfect Information (Weak) Perfect Bayesian Equilibrium

Definition An assessment (σ, µ) is a (weak) Perfect Bayesian Equilibrium if it is sequentially rational and satisfies Bayes’ rule on the path.

So in an extensive form game with imperfect information, to “solve” for a Weak PBE is to State the strategy profile σ. State the system of beliefs µ. Verify that the system of beliefs is derived from σ on the path. Verify that the strategy profile is sequentially rational relative to µ.

Jeffrey Ely Extensive-Form Games with Imperfect Information Problem with Weak PBE

¨¨ −1/2, 0, 0 ¨¨ A¨ r ¨¨ −1, 1, −1 ¨¨ H ¨ ¨ ¨ r ¨ ¨¨ Player 1 H Player¨¨ 3 H H ¨ H b H ¨ r H H H¨ p T H H ¨ p HH B HH ¨ p 1, −1, 1 H ¨ p PlayerH 2 ¨ 1 p r HH p H p rr H p ¨ 1, −1, 1 H p H ¨¨ T H p ¨ r H p ¨ HHp¨ Player 3 pH p H prr H T HH H −1, 1, −1 r

Jeffrey Ely Extensive-Form Games with Imperfect Information Problems with Weak PBE

Consider the following assessment: 1 Plays A 2 Plays T 3 plays T 3’s belief assigns probability greater than 1/2 to his top node. This is a weak PBE Sequentially rational Satisfies Bayes’ rule on the path. But it is not a subgame perfect Nash equilibrium. (The subgame is matching pennies.) The unique SPE has 1 playing B.

Jeffrey Ely Extensive-Form Games with Imperfect Information Related Problem

¨¨ 2, 2 ¨¨ A¨ r ¨¨ 3, 0 ¨¨ C ¨ ¨ ¨ r ¨ ¨¨ Player 1 H Player¨¨ 1 H H ¨ H b H Up ¨ r H H ¨ p D H H ¨ p HH B HH ¨ p 0, 3 H ¨ p PlayerH 2 ¨ 1 p r HH p H p rr H p ¨ 0, 3 H p C ¨¨ Down H p ¨ r H p ¨ HHp¨ Player 1 pH p H prr H D HH H 3, 0 r

Jeffrey Ely Extensive-Form Games with Imperfect Information Failure of OSDP for Weak PBE

Consider the following assessment. 1 Plays (A, C ) 2 Plays Down 1 assigns probability 1 to the top node in his information set. This is not sequentially rational for player 1 he can improve his payoff at the initial node by changing his strategy to B, D. But there is no profitable one stage deviation at any information set.

Jeffrey Ely Extensive-Form Games with Imperfect Information A Partial Fix

Bayes’ rule where possible. i.e. suppose All of the nodes in an information set have a common immediate predecessor And that predecessor is a singleton information set Then the mixed action at that predecessor should define the belief at the information set. There are other examples where it is possible to apply Bayes’ rule even at off-path information sets. Applying the “where possible” principle is a bit of a case-by-case matter.

Jeffrey Ely Extensive-Form Games with Imperfect Information Application: Take it or leave it offers

Single object for sale. Player 1 is the seller, Player 2 is a buyer. The buyer has a value v > 0 for the object. The seller has a prior belief µ about v. The seller demands a price p for the good. The buyer accepts or rejects. Payoffs from sale:

I v − p for the buyer I p for the seller Payoffs are zero if no sale.

Jeffrey Ely Extensive-Form Games with Imperfect Information Extensive Form

Game begins with a move by Nature. (By the way this is how we put incomplete information into an extensive-form game.) Nature selects v with an exogenous probability distribution given by the prior µ. Player 1 does not observe Nature’s move (A single information set includes them all.) Player 2 observes the offer made as well as Nature’s move (singleton information set.)

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Rationality For the Buyer

Because all of the buyer’s moves are at singleton information sets, sequential rationality pins down his continuation strategy, except when indifferent. The buyer accepts any offer strictly below his value. The buyer rejects any offer strictly above his value. Sequential rationality alone does not restrict his response to prices equaling his value. (Think of the complete information ultimatum game.)

Jeffrey Ely Extensive-Form Games with Imperfect Information Binary Values

Suppose v can take on two possible values {vl , vh} with vh > vl .

With probabilities µh and µl . Bayes’ rule now pins down the seller’s belief at his information set.

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Rationality for the Seller

We can immediately rule out certain price offers

No price offer less than vl can be sequentially rational.

No price offer between vl and vh can be sequentially rational.

No price offer greater than vh can be sequentially rational.

So the only possible sequentially rational price offers are p = vl and p = vh.

Jeffrey Ely Extensive-Form Games with Imperfect Information Marginal Revenue

Think back to textbook undergrad micro. A monopoly who lowers price faces a tradeoff Lower sales price More sales Our seller faces the same tradeoff, but “more sales” means higher probability of sale.

Jeffrey Ely Extensive-Form Games with Imperfect Information Marginal Revenue

At price vh, a sale occurs with probability µh < 1. At price vl : I Sale occurs with probability 1. The marginal sale earns revenue

µl · vl

I There is a loss of revenue on inframarginal sales equal to

µh(vh − vl )

Marginal revenue is the net:

µl · vl − µh(vh − vl )

Jeffrey Ely Extensive-Form Games with Imperfect Information Marginal Revenue

The seller will cut the price to vl if and only if marginal revenue is positive:

µh vl − (vh − vl ) > 0 µl

Jeffrey Ely Extensive-Form Games with Imperfect Information Perfect Bayesian Equilibrium

The seller offers vl if marginal revenue is positive, vh if negative. (Either if zero.) The buyer accepts the seller’s offer and any offer strictly greater than his value. The seller’s beliefs are given by Bayes’ rule

Jeffrey Ely Extensive-Form Games with Imperfect Information Marginal Revenue Is Cleaner In The Continuum Case

Suppose now that v is distributed according to a continuous CDF given by F with density f . f (v)v − [1 − F (v)] is the marginal revenue from reducing price below v (marginally). The seller wants to sell to type v if and only if

1 − F (v) v − > 0. f (v)

and so the optimal take-it-or-leave-it offer is the v at which marginal revenue is zero.

Jeffrey Ely Extensive-Form Games with Imperfect Information Kuhn’s Theorem

There is a sense in which we can interchangeably use mixed and behavioral strategies. For example, in a game of perfect information: Fix any mixed strategy. There is a behavioral strategy which, regardless of the opponent’s strategy, induces the same distribution over terminal nodes. Likewise for any behavioral strategy there is an equivalent mixed strategy. What about games with imperfect information?

Jeffrey Ely Extensive-Form Games with Imperfect Information Imperfect Recall

Wear Wear -1

Wash Wash

0 1

Consider the laundry game. Consider the behavioral strategy which randomizes with equal probability. This is the optimal strategy. It has no equivalent mixed strategy.

Jeffrey Ely Extensive-Form Games with Imperfect Information Kuhn’s Theorem

Theorem In a finite extensive-form game with perfect recall, for every mixed strategy there is at least one behavioral strategy which induces the same distribution over terminal nodes against all strategy profiles of the opponents. Likewise, for any behavioral strategy there is at least one mixed strategy which induces the same distribution over terminal nodes against any strategy profile of the opponents.

Jeffrey Ely Extensive-Form Games with Imperfect Information Consistency

If the behavioral strategy profile is fully mixed then all information sets are on the path. Beliefs are defined everywhere by Bayes’ rule. Thus, for any strategy profile, there are “nearby” strategy profiles which would give uniquely defined beliefs.

Definition An assessment (σ, µ) is consistent if there is a sequence of fully mixed behavioral strategy profiles which converges to σ such that the corresponding sequence of belief systems (derived using Bayes’ rule) converges to µ.

Jeffrey Ely Extensive-Form Games with Imperfect Information Sequential Equilibrium

Definition An assessment (σ, µ) is a sequential equilibrium if it is sequentially rational and consistent.

Every finite game has a sequential equilibrium. A sequential equilbirium is subgame-perfect. A sequential equilibrium has beliefs defined by Bayes’ rule wherever possible. When beliefs are consistent, the one-stage deviation principle holds. Additional restrictions are implied.

Jeffrey Ely Extensive-Form Games with Imperfect Information Forward Induction

1

Out In

2 2, 0 L R

1

l r l r 1, 3 0, 0 0, 0 3, 1

Jeffrey Ely Extensive-Form Games with Imperfect Information