Backwards Induction for Games of Infinite Horizon
Dietmar Berwanger
LaBRI Bordeaux
G Cambridge 2006
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 1 / 17 Games with many players
zero-sum two-player games: conflict I optimal behaviour – winning; I determinacy: ex-ante, non-interactive solutions.
non-zero sum, n-player games: tacit cooperation, coordination I deeper forms of rationality; I solution concepts: evolutionary, epistemic, operational.
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 2 / 17 Equilibria. Why not?
Nash Equilibrium: self-enforcing profile of strategies no player has an incentive to deviate explains why rational agents jointly choose a steady state
But games may have multiple equilibria . . . Prediction, prescription?
1. Coordination problems
L R
T ¡ + +
B + + ¡
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 3 / 17 Equilibria. Why not?
Nash Equilibrium: self-enforcing profile of strategies no player has an incentive to deviate explains why rational agents jointly choose a steady state
But games may have multiple equilibria . . . Prediction, prescription?
1. Coordination problems
L R
T ¡ + +
B + + ¡
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 3 / 17 2. Dynamic inconsistency: non-credible threats
+
+
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17 2. Dynamic inconsistency: non-credible threats
+
+
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17 2. Dynamic inconsistency: non-credible threats
+
+
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17
... even with few payoffs
+ ++
++ +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17
... even with few payoffs
+ ++
+ ++
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17
... even with few payoffs
+ ++
+ ++
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite
+ + + + +
+ + + + + + + +
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Arguments
a natural concept – the earliest in literature (Zermelo 1913); eliminates non-credible threats – subgame perfection (Selten 1965); implied by Common Knowledge of Rationality (Aumann 1995); promoted by iterated admissibility.
Nice to have in the infinite. But where to start from, and when to stop?
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 7 / 17 Reaching out for infinity
Idea Attach strategy automata to positions: I act as local maximisers (little players); I aware of the other automata’s programs; I take decisions in order to care for their vertex. Maintain one global authority for each (big) player i: I aggregates decision of local players; I distributes updates to their programs, accordingly. Background Attack subgames with zero-sum techniques. Use iterated admissibility as a foundation.
Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 8 / 17 Path Games
n players, perfect information, win-or-lose payoffs
¢¡ i
Γ = , (W )i ¡ i finite game graph V, E, V £ : set V of positions, E legal moves; ¤ ˙ i player-turn partition V = i Wi winning objective: ¡ ω-regular subset of Vω, paths in ; (may overlap) Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 9 / 17 Path Games n players, perfect information, win-or-lose payoffs ¢¡ i Γ = , (W )i ¡ i finite game graph V, E, V £ : set V of positions, E legal moves; ¤ ˙ i player-turn partition V = i Wi winning objective: ¡ ω-regular subset of Vω, paths in ; (may overlap) Play in (Γ, u): infinite path starting from u, formed interactively Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 9 / 17 Strategies Extensive form of a game: ¡ i Γ, u =ˆ T( ), (W )i ¡ ¡ T( ) ω-tree of histories, unravelling of = (V, E, Vi) from u; – T[v] := histories ending at v V i ¤ i – player-turn partition T := v V T[v] Strategy for player i starting from u: i ¡ i s : T T maps any p T to some successor. ¡ Irepresentation as a tree: colouring of T( ). i S [u] set of strategies starting from u V; i i S := ( S [u] )u V . Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 10 / 17 Subgames I Rooted subgame: Γh game induced by all histories comparable to h (extensive form). . . . assuming that h is reached. I Subgame (Γ, Q): strategy space restricted to Q S. i i i – here, only rectangular sets: Q = (Q )i Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 11 / 17 Parity Games: canonical on finite graphs ¡ i Parity game: regular game with Γ = , (Ω )i ¡ ¡ game graph over V = 0, . . . m ; i Ω play is winning ¢¤£ the least recurring state Ωi. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 12 / 17 Parity Games: canonical on finite graphs ¡ i Parity game: regular game with Γ = , (Ω )i ¡ ¡ game graph over V = 0, . . . m ; i Ω play is winning ¢¤£ the least recurring state Ωi. Prefix independence of objectives I history independence of subgames: ¡ ¡ Γh Γh Γ, v for any h, h T[v]. I strategies in parity games strategies in regular games on finite graphs Henceforth, restrict to parity games. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 12 / 17 A local elimination criterion Fix a subgame (Γ, Q, v). i A strategy s Q [v] is incautious if, in the subgame, player i can win all plays, but with s he loses some, or player i can win some plays, but with s he loses all . I Elimination set ∆(Q, v): all incautious strategies. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 13 / 17 Backwards Induction in the infinite Incorporate avoidance of incautious strategies into rationality. Enforce subgame perfection. Assume common knowledge of rationality. i i I Induction stages: Q0 := S ; i i i ¡ Qα+1 := s Qα : sh ∆(Q, v) for any v V , h T[v] . I Reach deflationary fixed point Q ¡ when Qα = Qα+1. i Solution concept: Q ¡ – inductive outcome. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 14 / 17 Interpretation A strategy automaton has several initial states, one for every position u. ¡ From the initial state for u, it runs on unravellings of , u (states V . . . ) Iterative construction: ¡ ¡ 0[u] recognises unravelling of , u. α ¡ α + 1: for every player i: I i £ for each v ¢ V , build an automaton [v] j recognising the incautious strategies at v, using ¤ α; I ¤ to construct ¤ α+1[u], start with α[u] and branch universally at every state (v, . . . ) to ¥¦£ [v]. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 15 / 17 Results Soundness 1 The set of backwards-inductive strategies is non-empty. 2 The induction on a parity game with m states terminates in nm steps. Finite-state compatibility For a regular game, the set of backwards inductive strategies is regular. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 16 / 17 Conclusion A solution technique for non-zero-sum games: Produces the backwards induction outcome in the finite. Respects iterative admissibility on the rationality path. Attains subgame perfection, even off the rationality path. Provides a handle to bounded rationality. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 17 / 17