Backwards Induction for Games of Infinite Horizon Dietmar Berwanger LaBRI Bordeaux Games Cambridge 2006 Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 1 / 17 Games with many players zero-sum two-player games: conflict I optimal behaviour { winning; I determinacy: ex-ante, non-interactive solutions. non-zero sum, n-player games: tacit cooperation, coordination I deeper forms of rationality; I solution concepts: evolutionary, epistemic, operational. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 2 / 17 Equilibria. Why not? Nash Equilibrium: self-enforcing profile of strategies no player has an incentive to deviate explains why rational agents jointly choose a steady state But games may have multiple equilibria . Prediction, prescription? 1. Coordination problems L R T ¡ + + B + + ¡ Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 3 / 17 Equilibria. Why not? Nash Equilibrium: self-enforcing profile of strategies no player has an incentive to deviate explains why rational agents jointly choose a steady state But games may have multiple equilibria . Prediction, prescription? 1. Coordination problems L R T ¡ + + B + + ¡ Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 3 / 17 2. Dynamic inconsistency: non-credible threats + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17 2. Dynamic inconsistency: non-credible threats + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17 2. Dynamic inconsistency: non-credible threats + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 4 / 17 ... even with few payoffs + ++ ++ + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17 ... even with few payoffs + ++ + ++ Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17 ... even with few payoffs + ++ + ++ Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 5 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Backwards Induction in the finite + + + + + + + + + + + + + Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 6 / 17 Arguments a natural concept { the earliest in literature (Zermelo 1913); eliminates non-credible threats { subgame perfection (Selten 1965); implied by Common Knowledge of Rationality (Aumann 1995); promoted by iterated admissibility. Nice to have in the infinite. But where to start from, and when to stop? Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 7 / 17 Reaching out for infinity Idea Attach strategy automata to positions: I act as local maximisers (little players); I aware of the other automata's programs; I take decisions in order to care for their vertex. Maintain one global authority for each (big) player i: I aggregates decision of local players; I distributes updates to their programs, accordingly. Background Attack subgames with zero-sum techniques. Use iterated admissibility as a foundation. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 8 / 17 Path Games n players, perfect information, win-or-lose payoffs ¢¡ i Γ = ; (W )i<n £ ¡ i finite game graph V; E; V £ : set V of positions, E legal moves; ¤ _ i player-turn partition V = i<nV ; Wi winning objective: ¡ !-regular subset of V!, paths in ; (may overlap) Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 9 / 17 Path Games n players, perfect information, win-or-lose payoffs ¢¡ i Γ = ; (W )i<n £ ; initial state u V ¡ i finite game graph V; E; V £ : set V of positions, E legal moves; ¤ _ i player-turn partition V = i<nV ; Wi winning objective: ¡ !-regular subset of V!, paths in ; (may overlap) Play in (Γ; u): infinite path starting from u, formed interactively Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 9 / 17 Strategies Extensive form of a game: ¡ i Γ; u =^ T( ); (W )i<n £ ¡ ¡ T( ) !-tree of histories, unravelling of = (V; E; Vi) from u; { T[v] := histories ending at v V i ¤ i { player-turn partition T := v V T[v] Strategy for player i starting from u: i ¡ i s : T T maps any p T to some successor. ¡ Irepresentation as a tree: colouring of T( ). i S [u] set of strategies starting from u V; i i S := ( S [u] )u V : Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 10 / 17 Subgames I Rooted subgame: Γh game induced by all histories comparable to h (extensive form). assuming that h is reached. I Subgame (Γ; Q): strategy space restricted to Q S. i i i { here, only rectangular sets: Q = (Q )i<n and Q = (Q [v])v V . assuming that any player i will use a strategy from Qi. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 11 / 17 Parity Games: canonical on finite graphs ¡ i Parity game: regular game with Γ = ; (Ω )i<n £ : ¡ ¡ game graph over V = 0; : : : m ; i Ω play is winning ¢¤£ the least recurring state Ωi. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 12 / 17 Parity Games: canonical on finite graphs ¡ i Parity game: regular game with Γ = ; (Ω )i<n £ : ¡ ¡ game graph over V = 0; : : : m ; i Ω play is winning ¢¤£ the least recurring state Ωi. Prefix independence of objectives I history independence of subgames: ¡ ¡ Γh Γh Γ; v for any h; h T[v]: I strategies in parity games strategies in regular games on finite graphs Henceforth, restrict to parity games. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 12 / 17 A local elimination criterion Fix a subgame (Γ; Q; v). i A strategy s Q [v] is incautious if, in the subgame, player i can win all plays, but with s he loses some, or player i can win some plays, but with s he loses all . I Elimination set ∆(Q; v): all incautious strategies. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 13 / 17 Backwards Induction in the infinite Incorporate avoidance of incautious strategies into rationality. Enforce subgame perfection. Assume common knowledge of rationality. i i I Induction stages: Q0 := S ; i i i ¡ Qα+1 := s Qα : sh ∆(Q; v) for any v V ; h T[v] : I Reach deflationary fixed point Q ¡ when Qα = Qα+1. i Solution concept: Q ¡ { inductive outcome. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 14 / 17 Interpretation A strategy automaton has several initial states, one for every position u. ¡ From the initial state for u, it runs on unravellings of ; u (states V : : : ) Iterative construction: ¡ ¡ 0[u] recognises unravelling of ; u. α ¡ α + 1: for every player i: I i £ for each v ¢ V , build an automaton [v] j recognising the incautious strategies at v, using ¤ α; I ¤ to construct ¤ α+1[u], start with α[u] and branch universally at every state (v; : : : ) to ¥¦£ [v]. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 15 / 17 Results Soundness 1 The set of backwards-inductive strategies is non-empty. 2 The induction on a parity game with m states terminates in nm steps. Finite-state compatibility For a regular game, the set of backwards inductive strategies is regular. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 16 / 17 Conclusion A solution technique for non-zero-sum games: Produces the backwards induction outcome in the finite. Respects iterative admissibility on the rationality path. Attains subgame perfection, even off the rationality path. Provides a handle to bounded rationality. Dietmar Berwanger (LaBRI) Backwards Induction / Infinite July 4, 2006 17 / 17.