Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 277 Games and Probabilistic Infinite-State Systems SVEN SANDBERG ACTA UNIVERSITATIS UPSALIENSIS ISSN 1651-6214 UPPSALA ISBN 978-91-554-6813-2 2007 urn:nbn:se:uu:diva-7607 ! " #$ #%%& %' ( ) ) ) * + , - * . .* #%%&* / 0 ) 1. . * 2 * #&&* 3# * * 0.45 6&316 1((7183 $1#* 9 : ) , , ) 1 , , * + ) , ) ) ) , * ; ) ) ) * 2 ) , , ) : ' * 0 , < , ) * / ) ) * ; ) ) 1 #1 1 = ))> * + : ) 1 ) ) * ; ) ) ) * + ) 1 * " , ) 1 * + * ; ) 1 =" : > ) * ! "! # * $ , * % , * ; ) * ; ) , ) ) * - , * ! 9. , 1 , * + ? 1 ) * ! , 4@ 9.* . : : , : * ; * &' ( ) : ) )) 4@ " : ) 1 + ) * ) + , --.) ) /.0120 ) ' A . #%%& 0..5 8( 18# 7 0.45 6&316 1((7183 $1# ' ''' 1&8%& = 'BB *:*B C D ' ''' 1&8%&> Till alla som gillar glass. ”Ska det va’ s˚asv˚art att fatta att detar ¨ falsk matematik” Peps Persson (om programvarufel) Contents 1 ListofPapers ...................................... 11 2 MyContribution .................................... 13 3 OtherPublications .................................. 15 4 Sammanfattning p˚a svenska (Summary in Swedish) . 17 5 Acknowledgments ................................... 21 6 Introduction ....................................... 23 6.1 Motivation...................................... 24 6.1.1 Verification.................................. 24 6.1.2 ModelChecking.............................. 25 6.1.3 Games..................................... 27 6.1.4 Probabilistic Systems . 29 6.1.5 Infinite-State Systems . 30 6.2 ModelsandObjectives............................ 32 6.2.1 From Probabilistic Systems to Games . 32 1 6.2.2 Formal Definition of 2 2 -PlayerGames ............ 33 6.2.3 Objectives.................................. 35 6.2.4 Infinite-State Probabilistic Models . 39 6.3 ProblemStatements.............................. 45 6.3.1 MemorylessDeterminacy....................... 45 6.3.2 ComputationProblems........................ 45 6.4 Contribution.................................... 47 6.4.1 MemorylessDeterminacy....................... 47 6.4.2 A Strategy Improvement Algorithm for Finite-State 1-PlayerLimit-AverageGames.................. 48 1 6.4.3 Classifying Infinite-State 2 -Player Systems: Eagerness 50 1 6.4.4 Quantitative Analysis of Infinite-State 2 -Player Systems50 1 6.4.5 Qualitative Analysis of Infinite-State 2 2 -Player Games 52 6.4.6 SummaryofPapers........................... 54 6.5 RelatedWork ................................... 55 6.5.1 Determinacy................................. 55 6.5.2 AlgorithmsforFinite-StateGames............... 57 6.5.3 Infinite-State Systems . 61 6.6 OpenProblemsandFutureWork.................... 64 6.6.1 Determinacy for Parity and Mean Payoff Games . 64 6.6.2 Strategy Improvement for Mean Payoff Games . 64 1 6.6.3 Classification of Infinite-State 2 -PlayerSystems..... 65 6.6.4 Forward Path Exploration Algorithms for Eager Infinite-State Systems . 65 1 6.6.5 Infinite-State 2 2 -PlayerGames.................. 66 6.6.6 Other Infinite-State Systems . 66 Bibliography .......................................... 67 A Memoryless determinacy of parity and mean payoff games: Asimpleproof...................................... 85 A.1 Introduction .................................... 87 A.2 ParityGames ................................... 90 A.3 Finite-DurationParityGames ...................... 90 A.4 Positional Strategies in Finite-Duration (Parity) Games . 91 A.5 Memoryless Determinacy of Finite-Duration Parity Games 93 A.6 Extension to Infinite-Duration Parity Games . 96 A.7 ExtensiontoMeanPayoffGames.................... 97 A.7.1 Finite and Infinite-Duration Mean Payoff Games . 97 A.7.2 Main Theorem for the Decision Version of Finite MeanPayoffGames........................... 98 A.7.3 Ergodic Partition Theorem for Mean Payoff Games . 99 A.8 Conclusions.....................................101 A.9 Bibliography....................................101 B A Combinatorial Strongly Subexponential Strategy ImprovementAlgorithmforMeanPayoffGames ...........105 B.1 Introduction ....................................107 B.2 Preliminaries....................................110 B.2.1 MeanPayoffGames...........................110 B.2.2 AlgorithmicProblemsforMPGs.................110 B.3 AHigh-LevelDescriptionoftheAlgorithm ............111 B.3.1 TheLongest-ShortestPathsProblem.............111 B.3.2 Relating the 0-Mean Partitioning and Longest-ShortestPathsProblems................113 B.3.3 TheAlgorithm...............................114 B.3.4 RandomizationScheme........................115 B.4 Retreats, Admissible Strategies, and Strategy Measure . 116 B.4.1 MeasuringtheQualityofStrategies ..............116 B.4.2 RequirementsfortheMeasure...................117 B.5 CorrectnessoftheMeasure.........................117 B.5.1 Attractiveness Implies Profitability . 118 B.5.2 Stability Implies Optimality . 120 B.6 EfficientComputationoftheMeasure ................122 B.7 ComplexityoftheAlgorithm .......................125 B.7.1 Complexity of Partitioning with Integer Thresholds . 125 B.7.2 Computing the Ergodic Partitioning . 126 B.8 The LSP Problem is in NP ∩ co-NP..................127 B.9 Exponential Sequences of Attractive Switches . 129 8 B.10ApplicationtoParityGames........................130 B.11Conclusions.....................................131 B.12Bibliography....................................132 C EagerMarkovChains ................................137 C.1 Introduction ....................................139 C.2 Preliminaries....................................142 C.3 Approximating the Conditional Expectation . 144 C.4 EagerAttractors.................................147 C.4.1 GR-Attractors...............................153 C.4.2 Probabilistic Lossy Channel Systems . 157 C.5 Bounded Coarseness . 158 C.6 Conclusion,Discussion,andFutureWork..............161 C.7 Bibliography....................................162 D Limiting Behavior of Markov Chains with Eager Attractors . 167 D.1 Introduction ....................................169 D.2 Preliminaries....................................172 D.3 ProblemStatements..............................174 D.4 OverviewoftheAlgorithms ........................176 D.5 TheSteadyStateDistribution ......................180 D.6 TheExpectedResidenceTime......................183 D.7 LimitingAverageExpectedReward..................187 D.8 ConclusionsandFutureWork.......................192 D.9 Bibliography....................................192 E StochasticGameswithLossyChannels ..................197 E.1 Introduction ....................................199 E.2 Preliminaries....................................202 E.3 Game Probabilistic Lossy Channel Systems (GPLCS) . 205 E.4 Reachability Games on GPLCS . 206 E.5 B¨uchi-GamesonGPLCS...........................209 E.6 ConclusionsandFutureWork.......................213 E.7 Bibliography....................................213 E.AAppendix:Proofs................................216 E.A.1 Section E.4 (Reachability Games on GPLCS) . 216 E.A.2 Section E.5 (B¨uchi-GamesonGPLCS)............217 Index ...............................................225 9 1. List of Papers This thesis is based on the following papers, which are referred to in the text by the capital letters A through E. A. Memoryless determinacy of parity and mean payoff games: Asimpleproof. Henrik Bj¨orklund, Sven Sandberg, and Sergei Vorobyov. In Theoretical Computer Science (TCS), 310(1-3):365–378, January 2004. B. A combinatorial strongly subexponential strategy improve- ment algorithm for mean payoff games. Henrik Bj¨orklund, Sven Sandberg, and Sergei Vorobyov. In Jiri Fiala, Vaclav Koubek, and Jan Kratochvil, editors, 29th In- ternational Symposium on Mathematical Foundations of Computer Science (MFCS ’04), volume 3153 of Lecture Notes in Computer Science (LNCS), pages 673–685. Springer, August 2004. C. Eager Markov chains. Parosh Aziz Abdulla, Noomene Ben Henda, Richard Mayr, and Sven Sandberg. In Susanne Graf and Wenhui Zhang, editors, Proceedings of the Fourth international symposium on Automated Technology for Veri- fication and Analysis (ATVA ’06), volume 4218 of Lecture Notes in Computer Science (LNCS), pages 24–38, Berlin Heidelberg, 2006. Springer Verlag. D. Limiting behavior of Markov chains with eager attractors. Parosh Aziz Abdulla, Noomene Ben Henda, Richard Mayr, and Sven Sandberg. In Pedro R. D’Argenio, Andrew Milner, Gerardo Rubino, editors, 3rd International Conference on the Quantitative Evaluation of Sys- Tems (QEST ’06), pages 253–262. IEEE, August 2006. E. Stochastic games with lossy channels. Parosh Aziz Abdulla, Noomene Ben Henda, Richard Mayr, and Sven Sandberg. Technical Report 2007-005, Department of Information Technology, Uppsala University, February 2007. Reprints were made with permission from the publishers. 11 2. My Contribution A. I suggested the structure of the inductive proof. All authors did approximately the same share of writing. B. I suggested the proofs of Theorems B.11 and B.12 and invented the method to save a factor n in each iteration.All