Scaling Empirical Game-Theoretic Analysis by Ben-Alexander Cassell A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy (Computer Science and Engineering) in the University of Michigan 2014 Doctoral Committee: Professor Michael P. Wellman, Chair Professor John E. Laird Professor Jeffrey K. MacKie-Mason Professor Demosthenis Teneketzis ©Ben-Alexander Cassell 2014 To Dad, Mom, Mary Rose, and Josh. ii Acknowledgments I would first like to extend my deepest thanks to my advisor, Michael Wellman. Michael always seemed to know exactly when I needed encouragement, and when I needed to be pushed, to bring out the best in me. He gave me the flexibility to pursue the topics that I found most interesting while providing invaluable guidance. I would also like to thank my dissertation committee, John Laird, Jeffrey MacKie-Mason, and Demosthenis Teneketzis, without whom this work would not be possible. I consider each member of my committee to be extraordinary scientists, teachers, and friends. I also owe a large amount of gratitude to Carey Bagdassarian at the College of William and Mary, my first research mentor, for encouraging me to aim high with my ambitions. I am deeply thankful to the administrative staff of the Computer Science and Engi- neering department. In particular, I must thank Dawn Freysinger, Rita Rendell, and Cindy Watts, who always greeted me with a smile and a determination to resolve my administra- tive challenges. I would also like to thank DCO, and especially Laura Fink, for helping me manage the servers that have been instrumental to my research. The rigors of the PhD program would have been more daunting were it not for a group of amazing friends and colleagues. I would like to especially thank the friends who put up with living with me over the years, and who are the source of some of my fondest graduate school memories: Daniel Fabbri, Jamie Kidwell, Erin Payne, Quang Duong, David Meis- ner, Steven Pelley, and Michael Chow. I am deeply indebted to my lab-mate Patrick Jordan, who set a sterling example of hard work and dedication, and demonstrates an unwavering belief in my ability, even when I question it myself. I would also like to thank Andrea Jor- dan, who has always treated me like family. I would like to thank my friends/collaborators, Timur Alperovich and Bryce Wiedenbeck, for always stimulating my brain with interesting discussion. I would be remiss if I did not acknowledge the past and present members of the Strategic Reasoning Group for their camaraderie and support. I must also thank my longtime friends from my hometown, St. Louis: Tony Flesor, Dan Pritt, Brandon Walsh, and Sarah Brandt. I would not have gotten this far without their love. I must thank my parents, Stu Cassell, Mary Rose Cassell, and Carol Rutherford, for always supporting me in my educational endeavors. The pride they have in me is conta- gious, bolstering me when I am low and sharing in the joy when I succeed. Finally, I wish to thank my brother and best friend, Josh. iii TABLE OF CONTENTS Dedication ....................................... ii Acknowledgments ................................... iii List of Figures ..................................... vii List of Tables ...................................... viii Chapter 1 Introduction ..................................... 1 1.1 Empirical Game-Theoretic Framework...................2 1.1.1 Strategic Games..........................2 1.1.2 Empirical Game Models......................3 1.1.3 Solution Concepts.........................4 1.1.4 Scaling EGTA...........................4 1.2 Overview of Contributions.........................7 1.2.1 Application: Wireless Access Point Selection...........8 1.2.2 EGTAOnline: Software Infrastructure for Experiment Management8 1.2.3 Efficient Analysis of Large Game Data Sets............8 1.2.4 Bootstrap Methods for Sequential Estimation of Nash Equilibria.9 1.2.5 Application: Equity Premium Estimation in Asset Pricing....9 1.3 Guide to Reading this Thesis........................ 10 2 Application: Wireless Access Point Selection ................... 11 2.1 Related Work................................ 12 2.2 Game Description.............................. 13 2.2.1 Multiple AP Selection....................... 13 2.2.2 Information Models........................ 14 2.3 Strategies.................................. 15 2.3.1 Association Policies........................ 15 2.3.2 Probing Policies.......................... 17 2.4 Experiments................................. 18 2.5 Results.................................... 19 2.5.1 Bulletin Board Model....................... 19 2.5.2 Probing Model........................... 20 2.5.3 Social Welfare........................... 22 2.6 Summary: AP Selection Game....................... 23 iv 2.7 Scaling Lessons............................... 23 3 EGTAOnline: Software Infrastructure for Experiment Management ..... 25 3.1 Related Work................................ 26 3.2 Role Symmetry............................... 27 3.3 Data Compatibility............................. 28 3.4 Architecture................................. 29 3.4.1 Simulators............................. 29 3.4.2 Observations............................ 30 3.4.3 Schedulers............................. 31 3.4.4 Simulations............................. 33 3.4.5 Profiles............................... 34 3.4.6 Games............................... 34 3.5 Data Reuse................................. 34 3.6 Automated Refinement of Game Models.................. 37 3.6.1 Exploration of Profile Space.................... 38 3.6.2 Sequential Estimation of Empirical Games............ 39 3.7 In Production................................ 41 4 Efficient Analysis of Large Game Data Sets .................... 42 4.1 Background: Database Management.................... 43 4.2 Representing Games in a Database..................... 44 4.3 Game-Theoretic Primitives......................... 46 4.4 Identifying PSNE.............................. 48 4.5 PSNE-Finding Performance......................... 51 4.5.1 Comparison of SQL Algorithms.................. 51 4.5.2 Comparison to Gambit....................... 52 4.6 Incremental Analysis............................ 56 4.6.1 Incremental Regret Maintenance.................. 57 4.6.2 Identifying Maximal Complete Subgames............. 57 4.7 Discussion.................................. 60 5 Bootstrap Methods for Sequential Estimation of Nash Equilibria ....... 62 5.1 The Bootstrap................................ 64 5.2 Using the Bootstrap in Sample Control................... 66 5.3 Experimental Data Sets........................... 68 5.4 Sequential Classification of Profiles as Nash Equilibria.......... 69 5.5 Sequential Search for δ-Equilibria..................... 72 5.6 Discussion.................................. 76 6 Application: Equity Premium Estimation in Asset Pricing ........... 79 6.1 Background: Agent Modeling....................... 80 6.2 Ambiguity Aversion and the Equity Premium Puzzle........... 82 6.3 Empirical Game Model of Asset Pricing.................. 83 6.3.1 Market and Asset Models..................... 83 6.3.2 Agent Strategy Composition.................... 84 v 6.3.3 Estimating the Empirical Game.................. 85 6.4 Simulation of Asset Pricing under Ambiguous Information........ 87 6.4.1 Market Conditions......................... 87 6.4.2 News................................ 88 6.4.3 Traders............................... 88 6.5 Experiments................................. 90 6.5.1 Equilibrium Analysis........................ 91 6.5.2 Equity Premium Estimation.................... 93 6.6 Discussion.................................. 96 7 Conclusion ...................................... 100 7.1 Contributions................................ 100 7.1.1 Software Systems and Methods.................. 100 7.1.2 Applications............................ 103 7.2 Future Work................................. 104 7.3 Final Remarks................................ 105 Bibliography ...................................... 106 vi LIST OF FIGURES 1.1 Iterative, search-based approach to empirical game-theoretic analysis......6 2.1 Lowest expected delay among found Nash equilibria for different rates of work clearing....................................... 22 3.1 Supporting the EGTA process with EGTAOnline................. 30 3.2 Fraction of unreduced-game profile space for an n-player reduction that is covered by smaller reductions, shown for varying size of strategy set, M.... 37 3.3 Implementing a sequential estimation procedure with EGTAOnline....... 40 4.1 Schema for empirical games............................ 45 4.2 Data dependence graph for calculating regret of profiles in an example game.. 49 4.3 Finding best responses in the symmetric aggregations table........... 51 4.4 Performance of two SQL methods for identifying all PSNE........... 52 4.5 Comparing performance of PSNE finding methods under two cache scenarios. 55 6.1 Workflow diagram for using EGTA to estimate a non-payoff variables V .... 84 6.2 Abstract view of market simulation........................ 87 7.1 Idealized automated empirical game-theoretic analysis.............. 101 vii LIST OF TABLES 2.1 Equilibrium analysis of the bulletin board model................. 19 2.2 Equilibrium analysis of the probing model.................... 21 5.1 Sequential classification performance....................... 71 5.2 Stopping rule performance............................. 75 6.1 Base market configuration for experiments...................