STOCHASTICPERTURBATIONSINGAMETHEORY ANDAPPLICATIONSTONETWORKS panayotis mertikopoulos Department of Physics National & Kapodistrian University of Athens A thesis submitted for the degree of Doctor of Philosophy Athens, June 2010 “But is mathematics not the sister, as well as the servant, of the arts? And does she not share the same touch of genius and melancholy madness?” — Harold Marston Morse, topologist “The happiest moments of my life have been those few which I have passed at home, in the bosom of my family.” — Thomas Jefferson This thesis can be dedicated to none other but my beloved family, the warm home that instilled in me the love of knowledge. STOCHASTICPERTURBATIONSINGAMETHEORY ANDAPPLICATIONSTONETWORKS Department of Physics National & Kapodistrian University of Athens This dissertation is accepted in its present form by the thesis committee of Panayotis Mertikopoulos as satisfying the thesis requirement for the degree of Doctor of Philosophy. THETHESISCOMMITTEE aris l. moustakas (Chair) Date andreas polydoros Date dimitris frantzeskakis Date vi ABSTRACT As far as complex organisms go, the Internet is a textbook specimen: its billions of man-made components might be relatively simple to understand on an individual basis, but the complexity of their interactions far outstrips the deductive capabilities of their creators. As a result, a promising way to analyze these interactions is provided by game theory, a discipline devoted to the (often chimeric) task of making sense of an even more obscure and complex entity, human rationality. Nevertheless, this inherent goal of game theory is also the source of a fundamental controversy: is it really possible to apply our understanding of human interactions and behavioral patterns to networks of machines, devoid as they are of the feelings and irrational drives that characterize the former? The way out of this impasse is to recall that game theory encompasses the study of “rationality” in its most abstract form, even when confronted with entities that have little or no capacity for rational thought (e.g. animals, bacteria, or, in our case, computers). This line of reasoning has led to the emergence of evolutionary game theory (EGT), an offshoot of game theory which is concerned with large populations of barely sentient species that interact with each other in competitive habitats that promote natural selection. As such, evolutionary methods appear particularly attractive as a means to understand the inner workings of communication networks: after all, it is a rare occasion when Internet users do not compete with one another for the resources of the network they live in. Perhaps the most well-studied model of population evolution in this context is that of the replicator dynamics, a dynamical system first introduced by Taylor and Jonker( 1978) to study the interactions of different phenotypes within biological species. These studies quickly attracted the interest of game theorists and, after about a decade, eventually culminated in the “folk theorem” of EGT, a theorem which links evolution and rationality by showing that what appears as the consequence of rational thought is, in fact, the byproduct of natural selection favoring the survival of the “fittest”. On the other hand, if rationality can be construed as the outcome of an evo- lutionary process, then, by providing appropriate selection criteria, evolution can also be steered to any state which is efficient with respect to these criteria. In this way, the replicator dynamics take on a different guise, that of a learning mechanism which the users of a network might employ in order to reach a “socially efficient” steady state; put differently, by manipulating the incentives of the “players”, the designers of the “game” can guide them to whatever state suits their purpose. The main issue that this dissertation seeks to address is what happens if, in addition to the interactions between the players of a game (e.g. the users of a network), the situation is exacerbated by the interference of an assortment of exogenous and unpredictable factors, commonly referred to as “nature”. We find that this random interference differentiates crucially between the evolutionary and learning approaches, leading to different (stochastic) versions of the replicator dynamics. Rather surprisingly, in the case of learning, we find that many aspects of rationality remain unperturbed by the effects of noise: regardless of the vii fluctuations’ magnitude, players are still able to identify suboptimal actions, something which is not always possible in the evolutionary setting. Even more to the point, the “strict (Nash) equilibria” of the game (an important class of steady states) turns out to be stochastically stable and attracting, again irrespective of the noise level. From the viewpoint of network theory (where stochastic perturbations are, quite literally, omnipresent), the importance of these results is that they guarantee the robustness of the replicator dynamics against the advent of noise. In this way, if the users of a stochastically fluctuating network adhere to a replicator learning scheme and are patient enough, we show that the flow of traffic in the network converges to an invariant (stationary) distribution which is sharply concentrated in a neighborhood of the network’s equilibrium point. DIAGRAMMATICOUTLINEOFTHETHESIS Chapter 1 Brief introduction Chapter 2 Chapter 3 Game theory essentials Itô calculus overview Chapter 4 Perturbations in Nash games Chapter 5 Perturbed population games Chapter 6 The routing problem Chapter 7 —— required Wireless network games – – – helpful viii PUBLICATIONS Some of the ideas presented in this thesis have appeared previously in the following publications: 1. P. Mertikopoulos and A. L. Moustakas: “Balancing traffic in networks: redundancy, learning and the effect of stochastic fluctuations”, submitted. URL: http://arxiv.org/abs/0912.4012. 2. P. Mertikopoulos and A. L. Moustakas: “The emergence of rational behavior in the presence of stochastic perturbations”, in The Annals of Applied Probability, vol. 20, no. 4, July 2010. URL: http://arxiv.org/ abs/0906.2094. 3. P. Kazakopoulos, P. Mertikopoulos, A. L. Moustakas and G. Caire: “Liv- ing at the edge: a large deviations approach to the outage MIMO capac- ity”, to appear in the IEEE Transactions on Information Theory. Available online: http://arxiv.org/abs/0907.5024. 4. P. Kazakopoulos, P. Mertikopoulos, A. L. Moustakas and G. Caire: “Dis- tribution of MIMO mutual information: a large deviations approach”, in ITW ’09: Proceedings of the 2009 IEEE Workshop on Networking and Information Theory. 5. P. Mertikopoulos and A. L. Moustakas: “Learning in the presence of noise”, in GameNets ’09: Proceedings of the 1st International Conference on Game Theory for Networks, May 2009. 6. P. Mertikopoulos and A. L. Moustakas: “Correlated anarchy in overlap- ping wireless networks”, in IEEE Journal on Selected Areas in Communi- cations, vol. 26, no. 7, special issue on the applications of game theory to communication networks, September 2008. URL: http://arxiv.org/ abs/0805.0963. 7. P. Mertikopoulos, N. Dimitriou and A. L. Moustakas: “Vertical handover between service providers”, in WiOpt ’08: Proceedings of the 6th Interna- tional Symposium on Modelling and Optimization of Wireless Networks, April 2008. 8. N. Dimitriou, P. Mertikopoulos and A. L. Moustakas: “Vertical handover between wireless standards”, in ICC’08: Proceedings of the 2008 IEEE International Conference on Communications, May 2008. 9. P. Mertikopoulos and A. L. Moustakas: “The Simplex Game: can selfish users learn to operate efficiently in wireless networks?”, in GameComm ’07: Proceedings of the 1st International Workshop on Game Theory for Com- munication Networks, October 2007. ix “Science, my lad, is made up of mistakes, but they are mistakes which it is useful to make, because they lead little by little to the truth.” — Jules Verne, A Journey to the Center of the Earth ACKNOWLEDGEMENTS Towards the end of “The Lord of the Rings” trilogy, and after having spent the most perilous part of a three-book journey keeping his master Frodo out of harm’s way, Sam Gamgee was confronted in the safety of the Shire by an angry Jolly Cotton who asked him why he is not “looking out for Frodo now that things are looking dangerous”. To that question, Sam surmised that he can either give a two-week answer or none at all, and, being pressed for time, chose to give the latter. In writing this acknowledgements section, I found myself in a similar dilemma because, for a long time, I had contemplated writing only two words: “Ari, thanks!” The reason that I chose not to take Sam’s approach and let two words carry the weight of many has more to do with my advisor’s character and modesty than with anything else: I doubt that he would find it fair or fitting to be favored in this way over the large number of people who have helped shape this thesis and its author. Nevertheless, this thesis would never have been written if it weren’t for him, so the credit is his, whether he ever claims it or not. The journey that I have Aris to thank for is a long one and I can only hope that it does not end with this thesis. Without a doubt, the going has not always been pleasant, but he was always there to carve a path where I failed. His door was always open, and he opened it as if I were an equal, not a student; he gave me the freedom to pursue what I wanted and was patient enough to see my choices through, even the failed ones (and there were many); whenever I despaired with a proof or a calculation, Aris was always there to lend a helping hand and a word of support; and in my meandering monologues of technical details, he would always identify the essence and the correct path to proceed. But, above all else, Aris has been a friend, a wise friend who gave advice carefully but freely, and on whose experience and kindness I could always draw when needed.