Dynamic Selfish Routing
Total Page:16
File Type:pdf, Size:1020Kb
DYNAMIC SELFISH ROUTING Von der Fakultat¨ fur¨ Mathematik, Informatik und Naturwissenschaften der Rheinisch-Westfalischen¨ Technischen Hochschule Aachen zur Erlangung des akademischen Grades eines Doktors der Naturwissenschaften genehmigte Dissertation vorgelegt von Diplom-Informatiker SIMON FISCHER aus Hagen Berichter: Universitatsprofessor¨ Dr. Berthold Vocking¨ Universitatsprofessor¨ Dr. Petri Mah¨ onen¨ Tag der mundlichen¨ Prufung:¨ 13. Juni 2007 Diese Dissertation ist auf den Internetseiten der Hochschulbibliothek online verfugbar.¨ Dynamic Selfish Routing SIMON FISCHER Aachen • June 2007 iv Abstract This thesis deals with dynamic, load-adaptive rerouting policies in game theoretic settings. In the Wardrop model, which forms the basis of our dynamic population model, each of an infinite number of agents injects an infinitesimal amount of flow into a network, which in turn induces latency on the edges. Each agent may choose from a set of paths and strives to minimise its sustained latency selfishly. Population states which are stable in that no agent can improve its latency by switching to another path are referred to as Wardrop equilibria. A variety of results in this model have been obtained recently. Most of these revolve around the price of anarchy, which measures the degrada- tion of performance due to the selfish behaviour of the agents as compared to a centrally optimised solution. Most of these analyses consider the no- tion of Wardrop equilibria as a solution concept of selfish routing games, but disregard the question of how such an equilibrium can be attained in the first place. In fact, common game theoretic assumptions needed for the motivation of equilibria in general strategic games are not satisfied for rout- ing games in large networks, the Internet being the prime example. These assumptions comprise accurate knowledge of the network and its latency functions as well as unbounded rationality and reasoning capabilities. The question of how Wardrop equilibria can be attained by a population of self- ish agents is the central topic of this thesis. Our first approach is inspired by evolutionary game theory. We show that Wardrop equilibria actually satisfy a stronger stability criterion, called evolutionary stability, which can be motivated by milder assumptions. We model a population of agents following some very simple randomised self- ish rules allowing them to exchange their path for a better one from time to time. An agent samples another path according to some probability dis- tribution and possibly exchanges its current path for the new one with a probability that increases with the latency gain. The behaviour of such a population over time can be described in terms of a system of differential equations. For a concrete choice of probability distributions involved in the above rules, these differential equations take the form of the so-called v replicator dynamics. As a first result, we show that a population following this dynamics converges towards the set of Wardrop equilibria. This convergence result implicitly assumes perfect information in that rerouting decisions made by other agents are observable by other agents immediately. In communication networks, however, such rerouting deci- sions may be observed only with a delay. In both theory and practise, it is known that rerouting decisions based on stale information may lead to undesirable oscillation effects which seriously harm performance. We con- sider an extension of our dynamic population model in which information is updated only at intervals of a given length. We show that, despite of this, convergence to Wardrop equilibria is still possible. For any class of latency functions with bounded slope and any finite update period length, policies from a large class converge towards Wardrop equilibria provided that they satisfy a certain smoothness condition. This condition requires the migra- tion rate to be reduced by a factor that is reciprocal to the maximum slope and the update period length. Finally, we show that Wardrop equilibria can be approached quickly. This is an issue of particular importance if the network or the flow demands themselves may change over time. To measure the speed of convergence, we consider the time to reach approximate Wardrop equilibria. We show that by applying a clever sampling technique it is possible to reach an ap- proximate Wardrop equilibrium in time polynomial in the approximation quality and the representation length of the network, e. g., for positive poly- nomial latency functions in coefficient representation. In particular, our bounds depend on the maximum slope of the latency functions only log- arithmically, which improves over earlier results in similar models which depend linearly on this parameter. We show that the crucial parameter that limits the speed of convergence is not the slope, but rather the elasticity of the latency functions. This parameter is closely related to the degree of polynomials. Based on these positive theoretical results, we design a dynamic traf- fic engineering protocol which we evaluate by simulations. Our protocol splits traffic bound for the same destination among alternative next-hop routers and continuously adjusts the splitting ratios. The simulation frame- work involves significantly more details than the Wardrop model does. For example, our simulations feature a full TCP implementation at packet level and include realistic HTTP workload generated on the basis of realistic sta- tistical properties of Web users. The simulation results are in good corre- spondence with theory. We see that our protocol in fact converges quickly and significantly improves the throughput of an autonomous system. vi vii Acknowledgements First of all, I thank Berthold Vocking¨ for being my advisor, not only with respect to this thesis, for his clear-sightedness, and for always knowing what is important and what is not. I thank Petri Mah¨ onen¨ for his interest in this work and for acting as a co-referee. I thank Harald Racke,¨ Nils Kammenhuber, and Anja Feldmann for the pro- ductive collaboration and many inspiring discussions. I am particularly grateful to Nils for introducing me to SSFNet, though this gratefulness is kind of ambivalent. I also thank the whole team of Computer Science 1 at Aachen for the nice working atmosphere, many interesting discussions, and also for assigning me to exercise six of the BuK 2007 test. I am in debt to Lars Olbrich for reading through early versions of some of the proofs. Finally, I thank Gisela for her substantial support, in particular during the last four weeks of writing up this thesis. This research was supported by the DFG Schwerpunktprogramm 1126 “Al- gorithmik großer und komplexer Netzwerke” and partially by the EU within the 6th Framework Programme under contract 001907 (DELIS). viii Table of Contents 1 Introduction 1 1.1 Foundations of Game Theory . 2 1.2 Evolutionary Game Theory: A Guided Tour . 5 1.2.1 The First Approach: Evolutionary Game Dynamics . 6 1.2.2 The Second Approach: Refined Solution Concepts . 12 1.2.3 Synthesis: Convergence Theorems . 17 1.2.4 Related Work . 20 1.3 Selfish Routing . 20 1.3.1 The Wardrop Model . 20 1.3.2 Wardrop Equilibria . 22 1.3.3 The Beckmann-McGuire-Winsten Potential . 23 1.3.4 Related Work . 24 1.4 Outline of the Thesis . 25 1.4.1 Bibliographical Notes . 27 2 Evolution of Selfish Routing 29 2.1 The Replicator Dynamics in the Wardrop Model . 30 2.1.1 Preliminaries . 32 2.2 Evolutionary Stability and Convergence . 33 2.2.1 Evolutionary Stability . 33 2.2.2 Convergence for Single-Commodity Instances . 34 2.2.3 Convergence of Multi-Commodity Instances . 36 2.3 Approximate Equilibria and Convergence Time . 37 2.3.1 Upper Bound for Single-Commodity Instances . 38 2.3.2 Upper Bound for Multi-Commodity Instances . 40 2.3.3 Lower Bound for Single-Commodity Instances . 41 2.4 Summary . 46 3 Coping with Stale Information 49 3.1 Related Work . 50 3.2 Smooth Rerouting Policies and Stale Information . 51 3.2.1 Smooth Rerouting Policies . 51 ix TABLE OF CONTENTS 3.2.2 Stale Information . 54 3.3 Stale Information Makes Convergence Hard . 55 3.3.1 Convergence Under Up-To-Date Information . 55 3.3.2 Oscillation of Best-Response . 56 3.4 Convergence Under Stale Information . 57 3.5 Convergence Time . 63 3.5.1 Uniform Sampling . 63 3.5.2 Proportional Sampling . 65 3.6 Summary . 66 4 Fast Convergence by Exploration and Replication 69 4.1 Introduction . 69 4.1.1 Our Results . 70 4.1.2 Related Work . 72 4.1.3 Preliminaries . 73 4.2 The Exploration-Replication Policy . 76 4.3 Convergence . 78 4.4 Convergence Time for Single-Commodity Instances . 82 4.4.1 Bicriteria Approximation . 82 4.4.2 Approximation of the Potential . 87 4.5 Convergence Time for Multi-Commodity Instances . 98 4.6 Lower Bounds . 103 4.6.1 Elasticity is Necessary . 103 4.6.2 Sampling with Static Probabilities is Slow . 105 4.7 Summary . 106 5 REPLEX – Dynamic Traffic Engineering 109 5.1 Introduction . 109 5.1.1 Related Work . 111 5.1.2 Notation . 112 5.2 The REPLEX Protocol . 112 5.2.1 Delegation of Routing Decisions . 113 5.2.2 Collaborative Performance Evaluation . 114 5.2.3 Measuring Performance . 115 5.2.4 Randomisation vs. Hashing . 116 5.2.5 Adapting Weights . 117 5.2.6 Summary of the Protocol and its Parameters . 117 5.3 Simulation Setup . 120 5.3.1 Realistic Web Workload and Traffic Demands . 120 5.3.2 Initialisation . 121 5.4 Exploring the Parameter Space using Artificial Topologies . 121 5.4.1 Convergence Speed vs. Oscillation . 121 x TABLE OF CONTENTS 5.4.2 Quality of the Solution . 125 5.4.3 Necessary Number of End Hosts . 125 5.4.4 Communication Helps . 127 5.4.5 Drifting Demands . 128 5.5 Simulation in a Real-World Topology .