Boston University Graduate School of Arts And
Total Page:16
File Type:pdf, Size:1020Kb
BOSTON UNIVERSITY GRADUATE SCHOOL OF ARTS AND SCIENCES Dissertation OVERLAY NETWORK CREATION AND MAINTENANCE WITH SELFISH USERS by GEORGIOS SMARAGDAKIS Diploma, Technical University of Crete, 2002 Submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy 2009 c Copyright by GEORGIOS SMARAGDAKIS 2008 Approved by First Reader Azer Bestavros, Ph.D. Professor of Computer Science Boston University Second Reader Nikolaos Laoutaris, Ph.D. Researcher Telef´onica Research, Barcelona Third Reader John W. Byers, Ph.D. Associate Professor of Computer Science Boston University ËØhÒ ÓikÓgèÒei ÑÓÙ¸ ÂeÑi× ØÓkÐ ¸ ³AÒÒa kai EÑÑaÒÓ٠к iv Acknowledgments In the last five years I have been privileged to work, study, and reside in a great academic environment that motivated me to challenge my limits. I would like to express my ac- knowledgments to colleagues and friends who encouraged and inspired my work during the graduate school years. I am grateful to my advisor, Azer Bestavros, for his support and encouragement throughout my graduate studies. Under his supervision, I was provided full academic freedom to build my research agenda and was encouraged to think outside the box. De- spite his extremely heavy schedule as the department chair, Azer was always available to discuss my research and academic development. I would like to express my gratitude to my co-advisor, Nikolaos Laoutaris. Nikos taught me how to attack challenging networking problems by peeling off non-essential details to reveal the underlying fundamental computer science problem. He also introduced me to game theory, approximation algorithms, and operational research, which are the algorithmic foundations of my research in this dissertation. My second co-advisor, John Byers, is one of the most effective teachers I have had. Taking coursework with him changed my outlook on computer networks and introduced me to new areas of research. The faculty, students, and staff of the computer science department at Boston Uni- versity provided a stimulating and personal environment. My special thanks go to the members of the WING group for their valuable feedback on my research. I would like to express my gratitude to my co-authors who contributed to my research efforts. I am also thankful to my office- and room-mate Vijay Erramilli, to Panagiotis Papapetrou, Niky Riga, Vassilis Athitsos, Michalis Potamias, and Irena Tsvetkova for being always there for v me. My research was financially supported by National Science Foundation, Boston Uni- versity, Telef´onica Research-Barcelona, and the European Commission. I dedicate this dissertation to my father, my mother, and my brother. Words are not strong enough to express my feelings for them. vi OVERLAY NETWORK CREATION AND MAINTENANCE WITH SELFISH USERS (Order No. ) GEORGIOS SMARAGDAKIS Boston University, Graduate School of Arts and Sciences, 2009 Major Professor: Azer Bestavros, Professor of Computer Science Department ABSTRACT Overlay networks have been used for adding and enhancing functionality to the end-users without requiring modifications in the Internet core mechanisms. Overlay networks have been used for a variety of popular applications including routing, file sharing, content distri- bution, and server deployment. Previous work has focused on devising practical neighbor selection heuristics under the assumption that users conform to a specific wiring proto- col. This is not a valid assumption in highly decentralized systems like overlay networks. Overlay users may act selfishly and deviate from the default wiring protocols by utilizing knowledge they have about the network when selecting neighbors to improve the perfor- mance they receive from the overlay. This thesis goes against the conventional thinking that overlay users conform to a spe- cific protocol. The contributions of this thesis are threefold. It provides a systematic evaluation of the design space of selfish neighbor selection strategies in real overlays, eval- uates the performance of overlay networks that consist of users that select their neighbors selfishly, and examines the implications of selfish neighbor and server selection to overlay protocol design and service provisioning respectively. This thesis develops a game-theoretic framework that provides a unified approach to modeling Selfish Neighbor Selection (SNS) wiring procedures on behalf of selfish users. The model is general, and takes into consideration costs reflecting network latency and user preference profiles, the inherent directionality in overlay maintenance protocols, and vii connectivity constraints imposed on the system designer. Within this framework the notion of user’s “best response” wiring strategy is formalized as a k-median problem on asymmetric distance and is used to obtain overlay structures in which no node can re-wire to improve the performance it receives from the overlay. Evaluation results presented in this thesis indicate that selfish users can reap substantial performance benefits when connecting to overlay networks composed of non-selfish users. In addition, in overlays that are dominated by selfish users, the resulting stable wirings are optimized to such great extent that even non- selfish newcomers can extract near-optimal performance through na¨ıve wiring strategies. To capitalize on the performance advantages of optimal neighbor selection strategies and the emergent global wirings that result, this thesis presents EGOIST: an SNS-inspired overlay network creation and maintenance routing system. Through an extensive measure- ment study on the deployed prototype, results presented in this thesis show that EGOIST’s neighbor selection primitives outperform existing heuristics on a variety of performance metrics, including delay, available bandwidth, and node utilization. Moreover, these re- sults demonstrate that EGOIST is competitive with an optimal but unscalable full-mesh approach, remains highly effective under significant churn, is robust to cheating, and incurs minimal overheads. This thesis also studies selfish neighbor selection strategies for swarming applications. The main focus is on n-way broadcast applications where each of n overlay user wants to push its own distinct file to all other destinations as well as download their respective data files. Results presented in this thesis demonstrate that the performance of our swarming protocol for n-way broadcast on top of overlays of selfish users is far superior than the performance on top of existing overlays. In the context of service provisioning, this thesis examines the use of distributed ap- proaches that enable a provider to determine the number and location of servers for optimal delivery of content or services to its selfish end-users. To leverage recent advances in virtu- alization technologies, this thesis develops and evaluates a distributed protocol to migrate servers based on end-users demand and only on local topological knowledge. Results under viii a range of network topologies and workloads suggest that the performance of the distributed deployment is comparable to that of the optimal but unscalable centralized deployment. ix Contents Abstract vii List of Tables xv List of Figures xvi List of Abbreviations xix 1 Introduction 1 1.1 Conceptual Contributions . ... 2 1.2 TechnicalContributions . ... 3 1.3 ThesisOrganization .............................. 6 2 Selfish Neighbor Selection 7 2.1 Background.................................... 10 2.2 Definitions..................................... 11 2.3 DerivingStableWirings . 12 2.3.1 TheBest-Responseofanode . 13 2.3.2 Connections between SNS Game and Facility Location . ...... 13 2.3.3 Equilibrium Wirings through Iterative Best Response ........ 15 2.3.4 A Lower Bound on the Cost of a Socially Optimal Wiring . 16 2.4 CharacterizationofStableWirings . ...... 17 2.4.1 SocialCostofStableWirings . 17 2.4.2 TopologyofStableWirings . 19 2.4.3 Contsraining the In-degree: A Doubly Constrained Overlay ..... 22 2.5 Overlay Neighbor Selection: Best Response vs. k-Random, k-Regular, and k-Closest ..................................... 23 x 2.5.1 Description and Design Methodology . 24 2.5.2 DescriptionoftheDatasets . 24 2.5.3 ComparisonofDifferentGraphs . 26 2.5.4 TheValueofBestResponse. 26 2.6 MinimizingtheMaximumDelay . 32 2.7 Overlay Neighbor Selection with variable out-degree . .......... 34 2.8 Overlay Neighbor Selection under scoped-flooding . ......... 35 2.8.1 A reformulation of Best Response for scoped-flooding . ...... 37 2.8.2 The value of Best Response for scoped-flooding . 38 2.8.3 Stable wirings under scoped-flooding . 39 2.9 ChapterSummary ................................ 42 3 The EGOIST Overlay Routing System 43 3.1 Background.................................... 44 3.2 Preliminaries ................................... 46 3.3 Architecture.................................... 47 3.3.1 BasicDesign ............................... 47 3.3.2 NeighborSelectionPolicies . 49 3.3.3 Dealingwithchurn............................ 49 3.3.4 DealingwithCheaters . .. .. .. .. .. .. 51 3.4 ExperimentalEvaluation. 52 3.4.1 CostMetrics ............................... 52 3.4.2 BaselineExperimentalResults . 54 3.4.3 Measurement and Re-wiring Overheads . 59 3.4.4 EffectofChurn.............................. 61 3.4.5 VulnerabilitytoAbuse. 64 3.5 ScalabilityIssues ............................... 66 3.5.1 ScalabilityviaSampling . 66 3.5.2 LayeredArchitecture. 69 xi 3.6 Applications.................................... 70 3.6.1 MultiplayerP2PGames . 71 3.6.2 MultipathFileTransfer . 72 3.6.3 Real-timeTrafficoverIP