DISS. ETH NO. 17731 TIK-Schriftenreihe Nr. 96 Dynamics and Cooperation Algorithmic Challenges in Peer-to-Peer Computing A dissertation submitted to the SWISS FEDERAL INSTITUTE OF TECHNOLOGY ZURICH for the degree of Doctor of Sciences presented by STEFAN SCHMID MSc in Computer Science, ETH Z¨urich born 31.07.1978 citizen of Hedingen ZH accepted on the recommendation of Prof. Dr. Roger Wattenhofer, ETH Zurich, examiner Prof. Dr. Boaz Patt-Shamir, Tel Aviv University, co-examiner Prof. Dr. Tim Roughgarden, Stanford University, co-examiner 2008 Abstract Peer-to-peer (p2p) computing is one of the most intriguing new network- ing paradigms of the last ten years, and many applications today use peer-to- peer technology, e.g. for large distributed computations, for file sharing, or for live media streaming. At the heart of the paradigm lies the idea of lever- aging the resources of the system's participants. Thus, potentially scalable and robust architectures can be built. However, making use of the decentralized resources is challenging. The peers are under the control of the individual users who may only connect to the network for a short period of time. Consequently, there are frequent mem- bership changes and p2p systems are highly dynamic. In addition to regular joins and leaves, the participating machines (often unreliable desktops) may crash. Peer-to-peer solutions are also faced with the fact that it is not always in the (anonymous) users' interest to contribute their resources. Rather, a user may be selfish and seek to exploit the system without reciprocating. This dissertation studies the challenges of the dynamics in p2p computing and of cooperation. We describe a system which is based on a hypercubic topology and which applies algorithms that maintain desirable network prop- erties despite worst-case joins and leaves; these algorithms can also be used for pancake graphs. Besides membership dynamics, we investigate dynamic changes of the available bandwidth between two peers, and we analyze the throughput of different transfer protocols. In order to emphasize the impor- tance of the cooperation challenge, we conduct a case study of BitTorrent| one of the most traffic intensive applications on the Internet|, and show that today's peer-to-peer networks still fail to fend off uncooperative peers. A game-theoretic analysis of a p2p network creation game is presented which estimates the impact of selfish behavior. We find that both the performance and the stability of a system can suffer severely. In addition, this dissertation introduces a mathematical framework which allows us to evaluate a system's robustness to malicious attacks; the framework is also useful for the analysis of social networks. The theoretic findings are complemented by a case study which identifies vulnerabilities in the popular Kad network. Zusammenfassung Peer-to-peer Computing ist eines der faszinierensten Paradigmen im Net- working Bereich des letzten Jahrzehnts. Bereits gibt es zahlreiche Anwen- dungen, beispielsweise die Berechnung schwieriger mathematischer Probleme, der Austausch grosser Dateien oder live Ubertragungen¨ von Sportveranstal- tungen. Dem Paradigma liegt die Idee zugrunde, dass durch Ausnutzung der Resourcen der teilnehmenden Rechner potentiell skalierbare und robuste Architekturen m¨oglich werden. Die Abh¨angigkeit von den dezentralen Resourcen der Teilnehmer bringt aber auch gewisse Schwierigkeiten mit sich. Benutzer verbinden sich oft nur f¨urkurze Zeit zum Netzwerk, weshalb das System oft sehr dynamisch ist. Meist sind die beteiligten Maschinen auch unzuverl¨assigund k¨onnenjederzeit ausfallen. Des weiteren ist es h¨aufignicht im Interesse eines (anonymen) Benutzers, selber Resourcen zum System beizutragen. Diese Dissertation befasst sich mit den Herausforderungen, die sich aus der Dynamik und aus der Notwendigkeit zur Kooperation ergeben. Insbeson- dere werden wir ein auf einem Hyperw¨urfelbasierendes System vorstellen, das trotz ununterbrochenen und schlimmstm¨oglichen Topologie¨anderungen n¨utzliche Eigenschaften sicherstellt. Unsere Algorithmen k¨onnenauch auf andere Topologien angewandt werden, beispielsweise auf Pancake Graphen. Neben der sogenannten Knotendynamik betrachten wir auch dynamische Anderungen¨ der verf¨ugbarenBandbreite zwischen zwei Peers und analysieren die Effizienz verschiedener Ubertragungsprotokolle.¨ Unsere Fallstudie in Bit- Torrent { eine der beliebtesten Anwendungen im Internet { zeigt auf, dass heutige peer-to-peer Systeme noch keine wirksamen Mechanismen gegen un- kooperatives Verhalten anwenden. Mittels einer spieltheoretische Analyse der Topologien unstrukturierter peer-to-peer Netzwerke kommen wir zum Schluss, dass ein strategisches Verhalten aber einen wesentlichen Einfluss auf die Effizienz eines Systems haben kann. Diese Arbeit f¨uhrtausserdem einen mathematischen Formalismus ein, der es erm¨oglicht, den potentiellen Schaden durch b¨oswilligeTeilnehmer abzusch¨atzen; dieser Formalismus ist auch bei der Analyse von sozialen Netzwerken einsetzbar. Die theoretischen Erkenntnisse werden erg¨anztdurch ein Fallbeispiel, welches Schwachstellen in Kad, einem popul¨arenpeer-to-peer Netzwerk, identifiziert. Acknowledgements Many people have contributed to this dissertation. First of all, I am grateful to my advisor Roger Wattenhofer for his support, and for giving me the opportunity to pursue a number of different and exciting projects during my PhD time. I benefitted from his excellent knowledge|and critical view|of past and current developments in the field, and I feel lucky that the European Space Agency chose another candidate three years ago! I am honored that two well-known researchers, Boaz Patt-Shamir and Tim Roughgarden, agreed to read through my dissertation and to serve on my committee board. I would like to acknowledge their work and thank for their comments which helped to improve the thesis. Parts of this thesis have been presented on various occasions. I am obliged to the following people for their invitations and their feedback: Aaron Zollinger (UC Berkeley), David Parkes (Harvard University), Bobby Klein- berg and Emin G¨unSirer (Cornell University), Stephan Eidenbenz (Los Alamos National Laboratories), and Christian Scheideler (TU M¨unchen). In particular, I would like to thank Sven Kosub for pointing me to [233]. Many thanks go to my colleagues in the Distributed Computing Group. First of all, I would like to thank Roland Flury for having been a super office mate|and Zurich marathon competitor! I am grateful for the generous help he offered on so many occasions, and I will never forget the SOLA Duo race. Special thanks go to Fabian Kuhn, who was my diploma thesis advisor; his marvelous way of tackling problems and his enthusiasm was certainly a mo- tivation for me to continue my studies as a PhD student. I thank Thomas Moscibroda for the fruitful collaboration which broadened my scientific hori- zon, for having been a pleasant Discrete Event Systems co-assistant, and for his sense of humor in various situations. Thanks also go to my|without loss of generosity!|most productive peer-to-peer collaborator Thomas \T.Lo." Locher for having been an arch-accomplice in stealing bits, for the Happy Times with time-shifted subtitles, and for making the publications he co- authored a pleasure to read. I thank Yvonne Anne Oswald for her interest in Al Capone, the families, Gaussian ski rental on trees|and Suannai, for hav- ing been a great traveling companion in Australia, China, Japan|and MMP, and for challenging the all-time science quiz record on the plane to Bangkok. I also thank Remo Meier for making Pulsar better than the SIGCOMM re- viewers were prepared to and for inviting me to the \Kanonenrohr Piste", Michael Kuhn for challenging discrete logarithm cryptography not only at the university of Berne and for the diligent use of the \rote Telefon", Pascal von Rickenbach for sorting out the Highway to Hell problem with me and for shedding light on the mysteries of the Piazza del Popolo, Aaron Zollinger for introducing me to the YMCA in Berkeley|and to a popular web search company, Keno Albrecht for the computer support and for the time2climb in the Gaswerk, Nicholas Burri for having been the only T¨oggelipartner with whom I could improve my ranking, Olga Goussevskaia for telling me when to \eat!" the opponent's queen at the TIK institute's (chess) summer fight, and Johannes Schneider for organizing, among many others things, the Indian Ooty trip and the Swiss 3 a.m./4190 m.a.s.l. Strahlhorn expedition. I would also like to thank the following former and new DCG members, office mates, people from the institute, collaborators, visitors, etc.: Andi Wetzel, Beat Futterknecht, Bernard Mans, Carmine Ventre, Caterina Sposato, Christoph Lenzen, Jin Li, Lino Sevcik, Marco Cicolini, Mario Strasser, Martin May, Miriam Fritz, Mirjam Wattenhofer, Monica Fricker, Philipp Sommer, Rainer Baumann, Regina O'Dell, Reto Sp¨ohel,Roland Mathis, Simon Heimlicher, Simon Schlachter, Thierry Dussuet, Thomas Steingruber, Uri Nadav, and Yves Weber. A number of student projects have been conducted in the context of this dissertation. It is great that (so far) four students, Raphael Eiden- benz, Thomas Locher, Remo Meier, and Yvonne Anne Oswald, have decided to start a PhD in our group as well. Moreover, it was interesting to wit- ness how Dominik Grolimund and Luzius Meisser's Kangoo/Wuala system evolved from a student project into a business plan. I wish the two of them an exciting future! Special thanks go to Joest Smit for his appetite for