Lecture Notes in Computer Science 2584 Edited by G. Goos, J. Hartmanis, and J. van Leeuwen 3 Berlin Heidelberg New York Barcelona Hong Kong London Milan Paris Tokyo André Schiper Alex A. Shvartsman Hakim Weatherspoon Ben Y. Zhao (Eds.)

Future Directions in

Research and Position Papers

13 Series Editors

Gerhard Goos, Karlsruhe University, Germany Juris Hartmanis, Cornell University, NY, USA Jan van Leeuwen, Utrecht University, The Netherlands

Volume Editors

André Schiper École Polytechnique Fédérale de Lausanne, Faculté Informatique et Communication IN-Ecublens, 1015 Lausanne, Switzerland E-mail: andre.schiper@epfl.ch Alex A. Shvartsman University of Connecticut, Computer Science and Engineering Unit 3155, Storrs, CT 06269, USA E-mail: [email protected] and [email protected] Hakim Weatherspoon Ben Y. Zhao University of California at Berkeley, Computer Science Division 447/443 Soda Hall, Berkeley, CA 94704-1776, USA E-mail: {hweather, ravenben}@cs.berkeley.edu

Cataloging-in-Publication Data applied for A catalog record for this book is available from the Library of Congress

Bibliographic information published by Die Deutsche Bibliothek Die Deutsche Bibliothek lists this publication in the Deutsche Nationalbibliographie; detailed bibliographic data is available in the Internet at .

CR Subject Classification (1998): C.2.4, D.1.3, D.2.12, D.4.3-4, F.1.2

ISSN 0302-9743 ISBN 3-540-00912-4 Springer-Verlag Berlin Heidelberg New York

This work is subject to copyright. All rights are reserved, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, re-use of illustrations, recitation, broadcasting, reproduction on microfilms or in any other way, and storage in data banks. Duplication of this publication or parts thereof is permitted only under the provisions of the German Copyright Law of September 9, 1965, in its current version, and permission for use must always be obtained from Springer-Verlag. Violations are liable for prosecution under the German Copyright Law. Springer-Verlag Berlin Heidelberg New York a member of BertelsmannSpringer Science+Business Media GmbH http://www.springer.de © Springer-Verlag Berlin Heidelberg 2003 Printed in Germany Typesetting: Camera-ready by author, data conversion by Olgun Computergraphik Printed on acid-free paper SPIN: 10872417 06/3142 543210 Preface

Every year we witness acceleration in the availability, deployment, and use of distributed applications. However building increasingly sophisticated applications for extant and emerging networked systems continues to be challenging for several reasons: – Abstract models of computation used in distributed systems research often do not fully capture the limitations and the unpredictable nature of realistic distributed computing platforms; – Fault-tolerance and efficiency of computation are difficult to combine when the com- puting medium is subject to changes, asynchrony, and failures; – Middleware used for constructing distributed software does not provide services most suitable for sophisticated distributed applications; – Middleware services are specified informally and without precise guarantees of effi- ciency, fault-tolerance, scalability, and compositionality; – Specification of distributed deployment of software systems is often left out of the development process; – Finally, there persists an organizational and cultural gap between engineering groups developing systems in a commercial enterprise, and research groups advancing the scientific state-of-the-art in academic and industrial settings. The objectives of this book are: (1) to serve as a motivation for defining future research programs in distributed computing, (2) to help identify areas where practitioners and engineers on the one hand and scientists and researchers on the other can improve the state of distributed computing through synergistic efforts, and (3) to motivate graduate students interested in entering the exciting research field of distributed computing. The title of this volume, Future Directions in Distributed Computing, captures the unifying theme for this collection of 38 position and research papers dealing with a vari- ety of topics in distributed computing, ranging from theoretical foundations to emerging distributed applications, and from advanced distributed data services to innovative net- work communication paradigms. Thirty-one of the 38 papers in this volume are based on the preliminary reports that appeared in the proceedings of the 2002 International Workshop on Future Directions in Distributed Computing, held on 3–7 June 2002 at the University of Bologna Residential Center located in the medieval hilltop town of Berti- noro, Italy. The workshop was organized by Ozalp¨ Babao˘glu,Ken Birman, and Keith Marzullo, and this book was envisioned by them. Without their efforts in organizing the Bertinoro workshop this book would not exist. The plan to compile this volume was developed during the workshop. The numerous technical discussions at the workshop inspired many of the workshop participants to VI Preface substantially revise their position papers, and several decided to produce completely new papers. This volume is the culmination of this development and, in addition to the refined workshop papers, it includes seven completely new contributions. All papers were anonymously cross-refereed; the editorial team accepted papers following their revision based on the referee comments. The papers are grouped into four parts, preceded by two papers with insights into technological and social processes that are a part of the development of distributed com- puting technology and its applications. The first paper presents a historical perspective on the development of group communication technology. Group communication services is an example of reusable and sophisticated building blocks that have been successfully used in developing dependable distributed applications and that improve our ability to construct such applications. The second paper discusses the social interactions along the boundary between academic distributed systems research and the real-world practice of distributed computing, and tackles the question of the academic research impact on the real world. The other papers are grouped in the following four parts. Part I includes 12 papers dealing with the theoretical foundations of distributed computing, selected current re- search projects, and future directions. Part II consists of eight papers presenting research on novel communication and network services for distributed systems. Part III includes eight papers dealing with data and file services, and coherence and replication in net- work computing. Part IV includes eight papers dealing with system solutions, pervasive computing, and applications of distributed computing technologies. We are certain that the papers in this volume contain numerous research topics that ought to be addressed in future research or that are already being addressed by forward- looking research in distributed computing. We are also certain that it is impossible to include all worthwhile future research topics in a single volume, and that some topics identified in this volume will turn out to be only moderately interesting. At the same time, we consider it necessary to identify the research topics that a broad cross-section of the research community views as important for the future of distributed computing.

January 2003 Andr´eSchiper Alex A. Shvartsman Hakim Weatherspoon Ben Y. Zhao Table of Contents

1. Practical Impact of Group Communication Theory Andr´eSchiper ...... 1

2. On the Impact of Academic Distributed Systems Research on Industrial Practice Michael D. Schroeder ...... 11

Part I. Foundations of Distributed Systems: WhatDoWeStillExpectfromTheory?...... 15

3. Using Error-Correcting Codes to Solve Distributed Agreement Problems: A Future Direction in Distributed Computing? Roy Friedman, Achour Most´efaoui, Sergio Rajsbaum, and Michel Raynal ... 17

4. Lower Bounds for Asynchronous Consensus ...... 22

5. Designing for Dependent Process Failures Flavio Junqueira and Keith Marzullo ...... 24

6. Comparing the Atomic Commitment and Consensus Problems Bernadette Charron-Bost ...... 29

7. Open Questions on Consensus Performance in Well-Behaved Runs Idit Keidar and Sergio Rajsbaum ...... 35

8. Challenges in Evaluating Distributed Algorithms Idit Keidar ...... 40

9. Towards Robust Optimistic Approaches Ricardo Jim´enez-Perisand Marta Pati˜no-Mart´ınez ...... 45

10. Towards a Practical Approach to Confidential Tolerance Jian Yin, Jean-Philippe Martin, Arun Venkataramani, Lorenzo Alvisi, and Mike Dahlin ...... 51 VIII Table of Contents

11. Modeling Complexity in Secure Distributed Computing Christian Cachin ...... 57

12. Communication and Data Sharing for Dynamic Distributed Systems Nancy Lynch and Alex Shvartsman ...... 62

13. Dissecting Distributed Computations ...... 68

14. Ordering vs Timeliness: Two Facets of Consistency? Mustaque Ahamad and Michel Raynal ...... 73

Part II. Exploring Next-Generation Communication Infrastructures andApplications...... 79

15. WAIF: Web of Asynchronous Information Filters Dag Johansen, Robbert van Renesse, and Fred B. Schneider ...... 81

16. The Importance of Aggregation Robbert van Renesse ...... 87

17. Dynamic Lookup Networks Dahlia Malkhi ...... 93

18. The Surprising Power of Epidemic Communication Kenneth P. Birman ...... 97

19. Topology-Aware Routing in Structured Peer-to-Peer Overlay Networks Miguel Castro, Peter Druschel, Y. Charlie Hu, and Antony Rowstron ...... 103

20. Uncertainty and Predictability: Can They Be Reconciled? Paulo Ver´ıssimo ...... 108

21. Fuzzy Group Membership Roy Friedman ...... 114

22. Toward Self-Organizing, Self-Repairing and Resilient Distributed Systems Alberto Montresor, Hein Meling, and Ozalp¨ Babao˘glu ...... 119

PartIII.ChallengesinDistributedInformationandDataManagement.....125

23. Dynamically Provisioning Distributed Systems to Meet Target Levels of Performance, Availability, and Data Quality Amin Vahdat ...... 127 Table of Contents IX

24. Database Replication Based on Group Communication: Implementation Issues Bettina Kemme ...... 132

25. The Evolution of Publish/Subscribe Communication Systems Roberto Baldoni, Mariangela Contenti, and Antonino Virgillito ...... 137

26. Naming and Integrity: Self-Verifying Data in Peer-to-Peer Systems Hakim Weatherspoon, Chris Wells, and John D. Kubiatowicz ...... 142

27. Spread Spectrum Storage with Mnemosyne Steven Hand and Timothy Roscoe ...... 148

28. Replication Strategies for Highly Available Peer-to-Peer Storage Ranjita Bhagwan, David Moore, Stefan Savage, and Geoffrey M. Voelker ... 153

29. A Data-Centric Approach for Scalable State Machine Replication Gregory Chockler, Dahlia Malkhi, and Danny Dolev ...... 159

30. Scaling Optimistic Replication Marc Shapiro and Yasushi Saito ...... 164

Part IV. System Solutions: Challenges and Opportunities in Applications of Distributed Computing Technologies ...... 169

31. Building a Bridge between Distributed Systems Theory and Commercial Practice Brian Whetten ...... 173

32. Holistic Operations in Large-Scale Sensor Network Systems: A Probabilistic Peer-to-Peer Approach Indranil Gupta and Kenneth P. Birman ...... 180

33. Challenges in Making Pervasive Systems Dependable Christof Fetzer and Karin H¨ogstedt ...... 186

34. Towards Dependable Networks of Mobile Arbitrary Devices – Diagnosis and Scalability Miroslaw Malek ...... 191

35. Technology Challenges for the Global Real-Time Enterprise Werner Vogels ...... 197

36. Middleware for Supporting Inter-organizational Interactions Santosh K. Shrivastava ...... 202 X Table of Contents

37. Hosting of Libre Software Projects: A Distributed Peer-to-Peer Approach Jes´usM. Gonz´alez-Barahonaand Pedro de-las-Heras-Quir´os ...... 207

38. System Support for Pervasive Applications Robert Grimm and Brian Bershad ...... 212

Author Index ...... 219