Challenged Networking

Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 450

An Experimental Study of new Protocols and Architectures

ERIK NORDSTRÖM

ACTA UNIVERSITATIS UPSALIENSIS ISSN 1651-6214 UPPSALA ISBN 978-91-554-7239-9 2008 urn:nbn:se:uu:diva-9002 !"#$" % & '$ (() '$*'# + + + , - . ,

/ 0 , ((), 1 /. , + . , , 2#(, 33 , , 4 5/ 6")!6'!##2!"$6!6,

7 . + 4 . , - 4 . . + . . , 4 . . . . . , 8 , 7 . + + , 7 + . , 5 . + . . . + . . , 8 + 9 : , 4 + . . , - + . . , . !! + + 4 -% ;/ - , . + + '( . 9 !:, 7 + < ,, . + , 7 , 8 . = , 4 , ; . . , 7 , 7 . . .,

1 . /. 7 /. - /. 8 /. - ,

! " # " $ % &&'" " (')*+) "

> / 0 (()

4 / '3#'!3'2 4 5/ 6")!6'!##2!"$6!6 * *** !6(( 9 *?? ,,? @ A * *** !6((: If I have seen further it is by standing on the shoulders of giants.

Sir Isaac Newton

Preface

Five years make a long time. It is certainly ample time to ponder on the reasons for pursuing a Ph.D., and for engaging in research. Many such reasons that are now obvious to me were not clear at the beginning of my journey – they have emerged gradually during the course of it. One of the reasons for engaging in research, which I believe in, is to contribute to a common pool of knowledge. There is a famous saying that we are standing on the shoulders of giants1. Furthering our knowledge is a collabora- tive process that consists of incremental steps. This process is how we in the long run affect the evolution of our society and, by extension, our world. Some thinkers of course produce more disruptive research than others. However, whether research is disruptive or not, often only time can tell. In either case, if research is not accessible, interesting, understandable, and educational for people, it might as well never have existed to begin with. Informing others of your research, especially those that are not professionals, I have learnt can be highly motivational and constructive for your work – but it is also a major challenge. Fortunately, this thesis is all about challenges. Thus, a goal has been to make this thesis accessible (at least in parts) also to people not necessarily familiar with computer networking. Whether I have succeeded or not is up to the reader to decide. This goal, however, does not imply that this thesis is free from technical jargon and intricate details. So beware! Often friends and family have asked me about my research. Equally often I have tried to avoid giving an answer. One reason for this unwillingness to explain is simply that I prefer not to talk about work when I am not working. Another reason is that I quickly learned that it was hard to explain in limited time (e.g., imagine doing it at a party before a person looses interest), or that it was obvious that the person was just being polite by asking – and in fact could not care less. The people that actually do want to know: here is your chance! Keep on reading. During ﬁve years many things can change – not only in terms of research, but also in life in general. This life I quickly learnt can suffer if you are sucked too deeply into what you are doing. I therefore decided early on that I would honor the life outside of work, and at the same time to embrace some of the perks of being a doctoral candidate in my area. These perks include free plan-

1This metaphor is attributed Bernard of Chartres, but was made famous by Sir Isaac Newton.

v ning of time (which can both be a blessing and a curse), working in an environment where you meet and interact with people of many different backgrounds, and last but not least the opportunity to travel to many interesting countries and places. These perks have not only been good for me as a researcher; they have also made me a better and more enlightened person. To have had the fortune to do all these things, such as enjoying perks and engaging in interesting research, has been a privilege. This privilege would not have been possible without the support of many people. My gratitude goes to all of these people, which I will here try to list without forgetting someone. My gratitude first and foremost goes to my closest family: my parents Jan- Eric and Eva, and my sister Jenny. They have not always understood what I have been doing in my research, but have always supported me in my decision to pursue it. When life has been difficult, in work or otherwise, they have always been there to pick me up. My closest family, and my other relatives, have my everlasting love and gratitude. My friends outside of work have been my retreat and safe haven from everything work related. They are too numerous to list here, and if I tried I would be afraid to forget somebody. You know who you are! I am grateful for having such diverse friends that do everything but researching challenged networking. They have given me broader perspectives, which undoubtedly have made me better in many ways, including in my research. My colleagues in the Communications Research (CoRe) group have been instrumental to my success as a Ph.D. candidate. These are: Richard Gold, my partner-in-crime and friend. Endless have our discussions been – sometimes heated, often challenging, always insightful. I am grateful for these insights, for the patience shown when listening to my complaints at times of low motivation, and for reminding me not to take work too seriously when I shouldn’t. Olof Rensfelt, whose research seems to be only one hobby of many that he excels in. Apart from his research, I especially appreciate his beer making skills and excellent cooking, which can always be enjoyed through a level of hospitality challenged only by his wife Agnes. Henrik Lundgren, who during my first year was my close collaborator and co-author on many of my papers. He is a friend that was a familiar face in Paris during my visit there. Chris- tian Rohner who has been a loyal work companion, and has always been able to put up with my sometimes strong opinions on how to do things. Christian Tschudin, who after he left the group has acted as my external supervisor. His refreshing ideas and unchallenged technical skills have been an inspiration. David Lundberg and Mattis Fjällström, who I have had the privilege working with early on in my studies. The rest of the colleagues in the CoRe group have always inspired and supported; Lars-Åke Larzon, Arnold Pears, Oskar Wibling, Ioana Rodhe, Fredrik Bjurefors, Thabotharan Kathiravelu, and Laura Feeney. Numerous are also the students, research assistants, alumni people, and visitors that I have been fortunate enough to have met and interacted with. All of these people have my gratitude. vi I am also thankful to my colleagues at the Thomson research lab in Paris, where I did an internship (2006-2007). Christophe Diot, who was kind enough to host me, and whose open mind (often spoken in a refreshingly direct manner) gave me fresh inspiration during the final years of my Ph.D. studies. My gratitude also goes to all of the others in the lab that made my stay in Paris a rememberable one: Amogh, Vivek, Andy, Augustin x 2, Laurent, Vincent, Letizia, Haakon, Ming, Abderrahmen, Theodoros, Anna-Kaisa, Fer- nando, Marianna, and others. Last, but not least, my supervisor and mentor Per Gunningberg. He has always believed in me, supported me, and given me the freedom that every researcher wants – but perhaps not always takes advantage of to the fullest. I owe him much, and for this he has my deepest gratitude and greatest respect.

vii

List of Included Papers

This thesis is based on the following papers, which are referred to in the text by their Roman numerals.

I Henrik Lundgren, David Lundberg, Johan Nielsen, Erik Nord- ström and Christian Tschudin, “A Large-scale Testbed for Re- producible Ad hoc Protocol Evaluations”. In Proceedings of The Third Annual IEEE Wireless Commmunications & Networking Conference (WCNC), March 2002. II Henrik Lundgren, Erik Nordström and Christian Tschudin, “Coping with Communication Gray Zones in IEEE 802.11b based Ad hoc Networks”. In Proceedings of The Fifth ACM International Workshop on Wireless Mobile Multimedia (WoWMoM), September 2002. III Erik Nordström, Per Gunningberg and Christian Tschudin, “Ro- bust and Flexible Internet Connectivity for Mobile Ad hoc Net- works”. An extended version of the paper: “Comparison of For- warding Strategies in Internet Connected MANETs”, published in ACM SIGMOBILE Mobile Computing and Communications Review (MC2R), Volume 8, Issue 4, Pages 72-76, October 2004. IV Christian Tschudin, Per Gunningberg, Henrik Lundgren and Erik Nordström, “Lessons from Experimental MANET Research”.In Elsevier Journal on Ad hoc Networks, Volume 3, Issue 2, Pages 221-233, March 2005. V Erik Nordström, Per Gunnningberg and Henrik Lundgren, “A Testbed and Methodology for Experimental Evaluation of Mobile Ad hoc Networks”. In Proceedings of The First International Con- ference on Testbeds and Research Infrastructures for the Devel- opment of Networks and Communities (TRIDENTCOM), Febru- ary 2005. VI Erik Nordström, Per Gunningberg, Christian Rohner and Oskar Wibling, “A Cross-Environment Study of Routing Protocols for Wireless Multi-hop Networks”. Technical Report 2007-016, Up- psala University, April 2007. VII Erik Nordström, Per Gunningberg, Christophe Diot and Augustin Chaintreau, “Interest-based Content Dissemination in Opportunistic Networks”. To be submitted for publication.

ix VIII Erik Nordström, Per Gunningberg, Christian Rohner, “A Search-based Network Architecture for Content-oriented and Opportunistic Communication”. Submitted for publication.

Reprints are made with permissions from the publishers.

Comments on my Participation Paper I: This paper is based on the work I did on the APE testbed in collaboration with Mattis Fjällström and David Lundberg. I developed large parts of the testbed and I invented the virtual mobility metric presented in the paper. It was developed further in discussions with the other authors.

Paper II: I came up with the ideas behind this paper during my implementation work with the AODV protocol. They were then developed in discussions with Henrik Lundgren and Christian Tschudin. The experimental results were produced by myself and Henrik Lundgren, who then took the role as lead author of the paper.

Paper III: I am the principal author of this paper. I came up with the ideas behind it and produced all the results myself. I developed and discussed the ideas with my co-authors.

Paper IV: I co-authored this paper, and speciﬁcally wrote the sections on the testbed and implementation experiences.

Paper V: I am the main author of this paper, and I participated in all parts of the work behind it. I formulated the methodologies described in discussions with my co-authors.

Paper VI: I implemented the AODV and DSR protocols that are evaluated in this paper. I participated in all parts of the experimental work and results analysis. I am the main author of this paper.

Paper VII: I am the main author of this paper. I took the lead on the work of collecting two of the three data sets used in the paper, and performed the analysis. I developed and implemented the simulator used to produce the results.

Paper VIII: I am the principal author of this paper. I developed the architecture described in the paper in collaboration with Christian Rohner. I also did a majority of the implementation work, and helped with the measurements. x Other Work In addition to the papers above, the following papers, posters, demos and workshops, represent work that I have participated in, presented, authored or co-authored. 1. Erik Nordström, Christophe Diot, Richard Gass, Per Gunningberg, “Ex- periences from Measuring Human Mobility using Bluetooth Inquiring De- vices”, MobiEval 2007, San Juan, Puerto Rico. 2. Erik Nordström, Per Gunningberg, Christian Rohner, Oskar Wibbling “Evaluating Wireless Multi-hop Networks Using a Combination of Simulation, Emulation, and Real World Experiments”, MobiEval 2007, San Juan, Puerto Rico. 3. Erik Nordström, Per Gunningberg, Christian Rohner, Oskar Wibling “A Cross-Environment Study of Routing Protocols for Wireless Multi-hop Net- works”, Presentation at Intimate workshop, July 2007, Paris, France. 4. Christian Rohner, Erik Nordström, Per Gunningberg, Christian Tschudin, “Interactions between TCP, UDP and Routing Protocols in Wireless Multi- hop Ad hoc Networks”, IEEE ICPS Workshop on Multi-hop Ad hoc Net- works: from theory to reality. REALMAN 2005, Santorini, Greece. 5. Erik Nordström, Richard Gold, Per Gunningberg, "Mythbusters: Whatever You Thought About MANET Routing, Think Again...", Technical Report 2005-039, Department of Information Technology, Uppsala University, November 2005. 6. Christian Rohner, Erik Nordström, Henrik Lundgren, “Using Ad hoc Net- working in Orienteering”, Demo at the 10th Annual International Confer- ence on Mobile Computing and Networking (MobiCom), September 2004, Philadelphia, USA. 7. Erik Nordström, Per Gunningberg, Christian Tschudin, “POSTER: Com- parison of Forwarding Strategies in Internet Connected MANETs”, Mobi- Hoc 2004, Tokyo, Japan. 8. Christian Tschudin, Henrik Lundgren, Erik Nordström, “Embedding MANETs in the Real World”, PWC 2003, Venice, Italy. 9. Henrik Lundgren, Erik Nordström, Christian Tschudin, “The Gray Zone Problem in IEEE 802.11b based Ad hoc Networks”, ACM SIGMOBILE Mobile Computing and Communications Review 2002. 10. The ﬁrst AODV Next Generation (AODVng) workshop, in conjunction with the SIGMOBILE MobiHoc Symposium in Lausanne, Switzerland, June 2002. 11. The ﬁrst AODV Interop, held at the University of California Santa Barbara, USA, March 2002. 12. Henrik Lundgren, David Lundberg, Johan Nielsen, Erik Nordström, Chris- tian Tschudin, “A Large-scale Testbed for Reproducible Ad hoc Proto- col Evaluations”, Technical Report 2001-029, Department of Information Technology, Uppsala University, November 2001.

Contents

1 Introduction ...... 1 1.1 Communication Protocols and Requirements ...... 2 1.2 Examples of Communication Challenges ...... 2 1.3 Research Issues and Topics ...... 6 1.3.1 New Networking Paradigms ...... 6 1.3.2 Heterogeneity: Compatibility and Interconnection ...... 7 1.3.3 Conﬁguration and Deployment ...... 7 1.3.4 Testing and Evaluation ...... 8 1.4 Research Issues Addressed in this Thesis ...... 8 1.4.1 Background and Motivation ...... 9 1.4.2 Issue 1: Repeatability in Real World Experiments ...... 10 1.4.3 Issue 2: Cross-environment Evaluation Methodologies . . . 10 1.4.4 Issue 3: Real World Performance Problems ...... 11 1.4.5 Issue 4: Novel Communication Architectures ...... 12 1.5 Contributions ...... 12 1.5.1 Research Results ...... 12 1.5.2 Implementations and Prototypes ...... 13 1.5.3 Standardization work ...... 15 2 Unchallenged Networking ...... 17 2.1 Switching Packets or Circuits? ...... 17 2.2 A Layered Architecture ...... 19 2.3 Intelligence on the Edge ...... 21 2.4 A Hierarchy of Networks ...... 22 2.5 Internet Routing ...... 23 2.6 Naming and Addressing ...... 24 3 Challenged Networking ...... 27 3.1 The Unshackling of the Internet ...... 27 3.2 Radio Frequency Communication ...... 29 3.2.1 Signal Propagation Effects ...... 29 3.2.2 Signal Propagation Ranges ...... 30 3.2.3 Shared Medium Transmission ...... 32 3.3 Wireless Link Technologies ...... 34 3.3.1 WiFi - IEEE 802.11 ...... 34 3.3.2 Bluetooth - IEEE 802.15.1 ...... 35 3.3.3 Other Wireless Link Technologies ...... 36 3.4 Challenges in Tetherless Communication ...... 36 3.4.1 Wireless Factors ...... 36 3.4.2 Mobility ...... 37 3.4.3 Multi-hop Communication ...... 39 3.4.4 Routing ...... 41 3.4.5 Sensing ...... 43 3.4.6 Disruptions and Disconnections ...... 44 3.4.7 Naming and Addressing ...... 47 3.4.8 Architectures ...... 50 3.4.9 Evaluation and Experimentation ...... 51 3.5 Summary and Conclusions ...... 53 4 Summaries of Included Papers ...... 55 4.1 Paper I: A Large-scale Testbed for Reproducible Ad hoc Pro- tocol Evaluations ...... 55 4.2 Paper II: Coping with Communication Gray Zones in IEEE 802.11b based Ad hoc Networks ...... 56 4.3 Paper III: Robust and Flexible Internet Connectivity for Mobile Ad hoc Networks ...... 57 4.4 Paper IV: Lessons from Experimental MANET Research . . . . 59 4.5 Paper V: A Testbed and Methodology for Experimental Eval- uation of Mobile Ad hoc Networks ...... 59 4.6 Paper VI: A Cross-Environment Study of Routing Protocols for Wireless Multi-hop Networks ...... 60 4.7 Paper VII: Interest-based Content Dissemination in Opportunistic Networks ...... 61 4.8 Paper VIII: A Search-based Network Architecture for Content- oriented and Opportunistic Communication ...... 62 5 Conclusions and Future Work ...... 65 6 Summary in Swedish ...... 67 Paper I: A Large-scale Testbed for Reproducible Ad hoc Protocol Evalu- ations ...... 81 1 Introduction ...... 83 2 APE Architecture and Implementation ...... 84 2.1 Choreography and Data Trafﬁc Generation ...... 84 2.2 Data Gathering Tools ...... 85 2.3 Data Analysis Tools ...... 86 2.4 Testbed Distribution ...... 86 3 Capturing Mobility in Ad hoc Networks ...... 86 3.1 A “Virtual Mobility” Metric ...... 86 4 Experimental Set-up ...... 88 4.1 Physical Environment ...... 88 4.2 Scenario Descriptions ...... 89 5 First Results ...... 90 5.1 vM Analysis ...... 90 5.2 Packet Loss and Hop Count Analysis ...... 93 xiv 6 Summary and Future Work ...... 96 Paper II: Coping with Communication Gray Zones in IEEE 802.11b based Ad hoc Networks ...... 103 1 Introduction ...... 105 2 Communication Gray Zones ...... 106 2.1 Performance Problems of Original AODV ...... 106 2.2 Conditions for the Forming of Communication Gray Zones 107 2.3 The Shape of Communication Gray Zones ...... 108 2.4 Unrealistic ns-2 simulations ...... 109 3 Eliminating Gray Zones ...... 109 3.1 Exchanging Neighbor Sets ...... 109 3.2 N–Consecutive HELLOs ...... 110 3.3 SNR Threshold for Control Packets ...... 110 4 Results ...... 111 4.1 Ping Delivery Success ...... 111 4.2 Continuous MP3 Streaming ...... 113 4.3 Intermittent HTTP Requests ...... 114 5 Discussion ...... 115 5.1 Comparison Against OLSR and LUNAR ...... 115 5.2 Protocols without Gray Zone Problem ...... 115 5.3 IEEE 802.11b is not Bidirectional ...... 116 6 Conclusions ...... 116 Paper III: Robust and Flexible Internet Connectivity for Mobile Ad hoc Networks ...... 123 1 Introduction ...... 125 2 Problem Diagnosis ...... 127 3 Related Work ...... 128 3.1 Internet drafts ...... 128 3.2 Evaluations of Systems ...... 129 4 Overview of MANET Internet Connectivity ...... 130 4.1 Destination Locality ...... 130 4.2 Gateway Discovery ...... 131 4.3 Forwarding ...... 132 5 Robust and Flexible Internet Connectivity ...... 135 5.1 Implementation Details ...... 136 6 Evaluation ...... 137 6.1 Simulation Results ...... 137 6.2 Experimental Results ...... 142 7 Conclusions ...... 144 Paper IV: Lessons from Experimental MANET Research ...... 147 1 Introduction ...... 149 2 MANET Research Approach ...... 150 2.1 Implementation ...... 151 2.2 Experimentation ...... 151

xv 2.3 Integration ...... 151 3 Lessons from Experimental Research within the IETF MANET Area ...... 152 3.1 APE Testbed – Reproducible Real World Testing ...... 152 3.2 AODV-UU – Protocol Implementation ...... 155 3.3 Gray Zones – Reality Complements Simulation ...... 155 3.4 The Ad Hoc Horizon – Focus on Achievable Goals . . . . . 157 4 Lessons from Research Outside the IETF MANET Area . . . . . 159 4.1 LUNAR – Targeting Simplicity ...... 160 4.2 Cross-Layer Optimization Between Link and Transport Layer ...... 162 5 Challenges to Ad Hoc Networking ...... 163 5.1 Dependence upon Link Layer Technology ...... 163 5.2 Limiting the Problem Space ...... 163 6 Outlook ...... 164 Paper V: A Testbed and Methodology for Experimental Evaluation of Wireless Mobile Ad hoc Networks ...... 169 1 Introduction ...... 171 2 Simulation, Emulation and Real World ...... 173 3 Related Work ...... 174 4 Methodology ...... 175 5 The APE Testbed ...... 176 5.1 Software Overview ...... 176 5.2 Experiment Design ...... 177 5.3 Measurements and Data Gathering ...... 179 6 Assessing Repeatability ...... 179 6.1 Topological Replay ...... 179 6.2 Assessment Metrics ...... 180 6.3 Assessment Example ...... 182 7 Experiment Example: Routing Protocol Comparisons and the Gray Zone effect ...... 183 8 Discussion and Conclusions ...... 185 Paper VI: A Cross-Environment Study of Routing Protocols for Wireless Multi-hop Networks ...... 189 1 Introduction ...... 191 2 Experimental Setup and Methodology ...... 192 2.1 Coupling the Real World, Emulation and Simulation . . . . 192 2.2 Traffic Configuration ...... 194 2.3 Scenario Descriptions ...... 194 2.4 Routing Protocols ...... 196 3 Evaluation ...... 198 3.1 Measurements and Metrics ...... 198 3.2 Performance when Varying the Evaluation Environment . 199 3.3 Performance of Sensing Mechanisms ...... 200 xvi 4 Related Work ...... 205 5 Conclusions ...... 206 Paper VII: Interest-based Content Dissemination in Opportunistic Net- works ...... 211 1 Introduction ...... 213 2 Related Work ...... 215 3 Content Dissemination Model ...... 216 3.1 The Node Profile ...... 216 3.2 Communities and Content Matching ...... 217 4 Experimental Data Sets ...... 218 4.1 Contact Data Sets ...... 219 4.2 Sociology of the Data Sets ...... 220 4.3 Interest Information ...... 221 5 Methodology ...... 222 5.1 Implementation ...... 222 5.2 Simulator Setup ...... 224 5.3 Dissemination Strategies ...... 225 5.4 Performance Metrics ...... 225 6 Results ...... 226 6.1 Performance from Contact Trace ...... 227 6.2 Impact of Interest Semantic ...... 228 6.3 Community Collaboration ...... 230 6.4 Community Size ...... 231 6.5 Dissemination Cost ...... 231 6.6 Delay and Path Length ...... 232 7 Discussion and Conclusion ...... 233 Paper VIII: A Search-based Network Architecture for Content-oriented and Opportunistic Communication ...... 237 1 Introduction ...... 239 2 Design Overview ...... 241 2.1 Desktop Searching ...... 241 2.2 Unified Metadata Namespace ...... 241 2.3 Search-based Networking Primitives ...... 243 2.4 Persistent and Searchable Storage ...... 246 2.5 Dissemination and Event Notification ...... 247 2.6 Searching as a Networking Abstraction ...... 248 3 The Haggle Architecture ...... 251 3.1 The Core System ...... 251 3.2 The Managers ...... 256 4 Implementation ...... 258 5 Evaluation ...... 260 5.1 Methodology and Experimental Setup ...... 260 5.2 Results ...... 261 6 Related Work ...... 263

xvii 7 Discussion and Conclusions ...... 264

xviii 1. Introduction

This thesis is concerned with the design, implementation and evaluation of communication protocols for challenged networks1. They are interesting from a communication perspective because they in many respects fundamentally change, or challenge, many established principles and assumptions in today’s communication networks – most notably the Internet. This has implications for the design, implementation, and evaluation of communication protocols, and requires us to think in new ways about how to approach computer communication. Challenged networks have emerged mainly as a result of developments in wireless and mobile communication technologies during the past couple of decades. Examples include wireless and mobile ad hoc networks (MANETs), wireless sensor networks (WSNs) consisting of small embedded sensor devices, and delay and disruption tolerant networks (DTNs) that allow postal service class data delivery. As technologies continue to evolve, even more types of networks will likely emerge, leading to an increasing diversity in the communication scenarios possible. In this chapter, we give the big picture introduction to the issues that arise when communication is built on top of challenged networks, discussing broadly the research problems in the area, and how this thesis contributes. In the next chapter, we give an introduction to unchallenged networking, i.e., the existing approaches to communication that has served well for a long time, but are today under increasing stress. This background is necessary to understand the issues brought forward by the challenged networking scenarios discussed in Chapter 3. In that chapter we also give an overview of the related work in the area. Chapter 4 gives short summaries of the included papers, whilst Chapter 5 presents our conclusions. Chapter 6 gives a short summary in Swedish. However, in the remaining sections of this chapter, we ﬁrst take a step back and look at the purpose for developing communication protocols to begin with, the requirements we put on them, and how they have traditionally been designed. Then follows a discussion on how these requirements are challenged by new communication scenarios, which subsequently expose the hidden assumptions that the requirements embed. Finally, we present the contributions of this thesis.

1Challenged networks were, to the author’s knowledge, ﬁrst used in this context in Kevin Fall’s seminal paper: A Delay-Tolerant Network Architecture for Challenged Internets [1].

1 1.1 Communication Protocols and Requirements Humans need languages to communicate and so do computers. A digital communication protocol is a common language decided upon, i.e., in which order zeros and ones are sent and received, such that two conversational end- points2 can individually interpret and understand the data sent between them. Each zero and one consumes a tiny piece of a limited and, perhaps, shared resource – such as a wire or the Ether. It is thus important that protocols do not only solve the problem of conveying data, but also do so efficiently, fairly and securely. These are typical requirements that are put on most communication protocols. Designing expressive enough protocols to communicate what is needed, while taking into account requirements like the above mentioned, is a major goal of computer communication research. Apart from these basic requirements, each new design of a communication protocol comes with its own unique requirements. These are decided by the services offered and the scenarios under which they operate. However, requirements are seldom orthogonal, and trade-offs are therefore an integral part of communication protocol design. For example, the cost of cryptographic computation often leads to solutions that trade security for efficiency. Striking a balance between requirements can be a delicate matter. In order to meet the design goals and requirements put on a protocol, explicit and implicit assumptions are made about the environment the protocol will operate in, the speed of the transmission media used, and so forth. As society and technology evolve, new requirements emerge that challenge the assumptions of the established communication protocol designs. New environments make old requirements obsolete, or even pose hurdles for deployment and efficient operation. Hence, the emerging computer networks, where the traditional communication requirements cannot be maintained (or only partially so), we call challenged networks. We now proceed to discuss how traditional computer communication is increasingly challenged.

1.2 Examples of Communication Challenges Judging by the immense success of the Internet and cellular communication systems, their initial design requirements have been met, or even exceeded. However, with this success comes also a cost. As more and more communication is incorporated into these systems – the Internet being the prominent example – we ﬁnd that some requirements become barriers to innovation, and hence challenges themselves. The most obvious example of such a barrier stems from the requirement that each node in the Internet must be allocated a unique address. The Internet Protocol (IP) is the the basic protocol that de-

2We use the generic term node for a computing device in a network, and the name end-point for a node that is actively communicating with another one, i.e., its peer.

2 fines Internet addressing. When IP was designed in the early 70:ies, it was assumed that 32 bits would be enough for the length of the address [2]3. This gives 232 (∼4 billion) unique addresses, which in reality is reduced because the addresses are allocated such that all of them are not used. Considering the rate of computerization, it is not hard to see that we are quickly running out of addresses, and that there is an increasingly urgent address depletion problem. With the appearance of the network address translation (NAT) principle [3], the address depletion problem has been alleviated by loosening the requirement that each node must be allocated a unique address. However, this does not address the core problem, and instead it illustrates the importance of designing for growth and flexibility – something which is truly a challenge in itself, since it is hard (or even impossible) to foresee future applications and technologies. Whilst unanticipated success can result in challenges as just described, recent Internet evolution promises more drastic challenges at the level of maintaining basic communication, or even establishing it from the beginning. As the Internet conquers the airways and infiltrates everything from the tiniest to the largest devices, its model of communication is spreading into areas for which it was not originally designed. Televisions, telephones, and every computing device imaginable is becoming Internet aware. We see a convergence in service delivery, where both Voice-Over-IP (VoIP) and IPTV are examples of traditional media services that are moving to an IP centric delivery model. Not only does this trend stress the address depletion problem, but it also raises new problems. So far, the Internet technologies have coped well with this disruptive evolution – which is a testament to the soundness of their initial design. How- ever, new scenarios are stretching the limits of the technologies. Examples include satellites networked into an inter-planetary Internet [4], postal service like data delivery into rural areas where communication infrastructure has not yet been, or might never be, developed [5]. Long delays, disrupted communications, and intermittent connectivity in these scenarios cause the Internet protocols to cease functioning. In some other scenarios the Internet protocols just work poorly. For instance, this is the case in ad hoc networks deployed in disaster areas where infrastructure has been destroyed, and in high speed inter-vehicular communication. These scenarios stress protocols by their high mobility, causing frequently changing communication paths. Furthermore, in sensor networks the protocols face constrained computing resources (such as CPU and memory), and strict power saving requirements. Together the above scenarios increase the heterogeneity of networking devices. This is itself a problem, because the Internet technologies were designed for a rather homogeneous network environment. It is today harder to make assumptions about the capabilities of a communication peer.

3Actually, the address length was 24 bits in the original proposal.

3 The problems described above are not only related to unanticipated growth, or hidden limiting assumptions in protocol designs. They challenge more fundamental principles of communication; the most prominent being the end-to- end principle. We will discuss it in more detail in Chapter 2, but for now it sufﬁces to say that it suggests that two end-points should manage their own conversation, preferably reducing the involvement of other nodes on the path that connects them. This principle is impossible to maintain if there never exists a fully connected path between the communicating peers. This is because they need to coordinate their conversation, e.g., to adapt the rate of message exchange, and therefore they need a reliable path over which they can give each other timely feedback. Thus, the communication assumes a physical connection between the end-points. This connectivity requirement might seem as an essential prerequisite for any type of communication to take place. However, the postal system is an example where it is not. As digital communication becomes more mobile, and people move between hotspots of connectivity, the connectivity requirement becomes increasingly restrictive. Therefore, principles like the end-to- end principle have to be reconsidered in order to support, e.g., postal service style digital store-carry-forward message delivery. This style of delivery carries information between intermittent communication opportunities, and might actually be an attractive means of enabling communication where it is otherwise impossible. However, intermittent connectivity does not only have consequences for end-to-end communication models. Without connected paths, there is no measurable upper bound on communication delay, and it may vary signiﬁcantly from time to time. In store-carry-forward systems, delay can be counted in hours, or even days or weeks. In conversational communication (e.g., voice), low latency is an explicit requirement as the information which is communicated quickly becomes outdated or obsolete. This is why early warning systems for tsunamis, to take one example, are not optimally built on top of the postal service. On the other hand, the postal service makes an excellent last resort replacement for communication in case the telecom infrastructure has been completely eradicated by disaster. To enable communication without end-to-end connectivity, we hence need to revisit the established models and principles of how we today communicate digitally. Good predictors of how well communication systems cope with different scenarios are how they approach naming and addressing, and how they resolve which nodes have which names and addresses. In the Internet, for example, hosts4 can have mnemonic names that are resolved through the Domain Name System (DNS) - a globally distributed name look-up service that maps the human readable names to the (for the user) unintuitive IP addresses. An IP ad-

4A host is an Internet node that runs applications or services, as opposed to a router that only forwards information between hosts.

4 dress is in turn resolved to a hardware address of a network interface, using the Address Resolution Protocol (ARP). Thus, hosts and other entities can exist and be named (or addressed) in different namespaces, and resolution is used to map between them. When, where, and how to do resolution is determining for many properties of networking systems and architectures; for instance, one such property is the support for mobility. There are attempts to generalize these namespaces, with the objective to better support, e.g., mobility, disconnections, and content oriented communication. However, generalizations are usually achieved at the cost of efficiency. The Internet predominantly does early binding of names and addresses, which is very efficient, but less flexible than late binding. Early binding makes sense in networks with low mobility, and where infrastructure is readily available. However, in networks without central lookup services (such as DNS), and where disconnections are frequent, it is often impossible to support early binding. Alternative resolution mechanisms must be developed that are flexible enough for these environments. An example of alternative resolution mechanism, which already exists in the Internet, is searching as provided by web search engines. They map content to hostnames, and they do it in flexible ways in that they return ranked lists of results (actually just the identifiers for files and hosts), which are as precise as the keywords given as input. Security is another important requirement in all communication, which often is challenged when moving into new communication domains. Attackers should not be able to easily disrupt communication in ways that deny users of their services. One class of attacks is generally referred to as Denial of service attacks (DoS). Challenged networking opens for new ways to launch DoS- attacks, such as jamming of wireless communications. Apart from providing resilience to DoS-attacks, a communication system also needs to protect sen- sitive data. For example, online banking requires secure authentication and encryption. In small and embedded devices, with limited computing power and energy, resource intensive cryptographic computation might be infeasi- ble. Further, if devices are mobile they might not have access to trusted authentication authorities, which can vouch for their identities. Hence, it is not straightforward to establish trust between challenged network parties. Secu- rity is, however, not a topic which is explicitly addressed in this thesis, but it remains an important issue in all networking, and should always be kept in mind. Finally, robustness is a harder to achieve when moving towards mobile and distributed communication. Communication systems should not break down because of an intermittent disconnection, or a single failure in the system (this is often referred to as the single point of failure). Rather, the system should be able to tolerate failures, exploit redundancy, and scale with the size of the network.

5 The example challenges discussed in this section, highlight the requirements of challenged networking that need to be addressed in some way or another. It is evident that the approaches to doing so vary depending on the scenario. As digital communication systems continue to evolve, along with our need for them, more challenges will undoubtedly emerge. This thesis does not propose a universal solution for challenged networking, as none will likely ever exist. However, it conveys a number of lessons, and propose incremental improvements as well as entirely new solutions. We now turn to discussing research issues in challenged networking, followed by the speciﬁc issues addressed in this thesis.

1.3 Research Issues and Topics The goal of researching challenged networks is to enable communication – no matter how resource constraint or challenged the environment – either by evolving the Internet protocols, or by developing entirely new ones. This thesis makes contributions in both aspects. Here we discuss ﬁrst, in a wider scope, the research issues that this thesis concerns.

1.3.1 New Networking Paradigms Tackling the problems of challenged networking sometimes requires entirely new communication paradigms, and “out-of-the-box” thinking. Consider, for instance, a mobile environment where people move between different locations. In this scenario, addresses that fix nodes to locations do not serve mobile users well, as they may continuously change their locations. Today in the In- ternet, care-of addresses are used, such that messages are resent from a node’s home network to its visited network. This is inefficient, and breaks down in challenged networks with intermittent connectivity. Thus, addresses that are bound to specific nodes are bad indicators for how, and where, to deliver messages. An alternative approach to fixed addressing is to exploit a priori knowledge about the behavior of mobile users, such as their habits and geographical movement patterns. This information can be used to quickly pinpoint one or a few possible locations, and hence improves the chances of delivery. A challenge, however, is from where to get such information. One approach, which has recently received increasing attention, is to exploit social networks [6, 7, 8, 9, 10]. An interesting observation in these networks is the so called small world phenomenon [11]. This notion comes from a set of experiments by the American social psychology researcher Stanley Milgram in 1967, and suggests that people in the United States are connected by, on average, six friendship links. This later became known as six degrees of separation [12]. His experiment tells us that, by using the social context, a message can be

6 forwarded to anyone using only six hops on average. Today, social networking websites exploiting friendship links, such as Facebook and MySpace, have literally invaded the Internet. They provide unique opportunities to study the utility of social networks in digital data dissemination.

1.3.2 Heterogeneity: Compatibility and Interconnection Radically new communication protocols, such as those exploiting social networks, risk breaking compatibility with legacy protocols. However, in many challenged networks it is still attractive to be able to communicate with legacy networks. Thus, it is desirable that new protocols remain compatible with existing ones. If this is not achievable, another approach is to design gateways that translate between the protocols of legacy and challenged networks [13]. Interconnecting different networks is thus an important research topic. As already pointed out, there is a convergence toward IP technology, but it is clear that not all challenged networks are able to support a full Internet protocol stack. Some of them instead have to rely on translation mechanisms in gateways, in order to bridge the gap between new and old networking paradigms. Resolution plays an important role in translation; network architectures that do late binding support translation better than those that do early binding. This is because anticipating the exact means of delivery, at the time information is sent, is difﬁcult (or impossible). Therefore, protocols that early bind themselves to one way of delivery are less likely to succeed.

1.3.3 Configuration and Deployment The consequences of challenged networking are more far-reaching than their effects on basic communication principles. There are implications for implementation and deployment of systems as well. The increasing ubiquity of small computing devices requires that network protocols are implemented with limited capabilities in mind. For example, embedded sensor motes have limited input/output interfaces that are not easily accessed by humans. Sen- sors and other small devices must therefore, to a high degree, be able to self-configure – something which is also important when they are deployed in remote areas, where they must act autonomously during long periods of time. A rover exploring Mars, or a sensor mote monitoring the icecap of the Antarctica, are difficult to update and administer once deployed. Temporary ad hoc networks deployed in disaster areas, where the communication infrastructure has been destroyed, cannot require extensive configuration by the people deploying it. Otherwise, they will be functional first when rescue operations are futile. To conclude, challenged networking also has implications for self- configuration, self-healing and fault-tolerance.

7 1.3.4 Testing and Evaluation Testing and evaluation of systems, which operate in challenged networks, also require our attention. There are considerable problems involved when designing testbeds and evaluation methodologies that ensure repeatable experiments and reproducible results. Wireless radio communication and mobility are difficult to deal with, as they have inherent random elements that are difficult to control and recreate from one experiment to another. Simulations therefore represent a convenient alternative. However, the random factors of wireless communication and mobility are often as difficult to model in simulation as they are to control in reality. Simulation models are hence continuously developed and refined. Emulation can be an effective middle-ground that runs protocols in real systems but with, e.g., an artificial wireless channel that can be controlled. This makes repeatability easier to achieve. Even with accurate simulations and emulations, experimental research is still needed to validate the models and emulation methods. One may ask if it is not best to rely solely on real world experiments, since its anyhow necessary to conduct them for simulation validation purposes? The answer is that simulations provide a way to scale scenarios to large sizes, to perform repeatable simulations with parameter variations, and to easily debug protocols. In comparison, real world experiments are extremely costly in terms of manpower, time and hardware resources. This is stressed even more in challenged networking for several reasons. First, most challenged networks do not actually exist at the time research is initiated (as opposed to the Internet). Therefore, testbeds and experimental networks have to be built from scratch. Second, the nature of challenged networks – often involving wireless communication, mobility and perhaps deployment in remote areas – contributes to a big investment when performing experiments. Finally, experiments, e.g., involving social network contexts, are difficult and sometimes impossible to repeat in the real world. Trace collection together with simulations can in this case be a powerful combination that bridges the real world and simulated environments, whilst at the same time ensuring a level of repeatability. Hence, simulations remain an important method for researching challenged networks. To conclude, the development of experimental testbeds that incorporate methodologies for repeatability, and which can bridge simulation, emulation and the real world, is important for challenged networking.

1.4 Research Issues Addressed in this Thesis The research in this thesis is systems oriented and spans several types of challenged networks and scenarios. It includes both experimental and theoretical work. If there is one focus of this work that should be stressed, it is the ambition to make protocols and systems work effortlessly in the real world. Much research stops short of implementing and experimentally evaluating systems

8 in reality. In challenged networking, prototyping and experimentation are especially important, because there are more hidden and unpredictable elements than in legacy networking. The impact of these elements, on the functionality of systems, cannot be foreseen in an all-theoretical approach. However, before we continue discussing the research in detail, we ﬁrst give some background, in order to motivate the issues addressed and the approaches taken.

1.4.1 Background and Motivation The earlier work of this thesis was carried out within the area of mobile ad hoc networks (MANETs). This was one of the first areas of challenged networks to draw serious attention from researchers. MANETs are loosely defined as networks of mobile nodes that use wireless means of communication, and rely on each other to set up connections over multiple hops. Under this generic definition, the scope of MANETs is quite broad, and can therefore encompass many different scenarios, with varying challenges. Historically, this broad and general definition has been an obstacle to focused research, which limits the problems to solve to a set of clearly defined ones [14]. Solutions have therefore also been general, and have not taken into account the circumstances of specific scenarios, such as the type of mobility or the applications used. Only in recent years have credible real world application scenarios appeared [15, 16], which more clearly define the circumstances for which researchers can design their protocols and systems. As a result of the early lack of focus, simulations have dominated the research, and they have been based on generic and randomized scenarios in terms of node movements and topography. The norm was for a long time simulations with random waypoint mobility (RWP) [17] over, e.g., a 1500x300 meter rectangular area, using simple free-space radio models [18] – and to some extent they still are. RWP mobility – although efficient for stressing protocols and studying their lower performance bounds – is not a very realistic mobility model, as users rarely (if ever) move in random patterns. A study published in 2006, revealed that among all MANET simulation papers published in the proceedings of the well regarded MobiHoc conference, less than 15% were repeatable [19]. Only 12% of the papers appeared to be “based on sound statistical techniques”. These are discouraging figures that illustrate the importance of developing rigorous methodologies that allow repeatability. While the study looked at technical flaws of simulations, it did not discuss the validity of the scenarios chosen, and the conclusions that can be drawn from the results produced. Random scenarios are quite useful to stress-test protocols in order to reveal bugs and general performance traits. But it is not well understood how well random scenarios, with simple models of mobility, radio, and environments, can help us predict the performance of MANET protocols in the real world. Experimental research can fill this gap,

9 but it requires the speciﬁcation of real scenarios and rigorous methodologies, in order to achieve repeatability. Today, there are numerous types of challenged networks, such as wireless sensor networks (WSNs), mesh networks, delay tolerant networks (DTNs) and vehicular ad hoc networks (VANETs), which can all be seen as having branched off from MANETs. With this development, the number of experimental networks and testbeds have also drastically increased, because nowadays there exist credible applications and scenarios that can be realized. We now describe the speciﬁc issues addressed.

1.4.2 Issue 1: Repeatability in Real World Experiments As the research of challenged networks in general, and MANETs in particular, has incorporated an increasing portion of experimental approaches, the design of testbeds has become an issue of major importance. The ad hoc protocol evaluation testbed (APE), which is described in Paper I, is one of our early contributions in this area. The research issue that we wanted to address with APE was how to experimentally evaluate challenged networking protocols in the real world, such that experiments can be repeated and results reproduced. At the time of its creation, APE was the first real world testbed of its kind, in that it could easily be reused by other researchers for their own experiments. It focuses on automating experiments, management and test repeatability, rather than on specific hardware setups that are hard for other researchers to recreate. Repeatability and reproducibility are central themes, and APE provides a unique approach to these issues. Instead of random and uncontrolled scenarios, APE is designed to run strictly controlled scenarios that use movement choreography and automated traffic generation.

1.4.3 Issue 2: Cross-environment Evaluation Methodologies Another research issue addressed in the context of APE is how to couple simulations, emulations and real world experiments, in order to improve evaluation methodologies for protocols and systems with complex interactions. In general, there are two established roles between simulations, emulations and real world experiments: 1) They are uncoupled, and used in separate evaluations in order to explore different aspects of protocols and systems. In this approach, simulations are used to study large scale trends that are impractical to study experimentally, whilst small scale experiments (real world or emulations) are used to illustrate applicability to the real world. 2) Simulations, emulations and real world experiments are coupled, such that feedback from experiments drive simulations (or emulations) directly, thus replacing some of the models. This feedback is also used to validate the simulation models, as it is possible to compare the results from the trace based simulations with the results from the modeled simulations.

10 We argue that the established roles above do not draw upon the individual strengths of each environment to the extent possible. For instance, network protocols can be very complex and, when run on top of a full networking stack, they also interact unpredictably with other parts of the system. A strength of simulations is that they isolate the evaluated protocols from complex system interactions. Emulations, on the other hand, have these interactions but lack, e.g., the radio environment or the mobility. As simulations (especially those using simple models) are free from many interactions , they provide a base line evaluation environment. By gradually exposing the protocol logic to the real world – starting from simulations with increasingly realistic models, followed by emulations as an intermediate step, ending with final real world experiments – it is easier to identify the exact reasons for poor performance. This identification can otherwise be very challenging. For instance, if there is a big discrepancy in the performance of a protocol in simulation versus the performance in the real world, it can be very difficult to figure out whether this is due to a protocol bug, an unexpected interaction with another protocol, or solely the effect of a real radio environment. By following the methodology above, it is possible to exclude some reasons for poor performance. The reasons that can be excluded are dependent on the step at which the performance starts to degrade. Thus, our argument is that simulations, emulations and real world experiments are rarely systematically combined in the way described. We therefore aim to provide better investigative tools that can be applied to the same evaluated scenario, and can draw upon the individual strengths of each tool. Our aim is not primarily to validate or improve simulations. Such validations typically use trace-based coupling to make the evaluation environments more similar. We instead try to exploit the differences of the environments as they are, with individual strengths and limitations. Some challenges toward our goal are to develop code that runs unaltered in each environment, to recreate scenarios in each environment such that they are comparable, and to develop methods that allow the results to be analyzed. Papers I, V and VI each contribute solutions to these challenges.

1.4.4 Issue 3: Real World Performance Problems The methodology described above provides means to unravel some of the hidden assumptions, and other problems, of traditional communication stacks – in this case the Internet protocol stack. The third research issue addressed is hence how to discover, and if possible reduce, the negative effects of hidden assumptions (and other problems) in communication protocols, as they are applied to challenged networking scenarios. The approach taken includes modifying or extending protocols to cope with hidden assumptions, and to develop protocols and systems that bridge the gaps between different types of networks. Papers II, III and VI address these issues.

11 1.4.5 Issue 4: Novel Communication Architectures As evident by our research described so far, it has focused on discovering, understanding, and addressing many problems that communication protocols experience in challenged settings. Naturally, this has also increased the understanding of the challenges faced, and the limitations of existing approaches. This experience has been used to explore radically new ways to do communication, in severely challenged scenarios. This leads us to the final issue addressed in this thesis, which is how to design and implement communication protocols and architectures for challenged networks. We do not claim that our research provide general solutions for any type of challenged network. Instead, we identify the issues to address in specific scenarios and propose novel solutions, which are not necessarily the only viable ones. For instance, in Paper VII, we illustrate the feasibility of content sharing in opportunistic networks through interest-based disseminations. In Paper VIII, we go one step further and present a network architecture for content-oriented and opportunistic networks, which is mainly motivated by two recent networking trends. First, networking is becoming increasingly content oriented, e.g., the Web, YouTube-style movie streaming, and peer-to- peer file sharing is dominating the application space. Second, we increasingly use searching to find, order, and access content – either on the Web or on our local computers and devices. As devices become smaller, and yet more capable, we are also likely to frequently access content when mobile. We will hence increase our mobile communication, and applications that drive this increase will likely be based on some sort of content sharing, in which searching is an integral part.

1.5 Contributions The research contributions of this thesis are divided into research results, software implementations and prototypes, and standardization work.

1.5.1 Research Results The following is a list of the most important research results in this thesis.

• Development of methodologies for experimentally evaluating MANET- based challenged networking protocols in a repeatable manner (Papers I and V). • Identiﬁcation and analysis of a number of phenomena that contribute to poor real world performance for established MANET routing protocols, such as AODV and DSR. These phenomena include Communication gray zones (Paper II), State replication problem (Paper III), TCP backlashing, Self interference and Link cache poisoning (Paper VI).

12 • An analysis of MANET Internet connectivity issues, identifying the importance of indirection support in gateway forwarding strategies (Paper III). • A design of robust and ﬂexible Internet connectivity in MANETs (Pa- per III). • A performance comparison of MANET routing protocols being standardized by the IETF. The compared protocols are AODV, DSR and OLSR (Paper VI). • A study of content-oriented dissemination in opportunistic networks (Pa- per VII). • A search-based network architecture for content-oriented and opportunistic communication (Paper VIII).

1.5.2 Implementations and Prototypes During the course of the work of this thesis, a number of software implementations of various network communication protocols and research tools, such as testbeds and simulators, have been created. The impact of these projects, on my understanding of network communication protocols and research methods on one hand, and on the visibility of my research on the other, should not be underestimated. In many cases, these implementations have generated far greater value, in terms of visibility and personal satisfaction, than the research work that they have enabled. All implementations have been, or aim to be, released under free software licenses [20]. This means that all the source code is in the public domain, and that the freedom of usage of the software and derivatives is protected. Perhaps, more importantly, the code allows research results to be independently veriﬁed and reproduced by other researchers. Be- low follows a list of the software implementations and prototypes that I have developed myself or in collaboration with others, along with a bit of history of their developments.

APE Testbed I was ﬁrst introduced to the research world in the summer of 2001, when I did an internship in the CoRe research group at the Information technology department of Uppsala University. I was working on a testbed for evaluating communication protocols in ad hoc networks, together with Mattis Fjällström. I continued working on this project as part of my diploma thesis [21], under the supervision of Henrik Lundgren and Christian Tschudin. I also collab- orated with David Lundberg, who was doing his thesis at the same time. Al- though I am responsible for most of the development of the testbed, which was later named the Ad hoc Protocol Evaluation (APE) testbed, all of the above mentioned people have in some way contributed to APE. APE was released as free software on SourceForge.net in November 2001 and has since been downloaded over 7,700 times.

13 AODV-UU AODV-UU is an implementation of the Ad hoc On-demand Distance Vector routing protocol [22], which was being proposed for standardization within the IETF’s5 MANET working group [23]. The UU suffix denotes the Uppsala University origin. AODV-UU is the most successful and well known implementation of this thesis work. It has been used in numerous external research projects, and many modifications and additions to the implementation have been made by other research groups. These include, several IPv6 versions [24, 25, 26], multicast extensions [27], support for SIP and alternative routing metrics [28], ports to various simulator environments [29], and a DYMO implementation [30]. I started work on AODV-UU for the Linux operating system after we found that the few existing MANET routing protocol implementations did function well. To evaluate the real world performance of these protocols, we first needed working implementations. I created AODV-UU myself, with some initial support from Henrik Lundgren. We hosted the code on our own servers from the first release in January 2002 until December 2007, at which point it was moved to SourceForge.net. We have no reliable figures for the total amount of downloads since the first release, but on SourceForge, AODV-UU has had around 150-200 downloads each month. Part of the success of AODV-UU can probably be attributed its ability to run in both the real world and in simulation environments, using the same un- modified code. Initial work on porting the code to the popular ns-2 network simulator [31] was done by Björn Wiberg. I have continued to maintain and improve the simulator version since then. The authors of the AODV protocol specification have officially recommended AODV-UU, and at one point they proposed it would become the official AODV implementation in ns-2, replacing the already existing one. However, this was rejected by the ns-2 maintain- ers on the basis that AODV-UU is mainly implemented in the C programming language, while ns-2 is implemented in C++. AODV-UU participated with success in the first AODV interoperability test in June 2002 [32].

DSR-UU The lack of MANET routing protocol implementations that functioned well, continued to be a problem during our work with APE. Our ambition with APE was, from the beginning, to do side-by-side routing protocol performance comparisons. The Dynamic Source Routing protocol (DSR) [17], was the main competitor to AODV and hence a candidate for comparison. I started implementing DSR-UU in 2006, and at that point there were no DSR implementations publicly available that was standard compliant, worked for our platforms, and had the features we needed. DSR-UU was implemented for Linux and ns-2 from the start and was implemented in C. It remains one of the

5Internet Engineering Task Force - www.ietf.org

14 few DSR implementations available. The code was, together with AODV-UU, moved to SourceForge.net in December 2007. It has around 50-70 downloads each month.

Simulator for Opportunistic Networks As part of my work with Paper VII of this thesis, I implemented a simulator for opportunistic networks. The complexity and overhead of more sophisticated simulators motivated the implementation of a simple simulator, speciﬁcally optimized for our purposes. These optimizations include customizations for trace driven simulations, using traces collected with iMotes6 [33]. The simulator is written in C and will be released, along with the traces, once no longer required for our own research.

Haggle Reference Implementation Haggle is a project in the Future and Emerging Technologies (FET) programme under the European Union’s Information Society Technologies. It is funded by the Situated and Autonomic Communication area of FET, which is part of the European Union’s Framework Programme 6 (FP6). Haggle aims to develop a new autonomic networking architecture designed for communication in the presence of intermittent network connectivity. The ﬁrst reference implementation of Haggle was created by Intel research in Cambridge, which at that point was a partner in the project. This research lab was later closed and Intel withdrew their Haggle partnership. Uppsala Univer- sity was assigned the task to create an updated and redesigned architecture, along with a reference implementation. I designed and implemented the second reference implementation together with Christian Rohner. It runs on Windows, Windows mobile, Linux and MacOS X. The Haggle reference implementation is not yet publicly released, and my work is ongoing. Much of the experience and knowledge that I have accumulated during my graduate years have gone into the design of Haggle.

1.5.3 Standardization work As part of my work on AODV-UU, I interacted frequently with the authors of the AODV protocol. Through this process I provided feedback on the AODV IETF draft, which later became an experimental RFC7 [34]. Moreover, I contributed updated text to sections of the draft, and subsequently the RFC, where I clariﬁed some aspects of the protocol.

6Small embedded sensor devices with Bluetooth radios. 7Request For Comments - a standards document within the IETF.

2. Unchallenged Networking

The success of the Internet is undeniable, and today its effects can be seen in almost every aspect of our daily lives. Despite this success, the Internet is not without its deficiencies. These become more apparent as the usage of the Internet evolves into new areas, which were not foreseen when the In- ternet was first designed. The Internet copes with many of these deficiencies through over-provisioning of bandwidth. Thus, there is an implicit assumption that new demanding usage scenarios stay within the wired domain, where bandwidth is cheap and exists in abundance1. Assumptions like this make the Internet relatively unchallenged, and it therefore serves as a reference for unchallenged networking. What follows is a brief overview of the structure and functionality of the Internet. The purpose is to understand how to meet the challenges that emerge as the underlying assumptions in the Internet are changed. The overview focuses on key design principles – some almost “holy”, and frequently the focus of much heated debates. Whilst many of these principles have served well for the Internet, and still do, they are often the ones that are first under attack when we consider challenged networking. The reader who is well versed with the functionality of the Internet may want to skip to the next chapter.

2.1 Switching Packets or Circuits? The Internet consists of hosts and routers, which are collectively referred to as nodes. Hosts sit at the edges of the network and run applications, while routers are dedicated to relaying data in-between hosts. The relaying is done using a technique called packet switching [35], and is illustrated in Figure 2.1. Packet switching divides data into smaller packets that each carry a protocol header containing the address of the end host they are destined for. This allows packets to be individually forwarded toward the destination – through a chain of routers – such that each packet can potentially take a different route. A packet must be completely received by a router, and maybe queued for transmission, before it can be passed on to the next router in the chain.

1In recent years, there have been frequent discussions about why, e.g., Google is buying Dark fiber. This type of fiber exists as fiber optic cables that have been dug down, but are currently not “lit up” due to unused capacity.

17 Packet switching stands in stark contrast to circuit switching, a technique prevalent in older telephone networks2. In those systems, a dedicated wire circuit is established between each pair of communicating peers, as shown in Figure 2.2. The data is not split into packets, and do not carry the overhead of individual headers, as the dedicated circuit instead guides the data to its destination. The advantage of this approach is that it is easy to guarantee quality of service (QoS), because the circuit allocates a ﬁxed amount of the total resources in the network. However, although both approaches have their ad- vantages and disadvantages, it is easy to point to the ﬂexibility and robustness of the packet switching approach.

1 d: B ... 0 d: B d: B n−1 d: B Host B

2 d: B n d: B

Host A Figure 2.1: Packet switching forwards packets individually to the destination based on the destination address in the packets’ headers.

Figure 2.2: Circuit switching establishes a dedicated circuit that guides the data to its destination in a continuous stream.

The strength of the packet switching approach is the discretization of information. Each packet is treated separately, and lost or corrupt packets do not affect the delivery of the rest of the packets. Depending on the application, lost packets can be resent, or if possible, the missing information recreated. Pack- ets can take different routes between a source and destination, as depicted in Figure 2.1, and may arrive out of order. Therefore, each packet carries a number so that the original order can be recreated at the destination, and missing packets identiﬁed. Packet switching is perhaps the single most important innovation that paved the way for the Internet, and digital communication networks

2Nowadays, telephone networks are more like packet switched networks, with circuit switching implemented on top of the packet switched network in the form of virtual circuits.

18 in general. As we move communication into more challenged scenarios, we ﬁnd that this corner stone of the Internet, works well even in the most extreme challenged networks.

2.2 A Layered Architecture Packet switching is the basic communication abstraction in the Internet, but it only provides a ﬂexible foundation, on top of which more sophisticated communication models are built. In the Internet, this sophistication is provided by the Internet reference model, which is split over ﬁve layers that each implement a number of protocols collectively called the TCP/IP protocol suite. Figure 2.3 depicts how the layers are arranged in a stack and how protocols relate to the individual layers. Each layer is subjected to different challenges as the assumptions underlying the design of the Internet are relaxed, or alto- gether changed. We need to understand these assumptions (that are sometimes implicit) and their effect on the different layers, in order to understand how to approach challenged networking.

Application HTTP, FTP, SMTP

Transport TCP, UDP

Network IP

Link 802.3, 802.11, 802.15

Physical Coaxial, Fiber, Copper

Figure 2.3: TCP/IP network layer stack with example protocols on the right.

The TCP/IP stack loosely resembles the OSI reference model, which today is used as a the reference layer model example in every textbook on computer networking. The OSI model was the result of the Open Systems Interconnec- tion effort to standardize networking. This effort was initiated in 1977 [36] by the International Organization for Standardization (ISO)3 [37]. The point of each layer is to provide an abstraction to layers above and beneath according to a predefined interface. This makes the stack modular and flexible. Within the domain of a layer, different protocols may provide, through the interface, a service that is defined by the layer. A common analogy for this protocol layering is the postal system. The analogy starts at the application layer with a person writing a message that it wants to deliver to a friend. The message is put in an envelope and a global address is written according to a standardized format that the system can understand.

3ISO is not an acronym, but instead based on the Greek word isos that means equal.

19 The envelope is then put in a post box and from that point the person is no longer concerned with the message. It is handed over, according to a well known interface, to the “lower layer”, i.e., the postal service that provides a delivery service with some guarantee of error free delivery. This service is in the Internet provided by the transport layer. Using the global address on the letter the message is delivered through a chain of postal offices. This is equivalent to the packet switching approach, which in the Internet is handled by the network layer that also provides the global addressing. The actual transfer between each postal office may involve different freight companies with their own protocols, similarly to how the Internet is linked together of different link layer technologies. The actual physical transport may be done by airplanes, trucks or boats. In the same way, the Internet involves many physical layer technologies, such as fiber, coaxial cables, and even the wireless Ether. The point of the layering principle, that the analogy tries to highlight, is that what happens below or above a certain layer is abstract. For example, the person writing the message could drop the letter directly in the recipient’s post box, thus circumventing all the layers below. This can be done without changing the protocol of communication used by the person herself, i.e., a message written in a language on a piece of paper. Another advantage of layering is that it allows modular designs, where the implementation of a layer can easily be changed. The Internet model lacks the session and presentation layers of the OSI model. The reason why the Internet did not adopt the OSI model was because the developments that led to the Internet stack preceded OSI. Both were inspired by the approaches taken in the ARPANET [38] and CYCLADES [39]. These networks preceded the Internet and they pioneered the principle of an interconnection layer that allowed different networks to interface. The layering principle was further developed in the OSI and TCP/IP models. We make two interesting observations from these two efforts. First, both models adopted the layered approach and ended up being very similar. Layering was a natural way to structure communication, although the exact structure of layers and interfaces may of course vary. Second, it is today evident that many of the problems and challenges now facing the TCP/IP model, such as mobility, could have been less severe if, for example, the session layer of the OSI model would have been part of the Internet stack. Today, we instead find similar functionality in the application layer (e.g., in protocols such as SIP [40]), or as new layer proposals (e.g., HIP [41]). Although there are risks associated with attempting generalizations, a conclusion is that elaborate designs might better stand the test of time than specific ones. This is a lesson well worth considering when new architectures are designed for challenged networks. How to structure and design future communication architectures is a major research topic. Although the layered approach has many intuitive and useful abstractions, it is not always flexible enough to accommodate more radical

20 communication paradigms. In Chapter 3, we look in detail at the drawbacks of a layered architecture in different types of challenged networking scenarios.

2.3 Intelligence on the Edge The Internet protocols of the TCP/IP stack are designed around a principle, which suggests that the intelligence of the network’s services should lie in the communicating end-points. The network in-between should be “stupid” and just relay data. This is referred to as the end-to-end principle [42]. It has proved very successful and is often, to a large degree, attributed the success of the Internet itself. The network layer, implemented by the IP protocol, is the only protocol every communicating device in the Internet needs to understand. The IP protocol is simple, and essentially provides a uniform packet switching layer, and addressing space, for all nodes in the Internet. The uniform network layer gives raise to the hour glass shape of the Inter- net networking stack. There is the single IP protocol in the middle of the stack, and an increasing number of different protocols as we move up or down in the layers. As IP is the only mandatory protocol, and the ones above are only required in end-hosts, whilst routers only need to implement IP and any speciﬁc protocols below. Thus, new protocols can easily be deployed in higher layers without involving any other nodes than the end-hosts. Similarly, different networks can operate different protocols beneath IP, e.g., Ethernet (IEEE 802.3), Token ring (IEEE 802.5), or WiFi (IEEE 802.11). IP thus makes it possible to connect together networks operating different link layer technologies. IP provides a packet switching layer that every node in the Internet can understand. However, as we argued earlier, sophistication is implemented on top of packet switching. In the Internet, this is provided primarily by TCP - the Transmission Control Protocol. It provides data transport services on top of IP, ensuring reliable and in order packet delivery, through a data stream abstraction to the application layer. TCP is really where the intelligence of the Internet is implemented, and there is no doubt that the Internet would not be the success it is today without it. TCP essentially implements a virtual circuit on top of the packet switched network layer. It establishes a connection between two conversational peers and makes sure that any packets sent between them are assembled in order, that lost packets are resent, and that the rate of transmission is adapted to the congestion in the network. The ingenious thing is that TCP needs no support from the network to achieve this – everything is handled by the two end hosts that communicate. There are, however, a number of assumptions made by TCP that are relevant in the context of challenged networks. TCP implicitly assumes a connected path between the end-points, because it relies on feedback from a connection peer, and on delay measurements over the entire path for making intelligent decisions. Hence, the unchallenged Internet is assumed to be well connected

21 through end-to-end paths – an assumption which is perfectly reasonable considering that the Internet, until recently, has consisted more or less exclusively of wired links.

2.4 A Hierarchy of Networks Whilst IP addressing ensures that every node in the Internet can be uniquely identified, the addresses are not simply assigned at random. The IP address is very similar to a zip code in a postal address, where the first numbers refer to the part of a country, the next to a specific city, the ones after that to a specific part of a city, etc. By varying the length of the prefix in the IP address, one can similarly home in on a specific node in the Internet. The prefixes hence structure the nodes of the Internet in a hierarchy. Each prefix corresponds to a subnet of the Internet, which can be further divided into smaller subnets, and so forth. This subnetting is what makes the Internet hierarchical. At the top of the Internet hierarchy we find autonomous systems (AS:es), which are collections of IP networks. An AS is governed by a single entity, e.g., an Internet service provider (ISP), or some other large organization. Each IP network is further divided into smaller networks, until reaching the smallest sized network of, e.g., a company, university, or other organization. As the IP address determines both the identity of a node and its place in the Internet hierarchy, it has dual semantic meanings. This property of the IP address, being both a locator and identifier, is one of the main reasons the Internet has been able to scale to such a large network, whilst retaining its efficiency. However, it is also the source of many of its deficiencies, most notably the inability to allow nodes the freedom to move between locations without changing addresses, and hence identities. We discuss this further in Chapter 3. Each router in the Internet maintains a forwarding table with IP prefixes, in which each prefix points to a specific outgoing connection leading to another router. In order to reach a destination, an IP packet’s destination address is indexed into each router’s table, using so called longest prefix matching. This is done through the chain of routers leading from the source to the destination, and is called forwarding. In order to build the forwarding tables, the routers must exchange information with neighboring routers, and from this information they construct a map of the Internet. Thanks to the hierarchical prefixes, each router only needs to know the least specific prefixes that aggregate all the subnets on an outgoing port. Analogously, if one is located in Europe, it is only necessary to know one route to, e.g., the US, and then care about the specific directions once there. This aggregation makes the Internet scale, as forwarding tables would otherwise explode in size and routing would involve significant overhead.

22 The task of exchanging preﬁxes and destination addresses, in order to construct the forwarding tables, is called routing. Routing will be discussed further in the next section. A general conclusion that we can make already now is that structures, such as hierarchies, can be very powerful to achieve efﬁciency and scalability. However, building and maintaining network structures in general, and hierarchies in particular, are expensive operations that require time and continuous communication. In many challenged network scenarios there may be neither time, nor resources available, to build such structures.

2.5 Internet Routing Although the Internet is structured and fairly well organized, it remains a giant web of inter-connected networks that together provide many different ways to get from point A to point B. How to find, or compute, the most effective path between two points is the task of routing protocols. As with much of the In- ternet terminology, there is some ambiguity about the meaning of routing. Routing is often used interchangeably with forwarding, which in most circumstances is fine. Here, however, we make a stricter separation between the two. In routing, nodes exchange information that allow them to build a view of the network. Since it takes time to exchange this information, the views of individual nodes are in most cases incomplete, and their accuracy depends on the particular routing protocol used. Exchanging lots of information consumes time and resources, and therefore the accuracy is a trade off against how quickly the routing converges. Forwarding, on the other hand, determines the next node to give a packet to using the the knowledge acquired through routing. In the Internet, this involves indexing a packet’s address into a forwarding table, and then transmitting it on the resulting out port (i.e., physical line connection). Thus, forwarding is conceptually simple and happens through a locally executed algorithm (although sophisticated algorithms are required to do forwarding at line speed). Routing, on the other hand, is a distributed algorithm that involves potentially every node in a subnet. The challenge of a routing protocol is to compute efficient paths between any two nodes (usually in terms of the number of hops), and to do so in an error free manner, i.e., by avoiding loops in the paths or paths that involve broken or no longer existing nodes. The two most well known routing algorithm types are link state protocols and distance vector protocols. The differences between the two types will not be detailed here. However, for the purpose of the further discussion in later chapters, we note that route computation in the Internet is something that is done before communication can take place, and involves sending out control messages at regular intervals. When to do route computation, or if it should be done at all, are questions that are important in the context of challenged networks.

23 2.6 Naming and Addressing The layered design of the Internet architecture implies a strict namespace separation. Each layer defines its own namespace; the application layer deals with, e.g., hostnames, whilst the transport layer deals with port numbers. IP furthermore uses IP addresses, and the link layer MAC4 addresses. In order to bind these entities to one another, various types of resolutions are used. For example, DNS and ARP (as discussed in Chapter 1) map application layer names to IP addresses, and IP addresses to link layer addresses, respectively. Although the separation of namespaces in the Internet stack is strict, the semantic meanings of them are not. The Internet architecture today defines two ways to identify end-points; namely hostnames and IP addresses. IP provides mandatory addressing and end-point identification, whilst the domain name system (DNS) provides optional names that map to IP addresses. These two namespaces exist in parallel. This is because DNS was invented relatively late in the Internet’s evolution, and was a response to the increasing difficulty of dealing with large numbers of IP addresses during Internet’s rapid expansion. Names are easier to remember than obscure numbers, and therefore a naming system was added to complement IP. Naming in the Internet is hence more of an historical afterthought than an intentional design – as evident by duplica- tion of functionality. The interpretation of names and addresses in the Internet is therefore somewhat convoluted. Salzer offers in [43], the following definitions of names, addresses and routes: • A name identifies what you want • An address identifies where it is • A route identifies a way to get there By these definitions, an IP address is both a name and an address, whilst the hostname is just an alias. This has the consequence that an address change en- tails also a name change. Although Dynamic DNS today allows hosts to keep a fixed alias, this alias cannot be remapped to another IP address without in- terrupting ongoing communications. This is because communication sessions are bound to the IP address, and not the name alias. Another issue is that the namespaces spanned by hostnames, IP addresses, and routes, are tightly coupled both temporally and spatially. That is, an application must specify a destination address before communication can be established (spatial coupling), and the source and destination must be coordi- nated during the information exchange (temporal coupling). This tight temporal coupling means that the Internet does early binding, and makes the Inter- net’s communication highly synchronous. The BSD sockets interface, which is the de facto standard Application Pro- gramming Interface (API) for communication, reflects the early binding of the Internet stack. It requires an application to specify a transport protocol (e.g.,

4MAC – Medium Access Control.

24 TCP or UDP), an IP address, and a port number – just to initiate a communication handle. This pins down the communication session locally to a specific network interface and a transport protocol, and remotely to a host and a specific application (as specified by the port number). This is very efficient in well connected scenarios as the bindings are done once, and data can then be forwarded without interruptions. There are, however, drawbacks to this approach when less well connected scenarios are considered, as we do in the next chapter.

3. Challenged Networking

As the ubiquity of wireless computing devices has increased, so has the challenges to the traditional communication models. Untethered communication provides the freedom of mobility, and as a result, an abundance of new networking scenarios have emerged. The challenges posed by wireless communication and mobility penetrate all the layers of the networking stack. In particular, the link layer, network layer and transport layer are affected in ways that in some cases force us to entirely rethink established communication models. In this chapter, we ﬁrst introduce the disruptive wireless technologies that are responsible for this development. We then look at the new types of networks and communication models that are currently being researched, and the challenges they pose to traditional networking. This chapter, not only provides an historical overview of challenged networking, but it also discusses the related work and developments in the area during its evolution.

3.1 The Unshackling of the Internet Wireless packet radio communication – the wireless equivalent of packet switching in the Internet – was pioneered by the ALOHANET in 1970 [44]. ALOHANET connected the Hawaiian islands using a network of radio towers. However, it was not until 1985, when unlicensed spread spectrum was ﬁrst made available in the US, that the initial seed for the wireless revolution we now experience was sown. The availability of unlicensed spectrum was subsequently copied in most other countries, and has since led to the development of several disruptive wireless radio technologies in the 2.4 GHz and 5 GHz frequency bands, such as WiFi (IEEE 802.11), Bluetooth (IEEE 802.15.1) and ZigBee (IEEE 802.15.4). Although the ﬁrst cellular technologies operating in licensed spectrum were quickly adopted with the spread of mobile telephony, they were not adapted to data packet services until the introduction of GPRS in the second generation cellular networks, and UMTS in third generation networks. The data services offered are even today slow in comparison to those operating in unlicensed spectrum, and they furthermore require costly subscriptions. Therefore, data communication using cellular technologies has not really taken off. In contrast, WiFi today provides theoretical speeds up to 248 Mbit/s and can be deployed at the cost of a wireless access point (AP) (around 60 e).

27 Cellular systems and WiFi of course have different trade-offs. Cellular packet networks operate over a wide area with a reach of several kilometers, and require elaborate and expensive infrastructure. In comparison, WiFi has a reach of only a couple of hundred meters in an outdoor environment, and signiﬁcantly shorter indoors. Hence, the usage scenarios are quite different. The relatively inexpensive devices operating in unlicensed spectrum are, however, very interesting in the context of challenged networking. Although infrastructure connectivity is available over the cellular systems, it might be more attractive to use technologies, such as WiFi and Bluetooth, to transfer bulk data for free, perhaps over multiple hops. Content sharing protocols that spread, e.g., pictures and music directly among mobile devices in peer-to- peer mode, might be a reality already within a couple of years. Perhaps the biggest obstacle is the resistance from major content providers, which are un- sure about how to cope with digital rights management (DRM). Most traditional wireless networks operate using a central coordinator, called a base station or an access point. The base station is part of a wireless infrastructure, which is usually deployed by a network operator, e.g., a cellular provider, or as part of a company, university, or home network. This infrastructure typically provides wireless edge access to client hosts that want to access an internal Local Area Network (LAN), or the Internet. The clients thus access the Internet over a wireless link, which connects them with the base station. With WiFi technology, these networks are often called Wireless LANs (WLANs). WiFi also allows direct peer-to-peer communication between wireless hosts by operating in so called ad hoc mode, without involving base stations or any type of infrastructure. Such networks are therefore called ad hoc networks. The spread of wireless link technologies has diversiﬁed the number of ways that computing devices can interact and exchange data. It also allows an un- precedented level of mobility. As these technologies evolve, they become smaller and less expensive, allowing them to be integrated in very small devices, such as sensors and RFID1 tags. This opens up a new era of communication with many opportunities for innovation. However, with the broad spectrum of technologies and device capabilities also comes many challenges. The Internet protocols struggle in mobile and wireless environments. They were not initially designed to operate over lossy wireless links, with large variances in delay. They furthermore do not cope well with intermittent connectivity, and long periods of disconnection. Thus, the unshackling of the Internet has spawned lots of research in how to improve the Internet protocols, but also in how to construct radically new ones.

1Radio frequency identiﬁcation.

28 3.2 Radio Frequency Communication Most wireless communication uses some kind of radio frequency (RF) technology. In this thesis, all wireless communication is assumed to be RF based. Other wireless technologies include infrared light (IR), laser light, and ultra sound. In RF communication, senders and receivers are often referred to as terminals. The communication is unguided, as the energy spreads freely through the air. In contrast, wired communication is guided because the energy is di- rected by the wire. Figure 3.1 illustrates the differences between unguided and guided transmissions. There are a number of salient properties inherent in RF communication that are often difﬁcult to deal with. Entire textbooks and countless research papers are dedicated to these issues. Here we only brieﬂy mention the most important aspects of RF communication, which are necessary to understand the rest of the thesis.

Figure 3.1: Signal propagation in unguided radio transmission versus guided wire transmission.

3.2.1 Signal Propagation Effects As Figure 3.1 shows, RF transmissions are usually omnidirectional, i.e., they spread the energy equally in all directions. Directional antennas may be used to focus the energy in speciﬁc directions, hence increasing the reach of the transmission. Here, we assume that omnidirectional antennas are used. If there are no obstacles, the energy spreads equally over a sphere surface that grows with the distance from the sender. The surface area s of this ever-increasing sphere hence grows with the distance d, according to the formula for a sphere’s surface s = 4πd2. The transmitted signal experiences free space loss, which is inverse proportional to the growth of the surface. The power of the received 2 signal at a receiver Pr is hence proportional to 1/d . Unless the signal travels in a vacuum, it experiences path loss –orattenuation – as it propagates through the air or other medium. This means that it looses energy to the medium, or obstacles in the way.

29 Obstacles in the path of the propagating signal can cause a number of different effects. On one hand, the signal can be completely blocked (or shad- owed) by large obstacles. On the other hand, reflections can cause a signal to go around obstacles bouncing off, e.g., a wall, similarly to how light is reflected in mirrors. Each time a signal is reflected it also becomes weaker, because some of the energy is absorbed by the reflecting material. Other effects include scattering, which splits the signal in several weaker copies, and diffraction and refraction, which have similar results on the signal. All these effects may happen at the same time, and can lead to severe degradation in the signal quality. Multi-path propagation is an impairment that is caused by the above mentioned effects and is illustrated in Figure 3.2. It means that the one receives many time-delayed copies of the same signal, all scrambled together. Powerful digital signal processors (DSPs) are used to recover the original signal.

Figure 3.2: Multi-path propagation of a radio frequency signal.

3.2.2 Signal Propagation Ranges By now, it should be evident that RF signals are not always received clearly, but that they vary a lot in quality – even when the sender and the receiver are both stationary. Adding mobility worsens the situation further. Figure 3.3 shows how the signal and noise power can vary with distance between two terminals. The measurement was performed in an indoor corridor, in a typical ofﬁce environment. One terminal was moving away from another at constant speed, while data packets where sent at a constant rate in order to sample the received signal levels. The distance is approximate. The ﬁgure illustrates that the variance in the received signal is quite large, even within the same distances (∼10 dB).

30 -30 signal noise -40

-50

-60 dBm -70

-80

-90

-100 0 5 10 15 20 25 30 35 40 Distance (m)

Figure 3.3: Received signal and noise level in an indoor corridor. dBm is the power ratio where 0 dBm equals one milliwatt. Every 3 dB increase doubles the power, whilst it is halved with every -3 dB decrease. -30 dBm equals 1 microwatt.

It is important to realize that when the signal-to-noise ratio (SNR) is very low, and the signal cannot be decoded by the receiver, it can still cause interference by adding to the background noise. As shown in Figure 3.3, this occurs at an approximate distance of 35-40 meters, for this example. The conclusion is that, within the interference range, an interfering signal can make another, otherwise clear signal, impossible to decode. If the interfering node is close enough to the sender, the interfering signal might be discernable from the background noise and hence detectable. Within this detection range, the receiver can take measure to, e.g., avoid transmitting at the same time. Fig- ure 3.4 shows the logical ranges of signal propagation. The free space loss formula in the previous section tells us that the power of a signal decides the range of the transmission. Increased power naturally means longer transmission range. But as the power is inverse square proportional to the distance, the gain in range per Watt quickly diminishes at larger distances. There is hence a threshold – that may vary with the application – where increasing the power does not make sense anymore. An alternative is to relay information over several hops. Under the right conditions the combined power usage of multiple hops can be lower than an equivalent range single hop.

31 interference

detection

transmission

distance

Figure 3.4: Ranges of signal propagation.

The bit rate of transmissions is another variable that affects the range. A higher data rate means that more bits are squeezed into the same time frame. Thus, the power output during that time frame is spread over more bits and each bit therefore has a smaller amount of energy. The result is that higher rate transmissions have shorter ranges than lower rate transmissions.

3.2.3 Shared Medium Transmission When terminals simultaneously act as both senders and receivers over a shared medium like the Ether, they need to coordinate their access to the medium. The “speakers” in the shared broadcast medium must take turns to speak. This is very much like coordinating the conversations of people in a crowded room. ALOHANET pioneered medium access control (MAC) protocols with the Aloha protocol. Today the descendants of this technology is used in both Ethernet-based LANs and WiFi-based Wireless LANs. The Ethernet LAN technology (IEEE 802.3), developed for shared cable transmissions, ﬁrst introduced multiple access with collision detection (CS- MA/CD). The basic CSMA scheme ﬁrst senses the medium, and then starts sending in case it is idle. The collision detection part (CD) allows Ethernet to detect collisions during ongoing transmissions, by sensing noise bursts. This detection allows quick aborts that save time and bandwidth. However, the detection can only be guaranteed if there is a maximum cable length, which gives an upper bound on how long a signal, or burst, needs to propagate in the wire. Stations thus know how long they need to wait, in order to determine whether a collision occurred or not. When dealing with the wireless Ether, it is not possible to set an upper bound on propagation, and guaranteed collision detection is hence impossible. Collision avoidance (CA) is instead used to reduce the likelihood of collisions. Terminals that sense a busy medium choose a random back-off interval,

32 reducing the chance of them trying to access the medium at the same time. This MAC scheme is hence called CSMA/CA, and is the underlying technique in the distributed coordination function (DCF) MAC protocol in WiFi (IEEE 802.11). As collisions still occur with CSMA/CA – sometimes without terminals noticing it – DCF also employs positive acknowledgements (ACKs) for each successfully received data frame. If an ACK is not received within a certain period of time, the failed frame is retransmitted. The ACKs are necessary because bit errors in wireless transmissions are much more common than in wired transmissions, since collision detection is not guaranteed. Although all transmissions are broadcasted in shared medium transmissions, there is a distinction made between unicast frames that are addressed to speciﬁc receivers, and broadcast frames that are addressed for everyone. Mul- ticast frames are similar to broadcast frames but are addressed for a subset of all terminals. The terminals that overhear a unicast frame not addressed for them, simply ignore it. ACKs are only sent for unicast frames because broadcast frames would be ACKed by everyone, causing unnecessary overhead and potential collisions. There are hence no reliability guarantees for broadcast or multicast frames.

Figure 3.5: Conﬁguration with “hidden terminals”.

The different ranges of wireless signal propagation, as discussed in the previous section, cause a number of adverse effects when terminals simultaneously try to access the medium. One of the most well known of these effects is the hidden terminal problem, which is illustrated in Figure 3.5. In this example, adjacent terminals are within each other’s transmission ranges, but terminals A and C are within interference range only. They are thus hidden to each other, and might try to transmit a message to terminal B simultaneously – hence causing a collision. CSMA/CA cannot detect hidden terminals, and therefore an additional mechanism is needed. Ahead of each transmission, a sending terminal transmits a request to send (RTS) message. When the destination terminal receive the RTS, it sends a clear to send (CTS) message, unless busy accessing the medium itself. Imagine that the requester in this case is terminal A in Figure 3.5, and that B sends the CTS. Terminal C over- hears the CTS, and then knows that the medium is busy for a period of time. C

33 therefore avoids transmitting during this period, which would otherwise cause a collision at terminal B. If terminal A never receives a CTS, it does not transmit either. DCF includes the RTS/CTS mechanism, but it is optional and can be turned off. In many cases, the reservation of the channel is less efﬁcient than just dealing with the collisions through retransmissions [45].

3.3 Wireless Link Technologies WiFi and Bluetooth are perhaps the two most disruptive wireless link technologies that have been introduced in recent years. They are both central to the work in this thesis, WiFi in particular. Below follow short overviews of these technologies.

3.3.1 WiFi - IEEE 802.11 As described previously, IEEE 802.11 – also called WiFi – is a CSMA/CA based standard that was ﬁrst standardized in 1997. Since then, numerous improvements to the basic protocol have been introduced that provide increased speed, mainly through new modulation schemes. WiFi radios are today found in almost every laptop sold, and is the technology in wireless home routers. Table 3.1 shows the development of WiFi over the years.

Protocol Year Frequency Max data rate 802.11 1997 2.4 GHz 2 Mbit/s 802.11a 1999 5 GHz 54 Mbit/s 802.11b 1999 2.4 GHz 11 Mbit/s 802.11g 2003 2.4 GHz 54 Mbit/s 802.11n 2009 2.4/5 GHz 248 Mbit/s

Table 3.1: Protocol variants of the IEEE 802.11 standard.

The protocol variants that operate in the same frequency bands (e.g., original/b/g/n) aim to be backward compatible. In order to achieve this, the standard deﬁnes two rate sets; the basic rate set and the data rate set. All MAC headers and preambles are sent at a basic rate of 1 Mbit/s for backwards compatibility. Broadcast frames are sent at a rate from the basic rate set, whilst unicast frames are sent at a rate from the data rate set. The basic rate set usually contains fewer, and lower, rates than the data rate set. Therefore, broadcast frames are often sent at a lower rate than unicast frames, e.g., 2 Mbit/s. Thus, broadcast frames at lower rates reach further than unicast frames at higher rates (see Section 3.2.2). This can be a source of problems if, for instance, broadcast beacons are used to probe for neighboring terminals. The detection range of such beacons does not match the data transmission range

34 at, e.g., higher than 2 Mbit/s. Therefore, communication gray zones appear in which detected neighbors cannot be communicated with. We describe this phenomenon in detail in Paper II. As previously mentioned, IEEE 802.11 uses the CSMA/CA medium access protocol and usually operates in the DCF mode. There is also a point coordination function (PCF) mode, which is built on-top of DCF. It allows a base station to fully coordinate the transmissions among clients by polling them in round-robin order. PCF can therefore be more fair, and can give better quality of service guarantees (QoS). PCF cannot be used in ad hoc mode, and therefore it is not of interest to this work. A hybrid mode (HCF), combines elements of DCF and PCF but is not supported as well as DCF, and is therefore also of no interest.

3.3.2 Bluetooth - IEEE 802.15.1 Bluetooth is a low-power open standard for Personal Area Networks (PANs) intended to replace cables. It is cheap, power efﬁcient, and has become a commodity technology in a large number of portable devices, such as mobile phones, laptops, PDAs. The technology does not involve infrastructure, and is entirely based on peer-to-peer communication. Here we give a short technical overview of Bluetooth, which aims to give an understanding of the functions and limitations of the technology. The Bluetooth physical layer operates in the 2.4 GHz unlicensed ISM frequency band, and employs a frequency hopping spread spectrum transceiver to combat interference and fading. The typical range is 10 meters, but it may vary depending on the power mode used. During operation, the radio channel is shared by a group of devices that are synchronized to a common clock and frequency hopping pattern. The device that provides the synchronization reference is called a master and the other devices are slaves. Synchronized devices form a piconet. Physical links are formed between any two devices that transmit packets between each other. The physical links are only formed between masters and slaves and not between slaves directly. This means that devices must pair before they can communicate, adding a slight overhead. A special physical channel is used for devices to discover each other. A device becomes discoverable by entering the inquiry substate where it can respond to inquiry requests. The inquiry scan substate is used to discover other devices. The discovering device iterates (hops) through all physical channel frequencies in a pseudo-random fashion. For each frequency, it sends an inquiry request and listens for re- sponses. From the discovery operation follows that a Bluetooth device cannot itself be discoverable while it scans for other devices. In [33], we describe the implications of this simplex operation when using Bluetooth to measure contact patterns among people. In Paper VII, we use the collected traces to simulate content-disseminations in opportunistic networks.

35 3.3.3 Other Wireless Link Technologies There are a number of new wireless link technologies that are related, but not central, to the work of this thesis. ZigBee (IEEE 802.15.4) is a short range, low power, wireless technology aimed mainly towards sensor networks. WiMAX (IEEE 802.16) is a wide area wireless link standard that competes with high speed wireless 3G, upcoming 3GPP long term evolution (LTE) technologies, and 4G cellular networks. These technologies will certainly have an impact on challenged networking in the future, but they are not relevant here and now.

3.4 Challenges in Tetherless Communication The “unshackling of the Internet” is indisputably the most important development that has facilitated the emergence of challenged networks. The ability of commodity wireless technologies to operate in ad hoc or peer-to-peer mode, without infrastructure or base stations, is also an important development. These two developments combined, open up new areas of innovation in tetherless computing, but they also constitute tremendous challenges to the traditional communication models discussed in Chapter 2. In the following sections, we discuss the most relevant aspects of these challenges. We take an historical approach, starting with the impact of wireless communication, and then we introduce new areas of challenged networking as they have developed.

3.4.1 Wireless Factors Whilst wired transmission mediums exhibit low delays and low error rates with small variances, wireless transmission through the ether in general has high delays and high error rates with large variances. The challenges of adapt- ing communication to high delays and high error rates are certainly major issues, but dealing with variance is often a much harder problem. Losses can be handled both at the link layer and in higher layer protocols such as TCP, while delay in general cannot be controlled. Applications therefore must adapt to the circumstances. Variance is something that is hard to handle in a single layer or protocol. Communication protocols are often tuned to certain conditions and when those change frequently and unpredictably, the performance can degrade quickly – in some cases to the point where the protocol fails. Variance effects from wireless communication are central in many aspects of challenged networking. They will therefore be discussed not only here, but whenever relevant. TCP is perhaps the Internet protocol that is most affected by wireless factors [46, 47], and the body of research on how to deal with them is very large. The TCP problem is that wireless packet loss is interpreted as a sign of a con- gested network, with packet losses at routers due to full buffers. TCP therefore

36 invokes congestion control, thus lowering the data rate, instead of taking specific action to handle the losses. Solutions include those that introduce mid- dlewares into the network that, for example, cache and resend packets at the link layer [48], or even those that split the TCP connection into one wireless and one wired part [49, 50]. Both are violations of the end-to-end principle, and hence the argument for edge intelligence. Other TCP modifications for wireless scenarios modify the end hosts only [51]. Both link layer and split connection approaches are in general unsuitable for challenged networks since, in networks without infrastructure, there is no predetermined place where middleboxes can be put. End-host modifications are viable options, but they create another challenge to maintain compatibility with existing TCP implementations. Others argue that TCP performance is hampered by the layered Internet architecture. The principle of layer independence forces TCP to take actions based solely on information inferred in its own layer. By braking the layer principle, cross-layer feedback mechanisms can inform TCP about, for example, losses or low quality links in the link layer, or temporary routing failures in the network layer. Many cross-layer approaches have been proposed, but determined layer advocates disapprove of every thought that violates the layering model. But the fact is that cross-layer feedback allows TCP to act quickly and more appropriately. Another problem faced by feedback mechanisms is that strict layering by definition makes it hard to implement them. Lower layers are usually implemented in hardware or closed device drivers. Passing information up the stack can be hard since it is against the stack’s architectural design. This thesis does not specifically deal with TCP related issues, but wireless related issues, along with issues in other layers, ultimately have implications also for TCP. Therefore, addressing wireless related problems in any layer is important for the overall functionality of systems in general, and for TCP in particular.

3.4.2 Mobility Mobility first became an issue in the Internet when portable laptop computers started to appear. People were no longer restricted to one physical location for their computing needs and could instead connect their laptops to the Internet wherever they travelled. This type of mobility was not anticipated when the Internet was first designed. When a computer changes network, it must also change IP address and hence identity, as discussed in Section 2.4. This has two implications. First, a computer is not reachable using its normal address assigned in its “home network”. Second, a change in IP address disrupts any active TCP connections or other communication sessions. Initially, the first issue was not a big problem, because mobile users rarely needed to be contacted through their computers. Today, however, the tradi-

37 tional client/server model of the Internet is being complemented with peer-to- peer models that often require host connectivity in both directions. In challenged network scenarios the peer-to-peer model is the often preferred as every mobile device is, in many aspects, assumed equally capable. The second issue is a major problem also in the Internet. With the advent of wireless communication it has become feasible to also maintain connectivity while moving in-between networks. In theory, communication sessions can be kept alive as long as there is support for roaming and hand-offs between networks. Early proposals for roaming and hand-offs suggested indirection mechanisms [52, 53], which later evolved into the Mobile IP standard [54]. The principles of Mobile IP is illustrated in Figure 3.6. It relies on the deployment of home agents and foreign agents in the network that allow data trafﬁc to be redirected to a mobile host’s current location. Whenever a mobile host visits a foreign network, it registers its location with its home agent. Any correspondent hosts that want to reach the mobile host can contact it via its home agent. The mobile host can also maintain its connections with correspondent hosts whilst moving between networks. The data is tunneled via the home agent using, e.g., IP-in-IP encapsulation, and this causes the triangular routing illustrated in Figure 3.6. More recent proposals to Internet mobility suggest indirection overlays [55, 56], which do not require agents to be deployed at predetermined locations.

Figure 3.6: Mobile IP routing between a mobile host and a correspondent host.

It is important to recognize that Mobile IP is an afterthought, and a response to the ﬁrst mobility scenarios facing the Internet. There is no native mobility support in the Internet architecture. Thus, it was natural for researchers to think about how mobile computers could organize themselves in networks that supported mobility, and that did not involve any infrastructure. The idea

38 was that mobile nodes would be able to network themselves ad hoc, and forward packets on behalf of each other, essentially forming their own Internet. However, in order to build forwarding tables they need routing as well. The existing intra-domain routing protocols, such as RIP and OSPF, are too slow and involve too much overhead to effectively route between mobile hosts. By modifying RIP, researchers came up with a routing protocol adapted to dynamic and self-configuring networks [57]. The protocol, called Destination Sequenced Distance Vector (DSDV) [57], was the first routing protocol for mobile ad hoc networks (MANETs). Further developments in the area later led to the forming of a MANET working group within the IETF [23]. We conclude that mobility is not something that is native to the Internet. There are many implicit assumptions, in terms of addressing, structure and stationarity of the network, which are challenged when mobility is added to the mix. How to deal with many of the effects of mobility is a central issue in this thesis. In Paper III, we look at issues of connecting MANETs to the Inter- net. The solution we propose allows micro mobility within the same MANET using a local routing protocol, while macro mobility between MANETs is facilitated by Mobile IP (Figure 3.6). Making these protocols work together efficiently and seamlessly is a major challenge.

3.4.3 Multi-hop Communication Multi-hop wireless communication has many potential usages, both as stan- dalone networks, but also to extend the reach of infrastructure, e.g., base stations. We have already discussed the fundamental challenges of wireless communication, involving packet loss, high delay and, e.g., effects from hidden- terminals. These are factors that play a signiﬁcant role in any wireless communication scenario. However, as communication is extended to multiple hops, several new wireless issues come into play. The difference between single- hop infrastructure networks and multi-hop ad hoc wireless networks is illustrated in Figure 3.7. Single hop communication, via e.g., a base-station, in most cases occurs within the same collision and interference domain. In the multi-hop case, the roles of collision and interference become more complex as adjacent links, and even links further separated, affect each other during transmission.

Figure 3.7: Single hop Wireless LAN versus multi-hop wireless ad hoc network.

39 While single hop wireless LANs and cellular networks are common today, it was multi-hop wireless packet radio networks that first attracted the attention of researchers when ALOHANET appeared. A number of early works on multi-hop packet radio networks look at issues of broadcast style routing and relaying [58, 59, 60] in stationary terrestrial networks. Wireless radio towers were networked in order to achieve robustness in primarily military scenarios, more or less forming a “wireless Internet”. These networks were the forerunners of today’s MANETs and derivatives. The major differences to the tower approaches are that MANETs, and challenged networks in general, have emerged from the availability of commodity wireless technologies, and allow a new level of mobility. This means that there is often, by default, no single entity that controls the nodes in the network, as is the case in the military scenario. Even if a network is built and controlled entirely by one operator, there is no guarantee that external nodes do not interfere. In the military scenario, interference and jamming is an active threat that networks are built to combat using redundancy. In the challenged network scenario such things may be an unintentional reality. Multi-hop wireless communication has many subtle performance implications, which are not visible in single hop networks. Holland and Vaidya show in [61] the effects of multi-hopping on TCP throughput. Since adjacent links in a multi-hop configuration interfere with each other, they have to share the wireless channel. A two-hop configuration hence effectively halves the available bandwidth. Links that are not adjacent, but are still within interference range (see Section 3.2.2 and Figure 3.4), also affect links further down a multi- hop path, reducing the link bandwidth even further. A number of TCP fairness issues have also been revealed [62]. In [63], we contribute to the understanding of transport layer issues by studying UDP performance in the presence of TCP flows in multi-hop ad hoc networks 2. An interesting recent development is the proliferation of commodity wireless access routers in homes. This has spawned research into so called mesh networks, which are essentially non-mobile wireless ad hoc networks that very much resemble the early multi-hop packet radio networks. Again, the major differences are the use of commodity hardware and the unplanned deployment of routers. Anyone should be able integrate, e.g., their home router into the mesh network through auto-configuration means. An urban rooftop mesh network is depicted in Figure 3.8. In contrast to MANETs, stationary mesh routers can self-optimize to a degree not possible in mobile scenarios. However, the unplanned structure of the meshes can still lead to severe interference. The research in the area focuses on how to deal with router failures and how to develop self-optimizing routing strategies. Several experimental mesh networks have been created for research purposes, MIT’s Roofnet [64] being the most well known one. Oth-

2This paper is not included in the thesis, but is still part of the work underlying it.

40 Figure 3.8: Rooftop mounted home routers form a mesh network in an urban area. ers include UCSB’s MeshNet [65], Microsoft research’s ofﬁce mesh network testbed [66], and the upcoming 802.11s standard [67] (draft status). Notable research efforts include studies of wireless link level interferences and performance effects from multi-hopping [68, 69]. Many improvements suggest new routing metrics [70, 71], automatic channel assignment strategies [72], cross-layer mechanisms [73], and opportunistic route optimizations [74].

3.4.4 Routing Routing is essential to the Internet and remains so also for challenged networks. Whilst routing in mesh networks is similar to routing in the Internet, other challenged networks may have constraints that require new, or highly modiﬁed, routing strategies. The main issue in mesh networks is how to adapt to a wireless and shared channel link layer, whilst in MANETs the high mobility causes constantly changing topologies that the routing must handle. In sensor networks, the main limitations are limited processing power and strict energy requirements. This has the consequence that the routing must do load balancing to preserve energy, and that large amounts of routing information should be avoided. Routing protocols supporting high levels of mobility have mainly been developed within the MANET area. A simple way to handle mobility, and hence dynamic topologies, is to increase the rate of routing updates sent in the network. But for wireless networks, which in general have lower bandwidth and also more contention for the channel than wired networks, this is a crude approach that wastes precious bandwidth. Thus, there are two main goals of MANET routing: 1) supporting dynamic and mobile environments, and 2) reducing the overhead of routing updates. The approaches to achieve this are generally classiﬁed as proactive, reactive or hybrid. Countless schemes have been proposed and developed over the years, but the MANET working group of the IETF now aims to standardize

41 one proactive and one reactive protocol [23]. IETF MANET focuses on routing protocols that are IP compatible and hence should be able to interface with existing IP networks. This slightly limits the spectrum of innovation. Proactive routing protocols function very similarly to the ones in the Inter- net. Routers send control messages to each other at regular intervals, so that they can keep the routing configuration up-to-date with the current network topology. To support higher levels of mobility they send routing updates more often, and make sure they spread more quickly. To counter the increased overhead, they include optimizations that enable more efficient dissemination of control messages. A popular proactive routing protocol is the Optimized Link State Protocol (OLSR) [75]. Reactive protocols aim to reduce the overhead in more drastic ways by only sending routing updates when the topology actually changes. Routing is performed on-demand and routes are discovered only when needed by an application. One can hence say that reactive schemes incorporate a degree of late binding between routes and destinations. This drastically reduces the overhead of routing and saves power, but also adds delay in cases there are no routes available when communication is initiated. When there are no active data flows over a route, the route state times out and is deleted. Maintaining the active route state is somewhat challenging, because the monitoring of data flows usually requires cross-layer mechanisms. As a result, reactive schemes are more difficult to implement in traditional networking stacks (e.g., TCP/IP), compared to proactive ones. Popular reactive protocols include Ad hoc On- demand Distance Vector (AODV) [22], Dynamic Source Routing (DSR) [17] and DYnamic MANET On-demand (DYMO) routing [76]. Hybrid protocols adopt principles of both strategies, and may mix proactive and reactive strategies in different parts of the network. An important difference between the proactive and reactive schemes is that proactive ones always have a complete view of the network, whilst reactive protocols at any time instant only “know” about the nodes that are actively communicated with. This makes it hard for reactive schemes to interface with other networks, because the schemes cannot easily discern which destinations are local to the ad hoc network, and which are located in another network – reachable only via a gateway. The problem could be solved by careful IP addressing, for example, by assigning a specific subnet prefix to each MANET. Then nodes can identify the location of other nodes from the IP address prefixes. However, this requires sophisticated distributed auto-configuration schemes, and has the downsides that mobile nodes cannot use their home network addresses and move seamlessly between different MANETs. Nodes that do not conform to the address scheme are hence excluded. Enforcing such an address scheme is furthermore very hard, since there is no central coordinator. We discuss network interconnection and gateway issues in Paper III, where we propose a way to solve these issues for the reactive AODV protocol.

42 3.4.5 Sensing Challenged networks often need accurate and timely sensing of their sur- roundings, in order to operate efficiently. This includes learning about the neighborhood (adjacent nodes), inferring the quality of links and paths in order to choose the best ones, and to make the most out of transient and time- limited node contacts. MANET routing is one example area where sensing is crucial for efficient operation. A challenge for sensing schemes is that the act of sensing causes changes in the phenomenon sensed. In more general terms, this is in science referred to as the observer effect. In networking, sensing often involves sending out packet probes in the network that may themselves contribute to interference. Many simultaneously probing devices may lead to inaccurate sensing that does not represent the normal network state. These effects are exacerbated by a shared wireless channel. There are two main reasons why sensing is important for challenged networks. First, wireless links are not binary in connectivity like wired links, as is illustrated in Figure 3.3. This means that sensing is needed to, e.g., filter out links that are of too low quality for effective usage. Second, in dynamic environments – where nodes change location frequently – it is necessary to continuously track changes in the network topology. The more dynamic the environment, the more accurate and frequent sensing is needed. However, the observer effect opposes this, because increased probing also increases interference. If there are no alternatives, such as passive sensing approaches, the effects can be devastating for performance. Thus, transient properties stress accurate and interference free sensing, whilst dynamic mobile environments stress frequent sensing. A considerable amount of work in this thesis focuses on sensing. In Pa- per II, we show how communication gray zones lead to inaccurate neighborhood sensing in MANETs, and then we study ways to combat these zones. In Paper VI, we report on sensing anomalies that seriously degrade the performance of popular MANET routing protocols. In [33], we study how limitations in the Bluetooth inquiry scan mechanism effect the ability to periodically sense the neighborhood. In [77], Liberatore et al. show how dual radios can help improve the sensing performance of Bluetooth. Sensing problems are far more severe in the real world than in simulations. Therefore, experimental “reality checks” and early prototyping are important ingredients in the protocol design process. Based on our results mentioned above, we argue that the initial approaches to MANET routing severely underestimated the importance of sensing. To improve the routing performance it is common to rely on neighborhood sensing through link layer feedback. This is easy to implement in simulation, but a lot more difficult in the real networking stacks. As a result, there is a big gap between the real world and simulated performances, which remains to this day, as we show in Paper VI.

43 Paper II was one of the ﬁrst papers identifying sensing issues in wireless ad hoc networks. [78] and [79] later produced similar results. More recent work use both active and passive sensing mechanisms to select optimal paths [70, 80].

3.4.6 Disruptions and Disconnections In mobile networks, such as MANETs, it is inevitable that nodes sometimes become disconnected from the rest of the network [81, 82]. In some circumstances a network might split into smaller networks that have intermittent connectivity with each other. The borders that define a network become blurred as networks may split and join in arbitrary ways. Other networks may not log- ically split, but may have to endure regular disruptions, for example, when connectivity is provided by a non-geostationary satellite orbiting the earth. Disruptions and disconnections pose considerable challenges to networking. The severity of disruptions and disconnections also determines the approaches taken to address them. The most challenging scenario is a completely disconnected one, only having intermittent pair-wise connectivity between mobile nodes as they move around. Kevin Fall formulated in his seminal paper [1], a delay tolerant network (DTN) architecture, which is designed to cope with these scenarios. It is a generalization of the models developed for deep space interplanetary communication [4]. The main concept of DTN is that data is carried between areas of connectivity, passing it on from node to node as they meet. This store- carry-forward approach we introduced in Chapter 1, and here we develop the discussion around it. We use the general term message for data that roughly corresponds to a complete piece of application data, e.g., an MP3 file, document, or text message. The size of a message can hence be arbitrary, but is in general large compared to IP packets. When dealing with large messages, it can be faster and cheaper to physically carry them in a storage device to the destination, rather than transmitting them, e.g., over a slow telephone link or a moderate speed satellite link. The benefits can be even greater if small messages are bundled together, thus reducing the overhead once forwarding opportunities occur. DTN bundles messages and delivers them through packet-switching – or more aptly named bundle- switching. In [83], Jain et al. formulate the DTN routing problem, and study the amount of knowledge needed to schedule bundle delivery with respect to delay and bandwidth consumption. Figure 3.9 depicts a typical DTN scenario, in which a bundle can be sched- uled for delivery either over a high bandwidth satellite link (present only during parts of the day), or a postal service man (which is slow, but can carry lots of information), or via an always present low bandwidth phone line. Depend- ing on the size of the bundle and the time of day, the delay optimal way to deliver the information can be different. In this case, a priori knowledge of

44 Figure 3.9: A delay tolerant network scenario with three scheduling possibilities for bundles. satellite orbits, the postman’s schedule, and other predictable delivery methods aid efﬁcient scheduling. Delay tolerant networks are envisaged to provide basic communication services to rural areas in undeveloped parts of the world, where communication infrastructure is sparsely deployed or not at all. KioskNet [84] is one example of how DTN can provide low-cost communication in rural areas. The Saami network connectivity (SNC) project [85] is another example, which provides communication in inaccessible areas. Similarly to MANETs, the DTN area has spawned many related network scenarios that are variations of the basic delay tolerant one. In [86], Zaho et al. study message ferrying, a DTN variation where special ferry nodes carry information between distinct points according to regular schedules. The contribution of message ferrying is how to exploit non- randomness by guiding ferries so that they optimize delivery. Wang et al. proposed Postmanet [87], which aims to turn the postal system into a generic digital delivery mechanism. The authors note that huge amounts of information can be bundled together on e.g., a DVD or BlueRay disc and then physically carried by the postal service to its destination. No expensive mobile communication devices need to be involved, and instead messages are carried on storage media in-between digital drop-off stations. In [88], Burgess et al. study how buses can aid delivery of messages in their DieselNet. Pocket switched networks [89] represent a DTN style networking scenario that focuses on information sharing between digital consumer devices, such as digital media players, smartphones, digital cameras, and other portable devices that people carry in their pockets. Figure 3.10 illustrates this scenario. People share messages directly between devices in their pockets as they meet. The contacts are not as predictable as in the general DTN case, and are therefore opportunistic, as contacts occur unpredictably. There may, however, be patterns in the contacts that can be exploited. For instance, people often move

45 between the same locations and have speciﬁc social interaction patterns. His- torical information, such as how and where people meet, can be used to es- timate the probability of meeting someone at a certain point in time in the future.

Figure 3.10: Pocket switched networks involve consumer devices and incorporate social aspects and opportunistic communication.

Countless routing and forwarding schemes have been devised to efficiently do store-carry-forward. The challenge is to decide which nodes are the best message carriers in the absence of end-to-end paths. Although the goal is to deliver messages from a source to one or more destinations, the approaches taken differ significantly from classical routing and forwarding schemes used in the Internet, MANETs and derivatives. Due to the intermittent connectivity there is no routing in the traditional sense (i.e., construction of end-to-end forwarding paths before the actual forwarding takes place). Instead, nodes decide whom to give certain messages to as they meet. This in situ3 decision is based either on a priori information as discussed above, or on local utility calculations on current and historical information collected during encounters, or both. Whom to forward messages to is hence often a late binding decision. The brute force approach to this problem floods the messages to every node encountered. In DTN terms, this is called epidemic forwarding [90]. A message spreads in the network according to an infectious decease, “infecting” those nodes that do not already carry the message. In an ideal situation – where resources in terms of buffer space, bandwidth and battery power are unlimited – an epidemic scheme is optimal because it always finds the “best” path simply by exhausting the available options. In reality, however, the obvious downside is that the overhead makes inefficient use of time-limited node contacts, buffer space, and battery power, due to the transmission of messages that in the end do not contribute to delivery. The epidemic forwarding scheme is information agnostic and memory-less. Variations of the epidemic approach aim to limit flooding and include randomized single/multiple copy schemes [91, 92]. Utility based schemes can be probabilistic (Prophet) [93], be based on message ferrying [86], use link state based algorithms [83], use priorities (MaxProp) [94], be resource aware

3A Latin phrase meaning in the place.

46 (RAPID) [88], use context and sociological information [95, 96, 97], and include countless variations of the aforementioned approaches. Forwarding schemes that rely solely on a priori information are not as well studied – because the information is difficult to acquire or synthetically generate. One of the few studies carried out is [98], where Hui et al. show how messages can be efficiently forwarded based only on labels of affiliation. The results are based on contact traces collected at conferences using Intel sensor motes, and exploited the fact that people that have the same affiliation often have contact with each other. By labeling a message, with the affiliation of the destination, it could be forwarded efficiently by only involving the people with the same affiliation. In comparison, the utility based schemes above try to infer such communities from continuous measurements and information gathering. In Paper VII, we contribute to the understanding of a priori information based forwarding by studying interest-based content dissemination. This is a more general approach than the one taken by Hui et al., as messages are disseminated to groups of people according to the match between their interests and the metadata of the messages’ content. This is a variation of publish/subscribe, which we discuss further in the next section.

3.4.7 Naming and Addressing As we consider more and more challenging network scenarios, it is evident that less and less of the Internet’s model of communication apply. Packet switching we have identified as an invariant [99] in all types of challenged networks. Other Internet traits, like its end-to-end principle, hierarchical structure, and layered design, are in discord with challenged networking. Neverthe- less, these traits in many ways still serve the Internet extremely well, and have allowed it to sustain continued growth. Naming and addressing, however, is an area where increased pressure is asserted even from within the Internet itself. Before we discuss why this is the case, we consider two important design issues of naming and addressing. The first is the definition of namespaces that allow entities (e.g., hosts) to be named and addressed in expressive ways. The second is how to determine bindings between the entities in different namespaces without restricting flexibility. But what do expressive and flexible mean in this context? The Internet’s layered namespaces, for example, limits the flexibility of the types of bindings that can be done between, e.g., names and addresses in the Internet. As layers have solidified, innovation has become rare and constrained, and therefore it is difficult to add new core functionality. For instance, the Host Identity Protocol (HIP) [41], is a standardization effort within the IETF that aims to solve the mobility problem in the Internet by adding a new identity namespace. This allows the separation of the identifier and locator roles of the IP address. However, a new namespace requires that a new layer

47 is added, and that has shown tremendously difficult – potentially every computer in the Internet needs to be updated, as well as most applications. Thus, the Internet’s layered namespaces are not very flexible. However, new namespaces can be added on top of the existing layers in the form of overlays (see for example [55, 100, 101]). But the drawback of these approaches are that they inflate the number of namespaces, and increase application incompatibilities when applications are designed specifically for the overlays. Further, the overlays still depend on the functionality of lower layer namespaces, which possibly restrict the efficiency. The Internet’s inability to accommodate changes is also evident by the increased pressure from content oriented communication. The Web, file-sharing applications, and video streaming sites, such as YouTube, have become hugely popular. Therefore, the bulk Internet traffic no longer consists of end-point conversations, but instead of content-oriented disseminations [102]. The problem, in this context, is that the Internet’s end-point oriented naming and addressing scheme does not properly support content addressing. Consider, for example, a number of users on a local area network (LAN), which access the same popular news website. With the Internet’s conversational model, routers forward one copy per user of the exact same content. In a disseminational model, the routers forward one copy that is split and forwarded to the users once the LAN gateway is reached, thus achieving higher efficiency. This example is intentionally simplified. In reality there exist some means to improve the situation, such as deploying caching proxys in the LAN. However, the main reason why the Internet still copes so well, is that the bandwidth in the network core is over-provisioned. This over-provisioning is a luxury that most challenged networks do not enjoy. Hence, they need to consider flexible addressing from the start, which can account for content oriented communication. In conclusion, it is clear that the Internet’s approach to naming and addressing – although efficient for conversations – is neither very expressive, nor flexible. How then, do alternatives look like? If we examine content addressing schemes, we can make the following broad classification of the available schemes: hash based, name based and metadata based ones. Cryptographic hash functions, such as SHA-1 [103], provide exact means to address content. Hashing generates an identifier from a piece of content, which is guaranteed (in a probabilistic sense) to be unique. The hash can then be used to identify and address content using distributed hash tables (DHTs) (see for example [100]). The content is usually resolved through route-by-key, which uses longest prefix matching on the hash in order to route to the location of the content. Name based content addressing generally uses flat identifier names, or more mnemonic URI-based names. Two resolution schemes dominate the name based approaches; lookup-by-name in a distributed database, or route-by-name to closest content copy. The DNS system, augmented with

48 content services such as AKAMAI4, is an example of the former approach. TRIAD [104] is an example of a network architecture that uses the latter one. Metadata addressing is not exact as hash or name based schemes are. Perhaps the best example of how metadata can be used to address content, is found in the area of operating systems. Semantic file systems [105], and metadata based search services, such as Google Desktop [106], have been developed to address and find content locally on computers. These services can be complemented with temporal and contextual information to enhance searches [107]. Metadata provides a very flexible alternative to ordinary file names and paths. In this case, resolution is done through searching on a local database. Existing metadata resolution systems usually resemble the ones used in name based naming and addressing. INS [108], is an example of a metadata based naming system that combines traditional DNS-style lookups with route-by-name resolution. A push-based scheme is used in so called publish-subscribe (pub/sub) systems [109]. This scheme use filters, in the form of regular expression or matching predicates [110], in order to push content to subscribers. We discuss pub/sub systems further in the next section. So, after having reviewed a number of alternative naming and addressing schemes, what do find in challenged networks? The DTN architecture [1], for example, uses names in the form of tuple identifiers consisting of a region name and an entity name. This scheme anticipates that DTN nodes move within geographical regions, which may be structured hierarchically. As bundles are forwarded, the entity part of the name tuple is bound once the bundle enters the destination region. DTN hence incorporates late binding resolution, in which only the portion of the name immediately needed for forwarding is used. Others advocate context aware addressing [96, 98]. These are similar to a priori schemes, but they in general combine a priori information with active measurements (the context), in order to select forwarding parties, as described in Section 3.4.6. In Paper VIII, we take a new approach to namespaces, resolution, and addressing in challenged networking. We recognize the central role of searching in today’s computing, and therefore explore ways to define networking primitives around search abstractions. These primitives apply to a unified metadata namespace, where they allow relationships between nodes and content to be constructed. This namespace thus spans both local file systems and networks, and allow novel ways to name and address content and nodes. This search based resolution is part of the Haggle architecture, which we further describe in the next section.

4www.akamai.com

49 3.4.8 Architectures At this point, we have mostly discussed how challenged networking can be ac- commodated in the existing Internet architecture. MANETs are, for example, adjusted to the existing Internet technologies in order to be compatible. As a result, MANETs have to cope with a number of awkward trade-offs, such as ambiguous interpretations of IP addresses, cross-layer designs, and trouble- some interconnections with IP networks. At some point in the network evolution process it is necessary to think about entirely new architectures, which are specifically designed for challenging scenarios. The DTN architecture provides complementary transport services not supported by “legacy” architectures (i.e., the Internet). It is hence not a replacement for the Internet as it runs in an overlay on top of it. Other approaches at- tempt cleaner breaks with the Internet by ridding themselves of layering [111], by providing mobility and indirection support [55, 112], or by supporting network pluralism [13]. The common goal of most architectures is to provide abstractions. The architectures are usually designed bottom-up, starting with abstractions for the major objective, e.g., indirection, mobility, or pluralism. In Chapter 2, we described the inflexibility of the Internet’s sockets API, which is designed around TCP with the intent to provide conversation abstractions. Publish-subscribe (pub/sub) architectures [109] provide an alternative to the Internet’s conversational sockets API. Pub/sub embeds the disseminational model of communication instead of the conversational one, which the Internet embeds. It further incorporates late binding, asynchronous communication, and content addressing. The pub/sub paradigm consists of three components: publishers that generate content, subscribers that specify interest in content, and infrastructure that matches the subscriber interests with the published content. The infrastructure decouples the publishers from the subscribers, such that content matching is performed without them interacting – in both the spatial and temporal sense. Disseminations hence occur as content is matched. Pub/sub systems are divided into two main types, depending on whether the matching is topic based or content based. In the former approach, content is published in channels that aggregate the content according to topics of interest. The latter approach is more general and allows subscribers to express interests using filters in the form of predicates [110], or regular expressions. The pub/sub paradigm fits well with many challenged network scenarios, since it incorporates late binding and asynchronous operation. In [113], Lenders et al. use a topic based pub/sub approach for wireless podcasting in disconnected MANETs. However, most pub/sub systems today are overlays on top of the Internet’s conversational communication model [110, 114]. Recent research argue for a top-down design of network architectures [115, 116, 117]. The authors of these papers advocate data and content-oriented services, and all of the architectures incorporate significant amounts of pub/sub

50 features. They provide asynchronous, late binding APIs that address data or content instead of end-points. All of them are also designed for wide area internetworks. In Paper VIII, we present Haggle: a layerless search based network architecture for content-oriented and opportunistic communication. The typical challenged network scenario targeted by Haggle is pocket switched networks [89]. Like the related works discussed above, Haggle incorporates pub/sub elements in its design, but adapts them to the opportunistic nature of the scenario, by adding persistent and searchable data storage. A key contribution of the Hag- gle architecture is a generic metadata format for application data that enables “searching as a networking abstraction”. This abstraction provides means to match content to nodes using searching instead of ﬁlters (as used in the general pub/sub approach). This is accomplished by unifying local searching services5 and network data dissemination. Every dissemination in Haggle is preceded by a search operation on stored content that determines what to disseminate, and to whom. An important consequence of this search based resolution is that it produces ranked search results, and this makes it possible to disseminate content based on its rank. Such ordered dissemination is, in general, speciﬁc to the forwarding algorithm used. Haggle provides this as a general service in the architecture that can be used by any forwarding algorithm.

3.4.9 Evaluation and Experimentation Challenged networking is here to stay, and it must therefore be studied in methodological ways. Prototyping and experimental evaluations are two important elements of this study process. Unsurprisingly, these elements are themselves major research challenges. The main challenge with experimental evaluations (as we briefly discussed in the introduction) is the difficulty of achieving repeatability. Another challenge is that such experiments are facilitated by testbeds – which often include expensive and specific hardware in elaborate setups – that are difficult to re- construct at other research labs. This makes it hard for other researchers to reproduce the results. In this thesis we make contributions to address both of these issues, and this work is found in papers I, V and VI. We here only briefly discuss these contributions, as they are also explained in the papers and the summaries of papers. Simulations represent a convenient and powerful alternative to cumbersome and costly experiments, but many argue that they are too simplistic, especially those that use simple models of mobility and radios [119]. User mobility patterns and radio propagation are two of the most significant factors for predict- ing the performance of challenged networks. Therefore, simulation models must be selected with care, and with an understanding of their limitations.

5Examples of search services are Google desktop [106] and Apple’s Spotlight [118]. They extract and index application speciﬁc metadata in order to provide fast local searching.

51 Takai et al., for instance, show that the wireless models used can greatly affect the outcome of simulations [120]. They compare MANET routing protocols, and show that the choice of physical layer model determines the relative rank- ing of the protocols. Models are, of course, continuously developed and refined, but simulators remain a complementary tool to real evaluations, as long as they cannot capture the full interactions of systems and environments – and that will probably forever remain an unachievable goal. Still, simulations are very attractive for repeatable and fully controllable evaluations at varying scales, which can not be carried out on real world testbeds. Emulations constitute an attractive middle-ground. They usually run proto- type implementations of protocols and systems on real hardware, but abstract away some of the unpredictable factors of radio and mobility. A common approach is to first measure these factors in the real world, and then use the collected mobility and radio traces (e.g., SNR measurements) to drive either simulations or emulations. With such traces the radio environment and mobility pattern can be repeated indefinitely. Zhang et al., describe in [121], an example of an emulation system for testing mobile ad hoc networks. They replace the wireless links with wired ones, and use packet filters to emulate mobility and radio connectivity. Thus, they entirely remove the wireless factors from the emulation. The systems presented in [122, 123, 124], use similar approaches, but instead of removing the wireless factors they use attenuators or digital signal processors to make them more realistic, albeit still controllable. These systems often rely on expensive hardware and, as in the case of [124], dedicated facilities in which experiments are carried out in radio isolated environments. Unless these emulation testbeds are opened up for public use (which in fact some of the are), it is nearly impossible to reproduce and verify the results produced in them. We believe that some testbed experiments should be exposed to all unpredictable factors, such that the interactions of the factors, as well as their full effect on protocols, can be studied. Instead of striving toward complete control of mobility and radio, we argue that for many purposes it is sufficient to be able to “manage” them. The experiments should be designed to allow variance of random factors – but only within measurable limits, such that the impact of the variance can be assessed. The conclusions drawn from the experiments are relative to the experienced variance. Based on this philosophy, we have created an Ad hoc Protocol Evaluation testbed (APE), which is described in detail in Paper I, whilst its methodology is described in Paper V. In short, APE is a complete Linux operating system that installs unintrusively on existing computers, e.g., laptops. This eliminates the risk of software misconfigurations and other incompatibilities. APE strives to be self-contained in that users can download the software and run experiments in an isolated and pre-configured software environment. This takes much of the hassle out of experiments, and experimentalists can literally be up and running experiments within minutes. The system performs exten-

52 sive logging of the environment (e.g., signal-to-noise ratio of packets), and of important parameters in the runtime system. This logging makes it straightforward to assess an experiment after is has finished. APE is pre-configured with scenario descriptions that automatically generate traffic and provide movement instructions to people carrying the laptops. The instructions make it, to some degree, possible to achieve repeatability from one experiment to another. For many evaluations this level of repeatability is sufficient. The scenarios can also be run in emulation mode, where packet filters emulate mobility by filtering out traffic from certain nodes at specific points in time. In Paper VI, we present a cross-environment methodology that uses simulations, emulations and real world experiments as parts of a toolbox. We design experiments that couple these methods, such that protocols can be gradually exposed to the random factors of the real world. This approach was described in Section 1.4.3. It allows us to pin down the factors that impact the performance, by making it possible to compare the results from all three methods.

3.5 Summary and Conclusions In this chapter, we have discussed the challenges of tetherless communication, described the background developments that led to these challenges, and presented the related work in the area. We conclude that the success of the In- ternet can to a large degree be attributed a systems-oriented research approach, which constructed live testbeds and real protocol implementations early on in the Internet’s infancy. The ARPANET and the ALOHANET are prime examples of such efforts, and these have undoubtedly played key roles in the success story that the Internet is. We further conclude that similar experimental and systems-oriented research is needed to also make challenged networking a reality. Cross-layer designs are needed that unfortunately undermine the layering principle of the Internet. Radio effects and mobility that are sensed in lower layers call for new protocols and architectures that closer interact with these realities. When delays and disconnections are severe, also the end-to-end principle is under- mined. We need new principles that are not formulated around strict requirements on, e.g., connectivity. The lessons learned have played important roles in the design of our Haggle architecture. Haggle is a “clean-slate” architecture in that it does not depend on legacy technologies, such as those underlying the Internet. However, Hag- gle can make use of such technologies when available. The work illustrates the opportunity to innovate without restrictions of legacy technologies. This is because many challenged networks are orthogonal to the Internet – they can operate in parallel, overlayed on top of the Internet, or by interfacing with it.

53 In the next chapter, we give summaries of the papers that present the detailed research results that underlie this thesis. After the summaries of papers follow our larger scope conclusions.

54 4. Summaries of Included Papers

4.1 Paper I: A Large-scale Testbed for Reproducible Ad hoc Protocol Evaluations The outset of this paper was the observation that there was a growing number of routing protocols for ad hoc networks, out of which only few had been systematically compared side-by-side with respect to performance in the real world. A typical MANET paper at the time suggested a new routing protocol (or modification of an existing one), and then evaluated it in simulation against its competitors. Although this produced many interesting papers, and novel routing ideas, there was an inadequate understanding of how the protocols behaved in the real world, which trade-offs they exhibited, and whether the assumptions underlying them were realistic. Moreover, evaluations of this kind risk intentional bias toward scenarios beneficial to the particular design studied. There existed no neutral ground on which fair side-by-side evaluations could be carried out. When we surveyed the existing real world evaluations, we found that most were carried out on tailor made testbeds, often involving complex and expensive equipment, such as GPS devices, cars, and specialized routers [125, 126]. None of the evaluations involved several protocols in repeatable experiments and, in general, they did not produce software and tools that could help other researchers reproduce the results of the experiments. We therefore constructed a reusable software platform for experimental evaluation of ad hoc network routing protocols, which aimed to substantially lower the barrier for experimentation. The result was the Ad hoc Protocol Evaluation (APE) testbed. It consists of a self-contained test execution environment, which can run from all common operating systems. The execution environment is pre-configured with the routing protocols under test, and with test scenarios that automate experiments. This configuration allows experiments to be easily repeated and scaled. There are no special hardware requirements to run experiments, except for, e.g., laptops equipped with wireless network interfaces. This paper describes the APE testbed and our software based approach. Ex- periments were scaled to 37 mobile nodes by involving students that moved in patterns according to instructions presented on their laptops. To assess the similarities of repeated runs we developed a virtual mobility metric. It uses the signal-to-noise ratio (SNR) from the wireless network interfaces to construct a mobility fingerprint for each test run. Our methodology, with movement in-

55 structions on the laptops and post-experiment mobility fingerprinting, ensures enough repeatability to compare routing protocols side-by-side. This is the main contribution of the paper. In hindsight, we had a somewhat naive approach to the scale of the experiments. Although the paper focuses on the testbed and methodology used, the protocols we ran did not perform well, and it was difficult to analyze the exact causes of the failed routing due to the complexity and sheer scale of experiments. The routing protocols, and their implementations, were simply not mature enough to perform in the real world, and especially not at the scale we attempted. Most of the problems were caused by bugs in the protocol implementations, which at the time were not well tested. Other problems were related to inappropriate assumptions about how the protocols operate in the real world, and the wireless effects they are subjected to. A lesson we learned was that these problems are more efficiently studied in smaller experiments, since the problems are not related to the scale of networks. This lesson we carried over to the work in papers II-VI.

4.2 Paper II: Coping with Communication Gray Zones in IEEE 802.11b based Ad hoc Networks The reality that struck us, during the work with paper I, pointed at performance problems in the implementations of the routing protocols. We therefore began working on our own implementation of the AODV routing protocol, which was one of the most well known protocols at the time. Our goal was to evaluate it in our APE testbed at a smaller scale, such that we would not be overwhelmed by information. By knowing the details of the implementation we also expected to be able to pinpoint the problems. For this purpose, we designed small scale scenarios involving only up to four nodes, whilst still allowing multi-hop communication and mobility. We performed meticulous testing of the AODV implementation to ensure that the results we produced were not biased by bugs in the code. However, irrespective of our efforts, we could not get AODV to perform as expected. Our scenarios were conﬁgured such that a potential multi-hop path was always available. Therefore, the percentage of delivered packets versus those sent, should have been close to 100%. Despite this effort, AODV always lost around 10% of the packets. We found that the loss consistently occurred in particular geographic regions in-between nodes. The intuitive explanation, for this unexpected packet loss, was that our scenarios had intermittent connectivity in these regions. However, we performed several measurements with static routing conﬁgurations that dismissed this hypothesis. The explanation that we eventually arrived at, was that AODV’s sensing mechanism suffers from so called communication gray zones. The cause of this phenomena is the behavior of the IEEE 802.11 MAC layer protocol.

56 AODV broadcasts neighbor detection messages (HELLOs) at a lower rate than unicasted data. As described in section 3.3, the neighbor detection range is therefore longer than the data transmission range. AODV thus discovers one hop neighbor routes that cannot be used until the neighbors move closer to each other. Earlier simulation studies of AODV did not identify this problem for two reasons. First, the wireless models in simulators did not have different ranges for broadcast and unicast. Second, the simulation studies almost exclusively use more efficient link layer neighbor detection, instead of broadcast beacons sent by the routing protocol. Such detection gives the best possible performance, but is generally not supported in real life network interface drivers and operating systems. The discovery of the communication gray zone effect is the first contribution of this paper. The second contribution is three different schemes to counter the effects of the gray zones. Two schemes are unintrusive modifications to AODV that make it more restrictive in selecting neighbors, and the third scheme uses cross-layer SNR feedback to the same end. The cross-layer scheme was the most successful and reliable one, and it brought the packet delivery success close to the 100% we expected. The efficiency of the cross-layer scheme can be attributed its accurate and timely operation. By feeding the SNR of received link layer frames up the networking stack, the routing protocol can filter out HELLO messages that are below a specified SNR threshold. The filtering effectively adjusts the range of broadcast transmissions to that of unicast transmissions. In fact, we found that if we reduced the range even further, we could force AODV to select more robust (but longer) routes, thus improving performance. Obviously, the performance increase relies on the availability of alternative routes to choose from. As other protocols also rely on beacon neighbor sensing, the gray zone effect is not only a problem for AODV. Our study therefore also included the OLSR routing protocol. However, we found that the significantly longer route convergence time of OLSR was its main performance bottleneck in our scenarios. In retrospect, we make the interesting observation that there is still a large gap between simulations and real world experiments. However, our work on more robust route selection was the precursor to the developments of a number of alternative routing metrics such as ETX, HOP, RTT and PktPair [70, 80], which have been developed over the years.

4.3 Paper III: Robust and Flexible Internet Connectivity for Mobile Ad hoc Networks Most MANET routing protocols do not consider interconnection with other networks, such as the Internet. Although many MANETs are IP based, they

57 differ from the Internet in several aspects, e.g., addressing, mobility, routing, and forwarding, making interconnection non-trivial. For example, addressing is often flat instead of hierarchical, and routing can be carried out reactively instead of proactively. Moreover, MANET nodes with direct Internet access may offer this connectivity to other nodes, and therefore there are no strict rules on how, when, and where Internet connectivity opportunities appear. Solutions have to be developed that account for these differences and circumstances – which may vary depending on the particular MANET routing protocol used, and the scenario considered. In this paper, we argue that a solution for Internet connectivity should be robust enough to cope with any circumstance and scenario, and flexible enough to exploit opportunities to do, e.g., multi-homing or load balancing. We survey the existing solutions to MANET Internet connectivity, but find them lacking with respect to robustness and flexibility. We describe the pros and cons of the solutions, along with an analysis of problems they exhibit. Most of the problems we find inherent in Internet connectivity schemes that do not support indirection, i.e., a way to enforce that packets are forwarded through a certain Internet gateway. Without indirection, MANET nodes are unable to signal their choice of gateway to other nodes that provide multi-hop connectivity to the gateway. This may interrupt ongoing data flows over the gateway when routing updates occur. The solution is to either agree on a MANET wide default gateway route, or to add signaling that replicates the gateway choices (indirection state) on intermediate nodes. The first solution is in discord with reactive hop-by-hop routing schemes (e.g., AODV), because they do not have a global view of the MANET, which is necessary for them to agree on a default route. The second solution can lead to a state replication problem, as we describe in the paper. This problem manifests itself, e.g., when a route toward a gateway updates to incorporate new nodes. In this case, the indirection states associated with the route are not replicated on the new nodes. The first contribution of this paper is the identification of the problems of MANET Internet connectivity, and the need for indirection. The second contribution is a comparison and analysis of two Internet connectivity schemes; one that achieves indirection through tunneling, and one that uses a default route without indirection. The results show that the indirection approach achieves up to 20% better performance. The third contribution is a design of a robust and flexible Internet connectivity scheme, which is implemented for the AODV routing protocol. We illustrate how the scheme integrates with Mobile IP to provide seamless connectivity across MANETs and the Internet, and how it supports multi-homing and load-balancing.

58 4.4 Paper IV: Lessons from Experimental MANET Research From the work with papers I-III, we noticed a growing frustration – among ourselves and within the MANET research community – with the lack of progress toward real world usage of MANET technologies. This paper is our take on how to improve the situation. The main lessons that we convey in the paper are as follows. First, research should focus on solving the small scale problems, and make sure that they work well. Only after those issues are solved should effort be put on scaling scenarios to larger sizes. Second, researchers should acknowledge that there is an ad hoc horizon, beyond which routing in ad hoc networks does not scale or work very well. Most user scenarios will likely involve only a small number of nodes (10-20), and up to around three hops. Therefore, routing protocols that operate beyond that scale are likely to be fringe designs, with only marginal deployments in the real world. Thus, targeting such scenarios for general solutions is not an efﬁcient approach. Third, the MANET research must involve more real world experimental work and early prototyping, which performs “reality checks” on protocols early on in the design process.

4.5 Paper V: A Testbed and Methodology for Experimental Evaluation of Mobile Ad hoc Networks This paper is a continuation of the work carried out in paper I. We formulate of a methodology for systematic and repeatable evaluations of protocols for wireless mobile ad hoc networks. The challenges faced in such evaluations involve, for instance, unpredictable radio environments and mobility, which are hard to completely control. The methodology presented suggests a way to handle these factors in our testbed. The methodology consists of a three step approach. (1) We identify the factors that cause large variances in the results. (2) We avoid, or reduce, the effects of them to the extent possible in careful design of the testbed. For example, we eliminated the risk of inconsistent conﬁgurations over test runs by a careful design of the testbed. (3) We provide tools and methods to assess the impact of remaining factors on the variance in the results. If we can deduce that large variances in our results are due to external environmental factors, which lie outside the area of study, we can exclude those results in our analysis. By following this methodology, we argue that it is possible to repeat experiments at a level acceptable for producing conclusive results. What constitutes an “acceptable” level depends on the type of experiment and hypothesis under test. A typical claim one would be able to make with some conﬁdence would be, e.g., under scenario x, protocol y consistently performs better than proto-

59 col z. We do not claim it is possible to exactly reproduce individual protocol performance numbers.

4.6 Paper VI: A Cross-Environment Study of Routing Protocols for Wireless Multi-hop Networks The initial goal with paper I was to carry out a large scale comparison of the most prominent competing MANET routing protocols. However, the reality at the time was that none of the routing protocol implementations available could face such a challenge. Furthermore, the methodologies did not allow systematic analysis of the results in a way that would be fair toward the compared protocols. A “grand comparison” of MANET routing protocols remained an elusive goal, which stayed with us for some time. At the time we started work on paper VI (which would eventually ful- fill our ambition of the grand comparison), we had already created one of the most recognized and well used AODV routing protocol implementations. However, its main competitor DSR, still had no real world implementations that were complete, and ran on our platform (i.e., Linux). We therefore created an implementation of the DSR protocol, which we could compare against AODV. In addition, we included a third party OLSR implementation that we earlier had tested in APE, and found reliable. These protocols together represent all the major routing strategies proposed for standardization within the IETF MANET working group. Thus, we were finally ready to put these protocols to the test in our APE testbed. We developed our testbed methodology further by incorporating simulations and emulations, which included the same protocol implementations and parameters as the real environment under test. This cross-environment methodology – with real world experiments, simulations, and emulations – is one of the contributions of the paper. By cross-examinations of the results, the methodology provided us a tool to better analyze the exact causes of performance problems in each of the protocols. A conclusion from the comparison is the importance of accurate and timely sensing in routing protocols. Such sensing includes determining the neighborhood and the quality of links. We found that DSR acts more aggressively than the other protocols in response to changes in the environment. Although DSR, in some situations, gains significant performance benefits through its aggres- sive style of routing, it also suffers because the sensing is not always accurate enough. The sensing underlies DSR’s decision making and is therefore crucial for efficient routing. As an undesired effect of inaccurate sensing, DSR builds up large queues of undelivered packets in order to later transmit when the environment is stable. This deferred forwarding cause delays up to 10 seconds in our experiments, and subsequent packet bursts when queues are emptied.

60 These bursts themselves affect the sensing negatively, leading to a destabilized environment that drives DSR out of steady-state. The cross-environment side-by-side comparison is the ﬁrst of its kind, and the main contribution of the paper. Other contributions include the methodology mentioned above, and the discovery of protocol issues that explain some of the performance anomalies exhibited in the protocols. These include self- interference from probe packets, TCP backlashing caused by the sudden increase in contention at the start TCP data transfers, and link-cache poisoning from erroneous routing state obtained from promiscuous listening of neighbor route updates.

4.7 Paper VII: Interest-based Content Dissemination in Opportunistic Networks During the work that underlies our previous papers, the MANET area began to prolificate – producing specialized offspring, such as mesh networks, sensor networks, and delay tolerant networks. It became obvious that the traditional ways of setting up routes and carrying out end-to-end forwarding in the best cases worked poorly, and in the worst cases did not work at all. Research into alternative approaches to routing and forwarding began to appear. In this paper we study how content can be disseminated efficiently in opportunistic networks, under the assumption that there is some correlation between the users’ interactions and their interests. For instance, when users interact in communities around their interests, it is efficient to disseminate content matching those interests only among the people of those communities. In order to study the mechanisms behind content dissemination based on people’s contacts and interests, we collected several contact traces using Blue- tooth equipped sensor devices. They were distributed at conferences to people who were asked to carry the devices at all times during the conferences. A device periodically registered the neighbor devices it could establish (radio) contact with, and from this information we could calculate the length of the contacts. We also instructed the participants to fill in forms that collected information about their professional interests, their background, and their affil- iations. We compiled the interests and traces into a number of data sets that we analyzed and used as input to a trace driven simulator, which allowed us to study different algorithms to disseminate content. We compared three content dissemination strategies: Flood, Content, and Wait. Flood transmits messages to all the nodes in contact – irrespective of their interests. Content, on the other hand, only disseminates to those nodes that have interests matching the content. Wait disseminates content only from the source nodes that generate messages, i.e., nodes do not forward any messages that they do not generate themselves. Flood and Wait hence provide the

61 upper and lower bounds in the simulations, whilst Content is a middle-ground. One goal with the paper is to establish these bounds and the middle-ground. The conclusion from the paper is that Content can disseminate messages almost as well as Flood in our scenarios, at a fraction of the cost. Even Wait performs quite well, at the cost of increased delay. This study is the first that verifies and quantifies the correlations between users movements and their interests using real world measurements. The results are, of course, biased toward the specific scenarios studied. It is a difficult challenge to provide studies from which more general conclusions can be drawn. However, this study is a first step toward that goal.

4.8 Paper VIII: A Search-based Network Architecture for Content-oriented and Opportunistic Communication In this paper we raise the question: How would a clean slate architecture for opportunistic and content-oriented communication look like? Our starting point is the observation that users increasingly find, access, and order content using searching. Examples of such searching include web based search engines and desktop search services. They rank and order content according to its match against a set of keywords. This style of searching is the central tenet of the Haggle architecture that we present in the paper. We identify the layered namespaces of the Internet architecture as one of the main reasons for its poor support for dynamic and intermittently connected environments. The Internet does early binding; before communication can take place it must map, e.g., a DNS name in one namespace to an IP address in another. This resolution is rigid and inflexible, as it can in general only bind end-points to locations. In Haggle, we instead introduce a unified metadata namespace, i.e., we can bind users to content in the same namespace. We define a set of search based networking primitives on the unified metadata namespace. The primitives map users to content in a way that is flexible and allow late binding. The type of resolution we achieve resembles that of search engines (e.g., Google), as a set of keywords return a ranked list of results (i.e., bindings). In our case, the keywords can represent the content a node wishes to receive and the results represent the content available in the network, or vice versa. In modern operating systems, searching is used to locate content on local file systems. Our contribution is an extension of this searching, such that files can also represent mobile devices in an opportunistic network. The interests of the mobile devices are represented as metadata in their corresponding files. With this extension, our networking primitives can resolve bindings that map content to mobile devices using searching. These bindings are resolved on mobile devices as they move around and encounter each other opportunisti-

62 cally. The bindings are used to decide which content to disseminate, and in which order to do it. In the paper, we describe our architecture and how it is designed around search based networking. We provide early results on the scaling properties of search based resolution for constrained devices. Our results indicate that it is feasible to support networks of up to hundreds of nodes, and thousands of unique pieces of content, even on the constrained devices we use.

5. Conclusions and Future Work

In this dissertation, we have described the problems of challenged networking, presented methodologies and evaluations of the problems, and proposed new protocols and architectures for future challenged networks. We draw the following conclusions from our research: • For challenged networking, the Internet protocols are at best inefficient but may also not work at all. The reasons for this are related to radio effects, mobility, long delays, and intermittent connectivity. The Internet paradigm of end-to-end connectivity is not feasible in challenged networking. We suggest that it is better to design future protocols and architectures around the principle that end-to-end connectivity is the exception, rather than the rule. • There is a surprisingly large discrepancy in the results produced by standard simulation tools and real world testbeds when evaluating ad hoc networks that involve mobile nodes. The value of cross-environment evaluation methodologies can in this case not be underestimated, since it is inher- ently difficult to create accurate simulation models and to create repeatable real world experiments. • In order to reach beyond the ad hoc network horizon, there must be progress both in shared radio medium protocols, and in higher layer protocols and architectures. In challenged networking, it is not possible to innovate only in higher layers, such as done in the Internet by constructing overlay networks. • Challenged networking calls for specialized communication, in which naming and addressing play key roles. The conversational abstraction of the Internet, where end-points are named and addressed, is not efficient in many challenged network scenarios. Disseminations and content-addressing are new models that promise efficient and flexible alternatives. However, such models need to be further studied before general conclusions can be drawn about their feasibility. In future work, we plan to evaluate new protocols and architectures by involving real users that run live applications. Experience from real users and applications is something that is currently lacking in the challenged networking area. Reliability, congestion control, and quality of service are issues that are increasingly important when the demands of users are taken into account. We anticipate that there will be many solutions that are tailored to the exact demands of the particular scenario considered.

6. Summary in Swedish

Dagens datakommunikation fordrar speciﬁka förutsättningar för att upprät- thållas, och de tas normalt för givet. Exempelvis så bygger Internet-teknologi i mångt och mycket på antagandet om att användare är kontinuerligt anslutna under tiden de nyttjar sina applikationer. Många applikationer slutar därför att fungera om användarens anslutning tillfälligt avbryts. Även under kontin- uerlig anslutning kan applikationer fungera dåligt i samband med, exempelvis, mobilitet och trådlösa förbindelser, vilka idag är allt vanligare. Förklaringen till detta är att Internet i huvudsak skapades för trådbundna datanätverk, där datorer förutsätts vara stationära och relativt homogena i sin kapacitet. När förutsättningarna således förändras, på grund av nya tillämpningar eller nya oförutsägbara miljöer, uppstår en mängd problem som leder till i bästa fall dålig prestanda, eller i sämsta fall till ingen kommunikation alls. Ett exempel är när kommunikations-infrastruktur slagits ut av en naturkatastrof. Mobiltele- foner slutar då att fungera, även om de inte i sig själva är trasiga. Teknologin som ﬁnns på plats skulle bättre utnyttjas om meddelanden, rörande exempelvis hjälpinsatser, kunde spridas direkt mellan mobiltelefoner. Tillämpningar som ska fungera under liknande problematiska förutsättningar ser vi idag utvecklas inom områden som exempelvis ad hoc-nätverk, sensornätverk, svarstidstoler- anta nätverk, och inom automobil-kommunikation. Datorer i dessa nätverk har begränsad processor-kapacitet, strikta energikrav, hög mobilitet, periodiska eller opportunistiska anslutningar, samt höga svarstider med stor varians. I denna avhandling studerar vi de problem som uppstår i dessa nätverk och föreslår nya protokoll och kommunikationsarkitekturer. Vi föreslår lösningar som antingen anpassar existerande teknologier, eller innebär helt nya sätt att bedriva datakommunikation. Metoden vi använder är systemorienterad – vi bygger och testar prototyper av system och protokoll, observerar samt exper- imenterar med olika förbättringar. En stor del av vårt arbete handlar om hur man bedriver experimentell forskning, t.ex. hur hanterar man de utmaningar som det innebär att testa system som är avsedda att fungera i extrema miljöer? En utmaning i detta sammanhang är repeterbarhet. Forskare måste kunna åter- skapa experiment, dels för att enstaka resultat inte är representativa och dels för att andra forskare ska kunna validera resultaten. Repeterbarheten försvåras avsevärt i och med att experimenten utförs i miljöer som är svåra att kon- trollera. I många sammanhang är det viktigt att data kan utbytas mellan olika typer av nätverk, inklusive ovan nämnda nya nätverk. Exempelvis så är Internet-

67 åtkomst attraktivt även i ad hoc-nätverk. Men de protokoll som fungerar som en brygga mellan Internet och andra nätverk är inte effektiva i ad hoc-nätverk, speciellt då mobila noder är involverade. Innehållsbaserad datakommunikation passar bättre för vissa av de nya typerna av nätverk. Standardmodellen i Internet bygger på kommunikation mellan två distinkta punkter. Därför är Internets protokoll för adressering och namngivning konstruerade att namnge och adressera speciﬁka datorer. I nätverk som opererar under extrema förhållanden kan det vara effektivare att istället adressera och namnge applikationsdata. Exempelvis i de fall där datan i sig är det viktiga och inte vilken dator den kommer ifrån.

Sammanfattningsvis är de vetenskapliga bidragen in denna avhandling: • Metoder för att utvärdera system och protokoll som avses fungera i miljöer som Internet ursprungligen inte var skapat för. • Identiﬁering av tidigare okända problem med routningsprotokoll för ad hoc-nätverk. Ett av dessa problem är det så kallade gråzons-problemet. • En prestandajämförelse mellan tre ad hoc-routningsprotokoll. • Beskrivning och analys av problem som uppstår i samband med att mobila ad hoc-nätverk ansluts till Internet. • Ett system som tillåter robust och ﬂexibel Internet-anslutning för mobila ad hoc-nätverk. • En studie av applikationsdata-orienterad disseminering i opportunistiska data-nätverk. • Ett förslag till kommunikationsarkitektur för innehållsbaserad kommunikation i opportunistiska nätverk. Arkitekturens design är baserad på sökprimitiver.

I avhandlingen beskriver vi det forskningsarbete som ligger till grund för dessa bidrag. Vi diskuterar konsekvenser av forskningsresultaten och drar slut- satser om hur framtida kommunikation kan byggas för utmanande miljöer.

68 Bibliography

[1] K. Fall. A delay-tolerant network architecture for challenged internets. In ACM SIGCOMM’03, August 2003.

[2] V. G. Cerf and R. E. Kahn. A protocol for packet network intercommu- nication. IEEE Transactions on Communications, com-22(5):637–648, May 1974.

[3] P. Tsuchiya and T. Eng. Extending the IP internet through address reuse. ACM SIGCOMM Computer Communications Review, 23(1):16– 33, January 1993.

[4] Internet Society IPN Special Interest Group. Homepage of the interplanetary internet project. http://www.ipnsig.org.

[5] Delay Tolerant Networking Research Group (DTNRG). Homepage of the delay tolerant networking research group. http://www.dtnrg.org.

[6] J. Kleinberg. Navigation in a small world. Nature, 406:845, 2000.

[7] O. Sandberg. Distributed routing in small-world networks. In Proceed- ings of the 8th Workshop on Algorithm Engineering and Experiments, 2006.

[8] J. Kleinberg. Complex networks and decentralized search algorithms. In Proceedings of the International Congress of Mathematicians (ICM), 2006.

[9] D. Kempe, J. Kleinberg, and E. Tardos. Maximizing the spread of in- ﬂuence through a social network. In Proceedings of 9th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2003.

[10] D. Kempe, J. Kleinberg, and E. Tardos. Inﬂuential nodes in a diffusion model for social networks. In Proceedings of 32nd International Col- loquium on Automata, Languages and Programming (ICALP), 2005.

[11] S. Milgram. The small world problem. Psychology Today, (1):61–67, 1967.

69 [12] J. Guare. Six Degrees of Separation: A Play. Vintage Books, New York, 1990.

[13] J. Crowcroft, S. Hand, R. Mortier, T. Roscoe, and A. Warﬁeld. Plutarch: An argument for network pluralism. In Workshop on Future Directions in Network Architecture (FDNA-03), 2003.

[14] C. Tschudin, H. Lundgren, and E. Nordström. Embedding MANETs in the Real World, volume 2775/2003 of Lecture Notes in Computer Science, pages 578–589. Springer Berlin / Heidelberg, 2003.

[15] M. Bechler, W. Franz, and L. Wolf. Mobile internet access in ﬂeetnet. February 2003.

[16] E. Huan, W. Hu, Jon Crowcroft, and I. Wassel. Towards commercial mobile ad hoc network applications: A radio dispatch system. In ACM MobiHoc, May 2005.

[17] David B. Johnson and David A. Maltz. Dynamic Source Routing in Ad Hoc Wireless Networks, chapter 5, pages 153–181. Mobile Computing. Kluwer Academic Publishers, 1996.

[18] J. Broch, D. Maltz, D. Johnson, Y-C. Hu, and J. Jetcheva. A performance comparison of multi-hop wireless ad hoc network routing protocols. In Proceedings of ACM International Conference on Mobile Computing and Networking (MobiCom), October 1998.

[19] S. Kurkowski, T. Camp, and M. Colagrosso. Manet simulation studies: The incredibles. ACM’s Mobile Computing and Communications Review, 9(4):50–61, October 2005.

[20] The Free Software Foundation. FSF free software licensing. http:// www.fsf.org/licensing/.

[21] E. Nordström. A large scale ad hoc network testbed for reproducible performance tests. Master’s thesis, Uppsala Uni- versity, 2002. http://www.csd.uu.se/courses/course-material/xjobb/ docs-reports/Nordstrom-2002.pdf.

[22] C. Perkins and E. Royer. Ad hoc on-demand distance vector routing. In Proceedings of 2nd IEEE Workshop on Mobile Computing Systems and Applications (WMCSA’99), February 1999.

[23] The IETF MANET working group. http://www.ietf.org/html.charters/ manet-charter.html.

[24] Peter Lee. AODV for ipv6 network. Simon Fraser University. http: //members.shaw.ca/aodv6-sfu/.

70 [25] Martin Dietze. An ipv6-patch for AODV-UU 0.9. University of Buck- ingham. http://webspace.buckingham.ac.uk/cadams/bnrg/aodv6/index. html.

[26] Anders Lundström. AODV-UU IPv6. Building Blocks for Net- work Centric Warfare. http://www.b2ncw.org/index.php?page=open_ deliveries.php.

[27] Multicast extensions of aodv (maodv). University of Maryland. http: //www.hynet.umd.edu/research/maodv/MAODV-UMD.html.

[28] Rui Prior. SIPAODV-UUETX. Karlstad University. http://www.cs.kau. se/cs/prtp/pmwiki/pmwiki.php?n=MeshWikipage.SIPAODV-UUETX.

[29] Alfonso Ariza Quintana. AODV-UU for the OMNeT++ simulator. Malaga University. http://webpersonal.uma.es/~AARIZAQ/.

[30] Francisco J. Ros. DYMO-UM. University of Murcia. http://masimum. inf.um.es/?Software:DYMOUM.

[31] The network simulator - ns-2 webpage. http://www.isi.edu/nsnam/ns/.

[32] E. Belding-Royer. Report on the aodv interop. UCSB Technical Report 2002-18, University of California Santa Barbara, June 2002.

[33] E. Nordström, C. Diot, R. Gass, and P. Gunningberg. Experiences of measuring human mobility using bluetooth inquiring devices. In Pro- ceedings of MobiEval’07, June 2007.

[34] Ad hoc on-demand distance vector (aodv) routing. Internet RFC 3561, July 2003. http://www.ietf.org/rfc/rfc3561.txt.

[35] L. Kleinrock. Information ﬂow in large communication nets. Technical report, RLE Quarterly Progress Report, July 1961.

[36] H. Zimmermann. OSI reference model – the ISO model of architecture for open systems interconnection. IEEE Transactions on Communica- tions, COM-28(4):425–432, April 1980.

[37] International Organization for Standardization (ISO). Homepage of the international organization for standardization. http://www.iso.org.

[38] L. G. Roberts and B. D. Wessler. Computer network development to achieve resource sharing. pages 543–549, 1970.

[39] L. Pouzin. Presentation and major design aspects of the cyclades computer network. In Proceedings of Thrid ACM-IEEE Communications Symposium, pages 80–87, November 1973.

71 [40] J. Rosenberg, H. Schulzrinne, G. Camarillo, A. Johnston, J. Peterson, R. Sparks, M. Handley, and E. Schooler. Sip: Session initiation protocol. IETF RFC 3261, June 2002. http://www.ietf.org/rfc/rfc3261.txt.

[41] R. Moskowitz, P. Nikander, P. Jokela, and T. Henderson. IETF RFC 5201, April 2008.

[42] J. Saltzer, D. Reed, and D. Clark. End-to-end arguments in system design. ACM Transactions on Computer Systems, 2(4):277–288, Novem- ber 1984.

[43] J. Salzer. On the naming and binding of network destinations. IETF RFC 1498, August 1993.

[44] Norman Abramson. Development of the ALOHANET. IEEE Transac- tions on Information Theory, 1985.

[45] K. Xu, M. Gerla, and S. Bae. How effective is the IEEE 802.11 RT- S/CTS handshake in ad hoc networks? In Global Telecommunications Conference (Globecom), November 2002.

[46] V. Tsaoussidis and I. Matta. Open issues on TCP for mobile computing. Wireless Communications and Mobile Computing, 2(2001-013):3–20, December 2001.

[47] H. ElAarag. Improving tcp performance over mobile networks. ACM Computing Surveys, 34:357–374, September 2002.

[48] H. Balakrishnan, S. Seshan, and R. Katz. Improving reliable transport and handoff performance in cellular wireless networks. Wireless Net- works, 1(4):469–481, 1995.

[49] A. Bakre and B. Badrinath. Implementation and performance evaluation of in-direct TCP. IEEE Transactions on Computers, 46(3):260– 278, 1997.

[50] Kevin Brown and Suresh Singh. M-TCP: TCP for mobile cellular networks. ACM Computer Communication Review, 27(5), 1997.

[51] C. Casetti, M. Gerla, S. S. Lee, S. Mascolo, and M. Sanadidi. Tcp with faster recovery. In MILCOM, October 2000.

[52] J. Ioannidis, D. Duchamp, and G. Q. Maguire Jr. Ip-based protocols for mobile internetworking. In ACM Computer Communications Review, volume 21, pages 235–245, September 1991.

[53] D. B. Johnson. Ubiquitous mobile host internetworking. In Fourth Workshop on Workstation Operating Systems, pages 85–90, October 1993.

72 [54] C. Perkins. IP mobility support for IPv4, January 2002. IETF Internet RFC 3220.

[55] I. Stoica, D. Adkins, S. Zhuang, S. Shenker, and S. Surana. Internet indirection infrastructure. In Proceedings of SIGCOMM, August 2002.

[56] S. Zhuang, K. Lai, I. Stoica, R. Katz, and S. Shenker. Host mobility using an internet indirection infrastructure. In Proceedings of MobiSys 2003, pages 129–144, June 2003.

[57] C. Perkins and P. Bhagwat. Highly dynamic destination-sequenced distance-vector routing (DSDV) for mobile computers. In ACM SIG- COMM, pages 234–244, 1994.

[58] I. Gitman, R. Van Slyke, and H. Frank. Routing in packet-switching broadcast radio networks. IEEE Transactions on Communications, 24(8):926–930, 1976.

[59] S. Sussman. A survivable network of ground relays for tactical data communications. IEEE Transactions on Communications, 28(9):1616– 1624, 1980.

[60] T. Anwar and C. Jones. A routing strategy idea using a self-supporting, modular navigational system for land-based mobile pr-nets. In Vehicu- lar Technology Conference, pages 619–623, May 1991.

[61] G. Holland and N. Vaidya. Analysis of TCP performance over mobile ad hoc networks. Wireless Networks, (8):275–288, 2002.

[62] K. Xu, M. Gerla, L. Qi, and Y. Shu. Enhancing TCP fairness in ad hoc wireless networks using neighborhood red. In Proceedings of Mobi- Com’03, San Diego, USA, September 2003.

[63] C. Rohner, E. Nordström, P. Gunningberg, and C. Tschudin. Interac- tions between TCP, UDP and routing protocols in wireless multi-hop ad hoc networks. In Proceedings of Realman: IEEE ICPS Workshop on Multi-hop Ad hoc Networks: from theory to reality, July 2005.

[64] B. A. Chambers. The grid roofnet: a rooftop ad hoc wireless network. Master’s thesis, MIT, June 2002.

[65] H. Lundgren, K. Ramachandran, E. Belding-Royer, K. Almeroth, M. Benny, A. Hewatt, A. Touma, and A. Jardosh. Experiences from the design, deployment, and usage of the ucsb meshnet testbed. IEEE Wireless Communications Magazine, 13(2):18–29, April 2006.

[66] Self-organizing neighborhood wireless mesh networks. http://research. microsoft.com/mesh.

73 [67] IEEE P802.11s/D1.10. Draft amendment to standard IEEE 802.11: ESS Mesh Networking, March 2008.

[68] K. Jain, J. Padhye, V. Padmanabhan, and L. Qiu. Impact of interference on multi-hop wireless network performance. In ACM MobiCom, September 2003.

[69] D. Aguayo, J. Bicket, S. Biswas, G. Judd, and R. Morris. Link-level measurements from an 802.11b mesh network. In Proceedings of ACM SIGCOMM 2004, pages 121–131, August 2004.

[70] R. Draves, J. Padhye, and B. Zill. Comparison of routing metrics for static multi-hop wireless networks. In SIGCOMM, volume 34, pages 133–144, October 2004.

[71] R. Draves, J. Padhye, and B. Zill. Routing in multi-radio, multi-hop wireless mesh networks. In ACM MobiCom, September 2004.

[72] K. N. Ramachandran, E. M. Belding, K. C. Almeroth, and M. M. Bud- dhikot. Interference-aware channel assignment in multi-radio wireless mesh networks. In Proceedings of Infocom, April 2006.

[73] V. Mhatre, F. Baccelli, H. Lundgren, and C. Diot. Joint mac-aware routing and load balancing in mesh networks. In Proceedings of CoNext, December 2007.

[74] S. Biswas and R. Morris. Opportunistic routing in multi-hop wireless networks. In SIGCOMM, August 2005.

[75] T. Clausen, P. Jacquet, A. Laouiti, P. Muhlethaler, and a. Qayyum et L. Viennot. Optimized link state routing protocol. In IEEE National Multi-Topic Conference (INMIC 2001), December 2001.

[76] I. Chakeres and C. Perkins. Dynamic MANET On-demand (DYMO) routing. Internet draft, February 2008. draft-ietf-manet-dymo-12.txt.

[77] Marc Liberatore, Brian Neil Levine, and Chadi Barakat. Maximizing transfer opportunities in bluetooth dtns. In Proceedings of ACM Con- ference on Future Networking Technologies (CoNext), December 2006.

[78] I. D. Chakeres and E. M. Belding-Royer. The utility of hello messages for determining link connectivity. In 5th International Symposium on Wireless Personal Multimedia Communications (WPMC), October 2002.

[79] S. Krco,ˇ Marina Dupcinov,ˇ and S. Murphy. Performance evaluation of an IEEE 802.11b ad-hoc network. FACTA UNIVERSITATIS (NIŠ) Electrical Energetics, 17(1):53–68, April 2004.

74 [80] D. De Couto, D. Aguayo, J. Bicket, and R. Morris. A high-throughput path metric for multi-hop wireless routing. In Proceedings of the 9th ACM International Conference on Mobile Computing and Networking (MobiCom), September 2003.

[81] Q. Li and D. Rus. Sending messages to mobile users in disconnected ad-hoc wireless networks. In Proceedings of the 6th annual international conference on Mobile computing and networking (MobiCom), 2000.

[82] J. Davis, A. Fagg, and B. N. Levine. Wearable computers and packet transport mechanisms in highly partitioned ad hoc networks. In Pro- ceedings of IEEE International Symposium on Wearable Computers (ISWC), pages 141–148, October 2001.

[83] S. Jain, K Fall, and R. Patra. Routing in a delay tolerant network. In ACM SIGCOMM’04, August 2004.

[84] A. Seth, D. Kroeker, M. Zaharia, S. Guo, and S. Keshav. Low-cost communication for rural internet kiosks using mechanical backhaul. In Proceedings of MOBICOM, pages 334–345, September 2006.

[85] A Doria, M. Uden, and D. P. Pandey. Providing connectivity to the saami nomadic community. In Development by Design Conference, 2002.

[86] W. Zhao, M. Ammar, and E. Zegura. A message ferrying approach for data delivery in sparse mobile ad hoc networks. In Proceedings of ACM Mobihoc, May 2004.

[87] R. Y. Wang, S. Sobti, N. Garg, E. Ziskind, and J. Lai. Turning the postal system into a generic digital communication mechanism. In Proceed- ings of SIGCOMM, September 2004.

[88] J. Burgess, B. Gallagher, D. Jensen, and B. N. Levine. Maxprop: Rout- ing for vehicle-based disruption-tolerant networks. In Proceedings of IEEE INFOCOM, April 2006.

[89] P. Hui, A. Chaintreau, J. Scott, R. Gass, J. Crowcroft, and C. Diot. Pocket switched networks and human mobility in conference environments. In ACM SIGCOMM Workshop on delay tolerant networking and related networks (WDTN-05), August 2005.

[90] A. Vahdat and D. Becker. Epidemic routing for partially connected ad hoc networks. Technical Report CS-200006, Duke University, April 2000.

75 [91] T. Spyropoulos, K. Psounis, and C. S. Raghavendra. Single-copy routing in intermittently connected mobile networks. In IEEE Communica- tions Society Conference on Sensor and Ad Hoc Communications and Networks (SECON), pages 235–244, October 2004.

[92] T. Spyropoulos, K. Psounis, and C. S. Raghavendra. Spray and wait: An efﬁcient routing scheme for intermittently connected mobile networks. In ACM SIGCOMM Workshop on Delay Tolerant Networking (WDTN), August 2005.

[93] A. Lindgren, A. Doria, and O. Schelén. Probabilistic routing in intermittently connected networks. In ACM SIGMOBILE Mobile Computing and Communications Review (POSTER), pages 19–20, July 2003.

[94] A. Balasubramanian, B. N. Levine, and A. Venkataramani. Dtn routing as a resource allocation problem. In SIGCOMM, August 2007.

[95] C. Mascalo and M. Musolesi. Scar: Context-aware adaptive routing in delay tolerant mobile sensor networks. In IWCMC, July 2006.

[96] C. Boldrini, M. Conti, J Jacopini, and A. Passarella. Hibop: a history based routing protocol for opportunistic networks. In World of Wireless, Mobile and Multimedia Networks (WoWMoM), June 2007.

[97] Pan Hui, Jon Crowcroft, and Eiko Yoneki. Bubble Rap: Social-based forwarding in delay tolerant networks. In 9th ACM International Sym- posium on Mobile Ad Hoc Networking and Computing (MobiHoc), May 2008.

[98] P. Hui and J. Crowcroft. How small labels create big improvements. In International Workshop on Intermittently Connected Mobile Ad hoc Networks in conjunction with IEEE PerCom, March 2007.

[99] B. Ahlgren, M. Brunner R. Hancock L. Eggert, and S. Schmid. Invari- ants – a new design methodology for network architectures. In ACM SIGCOMM 2004 Workshop on Future Directions in Network Architec- ture, August 2004.

[100] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan. Chord: A scalable peer-to-peer lookup service for internet applications. In SIGCOMM, August 2001.

[101] A. Rowstron and P. Druschel. Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems. In IFIP/ACM International Conference on Distributed Systems Platforms (Middle- ware), pages 329–350, November 2001.

76 [102] V. Jacobson. A new way to look at networking, August 2006. http: //video.google.com/videoplay?docid=-6972678839686672840.

[103] IPS 180-1. Secure hash standard. U.S. Department of Commerce/NIST, National Technical Information Service, April 1995. Springﬁeld, VA.

[104] D. R. Cheriton and M. Gritter. Triad: a scalable deployable nat-based internet architecture. Technical report, Departments of Computer Sci- ence and Electrical Engineering at Stanford University, January 2000.

[105] D. K. Gifford, P. Jouvelot, M. A. Sheldona, , and J. W. O’Toole Jr. Semantic ﬁle systems. In ACM Symposium on Operating System Prin- ciples, pages 16–25, 1991.

[106] Google desktop. http://desktop.google.com/.

[107] Craig A. N. Soules and Gregory R. Ganger. Connections: Using context to enhance ﬁle search. In ACM Symposium on Operating Systems Principles, October 2005.

[108] W. Adjie-Winoto, E. Schwartz, H. Balakrishnan, and J. Lilley. The design and implementation of an intentional naming system. In ACM Symposium on Operating System Principles, December 1999.

[109] P. T. Eugster, P. A. Felber, R. Guerraoui, and A.-M. Kermarrec. The many faces of publish/subscribe. ACM Computing Surveys, 35(2):114– 131, June 2003.

[110] A. Carzaniga and A. L. Wolf. Forwarding in a content-based network. In SIGCOMM, August 2003.

[111] R. Braden T. Faber and M. Handley. From protocol stack to protocol heap - role-based architecture. In HotNets-I, Princeton, NJ, October 2002.

[112] R. Gold, P. Gunningberg, and C. Tschudin. A virtualized link layer with support for indirection. In ACM SIGCOMM Workshop on Future Directions in Network Architecture (FDNA), August 2004.

[113] V. Lenders, G. Karlsson, and M. May. Wireless ad hoc podcasting. In IEEE Communications Society Conference on Sensor, Mesh, and Ad Hoc Communications and Networks (SECON), June 2007.

[114] V. Ramasubramanian, R. Peterson, and E. G. Sirer. Corona: A high performance publish-subscribe system for the world wide web. In Sympo- sium on Networked Systems Design and Implementation (NSDI), pages 15–28, May 2006.

77 [115] M. Gritter and D. R. Cheriton. An architecture for content routing support in the internet. In USENIX Symposium on Internet Technologies and Systems, March 2001. [116] M. Demmer, K. Fall, T. Koponen, and S. Shenker. Towards a modern communications api. In Sixth Workshop on Hot Topics in Networks (HotNets-VI), November 2007. [117] T. Koponen, M. Chawla, B-G. Chun, A. Ermolinskiy, S. Shenker K. H. Kim, and I. Stoica. A data-oriented (and beyond) network architecture. In Proceedings of ACM SIGCOMM, pages 181–192, August 2007.

[118] Apple spotlight. http://www.apple.com/macosx/features/300.html# spotlight. [119] David Kotz, Calvin Newport, Robert S. Gray, Jason Liu, Yougu Yuan, and Chip Elliott. Experimental evaluation of wireless simulation assumptions. In Proceedings of the ACM/IEEE International Symposium on Modeling, Analysis and Simulation of Wireless and Mobile Systems (MSWiM), October 2004. [120] M. Takai, J. Martin, and R. Bagrodia. Effects of wireless physical layer modeling in mobile ad hoc networks. In Proceedings of the ACM In- ternational Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), October 2001. [121] Y. Zhang and W. Li. An integrated environment for testing mobile ad- hoc networks. In Proceedings of 3rd ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), June 2002. [122] J. Kaba and D. Raichle. Testbed on a desktop: Strategies and techniques to support multi-hop manet routing protocol development. In Proccedings of 2nd ACM International Symposium on Mobile Ad Hoc Networking and Computing (MobiHoc), October 2001. [123] G. Judd and P. Steenkiste. Repeatable and realistic wireless experimentation through physical emulation. In Second Workshop on Hot Topics in Networks (HotNets-II, 2003. [124] Raychaudhuri, I. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa, H. Liu, and M. Singh. Overview of the orbit radio grid testbed for evaluation of next-generation wireless network protocols. In Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC), 2005. [125] D. Maltz, J. Broch, and D. Johnson. Quantitative lessons from a full-scale multi-hop ad hoc network testbed. In Proceedings of the

78 IEEE Wireless Communications and Networking Conference (WCNC), September 2000.

[126] R. Ramanathan and R. Hain. An ad hoc wireless testbed for scalable, adaptive qos support. In Proceedings of IEEE Wireless Communications and Networking Conference (WCNC), September 2000.

[127] A. Vahdat and D. Becker. Epidemic routing for partially-connected ad hoc networks. Technical Report CS-2000-06, Department of Computer Science, Duke University, July 2000.

[128] A. El Fawal, J-Y. Le Boudec, and K. Salamatian. Self-limiting epidemic forwarding. Technical Report LCA-REPORT-2006-126, EPFL, 2006.

[129] A. Lindgren, A. Doria, and O. Schelén. Probabilistic routing in intermittently connected networks. In Proceedings of the The First Inter- national Workshop on Service Assurance with Partial and Intermittent Resources (SAPIR 2004), August 2004.

[130] IEEE standards 802.15.1. IEEE, 2005. http://standards.ieee.org/ getieee802/.

79 Acta Universitatis Upsaliensis Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology 450

Editor: The Dean of the Faculty of Science and Technology

A doctoral dissertation from the Faculty of Science and Technology, Uppsala University, is usually a summary of a number of papers. A few copies of the complete dissertation are kept at major Swedish research libraries, while the summary alone is distributed internationally through the series Digital Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology. (Prior to January, 2005, the series was published under the title “Comprehensive Summaries of Uppsala Dissertations from the Faculty of Science and Technology”.)

ACTA UNIVERSITATIS UPSALIENSIS Distribution: publications.uu.se UPPSALA urn:nbn:se:uu:diva-9002 2008