Smart Messages: A Distributed Computing Platform for Networks of Embedded Systems ∗ Porlin Kang, Cristian Borcea, Gang Xu, Akhilesh Saxena, Ulrich Kremer, and Liviu Iftode Department of Computer Science, Rutgers University, Piscataway, NJ 08854 {kangp, borcea, gxu, saxena, uli, iftode}@cs.rutgers.edu Abstract or inter-car collaboration to improve traffic safety and fluidity. In this paper, we present the design and imple- To leverage the raw computing power provided mentation of Smart Messages, a distributed comput- by NES into new distributed applications, we have ing platform for networks of embedded systems based to overcome the scale, heterogeneity, and volatil- on execution migration. A Smart Message (SM) is ity that characterize these networks. Current ap- a user-defined distributed program which executes on proaches to NES programming are ad hoc and pro- nodes of interest, named by their properties, and uses vide limited flexibility; they are designed for specific an explicit lightweight migration to reach these nodes. classes of applications (e.g., querying the network During migrations, an SM carries its code and execu- for certain data) and can hardly accommodate new tion state, and it self-routes at each intermediate node applications or services after the network has been between two nodes of interest. The nodes in the net- deployed. As NES applications diversify, there will work cooperate to support the SM execution by provid- be an increasing demand for a common distributed ing a virtual machine and a shared memory region ad- computing platform to support arbitrary applica- dressable by names (tag space). To illustrate the flexi- tions over NES. bility of SMs to program real world applications, we A distributed platform for NES have to support describe EZCab, an application for booking cabs in simple development of new distributed applications. densely populated urban areas. We also present exper- It also has to allow applications to cope with the imental results to quantify the performance achieved uncertainty encountered in NES (e.g., the network by the SM prototype. topology as well as the resources at nodes are un- known a priori and can vary greatly over time). To answer these requirements, we need new program- ming abstractions since traditional message passing 1. Introduction does not work in highly dynamic network configu- rations. This model includes a number of charac- Following the rapid development of sensor net- teristics that render it unusable in NES: end-to-end work technologies [12, 26–28, 32], networks of em- data transfer between applications, fixed bindings bedded systems (NES) will become ubiquitous dur- between names and node addresses, and fixed rout- ing the next decade. We are already witnessing the ing. beginning of this new world in which cars, cam- A first problem with end-to-end data transfers eras, cell phones, and even watches have wireless is that they may complete very slowly, or may not network interfaces and are powerful enough to run complete at all in volatile networks [58]. Since ap- Linux [1]. NES offers the opportunity to program plications have no control over the network, they a large spectrum of applications, ranging from sim- are forced to wait indefinitely (or until the connec- ple data collection and data dissemination to com- tion times out) each time something goes wrong in plex distributed applications such as remote object the network. To be able to adapt quickly to network tracking using robots equipped with video cameras volatility, applications would like to regain the con- trol as soon as possible. Another problem with tra- ∗ This work is supported in part by the NSF grant ANI- ditional end-to-end data transfer is that it does not 0121416 allow in-network processing in order to reduce the size of data transferred by applications [26]. Reduc- found or after an application-set timeout. Since ap- ing the amount of traffic in the network is impor- plication’s code as well as its execution state are al- tant in mobile ad hoc networks, such as NES, since ready at the same node, the SM can quickly adapt it leads to reduced bandwidth and energy consump- to changes in the network. For instance, an SM can tion. Therefore, NES applications would also like to change dynamically its routing or its destination. be capable of performing in-network processing. Second, the content-based routing provides the flex- The fixed bindings between names and node ad- ibility to reach a node that offers a certain property dresses assumed in the message passing model rep- in an application-controlled manner. Third, the SM resent also a serious obstacle for NES applications. programming model eases the deployment of new After a fixed binding has been established during applications in the network after the network de- the name resolution phase, an application is forced ployment phase has ended. A user can inject ap- to contact the same node each time it needs to ac- plications at any node in the network, and conse- cess a resource of the same type. Commonly, name quently the applications migrate their code at every resolvers react slowly to network changes, and ap- node where they need to execute. And fourth, SMs plications would try to contact a node long time af- can significantly reduce the amount of traffic gener- ter this node has become unreachable, even though ated by certain classes of applications (e.g., process nodes with similar resources exist in the network. the data at the source for distributed image process- To prevent such a situation, more flexible naming is ing). needed in NES. We believe that content-based nam- Although the SM computing platform leverages ing [9, 30] can provide a solution because it allows work from various research areas, notably from mo- applications to contact any node has a certain re- bile agents [22,33] and active networks [19,40,54], its source. uniqueness comes from both the problem it solves, Since content-based naming makes fixed ad- programming distributed applications in NES, and dresses (e.g., IP) irrelevant, the routing and name its design that allows rapid adaptation of applica- resolution should be integrated in NES. Addition- tions to volatile network configurations. A detailed ally, given the diversity of applications, no sin- discussion and comparison between SMs and other gle routing will provide good performance for work is presented in Section 10. all applications. Therefore, similar to active net- To validate the proposed architecture, we have works [19, 40, 54], it would be desirable to let ap- developed an SM prototype in Java by modify- plications use the best-suited routing for their ing Sun’s Java KVM [2]. The prototype runs over needs. a testbed consisting of HP iPAQs equipped with 802.11 cards for wireless communication. This paper In this paper, we present the design and im- presents the implementation and evaluation of our plementation of Smart Messages (SMs) [15], a SM prototype. It also describes an application de- distributed computing platform for NES based veloped over this prototype, called EZCab, for book- on execution-migration, content-based nam- ing cabs in densely populated urban areas. ing, and self-routing [14]. Instead of passing data end-to-end between nodes, an SM application mi- The rest of this paper is organized as follows. Sec- grates to nodes of interest named by content tion 2 introduces the main SM concepts. The system and executes there. Each node has a virtual ma- support provided by nodes is described in Section 3. chine for SM execution and a name-based mem- Section 4 discusses the SM API. The self-routing ory, called tag space. The SMs use the tag space for mechanism is presented in Section 5. The basic se- content-based naming and persistent shared mem- curity architecture is explained in Section 6. Sec- ory. An SM carries its own routing code and routes tion 7 describes the implementation details of our itself at each node in the path toward a node of in- SM prototype, and Section 8 presents an evaluation terest. To perform routing, SMs store routing of this prototype. Section 9 describes the EZCab ap- information in the tag space at nodes. plication. We discuss the related work in Section 10 and conclude in Section 11. SMs represent an attractive alternative to tra- ditional distributed computing based on message passing for four reasons. First, SMs allow applica- 2. Smart Messages Architecture tions to adapt to highly dynamic network condi- tions. During migrations between nodes of interest, An SM is a component of a user-defined appli- the routing code can be instructed to return the cation whose execution is distributed over a series control to application as soon as a route cannot be of nodes using execution migration. The nodes on Application Migration // application data brick int numCabs, i; Routing Migration Location loc; // application code brick Application for(i=0; i<numCabs; i++){ migrate("FreeCab"); Virtual Virtual Virtual deleteTag("FreeCab"); Machine Machine Machine writeTag("Location", loc); Tag Code Tag Code Tag Code } Space Cache Space Cache Space Cache Node 1 Node 2 Node 3 (Node of interest) (Intermediate Node) (Node of interest) Figure 2. Smart Message Code Example Figure 1. Distributed Computing Using Smart Messages SMs, we present a very simple example consisting of an SM that books cabs in a densely populated city. Let us consider a group of people attending a confer- which SM applications execute, called “nodes of ence, who wants to return to the conference venue interest”, are named by content (tag names), dis- after an “off-site” lunch. Instead of calling a cab covered using application-controlled routing, and company or waiting on the street for a free cab, one switched when the SM application calls for exe- of them uses her handheld device to inject an SM in cution migration.