BraceForce: A Middleware to Enable Sensing Integration in Mobile Applications for Novice Programmers
Xi Zheng, Dewayne E. Perry, Christine Julien The Center for Advanced Research in Software Engineering Mobile and Pervasive Computing Lab The University of Texas at Austin [email protected],[email protected],[email protected]
ABSTRACT demonstrating the feasibility of applications, the develop- Even as the use of sensor networks to support mobile ap- ment of support services such as routing protocols or energy- plications grows, our ability to seamlessly and efficiently in- saving algorithms, or on advancement of hardware platforms corporate sensor network capabilities into our mobile ap- and operating systems. Little focus has been applied to ef- plications remains astoundingly difficult. Today, accessing fective development support for applications that integrate remote sensing data and integrating this data into the adap- the capabilities of networked sensing platforms in easy to use tive behavior of a dynamic user-facing mobile application re- and extensible ways. In addition to the variety of data for- quires interacting with multiple platforms, languages, data mats, communication technologies, and programming plat- formats, and communication paradigms. We present Brace- forms a developer must tackle, mobile applications also re- Force, an open and extensible middleware that allows de- quire handling network dynamics and energy constraints. velopers to access the myriad remote sensing capabilities This paper introduces BraceForce, a middleware for mo- inherent to today’s mobile computing spaces (where mo- bile application development that simplifies the development, bile devices and sensors are closely integrated) using very deployment, and debugging of mobile applications. Brace- minimal code. Further, BraceForce incorporates event- and Force targets novice developers, by which we refer to pro- model-driven data acquisition as first-class concepts to pro- grammers who have a good understanding of programming vide efficient access to sensing while retaining expressiveness in general but have limited prior experience in program- and flexibility for applications. We present the BraceForce ming sensor devices. For development, BraceForce sepa- architecture and key abstractions, describe their implemen- rates the application developer from the low-level interac- tations, and provide an empirical study using BraceForce to tions between the platform and sensors. For deployment, support mobile applications integrating sensing. BraceForce also allows the developer to connect the appli- cation to sensing assets in the deployment environment. Fi- nally, when applied to debugging tasks, BraceForce can mon- 1. INTRODUCTION itor a test environment using capabilities that may not be Developing and deploying mobile applications integrating available in the deployment environment. Architecturally, sensing involves a large amount of low-level programming BraceForce defines functional tiers that encapsulate related that requires interacting with different (often proprietary) aspects but coordinate in such a way that the tiers’ deploy- data formats, languages, and operating systems. In prac- ment to particular physical assets is flexible. Different tiers tice, applications are built for specific sensor platforms with can be deployed on user-facing devices or on sensing devices little potential for portability or integration with other sen- with limited capabilities, directly addressing and leveraging sors. Debugging mobile applications that integrate with a the specific capabilities and intentions of each device. Brace- physical environment requires not only integrating the appli- Force provides auto-discovery of new sensing and computing cation with sensing but also the use of a testing harness that, assets, allowing easy integration of new capabilities into ex- at debugging time, accesses sensed data about the physical isting applications or automatic and seamless extension of a environment for the purposes of validating the actions of debugging environment. the application. Integrating sensing for mobile applications Instead of using a proprietary programming language (e.g., and debugging support in a way that is easy, flexible, and nesC [9]), supporting a single hardware platform (e.g., ODK [5]), portable is essential for supporting current mobile applica- or creating a new standard (e.g., Dandelion [19] or DASH [7]), tion development. BraceForce presents a simple Java programming interface for In general, research in this space has been focused on integrating new sensing capabilities. To add a new sensor to BraceForce, a developer just needs to implement a sim- ple sensor driver interface that provides essential commands (e.g., open, close, query) and configuration. The driver im- plementation can be specific to the particular sensor but Permission to make digital or hard copies of all or part of this work for basically translates raw data into meaningful BraceForce personal or classroom use is granted without fee provided that copies are sensor data constructs, guided by the provided BraceForce not made or distributed for profit or commercial advantage and that copies interfaces. BraceForce handles issues such as thread man- bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific agement, networking, remote procedure call, etc. permission and/or a fee. BraceForce supports both pull- and push-based interac- Copyright 20XX ACM X-XXXXX-XX-X/XX/XX ...$15.00. tion with sensing devices. BraceForce also embraces model- communication scale of one machine, and connection mecha- driven data acquisition (MDDA) [14, 21, 22, 24, 27] to reduce nisms are among ROS components running on the same ma- energy costs of integrating sensing in mobile applications. chine; modern mobile systems require coordination among MDDA suppresses sensor readings that are predictable ac- distributed components. ROS also comes with a non-trivial cording to some a priori or learned model. BraceForce sup- learning curve. ports (i) temporal models based on previous sensor read- Arduino has gained wide popularity due to simplicity, high ings [24]; (ii) models based on underlying physics princi- degrees of usability, and the resultant enabling of rapid pro- ples [21]; (iii) models derived from applying data mining to totyping. While programming for Arduino is more straight- prior sensed data [14]; and (iv) models expressing correla- forward, programmers are still required to interact with the tions of data in space and time [27]. While the use of MDDA Arduino at a very low-level of abstraction. This does not is itself not novel to BraceForce, what is new is that Brace- allow for flexible updating and discovery of local sensing Force provides existing libraries for a few MDDA models devices. The Android Sensor Framework [1] provides a pro- and allows customized MDDA models to be plugged into the gramming interface to access hardware and software sensing framework through an extensible programming interface. components on Android. This framework is also limited in This paper describes BraceForce, from its flexible archi- its abstract capabilities; the developer is still entirely re- tecture to a prototype implementation. We directly assess sponsible for thread control and other essential concerns for BraceForce’s ability to ease mobile application development accessing sensors. Further, the Android Sensor Framework through an exploratory study applied to real mobile applica- does not enable connecting to external sensors connected to tions that rely on networked sensors. BraceForce provides a the device via Bluetooth, USB or other connectivity modes. clear design contract for integrating new sensors into existing Instead, the developer must use other libraries. applications. This paper makes the concrete contributions Integrating sensing in mobile applications requires high in three areas: level abstractions to provide automatic sensor discovery and 1. Supporting mobile application developers and users flexible and efficient access to the sensor data via event- and model-driven data acquisition. BraceForce unifies access to ◦ BraceForce provides a simple and unified programming a variety of sensor platforms and provides a flexible and interface to reduce programming barriers of mobile ap- expressive programming interface with high-level program- plications that integrate sensing. ming constructs tailored to networked sensor integration. ◦ BraceForce’s tiered architecture enables dynamic up- Sensor Programming Frameworks. Easing program- dates to mobile applications and their physical deploy- ming of sensor networks has received significant attention. ments without intervention by the users. Mat´e[18], for example, is a tiny virtual machine that allows 2. Supporting flexible and expressive mobile applications developers to concisely express sensor programs and cause ◦ BraceForce can be deployed to heterogeneous devices these programs to be dynamically deployed. A virtual ma- in different configurations to meet constraints of mobile chine approach is very flexible, but Mat´ecomes with the cost applications and devices. of increased complexity due to the severe and unmitigatable ◦ BraceForce supports both pull- and push-based inter- resource constraints of the target environment [12]. action with sensing devices. SensorWare [4] shares the resources of a single node among ◦ BraceForce embraces model-driven data acquisition to many applications. SensorWare’s primary abstractions have address energy concerns of mobile applications. a network focus and are intuitive for sensor and networking 3. Studying mobile application development experts but do not promote data and data integration for application developers. MiLAN [11] builds on networking ◦ We demonstrate and evaluate how BraceForce makes and discovery protocols using a plug-in mechanism to incor- both the development process more approachable and porate arbitrary protocols. Application developers use QoS the resulting mobile application more efficient. graphs to specify their sensing requirements, and MiLAN ◦ We demonstrate that MDDA within BraceForce can sig- uses this information along with the sensor network state to nificantly reduce the network overhead, which in turn determine how to configure the network and sensors to meet reduces energy consumption. the provided requirements. This high-level data-centric ab- The next section places this paper’s contributions in the straction is the style we target in our middleware. We cou- context of related work. In Section 3, we provide complete ple these abstractions with automatic sensor discovery and details of the BraceForce middleware. Section 4 presents the a distributed architecture that, by its nature, addresses the empirical design for our evaluation, while Section 5 describes mixed resource constraints of our target environment. its results. Dandelion [19] supports developing wireless body sensor applications on smartphones using a programming abstrac- 2. RELATED WORK tion called a senselet, which abstracts a device driver and al- lows applications to integrate data from that device through Sensor Platforms. TinyOS [13] is an operating system remote method invocation. This abstraction is data-centric for sensor networks that enables developers to use networked and provides a jumping-off point for BraceForce’s abstrac- sensors to solve distributed sensing, computational, and co- tions, but Dandelion does not incorporate higher-level pro- ordinating tasks. Its focus has largely been on supporting in- gramming constructs for aggregation, model-driven data ac- creasingly complex and sophisticated applications at the ex- quisition, and automatic sensor discovery and integration. pense of usability and flexibility [17]. The Robot Operating Perhaps most similar to our goals, Open Data Kit (ODK) System (ROS) [23] provides a component-oriented style of Sensors [5] simplifies deployment of smartphone applications programming, in which components communicate via pub- that rely on data from external sensors (e.g., connected via lish/subscribe mechanisms and two-way service calls, both USB and BlueTooth). A sensor driver developer implements using user-defined topics. However, ROS only supports a specific driver interfaces for configuring sensors, packaging acting with a sensor, from configuration, through starting, the data they generate, and connecting that data into the querying, reconfiguring, stopping, and cleaning up allocated Android framework. Application’s interactions with sensors resources. The sensor driver layer requires driver developers are then confined to pull-based acquisition with only locally to adhere to a design contract enabling these common func- connected sensors. ODK Sensors does not support MDDA, tionalities and unifying on-board and external sensors into limiting its applicability. a shared data packet format that can be used throughout We focus on easing the integration of distributed sensor the BraceForce architecture. A driver developer creates the data into mobile applications without limiting the expres- connection between the sensor and the driver layer by defin- siveness of sensing capabilities or over-utilizing precious con- ing how raw sensor data (e.g., in the form of a binary array) strained resources. This demands not only automatic sensor is converted into a BraceForce sensor data response, which discovery and integration but also abstractions for acquiring defines an abstract data type unified across all sensor data. sensor data in ways other than through direct polling. Each sensor data response maps a data type (or types) to a Model-Driven Data Acquisition. Model-driven data value (or values) and associates a timestamp with the data. acquisition (MDDA) can limit costly communication with This timestamp ensures that data used by higher layers is networked sensors by suppressing polling and notifications fresh within the requirements of the application. from the sensors except when the sensor readings deviate from some pre-defined or learned model. Gupta et al. stud- ied the problem of collecting spatially correlated data in a wireless sensor network, based on the theory of dominating sets [10]. Other work has focused on achieving data sup- pression using temporal and spatial data correlation either Mobile)Applica:on)Node) by dividing the network into clusters [20], exploiting domain knowledge regarding reasonable ranges of sensed values [26], Distributed)Data)) Model)driven) or combining temporal and spatial correlations [27]. Other Cache)and)Aggregator) )))))data)acquisi:on) approaches use simple models of sensed value trends to gen- Distribu:on)Layer) D2D)Discovery) Distribu:on)Layer) erate readings only when the sensed value deviates from the model’s predicted value. Simple linear models are very effec- Data)Layer) Data)Layer) tive [24], and the approach can also be applied to in-network Model)driven) Model)driven) aggregation of raw values [8]. )))))data)acquisi:on) )))))data)acquisi:on) We are motivated to use similar techniques in BraceForce Sensor)Driver)Layer) Sensor)Driver)Layer) to suppress data without sacrificing the quality of knowledge about the sensed entity. BraceForce allows mobile applica- On#board) On#board) Driver) Sensors) Driver) Sensors) tion developers to incorporate different models of MDDA both at the level of individual sensors and at the level of an entire distributed application to handle different deployment scenarios and meet specific energy requirements of mobile applications. We avoid approaches that demand a particu- External) External) Sensors) lar topological structure (e.g., designated clusters of nodes) Sensors) to avoid unnecessary rigidity in a dynamic network of sup- porting sensors and rely on simple models that have shown Figure 1: BraceForce: Uniform Environment significant gains. Data Layer. Mobile operating systems allow sensors to 3. BRACEFORCE be accessed directly using Bluetooth, Near Field Communi- We first describe BraceForce from an abstract architec- cation (NFC), USB, and proprietary interfaces to on-board tural perspective; we then briefly discuss our prototype. sensors. Access is often device-specific and requires a non- trivial amount of low-level development and testing using 3.1 BraceForce Abstract Architecture interfaces that are not portable across platforms. These BraceForce comprises five abstraction layers: the sensor programming methods also usually entail a steep learning driver layer, the data layer, the distribution layer, the dis- curve. BraceForce encapsulates these interactions in the tributed data cache and aggregator layer, and the mobile ap- data layer, explicitly separating user-application code above plication layer. Fig. 1 shows one example BraceForce de- from sensor- and platform-specific code below. The data ployment with these five layers. layer also controls data retrieval for locally connected sen- sors (both on-board and external), where the options include 3.1.1 BraceForce Layered Architecture push, in which a sensor pushes every data reading to the data Sensor Driver Layer. In the current state of the art, mo- layer; pull, in which the data layer periodically polls the con- bile application developers must understand low-level proto- nected sensor; and event-driven, in which a sensor notifies cols and hardware-specific aspects of sensors to be able to use the data layer only upon the occurrence of some predefined sensor data. Moreover, raw sensor data has no open stan- “event.” At the data layer, BraceForce introduces single- dard and often loses its original temporal information. These device MDDA, whereby simple models of temporal correla- issues demonstrate a gap between the low-level sensing capa- tion can be used to suppress sensor readings as long as they bilities of both on-board and external sensors and the appli- follow an expected model [24]. cation space. At BraceForce’s base, the sensor driver layer Distribution Layer. Modern mobile applications require bridges this gap by encapsulating functions related to inter- a distributed view of active and available sensors, both on the local device and available remotely on other devices in tecture, depending on a particular device’s capabilities and the network. Programmers must use message passing mech- functional requirements. In any deployment, the sensor driver anisms or RPC-like connectors to make devices in the net- layer and the discovery layer must be present on any sensing- work communicate and coordinate their actions. Develop- capable device, as they are essential to connecting to and ers must also handle location transparency and other intrin- getting data from sensors. On moderately capable devices, sic issues related to distributed programming (e.g., concur- the addition of the data layer adds multiple modalities for rency, failures, and time synchronization). In the distribu- driving data acquisition and the potential for expressive tion layer, BraceForce provides automatic discovery of other temporal-based MDDA. Fig. 2 shows an alternative deploy- BraceForce-enabled devices, and thereby other BraceForce ment in which the system consists of heterogeneous devices; connectable sensors. BraceForce uses a combination of best in this example, the application runs on a dedicated high- effort protocols for the on-the-fly discovery and reliable pro- powered device (e.g., a laptop) that connects to a variety tocols for initiating and accepting remote method calls to of sensing devices of different capabilities, glued together by transfer sensor data from discovered sensors. The distri- the distribution layer. bution layer also extends the data acquisition modalities of the data layer by defining data listeners that bridge between the data layer below and the aggregation layer above and, more importantly, across distributed devices. One specific function of this bridge is to use guidance from cross-device Mobile)Applica:on)Node) MDDA performed at the aggregation layer (described next) to correctly configure and query underlying (distributed) Distributed)Data)) Model)driven) sensors. Within the distribution layer, BraceForce’s auto- Cache)and)Aggregator) )))))data)acquisi:on) matic detection of distributed sensors can be deployed along- side the Sensor Driver Layer, circumventing the data layer, Discovery)Ini:ator) simplifying the BraceForce deployment on devices that have D2D)Discovery) significant computation and storage constraints. Discovery)Target) Distributed Data Cache and Aggregator. Mobile appli- Discovery)Target) cations that require large scale deployments often include Sensor)Driver)Layer) sensing devices that have severe computation and storage Data)Layer) Model)driven) constraints. In this case, it is highly ideal that data storage )))))data)acquisi:on) On.board) and processing are done in backend servers or other more Driver) Sensors) powerful devices in the network. Programming these fea- Sensor)Driver)Layer) tures in a scalable and reliable way is a challenge for mo- On.board) bile application developers. BraceForce’s aggregation layer Driver) Sensors) is specifically designed to deal with this challenge through External) a unified view of the dynamic sensing capabilities of the Sensors) networked environment. As new sensors are discovered on- the-fly or known sensors disappear, these dynamics are han- dled seamlessly by BraceForce. Given a distributed view External) of aggregate sensing capabilities, the aggregation layer can Sensors) perform high-level MDDA, for example, by using more so- phisticated models based on spatial correlations or learned Figure 2: BraceForce: Mixed Environment relationships among sensed data [22]. In our prototype, we demonstrate the potential for this high-level MDDA using We envision three primary use cases for BraceForce. In the a spatial correlation model that suppresses sensor readings obvious case, BraceForce is integral to the mobile applica- that are similar to neighboring readings. More generally, tion. Take a smart home application in which a home con- from the aggregation layer, BraceForce can expressively di- troller connects to various sensing devices integrated with rect data acquisition (e.g., by choosing between event-driven the home to control the ambient environment. In this case, data acquisition, push, and pull) for all of the sensors under the homeowner may introduce and remove components from its purview. This layer also encapsulates data mining and the home over its lifetime, and, using BraceForce, the chang- compression. ing capabilities are seamlessly reflected to the application. Mobile Application Node. To support the mobile appli- As a second case, we envision BraceForce deployed for cation, BraceForce maintains a registry of aggregation and large scale ad-hoc sensor networks (e.g., to monitor a fire in sensing capabilities, and the programming interface allows a forest). The Sensor Driver Layer and Discovery Layer can the application to subscribe to them. Based on the appli- be combined into a service to be deployed on cheap sensor cation’s goals, these sensors can be integrated as part of a devices with basic sensing and networking capacity and lim- testing harness at debugging time or can provide essential ited computation and storage resources. These devices can functional information at deployment time. be deployed at a mass scale. The Distributed Data Cache and Aggregator can be deployed to more powerful Android 3.1.2 BraceForce Deployment Scenarios devices or a cluster of backend servers for data processing. Fig. 1 showed just one possible deployment scenario in The sensor data then will be passed to the mobile appli- which all of the devices are homogeneous (e.g., smart phones) cation node, which runs on user-facing Android devices to and have the same capabilities. In BraceForce, different notify users (e.g., first responders) of the real-time situation physical assets can support different pieces of the archi- to allow them to take informed actions. As a third case, consider an autonomous robot application Android provides sensor data subscription only for inter- wrapped in a test harness that connects to expressive sensing nal sensors. To subscribe for sensor data from external sen- capabilities available in the debugging environment but not sors (e.g., sensors connected via USB), developers have to available in the deployment environment. At deployment create a subscription model themselves (e.g., using the ob- time, the robot may be placed in an unknown territory and server design pattern) and write a large amount of low-level expected to perform some tasks. At debugging time, how- code to access sensor data. Retrieving data from other de- ever, the developer may control the environment and may vices is even more difficult. In BraceForce, we provide a be able to monitor various aspects of the robot using sen- unified subscription interface for developers to access inter- sors embedded in the environment (e.g., overhead cameras). nal sensors, external sensors, and networked sensors; this The debugging program that surrounds the actual control abstraction allows generic ”plug-and-play” of sensors at run- application can use BraceForce to access these sensors for time. Fig. 3 shows the BraceForce API used to subscribe debugging; at deployment time, these connections to Brace- sensed data from an internal accelerometer (line 1), a tem- Force and the debugging environment are removed. perature sensor (Dallas DS18B20) connected via USB (line 2), and all accelerometers on networked devices (line 3). 3.2 BraceForce Implementation Line 4 shows how to retrieve sensor data; it is the same To demonstrate and evaluate BraceForce, we have built regardless of the type of sensor connection. The retrieved the entire architecture in the Android operating system. We data contains a timestamp of the data and where the data choose Android as our initial platform as it is open source is from; data values are accessed through meaningful keys and has extensive support for background processes, includ- instead of array indices provided by Android. ing several built-in constructs for inter-application commu- 1 BraceSensorManager. subscribeSensorEvent( nication [25]. BraceForce is not particular to Android, and BuiltInSensorType .ACCELEROMETER. name() , this , we avoid using low-level constructs and interfaces specific to this ); Android and not replicated on other platforms. 2 BraceSensorManager. subscribeSensorEvent(”DS18B20”, this , this , SensorAutoDetection.USB ); 3 BraceSensorManager. subscribeSensorEvent( 3.2.1 Android Programming Idiosyncrasies BuiltInSensorType .ACCELEROMETER. name() , this , A handful of accidental complexities arise from our choice this , true ); of Android; below, we describe these challenges and our so- 4 BraceSensorManager. retrieveSensorData(event); lutions as they arise. In addition, to understand the dis- Figure 3: Interface for data subscription cussions of the architecture, we briefly review a vocabulary related to Android: 3.2.3 Thread Management and Android services Intent: a passive data structure holding an abstract de- Android provides much support for multi-threaded appli- scription of an operation to be performed; something that cations. As with any multithreaded environment, the added has happened and is being announced. flexibility comes with a significant increase in complexity. Android Interface Definition Language (AIDL): allows In Android applications that interface with sensing, devel- definition of the programming interface that the client and opers must implement (and debug) threads to listen for and service use to communicate with each other using inter- connect to multiple connections. These tasks are far from process communication (IPC). trivial, as they require the developer to have a deep under- standing of the relationships between the operating system Bundle: a data structure of key-value mappings but not and the life cycles of application components. As our user limited to a single String/Object mapping. study (described in Section 5) demonstrates, developers have Android does not provide significant high-level abstrac- a difficult time navigating the complexities of the Android tions for programming coordination among distributed de- APIs related to thread management and concurrency. vices. From a model perspective, BraceForce assumes such BraceForce exposes thread managers that explicitly en- capabilities, e.g., in the form of Java RMI. To support our able thread-safe access to the shared sensor data available to BraceForce prototype on Android, we therefore implement the mobile applications. To enable communication beyond our own version of RMI by building on several open source shared memory, we expose the threads and their embodied projects. We use JsonBeans [15] to serialize and deserialize data using the Android Binder service’s remote procedure Java object graphs to and from JSON [6]. JsonBeans is an call capabilities. To conserve energy, BraceForce activates obvious choice for this task because it is very lightweight threads only when necessary. This is guided by developers (45KB) with no external dependencies. We use ASM [2, 16] through the interface in Fig. 3, e.g., by specifying the types to dynamically generate classes involved in the RMI process of communication interfaces to attach to event listeners. For in binary form. example, in Fig. 3, line 2 specifically indicates that Brace- Force should activate the USB thread to communicate with 3.2.2 Unified Data and Subscription Interfaces the connected temperature sensor. Alternatively, a devel- Programming for Android requires interacting with the oper with less experience in low-level details can indicate, Dalvik virtual machine, and it is inevitable that our im- via a boolean parameter, activation of all the networking plementation accesses some proprietary Android constructs threads, albeit at increased cost. (e.g., Bundle). To remain as general as possible, we imple- ment a wrapper that encapsulates these proprietary compo- 3.2.4 Networking nents and presents a generic interface to the BraceForce im- Our prototype supports simple device-to-device discov- plementation. Within this wrapper, BraceForce translates ery based on UDP. The relevant threads for discovery and Android data structures to reusable and portable Java data exposed Android Binder services are built in the distribu- structures. tion layer. When each discoverable service is started within BraceForce, the distribution layer implementation creates a BraceForce SensorDataResponse, and make it available for new Android intent and attaches the discovery capability to the higher layers of the architecture. The manager’s inter- that intent. When the discovery layer starts a service, it face allows the higher level (and ultimately the application listens on the UDP port specified in its intent and reacts to developer) to specify how BraceForce should interact with discovery requests received on this port. The discovery layer the sensor. In the case of push-based interactions or event- also actively engages in discovery by sending UDP packets driven data acquisition, the application developer defines the to neighboring devices. Each node maintains an active list behavior of a listener, as shown in Fig. 5. of other nodes; this list is maintained by removing any nodes that have failed to respond to three consecutive periodic re- 1 public interface SensorDataChangeListener { 2 void bindDataProvider(SensorDataProvider quests. Using UDP for discovery is a simple approach to p r o v i d e r ) ; enable automatic device discovery. BraceForce could poten- 3 void onDataChanged(SensorDataChangeEvent event) ; tially integrate other on-the-fly discovery mechanisms in the 4 } 5 public interface SensorDataProvider { distribution layer, e.g., [3], so that devices can talk to each 6 L i s t