Extending a Networked Robot System to Include Humans, Tiny Devices, and Everyday Objects To the women of three generations: my mother Rezia Rashid, wife Mari Rashid, and our sweet little girl Marissa Ida Rashid. Örebro Studies in Technology 47 Md. Jayedur Rashid Extending a Networked Robot System to Include Humans, Tiny Devices, and Everyday Objects © Md. Jayedur Rashid, 2011 Title: Extending a Networked Robot System to Include Humans, Tiny Devices, and Everyday Objects. Publisher: Örebro University 2011 www.publications.oru.se [email protected] Printer: Intellecta Infolog, Kållered 03/2011 issn 1650-8580 isbn 978-91-7668-792-5 Abstract In networked robot systems (NRS), robots and robotic devices are distributed in the environment; typically tasks are performed by cooperation and coordi- nation of such multiple networked components. NRS offer advantages over monolithic systems in terms of modularity, flexibility and cost effectiveness, and they are thus becoming a mainstream approach to the inclusion of robotic solutions in everyday environments. The components of a NRS are usually robots and sensors equipped with rich computational and communication facilities. In this thesis, we argue that the capabilities of a NRS would greatly increase if it could also accommodate among its nodes simpler entities, like small ubiquitous sensing and actuation devices, home appliances, or augmented everyday objects. For instance, a do- mestic robot needs to manipulate food items and interact with appliances. Such a robot would benefit from the ability to exchange information with those items and appliances in a direct way, in the same way as with other networked robots and sensors. Combining such highly heterogeneous devices inside one NRS is challeng- ing, and one of the major challenges is to provide a common communication and collaboration infrastructure. In the field of NRS, this infrastructure is com- monly provided by a shared middleware. Unfortunately, current middlewares lack the generality needed to allow heterogeneous entities such as robots, sim- ple ubiquitous devices and everyday objects to coexist in the same system. In this thesis we show how an existing middleware for NRS can be ex- tended to include three new types of “citizens” in the system, on peer with the other robots. First, we include computationally simple embedded devices, like ubiquitous sensors and actuators, by creating a fully compatible tiny version of the existing robotic middleware. Second, we include augmented everyday ob- jects or home appliances which are unable to run the middleware on board, by proposing a generic design pattern based on the notion of object proxy. Finally, we go one step further and include humans as nodes in the NRS by defining the notion of human proxy. While there exist a few other NRS which are able to include both robots and simple embedded devices in the same system, the use I II of proxies to include everyday objects and humans in a generic way is a unique feature of this work. In order to verify and validate the above concepts, we have implemented them in the Peis-Ecology NRS model. We report a number of experiments based on this implementation, which provide both quantitative and qualitative evaluations of its performance, reliability, and interoperability. Acknowledgements First, I would like to thank my primary supervisor Dr. Mathias Broxvall and co-supervisor Professor Alessandro Saffiotti for giving me the opportunity to conduct my PhD study at AASS. They have supported me by their remark- able supervision, valuable suggestions and all fruitful discussions during this research study. They were always there whenever I needed them despite of their very busy schedule. Thanks to Vetenskapsrådet (the Swedish Research Council) for financing this work and CUGS (the Swedish National Graduate School in Computer Sci- ence) for allowing me to participate several courses. Two person at AASS, Bo-Lennart Silfverdal and Per Sporrong, have made my life, as a PhD student, easier by their experience and knowledge of designing electronic equipments. All customized devices, which have been used in this thesis for different experiments, are prepared by their help. Many thanks to Bo and Per. Barbro Alvin, Jenny Tiberg, Kicki Ekberg and Elin Abelson deserve thanks for helping me to sort out all administrative works and real-life matter such as renting apartment, arrangement for traveling to different scientific events, etc. different times during my staying here. A special thank goes to Abdelbaki who has been my mentor since the day I started at AASS. He has given me his valuable time to adapt at AASS as well as at Örebro city. A lot of weekends we have spent together and explored the city which I shall always remember. Thanks to all my colleagues (former and present) at AASS for accepting me as one of their members and giving me a lot of time at different occasions. Spe- cial thanks to Federico (Pex) for taking a quick look of the thesis draft, office- mate Johan Larsson for reading several of my pre-submitted articles, Achim Lilienthal for arranging several sporting events, the Italians – Marcello, Matteo and Marco for joining me at as well as arranging many social events, Robert Lund, Henrik, Luigi, Rehan, Mitko and Fabian for playing badminton with me. I express my special gratitude to my best friend and wife, Mari Teresa Rashid (former name Mari Teresa Vähäkuopus) who has inspired me every III IV moments as well as has to sacrifice many hours due to my late presence at home. Last but not the least, thanks to my family in Bangladesh, particularly my mother, eldest sister Feroza Begum and her husband Md. Enayet Ullah, for their unconditional love, prayer, moral support and inspiration. Contents 1 Introduction 1 1.1 Motivation . .2 1.2 Research Questions . .5 1.3 Goals of The Thesis . .5 1.4 Methodology . .6 1.4.1 Step 1: Including Tiny Embedded Devices in a NRS . .7 1.4.2 Step-2: Including Everyday Objects in a NRS as Proxies .8 1.4.3 Step-3: Including Humans in a NRS . .8 1.4.4 Evaluation . .9 1.5 Publications . 10 1.6 Thesis Outline . 11 2 Related Work 13 2.1 Network Robot Systems (NRS) . 13 2.1.1 Representative NRS Projects . 15 2.2 Wireless Sensor Network (WSN): Ubiquitous Sensing Technology 19 2.3 Middleware . 20 2.3.1 Classical Middleware for Distributed Systems (DS) . 21 2.3.2 NRS Middleware . 21 2.3.3 WSN Middleware . 28 2.4 Including Tiny Devices (WSN nodes) in NRS . 30 2.4.1 Existing Approaches to Accomodate WSN into NRS . 31 2.4.2 Reconfiguration in Hybrid Middlewares Consisting of NRS and WSN Nodes . 32 2.5 Smart Everyday Objects . 33 2.5.1 Standalone Smart Objects . 34 2.5.2 Network of Smart Objects . 35 2.5.3 A Missing Link . 37 2.6 Including Everyday Objects (Smart/Dumb) in NRS . 37 2.6.1 Using Cognitive Robotics Techniques . 38 2.6.2 Using Full-Scale Processors . 38 V VI CONTENTS 2.6.3 Using Tiny Devices . 38 2.6.4 Using Teeny Devices . 39 2.7 Including Humans in NRS . 40 2.7.1 No Interaction with the System . 40 2.7.2 Humans Interact with the Robot . 40 2.7.3 Humans Interact with the Environment . 41 2.7.4 Teams of Humans and Robot . 42 2.8 Discussion . 43 3 Background: The Peis-Ecology NRS 45 3.1 The History and Concept . 45 3.2 What The Peis-Ecology Offers . 49 3.3 The Peis-Ecology Realization . 50 3.4 Shared Memory Model and Configuration . 51 3.5 The Peis-Middleware . 53 3.5.1 The PeisInit . 55 3.5.2 Distributed Tuplespace . 57 3.5.3 The Peis-Kernel . 59 3.6 The Peis-Home: Peis-Ecology Testbed . 61 3.7 What’s Missing . 63 4 Including Tiny Devices 65 4.1 Requirements for Including Tiny Devices . 65 4.1.1 Tiny Devices . 66 4.1.2 Requirements from the Tiny Device’s Constraints . 67 4.1.3 Requirements for the Interoperability . 67 4.2 Approaches to Include Tiny Devices . 68 4.2.1 Traditional NRS Approach . 68 4.2.2 Wireless Sensor Network (WSN) Approach . 68 4.2.3 Downgrade Approach . 69 4.2.4 The “Twin-Version” Approach . 69 4.2.5 The Approach Demonstrated in This Thesis . 70 4.3 Requirements for the Peis-Ecology NRS Implementation . 71 4.3.1 Requirements Posed by the Peis-Ecology NRS . 71 4.3.2 Requirements Posed by the Tiny Network Protocol . 71 4.4 Tiny Peis-Kernel and Tiny Tuple . 72 4.4.1 Assumptions for Platform and the Tiny Kernel . 72 4.4.2 The Tiny Peis-Kernel . 74 4.4.3 Tiny Tuple . 75 4.5 Network Translator: The Tiny-gateway . 77 4.6 Tiny Meta-Tuple: Dynamic Reconfiguration of Tiny Devices . 79 4.7 An Example Tiny Kernel Implementation on the TinyOS . 81 4.8 The Tiny-gateway Implementation . 87 4.9 Physical Realization . 90 CONTENTS VII 4.10 Use Case: Sharing Functionalities between Full-Peis and tiny-Peis 91 4.11 Handling Mobility . 93 4.12 Discussion . 93 5 Including Everyday Objects 95 5.1 Approach to Include Everyday Objects . 96 5.1.1 Traditional Approach . 96 5.1.2 Our Approach . 97 5.2 Requirements . 98 5.3 The Design Pattern: Concept of Proxies . 99 5.3.1 Proxied-Objects: . 99 5.3.2 Interface . 100 5.3.3 Proxy . 100 5.3.4 Proxy Manager . 103 5.3.5 Signature . 103 5.3.6 Conceptual Examples . 104 5.4 Peis-Ecology Implementation . 105 5.5 Use Cases . 109 5.6 Discussion . 114 6 Including Humans 117 6.1 Approach to Include Humans .