Building Ubiquitous Robot Systems M. Broxvall, A. Saffiotti Oktober 29th, San Diego Supported by: Electronics and Telecommunications Research Institute, Korea The Swedish Research Council (Vetenskapsrådet), Sweden Knowledge Foundation, Sweden Outline ● Theoretical – Concepts and definitions – A case study: The PEIS Ecology – Challenges in Ubiquitous Robotics ● Practical using the PEIS Ecology – Demonstrator environments – Existing components and their use – Dissecting an experiment – Using the reference middleware (OpenSource) – Other issues Ubiquitous Robot Systems ● Classical robotics: robot and environment as two distinct entities ● Ambient Intelligence – Distribute computational capability and intelligence in the environment ● Ubiquitous Robotics and the Ecologies of PEIS1 Project – Combine ambient intelligence with AI and classical robotics. Artificial Intelligence Autonomous Ambient Robotics Intelligence Ubiquitous Robotics Computing Sensor Networks 1PEIS is pronounced [pace]. Related Fields Ambient Autonomous Artificial Intelligence Intelligence Robotics Ubiquitous Robotics Computing Sensor Networks Related Fields Ambient Autonomous Artificial Intelligence Intelligence Robotics Ubiquitous Robotics Computing Aims at the creation of a universal robot, but this is beyond our current capabilities A PEIS Ecology exploits the interaction of many connected specialized devices to perSfoernms oar t aNsektworks Related Fields Ambient ConAnuetcto nwiodemo aurras ys of Artificial Intelligence Intelligence small devicesRobot forics remote sensing, but they do not deal with cognition and (complex) actuation A PEIS Ecology can perform tasks that require Ubiquitous Robotics cognition and physical Computing action Sensor Networks Related Fields Ambient Autonomous Artificial Intelligence Intelligence Robotics Ubiquitous Robotics Computing It is mainly an information technology, but it will not help us to perform full physical tasks A PEIS Ecology will bring physical action and interaction into the user’s environment Sensor Networks Scenario Johanna is 72 years old. She lives in a small house. Before she wakes up, her fridge realizes that something is smelling bad. Pippi, a robot carrying a capable artificial nose, goes to the fridge and performs a careful inspection. The milk is found to be bad, and the autonomous thrashcan is called to fetch it. Soon after, the coffee machine turns on. Johanna’s personal robot, Emil, brings her a cup of coffee. It also tells her about the milk, and asks if she would like the trolley to go out and buy a new bottle. Keypoints in scenario ● Many specialized robotic devices – Autonomous trolley, cleaning robot, monitoring cameras – Highly heterogeneous ● Devices communicate and cooperate – Functionality and intelligence emerge from cooperation ● Assistive task performed by the whole house – As opposed to building one single “super-robot” What is ubiquitous robot systems? ● Sensing and Acting (robotic) components distributed ubiquitously in an intelligent living environment – Provide users the right services at anytime and anywhere ● Illustrated by the Ecologies of PEIS project – A ubiquitous robot system consisting of a number of Physically Embedded Intelligent Systems – OpenSource middleware and components aiding the development of ubiquitous robot systems ● Additional requirements such as distributed, decentralized intelligence and scalability of devices. Not required for all ubiquitous robotic systems Outline ● Theoretical – Concepts and definitions – A case study: The PEIS Ecology – Challenges in Ubiquitous Robotics ● Practical using the PEIS Ecology – Demonstrator environments – Existing components and their use – Dissecting an experiment – Using the reference middleware (OpenSource) – Other issues The PEIS Ecology approach ● Uniform notion of “robot” – PEIS = Physically Embedded Intelligent System ● Uniform communication model PEIS – Distributed tuple space ● Uniform cooperation model – Lending functionalities to one another ● PEIS Ecology – A system consisting of a set of cooperating PEIS PEIS: Physically Embedded Intelligent System ● PEIS Component Input Output – Software module – Implements a functionality Sensor Actuator – Has input/output ports Environment – Can have sensors/actuators ● PEIS – A set of PEIS-Components M D – Includes sensing and/or actuation – Resides in one physical entity P C Environment PEIS Ecology ● A collection of PEIS – that are embedded in the same environment – that can borrow capabilities from each-other Eg. A camera tracking system Eg. An autonomous vacuum cleaner Eg. A camera tracking system localizing an autonomous vacuum cleaner EnvironmentEnvironment Environment A Simple PEIS Ecology ● An autonomous vacuum cleaner ● An overhead monitoring system ● An unknown box Where am I? Can I push you away? A Simple Ecology ● An autonomous vacuum cleaner ● An overhead monitoring system ● An unknown box weight The parcel The vacuum cleaner The tracking system M D M D M D P C P C P C location Environment The PEIS home ● Testbed environment built-up to test concepts and implementations. ● Implemented hardware and software components – Refrigerator – Cameras – Mobile robots – RFID-tagged floor – Electronic noses – Actuated window blinds, lamps – plants ... PEIS Ecology Example ● Example run in an implemented PEIS Ecology Benefits of a PEIS Ecology ● Technical – by-pass hard problems of todays robots – perception & action replaced by direct communication – robustness from redundancy The parcel shape = {.......} M D The vacuum cleaner The tracking system M D M D P C P C P C Environment Benefits of a PEIS Ecology ● Technical – by-pass hard problems of today robots – perception & action replaced by direct communication – robustness from redundancy ● Pragmatical – can be easily customized – can be built and upgraded incrementally – easier to accept and afford than a full powerful robot ⇒ Smooth path to introduce robotic technologies into everyday life Requirements on a PEIS Ecology ● Hardware and software – Sensors and actuators distributed in environment – Processors running software components ● Mechanism implementing PEIS-components – PEIS to PEIS communication – Component to component communication ● Collaborations – Sharing data, requesting functionalities – Creating configurations – Providing semantic information Outline ● Theoretical – Concepts and definitions – A case study: The PEIS Ecology – Challenges in Ubiquitous Robotics ● Practical using the PEIS Ecology 1. Heterogeneity – Demonstrator environments 2. Link physical and digital world 3. Self-configuration – Existing components and their use – Dissecting an experiment – Using the reference middleware (OpenSource) – Other issues Heterogeneity Where am I? Can I push you away? ● Very different devices and functionalities PEIS ● 3G Different platforms n a Z L i W g b – IR e processing hardware / software e R IR F ID – communication protocol / media ● Dynamic world PEIS can join and leave the ecology at any time ⇒ need middleware for integration in a common ecology The PEIS Middleware ● Provides uniform communication layer – Hides platform and network heterogeneity ● Manages all interactions – All PEIS components communicate via the middleware ● Implemented in the “PEIS Kernel” – All PEIS components must use it Component processes PEIS Kernel OS CPU PEIS Middleware: Example Where am I? Can I push you away? PEIS Middleware: Example The Parcel M D The Monitoring System The Vacuum Cleaner P C M D M D P C P C Environment PEIS Middleware: Example Distributed Tuple-Space The Parcel M D The Monitoring System The Vacuum Cleaner P C M D M D P C P C Environm ent PEIS Middleware ● The PEIS Kernel – Communication model: shared tuple-space + events – Distributed over ad-hoc P2P network – Run-time C library (open source) nts e n PTL o Meta layer P mp Vision TC EI o Semantic discovery C S Nose Player S Configurator M P EI i d EI P Tuplespace layer Distribution d S l Topology e K Subscriptions w e a r r Communication layer n e e Network communication l P2P Discovery Hardware & OS PEIS Middleware: Communication model ● Through the exchange of tuples < PEIS-ID, Component-ID, Key, Value(s) > ● Tuples live in a distributed tuple-space – Shared by all PEIS components – Physically distributed across PEIS – <PEIS-ID,Component-ID> as a common address space ● Communication triggered by events – PEIS subscribe to tuples by content – Subscribers notified when a matching tuple is inserted/updated PEIS Middleware: Communication model Subscribe: *.*.roomba-at = * Insert: cam1.track.roomba-at = (3,5) Notify: cam1.track.roomba-at = (3,5) ... ... Insert: cam2.track.roomba-at = (1,-2) Notify: cam2.track.roomba-at = (1,-2) Time PEIS Middleware Meta-level components ● EIS The P• PEIS initKernelialization – Co•m semanmunictica advertion tismemenodelts: sandha rdisedco tvuepryle-space + events Hardware• c/ onOfSigur poarttoarsbility Linu–x (rDobotis• .ts..r andibu tPCed) over ad-hoc P2P network MacOS (home PC) – Run-time C library (open source) OpenR nts TinyOSe n PTL ... o Meta layer P mp Vision TC EI o Peis-Init Semantic discovery C S Nose Player S Configurator M P EI i d EI P Tuplespace layer Distribution d S l Topology e K Subscriptions w e a r r Communication layer n e e Network communication l P2P Discovery Hardware & OS Outline ● Theoretical – Concepts and definitions – A case study: The PEIS Ecology – Challenges in Ubiquitous Robotics ● Practical using the PEIS Ecology 1. Heterogeneity – Demonstrator environment 2. Link physical and digital world 3.