The Penetration of Internet of Things in Robotics: Towards a Web of Robotic Things

Home , Home automation

Journal of Ambient Intelligence and Smart Environments 12 (2020) 491–512 491 DOI 10.3233/AIS-200582 IOS Press The penetration of Internet of Things in robotics: Towards a web of robotic things

Andreas Kamilaris a,b,* and Nicolò Botteghi b a Research Centre on Interactive Media, Smart Systems and Emerging Technologies (RISE), Nicosia, Cyprus E-mail: [email protected] b Dept. of Electrical Engineering, Mathematics and Computer Science, University of Twente, Enschede, The Netherlands E-mail: [email protected]

Abstract. As the Internet of Things (IoT) penetrates different domains and application areas, it has recently entered also the world of robotics. Robotics constitutes a modern and fast-evolving technology, increasingly being used in industrial, commercial and domestic settings. IoT, together with the Web of Things (WoT) could provide many beneﬁts to robotic systems. Some of the beneﬁts of IoT in robotics have been discussed in related work. This paper moves one step further, studying the actual current use of IoT in robotics, through various real-world examples encountered through a bibliographic research. The paper also examines the potential of WoT, together with robotic systems, investigating which concepts, characteristics, architectures, hardware, software and communication methods of IoT are used in existing robotic systems, which sensors and actions are incorporated in IoT-based robots, as well as in which application areas. Finally, the current application of WoT in robotics is examined and discussed.

Keywords: Internet of Things, robotics, robots, sensors

1. Introduction – Mechanics, i.e. motors, pistons, grippers, wheels and gears that make the robot move, grab, turn, Robots are increasingly being used in industrial, and lift. commercial and domestic applications, as well as for – Sensors, i.e. sensing equipment that helps the rescue operations where there are safety risks for hu- robot perceive its surroundings. mans [126]. Robot comes from the Czech word robota Robotics constitutes a modern and fast-evolving which means forced work or labour. The word robot technology [106]. Robotics can be deﬁned as the today means any man-made machine that can perform branch of engineering that involves the conception, de- work or other actions normally performed by humans, sign, manufacture and operation of robots [33]. Al- either automatically or by remote control. Robots are though robots have been mostly used in industrial ap- employed because it is often cheaper to use them over plications till date, recent technological progress in the humans, easier for robots to do some job and some- emerging domains of cognition, manipulation and in- times the only possible way to accomplish some tasks. teractions is moving the robotics industry toward ser- An example of a robot with a human-like shape is pro- vice robots and human-centric design [136]. vided in Fig. 1. Most robots are composed of the fol- The Internet of things (IoT) [133] is the extension lowing components: of Internet connectivity into physical devices and everyday objects. In this context, physical things become – A controller, i.e. the brain of the robot. uniquely addressable and interconnected through the Internet, providing support for the IPv4 or (more re- *Corresponding author. E-mail: [email protected]. cently) the IPv6 protocol [79]. The Internet Protocol

As the IoT and the WoT penetrate different domains, application areas and scientific disciplines [80], it is worth the effort to examine their impact, interaction and application in the emerging research area of robotics, as they transition from industrial applications into the everyday lives of people in home settings, buildings, commercial centres, communities, air- Fig. 1. An example robot pouring some liquid from one cup to an- ports, supermarkets, etc. other (source: [106]). The interconnection and relationship between the IoT and robotics has been defined as Internet of (IP) is the principal communications protocol in the In- Robotic Things (IoRT) [96,108,125]. IoRT has been ternet protocol suite. IPv6 is the most recent version of defined as a a global infrastructure enabling advanced the IP, intended to replace IPv4 by offering support for robotic services by interconnecting robotic things trillions of devices connected to the Internet. based on existing and evolving interoperable informa- Similar to the IoT, the Web of Things (WoT) [39, tion and communication technologies such as cloud 134], is about approaches, software architectural styles computing, cloud storage and other existing Internet and programming patterns that allow real-world ob- technologies. IoT allows robots to communicate by jects to be part of the World Wide Web. Via the WoT, means of the IP protocol, especially its IPv6 version, physical things have access to the Web, interacting which is designed for billions of Internet-connected through services and application programming inter- objects [49]. Internet connection permits updating in- faces (APIs) based on the Hypertext Transfer Protocol formation (and possibly the robot’s firmware) in real- (HTTP). HTTP is the foundation of data communica- time [122], storing/processing data on the cloud and tion for the World Wide Web, where hypertext doc- taking advantage of Internet protocols for security, uments include hyperlinks to other resources that the authentication, data integrity, message routing, etc. user can easily access. Enhanced with existing Web [110,114]. services, physical devices offer advanced features and The main motivation for preparing this paper stems intelligence, such as smart umbrellas that warn the user from the fact that related work in this field [1,21,38, when it is about to rain or kettles that heat water when 96,108,125] has discussed only generally some of the the electricity tariff is low [124]. This new possibil- opportunities of IoT in robotics. There is a gap in ity of blending physical sensor-based capabilities and the literature in relation to the actual degree of pen- functionality, together with real-time Web services and etration of IoT in robotic systems and services, as APIs has been defined as physical mashups [40]. well as how IoT has been used in robot systems till Numerous research areas and application domains date. have been influenced in the past by the introduction of One might argue that robots constitute just some IoT and WoT. These areas include home automation extra “things” in the IoT ecosystem. However, robots [61,113], smart buildings [18], smart grid [23], remote constitute complex systems with a wide range of tech- healthcare [137], agriculture [56], as well as a limited nologies, behaviours and components, which makes number of smart city applications [58,59]. their research domain quite unique. Nevertheless, the Examples of this influence by IoT/WoT involve re- two research fields of robotics/IoT have started inter- duced risks of vendor lock-in, adopting machinery (i.e. acting in the last decade, linking the research com- for agriculture) and sensing/automation systems from munities together in many aspects, creating a new in- different companies, as these could become easily in- terdisciplinary field. According to [1], IoRT is at a teroperable in the overall smart systems, easier data ex- level more advanced than IoT, due to the need to in- change among different, heterogeneous components, tegrate numerous modern technologies together, in or- increased automation with less effort by means of In- der to satisfy the needs and requirements of high- ternet and Web standards etc. Use of open standards tech robotic systems that operate beyond the industrial brings seamless connectivity and advanced interoper- level. ability, while the publishing of data produced by IoT Moreover, we argue that IoRT should also em- sensors as open data on the Web promotes knowledge brace the WoT, which has not yet been considered in sharing and is important for the advancement of re- the survey papers mentioned above. WoT could of- search in these fields. fer additional benefits to the IoT-enabling of robotics- A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 493 based systems, especially in terms of higher interop- 2. Related surveys erability among the robot’s sensory components, but also robot-to-robot (R2R) and robot-to-human (R2H) As stated before, related work in this field [21,38,96, communication at the application layer. Additionally, 108,125] has not discussed elaborately the use, charac- WoT could facilitate the combination of Web ser- teristics, developments, technologies and opportunities vices and robotic services, towards physical robotic of IoT in robotics. mashups, realizing the notion of a Web of Robotic In particular, the work in [108] focuses on the lower Things (WoRT). These potential benefits are discussed and higher-level abilities of IoT-enabled robots, while in Section 6, in more detail, together with a small Vermesan et al. [125] discuss generally some technolo- historical journey on the first robots that had pres- gies of IoT that can support robotic systems. Ray [96] lists some examples of existing robots envisaged for ence and allowed remote control via the Web, back in an IoRT architecture, but the linking between the ex- 1995. amples mentioned in the paper with the IoT is un- Therefore, the contribution of this paper is to study clear and not convincing. The differences between this the current use of IoT in robotics, through various real- survey and the relevant ones are summarized in Ta- world examples encountered through bibliography- ble 1. based research and to examine the potential of the WoT Further, the current application and future potential in robotic systems. To the authors’ knowledge, it is the of WoT in robotics have not been discussed in any rel- most complete survey paper to date, aiming to present evant survey paper, except from [38], where the aspect the actual research taking place at the intersection of of semantic consensus has been generally discussed, these research domains. suggesting that this issue could be approached via Se- The rest of this paper is organized as follows: Sec- mantic Web technologies. tion 3 describes the methodology adopted and Sec- tion 4 introduces the concept of robotics, clarifying its relationship with IoT. Section 5 presents the existing 3. Methodology penetration of IoT in robotics, while Section 6 studies the connection between WoT and robotic systems. Fi- This paper aims to fill the aforementioned gaps in nally, Section 7 discusses the overall findings and Sec- the literature, by addressing the following research tion 8 concludes the paper. questions:

Table 1 Differences between this survey and related ones Characteristic/Paper [96][125][108][21][38] Our paper Review of papers IoRT-based Only general Only general Only general IoRT-based demonstrating robots ones ones Description of IoRT X Focus on Only cloud IoT security, hardware X Technologies software robotics platforms platforms and interoperability Linking between robots, Crossover of Link with Only general X sensors, actions and IoRT into nine applications applications robotic system abilities Linking of robots with IoRT Link between IoRT X technologies technologies and EU projects Research challenges X Only general X Presenting the whole picture XX X around IoRT Linking of robots with WoRT Semantic consensus X technologies towards Semantic WoT 494 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

1. Which concepts, characteristics, architectures, platforms, software, hardware and communication standards of IoT have been used by existing robotic systems and services? 2. Which sensors and actuators have been incorporated in IoT-based robots? 3. In which application areas has IoRT been applied? 4. Which technologies of IoT have been used in robotics till today? Fig. 2. A Venn diagram showing research areas involved. 5. Has WoT been used in robotics? If yes, by means of which technologies? and WoT, aiming to close the existing gap in the lit- 6. Which is the overall potential of WoT in combi- erature, as described in the introductory section. This nation with robotics, towards a WoRT? paper does not intend to compare specific hardware As the IoT has been defined in different ways in lit- and software platforms used for sensors/robots devel- erature, it must be clarified that IoT – in the context of opment and/or development of IoT-based systems. For this paper – is perceived as a system that involves real- such comparisons, the reader should consider relevant world things, which can communicate/interact over the studies [20,34,55,78,98,125,136], which cover quite Internet and they can be remotely monitored and con- well the spectrum of embedded hardware/software de- trolled via Internet protocols. These devices involve velopment, plus low-power communications. The re- robots and robotic systems in the context of this work. search areas targeted by this paper are displayed as a Search for related work was performed through the Venn diagram in Fig. 2. WoT is considered a subset of Web scientific indexing services Web of Science and IoT and they both interact with robotics via the emerg- Google Scholar. The following query was used: ing research areas of IoRT and WoRT. Additional cir- Robotics AND [“Internet of Things” OR “Web of cles in the figure could be sensor technologies and Things”] communication protocols, as they are extensively used Thirty seven (37) papers were found via this ap- both by IoT and robotics. proach. To increase the range of our bibliography, a We note that IoRT is mostly about technical aspects search of the related work as appeared in these 37 pa- of robotic systems, as well as technologies for com- pers was also performed. This effort allowed to in- munication and message exchange or services and data crease the number of papers discovered to 61. From understanding. It is not about artificial intelligence, these 61 papers, 12 papers (20%) were discarded for robot perception and empathetic behaviour, where the following reasons: robotics touches upon other research disciplines. 1. They did not involve a real-world implementation of a robotic system but only mentioned theo- 4. Robotics retically how a robotic system could benefit from the connection to the Internet/Web, or from the Robots are becoming a fundamental part of our so- concepts of IoT. ciety and they will become even more important in 2. They did involve a real-world implementation of the future. The past decades were characterized by the some robot or robotic system, however, this robot massive automation in the industry, as for example in did not somehow relate to the concepts of IoT the case of automatic machines and industrial manip- and/or WoT. ulators. In this context, the robots work in a perfectly- Forty nine (49) papers were finally selected in order known and modeled environment and safety layers are to be analyzed in more detail. Each of these papers was built around them to prevent harming people and other studied in detail, aiming to address the aforementioned machinery. Furthermore, the actions of these robots are six research questions. The results of our research are completely programmed in order to avoid any unpre- presented in the next sections. dictable behaviours. The robots work fast and accu- We note that this paper focuses on the connection rately in order to improve the efficiency of industrial between robotics and the concepts/principles of IoT processes. A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 495

However, to further progress in the integration with tioned before, robots need to interact with their en- everyday life, robots, usually referred to as service vironment, i.e. other robots, devices and machines robots in this context, need to be able to perceive and (R2R) and also with humans (R2H). To achieve this understand unknown and complex environments and in a highly interoperable way towards true machine- to be capable of planning and acting in unforeseen sit- to-machine (M2M) interaction involving context un- uations. In robotics, these concepts are usually gath- derstanding, considering the wide variety of proto- ered together under the word cognition. Cognition is cols and hardware/software/communication architec- the key element in order to deal with the variety of en- tures and solutions available, the IoT and WoT come vironmental aspects, parameters and tasks in which the into play. The particular benefits from a WoT integra- service robots operate, i.e. houses, warehouses and of- tion are listed in Section 4.2 below. The main differ- fices. The second major challenge is the manipulation ence between IoT and WoT is that IoT operates in the of a high variety of objects, deformable or not, in the lower layers of the OSI stack, while WoT mainly at operational environment. The final hurdle for robots the application layer. Open Systems Interconnection is the interaction. Robots and humans are expected to (OSI) protocols are a family of information exchange end up in the same environments, working in paral- standards developed jointly by the ISO and the ITU- lel and collaborating together, without any safety layer T standardization bodies. OSI is a conceptual model to keep them apart. Thus, the robots’ behaviour must that characterises and standardises the communication be predictable and safe for humans, for the surround- functions of a telecommunication or computing sys- ings and eventually for other co-operating robots. The tem without regard to its underlying internal structure concept of interaction, however, cannot be limited only and technology. In this context, some overlap between to physical and safe interaction between humans and IoT and WoT may exist at the presentation and ses- robots, but it must be extended to more abstract ways sion layers of the OSI stack. This difference is illus- of interacting. Interpreting verbal and non-verbal com- trated in Fig. 3. The right part of the figure lists many munication such as facial expression, body movements of the technologies and acronyms mentioned through and gestures, as well as an understanding of social in- this survey, under the relevant part of the ISO stack teractions are clearly necessary steps to be taken in the where they belong, as well as whether they constitute near future. mainly IoT or WoT technologies. IoT could improve robotics with higher productiv- 4.1. Internet of Things and robotics ity (i.e. by re-using well-accepted and understood software and protocols), lower costs, better customer ex- Since robots are entering the everyday life of hu- periences due to the easier integration with existing mans, supporting their tasks and interacting with them, components of the nearby environment and with cloud the sensory equipment, hardware platforms, software computing, high-quality data in terms of semantics and communication patterns of the robot machines awareness, context understanding and many other pos- become more complicated and demanding. As men- sibilities listed in related work [80,133,134].

Fig. 3. Layers of the OSI stack where IoT and WoT are located. 496 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

Moreover, robotics can benefit from the plethora of primitives to users with little programming expe- research and development in IoT, in terms of resource- rience to perform advanced tasks. Users may se- constraint hardware and software, low-power commu- lect any programming language that supports the nication algorithms and protocols, as well as opti- HTTP protocol, such as Python, Java, Ruby, C, mal solutions for wireless sensor networks (WSN), PHP, JavaScript, etc. such as networking, mobility, data propagation, topol- – Exposing robotic services as APIs would facil- ogy building and maintenance etc. [97]. Finally, the itate application-layer interoperability between unique naming and addressing capabilities provided by robots (R2R) and humans (R2H). When these the IPv6 protocol, allow robots and robotic systems APIs become standardized, enhanced with se- to become uniquely addressable citizens of the Inter- mantic technologies, interaction between robots net, exploiting the TCP/IP protocols for device discov- and humans can become automatic and in real- ery, message exchange, security etc. The Transmission time, using the Web as the common platform and Control Protocol/Internet Protocol (TCP/IP) is a suite language for communication. of communication protocols used to interconnect net- – Combining robotic services with Web services work devices on the internet. TCP operates at the trans- and resources would allow the creation of Web- port layer of the OSI stack, while IP at the network based robotic mashups, where the robots exploit layer (see Fig. 3). seamlessly knowledge, information and context already available at the Web, towards more in- 4.2. Web of Things and robotics formed choices and aware behaviour. – Uniform access to heterogeneous embedded de- One of the first robots having a basic presence on vices installed on the robot, where the robot it- the Web was MERCURY [36], which started its opera- self becomes a homogeneous environment (i.e. at tion in 1994. It was one of the first teleoperated manip- the application layer), where any sensor/actuation ulators on the Web. It enabled Web users to excavate can be individually accessed in a standardised artefacts buried in a sand-filled terrarium. The TELE- way, facilitating coordination, action-taking and GARDEN robot in 1995 [37], successor of MER- control. This homogeneous access would allow CURY, allowed people to control the planting of flow- easier connection between robotic systems and ers via a Web interface, being able to coordinate re- cloud computing, for more advanced real-time quests by multiple users. XAVIER followed in 2002 computing processing and for permanent storage [107]. Although it had only a basic Web interface al- of information such as sensory measurements. lowing only basic interaction possibilities, it became – Harnessing well-defined protocols used for years quite successful due to the possibilities of remote con- on the Web for device and service discovery, ser- trol of a robot via any Web browser around the world. vice and data description, semantics understand- As mentioned in the introduction, WoT could of- ing, security and privacy, orchestration and rout- fer more benefits to robotic systems than IoT alone ing. [39,134], especially in terms of interoperability at the – Particularly related to the semantics of services application layer among robotic components and ser- and information, the Semantic Web (see Sec- vices, but also between robots (R2R) and humans tion 6.1 below) involves numerous technologies (R2H), as well as among other machines (M2M). and implementations for uniformly describing de- Some of these benefits are listed below, in more detail, vices, services and data, allowing common un- in relation to robotic systems: derstanding and advanced reasoning between different entities. The Semantic Web is an exten- – Data from the sensors of the robots may be easily sion of the World Wide Web through standards exported into Web applications in popular, well- set by the World Wide Web Consortium (W3C). understood standard formats, for easier reuse. The goal of the Semantic Web is to make Internet Representation formats may be negotiated in real- data machine-readable, enabling the encoding of time, depending on the formats supported by the semantics with the data. machines involved. – Exposing the services provided by the robots (and We note that APIs, in the WoT research area, are ex- their individual sensing and actuating compo- pected to be resource-oriented and to follow the REp- nents) as interoperable APIs, would provide the resentational State Transfer (REST) [31]. REST is an A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 497 architectural style for developing Web Services and it marizes in which ways the IoT-enabled robots of re- has been adopted by WoT due to its simplicity and the lated work have made actual use of Internet technolo- fact that it builds upon existing systems and features of gies. the HTTP protocol. A RESTful API follows the principles of REST for providing Web Services to the users, 5.1. Applications based on the HTTP request/response protocol. Thus, basic HTTP-based interfaces that do not follow REST Table 2 lists the different application domains and can not be considered part of the WoT. However, they their specific application areas, where IoT-based robots can still be considered part of the IoT, as long as they and robotic services have been used. The most popu- employ TCP/IP. lar categories are entertainment (8 papers), health (7 papers), education (6 papers), surveillance (6 papers) 5. Analysis of the application of Internet of Things and culture (5 papers). Some categories are overlap- in robotics ping, such as health with domestic support, surveillance with military, as well as emergency/disaster re- This section addresses the first three research ques- sponse with rescue operations. Autonomous cars are tions as defined in Section 3. First, Section 5.1 lists used as moving robots in the area of transportation, the existing applications of IoT in robotics, presenting while a warfare robot car has been developed for mili- how sensors and robot actions have been used in dif- tary purposes in [48]. Unmanned aerial vehicles (UAV) ferent application domains. Then, Section 5.2 shows (i.e. drones) are considered as flying robots in surveil- some of the hardware elements and sensory equip- lance, emergency/disaster response and rescue opera- ment used in the work under study. Afterwards, Sec- tions. UAV/drones are small aircrafts without a human tion 5.3 presents the software platforms and communi- pilot onboard. cation protocols used during programming and control Example robots as they appear in different applica- of IoT-based robotic systems. Finally, Section 5.4 sum- tion areas of the work under study are shown in Fig. 4.

Table 2 Applications of IoT in robotics Application area Application Industry Manufacturing [15,129], material handling [128], 3D assembly operations [131] Customer support Office operations’ support [81,107] Transportation Autonomous cars [35], car parking system [50] Environment Water quality monitoring [130], smoke detection [12], air quality [123], space exploration [11] Health Diabetes management [6–8], measuring reflexes [27], tele-surgery [19], tele-echography [74], remote treatment [115] Education Teaching computing, programming and robotics in schools and universities [17,42,122], collaborative learning [93], educate people in public places [16] Domestic support Medical and health care [45], support of people with dementia [109], support of elderly and disabled residents [51], independent living [25] Entertainment Pet robot [64,71], remote painting [111], robot that sings and dances [69], entertain people in public places [16], allow low-cost public access to a tele-operated robot [36], interact with a remote garden filled with living plants [37], moving in a wooden labyrinth trying to get out of the maze [100] Sports Ball detection and catching [116], measuring reflexes [27], control of a football robot team [13] Culture Remote tour guiding at a museum [16,72,90], interactive tour guide in museums [16,118] Surveillance UAV [82,83,102], monitoring activities in factories, offices and industrial sites, remote control [102,105], detect internal condition of working areas [123], drones as a service [68] Military Warfare robot car [48], land mining and field surveillance [9] Emergency/Disaster response Emergency response system [43,138], drones for emergency use [68], crime situations [30] Rescue operations UAV for object tracking [66], UAV for disaster rescue operations [3] Agriculture Soil moisture sensing and remote crop monitoring [130], live streaming of crops, seed sowing, pesticide sprinkling and automatic irrigation [44] 498 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

Fig. 4. Example robots as presented in related work under study.

Figure 5 maps the sensing/actuation capabilities of 5.2. Hardware and sensors the IoT robots, together with their actions, in relationship to the different application areas as encountered 5.2.1. Hardware in this study by analyzing the related bibliography. Besides the mechanical parts which vary signifi- The most popular sensors, actions and application ar- cantly per related work under study, there is some eas are highlighted in blue color. As the figure shows, common hardware used among the surveyed papers, the most commonly used sensors are the ones that such as Raspberry Pi [12,17,27,44,48,66,105,123,129] measure proximity (for the actions of touching, pick- and Arduino [9,105,130]. Both Raspberry Pi and Ar- ing and placing, reminding), vision (for the actions of duino constitute open-source hardware and electronic prototyping platforms, enabling users to create inter- picking and placing, moving, observing, flying, spray- active electronic objects. In the context of this sur- ing and pruning, reminding), voice (for the actions of vey, researchers in related work have used them as speaking, listening and voice recognition) and motion mini computers, connecting external sensors, actuators (for the actions of picking and placing, speaking, ob- and mechanical parts, in order to give intelligence to serving, climbing, swimming and displaying text). their robots. TelosB was the sensor platform selected The most popular actions involve moving, observ- in [102]. ing, flying and navigating. Some actions are appropri- Some efforts tried to develop humanoid robots [16, ate only for specific application areas, such as spray 27,109,118], while others employed autonomous ve- and prune for agriculture, reminders for health and do- hicles [35,129,130], a space exploration rocket [11], mestic support, pick-and-place for customer support a warfare car robot [48] and UAV/drones [3,30,66,68, and industrial applications, etc. 82,83,102,130,138]. An UAV enhanced with IoT sen- In relation to the possible actions performed by the sors has been introduced in [102], where the UAV in- robot, based on its sensory equipment, the applica- teracts better with its environment towards more effec- tion areas where most actions have been recorded are tive surveillance, gathering data coming from temper- agriculture (7 actions), domestic support (6 actions), ature, humidity and light sensors. surveillance (6 actions), military (6 actions), rescue op- The remaining papers used robots with application- erations (5 actions) and entertainment (5 actions). specific mechanics and characteristics. The interesting A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 499

Fig. 5. The connection between IoT robot sensors, actions and IoT application areas, as they appear in related work under study. concept of biobots was introduced in [43], which was The relationship between the aforementioned sen- about real animals (in this case, dogs) equipped with sors and the robots’ sensing/actuation capabilities, as sensors such as cameras or gas detectors. These sen- they appear in related work, is presented in Table 3. sors allowed the animals to sense additional environ- The relationship between the intended use of these sen- mental aspects (i.e. search rubble for casualties or de- sors, the desired actions and targeted application areas tect dangers such as a gas leak). In this case, the me- is illustrated in Fig. 5. chanical parts of a robot are replaced by the physical capabilities of the animals involved. 5.3. Software and communications

5.2.2. Sensors 5.3.1. Software Robots were equipped with a wide variety of dif- In regards to software, some papers [3,30,66,102] ferent sensors, such as radio-frequency identification employed the popular Linux-based Robot Operating (RFID) [15,45,128–130], video cameras [11,35,36,51, System (ROS) [95], which provides the communica- 81,93,100,116,131], infrared and light sensors [81], tions infrastructure to program, operate, debug and smoke sensors [12], temperature and gas [9,123], tem- control the robot as a system of systems. perature, humidity and light sensors [102], medical ROS is an open-source framework for writing soft- sensors [6,7,45], accelerometers and gyroscopes [42, ware for robotic systems. It consists of a number of li- 64], occupancy [25] and infrared sensors [44,81,100]. braries, tools and sets of conventions to simplify the Global Positioning System (GPS) receivers were also task of writing software for complex mechatronic sys- installed in many robots for localization and navigation tems [95]. ROS includes a large variety of algorithms [30,35,42,48,66,68,130]. and functions for creating new software components RFID uses electromagnetic fields to automatically and drivers. ROS supports multiple sensor technolo- identify and track tags attached to objects. GPS re- gies as well as programming languages, of which C++ ceivers exploit a satellite-based radio-navigation sys- and Python are the most important. Nowadays, ROS tem to deduce the geo-location of objects they are at- is one of the most exploited tools for developing al- tached to [57]. gorithms in the context of robotics due to its flexi- 500 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

Table 3 cloud-based robotics platforms that could be used to- Sensors and intended use gether with IoRT architecture. As an example, Simoens Sensor Intended use et al. [109] employed the DYAMAND middleware Microphone Voice recognition [85] for abstracting the different protocols and inter- Gyrometer, accelerometer Rotational motion faces of the installed sensors of the robot. RFID, Infrared sensor Sense proximity 5.3.2. Communication Occupancy, infrared sensor Sense motion, perform some mechanical action In terms of communication protocols, the major- Video camera Computer vision, human remote ity of related work used Wi-Fi (i.e. 16 papers), while vision IEEE802.15.4 and ZigBee [45,48,102,128,129], third Pressure sensor Sense pressure on something or fourth generation of broadband cellular network Medical sensors Measure health indications of technology (3G/4G) [6,7,66] and Bluetooth [6,7,45, humans 48,64,66,109] were also used. Regarding Bluetooth in GPS receiver Location particular, the Bluetooth Low Energy (BLE) technol- Gas/chemical sensor Sense hazardous materials ogy was used for communication among robot compo- Smoke sensor Sense fire nents. Temperature/humidity sensor Sense weather conditions, Wi-Fi has been used where wide coverage was re- surveillance quired (i.e. up to 100 meters) and/or it was not possi- Light sensor Measure illumination, ble to propagate messages via intermediate nodes [5]. surveillance Since Wi-Fi consumes much energy, it has been used where autonomy had not been an important issue (e.g. bility and simplicity. However, it is worth mentioning education, health, tour guiding in museums). 3G/4G that ROS has no real-time capabilities. This means that has also been used in scenarios where connectivity was ROS does not provide guarantees about the timing of difficult (e.g. rescue operations) but it consumes more operations, hence it is not intended for operations that energy in comparison to Wi-Fi. On the other hand, Zig- have strict timing requirements. Bee and BLE have been used for short-range and low- In the case of safety critical systems in which hard energy communication scenarios, where there was a real-time constraints exist, real-time operating sys- specific (indoor) topology/infrastructure and need for tems must be adopted. The real-time operating system autonomy (e.g. industry, domestic support, entertain- (RTOS) is one such system, used in [42], which han- ment). Zigbee is a worldwide standard for low power, dles the execution of tasks in order to meet their time self-healing, mesh networks offering a complete and deadline, but also facilitates memory management and interoperable IoT solution for home and building au- accessing resources. The two main design philoso- tomation. phies are: event-driven and time sharing. An event- An aspect not explicitly addressed in the surveyed driven scheduler switches between tasks when an event papers is the security of software, communication mes- of higher priority requires to be accommodated. On the sages and the actual data involved. Two papers men- other hand, a time sharing scheduler switches among tioned the use of the HTTPS protocol for secure com- tasks based on a periodic clock signal. munication [6,7] while NASA employed the WITS Other papers used GOLEX [16,118], Embedded C Encrypted Data Delivery System (WEDDS) and its [9], the Multi-target Robot Language (MRL) [81], public key infrastructure during the Mars polar lander Node.js [105] as well as OpenWSN [102]. OpenWSN mission [11]. HTTPS is an extension of the HTTP pro- is an open-source implementation of the IEEE/IETF tocol for secure communication. 6TiSCH protocol stack [132]. 6TiSCH is a promising Working Group that aims to achieve industrial- 5.4. Summary grade performance in terms of jitter, latency, scalabil- ity, reliability and low-power operation for IPv6 over It is worth investigating in which ways the IoT- IEEE802.15.4e TSCH. Timeslotted Channel Hopping enabled robots of the related work under study make (TSCH) is a Medium Access Control (MAC) proto- actual use of Internet technologies, according to the col based on Time Division Multiple Access (TDMA) definition of IoRT [96,108,125], as mentioned in the schedule. introduction. Table 4 presents the classification of the Besides the aforementioned software systems and surveyed papers in different classes, based on how they platforms, Ray [96] has listed numerous emerging use Internet technologies. A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 501

Table 4 Use of Internet technologies in IoRT Type of use Related work Not any details [3,64,82,83] IPv4 communication [6,8,11–13,16,17,25,27,36,37,42,45,48,69,71,72,90,93,100,105,107,109,111, 115,116,118,123,130] Java Object Request Broker (ORB) architecture CORBA [51], HORB [74] IPv6 architectures 6LoWPAN [15], 6TiSCH technology (COAP, RPL, 6LoWPAN) [102] Cloud robotics [9,30,30,35,44,66,68,128,129,138]

The majority of the surveyed papers use TCP/IP Table 5 communications for interaction between the robot and Use of web technologies in WoRT the outside world [98]. Some of these papers incor- Type of use Related work porate principles of the WoT and they would be ana- HTTP-based communication [30,102] lyzed in the Section 6. Java ORB is used in [51,74]for Basic HTTP interface for [6,8,9,11,12,16,36,37,44,51,72, managing distributed program objects, while Brizzi et interacting with the robot 81,90,100,107,111,118,130] al. [15] employ an IPv6-based 6LoWPAN architecture Web server on the robot [42,105] for communication between robots and wireless sen- REST API for robot control [27,66,123] sor networks. 6LoWPAN is a working group and stan- Semantic Web technologies [15,109] dard for the application of IPv6 over low-power sen- Publish/Subscribe architectures [27,109] sors and wireless sensor networks [84]. An implemen- (Message brokers) tation of 6LoWPAN via the 6TiSCH technology [132] was performed in [102], combining 6LoWPAN with any Web technologies have been omitted from the ta- relevant software protocols and implementations such ble. Most of the papers use a basic HTTP interface as the Constrained Application Protocol (COAP) [104] for interacting with the robot, while the underlying and the RPL IPv6 routing protocol [135]. communication is realized using different communica- Finally, the last category of Table 4 is about pa- tion protocols. In Section 4.2, it was mentioned that pers using cloud services for storage, processing, up- WoT-based developments should employ REST and dates and management/control. These papers touch not only basic HTTP interfaces. However, we still in- upon the research area of cloud robotics [46], which clude in Table 5 papers with only basic HTTP inter- deals with infrastructures and protocols for machine- faces for interaction with robots, because they consti- to-cloud (M2C) communications. Some interesting tute the majority of related work. relevant concepts mentioned in related work [35,129] Actual Web servers on the robot are installed in [42] are Information Centric Networking (ICN) [2] and (i.e. commercial Keil Tools C/C++ cloud-based com- Software-Defined Networking (SDN) [77]. ICN is an piler) and [105] (i.e. NodeJS server platform), while approach to evolve the Internet infrastructure away some papers move one step further, creating RESTful from a host-centric paradigm based on the end-to-end APIs for interacting with the robot’s features and op- principle, to a network architecture in which informa- erations [27,66,123]. NodeJS is a platform built on the tion is the focal point. SDN is an architecture that aims browser Chrome’s JavaScript runtime for easily build- to make networks agile and flexible, improving net- ing fast and scalable network applications. work control. In the context of IoRT, ICN and SDN are To enable Web-based interaction, some papers used approaches/architectures for facilitating robot control, platforms such as the WebIOPi IoT framework [123], as well as communication and networking between the Web interface for telescience (WITS) [11], the robots and the outside world through the Internet. WAMP server [27], the MASSIF platform [14,109] and NodeJS [105]. These platforms provide Web servers and support for exposing robot services as API 6. Analysis of the application of Web of Things in calls, control and debugging of the software used for robotics programming the robots (e.g. WebIOPi for Raspberry Pi), control dashboards and easy installation setups Table 5 lists which Web technologies have been (e.g. WAMP), remote access to the robot (e.g. WITS used in the surveyed papers. Papers that do not use for the planetary rover mission to Mars [11]), man- 502 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things agement of asynchronous messaging and handling of source Description Framework (RDF) is a general- thousands of concurrent connections (e.g. NodeJS), purpose language for representing information on the semantic annotation, reasoning and integration of IoT Web, while the Web Ontology Language (OWL) is a data (e.g. MASSIF [14]) etc. formal language for representing ontologies in the Se- Shin et al. [105] used the Web Service Description mantic Web. Semantic query languages are necessary Language (WSDL) to describe the services provided for retrieving and manipulating data stored in descrip- by the REST API of their UAV [22]. WSDL [22], simi- tion languages such as RDF/OWL, being able to an- lar to Web Application Description Language (WADL) swer complex queries and produce advanced knowl- which is generally more suitable for Web-based appli- edge by combining different sources of information to- cations [41], is used to describe service and their se- gether. mantics in order for humans and machines to be able From the surveyed papers, Simoens et al. [109]em- automatically to use these services by creating the ap- ployed semantic technologies to describe the robots’ propriate requests via TCP/IP calls. produced data by means of the WSN and SSN ontolo- Finally, Web-based message brokers, such as Rab- gies while Brizzi et al. [15] described robotic services bitMQ and Crossbar.io, are used in [27,109]. Message by means of semantic web services. Semantic Web ser- brokers are intermediary computer program modules vices [76] are similar to Web Services, but they ad- that translate a message from the formal messaging ditionally employ standards for the interchange of se- protocol of the sender to the formal messaging proto- mantic data. col of the receiver. These messaging brokers are suitable for high-performance, scalable distributed messaging where multiple publishers and subscribers of 7. Discussion information are involved (e.g. health monitoring scenarios with care-providers involved [27,109]). In this This section discusses the general findings of the case, they are called publish/subscribe architectures or survey. Specifically, Section 7.1 refers to the techni- infrastructures. cal aspects of the surveyed work and Section 7.2 examines the actual relationship between related work 6.1. Web semantics under study and the general principles of IoT/WoT. An advanced aspect of WoT towards seamless M2M Then, Section 7.3 captures the big picture in relation communication and understanding is the use of Se- to research and development in the area of robotic IoT, mantic Web technologies to describe services and data, while Section 7.5 provides future research opportuni- towards a semantic WoT [92]. The semantic WoT ties in the IoRT domain. Finally, Section 7.6 summa- involves technologies for uniformly describing WoT rizes the take-home messages of this survey paper. data streams, devices and services, allowing easy and fast integration with other sources of information, plat- 7.1. Technical aspects forms and applications towards advanced knowledge and reasoning [59]. A large percentage of the research work under study Semantics are particularly important in robotics, for (36%) employed open-source hardware and prototyp- the understanding of space, ambient environment and ing platforms, connecting them to a wide variety of surroundings of the robot and for better reasoning. Se- sensors (i.e. 14 different sensor types). The mechan- mantics can be described and analyzed by means of ical parts of the robots involved 16 different actions, ontologies and vocabularies (e.g. OWL, SSN [24], IoT with movement, observation and navigation being the [103]). Ontologies include concepts and categories in most popular ones. Sixteen application areas have been a subject area or domain (i.e. where the robot is op- recorded, with health being the most popular applica- erating) that show, describe and explain their proper- tion domain for IoT-based robotics. This makes sense, ties and the relations between them. Ontologies are considering that health applications, especially at do- not enough though and need to be accompanied by mestic level, require easy interconnection with other description languages (e.g. RDF, OWL), as well as devices of a smart home or fast notification/alerting query languages (e.g. SPARQL, CQEL). Description and communication with care providers. Thus, the languages facilitate consistent encoding, exchange and IoT/WoT protocols are appropriate for this intercon- processing of semantically-annotated content. The Re- nection with low effort. A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 503

In regards to communications, Wi-Fi was the most As mentioned before, cloud networked robotics is popular technology (36% of the papers), followed by a modern, promising research area [46]. New cloud- Bluetooth (16%) and ZigBee (10%). The characteris- based software systems make the integration between tics of these communication protocols are described robotics and IoT much easier [42,120]. The ROS could in [5] and researchers should consider the most ap- be used for connecting robots to the cloud [95](i.e. propriate technology for their implementations, taking together with the FIROS tool [67]). Such possibili- into account application requirements such as range ties are also provided by RoboEarth [127], a system and coverage, energy consumption and autonomy of for sharing knowledge between robots. Rapyuta, as the the robot, security, mobility aspects etc. RoboEarth cloud engine, helps robots to offload heavy Unfortunately, only 2 papers (4%) explicitly men- computation by providing secured customizable com- tioned security measures during message communica- puting environments in the cloud [47]. Towards WoRT, tion. Some papers might have used the security fea- theworkin[63] allows to build REST APIs for ROS, tures provided by the underlying platforms and oper- while development of Web-based services for robotic ating systems used (e.g. ROS, RTOS, MRL), but this devices is possible via [75]. has not been specified by the authors explicitly. We argue that nowadays IoT offers a wide range of secure 7.3. The big picture communication protocols [86] and they should be har- nessed by researchers to increase the security of their To complete the discussion on the IoRT/WoRT top- ics, we try to capture here the big picture of this re- robot implementations. IoT could also be used to en- search area. Figure 6 depicts the general suggested hance humans’ access control via smart authentica- architecture for IoRT/WoRT systems, considering the tion, i.e. by means of various biometrics, face or voice principles of IoT and WoT applied in robotics. Com- recognition etc. munication between sensors and electronic platforms is via the IPv6 protocol, while sensory platforms can 7.2. Connection to the Internet/Web of Things embed Web servers themselves [61]. Figure 6 lists some of the operating systems, as identified in related Our research shows that some papers claimed to be work under study for sensors’ and robots’ program- IoT-enabled without giving any details of the connec- ming, or for deploying Web servers on the robots. tion between IoT and robotics [3,64,82,83]. Also, the Some promising platforms for robots’ programming majority of the surveyed papers (29 papers, 59%) use not employed in the surveyed papers but still worth merely TCP/IP communication, which is only a small mentioning are FIROS [67], a tool for connecting mo- part of the concepts of IoT and IoTR [96,108,125]. In bile robots to the cloud (i.e. by using ROS), BrainOS,1 regards to the WoT, many of the surveyed papers (20 an autonomous navigation platform based on computer papers, 40%) perform HTTP-based communication or vision and artificial intelligence, as well as the Middle- provide only some basic HTTP interface for interact- ware for Robotic Applications (MIRA) [29], a cross- ing with the robots. platform framework for the development of robotic ap- The most complete papers in IoRT are those em- plications. ploying cloud robotics (see Table 4), while the most Towards full Web integration, robots may expose complete ones in WoRT are those using REST APIs for their services as a REST API, for easier reuse of robot control and/or semantic web technologies (see their capabilities and features by authorized third par- Table 5). The observations that a limited number of pa- ties. Robots may also incorporate semantic engines pers employ IPv6 architectures or cloud robotics (12 for annotating semantically their services and data, papers, 24%) and/or use REST APIs or semantics (5 towards seamless M2M interaction. As mentioned in papers, 10%) are indications that the penetration of Section 6.1, researchers employed various ontologies IoT/WoT in robotics is still low and that the IoT/WoT to describe robotic data (i.e. WSN, SSN) and robotic are still not largely and properly used in robotics. This services (i.e. Brizzi et al. [15]). A set of interesting phenomenon has been observed also in WoT frame- ontologies that could be used in future IoRT systems works in the past [55], where the authors claimed to is the IEEE Ontologies for Robotics and Automation have WoT-ready frameworks, however their developments missed some important elements of the WoT 1BrainCorp, BrainOS, https://www.braincorp.com/brainos- principles. autonomous-navigation-platform. 504 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

Fig. 6. The general architecture for IoRT/WoRT systems.

(ORA) [94]. These ontologies could also describe the integration to existing systems and infrastructures, as relationship between a robot agent and its physical well as use of well-known and popular technologies environment. As an example, Jorge et al. [52]used for programming, management and control [39,134]. the ORA ontology for spatial reasoning between two Some of these benefits have been highlighted in Sec- robots that must coordinate for providing a missing tions 4.1 and 4.2. As mentioned before however, more tool to a human. Another interesting set of semantics studies are needed in order to further validate the im- is proposed in [117], for projecting the effects of ac- pact of the IoT/WoT protocols and technologies on the tions and processes performed by the robots and their real-time performance of robots. sequence. Besides ontologies, query languages and engines (e.g. SPARQL, CQEL) are important for seman- 7.4. Evaluation of IoRT systems and performance tic reasoning. metrics For completing IoT integration, especially in cases where robot swarms are involved or high scalabil- It is important to define and accept some standard ity/performance are needed, cloud services could be performance metrics that can be used to assess the the solution. The cloud could be used for more ad- performance and/or compare various robotic systems vanced processing and for storage of information, but [112]. In traditional robotics, many performance met- also for efficient messaging via publish/subscribe in- rics exist in order to assess the quality of a robot or frastructures. Concepts such as ICN [2] and SDN [77] a network/swarm of robots, while the ones most com- could also be employed for better overall management monly adopted are power/energy consumption, band- and control of the robots in a more abstract and generic width, latency, throughput, resilience to errors, packet manner. loss and locality of program execution. Some metrics As mentioned in Section 3, comparison of IoT/WoT are more suitable for UAV, such as setup time, flight existing software and hardware platforms is out of the time, inaccuracy of land, haptic control effort and cov- scope of this paper. It is worth commenting however erage ratio [53]. that the inclusion of IoT/WoT protocols gives impor- Unfortunately, our analysis cannot get in depth on tant benefits to (not only) robotic systems, such as in- this aspect because none of the cited authors discussed, teroperability, seamless M2M communication, easier mentioned or evaluated their proposed robotic systems A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 505 using any assessment metrics and/or comparing their and kind of robots, objectives, tasks, environments) systems with existing similar robotic implementations. [125]. These contextual changes can occur very fast in The only exception was the work in [102], where the an Internet/Web-enabled system, because the robot has authors assessed the quality of service of their robotic real-time access to information and content, provided system (i.e. UAV with Telosb sensors), expressed in by Web services, APIs, other robots of the swarm or terms of network joining time, data retrieval delay and other citizens of the IoT ecosystem. packet loss ratio. Their conclusion was that the performance of the UAV based on the aforementioned met- 7.5. Challenges and future opportunities rics satisfied the mission requirements. Generally, authors of work under study preferred Robotics, together with IoT, constitute dynamic and to focus their research on the feasibility and demon- active research fields and there is much ongoing re- stration of connecting robots to the Internet/Web, as search in these areas. This section focuses on the exist- well as to the new possibilities (i.e. sensing, actions, ing challenges and barriers, as well as future research application domains) that arose by their robot mod- opportunities that arise from the combination of these els/implementations (see Fig. 5). We acknowledge the technologies together in the future. Some of these chal- fact that in many cases there had not been any similar lenges and/or opportunities are the following: robotic systems to compare with. We will try here to predict the metrics which should – Although the impact of IoT/WoT technologies be used for the assessment of IoRT systems as this and protocols on embedded devices may be low trend is emerging around the world. The fast growth in [28,49,61,98,101], this impact might still be con- autonomy and new functionality/capabilities of robots siderable in time-critical robotic systems, where (including their Internet/Web-enabling) is followed by decisions need to be taken in fragments of a sec- the fast growth in complexity of algorithms as well. ond. We welcome research efforts dealing with This translates in higher demands for data, computa- this aspect, studying in detail how different tech- tional and storage resources (that is common, for ex- nologies and protocols of IoT and WoT affect per- ample, for machine learning-based algorithms). How- formance. This could be well related to the se- ever, due to the limited amount of computing and stor- curity of the messages exchanged between robots age capabilities of (mobile) robots, the energy con- and infrastructures [114]. sumption has great importance in IoRT, more than in – As mentioned in Section 7.3, it is unfortunate general IoT applications. Furthermore, in the near fu- that no comparisons of the robots with exist- ture, IoRT is expected to involve more operations that ing literature in terms of performance metrics require real-time and safety-critical robot functionali- was performed in the related work under study. ties (e.g. R2H interaction). In such scenarios, the con- We expect that future research will focus more sistent reduction of the latency plays a crucial role [1]. on performance, studying the possibilities and It is still important to quantify and assess the impact of constraints of the robots considering low-level the TCP/IP stack and/or Web technologies to the real- metrics such as energy consumption and auton- time operations of robotic systems. The papers under omy, bandwidth, latency, throughput, resilience study have not discussed this issue. Although other re- and fault tolerance, as well as high-level metrics search works demonstrated that the impact of IoT/WoT such as ease of use and control of the robot, over- on embedded devices is low [28,49,61,98,101], this all engagement and social behaviour, safety etc. It might not be the case for time-critical robots. would be also important to study the penalty and Robots are non-homogeneous and complex com- trade-offs in energy consumption and autonomy positions of sensors, actuators, control systems, me- when Web servers and the IPv6 stack are added chanical parts and artificial intelligence algorithms, to the robots [49,61]. which are meant to solve different and continuously- – In relation to the above, an important chal- changing tasks. The flexibility and adaptability of lenge(and desired characteristic) of robots is au- robotic systems must be reflected in the IoRT/WoRT tonomy in operation. IoT has active research in systems as well. Thus, an important aspect for the breakthrough high-performance architectures, al- further development of IoRT systems is the abil- gorithms and hardware that will allow wireless ity to quickly adjust to changes of context, without networks to be highly efficient [73], powered by application-specific adaptations (e.g. different number tiny batteries, energy-harvesting [54], or over-the- 506 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

air power transfer. Moreover, new communica- – In industrial applications and logistics, technolo- tion systems based on biology and chemistry are gies that blend the physical and digital context expected to evolve in IoT research, enabling a are important. Broader and more expansive in- wide range of new micro- and macro-scale appli- formation capture and processing via IoT/WoT, cations [4]. combined with smarter manipulation and move- – Another important research challenge for robots ment of physical materials via robots, can de- is the collective behaviour of robot swarms, or the liver new benefits such as higher efficiency of op- coordination and control of multi-robot systems erations, more insights and visibility, as well as towards an optimized outcome. Such swarms better interaction between components, systems have been proposed for collective industrial con- and actors. Blockchain could be relevant, to keep struction [91], transportation and box pushing an immutable distributed ledger among IoT sen- [89], as well as for smart homes [99] and search sors and robotic systems of untrusted partners and rescue [26]. More generally, the authors in involved in some supply chain ecosystem [119, [70] propose a plan-based approach, based on 125]. A blockchain is a growing list of records, PEIS Ecology [99], to automatically generate a called blocks, that are linked using cryptography preferred configuration of a robot ecology given [62]. a task, environment and set of resources. In this – The trend towards human-centric design of robot swarm ecosystem, IoT can offer effective robotic systems is expected to continue [136] and solutions for networking, communication among robots will become more integrated in our every- the robots and mobility [26]. day lives, either for assistance in common tasks – Just like there exist complete operating systems or for actual support of people in need, e.g. move- (e.g. TinyOS, Contiki) and application-layer pro- ment of people with paralysis. Thus, they need to tocols (e.g. COAP, ZigBee) for IoT sensors and capture humans’ emotions and social behaviour devices, we expect similar operating systems to understand how to react in different situations to appear for robots [136]. As mentioned be- [32]. This information might come from IoT sen- fore, early efforts in this direction are BrainOS sors, either installed on the robots or in the nearby and MIRA. A culture of developers and re- environment (i.e. smart homes/buildings) or from searchers still needs to be built around these plat- the online social networking presence and activ- forms/frameworks, sharing code, experiences and ity of humans. Ethics is an important dimension in this direction [65] and needs to be considered solutions to the community. Worth mentioning a-priory. is the Brains For Bots SDK by Neurala,2 which – Towards human-centric robotics, the teaching of includes numerous features for creating applica- robots to take actions by means of natural lan- tions that can learn, recognize, find and track ob- guage instructions is a key aspect. This challenge jects in real-time. Neurala incorporates in its SDK could derive knowledge and ideas from Web Se- deep learning techniques, to support cognitive re- mantic technologies, as well as from ontology- quirements in robot applications. based natural language interfaces for controlling – Combining online social networking with IoT/ IoT devices, such as im4Things [87] and home WoT-enabled robots [121]. In this case, online automation [10]. These language interfaces allow social networks could be used for storing and people to interact with machines/robots using a sharing links to resources of interest for the human language. R2R and R2H interactions, facilitating sharing – We expect service robots to enter new real-world of robotic services among online trustful con- environments in which IoT has already penetrated tacts [60]. Robots could recognize or authenticate [58]. These environments could be sports (i.e. as- users, giving them access to some of their con- sistance and safety in sports having risks such as trols, based on the peoples’ online profiles and en- climbing and parachuting), health (i.e. not only dorsements they have from other authorized peo- monitoring patients but also taking first-aid ac- ple. tions if needed), ecology and environmental monitoring, surveillance and security in social spaces 2Neurala, Brains For Bots, https://www.welcome.ai/tech/ (i.e. schools, airports, urban hotspots etc.), gam- hardware-iot/neurala-brains-for-bots-sdk. ing (i.e. robots become comrades or opponents A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 507

in real-world gaming scenarios), customer service ploited in robotic systems. Services of the robots (i.e. restaurants, hotels, gyms, tourism), the movie are not generally exposed as a REST API, nor the industry and many others. The autonomous learn- services and data are described via Web semantic ing capabilities of robots, together with their po- technologies. tential actions due to their mechanical parts, could – There is an increasing number of operating sys- offer more opportunities in these new environ- tems, software tools and platforms for developing ments. IoRT/WoRT-based robotic systems. Most of this – In recent year, research has mostly progressed in software enables access to cloud computing and the direction of developing novel applications and publish-subscribe infrastructures for scaling com- combinations of IoT/WoT and robotics. While putation and storage capabilities and for support- few theoretical frameworks have been proposed ing robot swarms more easily. for IoT and WoT [88], there has not yet been par- – Related work mostly focuses on feasibility stud- ticular focus and wide interest on studying a gen- ies and demonstrations, which showcase the ben- eral theoretical framework for IoRT and WoRT. eﬁts of IoT/WoT in robotics, mainly in terms This task, although not yet studied enough, can of ubiquitous access via Internet/Web and easy play an important role for further strengthening interoperability with other systems. Aspects of and broadening the applicability of IoRT/WoRT performance, especially considering the overhead to new robotics challenges. produced by the TCP/IP protocol, have not been well studied. 7.6. Take-home messages – There are still numerous open issues and gaps for future research in this emerging intersection of re- Summarizing the study performed in this paper, search areas (see Section 7.5). some take-home messages can be the following:

– As the IoT penetrates different domains, applica- 8. Conclusion tion areas and scientific disciplines, it started to penetrate also the research area of robotic sys- This paper studied the current use of the Internet of tems. Things (IoT) in robotics, through various real-world – There is a wide range of sensors used, intended examples encountered through a research based on a actions of the robots and application areas where bibliographic-based method. The concepts, character- the IoRT/WoRT-based robots operate. The most istics and architectures of IoT, as they are being used popular actions involve moving, observing via in existing robotic systems, have been recorded and computer vision, flying and navigating. The most listed, together with popular software, hardware and popular application areas are entertainment, communication methods. Moreover, the application ar- health, education and surveillance. eas, sensors and robot services/actions incorporated in – A large percentage of the surveyed papers em- IoT-based robots are presented. Further, the current ap- ployed open-source hardware and prototyping plication of the Web of Things (WoT) in robotics has platforms, making the physical connection to a been investigated and the overall potential of the Web wide variety of different sensors possible. of Robotic Things has been discussed in the paper. – Regarding communications, various protocols are A general observation is that some of the advanced being used, with Wi-Fi being the most popular, concepts of IoT/WoT are not yet being used by re- followed by IEEE802.15.4 and Bluetooth. The searchers in robotics. Finally, future research direc- decision on which communication protocol to use tions and opportunities are proposed. was mainly influenced by the coverage range re- quired, as well as the need for autonomy and longer lifetime of operation (i.e. energy consump- Acknowledgements tion). – The IoT is still not used at its full potential in Andreas Kamilaris has received funding from the robotics. Cloud robotics and the IPv6 protocol are European Union’s Horizon 2020 research and inno- not fully utilized. The same holds for the prin- vation programme under grant agreement No 739578 ciples of WoT, which has not yet been fully ex- complemented by the Government of the Republic of 508 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

Cyprus through the Directorate General for European [12] S. Bhutada, L.S. Tejasri and R. Tejaswini, IoT based mobile Programmes, Coordination and Development. robot for smoke detection, International Research Journal Nicolò Botteghi has received funding from Smart of Engineering and Technology (IRJET) 5(4) (2018), 1381– 1383. Tooling. Smart Tooling is an Interreg Flanders- [13] Z. Bi, G. Wang, L.D. Xu, M. Thompson, R. Mir, J. Nyikos, Netherlands project sponsored by the European Union A. Mane, C. Witte and C. Sidwell, IoT-based system for com- focused on automation in the process industry: making munication and coordination of football robot team, Internet maintenance safer, cheaper, cleaner, and more efficient Research 27(2) (2017), 162–181. doi:10.1108/IntR-02-2016- 0056. by developing new robot prototypes and tools. [14] P. Bonte, F. Ongenae, F. De Backere, J. Schaballie, D. Arndt, S. Verstichel, E. Mannens, R. Van de Walle and F. De Turck, The MASSIF platform: A modular and semantic platform References for the development of flexible IoT services, Knowledge and Information Systems 51(1) (2017), 89–126. doi:10.1007/ s10115-016-0969-1. [1] I. Afanasyev, M. Mazzara, S. Chakraborty, N. Zhuchkov, [15] P. Brizzi, D. Conzon, H. Khaleel, R. Tomasi, C. Pastrone, A. Maksatbek, A. Yesildirek, M. Kassab and S. Distefano, A. Spirito, M. Knechtel, F. Pramudianto and P. Cultrona, Towards the Internet of robotic things: Analysis, architec- Bringing the Internet of Things along the manufacturing line: ture, components and challenges, in: 12th International Con- A case study in controlling industrial robot and monitor- ference on Developments in ESystems Engineering (DeSE), ing energy consumption remotely, in: Emerging Technologies 2019, ISBN 9781728130217. doi:10.1109/dese.2019.00011. and Factory Automation (ETFA), 18th Conference on, IEEE, [2] B. Ahlgren, C. Dannewitz, C. Imbrenda, D. Kutscher and 2013, pp. 1–8. B. Ohlman, A survey of information-centric networking, [16] W. Burgard, A.B. Cremers, D. Fox, D. Hähnel, G. Lake- IEEE Communications Magazine 50(7) (2012). meyer, D. Schulz, W. Steiner and S. Thrun, Experiences with [3] T. Ahn, J. Seok, I. Lee and J. Han, Reliable flying IoT net- an interactive museum tour-guide robot, Artificial intelligence works for UAV disaster rescue operations, Mobile Informa- 114(1–2) (1999), 3–55. tion Systems 2018 (2018). [17] V. Callaghan, Buzz-Boarding; practical support for teaching [4] I.F. Akyildiz and J.M. Jornet, The Internet of nano-things, computing based on the internet-of-things, in: 1st Annual IEEE Wireless Communications 17(6) (2010), 58–63. doi:10. Conference on the Aiming for Excellence in STEM Learning 1109/MWC.2010.5675779. and Teaching, Imperial College, London and The Royal Geo- [5] S. Al-Sarawi, M. Anbar, K. Alieyan and M. Alzubaidi, Inter- graphical Society, 2012, pp. 12–13. net of Things (IoT) communication protocols, in: 8th Interna- [18] M. Casini, Internet of things for energy efficiency of build- tional Conference on Information Technology (ICIT), IEEE, ings, International Scientific Journal Architecture and Engi- 2017, pp. 685–690. neering 2(1) (2014), 24–28. [6] M.A. Al-Taee, W. Al-Nuaimy, A. Al-Ataby, Z.J. Muhsin and [19] M.C. Çavu¸soglu,˘ W. Williams, F. Tendick and S.S Sastry, S.N. Abood, Mobile health platform for diabetes management Robotics for telesurgery: Second generation Berkeley/UCSF based on the Internet-of-Things, in: Applied Electrical Engi- laparoscopic telesurgical workstation and looking towards the neering and Computing Technologies (AEECT), IEEE Jordan future applications, Industrial Robot: An International Jour- nal 30(1) (2003), 22–29. doi:10.1108/01439910310457670. Conference on, 2015, pp. 1–5. [20] T.V. Chien, H.N. Chan and T.N. Huu, A comparative study [7] M.A. Al-Taee, W. Al-Nuaimy, Z.J. Muhsin and A. Al-Ataby, on operating system for wireless sensor networks, in: Inter- Robot assistant in management of diabetes in children based national Conference on Advanced Computer Science and In- on the Internet of things, Interaction 11 (2017), 13. formation Systems, IEEE, 2011, pp. 73–78. [8] M.A. Al-Taee, A.H. Sungoor, S.N. Abood and N.Y. Philip, [21] A.R. Chowdhury, IoT and robotics: A synergy, PeerJ Web-of-Things inspired e-Health platform for integrated di- Preprints 5 (2017), e2760v1. abetes care management, in: Applied Electrical Engineering [22] E. Christensen, F. Curbera, G. Meredith, S. Weerawarana et and Computing Technologies (AEECT), IEEE Jordan Confer- al., Web services description language (WSDL) 1.1, Citeseer, ence on, 2013, pp. 1–6. 2001. [9] M. Ashokkumar and T. Thirumurugan, Integrated IOT based [23] S.E. Collier, The emerging enernet: Convergence of the design and Android operated multi-purpose field surveillance smart grid with the Internet of Things, IEEE Industry Appli- robot for military use, Advances in Engineering Research cations Magazine 23(2) (2016), 12–16. doi:10.1109/MIAS. (AER) 142(1) (2018), 236–243. 2016.2600737. [10] C.J. Baby, F.A. Khan and J. Swathi, Home automation using [24] M. Compton, P. Barnaghi, L. Bermudez, R. García-Castro, IoT and a chatbot using natural language processing, in: In- O. Corcho, S. Cox, J. Graybeal, M. Hauswirth, C. Henson, novations in Power and Advanced Computing Technologies A. Herzog et al., The SSN ontology of the W3C semantic (i-PACT), IEEE, 2017, pp. 1–6. sensor network incubator group, Web semantics: science, ser- [11] P.G. Backes, K.S. Tso, J.S. Norris, G.K. Tharp, J.T. Slostad, vices and agents on the World Wide Web 17 (2012), 25–32. R.G. Bonitz and K.S. Ali, Internet-based operations for the [25] S. Coradeschi, A. Cesta, G. Cortellessa, L. Coraci, J. Gon- Mars polar lander mission, in: Robotics and Automation zalez, L. Karlsson, F. Furfari, A. Loutfi, A. Orlandini, (ICRA), IEEE International Conference on, Vol. 2, IEEE, F. Palumbo et al., Giraffplus: Combining social interaction 2000, pp. 2025–2032. and long term monitoring for promoting independent living, A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 509

in: Human System Interaction (HSI), 6th International Con- [41] M.J. Hadley, Web application description language (WADL), ference on, IEEE, 2013, pp. 578–585. 2006. [26] S.M. Das, Y.C. Hu, C.G. Lee and Y.-H. Lu, Mobility-aware [42] J.O. Hamblen and G.M. Van Bekkum, An embedded systems ad hoc routing protocols for networking mobile robot teams, laboratory to support rapid prototyping of robotics and the Journal of communications and networks 9(3) (2007), 296– Internet of Things, IEEE Trans. Education 56(1) (2013), 121– 311. doi:10.1109/JCN.2007.6182857. 128. doi:10.1109/TE.2012.2227320. [27] P. Doxopoulos, K.L. Panayiotou, E.G. Tsardoulias and [43] T. Harbert, On the Internet of Things, Nobody knows you’re A.L. Symeonidis, Creating an extrovert robotic assis- a dog, 2014, tant via IoT networking devices, 2018, arXiv preprint http://spectrum.ieee.org/computing/embedded-systems/ arXiv:1804.04361. on-the-internet-of-things-nobody-knows-youre-a-dog. [28] A. Dunkels et al., Efficient application integration in IP-based [44] P. Hemalatha, K. Dhanalakshmi, S. Matilda and M.B. Anand, sensor networks, in: Proceedings of the First ACM Workshop Farmbot-a smart agriculture assistor using Internet of Things, on Embedded Sensing Systems for Energy-Efficiency in Build- International Journal of Pure and Applied Mathematics ings, ACM, 2009, pp. 43–48. doi:10.1145/1810279.1810289. 119(10) (2018), 557–566. [29] E. Einhorn, T. Langner, R. Stricker, C. Martin and H.- [45] F. Hu, D. Xie and S. Shen, On the application of the Internet M. Gross, Mira-middleware for robotic applications, in: of Things in the field of medical and health care, in: Interna- IEEE/RSJ International Conference on Intelligent Robots and tional Conference on Cyber, Physical and Social Computing, Systems, 2012, pp. 2591–2598. IEEE, 2013, pp. 2053–2058. [30] G. Ermacora, A. Toma, B. Bona, M. Chiaberge, M. Silvagni, [46] G. Hu, W.P. Tay and Y. Wen, Cloud robotics: Architecture, M. Gaspardone and R. Antonini, A cloud robotics architec- challenges and applications, IEEE Network 26(3) (2012). ture for an emergency management and monitoring service in doi:10.1109/MNET.2012.6201212. a smart city environment, Polytech. Univ. Turin, Turin, Italy, [47] D. Hunziker, M. Gajamohan, M. Waibel and R. D’Andrea, Tech. Rep., 2013. Rapyuta: The RoboEarth cloud engine, in: ICRA, Citeseer, [31] X. Feng, J. Shen and Y. Fan, REST: An alternative to RPC 2013, pp. 438–444. for web services architecture, in: First International Confer- [48] L.B. Imran, M. Farhan, R.M.A. Latif and A. Rafiq, Design of ence on Future Information Networks, IEEE, 2009, pp. 7–10. an IoT based warfare car robot using sensor network connec- doi:10.1109/ICFIN.2009.5339611. tivity, in: 2nd International Conference on Future Networks [32] T. Fong, I. Nourbakhsh and K. Dautenhahn, A sur- and Distributed Systems, ACM, 2018, p. 55. vey of socially interactive robots, Robotics and au- [49] A.J. Jara, M.A. Zamora and A. Skarmeta, Glowbal IP: An tonomous systems 42(3–4) (2003), 143–166. doi:10.1016/ adaptive and transparent IPv6 integration in the Internet of S0921-8890(02)00372-X. Things, Mobile Information Systems 8(3) (2012), 177–197. [33] J.L. Fuller, Robotics, Prentice Hall PTR, 1998. doi:10.1155/2012/819250. [34] P. Gaur and M.P. Tahiliani, Operating systems for IoT de- [50] Z. Ji, I. Ganchev, M. O’Droma, L. Zhao and X. Zhang, vices: A critical survey, in: IEEE Region 10 Symposium, A cloud-based car parking middleware for IoT-based smart IEEE, 2015, pp. 33–36. cities: Design and implementation, Sensors 14(12) (2014), [35] M. Gerla, E.-K. Lee, G. Pau and U. Lee, Internet of vehi- 22372–22393. doi:10.3390/s141222372. cles: From intelligent grid to autonomous cars and vehicular clouds, in: Internet of Things (WF-IoT), IEEE World Forum [51] S. Jia and K. Takase, Internet-based robotic system using on, 2014, pp. 241–246. CORBA as communication architecture, Journal of Intelli- [36] K. Goldberg, M. Mascha, S. Gentner, N. Rothenberg, C. Sut- gent and Robotic Systems 34(2) (2002), 121–134. doi:10. ter and J. Wiegley, Desktop teleoperation via the world wide 1023/A:1015603327545. web, in: Proceedings of 1995 IEEE International Conference [52] V.A. Jorge, V.F. Rey, R. Maffei, S.R. Fiorini, J.L. Carbon- on Robotics and Automation, Vol. 1, IEEE, 1995, pp. 654– era, F. Branchi, J.P. Meireles, G.S. Franco, F. Farina, T.S. Da 659. doi:10.1109/ROBOT.1995.525358. Silva et al., Exploring the IEEE ontology for robotics and [37]K.Goldberg,J.Santarromana,G.Bekey,S.Gentner,R.Mor- automation for heterogeneous agent interaction, Robotics ris, J. Wiegley and E. Berger, The telegarden, in: Proc. of and Computer-Integrated Manufacturing 33 (2015), 12–20. ACM SIGGRAPH, 1995, pp. 135–1140. doi:10.1016/j.rcim.2014.08.005. [38] L.A. Grieco, A. Rizzo, S. Colucci, S. Sicari, G. Piro, [53] C. Ju and H. Son, Multiple UAV systems for agricultural D. Di Paola and G. Boggia, IoT-aided robotics applications: applications: Control, implementation, and evaluation, Elec- Technological implications, target domains and open issues, tronics 7(9) (2018), 162. doi:10.3390/electronics7090162. Computer Communications 54 (2014), 32–47. doi:10.1016/j. [54] P. Kamalinejad, C. Mahapatra, Z. Sheng, S. Mirabbasi, comcom.2014.07.013. V.C. Leung and Y.L. Guan, Wireless energy harvesting for the [39] D. Guinard, V. Trifa, F. Mattern and E. Wilde, From the In- Internet of Things, IEEE Communications Magazine 53(6) ternet of Things to the Web of Things: Resource-oriented ar- (2015), 102–108. doi:10.1109/MCOM.2015.7120024. chitecture and best practices, in: Architecting the Internet of [55] A. Kamilaris and M.I. Ali, Do Web of Things platforms truly Things, Springer, 2011, pp. 97–129. doi:10.1007/978-3-642- follow the Web of Things? in: Internet of Things (WF-IoT), 19157-2_5. IEEE 3rd World Forum on, 2016, pp. 496–501. [40] D. Guinard, V. Trifa, T. Pham and O. Liechti, Towards physi- [56] A. Kamilaris, F. Gao, F.X. Prenafeta-Boldu and M.I. Ali, cal mashups in the web of things, in: Networked Sensing Sys- Agri-IoT: A semantic framework for Internet of Things- tems (INSS), Sixth International Conference on, IEEE, 2009, enabled smart farming applications, in: IEEE World Forum on pp. 1–4. Internet of Things (WF-IoT), IEEE, Reston, VA, USA, 2016. 510 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

[57] A. Kamilaris and F.O. Ostermann, Geospatial analysis and Robots and Systems, IEEE/RSJ International Conference on, the Internet of Things, ISPRS International Journal of Geo- Vol. 2, 2001, pp. 637–642. Information, Special Issue “Geospatial Applications of the [73] M.S. Mahmoud and A.A. Mohamad, A study of effi- Internet of Things (IoT)” 7(7) (2018). cient power consumption wireless communication tech- [58] A. Kamilaris and A. Pitsillides, Mobile phone computing and niques/modules for internet of things (IoT) applications, the Internet of Things: A survey, IEEE Internet of Things 2016. (IoT) Journal 3(6) (2016), 885–898. doi:10.1109/JIOT.2016. [74] K. Masuda, E. Kimura, N. Tateishi and K. Ishihara, Three 2600569. dimensional motion mechanism of ultrasound probe and its [59] A. Kamilaris, A. Pitsillides, F.X. Prenafeta-Boldu and application for tele-echography system, in: Intelligent Robots M.I. Al, A Web of Things based eco-system for urban com- and Systems, IEEE/RSJ International Conference on,Vol.2, puting – towards smarter cities, in: 24th International Confer- IEEE, 2001, pp. 1112–1116. ence on Telecommunications (ICT), Limassol, Cyprus, 2017. [75] S. Mayer, Web-based service brokerage for robotic de- [60] A. Kamilaris, G. Taliadoros, A. Pitsillides and D. Papadiomi- vices, in: ACM Conference on Ubiquitous Computing, 2012, dous, The practice of online social networking of the physi- pp. 863–865. cal world, International Journal of Space-Based and Situated [76] S.A. McIlraith, T.C. Son and H. Zeng, Semantic web services, Computing 2(4) (2012), 240–252. doi:10.1504/IJSSC.2012. IEEE intelligent systems 16(2) (2001), 46–53. doi:10.1109/ 050007. 5254.920599. [61] A. Kamilaris, V. Trifa and A. Pitsillides, HomeWeb: An ap- [77] N. McKeown, Software-defined networking, INFOCOM plication framework for web-based smart homes, in: 18th In- keynote talk 17(2) (2009), 30–32. ternational Conference on Telecommunications, IEEE, 2011, [78] J. Mineraud, O. Mazhelis, X. Su and S. Tarkoma, A gap anal- pp. 134–139. ysis of Internet-of-Things platforms, Computer Communica- [62] A.F.A. Kamilaris and F.X. Prenafeta-Boldu, The rise of tions 89 (2016), 5–16. doi:10.1016/j.comcom.2016.03.015. blockchain technology in agriculture and food supply chains, [79] D. Minoli, Building the Internet of Things with IPv6 and Trends in Food Science and Technology Journal 91(1) (2019), MIPv6: The Evolving World of M2m Communications, John 640–652. doi:10.1016/j.tifs.2019.07.034. Wiley & Sons, 2013. [63] F.L. Keppmann, M. Maleshkova and A. Harth, Building [80] D. Miorandi, S. Sicari, F. De Pellegrini and I. Chlamtac, Inter- REST APIs for the robot operating system-mapping concepts, net of things: Vision, applications and research challenges, Ad interaction, in: SALAD@ ESWC, 2015, pp. 10–19. hoc networks 10(7) (2012), 1497–1516. doi:10.1016/j.adhoc. [64] M.S. Kim, J.G. Lee and S.J. Kang, A self-organizing interac- 2012.02.016. tion and synchronization method between a wearable device [81] F. Mizoguchi, H. Hiraishi and H. Nishiyama, Human-robot and mobile robot, Sensors 16(6) (2016), 842. doi:10.3390/ collaboration in the smart office environment, in: Advanced s16060842. Intelligent Mechatronics, IEEE/ASME International Confer- [65] T. Körtner, Ethical challenges in the use of social service ence on, IEEE, 1999, pp. 79–84. robots for elderly people, Zeitschrift für Gerontologie und [82] M. Mozaffari, W. Saad, M. Bennis and M. Debbah, Mobile Geriatrie 49(4) (2016), 303–307. doi:10.1007/s00391-016- Internet of Things: Can UAVs provide an energy-efficient 1066-5. mobile architecture? in: Global Communications Conference [66] A. Koubya and B. Quershi, Dronetrack: Cloud-based real- (GLOBECOM), IEEE, 2016, pp. 1–6. time object tracking using unmanned aerial vehicles, IEEE [83] M. Mozaffari, W. Saad, M. Bennis and M. Debbah, Mo- Access (2018). bile unmanned aerial vehicles (UAVs) for energy-efficient [67] R. Limosani, A. Manzi, L. Fiorini, P. Dario and F. Cavallo, Internet of Things communications, IEEE Transactions on Connecting ROS and FIWARE: Concepts and tutorial, in: Wireless Communications 16(11) (2017), 7574–7589. doi:10. Robot Operating System (ROS), Springer, 2019, pp. 449–475. 1109/TWC.2017.2751045. doi:10.1007/978-3-319-91590-6_13. [84] G. Mulligan, The 6LoWPAN architecture, in: Proceedings of [68] S.W. Loke, The internet of flying-things: Opportunities and the 4th Workshop on Embedded Networked Sensors,ACM, challenges with airborne fog computing and mobile cloud in 2007, pp. 78–82. doi:10.1145/1278972.1278992. the clouds, 2015, arXiv preprint arXiv:1507.04492. [85] J. Nelis, T. Verschueren, D. Verslype and C. Develder, Dya- [69] X. Lu, W. Liu, H. Wang and Q. Sun, Robot control design mand: Dynamic, adaptive management of networks and de- based on smartphone, in: Control and Decision Conference vices, in: 37th Annual IEEE Conference on Local Com- (CCDC), 25th Chinese, IEEE, 2013, pp. 2820–2823. doi:10. puter Networks, 2012, pp. 192–195. doi:10.1109/LCN.2012. 1109/CCDC.2013.6561425. 6423604. [70] R. Lundh, L. Karlsson and A. Saffiotti, Dynamic self- [86] K.T. Nguyen, M. Laurent and N. Oualha, Survey on se- configuration of an ecology of robots, in: IEEE/RSJ Interna- cure communication protocols for the Internet of Things, Ad tional Conference on Intelligent Robots and Systems, 2007, Hoc Networks 32 (2015), 17–31. doi:10.1016/j.adhoc.2015. pp. 3403–3409. 01.006. [71] R.C. Luo, K.L. Su, J.H. Tzou and S.H. Phang, Multisensor [87] J.Á. Noguera-Arnaldos, M.A. Paredes-Valverde, M.P. Salas- based control of pet robot through the Internet, in: Industrial Zárate, M.Á. Rodríguez-García, R. Valencia-García and Electronics Society (IECON), 27th Annual Conference of the, J.L. Ochoa, im4Things: An ontology-based natural language Vol. 1, IEEE, 2001, pp. 416–421. interface for controlling devices in the Internet of Things, [72] S. Maeyama, S. Yuta and A. Harada, Remote viewing on in: Current Trends on Knowledge-Based Systems, Springer, the web using multiple mobile robotic avatars, in: Intelligent 2017, pp. 3–22. doi:10.1007/978-3-319-51905-0_1. A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things 511

[88] J.H. Nord, A. Koohang and J. Paliszkiewicz, The Internet of monitoring, IET wireless sensor systems 7(5) (2017), 155– Things: Review and theoretical framework, Expert Systems 162. doi:10.1049/iet-wss.2017.0032. with Applications (2019). [103] N. Seydoux, K. Drira, N. Hernandez and T. Monteil, IoT- [89] L.E. Parker and F. Tang, Building multirobot coalitions O, a core-domain IoT ontology to represent connected de- through automated task solution synthesis, Proceedings of the vices networks, in: European Knowledge Acquisition Work- IEEE 94(7) (2006), 1289–1305. doi:10.1109/JPROC.2006. shop, Springer, 2016, pp. 561–576. 876933. [104] Z. Shelby, K. Hartke and C. Bormann, The constrained appli- [90] E. Paulos and J. Canny, Delivering real reality to the world cation protocol (CoAP), Technical report, 2014. wide web via telerobotics, in: Robotics and Automation, IEEE [105] M.S. Shin, B.C. Kim, S.M. Hwang and M.C. Ko, Design and International Conference on, Vol. 2, 1996, pp. 1694–1699. implementation of IoT-based intelligent surveillance robot, doi:10.1109/ROBOT.1996.506956. Studies in Informatics and Control 25(4) (2016), 422. doi:10. [91] K.H. Petersen, R. Nagpal and J.K. Werfel, Termes: An au- 24846/v25i4y201603. tonomous robotic system for three-dimensional collective [106] B. Siciliano, L. Sciavicco, L. Villani and G. Oriolo, Robotics: construction, 2011. Modelling, Planning and Control, Springer Science and Busi- [92] D. Pfisterer, K. Romer, D. Bimschas, O. Kleine, R. Mi- ness Media, 2010. etz, C. Truong, H. Hasemann, A. Kr’oller,´ M. Pagel, [107] R. Simmons, R. Goodwin, S. Koenig, J. O’Sullivan and M. Hauswirth et al., SPITFIRE: Toward a semantic web of G. Armstrong, Xavier: An autonomous mobile robot on the things, IEEE Communications Magazine 49(11) (2011), 40– web, Beyond Webcams: an introduction to online robots 48. doi:10.1109/MCOM.2011.6069708. (2002), 81. [93] I. Plauska and R. Damaševicius,ˇ Educational robots for [108] P. Simoens, M. Dragone and A. Saffiotti, The Internet of Internet-of-Things supported collaborative learning, in: Inter- Robotic Things: A review of concept, added value and national Conference on Information and Software Technolo- applications, International Journal of Advanced Robotic gies, Springer, 2014, pp. 346–358. doi:10.1007/978-3-319- Systems 15(1) (2018), 1729881418759424. doi:10.1177/ 11958-8_28. 1729881418759424. [94] E. Prestes, J.L. Carbonera, S.R. Fiorini, V.A. Jorge, M. Abel, [109] P. Simoens, C. Mahieu, F. Ongenae, F. De Backere, S. De R. Madhavan, A. Locoro, P. Goncalves, M.E. Barreto, Pestel, J. Nelis, F. De Turck, S.A. Elprama, K. Kilpi, C. Jew- M. Habib et al., Towards a core ontology for robotics and au- ell et al., Internet of robotic things: Context-aware and per- tomation, Robotics and Autonomous Systems 61(11) (2013), sonalized interventions of assistive social robots, in: 5th IEEE 1193–1204. doi:10.1016/j.robot.2013.04.005. International Conference on Cloud Networking (Cloudnet), [95] M. Quigley, K. Conley, B. Gerkey, J. Faust, T. Foote, J. Leibs, 2016, pp. 204–207. R. Wheeler and A.Y. Ng, ROS: An open-source robot oper- [110] W. Stallings, Computer Networking with Internet Protocols ating system, in: ICRA Workshop on Open Source Software, and Technology, Pearson/Prentice Hall, Upper Saddle River, Vol. 3, Kobe, Japan, 2009. NJ, USA, 2004. [96] P.P. Ray, Internet of robotic things: Concept, technologies, [111] M.R. Stein, Painting on the world wide web: The PumaPaint and challenges, IEEE Access 4 (2016), 9489–9500. doi:10. project, in: Telemanipulator and Telepresence Technologies 1109/ACCESS.2017.2647747. V, Vol. 3524, International Society for Optics and Photonics, [97] C. Razafimandimby, V. Loscri and A.M. Vegni, A neural net- 1998, pp. 201–210. doi:10.1117/12.333686. work and IoT based scheme for performance assessment in [112] A. Steinfeld, T. Fong, D. Kaber, M. Lewis, J. Scholtz, Internet of robotic things, in: 2016 IEEE First International A. Schultz and M. Goodrich, Common metrics for Conference on Internet-of-Things Design and Implementa- human-robot interaction, in: Proceedings of the 1st ACM tion (IoTDI), IEEE, 2016, pp. 241–246. doi:10.1109/IoTDI. SIGCHI/SIGART Conference on Human–Robot Interaction, 2015.10. ACM, 2006, pp. 33–40. doi:10.1145/1121241.1121249. [98] J.J. Rodrigues and P.A. Neves, A survey on IP-based wireless [113] B.L.R. Stojkoska and K.V. Trivodaliev, A review of Internet sensor network solutions, International Journal of Communi- of Things for smart home: Challenges and solutions, Journal cation Systems 23(8) (2010), 963–981. of Cleaner Production 140 (2017), 1454–1464. doi:10.1016/ [99] A. Saffiotti and M. Broxvall, PEIS ecologies: Ambient in- j.jclepro.2016.10.006. telligence meets autonomous robotics, in: Proceedings of the [114] H. Suo, J. Wan, C. Zou and J. Liu, Security in the Internet Joint Conference on Smart Objects and Ambient Intelligence: of Things: A review, in: Computer Science and Electronics Innovative Context-Aware Services: Usages and Technolo- Engineering (ICCSEE), International Conference on,Vol.3, gies, ACM, 2005, pp. 277–281. IEEE, 2012, pp. 648–651. [100] P. Saucy and F. Mondada, KhepOnTheWeb: Open access to [115] N. Takanobu, R. Soyama, A. Takanishi, K. Ohtsuki, D. Ozawa a mobile robot on the Internet, IEEE robotics & automation and M. Ohnishi, Remote treatment with mouth opening and magazine 7(1) (2000), 41–47. doi:10.1109/100.833574. closing training robot, in: Intelligent Robots and Systems [101] L. Schor, P. Sommer and R. Wattenhofer, Towards a zero- (IROS), IEEE/RSJ International Conference on, Vol. 1, IEEE, configuration wireless sensor network architecture for smart 2000, pp. 553–558. buildings, in: First ACM Workshop on Embedded Sensing [116] K. Tanaka, E. Nakagawa, M. Ito, N. Mizuno, T. Yamada, Systems for Energy-Efficiency in Buildings, 2009, pp. 31–36. E. Shimizu and K. Kagayama, An Internet-based tele-robot [102] V. Scilimati, A. Petitti, P. Boccadoro, R. Colella, D. Di Paola, environment for a time critical task, in: Systems, Man, and Cy- A. Milella and L.A. Grieco, Industrial Internet of Things at bernetics (SMC), IEEE International Conference on,Vol.5, work: A case study analysis in robotic-aided environmental 1999, pp. 1106–1110. 512 A. Kamilaris and N. Botteghi / The penetration of Internet of Things in robotics: Towards a web of robotic things

[117] M. Tenorth and M. Beetz, A unified representation for rea- [127] M. Waibel, M. Beetz, J. Civera, R. d’Andrea, J. Elfring, soning about robot actions, processes, and their effects on ob- D. Galvez-Lopez, K. Häussermann, R. Janssen, J. Montiel, jects, in: IEEE/RSJ International Conference on Intelligent A. Perzylo et al., Roboearth, IEEE Robotics and Automa- Robots and Systems, IEEE, 2012, pp. 1351–1358. tion Magazine 18(2) (2011), 69–82. doi:10.1109/MRA.2011. [118] S. Thrun, M. Bennewitz, W. Burgard, A.B. Cremers, F. Del- 941632. laert, D. Fox, D. Hahnel, C. Rosenberg, N. Roy, J. Schulte [128] J. Wan, S. Tang, Q. Hua, D. Li, C. Liu and J. Lloret, Context- et al., MINERVA: A second-generation museum tour-guide aware cloud robotics for material handling in cognitive indus- robot, in: Robotics and Automation, IEEE International Con- trial Internet of Things, IEEE Internet of Things Journal 5(4) ference on, Vol. 3, IEEE, 1999. (2018), 2272–2281. doi:10.1109/JIOT.2017.2728722. [119] F. Tian, A supply chain traceability system for food safety [129] J. Wan, S. Tang, Z. Shu, D. Li, S. Wang, M. Imran and based on HACCP, blockchain & Internet of Things, in: Inter- A.V. Vasilakos, Software-defined industrial Internet of Things national Conference on Service Systems and Service Manage- in the context of industry 4.0, IEEE Sensors Journal 16(20) ment, IEEE, 2017, pp. 1–6. (2016), 7373–7380. [120] R. Toris, J. Kammerl, D.V. Lu, J. Lee, O.C. Jenkins, S. Osen- [130] J. Wang, E. Schluntz, B. Otis and T. Deyle, A new vision toski, M. Wills and S. Chernova, Robot web tools: Efficient for smart objects and the internet of things: Mobile robots messaging for cloud robotics, in: IROS, 2015, pp. 4530–4537. and long-range UHF RFID sensor tags, 2015, arXiv preprint arXiv:1507.02373. [121] C. Turcu and C. Turcu, The social Internet of Things and the [131] L. Wang, A. Mohammed and M. Onori, Remote robotic as- RFID-based robots, in: Ultra Modern Telecommunications sembly guided by 3D models linking to a real robot, CIRP and Control Systems and Workshops (ICUMT), 4th Interna- Annals-Manufacturing Technology 63(1) (2014), 1–4. doi:10. tional Congress on, IEEE, 2012, pp. 77–83. 1016/j.cirp.2014.03.013. [122] C. Turcu, C. Turcu and V. Gaitan, Integrating robots into the [132] T. Watteyne, X. Vilajosana, B. Kerkez, F. Chraim, K. Weekly, Internet of Things, International Journal of Circuits, Systems Q. Wang, S. Glaser and K. Pister, OpenWSN: A standards- and Signal Processing 6(6) (2012), 430–437. based low-power wireless development environment, Trans- [123] S. Valluri, A.G. Sagar and S.G. Shanmugam, IOT based actions on Emerging Telecommunications Technologies 23(5) real time monitoring mobile robot, International Journal for (2012), 480–493. doi:10.1002/ett.2558. Research in Applied Science and Engineering Technology [133] R.H. Weber and R. Weber, Internet of Things, Vol. 12, (IJRASET) 5(V) (2017), 105–109. Springer, 2010. [124] J.I. Vazquez, J. Ruiz-de-Garibay, X. Eguiluz, I. Doamo, [134] E. Wilde, Putting things to REST, 2007. S. Rentería and A. Ayerbe, Communication architectures and [135] T. Winter, P. Thubert, A. Brandt, J. Hui, R. Kelsey, P. Levis, experiences for web-connected physical smart objects, in: K. Pister, R. Struik, J.-P. Vasseur and R. Alexander, RPL: 2010 8th IEEE International Conference on Pervasive Com- IPv6 routing protocol for low-power and lossy networks, puting and Communications Workshops (PERCOM Work- Technical report, 2012. shops), IEEE, 2010, pp. 684–689. doi:10.1109/PERCOMW. [136] L. Wood, Service robots: The next big productivity plat- 2010.5470521. form, 2016, https://usblogs.pwc.com/emerging-technology/ [125] O. Vermesan, A. Bröring, E. Tragos, M. Serrano, D. Bacciu, service-robots-the-next-big-productivity-platform/. S. Chessa, C. Gallicchio, A. Micheli, M. Dragone, A. Saf- [137] Y. Yuehong, Y. Zeng, X. Chen and Y. Fan, The Internet of fiotti et al., Internet of robotic things: Converging sens- Things in healthcare: An overview, Journal of Industrial In- ing/actuating, hypoconnectivity, artificial intelligence and IoT formation Integration 1 (2016), 3–13. doi:10.1016/j.jii.2016. platforms, in: Cognitive Hyperconnected Digital Transforma- 03.004. tion: Internet of Things Intelligence Evolution, 2017, pp. 97– [138] J. Zander, P.J. Mosterman, T. Padir, Y. Wan and S. Fu, Cyber- 155. physical systems can make emergency response smart, Proce- [126] J. Vertut, Teleoperation and Robotics: Applications and Tech- dia Engineering 107 (2015), 312–318. doi:10.1016/j.proeng. nology, Vol. 3, Springer Science and Business Media, 2013. 2015.06.086.