Collaborative Multi-Robot Systems for Search and Rescue: Coordination and Perception
Total Page:16
File Type:pdf, Size:1020Kb
1 Collaborative Multi-Robot Systems for Search and Rescue: Coordination and Perception Jorge Pena˜ Queralta1, Jussi Taipalmaa2, Bilge Can Pullinen2, Victor Kathan Sarker1, Tuan Nguyen Gia1, Hannu Tenhunen1, Moncef Gabbouj2, Jenni Raitoharju2, Tomi Westerlund1 1Turku Intelligent Embedded and Robotic Systems, University of Turku, Finland Email: 1fjopequ, vikasar, tunggi, toveweg@utu.fi 2Department of Computing Sciences, Tampere University, Finland Email: 2fjussi.taipalmaa, bilge.canpullinen, moncef.gabbouj, jenni.raitoharjug@tuni.fi Abstract—Autonomous or teleoperated robots have been play- ing increasingly important roles in civil applications in recent years. Across the different civil domains where robots can sup- port human operators, one of the areas where they can have more impact is in search and rescue (SAR) operations. In particular, multi-robot systems have the potential to significantly improve the efficiency of SAR personnel with faster search of victims, initial assessment and mapping of the environment, real-time monitoring and surveillance of SAR operations, or establishing emergency communication networks, among other possibilities. SAR operations encompass a wide variety of environments and situations, and therefore heterogeneous and collaborative multi- robot systems can provide the most advantages. In this paper, we review and analyze the existing approaches to multi-robot (a) Maritime search and rescue with UAVs and USVs SAR support, from an algorithmic perspective and putting an emphasis on the methods enabling collaboration among the robots as well as advanced perception through machine vision and multi-agent active perception. Furthermore, we put these algorithms in the context of the different challenges and constraints that various types of robots (ground, aerial, surface or underwater) encounter in different SAR environments (maritime, urban, wilderness or other post-disaster scenarios). This is, to the best of our knowledge, the first review considering heterogeneous SAR robots across different environments, while giving two complimentary points of view: control mechanisms and machine perception. Based on our review of the state-of-the-art, we discuss the main open research questions, and outline our insights on the current approaches that have potential to improve the real-world performance of multi-robot SAR systems. (b) Urban search and rescue with UAVs and UGVs Index Terms—Robotics, search and rescue (SAR), multi-robot systems (MRS), machine learning (ML), active perception, active arXiv:2008.12610v1 [cs.RO] 28 Aug 2020 vision, multi-agent perception, autonomous robots. I. INTRODUCTION Autonomous robots have seen an increasing penetration across multiple domains in the last decade. In industrial environments, collaborative robots are being utilized in the manufacturing sector, and fleets of mobile robots are swarming in logistics warehouses. Nonetheless, their utilization within civil applications presents additional challenges owing to the interaction with humans and their deployment in potentially unknown environments [1]–[3]. Among civil applications, (c) Wilderness search and rescue with support UAVs search and rescue (SAR) operations present a key scenario where autonomous robots have the potential to save lives by Fig. 1: Different search and rescue scenarios where heteroge- enabling faster response time [4], [5], supporting in hazardous neous multi-robot systems can assist SAR taskforces. environments [6]–[8], or providing real-time mapping and [10], among other possibilities. In this paper, we perform a monitoring of the area where an incident has occurred [9], literature review of multi-robot systems for SAR scenarios. 2 System Level Perspective of Multi-Robot SAR Systems Equipment Operational Human Shared Communication and Sensors Environments Detection Autonomy Section III-A Section III-B Section III-C Section III-D Section III-E (a) Aspects of multi-robot SAR systems discussed in Section III of this paper. Algorithmic Perspective of Multi-Robot SAR Systems Coordination Algorithms Perception Algorithms Formation Control Multi-Agent Decision Active Multi-Agent Segmentation and Multi-Modal and Area Coverage Making and Planning Perception Object Detection Sensor Fusion Section IV-A to IV-C Section IV-D to IV-H Section V-A to V-C Section V-D to V-E Section IV Section VI Section V (b) Division of multi-robot SAR systems into separate components from an algorithmic point of view. Control, planning and coordination algorithms are described in Section IV, while Section V reviews perception algorithms from a machine learning perspective. Section VI then puts these two views together by reviewing the works in single and multi-agent active perception. Fig. 2: Summary of the different aspects of multi-robot SAR systems considered in this survey, where we have separated (a) system-level perspective, and (b) planning and perception algorithmic perspective. These systems involve SAR operations in a variety of envi- of multi-UAV systems from the point of view of communica- ronments, some of which are illustrated in Fig. 1. With the tion, and for a wide range of applications from construction wide variability of SAR scenarios, different situations require or delivery to SAR missions. An extensive classification robots to be able to operate in different environments. In of previous works is done taking into account the mission this document, we utilize the following standard notation to and network requirements in terms of data type, frequency, refer to the different types of robots: unmanned aerial vehi- throughput and quality of service (latency and reliability). cles (UAVs), unmanned ground vehicles (UGVs), unmanned In comparison to [11], our focus is on multi-robot systems surface vehicles (USVs), and unmanmed underwater vehicles including also ground, surface, or underwater robots. Another (UUVs). These can be either autonomous or teleoperated, recent review related to civil applications for UAVs was carried and very often a combination of both modalities exists when out in [3].In [3], the authors provide a classification in terms considering heterogeneous multi-robot systems. In maritime of technological trends and algorithm modalities utilized in SAR, autonomous UAVs and USVs can support in finding research papers: collision avoidance, mmWave communication victims (Fig. 1a). In urban scenarios, UAVs can provide real- and radars, cloud-based offloading, machine learning, image time information for assessing the situation and UGVs can processing and software-defined networking, among others. access hazardous areas (Fig. 1b). In mountain scenarios, UAVs A recent survey [12] focused on UAVs for SAR operations, can help in monitoring and getting closer to the victims that with an extensive classification of research papers based on (i) are later rescued by a helicopter (Fig. 1c). sensors utilized onboard the UAVs, (ii) robot systems (single In recent years, multiple survey papers addressing the uti- or multi-robot systems, and operational mediums), and (iii) lization of multi-UAV systems for civil applications have been environment where the system is meant to be deployed. In a published. In [11], the authors perform an exhaustive review study from Grayson et al. [13], the focus is on using multi- 3 robot systems for SAR operations, with an emphasis on task summary, our contribution focuses on reviewing the different allocation algorithms, communication modalities, and human- aspects of multi-robot SAR operations with robot interaction for both homogeneous and heterogeneous 1) a system-level perspective for designing autonomous multi-robot systems. In this work, we review also heteroge- SAR robots considering the operational environment, neous multi-robot systems. However, rather than focusing on communication, level of autonomy, and the interaction describing the existing solutions at a system level, we put an with human operators, emphasis on the algorithms that are being used for multi-robot 2) an algorithmic point of view of multi-robot coordination, coordination and perception. Moreover, we describe the role multi-robot search and area coverage, and distributed of machine learning in single and multi-agent perception, and task allocation and planning applied to SAR operations, discuss how active perception can play a key role towards 3) a deep learning viewpoint to single- and multi-agent the development of more intelligent robots supporting SAR perception, with a focus on object detection and tracking operations. The survey is further divided into three main and segmentation, and a description of the challenges sub-categories: 1) planning and area coverage algorithms, and opportunities of active perception in multi-robot 2) machine perception, and 3) active perception algorithms systems for SAR scenarios. combining the previous two concepts (Fig. 2). The remainder of this paper is organized as follows: Sec- While autonomous robots are being increasingly adopted tion II describes some of the most relevant projects in SAR for SAR missions, current levels of autonomy and safety of robotics, with an emphasis on those considering multi-robot robotic systems only allow for full autonomy in the search systems. In Section III, we present a system view on SAR part, but not for rescue, where human operators need to robotic systems, describing the different types of robots being intervene [14]. This leads to the design of shared autonomy utilized, particularities of SAR environments, and different interfaces and