UNIVERSIDAD POLITÉCNICA DE MADRID ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES A NOVEL ROBOTIC FRAMEWORK FOR SAFE INSPECTION AND TELEMANIPULATION IN HAZARDOUS AND UNSTRUCTURED ENVIRONMENTS DOCTOR OF PHILOSOPHY AUTOMATION AND ROBOTICS MARIO DI CASTRO September 27, 2019 UNIVERSIDAD POLITÉCNICA DE MADRID ESCUELA TÉCNICA SUPERIOR DE INGENIEROS INDUSTRIALES CERN - EUROPEAN ORGANIZATION FOR NUCLEAR RESEARCH A NOVEL ROBOTIC FRAMEWORK FOR SAFE INSPECTION AND TELEMANIPULATION IN HAZARDOUS AND UNSTRUCTURED ENVIRONMENTS DOCTOR OF PHILOSOPHY AUTOMATION AND ROBOTICS Author: Mario Di Castro Electronic Engineer Advisors: Dr. Manuel Ferre Pérez Universidad Politécnica de Madrid Dr. Alessandro Masi CERN - European Organization for Nuclear Research September 27, 2019 A NOVEL ROBOTIC FRAMEWORK FOR SAFE INSPECTION AND TELEMANIPULATION IN HAZARDOUS AND UNSTRUCTURED ENVIRONMENTS Autor: Mario Di Castro Tribunal Presidente: Dr. Claudio Rossi Secretario: Dr. Paloma de la Puente Vocal A: Dr. Cristina Adorisio Vocal B: Dr. Antonio Giménez Fernández Vocal C: Dr. Raúl Marín Prades Suplente A: Dr. Marco Pezzetti Suplente B: Dr. Concepción Alicia Monje Micharet Acuerdan otorgar la calificatión de: Madrid, de, 2019 Acknowledgments First of all, I would like to express my deep gratitude to my research supervisor, Professor Manuel Ferre Pérez, for his patient guidance, enthusiastic encouragement and useful critiques of this thesis. An immense thankfulness goes to my CERN supervisor, Doctor Alessandro Masi, who has believed in my potential and who has always supported me during the last years, being my mentor and my inspiration. Thanks to the unconditional support of Roberto Losito for the direction of the Engineering Department at CERN, that provides extraordinary opportunities and individual development to its scientists. I would like to express my great appreciation and deep respect to Professor Raul Marin for the support for writing of the thesis, help and profound discussions on teleoperation and robotic controls. Also, I would like to offer my special thanks to my colleagues of the Mechatronics, Robotics and Operation Section within the Survey, Mechatronics and Measurement Group at CERN for their inestimable help in the building-up of this work. In the specific, Giacomo Lunghi, Luca Buonocore, Carlos Veiga Almagro, Leanne Attard, Jorge Camarero Vera, Zsolt Pasztori, Santiago Solis Paiva, Arturs Ivanovs and Maria Laura Baiguera Tambutti. My grateful thanks are directed to all collegues at CERN, and in the specific the Radioprotection group, that inspired me with problems and discussions. My warmest thanks go to my children Giada, Lorenzo and Federico, for bringing joy and source of inspiration to reach what looks impossible. The deepest thanks goes to my wife Valeria, that has supported me during these years with trust and love, to her I dedicate this thesis. iii Abstract Intelligent robotic systems are becoming essential for space applications, industries, nu- clear plants and for harsh environments in general, such as the European Organization for Nuclear Research (CERN) particles accelerator complex and experiments. Robotics tech- nology has huge potential benefits for people and its ultimate scope depends on the way this technology is used. In order to increase safety and machine availability, robots can perform repetitive, unplanned and dangerous tasks, which humans either prefer to avoid or are unable to carry out due to hazards, size constraints, or the extreme environments in which they take place. Nowadays, mechatronic systems use mature technologies that allow their robust and safe use, even in collaboration with human workers. Over the past years, the progress of robots has been based on the development of smart sensors, artificial intel- ligence and modular mechanical systems. Due to the multiple challenges that hazardous and unstructured environments have for the application of autonomous industrial systems, there is still a high demand for intelligent and teleoperation systems that give the con- trol of a robot (slave) to a human operator via haptic input devices (master), as well as using human-supervised telerobotic control techniques. Modern techniques like simulation and virtual reality systems can facilitate the preparation of ad-hoc mechatronic tools and robotic intervention including recovery scenarios and failure mode analysis. The basic contribution of this thesis is the development of a novel robotic framework for autonomous inspections and supervised teleoperations in harsh environments. The proposed framework covers all aspects of a robotic intervention, from the specification and operator training, the choice of the robot and its material in accordance with possible radiological contamination risks, to the realization of the intervention, including procedures and recov- ery scenarios. In a second set of contributions, new methods for mutirobots maintenance operations are developed, including intervention preparation and best practices for remote handling and advanced tools. The third set of contributions is built on a novel multimodal user-friendly human-robot interface that allows operator training using virtual reality sys- tems and technicians not expert in robot operation to perform inspection/maintenance tasks. In this thesis, we exploit a robotic system able to navigate autonomously and to inspect unknown environments in a safe way. A new real-time control system has been implemented in order to guarantee a fast response to environmental changes and adaptation to different type of scenarios the robot may find in a semi-structured and hazardous environment. The proposed new robotic control system has been integrated on different robots, tested and validated with several robotic interventions in the CERN hazardous particle accelerator complex. v Resumen Los sistemas robóticos inteligentes se están convirtiendo en algo esencial para las aplica- ciones espaciales, industriales, las centrales nucleares y los entornos hostiles en general, tales como las complejas instalaciones del acelerador de partículas de la Organización Europea para la Investigación Nuclear (CERN) y sus experimentos. La tecnología robótica tiene un enorme potencial para el beneficio de las personas y su ámbito de aplicación final depende de la forma en que esta es utilizada. Con el fin de incrementar la seguridad y disponibilidad de las maquinas, los robots pueden llevar a cabo tareas repetitivas, inesperadas y peligrosas, las cuales prefieren ser evitadas por los humanos o bien, son incapaces de realizarlas debido a tales peligros, limitaciones de espacio, o los entornos extremos donde tienen lugar. En la actualidad, los sistemas de mecatrónica utilizan tecnologías consolidadas que les permite un uso robusto y seguro, incluso en colaboración con trabajadores humanos. En los úl- timos años, el avance de los robots se ha basado en el desarrollo de sensores inteligentes, inteligencia artificial y sistemas mecánicos modulares. Debido a la multitud de desafíos que los entornos peligrosos y desestructurados generan a la hora de aplicar sistemas industriales autónomos, todavía existe una amplia demanda de sistemas inteligentes y tele-operados que entregan el control de un robot (esclavo) a un operador humano a través de dispositivos de entrada hápticos (maestro). Técnicas modernas como los sistemas de simulación y la realidad virtual pueden facilitar la preparación de herramientas mecatrónicas puntuales e intervenciones robóticas, incluyendo escenarios de recuperación y modo de análisis de fallos. La contribución base de esta tesis es el desarrollo de un novedoso framework robótico para inspecciones autónomas y teleoperaciones supervisadas en entornos hostiles. El framework propuesto cubre los aspectos de una intervención robótica al completo, desde la especificación y adiestramiento del operador, la elección del robot y sus materiales acorde con los posi- bles riesgos de contaminación radiológica, hasta la realización de la intervención, incluyendo procedimientos y planes de recuperación. En un segundo conjunto de contribuciones, se de- sarrolla un método para las operaciones de mantenimiento mediante multirobots, incluyendo la preparación de las intervenciones y las mejores prácticas para la manipulación remota y herramientas avanzadas. El tercer grupo de contribuciones se basa en una nueva interfaz multimodal humano-robot fácil de usar, la cual permite entrenar operadores inexpertos en operaciones robóticas, haciendo uso de sistemas y técnicas de realidad virtual, para tareas de inspección y/o mantenimiento. En esta tesis, explotamos un sistema robótico capaz de navegar de forma autónoma, inspeccionando entornos desconocidos de una forma segura. Un nuevo sistema de control en tiempo real ha sido implementado con el fin de garantizar vi una respuesta rápida a los cambios del entorno y la adaptación a los diferentes tipos de escenarios que el robot podría encontrar en un entorno hostil y semiestructurado. El nuevo sistema para el control de robots propuesto, ha sido integrado en diferentes robots, además de testeado y validado en un gran número de intervenciones robóticas dentro de las instalaciones del complejo acelerador de partículas del CERN. Contents Acknowledgment ii Abstract v Resumen vi 1 Introduction1 1.1 Remote Interventions . 3 1.2 Motivation . 5 1.3 Thesis Hypothesis and Contributions . 7 1.4 List of Publications . 8 1.5 Outline . 11 2 State of the Art on Robotic Systems 12 2.1 Artificial Intelligence . 17 2.2 Telemanipulation . 19 2.3 Haptic Telepresence