Contributions to Adaptive Mission Planning for Cooperative Robotics in the Internet of Things

Universidad Politécnica de Madrid Departamento de Ingeniería Telemática y Electrónica

Author: Néstor Lucas Martínez, MSc

Supervisor: José Fernán Martínez Ortega, PhD

This dissertation is submitted for the degree of PhD. in Systems and Services Engineering for the Information Society

Escuela Técnica Superior de Ingeniería y Sistemas de Telecomunicación March 2021

DOCTORADO EN INGENIERÍA DE SISTEMAS Y SERVICIOS PARA LA SOCIEDAD DE LA INFORMACIÓN

Tesis Doctoral Título Contributions to Adaptive Mission Planning for Cooperative Robotics in the Internet of Things Autor Néstor Lucas Martínez

VºBº Director Dr. José-Fernán Martínez Ortega

Tribunal Presidente M. Encarnación Pastor Martín

Secretario Lourdes López Santidrián

Vocal Celeste Campo Vázquez

Vocal Miguel Ángel López Carmona

Vocal Roemi Emilia Fernández Saavedra

Suplente Juan Ramón Velasco Pérez

Suplente Rodolfo E. Haber Guerra

Lugar de lectura E.T.S.I. y Sistemas de Telecomunicación (U.P.M.)

Fecha de lectura

Calificación

El Presidente El secretario Los vocales

Tesis Doctoral para la obtención del título de Doctor por la Universidad Politécnica de Madrid

This Ph.D. Thesis is dedicated to my family and friends.

With a special memory to all the people affected by the COVID-19 global pandemic.

Declaration

I hereby declare that except where specific reference is made to the work of others, the contents of this dissertation are original and have not been submitted in whole or in part for consideration for any other degree or qualification in this, or any other university. This dissertation is my own work and contains nothing which is the outcome of work done in collaboration with others, except as specified in the text and Acknowledgements. This dissertation contains fewer than 80,000 words including appendices, bibliography, footnotes, tables and equations and has fewer than 150 figures.

Néstor Lucas Martínez, MSc March 2021

Acknowledgements

I would like to share with you part of my journey to reach this dissertation. One could say that it all started when I was a little kid. One of my first memories is about wondering about how things worked, always looking and asking for understandable and useful information. My parents always encouraged me to do so, as some of my teachers did too. I remember searching and reading about mathematics, physics, and when I got my first computer, technology.

Being a teenager I learned the basics of computer programming. I even read some books and computer magazines where they talked about artificial intelligence, and I learned some basic concepts about tree search and expert systems. I also was interested in electronics and communications, and despite not knowing at the time the exact definition of “telematics” (coined in the Nora-Minc report, as I learned later), I decided to continue my studies with a Telecommunications Engineering degree majoring in Telematics.

At the same time, I was admitted to the university, I was also admitted as a civil defense volunteer in the Madrid civil defense volunteer corps. As a volunteer I was promoted multiple times, assuming increasing responsibilities, which included coordinating services and managing other volunteers. It was during this time when I learned about the importance of “mission management for multiple agents” in environments where adaptation to the context was required.

Going back to the academic part, I always wanted to do a PhD and dedicate myself to teaching and research. At the time I entered university, following this path was quite complicated, since there were hardly any opportunities.

Time passed and I wondered what I would do when I got my undergraduate degree. I was not attracted to most of the job possibilities at the time, and the opportunities to pursue a PhD, or even just doing some research were scarce. But I had to finish my studies, and to do so, I had to do my final project.

Looking for offers to do my final degree project, one of them caught my attention. It was about doing some work related to ubiquitous computing, and with the possibility of continuing after graduation. I applied, and the professor in charge of the offer called me to interview me.

I will always remember that interview. As I said before, I was not too interested in the usual job offers I could find related to my studies. The main reason was that they were essentially related to just programming at upper layers, with little to no relation to telematics. However, this professor explained to me what they were doing in their research group. Without knowing it, it was precisely what I had been looking for. But there was more. Once we agreed on doing my final project with them, he also asked me about my interest in pursuing a PhD. You can figure out that I was more than pleased to answer that question, confirming my interest.

That is when I started my research career at the Next-Generation Networks and Services research group (GRyS), and the professor that interviewed me was José Fernán Martínez, or JF, as we all call him, and who is now my thesis supervisor.

Few years have passed since then. I have got a MSc in Systems and Services Engineer- ing while working at GRyS. I have participated in several European research projects, traveling around Europe for meetings, developments, integrations and demonstrations. I have met a lot of interesting people from whom I have learned a lot. As if that were not enough, in recent years I have also had the opportunity to carry out some teaching collaborations. And I have experienced that I really like both, teaching and researching.

I hope that this dissertation is not just the end of this journey, but just a required milestone to continue enjoying and learning doing research and teaching, and with my colleagues and students.

In closing, and making a good use of the title of this section, I would like to express the following acknowledgements:

• To my parents, Mayka and Clemente, for always supporting me, despite not having a background either in science or engineering. Just by feeding my curiosity they were able to guide me here.

• To Mónica, my partner in life, my significant other, for always being there, in good times and in bad times.

• To my friend-like-a-brother Antonio and his significant other, Cristina, for all the good that it means to me to have both of you by our side.

• To JF, I mean, “Professor José Fernán Martínez”, for the opportunity he has given me to carry out this thesis under his supervision, and for the infinite patience he has shown me during the whole process of writing this thesis.

• To the rest of the current members of the GRyS team, in no particular order: Lour- des, Pedro, Gregorio, Sara, José Antonio, Mario, Marta, and especially Vicente, with whom I have spent great time in all the projects in which we have collaborated, and with whom I have a pending ping pong game when we have the opportunity to do so.

• To the former members of the GRyS team, in no particular order: Daniel, Victoria, David, Carlos, José Antonio, Rubén, Esther, Alexandra, Jesús, and especially

– x – Raúl, from whom I learned a lot about self-adaption, control systems, and also about music and other things we talked about in our spare time together.

• To our former Chinese colleagues at GRyS, Huang Yuanjiang, Xin Li, Ning Li, Xin Yuan, Chen Yuwen and Zhaoyu Zhai. To all of them I want to say 非常感谢你的一切1 and 很高兴与您合作2. In the same sense, to our former colleague Saloum Jimbara, with whom I shared nice conversations and experiences while he was finishing his Master degree with us.

• To all the researchers and developers that I have met during my work in the several projects I have already participated in. The list is too long, but I would like to at least mention the following people:

– My colleagues at the Mälardalen University (MDH), especially my friends Afshin, Gita, Branko and Jean. No one will ever understand us when we laugh just talking about hot dogs. And let me tell you this: our game nights have been really helpful to disconnect, at least briefly, when I was doing the final changes to this manuscript. – Sonia, Belén, and the rest of the people from Tecnalia I worked with in the SWARMs and AFarCloud projects. It really has been a pleasure working with you, and I look forward to meeting you again in further projects. – Coen, Yolanda, Julio, Stefano, Roshan, Soumya and with a special mention to †Zoltan for always welcoming us so well when we traveled to The Hague for the integrations of the DEMANES project. – Christophe, Natividad, Félix, Víctor Manuel, María, Rubén, Raimon, Carlos, Josep, Marc, Joan, Natxo, Miguel Lláçer and Miguel Montesinos for their support during the integrations of the WoO project.

• To my co-students during my master degree complementary subjects:

– Alejandro Duque, who was my partner in “Diseño Digital I”, with whom I spent countless moments talking and preparing the activities of the subject. – David Teva, who was my partner in “Diseño Digital II”, sharing the moments of stress during the final project for the subject. – Jorge Montes, who was my partner in “Sistemas Basados en Microproce- sador”, and who has also become my friend, and with whom I can have a nice technical conversation, or a funny geek talk.

1Thank you very much for everything. 2It has been a pleasure working with you.

– xi – • To Jorge García Paredes, who did his final degree project with us in relation to the contents of this thesis, resulting in interesting and fruitful conversations that helped me to advance with my dissertation. We will always have the talks about the description in PDDL about how to make a Spanish omelet, considering those who prefer it with onions, and those who prefer it without it. And as we said the last time we talked, Qapla’3

• To all the students I had the opportunity to collaborate with their teaching in subjects like “Software de Comunicaciones”, “Redes de Ordenadores”, “Señalización y Conmutación”, “Aplicaciones Telemáticas Avanzadas”, and more recently “Pro- gramación II”. I have learned a lot assisting in your learning, and I hope to have helped you at least a bit.

• Finally, to all those people that I have not mentioned, and who in one way or another have been there. There are so many of you that I’m sure some names have been overlooked. But rest assured that I remember and acknowledge all of you.

3Success, in Klingon language.

– xii – Abstract

In recent years, robotics has experienced a growing interest thanks to the impetus received by the advances on the various technologies on which it relies. Of all the aspects in which robotics is making its way, one of the most relevant is related to autonomous robotics, where robots are capable of performing assigned tasks with minimal human intervention. A simple example is the now common Unmanned Aerial Vehicle (UAV), capable of flying between points without the need for a human to carry out the piloting tasks. This ability to carry out assigned tasks with minimal human intervention has its main advantages in those tasks that are carried out in harsh, dangerous, or even distant environments.

The usual way of working with this type of robot starts with the definition of some goals resulting in what is known as a mission. A plan is defined to achieve the mission goals. In this context the definition of a plan is limited to a sequence of actions that therobot must carry out, without alternative branches of execution. This approach is acceptable when it is possible to control the conditions of the environment in which the plan is to be executed. However, the environments where there is greater interest for the use of autonomous robots, such the ones with peril or considerable distances, are usually open. This implies that in those environments may occur situations that prevent the correct execution of the plan, being necessary to adapt the mission to such situations.

Traditionally, the adaptation of a mission when situations that prevent the execution of the plan has done in two ways:

1. Delegating the ability to adapt to robots. 2. Updating the mission plan, either repairing it or creating a new one for the situation detected (re-planning).

Both options have their drawbacks. On the one hand delegation is not always possible, far from easy. And even in those cases in which a certain adaptive ability can be delegated to the robots, it is still possible that there are situations to which the robot cannot adapt. On the other hand, updating the mission plan is a time-consuming process, which would negatively affect the fulfillment of the mission. Furthermore, if several robots are participating cooperatively in a mission, it is possible that the situation detected by one of them requires the adaptation of the plan for others. And neither delegation, nor re-planning or plan repairing cover this possibility.

Additionally, there are other types of situations that can be detected during the execution of a mission that do not imply the need to adapt the plan, but rather the presence of an opportunity to achieve other desirable goals. This thesis proposes a contribution to the adaptation of mission plans for cooperative robotics within the framework of the Internet of Things (IoT), with the following objectives: 1) define an improved structure of a plan, compatible with its classic definition, and which allows the use of existing knowledge to anticipate possible adaptations, as well as to identify opportunities outside the original plan; 2) define a reference middleware architecture for mission management that, using the previous structure, serves as a guide for the design of concrete architectures for specific systems.

The new structure defined, called “strategy” in this dissertation, incorporates the classic structure of a plan complemented with the possible hierarchical decomposition of the actions that constitute it, the inclusion of decision nodes and the consideration of alternative plans for identified opportunities. This structure is complemented by the proposal of a common reference architecture for mission management, called “CoMMMA” in this thesis. CoMMMA includes the necessary functionalities to facilitate adaptation to events and detection of opportunities, maintaining a close relationship with the Internet of Things (IoT) reference model.

As proof of concept and validation of the proposal, this model has been used to define a mission manager component for the architecture of the SWARMs European Research Project. The SWARMs project was aimed to expand the use of underwater and surface autonomous robotics, using autonomous vehicles to carry out tasks in the underwater environment, in which the conditions of danger and distance are met. The manager component employs the necessary CoMMMA concepts that apply to the specific requirements of the project, and it has been successfully tested in the final demonstrator for the project, obtaining promising results.

The CoMMMA model presented in this thesis has also been used in the design of the mission management component for the architecture of the European Research Project AFarCloud, framed in the field of precision agriculture, and pending evaluation atthe time of writing these lines.

The foundations and outcomes presented in this dissertation are mainly contextualized in the following European Research Projects: WoO (ITEA2 code: 10028), DEMANES (Artemis code: 295372), ACCUS (Artemis code: 333020), SWARMs (ECSEL code: 662107) and AFarCloud (ECSEL code: 783221).

Keywords: robotics, ai planning, adaptive systems, decentralized adaptive control, multi-agent systems, mission management, internet of things.

– xiv – Resumen

En los últimos años la robótica ha experimentado un creciente interés gracias al impulso recibido por el avance de las diversas tecnologías en las que se apoya. De todos los aspectos en los que la robótica está abriéndose paso, uno de los más relevantes es el relacionado con la robótica autónoma, donde los robots son capaces de realizar las tareas asignadas con la mínima intervención humana. Un ejemplo sencillo es el ya habitual vehículo aéreo no tripulado (UAV), capaz de realizar vuelos entre puntos sin necesidad de que un humano realice las labores de pilotaje. Esta capacidad para llevar a cabo tareas encomendadas con mínima intervención humana presenta sus principales ventajas en aquellas tareas que se llevan a cabo en entornos peligrosos o distantes. La forma habitual de trabajo con este tipo de robots comienza con la definición de unos objetivos que dan lugar a lo que se conoce como misión. Para alcanzar los objetivos de la misión se define un plan, que en este contexto se limita a una secuencia fija de acciones que el robot debe llevar a cabo, sin alternativas ni ramificaciones. Este enfoque es aceptable cuando pueden controlarse las condiciones del entorno en el que va a ejecutarse el plan. Sin embargo, los entornos de mayor interés para el uso de robots autónomos, como son los ya citados entornos en los que exista un peligro o una distancia, suelen ser abiertos, implicando que sus condiciones no pueden ser controladas. Esto implica que pueden ocurrir situaciones que impidan la ejecución correcta del plan, dando lugar a la necesidad de adaptarse a dichas situaciones. La adaptación de la misión cuando ocurren situaciones que impiden la ejecución del plan se ha realizado tradicionalmente de dos formas:

1. Delegando la capacidad de adaptación en los robots. 2. Actualizando el plan de la misión, ya sea reparando el existente, o creando un nuevo para la situación detectada.

Ambas opciones tienen sus inconvenientes. Por un lado, la delegación no es siempre posible, ni mucho menos fácil. Y aun en aquellos casos en los que pueda delegarse una cierta capacidad adaptativa en los robots, sigue siendo posible que se den situaciones a las que el robot no pueda adaptarse. Por otro lado, la actualización del plan de la misión es un proceso que lleva tiempo, lo que afectaría negativamente al cumplimiento de la misión. Además, si son varios los robots los que están participando de forma cooperativa en una misión, es posible que la situación detectada por uno de ellos necesite de la adaptación del plan de otros. Y ni la delegación, ni la reparación o la regeneración del plan cubren esta posibilidad. Adicionalmente existen otro tipo de situaciones que pueden detectarse durante la ejecución de una misión que no impliquen la necesidad de adaptar el plan, sino la presencia de una oportunidad para alcanzar otros objetivos deseables. En esta tesis se propone una contribución a la adaptación de planes de misiones para la robótica cooperativa en el marco de Internet de las Cosas (IoT) con los siguientes objetivos: 1) definir una estructura mejorada de un plan, compatible consu definición clásica, y que permita utilizar el conocimiento existente para anticipar posibles adaptaciones, así como identificar oportunidades ajenas al plan original; 2) definir una arquitectura de intermediación de referencia para la gestión de misiones que, usando la estructura anterior, sirva de guía para el diseño de arquitecturas concretas para sistemas específicos. La nueva estructura definida, llamada «estrategia» en esta tesis, incorpora laes- tructura clásica de un plan complementada con la posible descomposición jerárquica de las acciones que lo constituyen, la inclusión de nodos de decisión y la consideración de planes alternativos para oportunidades identificadas. Esta estructura se complementa con la propuesta de una arquitectura común de referencia para la gestión de misiones, llamada «CoMMMA» en esta tesis. CoMMMA incluye las funcionalidades necesarias para facilitar la adaptación ante eventos y ante la detección de oportunidades, man- teniendo una estrecha relación con el modelo de referencia de Internet de las Cosas (IoT). Como prueba de concepto y validación de la propuesta, este modelo se ha utilizado para definir un componente gestor de misiones para la arquitectura del Proyecto Europeo de Investigación SWARMs. El objetivo del proyecto SWARMs ha sido expandir el uso de robots submarinos y de superficie, empleando vehículos autónomos para la realización de tareas en el entorno submarino, en el que se cumplen las condiciones de peligrosidad y distancia. El componente gestor emplea los conceptos de CoMMMA necesarios que se aplican a los requisitos específicos del proyecto, y ha sido probado con éxito enel último demostrador del proyecto obteniendo resultados prometedores. El modelo CoMMMA presentado en esta tesis se ha empleado también en el diseño del componente de gestión de misiones para la arquitectura del Proyecto Europeo de Investigación AFarCloud, enmarcado en el ámbito de la agricultura de precisión, estando pendiente de evaluar en el momento de escribir estas líneas. Los fundamentos y resultados presentados en esta tesis se contextualizan princi- palmente en los siguientes Proyectos Europeos de Investigación: WoO (ITEA2 código: 10028), DEMANES (Artemis código: 295372), ACCUS (Artemis código: 333020), SWARMs (ECSEL código: 662107), y AFarCloud (ECSEL código: 783221).

Palabras clave: robótica, planificación automática, sistemas adaptativos, control adaptativo descentralizado, sistemas multi-agente, gestión de misiones, internet de las cosas.

– xvi – Table of contents

Acknowledgements vii

Abstract xii

Resumen xiv

List of figures xxiii

List of tables xxix

List of equations xxxi

Acronyms xxxiii

I Introduction1

1 Introduction3 1.1 Motivation ...... 4 1.1.1 Robotics Uses Overview ...... 4 1.1.2 Robotics Emerging Research Overview ...... 7 1.1.3 Adaptive Mission Planning Open Issues ...... 10 1.2 Objectives and Thesis Contributions ...... 11 1.3 Research Framework ...... 13 1.4 Dissertation Outline ...... 17

II State of the Art 19

2 State of the Art on Related Topics to Adaptive Mission Planning 21 2.1 Introduction ...... 21 2.2 State of the Art in Automated Planning Paradigms ...... 23 2.2.1 Introduction ...... 23 2.2.2 Classical Planning ...... 23 Table of contents

2.2.3 Probabilistic Planning: Markov Decision Processes ...... 25 2.2.4 Partially Observable Markov Decision Process ...... 27 2.2.5 Decision Trees ...... 29 2.2.6 Hierarchical Task Networks ...... 31 2.2.7 Opportunistic Planning ...... 32 2.2.8 Conclusions ...... 34 2.3 State of the Art in Automated Planning Description Languages . . . . . 34 2.3.1 Introduction ...... 34 2.3.2 Stanford Research Institute Problem Solver ...... 34 2.3.3 ADL ...... 37 2.3.4 PDDL ...... 37 2.3.5 Conclusions ...... 41 2.4 State of the Art in Self-Adaptive Models ...... 42 2.4.1 Introduction ...... 42 2.4.2 Adaptive Systems Definition ...... 42 2.4.3 The OODA loop ...... 44 2.4.4 ECA ...... 45 2.4.5 SPA ...... 46 2.4.6 MAPE-K ...... 47 2.4.7 Decentralized Control Patterns for Self-Adaptive Systems . . . . 49 2.4.8 Conclusions ...... 53 2.5 State of the Art in Architectural Reference Models ...... 54 2.5.1 Introduction ...... 54 2.5.2 Reference Model for Service Oriented Architecture ...... 54 2.5.3 IoT Standards ...... 57 2.5.4 IoT-A ...... 61 2.5.5 Conclusions ...... 66 2.6 Chapter Conclusions ...... 66

3 Survey on Mission Management Architectures for Unmanned Vehicles 69 3.1 Introduction ...... 69 3.2 Description of the selected features for a mission manager ...... 70 3.2.1 Mission plan specification ...... 70 3.2.2 Use of Self-Adaptive Models ...... 72 3.2.3 The Use of Heterogeneous Vehicles: Virtualization ...... 72 3.2.4 Mission plan Dispatching ...... 75 3.3 Description of the Selected Proposals ...... 75 3.3.1 T-REX: A Teleo-Reactive EXecutive Architecture ...... 77 3.3.2 RAUVI and its Mission Control System (MCS) ...... 80

– xviii – Table of contents

3.3.3 Intelligent Control Architecture (ICA-Trident) ...... 83 3.3.4 Continuous Planning and Execution Framework (CPEF) . . . . . 86 3.3.5 TRAC ...... 88 3.3.6 ROSPlan: Planning for the Robot Operating System ...... 92 3.4 Summary ...... 95 3.4.1 Comparison of the Proposals ...... 95 3.4.2 Open issues ...... 96 3.5 Conclusions ...... 97

III Research Contributions: Formalization and Design 99

4 Mission Management Domain Model 101 4.1 Introduction ...... 101 4.2 The Mission Plan Concept Problem ...... 102 4.3 Strategy Formalization ...... 107 4.4 Strategy Domain Model ...... 108 4.4.1 Strategy ...... 109 4.4.2 Plan ...... 109 4.4.3 Opportunity ...... 109 4.4.4 Decision Making Function ...... 110 4.4.5 Action ...... 110 4.4.6 State ...... 110 4.4.7 Goal ...... 110 4.4.8 Decision ...... 111 4.5 Mission Management Domain Model ...... 111 4.5.1 Mission ...... 112 4.5.2 Mission Description ...... 112 4.5.3 Agent ...... 112 4.5.4 Mission Manager ...... 113 4.6 Relation of the Mission Management and the IoT Domains ...... 113 4.7 Conclusions ...... 115

5 Towards a Common Mission Management Middleware Architecture 117 5.1 Introduction ...... 117 5.2 Overview of the CoMMMA Approach ...... 118 5.2.1 Balance of Adaptation ...... 119 5.3 Requirements ...... 120 5.3.1 Functional requirements ...... 120 5.3.2 Non-Functional requirements ...... 121

– xix – Table of contents

5.4 CoMMMA Functional Model ...... 123 5.4.1 Mission Management Functional Group ...... 123 5.4.2 Strategy Management Functional Group ...... 125 5.4.3 Report Management Functional Group ...... 126 5.4.4 Agent Abstraction Functional Group ...... 127 5.4.5 Data Management Functional Group ...... 127 5.4.6 Cross-Layer Services Functional Group ...... 128 5.4.7 Communication Layer ...... 129 5.5 Computational Analysis ...... 129 5.5.1 Detailed Component Diagram ...... 129 5.5.2 Use Cases ...... 131 5.5.3 CoMMMA Workflows ...... 135 5.6 Conclusions ...... 152

IV Research Contributions: Implementation and Analysis 155

6 The SWARMs Mission Manager: The MTRR. Design and Validation 157 6.1 Introduction ...... 157 6.2 Overview of the SWARMs Architecture ...... 159 6.2.1 The SWARMs Architecture ...... 159 6.2.2 The Mission Plan ...... 163 6.3 Description of the MTRR ...... 165 6.3.1 AUV Virtualization at the MTRR ...... 169 6.3.2 Activities performed by the MTRR ...... 171 6.4 Validation Details ...... 182 6.4.1 Validation Goals ...... 182 6.4.2 Validation Scenario ...... 183 6.4.3 Technical details ...... 185 6.4.4 Details for the validation tests ...... 186 6.4.5 Details for the mission plans and execution used for the trials . . 188 6.5 Results ...... 196 6.5.1 Starting a mission ...... 196 6.5.2 Processing task status reports ...... 201 6.5.3 Processing environmental reports ...... 204 6.6 Conclusions ...... 208

– xx – Table of contents

V Conclusions and Future Works 209

7 Conclusions and Future Works 211 7.1 Conclusions ...... 211 7.2 Future Works ...... 216

VI Appendix 221

A Published Works and Dissemination Activities 223 A.1 List of Publications ...... 223 A.2 Congress and Conference Papers ...... 225 A.3 Participation in Research Projects ...... 225 A.4 Technological Transfer ...... 226 A.4.1 Self-Adaptive Transmission Power for the Internet of Things . . . 226 A.4.2 Software para la transferencia de información entre aplicaciones de control y vehículos acuáticos ...... 227 A.5 Other Merits ...... 228 A.5.1 Final Degree Projects Guidance ...... 228 A.5.2 Pre-doctoral Contract of the UPM ...... 228

Glossary 229

Index 257

– xxi –

List of figures

1.1 High level markets identified in the MAR for robotics in Europe [43]...8 1.2 Robotic system technologies identified in the MAR for robotics in Europe [43]...... 9 1.3 Project logos where contributions to this dissertation have been carried out 14 1.4 Contributions from each research project to this thesis dissertation . . . 16 1.5 Dissertation Roadmap ...... 18

2.1 Mission life cycle stages ...... 22 2.2 Graph description of a classical plan ...... 24 2.3 Classical plan domain model ...... 24 2.4 Typical classic plan execution ...... 24 2.5 Graph description of Markov Decision Process (MDP)...... 26 2.6 MDP domain model ...... 27 2.7 Graph description of Partially Observable Markov Decision Process (POMDP)...... 28 2.8 POMDP domain model ...... 28 2.9 Classical graph description of a decision tree ...... 29 2.10 Modified graph description of a decision tree ...... 30 2.11 Modified graph description of a decision tree including the states . 30 2.12 Domain model for a decision tree ...... 31 2.13 Graph description of a Hierarchical Task Network (HTN)...... 32 2.14 Domain model example of a HTN...... 32 2.15 Graph description of an opportunistic planning ...... 33 2.16 Domain model example of an opportunistic planning ...... 33 2.17 Example world model ...... 35 2.18 The Observe, Orientate, Decide, Act (OODA) Loop ...... 45 2.19 The Event-Condition-Action (ECA) rule approach for reacting to an event 46 2.20 The Sense-Plan-Act (SPA) model ...... 46 2.21 Architectural blueprint for autonomic computing from IBM ...... 47 2.22 Functional description of a Monitor, Analyze, Plan, and Execute with Knowledge (MAPE-K) loop ...... 48

– xxiii – List of figures

2.23 Example of a hierarchy of MAPE-K autonomic managers ...... 49 2.24 Legend for the decentralized control pattern diagrams ...... 49 2.25 Coordinated control ...... 50 2.26 Information sharing ...... 51 2.27 Master/Slave pattern ...... 51 2.28 Master/Slave pattern example ...... 52 2.29 Regional planning ...... 52 2.30 Hierarchical control pattern ...... 53 2.31 Hierarchical control pattern example ...... 53 2.32 Conceptual map for the Service Oriented Architecture (SOA) reference model ...... 55 2.33 IoT-A Domain Model ...... 63 2.34 Internet of Things Architecture (IoT-A) information model ...... 64 2.35 IoT-A functional model ...... 65

3.1 Mission plan specification assessment diagram (1st group) ...... 71 3.2 Diagram for the assignment of categories for the self-adaptation feature 73 3.3 Diagram for the assignment of categories for the virtualization feature. . 74 3.4 Diagram for the assignment of categories for the dispatching feature . . 76 3.5 An example Teleo-Reactive Executive (T-REX) agent composed of four reactors ...... 77 3.6 An example of the design of a T-REX integrated system ...... 78 3.7 An overview of the RAUVI software architecture ...... 81 3.8 Reconfigurable AUV for Intervention (RAUVI) architecture ...... 81 3.9 Architectural concepts of the Intelligent Control Architecture (ICA).... 84 3.10 ICA ontology ...... 84 3.11 Internal structure of an ICA agent ...... 85 3.12 Continuous Planning and Execution Framework (CPEF) overview . . . . 87 3.13 Technologies for Reliable Autonomous Control (TRAC) functional architecture ...... 90 3.14 The ACE collaborative diagram ...... 91 3.15 ACE Action Manager components ...... 91 3.16 ROSPlan framework overview ...... 93 3.17 ROSPlan Plan Dispatch node ...... 94

4.1 Scenario for the mission problem example. There is an object of interest in Room C represented as a green diamond. The robot (blue circle) and the human (yellow circle) are both in Room A ...... 102 4.2 Possible mission plan for solving the example mission problem . . . . . 103

– xxiv – List of figures

4.3 Possible decision tree for solving the example mission problem . . . . . 104 4.4 Possible decision tree with an opportunistic layer for solving the example mission problem ...... 105 4.5 A strategic solution for solving the example mission problem ...... 106 4.6 Representation of the Strategy Domain Model ...... 108 4.7 Full Mission Management Domain Model ...... 111 4.8 Relation of the Mission Management and the IoT Domains ...... 114

5.1 Functional view of the Common Mission Management Middleware Archi- tecture (CoMMMA)...... 124 5.2 Component diagram of the CoMMMA...... 130 5.3 CoMMMAUses...... 131 5.4 Sequence diagram for the initial common interactions at the beginning of a mission strategy execution ...... 136 5.5 Activity diagram used by the Mission Manager component for the execution of a strategy depending on the execution capabilities of the agents, and without considering opportunities ...... 137 5.6 Sequence diagram for sending a strategy to an agent capable of understanding and executing it ...... 137 5.7 Sequence diagram for sending a deterministic mission plan generated to an agent not capable of understanding strategies, and expecting a full plan138 5.8 Sequence diagram for executing a mission strategy with an agent not capable of understanding strategies and expecting just one action . . . 139 5.9 Sequence diagram for executing opportunities when an agent is not capable of understanding them directly, and the opportunity is identified byanevent ...... 141 5.10 Sequence diagram for executing opportunities when an agent is not capable of understanding them directly, and the opportunity is identified by an environment condition ...... 142 5.11 Sequence diagram for mission adaptation. The specifics interactions for the generation of a plan from the strategy or the selection of the next action from a strategy must follow their corresponding sequence diagrams for mission strategy execution ...... 144 5.12 Activity diagram used by the Mission Manager component for the adaptation of the mission execution based on the strategy execution capabilities of the affected agent ...... 145 5.13 Sequence diagram with the interactions for retrieving a plan by its id . . 146 5.14 Sequence diagram with the interactions for retrieving a list of plans by matching given goals ...... 146

– xxv – List of figures

5.15 Sequence diagram with the interactions for retrieving a list of plans by similarity to a given plan ...... 146 5.16 Sequence diagram with the interactions for retrieving a strategy by its id 147 5.17 Sequence diagram with the interactions for retrieving a list of strategies by matching given goals ...... 147 5.18 Sequence diagram with the interactions for retrieving a list of strategies by similarity to a given strategy ...... 147 5.19 Sequence diagram with the interactions for retrieving a list of strategies by containing to a given plan ...... 148 5.20 Sequence diagram with the interactions for the generation of a strategy using aggregation with plans and/or strategies retrieved from the historical repositories ...... 149 5.21 Sequence diagram with the interactions for the update of a strategy . . . 150 5.22 Sequence diagram with the interactions for the notification of a mission status ...... 150 5.23 Sequence diagram with the interactions for the notification of an event . 151 5.24 Sequence diagram with the interactions for the notification of environmental information ...... 152

6.1 Process of the generation of the MTRR ...... 159 6.2 SWARMs architecture overview ...... 160 6.3 SWARMs mission and planning ontology ...... 164 6.4 Structure of a SWARMs mission plan ...... 164 6.5 Example of a mission for three Autonomous Underwater Vehicles (AUVs) named NTNU, PORTO 1 and PORTO 2. In this figure GW refers to the low-level tasks GOTO_WAYPOINT, and FR refers to the low-level task FOLLOW_ROW ...... 165 6.6 MTRR-centered SWARMs components diagram ...... 168 6.7 MTRR component class diagram ...... 169 6.8 IoT-A domain model related to the virtual representation of a physical entity [200]...... 170 6.9 Overview of the vehicle virtualization used at the MTRR ...... 170 6.10 Functional overview of the start of a mission ...... 172 6.11 Flow diagram for the plan validation undertaken at the Mission and Tasks Register and Reporter (MTRR)...... 173 6.12 Flow diagram for mission data extraction and storage done at the MTRR 174 6.13 Flow diagram for the task assignment when done by full plan ...... 175 6.14 Flow diagram for the task assignment when done by wait for completion 176 6.15 Sequence diagram for the startMission procedure performed by the MTRR177

– xxvi – List of figures

6.16 Functional overview of the processing of a mission status report from a vehicle ...... 178 6.17 Flow diagram for assigning the next task after receiving a COMPLETED task status ...... 179 6.18 Sequence diagram for the task report method in the MTRR ...... 180 6.19 Functional overview of the processing of an environmental report . . . . 180 6.20 Sequence diagram for the management of an environmental report . . . 181 6.21 Sequence diagram for the management of an event report ...... 181 6.22 Sequence diagram for aborting a vehicle plan ...... 182 6.23 Sequence diagram for aborting the full mission ...... 182 6.24 Final SWARMs demo test area used for validation ...... 184 6.25 Control Station distribution ...... 184 6.26 Vehicles used during for the final audit of SWARMs ...... 186 6.27 Hierarchical representation of mission 35 plan ...... 188 6.28 Estimated timeline for mission 35 plan ...... 189 6.29 Mission 35 progress as shown at the Mission Management Tool (MMT) developed by MDH...... 189 6.30 Hierarchical representation of mission 36 plan ...... 190 6.31 Estimated timeline for mission 36 plan ...... 191 6.32 Mission 36 progress as shown at the MMT developed by MDH..... 191 6.33 Mission 36 progress as shown at the Context Awareness Framework (CAF) Graphical User Interface (GUI) used in Smart and Networking Underwater Robots in Cooperation Meshes (SWARMs)...... 192 6.34 Hierarchical representation of mission 44 plan ...... 193 6.35 Estimated timeline for mission 44 plan ...... 193 6.36 Mission 44 progress as shown at the MMT developed by MDH..... 193 6.37 Hierarchical representation of mission 45 plan ...... 194 6.38 Estimated timeline for mission 45 plan ...... 195 6.39 Mission 45 progress as shown at the MMT developed by MDH..... 195 6.40 Distribution of times for starting a mission in the MTRR ...... 198 6.41 Relation between the size of the mission plan and the time required to start it ...... 198

6.42 (a) Boxplot for the mission plan length (lplan). The median is completely displaced to the lower quartile, and there are also a couple of outliers; (b)

boxplot for the time required to start a mission (tstartMission)...... 199 6.43 Pareto chart for the average distribution of time for each activity performed by the MTRR when starting a mission ...... 200

– xxvii – List of figures

6.44 Pareto chart for the average distribution of times for each activity performed by the MTRR when starting a mission without consideration for the logging of the actions in the mission plan ...... 200 6.45 Pareto chart for the average distribution of time for each activity performed by the MTRR when starting a mission, averaged to the number of actions in the mission plan and the number of vehicles as appropriate ...... 201 6.46 Average time distribution for the processing of the task status reports . . 203 6.47 Pareto chart for the processing times of the task status reports . . . . . 203 6.48 Distribution of time required for processing the state vectors ...... 206 6.49 Pareto chart for the average total time required for each part in processing the state vector ...... 206 6.50 (a) Boxplot for times required to store the state vector in the SWARMs ontology; (b) boxplot for the times required store the state vector in the historical database ...... 207

A.1 Register for the intellectual property of the “Self-Adaptive Transmission Power for the Internet of Things.” ...... 227 A.2 Register for the intellectual property of “Software para la transferencia de información entre aplicaciones de control y vehículos acuáticos.” . . . . 227 A.3 UPM’s own program predoctoral contract credential...... 228

– xxviii – List of tables

1.1 Robotic system abilities identified in the Multi-Annual Roadmap (MAR) for robotics in Europe [43]...... 8

2.1 Fields of a domain in Planning Domain Definition Language (PDDL).. 38 2.2 Fields of a problem description in PDDL...... 39 2.3 Versions of PDDL ...... 40 2.4 Other variants and extensions of PDDL ...... 41 2.5 ITU-T Recommendations on IoT...... 58 2.6 ISO/IEC Standards ...... 60 2.7 IEEE Standards ...... 61

3.1 Mission plan specification assessment categoriesst (1 group) ...... 71 3.2 Mission plan specification assessment categoriesnd (2 group) ...... 72 3.3 Self-adaptation assessment categories ...... 73 3.4 Virtualization assessment categories ...... 75 3.5 Mission plan dispatching assessment categories ...... 76 3.6 T-REX architecture assessment ...... 80 3.7 RAUVI architecture assessment ...... 83 3.8 ICA assessment ...... 87 3.9 CPEF assessment ...... 89 3.10 TRAC assessment ...... 92 3.11 ROSPlan assessment ...... 94 3.12 Comparison of the categories assigned to the reviewed systems . . . . 95

5.1 Functional Requirements for CoMMMA...... 122 5.2 Non-Functional Requirements for CoMMMA...... 123 5.3 Description of the Mission Execution related use cases ...... 132 5.4 Description of the Historical Plan Retrieval use case ...... 132 5.5 Description of the Historical Strategy Retrieval use case ...... 132 5.6 Description of the Strategy Generation use case ...... 133 5.7 Description of the Strategy Update use case ...... 133

– xxix – List of tables

5.8 Description of the Mission Adaptation use case ...... 133 5.9 Description of the Opportunity Execution use case ...... 134 5.10 Description of the Mission Status Notification use case ...... 134 5.11 Description of the Event Notification use cases ...... 134 5.12 Description of the Environmental Information use cases ...... 135

6.1 Relations between the MTRR and other middleware components . . . . 167 6.2 Details of the vehicles used during the tests ...... 185 6.3 Details for each registered mission ...... 186 6.4 Times for Starting a Mission in the MTRR after receiving it from the MMT (in ms) ...... 196 6.5 Values for processing the task status reports and, if required, send the next action to the corresponding vehicle. Times are given in milliseconds 201 6.6 Average times for processing the task status reports ...... 204 6.7 Values for processing the state vector report. Times are given in milliseconds204 6.8 Average times per activity for processing the state vector reports . . . . 207

– xxx – List of equations

2.1 Formal definition of a plan ...... 23 2.2 Sum of probabilities for all the actions and for all the states of a MDP.... 25 2.3 Reward definition of a MDP...... 25 2.4 Stanford Research Institute Problem Solver (STRIPS) solution for the example exercise...... 37 2.5 Principle of negative feedback ...... 43 2.6 Definition of the event E as a stimulus for the system S . . . . . 43 2.7 Formal definition of the Law of Adaptation ...... 44 4.1 Formal definition of a strategy ...... 108 5.1 Time balance for adaptation in a planning problem where the deliberation is made apart from the robot...... 119 5.2 Total time for the communication of an event and the corresponding adapted plan...... 119 5.3 Adaptation time improvement by using decision on a predefined strategy instead of a complete new deliberation ...... 120

– xxxi –

Acronyms

AADL Architecture Analysis and Design Language. ACCUS Adaptive Cooperative Control in Urban (sub) Systems. ADL Action Description Language. AED Automated External Defibrillator. AFarCloud Aggregate Farming in the Cloud. AI Artificial Intelligence. ARM Architectural Reference Model. AUV Autonomous Underwater Vehicle. CAF Context Awareness Framework. CANON Controlled, Agile, and Novel Ocean Network. CBR Case-Based Reasoning. CCS Central Control Station. CDT Control Data Terminal. CoMMMA Common Mission Management Middleware Architecture. CORA Core Ontology for Robotics and Automation. COVID-19 Coronavirus disease 2019. CPEF Continuous Planning and Execution Framework. CPR Cardiopulmonary Resuscitation. CPS Cyber-Physical System. DDS Data Distribution Service. DEMANES Design, Monitoring and Operation of Adaptive Networked Embedded Systems. DOI Digital Object Identifier. DSS Decision Support System. EBNF extended Backus-Naur form. ECA Event-Condition-Action. ECS Electronic Components and Systems. GUI Graphical User Interface. HDDL Hierarchical Domain Definition Language.

– xxxiii – Acronyms

HTN Hierarchical Task Network. ICA Intelligent Control Architecture. IEC International Electrotechnical Commission. IEEE Institute of Electrical and Electronics Engineers. IoAT Internet of Autonomous Things. IoRT Internet of Robotic Things. IoT Internet of Things. IoT-A Internet of Things Architecture. IPC International Planning Competition. ISO International Organization for Standardization. ITU International Telecommunications Union. ITU-T ITU’s Telecommunication Standardization Sector. JCR Journal Citation Reports. JTC Joint Technical Committee. MA-PDDL Multi-Agent PDDL. MAPE-K Monitor, Analyze, Plan, and Execute with Knowledge. MAPL Multi-Agent Planning Language. MAR Multi-Annual Roadmap. MBARI Monterey Bay Aquarium Research Institute. MCL Mission Control Language. MDH Mälardalen University. MDP Markov Decision Process. MMT Mission Management Tool. MTRR Mission and Tasks Register and Reporter. NDDL New Domain Description Language. OASIS Organization for the Advancement of Structured Information Standards. OODA Observe, Orientate, Decide, Act. OWL Web Ontology Language. PDDL Planning Domain Definition Language. POMDP Partially Observable Markov Decision Process. PPDDL Probabilistic PDDL. PPP Public-Private Partnership. RAUVI Reconfigurable AUV for Intervention. RDDL Relational Dynamic influence Diagram Language. ROS Robot Operating System. ROV Remote Operated Vehicle. RSOA Robotic System On-board Architecture. SMPA Sense-Model-Plan-Act.

– xxxiv – Acronyms

SOA Service Oriented Architecture. SPA Sense-Plan-Act. SPARQL SPARQL Protocol and RDF Query Language. SRA Strategic Research Agenda. SRI Stanford Research Institute. SRIDA Strategic Research Innovation and Deployment Agenda. SRP Single-Responsibility Principle. STO Scientific and Technical Objectives. STRIPS Stanford Research Institute Problem Solver. SUMO Suggested Upper Merged Ontology. SWARMs Smart and Networking Underwater Robots in Cooperation Meshes. T-REX Teleo-Reactive Executive. TBS Trondheim Biological Station. TRAC Technologies for Reliable Autonomous Control. UAV Unmanned Aerial Vehicle. UGV Unmanned Ground Vehicle. UML Unified Modeling Language. USAF United States Air Force. USV Unmanned Surface Vehicle. VDT Vehicle Data Terminal. wff well-formed formula. WoO Web of Objects. XML Extensible Markup Language.

– xxxv –

Part I

Introduction

Chapter 1

Introduction

“‘Which methods might work for the problem I’m facing—and which representations are likely to work well with those methods?’

Most computer programs still, today, can do only one particular kind of task, using only a single representation—whereas our human brains accumulate multiple ways to describe each of the Types of Problem we face. ...we need to learn how to switch to another alternative whenever the method we’re using fails”

— Marvin Minsky, The Emotion Machine

Robotics is one of the fields that has grown the most in recent years thanks tothe evolution of multiple associated technologies. In particular, the improvement in the autonomous capabilities of robots is opening a new set of opportunities that present interesting technological challenges. This is the case of autonomous robots, which, considered as robotic agents, are used to achieve goals by performing tasks following a plan. The goals defined for the robot, and the tasks assigned to achieve them are known respectively as mission and mission plan.

The use of autonomous robots in open environments implies that new situations may arise not contemplated in the original plan, and that prevent the achievement of the mission goals. Hence the need to develop adaptive solutions that allow adaptation to such situations. The adaptation can be carried out both in the robot itself and in the entire system related to the mission, its planning and its execution. In particular, adapting the mission requires the involvement of all parties in the case where multiple robots cooperatively participate in it.

– 3 – Introduction

One of the technologies that is of interest to robotics is the Internet of Things (IoT). The main concept of the IoT is to interconnect “things”, such as robots, and in particular autonomous robots that participate in a mission. Therefore, using IoT in conjunction with robotics seems promising to contribute to the improvement in the adaptation of missions.

In this dissertation we study this problem, and propose a novel structure for the mission plan and a reference architecture for mission management. This reference architecture is used later as a guide for the design, implementation and validation of a mission manager component for the SWARMs European project, in which multiple autonomous robots are used for missions in the underwater environment.

The rest of this chapter is organized as follows. First, Section 1.1 explains the main motivations for this thesis, including the general robotics scope in which it is framed, the general robotics emerging research interests identified in several research agendas, and a more detailed view of the specific challenges in relation to the adaptive mission planning for cooperative robotics problem. Second, Section 1.2 describes the objectives and contributions of this dissertation. Third, the research framework where this work has been carried out is presented in Section 1.3. Finally, the outline of this thesis is presented in Section 1.4.

1.1 Motivation

1.1.1 Robotics Uses Overview

Technological advances in recent years have given new impetus to robotics. In particular, the improvement in the autonomous capabilities of robots has made it possible to explore new possibilities for their use, whether it is for exploring new solutions to societal challenges from ageing to health, smart transport, security, energy and environment, or for being used in harsh environments, surveying dangerous, remote and inaccessible environments.

An example scenario in which the conditions of danger, inaccessibility and distance occur is the underwater environment [1]. Robotics used in underwater environments are mainly based on AUVs, maybe with the support of Unmanned Surface Vehicles (USVs). For instance, AUVs have been used in military interventions [2,3] like the design of large diameter mine countermeasures [4]. They have also been used for scientific interventions, like the survey and exploration of the Arctic Ocean[5,6], and also for underwater infrastructure maintenance [7–9], and oil spill response [10–12]. The improvement of the autonomous capabilities of this type of robots in underwater environments has been explored in the SWARMs project, in which part of the work of this dissertation has been carried out.

– 4 – 1.1 Motivation

Another scenario that has received increasing interest in recent years is that of precision farming. Precision farming is a management approach that focuses on (near real-time) observation, measurement, and responses to variability in crops, fields and animals [13]. Such an approach to management can benefit from the use of robotics. For instance, UAVs can be used for rice pest detection as suggested in [14], or in general to monitor the health of crops using, for example, hyper-spectral cameras [15, 16]. Unmanned Ground Vehicles (UGVs) can be used for any of the tasks in the fertilizing, planting, spraying and harvesting cycle [17]. And the combined use of different technologies, including robotics, in support of precision farming is one of the objectives of the Aggregate Farming in the Cloud (AFarCloud) project [18, 19] in which part of the work of this thesis has also been carried out.

A striking use case is the use of an ambulance drone to transport an Automated External Defibrillator (AED) with the aim of providing advanced Cardiopulmonary Resuscitation (CPR) support until the arrival of specialized personnel [20, 21].

Going a bit further, the use of autonomous robots for disaster mitigation has also been explored [22]. A proposal for using UAVs for collecting information during catastrophes is presented in [23], taking advantage of their ability to fly over land that may be inaccessible. In the same line, [24] explores the use of UAVs during disasters in Japan, for both surveying areas and transporting goods that may be needed and cannot be delivered by other means.

Robotics has also been especially relevant in various activities associated with the management of the global Coronavirus disease 2019 (COVID-19) pandemic. Indeed, during the European Robotics Week in late November 2020, Marina Bill, ABB’s Global Head of Sales for Robotics and Discrete Automation talked about “the crucial role robotics can play during the pandemic” [25]. Some of the identified uses of robotics for activities related to the COVID-19 are mentioned below.

Draganfly Inc., Vital Intelligence Inc. and the University of South Australia announced in March, 2020 that they were working in the integration of cameras powered with the Virtual Intelligence solutions and Draganfly’s Commander AUV for two possible interventions related to the COVID-19 pandemic: a thermal commander able to transmit temperature, cough, and heart and respiratory rate, and a social distancing commander able to detect breaches in social distancing [26].

Khan and Javaid discuss the use of autonomous robots for tasks like the delivery of medical supplies and the remote monitoring of patient symptoms in [27]. Indeed, in August 2020 the MIT announced a partnership with Boston Dynamics and Brigham and Women’s Hospital to reduce the risk to healthcare workers treating COVID-19 patients by using robots to remotely measure their vital signs [28].

– 5 – Introduction

Kumar et al. propose the use of UAVs for aiding in the COVID-19 monitoring, sanitization, social distancing, data analysis, and statistics generation for the control room in [29].

In [30] a novel autonomous robot, called i-Robot UVC, is presented. This robot is designed to perform ultraviolet germicidal irradiation [31], and can do it either autonomously or remotely operated. It is an autonomous device and can be controlled manually, or work in a fully automated mode.

Yang et al. discuss the role of robotics in managing public health and infectious diseases in [32]. In their own words “the impact of COVID-19 may drive further research in robotics to address risks of infectious diseases”. Their work identifies three main opportunities for the use of autonomous robots in this scenario: (1) disinfection, (2) screening, (3) goods delivery (samples and medicines).

Finally, Murphy et al. have reviewed 262 reports between March 27 and July 4, 2020 reporting actual use of different types of robots for the COVID-19 response [33]. In their survey they have identified up to 74 robot uses in public safety, 46 in clinical care,27 in continuity of work and education, 22 in quality of life and 21 in laboratory and supply chain automation.

1.1.1.1 Relation with IoT

IoT has been formally defined as “a global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies”[34]. A similar definition is given in[35], with a specific mention to the capability of this global infrastructure to process and react to the information from the physical and virtual world.

In plain words, IoT provides the means for things to communicate. And it is not surprising that as robotics and IoT have progressed, proposals have emerged that interrelate both technologies. That is, for instance, the case of the Internet of Autonomous Things (IoAT) for including autonomous devices as part of an IoT ecosystem as mentioned in [36–38]. In a different proposal, the concept of Internet of Robotic Things (IoRT) was defined as “a global infrastructure for the information society enabling advanced robotic services by interconnecting robotic things”, even suggesting a possible IoRT architecture [39], in a clear reference to the formal definition from ITU’s Telecommunication Standardization Sector (ITU-T) given in [34].

In short, IoT can be considered as one of the enabling technologies for Robotics, and is especially useful in the case of multiple robots working cooperatively to carry out a mission.

– 6 – 1.1 Motivation

1.1.2 Robotics Emerging Research Overview

The growing interest in Robotics has attracted the attention of multiple research, development and innovation collectives. That is the case of the SPARC Public-Private Partnership (PPP) between the European Commission and the robotics community represented by euRobotics, in whose Strategic Research Agenda (SRA) for robotics in Europe expressly mentions that “Robotics Technology will become dominant in the coming decade. It will influence every aspect of work and home. Robotics has the potential to transform lives and work practices, raise efficiency and safety levels, provide enhanced levels of service and create jobs. Its impact will grow over time as will the interaction between robots and people”[40]

This is also the case of three relevant industry associations, AENEA, Artemis-IA and EPoSS, who have also published their own SRA for Electronic Components and Systems (ECS)-based applications, in which they affirm thatthe “ advent of new technologies like IoT, Artificial Intelligence (AI), Robotics and the like will shape new ways of how people interact with the world and with each other”[41].

The AI, Data and Robotics partnership, formed by BDVA, Claire, Ellis, EurAI and eu- Robotics, has also published its own Strategic Research Innovation and Deployment Agenda (SRIDA), paying special attention to the integration of AI, Big Data and robotics technologies, and mentioning, in regards of robotics, that “robots are unique because they create value by performing physical tasks that people cannot, should not, or will not do”[42].

The Robotics 2020 Multi-Annual Roadmap [43] provides a detailed guide for the aspects included in the SPARC SRA for robotics in Europe. The approach used in this document identifies some market domains, which set the requirements for the system abilities, and maps such domains with the technologies that provide the capabilities to fulfill those requirements. The roadmap identifies seven end user market domains: Commercial, Logistics and Transport, Civil, Agriculture, Consumer, Manufacturing, and Healthcare. These seven domain markets are illustrated in Figure 1.1.

Continuing with the Robotics 2020 Multi-Annual Roadmap, it also identifies nine system abilities that a robotic system can possess, and that are briefly described in Table 1.1.

The last area covered in the SPARC roadmap is about technologies identified as relevant to robotics. In total, the roadmap identifies 26 technologies, grouped into six clusters characterized by four purposes:

• Systems Development: Better systems and tools

• Human Robot Interaction: Better interaction

– 7 – Introduction

• Mechatronics: Making better machines

• Perception, Navigation and Cognition: Better action and awareness

Figure 1.1. High level markets identified in the MAR for robotics in Europe [43]

Table 1.1. Robotic system abilities identified in the MAR for robotics in Europe [43]

Ability name Description Configurability The ability of the robot to be configured to perform a task, or reconfigured to perform different tasks. Adaptability The ability of the system to adapt itself to different work scenarios, different environments and conditions. Interaction The ability of a system to interact physically, cognitively and socially either with users, operators or other systems around it, including other robots. Dependability The ability of the system to perform its given tasks without systematic errors. Motion The ability of the system to move. Manipulation The ability of the system to handle objects. Perception The ability of the robot to perceive its environment. Decisional Autonomy The ability of the robot to act autonomously. Cognitive The ability to interpret the task and environment such that tasks can be effectively and efficiently executed even where there exists environmental and/or task uncertainty.

Figure 1.2 illustrates the grouping of the technologies identified into their corresponding clusters.

– 8 – 1.1 Motivation

Figure 1.2. Robotic system technologies identified in the MAR for robotics in Europe [43]

Using the market domains, the systems abilities and the technologies outlined, the SPARC roadmap provides a comprehensive catalog of opportunities organized by area.

In the framework of this thesis, the opportunities identified for planning and system architectures are of particular interest. In the case of planning, the SPARC roadmap identifies the following technological barriers:

• Computational complexity of the combined hybrid planning problems.

• Computational complexity of planning when uncertainty is taken into account, e.g., in decision-theoretic approaches.

In the case of the system architectures, among all identified opportunities, the following are of interest for the framework of this thesis:

• Development of design patterns and the creation of reference architectures.

• Reference architectures for several domains that work across multiple operating environments and robot configurations.

• Resource level autonomy – Selecting and implementing among pre-defined reaction scenarios.

– 9 – Introduction

• Resource level autonomy – Generating and implementing new reaction scenarios based on previous knowledge.

• Models and domain specific languages for tasks, resources, platforms, etc.

• Robots integrated into IoT and making use of big data methods and semantic web technology.

1.1.3 Adaptive Mission Planning Open Issues

This dissertation is focused on cooperative autonomous robots working together to achieve some goals. The list of these goals constitutes what is defined as a mission [44], with the mission plan being the sequence of actions created to achieve such goals [45]. Mission management can then be defined as the set of activities related to the creation, validation, transfer and execution of a mission, and hence its relevance in the adaptive mission planning.

While working on the design of the brokering component for mission management in the SWARMs European project, and later for a similar component in the AFarCloud European project, the following problems related to adaptive mission planning were identified.

Problem 1: A mission plan is defined as the sequence of actions needed to achieve the mission goals [45]. A mission plan can be done manually, or using anAI planning algorithm adapted to the specific mission scenario. In any case, this definition ofa mission plan is deterministic in the sense that after executing an action in the plan, the conditions of the environment are expected to meet the requirements to execute the next action, and so on until the goal is reached. Its common in robotics to use a subsumption approach [46–48], delegating to the robotic agent the adaptation to the situations that it may face when carrying out a mission plan. Even so, it is not always possible for the robotic agent to adapt to the situations it may encounter. Thus, the use of deterministic plans is a problem for the adaptation of missions to new situations.

Problem 2: The execution of a mission plan in an open real world scenario is subject to uncertain and even unknown events [42]. Indeed, as David Poole states in [49]: “An agent cannot just plan a fixed sequence of steps; the result of planning needs to bemore sophisticated. Planning must take into account the fact that an agent in the real world does not know what will actually happen when it acts. An agent should plan to react to its environment”. One of the possibilities is using a non-deterministic approach by posing the mission planning problem as a MDP or POMDP problem. In this case, instead of a plan, the solution is a policy, defined as a function that calculates the best action to reach a goal given the current state observation, or partial observation. The definition of

– 10 – 1.2 Objectives and Thesis Contributions this function is usually complex, and still it may be required to redefine it for adapting to new situations.

Problem 3: Adaptive planning has been commonly defined as the change of the current plan to meet the demands of some new planning situation [50]. This process is typically done by repairing or modifying the current plan [51, 52], or by making a new plan (re- planning) [53]. Several issues arise whenever it is required to repair a plan or to re-plan. First, this is part of the deliberative process, and it is time consuming. Second, it may also require updating the model of the world used to create the original plan, as it may no longer apply to the new situation.

Problem 4: Since the deliberative process is time consuming and requires some computing power, it is common for mission plan repair or re-planning to be carried out independently of the robotic agents who were executing the mission plan, in a dedicated machine at a mission control or similar. This is affected by the communications as well, as in certain scenarios its constrained nature introduces several concerns. That is the case, for example, of the underwater environment [1] in which the activities of the SWARMs European project took place. In General, opportunities for communication with remote autonomous agents are often scarce, whether in space, underwater, or disaster-recovery environments [54]. This characteristic creates a significant problem for mission adaptation in the form of considerable delays that can affect mission success.

Problem 5: As the use of autonomous robots grows, their heterogeneity also grows, especially in relation to the management of missions carried out in the robotic agent itself. This is because usually each mission management solution that is developed is made specifically for each case, and for the robot agents that will be used in it. This contributes to a fragmented scenario of mission management solutions, in the same way as other fragmentation issues mentioned in [42].

Problem 6: During the execution of a mission, situations may occur that do not imply a need for adaptation, but do offer an interesting opportunity not originally contemplated in the mission [54]. Identifying and being able to take advantage of these opportunities remains an open issue in mission management.

1.2 Objectives and Thesis Contributions

The main motivation of this thesis is to contribute to the research of adaptive mission planning management for the cooperation between multiple robotic agents in different scenarios for the Internet of Things. In order to fulfill this motivation, the work presented in this manuscript is centered on the mission plan description and execution in the domain of multi-agent robots, provided by means of convergence of several paradigms in the field ofAI planning paradigms, adaptive systems and the Internet of Things. The

– 11 – Introduction combination of such approaches provides the basis for a promising line of research in the field of next-generation mission management for autonomous systems.

The core objectives of this dissertation are summarized as follows. Around these starting point requirements, main contributions of this thesis are provided.

• To formalize a novel structure and domain model for a mission plan, called strategy, with the aim of improving adaptive capabilities of the agents taking part in a mission. This new domain model uses the concepts from four main planning paradigms. First, it uses a classical planning approach for those actions without a known alternative. Second, it uses decision nodes at selected parts of the classical plan when there is a known alternative for different environmental conditions at the time of making the decision. Third, it uses a hierarchical decomposition of some actions taking into consideration the heterogeneous capabilities of the different agents that can participate in a plan. And finally, it uses a secondary opportunistic layer for leading new branches of execution in case a chance is given for it.

• To formalize a mission management domain model that integrates the proposed strategy domain model, and that serves as a conceptual basis for an Architectural Reference Model (ARM).

• To formalize and design an ARM for mission management for cooperative robotics. This architectural reference model, called CoMMMA, guides the definition of specific architectures capable of using different mission planning description formats, enabling adaptive mission execution both from the agents capable of doing it, and from the mission manager itself whenever the agents are not directly capable, and for that situations, providing a virtualization of the agents’ capabilities regarding mission parsing and execution.

• To formalize, design, implement and validate a specific mission manager designed using CoMMMA as a guideline.

In order to achieve the previous objectives, the following analyses will be conducted as the main state-of-the-art evaluation contributions of this dissertation:

1. Analysis of the state-of-the-art of automated planning paradigms and description languages. This study will enable the understanding of the background on these concepts, as well as how they can be used and integrated in mission planning and execution for cooperative agents.

2. Analysis of the state-of-the-art of adaptation and adaptive models. This includes the exploration of what is formally defined as adaptation, and the most prominent

– 12 – 1.3 Research Framework

adaptive models used for autonomous agents, and a selection of patterns for decentralized self-adaptive systems.

3. Analysis of the state-of-the-art of architecture reference models. This study focuses on the reference models for service oriented applications, the IoT, and some other relevant and recent proposals aimed at autonomous and robotic things.

4. Analysis of the state-of-the-art of mission management for cooperative robotics. This study is centered in surveying the main research trends on mission management architectures, frameworks and models, paying special attention to how they use planning and adaptation, and how they relate to the architecture reference models explored previously.

From the conclusions obtained in these analyses, main formalization and practical application contributions provided by this dissertation can be classified as follows:

1. Formal description of a new proposal for a mission structure called strategy, which combines the characteristics of classic mission plans, with the use of policies defined as decision-making functions, the possibility of hierarchical decomposition of actions and the consideration towards possible opportunities during the mission execution.

2. Abstract formalization of both a Strategy and a Mission Management Domain Models for adaptive mission planning.

3. Abstract formalization of a common mission management middleware architecture (CoMMMA) as a reference model for the creation of specific mission managers for specific domains of application.

4. Practical approach to adaptive mission management. It represents the practical implementation of a specific mission manager for the SWARMs European project specific needs using the CoMMMA proposal as guidance for its design.

1.3 Research Framework

The work presented in this dissertation has been developed within the framework of several projects funded by the European Commission in the fields of pervasive and adaptive computing, the Internet of Things and cooperative robotics. In order to illustrate the research context of this thesis, next paragraphs summarize the main objectives of these European Projects.

– 13 – Introduction

Figure 1.3. Project logos where contributions to this dissertation have been carried out

• AFarCloud (ECSEL code: 783221, start date: 09/2018, end date: 08/2021) [55, 56]: AFarCloud addresses the urgent need for a holistic and systematic approach. It will provide a distributed platform for autonomous farming, which will allow the integration and cooperation of Cyber Physical Systems in real-time for increased agriculture efficiency, productivity, animal health, food quality and reduced farm labor costs. This platform will be integrated with farm management software and will support monitoring and decision-making, based on big data and real time data mining techniques.

AFarCloud also aims to make farming robots accessible to more users by enabling farming vehicles to work in a cooperative mesh, opening up new applications and ensuring re-usability, as various standard vehicles can combine their capabilities in order to boost farming efficiency. The achievements from AFarCloud will be showcased in early laboratory trials and holistic demonstrators, including cropping and livestock management scenarios. Local demonstrators will test specific functionalities and validate project results in relevant environments located in different European regions.

• SWARMs (ECSEL code: 662107, start date: 07/2015, end date: 06/2018) [57, 58]: SWARMs will expand the use of AUVs/ROVs (Autonomous Unmanned Ve- hicles and Remotely Operated Vehicles) and facilitate the creation, planning and execution of maritime and offshore operations. This will make autonomous operations a viable option for new and existing industries, reducing the operational cost, increasing the safety of tasks and helping to expand the offshore sector.

– 14 – 1.3 Research Framework

The project will improve the autonomy and cooperation of underwater Cyber- Physical Systems (CPS), introducing the CPS concept for subsea operations to boost their cost-effectiveness. This will improve the industrialization and automation of underwater inspection, maintenance and construction operations in the harsh conditions inherent to the oceanic/maritime environment. By contributing to guaranteeing interoperability, cooperation, reuse and technological improvements, SWARMs will reduce both development and operating costs, yet contribute to increasing safety in underwater operations by reducing the risks associated with human operators.

• Adaptive Cooperative Control in Urban (sub) Systems (ACCUS) (Artemis code: 333020, start date: 06/2013, end date: 01/2016) [59, 60]: Adaptive Coopera- tive Control in Urban (sub) Systems like traffic, energy, and outdoor lighting are managed by self-contained embedded systems. ACCUS aims to establish long- term sustainable urban areas and develop the necessary infrastructure to enable the sharing of urban subsystem services, demonstrating this through selected use-cases in a real-life urban setting.

• Design, Monitoring and Operation of Adaptive Networked Embedded Systems (DEMANES) (Artemis code: 295372, start date: 05/2012, end date: 04/2015) [61, 62]: DEMANES aims to provide component-based methods, framework and tools for the development of runtime adaptive systems, enabling them to react to changes in themselves, in their environment and in user needs. Major societal challenges require large-scale monitoring and control solutions. Technological developments will make it possible to design and build these large systems. However, a versatile methodology is lacking to design and implement these systems so DEMANES aims to model the architecture and the operation of adaptive systems and support the design process. DEMANES will incorporate recent advances in systems and control engineering. The concept, methodology and tools developed in DEMANES will be validated and demonstrated in three use cases: smart urban transport, smart airport and smart home.

• Web of Objects (WoO) (ITEA2 code: 10028, start date: 01/2012, end date: 12/2014) [63, 64]: The ITEA 2 Web of Objects project is developing a web-of- objects (WoO) based approach to building automation in which smart distributed applications combine information from different building-control domains which are currently isolated from each other. Web of Objects aims to establish a unified service concept based on a service-oriented architecture, one that also includes the network infrastructure and its capabilities as services.

– 15 – Introduction

Figure 1.4 illustrates the map of contributions from each research project mentioned to this dissertation, and that can be summarized as follows:

• From the WoO project:

– The concept of service composition has served as reference for the integration of the concepts of compound and primitive actions as part of the strategy proposal. – The concept of device virtualization has served as reference for the definition of the robotic agent virtualization as part of the CoMMMA proposal.

• From the DEMANES and the ACCUS projects:

– The self-adaptive control models and the mediation patterns used in both projects have served as reference for the formalization of the CoMMMA proposal.

• From the SWARMs and the AFarCloud projects:

– The experience with mission management and automated planning have served as reference for the formalization of the CoMMMA proposal. – The experience with automated planning has also served as reference for the strategy proposal.

Figure 1.4. Contributions from each research project to this thesis dissertation

– 16 – 1.4 Dissertation Outline

1.4 Dissertation Outline

The work presented in this PhD thesis is organized in five differentiated parts, consisting in a total of seven chapters and one appendix. The main contents of the aforementioned parts are described in the following paragraphs, and illustrated in the roadmap shown in Figure 1.5.

Part I corresponds to the introduction to the PhD thesis.

• Chapter1 , which introduces the main concepts and objectives for this PhD Thesis. It introduces the use of autonomous agents, like robots, in open and unrestricted real world scenarios, and how the use of automated planning, together with some basic adaptive approaches is the current option of choice. This chapter also describes the usual problems that the stat-of-the-art solutions face, and gives some insights on how they may be solved with the proposals included in this dissertation.

Part II covers the state-of-the-art for the concepts introduced in the Chapter1.

• Chapter2 , analyzes the topics of interest for adaptive mission planning for cooperative robotics in the IoT. • Chapter3 , surveys a comprehensive selection of mission management architectures in relation to the topics of interest identified in Chapter2.

Part III describes the proposals made in this dissertation in order to contribute to adaptive mission planning for cooperative robotics.

• Chapter4 delves into the need for a new structure beyond plans and policies, describing a novel strategic domain model for planning that could be integrated in a more generic formal mission specification. • Chapter5 introduces a reference architecture for mission management that could be used to guide the design of specific architectures for adaptive missions in cooperative robotics, named CoMMMA. CoMMMA follows the SOA and IoT-A reference models, and provides the required components to deal with both classical and non-classical planning paradigms and description languages, as well as the novel strategy formulation from Chapter4.

Part IV explores the experimental design and discusses the obtained results.

• Chapter6 describes the whole process of designing a specific mission manager for the SWARMs European project, the MTRR, using the CoMMMA proposal as a guideline. This chapter provides a brief overview of the whole

– 17 – Introduction

SWARMs architecture, how the MTRR was designed to be an integral part of it, and how it was finally validated during the SWARMs final audit.

Part V summarizes the conclusions obtained by this research, and outlines possible future works.

• Chapter7 , concludes this dissertation remarking the final conclusions and providing an overview of future directions for this research work.

Part VI includes one appendix with additional information for this dissertation.

• AppendixA summarizes and presents the main scientific dissemination results which are related to the work presented and discussed in this doctoral thesis, which justifies the novelty and impact of our contribution.

Figure 1.5. Dissertation Roadmap

– 18 – Part II

State of the Art

Chapter 2

State of the Art on Related Topics to Adaptive Mission Planning

A complex adaptive system acquires information about its environment and its own interaction with that environment, identifying regularities in that information, condensing those regularities into a kind of "schema", or model, and acting in the real world on the basis of that schema

— Murray Gell-Mann

2.1 Introduction

The contribution to adaptive mission planning for cooperative robotics on the Internet of Things (IoT) includes topics from several fields. First, it is necessary to have a basic knowledge of what is understood by a mission in the field of cooperative robotics. From a common point of view, Collins Dictionary defines a mission as “a specific task or duty assigned to a person or group of people”. That definition can be modified to refer toan agent or a group of agents instead of just humans. And to be more specific, in the field of cooperative robotics, a mission is usually defined as the set of tasks that an agent must perform [45], or as a set of objectives to be achieved, with the schema for achieving those objectives being called a mission plan [44].

The mission follows a life cycle that has been defined in [45], and that is illustrated in Figure 2.1. It starts with the mission creation. The mission is then validated, and transferred to the agent that executes it. Additionally, this proposed life cycle includes a final step for replaying the mission in a simulator for offline analysis.

– 21 – State of the Art on Related Topics to Adaptive Mission Planning

Figure 2.1. Mission life cycle stages

The definition of a mission, or a mission plan, is equivalent to the definition ofaclassical plan fromAI automated planning as a sequence of actions to achieve a goal [65]. Thus, it is a common interest to explore the generation of missions or mission plans using automated planning algorithms. And in particular, for the interest of this dissertation, it is important to have a better grasp of the differentAI automated planning paradigms and description languages that make up the state-of-the-art in this field.

So far this covers the “mission planning” part of this dissertation. With regard to adaptation, [50] described the basic concepts for adaptive planning. In a first approach, the adaption of a plan occurs when the plan cannot be continued, triggering the need for repairing the plan [51, 52], or making a new plan (re-planning) [53]. It also included an interesting concept, that is, the use of pre-stored plans to be refitted according to the current situation, laying the foundations of Case-Based Reasoning (CBR). In addition, adaptation and adaptive models need to be explored in order to better understand how they can be used to contribute to adaptive mission planning.

Finally, cooperation, and especially cooperation in the framework of the IoT, involves the study of related reference architectures.

Thus, in summary, this chapter contains a study of the paradigms and descriptive languages ofAI automatic planning, of adaptation and adaptive models, and of reference architectures. In addition, it includes an additional section dedicated to exploring the state- of-the-art in relation to mission managers used in cooperative robotics, and evaluated in the terms explored previously in this chapter.

The rest of this chapter is organized as follows. Section 2.2 presents the most relevant automated planning paradigms that are of interest for mission planning for cooperative agents. Section 2.3 introduces the concept of planning description languages, and summarizes the current state-of-the-art in formal languages for automated planning. Section 2.4 provides a comprehensive review of what is formally understood as adaptation related to a system, and gives some details of the most commonly used adaptive models in robotics. Section 2.5 briefly describe the architectural reference models that are of use for a SOA, and more specifically, in relation to the IoT. Finally, Section 2.6 summarizes the main conclusions of this chapter.

– 22 – 2.2 State of the Art in Automated Planning Paradigms

2.2 State of the Art in Automated Planning Paradigms

2.2.1 Introduction

This section studies the mainAI planning paradigms that can be of interest for mission planning. It is organized as follows. Section 2.2.2 explores the classical definition of planning and automated planning. Section 2.2.3 studies the basics of probabilistic planning and Markov Decision Processes (MDPs). Section 2.2.4 describes the concept of Partially Observable Markov Decision Processes (POMDPs). Section 2.2.5 explains the basic details of decision trees. Section 2.2.6 examines the Hierarchical Task Network (HTN) planning paradigm. And finally, Section 2.2.7 studies the idea of opportunistic planning.

2.2.2 Classical Planning

Classical planning has been defined as the reasoning side of acting [66]. Thus, classical planning is concerned with choosing and organizing actions for changing the state of a system to achieve a goal. The conceptual model for planning is described using the general model for a dynamic system. This general model, applied for planning, is based in the state-transition systems model [66]. Formally, a state-transition system is a 4-tuple ⟨Σ = (S, A, E, γ)⟩, where:

• S = {s1, s2, ...} is a finite or recursively enumerable set of states.

• A = {a1, a2, ...} is a finite or recursively enumerable set of actions.

• E = {e1, e2, ...} is a finite or recursively enumerable set of events.

• γ : S × A × E− > 2S is a state-transition function.

Consequently, a plan, formally defined as the sequence of actions to achieve agoal[65], can be expressed as:

πplan ≡ {S, A, G} (2.1)

Figure 2.2 illustrates the concept of a classical plan as a graph. The top row represents the states of the system, from the initial state S0 to the final goal state represented as Sn.

The bottom row represents the actions that constitute the plan, from the initial action A0 to the final action An−1 that reaches the final goal state Sn from state Sn−1. As depicted,

– 23 – State of the Art on Related Topics to Adaptive Mission Planning

Figure 2.2. Graph description of a classical plan

in classical planning it is expected a sequential progress: from state S0 action A0 is executed leading to state S1, from where action A1 is executed, and so on. The domain model from this formal definition can be made as shown in Figure 2.3. In this figure, one plan is composed of one or more actions that lead to a state thatcan be a goal. Each action also has an execution order, that can be just the order in the sequence, a temporal constraint, or any other.

Figure 2.3. Classical plan domain model

The way of executing a classical plan is to feed the sequence of actions to a controller [66]. First, a domain expert and a planning expert define the description of the domain and the problem, both descriptions are used by the planner, who generates the plan that is sent to the controller. The controller acts on the system, and if it has a feedback loop, it can detect when the plan cannot continue its execution, and either try to repair the plan [51, 52], make a new plan or re-plan [53], or just stop. This process is illustrated in Figure 2.4.

Figure 2.4. Typical classic plan execution

– 24 – 2.2 State of the Art in Automated Planning Paradigms

Finally, it is important to notice that the classical planning paradigm only addresses problems for deterministic systems, where the later state of the system is determined by the earlier ones.

2.2.3 Probabilistic Planning: Markov Decision Processes

In order to overcome the limitations of the classical planning paradigm for non-deterministic systems, there is the probabilistic planning paradigm. In non-deterministic systems, the future states are not always predictable, and even when they are, their modeling is excessively complex and costly. In probabilistic planning, instead of knowing exactly the state that will be reached when executing an action, an estimated likelihood of the effects is calculated.

The usual model for a probabilistic plan is a Markov Decision Process (MDP)[67]. A MDP is a stochastic state-transition system with a probability distribution. It can also be seen as a sequence of stages with actions and rewards, or as a decision network [49]. The formal characterization of a MDP consists of a 4-tuple ⟨S, A, P, R⟩ [49], where:

• S is a set of states of the world.

• A is a set of actions (with As being the finite set of actions available from state s).

• P : S × S × A → [0, 1], usually also written as P(s′|s, a), is the probability that action a in state s will transition to state s′. The sum of all the probabilities for an action a at a given state s must be equal to 1, as in Equation 2.2.

∀s ∈ S, ∀a ∈ A ⇒ ∑ P(s′|s, a) = 1 (2.2) s′∈S

• R : S × A × S → ℜ, also expressed as R(s, a, s′), is the immediate reward (or expected immediate reward) received after transitioning from state s to state s′, due to action a. The reward for an action a at a given state s, or R(s, a) is the sum of the product of each reward to transition to state s′ from s doing action a times the probability of transitioning to state s′ from s doing action a, as in Equation 2.3.

R(s, a) = ∑ R(s, a, s′) ∗ P(s′|s, a) (2.3) s′

Consequently, the plan for a non-deterministic system Σ can be represented as a policy π used to estimate the action a from the current state s. Then the problem of a MDP is

– 25 – State of the Art on Related Topics to Adaptive Mission Planning to find a policy function π(s) that returns the action to be executed at state s [68]. Once a policy has been defined for a MDP, the action to be executed at each state can be easily decided, and this together behaves as a Markov chain.

Figure 2.5 shows an example of a simple MDP graph with an initial state S0, three intermediate states S1 to S3 and a goal state G. To keep the graph as simple as possible, it shows neither the probabilities nor the rewards. In this example the initial state S0 has two possible actions, A0,0 and A0,1, each of one with their corresponding probabilities to reach a different state. For instance, action A0,0 can reach either state S1 or state S2 with a probability and a reward that are considered by the policy π to decide whether to choose one action or the other.

Figure 2.5. Graph description of MDP

A rough domain model for a MDP is shown in Figure 2.6. In this figure, the plan, rather than actions, is composed of a sequence of goals intended to guide the decisions of the decision functions. The policy, as an implementation of a plan, is composed of one or more decision functions. The decision functions are related to the actions and the states. They use the probability of reaching a desired state from the current one by doing an action, as well as the reward for the same transition. And as with classical planning, some states can also be goals.

Executing a non-deterministic plan based on MDPs is easy once the policy is known. The agent first observes the current state s, and if s ∈ S, that is, if the current state is a known state, it just executes the action returned by π(s).

In conclusion, the use of MDPs extends the capabilities of classical planning by incor- porating non-deterministic models through the use of probability functions. However, a

– 26 – 2.2 State of the Art in Automated Planning Paradigms

Figure 2.6. MDP domain model

MDP requires the agent executing the policy to know exactly its current state at every moment.

2.2.4 Partially Observable Markov Decision Process

Partially Observable Markov Decision Process (POMDP) (pronounced “pom-dee-pee”) is a generalization of a MDP for those situations where the states are not fully observable [49, 65, 67]. Built on the basis of a MDP, a POMDP includes a set of possible observations and their probabilities. Consequently, a POMDP can be defined as a 7-tuple ′ ′ ⟨S, A, O, P(S0), P(S |S, A), R(S, A, S ), P(O|S)⟩, where:

• S is a set of states.

• A is a set of actions.

• O is a set of possible observations.

• P(S0) is the probability distribution of the initial state.

• P(S′|S, A) is the probability of reaching state S′ by doing action A from state S.

• R(S, A, S′) is the expected reward for starting in state S, doing action A, and reaching state S′.

• P(O|S) is the probability of observing O in state S.

The operational mode of a POMDP is first to make the partial observation, and using the probability P(O|S) infer the most probable current state s, from where decide which action a to take.

Figure 2.7 shows an example of a simple POMDP graph, based on the one used for

MDP. As before, this graph has an initial state S0, three intermediate states S1 to S3 and a goal state G. And again, to keep the graph as simple as possible, it shows neither

– 27 – State of the Art on Related Topics to Adaptive Mission Planning

the probabilities nor the rewards. In this example the observation O0 is linked to the state S0, meaning that observing O0 implies that the most probable current state of the system is S0, where there are two possible actions, A0,0 and A0,1, each of one with their corresponding probabilities to reach a different state. Assuming action A0,0, subsequent observations are expected to be either O1, related to state S1, or O2, related to state S2. And as with the MDP example, taking this action has its corresponding reward.

Figure 2.7. Graph description of POMDP

The domain model for a POMDP shown in Figure 2.8 is a modified version of the information model for a MDP presented in the previous section, with the addition of the observation. With this modification, there is an observation that is used by the decision function in the policy to infer the current state and use this information to select the best action to execute.

Figure 2.8. POMDP domain model

– 28 – 2.2 State of the Art in Automated Planning Paradigms

2.2.5 Decision Trees

According to [69], “a decision tree is an explicit representation of all scenarios that can possibly emanate from a given decision”. Knowing a decision tree for a given problem enables a rational agent to decide on a plan to reach a desired goal [49].

A decision tree is just a tree graph with three types of nodes:

• Decision nodes, where the decisions are made.

• Chance nodes, where some chances can happen after making a decision.

• End nodes, where no more decisions are considered or cannot be taken.

Figure 2.9 illustrates a simple generic example of a decision tree using its classical representation. Squared nodes represent decision nodes, and circle nodes represent chance nodes [69]. In this graph each decision node is expressed as Di,j, where i represents the level of the decision node, and j is just the order of the decision node at that level. In turn, actions are expressed as Ai,j,k, where i and j have the same meaning as for the decision nodes, and k represents the id of the action for the pair

(i, j). In this example the root of the decision tree, D0,0, can take two different actions,

A0,0,0 and A0,0,1. Each of this actions are followed by new decision nodes, D1,0 and D1,1 respectively, that have their corresponding actions. And after these actions there are some chance nodes, representing that the system can transition to one state or another. In this example the chance nodes are only at the end of the graph, leading to different final states SFi,j. But in more complex decision trees, chances can happen at any layer of the tree.

Figure 2.9. Classical graph description of a decision tree

– 29 – State of the Art on Related Topics to Adaptive Mission Planning

The same example is illustrated in Figure 2.10 using the graph style that has been used for the classical planning, MDPs and POMDPs.

Figure 2.10. Modified graph description of a decision tree

And a possible mapping of Figure 2.10 to the states of a system is illustrated in Fig- ure 2.11. In this example, state S0 is linked to the decision node D0,0, where a decision is made to reach state S1 choosing the action that will get to that state from the current context. And the same happens at the next level, choosing the best suited action to reach the goal state G from the corresponding decision node, that can be either D1,0 or

D1,1, depending on which path was executed at S0.

Figure 2.11. Modified graph description of a decision tree including the states

Of course, this is just a simple example. Each of the decision nodes could have more than two possible actions, and any of them can lead to a different known state.

– 30 – 2.2 State of the Art in Automated Planning Paradigms

A possible domain model for a decision tree is shown in Figure 2.12. In this diagram, using the classical planning one as a reference, one plan has one or more decision functions, and each decision function decides on each action.

Figure 2.12. Domain model for a decision tree

2.2.6 Hierarchical Task Networks

A HTN is a planning technique that introduces a hierarchical decomposition of tasks in the classical planning paradigm. By definition, a HTN planning problem includes an initial state, a task network as an objective to be achieved, and the domain knowledge consisting of primitive and compound tasks [70]. The set of tasks that can be part of the HTN are:

• Primitive tasks, which are essentially the actions of a classical plan.

• Compound tasks, which are a composed set of primitive tasks.

• Goal tasks, which are like the goals of a classical plan, but in a more general sense.

Thus, the plan in a HTN problem is the sequence of primitive tasks that can be obtained from the HTN by the decomposition of the compound tasks into their primitive ones.

Figure 2.13 shows a simple example graph of a HTN. This example is intentionally similar to the one in Figure 2.2 to better illustrate the contribution of a HTN. In this simple example the HTN is the graph of actions and their hierarchies (here just with one level of composition). The plan to be executed, as explained before, is obtained by decomposing the compound tasks into their primitive ones. In this example, task A0 is a compound task, composed by tasks A0,0 to A0,m. Therefore, the plan execution starts with task

A0,0, and continues to task A0,m, and then to task A1, and so on until reaching the goal state Sn. However, it is important to keep in mind that the HTN can be as complex as needed, and have as many hierarchical layers as required.

– 31 – State of the Art on Related Topics to Adaptive Mission Planning

Figure 2.13. Graph description of a HTN

Finally, a domain model approach of a HTN is shown in Figure 2.14. This diagram is based on the classical planning one, but it can also be applied to any other, as the relevant change is related to the action class. In a HTN an action, representing what in HTN is called a task, has two subclasses: primitive action and compound action. And a compound action class is composed of one or more primitive actions.

Figure 2.14. Domain model example of a HTN

2.2.7 Opportunistic Planning

Sometimes, during the execution of a plan, opportunities arise that allow an agent to achieve objectives that are complementary to those originally defined [54]. In a first approach to opportunistic planning, Cashmore et al. defined an opportunistic planning model for deterministic planning problems with temporal constraints [71]. In their proposal an opportunity is considered as a dynamic soft goal that can happen at any time during the execution of a mission plan. And if an opportunity is discovered, a new plan is generated taking into account the time passed from the start of the mission in order to avoid further issues with time-constrained actions from the original plan.

– 32 – 2.2 State of the Art in Automated Planning Paradigms

From a conceptual point of view, an opportunistic plan can be seen as an alternative branch of a plan that can be followed only when the opportunity appears, and as long as certain conditions are met. As a simple graphical approximation, using the same graph descriptions from the previous planning paradigms, Figure 2.15 illustrates the graph for an opportunistic plan over a deterministic plan. In this graph the big hexagonal node at the center represents one opportunity. This opportunity is evaluated at every state, and if the opportunity arises, the execution of the plan is diverted to the opportunistic branch. In this graph the opportunity reaches a new goal, and finishes the plan. But it could also return to the original state that triggered the opportunistic branch, or even a new plan could be generated for reaching the original main goal.

Figure 2.15. Graph description of an opportunistic planning

An example of an opportunistic planning domain model is shown in Figure 2.16. In this example, one plan can have several opportunities that can alter the course of the plan execution.

Figure 2.16. Domain model example of an opportunistic planning

As with HTN, opportunistic planning can be applied to both deterministic and non- deterministic plans.

– 33 – State of the Art on Related Topics to Adaptive Mission Planning

2.2.8 Conclusions

This section has explored the basic concepts ofAI planning. Firstly, it has presented the classical approach forAI planning for deterministic systems. For non-deterministic systems, it has explored the use of MDPs and POMDPs. And finally, for both deterministic and non-deterministic systems, it has covered the use of decision trees for guiding the plan execution, hierarchical task networks for actions decomposition, and opportunistic planning, to take advantage of desirable situations that may occur during the execution of a plan.

2.3 State of the Art in Automated Planning Description Languages

2.3.1 Introduction

This section presents a description of a selected number of formal description languages used in automated planning, for either the description of the domain (the world model), the planning problem, or the proposed solution as a plan.

The rest of this section is organized as follows. Section 2.3.2 describes the Stanford Re- search Institute Problem Solver (STRIPS) description language. Section 2.3.3 describes the Action Description Language (ADL) description language. Section 2.3.4 describes the Planning Domain Definition Language (PDDL) description language. And finally, Section 2.3.5 summarizes some conclusions for this section.

2.3.2 Stanford Research Institute Problem Solver

Stanford Research Institute Problem Solver (STRIPS) is both the name of the automated planner and the description language developed in 1971 by Richard Fikes and Nils Nilsson at the Stanford Research Institute (SRI)[72]. Despite its age, it is still used in several automated planning related topics [73, 74]. A problem in STRIPS is defined using an initial world model, a set of available operators and their effects on the world model, and the goal. The world model, also known as the domain, is represented using well-formed formulas (wffs), in this case, a finite sequence of logical descriptive instances. For instance, a simple wff can be AT(A, a), meaning that object A is located at position a.

The operators of the STRIPS language have two main parts:

• The conditions under which the operator is applicable, or preconditions.

• The effects of the operator in the world model.

– 34 – 2.3 State of the Art in Automated Planning Description Languages

For instance, assuming an agent with the ability to move freely from one position to another, the operator goto(a, b) have:

• Preconditions:

– The agent must be at a. – The location b must be free.

• Effects:

– The agent is no longer at position a. – The agent is now at position b.

The operators are described as schemes, including the preconditions and the effects. The effects are described by the use of two lists: delete and add. The delete list is used to specify the clauses, or wffs, that are no longer true in the updated model of the world. Likewise, the add list is used to detail the wffs that are true in the updated model of the world.

As a basic exercise, Figure 2.17 illustrates a simple example world model for a problem. In this example there is a robot located at position a, a box located at position b, and an empty space at position c. The goal for the exercise is for the robot to move the box from a to c.

Figure 2.17. Example world model

The initial world model for this example can be described in STRIPS as follows:

ATROBOT(a) AT(BOX,b) ROBOTLOAD(0) ONFLOOR(BOX)

In this exercise there are three: goto, pick and drop. The schema for them is as follows:

– 35 – State of the Art on Related Topics to Adaptive Mission Planning

goto(m): Robot goes to location m.

• Preconditions:

• Effects:

– Delete list: ATROBOT($) – Add list: ATROBOT(m)

pick(m): Robot picks the object m.

• Preconditions:

– NEXTTO(ROBOT,m) – ROBOTLOAD(0) – ONFLOOR(m)

• Effects:

– Delete list: ∗ ONFLOOR(m) ∗ AT(m) ∗ NEXTTO(ROBOT,m) ∗ ROBOTLOAD(0) – Add list: ∗ ROBOTLOAD(m)

drop(m): Robot drops the object m.

• Preconditions:

– ROBOTLOAD(m)

• Effects:

– Delete list: ∗ ROBOTLOAD(m) – Add list: ∗ ROBOTLOAD(0) ∗ ONFLOOR(m) ∗ AT(m) ∗ NEXTTO(ROBOT,m)

The goal for this planning problem is to have the box at location c. This is expressed in STRIPS with the wff AT(BOX,c). The solution to a STRIPS problem is a sequence of

– 36 – 2.3 State of the Art in Automated Planning Description Languages specific operators that transforms the current model of the world to one that matchesthe wff for the goal. For instance, a possible STRIPS solution for this exercise would be:

{goto(b), pickup(BOX), goto(c), drop(BOX)} (2.4)

2.3.3 ADL

In 1987 Edwin Pednault introduced the concept of action languages for the description of actions and their effects [75]. This language was conceived as a combination of the adventages of the notation from STRIPS language and the semantics and expressive power of the situation calculus. The main contributions of Action Description Language (ADL) to STRIPS can be summarized as:

• Introduction of disjunctive and negative preconditions.

• Introduction of conditional effects.

• Open-world assumption.

• Equality expressions.

• Support for typed data.

The syntax for ADL is essentially the same as the one from STRIPS, with the additional elements related to the already mentioned contributions. For example, with ADL, instead of expressing that the box from the exercise in Section 2.3.2 is on the floor on location b, it can be expressed that it is not on location c (negative precondition); and for the pick operator, a precondition for asserting that the parameter is not referring to the self can also be used (equality expressions).

2.3.4 PDDL

The Planning Domain Definition Language (PDDL) was the first attempt to create a standard definition for planning languages. It was created in 1998 for the Artificial Intelligence Planning Systems Conference (AIPS) planning competition. PDDL follows the syntactic features of STRIPS and is a descendant of ADL among others [76].

PDDL uses an extended Backus-Naur form (EBNF) notation to provide the linguistic tools necessary to formally describe a planning problem, including the domain of the problem, and the problem description itself, both given in separate files.

– 37 – State of the Art on Related Topics to Adaptive Mission Planning

The domain is the model of the world for the planning problem. It must declare every aspect of relevance to solve the problem. To do this, PDDL provides the fields that can be part of the structure of a domain description. These fields are listed in Table 2.1 with their corresponding description.

Table 2.1. Fields of a domain in PDDL

Name (in EBNF) Type Description :extends Optional Declaration that this domain inherits all the declarations from other domains. :requirements Optional Declaration of the capabilities required for the planner for understanding the domain. If no requirements are declared, a :strips requirement is used as default. :types Optional Declaration of the types of a list of entities. :predicates Required List of declarations of predicates, in the same sense as in first-order logic. :axioms Optional Relationship assertions among propositions in the domain. The use of :axioms requires the :requirements declaration of :domain-axioms. :actions Required Description of the possible actions in the domain, also given in EBNF notation. An action can have parameters, preconditions and effects.

Using the same planning problem exercise from Section 2.3.2, the domain of the problem can be expressed in PDDL as in Listing 2.1. Please note that the STRIPS operator goto has been here renamed to the action move. Also, this example is similar to the one used in the “PDDL by Example” course [77].

(define (domain gripper−example ) (:predicates (location ?l) (box ?b) (gripper ?g) (at ?b ?r) (at−robot ?r) (free ?g) (carry ?o ?g))

(: action move :parameters (?from ?to) :precondition ( and (location ?from) (location ?to) ( at−robot ?from)) : e f f e c t ( and ( at−robot ? to ) ( not ( at−robot ?from))))

(:action pick :parameters (?obj ?location ?gripper)

– 38 – 2.3 State of the Art in Automated Planning Description Languages

:precondition ( and ( box ? obj ) (location ?location) (at ?obj ?location) ( at−robot ?location) (free ?gripper)) : e f f e c t ( and (carry ?obj) ( not (at ?obj ?location)) ( not (free ?gripper))))

(:action drop :parameters (?obj ?location ?gripper) :precondition ( and ( box ? obj ) (location ?location) (gripper ?gripper) (carr ?obj ?gripper) ( at−robot ?location)) : e f f e c t ( and (at ?obj ?location) (free ?gripper) ( not (carry ?obj ?gripper))))) Listing 2.1. Domain example expressed in PDDL

The PDDL problem description is used to declare the initial state of the world, and the goal to be achieved.

Table 2.2. Fields of a problem description in PDDL

Name (in EBNF) Description :domain Reference to the domain where the planning problem is applied. :objects List of objects that exist in the problem. :init Initial description of the system, using the predicates declared in the domain. :goal The goal description, expressed as the predicates that must be true for solving the problem.

Continuing with the same example problem as before, Listing 2.2 shows the description of the planning problem using PDDL.

(define (problem gripper−example−problem ) (:domain gripper−example ) (:objects locationA locationB locationC box1 gripper1) (: init (location locationA)

– 39 – State of the Art on Related Topics to Adaptive Mission Planning

(location locationB) (location locationC) ( box box1 ) (gripper gripper1) ( at−robot location1) (free gripper1) (at box1 locationB)) (:goal (at box1 locationC)))

Listing 2.2. Problem description for the example domain expressed in PDDL

PDDL, like STRIPS and ADL, does not provide a notation for describing the solution to the planning problem, that is, the plan. However, a possible solution for this example could be the sequence of actions expressed as LISP calls in Listing 2.3

(move locationA locationB) (pick box1 locationB gripper) (move locationB locationC) (drop box1 locationC gripper)

Listing 2.3. Possible solution to the example problem

Since the first release of PDDL there has been several updates to the notation. Table 2.3 collects the released versions of PDDL, with the year of the release, and their main contributions to the language.

Table 2.3. Versions of PDDL

Version Year Description 1.2 1998 Created for the 1st IPC. This was the first version released, and it provided the basis for the description of the domain and the problem of a planning problem [76]. 2.1 2002 Created for the 2nd IPC. It introduced numeric fluents, plan metrics and continuous durative actions [78]. 2.2 2004 Official language of the deterministic track of the 4th IPC. It introduced derived predicates and timed initial literals [79]. 3.0 2006 Official language of the deterministic track of the 5th IPC. It introduced state-trajectory constraints and preferences [80–82]. 3.1 2008 Official language of the deterministic track of the 6th and7th IPC. It introduced object-fluents [83].

Besides the updates to PDDL, there has been notable contributions to this description language with several variants created to cover certain aspects of planning problems

– 40 – 2.3 State of the Art in Automated Planning Description Languages that were not covered by the PDDL version available at the time. Table 2.4 collects a summary of these variants and their contributions.

Table 2.4. Other variants and extensions of PDDL

Name Year Description NDDL[84] 2002 It is an extension developed by NASA for its Europa-2 project. It added the use of variable/value representations and of intervals instead of actions. MAPL[85] 2003 It is an extension for PDDL 2.1 introducing modifications to enable multi-agent planning problems. PPDDL[86] 2004 It is an extension that adds probabilistic effects, reward fluents, goal rewards and goal-achieved fluents to PDDL 2.1 for expressing non-deterministic planning problems in fully observable domains for realizing MDPs. RDDL[87] 2011 Conceptually based on PPDDL 1.0 and PDDL 3.0, it introduces partial observability among other syntactically and semantically changes, allowing the description of both MDPs and POMDPs non-deterministic planning problems. MA-PDDL[88] 2012 It is an extension of PDDL 3.1 allowing planning for multiple agents, solving most of the issues detected in MAPL. HDDL[89] 2020 It is an extension to PDDL for expressing HTN planning problems.

2.3.5 Conclusions

This section has covered the three most prominent description languages used inAI planning, STRIPS, ADL and PDDL, with a brief mention to some of the many PDDL variants that have appeared over the years.

One of the main restrictions of these languages is that they are mainly focused on the description of the domain and the problem, but the expression of the solution lies out of their scope.

Another drawback that can be identified is that they are specially designed for their use inAI planning exploration and development. That is clearer when noticing that most of the updates and contributions are guided by the interests of each new IPC. For instance, the novel Hierarchical Domain Definition Language (HDDL) has been presented in 2020 for the IPC for that year. The same applies to Relational Dynamic influence Diagram Language (RDDL) in 2011. On the one hand, their usage in other fields is usually complicated, and requires the aid of experts inAI planning. On the other hand, this implies that usually, whenAI planning is required in a practical project, other languages

– 41 – State of the Art on Related Topics to Adaptive Mission Planning are defined, like the already mentioned NDDL, or ad-hoc solutions are developed as shown in some of the mission managers explored in Chapter3.

Finally, one of the objectives of these languages is to provide the syntactical tools to formally describe the domain of a planning problem. That could be related to the use of ontologies and semantic models that have gained attention and use in recent years, especially in relation to IoT systems and even robotics. Indeed, there have been some attempts to define Web Ontology Language (OWL) ontologies for providing a framework for knowledge engineering and planning, but often the domains represented in languages like PDDL are made for a specific application, and therefore not easy to reuse[90].

2.4 State of the Art in Self-Adaptive Models

2.4.1 Introduction

The use of adaptive models can be practical for the adaptation of the missions being planned for cooperative robotics. These models can provide the foundation on how to adapt a mission plan when necessary. In addition, the use of decentralized adaptation patterns may also be of interest for their application in multi-agent systems, such as those that involve the cooperative work of several robots.

In order to have a common knowledge base on what it is understood by adaptation, Section 2.4.2 provides a formal definition of this concept. Sections 2.4.3 to 2.4.6 describe different adaptive models, from the Observe, Orientate, Decide, Act (OODA) loop to the most mentioned one in Cyber-Physical Systems (CPSs), which is the Monitor, Analyze, Plan, and Execute with Knowledge (MAPE-K) model. Section 2.4.7 presents some decentralized control patterns using the MAPE-K adaptive model as a reference. Finally, Section 2.4.8 summarizes the main conclusions on adaptation and adaptive models.

2.4.2 Adaptive Systems Definition

Collins Dictionary defines adaptation as “the act of changing something or changing your behavior to make it suitable for a new purpose or situation”. Cambridge Dictionary defines it as “the process or act of changing to suit a new situation”. And Merriam-Webster defines it as “the act or process of making something fit”, and also as the “modification of an organism or its parts that makes it more fit for existence under the conditions of its environment”. It seems quite reasonable that adapting is changing something to fit a new situation.

The first studies on adaptation in computer science and engineering were mostly inspired by the physiological homeostatic principles [91]. It was Norbert Wiener who introduced

– 42 – 2.4 State of the Art in Self-Adaptive Models the concept of cybernetics applied to self-regulating mechanisms, using the classical equation for a negative feedback as a reference, as shown in Equation 2.5

δJ x = −µ (2.5) δx

In this equation, x is the control action, while J is the objective to be minimized, and µ is a parameter that adjusts the amplitude of the system’s response. In other words, adaptation from this mathematical approach is action that adjusts some system behavior. Consequently, an adaptive system is a set of interacting elements and at least one process controlling the system’s adaptation [92].

Another formal definition is provided by Martin et al. in[92]. This definition about what is adaptation starts by defining a non-adaptive behavior. Formally, given a system S, it is said that a physical event E is a stimulus for the system S if and only if the probability P(S → S′|E) that the system suffers a change or be perturbed (in its elements or in its processes) when the event E occurs is strictly greater than the prior probability that S suffers a change independently of E, as expressed in Equation 2.6.

P(S → S′|E) > P(S → S′) (2.6)

What this equation represents is that for the given system S, it is more probable to change to S′ due to the presence of a stimulus E than when no stimulus is present at all. Using mission planning for cooperative robotics as an example, a team of cooperative robots (the system) may be executing a plan to achieve some goals when some external event occurs preventing them to fulfill the mission (the event), and leading to afailed mission (the modified system S′).

Martin et al. suggested that any system with adaptive capabilities must have some motivations to change their behavior, expressing it as the “Law of Constancy”:

Law (of Constancy). Every adaptive system requires a source of constancy to converge and it tends to that source of constancy.

This law implies that any adaptive system will react to stimuli that may change their source of constancy by performing some kind of adaptation. Or expressed in other terms, no adaptive system will change their behavior if there are no external stimuli. This principle is expressed as the “Principle of Justified Persistence”:

– 43 – State of the Art on Related Topics to Adaptive Mission Planning

Principle (of the Justified Persistence). If an organism or system exists in a specific state of its environment, then the maximum a priori probability for surviving (avoiding extinction) is obtained when the environmental conditions are constant or the change in the environment is highly smooth.

As generalization of this, Martin et al. proposed also the “Law of Inertia” of adaptive systems:

Law (of Inertia of adaptive systems). An adaptive system that is in a steady-state and without internal or external stimuli will remain in a steady-state, that is, without changing its behavior.

Together, these two laws and the principle give rise to a general “Law of Adaptation” that encloses the behavior of an adaptive system.

Law (of Adaptation). Every adaptive system converges to a state in which all kinds of stimulation ceases.

A formal description of this law is, given S as an arbitrary system subject to changes in the time dimension t, and E as an arbitrary event that is a stimulus to S, it is said that S is an adaptive system if and only if when t tends to infinity, the probability that the system

S changes its behavior in a time step t0 given E is the same has the probability that S changes its behavior independently of the event E [92]. Mathematically:

′ ′ lim Pt(S → S |E) = Pt(S → S ) (2.7) t→∞

In summary, and using again mission planning for cooperative robotics as an example, a team of cooperative robots (the system) has some mission goals (the source of constancy), and when an event occurs (an external stimulation), the team change their actions to still reach the mission goals (converge to its source of constancy). And that is an adaptive behavior.

2.4.3 The OODA loop

The OODA loop model was introduced by United States Air Force (USAF) Colonel John Boyd as a strategic concept for combat operations [93–95]. Since its introduction, it has been applied to several other fields where decision making is required. For instance, it has been integrated as part of the adaptive process in mission management architectures

– 44 – 2.4 State of the Art in Self-Adaptive Models like ICA, which will be studied in Section 3.3.3. It also has been proposed as part of adaptive mission planning [96] for deciding when a mission plan needs to be updated.

The OODA loop as proposed by Boyd is shown in Figure 2.18. It consists of four steps: Observation, Orientation, Decision and Action.

Figure 2.18. The OODA Loop

The first step, theObservation “ ”, corresponds to the assessment of the environment, the self, and the interaction between both.

The next step is “Orientation”, using all the available knowledge to analyze the inputs from the observation. In the original definition of the loop, this knowledge was composed by the cultural traditions, the genetic heritage, the previous experiences, and the new information. Those concepts can easily be adapted to any other domain beyond the original one.

The third step is “Decision”, which is just the choice of a course of action based on the analysis result from the previous step. An important part of the decision step is also to foresee the possible outcomes of the action chosen.

The final step isAction “ ”, where the decision made in the previous step is implemented, and the possible outcomes that were predicted are verified.

According to Boyd, the OODA loop is non-linear, using an assorted number of feedback and feed-forward links, as they were already shown in Figure 2.18. In Boyd’s words, the OODA loop is “an evolving, open-ended, far from equilibrium process of self-organization, emergence, and natural selection”.

2.4.4 ECA

The Event-Condition-Action (ECA) model is not really an adaptive model, although it matches the definition of adaptation. It is indeed a classical computing paradigm[97]

– 45 – State of the Art on Related Topics to Adaptive Mission Planning that rules the reaction of a system when an event occurs, by checking a predefined set of conditions, and if one holds, execute the corresponding action [98]. It has been mostly used in database systems [99, 100], but it also has been considered as one of the possibilities for the planning element of a MAPE-K model [101]. As mentioned before, reacting to an event according to a rule matches the definition of adaptation given in Section 2.4.2. However, there is no formal definition of any of the elements ofan ECA model. Therefore, the way in which an event is detected and identified, the selection and type of the condition to evaluate, and the action to be taken, are left to the specific needs of each case.

As a simple reference, Figure 2.19 illustrates the common flow of an ECA solution, from the event to the corresponding action, through the evaluation of a known condition.

Figure 2.19. The ECA rule approach for reacting to an event

2.4.5 SPA

Originally mobile robot architectures were designed using a Sense-Model-Plan-Act (SMPA) framework [47, 102]. However, it was soon found that the modeling stage was problematic, preventing progress in developments, and giving rise to a simplified framework model known as Sense-Plan-Act (SPA)[46, 102, 103].

The functional description of a SPA model is depicted in Figure 2.20.

Figure 2.20. The SPA model

From the robotics perspective, a SPA architecture consists of three functional elements that give name to the model. The “sensing” element is in charge of the interpretation of the raw data gathered from the different sensor elements in the robot. This information is passed to the “planning” element, which uses it in conjunction with an already available internal model of the world and a given goal to plan the next course of action. Finally, the “acting” element, generally called the “executor”, executes the actions in the plan.

The SPA model may look similar to the ECA model already discussed. But there are two distinguishing features. ECA is guided by events, while SPA is guided by observations of the world. Of course, an event is detected by an observation, but not all observations are events. Also, in ECA the actions to adapt the system are decided by rules that can be

– 46 – 2.4 State of the Art in Self-Adaptive Models defined in different ways, like just fixed conditional rules, fuzzy rules, and so on. Unlike this approach, in SPA the actions originally were decided by a planner.

Finally, as the SPA model is guided by the sensing element, it still matches the definition of adaptation given in Section 2.4.2, since it plans the actions to suit the given objective and the observed state of the world. Indeed, a variation of the SPA model has been used in the T-REX architecture, as mentioned later in Section 3.3.1.

2.4.6 MAPE-K

Perhaps the most widely used adaptive model in CPSs is the Monitor, Analyze, Plan, and Execute with Knowledge (MAPE-K) control loop [104] from the autonomic computing model defined by IBM in 2005 [105]. The autonomic computing model defines an architecture distributed in four elements, as depicted in Figure 2.21, and that are described as follows:

• Managed elements. These are the elements representing any software or hardware resource that is given autonomic behavior by coupling it with an autonomic manager.

• Sensors. These are the elements that collect information about the managed element.

• Effectors. These are the elements that are responsible for carrying out the changes to the managed elements decided at the autonomic manager.

• Autonomic manager. This is the element responsible for monitoring the environment, analyzing the collected information, defining the corresponding plan, and executing the plan if required. An autonomic manager can manage one or many elements, depending on the system specific design.

Figure 2.21. Architectural blueprint for autonomic computing from IBM

– 47 – State of the Art on Related Topics to Adaptive Mission Planning

The main element of this model is the autonomic manager. It is made up of five functional elements that give MAPE-K its name: Monitor, Analyze, Plan, Execute and Knowledge. The functional description of an autonomic manager with its five elements is illustrated in Figure 2.22, and the description of the elements is as follows:

• A“Monitor” that gathers the information of the managed resource through the sensors.

• An “Analyze” that correlates the gathered information from the monitor to known situations.

• A“Plan” that provides the relevant actions to be executed according to the results from the analysis in the analyze element.

• An “Execute” that dispatches and controls the execution of the actions resulting from the plan element.

• A“Knowledge” that is shared among the rest of the elements of the autonomic manager.

Figure 2.22. Functional description of a MAPE-K loop

An interesting feature of the MAPE-K proposal is that it allows for the hierarchical distribution of autonomic managers, so that the autonomic managers themselves can in turn be elements managed by other autonomic managers. Figure 2.23 illustrates an example of a two-level autonomic management hierarchy. In this example, there is one autonomic manager at the high-level, that manages two elements in the mid-level. These two elements are in turn autonomic managers of the three managed elements at the low-level, where the first mid-level autonomic manager just manages one low-level element, and the second mid-level autonomic manager manages the other two low-level elements.

– 48 – 2.4 State of the Art in Self-Adaptive Models

Figure 2.23. Example of a hierarchy of MAPE-K autonomic managers

2.4.7 Decentralized Control Patterns for Self-Adaptive Systems

As explained in Section 2.4.6, MAPE-K introduces the concept of managed and managing systems, and how this concept can be escalated in such a way that the managed system is in turn a manager itself for another managed system or element, thus creating a distributed self-adaptive system. Weyns et al. propose some patterns for decentralized control in [106] using the MAPE-K model. Their proposed patterns abstract the knowledge element from MAPE-K and how that knowledge is shared among the rest of the MAPE-K elements, either in a unique autonomic manager or in a distributed adaptive system. This is done mainly because the treatment of the knowledge element is heavily dependent on the domain characteristics. The legend for the diagrams used to represent both the patterns and the examples is shown in Figure 2.24.

Figure 2.24. Legend for the decentralized control pattern diagrams

The identified patterns in the work from Weyns et al. are the ones listed below, witha reference to the subsection where are described next.

• Coordinated control (Section 2.4.7.1).

• Information sharing (Section 2.4.7.2).

• Master/slave (Section 2.4.7.3).

• Regional planning (Section 2.4.7.4).

• Hierarchical control (Section 2.4.7.5).

– 49 – State of the Art on Related Topics to Adaptive Mission Planning

2.4.7.1 Coordinated Control

Sometimes it is not possible to centralize the control needs for self-adaptation. It may be because of the distributed nature of the information needed to decide when and how to adapt, to the scale of the system or even just that the systems that are integrated together are owned by different entities. But in those cases there may still be some situations where a coordinated adaptation between the participating systems is desired.

The proposed pattern suggests that any participating system has its own MAPE-K, where each functional element of the loops is coordinated with their peers in other loops from other systems. Figure 2.25a shows the diagram for the coordinated control pattern, and Figure 2.25b shows a possible example.

(a) Pattern (b) Example of use

Figure 2.25. Coordinated control

An example implementation of a decentralized control self-adaptive system can be found in [107], despite being even prior to the original proposal of the MAPE-K model. In that example, the authors used a local component manager deployed at each node of a distributed application. At the same time, each component manager cooperates with the others to keep the architecture under certain constraints in the face of certain types of events.

2.4.7.2 Information Sharing

In a similar situation like the one for the previous pattern, there can be scenarios where loosely connected systems can adapt locally, but they may need information from other systems, and just information. Consequently, the pattern for these situations is similar to the coordinated pattern, but restricted to the cooperation between the monitoring functional elements from the MAPE-K loops. Figure 2.26a shows the diagram for the information sharing pattern, and Figure 2.26b shows a possible example.

An example of use of this pattern can be found in [108]. In that work the information sharing pattern is used for the self-healing management of a traffic monitoring system. In this system, a group of traffic cameras monitor the conditions of the road, sharing

– 50 – 2.4 State of the Art in Self-Adaptive Models

(a) Pattern (b) Example of use

Figure 2.26. Information sharing information that is used to dynamically organize the system to span the control of detected traffic jams.

2.4.7.3 Master/Slave

Occasionally the distribution of a system requires that the monitoring and acting parts of the adaptation loop are performed locally, at the managed elements. But the analysis and planning of the actions to be executed for adaptation requires for a certain degree of global control. In these cases, the local autonomic managers will have only the functional monitoring and execution elements of a MAPE-K. The analysis and planning elements will be separate, shared with the rest of autonomic managers, with whom they will interact. Figure 2.27 shows the diagram for the master/slave pattern, and Figure 2.28 shows a possible example.

Figure 2.27. Master/Slave pattern

An example of use of a master/slave pattern can be found in [109], where it is used for managing the adaptation of cloud resources in a virtualized cloud computing infrastructure.

– 51 – State of the Art on Related Topics to Adaptive Mission Planning

Figure 2.28. Master/Slave pattern example

2.4.7.4 Regional Planning

The regional planning pattern is conceived for those scenarios where there is a need for planning the execution in certain managed elements due to the monitoring and analysis results from other managed elements. Figure 2.29a shows the diagram for the regional planning pattern, and Figure 2.29b shows a possible example.

(a) Pattern (b) Example of use

Figure 2.29. Regional planning

As examples of use of the regional pattern, the framework presented in [110] decides the optimal deployment of certain software at runtime, using centralized analyzers at master hosts that can plan the actions to execute at slave hosts. And an adaptive management of composite services is presented [111] using also a regional pattern for adaptation. In this case the proposed architecture uses a common adaptation manager to decide the adaptation of concrete services by using information from several of them.

2.4.7.5 Hierarchical Control

Another type of adaptation that can be found in complex distributed systems is when every entity in the system has its own adaptive requirements, but also can be part of the adaptation needs decided at upper levels in a hierarchical distribution of responsibilities. This is the motivation for a hierarchical control pattern, Figure 2.30 shows the diagram for the hierarchical control pattern, and Figure 2.31 shows a possible example.

– 52 – 2.4 State of the Art in Self-Adaptive Models

Figure 2.30. Hierarchical control pattern

Figure 2.31. Hierarchical control pattern example

The most classic example of use for this pattern is the one provided in the MAPE-K proposal from IBM [105], and that resembles the generic example illustrated in Figure 2.23, where high-level managers manage elements that are in turn managers of managed elements at a lower level.

2.4.8 Conclusions

This section started by giving a formal definition of adaptation and adaptive systems in order to have a common ground for the description of a selection of adaptive models. The four adaptive models discussed here are of relevance as they are typically referred as the foundations for adaptation in robotics, mission management architectures and CPSs. What is more, the MAPE-K architecture is useful to describe five possible patterns for decentralized control in self-adaptive systems.

The concepts and topics covered in this section are helpful in extending the possibilities of adaptive mission planning. Using the definition of adaptive planning from [50], in

– 53 – State of the Art on Related Topics to Adaptive Mission Planning conjunction with the definition of adaptation given in this section and with the commonalities from the studied adaptive models, it seems interesting to explore the possibility of improving the capabilities of cooperative robotics as a system of systems [112–114]. This improvement could be achieved by focusing on two specific points:

1. By defining a distributed architecture for mission management in cooperative robotics that, enabling the use of a decentralized control pattern for adaptation, includes some improvement in the decision/condition/plan part of the adaptive model.

2. Taking into account the realization of the distributed architecture mentioned in the previous point, by considering the use of alternative execution branches for events that alter the mission progress, thus moving towards an anticipatory system [115].

What lies ahead is to study some reference models for architecting in the IoT that can be considered for cooperative robotics, and that can be of interest for the aforementioned two points of interests.

2.5 State of the Art in Architectural Reference Models

2.5.1 Introduction

The previous sections have studied the concepts and current state-of-the-art in relation to automated planning and adaptation. With those concepts, it has been possible to formulate the opportunity to improve adaptive mission planning for cooperative robotics in two ways: with a distributed architecture, and with the consideration of anticipated possible branches of execution in case of events. Continuing with the interest of this dissertation, the distributed architecture should focus on the IoT. Thus, this section explores the current state of reference architectures for the IoT.

First, Section 2.5.2 describes the foundations of Service Oriented Architecture (SOA), since they are one of the main considerations in IoT models. It is followed by Section 2.5.3, in which the advances in the standardization of the architectural reference models for IoT are related. Next, Section 2.5.4 introduces the Internet of Things Architecture (IoT-A) reference model, as the first complete approach to an architectural reference model, being the result of the IoT-A European project. Finally, Section 2.5.5 summarizes the conclusions from the studied reference models.

2.5.2 Reference Model for Service Oriented Architecture

By definition, a Service Oriented Architecture (SOA) is “a paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership

– 54 – 2.5 State of the Art in Architectural Reference Models domains” [116]. In order to provide guidance for the design of specific service oriented architectures, Organization for the Advancement of Structured Information Standards (OASIS) has published a consolidated reference model [116] as well as a reference architecture foundation [117]. This section describes the main concepts of interests from the reference model.

Figure 2.32 shows the conceptual map for the SOA reference model. In this figure, the main concepts are the ones in dark purple.

Figure 2.32. Conceptual map for the SOA reference model

Descriptions of each of the main concepts from the SOA reference model are listed below. In these descriptions the key words must, must not, required, not required, shall, shall not, should, should not, recommended, may and optional are to be interpreted as described in [118].

Service is the main concept of a SOA, and is defined as the mechanism to enable the access to one or more capabilities, with the access being defined through an interface, and done according to the restrictions and policies detailed in the service description. A service is provided by a service provider, and accessed or consumed by a service consumer.

Visibility is part of the dynamic perspective of a service, and it is related to how the service provider and the service consumer see each other. There are three preconditions to visibility: awareness, willingness and reachability. And they are mandatory, as the initiator in service interaction must be aware of the other parties, must be predisposed to interaction, and must be able to interact.

– 55 – State of the Art on Related Topics to Adaptive Mission Planning

Interaction is also part of the dynamic perspective of a service, and involves performing actions against it. It requires an information model, that is, the characterization of the information that may be exchanged with the service, and a behavior model, that is, the knowledge of the actions invoked against the service and the process or temporal aspects of interacting with it.

Real World Effect is the last concept related to the dynamic perspective of a service, and can be the response to a request for information, or a change in the state of some defined entities shared by the service participants. However, each partyin a service interaction should not have explicit knowledge of the internal actions performed by their peers as a result of their participation in a service interaction.

Service Description is the information needed to use a service. The purpose of this description is to facilitate the interaction and the visibility, and it should be represented in a standard referenceable format. The information that a service consumer needs is as follows:

• The existence and reachability of a service should be sufficient to enable the interaction between the consumer and the provider of a service. • The function(s) and the real world effects of a service, that should be expressed unambiguously, and should be generally understandable by consumers. • The constraints and policies associated with the service, if any. • The service interface, which should be syntactically represented in a standard format. A usable service must be represented in a format that can be understood by its consumers.

This information should be represented in any service description, but it is not required to include it explicitly.

Contract and Policy refer to the agreement by two or more parties (the contract), and the constraints or conditions on the use (the policy). Both contracts and policies should be written in a form that is understandable to, and processable by, the parties to whom they are directed. They may also be automatically interpreted.

Execution Context is the set of structured elements, process entities, policy assertions and agreements that are identified as part of a service instance interaction. Itis not limited to one side of the interaction, and allows to distinguish between different instances of the same service.

– 56 – 2.5 State of the Art in Architectural Reference Models

2.5.3 IoT Standards

The IoT has attracted the interest of reference bodies in terms of regularization and standardization. Mainly, the International Telecommunications Union (ITU), the Interna- tional Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) and the Institute of Electrical and Electronics Engineers (IEEE) have generated abundant reference documentation to guide the development of IoT-based technologies. The next subsections describe the works made by each of these bodies, and how they are relevant from an architectural reference model point of view.

2.5.3.1 ITU Standards

The ITU, being the agency who is first recognized as using the expression “Internet of Things” [119], has published an important number of recommendations on the matter. The ITU-T study group SG20, as the group for Internet of Things and smart cities and communities (SC&C), is responsible for the ITU-T Y series recommendations for global information infrastructure, Internet protocol aspects, next-generation networks, Internet of Things and smart cities. In particular, recommendations Y.4000 to Y.4999 are assigned for Internet of Things and smart cities and communities, with the following organization:

• Y.4000 – Y.4059: General.

• Y.4050 – Y.4099: Definitions and terminologies.

• Y.4100 – Y.4249: Requirements and use cases.

• Y.4250 – Y.4399: Infrastructure, connectivity and networks.

• Y.4400 – Y.4549: Frameworks, architectures and protocols.

• Y.4550 – Y.4699: Services, applications, computation and data processing.

• Y.4700 – Y.4799: Management, control and performance.

• Y.4800 – Y.4899: Identification and security.

• Y.4900 – Y.4999: Evaluation and assessment.

Some of the Y-series recommendations about IoT related topics were previously published with another numbering, being reassigned in mid-2016 to the range Y.4000–Y.4999. Table 2.5 shows a selection of the published recommendations, with their recommendation number, the year in which they came into force, their current status, and their title.

– 57 – State of the Art on Related Topics to Adaptive Mission Planning

Table 2.5. ITU-T Recommendations on IoT

Number Year Status Title Y.4000 2012 In force Overview of the Internet of Things Y.4050 2012 In force Terms and definitions for the Internet of Things Y.4100 2014 In force Common requirements of the Internet of Things Y.4101 2017 In force Common requirements and capabilities of a gateway for Internet of Things applications Y.4102 2015 In force Requirements for Internet of Things devices and operation of Internet of Things during disasters Y.4103 2014 In force Common requirements for Internet of Things applications Y.4111 2016 In force Semantics based requirements and framework of the Internet of Things Y.4112 2016 In force Requirements of the plug and play capability of the Inter- net of Things Y.4113 2016 In force Requirements of the network for the Internet of Things Y.4114 2017 In force Specific requirements and capabilities of the Internet of things for big data Y.4115 2017 In force Reference architecture for IoT device capability exposure Y.4117 2017 In force Requirements and capabilities of the Internet of Things for support wearable related services Y.4118 2018 In force Internet of Things requirements and technical capabilities for support of accounting and charging Y.4119 2018 In force Requirements and capability framework for IoT-based automotive emergency response system Y.4120 2018 In force Requirements of Internet of Things applications for smart retail stores Y.4121 2018 In force Requirements of an Internet of Things enabled network for support of applications for global processes of the Earth Y.4203 2019 In force Requirements of things description in the Internet of Things Y.4204 2019 In force Accessibility requirements for the Internet of Things applications and services Y.4208 2020 In force Internet of things requirements for support of edge computing Y.4401 2015 In force Functional framework and capabilities of the Internet of Things Y.4417 2018 In force Framework of self-organization network in Internet of Things environments (continued on next page)

– 58 – 2.5 State of the Art in Architectural Reference Models

Table 2.5. ITU-T Recommendations on IoT (continued)

Number Year Status Title Y.4418 2018 In force Gateway functional architecture for Internet of Things applications Y.4453 2016 In force Adaptive software framework for Internet of Things devices Y.4455 2017 In force Reference architecture for Internet of Things network service capability exposure Y.4459 2020 In force Digital entity architecture framework for Internet of Things interoperability Y.4460 2019 In force Architectural reference models of devices for Internet of things applications Y.4472 – Draft Open data application programming interface (APIs) for IoT data in smart cities and communities Y.4552 2016 In force Application support models of the Internet of Things Y.4555 2019 In force Service functionalities of self-quantification over Internet of things Y.4556 2019 In force Requirements and functional architecture of smart resi- dential community Y.4558 2019 In force Requirements and functional architecture of smart fire smoke detection service Y.4559 – Draft Requirements and functional architecture of base station inspection services using unmanned aerial vehicles Y.4702 2016 In force Common requirements and capabilities of device management in the Internet of things Y.4806 2017 In force Security capabilities supporting safety of the Internet of Things

Along with these recommendations, ITU-T SG20 has also published some supplements to the Y-series, such as Supplement 53 that complements the use cases covered by other recommendations in the series.

One of things that can be noticed from the overwhelming number of standards and recommendations published by ITU is that besides giving general references, they also have published some recommendations for specific scenarios. For instance, Y.4117 is about the requirements and capabilities for services related to wearable things, while Y.4119 is about automotive emergency response systems, Y.4120 is about the requirements for smart retail stores applications, and so on.

Another matter of relevance about ITU standards and recommendations about IoT is that they are mostly focused on the communications layer.

– 59 – State of the Art on Related Topics to Adaptive Mission Planning

2.5.3.2 ISO/IEC Standards

The scope of the ISO/IEC Joint Technical Committee (JTC) 1/SC 41 is the standardization in the area of IoT and related technologies [120]. In their ongoing work, they have produced, or are working on, various standards. Table 2.6 contains a selection of the most relevant ones and their current status at the time of writing this dissertation. An “in development” status means that the standard has been approved for development, but has not produced any draft yet.

Table 2.6. ISO/IEC Standards

Standard Year Status Title 29161:2016 2016 Published Information technology – Data structure – Unique identification for the Internet of Things TR 22417:2017 2017 Published Information technology – Internet of Things use cases 29181-9:2017 2017 Published Information technology – Future Net- work – Problem statement and requirements – Part 9: Networking of everything 20924:2018 2018 Published Information technology – Internet of Things – Vocabulary 30141:20181 2018 Published Internet of Things – Reference Archi- tecture 21823-1:2019 2019 Published Internet of Things – Interoperability for IoT systems – Part 1: Framework 21823-2:2020 2020 Published Internet of Things – Interoperability for IoT systems – Part 2: Transport interoperability 21823-3 2020 Draft Internet of Things – Interoperability for IoT systems – Part 3: Semantic interoperability 30147 – In development Information technology – Internet of things – Methodology for trustworthiness of IoT system/service 30149 – In development Internet of Things – Trustworthiness framework 30173 – Draft Digital twin – Concepts and terminology 30172 – Draft Digital twin – Use cases 30167 – In development Internet of Things – Underwater communication technologies for IoT

1This standard was not available at the time of developing the proposal presented in this dissertation.

– 60 – 2.5 State of the Art in Architectural Reference Models

As said before, the standardization work is a continuous process in which, in addition to developing new standards, those already published are reviewed and updated. That is the case, for instance, of standard ISO/IEC 20924, on the IoT vocabulary. The ISO/IEC JTC 1/SC 41/WG3 is currently working on a new draft, the latest version of which has been approved in 2020, and the final version of which is scheduled for May 2021. The same applies for ISO/IEC 30141, on the reference architecture for IoT, currently in an early draft for an update whose next draft version is expected by March 2021, and whose final version is planned for the end of 2023.

2.5.3.3 IEEE Standards

IEEE has also paid attention to standardization in the field of IoT. Table 2.7 contains information of IEEE standards, published or in development.

Table 2.7. IEEE Standards

Standard Year Status Title P24132 2020 Active Architectural framework for the IoT P2413.1 2018 In development Reference Architecture for Smart City (RASC) P2413.2 2019 In development Reference Architecture for Power Distribution IoT (PDIoT) P1451-99 2020 In development Harmonization of IoT devices and systems P1931.1 2016 In development Architectural Framework for Real-time Onsite Opera- tions Facilitation (ROOF) for the Internet of Things P2668 2018 In development Maturity Index of Internet-of-things: Evaluation, Grad- ing and Ranking

2.5.4 IoT-A

The only completed and published ARM for IoT at the time of designing the architectural proposal for this dissertation, and which is later presented in Chapter5, was the one published as the result of the IoT-A European project [121]. The full ARM was collected in [122], with its reference model published in [123] and its reference architecture published in [124].

2.5.4.1 IoT-A Reference Model

The IoT reference model from IoT-A is divided in five main models:

• IoT Domain Model, which describes the concepts that are relevant in the IoT.

2This standard was not available at the time of developing the proposal presented in this dissertation.

– 61 – State of the Art on Related Topics to Adaptive Mission Planning

• IoT Information Model, which defines the structure of all the information ona conceptual level.

• IoT Functional Model, which is an abstract framework for understanding the main functionality groups and their interactions.

• IoT Communication Model, which defines the main communication paradigms for connecting elements.

• IoT Trust, Security and Privacy Model, which describes the mechanisms that prevent data of a subject to be used improperly.

In the contribution for adaptive mission planning for cooperative robotics, the models that are most useful are fundamentally the domain, the information and the functional models. For sure, communications and security are of interest, but they fall outside the scope of this thesis.

2.5.4.1.1 IoT Domain Model

The IoT domain model from IoT-A has been described in [125], and it describes the concepts used in the ARM. There are five core concepts, that are described as follows:

• Augmented Entity is the combination of the real-world object (Physical Entity) and its digital representation (Virtual Entity).

• User is who uses an IoT system, and it can either be human beings, machines, devices, services, etc.

• Device is the hardware to monitor, or to interact with, real world objects. They can also provide connectivity to other systems.

• Resource is the computational element that gives access to information about, or actuation capabilities on, a real-world object.

• Service in the same sense as the definition of a service given in the SOA reference model, a service exposes the Resources through a common interface, making them available for applications and for other services.

The relationship between these concepts and other related ones also defined in the IoT domain model is shown in Figure 2.33.

The concept of augmented entity is the way to enable the convergence of the real world and the digital one, by providing a virtual counterpart for each physical entity participating

– 62 – 2.5 State of the Art in Architectural Reference Models

Figure 2.33. IoT-A Domain Model in an IoT system. In the domain model this is represented by the concept of virtual entity, which in turn can be of two types: active or passive digital artefact. Active digital artefacts are running software applications, while passive digital artefacts are passive software elements, like database entries.

The concept of user is applied to those entities that can interact with an IoT system, and therefore it includes the concepts of human user and the aforementioned active digital artifact.

The concept of device applies to the physical entities, and they can be either actuators, sensors, or tags.

The concept of resource refers to the software components that implements functionalities. As such, it can be hosted in a device, or be accessible on the network.

Finally, the concept of service, matching the one from the SOA reference model, is the way users can access resources.

– 63 – State of the Art on Related Topics to Adaptive Mission Planning

2.5.4.1.2 IoT Information Model

The IoT information model defines the structure of all the information on a conceptual level. In the case of the IoT ARM from IoT-A, the information model focuses on the Virtual Entity element (not to be confused with the concept with the same name from the domain model). The other two main elements from the information model are the Service Description and the Association. The diagram in Figure 2.34 illustrates the relations between these information elements. As shown, the Association is used to model the connection between the Virtual Entity and the Service Description.

Figure 2.34. IoT-A information model

The Service Description element contains all the relevant information of a service, like its interface description, in the same way as it does in the SOA reference model.

2.5.4.1.3 IoT Functional Model

As mentioned before, the IoT functional model is an abstract framework for understanding the main functionality groups and their interactions. In the case of the IoT ARM it contains five longitudinal functional groups and two transversal ones, as shown in Figure 2.35.

The description and interactions of each functional group are defined below:

• IoT Process Management relates the conceptual integration of process management systems with the IoT ARM.

– 64 – 2.5 State of the Art in Architectural Reference Models

Figure 2.35. IoT-A functional model

• Service Organization is central to the model, and acts as a communication hub between several other functionality groups.

• Virtual Entity contains the functions for interacting with the IoT system on the basis of the concept of virtual entity from the IoT domain model.

• IoT Service contains the services and functionalities for discovery, look-up and name resolution of such services.

• Communication abstracts the variety of interaction schemes derived from the many technologies belonging to IoT systems.

• Management combines all functionalities that are needed to govern an IoT system.

• Security provides the mechanisms for ensuring the security and privacy of IoT systems.

2.5.4.2 IoT Reference Architecture

The IoT reference architecture defines the usage views and perspectives in the IoT ARM. There are three main views in it [124]:

• IoT Functional View. It describes the functional components for each of the functional groups in the IoT functional model.

• IoT Information View. It describes how the information is to be presented and handled in an IoT system.

– 65 – State of the Art on Related Topics to Adaptive Mission Planning

• IoT Deployment and Operation View. It provides a set of guidelines for users of the IoT reference model interested in the design of actual implementations of IoT systems.

2.5.5 Conclusions

This section has started by collecting the concepts from the SOA reference model, as service orientation is one of the aspects to take into consideration for IoT systems. Then, it has provided a relation of current standards on IoT from reference bodies such as ITU-T, ISO and IEEE. Finally, it has described the architectural reference model for IoT available at the time of defining the proposal for this dissertation.

2.6 Chapter Conclusions

This chapter has undertaken a diverse and extensive exploration of the state-of-the-art on topics that are of relevance for contributing to adaptive mission planning for cooperative robotics in the IoT.

Section 2.1 introduced the concept of mission, presenting the problems of its planning, adaptation and management in the field of cooperation between multiple agents.

Section 2.2 explored the paradigms onAI planning, as they are closely related to mission planning. In summary, there are several ways to approach a planning problem. It can be done by using a deterministic approach by delegating the responsibilities to adapt to non-deterministic scenarios to the agents in charge of executing a classical plan. It can be done using a non-deterministic approach by formulating a policy for a MDP or a POMDP, although it is a very complicated task. It can be eased by using a decision tree for known possible branches of execution. It can be decomposed in a hierarchy according to the capabilities of the agents. And if this were not enough, it can also include an opportunistic layer of actions.

Section 2.2 described the basics ofAI planning description languages, with PDDL being the most prominent, and the base for multiple extensions to include support for multiple agents, probabilistic planning and hierarchical planning among others. However, PDDL and the rest of planning description languages are designed for the description of the domain of a problem and the problem itself, as these languages were mostly designed to provide a common input for testing planning algorithms. And this translates into two major drawbacks:

1. They are not designed to describe the resulting plans from solving the planning problem.

– 66 – 2.6 Chapter Conclusions

2. In relation to the description of the domain, that is, the model of the world, they are not related in any way to other descriptions of models of the world used nowadays in complex systems, like semantic descriptions using ontologies, such as the Core Ontology for Robotics and Automation (CORA)[126], or any other robotics related ontology.

Section 2.4 formally defined adaptation, described some adaptive models commonly used in automated systems, such as OODA, ECA, SPA, and specially MAPE-K, and studied some already defined decentralized control patterns that can be of interest for distributed adaptation. These concepts are of interest for adapting the mission plan for scenarios with both one and multiple agents.

Finally, Section 2.5 described some architectural reference models of interest for IoT systems, like the SOA reference model, and the IoT-A architectural reference model. These models provide some guidance for the design of specific architectures, like the ones that could be developed for cooperative robotics in the IoT.

– 67 –

Chapter 3

Survey on Mission Management Architectures for Unmanned Vehicles

Humans are allergic to change. They love to say, “We’ve always done it this way”. I try to fight that. — Rear Admiral Grace Murray Hopper

3.1 Introduction

The previous chapter has explored the concepts and details of relevance related to theAI planning paradigms,AI planning description languages, adaptation and adaptive models, and ARMs for IoT systems. One of the possible research opportunities opened after such study is mission management to improve mission adaptability.

This chapter provides a comprehensive survey of selected works in mission management architectures for cooperative autonomous vehicles, paying special attention to the ones developed for AUVs and underwater operations, in consideration to the SWARMs European project in which part of the work of this thesis has been carried out.

It is important to remark that this chapter has already been published in [127] as part of the dissemination efforts for this dissertation. The contents presented here have been adapted for its integration with the state-of-the-art, and extended with some final conclusions.

– 69 – Survey on Mission Management Architectures for Unmanned Vehicles

The rest of this chapter is organized as follows. Section 3.2 describes the features against which the selected architectures will be evaluated. Section 3.3 performs the assessment of the selected architectures against the features presented in the previous section. Section 3.4 summarizes the observations from the survey, and provides some related conclusions. Finally, Section 3.5 provides some general conclusions highlighting the identified topics of interest for further exploration that are covered in this disseration.

3.2 Description of the selected features for a mission manager

This section discusses in detail the features related to the topics that have been presented earlier in this chapter. In particular, there are four different features that have been defined as of interest for the design of a mission management architecture. Toensure the assessment is as objective and accurate as possible, a set of categories have been defined for each of the selected features.

It is important to recall that these categories provide only a classification for the studied architectures, but they do not establish a ranking system.

3.2.1 Mission plan specification

As stated in Section 2.1, the mission is the essential information structure for cooperative robotics operations. Its main constitutive part, the mission plan, matches the formal definition of a plan as a sequence of actions to achieve agoal[65]. In consequence it is common to use anAI automated planning approach to generate the mission plan.

Whether the mission plan follows a deterministic or a non-deterministic paradigm, with or without temporal constraints, and either considering one or multiple agents, from an architectural point of view it is important to know how the mission plan is specified. This survey takes into account the selection of planning paradigms from Thompson and Guihen [128], with the additional consideration of the use of a hierarchical task description, for instance, a HTN[129], as already described in Section 2.2.

The mission plan specification feature is categorized regarding the planning paradigm identified in the surveyed architecture, with special attention to the way the mission plan is described. Already existing formal description languages, as described in Section 2.3, are commonly used for automated planning. They also have been considered for the determination of the categories. In particular: the STRIPS[72]; the PDDL[76]; the RDDL [87]; or any ad-hoc specification created for the architecture proposal. For PDDL, any of its versions and extensions have also been taken into consideration. These formal description languages have already been described in 2.3.

– 70 – 3.2 Description of the selected features for a mission manager

The categories that have been identified for this feature are summarized in Tables 3.1 and 3.2. It has been decided to split the assessment into two different categorical groups, as the categories from the second group can be present or absent indistinctly in any of the categories of the first group. The first group focuses on the use of deterministic ornon- deterministic planning paradigms. A graphical scheme for the proposed assignment of categories for the first group is shown in Figure 3.1, and the description of the suggested assessment is summarized in Table 3.1.

Figure 3.1. Mission plan specification assessment diagramst (1 group)

Table 3.1. Mission plan specification assessment categoriesst (1 group)

Category Description 1 The architecture proposal does not describe the way the mission plan is specified. 2 The architecture proposal follows a deterministic approach and uses an ad-hoc mission plan specification, or proposes a new one. 3 The architecture proposal follows a deterministic approach and uses either STRIPS, any of the PDDL versions, or any other formal description language already defined for automated planning. 4 The architecture proposal follows a non-deterministic approach and uses an ad-hoc mission plan specification or proposes a new one. 5 The architecture proposal follows a non-deterministic approach and uses PPDDL, RDDL or any other formal description language already defined for automated planning.

The second assessment group focuses on the use of other planning paradigms that overlap the assessment made in the first group. This assessment includes the aforementioned multi-agent, temporal constraint and hierarchical tasks paradigm that can be

– 71 – Survey on Mission Management Architectures for Unmanned Vehicles used both for deterministic and non-deterministic planning. In this case the category is identified by a letter that is added to the first group category according to the assessment made. For instance, an architecture that uses deterministic planning with a formal specification and a multi-agent approach with temporal constraints will be categorized as “3MT”. The description of the assessment categories for this second group is summarized in Table 3.2.

Table 3.2. Mission plan specification assessment categoriesnd (2 group)

Category Description M The architecture proposal includes a multi-agent approach. T The architecture proposal includes a temporal constraint approach. H The architecture proposal includes a hierarchical task network.

3.2.2 Use of Self-Adaptive Models

The underwater environment is by definition uncertain and unreliable. This implies that the vehicles, as agents entrusted with the task of executing the mission plan, cannot ensure that they will be able to achieve the mission goals [49]. To improve their reliability, it is common to design mission management architectures considering self-adaptation to certain unforeseen events. The definition of adaptation, self-adaptation along with the description self-adaptive models has already been covered in Section 2.4.2.

The categorization defined for the use of self-adaptation in the explored architectures is illustrated in Figure 3.2. The first category is used for architectures that do not provide a description of the use of any self-adaptation model, and none has been identified from the proposal. The second category is for architectures that only consider the use of self-adaptation models at the vehicle mission management level, while the third category is for architectures that only consider the use of self-adaptation at the global mission management level. The fourth and fifth categories are for architectures that use self-adaptation models at both levels of management, independently or using a decentralized control pattern respectively. Table 3.3 summarizes the assessment for the self-adaptation feature as it has been defined.

3.2.3 The Use of Heterogeneous Vehicles: Virtualization

Cooperative underwater operations using multiple AUVs require their mission management capabilities to be taken into account by mission management architectures. A simplistic approach could be that participation in cooperative missions be limited to AUVs with homogeneous capabilities tightly represented within the mission management architecture. However, the increasing interest in the use of AUVs for underwater operations

– 72 – 3.2 Description of the selected features for a mission manager

Figure 3.2. Diagram for the assignment of categories for the self-adaptation feature

Table 3.3. Self-adaptation assessment categories

Category Description 1 Self-adaptation is not mentioned and has not been identified in the proposed architecture. 2 Self-adaptation is only considered at the vehicle level. 3 Self-adaptation is only considered at the global mission management level. 4 Self-adaptation is considered both at the vehicle and the global mission management levels, but done independently. 5 Self-adaptation is considered both at the vehicle and the global mission management levels, in a cooperative way using a decentralized control pattern. implies the also increasing availability of multiple AUVs with different capabilities. Mis- sion management architectures that want to consider the possibility of using AUVs with heterogeneous capabilities may also rely on using a tightly representation of the AUVs mission management capabilities by restricting the heterogeneity. Another approach is to use a well-defined soft-representation of a common mission management capability defined for the architecture, enabling the possibility of using legacy andnew AUVs in cooperative missions.

A virtual representation of a physical entity can be described as an abstraction mechanism that masks the specifics of such a physical entity, providing a common interface to access its resources. The use of virtualization for abstracting the capabilities of heterogeneous entities has already been used in other domains. For instance, a model for the virtualization of event sources inspired by the IoT-A domain model was described in [123, 130]. This IoT-A model, already discussed in Section 2.5.4, provides the access to the

– 73 – Survey on Mission Management Architectures for Unmanned Vehicles capabilities of a physical entity by the use of the services and resources exposed through its virtual representation as a virtual entity. In regards to virtualization, this domain model represents the most usual way for the virtualization of physical entities like AUVs.

From the global mission management point of view, the use of AUV virtualization may enable the chance to integrate both legacy and novel AUVs in a cooperative mission, dealing with their heterogeneity regarding their capabilities by using their virtual representations instead of a direct and tightly coupled one.

The schema for the assignment of the categories that we have defined for the virtualization feature is shown in Figure 3.2. The first category is for architectures that do not mention the use of virtualization or abstraction methods in regards to mission management. The second category is for architectures that do not use virtualization or abstraction at all, relying on a tightly coupled integration of the vehicles within the mission. Architectures in this category can use both homogeneous and heterogeneous vehicles, requiring for the specific integration of the capabilities of each vehicle. The third category is for architectures that use virtualization just for homogeneous vehicles using a tightly coupled virtual representation. The fourth category is for architectures that use virtualization for heterogeneous vehicles with a tightly coupled virtual representation. And finally, the fifth category is for architectures that use virtualization for heterogeneous vehicles with a loosely coupled virtual representation. It is important to notice that architectures within category 2 are similar to architectures in categories 3 and 4, with the difference that architectures in category 2 will not use any virtual representation of the AUVs, while architectures in categories 3 and 4 will do, although being tightly coupled to the capabilities of the AUVs. Table 3.3 summarizes the assessment of the virtualization feature categories as the have been defined.

Figure 3.3. Diagram for the assignment of categories for the virtualization feature.

– 74 – 3.3 Description of the Selected Proposals

Table 3.4. Virtualization assessment categories

Category Description 1 The architecture does not mention vehicle virtualization or vehicle abstraction. 2 The architecture does not use vehicle virtualization, and either every vehicle is considered to have the same capabilities, or they have capabilities that are tightly coupled into the architecture. 3 The architecture uses vehicle virtualization, but every vehicle is expected to have the same capabilities. 4 The architecture uses vehicle virtualization for a set of heterogeneous vehicles with different capabilities, but using a tightly coupled virtual representation. 5 The architecture uses vehicle virtualization for a set of heterogeneous vehicles with different capabilities enabling the possibility of including new vehicles in a mission.

3.2.4 Mission plan Dispatching

The last feature considered is related to the way the mission plan is dispatched and executed to the AUVs from the global mission management point of view. The classical procedure implied to pre-load a pre-defined mission plan in the AUVs before launching them [131]. However, the improvement in underwater communications in recent years has the potential to consider other ways of dispatching the mission plan to the AUVs participating in a mission [132].

For this feature a set of five possible categories have been identified, following the diagram shown in Figure 3.4. The first category, as with the rest of the features considered before, is for architectures that do not give a description of how the mission plan is delivered. The second category is for architectures that use the pre-loading of the full mission plan into the AUVs. The third and fourth categories are for architectures that include the dispatching of the mission plan from the global mission management, either done task-by-task (for the third category), or by sending them the full plan (for the fourth category). The fifth and last category is for architectures that using anonline dispatching from the global mission management (categories three and four), also take into consideration the possibility of live updating the mission plan. Table 3.5 summarizes the mission plan dispatching assessment.

3.3 Description of the Selected Proposals

A list of well-selected mission management architectures is provided in this section, with brief introductions and comprehensive analyses based on the set of technical

– 75 – Survey on Mission Management Architectures for Unmanned Vehicles

Figure 3.4. Diagram for the assignment of categories for the dispatching feature

Table 3.5. Mission plan dispatching assessment categories

Category Description 1 The proposal does not mention how the mission plan or its tasks are dispatched to the vehicles. 2 The mission plan is fully loaded into the vehicles offline. 3 The mission plan is dispatched to the vehicles task-by-task, thus the global mission manager being responsible for active supervising of the task execution progress before dispatching the next task to a vehicle. 4 The mission plan is dispatched completely to the vehicles, thus the supervision of the execution of each task in the mission plan is done at the vehicle level, and without regards if the mission manager is also actively or passively monitoring the execution progress. 5 Same as categories three and four, but also including the possibility of updating the mission plan in real time from the global mission manager. features that was introduced in Section 3.2. This list presents a holistic view of the status of mission management architectures. Some of the systems included in the study were not initially conceived for being used with underwater environments. However, all these systems can be easily adapted to different environments when just focusing on mission management. For instance, the Teleo-Reactive Executive (T-REX) has been successfully implemented for underwater robotics, but also for service robotics [133]. Even a generic executive reactor for any robotic platform [134] has been developed for the T-REX architecture. The ROSPlan framework, developed for Robot Operating System (ROS) and designed for any robotics, was created as part of the PANDORA project for its use with AUVs.

– 76 – 3.3 Description of the Selected Proposals

3.3.1 T-REX: A Teleo-Reactive EXecutive Architecture

Teleo-Reactive Executive (T-REX)[135, 136] was developed by the Autonomy Group at Monterey Bay Aquarium Research Institute (MBARI), and has been validated in several real case scenarios. For instance, it has been successfully used in some of the experiments of the Controlled, Agile, and Novel Ocean Network (CANON) initiative at MBARI[137, 138], as well as other marine and service robotics experiments [133] and simulations [139]. It also had an executive implementation as a node part of ROS up until the C Turtle release [140].

The T-REX architecture was conceived as a goal-oriented system, where the typical mission plan, instead of being a sequence of detailed tasks, is specified by a sequence of high-level goals and constraints. The reasoning for this change is to deal with the increasing mission uncertainty in the underwater environment.

The essentials of T-REX are inspired by the SPA adaptive model, discussed in Sec- tion 2.4.5, using a number of concurrent control loops that provide the response to exogenous events and enable the distribution of the deliberation processes among the components of a T-REX agent. An example of a T-REX agent is described in [136], and illustrated in Figure 3.5. In this case the T-REX agent has four components, formerly named Teleo-Reactor, or just simply reactors: a Mission Manager, a Science Operator, a Navigator, and an Executive. Each of the reactors in the figure provide the goals for other reactors (full arrows), and monitors their current state through the provided observations (dotted arrows).

Figure 3.5. An example T-REX agent composed of four reactors

Each reactor in T-REX is capable of planning the specific tasks to achieve the goals provided, and to dispatch them to other reactors as their own goals. McGann et al.

– 77 – Survey on Mission Management Architectures for Unmanned Vehicles provided in [141] the example of an integrated system design with three main reactors with their internal main components that is shown in Figure 3.6. In this example, there are three reactors in a hierarchical distribution: a vehicle control subsystem (VCS) at bottom, a navigator at middle, and a mission manager at top. As the figure illustrates, each reactor receives goals from the reactor in its upper layer reactor, and sends their own generated goals to its lower layer reactor. At the same time each reactor receives the observations from its lower layer reactor, and generates their own observations to its upper layer reactor. Each agent has a local database for storing the received observations, and as shown in the figure, to store also plans, models and algorithms. The reactors in the example include a planner, a synchronizer, and a dispatcher. The planner uses automated planning techniques for adapting incomplete plans from the database to the current problem. The synchronizer is used to ensure plan consistency and completeness. And finally, the dispatcher is a simple algorithm used to publish goals to other reactors.

Figure 3.6. An example of the design of a T-REX integrated system

– 78 – 3.3 Description of the Selected Proposals

The system proposed as an example in Figure 3.6 can be used as follows: The Mission Manager reactor receives the goals to fulfill a mission, for instance a mission to survey an area. With that information and the data available in its local database, the Mission Manager generates a plan with the navigator steps that is dispatched as goals to the Navigator. In turn, the Navigator takes these goals, and using the data available in its local database generates a plan for the VCS consisting of the specific commands the vehicle has to execute. And as before, the plan generated by the Navigator is dispatched to the VCS as the goals to the VCS.

The dispatching algorithm is ruled by time windows. Each goal has an assigned starting time to be dispatched to the corresponding reactor. The time is calculated taking into account the latency of the reactor, its planning horizon and the execution frontier for the current task in execution, if any.

As for the features being assessed, in regards to the plan specification feature, the T-REX proposal does not describe how the plan is specified at any level. Therefore, it can only be assumed that it is done using an ad-hoc format. This corresponds to a category 2 for this feature. Besides, the use of multiple reactors allows for a multi-agent structure with a hierarchical mission plan, and the description of the dispatching algorithm expresses that each task has a temporal constraint. Consequently, this feature also has the “MHT” modifiers, resulting in a final assessment of a category 2MHT.

In regards to the adaptation feature, the literature about the T-REX architecture describes explicitly that it is based on the “sense-plan-act” self-adaptive model [141]. Specifically, the architecture uses a partitioned control where each reactor has its own internal self- adaptive loop working in a coordinated pattern with other reactors. In consequence the assignment for T-REX regarding the adaptation feature is of a category 5.

About the virtualization feature, the T-REX literature mentions that a T-REX agent uses a single model for control at various levels of abstraction [136]. However, no specifics are given about if there is any use of virtualization for the management capabilities of the AUVs participating in a mission, and as a result the corresponding category for this feature is category 1.

Finally, the dispatching of tasks in T-REX, as described in [136], uses time windows, with new tasks being dispatched when their time window starts. Thus, the category assigned for this feature is category 3.

A summary of the categorization for the measured features is included in Table 3.6.

– 79 – Survey on Mission Management Architectures for Unmanned Vehicles

Table 3.6. T-REX architecture assessment

Feature Category Description Plan Specification 2MHT The reviewed literature for T-REX does not provide any insights of how the plan is specified, thus it seems reasonable to assume that the plan specification is done using an ad-hoc format. Theuse of different reactors as agents capable of receiving goals and generating plans seems compatible with a multi-agent approach. It also means that the mission plan, seen from the top, can be considered as a hierarchical plan, since a plan from a reactor can be seen as the lower-level plan for the goals provided by another reactor. Adaptation 5 As T-REX is based in the SPA model, it uses self- adaptation by default. The partitioning of the control loops in the executive is done using a coordinated pattern. Virtualization 1 There is no mention to virtualization or abstraction. Dispatching 3 The dispatcher sends the goals to the corresponding reactors one by one. It is important to notice that in this case the description of the algorithm implies that there is no real supervision made from the upper reactor, and that the dispatching of a new goal is just ruled by its planned starting time.

3.3.2 RAUVI and its Mission Control System (MCS)

The RAUVI[142] was a research project aimed at the development of a new reconfigurable AUV capable of performing underwater operations using a robotic arm [143]. A distributed architecture was proposed [144] and validated through simulations and experiments in controlled scenarios [145]. The proposed architecture was designed to combine two pre-existing architectures with a Mission Control System (MCS), as shown in Figure 3.7. These two architectures are the manipulation and the vehicle architectures respectively.

The manipulation architecture is aimed to control the robotic arm in the AUV, and consists mainly of two modules: perception and action. On the other hand, the vehicle architecture is in charge of ensuring the functionality of the AUV, and includes three main modules: robot interface, perception and control. The third piece in the proposed distributed architecture is the aforementioned MCS, in charge of defining the task execution flow to fulfill the mission. A detailed functional diagram of the proposed architecture isshownin Figure 3.8. Four main functional blocks are shown in the diagram. At top sits the MCS,

– 80 – 3.3 Description of the Selected Proposals

Figure 3.7. An overview of the RAUVI software architecture with the “Architecture Abstraction Layer”. At bottom there is the manipulator architecture on the left, and the vehicle architecture on the right. Finally, in the middle there is a centralized blackboard used as a communication interface between the manipulator and the vehicle architecture.

Figure 3.8. RAUVI architecture

The mission plan in RAUVI must be provided by the mission programmer using a graphical user interface to generate a mission file expressed in Extensible Markup Language (XML), using a Mission Control Language (MCL)[146]. The generated file is then parsed by a MCL compiler that generates a Petri net mission file. The MCS receives the Petri net, and executes it through the abstraction layer, where every task has the exact same interface, and is dispatched to either the manipulator or the vehicle architecture.

– 81 – Survey on Mission Management Architectures for Unmanned Vehicles

The manipulator architecture receives the task through the action layer. This module can receive two types of actions: control and perceptual. The control actions are fed to the robot abstraction layer, where they are executed by the manipulator robot itself. On the other hand, the perceptual actions are fed to the perception layer, updating the robot internal perception. The perception layer also receives information from other parts of the manipulator robot, as internal perceptions, and from the vehicle architecture through the centralized blackboard, as external perceptions. Finally, if some perceptions meet some conditions at the condition layer, an event is sent to the MCS through the Abstraction Layer for further processing.

The vehicle architecture receives the tasks from the MCS in a control module that matches them to the primitives that are sent to the coordinator that manages their execution using the vehicle velocity controller. Besides the tasks received from the MCS, the vehicle architecture has a perception module that receives internal perceptions from the vehicle navigator and obstacle detector, and external perceptions from the manipulator architecture through the centralized blackboard. These perceptions can be used by the primitives in the control module.

Regarding the features being considered for this study, in RAUVI the mission file uses an ad-hoc specific format, the MCL. This format allows a deterministic approach, without multi-agent considerations, hierarchical structures or temporal constraints. Consequently, the corresponding category for the plan specification feature is a category 2.

With respect to the adaptation feature, there is no mention at all to the use of any self-adaptive model in the architecture, and no one has been identified. Therefore, the assigned category for the adaptation feature is category 1.

In regards to the virtualization feature, the RAUVI architecture specifically mentions the use of an abstraction layer [144] between the MCS and the vehicle/manipulator architectures. The purpose of this abstraction layer is to adapt the actions and the events to and from the corresponding instances of each architecture. This enables the virtualization of the capabilities of the different architectures being used, providing a common access to both the manipulator onboard a vehicle, and the vehicle itself. Therefore, the assigned category in regards to the virtualization feature is category 5.

Finally, for the dispatching feature, in the RAUVI architecture the mission, once compiled in the MCL, is fully loaded into the vehicles before being launched. Therefore, the corresponding category for the dispatching feature is category 2.

A summary of the categorization for the measured features is included in Table 3.7.

– 82 – 3.3 Description of the Selected Proposals

Table 3.7. RAUVI architecture assessment

Feature Category Description Plan Specification 2 The RAUVI proposal includes the definition of a Mission Control Language to define the mission tasks of interest in a friendly high-level language. The mission plan follows a deterministic approach with no multi-agent capabilities, no temporal constraints, and no use of a hierarchical distribution of tasks. Adaptation 1 Self-adaptation is not mentioned in the description of the proposal. Virtualization 5 According to the description provided in [144] the architecture abstraction layer is capable of adapting the actions and events to the corresponding instances of the target architectures, allowing the MCS to be architecture independent. Dispatching 2 The mission is fully loaded into the vehicle before launch [144].

3.3.3 Intelligent Control Architecture (ICA-Trident)

The TRIDENT project [147] explored and proposed a new methodology to provide multi- purpose dexterous manipulation capabilities for intervention operations in unknown, unstructured and underwater environments. As part of this project the ICA was proposed as a novel cognitive control architecture for AUVs[148, 149], enabling the use of multiple vehicles in underwater intervention missions. The ICA was validated in the experiments carried out in the TRIDENT project, including simulations and real experiments in the sea [150].

An overview of ICA is shown in Figure 3.9. The components are distributed as in the usual three-layer architecture used in robotics [46]. At the deliberation layer the system uses a hierarchical mission plan specified by the main goal and its decomposition in subsequent sub-goals. Each goal is then planned, producing an agent plan that is a sequence of activities, or command messages. Once an agent plan is produced, its activities are matched to the available services that can be discovered by other agents, and can be either basic or composed services following a Service Oriented Architecture approach [116].

The ontology of the ICA, as shown in Figure 3.10, shows the relations among its core elements. The system, identified as an Autonomous Maritime Robot (AMR) fulfills a mission that has one or more plans and one or more goals, and that is carried out by

– 83 – Survey on Mission Management Architectures for Unmanned Vehicles

Figure 3.9. Architectural concepts of the ICA an agent. Each goal is achieved by a plan that involves one or more capabilities. The agents have plans that achieve goals and carry out them by means of activities that are carried out by services.

Figure 3.10. ICA ontology

– 84 – 3.3 Description of the Selected Proposals

The internal structure of an ICA agent is shown in Figure 3.11. From the robotics model point of view, it is composed of five main blocks: the user interface, the deliberation unit, the execution unit, the behavior unit, and the world model.

Figure 3.11. Internal structure of an ICA agent

The user interface is the block responsible to provide the end user with an interface to manage the mission and check its progress.

The deliberation unit represents the deliberation layer of a three-layer robotics architecture, and has three components related to the generation of a mission plan: the mission communicator, used to communicate with the human operator through the social model; the mission planner, used to generate the mission plan; and the mission reasoner, used to decide what to do next according to the data perceived from the sensors.

The execution unit represents the execution layer of a three-layer robotics architecture, and is responsible for the execution of the mission plan through the mission spooler.

The behavior unit represents the behavior layer of a three-layer robotics architecture, and is responsible for executing the commands dispatched by the mission spooler according to the services available in the agent.

Finally, the world model is a repository composed of three models: the social model, that describes the context for the agent; the mental model, that describes what the agent knows about itself; and the geospatial model, that contains the environmental data gathered from the sensors.

ICA agents also use a situation awareness [151] approach that uses a decision making cycle defined as an OODA loop. In this OODA loop the observation and orientation

– 85 – Survey on Mission Management Architectures for Unmanned Vehicles stages are matched by the belief blocks at the world model, provided by the social and geospatial models, and the decision and action stages are matched by the intention block, that is, the mission spooler module.

Regarding the assessment of the features, with respect to the plan specification feature, the ICA proposal explicitly states that it “moves away from fixed mission plans”. And it does so by including a mission reasoner able to adapt a mission plan based on the current situation of the mission. However, a mission plan is still generated by the mission planner from a hierarchical set of goals, although no description is given in the ICA literature about how this mission plan is specified. Additionally, ICA provides the mechanisms for using multiple agents from the operator point of view, assigning specific agent plans to each agent involved in a mission. As a result, ICA belongs to a category 2MH for the plan specification feature.

In regards to the adaptation feature, the ICA proposal explicitly mentions the use of an OODA loop for the decision making cycle embedded in each agent. Therefore, a category 2 is assigned to ICA for the adaptation feature.

With respect to the virtualization, or any other abstraction mechanism, the ICA literature does not mention any approach, and no one can be identified from the proposal, resulting in a category 1 assignment for this category.

Finally, although the ICA specifically mentions the use of a mission spooler component that dispatches the mission plan to the service matchmaker component, there is no description on how the dispatching is done. Consequently, the assessment for the dispatching feature is a category 1.

A summary of the categorization for the measured features is included in Table 3.8.

3.3.4 Continuous Planning and Execution Framework (CPEF)

The CPEF[152, 153] is a framework designed for plan generation, execution, monitoring and repairing aimed to solve complex tasks in unpredictable and dynamic environments. It was developed as part of the Defense Advanced Research Projects Agency (DARPA) Joint Forces Air Component Commander (JFACC) program, and validated through several demonstrations [154–157]. Although it was conceived for its use with UAVs, the proposed design for the framework could be of use for any other application domain.

Figure 3.12 shows the functional overview of the six components that make up CPEF. The Plan Manager is the core component, responsible for the control of the system. It is in charge of controlling the generation of plans, supervising their execution, providing knowledge for both plans and plan executions, responding to unexpected events and controlling the adaptation of the plan in the event of plan failures that require plan updates.

– 86 – 3.3 Description of the Selected Proposals

Table 3.8. ICA assessment

Feature Category Description Plan Specification 2MH From the architectural overview it is clear that the mission specification is provided as a hierarchical set of goals used to generate agent plans, that are in essence a sequence of commands without following or proposing a formal description, and thus using an ad-hoc one. AS the agent plans can be assigned to different agents, it can be considered that ICA provides a multi-agent view. Adaptation 2 ICA includes an OODA loop approach for the situation awareness performed by an AUV agent. Virtualization 1 There is no mention to virtualization or abstraction of the agents in the ICA proposal. Dispatching 1 Although the internal structure of an ICA agent includes a component named “Mission Spooler” that feeds the “Service Matchmaker” with agent plans, there is no mention to how these agent plans are really dispatched to the AUV agents.

The supervision of the execution of the plans from the Plan Manager uses a flow model in which the plan manager waits for the outcome of individual actions.

Figure 3.12. CPEF overview

Besides the tracking of the execution through the flow control model used in the Plan Manager, CPEF resorts to the use of monitors, defined as event-response rules. These monitors can trigger the execution of a predefined response whenever they detect a specific event. There are three main categories of monitors defined in CPEF:

– 87 – Survey on Mission Management Architectures for Unmanned Vehicles

• Failure monitors. They provide the responses to the failures that can happen while a plan is being executed.

• Knowledge monitors. They provide the responses for decision-making regarding the changes in the world-state.

• Assumption monitors. The provide the responses for changes in the situation of the plan execution that violate the assumptions on which the plan relies.

Regarding adaptation, CPEF, using the monitors already described, defines generalized failure models for assessing the detected failures, and deciding if they are relevant enough to require for the repairing of the plan in execution.

As for the features being considered, the literature about CPEF does not provide a description about the mission plan specification, and therefore the category for the plan specification feature is category 1.

In regards to the adaptation feature, the CPEF proposal explicitly mentions that the plan manager is able to adapt the plan to the events that trigger the plan repair mechanisms. This adaptation is made at the agents implementing the CPEF architecture, without specific mention to the use of a possible decentralized control pattern. Therefore, the category assigned to CPEF for the adaptation feature is category 3.

As with the plan specification feature, CPEF does not describe any virtualization or abstraction mechanism, subsequently being categorized as category 1 for the virtualization feature.

Finally, the reviewed material about CPEF does not include how the mission plan is dispatched to the agents. However, it is explicitly explained that the plan manager uses a flow model waiting for the status reports of each task in the mission planbefore dispatching the next one. Thus the assigned category for the dispatching feature is category 3.

A summary of the categorization for the measured features is included in Table 3.9.

3.3.5 TRAC

The Technologies for Reliable Autonomous Control (TRAC) of UAVs was developed as part of a DARPA effort under the Software Enabled Control (SEC) initiative [158]. It was created to enable greater mission capability, higher reliability and safety, and greater adaptability to vehicle and mission environment variability, and was in essence the same as the Reliable Autonomous Control Technologies (ReACT) for UAVs created by the same authors as a NASA effort for the revolutionary concepts (RevCon) initiative [158].

– 88 – 3.3 Description of the Selected Proposals

Table 3.9. CPEF assessment

Feature Category Description Plan Specification 1 The description of CPEF does not include a reference of how the plan has to be specified. Besides, the proposal of CPEF predates the specification of PDDL and its updates and extensions, as well as RDDL. Adaptation 3 The Plan Manager in CPEF is able to adapt the plan to the events if they trigger the need for repairing the plan. However there is no mention about the use of adaptation in the agents executing the plan. Virtualization 1 There is no mention in the proposal about the use of virtualization or abstraction of the agents executing the plan. Dispatching 3 Although there is no specific mention on how the plan is dispatched to the agents, the CPEF proposal mentions that the Plan Manager uses a flow model in which it waits for the status reports of each individual action in the plan.

TRAC/ReACT were developed and demonstrated through simulations [159] and real scenarios demonstrations [158].

As with CPEF, TRAC was not intended for being used in underwater missions. However, some of the concepts introduced by TRAC could be of interest when designing a mission management architecture for underwater operations. Figure 3.13 illustrates the functional architecture of TRAC.

The components of the TRAC architecture include:

• A beacon-based Exception Analysis for Maintenance (BEAM) that monitors the data bus and detects anomalies, and isolates and characterizes failures.

• A Spacecraft Health Inference Engine (SHINE), used in conjunction with BEAM for diagnosis and prognosis.

• A Closed Loop Execution and Recovery (CLEaR) that provides high-level mission plan management and generates the sequence of commands to the UAV to execute the mission plan.

• An Autonomous Command Executive (ACE) that supervises the execution of the mission plan created by the CLEaR component.

– 89 – Survey on Mission Management Architectures for Unmanned Vehicles

Figure 3.13. TRAC functional architecture

These four components are linked together through the Active State Cache, which is a data exchange repository for keeping and synchronizing all global information regarding the mission.

The CLEaR component, which later evolved as an independent framework [160], is just in charge of generating the mission plan and coordinating goal-driven and event- driven behavior. This is partially done by sharing plan information and continuous status updates both with the planner and the executive. The decision about if a plan conflict must be handled by the planner or by the executive is provided by some heuristics that depend on the mission and the domain.

The other relevant component from the mission management point of view is the ACE [159]. Figure 3.14 shows the functions performed by ACE, and how they are related to each other.

As illustrated, ACE includes an Action Manager that receives the actions from the planner requests and the unplanned events to be handled. The action manager is responsible for the execution of the actions in the vehicles through the VMS, and the decision making about the unexpected events. If an event prevents the normal completion of an action, the action manager will notify the planner, so a new plan can be generated if required to fulfill the mission.

– 90 – 3.3 Description of the Selected Proposals

Figure 3.14. The ACE collaborative diagram

Figure 3.15 shows the components of the action manager within ACE. Each component is responsible for sampling the current state at the assigned step and calculating if a reaction is needed taking into account the operational limits.

Figure 3.15. ACE Action Manager components

The TRAC description does not mention how the mission plan is specified, and therefore the assigned category for the plan specification feature is category 1.

Regarding the adaptation feature, the TRAC proposal does also not mention the use of any self-adaptive model. However, the proposed architecture is continuously evaluating the current state of the system, triggering the re-planning of the mission plan if required,

– 91 – Survey on Mission Management Architectures for Unmanned Vehicles both at the mission control (global mission management), and at the vehicles, in a collaborative way. Hence the assigned category for the adaptation feature is category 5.

As with the plan specification, the TRAC proposal does not mention any kind of virtualization or abstraction. Therefore, the assigned category is category 1 for the virtualization feature.

Finally, although it is not explicitly explained in the proposal, the description of the ACE component of the TRAC architecture seems to use a task-by-task dispatching. Consequently, the assigned category for the dispatching category is category 3.

A summary of the categorization for the measured features is included in Table 3.10.

Table 3.10. TRAC assessment

Feature Category Description Plan Specification 1 The proposal does not imply the use of a predefined mission plan specification. Adaptation 5 Although there is no clear mention of the use of self-adaptation, the TRAC architecture evaluates the current state of the system continuously, allowing for the re-planning of the mission plan if required, both at the mission control and at the vehicles. Virtualization 1 There is no mention of the use of virtualization or abstraction of the vehicles management capabilities. Dispatching 3 From the ACE description it seems that the dispatching of the actions to the vehicles is performed task by task.

3.3.6 ROSPlan: Planning for the Robot Operating System

ROSPlan [161] is a framework that, according to their authors, provides a collection of tools for AI planning in a ROS system. Although ROSPlan is designed for any robotics, it was originally developed for the PANDORA project [162] for its use with AUVs, and was validated in the experiments that were carried out in said project [163].

ROSPlan uses a modular design where a number of ROS nodes provide the functionalities for problem generation, planning and plan execution. Initially presented in [164], a complete new version was presented in June 2018.

The components of ROSPlan are defined as ROS nodes, and the general overview of the system is shown in Figure 3.16. There are five ROS nodes defined [165]:

– 92 – 3.3 Description of the Selected Proposals

• The Knowledge base, used to store plan models using PDDL.

• The Problem Interface, used to generate the description of a problem using PDDL, and publish it on a ROS topic or write it to a file.

• The Planner Interface, used to invoke a planner and either publish the resulting plan to a ROS topic, or write it to a file.

• The Parsing Interface, used to translate a PDDL plan into ROS messages.

• The Plan Dispatch, which is responsible for the plan execution.

Figure 3.16. ROSPlan framework overview

In ROSPlan the process starts by the specification of a problem asa PDDL problem instance using the Problem Interface node and the Knowledge Base. When the problem description is available, it is published to a ROS topic. A Planner Interface subscribed to that topic will receive the problem description, and using an AI planner will try to find a solution. If the planner is successful, the solution is published toanother ROS topic. In this case it will be addressed to a subscribed Parsing Interface that will process the published solution to generate a plan representation that can be a Petri-Net plan, an ESTEREL plan, or a sequential plan. This plan representation is again published to another ROS topic addressed to a subscribed Plan Dispatch. The Plan Dispatch receives the plan as a whole, and dispatches each action individually. A diagram for the Plan Dispatch node is shown in Figure 3.17.

– 93 – Survey on Mission Management Architectures for Unmanned Vehicles

Figure 3.17. ROSPlan Plan Dispatch node

Regarding the assessment of categories for the features being considered, the ROSPlan proposal is quite clear in the use of PDDL for the specification of the mission plans. There have been some efforts on developing additional ROS nodes capable of handling probabilistic plans described with RDDL[166], but they are still not part of the ROSPlan framework. Hence the category assigned for the plan specification feature is category 3.

However, ROSPlan does not describe either an adaptation or a virtualization mechanism. Therefore, for both features corresponds category 1.

Finally, the dispatch of the tasks in the mission plan is done in a sequential way, and subsequently the assigned category for the dispatch feature is category 3.

A summary of the categorization for the measured features is included in Table 3.11.

Table 3.11. ROSPlan assessment

Feature Category Description Plan Specification 3 All the plans managed by ROSPlan are described using PDDL. Adaptation 1 There is no mention of any self-adaptation capabilities provided by ROSPlan, and none can be inferred from the proposal description. Virtualization 1 There is no mention to any kind of virtualization or abstraction of the vehicles. Dispatching 3 The actions are dispatched one by one in a sequential way. The Plan Dispatch node just returns an error if a failure occurs while executing a plan, or a success message if all the actions in the plan are successfully executed.

– 94 – 3.4 Summary

3.4 Summary

After presenting and introducing the selected architectures and frameworks individually in the previous section, it is of interest to make a comparison of them to have a clearer understanding of the status of mission management solutions. A summary of the presented architectures and frameworks is discussed in Section 3.4.1, and the challenges identified are discussed in Section 3.4.2.

3.4.1 Comparison of the Proposals

In order to consider the overall assessment that has been done with all the proposals included in Section 3.3, Table 3.12 shows the categories for each one according to the criteria that were described in Section 3.2.

Table 3.12. Comparison of the categories assigned to the reviewed systems

Architecture Plan specification Adaptation Virtualization Dispatching T-REX 2MHT 5 1 3 RAUVI 2 1 5 2 ICA 2MH 2 1 1 CPEF 1 3 1 3 TRAC 1 5 1 3 ROSPlan 3 1 1 3

When looking at Table 3.12 it is important to recall that a lower category does not necessarily mean a lower achievement at the specific feature. It is normal that different scenarios have different requirements, thus needing different capabilities on the selected features. However, it is quite remarkable that none of the architectures and frameworks evaluated have considered the use of probabilistic planning. Most of the solutions to deal with the unreliable nature of the underwater environment are related to reacting to unexpected events, either at the vehicle level or at the global mission control level. Continuing with planning, it is interesting that only two of the selected architectures, T-REX and ICA, consider the use of multi-agent planning and hierarchical planning, and only T-REX included temporal constraints in the mission plan.

The same can be said about the virtualization of the vehicles from the point of view of mission management. Only RAUVI defined some mechanism to virtualize or abstract the capabilities of vehicles. And this can be a potential issue to adapt the architecture to work with new vehicles with different capabilities.

Regarding the adaptation, there is greater diversity. TRAC describes an adaptation mechanism where the information is shared among all the agents in the architecture and

– 95 – Survey on Mission Management Architectures for Unmanned Vehicles the adaptation is performed using a decentralized control pattern. T-REX also mentions the use of a coordination control using self-adaptive loops at the reactor level. In CPEF the adaptation of the mission is only mentioned at the plan management level, while in ICA it is only mentioned at the vehicle level. Finally, RAUVI and ROSPlan do not mention adaptation at all, and the use of adaptation models cannot be inferred from their description.

Finally, regarding the dispatching of the mission plan to the vehicles, there is a great vari- ance for the architectures designed for specific use of underwater vehicles (T-REX,RAUVI and ICA). On the other hand, CPEF, TRAC and ROSPlan all use a task-by-task dispatch, possibly because they are not considering the constrained nature of underwater communications.

3.4.2 Open issues

Based on the analyses on the mission management architectures, a set of challenges have been identified and discussed as open issues in the following subsections.

3.4.2.1 High Dependence on Deterministic Plans for Non-Deterministic Environments

Probabilistic planning is hard. Instead of generating a sequence of actions, tasks, or even goals, a probabilistic plan has to provide a policy, that is, a function that given a state returns the next action with higher probability and reward to reach another state. However, an agent using a deterministic in an uncertain environment cannot guarantee to satisfy its goals [49]. As the underwater environment is not only uncertain, but also unreliable and prone to unpredictable events, the typical approach has been to use a deterministic plan to set the sequence of actions to achieve the desired goals, and also use a subsumption architecture [46–48] where the agents react to the unexpected events and try to adapt themselves to continue with the plan. The problem is when the plan cannot be fulfilled, and a new mission plan has to be created, either for repairing the current plan, or just to re-plan a whole new mission.

The use of other approaches for planning under uncertainty could provide an extra mechanism to improve the response to unexpected events that can be characterized with a stochastic model.

3.4.2.2 Lack of Common Specification for Mission Plans

Continuing with planning, most architectures use an ad-hoc mission plan specification, preventing its reuse in other projects. It is true that there are formal description languages from AI automated planning, like PDDL and RDDL, that could be used. But it is also

– 96 – 3.5 Conclusions true that those languages are hard to understand for non-experts in automated planning. Besides, an interesting thought about the specification of the mission plan, including the mission domain and mission problem, is that nowadays systems also use context awareness descriptions that should be synchronized to the domain description of the problem. That being said, it could be worth to consider not only the specification of a common description language for mission plans, but also a mediator component integrated with both the mission manager and the context manager that could adapt the descriptions in different formats to the needs of the components that will use them (like mission planners and reasoners), keeping the information synchronized for every participating agent.

3.4.2.3 Lack of Decentralized Self-Adaptive Control

Self-adaption should be considered both at the agents executing the mission plan and at the global mission manager, both independently and coordinately. In general, self-adaptation is only considered either at the vehicles, for adapting themselves while executing a mission to still be able to achieve the goal, or at the mission management, for re-planning when the outcomes of a task in a mission deviates enough to require a new plan. however, it could be interesting to explore the managing-managed approach from MAPE-K allowing the mission manager not only to react to events notified by the vehicles, but also to update the mission plan if there is new information from other sources that suggest that the update will improve the mission outcome.

3.4.2.4 Tightly Coupling of the Mission Management to the Specific Vehicles

In general, the reviewed architectures generate mission plans that are tightly coupled to the vehicles that are being used in their respective projects, or are too generic to delegate the responsibility of matching the vehicles capabilities to the specific implementation for each scenario. However, to improve the reuse of the proposed architectures, a virtualization approach should be considered. By using a virtual representation of the vehicles within the scope of the architecture, it is possible to integrate new and legacy vehicles by just implementing an adapter that on the one side matches the specifics of the vehicle, and on the other side matches the ones of the virtual model defined for the architecture.

3.5 Conclusions

This chapter has offered a survey of some selected mission management architectures for cooperative robotics used mostly in underwater operations, but also including ones

– 97 – Survey on Mission Management Architectures for Unmanned Vehicles for AUVs. This survey found that usually mission management is designed ad-hoc, with no guidance at all.

In conclusion, we have found three interesting topics to explore in this dissertation:

1. The formal definition of a plan is too restrictive for enabling adaptive mission planning and management. The formal definition of a policy is too broad forthe usual needs of adaptive mission planning and management. Thus, there is a need for at least some guidance to define a mission plan structure that allows decision nodes, hierarchical decomposition and opportunistic layers.

2. The planning description languages are mostly focused on the definition of the domain and the problem, with no relation to other real world model descriptions that can already be present in an IoT system. Consequently, there is a need for a common description language that can be used both by planning algorithms and IoT systems.

3. There is no directly applicable reference model for designing adaptive mission management architectures for cooperative robotics, especially in the field of the IoT.

– 98 – Part III

Research Contributions: Formalization and Design

Chapter 4

Mission Management Domain Model

I love when a plan comes together — Col. Hannibal Smith (The A-Team)

4.1 Introduction

The growing interest in the use of autonomous robots to perform different tasks in the real world is highlighting the problem that an agent cannot just follow a simple sequence of actions, or in other words, the structure of a mission plan must be more sophisticated [49]. In Chapter2 we have found that there is still a high dependence in using deterministic plans for non-deterministic environments, giving rise to the need to generate a new plan each time an event occurs that prevents continuing with the execution of the current plan.

In this chapter we present our proposal for a novel Mission Management Domain Model with the aim of defining the concepts for a new structure that is open enough to allow itto be used in different scenarios and with different requirements. Here we use the definition of a domain model as a description of concepts belonging to a particular area of interest [123, 167], including the relationship among those concepts. In this case, our proposal defines the concepts belonging to the area of mission planning and management and their relationships. This domain model has also been defined so that it can be usedas an abstract framework that serves as reference to understand such relations [116].

Finally, as the scope of this thesis also includes the use of autonomous robots in the framework of IoT, we have established the relation between our proposed model and the IoT Domain Model already presented in Section 2.5.4.1.1.

– 101 – Mission Management Domain Model

This chapter proposes our domain model for mission management for cooperative agents in the context of this dissertation. Section 4.2 draws attention to the problems in mission planning using the paradigms studied in Chapter2. Section 4.3 provides a brief formal description of our strategy proposal, and Section 4.4 presents the domain model for that concept as used in our research, with its related concepts applied to mission planning. The whole domain model for mission management with the integration of the strategy model is presented in Section 4.5. Section 4.6 describes the relation of the mission management and the IoT models. Finally, the main conclusions of this chapter are summarized in Section 4.7.

4.2 The Mission Plan Concept Problem

In this section we are going to describe a plausible scenario with just one robotic agent where we will see the problems that can arise using a classical planning approach, and the possible workarounds that can be used as part of a novel structure.

Let’s start with a simple scenario for a mission planning problem. In this scenario there are three rooms, a person, a robot, and an object. The rooms are separated by doors that can be either opened or closed. In the initial state, the person and the robot are both in the same room, Room A, and the object is in Room C. Figure 4.1 illustrates this scenario.

Figure 4.1. Scenario for the mission problem example. There is an object of interest in Room C represented as a green diamond. The robot (blue circle) and the human (yellow circle) are both in Room A

Using this scenario we are going to start from a very simple situation. In all the cases the goal is going to be the same: the person has to have the object from RoomC. Or in other words the robot has to pick up the object and give it to the person.

The initial version of the robot is able to execute the following commands, that can be translated to actions in a planning problem domain description:

– 102 – 4.2 The Mission Plan Concept Problem

• GOTO(x): Goes to location x.

• PICK(o): Picks up object o.

• DROP(o): Drops object o.

Thus, a very basic mission plan could be as the one illustrated as a graph in Figure 4.2.

Figure 4.2. Possible mission plan for solving the example mission problem

But, what if a door is unexpectedly closed? The plan execution would just fail. The typical approaches for whenever this happens are:

1. Limit the possibilities of non-deterministic events in the scenario. In this case, for example, preventing the doors from being closed.

2. Reset the scenario to its expected state, and execute again the mission plan. In this case, for example, opening the closed door and execute again the mission plan.

3. Update the problem domain description with the new conditions of the scenario, and generate a new mission plan.

4. Upgrade the robot to be able to open doors.

Both the first and second options are typical where the possibility of non-deterministic events can be limited to some extent, like in industrial settings. But that is not the case in more open scenarios, in which non-deterministic events cannot be controlled, at least not to a practical level.

The third option is the typical for an adaptive planning approach, and requires either an easy to update domain description, or the aid of an expert to create the new one.

And the last option is to upgrade the robot, for instance with a robotic arm capable of opening doors, thus adding a new OPEN(d) command to its catalogue of possible commands. Therefore, now it is possible to add this new command to a mission plan, and it is also possible to take into consideration a classical plan repairing or re-planning approach.

Indeed, let’s assume that we still use the same mission plan as in Figure 4.2, and again a door is closed (it does not matter if it is DoorAB or DoorBC). Then the controller executing the mission may request a plan repair or a re-plan. And since the robot now

– 103 – Mission Management Domain Model has the OPEN(d) command available, the plan can be repaired by simply inserting an OPEN(d) command at the break point.

Could this be improved? The answer is yes.

The robot could also be updated to detect if a door is opened or closed and react accordingly. Then, the original mission plan is still valid, as the adaptation to non- deterministic events is delegated to the robot following a subsumption approach [46, 48].

But in case the robot could not have this kind of update, there are still other possibilities. For instance, we could use a non-deterministic approach for solving the problem, and instead of a plan, generate a policy, considering the scenario as a POMDP problem. For simplicity, the mission plan for this new approach could be like a decision tree, like the one shown in Figure 4.3.

Figure 4.3. Possible decision tree for solving the example mission problem

One important thing to notice is that not every step is a decision node. For instance, in this example, if it is necessary to open a door, after doing it there is no decision node, but just the corresponding action to go to the next room. The generation of these kinds of trees can also be automated as a plan aggregation [168, 169].

But there is more. The human in the scenario can move freely. And remember, the goal was that “the person has to receive the object from RoomC”.

– 104 – 4.2 The Mission Plan Concept Problem

For this situation we can recall the opportunistic paradigm that was studied in Sec- tion 2.2.7. Thus, the mission could include an opportunity defined asif “ the robot has the object, and meets the person, drop the object”.

So far, the mission plan structure we have been working with has evolved from a formal plan as a sequence of actions, to a decision tree used as a policy for a POMDP problem including an opportunistic layer. And what is more, this opportunistic layer is defined here not as a condition for re-planning, but as an alternative branch that is enabled after picking the object. Figure 4.4 illustrates a possible diagram for this solution.

Figure 4.4. Possible decision tree with an opportunistic layer for solving the example mission problem

One last thing that we should also take into consideration is that it is not always possible to upgrade the robot to carry out actions autonomously, even if it has the ability to perform them as a succession of primitives. This question is related to the hierarchical decomposition of tasks, as studied in Section 2.2.6, and can be addressed in a similar way.

Let’s suppose that the robot from the example problem is equipped with a robotic arm, but does not know how to open a door. Let’s assume that it still has the ability to detect whether the door is closed or not, and follow one course of action or another accordingly. Let’s assume also that now the robot has a complementary set of commands to control the robotic arm. For instance, instead of the OPEN(d) command, it now can have the following commands in addition to GOTO(x), PICK(o) and DROP(o):

– 105 – Mission Management Domain Model

• MOVE-ARM(d,a): Moves the arm a distance d in the axis a.

• OPEN-GRIPPER(g): Opens the gripper g.

• CLOSE-GRIPPER(g): Closes the gripper g.

• ROTATE-GRIPPER(g,a): Rotates the gripper g a certain angle a clockwise.

Now, the old OPEN(d) command can be considered as a compound action, that is composed using the new commands as primitive actions for each door in the scenario. Figure 4.5 illustrates a possible graph for solving the example scenario with the integration of the hierarchical decomposition of the OPEN(d) action with the decision tree, and with an opportunistic layer.

Figure 4.5. A strategic solution for solving the example mission problem

In conclusion, this simple example illustrates how a classical planning approach can easily fail in an open domain (like the real world), and how it is required to develop a new concept that incorporates the solutions that other planning paradigms have already solved independently for each kind of problem. This new concept is essentially a set of guidelines, including the possibility of strict sequence of actions, decision making nodes, hierarchical decomposition and opportunistic plans. And consequently, we call it a strategy.

– 106 – 4.3 Strategy Formalization

The concept of strategy allows including the possible branches of a plan that may be known at the time of its generation. Having multiple branches facilitates the rapid adaptation of the mission in the event of situations that are included in the mission strategy. In addition, the use of an opportunistic layer allows identifying and activating situations of interest to the mission. And the consideration towards a possible hierarchical decomposition of actions when necessary enables the possibility of using robots with different capabilities in relation to the interpretation and execution of the mission strategy.

Some practical examples of using mission strategies are:

• In a marine robotics domain, like the one explored in the SWARMs[57, 58], we can imagine a scenario like the one described next. A set of AUVs and USVs are assigned to a mission to survey an area. Some of the AUVs may have the capability for deciding by themselves how to survey their assigned area, and some may not, requiring the use of a hierarchical decomposition. Additionally an AUV, while surveying its assigned area, can detect a plume, implying an opportunity to explore it. Finally, some foreseeable events can happen, like an AUV being unable to access its assigned area because its path is blocked by a school of salmon. An alternative route, or area assignment could be included in the strategy.

• In a precision farming domain, like the one explored in the AFarCloud[55, 56], we can imagine a scenario like the one described next. A set of UAVs and UGVs are assigned to a mission. In this case one UGV can be assigned to perform some treatment on a crop field, while some AUVs can be assigned to monitor and track the livestock. In this case several opportunities can be defined. For instance, the UGV can detect an unexpected obstacle in the crop field, meaning an opportunity to divert an AUV to its location to identify the obstacle. Of course, the UGV would adapt its assigned task to the current situation according to the options included in the strategy. Another opportunity can be identified if an AUV assigned to monitor and track the livestock detects a possible threat. In this case the AUV can follow a given procedure as part of the strategy to minimize the threat.

4.3 Strategy Formalization

As discussed in the previous section, there is a need for the definition of a new structure that combines the advantages of using classical plans, policies (expressed as decision functions), hierarchical decomposition of actions, and opportunistic planning. Previously we have called this structure a strategy, and it can be formally defined as the set of plans, policies and opportunities that can be used to achieve a set of goals. This formal definition can be expressed as in Equation 4.1.

– 107 – Mission Management Domain Model

σstrategy ≡ {S, Πplan, Π(s), O(s), G} (4.1)

In this equation:

• S is a finite or recursively enumerable set of states.

• Πplan is a finite or recursively enumerable set of classical plans as definedin Section 2.2.2.

• Π(s) is a finite or recursively enumerable set of policies as defined inSection 2.2.4.

• O(s) is a finite or recursively enumerable set of opportunities as defined inSec- tion 2.2.7.

• G is a finite or recursively enumerable set of goal states.

4.4 Strategy Domain Model

The Strategy Domain Model defines the concepts related to the proposed strategy structure. Figure 4.6 shows a representation of the model. The concepts that appear in this diagram and their relationships are explained below in their respective subsections.

Figure 4.6. Representation of the Strategy Domain Model

The descriptions of each concept are discussed in the following subsections. The key words must, must not, required, shall, shall not, should, should not, recommended, may and optional in this section are to be interpreted as described in [118].

– 108 – 4.4 Strategy Domain Model

4.4.1 Strategy

The term strategy has been defined in several ways, keeping always a similar concept, as a “deliberate conscious set of guidelines that determines decisions into the future” [170]. Other definitions include, for instance, the most referred in management theory, known as the Chandler definition, which states that “strategy is the determination of the basic long-term goals of an enterprise, and the adoption of courses of action and the allocation of resources necessary for carrying out these goals” [171]. Keeping the same spirit, we define the Strategy concept as the set of guidelines to achieve one or more Goals by using Plans and Decision Making Functions for adapting to the Plan in execution according to the observed (or partially observed) State, and taking into consideration possible Opportunities that can divert the Plan from its current course.

A Strategy must have at least one Goal and one Plan to achieve such Goal. It may also have some Opportunities and Decision Making Functions. In other words, Opportunities and Decision Making Functions are optional.

4.4.2 Plan

The concept of Plan is exactly the same as defined forAI planning, that is, a sequence of Actions to achieve a Goal [65]. A Plan must have at least one Action, and can be updated as a result of a Decision made by a Decision Making Function with the observation (or partial observation) of the current State, and using an Opportunity if applicable.

The sequence of Actions may be given by a specific order of execution, a temporal constraint, or any other method needed for each specific implementation of the model.

4.4.3 Opportunity

An Opportunity is a situation where an alternative Goal is identified as of interest and reachable from the current State.A Decision Making Function uses an Opportunity description together with the monitored State to make a Decision that might update the Plan, usually with a previously defined opportunistic Plan.

An Opportunity may include conditions that complement the observed State to determine the Decision to execute the opportunistic Plan. For instance, an Opportunity may require that some actions from the original Plan have been completed successfully before the Opportunity is considered feasible.

As defined, an Opportunity implies the activation of a previously defined opportunistic Plan. This Plan may substitute the current one, or be executed in parallel, with other agents available in the mission, as defined later in the Mission Management Do-

– 109 – Mission Management Domain Model main Model. Similarly, if the Opportunity changes the current Plan, the Strategy may contemplate recovering the previous one at the point it was diverted.

4.4.4 Decision Making Function

A Decision Making Function is any mechanism used to analyze the observed State, and together with the description of the possible Opportunities included in the Strategy, make Decisions that can update the Plan.

Decision Making Functions are used both for the representation of policies included in the Strategy, and for the identification of Opportunities that can be exploited.

The observation of the State used by the Decision Making Function may be partial, as in a POMDP problem.

4.4.5 Action

An Action is the fundamental part of a Plan, and represents what an agent is capable of parsing and executing autonomously.

An Action may lead to the achievement of a Goal.

Actions may also be composed of other Actions, following a hierarchical decomposition where the parent Action is to be considered as a compound Action, and a childless Action is considered as a primitive Action.

4.4.6 State

A State is a representation of the current condition of the world, restricted to the interest of a mission planning problem. It may be partial, and it is monitored by Decision Making Functions to be analyzed and make Decisions accordingly.

4.4.7 Goal

A Goal is a special case of a State, and corresponds to the description of a desired one. The Strategy must have at least one Goal. If there are more than one, there may be some relation between the list of Goals in the Strategy. For instance, one Goal may be required to be achieved before another. On the other hand, there may not be any relation between Goals, meaning that they can be achieved in any order, with no restrictions.

Goals are achieved by Actions, and may be considered as main Goals if they are achieved by Actions in the main Plan for the Strategy, or as opportunistic Goals if they are achieved by Actions in opportunistic Plans activated after an Opportunity has been identified in a Decision made by a Decision Making Function.

– 110 – 4.5 Mission Management Domain Model

4.4.8 Decision

A Decision is the result of a Decision Making Function, and that updates the Plan either because it takes an observed Opportunity, or just because the current State implies the selection of a different branch in the Strategy, that is, another sequence of Actions, or what is the same, another Plan.

4.5 Mission Management Domain Model

The new structure proposed in the previous section requires for a full new domain model that includes those concepts related to the execution of a mission expressed in the terms of the Strategy Domain Model. We call this full domain model a Mission Management Domain Model. Figure 4.7 shows the full Mission Management Domain Model with the Strategy Domain Model integrated. In essence, the corresponding part to mission management can be interpreted as the following sentence: A Mission uses one or more Strategies to achieve one or more Goals with the participation of one or more Agents, with the Mission, the Agents and the Strategies being managed by a Mission Manager.

Figure 4.7. Full Mission Management Domain Model

Our Mission Management Domain Model proposal is presented here together with our Strategy Domain Model proposal. However, the concepts that are specific in the mission management model, shown in red in the diagram from Figure 4.7, can be associated with other structures than the one proposed. For instance, instead of a Strategy, they

– 111 – Mission Management Domain Model could be applied to just a Plan from the classical planning paradigm, or a Policy from the non-deterministic planning paradigm.

4.5.1 Mission

A mission has already been defined as the set of tasks an agent must perform [45], or as a set of objectives to be achieved [44]. Given that the tasks to be carried out by an agent are directly related to the objectives to be achieved, a Mission is defined in the Mission Management Domain Model as a set of Goals to be achieved by some Agents that participate in the Mission.

A Mission must have at least one Goal, and have at least one Agent participating in the Mission.

4.5.2 Mission Description

The Mission Description represents the information needed to properly manage a Mission, together with the Agents participating in it, and the Strategies being used.

The Mission Description can contain any information element that is considered necessary for each specific implementation. However, it is recommended to include a unique identification for each Mission, especially when there is more than one to be managed. It is also recommended to include any relevant information about the Agents.

4.5.3 Agent

The agent concept is used here in the sense of the definition from the ITU-T Y.101 recommendation, that is, as “an element that performs some task on behalf of some party rather than having the party itself perform the task” [172]. In the scope of the Mission Management Domain Model, an Agent may be any entity capable of executing the Actions in the Plan from the Strategy being used in the Mission.

The Mission Management Domain Model does not enforce that any Agent participating in a Mission has an active role. This means that Agents may participate in Missions passively, awaiting for an Opportunity to be activated.

Every Agent participating in a Mission must have one Plan that is part of the Strategy followed by the Agent and used in the Mission. However, if the Agent has no active role in the Mission, its Plan should be an opportunistic Plan.

– 112 – 4.6 Relation of the Mission Management and the IoT Domains

4.5.4 Mission Manager

According to the American Management Association (AMA), management can be defined as “the act of getting things done through others and having them do it willingly” [173]. Thus, using this definition as a reference, a Mission Manager is defined as the responsible for getting the Mission accomplished. To do this, it uses the Mission Description in order to manage the Mission itself, the Agents participating in it, and the Strategies used in the Mission and followed by the Agents.

A Mission Manager must use at least one Mission Description, and should manage at least one Mission, one Agent and one Strategy.

4.6 Relation of the Mission Management and the IoT Domains

The scope of this dissertation includes the use of autonomous robots in the framework of IoT. Therefore, it is of interest to analyze how the proposed Mission Management and the IoT Domain models are interrelated. Figure 4.8 graphically shows the relationship between both models. This figure uses a simplified version ofthe IoT Domain Model for clearness. For the full IoT Domain Model diagram please refer to Figure 2.33.

There are three relations between the Mission Management and the IoT Domain models represented in Figure 4.8. The most important relationship is the one between the concept of Agent from the Mission Management Domain model and the set of concepts from the IoT Domain model consisting of the Physical Entity, the Virtual Entity that represents it, the Augmented Entity resulting from the composition of both, the Active Digital Artifact and the Digital Artifact. Considering an Agent as an autonomous robot that interacts with the physical world, it is evident that it is a Physical Entity. Consequently, an Agent can have a virtual representation as a Virtual Entity. Also, an Agent provides some services in the context of mission planning and execution. For instance, one Agent may provide the service of planning a compound Action by itself instead of requiring the primitive Actions. Finally, from the conceptual point of view, the virtual representation of the Agent as a Virtual Entity is indeed a Digital Artifact, and it is an active one by definition.

Another relationship shown in Figure 4.8 is the one between the Mission Manager and the Digital Artifact. Although the definition given for the Mission Manager in the Mission Management Domain Model is intentionally open, it usually refers to a piece of software that is in charge of the management of the Missions, Agents and Strategies. Therefore, it is fact a Digital Artifact, specifically an Active Digital Artifact. And consequently it has Services related to mission management.

– 113 – Mission Management Domain Model iue4.8. Figure Domains IoT the and Management Mission the of Relation

– 114 – 4.7 Conclusions

Finally, the last relationship shown in Figure 4.8 is the one from the User concept from the IoT Domain model and the Mission Manager. This is a direct relation, as the Mission Manager is defined to be used. The User here is considered in a broad sense, so it can be a human user, or an application, depending on how the Mission Manager is realized for each specific implementation.

4.7 Conclusions

In this chapter we have explored the problem of using a classical deterministic paradigm for mission planning for the intervention of robots in non-deterministic scenarios. Sec- tion 4.2 has provided a simple example of a mission that might fail in several ways by using a deterministic plan, how it is usually solved, and how it can be improved by using together some of the paradigms studied in the state-of-the-art in Chapter2. We have used this example to propose a novel structure that we have called strategy. To shape this proposal, Section 4.4 has described a new Strategy Domain Model where the related concepts have been defined, and Section 4.5 has described the integration of this Strategy Domain Model with a Mission Management Domain Model that we propose for its use in the design and implementation of Mission Management architectures. Finally, in Section 4.6 we have discussed the relations existing between the proposed Mission Management Domain Model and the IoT Domain Model, and how concepts handled in both are interrelated, thus showing the possibilities of defining mission management systems within the framework of IoT.

– 115 –

Chapter 5

Towards a Common Mission Management Middleware Architecture

Your mission, should you choose to accept it... — IMF Mission Commander (Mission: Impossible)

5.1 Introduction

As it has already been mentioned in this dissertation, the typical approach to working with autonomous robots is based on the definition of a mission plan that specifies the ordered sequence of actions that allow them to achieve the goals of said mission. The problem with this approach is that in open environments it is common to face situations that prevent the execution of the plan from being completed. The two usual solutions that are common to tackle this problem are to delegate to the robots the responsibility to adapt to certain solutions, and to repair the plan, or make a new one when the robot is not able to adapt itself to the situation. But, as already discussed in previous chapters, these solutions still have problems. On the one hand, a situation detected by a robot may also affect others in a mission, and just delegating the adaptation to the robot that detects the situation fails to adapt the plan for the rest. On the other hand, repairing the plan or making a new one is a process that takes time, being especially significant when communications are limited, as can happen in scenarios where the use of autonomous robots is more useful due to danger and distance from the scene.

In Chapter4 we have proposed a new structure that, starting from the classical structure of a deterministic plan, and taking ideas from other planning paradigms, provides infor-

– 117 – Towards a Common Mission Management Middleware Architecture mation that can improve the adaptive capability of the robotic agents participating in a mission.

In this chapter we address our proposal of a common Architectural Reference Model (ARM) for mission management that can both use a classic deterministic plan and the proposed new structure. We are focusing on the management of the mission considering management as the process consisting of planning, organizing, commanding, coordinating and controlling [174]. Hence, a mission management architecture should provide the means for those steps.

The proposal that is presented in this chapter includes the fundamental concepts and properties for a mission management system designed to mediate between any compliant mission management application and the robots participating in a mission. Therefore, as this follows the formal definition of a middleware architecture, it has been conceived as one.

The rest of this chapter is organized as follows. Section 5.2 describes the approach of the architectural proposal, including a brief commentary on the balance of adaptation problem. Section 5.3 details the requirements for the architecture. Section 5.4 describes the functional model of the architecture, while the computational analysis is detailed in Section 5.5. Finally, Section 5.6 summarizes the conclusions from this chapter.

5.2 Overview of the CoMMMA Approach

The Common Mission Management Middleware Architecture (CoMMMA) ARM proposal represents the specification of a set of guidelines for the design of mission management architectures suitable for providing advanced adaptive mission planning and execution capabilities for cooperative robotics. With that intention, CoMMMA is based on the use of the proposed strategy structure described in the previous chapter, including the special case when the strategy is just a classical deterministic plan.

The components related to mission management that are defined in CoMMMA are designed following the MAPE-K adaptive model. This also enables the possible use of decentralized control adaptive patterns. Using this approach alone allows CoMMMA- based systems to provide cooperative adaptation mechanisms for multi-robotic agent missions. Furthermore, using CoMMMA together with the strategy model proposed in Chapter4, makes possible to improve the balance of adaptation defined in Section 5.2.1.

Additionally, CoMMMA uses a virtual agent representation inspired by the IoT-A reference model, with the intention of framing it in the IoT domain. This virtual representation is aimed also to effectively decouple the mission management from the specifics of the agents participating in a mission.

– 118 – 5.2 Overview of the CoMMMA Approach

5.2.1 Balance of Adaptation

Adaptation can be performed at different points in an adaptive mission. Using a decentralized control pattern, like the ones explored in Section 2.4.7, the adaptation can be done in the agents executing the mission, in a global mission manager (mission control), or in both. The simplest scenario is when adaptation is fully delegated in the agents. However, that prevents the coordinated adaptation that may be required when multiple agents are working together.

In the case of a coordinated adaptation, it may be necessary to generate a new plan or repair the existing one, either for a single agent, for several, or for all. This process is a deliberative one, and it is time consuming. With that idea in mind we can express the balance of adaptation for a cooperative mission as in Equation 5.1.

tadaptation = tcommunication + tdeliberation (5.1)

In Equation 5.1 tcommunication is the total time required for the communication, both for the information generated at the agent that triggers the need for adaptation, and for the new plan, or updates to the plan, sent from the mission control to the agents involved. And

tdeliberation is the total time required to generate an updated or a new plan, as required for the adaptation needs.

With the aim of improving adaptation times, taking into account the balance of adaptation, it is only possible to act either in tcommunication or in tdeliberation.

Acting on tcommunication is beyond the scope of this dissertation, since the technologies used for the communication are outside the scope of an architectural reference model proposal. Anyway, it is important to know that tcommunication can be expressed also as in Equation 5.2. In this equation, ttransmissionevent refers to the time to transmit the event message that an agent sends to the mission control, tpropagationevent refers to the time required for the message to reach the mission control once transmitted from the agent,

ttransmissionplan_update is the time required to transmit the plan update from the mission control to the agent, and tpropagationplan_update is the time for the message with the new plan to reach the agent once transmitted from the mission control.

tcommunication = ttransmission + tpropagation event event (5.2)

+ ttransmissionplan_update + tpropagationplan_update

– 119 – Towards a Common Mission Management Middleware Architecture

Therefore, to improve tcommunication means to improve either ttransmission, tpropagation or both. In the case of tpropagation, it depends on the physical medium used for the communication. And in the case of ttransmission, considering the transmission of the full message, it depends on both the way the transmission is performed (technology, modulation, codifi- cation), and the length of the message to be transmitted. As can be seen, in any of the cases these actions are outside an architectural proposal.

On the other hand, tdeliberation can be improved if the deliberation is done early. Here we can use the advantages of the strategy structure proposed in Section 4.4. If the mission strategy covers the reported event with some alternative branch, the deliberation process is then limited to selecting the appropriate branch and notifying the agent. This can be expressed as in Equation 5.3.

< ⇒ < (5.3) tdecision tdeliberation tadaptationdecision tadaptationdeliberation

In conclusion, an architectural proposal for mission management can improve the time to adaptation of a mission plan if it uses the strategy proposal from Section 4.4, and has some functional components that take into account the balance of adaptation.

5.3 Requirements

This section includes the description of the requirements found for a mission management architecture with adaptive capabilities, and is divided into the functional and non-functional requirements.

5.3.1 Functional requirements

The functional requirements are related to the capabilities expected for an adaptive mission management architecture for cooperative robotics. Hence, the list of the functional requirements found is as follows:

1. Execute Mission: a CoMMMA-based architecture must be able to execute a mission, once it is provided with the mission strategy. The mission strategy can be as simple as a single classic plan, or as complex as a plan with decision nodes and opportunistic plans.

2. Historical Plan Retrieval: although not directly related to mission management, the capability of retrieving historical plans is required to provide the means for analyzing past missions and their plans, and even use them for the generation of new plans or strategies.

– 120 – 5.3 Requirements

3. Historical Strategy Retrieval: as same as with the historical plan retrieval, the historical strategy retrieval is needed to provide a way to analyze previous missions, and to help in the generation of new strategies.

4. Strategy Generation: the strategy for a mission can be generated at the application level, with a dedicated application, or at the architectural level, using the services from a specific component.

5. Strategy Update: a strategy can be updated whether it is being executed or if it corresponds to a historical record in the corresponding repository. Updating a strategy while it is being executed has a special treatment, as the updates have to be taken into consideration by the mission execution, and be notified to the robots participating in the mission accordingly.

6. Mission Adaptation: the execution of a mission may require to be adapted when a new situation is detected after the reception of an event notification. The adaptation may be arranged by the robot itself or by the mission manager in the architecture depending on the robot capabilities. It also may affect other robots participating in the mission.

7. Opportunity Execution: the detection of an opportunity by the analysis of the environmental notifications may trigger the chance for executing an alternative plan for such opportunity, that is, an opportunistic plan.

8. Mission Status Notification: the robots participating in a mission must be ableto notify their progress of the mission execution.

9. Event Notification: the robots participating in a mission must be able to notify the events they detect, enabling the possibility of a decentralized adaptation pattern, or in other words, that other robots in the mission can adapt their plans if necessary due to the notified event.

10. Environmental Notification: the robots, and maybe other devices in the system, like sensors or other sources of information, must be able to provide environmental information. This information is used among other things to evaluate if an opportunity is present.

This list of functional requirements is summarized in Table 5.1.

5.3.2 Non-Functional requirements

The non-functional requirements identify other qualities of a mission management architecture not related to its capabilities, but to other constraints. In the case of CoMMMA the list of non-functional requirements identified is as follows:

– 121 – Towards a Common Mission Management Middleware Architecture

Table 5.1. Functional Requirements for CoMMMA

Identifier Name Description CoMMMA-FUN-01 Execute Executing a mission given a classic determin- Mission istic plan or a novel strategy, considering the former as a special case of the latter. CoMMMA-FUN-02 Historical Plan Recovering information regarding plans Retrieval stored in a historical repository, including the plans for missions previously executed. CoMMMA-FUN-03 Historical Recovering information regarding strategies Strategy stored in a historical repository, including the Retrieval strategies from missions previously executed. CoMMMA-FUN-04 Strategy Generation of new strategies using informa- Generation tion provided by an operator and recovered from the historical repositories. CoMMMA-FUN-05 Strategy Updating a strategy manually by an operator. Update CoMMMA-FUN-06 Mission Adapting the mission execution to new situa- Adaptation tions using the mission strategy. CoMMMA-FUN-07 Opportunity Take advantage of an opportunity described Execution in the mission strategy, and detected on the environmental conditions of the mission. CoMMMA-FUN-08 Mission Status Being able to keep informed about the Notification progress of the execution of a mission. CoMMMA-FUN-09 Event Being able to be informed about the detection Notification of events during the execution of a mission. CoMMMA-FUN-10 Environmental Being able to periodically receive environmen- Information tal information during the execution of a mission.

1. Open Specification: the specification of the CoMMMA must be as open as possible, in the spirit of the European robotics-related research agendas [42, 43].

2. Open Source: the code for generic implementations and demonstrations of CoM- MMA must be open source licensed to promote its adoption and to provide examples that enable relevant robotics research on the mission management topic.

3. Distributed system: considering the distributed nature of having multiple agents cooperating in the fulfillment of a mission, it is expected to have a certain degree of distribution of a mission management system.

4. Service orientation: the mission management capabilities provided by CoMMMA can be considered as services using the [116] definition.

– 122 – 5.4 CoMMMA Functional Model

5. IoT relation: as IoT is considered one of the enabling technologies for robotics, it is desirable to use the IoT reference models and architectures as guidance for the definition of CoMMMA or any other mission management architecture.

Table 5.2. Non-Functional Requirements for CoMMMA

Identifier Name Description CoMMMA-NFR-01 Open Specification The specification of the architecture must be open in the spirit of openness promoted in [42, 43]. CoMMMA-NFR-02 Open Source The source code for generic implementations and demonstrations of CoMMMA must be open source licensed, as it is considered essential for quickly advancing in relevant robotics research [43]. CoMMMA-NFR-03 Distributed System The functionalities of CoMMMA may be distributed among several entities in the system, including the robots participating in the missions. CoMMMA-NFR-04 Service Oriented Considering the CoMMMA as a set of mechanisms to access one or more capabilities of robotic agents participating in a mission, it is reasonable to apply a service-oriented approach. CoMMMA-NFR-05 IoT Relationship CoMMMA must take into consideration the architectural references for IoT.

5.4 CoMMMA Functional Model

For the design of the CoMMMA a functional decomposition approach has been used, identifying the necessary functionalities to cover the requirements defined in previous sections. Using the Single-Responsibility Principle (SRP), the functional components associated with the identified functionalities have been defined, and they have been grouped according to the relationship between them.

Figure 5.1 illustrates a high-level abstract representation of the CoMMMA functional view, showing the six functional groups identified.

5.4.1 Mission Management Functional Group

The Mission Management Functional Group is associated with the mission management functions, and more specifically with organizing, commanding, coordinating and

– 123 – Towards a Common Mission Management Middleware Architecture

Figure 5.1. Functional view of the CoMMMA controlling the execution of a mission given a strategy. It is one of the two high-level functional groups, and consists of five functional components:

• Mission Manager • Mission Parser • Mission Dispatcher • Mission Monitor • Mission Analyzer

The Mission Manager functional component provides the coordination and mediation functionalities to the group. It acts as an intermediary between the application and the rest of the mission management functionalities. Thus, it is who receives the request to execute a mission from the application, with the corresponding strategy to follow, and who notifies to the application the information regarding the progress of the mission.

The other four components are designed after the MAPE-K adaptive model, with the knowledge component being represented by the strategy being followed.

The Mission Monitor collects the information from the agents participating in a mission, and correlates them to strategy related data that can be analyzed. The Mission Mon- itor can aggregate, correlate and filter the received information with other information available in the system.

– 124 – 5.4 CoMMMA Functional Model

The Mission Analyzer provides the mechanisms to observe and analyze situations to determine if some change needs to be made. The decision can be made by the analyzer, or supported by the use of a decision function available at the Decision Support System component from the Cross-Layer Services.

The Mission Parser is responsible for the analysis and interpretation of the strategy. It represents the “plan” element from the MAPE-K model.

The Mission Dispatcher has the functions for sending the strategy, the parsed plan, or the next action to be executed to the agents participating in the mission. It is, in a way, providing the mechanisms to execute the mission, and consequently it represents the “execute” element from the MAPE-K model.

CoMMMA does not enforce any specific implementation of any of these components. This means that for specific architectures there may be multiple implementations of any of the MAPE-K related components. In those situations, the mission manager is expected to act as a mediator, encapsulating the communication between them.

CoMMMA does not impose a fixed adaptive model either. It is possible to establish the necessary interactions between the agents that participate in a mission and the components in this functional group to carry out any decentralized adaptive control pattern.

5.4.2 Strategy Management Functional Group

CoMMMA is designed so that applications can also retrieve and reuse previous plans and strategies, create new strategies from both deterministic plans and other strategies, and update strategies either in execution or stored from previous missions. In this way, this group provides the necessary functionalities for the four steps of a CBR workflow [175]. The Strategy Management Functional Group is the second high-level group and has four functional components conceived for this purpose:

• Historical Plan Retrieval • Historical Strategy Retrieval • Strategy Generation • Strategy Update

The Historical Plan Retrieval and Historical Strategy Retrieval have the functionalities to retrieve either classical plans or full strategies from the repositories, respectively. They provide functions not only for requesting plans or strategies by their identifications in the repositories, but also for looking for plans or strategies that can achieve some goals, or that are similar to given plans or strategies. And in the case of a strategy

– 125 – Towards a Common Mission Management Middleware Architecture retrieval, the Historical Strategy Retrieval has also the functionality to retrieve a list of strategies that include a given plan.

The Strategy Generation offers the functionalities to create new strategies. For instance, it can use a plan aggregation algorithm, like the ones proposed in [168] or in [169]. The aggregation to create a strategy can be done using a provided list of plans or strategies provided by the application. Or it can be done by extracting the plans or the strategies from the repositories according to the conditions requested by the application (fulfillment of goals, similarity to given plans or strategies, or the inclusion of a given plan).

Finally, the Strategy Update contains the functions to update a strategy. For now, the CoMMMA proposal only contemplates the manual update of the strategy carried out from the application. However, it is possible to use automated strategy update mechanisms as long as the specific CoMMMA-based architecture includes them.

5.4.3 Report Management Functional Group

The Report Management Functional Group is one of the mid-level groups, and gathers together the three functional components dedicated to the generation of reports from the information received from the agents participating in a mission. The three functional components are:

• Mission Status Reporter

• Event Reporter

• Environmental Reporter

The Mission Status Reporter tracks and reports about the progress of the missions carried out by the agents. This information is stored in a repository using the functionalities from the Data Access Manager component, and is also sent to the Mission Monitor so that it can perform its function of monitoring the progress of the mission.

The Event Reporter collects any kind of event sent by the agents, and generates the corresponding report. In the same way as it happens with the Mission Status Reporter, this report is also stored in a repository, and it is also sent to the Mission Monitor to process it and act accordingly.

The Environmental Reporter receives information captured by the sensors used by the agents, and that is sent periodically. As in the other reporters, this information is used to generate a report that in this case, in addition to being stored in a repository, is sent to the Context Management to update the context information of the system if necessary.

– 126 – 5.4 CoMMMA Functional Model

CoMMMA does not enforce the use of any specific type of repository to store any ofthe reports. Such a decision is left open for the specific needs of each CoMMMA-based architecture.

5.4.4 Agent Abstraction Functional Group

The Agent Abstraction Functional Group is the other mid-level group, and encompasses the functional components related to the virtual representation of the agents, and which define how to interact with them. From the point of view CoMMMA, this abstraction is limited to the execution of the mission, and is inspired by the virtual entity model from the IoT-A ARM. In fact, it has no relation to the concept of the digital twin [176]. There are three functional components included in this group:

• Virtual Agent Discovery (VA Discovery),

• Virtual Agent Monitoring (VA Monitoring),

• Virtual Agent Service (VA Service).

The Virtual Agent Service represents the services related to mission management exposed by the agents registered in the system. These services can be for executing a single action, a sequence of commands (a plan), a partial strategy or a full strategy.

The Virtual Agent Monitoring is responsible for automatically detecting any new relevant information from the virtual agent physical counterpart. This can be either information addressed to the Report Management Functional Group for the generation of a report, or information related to the Virtual Agent Discovery functional component.

The Virtual Agent Discovery handles the discovery of virtual agents. The mission management capabilities that a CoMMMA-based system can offer depend on the availability of the right set of agents at the right time. A mission management system can have similar agents with different characteristics and qualities, and even have exactly the same type of agent. Moreover, some agents may become unavailable, and the system must be able to take this into account to continue to provide the same mission management capabilities. Thus, the Virtual Agent Discovery allows for dynamic association and updating between a physical agent and its virtual representation, providing the means to insert, delete and update said association.

5.4.5 Data Management Functional Group

The Data Management Functional Group is one of the two transversal functional groups, and is related to the organization and control of the data available in the system. There is no restriction with respect to the use of relational and non-relational databases,

– 127 – Towards a Common Mission Management Middleware Architecture as the Data Access Manager component is provided to abstract the access to the information stored in the available repositories. However, there are three databases required for a CoMMMA-based system:

• Historical Mission Plans • Historical Strategies • Ontology

The Historical Mission Plans and the Historical Strategies are two repositories needed to retain and retrieve mission plans and strategies that have been executed in the past.

The Ontology comprises the representation, formal naming and definition of the categories, properties and relations between the concepts, data and entities that substantiate at least the mission management domain, and possibly others related to the specific nature of the missions.

5.4.6 Cross-Layer Services Functional Group

The Cross-Layer Services is the other transversal group, and encompasses several functional components not directly related to each other, but that have in common that the can be accessed at any level of the CoMMMA. It consists of five functional components:

• Decision Support System • Reasoner • Context Management • Security • P/S Manager

The Decision Support System is defined as a mediating component to facilitate the access to Data Distribution Service (DDS) implementations that can be used to support decision-making in the context of mission management. As a possible use case, CoM- MMA contemplates the possibility of using Decision Support System (DSS) functions to decide whether an event notified by an agent requires plan adaptation according tothe mission strategy or not.

The Reasoner has the functionalities to infer the logical consequences from a set of rules. In CoMMMAa Reasoner is used to deduce if the context of the mission, that is, the information about the environment in which it is being executed, corresponds to any of the opportunities that can be part of a mission strategy.

– 128 – 5.5 Computational Analysis

The Context Management provides the CoMMMA with the basic capabilities for context awareness. It gives CoMMMA the capability to determine or influence a next action ina mission strategy by referring to the information that characterizes the environment where the mission is being conducted.

The Security component is responsible for ensuring the security and privacy of a CoM- MMA compliant system. It provides the functionalities to preserve the confidentiality, integrity and availability of the information, with consideration to its authenticity, account- ability, non-repudiation and reliability.

The P/S Manager provides the functions to use the publish/subscribe paradigm for an event-driven communication approach between the components of the CoMMMA. It is not mandatory to use a publish/subscribe paradigm, as it depends on the specific implementation for each case. However, it is useful for decoupling communications between components, such as when it comes to asynchronous notifications.

5.4.7 Communication Layer

Although not properly identified as a functional group, but as a layer, the Communica- tion Layer abstracts the specifics of the communication between the entities in which CoMMMA is distributed, including the agents that can participate in a mission. It is in charge of the proper handling of the messages to be sent and received from any entity in the system.

5.5 Computational Analysis

This section discusses the computational analysis of the CoMMMA proposal. First, it provides a Unified Modeling Language (UML) component diagram for the architecture following the functional model described in Section 5.4. Second, it provides the description of the use cases identified. Finally, it describes the interactions between the components for each of the use cases.

5.5.1 Detailed Component Diagram

Based on the abstract formalization provided in Section 5.4, a concrete formal model according to a UML component diagram of CoMMMA is proposed as illustrated in Figure 5.2. The interactions between the components, materialized through the interfaces shown in the diagram, are described in detail in Section 5.5.3, and the description of the use cases is discussed in Section 5.5.2.

– 129 – Towards a Common Mission Management Middleware Architecture iue5.2. Figure CoMMMA the of diagram Component

– 130 – 5.5 Computational Analysis

5.5.2 Use Cases

The use cases considered for CoMMMA usage are illustrated in Figure 5.3. In this figure it is possible to identify two types of actors. On the one side there are the operators, identified here as a Mission Operator and as a Strategy Operator. The Mission Operator is the actor who interacts with the mission management functionalities from the CoMMMA, while the Strategy Operator is the actor that interacts with the strategy management functionalities. On the other hand, there are the robotic agents, identified here as Capable of Strategy, Capable of Plan and Capable of Action, for those robots capable of understanding and following a strategy, just a plan, or just actions respectively.

Figure 5.3. CoMMMA Uses

Tables 5.3, 5.4, 5.5, 5.6, 5.7, 5.8, 5.9, 5.10, 5.11 and 5.12 collect the specific details for each use case identified.

[Remainder of page intentionally left blank]

– 131 – Towards a Common Mission Management Middleware Architecture

Table 5.3. Description of the Mission Execution related use cases

USE CASE: Mission Execution (CoMMMA-FUN-001) Scope & Objectives It is used to execute a mission given a strategy or a plan, the latter being considered as a special case of a strategy Actors Mission Operator Preconditions CoMMMA-based system deployed, Robotic Agents available and ready to receive a strategy, a plan or an action to be executed Postconditions Robotic Agents assigned to the mission are listed as in use Sequence As explained in Section 5.5.3.1 Exceptions The mission strategy, or where appropriate the mission plan, may not be accepted because it does not pass the validation carried out by the system

Table 5.4. Description of the Historical Plan Retrieval use case

USE CASE: Historical Plan Retrieval (CoMMMA-FUN-02) Scope & Objectives Retrieves a plan, or a list of plans, from the historical repository. It can return a specific plan given its identification, or a listof plans either by matching a list of goals, or by similarity to a given plan Actors Strategy Operator Preconditions A historical plan repository has to be available Postconditions Information containing a plan, or a list of plans, is returned to the actor Sequence As explained in Section 5.5.3.2.1 Exceptions There may not be any plan matching the request conditions

Table 5.5. Description of the Historical Strategy Retrieval use case

USE CASE: Historical Strategy Retrieval (CoMMMA-FUN-03) Scope & Objectives Retrieves a strategy, or a list of strategies, from the historical repository. It can return a specific strategies given its identification, or a list of strategies either by matching a list of goals, by similarity to a given strategy, or by containing a given plan Actors Strategy Operator Preconditions A historical strategy repository has to be available Postconditions Information containing a strategy, or a list of strategies, is returned to the actor Sequence As explained in Section 5.5.3.2.1 Exceptions There may not be any strategy matching the request conditions

– 132 – 5.5 Computational Analysis

Table 5.6. Description of the Strategy Generation use case

USE CASE: Strategy Generation (CoMMMA-FUN-04) Scope & Objectives It is used to generate new strategies Actors Strategy Operator Preconditions A strategy generation algorithm, like a plan aggregation, must be provided in a CoMMMA-based system Postconditions A strategy is generated Sequence As explained in Section 5.5.3.2.2 Exceptions The information provided by the actor may not be enough to generate a valid strategy

Table 5.7. Description of the Strategy Update use case

USE CASE: Strategy Update (CoMMMA-FUN-05) Scope & Objectives It is used to update a strategy, for both the ones stored in the historical strategies repository, and for the one being executed, if any Actors Strategy Operator Preconditions The strategy to be updated must exist in the repository Postconditions The strategy is updated Sequence As explained in Section 5.5.3.2.3 Exceptions The referred strategy to be updated may not exist

Table 5.8. Description of the Mission Adaptation use case

USE CASE: Mission Adaptation (CoMMMA-FUN-06) Scope & Objectives This case is used when an event is notified and the event requires the adaptation of the mission to a new situation Actors The three robotic agents Preconditions CoMMMA-based system deployed, a mission strategy in execution Postconditions The mission execution is adapted to the new situation Sequence As explained in Section 5.5.3.1.2 Exceptions The new situation may not be covered by the current strategy

– 133 – Towards a Common Mission Management Middleware Architecture

Table 5.9. Description of the Opportunity Execution use case

USE CASE: Opportunity Execution (CoMMMA-FUN-07) Scope & Objectives It is used for the execution of an opportunity once it is detected from the observation of the environmental conditions Actors The three robotic agents Preconditions CoMMMA-based system deployed, a mission strategy in execution with opportunistic plans Postconditions The opportunistic plan is executed to take advantage of the detected opportunity Sequence For those agents not capable of understanding strategies, it is explained in Section 5.5.3.1.1 Exceptions The strategy may not have any opportunistic plan included

Table 5.10. Description of the Mission Status Notification use case

USE CASE: Mission Status Notification (CoMMMA-FUN-08) Scope & Objectives It is used to notify the progress of a mission being executed Actors The three robotic agents Preconditions CoMMMA-based system deployed, a mission being executed Postconditions The status notification is received and processed by the CoM- MMA-based system Sequence As explained in Section 5.5.3.3.1 Exceptions A problem with the communications may mean the loss or delay in the notification of the status

Table 5.11. Description of the Event Notification use cases

USE CASE: Event Notification (CoMMMA-FUN-09) Scope & Objectives It is used to notify events during the execution of a mission Actors The three robotic agents Preconditions CoMMMA-based system deployed, a mission being executed Postconditions The event is notified to the CoMMMA-based system. Depend- ing on the event, it may require the use of mission adaptation Sequence As explained in Section 5.5.3.3.2 Exceptions A problem with the communications may mean the loss or delay in the notification of the event

– 134 – 5.5 Computational Analysis

Table 5.12. Description of the Environmental Information use cases

USE CASE: Environmental Information (CoMMMA-FUN-10) Scope & Objectives It is used to provide environmental information gathered by the agents and other entities capable of sensing during the execution of a mission Actors The three robotic agents Preconditions CoMMMA-based system deployed, a mission being executed Postconditions The environmental information received is used to update the context information used in the system. Certain conditions may identify an opportunity meaning the activation of an opportunistic plan Sequence As explained in Section 5.5.3.3.3 Exceptions A problem with the communications may mean the loss or delay in the notification of the environmental information

5.5.3 CoMMMA Workflows

5.5.3.1 Mission Management Related Workflows

This section describes the details for the interactions related to the mission management, including the mission execution and the mission adaptation to new scenarios.

5.5.3.1.1 Mission Strategy Execution

This workflow starts when an application gets a strategy and requests the CoMMMA compliant mission manager architecture to deploy it to the agents assigned to the mission. The strategy can be either a single level deterministic plan with no opportunistic chances, or any of the multiple possible options contemplated in the conceptual description of the structure of a strategy proposed in Section 4.4.

The execution of a mission strategy can follow different paths, depending on the mission related capabilities of the agents assigned to the mission. However, the first interactions are always the same, as illustrated in Figure 5.4. The strategy is received by the Mission Manager component, who validates it using the corresponding services from the Mission Parser. If the strategy is invalid, it is discarded and the application is notified with the reasons. If the strategy is valid, it is stored inthe Historical Strategies repository using the Data Access Manager. Afterwards, if the strategy has compound actions, it is filtered for each agent, also using the Mission Parser. The result of this filtering is a simplified version of the strategy, consisting of either the compound actions or their corresponding primitives, depending on the capabilities of the agents.

– 135 – Towards a Common Mission Management Middleware Architecture

Figure 5.4. Sequence diagram for the initial common interactions at the beginning of a mission strategy execution

Continuing with the strategy process for each agent, various scenarios may occur from this point on:

1. The agent is fully capable of understanding and executing strategies

2. The agent is not capable of understanding strategies:

(a) The agent is unable to understand decision nodes: i. The agent expects to receive a deterministic mission plan ii. The agent waits for an action, after which it will request the next one, and so on until the mission is finished (b) The agent is unable to understand opportunistic plans

It is important to note that scenarios 2.a and 2.b are not mutually exclusive. Figure 5.5 illustrates these scenarios, without considering opportunities.

The first scenario is the simplest of all, as it only requires to send the strategytothe agent through the Virtual Agent Service component using the Mission Dispatcher, as shown in Figure 5.6.

In scenario 2.a.i the agent is not capable of understanding the decision nodes that can be part of a strategy, and expects a full deterministic mission plan to execute. In this

– 136 – 5.5 Computational Analysis

Figure 5.5. Activity diagram used by the Mission Manager component for the execution of a strategy depending on the execution capabilities of the agents, and without considering opportunities

Figure 5.6. Sequence diagram for sending a strategy to an agent capable of understanding and executing it case the Mission Manager is able to generate the mission plan for the agent using the received strategy, the Mission Parser, and the Decision Support System component to estimate the current most probable decision for each decision node. The obtained mission plan is then sent to the agent from the Mission Dispatcher through the Virtual Agent Service. This sequence of actions is illustrated in Figure 5.7.

The case for scenario 2.a.ii requires a different approach. In this case, as the agent does not need a full deterministic plan, there is no need to generate one. Therefore, the Mission Manager, using the Mission Parser, checks if the current node in the

– 137 – Towards a Common Mission Management Middleware Architecture

Figure 5.7. Sequence diagram for sending a deterministic mission plan generated to an agent not capable of understanding strategies, and expecting a full plan strategy is an action or a decision node. If it is an action, it is sent to the agent from the Mission Dispatcher through the Virtual Agent Service as usual. If instead it is a decision node, then the Mission Manager uses the Decision Support System to decide the next action for the current context, and sends the obtained action to the agent from the Mission Dispatcher using the Virtual Agent Service. The rest of the actions that the agent will have to execute according to the strategy will be sent later, as the mission progresses. When the agent is ready to receive the next action, if any, it notifies it as a mission status. Therefore, the Virtual Agent Monitor will receive first this status, that will be notified to the Mission Status Reporter, who will store it in the repository using the Data Access Manager, and will also notify the Mission Monitor. The Mission Monitor will identify this status, and using the Mission Parser will retrieve the next node for this agent in the strategy. And as previously, if the node is an action node, it will be sent directly to the agent. And if it is a decision node, the action will be decided using the Decision Support System and sent to the agent afterwards. These interactions are shown in Figure 5.8.

[Remainder of page intentionally left blank]

– 138 – 5.5 Computational Analysis Sequence diagram for executing a mission strategy with an agent not capable of understanding strategies and expecting just one action Figure 5.8.

– 139 – Towards a Common Mission Management Middleware Architecture

The last scenario is when the agent is not capable of understanding opportunities. An opportunity can be identified either after an event, or as a consequence ofan environmental observation. In the first case, the event detected is received atthe Virtual Agent Monitor and then notified to the Event Reporter. The event report is stored in the repositories using the Data Access Manager, and notified to the Mission Monitor. The Mission Monitor extracts the relevant information from the report and sends it to the Mission Analyzer for further analysis, following a MAPE-K model. Simultaneously, the Mission Monitor can notify the application about the event if required. Meanwhile, the Mission Analyzer uses the Decision Support System to detect if the event activates an opportunity. If so, the Mission Parser is used to extract the opportunistic plan associated with the identified opportunity. Then, the opportunistic plan sent to the agent by the Mission Dispatcher using the Virtual Agent Service, and completing the MAPE-K loop. The opportunistic plan is sent according to the type of agent, that is, taking into account whether the agent expects to receive a plan or an action, as explained for scenarios 2.a.i and 2.a.ii. This sequence is shown in Figure 5.9. The part that corresponds to sending the plan can be done using any of the sequences described previously for scenarios 2.a.i and 2.a.ii.

The second case occurs when the opportunity is associated with an environmental condition. In this case the environmental information is received at the Virtual Agent Monitor, who notifies it to the Environmental Reporter. Then an environmental report is stored in the repositories using the Data Access Manager, and in this case it is also sent to the Context Manager to update the context of the mission. A Reasoner will be activated with the context update, and it will assess whether the new environmental conditions match with any known opportunities in the strategy. If this is the case, the associated opportunistic plan will be selected in the Mission Parser, and sent to the agent by the Mission Dispatcher using the Virtual Agent Service as usual. This sequence of interactions is illustrated in Figure 5.10.

Opportunities can also arise for events or observations on environmental conditions detected elsewhere in the system. In these cases, the Mission Manager identifies the agents affected by the detected opportunity, and is in charge of activating it taking into account the capabilities of each agent regarding the execution of mission strategies.

If the agent for whom the opportunity is to be activated is able to follow the mission strategy, then it is only necessary to notify it of the event or the environmental condition that has activated said opportunity.

If, on the contrary, the agent is not able to follow the mission, his plan is updated from the Mission Manager to the corresponding one for the opportunity.

– 140 – 5.5 Computational Analysis Sequence diagram for executing opportunities when an agent is not capable of understanding them directly, and the opportunity is Figure 5.9. identified by event an

– 141 – Towards a Common Mission Management Middleware Architecture dniidb nevrnetcondition an environment by identified 5.10. Figure eunedarmfreeuigopruiiswe naeti o aal fudrtnigte iety n h potnt is opportunity the and directly, them understanding of capable not is agent an when opportunities executing for diagram Sequence

– 142 – 5.5 Computational Analysis

5.5.3.1.2 Mission Plan Adaptation

Considering the formal definition of a plan as a sequence of actions, the adaptation of a mission plan occurs when a situation is detected during its execution that implies the need to change it. Such a situation can be detected in one of two ways: by receiving and analyzing an event reported by an agent participating in the mission, or by receiving and analyzing an event from some other element used in the mission (for example, sensors). Furthermore, a situation that requires adaptation of the mission plan is not necessarily limited to the agent who detected it. It is possible that a situation affects another agent participating in the mission, or several.

CoMMMA relies on the use of the strategy structure to handle adaptation. This provides anticipated knowledge that enables the adaptive response to events to be improved, and thus improves the balance of adaptation. Moreover, the Mission Management Functional Group components are defined after the MAPE-K adaptive model. Using this approach, CoMMMA provides the basis for defining specific architectures that can employ any decentralized adaptive control pattern.

When a received event notification corresponds to a situation that requires updating the mission plan, that is, its adaptation, it may or may not be that the situation was already anticipated in the mission strategy. If the situation were not foreseen, it would be the responsibility of the operator, through the Mission Management Application using CoMMMA, to decide what to do, being able to ignore the situation, update the strategy to face it, or abort the mission.

If, on the contrary, the situation was anticipated in the strategy, it would be enough to apply it.

The adaptation process starts with the notification of an event received at the Mission Monitor. The event is sent to the Mission Analyzer, and if it detects a situation that requires adaptation, it is notified to the Mission Manager. The Mission Manager identifies the agents affected by the required adaptation, and proceeds according totheir strategy execution capabilities.

If the agent understands strategies, and if the event was reported by the agent, then there is nothing else to do, because the agent just needs to follow the strategy to adapt to the situation. But if the agent did not report the event, then the Mission Manager has to notify the event to the agent using the Virtual Agent Service, so the agent is aware of the situation, and can follow the strategy to adapt to it.

If, on the other hand, the agent does not understand strategies, then it is not relevant whether it was the agent who reported the event or not. In this case, if the agent is executing a precalculated plan, then the Mission Manager has to generate the adapted

– 143 – Towards a Common Mission Management Middleware Architecture plan and send it to the agent. And if the agent was executing actions, then the Mission Manager, being responsible for selecting and sending the next actions, only has to follow the strategy.

Figure 5.11 shows the sequence of interactions for the adaptation of the mission plan to an event according to the description provided. The specific details of the generation of the adapted plan and the selection of the next action for agents incapable of following strategies have been deliberately omitted, as they have already been described for the mission strategy execution. The process followed by the Mission Manager component, as described before, is illustrated in the activity diagram from Figure 5.12.

Figure 5.11. Sequence diagram for mission adaptation. The specifics interactions for the generation of a plan from the strategy or the selection of the next action from a strategy must follow their corresponding sequence diagrams for mission strategy execution

– 144 – 5.5 Computational Analysis

Figure 5.12. Activity diagram used by the Mission Manager component for the adaptation of the mission execution based on the strategy execution capabilities of the affected agent

5.5.3.2 Strategy Management Related Workflows

5.5.3.2.1 Plan/Strategy Retrieval

The retrieval of a plan or a strategy can be done in three ways: by using a plan/strategy identification, by searching for plans/strategies that meet certain mission goals, orby searching for plans/strategies similar to given ones. There is an additional case for the retrieval of strategies containing a given plan. In all the cases the interactions are similar. First, the application sends the request to either the Historical Plan Retrieval, if it is looking just for plans, or to the Historical Strategy Retrieval, if it is looking for full strategies. The request is handled, and using the Data Access Manager, the plan, a list of plans, the strategy or a list of strategies is retrieved from the repositories. The retrieved information is then sent back to the application.

Figures 5.13, 5.14 and 5.15 show the interactions for the retrieval of a plan by its id, of a list of plans by matching goals, and of a list of plans by similarity to a given plan. Figures 5.16, 5.17, 5.18 and 5.19 show the interactions for the retrieval of a strategy by its id, of a list of strategies by matching goals, of a list of strategies by similarity to a given strategy, and of a list of strategies that contain a given plan.

– 145 – Towards a Common Mission Management Middleware Architecture

Figure 5.13. Sequence diagram with the interactions for retrieving a plan by its id

Figure 5.14. Sequence diagram with the interactions for retrieving a list of plans by matching given goals

Figure 5.15. Sequence diagram with the interactions for retrieving a list of plans by similarity to a given plan

– 146 – 5.5 Computational Analysis

Figure 5.16. Sequence diagram with the interactions for retrieving a strategy by its id

Figure 5.17. Sequence diagram with the interactions for retrieving a list of strategies by matching given goals

Figure 5.18. Sequence diagram with the interactions for retrieving a list of strategies by similarity to a given strategy

– 147 – Towards a Common Mission Management Middleware Architecture

Figure 5.19. Sequence diagram with the interactions for retrieving a list of strategies by containing to a given plan

5.5.3.2.2 Strategy Generation

The generation of a new strategy can be done using any suitable algorithm. In the case of using an aggregation approach, as the one mentioned in Section 5.4.2, the application can provide a list of plans or strategies to be aggregated. It can also request the aggregation by providing a list of plans and strategies identification from the repositories. It can even request the aggregation of the best plan or strategy matches for given goals, or for similarity to a given plan or strategy.

The former case is straightforward. The application provides the list of plans or strategies to be aggregated to the Strategy Generation component, obtaining the generated strategy as a response.

The latter cases require the Strategy Generation to ask for the list of plans or strategies either to the Historical Plan Retrieval or to the Historical Strategy Retrieval. Once the Strategy Generation gets the plans or the strategies for the aggregation, it can use any algorithm according to the specific needs of each implementation. Figure 5.20 shows this sequence of interactions from a general point of view.

[Remainder of page intentionally left blank]

– 148 – 5.5 Computational Analysis

Figure 5.20. Sequence diagram with the interactions for the generation of a strategy using aggregation with plans and/or strategies retrieved from the historical repositories

5.5.3.2.3 Strategy Update

The update of a strategy is made by a request from an application. It can be done over a stored strategy, or over the one that is being followed as part of the mission execution. In both cases the interaction starts at the application. The application has the identification of the stored strategy that is going to be updated, and with that information, and the desired changes to the strategy, it requests the update to the Strategy Update. This component does two things. The first one is to store the updated strategy into the historical strategies repository using the Data Access Manager. The second thing is to notify the Mission Manager that a strategy has been updated. It is done this way because according to the SRP, neither the Strategy Update nor any other components of the Strategy Management Functional Group need to know anything about missions. If the updated strategy is the one being used during the mission, the Mission Manager, having the updated strategy, will propagate it to the agents participating in the mission. Specifically, it will send the updated strategy to the agents who know how to followit, it will update the plans for those agents that require a plan, and it will apply the new

– 149 – Towards a Common Mission Management Middleware Architecture strategy directly to the rest of the agents that are just waiting for actions. Figure 5.21 illustrates these interactions.

Figure 5.21. Sequence diagram with the interactions for the update of a strategy

5.5.3.3 Report Management Workflows

5.5.3.3.1 Mission Status Report

The progress of the mission in execution is controlled by the mission status reports received from the vehicles. This interaction starts when the Virtual Agent Monitor receives a status message from its physical counterpart. The Virtual Agent Monitor sends the received status message to the Mission Status Report, who generates a report for the received status and sends it to the Mission Monitor, and also stores it into some repository using the Data Access Manager. The Mission Monitor then notifies the status to the Mission Manager, who notifies it to the application, and uses itto decide if it has to send the next action to an agent that is ready for them, as already described in Section 5.5.3.1.1. These interactions are shown in Figure 5.22

Figure 5.22. Sequence diagram with the interactions for the notification of a mission status

– 150 – 5.5 Computational Analysis

5.5.3.3.2 Event Report

The notification of an event starts when the Virtual Agent Monitor receives the event message from their physical counterpart. The event message is then sent to the Events Reporter, who generates the corresponding report. This report is then notified to the Mission Monitor, and stored in a repository using the Data Mission Manager. The Mission Monitor notifies the event to the Mission Analyzer, where it is analyzed to detect if it corresponds to a situation that requires adaptation, and to the Mission Manager to be forwarded to the application. If the Mission Analyzer detects a situation that requires adaptation, it notifies then the Mission Manager, as already explained for the mission adaptation in Section 5.5.3.1.2, and illustrated in Figure 5.11. Figure 5.23 shows the sequence diagram with these interactions, except those corresponding to adaptation, already explained.

Figure 5.23. Sequence diagram with the interactions for the notification of an event

5.5.3.3.3 Environmental Report

The environmental information gathered by the agents participating in a mission, also known as telemetry, is received by the Virtual Agent Monitor, as with the rest of the reports. This information is then sent to the Environmental Report to generate the corresponding report. The report is stored in the repositories using the Data Access Manager, and notified to the Context Management component, to update the system context. When the context is updated, a Reasoner may be activated to assess whether the new environmental conditions match with any known opportunities in the strategy. Figure 5.24 illustrates these interactions. Further interactions related to the management of opportunities for agents that require a mission plan have already been shown in Figure 5.10.

[Remainder of page intentionally left blank]

– 151 – Towards a Common Mission Management Middleware Architecture

Figure 5.24. Sequence diagram with the interactions for the notification of environmental information

5.6 Conclusions

This chapter has described the formal proposal for Common Mission Management Middleware Architecture (CoMMMA) ARM. Section 5.2 has provided an overview of the approach used to design the architectural proposal. As part of this overview, the concept of the balance of adaptation in a coordinated and distributed adaptation pattern was presented. The balance of adaptation refers to the time required by a system to adapt to a new situation. And, in a distributed system, like when working with cooperative autonomous robots, this time is determined by the time necessary to communicate the new situation to the rest of the entities participating in a mission, and by the time required to decide the action with which to adapt to the given situation. The CoMMMA proposal is able to improve this balance by acting on the deliberation part with the use of the strategy model from Chapter4. It is also able to provide a decentralized adaptation, meaning that even if a robotic agent is able to adapt itself to a situation, the architecture provides the means for other robotic agents to be aware of such, and adapt themselves if necessary.

A comprehensive list of the requirements for a mission management architecture was collected in Section 5.3. This list describes the functional and non-functional requirements for the CoMMMA ARM.

Once the requirements analysis is complete, Section 5.4 describes the functional model of the CoMMMA in detail. The functional model is done by identifying the functionalities and grouping them together by similarity, resulting in six functional groups: Mission Management, Strategy Management, Data Management, Report Management, Agent Abstraction, and a transversal group with a generic name of Cross-Layer Services.

Finally, Section 5.5 provides the computational analysis of the CoMMMA. This analysis begins with a detailed component diagram for the full architecture including the relations between the functional components identified in the functional model. It is followed by

– 152 – 5.6 Conclusions the description of the use cases, and completed with the interactions for each of the use cases.

The CoMMMA proposal is defined to serve as a guideline to design specific mission management architectures for cooperative robotics. The design of specific mission management architectures depends on the specific characteristics of each domain. Therefore, it is not necessary for a specific architecture to implement all the functionalities described here.

– 153 –

Part IV

Research Contributions: Implementation and Analysis

Chapter 6

The SWARMs Mission Manager: The MTRR. Design and Validation

Management of many is the same as management of few. It is a matter of organization — Sun Tzu

6.1 Introduction

This chapter explores and analyzes the adaptation of the CoMMMA proposal described in Chapter5 to the specific scenario of the SWARMs European research project [57, 58]. The primary goal for the SWARMs project was to expand the use of underwater and surface vehicles (AUVs, Remote Operated Vehicles (ROVs), USVs) to facilitate the conception, planning and execution of maritime and offshore operations and missions.

In order to provide some context, it is important to recall the approaches already defined for mission management in previous projects related to maritime operations. For instance, in [177] an object-oriented system for AUV control called Administrative System for AUV Control Software (ASACS) was defined. The ASACS architecture focused on the control at the vehicle level, and included a plan dispatch subsystem, also called captain in the proposal, responsible for activating the items from the mission plan, that is, the tasks (or actions) that are part of it. This work has been used as a reference for other architectures, like the hybrid Command and Control (C2) architecture proposed in [178, 179]. In this later work, the authors borrowed the “captain” idea from the ASACS proposal, and included it at the supervisory level of a three-layer architecture, being responsible for the start, coordination, supervision and control of the execution of the mission. In this proposal, the captain was in direct connection with the “executive officer” component

– 157 – The SWARMs Mission Manager: The MTRR. Design and Validation at the mission level, which was responsible for the interpretation of the mission plan provided by the “captain” and passing to the “navigator” component the required actions according to the plan.

In [180] the authors proposed a Mission Management System (MMS) to guide the AUV Martin [181] during the execution of pre-programmed survey missions. The proposed MMS consisted of a group of modules, including a mission control module named also “captain”, and a mission execution module for controlling the execution of the mission plan. In this proposal the mission control was in charge of starting and stopping a mission, triggering the re-planning when needed, and applying the emergency procedures as required. The mission execution module received the detailed parts of the plan and sent the required commands to the “navigator” module.

T-REX, already presented and discussed in Section 3.3.1, was conceived as a goal- oriented system with embedded automated planning and adaptive execution, and was mostly considered at the vehicle level. The same applies to ICA, discussed in Sec- tion 3.3.3, and RAUVI, discussed in Section 3.3.2.

So far, mission management for autonomous vehicles in maritime operations are considered mainly at the vehicle level. Indeed, in [131] it is specified that, for the automation part, a pre-programmed mission shall be stored previously in the AUV and that the execution shall begin after a starting order. The same approach is mentioned in other surveys regarding mission planning and management systems for AUVs[182–184]

However, SWARMs explored other approaches for mission management trying to improve in at least two specific issues. On the one hand, as already presented in this introduction, the typical procedure for maritime operations was to load a predefined plan into the vehicles, and only manage their execution at the vehicle level. Therefore, any adaptive approach was completely delegated in the vehicles. SWARMs added an extra layer of mission management with the Mission and Tasks Register and Reporter (MTRR), allowing for the use of a decentralized control pattern for plan adaptation. On the other hand, another issue found in previous systems was that they were tightly coupled to the vehicles used. SWARMs allowed for the use of heterogeneous vehicles through the use of a virtualization approach embedded in the MTRR.

Keeping in mind the specific requirements for the SWARMs project, the MTRR was designed using the CoMMMA proposal as a guide following the process depicted in Figure 6.1.

The rest of this chapter has been mostly published in [185] as part of the dissemination efforts for this thesis, and is organized as follows: Section 6.2 gives an overview of the SWARMs architecture focusing on those parts related to the MTRR and how they

– 158 – 6.2 Overview of the SWARMs Architecture

Figure 6.1. Process of the generation of the MTRR interact; Section 6.3 describes in detail the specific design of the MTRR; Section 6.4 provides the details for the validation, including the validation goals, the scenario, and the specific characteristics of each part used during the validation; Section 6.5 discusses the results obtained during the validation tests; and finally, Section 6.6 explores some conclusions obtained from the validation.

6.2 Overview of the SWARMs Architecture

In order to have a better grasp of this mission manager, this section introduces an overview of the SWARMs architecture [186, 187], paying special attention to the components related to the MTRR and the used information models.

6.2.1 The SWARMs Architecture

Figure 6.2 shows an overview of the SWARMs architecture. The top layer is the application layer, where two applications were developed:

• A Mission Management Tool (MMT)[188], used by a human operator to supervise the global mission, to track the situational changes, and to generate the mission plans.

• A Context Awareness Framework (CAF) application, used to provide a visual interface to the context awareness to a human operator.

The applications at the application layer interact directly with the SWARMs middleware. Located between the communication network with the vehicles and the application layer, the middleware is responsible for:

– 159 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.2. SWARMs architecture overview

• Centralize the communications.

• Serve as a mission executive, providing execution and control mechanisms.

• Define a common information model.

• Use a semantic management.

• Provide a registry of all available vehicles and services.

• Track and report the progress status of a mission.

• Define mechanisms and functionalities for understanding the surrounding environment of the vehicles participating in a mission.

• Define mechanisms for data validation.

With that into consideration, the SWARMs architecture follows a hierarchical control pattern for decentralized self-adaptive control [106].

– 160 – 6.2 Overview of the SWARMs Architecture

Finally, to make transparent the SWARMs Data Distribution Service (DDS) communication through the acoustic network between the Central Control Station (CCS) where the middleware will be running and the vehicles, two interface modules are used [132, 189]:

• The Control Data Terminal (CDT), that provides an interface between the middleware components at the CCS and the acoustic communication system used to communicate with the vehicles.

• The Vehicle Data Terminal (VDT), that provides an interface between the communication system and the vehicle.

Among all the components in the SWARMs middleware, the mission manager called here MTRR is the component responsible for the treatment and supervision of all the mission related interactions within the middleware. In a general overview, it is responsible for mediating between a MMT application, used by a human operator to generate the mission plan and supervise the mission progress, and a DDS[190] used as the paradigm of communications with the vehicles participating in a mission [187]. In the middleware, the MTRR interacts with the Semantic Query from the high-level services block, with the Publish/Subscription Manager component from the Data Management block, and with all the three reporter components from the low-level services block. It also makes use of the mission plan structure defined in the SWARMs information model [191]. The following subsections give a brief description of the components interacting with the MTRR and the mission plan model being used.

6.2.1.1 The Semantic Query Component and the Data Repositories

There are several repositories defined in SWARMs:

• A semantic ontology has been created for the SWARMs project, as described in [191]. This ontology defines a common information model that provides anon- ambiguous representation of a swarm of maritime vehicles and missions. It is also in charge of storing the information obtained from different domains, including mission and planning, and environment recognition and sensing among others. Besides the information model, the ontology is also used to store the last updated values, as they are also used automatically to infer events by means of the use of semantic reasoners [191–193].

• In order to perform requests on the SWARMs ontology, Apache Jena Fuseki has been utilized, as it can provide server-like capabilities for resource storage, along with the protocols used for the SPARQL Protocol and RDF Query Language

– 161 – The SWARMs Mission Manager: The MTRR. Design and Validation

(SPARQL) queries needed to retrieve data from the SWARMs ontology. The combined Fuseki server sending SPARQL requests to the SWARMs ontology that is accessed and the ontology itself are referred here as the semantic repository. There are several reasons for having chosen Fuseki as the way to interact with the ontology. To begin with, it offers the SPARQL protocol version 1.1 for queries and updates, enabling easier access to the ontology. Secondly, by having a server with a web-based front end, data management and monitoring of the ontology status becomes easier. Lastly, specific resources can be provided to the server in terms of, for example, memory allocation, thus making it more flexible to execute depending on the capabilities of the hardware that runs it. It must be noted that the MTRR has been conceived to send and store information used for mission planning, execution and data retrieval, and it is oblivious to the ontology that is used for this purpose, due to the fact that the MTRR has been designed, implemented and tested as software capable of sending information to an ontology, regardless of which one is used. Development works strictly related to ontologies and ontologies for underwater vehicles are covered by their authors in [191].

• A historical data repository that allows to have a record of the progress of the missions, and to be used by decision-making algorithms to compute the recommended choices using the information from past situations. As it was expected that the volume of the historical data would be large, it was decided to use a relational database for this repository instead of the ontology.

The access to both repositories from the MTRR is done through the Semantic Query component. This component abstracts the use of the Data Access Manager component for inserting and retrieving information for any of the repositories. Even though the name suggests that all the queries requests sent to it will be semantic and, therefore, related to the ontology, in fact it provides the methods for accessing both the ontology and the relational database. It also provides a high-level interface to the application layer so the applications can forward the queries they need.

6.2.1.2 The Publish/Subscription Manager Component

The Publish/Subscription Manager is a DDS-based component responsible to provide reliable non-blocking communications between the middleware and the underwater vehicles. It uses a publish/subscribe paradigm for data communications, generating and managing a collection of DDS topics. A detailed description of how DDS communications are managed in SWARMs is given by the authors in [187].

– 162 – 6.2 Overview of the SWARMs Architecture

6.2.1.3 The Reporters

The reporters are located at the low-level services section of the middleware and are in charge of receiving the raw data from the vehicles through the DDS Manager, processing the data to generate a report and delivering the report to the MTRR. The procedure followed is:

• The Task Reporter receives the data regarding the status of the tasks being executed in a mission carried out by a vehicle, generates the corresponding report, and dispatches the report to the MTRR for further processing.

• The Environment Reporter receives the environmental data that the vehicles publish periodically to the DDS Manager, generates the corresponding report, and dispatches the report to the MTRR for further processing.

• The Event Reporter receives the events published by the vehicles to the DDS Manager, generates the corresponding report, and dispatches the report to the MTRR for further processing.

6.2.2 The Mission Plan

The SWARMs ontology [191] defines a mission plan as a “sequence of scheduled low- level tasks (operator and vehicle tasks) that need to be carried out to achieve a mission, with dependencies between tasks and approximate time duration”. Figure 6.3 illustrates the SWARMs mission and planning ontology.

The mission plan is generated at the MMT using one of the available high-level planners [194–196] and sent to the MTRR through an Apache Thrift [197, 198] interface using an ad-hoc format whose structure is shown in Figure 6.4.

Each mission plan is identified by a missionId, and consists of a hierarchical list of actions and the list of vehicles participating in the mission. The rest of the attributes shown in Figure 6.4 are only used at the MMT to feed the planners with the areas and points of interest for a mission.

Every action in the action list is assigned to a vehicle. They can also have a parent action, forming a hierarchical structure where the high-level actions are those without a parent, and the low-level actions are the vehicle-level tasks from the SWARMs ontology. A graphical depiction of this structure is shown in Figure 6.5. This figure illustrates a mission plan that includes three AUVs named NTNU, PORTO 1 and PORTO 2. For each AUV there is a two-level hierarchical mission plan.

For instance, the mission plan corresponding to the NTNU AUV in Figure 6.5 has a high-level plan consisting of three high-level tasks, starting with a TRANSIT, followed

– 163 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.3. SWARMs mission and planning ontology

Figure 6.4. Structure of a SWARMs mission plan by a SURVEY and finishing with another TRANSIT. And the low-level plan consists of decomposing the high-level tasks into low-level ones: both TRANSIT high-level tasks are decomposed into just one GOTO_WAYPOINT each; and the high-level SURVEY is decomposed in a sequence of FOLLOW_ROW and GOTO_WAYPOINT low-level tasks.

The vehicles in SWARMs may be equipped with an onboard planner, meaning that these vehicles are capable of understanding high-level tasks and generating the required

– 164 – 6.3 Description of the MTRR

Figure 6.5. Example of a mission for three AUVs named NTNU, PORTO 1 and PORTO 2. In this figure GW refers to the low-level tasks GOTO_WAYPOINT, and FR refers tothe low-level task FOLLOW_ROW low-level tasks. In consequence, only one of the two levels of the hierarchy needs to be dispatched to the vehicles: the high-level tasks when the vehicles have an onboard planner, and the low-level tasks otherwise.

6.3 Description of the MTRR

In a general sense, the MTRR is responsible for the mission execution and control at the middleware layer. From top-down, the MTRR receives the mission plan generated at the MMT, processes it and dispatches the tasks to the corresponding vehicles, storing the mission-related information in the SWARMs repositories. From bottom-up, the MTRR receives the reports from the vehicles participating in a mission, processes and stores them into the SWARMs repositories, and if required, notifies the MMT about the events of interest. In a more detailed description, these are the functionalities defined for the MTRR:

• Mission information retrieval. The different parameters that are fixed at the MMT in the mission plan as a way to establish what kind of plan will be executed or what maritime vehicles should be involved are transferred to the MTRR. Those

– 165 – The SWARMs Mission Manager: The MTRR. Design and Validation

parameters are: coordinates of the area to be inspected or surveyed, stages of the plan to be executed (going to a specific point, having an autonomous vehicle executing a particular trajectory) or transferring back to the MMT data about the mission progress.

• Mission status collection. The MTRR provides information to the high end of the system regarding what stages of the mission were completed. Depending on the actions to be carried out by the vehicles, high-level tasks such as “SURVEY ” (used by the AUV to maneuver in a land mower pattern), “TRANSIT ” (used to go to a given coordinate) or “HOVER” (used by USVs to navigate the surface of an area to be surveyed in a circular pattern) will need to be sent. Missions are created following a sequential order for the tasks to be executed, so each of those tasks will be sent to the vehicles through the MTRR, which will notify when they are completed. Also, the MTRR is in charge of sending the “ABORT ” signal to the vehicles when the mission has to be finished abruptly due to any major reason.

• Information storage invocation. Data about how vehicles have progressed in a specific mission are retrieved via state vectors that contain, among other piecesof information, their location, speed, depth or altitude from the seabed, according to a data format specifically developed for the project [199]. The MTRR invokes the necessary operations that involve other software components in the middleware responsible for storing the data gathered. In this way, the MTRR makes the information available for all the other applications that require the data to perform their actions, such as the context awareness component (used to assess the level of danger in certain situations) or a plume simulator (which would simulate information about salinity concentration during a pollution leak event).

As anticipated in Section 6.2.1, the MTRR is a software component with close interactions with other parts of the SWARMs architecture. Those that are most notorious are related to the MMT, but there are also other components that make use of the information retrieved to perform their own functionalities. While these other components are usually not directly involved in the execution of missions, they provide several functionalities to the overall middleware architecture and rely in a critical manner on the MTRR. These interactions have been further summarized in Table 6.1.

– 166 – 6.3 Description of the MTRR

Table 6.1. Relations between the MTRR and other middleware components

Component Component Interaction with the Location name Functionality MTRR

Sends the mission content to the MTRR and receive data back Used to show data Supervise Global Mission from it about stages and create Mission Management Tool completed and state missions vector data collected from the autonomous vehicles

It is invoked by the MTRR in order to Used to access the transfer the information storage means of that has been gathered Semantic Query Data Management the middleware from the maritime (ontology, vehicles to the database database) or the ontology when it is required

Used to set the paradigm of The MTRR invokes communications some of its Publish/Subscribe between the functionalities to Data Management (PS) Manager middleware and subscribe to the topic the maritime that is used for autonomous communications vehicles

Used to report task Invokes the MTRR in Low-Level Task Reporter status information order to collect Services during the mission information

Used to highlight Invokes the MTRR in Low-Level Events Reporter unusual events order to collect Services during the mission information

Used to report information Invokes the MTRR in Environment Low-Level collected about the order to collect Reporter Services surroundings of the information vehicle

– 167 – The SWARMs Mission Manager: The MTRR. Design and Validation

A UML perspective of these relations has been depicted in Figure 6.6. Note that each of the locations of the middleware components that interact with the MTRR has also been included as well so there is a better understanding on the application domain of their functionalities.

Figure 6.6. MTRR-centered SWARMs components diagram

Additionally, it is shown in Figure 6.7 how the different interfaces and classes of the component have been designed and implemented. There are two interfaces, MTReporter and MTFeeder, which define the methods that must be used to transfer the information to the components responsible for reports (which require either raw or processed data and an identifier for the mission that is taking place) in case of the former, and what kind of actions are to be expected from missions for the latter. Some other classes are used for information parsing regarding vehicle information and data format (like Mission-

– 168 – 6.3 Description of the MTRR

Parser and MessageFormatter), whereas other ones are used to compare actions of local or global nature (that is to say, ActionComparator and ActionComparatorGlobal). An additional class called ThriftClientToMMT is used to interface the MMT, whereas MessageCONSTANTS and MTRRException are support classes that are used to include constants and throw exceptions when required.

Figure 6.7. MTRR component class diagram

6.3.1 AUV Virtualization at the MTRR

Virtualization can be defined as the creation of a virtual representation of physical entities, including hardware platforms like AUVs. The IoT-A reference model provides an interesting domain model that includes the virtualization of physical entities by associating their capabilities to resources that are exposed as services [200]. Figure 6.8 illustrates this part of the IoT-A domain model.

The MTRR adopts a similar approach for the AUVs and USVs used in a mission. Fig- ure 6.9 shows an overview of the used internal virtualization.

– 169 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.8. IoT-A domain model related to the virtual representation of a physical entity [200]

Figure 6.9. Overview of the vehicle virtualization used at the MTRR

The physical vehicle has an onboard computer. In this onboard computer there is a software robot supervision module and depending on its capabilities, a robot planner module. On the other hand, running inside the MTRR (hence identified as a “network resource” following the IoT-A terminology), there is the “plan controller”, accessed as a part of the virtual vehicle that represents the physical one.

– 170 – 6.3 Description of the MTRR

The virtual representation of a vehicle is created when the MTRR receives a new mission plan. As part of the procedure for the start of a mission, the MTRR parses the vehicles assigned to the mission, and extracts their corresponding mission plan, creating an internal virtual representation. The purpose of this virtualization is to abstract the dispatcher part of the MTRR regardless of whether the vehicles have an on-board planner or not. In this way, the dispatcher part of the MTRR only has to request to dispatch the next task in the plan for the virtual representation of the vehicle. And it is this virtual representation the one that is responsible for selecting if it has to follow the high-level or the low-level tasks of the corresponding part of the mission plan.

6.3.2 Activities performed by the MTRR

The MTRR performs the following activities:

• Starting a mission.

• Receiving and recording mission status reports.

• Receiving and recording environmental reports.

• Receiving and recording events reports.

• Aborting a mission.

These activities are discussed in detail in the following subsections.

6.3.2.1 Starting a Mission

Figure 6.10 shows the functional overview of the start of a mission within the SWARMs architecture. From the MTRR point of view, the start of a mission begins when it receives a new mission plan from the MMT (step 1). After verifying that the plan is valid and parsing it into the corresponding vehicle plans, the MTRR stores the information in the SWARMs repository (both historic data at the relational database and semantic data at the ontology) through the Semantic Query component (step 2). Then the MTRR publishes the mission plan to the DDS through the Publish/Subscription Manager component (step 3), and it is finally delivered to the vehicles (step 4).

Internally, when the MTRR receives a new mission from the MMT, it performs the following actions in the given order:

1. Validation of the mission plan.

2. Storage of the reference coordinates into the semantic repository (that is, the ontology via Fuseki server).

– 171 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.10. Functional overview of the start of a mission

3. Storage of the mission plan into the semantic repository.

4. Parsing the vehicles assigned to the mission plan.

5. Storage of assigned vehicles into the semantic repository.

6. Subscription to vehicles events.

7. Parsing the actions assigned to each vehicle in the mission plan.

8. Publication of the first assigned action for each vehicle in the SWARMs DDS partition.

The validation of the mission plan (activity 1) is just the sequence of verification depicted in Figure 6.11. In essence, the validation just checks if there is no other active mission, if the mission is not empty, if it has a plan, and finally, if it has assigned vehicles. If the validation is passed, then the corresponding new mission ID is stored in the P/S Manager.

Once the validation result has been positive, the MTRR stores the relevant information in the ontology using the methods provided by the Semantic Query component (activities 2 and 3). Afterwards, the MTRR checks if the statuses from the vehicles have been updated, and if not, it requests them for an update. Usually this request has been made

– 172 – 6.3 Description of the MTRR

Figure 6.11. Flow diagram for the plan validation undertaken at the MTRR previously by the MMT, just before making a new plan and through the proper MTRR service created for that. After the MTRR confirms that the statuses from the vehicles

– 173 – The SWARMs Mission Manager: The MTRR. Design and Validation are updated, it subscribes itself to the events from them (activity 6). This part of the startMission method is depicted in Figure 6.12.

Figure 6.12. Flow diagram for mission data extraction and storage done at the MTRR

Finally, the MTRR assigns the actions in the plan to each vehicle (activities 7 and 8). Each vehicle is able to perform high-level actions (if they have an onboard planner) or low-level actions (if they do not have an onboard planner), and the MTRR is responsible for sending them the adequate action list according to their capabilities. This is supported

– 174 – 6.3 Description of the MTRR by a MissionParser as part of the MTRR component. Also the assignment of the plan to each vehicle can be done in two ways:

1. Sending the full plan to each vehicle.

2. Sending just the first action in the plan to each vehicle. Subsequent actions willbe sent from the TaskReport whenever requested by the vehicle.

The second style of assignment was added to the startMission in the MTRR as requested by some vehicle providers in the SWARMs project. Figures 6.13 and 6.14 show the diagrams for both ways of assignment, respectively. The full sequence diagram is detailed in Figure 6.15.

Figure 6.13. Flow diagram for the task assignment when done by full plan

– 175 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.14. Flow diagram for the task assignment when done by wait for completion

– 176 – 6.3 Description of the MTRR Sequence diagram for the startMission procedure performed by the MTRR Figure 6.15.

– 177 – The SWARMs Mission Manager: The MTRR. Design and Validation

6.3.2.2 Receiving and Recording Mission Status Reports

Figure 6.16 shows the functional overview of how a mission status report sent by a vehicle is processed within the SWARMs architecture. It starts with the vehicle publishing the corresponding mission status part to a specific DDS topic (step 1). The Task Reporter component, which is subscribed to that topic, receives the published data (step 2), processes it, and sends the generated report to the MTRR (step 3). The MTRR then processes the report, stores the relevant information into the SWARMs repository through the Semantic Query component (step 4), notifies the MMT (step 5), and if there are pending tasks in the vehicle plan, publishes the next one to the specific DDS topic through the P/S Manager (step 6).

Figure 6.16. Functional overview of the processing of a mission status report from a vehicle

In summary, the reception of a new mission status report in the MTRR has three main responsibilities:

1. To extract and store the status report in the ontology using the Semantic Query component.

2. Notifying the MMT if there is any important status update, like the vehicle being unable to fulfill the mission.

– 178 – 6.3 Description of the MTRR

3. If the assignment mode was set to wait until task completion for requesting the next task in the mission plan, then the assignment of the tasks to the vehicles besides the first one is performed also when receiving a mission status report with a status of COMPLETED.

The first part is straightforward: it is just the extraction of the status from the statusreport and its storage into the ontology using the interface provided by the Semantic Query.

The second part is also straightforward: if the received status is of interest for the MMT, then the MTRR will send a notification to it.

The third part is also very simple, as depicted in Figure 6.17. If the task status report indicates that the task has been completed, the next task for the vehicle, if any, is extracted from the vehicle plan and published through the P/S Manager.

Figure 6.17. Flow diagram for assigning the next task after receiving a COMPLETED task status

Figure 6.18 shows the sequence diagram for the full procedure performed by the MTRR when receiving a new task status report.

6.3.2.3 Receiving and Recording Environmental Reports

The functional overview for processing an environmental report is shown in Figure 6.19. The flow is similar to that for the mission status reports. As before, it starts whenthe vehicle publishes data, in this case environmental data from the onboard sensors, to the DDS topic for environmental information (step 1). This time the Environmental Reporter component is the one that is subscribed to that topic. Consequently, it receives the published environmental data (step 2), generates an environmental report from the received data, and sends the report to the MTRR (step 3). Finally, the MTRR processes the environmental report and stores the information in the SWARMs repository through the Semantic Query (step 4).

– 179 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.18. Sequence diagram for the task report method in the MTRR

Figure 6.19. Functional overview of the processing of an environmental report

When the MTRR receives an environmental report it stores the data from the report in the historical database and in the ontology using the interface provided by the Semantic Query. It also checks if the report includes salinity concentration information, as it requires a specific method in the interface to the Semantic Query. The full sequence diagram for this method is shown in Figure 6.20.

– 180 – 6.3 Description of the MTRR

Figure 6.20. Sequence diagram for the management of an environmental report

6.3.2.4 Receiving and Recording Events Reports

The reception of an event report implies notifying the MMT about the event as well as storing the event into the database. The functional overview is essentially the same as for the mission status and the environmental reports, with the only difference of using the Events Reporter component. The sequence diagram for this method is provided in Figure 6.21.

Figure 6.21. Sequence diagram for the management of an event report

6.3.2.5 Aborting a mission

A mission can be aborted manually for whatever reason the operator considers appropriate. The operator can decide whether to just abort the part of the mission assigned to a vehicle, or the full mission. Figures 6.22 and 6.23 show the sequence diagrams for aborting a vehicle plan and the full mission respectively.

– 181 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.22. Sequence diagram for aborting a vehicle plan

Figure 6.23. Sequence diagram for aborting the full mission

6.4 Validation Details

6.4.1 Validation Goals

The main purpose of the validation of the MTRR is to verify its effectiveness and compliance with the SWARMs project Scientific and Technical Objectives (STO), and in particular with the development of a semantic middleware and common information

– 182 – 6.4 Validation Details model for managing the interoperability, cooperation and coordination [201]. Specifi- cally, the MTRR is responsible for the mission management from within the SWARMs middleware, having a significant role.

In summary, the validation goals for the MTRR are:

1. Verification of the effectiveness of the procedures for starting a mission, processing a task status report and dispatching the next task whenever there is any pending assignment, and processing the received environmental reports. The MTRR will be considered effective if it produces the expected results in terms of interoperability, cooperation and coordination.

2. Characterization of the startMission activity of the MTRR by measuring the times required for each of its internal sub-activities, and identifying possible bottlenecks and correlations.

3. Characterization of the reportTask activity of the MTRR by measuring the times required for each of its internal sub-activities, and identifying possible bottlenecks.

4. Characterization of the reportEnvironment activity of the MTRR by measuring the times required for each of its internal sub-activities, identifying possible bottlenecks and comparing the efficiency of storing the state vector in the semantic andthe relational databases.

6.4.2 Validation Scenario

The MTRR was validated during the integration and final demonstration for the SWARMs project. These tests were done at the Trondheim Biological Station (TBS) in Norway. The test area used for the validation is shown in Figure 6.24.

A CCS was deployed at the TBS facilities as the on-shore control operations room. The CCS consisted of five computers connected to the same private IP network. Figure 6.25 shows how the different parts of the system were distributed among the CCS computers. Note that the MMT and CAF computers are in charge of running the MMT and CAF applications respectively, and two more computers are used for plume and vehicle simulations.

Regarding the middleware and the repositories of information, a virtual machine solution was chosen. Both the relational database and the Fuseki server used to query the ontology were deployed in the same virtual machine as the middleware.

All the communications from the middleware to the vehicles, as explained in Section 6.2, were done through the CDT. If the mission was simulated, the CDT was configured to

– 183 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.24. Final SWARMs demo test area used for validation

Figure 6.25. Control Station distribution connect to the VDTs from the simulator computer at the same CCS. Should the mission use real vehicles deployed underwater, the CDT would be configured to communicate to the VDTs on board those vehicles. Therefore, from the point of view of the rest of the system, the use of a simulator or real vehicles was transparent.

The plume simulator [202] that is also shown in Figure 6.25 was used in the following way: when the Environment Reporter component from the middleware received a new report from the vehicles (either real or simulated), it queried the plume simulator for the concentration values corresponding to the position of the vehicle. The returned data, if any, were aggregated to the report sent to the MTRR.

– 184 – 6.4 Validation Details

6.4.3 Technical details

6.4.3.1 The middleware running environment

The MTRR, as part of the SWARMs middleware, was deployed in a VMware virtual machine running Ubuntu 14.04 LTS with a Linux kernel version 4.2.0-42-generic. The virtual machine was configured with 2 GB of RAM, 4 cores and 20 GB of harddisk storage (prior tests were carried out with just 1 core). It was run on top of a Windows 10 machine with 16 GB of RAM and 8 i7 cores using VMware Workstation Player 12.

The MTRR was integrated with the rest of the SWARMs middleware as a Java application, and executed using a Java SE Runtime Environment version 8u101 from Oracle. Also, although the middleware could be run as standalone software, it was decided to run it from the development environment used, which was Eclipse Mars. All the invocations between the middleware components were done through calls to Java methods among the different classes implementing them.

The historical database was deployed as a relational database using MySQL Server version 5.5.55 (package version 5.5.55-0ubuntu0.14.04.1 installed directly from the Ubuntu repositories) and it was deployed in the same virtual machine as the SWARMs middleware (and therefore the MTRR). The required resources to query the SWARMs ontology were deployed as a SPARQL server using Apache Jena Fuseki version 3.4.0, and it was also deployed in the same virtual machine as the SWARMs middleware, with a specific maximum memory allocation of 1200 MB.

6.4.3.2 Other Equipment and Materials Used for the Tests Involving the MTRR

The integration and final demonstration for SWARMs was carried out through simulations and real deployments. The real deployments used a group of AUVs, an USV or both. The details of the vehicles used in the tests are listed in Table 6.2, and the vehicles themselves are shown in Figure 6.26.

Table 6.2. Details of the vehicles used during the tests

Vehicle name Type Codename Noptilus-1 AUV PORTO1 Noptilus-2 AUV PORTO2 Fridtjof AUV NTNU Telemetron USV MAROB

A hardware-in-the-loop approach was used for the simulations. This approach allowed us to test everything as if real vehicles were used. As shown in Figure 6.25 the simulated vehicles were run in a dedicated computer deployed at the CCS. They used exactly the

– 185 – The SWARMs Mission Manager: The MTRR. Design and Validation

(a) AUVs (b) Telemetron (USV)

Figure 6.26. Vehicles used during for the final audit of SWARMs same software as the real vehicles, and it was executed using a Gazebo simulator for underwater intervention and multi-robot simulation [203, 204] running in the simulator computer. In consequence, from the middleware perspective the use of a simulated or a real vehicle was transparent, with no impact in its internal operations.

6.4.4 Details for the validation tests

During the integration and the final demonstration several tests were conducted. Re- garding the validation for the MTRR, a total of 45 missions were registered. The basic details of each mission is shown in Table 6.3, including the total number of actions in the mission, how many of them were high-level ones, and how many were low-level, the vehicles assigned for the mission, and whether the mission was real or simulated.

Table 6.3. Details for each registered mission

Start Actions Mission Mission Assigned Vehicles Time H L T Type 1 10:01:27 9 14 23 NTNU,PORTO2,MAROB Real 2 10:17:56 9 0 9 NTNU,PORTO2,MAROB Real 3 11:57:17 9 14 23 NTNU,PORTO2,MAROB Simulated 4 12:57:58 9 34 43 NTNU,PORTO2,MAROB Real 5 13:25:55 9 34 43 NTNU,PORTO2,MAROB Real 6 15:31:05 9 34 43 NTNU,PORTO2,MAROB Simulated 7 17:10:09 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 8 17;58:57 3 16 19 MAROB Real 9 10:29:28 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 10 11:01:59 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 11 11:13:54 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 12 12:21:25 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 13 12:30:29 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 14 12:41:11 9 0 9 NTNU,PORTO1,PORTO2 Real 15 12:55:57 9 0 9 NTNU,PORTO1,PORTO2 Real (continued on next page) – 186 – 6.4 Validation Details

Table 6.3. Details for each registered mission (continued)

Start Actions Mission Mission Assigned Vehicles Time H L T Type 16 13:17:36 9 0 9 NTNU,PORTO1,PORTO2 Simulated 17 13:39:19 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 18 14:12:16 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 19 15:19:59 9 48 57 NTNU,PORTO1,PORTO2 Real 20 15:37:19 3 16 19 MAROB Real 21 15:40:54 3 16 19 MAROB Real 22 16:38:04 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 23 17:38:34 12 50 62 NTNU,PORTO1,PORTO2,MAROB Simulated 24 19:33:52 3 10 13 MAROB Real 25 08:26:21 3 12 15 MAROB Real 26 09:01:21 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 27 09:47:54 3 16 19 MAROB Real 28 10:00:23 3 0 3 MAROB Real 29 10:47:27 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 30 12:05:20 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 31 12:25:24 3 6 9 MAROB Real 32 12:37:14 3 10 13 MAROB Real 33 13:20:55 3 10 13 MAROB Real 34 13:41:14 3 16 19 MAROB Real 35 14:44:52 3 16 19 MAROB Real 36 15:11:33 12 50 62 NTNU,PORTO1,PORTO2,MAROB Real 37 10:36:45 3 8 11 MAROB Real 38 10:59:09 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 39 11:15:00 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 40 11:36:15 3 8 11 MAROB Real 41 11:38:50 3 8 11 MAROB Real 42 11:55:27 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 43 12:11:50 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real 44 14:17:02 3 8 11 MAROB Real 45 14:36:24 12 0 12 NTNU,PORTO1,PORTO2,MAROB Real

The approach used for the mission assignment for all the missions was by individual tasks due to the constraints in the Robotic System On-board Architecture (RSOA) used in the vehicles. This means that when a mission starts, only the first action for each vehicle is sent to them. The rest of the actions in the mission plan are sent when a COMPLETED status report is received from a vehicle for its current action.

Mission 27 was aborted after sending the first action in the mission plan to each vehicle as there were underwater unexpected obstacles that prevented them from fulfilling the

– 187 – The SWARMs Mission Manager: The MTRR. Design and Validation mission. Missions 35, 36, 44 and 45 were those performed during the SWARMs final audit.

6.4.5 Details for the mission plans and execution used for the trials

This section provides the specific details for the mission plans and their execution for the missions performed during the final audit for the SWARMs project. The rest of the missions considered for the validation, and that were performed during the preparations for the final audit were essentially the same, with minor modifications.

6.4.5.1 Audit day 1: first mission

The first mission for the first audit day consisted of the survey of one surface areabya USV. This mission, numbered as mission #35 in Table 6.3 used only one vehicle, the MAROB USV, with a mission plan consisting in 3 high-level and 16 low-level actions, for a total of 19 actions. The hierarchical representation of the mission plan is shown in Figure 6.27. The top row, with the black-filled boxes, represents high-level actions. In this case the three high-level actions were a TRANSIT, a SURVEY and another TRANSIT.

Figure 6.27. Hierarchical representation of mission 35 plan

The bottom row, with the grey-filled boxes, represents the low-level actions in which the high-level ones are decomposed. In this case the first and second high-level TRAN- SIT actions are decomposed in just one low-level GOTO_WAYPOINT each. and the high-level SURVEY is decomposed in a succession of low-level FOLLOW_ROW and GOTO_WAYPOINT actions.

The mission plan provided by the MMT also included a time estimation for each of the actions. Figure 6.28 shows the timeline for the mission plan considering only the high-level actions.

The mission was executed without problems, and the progress of the mission as reported by MMT to the mission operator is shown in Figure 6.29.

As the only vehicle participating was an USV, no underwater communications were used during this mission. The request for starting a new mission was received at the MTRR at 14:44:52, and the first task assignment was delivered to the USV at 14:44:53, with the first task report from the USV confirming the reception of the action being received

– 188 – 6.4 Validation Details

Figure 6.28. Estimated timeline for mission 35 plan

Figure 6.29. Mission 35 progress as shown at the MMT developed by MDH

491 milliseconds later. The USV reached the final point in its mission plan at 14:58:52, meaning that the mission took 14 minutes to finish.

6.4.5.2 Audit day 1: second mission

The second mission for the first audit day consisted of the survey of the same underwater area at three different depth levels performed by three AUVs, and with the support of a USV. Therefore, this mission, numbered as mission #36 in Table 6.3 used three AUVs and one USV acting as a support vehicle for the relay of the communications to and from the AUVs. The mission plan consisted of 12 high-level and 50 low-level actions, for a total of 62 actions. The hierarchical representation of the mission plan is shown in Figure 6.30. Each lane represents the part of the mission plan that corresponds to each

– 189 – The SWARMs Mission Manager: The MTRR. Design and Validation vehicle, that is, a vehicle plan. For each vehicle plan, the top row, with the black-filled boxes, represents the high-level actions, and the bottom row, with the grey-filled boxes, represents the low-level actions.

Figure 6.30. Hierarchical representation of mission 36 plan

The plan for each AUV was essentially the same, differing only in the parameters for the actions. They consisted of a high-level TRANSIT, followed by a high-level SURVEY, and finally a high-level TRANSIT. As before, the high-level TRANSIT actions were decomposed in low-level GOTO_WAYPOINT actions, and the high-level SURVEY action was decomposed in a succession of low-level FOLLOW_ROW and GOTO_WAYPOINT actions.

The plan for the USV consisted of a high-level TRANSIT, followed by a high-level INSPECT, and another TRANSIT. As before, the high-level TRANSIT actions were decomposed in low-level GOTO_WAYPOINTS actions. The high level INSPECT action was considered as a primitive action for this type of vehicle, and was considered as a low level action also.

– 190 – 6.4 Validation Details

The mission plan provided by the MMT also included a time estimation for each of the actions. Figure 6.31 shows the timeline for the mission plan considering only the high-level actions.

Figure 6.31. Estimated timeline for mission 36 plan

The mission was executed without problems, and the progress of the mission provided by the MMT to the mission operator is shown in Figure 6.32. In this case, as the AUVs were operating underwater, and underwater communications are still unreliable, some reports were missing, causing some jumps in the progress information about the mission execution of some AUVs.

Figure 6.32. Mission 36 progress as shown at the MMT developed by MDH

This mission also used the plume simulator mentioned in the description of the validation scenario. The plume defined in the simulator was inside the three areas being explored by the AUVs. The information reported by the vehicles, and managed by the MTRR, was shown to the human operator using a CAF application. Figure 6.33 illustrates the main

– 191 – The SWARMs Mission Manager: The MTRR. Design and Validation screen of the CAF showing the plume as being detected by the AUVs participating in the mission.

Figure 6.33. Mission 36 progress as shown at the CAF GUI used in SWARMs

The request for starting a new mission was received at the MTRR at 15:11:33. The first task assignment for each vehicle was registered also at 15:11:33 for each of the AUVs and the USV. The first task reports from each vehicle confirming the reception ofthe task assignment after their corresponding delivery were received 523 ms later for the USV, 782 ms later for the PORTO1 AUV, 1536 ms later for the NTNU AUV, and 1583 ms later for the PORTO2 AUV. The last AUV emerged at 15:42:15, and the mission was registered as finished at 15:43:11, meaning that the mission took 30 minutes and42 seconds to finish.

6.4.5.3 Audit day 2: first mission

The first mission for the second audit day consisted of the survey of one surface area by a USV. This mission, numbered as mission #44 in Table 6.3 used only the MAROB USV, with a mission plan consisting in 3 high-level and 8 low-level actions, for a total of 11 actions. The hierarchical representation of the mission plan is shown in Figure 6.34. The top row, with the black-filled boxes, represents high-level actions. In this casethe three high-level actions were a TRANSIT, a SURVEY and another TRANSIT.

– 192 – 6.4 Validation Details

Figure 6.34. Hierarchical representation of mission 44 plan

The mission plan provided by the MMT also included a time estimation for each of the actions. Figure 6.35 shows the timeline for the mission plan considering only the high-level actions.

Figure 6.35. Estimated timeline for mission 44 plan

The mission was executed without problems, and the progress of the mission provided by the MMT to the mission operator is shown in Figure 6.36.

Figure 6.36. Mission 44 progress as shown at the MMT developed by MDH

As the only vehicle participating was an USV, no underwater communications were used during this mission. The request for starting a new mission was received at the MTRR at

– 193 – The SWARMs Mission Manager: The MTRR. Design and Validation

14:17:02, and the first task assignment was delivered to the USV also at 14:17.02, with the first task report from the USV confirming the reception of the action being received 515 milliseconds later. The USV reached the final point in its mission plan at 14:24:54, meaning that the mission took 7 minutes and 52 seconds to finish.

6.4.5.4 Audit day 2: second mission

The second mission for the second audit day consisted of the survey of three different underwater areas at three different depth levels performed by three AUVs, and with the support of a USV. Therefore, this mission, numbered as mission #45 in Table 6.3 used three AUVs and one USV acting as a support vehicle for the relay of the communications to and from the AUVs. The mission plan consisted of just 12 high-level and none low-level actions. The representation of the mission plan is shown in Figure 6.37. As in previous figures, each lane represents the part of the mission plan that corresponds toeach vehicle, that is, their vehicle plan.

Figure 6.37. Hierarchical representation of mission 45 plan

The plan for each AUV was essentially the same, differing only in the parameters for the actions. They consisted of a high-level TRANSIT, followed by a high-level SURVEY, and finally a high-level TRANSIT. The plan for the USV consisted of a high-level TRANSIT, followed by a high-level INSPECT, and another TRANSIT.

The mission plan provided by the MMT also included a time estimation for each of the actions. Figure 6.38 shows the timeline for the mission plan considering only the high-level actions.

– 194 – 6.4 Validation Details

Figure 6.38. Estimated timeline for mission 45 plan

The mission was executed without problems, and the progress of the mission provided by the MMT to the mission operator is shown in Figure 6.39. In this case, as the AUVs were operating underwater, and underwater communications are still unreliable, some reports were missing, causing some jumps in the progress information about the mission execution of some AUVs.

Figure 6.39. Mission 45 progress as shown at the MMT developed by MDH

The request for starting a new mission was received at the MTRR at 14:36:24. The first task assignment for each vehicle was registered also at 15:11:33 for each of the AUVs and the USV. The first task reports from each vehicle confirming the reception of thetask assignment after their corresponding delivery were received 407 ms later for the USV, 457 ms later for the NTNU AUV, 624 ms later for the PORTO1 AUV, and 681 ms later for the PORTO2 AUV. The last AUV emerged at 14:43:49, and the mission was registered as finished at 14:44:48, meaning that the mission took 8 minutes and 24 secondsto finish.

– 195 – The SWARMs Mission Manager: The MTRR. Design and Validation

6.5 Results

This section includes the analysis of the results obtained from the records acquired from the missions listed in the previous section.

The validation of the MTRR focuses on the processing time for each of the activities it executes as described in Section 6.3.2, as the processing time could introduce issues in the mission management when the task assignment is done by task. As a reminder, these activities are:

• Starting a mission.

• Reporting the status of a vehicle.

• Reporting the state vector from a vehicle.

The results obtained for each activity are discussed in the following subsections.

6.5.1 Starting a mission

From the MTRR perspective, the start of a mission begins when it receives a new mission plan from the MMT. When this happens, the MTRR performs the activities described in 6.3.2. For debugging purposes, the MTRR also recorded logs of the mission plan that just consisted of parsing the mission and logging it into a log file for further verification.

The time registered for each sub-activity, as well as the total time required for the starting mission activity, is shown in Table 6.4. The time for parsing the actions assigned to each vehicle in the mission plan and to publish the first assigned action for each one is included as tpubaction . The time for validating the mission plan, (re)loading the MTRR configuration, and checking if there are AUVs in the mission is included as tother.

Table 6.4. Times for Starting a Mission in the MTRR after receiving it from the MMT (in ms)

Mission tlogplan tstorecoords tstoreplan tstoreveh tparseveh tsubsevents tpubaction tother 1 91 17 140 13 0 3 7 6 2 32 5 60 12 9 2 25 9 3 73 14 121 9 1 4 10 9 4 111 9 210 11 1 3 10 4 5 170 12 239 17 1 6 7 12 6 182 10 212 12 1 2 7 5 7 170 8 361 14 1 2 8 5 8 129 10 141 14 1 1 2 11 9 38 7 74 16 1 5 12 4 (continued on next page)

– 196 – 6.5 Results

Table 6.4. Times for Starting a Mission in the MTRR after receiving it from the MMT (in ms) (continued)

Mission tlogplan tstorecoords tstoreplan tstoreveh tparseveh tsubsevents tpubaction tother 10 48 7 70 25 1 2 9 6 11 38 6 62 17 4 5 8 13 12 59 8 70 18 2 11 23 7 13 56 7 61 15 1 4 19 8 14 37 5 37 14 0 2 8 10 15 42 6 41 15 1 2 6 8 16 35 11 102 28 1 10 15 5 17 28 7 57 14 0 5 10 5 18 52 7 67 16 0 3 12 5 19 162 10 265 9 2 3 6 5 20 135 21 107 5 1 1 4 35 21 90 9 122 5 0 2 2 32 22 200 6 248 12 1 4 12 19 23 191 10 317 12 0 3 7 8 24 62 6 65 6 0 1 3 7 25 53 16 71 4 0 1 2 12 26 204 6 293 22 2 16 32 13 27 94 8 147 5 1 1 5 17 28 17 6 16 10 1 1 4 26 29 162 10 268 15 1 4 10 4 30 241 31 265 13 1 2 6 8 31 34 6 51 9 0 0 4 5 32 51 8 73 4 0 1 2 6 33 113 11 199 7 0 1 6 9 34 94 10 125 4 0 2 3 10 35 217 33 144 4 1 1 2 18 36 164 6 286 16 2 5 12 5 37 138 30 73 4 1 1 5 30 38 41 6 50 16 2 4 13 7 39 43 10 104 19 1 6 8 23 40 67 5 80 5 0 1 3 17 41 87 10 338 5 0 1 3 7 42 65 13 83 19 0 4 16 4 43 74 11 87 38 1 6 10 9 44 61 10 81 11 0 1 3 7 45 48 7 59 16 1 7 18 10

In Figure 6.40 we can see the distribution of the times required for each sub-activity and each mission separately, along with the length of the mission plan for each mission.

– 197 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.40. Distribution of times for starting a mission in the MTRR

The graph depicted in Figure 6.40 seems to suggest a possible correlation between the mission plan length (lplan) and the time required to start it (tstartMission). Figure 6.41 represents a scatterplot for the relationship between both.

Figure 6.41. Relation between the size of the mission plan and the time required to start it

The scatter plot also suggests a definite positive correlation between lplan and tstartMission. However, as shown in Figure 6.42, both sets of data do not seem to follow a normal distribution.

In the first case, as illustrated in Figure 6.42a, the whiskers for lplan are of approximate equal length, but the median is too close to the lower quartile, and there are also a couple of outliers. In the second case, as illustrated in Figure 6.42b for tstartMission, the distance between the lower quartile and the lower whisker is shorter than the upper one, and the median is also displaced from the center. Therefore, a Spearman’s correlation

– 198 – 6.5 Results

(a) (b)

Figure 6.42. (a) Boxplot for the mission plan length (lplan). The median is completely displaced to the lower quartile, and there are also a couple of outliers; (b) boxplot for the time required to start a mission (tstartMission) is considered to determine the relationship between the 45 mission plan lengths and the total time required for starting each mission. As a result, there is a strong positive monotonic correlation between these two values (ρs = 0.79, n = 45, p < 0.001). To better understand the contribution of each sub-activity to the total time required for starting a mission, Figure 6.43 shows the Pareto chart with the average distribution of time for each sub-activity. It is quite clear that the sub-activity contributing most is the one in charge of storing the mission plan into the semantic repository, followed by the logging of the mission plan actions to the human operator, other activities performed during the start of a mission, and the storing of the vehicles assigned to the mission and the reference coordinates into the semantic repository. If we take into consideration that the logging of the mission plan actions to the human operator was done for debugging purposes and it is not required for the normal operation of the MTRR, we can affirm that the most consuming activities when starting a mission are those related to the storage of information into the semantic repository.

By removing the time for logging the mission to the operator, it can be observed how storing the mission plan into the semantic repository is the activity that takes more time, representing 67.37% of the total time required for the procedure of starting a mission. This is better illustrated in Figure 6.44. Indeed, all the activities related to the storage of information in the semantic repository represent a 78.83% of the total time.

The length of the plan and the number of assigned vehicles have an impact on the performance of the parts that handle the mission plan and the list of assigned vehicles. From the MTRR perspective these actions are done as one, and thus, only an estimate of the distribution of time can be made. To do so it is required to divide the time of each action by the length of the mission plan, or the number of assigned vehicles as

– 199 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.43. Pareto chart for the average distribution of time for each activity performed by the MTRR when starting a mission

Figure 6.44. Pareto chart for the average distribution of times for each activity performed by the MTRR when starting a mission without consideration for the logging of the actions in the mission plan appropriate. Figure 6.45 illustrates the contributions from each part considering the aforementioned estimation. The storage and the logging of the mission plan have been averaged taking into account the total number of actions in the mission plan, which are provided in Table 6.3. The storage of the assigned vehicles, the subscription to events and the assignment of the first action for each vehicle have been averaged taking into account the number of vehicles for each mission also provided in Table 6.3.

– 200 – 6.5 Results

Figure 6.45. Pareto chart for the average distribution of time for each activity performed by the MTRR when starting a mission, averaged to the number of actions in the mission plan and the number of vehicles as appropriate

6.5.2 Processing task status reports

Table 6.5 contains the collected information about the total times spent for processing the task status report, sorted by mission. Each row includes how many task status reports are received in the mission (#Rpts), how many times a next action assignment was performed (#Actns), the total time required for retrieving the vehicle information from the database (tveh), the total time used in the mission for storing the task status reports in the database (tstostat ), to send the status report to the MMT( tsendstat ), to send the next action assignment to the vehicle, if any (tsendact ) and for other activities not relevant for the processing of the status report (tother). There were no records regarding the processing of the tasks status reports during mission 27.

Table 6.5. Values for processing the task status reports and, if required, send the next action to the corresponding vehicle. Times are given in milliseconds

Mission #Rpts. #Actns. tveh tstostat tsendstat tsendact tother ttotal 1 170 5 1190 226 44 20 18 1498 2 215 7 1123 228 49 43 27 1470 3 199 7 934 230 56 39 16 1275 4 149 3 677 158 40 19 14 908 5 215 7 1077 263 44 25 19 1428 6 229 8 1069 274 68 38 16 1465 7 301 10 1351 331 58 39 30 1809 8 71 3 482 106 30 14 5 637 9 43 1 211 47 12 5 5 280 (continued on next page)

– 201 – The SWARMs Mission Manager: The MTRR. Design and Validation

Table 6.5. Values for processing the task status reports and, if required, send the next action to the corresponding vehicle. Times are given in milliseconds (continued)

Mission #Rpts #Actns tveh tstostat tsendstat tsendact tother ttotal 10 285 8 1322 299 54 42 24 1741 11 157 4 705 287 39 21 9 1061 12 299 8 1413 338 58 30 27 1866 13 119 3 566 140 19 18 15 758 14 169 4 658 173 43 29 15 918 15 217 6 956 257 58 39 17 1327 16 253 8 1151 263 66 31 18 1529 17 243 7 1116 227 51 63 19 1476 18 261 9 1369 330 78 45 20 1842 19 161 3 562 119 39 30 8 758 20 39 1 290 56 11 12 1 370 21 39 1 310 82 9 5 9 415 22 127 3 590 127 32 21 15 779 23 394 13 1544 387 81 54 28 2094 24 71 3 503 79 29 10 11 632 25 71 3 435 92 16 16 8 567 26 301 10 1200 329 75 57 22 1683 27 — — — — — — — — 28 71 3 413 113 22 11 9 568 29 233 6 977 268 57 35 32 1369 30 201 4 979 203 63 26 14 1285 31 63 2 396 98 31 7 4 536 32 41 1 262 61 31 5 7 366 33 71 3 474 92 28 17 12 623 34 71 3 424 98 21 19 6 568 35 41 1 285 49 15 7 2 358 36 237 6 1080 281 50 25 17 1453 37 71 3 576 93 36 13 6 724 38 301 10 1250 259 44 44 20 1617 39 301 10 1226 309 58 32 23 1648 40 41 1 288 59 16 6 0 369 41 65 2 456 99 33 17 11 616 42 227 6 1231 265 43 40 28 1607 43 259 6 1195 263 56 37 14 1565 44 71 3 495 163 28 15 13 714 45 267 8 1133 287 65 32 13 1530

In order to better understand which part of the processing of the report contributes the most to the total time, the average time is calculated for each mission and shown in Figure 6.46. The average time for retrieving the vehicle information from the database,

– 202 – 6.5 Results storing the task status in the database, sending the task status to the MMT and the rest of the activities not relevant for the processing of the report is calculated using the total number of reports registered for each mission. The average time for the next action assignment is calculated using the number of next action assignments that occurred during each mission.

Figure 6.46. Average time distribution for the processing of the task status reports

Finally, in order to understand which part of the report processing contributes the most to the full processing time, Figure 6.47 shows the Pareto chart for the averages for each part and all the missions. The part contributing most is the next action assignment, with 48.69% of the total time, followed by the retrieval of the vehicle information, with 39.39% of the total time. The other activities done and the notification of the status report tothe MMT are the parts contributing less with 0.70% and 2.13% respectively. The figures for the average time are collected in Table 6.6.

Figure 6.47. Pareto chart for the processing times of the task status reports

– 203 – The SWARMs Mission Manager: The MTRR. Design and Validation

Table 6.6. Average times for processing the task status reports

Activity Average time (ms) Contribution (%) Getting vehicle info 5.26 39.39 Storing task status in database 1.21 9.09 Sending task status to MMT 0.28 2.13 Assigning next action 6.50 48.69 Other activities 0.09 0.70 TOTAL AVERAGE TIME 13.35 100.00

6.5.3 Processing environmental reports

The vehicles are set to send periodically environmental reports, known as state vectors, which always have the same size. Table 6.7 collects the total time used for processing the state vectors for each mission. Each row includes the mission number, the total number of reports received during the mission (#Rpts), the total number of reports that included salinity information (#Rptssalt), the total time spent for storing the state vector in the relational database (tstodb ) and in the ontology (tstoont ), the total time for storing the salinity information in the ontology (tstosalt ) and the total time used for other activities not relevant to the processing of the report (tother). There were no records regarding the processing of environmental reports during mission 27.

Table 6.7. Values for processing the state vector report. Times are given in milliseconds

Mission #Rpts #Rpts t t t t t salt stovdb stovont stosalt other total 1 156 0 1694 10962 — 86 12742 2 87 0 884 5320 — 56 6260 3 128 0 1330 7037 — 72 8439 4 334 14 3072 17470 398 171 21111 5 528 31 5202 26807 878 274 33161 6 305 26 3013 15942 751 182 19888 7 758 139 7063 37239 4880 417 49599 8 130 0 1173 6620 — 96 7889 9 1608 0 14868 77662 — 779 93309 10 278 0 2832 13490 — 119 16441 11 48 0 390 2551 — 31 2972 12 303 0 2782 14401 — 148 17331 13 124 0 1177 6196 — 75 7448 14 92 0 776 4296 — 41 5113 15 493 0 4668 23073 — 268 28009 16 181 0 1859 11483 — 69 13411 17 276 0 2307 15626 — 128 18061 (continue on next page)

– 204 – 6.5 Results

Table 6.7. Values for processing the state vector report. Times are given in milliseconds (continued)

Mission #Rpts #Rpts t t t t t salt stovdb stovont stosalt other total 18 141 0 1308 8423 — 68 9799 19 279 48 2484 14219 1354 172 18219 20 13 0 164 867 — 4 1035 21 46 0 485 2372 — 30 2887 22 289 19 2557 14106 517 166 17346 23 851 137 7259 41090 4554 504 53405 24 94 0 873 4923 — 38 5834 25 216 0 1808 11413 — 79 13300 26 490 69 4504 23836 1989 271 30600 27 — — — — — — — 28 180 0 1790 8297 — 104 10191 29 321 59 3173 15605 1786 208 20772 30 176 18 1619 10429 499 86 12633 31 54 0 524 2824 — 23 3371 32 14 0 137 810 — 9 956 33 110 0 1040 5278 — 64 6382 34 155 0 1513 7225 — 59 8797 35 127 0 1192 6132 — 66 7390 36 443 43 4102 20727 1194 211 26234 37 53 0 471 3663 — 36 4170 38 336 22 3359 17326 617 189 21491 39 228 8 2242 10762 219 107 13330 40 7 0 75 438 — 5 518 41 45 0 404 2828 — 21 3253 42 96 0 961 4807 — 57 5825 43 160 5 1473 9557 129 57 11216 44 56 0 575 2955 — 21 3551 45 163 2 1563 8174 50 99 9886

The distribution of the time required for processing the state vectors is also shown in Figure 6.48. It is clearly seen that for every mission the most contributing part was the storage of the state vector in the ontology, followed by the storage of the salinity information also in the ontology whenever this information was available. The averages for the storage of the state vector in the relational database and in the ontology, as well as for the other activities not relevant to the processing of the report, are calculated using the total number of reports registered for each mission. The averages for storing the salinity information in the ontology are calculated using just the number of reports that included salinity information.

– 205 – The SWARMs Mission Manager: The MTRR. Design and Validation

Figure 6.48. Distribution of time required for processing the state vectors

The Pareto chart shown in Figure 6.49 illustrates how each part done for processing the state vector contributes to the total time required. As was clearly seen in the distribution of time from Figure 6.48, the part of the processing time that contributes the most is the storage of the state vector in the ontology, followed by the time required for storing the salinity information in the ontology whenever this information was available. Other activities performed for processing the state vector made negligible contributions to the total.

Figure 6.49. Pareto chart for the average total time required for each part in processing the state vector

As mentioned before, the part contributing most was the storage of the state vector in the ontology, with a 55.34% of the average total time. The storage of the salinity information in the ontology follows with a 33.85%. The time spent for the other activities not relevant

– 206 – 6.5 Results for the processing of the environmental report are negligible, with a contribution of 0.57%. The figures for the average times are provided in Table 6.8.

Table 6.8. Average times per activity for processing the state vector reports

Activity Average time (ms) Contribution (%) Storing vector in the database 9.36 10.24 Storing vector in the ontology 50.61 55.34 Storing salinity in the ontology 30.96 33.85 Other activities 0.52 0.57 AVERAGE TOTAL 91.45 100.00

From the point of view of the MTRR, the storage of the state vector and the salinity information are undertaken as one unique, indivisible action provided by the Data Access Manager component in the SWARMs architecture. Thus, from the MTRR perspective it can only be asserted that storing information in the ontology took longer. Considering that the state vector always has the same size, the average time for storing it in the semantic repository was 12339 ms with a standard deviation of 13435 ms. As for the relational database, the average time for storing the state vector was 2283 ms with a standard deviation of 2564 ms. Figure 6.50 illustrates the distribution of both series of time.

(a) (b)

Figure 6.50. (a) Boxplot for times required to store the state vector in the SWARMs ontology; (b) boxplot for the times required store the state vector in the historical database

In terms of efficiency it can be affirmed that storing the information in the relational database represents only an average of 18.50% of the time required to store the same information into the semantic repository.

– 207 – The SWARMs Mission Manager: The MTRR. Design and Validation

6.6 Conclusions

This chapter has covered the design, implementation and validation of a specific mission manager, the MTRR, for a specific scenario, the SWARMs European research project, using the CoMMMA proposal from Chapter5 as a guideline. Section 6.2 has given a detailed overview of the SWARMs architecture and its requirements. The specific design of the MTRR has been provided in Section 6.3. This specific mission manager has been validated during the SWARMs project final audit following a validation design as described in Section 6.4, and obtaining the results discussed in Section 6.5.

In particular, the results obtained from the validation suggest that the impact of the MTRR specific actions in the whole architecture was negligible, allowing the identification of some potential issues that should be handled carefully when designing and implementing such specific mission manager:

1. Dealing with repositories for storing and retrieving information related to the mission is usually the most time-consuming activity.

2. The size of a mission also has an impact on its processing and dispatching. Constrained scenarios should also take this into account.

Besides those issues, this design, implementation and validation suggests also that dealing with heterogeneous vehicles regarding their individual mission management capabilities is possible when using a distributed mission management approach like the one proposed in CoMMMA, also enabling a decentralized control of the plan adaptation.

– 208 – Part V

Conclusions and Future Works

Chapter 7

Conclusions and Future Works

The best way to predict the future is to invent it —Alan Kay

In this dissertation we have presented the results of our research on adaptive mission planning for cooperative robotics in the framework of the IoT. Such results have given rise to a series of contributions that can be summarized as follows: the proposal of a formal description for a new mission structure called strategy; the proposal of a new domain model for the management of missions; the proposal of a new Architectural Reference Model (ARM) for the design of mission management architectures called Common Mission Management Middleware Architecture (CoMMMA); the proposal of an equation to determine the balance of an adaptation in the scope of mission management; and the development of a specific mission management component for the architecture of the SWARMs European research project using the proposed CoMMMA ARM as a reference.

In this chapter we provide a comprehensive summary of the major conclusions and a selection of possible future lines of research that could be explored afterwards. The rest of the chapter is organized as follows. Section 7.1 summarizes and highlights the main conclusions drawn from the contributions provided in this dissertation, while some lines of further work are proposed in Section 7.2.

7.1 Conclusions

The use of advanced mission management systems for autonomous robots can be considered as an enabler for a better adaptation to new situations during the execution of a mission. In this thesis we have proposed the CoMMMA ARM, along with a Mission Management Domain Model and a novel strategy structure. Together, these proposals

– 211 – Conclusions and Future Works provide enough guidance for the design of specific architectures for mission management, as has been proved with the design, implementation and validation of the Mission and Tasks Register and Reporter (MTRR) mission manager for the SWARMs European project.

Furthermore, the combined use of the strategy structure, the Mission Management Domain Model, and the CoMMMA, enable a reliable and smart adaptation for the execution of a mission. Moreover, the use of the strategy model can be considered also as an enabler for an anticipative behavior. Indeed, according to [115], an anticipatory system is “a system containing a predictive model of itself and/or of its environment, which allows it to change state at an instant in accord with the model’s predictions pertaining to a later instant”. Consequently, the use of the strategy model can be considered as a predictive model as follows: for a given policy, deciding what action is the better option to achieve a goal can be also interpreted as changing state (performing an action) according to the model’s predictions pertaining to a later instant (according to the policy in order to achieve a goal). This behavior also matches the definition of anticipation given in [92] as “every kind of behavior which is influenced by some kind of knowledge, expectation, belief or intuition about the future”. Additionally, the use of the CoMMMA also enables the robotic agents participating in mission to improve their interaction levels, as described in the MAR for robotics in Europe [43], to at least Level 5, team coordination, even for robots that are only capable of Levels 1-3.

Regarding the core objectives described for this thesis in Section 1.2, the achievements of this thesis are summarized below.

First, a complete state-of-the-art study has been carried out in PartII, covering multiple concepts and technologies. These concepts and technologies were necessary to identify possible research and innovation opportunities for the adaptation of mission plans in the field of cooperative robotics for the IoT. Specifically, Chapter2 has explored the main topics of interests in regards to adaptive mission planning and management, and Chapter3 has surveyed a selection of mission management architectures taking into account the topics studied in the previous chapter.

In regards to the exploration carried out in Chapter2:

1. Section 2.2 explored the relevant AI planning paradigms, with the goal of better understanding how a planning problem is formally defined. In a first approximation, two ways of modeling a planning problem were identified: deterministic and non- deterministic. Additionally, and in a complementary way to both, the concepts of hierarchical task decomposition and opportunistic planning were also covered.

– 212 – 7.1 Conclusions

2. Section 2.3 studied the relevant planning description languages, used to describe both a planning problem and the model of the world in which that problem is framed, and which in the context ofAI planning is called domain. Beyond the historical approach to languages such as STRIPS and ADL, PDDL was identified as the reference language for the description of domains and planning problems.

3. Section 2.4 investigated the topic of adaptation. First, it identified a formal definition of adaptation to have a clear understanding of the concept. Subsequently, the different models commonly used in the design of adaptive systems were studied. The MAPE-K model was identified as the most relevant for CPSs, and was used to describe some decentralized control patterns for distributed adaptation.

4. Section 2.5 examined the relevant reference models for IoT systems. As an IoT system is primarily service-oriented, this section first described the SOA reference model, where the concepts for service orientation are defined. Next, the standardization efforts on IoT from ITU-T, ISO/IEC and IEEE were explored, in particular on ARMs. Finally, it describes the IoT ARM from the IoT-A European project, as it was the only full ARM available at the time of designing the proposal from this dissertation.

Chapter3 has performed a survey of a selection of mission management architectures that were developed in the field of cooperative robotics. This study compared the selected architectures in relation to the concepts and technologies explored in Chapter2. As a result, we found the following research opportunities:

1. Every studied architecture used their own mission plan structure, designed ad-hoc for it, and most of them relied on the use of deterministic approaches, delegating the adaptation to events to the robots’ capabilities. Consequently, there is a need for a new structure that encompasses the possibility of using sequences of actions as plans for deterministic situations together with policies based on decision- making functions that allow to plan the reaction to events, and do so following multiple decentralized patterns. It is convenient that this new structure also allows the hierarchical decomposition of the actions, as well as the consideration of opportunities following the opportunistic planning model.

2. In the same sense, every explored architecture was designed ad-hoc, without any common reference model identified. Also, these architectures were tightly coupled with the mission plan format decided for the architecture, and/or with the specific mission parsing and execution capabilities of the robots being used. Consequently, there is a need for a reference model that can provide guidance in the design

– 213 – Conclusions and Future Works

of specific mission management architectures, especially taking into account the possibility of using any decentralized adaptive pattern, and decoupling the mission parsing and execution capabilities of the robots from the mission management itself. Besides, this reference model should take into account the standardization efforts made in the scope of IoT, and be in line with them.

In line with the identified opportunities, this thesis presents two proposals: anovel information domain model for adaptive mission management for cooperative robotics, that includes the definition of a novel structure for a mission, and a common reference architecture for mission management that takes advantage of the aforementioned domain model and structure.

Part III included the detailed description of both proposals. Specifically, Chapter4 covered the description of the new information model and the associated new structure, and Chapter5 described the architectural reference model proposal in detail.

The first proposal started with the analysis of a basic scenario for a mission planning problem in Section 4.2, and with the step by step introduction of why it is necessary to define a new structure for a mission, and how it can be created. This new structure, which we have called strategy, is one of the contributions of this thesis, and was formally described in Section 4.3. Afterwards, the proposal presented the Strategy Domain Model in Section 4.4, where were defined the concepts associated with the strategy structure. This domain modal was later integrated as part of a novel Mission Management Domain Model, that is another contribution of this thesis, and that was described in Section 4.5. Finally, as one of the objectives of this thesis is to frame the adaptive mission planning for cooperative robotics in the IoT, the relationship of the new proposed Mission Management Domain Model with the IoT Domain Model was discussed in Section 4.6.

The second proposal introduced the needs for an ARM for adaptive mission management, with special attention to how the adaptation is integrated within the ARM. Section 5.2.1 introduced the concept of the balance of adaptation as a measure of the time required for a robot to adapt to an event during the execution of a mission. This concept takes into consideration not only whenever the robot is able to adapt to an event detected by itself, but also when the event is detected by some other entity in the system, or the robot has no direct capability for the adaptation. We identified two major contributors to this balance: the time due to communications, and the time due to the deliberative process for the event. The deliberative process is the one that is considered from the architectural point of view, as it can be improved by using a mission structure like the strategy proposed in Chapter4. Moreover, it can also be improved by embedding the characteristics of an adaptive system in the ARM, allowing for the use of different decentralized adaptation patterns.

– 214 – 7.1 Conclusions

The proposed ARM, called CoMMMA, uses the MAPE-K adaptive model and the IoT-A reference model as references. The MAPE-K has been used to define the architectural components directly related to mission management, with the intention to provide the core functionalities for an autonomic manager. On the other hand, the IoT-A reference model has been used to provide a common framework considering the robots as agents that can be represented as entities within the IoT-A Domain Model.

Continuing with the CoMMMA, Section 5.3 described the identified requirements, both functional and non-functional. The CoMMMA functional model was comprehensively described in Section 5.4, and the computational analysis was carried out in Section 5.5, including the use cases and interactions between the components of the architecture.

In summary, regarding both proposals, this thesis has provided the following contributions:

1. A new structure, called strategy, that encompasses the advantages of using sequences of actions (as classical plans), together with policies, identified as decision functions, for guiding the reaction to events, and with consideration to the hierarchical decomposition of actions, and opportunities evaluation.

2. A new domain model, called Mission Management Domain Model, that provides the formal definition of adaptive mission management related concepts, including those related to the use of the strategy structure.

3. A new architectural reference model, called CoMMMA, proposed for guiding the design of specific mission management architectures.

4. Additionally, this thesis has also presented the balance of adaptation as an equation that allows architecture designers and software engineers to evaluate the adaptation capabilities of a mission management architecture.

The CoMMMA proposal has been validated with the design, implementation, integration and analysis of a specific mission manager for the SWARMs European project, as described in Chapter6. In this chapter we illustrated the use of CoMMMA as a reference guide for the design of a specific mission manager component for the SWARMs architecture. To have a better understanding of the requirements, Section 6.2 provided an overview of the SWARMs architecture. Section 6.3 outlined in detail the specific design of the mission management component designed for the SWARMs architecture, and that was called MTRR. This description of the MTRR included information about the use of a virtual representation of the robotic vehicles used in the missions of the project, following both the IoT-A and CoMMMA references. It also described the interactions with

– 215 – Conclusions and Future Works the rest of the components of the SWARMs architecture, again using as a reference the CoMMMA proposal from this thesis.

The MTRR was integrated and validated in the final demonstration of the SWARMs project in Trondheim, Norway. The description of the validation details, including the validation goals, the scenario, the technical details, and the performed tests, was thoroughly described in Section 6.4. The obtained results were discussed in Section 6.5. From the MTRR point of view, these tests were focused on efficiency, measuring the MTRR processing times for each of its activities related to the management of the missions. In particular, the interest was to measure the possible impact of the MTRR on the aforementioned balance of adaptation, as it had to decide whether to dispatch the next action to a robotic vehicle according to its last status update and its vehicle plan. The obtained results showed promising figures, meaning that the impact of the MTRR on the whole system was negligible.

In conclusion, in this dissertation we have presented two proposals for improving the design of adaptive mission management architectures, thus contributing to the adaptive mission planning for cooperative robotics. And it has been done in line with the current standardization efforts in IoT architectures. As a result of both proposals we have achieved four contributions: A novel mission structure called strategy; A novel Mission Management Domain Model; A novel ARM called CoMMMA as a reference for the design of specific mission management architectures; and the formalization of the conceptof the balance of adaptation applied to adaptive mission planning. As a final result, the CoMMMA ARM has been used to design the MTRR, a specific mission manager for the SWARMs European project architecture. And finally, with the results obtained during the validation of this specific mission manager we have verified that following the CoMMMA ARM it is possible to design mission managers with negligible impact on the balance of adaptation.

7.2 Future Works

Based on the above conclusions and considering the limitations of the current work, future research can be carried out in the following main areas: (i) Strategy structure and domain (ii) Mission management domain (iii) CoMMMA formalization, (iv) CoMMMA uses and validation, (v) CoMMMA functional groups and components, (vi) Other areas.

(i) For the strategy structure and domain:

• The strategy structure and domain model presented in Section 4.4 can be expanded to accommodate other execution possibilities in addition to the proposed ones. For instance, it is quite common to define fail-safe mech-

– 216 – 7.2 Future Works

anisms for autonomous robots. The fail-safe mechanisms can be defined as fixed mechanisms for the type of robot and its possible actions, oras specific mechanisms for the mission to be executed. It is the last typethat can be considered to be integrated in the strategy structure. In fact, with the proposed definition for the strategy structure, fail-safe mechanisms could already be defined as opportunistic plans. However, given that failures are not exactly opportunities, it might be of interest to explore the definition of a specific concept for fail-safe mechanisms in the strategy domain model, with its corresponding part in the strategy structure. • Another interesting line of work in relation to the strategy structure proposal is how to generate a strategy automatically. As already mentioned in Sec- tion 5.4.2, an option is to use plan aggregation, like it was used for the generation of policies in MDP in [168]. In a separate work a similar approach was explored in [169] for the generation of decision trees by aggregating the results from an automated planning algorithm. And in [205] the authors propose a similar technique for the generation of a behavior tree. Consequently, there seems to be an interest in similar techniques that could be useful for the automatic generation of strategies.

(ii) For the mission management domain:

• One of the conclusions that we can extract from the work carried out during this dissertation is that there is no reference specification for the description of a mission, resulting in each mission management solution to define their own ad-hoc mission description, as it occurred in SWARMs. However, missions for cooperative agents share some commonalities, and consequently it seems interesting the effort on defining a common general specification for mission descriptions, including the mission plans and other related and relevant information. This common general specification could be explored as an extension for the Mission Management Domain model proposed in Section 4.5. Similarly, either separately or as a part derived from the previous idea, the development of a common mission description language could also be explored. In both cases, a new formal mission specification is required to ease reusability, and to provide the enhanced management modes required for the IoT.

(iii) For the CoMMMA formalization:

• The CoMMMA proposal is designed to be used in the development of middleware solutions for the management of missions for cooperative robotics

– 217 – Conclusions and Future Works

in the framework of the IoT. Consequently, the IoT ARM of the IoT-A project has been used as a guide in the design of the CoMMMA proposal. However, since the beginning of the works in the definition of the CoMMMA proposal, progress has been made by others in the standardization of a reference architecture for IoT. That is the case of the ISO/IEC 30141:2018 standard [206], currently in force, and being reviewed for an upcoming update. Therefore, an interesting line of future work would be the review of CoMMMA compliance with the ISO/IEC 30141:2018 standard, and its upgrade and adaptation to it if necessary. • The CoMMMA proposal has been described in this dissertation using a generic functional view and the description of its main workflows. Another interesting line of future work could be the formalization of an architectural description of the CoMMMA proposal using the ISO/IEC/IEEE:2011 42010 [207] standard. • Along the same lines as the previous point, it could also be of interest to work on the description of the CoMMMA proposal using Architecture Analysis and Design Language (AADL)[208–210]. AADL is an analysis and design language created to address the common problems in the development of software for mission- and safety-critical systems. Having an AADL description of the CoMMMA proposal could be of interest for its use in the design of such systems.

(iv) For the CoMMMA uses and validation:

• Being conceived as a reference architecture, one of the lines of work of interest is to explore the design of more mission management architectures using CoMMMA as a guide, and use the obtained results as feedback to improve the reference model. In that line we are currently working in the design, implementation and validation of a mission management middleware solution for the AFarCloud European project. This solution is specific for the AFarCloud requirements, and therefore it has notable differences with the solution developed previously for the SWARMs European project. In particular, the new solution has to manage different types of robotic vehicles with different types of mission plans, resembling the strategy proposal from this dissertation. • The solution developed for the SWARMs European project, whose design and validation is included in Chapter6, used a decentralized hierarchical control pattern for adaptation. However, the adaptation performed from the mission manager was manually addressed, with no alternative branches of

– 218 – 7.2 Future Works

execution previously defined as part of the mission plan. Consequently, it seems of interest to explore new designs and implementations based on the CoMMMA proposal in order to test the efficiency and efficacy of a strategic approach to improve the balance of adaptation as described in Section 5.2.1.

(v) For the CoMMMA functional groups and components:

• Regarding the Strategy Management Functional Group: – An interesting line of work not explored in this dissertation is on the possible techniques for managing strategies. In Chapter5 it was mentioned that both classical plans and strategies could be stored in dedicated repositories for further use. Given that missions that are carried out in the same scenario are usually repeated, it is reasonable to think about the possibility of reusing missions previously executed. This idea resembles the Case-Based Reasoning (CBR) process. Thus, managing strategies could be defined as a CBR-based management. A CBR process is usually formalized as a continuous loop of four steps: retrieve, reuse, revise and retain [175, 211]. Then, further exploration of a CBR-based strategy management opens up several research opportunities, beginning with the definition of a strategy representation method to enabling all four steps of a CBR, and followed by the investigation on each of the steps. – Another interesting line of work is on the update of the strategies during the execution of a mission to cope with new situations. Initially these updates are expected to be done by hand, but it could be worth exploring the use of machine learning techniques that can at least help with the manual updating, and at best do it automatically. An interesting approach to take into consideration is the Planning, Execution and Learning Ar- chitecture [212] (PELA), where a loop is defined to upgrade the action model of the planning component. A similar idea could be adapted to upgrade a strategy. • Regarding the Abstract Agents Functional Group: – Section 5.4.4 described the possibility of using discovery capabilities for handling the generation and actualization of virtual agents and their corresponding available mission management related services. The mission manager designed and implemented for the SWARMs European project, the MTRR, used internally a virtual representation of the agents. However, this representation was fixed, and no discovery was carried out at all. Therefore, an interesting line of work for future research could

– 219 – Conclusions and Future Works

be to explore the discovery functionalities and techniques applied to a mission management architecture. • Regarding the rest of the CoMMMA functional components and groups, this dissertation has focused mainly on those directly related to mission management, and partially on those related to strategy management. Therefore, the exploration of the rest of the components included in the CoMMMA proposal remains open.

(vi) For other areas related to the objectives of this dissertation:

• An idea that could be of interest for future lines of work is in relation to the hierarchical decomposition of actions. So far it has been considered in this dissertation that the decomposition of actions is carried out for those agents who are unable to execute more complex actions. However, other possibilities worth exploring would be:

– Using agents capable of learning fixed sequences of actions as new compound actions. In this case, if a compound action is always decomposed in the same sequence of primitive actions, then it would be only required to send the fixed sequence once, with a label association tothe corresponding compound action name. – Using agents capable of using parameterized actions. In this case, if a compound action can be parameterized, then it would be only required to send the parameters for the decomposition of the action at the agent.

• IoT systems commonly use ontologies for representing the models of the world used by those systems.AI planning systems commonly use PDDL for representing the model of the world associated with the planning problem. It has already been tried to resolve this divergence with some proposals such as [213]. However, this question still remains open, and could be an interesting topic for further research.

– 220 – Part VI

Appendix

Appendix A

Published Works and Dissemination Activities

Ich habe nichts dagegen wenn Sie langsam denken, Herr Doktor, aber ich babe etwas dagegen wenn Sie rascher publizieren als denken

(I do not mind if you think slowly, doctor, but I would mind if you publish faster than you think). — Wolfgang Pauli

This appendix provides a comprehensive list of all the scientific publications, research projects and other results that can be considered as part of this thesis. They are spread over a period of several years, from the first to the last. Currently there are other publications in progress that are yet to be finished and submitted to Journal Citation Reports (JCR) indexed journals for their consideration.

A.1 List of Publications

Below is a list of the related scientific papers that have been published in JCR indexed journals. Each reference is completed with the JCR impact factor for the year of publication and its associate quartile, the Digital Object Identifier (DOI) reference, and a brief description of how each paper provides a contribution to the thesis.

1. Lucas Martínez, N.; Martínez Ortega, J.F.; Hernández Díaz, V., “Virtualization of Event Sources in Wireless Sensor Networks for the Internet of Things”, Sensors, vol. 14, no. 12, pp. 22737-22753, 2014. DOI: 10.3390/s16050684. Impact Factor: 2.245 (Q1) [130].

– 223 – Published Works and Dissemination Activities

2. Hernández Díaz, V.; Martínez Ortega, J.F.; Lucas Martínez, N.; Del Toro Mata- moros, R.M., “Self-Adaptive Strategy Based on Fuzzy Control Systems for Im- proving Performance in Wireless Sensors Networks”, Sensors, vol. 15, no.9, pp. 24125-24142, 2015. DOI: 10.3390/s150924125. Impact Factor: 2.033 (Q1) [214].

3. Li, X.; Martínez Ortega, J.F.; Rodríguez-Molina, J.; Lucas Martínez, N., “A Survey on Intermediation Architectures for Underwater Robotics”, Sensors, vol. 16, no. 2, p. 190, 2016. DOI: 10.3390/s16020190. Impact Factor: 2.677 (Q1) [215].

4. Lucas Martínez, N.; Martínez Ortega, J.F.; Hernández Díaz, V.; Del Toro Mata- moros, R.M., “Communication Range Dynamics and Performance Analysis for a Self-Adaptive Transmission Power Controller”, Sensors, vol. 16, no. 5, p. 684, 2016. DOI: 10.3390/s16050684. Impact Factor: 2.677 (Q1) [216].

5. Zhai, Z.; Martínez Ortega, J.F.; Lucas Martínez, N.; Rodríguez-Molina, J., “A Mis- sion Planning Approach for Precision Farming Systems Based on Multi-Objective Optimization”, Sensors, vol. 18, no. 6, p. 1795, 2018. DOI: 10.3390/s18061795. Impact Factor: 3.031 (Q1) [217].

6. Zhai, Z.; Martínez Ortega, J.F.; Lucas Martínez, N.; Castillejo, P., “A Rule-Based Reasoner for Underwater Robots Using OWL and SWRL”, Sensors, vol. 18, no. 10, p. 3481, 2018. DOI: 10.3390/s18103481. Impact Factor: 3.031 (Q1) [193].

7. Zhai, Z.; Martínez Ortega, J.F.; Beltrán, V.; Lucas Martínez, N., “An Associated Representation Method for Defining Agricultural Cases in a Case-Based Reasoning System for Fast Case Retrieval”, Sensors, vol. 19, no. 23, p. 5118, 2019. DOI: 10.3390/s19235118. Impact Factor: 3.275 (Q1) [211].

8. Lucas Martínez, N.; Martínez Ortega, J.F.; Rodríguez-Molina, J.; Zhai, Z., “Pro- posal of an Automated Mission Manager for Cooperative Autonomous Underwater Vehicles”, Applied Sciences, vol. 10, no. 3, p. 855. DOI: 10.3390/app10030855. Impact Factor: 2.474 (Q2) [For 2019] [185].

9. Zhai, Z.; Martínez Ortega, J.F.; Beltrán, V.; Lucas Martínez, N., “Decision Support Systems for Agriculture 4.0: Survey and Challenges”, Computers and Electronics in Agriculture, vol. 170, p. 105256, 2020. DOI: 10.1016/j.compag.2020.105256. Impact Factor: 3.858 (Q1) [For 2019] [218].

10. Lucas Martínez, N.; Martínez Ortega, J.F.; Castillejo, P.; Beltrán Martínez, V., “Survey of Mission Planning and Management Architectures for Underwater Co- operative Robotics Operations”, Applied Sciences, vol. 10, no. 3, p. 1086, 2020. DOI: 10.3390/app10031086. Impact Factor: 2.474 (Q2) [For 2019] [127].

– 224 – A.2 Congress and Conference Papers

11. Zhai, Z.; Martínez Ortega, J.F.; Lucas Martínez, N.; Hernández Díaz, V., “Apply- ing case-based reasoning and a learning-based adaptation strategy to irrigation scheduling in grape farming”, Computers and Electronics in Agriculture, vol. 178, 105741, 2020. DOI: 10.1016/j.compag.2020.105741. Impact Factor: 3.858 (Q1) [For 2019] [219].

12. Zhai, Z.; Martínez Ortega, J.F.; Lucas Martínez, N.; Xu, H., “An Efficient Case Retrieval Algorithm for Agricultural Case-Based Reasoning Systems, with Consid- eration of Case Base Maintenance”, Agriculture, vol. 10, no. 9, p. 387, 2020. DOI: 10.3390/agriculture10090387. Impact Factor: 2.072 (Q2) [For 2019] [218].

A.2 Congress and Conference Papers

The following contributions have been made to conferences regarding topics related to the scope of this thesis:

1. Lucas Martínez, N.; Martínez Ortega, J.F.; Hernández Díaz, V.; Virtualization of event sources in Wireless Sensor Networks for the Internet of Things. 1st International Electronic Conference on Sensors and Applications (ECSA), 1-16 June 2014. DOI: 10.3390/ecsa-1-d009.

2. Lucas Martínez, N.; Martínez Ortega, J.F.; Hernández Díaz, V.; Communication Range Dynamics Using an Energy Saving Self-Adaptive Transmission Power Controller in a Wireless Sensor Network. 2nd International Electronic Conference on Sensors and Applications (ECSA), 15-30 November 2015. DOI: 10.3390/ecsa- 2-D006.

A.3 Participation in Research Projects

As it was already mentioned in Chapter1, this thesis has been done in the context of the following European research projects:

1. WoO: Web of Objects (ITEA2) [64] was set out to address specific issues relating to the increasing integration of internet-connected devices in existing business applications, proposing a modular solution kit to enable the development of industrial and consumer applications with smart objects as actors, across multiple layers from object nodes towards user application [63]. Device virtualization and service composition were explored in this project, giving the insights for the agent virtualization and the hierarchical composition of actions in a mission plan that are used in this thesis.

– 225 – Published Works and Dissemination Activities

2. DEMANES (ARTEMIS) [62] aimed to provide component-based methods, framework and tools for the development of runtime adaptive systems, enabling them to react to changes in themselves, in their environment and in user needs [61]. The use of self-adaptive systems was explored in this project, serving as the foundations for self-adaptive architectural designs.

3. ACCUS (ARTEMIS) [60] aimed to provide an integration and coordination platform for urban systems to ease the deployment of cross-domain application/services to optimize the combined performance of interacting urban processes (e.g. environment, energy, traffic, security, etc.) [59].

4. SWARMs: Smart and Networking Underwater Robots in Cooperation Meshes (ECSEL) [58] primary goal was to expand the use of underwater and surface vehicles (AUVs, ROVs, USVs) to facilitate the conception, planning and execution of maritime and offshore operations and missions [57].

5. AFarCloud (ECSEL) [56] addresses the urgent need for a holistic and systematic approach for precision farming, and will provide a distributed platform for autonomous farming, which will allow the integration and cooperation of CPSs in real-time for increased agriculture efficiency, productivity, animal health, food quality and reduced farm labor costs [55].

A.4 Technological Transfer

In addition to the scientific publications, the works that contributed to this thesis have also provided some technological transfer in the form of registered software as original contributions.

A.4.1 Self-Adaptive Transmission Power for the Internet of Things

The adaptive controller that was developed for the DEMANES European research project has been registered as an original contribution under the denomination “Self-Adaptive Transmission Power for the Internet of Things”, with the registration request number M-007848/2015, and record number 16/2016/2001. The documentation required for the registration, shown in Figure A.2, clearly identifies the author of this thesis asan active co-operator in the design, implementation and testing of this software. Additionally, this software has been released as an open source project available at the DEMANES GitHub repository[220, 221].

– 226 – A.4 Technological Transfer

Figure A.1. Register for the intellectual property of the “Self-Adaptive Transmission Power for the Internet of Things.”

A.4.2 Software para la transferencia de información entre aplicaciones de control y vehículos acuáticos

The mission manager component developed for the SWARMs European project, that is, the MTRR, has been registered as an original contribution under the denomination of “Software para la transferencia de información entre aplicaciones de control y vehículos acuáticos” (Software for information transferring between control applications and aquatic vehicles), with the registration request number M-007945/2018, and record number 16/2019/2069.

Figure A.2. Register for the intellectual property of “Software para la transferencia de información entre aplicaciones de control y vehículos acuáticos.”

– 227 – Published Works and Dissemination Activities

A.5 Other Merits

A.5.1 Final Degree Projects Guidance

In relation to some of the topics developed during the research work associated with this doctoral thesis, the author has provided some guidance to the following Final Degree Project:

• Jorge García Paredes. “Anticipative System for a Planner through Machine Learn- ing Techniques”. Defended on July 23, 2018, at E.T.S.I. y Sistemas de Telecomu- nicación (UPM), for the Ingeniería Telemática y Electrónica Department (DTE).

A.5.2 Pre-doctoral Contract of the UPM

During the development of this thesis, the author was granted with a pre-doctoral contract of the UPM.

Figure A.3. UPM’s own program predoctoral contract credential.

– 228 – Glossary

Agent From [172]: “An agent is an element that performs some task on behalf of some party (i.e. a user, machine, application, or another agent) rather than having the party itself perform the task”.

Anticipation From [92]: “Every kind of behavior which is influenced by some kind of knowledge, expectation, belief or intuition about the future”.

Anticipatory system From [115]: “A system containing a predictive model of itself and/or of its environment, which allows it to change state at an instant in accord with the model’s predictions pertaining to a later instant”.

Architecture In [222], an architecture is defined as the “fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution”.

Artificial Intelligence According to [223] artificial intelligence is an “interdisciplinary field, usually re- garded as a branch of computer science, dealing with models and systems for the performance of functions generally associated with human intelligence, such as reasoning and learning”.

Autonomous robot See semi-autonomous robot and fully autonomous robot.

– 229 – Glossary

Autonomous Underwater Vehicle (AUV) According to [224] an autonomous underwater vehicle, or AUV, is “a self-propelled, unmanned, untethered underwater vehicle capable of carrying out simple activities with little or no human supervision”.

Context From [225]: “The information that can be used to characterize the environment of a user. Context information may include where the user is, what resources (devices, access points, noise level, bandwidth, etc.) are near the user, at what time the user is moving, interaction history between person and objects, etc. According to specific applications, context information can be updated”.

Context Awareness From [226]: “Context awareness is a capability to determine or influence a next action in telecommunication or process by referring to the status of relevant entities, which form a coherent environment as a context”.

Fully autonomous robot According to [126] a fully autonomous robot is “a role for a robot performing a given task in which the robot solves the task without human intervention while adapting to operational and environmental conditions”.

Internet of Things From [34]: “A global infrastructure for the information society, enabling advanced services by interconnecting (physical and virtual) things based on existing and evolving interoperable information and communication technologies”.

Internet of Things Architecture (IoT-A) According to [200], the Internet of Things Architectural Reference Model (IoT-A) “can be visualized as the ‘Matrix’ that eventually gives birth ideally to all concrete architectures”.

Middleware From [172]: “The mediating entity between two information elements. Such an element can be, for example, an application, infrastructure component, or another mediating entity”. In [227] a middleware is defined as “a software layer between applications and operating systems that provides application programmers with higher level of abstractions, such as remote procedure invocation, reliable message exchange or transactions”.

– 230 – Glossary

Robot According to [228] a robot is “a device with a processing unit often accompanied by sensors and actuators that can work with real-world phenomena and entities” [126] also defines a robot as “an agentive device (Agent and Device in Suggested Upper Merged Ontology (SUMO)) in broad sense, purposed to act in the physical world in order to accomplish one or more tasks. In some cases, the actions of a robot might be subordinated to actions of other agents (Agent in SUMO), such as software agents (bots) or humans. A robot is composed of suitable mechanical and electronic parts. Robots might form social groups, where the interact to achieve a common goal. A robot (or a group of robots) can form robotic systems together with special environments geared to facilitate their work”.

Semi-autonomous robot According to [126] a semi-autonomous robot is “a role for a robot performing a given task which the robot and a human operator plan and conduct the task, requiring various levels of human interaction.”.

Service Oriented Architecture (SOA) According to [116], a Service Oriented Architecture (SOA) “is a paradigm for organizing and utilizing distributed capabilities that may be under the control of different ownership domains”. The key concepts for this paradigm are: visibility, interaction and effect.

Thing From [34]: “With regard to the Internet of Things, this is an object of the physical world (physical things) or the information world (virtual things), which is capable of being identified and integrated into communication networks”.

Unmanned Surface Vehicle (USV) An unmanned surface vehicle, or USV, is, in a similar way to an AUV, a self- propelled, unmaned surface vehicle capable of carrying out simple activities with little or no human supervision.

– 231 –

Bibliography

[1] Pedro Patrón and Yvan R. Petillot. “The underwater environment: A challenge for planning”. In: 27th Workshop of the UK PLANNING AND SCHEDULING Special Interest Group, PlanSIG. Dec. 2008. URL: http://www.macs.hw.ac.uk/~ruth/ plansig08/ukplansig09_submission_21.pdf (visited on 2020-12-15).

[2] Barbara Fletcher. “UUV master plan: a vision for navy UUV development”. In: OCEANS 2000 MTS/IEEE Conference and Exhibition. Conference Proceedings. Vol. 1. IEEE, Sept. 2000, pp. 65–71. ISBN: 0-7803-6551-8. DOI: 10.1109/OCEANS. 2000.881235. URL: http://ieeexplore.ieee.org/document/881235/.

[3] Jinyeong Heo, Junghoon Kim, and Yongjin Kwon. “Technology Development of Unmanned Underwater Vehicles (UUVs)”. In: Journal of Computer and Commu- nications 05.07 (May 2017), pp. 28–35. ISSN: 2327-5219. DOI: 10.4236/jcc.2017. 57003. URL: http://www.scirp.org/journal/doi.aspx?DOI=10.4236/jcc.2017.57003.

[4] Kwang Sub Song. “Design of Large Diameter Mine Countermeasure Hybrid Power Unmanned Underwater Vehicle”. In: Autonomous Vehicle. InTech, Sept. 2016. DOI: 10.5772/63795. URL: http://www.intechopen.com/books/autonomous- vehicle / design - of - large - diameter - mine - countermeasure - hybrid - power - unmanned-underwater-vehicle.

[5] Chris Kaminski et al. “12 days under ice – an historic AUV deployment in the Canadian High Arctic”. In: 2010 IEEE/OES Autonomous Underwater Vehicles. IEEE, Sept. 2010, pp. 1–11. ISBN: 978-1-61284-980-5. DOI: 10.1109/AUV.2010. 5779651. URL: http://ieeexplore.ieee.org/document/5779651/.

[6] Lionel Camus et al. “Autonomous surface and underwater vehicles reveal new discoveries in the Arctic Ocean”. In: OCEANS 2019 - Marseille. IEEE, June 2019, pp. 1–8. ISBN: 978-1-7281-1450-7. DOI: 10.1109/OCEANSE.2019.8867181. URL: https://ieeexplore.ieee.org/document/8867181/.

[7] Gerardo G. Acosta, Oscar A. Calvo Ibañez, Hugo. J. Curti, and Alejandro F. Rozenfeld. “Low-cost Autonomous Underwater Vehicle for pipeline and cable inspections”. In: 2007 Symposium on Underwater Technology and Workshop

– 233 – Bibliography

on Scientific Use of Submarine Cables and Related Technologies. IEEE, Apr. 2007, pp. 331–336. ISBN: 1-4244-1207-2. DOI: 10.1109/UT.2007.370799. URL: http://ieeexplore.ieee.org/document/4231129/. [8] Byrel Mitchell, Nina Mahmoudian, and Guy Meadows. “Autonomous underwater pipeline monitoring navigation system”. In: SPIE, Automatic Target Recognition XXIV. Vol. 9090. June 2014. DOI: 10.1117/12.2055178. URL: http://proceedings. spiedigitallibrary.org/proceeding.aspx?doi=10.1117/12.2055178. [9] Imad Jawhar, Nader Mohamed, Jameela Al-Jaroodi, and Sheng Zhang. “An Architecture for Using Autonomous Underwater Vehicles in Wireless Sensor Networks for Underwater Pipeline Monitoring”. In: IEEE Transactions on Industrial Informatics 15.3 (Mar. 2019), pp. 1329–1340. ISSN: 1551-3203. DOI: 10.1109/TII. 2018.2848290. URL: https://ieeexplore.ieee.org/document/8387464/. [10] Arne R. Diercks et al. “NIUST - Deepwater horizon oil spill response cruise”. In: OCEANS 2010 MTS/IEEE SEATTLE. IEEE, Sept. 2010, pp. 1–7. ISBN: 978-1- 4244-4332-1. DOI: 10.1109/OCEANS.2010.5664443. URL: http://ieeexplore.ieee. org/document/5664443/. [11] Amit Shukla and Hamad Karki. “Application of robotics in offshore oil and gas industry— A review Part II”. In: Robotics and Autonomous Systems 75 (Jan. 2016), pp. 508–524. ISSN: 09218890. DOI: 10.1016/j.robot.2015.09.013. URL: https://linkinghub.elsevier.com/retrieve/pii/S0921889015002018. [12] Bingxi Li, Barzin Moridian, and Nina Mahmoudian. “Autonomous Oil Spill Detec- tion: Mission Planning for ASVs and AUVs with Static Recharging”. In: OCEANS 2018 MTS/IEEE Charleston. IEEE, Oct. 2018, pp. 1–5. ISBN: 978-1-5386-4814- 8. DOI: 10.1109/OCEANS.2018.8604490. URL: https://ieeexplore.ieee.org/ document/8604490/.

[13] Precision farming - European Commission. URL: https://ec.europa.eu/eip/ agriculture/en/digitising-agriculture/developing-digital-technologies/precision- farming-0 (visited on 2021-01-02). [14] Sourav Kumar Bhoi et al. “An Internet of Things assisted Unmanned Aerial Vehicle based artificial intelligence model for rice pest detection”. In: Microprocessors and Microsystems 80 (Feb. 2021), p. 103607. ISSN: 01419331. DOI: 10.1016/ j . micpro . 2020 . 103607. URL: https : / / linkinghub . elsevier. com / retrieve / pii / S0141933120307560. [15] Achilles D. Boursianis et al. “Internet of Things (IoT) and Agricultural Unmanned Aerial Vehicles (UAVs) in smart farming: A comprehensive review”. In: Internet of Things (Mar. 2020), p. 100187. ISSN: 25426605. DOI: 10.1016/j.iot.2020.100187. URL: https://linkinghub.elsevier.com/retrieve/pii/S2542660520300238.

– 234 – Bibliography

[16] Sachin Vishnudas Pathak, Atul G Mohod, and Abhiman Arjun Sawant. “Review on effective role of UAV in precision farming”. In: Journal of Pharmacognosy and Phytochemistry 9.4 (June 2020), pp. 463–467. ISSN: 22784136. URL: https: //www.phytojournal.com/archives/2020/vol9issue4/PartF/9-3-36-382.pdf (visited on 2021-01-03).

[17] Pablo Gonzalez-De-Santos et al. “Unmanned Ground Vehicles for Smart Farms”. In: Agronomy - Climate Change and Food Security. IntechOpen, July 2020. DOI: 10.5772/intechopen.90683. URL: https://www.intechopen.com/books/agronomy- climate-change-food-security/unmanned-ground-vehicles-for-smart-farms.

[18] Pedro Castillejo et al. “The AFarCloud ECSEL Project”. In: 2019 22nd Euromicro Conference on Digital System Design (DSD). IEEE, Aug. 2019, pp. 414–419. ISBN: 978-1-7281-2862-7. DOI: 10.1109/DSD.2019.00066. URL: https://ieeexplore. ieee.org/document/8875170/.

[19] Pedro Castillejo et al. “Aggregate Farming in the Cloud: The AFarCloud ECSEL project”. In: Microprocessors and Microsystems 78 (Oct. 2020), p. 103218. ISSN: 01419331. DOI: 10.1016/j.micpro.2020.103218. URL: https://linkinghub.elsevier. com/retrieve/pii/S0141933120303793.

[20] Alec Momont. Ambulance Drone. URL: https://www.tudelft.nl/en/ide/research/ research-labs/applied-labs/ambulance-drone/ (visited on 2021-01-02).

[21] Patrick Van de Voorde et al. “The drone ambulance [A-UAS]: golden bullet or just a blank?” In: Resuscitation 116 (July 2017), pp. 46–48. ISSN: 03009572. DOI: 10.1016/j.resuscitation.2017.04.037. URL: https://linkinghub.elsevier.com/retrieve/ pii/S0300957217301995.

[22] Robin R. Murphy, Satoshi Tadokoro, and Alexander Kleiner. “Disaster Robotics”. In: Springer Handbook of Robotics. Cham: Springer International Publishing, 2016, pp. 1577–1604. DOI: 10 . 1007 / 978 - 3 - 319 - 32552 - 1 _ 60. URL: http : //link.springer.com/10.1007/978-3-319-32552-1_60.

[23] Yasuki Miyazaki et al. “Acquisition of Disaster Emergency Information Using a Terrain Database by Flying Robots”. In: Journal of Robotics and Mechatronics 30.3 (June 2018), pp. 443–452. ISSN: 1883-8049. DOI: 10.20965/jrm.2018.p0443. URL: https://www.fujipress.jp/jrm/rb/robot003000030443.

[24] Koki Yakushiji et al. “Short-Range Transportation Using Unmanned Aerial Vehicles (UAVs) during Disasters in Japan”. In: Drones 4.4 (Oct. 2020), p. 68. ISSN: 2504- 446X. DOI: 10 . 3390 / drones4040068. URL: https : / / www. mdpi . com / 2504 - 446X/4/4/68.

– 235 – Bibliography

[25] The value of robotics in a pandemic. Dec. 2020. URL: https://www.eu-robotics. net/robotics_week/newsroom/press/powerful- tech- talented- people.html? changelang=4 (visited on 2021-01-01).

[26] Vital Intelligence - Draganfly - Touchless Health Monitoring Technology. URL: https://draganfly.com/products/vital-intelligence/ (visited on 2020-12-22).

[27] Ibrahim Haleem Khan and Mohd Javaid. “Automated COVID-19 emergency response using modern technologies”. In: Apollo Medicine 17 (5 Aug. 2020). ISSN: 0976-0016. DOI: 10.4103/am.am_68_20. URL: https://www.apollomedicine. org/article.asp?issn=0976-0016;year=2020;volume=17;issue=5;spage=58; epage=61;aulast=Khan.

[28] Robot takes contact-free measurements of patients’ vital signs | MIT News | Massachusetts Institute of Technology. URL: https://news.mit.edu/2020/spot- robot-vital-signs-0831 (visited on 2020-12-29).

[29] Adarsh Kumar et al. “A drone-based networked system and methods for combating coronavirus disease (COVID-19) pandemic”. In: Future Generation Computer Systems 115 (Feb. 2021), pp. 1–19. ISSN: 0167739X. DOI: 10.1016/j.future.2020. 08.046. URL: https://linkinghub.elsevier.com/retrieve/pii/S0167739X20317064.

[30] Moez Guettari, Ines Gharbi, and Samir Hamza. “UVC disinfection robot”. In: Environmental Science and Pollution Research (Oct. 2020). ISSN: 0944-1344. DOI: 10.1007/s11356-020-11184-2. URL: http://link.springer.com/10.1007/ s11356-020-11184-2.

[31] Christine R. Kovach et al. “Evaluation of an ultraviolet room disinfection protocol to decrease nursing home microbial burden, infection and hospitalization rates”. In: BMC Infectious Diseases 17.1 (Dec. 2017), p. 186. ISSN: 1471-2334. DOI: 10.1186/s12879- 017- 2275- 2. URL: http://bmcinfectdis.biomedcentral.com/ articles/10.1186/s12879-017-2275-2.

[32] Guang-Zhong Yang et al. “Combating COVID-19—The role of robotics in managing public health and infectious diseases”. In: Science Robotics 5.40 (Mar. 2020), eabb5589. ISSN: 2470-9476. DOI: 10.1126/scirobotics.abb5589. URL: https://robotics.sciencemag.org/lookup/doi/10.1126/scirobotics.abb5589.

[33] Robin R. Murphy, Vignesh Babu Manjunath Gandudi, and Justin Adams. “Appli- cations of robots for COVID-19 response”. In: (2020), p. 7. arXiv: 2008.06976. (preprint).

[34] ITU Telecommunication Standardization Sector (ITU-T). Recommendation ITU-T Y.2060: Overview of the Internet of things. ITU Telecommunication Standardiza-

– 236 – Bibliography

tion Sector (ITU-T), 2012, p. 22. URL: https://www.itu.int/rec/T-REC-Y.2060- 201206-I/en (visited on 2020-12-01).

[35] ISO/IEC JTC 1/SC 41. ISO - ISO/IEC 20924:2018 - Information technology — Internet of Things (IoT) — Vocabulary. ISO/IEC. Dec. 2018, p. 7. URL: https: //www.iso.org/standard/69470.html (visited on 2021-01-02).

[36] Tom Keeley. IoT to IoAT: Internet of Autonomous Things devices provides solutions. Apr. 2016. URL: https://www.controleng.com/articles/iot-to-ioat-internet-of- autonomous-things-devices-provides-solutions/ (visited on 2020-01-20).

[37] Ovidiu Vermesan and Joël Bacquet, eds. Cognitive Hyperconnected Digital Transformation. River Publishers, 2017, p. 310. ISBN: 9788793379824. DOI: 10.13052/rp-9788793609105. URL: http://riverpublishers.com/dissertations_xml/ 9788793609105/9788793609105.xml.

[38] Ovidiu Vermesan et al. “The next generation internet of things-Hyperconnectivity and embedded intelligence at the edge”. In: Next Generation Internet of Things: Distributed Intelligence at the Edge and Human Machine-to-Machine Cooperation. 2018, pp. 19–102. ISBN: 9788770220071. URL: http://hdl.handle.net/11250/ 2575161.

[39] Partha Pratim Ray. “Internet of Robotic Things: Concept, Technologies, and Challenges”. In: IEEE Access 4 (2016), pp. 9489–9500. ISSN: 2169-3536. DOI: 10.1109/ACCESS.2017.2647747. URL: http://ieeexplore.ieee.org/document/ 7805273/.

[40] SPARC. Strategic Research Agenda for Robotics in Europe 2014-2020 (Slide). 2014, p. 108. URL: https://www.eu-robotics.net/sparc/upload/topic_groups/ SRA2020_SPARC.pdf (visited on 2021-01-03).

[41] EPoSS. Strategic Research Agenda. June 2019. URL: https://www.ecsel.eu/sites/ default/files/2019-02/ECS-SRA%202019%20FINAL.pdf (visited on 2021-01-03).

[42] BDVA et al. Strategic Research Innovation and Deployment Agenda (SRIDA) for the AI, Data and Robotics Partnership. Sept. 2020. URL: https://www.eu-robotics. net/cms/upload/downloads/ppp- documents/AI- Data- Robotics- Partnership- SRIDA-V3.0.pdf (visited on 2021-01-03).

[43] SPARC. Robotics 2020 Multi-Annual Roadmap. Dec. 2016. URL: https://www. eu- robotics.net/cms/upload/topic_groups/H2020_Robotics_Multi- Annual_ Roadmap_ICT-2017B.pdf (visited on 2020-12-15).

– 237 – Bibliography

[44] Mangal Kothari et al. “Robust Mission Planning for Underwater Applications: Issues and Challenges”. In: IFAC Proceedings Volumes 45.5 (Apr. 2012). Ed. by Pereira Fernando, pp. 223–229. ISSN: 14746670. DOI: 10.3182/20120410-3-PT- 4028.00037. URL: https://linkinghub.elsevier.com/retrieve/pii/S1474667016306061.

[45] Enrique Fernández Perdomo, Jorge Cabrera Gámez, Antonio Carlos Domínguez Brito, and Daniel Hernández Sosa. “Mission specification in underwater robotics”. In: Journal of Physical Agents (JoPha) 4.1 (2010), pp. 25–33. ISSN: 1888-0258. DOI: 10.14198/JoPha.2010.4.1.05. URL: http://hdl.handle.net/10045/13293.

[46] David Kortenkamp and Reid Simmons. “Robotic Systems Architectures and Programming”. In: Springer Handbook of Robotics. Berlin, Heidelberg: Springer Berlin Heidelberg, 2008, pp. 187–206. DOI: 10.1007/978-3-540-30301-5_9. URL: http://link.springer.com/10.1007/978-3-540-30301-5_9.

[47] Rodney Brooks. “A robust layered control system for a mobile robot”. In: IEEE Journal on Robotics and Automation 2.1 (Mar. 1986), pp. 14–23. ISSN: 0882-4967. DOI: 10.1109/JRA.1986.1087032. arXiv: 1010.0034. URL: http://ieeexplore.ieee. org/lpdocs/epic03/wrapper.htm?arnumber=1087032.

[48] Mario Paulo Brito et al. “The Role of adaptive mission planning and control in persistent autonomous underwater vehicles presence”. In: 2012 IEEE/OES Autonomous Underwater Vehicles (AUV). IEEE, Sept. 2012, pp. 1–9. ISBN: 978- 1-4577-2056-7. DOI: 10.1109/AUV.2012.6380748. URL: http://ieeexplore.ieee.org/ document/6380748/.

[49] David L. Poole and Alan K. Mackworth. Artificial Intelligence: Foundations of Computational Agents. 2nd ed. Cambridge University Press, 2017, p. 820. ISBN: 9781107195394. URL: https://artint.info/2e/index.html (visited on 2020-10-14).

[50] Richard Alterman. “Adaptive planning”. In: Cognitive Science 12.3 (Sept. 1988), pp. 393–421. ISSN: 03640213. DOI: 10.1016/0364- 0213(88)90028- 6. URL: http://doi.wiley.com/10.1016/0364-0213(88)90028-6.

[51] Roman Van Der Krogt and Mathijs De Weerdt. “Plan Repair as an Extension of Planning”. In: International Conference on Automated Planning and Scheduling (ICAPS). Ed. by Susanne Biundo, Karen Myers, and Kanna Rajan. Monterey, CA: The AAAI Press, 2005, pp. 161–170. ISBN: 1577352203. URL: http://www.aaai. org/Papers/ICAPS/2005/ICAPS05-017.pdf.

[52] Roman Van Der Krogt. “Plan Repair in Single-Agent and Multi-Agent Systems”. PhD Thesis. Technical University of Delft, Dec. 2005, p. 384. URL: http://resolver. tudelft.nl/uuid:19688ab3-2dcf-4db8-8c3b-dae7ddf7121c.

– 238 – Bibliography

[53] Maria Fox, Alfonso Gerevini, Derek Long, and Ivan Serina. “Plan stability: re- planning versus plan repair”. In: Proceedings of International Conference on AI Planning and Scheduling (ICAPS). Ed. by Derek Long, Stephen F. Smith, Daniel Borrajo, and Lee McCluskey. Cumbria, UK: The AAAI Press, 2006, pp. 212–221. URL: http://strathprints.strath.ac.uk/2776/.

[54] Amanda Coles. “Opportunistic branched plans to maximise utility in the presence of resource uncertainty”. In: Frontiers in Artificial Intelligence and Applications 242 (2012), pp. 252–257. ISSN: 09226389. DOI: 10.3233/978-1-61499-098-7-252.

[55] AFarCloud | ECSEL Joint Undertaking. 2020. URL: https://www.ecsel.eu/projects/ afarcloud (visited on 2020-11-10).

[56] AFarCloud. 2018. URL: http://www.afarcloud.eu/ (visited on 2020-11-10).

[57] SWARMs | ECSEL Joint Undertaking. 2015. URL: https://www.ecsel.eu/projects/ swarms (visited on 2020-11-10).

[58] SWARMs. 2018. URL: http://swarms.eu/ (visited on 2020-11-10).

[59] Artemis Project - 333020 ACCUS. 2016. URL: https://artemis-ia.eu/project/50- accus.html (visited on 2020-11-10).

[60] Project ACCUS. 2016. URL: https://web.archive.org/web/20160109224610/http: //projectaccus.eu/ (visited on 2020-11-10). (archived).

[61] Artemis Project - 295372 DEMANES. 2015. URL: https://artemis-ia.eu/project/37- demanes.html (visited on 2020-11-20).

[62] DEMANES. 2015. URL: https://web.archive.org/web/20150831030642/http: //www.demanes.eu/ (visited on 2020-11-10). (archived).

[63] ITEA 2 · Project · 10028 Web of Objects. 2015. URL: https://itea3.org/project/web- of-objects.html (visited on 2020-11-10).

[64] ITEA 2 Project - WoO Project. 2014. URL: https : / / web . archive . org / web / 20140121220344/http://www.web- of- objects.com/ (visited on 2020-11-10). (archived).

[65] Stuart Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. 3rd. Pearson, 2009. ISBN: 978-0-13-604259-4. URL: http://www.mypearsonstore. com/bookstore/artificial-intelligence-a-modern-approach-9780136042594?xid= PSED.

[66] Malik Ghallab, Dana Nau, and Paolo Traverso. Automated Planning. San Fran- cisco, CA, United States: Morgan Kaufmann Publishers Inc., 2004. ISBN: 978-1- 55860-856-6. DOI: 10.1016/B978-1-55860-856-6.X5000-5.

– 239 – Bibliography

[67] Malik Ghallab, Dana Nau, and Paolo Traverso. Automated Planning and Acting. Cambridge, UK: Cambridge University Press, 2014. ISBN: 9781139583923. DOI: 10.1017/CBO9781139583923.

[68] Sérgio Esteves and Luís Veiga. “Planning and Scheduling Data Processing Workflows in the Cloud with Quality-of-Data Constraints”. In: vol. 8377. Springer, Cham, 2014, pp. 324–338. DOI: 10.1007/978-3-319-06859-6_29.

[69] Judea Pearl. Probabilistic Reasoning in Intelligent Systems. San Francisco, CA, United States: Morgan Kaufmann Publishers Inc., 1988, p. 557. ISBN: 9780080514895. DOI: 10.1016/C2009-0-27609-4.

[70] Ilche Georgievski and Marco Aiello. “HTN planning: Overview, comparison, and beyond”. In: Artificial Intelligence 222 (May 2015), pp. 124–156. ISSN: 00043702. DOI: 10.1016/j.artint.2015.02.002. URL: http://linkinghub.elsevier.com/retrieve/pii/ S0004370215000247.

[71] Michael Cashmore et al. “Opportunistic Planning in Autonomous Underwater Missions”. In: IEEE Transactions on Automation Science and Engineering 15.2 (Apr. 2017), pp. 519–530. ISSN: 1545-5955. DOI: 10.1109/TASE.2016.2636662. URL: https://ieeexplore.ieee.org/document/7828036/.

[72] Richard E. Fikes and Nils J. Nilsson. “Strips: A new approach to the application of theorem proving to problem solving”. In: Artificial Intelligence 2.3-4 (Dec. 1971), pp. 189–208. ISSN: 00043702. DOI: 10.1016/0004- 3702(71)90010- 5. URL: http://www.sciencedirect.com/science/article/pii/0004370271900105.

[73] Diego Aineto, Sergio Jiménez, and Eva Onaindia. “Learning STRIPS Action Models with Classical Planning”. In: Proceedings of the International Conference on Automated Planning and Scheduling 28.1 (June 2018). URL: https://ojs.aaai. org/index.php/ICAPS/article/view/13870 (visited on 2021-03-12).

[74] Alejandro Suárez-Hernández, Javier Segovia-Aguas, Carme Torras, and Guillem Alenyà. “STRIPS Action Discovery”. In: AAAI 2020 conference. Jan. 2020. arXiv: 2001.11457. URL: http://arxiv.org/abs/2001.11457.

[75] Edwin P.D. Pednault. “Formulating multiagent, dynamic-world problems in the classical planning framework”. In: Reasoning About Actions & Plans. Elsevier, 1987, pp. 47–82. DOI: 10.1016/B978- 0- 934613- 30- 9.50006- 8. URL: https: //linkinghub.elsevier.com/retrieve/pii/B9780934613309500068.

[76] Craig McDermott et al. PDDL : The Planning Domain Definition Language. 1998. URL: http://www.plg.inf.uc3m.es/ipc2011-deterministic/attachments/Resources/ mcdermott-et-al-tr-1998.pdf (visited on 2021-02-23).

– 240 – Bibliography

[77] Manuela Veloso. PDDL by Example. 2002, p. 10. URL: http://www.cs.toronto.edu/ ~sheila/384/w11/Assignments/A3/veloso-PDDL_by_Example.pdf (visited on 2020-10-12).

[78] Maria Fox and Derek Long. “PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains”. In: Journal of Artificial Intelligence Research 20 (Dec. 2003), pp. 61–124. ISSN: 1076-9757. DOI: 10.1613/jair.1129. arXiv: 1106. 4561. URL: http://www.aaai.org/Papers/JAIR/Vol20/JAIR-2002.pdf (visited on 2020-11-17).

[79] Stefan Edelkamp and Jörg Hoffmann. PDDL2.2 : The Language for the Classical Part of the 4th International Planning Competition. Tech. rep. 195. 2004, p. 21. URL: https://helios.hud.ac.uk/scommv/IPC-14/repository/edelkamp-hoffmann- tr-2004.pdf (visited on 2021-03-03).

[80] Alfonso Gerevini and Derek Long. Plan Constraints and Preferences in PDDL3. Tech. rep. Universitá di Brescia, 2005, p. 12. URL: http://zeus.ing.unibs.it/ia/ PDDL2-151003/pddl3-0.ps (visited on 2020-10-15).

[81] Alfonso Gerevini and Derek Long. BNF description of PDDL3. 0. 2005, p. 7. URL: http://www.plg.inf.uc3m.es/ipc2011- deterministic/attachments/Resources/ kovacs-pddl-3.1-2011.pdf (visited on 2021-02-26).

[82] Alfonso Gerevini and Derek Long. “Preferences and Soft Constraints in PDDL3”. In: ICAPS-2006 Workshop on Preferences and Soft Constraints in Planning. 2006, pp. 46–54. URL: http://strathprints.strath.ac.uk/3149/ (visited on 2020-10-15).

[83] Daniel L. Kovacs. BNF definition of PDDL. 3.1 Tech. rep. 2011, p. 5. URL: http: //www.plg.inf.uc3m.es/ipc2011-deterministic/attachments/Resources/kovacs- pddl-3.1-2011.pdf (visited on 2020-11-03).

[84] Jeremy Frank and Ari Jónsson. “Constraint-Based Attribute and Interval Planning”. In: Constraints 8.4 (2003), pp. 339–364. ISSN: 1572-9354. DOI: 10.1023/A: 1025842019552.

[85] Michael Brenner. “A Multiagent Planning Language”. In: ICAPS International Workshop on PDDL. 2003, pp. 33–38. URL: http://users.rsise.anu.edu.au/ ~thiebaux/workshops/ICAPS03/proceedings/PDDL- ICAPS03.pdf#page=41 (visited on 2020-11-03).

[86] Håkan L.S. Younes and Michael L. Littman. PPDDL1. 0: An extension to PDDL for expressing planning domains with probabilistic effects. Tech. rep. 2004, p. 23. URL: http://reports-archive.adm.cs.cmu.edu/anon/anon/home/ftp/usr0/ftp/2004/CMU- CS-04-167.pdf (visited on 2020-10-17).

– 241 – Bibliography

[87] Scott Sanner. Relational dynamic influence diagram language (rddl): Language description. Tech. rep. 2010. URL: https://users.cecs.anu.edu.au/~ssanner/IPPC_ 2011/RDDL.pdf (visited on 2021-03-05).

[88] Daniel L. Kovacs. “A Multi-Agent Extension of PDDL3.” In: 3rd Workshop on the International Planning Competition (IPC). Sao Paulo, 2012, p. 9. URL: http: //home.mit.bme.hu/~dkovacs/pubs/d.l.kovacs_2012_ICAPS-WIPC.pdf (visited on 2020-11-17).

[89] Daniel Höller et al. “HDDL: An Extension to PDDL for Expressing Hierarchical Planning Problems”. In: Proceedings of the AAAI Conference on Artificial Intelli- gence 34.06 (Apr. 2020), pp. 9883–9891. ISSN: 2374-3468. DOI: 10.1609/aaai. v34i06.6542. URL: https://aaai.org/ojs/index.php/AAAI/article/view/6542 (visited on 2021-01-03).

[90] Eric Bouillet et al. “A knowledge engineering and planning framework based on OWL ontologies”. In: International Competition on Knowledge Engineering for Planning and Scheduling (ICKEPS) (2007). URL: https://icaps07-satellite.icaps- conference.org/ickeps/OWL-ICKEPS07_CamRdy.pdf (visited on 2021-03-06).

[91] Walter Bradford Cannon. The wisdom of the body. London, UK: Kegan Paul and Co., Ltd., 1934, p. 312. DOI: 10.1038/133082a0.

[92] José Antonio Martín H., Javier de Lope, and Darío Maravall. “Adaptation, anticipation and rationality in natural and artificial systems: computational paradigms mimicking nature”. en. In: Natural Computing 8.4 (Dec. 2009), pp. 757–775. ISSN: 1567-7818. DOI: 10.1007/s11047-008-9096-6. URL: http://link.springer.com/10. 1007/s11047-008-9096-6.

[93] Martin Revay and Miroslav Liska. “OODA loop in command and control systems”. In: 2017 Communication and Information Technologies (KIT). IEEE, Oct. 2017, pp. 1–4. ISBN: 978-80-8040-550-2. DOI: 10.23919/KIT.2017.8109463. URL: http://ieeexplore.ieee.org/document/8109463/.

[94] John R. Boyd. A Discourse on Winning and Losing. Ed. by Grant T. Hammond. Al- abama, AL, USA: Air University, 2018, pp. 383–386. URL: https://www.airuniversity. af.edu/Portals/10/AUPress/Books/B_0151_Boyd_Discourse_Winning_Losing. PDF (visited on 2020-12-30).

[95] Chet Richards. “Boyd ’s OODA Loop”. In: Necesse 5.1 (2020), pp. 142–165. URL: https://hdl.handle.net/11250/2683228.

[96] Pedro Patrón and David M. Lane. “Adaptive mission planning: the embedded OODA loop”. In: Proceedings of the 3rd SEAS DTC Technical Conference. 2008,

– 242 – Bibliography

p. 10. URL: http://osl.eps.hw.ac.uk/files/uploads/publications/2008_SEAS_DTC_ PPatron.pdf (visited on 2020-10-12).

[97] Yepang Liu and Shing-Chi Cheung. A Survey of Context-Aware Pervasive Ap- plications: From Development Support to Quality Assurance. Tech. rep. Hong Kong: The Hong Kong Univeristy of Science and Technology, 2011, p. 27. DOI: 10.13140/RG.2.2.28955.16163.

[98] Corporate ACT-NET Consortium. “The active database management system manifesto”. In: ACM SIGMOD Record 25.3 (Sept. 1996), pp. 40–49. ISSN: 0163- 5808. DOI: 10.1145/234889.234896. URL: https://dl.acm.org/doi/10.1145/234889. 234896.

[99] Norman W. Paton and Oscar Díaz. “Active database systems”. In: ACM Com- puting Surveys 31.1 (Mar. 1999), pp. 63–103. ISSN: 0360-0300. DOI: 10.1145/ 311531.311623. URL: https://dl.acm.org/doi/10.1145/311531.311623.

[100] D. Goldin, S. Srinivasa, and V. Srikanti. “Active databases as information systems”. In: Proceedings. International Database Engineering and Applications Symposium, 2004. IDEAS ’04. IEEE, 2004, pp. 123–130. ISBN: 0-7695-2168-1. DOI: 10.1109/IDEAS.2004.1319786. URL: http://ieeexplore.ieee.org/document/ 1319786/.

[101] Markus C Huebscher and Julie A McCann. “A survey of autonomic computing—degrees, models, and applications”. In: ACM Computing Surveys 40.3 (Aug. 2008), pp. 1–28. ISSN: 03600300. DOI: 10.1145/1380584.1380585. URL: http://portal.acm.org/citation.cfm?doid=1380584.1380585.

[102] Rodney A. Brooks. Intelligence Without Reason. Tech. rep. Boston, MA, USA: Massachusetts Institute of Technology, 1991, p. 28. URL: https://dspace.mit.edu/ bitstream/handle/1721.1/6569/AIM-1293.pdf.

[103] Jiríˇ Wiedermann and Jan van Leeuwen. “Towards Minimally Conscious Cyber- Physical Systems: A Manifesto”. In: 47th International Conference on Current Trends in Theory and Practice of Computer Science, SOFSEM 2021. Bolzano- Bozen, Italy, Jan. 2021, pp. 43–55. DOI: 10.1007/978-3-030-67731-2_4. URL: http://link.springer.com/10.1007/978-3-030-67731-2_4.

[104] Juan Vizcarrondo, Jose Aguilar, Ernesto Exposito, and Audine Subias. “MAPE-K as a service-oriented architecture”. In: IEEE Latin America Transactions 15.6 (June 2017), pp. 1163–1175. ISSN: 1548-0992. DOI: 10.1109/TLA.2017.7932705. URL: http://ieeexplore.ieee.org/document/7932705/.

– 243 – Bibliography

[105] IBM. An architectural blueprint for autonomic computing. Tech. rep. 2006, p. 34. URL: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.150.1011&rep= rep1&type=pdf (visited on 2020-10-02). [106] Danny Weyns et al. “On Patterns for Decentralized Control in Self-Adaptive Systems”. In: Software Engineering for Self-Adaptive Systems II. 2013, pp. 76– 107. DOI: 10.1007/978-3-642-35813-5_4. URL: http://link.springer.com/10.1007/ 978-3-642-35813-5_4. [107] Ioannis Georgiadis, Jeff Magee, and Jeff Kramer. “Self-organising software architectures for distributed systems”. In: Proceedings of the first workshop on Self-healing systems - WOSS ’02. New York, New York, USA: ACM Press, 2002, p. 33. ISBN: 1581136099. DOI: 10 . 1145 / 582128 . 582135. URL: http : //portal.acm.org/citation.cfm?doid=582128.582135. [108] Danny Weyns, Sam Malek, and Jesper Andersson. “On decentralized self- adaptation”. In: Proceedings of the 2010 ICSE Workshop on Software Engi- neering for Adaptive and Self-Managing Systems - SEAMS ’10. New York, New York, USA: ACM Press, 2010, pp. 84–93. ISBN: 9781605589718. DOI: 10.1145/1808984.1808994. URL: http://portal.acm.org/citation.cfm?doid= 1808984.1808994. [109] Giovanni Toffetti, Alessio Gambi, Mauro Pezzè, and Cesare Pautasso. “Engineer- ing Autonomic Controllers for Virtualized Web Applications”. In: 2010, pp. 66–80. DOI: 10.1007/978-3-642-13911-6_5. URL: http://link.springer.com/10.1007/978- 3-642-13911-6_5. [110] Sam Malek, Nels Beckman, Marija Mikic-Rakic, and Nenad Medvidovic. “A Frame- work for Ensuring and Improving Dependability in Highly Distributed Systems”. In: 2005, pp. 173–193. DOI: 10.1007/11556169_8. URL: http://link.springer.com/ 10.1007/11556169_8. [111] Valeria Cardellini, Emiliano Casalicchio, Vincenzo Grassi, and Francesco Lo Presti. “Adaptive Management of Composite Services under Percentile-Based Service Level Agreements”. In: 2010, pp. 381–395. DOI: 10.1007/978-3-642- 17358-5_26. URL: http://link.springer.com/10.1007/978-3-642-17358-5_26. [112] Russell L. Ackoff. “Towards a System of Systems Concepts”. In: Management Science 17.11 (July 1971), pp. 661–671. ISSN: 0025-1909. DOI: 10.1287/mnsc. 17.11.661. URL: http://pubsonline.informs.org/doi/abs/10.1287/mnsc.17.11.661. [113] Mark W. Maier. “Architecting principles for systems-of-systems”. In: Systems Engineering 1.4 (1998), pp. 267–284. ISSN: 1098-1241. URL: https://onlinelibrary. wiley.com/doi/10.1002/(SICI)1520-6858(1998)1:4%3C267::AID-SYS3%3E3.0. CO;2-D.

– 244 – Bibliography

[114] Marietheres Dietz and Günther Pernul. “Digital Twin: Empowering Enterprises Towards a System-of-Systems Approach”. In: Business & Information Systems Engineering 62.2 (Apr. 2020), pp. 179–184. ISSN: 2363-7005. DOI: 10.1007/ s12599-019-00624-0. URL: http://link.springer.com/10.1007/s12599-019-00624- 0.

[115] Robert Rosen. Anticipatory Systems. Vol. 1. IFSR International Series on Sys- tems Science and Engineering. New York, NY: Springer New York, 2012. ISBN: 978-1-4614-1268-7. DOI: 10.1007/978-1-4614-1269-4. URL: http://link.springer. com/10.1007/978-1-4614-1269-4.

[116] C. Matthew MacKenzie et al. Reference Model for Service Oriented Architecture 1.0. Tech. rep. OASIS, 2006, p. 31. URL: http : / / docs. oasis - open . org / soa - rm/v1.0/soa-rm.pdf (visited on 2010-10-02).

[117] Reference Architecture Foundation for Service Oriented Architecture Version 1.0. Tech. rep. OASIS, 2012, p. 118. URL: http://docs.oasis-open.org/soa-rm/soa- ra/v1.0/cs01/soa-ra-v1.0-cs01.pdf (visited on 2020-10-15).

[118] S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. Tech. rep. Mar. 1997. DOI: 10.17487/rfc2119. URL: https://www.rfc-editor.org/info/rfc2119.

[119] International Telecommunication Union (ITU). The Internet of Things. Tech. rep. 2005, p. 212. URL: https://www.itu.int/net/wsis/tunis/newsroom/stats/The-Internet- of-Things-2005.pdf (visited on 2020-10-13).

[120] IEC - ISO/IEC JTC 1/SC 41 Dashboard > Scope. URL: https://www.iec.ch/dyn/ www/f?p=103:7:14352973279623::::FSP_ORG_ID,FSP_LANG_ID:20486,25 (visited on 2020-12-14).

[121] Internet of Things Architecture | IoT-A Project | FP7 | CORDIS | European Commission. URL: https : / / cordis . europa . eu / project / id / 257521 / es (visited on 2020-12-14).

[122] Alessandro Bassi et al., eds. Enabling Things to Talk. Berlin, Heidelberg: Springer, Berlin, Heidelberg, 2013, p. 349. ISBN: 978-3-642-40402-3. DOI: 10.1007/978-3- 642-40403-0. URL: http://link.springer.com/10.1007/978-3-642-40403-0.

[123] Martin Bauer et al. “IoT Reference Model”. In: Enabling Things to Talk. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 113–162. DOI: 10.1007/978-3- 642-40403-0_7. URL: http://link.springer.com/10.1007/978-3-642-40403-0_7.

[124] Martin Bauer et al. “IoT Reference Architecture”. In: Enabling Things to Talk. Berlin, Heidelberg: Springer Berlin Heidelberg, 2013, pp. 163–211. DOI: 10.1007/ 978-3-642-40403-0_8. URL: http://link.springer.com/10.1007/978-3-642-40403- 0_8.

– 245 – Bibliography

[125] Stephan Haller, Alexandru Serbanati, Martin Bauer, and Francois Carrez. “A Domain Model for the Internet of Things”. In: 2013 IEEE International Conference on Green Computing and Communications and IEEE Internet of Things and IEEE Cyber, Physical and Social Computing. IEEE, Aug. 2013, pp. 411–417. ISBN: 978-0-7695-5046-6. DOI: 10.1109/GreenCom-iThings-CPSCom.2013.87. URL: http://ieeexplore.ieee.org/document/6682101/.

[126] IEEE Robotics and Automation Society. 1872-2015 – IEEE Standard Ontologies for Robotics and Automation. IEEE, 2015, p. 60. ISBN: 9780738196503. DOI: 10.1109/IEEESTD.2015.7084073.

[127] Néstor Lucas Martínez, José-Fernán Martínez-Ortega, Pedro Castillejo, and Victoria Beltrán Martínez. “Survey of Mission Planning and Management Archi- tectures for Underwater Cooperative Robotics Operations”. In: Applied Sciences 10.3 (Feb. 2020), p. 1086. ISSN: 2076-3417. DOI: 10.3390/app10031086. URL: https://www.mdpi.com/2076-3417/10/3/1086.

[128] Fletcher Thompson and Damien Guihen. “Review of mission planning for autonomous marine vehicle fleets”. In: Journal of Field Robotics 36.2 (Mar. 2019), pp. 333–354. ISSN: 15564959. DOI: 10.1002/rob.21819. URL: http://doi.wiley.com/ 10.1002/rob.21819.

[129] Malik Ghallab, Dana Nau, and Paolo Traverso. “Hierarchical Task Network Plan- ning”. In: Automated Planning. Elsevier, 2004, pp. 229–261. ISBN: 9781558608566. DOI: 10.1016/B978-155860856-6/50017-X. URL: https://linkinghub.elsevier.com/ retrieve/pii/B978155860856650017X.

[130] Néstor Lucas Martínez et al. “Virtualization of Event Sources in Wireless Sensor Networks for the Internet of Things”. In: Sensors (Basel, Switzerland) 14.12 (2014), pp. 22737–22753. ISSN: 1424-8220. DOI: 10.3390/s141222737.

[131] DNV (Det Norske Veritas). RULES FOR CLASSIFICATION Underwater technology Part 5 Types of UWT systems Chapter 8 Autonomous underwater vehicles. Tech. rep. Dec. 2015. URL: http : / / rules . dnvgl . com / docs / pdf / DNVGL / RU - UWT/2018-01/DNVGL-RU-UWT-Pt5Ch8.pdf (visited on 2020-12-21).

[132] Francesco Pacini et al. “The SWARMs Approach to Integration of Underwater and Overwater Communication Sub-Networks and Integration of Heterogeneous Underwater Communication Systems”. In: Volume 7A: Ocean Engineering. Amer- ican Society of Mechanical Engineers, June 2018. ISBN: 978-0-7918-5126-5. DOI: 10.1115/OMAE2018-78772. URL: https://asmedigitalcollection.asme.org/OMAE/ proceedings/OMAE2018/51265/Madrid,%20Spain/277727.

– 246 – Bibliography

[133] Frédéric Py, Kanna Rajan, and Conor Mcgann. “A systematic agent framework for situated autonomous systems”. In: 9th International Conference on Autonomous Agents and Multiagent Systems. Vol. 2. Toronto, Canada, 2010, pp. 583–590. URL: https://dl.acm.org/doi/10.5555/1838178.1838183 (visited on 2020-12-30).

[134] Fernando Ropero, Pablo Muñoz, and María D. R-Moreno. “A Versatile Executive Based on T-REX for Any Robotic Domain”. In: 2018, pp. 79–91. DOI: 10.1007/978- 3-030-04191-5_6. URL: http://link.springer.com/10.1007/978-3-030-04191-5_6.

[135] MBARI - Autonomy - TREX. URL: https://web.archive.org/web/20140903170721/ https://www.mbari.org/autonomy/TREX/index.htm (visited on 2019-11-27).

[136] Conor McGann et al. “A deliberative architecture for AUV control”. In: 2008 IEEE International Conference on Robotics and Automation. IEEE, May 2008, pp. 1049–1054. ISBN: 978-1-4244-1646-2. DOI: 10.1109/ROBOT.2008.4543343. URL: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4543343% 20http://ieeexplore.ieee.org/document/4543343/.

[137] CANON: Controlled, Agile, and Novel Ocean Network. URL: http://www.mbari. org/canon/ (visited on 2020-01-20).

[138] Rishi Graham et al. “Exploring Space-Time Tradeoffs in Autonomous Sampling for Marine Robotics”. In: Experimental Robotics. Heidelberg: Springer, 2013, pp. 819–839. DOI: 10.1007/978-3-319-00065-7_55. URL: http://link.springer.com/ 10.1007/978-3-319-00065-7_55.

[139] Py and Rajan. “T-REX: partitioned inference for AUV mission control”. In: Fur- ther Advances in Unmanned Marine Vehicles. Institution of Engineering and Technology, pp. 171–199. DOI: 10.1049/PBCE077E_ch9. URL: https://digital- library.theiet.org/content/books/10.1049/pbce077e_ch9.

[140] Executive T-Rex for ROS. URL: http://wiki.ros.org/executive_trex (visited on 2019-11-27).

[141] Conor McGann et al. “Adaptive control for autonomous underwater vehicles”. In: Proceedings of the National Conference on Artificial Intelligence 3 (2008), pp. 1319–1324. URL: https://www.aaai.org/Papers/AAAI/2008/AAAI08-209.pdf (visited on 2020-10-03).

[142] RAUVI: Reconfigurable AUV for Intervention. URL: http://www.irs.uji.es/rauvi/ news.html (visited on 2019-11-27).

[143] Gianluca De Novi et al. “A new approach for a Reconfigurable Autonomous Un- derwater Vehicle for Intervention”. In: 2009 3rd Annual IEEE Systems Conference. IEEE, Mar. 2009, pp. 23–26. ISBN: 978-1-4244-3462-6. DOI: 10.1109/SYSTEMS. 2009.4815765. URL: http://ieeexplore.ieee.org/document/4815765/.

– 247 – Bibliography

[144] Narcís. Palomeras et al. “A distributed architecture for enabling autonomous underwater Intervention Missions”. In: 2010 IEEE International Systems Confer- ence. IEEE, Apr. 2010, pp. 159–164. ISBN: 978-1-4244-5882-0. DOI: 10.1109/ SYSTEMS.2010.5482349. URL: http://ieeexplore.ieee.org/document/5482349/.

[145] Mario Prats et al. “Reconfigurable AUV for intervention missions: a case study on underwater object recovery”. In: Intelligent Service Robotics 5.1 (Jan. 2012), pp. 19–31. ISSN: 1861-2776. DOI: 10.1007/s11370- 011- 0101- z. URL: http: //link.springer.com/10.1007/s11370-011-0101-z.

[146] Juan C. García et al. “Towards specification, planning and sensor-based control of autonomous underwater intervention”. In: IFAC Proceedings Volumes 44.1 (Jan. 2011), pp. 10361–10366. ISSN: 14746670. DOI: 10.3182/20110828-6-IT-1002. 02456. URL: https://linkinghub.elsevier.com/retrieve/pii/S1474667016452778.

[147] Trident FP7 European Project. URL: http://www.irs.uji.es/trident/aboutproject.html (visited on 2019-11-28).

[148] Carlos C. Insaurralde, Joel J. Cartwright, and Yvan R. Petillot. “Cognitive Con- trol Architecture for autonomous marine vehicles”. In: 2012 IEEE International Systems Conference SysCon 2012. IEEE, Mar. 2012, pp. 1–8. ISBN: 978-1-4673- 0750-5. DOI: 10.1109/SysCon.2012.6189542. URL: http://ieeexplore.ieee.org/ document/6189542/.

[149] Carlos C. Insaurralde and Yvan R. Petillot. “Intelligent autonomy for collaborative intervention missions of unmanned maritime vehicles”. In: OCEANS 2013 MTS/IEEE - San Diego: An Ocean in Common (2013), pp. 1–6. DOI: 10.23919/ OCEANS.2013.6741354.

[150] Pedro J. Sanz et al. “TRIDENT An European project targeted to increase the autonomy levels for underwater intervention missions”. In: 2013 OCEANS. San Diego, CA, USA: IEEE, 2013, pp. 1–10. DOI: 10.23919/OCEANS.2013.6741370.

[151] Mica R. Endsley. Designing for Situation Awareness. 2nd. Boca Raton: CRC Press, Apr. 2016, p. 396. ISBN: 9780429146732. DOI: 10.1201/b11371. URL: https://www.taylorfrancis.com/books/9781420063585.

[152] Karen L Myers. “Towards a Framework for Continuous Planning and Execution”. In: Proceedings of the AAAI Fall Symposium on Distributed Continual Planning. AAAI Press, 1998, p. 6. URL: http://www.ai.sri.com./~cpef/papers/fss98.pdf (visited on 2021-03-06).

[153] Continuous Planning and Execution. URL: http://www.ai.sri.com/~cpef/ (visited on 2020-01-28).

– 248 – Bibliography

[154] Karen L Myers. JFACC Continuous Planning and Execution. Tech. rep. SRI INTERNATIONAL, 2000, pp. 1–86. URL: https://apps.dtic.mil/docs/citations/ ADA383279 (visited on 2020-10-15).

[155] CPEF Baseline Demonstration. 1997. URL: http://www.ai.sri.com/~cpef/jfacc/acp- demo-dec97.html (visited on 2020-01-28).

[156] CPEF Phase 2 Demonstration. 1998. URL: http://www.ai.sri.com/~cpef/jfacc/ifd2. html (visited on 2020-01-28).

[157] CPEF Final Demonstration. 1999. URL: http://www.ai.sri.com/~cpef/jfacc/final- status.html (visited on 2020-01-28).

[158] P. Schaefer et al. “Reliable autonomous control technologies (ReACT) for unin- habited air vehicles”. In: 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542). Vol. 2. IEEE, pp. 2/677–2/684. ISBN: 0-7803-6599-2. DOI: 10. 1109/AERO.2001.931247. URL: http://ieeexplore.ieee.org/document/931247/.

[159] Timothy L. Johnson et al. “The TRAC mission manager autonomous control executive”. In: 2001 IEEE Aerospace Conference Proceedings (Cat. No.01TH8542). Vol. 2. IEEE, pp. 2/639–2/646. ISBN: 0-7803-6599-2. DOI: 10.1109/AERO.2001. 931243. URL: http://ieeexplore.ieee.org/document/931243/.

[160] Forest W Fisher et al. “CLEaR: Closed Loop Execution and Recovery - A Frame- work for Unified Planning and Execution”. In: Interplanetary Network Directorate Technology and Science News. 16. Pasadena, CA, USA, 2002, p. 6. URL: http: //hdl.handle.net/2014/10168 (visited on 2020-10-15).

[161] What is ROSPlan? URL: https://kcl- planning.github.io/ROSPlan/ (visited on 2019-11-27).

[162] PANDORA. Persistent Autonomy through learNing, aDaptation, Observation, and Re-plAnning. 2012. URL: https://web.archive.org/web/20190818215121/http: //persistentautonomy.com/ (visited on 2021-03-06). (archived).

[163] Francesco Maurelli et al. “The PANDORA project: A success story in AUV autonomy”. In: OCEANS 2016 - Shanghai. IEEE, Apr. 2016, pp. 1–8. ISBN: 978-1- 4673-9724-7. DOI: 10.1109/OCEANSAP.2016.7485618. URL: http://ieeexplore. ieee.org/document/7485618/.

[164] Michael Cashmore et al. “ROSPlan: Planning in the Robot Operating System”. In: Proceedings of International Conference on AI Planning and Scheduling (ICAPS). Jerusalem, Israel: AAAI Press, 2015, pp. 333–341. ISBN: 9781577357315. URL: https://www.aaai.org/ocs/index.php/ICAPS/ICAPS15/paper/view/10619 (visited on 2021-01-04).

– 249 – Bibliography

[165] ROSPlan Overview. URL: https://kcl-planning.github.io/ROSPlan/documentation/ (visited on 2019-11-29).

[166] Gerard Canal et al. “Probabilistic Planning for Robotics with ROSPlan”. In: 2019, pp. 236–250. DOI: 10.1007/978-3-030-23807-0_20. URL: http://link.springer.com/ 10.1007/978-3-030-23807-0%7B%5C_%7D20.

[167] The Consultative Committee for Space Data Systems. Information Architec- ture Reference Model. Tech. rep. June. NASA, 2006, p. 58. URL: https://cwe. ccsds.org/sea/docs/Work%20Completed%20(Closed%20WGs)/Information% 20Architecture%20Working%20Group/Draft%20Documents/IA%20Reference% 20Model/RASIM-060709.pdf (visited on 2020-12-13).

[168] Florent Teichteil-Königsbuch, Ugur Kuter, and Guillaume Infantes. “Incremental plan aggregation for generating policies in MDPs”. In: Proceedings of the Interna- tional Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS. Vol. 2. 1. 2010, pp. 1231–1238. ISBN: 9781617387715. URL: http://www.ifaamas. org/Proceedings/aamas2010/pdf/01%20Full%20Papers/25_04_FP_0529.pdf (visited on 2020-11-04).

[169] Jorge García-Paredes. “Anticipative system for a planner through machine learning techniques”. Trabajo Fin de Grado. Universidad Politécnica de Madrid, July 2018, p. 118. URL: http://oa.upm.es/53110/.

[170] Henry Mintzberg. “Patterns in Strategy Formation”. In: Management Science 24.9 (May 1978), pp. 934–948. ISSN: 0025-1909. DOI: 10.1287/mnsc.24.9.934. URL: http://pubsonline.informs.org/doi/abs/10.1287/mnsc.24.9.934.

[171] Alfred D. Chandler Jr. Strategy and structure: chapters in the history of the industrial enterprise. Washington, D.C., USA: Beard Books, 1962.

[172] ITU Telecommunication Standardization Sector (ITU-T). Recommendation ITU- T Y.101: Global Information Infrastructure terminology: Terms and definitions. Tech. rep. 2000, p. 11. URL: https://www.itu.int/rec/T-REC-Y.101/en (visited on 2020-10-15).

[173] Bharat S. Thakkar et al. Paradigm Shift in Management Philosophy. Ed. by Bharat S. Thakkar. 1st. Cham: Springer International Publishing, 2020, p. 260. ISBN: 978-3-030-29709-1. DOI: 10.1007/978-3-030-29710-7. URL: http://link.springer. com/10.1007/978-3-030-29710-7.

[174] Henri Fayol. General and Industrial Management. Trans. by Constance Storrs. Eastford, CT, USA: Martino Fine Books, Aug. 2013, p. 142. ISBN: 9781614274599.

– 250 – Bibliography

[175] Agnar Aamodt and Enric Plaza. “Case-Based Reasoning: Foundational Issues, Methodological Variations, and System Approaches”. In: AI Communications 7.1 (1994), pp. 39–59. ISSN: 09217126. DOI: 10.3233/AIC- 1994- 7104. URL: https://content.iospress.com/articles/ai-communications/aic7-1-04.

[176] Abdulmotaleb El Saddik. “Digital Twins: The Convergence of Multimedia Tech- nologies”. In: IEEE MultiMedia 25.2 (Apr. 2018), pp. 87–92. ISSN: 1070-986X. DOI: 10.1109/MMUL.2018.023121167. URL: https://ieeexplore.ieee.org/document/ 8424832/.

[177] Ørnulf Jan Rødseth. “Object-oriented software system for AUV control”. In: Engi- neering Applications of Artificial Intelligence 4.4 (Jan. 1991), pp. 269–277. ISSN: 09521976. DOI: 10.1016/0952-1976(91)90041-4. URL: https://linkinghub.elsevier. com/retrieve/pii/0952197691900414.

[178] Tan Yew Teck, Mandar Anil Chitre, Prahlad Vadakkepat, and Shiraz Shahabudeen. Design and Development of Command and Control System for Autonomous Underwater Vehicles. Tech. rep. 2008, pp. 1–6. URL: https://arl.nus.edu.sg/twiki6/ pub/ARL/BibEntries/Tan2009a.pdf (visited on 2020-12-15).

[179] Tan Yew Teck. “Design and development of command and control system for autonomous underwater vehicles”. Master Thesis. National University of Singapore, 2008, p. 103. URL: http://scholarbank.nus.edu.sg/handle/10635/16691.

[180] Henrik Østergaard Madsen. “Mission Management System for an Autonomous Underwater Vehicle”. In: IFAC Proceedings Volumes 30.22 (Sept. 1997), pp. 59– 63. ISSN: 14746670. DOI: 10.1016/S1474- 6670(17)46490- 1. URL: https:// linkinghub.elsevier.com/retrieve/pii/S1474667017464901.

[181] Henrik Østergaard Madsen, Anders Bjerrum, Bo Krogh, and Maridan Aps. “MAR- TIN - an AUV for Offshore Surveys”. In: Oceanology International 96. March. Brighton, 1996, pp. 5–8. URL: http://henrik.ostergaard.net/wp-content/uploads/ sites/2/2020/01/Control- system- for- an- AUV- pagenumbers.pdf (visited on 2020-10-15).

[182] Adham Atyabi, Somaiyeh MahmoudZadeh, and Samia Nefti-Meziani. “Current advancements on autonomous mission planning and management systems: An AUV and UAV perspective”. In: Annual Reviews in Control 46 (2018), pp. 196– 215. ISSN: 13675788. DOI: 10 . 1016 / j . arcontrol . 2018 . 07 . 002. URL: https : //linkinghub.elsevier.com/retrieve/pii/S1367578818300257.

[183] Somaiyeh MahmoudZadeh, David M. W. Powers, and Reza Bairam Zadeh. “State- of-the-Art in UVs’ Autonomous Mission Planning and Task Managing Approach”. In: Autonomy and Unmanned Vehicles. Singapore: Sprigner, 2019, pp. 17–30.

– 251 – Bibliography

DOI: 10.1007/978-981-13-2245-7_2. URL: http://link.springer.com/10.1007/978- 981-13-2245-7_2.

[184] Somaiyeh MahmoudZadeh, David M. W. Powers, and Reza Bairam Zadeh. “State- of-the-Art in UVs’ Autonomous Mission Planning and Task Managing Approach”. In: Autonomy and Unmanned Vehicles. Springer, Singapore, 2019. Chap. 2, pp. 17–30. ISBN: 978-981-13-2245-7. DOI: 10.1007/978-981-13-2245-7_2. URL: http://link.springer.com/10.1007/978-981-13-2245-7_2.

[185] Néstor Lucas Martínez, José Fernán Martínez-Ortega, Jesús Rodríguez-Molina, and Zhaoyu Zhai. “Proposal of an Automated Mission Manager for Cooperative Autonomous Underwater Vehicles”. In: Applied Sciences 10.3 (Jan. 2020), p. 855. ISSN: 2076-3417. DOI: 10.3390/app10030855. URL: https://www.mdpi.com/2076- 3417/10/3/855.

[186] Sonia Bilbao et al. D3.2 Architecture Analysis and Specification (v2). SWARMs, 2016, p. 82.

[187] Jesús Rodríguez-Molina et al. “An Optimized, Data Distribution Service-Based Solution for Reliable Data Exchange Among Autonomous Underwater Vehicles”. In: Sensors 17.8 (Aug. 2017), p. 1802. ISSN: 1424-8220. DOI: 10.3390/s17081802. URL: http://www.mdpi.com/1424-8220/17/8/1802.

[188] Afshin E. Ameri, Baran Cürüklü, Branko Miloradovic, and Mikael Ektröm. “Plan- ning and Supervising Autonomous Underwater Vehicles through the Mission Management Tool”. In: Global OCEANS 2020. Singapore, 2020, pp. 1–7.

[189] Francesco Pacini et al. “Integrated comunication network for underwater applications: the SWARMs approach”. In: 2018 Fourth Underwater Communications and Networking Conference (UComms). IEEE, Aug. 2018, pp. 1–5. ISBN: 978-1- 5386-6442-1. DOI: 10.1109/UComms.2018.8493214. URL: https://ieeexplore.ieee. org/document/8493214/.

[190] About the Data Distribution Service Specification Version 1.4. URL: https://www. omg.org/spec/DDS/ (visited on 2019-12-16).

[191] Xin Li et al. “SWARMs Ontology: A Common Information Model for the Coopera- tion of Underwater Robots”. In: Sensors 17.3 (Mar. 2017), p. 569. ISSN: 1424- 8220. DOI: 10.3390/s17030569. URL: http://www.mdpi.com/1424-8220/17/3/569.

[192] Xin Li, José-fernán Martínez, Gregorio Rubio, and David Gómez. “Context Rea- soning in Underwater Robots Using MEBN”. In: The Third International Con- ference on Cloud and Robotics (ICCR 2016). November. Saint Quentin, 2016, pp. 22–23. URL: https://arxiv.org/abs/1706.07204.

– 252 – Bibliography

[193] Zhaoyu Zhai et al. “A Rule-Based Reasoner for Underwater Robots Using OWL and SWRL”. In: Sensors 18.10 (Oct. 2018), p. 3481. ISSN: 1424-8220. DOI: 10.3390/s18103481. URL: http://www.mdpi.com/1424-8220/18/10/3481.

[194] Branko Miloradovic, Baran Curuklu, and Mikael Ekstrom. “A genetic planner for mission planning of cooperative agents in an underwater environment”. In: 2016 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Dec. 2016, pp. 1–8. ISBN: 978-1-5090-4240-1. DOI: 10.1109/SSCI.2016.7850163. URL: http://ieeexplore.ieee.org/document/7850163/.

[195] Branko Miloradovic,´ Baran Çürüklü, and Mikael Ekström. “A Genetic Mission Planner for Solving Temporal Multi-agent Problems with Concurrent Tasks”. In: 2017, pp. 481–493. DOI: 10.1007/978- 3- 319- 61833- 3_51. URL: http://link. springer.com/10.1007/978-3-319-61833-3_51.

[196] Itziar Landa-Torres, Diana Manjarres, Sonia Bilbao, and Javier Del Ser. “Under- water Robot Task Planning Using Multi-Objective Meta-Heuristics”. In: Sensors 17.4 (Apr. 2017), p. 762. ISSN: 1424-8220. DOI: 10 . 3390 / s17040762. URL: http://www.mdpi.com/1424-8220/17/4/762.

[197] Apache Thrift - Home. URL: https://thrift.apache.org/ (visited on 2019-12-16).

[198] Mark Slee, Aditya Agarwal, and Marc Kwiatkowski. Thrift : Scalable Cross- Language Services Implementation. Palo Alto, CA, 2007. URL: https://users. cs.jmu.edu/bernstdh/Web/CS462/thrift-20070401.pdf (visited on 2020-10-15).

[199] Jesús Rodríguez-Molina, Belén Martínez, Sonia Bilbao, and Tamara Martín- Wanton. “Maritime Data Transfer Protocol (MDTP): A Proposal for a Data Trans- mission Protocol in Resource-Constrained Underwater Environments Involving Cyber-Physical Systems”. In: Sensors 17.6 (June 2017), p. 1330. ISSN: 1424- 8220. DOI: 10.3390/s17061330. URL: http://www.mdpi.com/1424-8220/17/6/1330.

[200] Martin Bauer et al. Deliverable D1.5 – Final architectural reference model for the IoT v3.0, FP7-257521 "Internet of Things - Architecture IoT-A". IoT-A, 2013, p. 499.

[201] SWARMs Overview. URL: http://swarms.eu/overview.html (visited on 2019-12-16).

[202] GitHub - uuvsimulator/uuv_plume_simulator: ROS nodes to generate a turbulent plume in an underwater environment. URL: https://github.com/uuvsimulator/uuv_ plume_simulator (visited on 2019-12-11).

[203] Musa Morena Marcusso Manhaes et al. “UUV Simulator: A Gazebo-based package for underwater intervention and multi-robot simulation”. In: OCEANS 2016 MTS/IEEE Monterey. IEEE, Sept. 2016, pp. 1–8. ISBN: 978-1-5090-1537-5. DOI:

– 253 – Bibliography

10.1109/OCEANS.2016.7761080. URL: http://ieeexplore.ieee.org/document/ 7761080/.

[204] GitHub - uuvsimulator/uuv_simulator: Gazebo/ROS packages for underwater robotics simulation. URL: https://github.com/uuvsimulator/uuv_simulator (visited on 2019-12-11).

[205] Michele Colledanchise, Ramviyas Parasuraman, and Petter Ogren. “Learning of Behavior Trees for Autonomous Agents”. In: IEEE Transactions on Games 11.2 (June 2019), pp. 183–189. ISSN: 2475-1502. DOI: 10.1109/TG.2018.2816806. URL: https://ieeexplore.ieee.org/document/8319483/.

[206] ISO/IEC. ISO/IEC 30141:2018 - Internet of Things (loT) — Reference Architecture. URL: https://www.iso.org/standard/65695.html (visited on 2020-12-27).

[207] ISO/IEC. ISO/IEC/IEEE 42010:2011 - Systems and software engineering — Architecture description. URL: https://www.iso.org/standard/50508.html (visited on 2020-12-27).

[208] Architecture Analysis and Design Language. URL: https://www.sei.cmu.edu/our- work/projects/display.cfm?customel_datapageid_4050=191439 (visited on 2020-12-27).

[209] Peter H Feiler, David P Gluch, and John J Hudak. The Architecture Analysis & Design Language ( AADL ): An Introduction. Tech. rep. February. 2006, CMU/SEI– 2006–TN–011. DOI: 10.1184/R1/6584909.v1. URL: http://www.sei.cmu.edu/ library/abstracts/reports/06tn011.cfm.

[210] Peter H Feiler and David P Gluch. Model-Based Engineering with AADL: An Introduction to the SAE Architecture Analysis & Design Language. 1st. Boston, MA, USA: Addison-Wesley Professional, 2012. ISBN: 0321888944.

[211] Zhai, Martínez Ortega, Beltran, and Lucas Martínez. “An Associated Representa- tion Method for Defining Agricultural Cases in a Case-Based Reasoning System for Fast Case Retrieval”. In: Sensors 19.23 (Nov. 2019), p. 5118. ISSN: 1424-8220. DOI: 10.3390/s19235118. URL: https://www.mdpi.com/1424-8220/19/23/5118.

[212] Sergio Jiménez Celorrio. “Planning and Learning under Uncertainty”. PhD. Uni- versidad Carlos III de Madrid, 2011, p. 197. URL: https://e-archivo.uc3m.es/ handle/10016/12574.

[213] Z. Kootbally et al. “Towards robust assembly with knowledge representation for the planning domain definition language (PDDL)”. In: Robotics and Computer- Integrated Manufacturing 33 (June 2015), pp. 42–55. ISSN: 07365845. DOI: 10.1016/j.rcim.2014.08.006. URL: https://linkinghub.elsevier.com/retrieve/pii/ S0736584514000672.

– 254 – Bibliography

[214] Vicente Hernández Díaz, José-Fernán Martínez Ortega, Néstor Lucas Martínez, and Raúl del Toro Matamoros. “Self-Adaptive Strategy Based on Fuzzy Control Systems for Improving Performance in Wireless Sensors Networks”. In: Sensors 15.9 (2015), pp. 24125–24142. ISSN: 1424-8220. DOI: 10.3390/s150924125. URL: http://www.mdpi.com/1424-8220/15/9/24125/.

[215] Xin Li, José-Fernán Martínez, Jesús Rodríguez-Molina, and Néstor Martínez. “A Survey on Intermediation Architectures for Underwater Robotics”. In: Sensors 16.2 (2016), p. 190. ISSN: 1424-8220. DOI: 10 . 3390 / s16020190. URL: http : //www.mdpi.com/1424-8220/16/2/190.

[216] Néstor Lucas Martínez, José-Fernán Martínez Ortega, Vicente Hernández Díaz, and Raúl M. Del Toro Matamoros. “Communication Range Dynamics and Perfor- mance Analysis for a Self-Adaptive Transmission Power Controller”. In: Sensors 16.5 (2016), p. 684. DOI: 10.3390/s16050684.

[217] Zhaoyu Zhai, José-Fernán Martínez Ortega, Néstor Lucas Martínez, and Jesús Rodríguez-Molina. “A Mission Planning Approach for Precision Farming Systems Based on Multi-Objective Optimization”. In: Sensors 18.6 (June 2018), p. 1795. ISSN: 1424-8220. DOI: 10.3390/s18061795. URL: http://www.mdpi.com/1424- 8220/18/6/1795.

[218] Zhaoyu Zhai, José Fernán Martínez, Victoria Beltran, and Néstor Lucas Martínez. “Decision support systems for agriculture 4.0: Survey and challenges”. In: Com- puters and Electronics in Agriculture 170 (Mar. 2020), p. 105256. ISSN: 01681699. DOI: 10.1016/j.compag.2020.105256. URL: https://linkinghub.elsevier.com/ retrieve/pii/S0168169919316497.

[219] Zhaoyu Zhai, José Fernán Martínez, Néstor Lucas Martínez, and Vicente Hernán- dez Díaz. “Applying case-based reasoning and a learning-based adaptation strategy to irrigation scheduling in grape farming”. In: Computers and Electronics in Agriculture 178.August (2020), p. 105741. ISSN: 01681699. DOI: 10.1016/j. compag.2020.105741. URL: https://doi.org/10.1016/j.compag.2020.105741.

[220] DEMANES Repository. 2015. URL: https://github.com/demanes (visited on 2018-09-16).

[221] Néstor Lucas Martínez et al. Reasoning engine for SunSPOT. 2015. URL: https: //github.com/DEMANES/Reasoning-Engine-for-SunSPOT (visited on 2020-12- 15).

[222] IEEE. 42010-2011 - ISO/IEC/IEEE Systems and software engineering – Archi- tecture description. IEEE, 2011, p. 46. URL: https://ieeexplore.ieee.org/stamp/ stamp.jsp?arnumber=6129467 (visited on 2020-12-20).

– 255 – Bibliography

[223] ISO/IEC. ISO/IEC 2382:2015 - Information technology — Vocabulary. 2015. URL: https://www.iso.org/standard/63598.html (visited on 2020-12-15).

[224] J.G. Bellingham. “Platforms: Autonomous Underwater Vehicles”. In: Encyclo- pedia of Ocean Sciences. Elsevier, 2009, pp. 473–484. DOI: 10.1016/B978- 012374473 - 9 . 00730 - X. URL: https : / / linkinghub. elsevier. com / retrieve / pii / B978012374473900730X.

[225] ITU Telecommunication Standardization Sector (ITU-T). Recommendation ITU-T Y.2002: Overview of Ubiquitous Networking and of its Support in NGN. 2002. URL: https://www.itu.int/rec/T-REC-Y.2002/en.

[226] ITU Telecommunication Standardization Sector (ITU-T). Recommendation ITU-T Y.2201: Requirements and Capabilities for ITU-T NGN. ITU Telecommunication Standardization Sector (ITU-T), p. 76. URL: https://www.itu.int/rec/T- REC- Y.2201/en (visited on ).

[227] Wolfgang Emmerich, Mikio Aoyama, and Joe Sventek. “The impact of research on middleware technology”. In: ACM SIGOPS Operating Systems Review 41.1 (Jan. 2007), p. 89. ISSN: 01635980. DOI: 10 . 1145 / 1228291 . 1228310. URL: http://portal.acm.org/citation.cfm?doid=1228291.1228310.

[228] ITU Telecommunication Standardization Sector (ITU-T). Recommendation ITU-T- Y.4106: Requirements and Functional Model for a Ubiquitous Network Robot Plat- form that Supports Ubiquitous Sensor Network Applications and Services. Ed. by ITU Telecommunication Standardization Sector (ITU-T). ITU Telecommunication Standardization Sector (ITU-T), Mar. 2016, p. 26. URL: https://www.itu.int/rec/T- REC-F.747.3/en (visited on 2020-12-15).

– 256 – Index

ACCUS, 15, 226 POMDP, see POMDP Adaptation Autonomous Robot,3,5, 10, 229 Decentralized Control Patterns AUV, 72, 107, 157, 230 Coordinated Control, 50 Balance of Adaptation, 119 Hierarchical Control, 52 Information Sharing, 50 CAF, 159, 183, 191 Master/Slave, 51 Case Based Reasoning, see CBR Regional Planning, 52 CBR, 22, 125, 219 Definition, 42 CDT, 183 Adaptive Models Central Control Station, 183 ECA, see ECA Classical Planning, 23, 66 MAPE-K, see MAPE-K CoMMMA, 12, 61, 118, 127, 157, 215–220 OODA, see OODA Computational Analysis, 129 SPA, see SPA Use Cases, 131 ADL, 37, 37, 213 Workflows, 135 AFarCloud, 10, 14, 107, 218, 226 Functional Model, 123 Agent,3, 10, 21, 26, 35, 42, 70, 101, 118, Overview, 118 229 Requirements, 120 Architectural Reference Model, 12, 61, 118, CPEF, 86, 87, 89, 96 211, 218 CPS, 213 Artificial Intelligence, 22, 23, 37, 41, 69, 229 DDS, 171, 178 Automated Planning Paradigms Decision Trees, 29, 66 Classical Planning, see Classical Plan- DEMANES, 15, 226 ning Domain Models Decision Trees, see Decision Trees Mission Management Domain Model, Hierarchical Task Networks, see HTN see Mission Management Domain MDP, see MDP Model Opportunistic Planning, see Opportunis- Strategy Domain Model, see Strategy tic Planning Domain Model

– 257 – Index

ECA, 45, 46, 67 Use of self-adaptive models, 72 European Research Projects Virtualization, 72 ACCUS, see ACCUS Mission Management Domain Model, 101, AFarCloud, see AFarCloud 110, 111 DEMANES, see DEMANES, 226 MMT, 159, 163, 171, 178, 181, 183, 196 SWARMs, see SWARMs, 227 MTRR, 157, 158, 182, 215, 219, 227 WoO, see WoO Aborting a mission, 181 Activities, 171 HDDL, 41 Description, 165 Hierarchical Task Networks, see HTN Environmental Reports, 179 HTN, 31, 41, 66, 70 Events Reports, 181 ICA (Trident), 45, 83, 84, 85, 87, 95, 96, Status Reports, 178 158 Validation Conclusions, 208 Internet of Things, see IoT Validation Details, 182 IoT,4,6, 54, 118, 123, 212, 213, 218, 230 Validation Results, 196 Standards, 57 Virtualization, 169 IoT-A, 61, 67, 118, 127, 169, 215, 218 NDDL, 41 JCR, 223 OASIS, 55

MA-PDDL, 41 OODA, 44, 67, 85, 86 MAPE-K, 46, 47, 49, 67, 97, 118, 124, 125, Opportunistic Planning, 32, 66

140, 143, 213 PDDL, 37, 66, 70, 71, 93, 96, 213, 220 MAPL, 41 Plan,3 MDP, 25, 27, 41, 66 Planning Description Languages, 34 Middleware, 118 ADL, see ADL Mission,3, 112 PDDL, see PDDL Mission Control Language (MCL), 81 STRIPS, see STRIPS Mission Management Architectures POMDP, 27, 66, 110 Architectures PPDDL, 41, 71 CPEF, see CPEF ICA (Trident), see ICA (Trident) RAUVI, 80, 81, 83, 95, 96, 158 RAUVI, see RAUVI RDDL, 41, 70, 71, 96 ROSPlan, see ROSPlan Reference Architectures T-REX, see T-REX IoT-A, see IoT-A TRAC, see TRAC SOA, see SOA Features ROSPlan, 92, 93, 94, 96 Mission plan dispatching, 75 ROV, 157 Mission plan specification, 70 SOA, 54, 54, 62, 67, 213

– 258 – Index

SPA, 46, 67, 77 State of the Art Automated Planning Paradigms, see Automated Planning Paradigms Strategic Research Agendas,7 Strategy, 12, 106, 108, 214 Definition, 109 Domain Model, 108 Formalization, 107 Strategy Domain Model, 108 STRIPS, 34, 37, 70, 71, 213 SWARMs, 10, 11, 14, 69, 107, 157, 159, 188, 215, 217–219, 226 Architecture, 159 P/S Manager, 162 Reporters, 163 Semantic Query, 161 Mission plan, 163 SWARMs project, 183

T-REX, 47, 77, 80, 95, 96, 158 TRAC, 88, 88, 90, 92, 95

UAV, 88, 107 UGV, 107 USV, 107, 157, 231

WoO, 15, 225

– 259 –