Software Engineering for Robotics Ana Cavalcanti • Brijesh Dongol • Rob Hierons • Jon Timmis • Jim Woodcock Editors

Software Engineering for Robotics Editors Ana Cavalcanti Brijesh Dongol Department of Computer Science University of Surrey University of York Guildford, UK York, UK

Rob Hierons Jon Timmis University of Shefﬁeld University of Sunderland Shefﬁeld, UK Sunderland, UK

Jim Woodcock Department of Computer Science University of York York, UK

ISBN 978-3-030-66493-0 ISBN 978-3-030-66494-7 (eBook) https://doi.org/10.1007/978-3-030-66494-7

Chapters ‘Towards Autonomous Robot Evolution’, ‘Composition, Separation of Roles and Model-Driven Approaches as Enabler of a Robotics Software Ecosystem’ and ‘Verifiable Autonomy and Responsible Robotics’ are licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/). For further details see license information in the chapters. This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG. The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland Foreword

As a Professor of Robotics and Autonomous Systems, I have been carrying out research into advanced robotics for over 30 years, half of that as Deputy Director of the Bristol Robotics Laboratory, which is one of the most comprehensive robot research laboratories in the UK. During that period, I have been lucky enough to have the pleasure of getting to know many of this book’s editors and chapter co-authors, and, indeed, I have worked closely with some of them. During that period also, all of those involved in writing or coordinating the book’s contents have become eminent figures in their own right, in recognition of their individual achievements in advancing the field of Software Engineering for Robotics. As a result, I am pleased and honoured that I have been asked to write this foreword. Robotics and the sectors of software engineering associated with it are critically linked. There is a growing public perception that robots will soon emerge from the confines of “factory shop-floor”-style repetitive motion sequences to work with us and amongst us, thus improving both our domestic and professional lives. However, for this aspiration to succeed, it is essential that critical aspects of software engineering and advanced robotics develop together in close synchrony. On one hand, robotics provides a physically observable manifestation that enables software systems to interact with human beings and other artefacts in an embedded and instantiated manner. On the other hand, meeting the challenges presented by the many requirements arising from the typically complex nature of those interactions is fully reliant on the success of the software engineering design efforts that are described, demonstrated and proposed in this book. Taken together, the chapter authors of this book have achieved a tour de force that ranges freely across the wide scope of the field of Software Engineering for Robotics and the exceptional advances currently being made in it. This rich content has been linked together deftly by its editors. If a reader seeks to discover all that is important in this field, and understand the pivotal issues that it faces, then this book should be one of that reader’s first ports of call. By absorbing its content, the reader will come to understand many of the most fundamental, and often interlinked, challenges that must be overcome in order to bring the promised “robot revolution” to successful fruition, including the

v vi Foreword society-shifting beneﬁts that could be realised by it, as well as the crucial dangers and pitfalls that must be circumvented in doing so.

Professor of Robotics and Autonomous Systems Tony (Anthony) Pipe Bristol Robotics Laboratory University of the West of England Bristol, UK Preface

The origin of this book is a 2-day event, entitled RoboSoft, that took place in November 2019 in London. Organised with the generous support of the Royal Academy of Engineering and the University of York, UK, RoboSoft brought together more than 100 scientists, engineers and practitioners from all over the world, representing 70 institutions to discuss the state of the art and practice on Software Engineering for Robotics. The lively discussions that followed each of the talks and the panel session provide evidence for the importance of this area and for the exciting challenges that lie ahead. The chapters of this book are co-authored by a RoboSoft speaker, and provide a snapshot of the diversity of the work that is being carried out in this area, of the research and insights available and of future possibilities. The list of these speakers, with a short biography, follows. The topics covered range from programming languages and environments, via approaches for design and verification, to issues of ethics, accident investigation and regulation. In terms of techniques, there are results on product lines, mission specification, component-based development, simulation, testing and proof. Appli- cations range from manufacturing to service robots, to autonomous vehicles, and even to robots that evolve in the real world. The views put forward by the various authors are also diverse. Some cover approaches that are successful in other areas and can be translated for use in robotics. Some authors state that there is nothing special about Software Engineering for Robotics, while the starting point for others are the peculiarities of this domain. Some take the view that there is no time for toy examples; others use such examples to drive foundational research in robotics. Some focus on the decision aspects of an autonomous robot, with the view that dealing with functional aspects is a standard task. Others understand that models for decision components are formal in nature, so the challenge is verification of functional components. So, the reader, has a breadth of perspectives and arguments, and the opportunity to make their own mind up. The structure of the book follows that adopted in organising the RoboSoft presentations, to cover the topics of algorithms, modelling, testing, ethics and formal

vii viii Preface verification. A final chapter summarises the discussions on ethics and regulation, including the material based on the panel at the end of RoboSoft. The intended readership of this book includes researchers and practitioners with all levels of experience interested in working in the area of robotics, and software engineering more generally. The chapters are all written to be accessible to a wide readership and are self-contained, including explanations of the core concepts that they cover to aid the reader. The chapters also provide links with the material in other chapters to give the book a cohesive and comprehensive account of the state of the art and practice. Finally, the chapters include a discussion of directions for further work. We are grateful for the contribution of all speakers and attendees of RoboSoft and of all co-authors of the RoboSoft speakers. We also would like to thank the administrators that worked on behalf of the Royal Academy of Engineering and of the University of York to make RoboSoft and, therefore, this book possible. The enthusiasm of all these colleagues has given us all extra encouragement to produce this book as a snapshot of Software Engineering for Robotics as an emergent field of study. We look forward to working together, as a community, to contribute to this fascinating multidisciplinary area. It will no doubt have a significant and positive impact on the quality of life of our society. We envisage advancements to economy, manufacturing, transportation, health, entertainment, safety and much more. Our hope is that, in 5 or 10 years, we will look back and decide that it is definitely time for a second edition.

York, UK Ana Cavalcanti Guildford, UK Brijesh Dongol Shefﬁeld, UK Rob Hierons Sunderland, UK Jon Timmis York, UK Jim Woodcock October 2020 Contents

1 Software Product Line Engineering for Robotics ...... 1 Davide Brugali 2 Towards Autonomous Robot Evolution ...... 29 Agoston E. Eiben, Emma Hart, Jon Timmis, Andy M. Tyrrell, and Alan F. Winfield 3 Composition, Separation of Roles and Model-Driven Approaches as Enabler of a Robotics Software Ecosystem ...... 53 Christian Schlegel, Alex Lotz, Matthias Lutz, and Dennis Stampfer 4 Testing Industrial Robotic Systems: A New Battlefield! ...... 109 Arnaud Gotlieb, Dusica Marijan, and Helge Spieker 5 Gaining Confidence in the Trustworthiness of Robotic and Autonomous Systems...... 139 Kerstin Eder 6 Robot Accident Investigation: A Case Study in Responsible Robotics...... 165 Alan F. T. Winfield, Katie Winkle, Helena Webb, Ulrik Lyngs, Marina Jirotka, and Carl Macrae 7 Verifiable Autonomy and Responsible Robotics...... 189 Louise Dennis and Michael Fisher 8 Verification of Autonomous Robots: A Roboticist’s Bottom-Up Approach ...... 219 Félix Ingrand 9 RoboStar Technology: A Roboticist’s Toolbox for Combined Proof, Simulation, and Testing ...... 249 Ana Cavalcanti, Will Barnett, James Baxter, Gustavo Carvalho, Madiel Conserva Filho, Alvaro Miyazawa, Pedro Ribeiro, and Augusto Sampaio

ix x Contents

10 CorteX: A Software Framework for Interoperable, Plug-and-Play, Distributed, Robotic Systems of Systems ...... 295 Ipek Caliskanelli, Matthew Goodliffe, Craig Whifﬁn, Michail Xymitoulias, Edward Whittaker, Swapnil Verma, Craig Hickman, Chen Minghao, and Robert Skilton 11 Mutation Testing for RoboChart ...... 345 Robert M. Hierons, Maciej Gazda, Pablo Gómez-Abajo, Raluca Lefticaru, and Mercedes G. Merayo 12 Languages for Specifying Missions of Robotic Applications...... 377 Swaib Dragule, Sergio García Gonzalo, Thorsten Berger, and Patrizio Pelliccione 13 RoboStar Technology: Modelling Uncertainty in RoboChart Using Probability ...... 413 Jim Woodcock, Simon Foster, Alexandre Mota, and Kangfeng Ye 14 Panel Discussion: Regulation and Ethics of Robotics and Autonomous Systems...... 467 Brijesh Dongol, Ron Bell, Ibrahim Habli, Mark Lawford, Pippa Moore, and Zeyn Saigol Editors and Contributors

We list here the speakers and organisers of the RoboSoft event. We acknowl- edge and appreciate also the contribution of all co-authors of the chapters. Ron Bell (Engineering Safety Consultants) is one of the most influential figureheads in the field, having been awarded an OBE. In 1998, he was appointed as one of the five UK members of the binational Channel Tunnel Safety Authority, which is a post he held for 13 years. He chairs one of the two IEC working groups responsible for IEC 61508 (the international standard dealing with safety-critical systems), a post that he has held since 1987. In 2005, he received the IEC 1906 Award for his work on functional safety and IEC 61508. He held a 3-year appointment (2015–2018) as a Royal Academy of Engineering Visiting Professor at Liverpool John Moores University.

Davide Brugali (University of Bergamo) graduated in Electronic Engineering at Politecnico di Milano in 1994; he received a PhD in Computer Science from Politecnico di Torino in 1998. Since 2011, he is Associate Professor at the University of Bergamo. He has been Visiting Researcher at the CMU Robotics Institute in 1997 and Visiting Professor at NASA Jet Propulsion Laboratory in 2006. From 2000 to 2020, he has been co-chair of the IEEE RAS Technical Committee on “Software Engineering for Robotics and Automation”. He was editor and co-author of a Springer STAR book on Software Engineering for Experimental Robotic (2006). He is the main author of the book Software Development: Case Studies in Java published by Addison-Wesley in 2005. xi xii Editors and Contributors

Ana Cavalcanti (University of York) is Professor of Software Veriﬁcation and Royal Academy of Engi- neering Chair in Emerging Technologies working on Software Engineering for Robotics: modelling, validation, simulation and testing. She currently leads the RoboStar research group at the University of York. She held a Royal Society Wolfson Research Merit Award and a Royal Society Industry Fellowship to work with QinetiQ in avionics. She has chaired the programme committee of various well-established international conferences, is on the editorial board of four international journals and is chair of the board of the Formal Methods Europe Association. She is, and has been, Principal Investigator on several large research grants. Her current research is on the theory and practice of veriﬁcation and testing for robotics.

Brijesh Dongol (University of Surrey) is a Senior Lec- turer. His research is on the formal techniques and ver- iﬁcation methods for concurrent and real-time systems. This includes concurrent objects, transactional memory and associated correctness conditions; weak memory models; algebraic techniques; and hybrid systems. He completed his PhD in 2009 from the University of Queensland. He was a postdoctoral researcher at the University of Shefﬁeld and lecturer at Brunel University London before moving to Surrey. He leads several projects funded by the EPSRC, research institutions and industrial partners. He is a member of the Formal Methods Teaching Committee.

Kerstin Eder (University of Bristol) is Professor of Computer Science and leads the Trustworthy Systems Laboratory at the University of Bristol, as well as the Verification and Validation for Safety in Robots research theme at the Bristol Robotics Laboratory. Her research is focused on specification, verification and analysis techniques to verify or explore a system’s behaviour in terms of functional correctness, safety, performance and energy efficiency. Kerstin has gained extensive expertise in verifying complex microelec- tronic designs at leading semiconductor design and EDA companies. She seeks novel combinations of formal methods with state-of-the-art simulation and Editors and Contributors xiii

test-based approaches to achieve solutions that make a difference in practice. She holds a PhD in Computa- tional Logic, an MSc in Artiﬁcial Intelligence and an MEng in Informatics. In 2007, she was awarded a Royal Academy of Engineering “Excellence in Engineering” prize.

Gusz Eiben (Vrije Universiteit Amsterdam) is Professor of Artificial Intelligence at the Vrije Universiteit Ams- terdam, where he leads the Computational Intelligence Group, and Visiting Professor at the University of York, UK. His research lies in the fields of Artificial Intelli- gence, Artificial Life, and Adaptive Collective Systems. His approach to Artificial Intelligence is based on Evo- lutionary Computing. Over more than 30 years, he has worked on the theoretical foundations and applications in health, finance and traffic management, built a system to evolve Mondriaan- and Escher-style art and exhib- ited it in the Haags Gemeentemuseum, researched how artificial societies can emerge in the computer through evolution and learning, invented and tested reproduction mechanisms that use more than two parents and studied how evolutionary processes can be (self-)calibrated. Lately he has been active in Evolutionary Robotics.

Michael Fisher (University of Manchester) holds a Royal Academy of Engineering Chair in Emerg- ing Technologies on the theme of “Responsible Autonomous Systems”. His research particularly concerns formal verification and autonomous systems, as well as safety and ethics in autonomous robotics. He serves on the editorial boards of both the Journal of Applied Logic and the Annals of Mathematics and Arti- ficial Intelligence and is a corner editor for the Journal of Logic and Computation. He is a Fellow of both the BCS and the IET, is a member of the British Standards Institute’s AMT/10 “Robotics” standards committee and is a member of the IEEE’s P7009 “Fail-Safe Design of Autonomous System” standards committee. He is involved in a range of EPSRC projects across robotics for hazardous environments and sensor network analysis and is co-chair of the IEEE’s international Technical Committee on the Verification of Autonomous Systems. xiv Editors and Contributors

Arnaud Gotlieb (Simula Research Laboratory)isChief Research Scientist and Research Professor at Simula Research Laboratory in Norway. His research interests are on the application of Artificial Intelligence to the validation of software-intensive systems, cyber-physical systems including industrial robotics, and autonomous systems. Arnaud has co-authored more than 120 publications on Artificial Intelligence and Software Engi- neering and has developed several tools for testing critical software systems. He was the scientific coordinator of the French ANR-CAVERN project (2008– 2011) for Inria and led the Research-Based Innovation Center Certus dedicated to Software Validation and Verification (2011–2019) at Simula. He was recently awarded with the prestigious RCN FRINATEK grant for the T-LARGO project on testing learning robots (2018– 2022). He leads the industrial pilot experiments of the H2020 AI4EU Project (2019–2022).

Ibrahim Habli (University of York) is a Senior Lecturer. His research interests are in the design and assur- ance of safety-critical systems, with a particular focus on intelligent systems (for instance, autonomous and connected driving) and digital health (for instance, e- Prescribing and self-management apps). In 2015, he was awarded a Royal Academy of Engineering Indus- trial Fellowship through which he collaborated with the NHS on evidence-based means for assuring the safety of digital health systems. His research is empirical and industry-informed, with collaborative projects with organisations such as Rolls-Royce, NASA, Jaguar Land Rover and NHS Digital. Ibrahim is an academic lead on the Assuring Autonomy International Programme, a £12 million initiative funded by the Lloyd’s Register Foundation and the University of York. He has been a member of several safety standardisation committees (e.g. DO178C, MISRA and BSI). Editors and Contributors xv

Rob Hierons (University of Sheffield) received a BA in Mathematics (Trinity College, Cambridge) and a PhD in Computer Science (Brunel University). He then joined the Department of Mathematical and Computing Sciences at Goldsmiths College, University of London, before returning to Brunel University in 2000. He was promoted to full professor in 2003 and joined the University of Sheffield in 2018. His research concerns the automated generation of efficient, systematic test suites on the basis of program code, models or speci- fications. He is joint Editor-in-Chief of the journal of Software Testing, Verification, and Reliability (STVR) and is a member of the editorial boards of The Computer Journal and Formal Aspects of Computing.

Félix Ingrand (LAAS/CNRS) is a tenured researcher at CNRS. After his PhD from the University of Grenoble (1987), he spent 4 years at SRI International (Menlo Park, CA) where he worked on procedural reasoning. He joined the Robotics and Artiﬁcial Intelligence Group at CNRS/LAAS in 1991. His work deals with architecture for autonomous systems with an emphasis on the decisional aspect. He has been invited to NASA Ames Research Center to work on various robotics platforms (K9 and Gromit) to study the use of the LAAS Architecture and tools on those platforms, and conduct research on the development of a temporal planner and execution control system based on the IDEA/Europa planner. Recently, Félix has worked on extending the LAAS Architecture to support formal validation, veriﬁ- cation and correct controller synthesis. xvi Editors and Contributors

Mark Lawford (McMaster University) is Chair of McMaster University’s Department of Computing and Software and Director of the McMaster Centre for Soft- ware Certification. He has a BSc (1989) in Engineer- ing Mathematics from Queen’s University, Kingston, where he received the University Medal in Engineering Mathematics. His MASc (1992) and PhD (1997) are from the University of Toronto. He worked at Ontario Hydro as a real-time software verification consultant on the Darlington Nuclear Generating Station Shutdown Systems Redesign project, receiving the Ontario Hydro New Technology Award for Automation of System- atic Design Verification of Safety Critical Software in 1999. Since 2012, he has been involved in automotive software research, and in 2014, he was a co-recipient of a Chrysler Innovation Award. He serves on the steering committee of the Software Certification Con- sortium (SCC). He is a licensed Professional Engineer in Ontario and a senior member of the IEEE.

Pippa Moore (Civil Aviation Authority) worked for GEC-Marconi Avionics prior to joining the UK CAA. While with this organisation, she worked on the development of a range of safety-critical flight control com- puters in both the military and civil fields. She has been a Design Surveyor with the CAA since 1996, specialising in airborne software, airborne electronic hardware and safety assessment. In that time, Pippa has worked as a CAA, JAA and EASA systems specialist on civil aircraft certification projects such as the Boe- ing 737 and 767, A330/340 and A380. She has also worked on numerous engine certification and validation projects. Additionally, Pippa has worked with the regulatory authority teams for several UAS programmes and undertaken research on aviation safety topics that have directly changed aircraft safety regulations. She has spent the last 3 years as the technical lead for the CAA’s Cyber Oversight Programme. Editors and Contributors xvii

Patrizio Pelliccione (University of L’Aquila)isFull Professor at Gran Sasso Science Institute (GSSI) and Associate Professor at the Department of Computer Science and Engineering at Chalmers University of Technology and University of Gothenburg. He received his PhD in 2005 at the University of L’Aquila, and since 2014, he is Docent in Software Engineering, a title given by the University of Gothenburg. His research topics are in software architectures modelling and ver- iﬁcation, autonomous systems and formal methods. He has co-authored more than 120 publications in journals and international conferences and workshops. He has been on the programme committees for several top conferences, is a reviewer for top journals and has chaired the programme committee of several international conferences. He is very active in European and national projects. He is the PI for the Co4Robots H2020 EU project for the University of Gothenburg. In his research activity, he has pursued extensive and wide collaboration with industry.

Zeyn Saigol (Connected Places Catapult)isPrincipal Technologist at Connected Places Catapult, specialising in verification, validation and regulatory approval of autonomous vehicles (AVs). He is the technical lead for the MUSICC project, which is a key pillar of the UK Department for Transport’s contribution to multi- national AV certification. Zeyn is also an interface architect on the VeriCAV project, which is creating a smart simulation testing framework, and is a member of ISO and ASAM committees working on standards for AV testing. His background includes a BSc in Physics from the University of Bristol, master’s degrees from Imperial College and the University of Edinburgh and a PhD in AI from the University of Birmingham. His research interests covered planning under uncertainty, knowledge representation, mapping, path planning and machine learning. He has worked with a variety of autonomous systems, including wheeled, flying and marine robots, in roles spanning academia and industry. xviii Editors and Contributors

Christian Schlegel (Technische Hochschule Ulm)is Head of the Service Robotics Research Group, Pro- fessor for Real-Time and Autonomous Systems in the Computer Science Department since 2004 and co-opted member of the Faculty of Engineering, Computer Sci- ence and Psychology of the University of Ulm. He is the technical lead of the EU H2020 RobMoSys project and the elected coordinator of the euRobotics Topic Group on Software Engineering, System Integration, System Engineering. Christian is co-founder and Associate Editor of the open access journal JOSER—Journal of Software Engineering for Robotics—and co-organiser of the series of International Workshop on Domain- Speciﬁc Languages and Models for Robotics Systems (DSLRob). He was awarded a Diploma and PhD in Computer Science in 1993 and 2004, respectively.

Rob Skilton (Remote Applications in Challenging Environments—RACE) is Head of Research at RACE, a UK centre for Remote Applications in Challenging Environments, where he leads a team specialising in control systems, autonomy and perception for robotic operation and inspection in hazardous environments. Robert graduated with an MSc in Cybernetics in 2011 and is currently studying for a PhD in Autonomous Robotics and Machine Learning at the Surrey Tech- nology for Autonomous Systems and Robotics (STAR) Lab. Robert is a Chartered Engineer, brings experience in developing robotic systems for hazardous environments and has developed numerous robotic and software platforms for use in nuclear and other extreme environments. He has experience from a wide range of roles in industrial engineering and R&D projects including in telerobotics and is currently leading various activities including the Robotics and AI in Nuclear (RAIN) work on the teleoperation of industrial robots. Editors and Contributors xix

Jon Timmis (University of Sunderland) is Professor of Intelligent and Adaptive Systems and Deputy Vice- Chancellor. He is Visiting Professor at the University of York. Jon has worked for over 20 years in the area of biologically-inspired systems and computational biol- ogy. His research cuts across many areas, but the major- ity of his work revolves around immunology, either developing computational models of immune func- tion (computational immunology), or fault-tolerance achieved via bio-inspired engineering with a focus on the immune system and evolutionary processes. Jon has worked extensively on swarm robotic systems and adaptive autonomous robotic systems. Jon is a previous recipient of a Royal Society Wolfson Research Merit Award and a Royal Academy of Engineering Enterprise Fellowship. Jon co-founded Simomics Ltd in 2014 to commercialise his research.

Alan Winﬁeld (University of the West of England) is Professor of Robot Ethics, Visiting Professor at the University of York and Associate Fellow of the Cambridge Centre for the Future of Intelligence. He co-founded and led APD Communications Ltd until taking up appointment at the University of the West of England (UWE), Bristol, in 1992. Alan co-founded the Bristol Robotics Laboratory, where his research is focused on the science, engineering and ethics of cognitive robotics. He is passionate about communi- cating research and ideas in science, engineering and technology; he led the UK-wide public engagement project “Walking with Robots” and was awarded the 2010 Royal Academy of Engineering Rooke medal for public promotion of engineering. Alan sits on the execu- tive of the IEEE Standards Association Global Initiative on Ethics of Autonomous and Intelligent Systems and chairs Working Group P7001, drafting a new IEEE standard on Transparency of Autonomous Systems. He is a member of the World Economic Forum Global AI Council. xx Editors and Contributors

Jim Woodcock (University of York) is Professor of Software Engineering, a Fellow of the Royal Academy of Engineering and an award-winning researcher. He is also Professor of Digital Twins at Aarhus Univer- sity. He has 40 years’ experience in formal methods. His research interests are in unifying theories of programming (UTP), robotic digital twins and industrial applications. Formerly, he worked on applying the Z notation to the IBM CICS project, helping to gain a Queen’s Award for Technological Achievement in 1992. He created the theory and practical veriﬁcation for NatWest Bank’s Mondex smart-card system, the ﬁrst commercial product to achieve ITSEC Level E6 (Common Criteria EAL 7). For the last decade, he has researched the theory and practice of cyber-physical systems and robotics. He led the team that developed extensive UTP theories and the Isabelle/UTP theorem prover. He is Editor-in-Chief of the prestigious Springer journal Formal Aspects of Computing.