MAPPING THE DEVELOPMENT OF AUTONOMY IN WEAPON SYSTEMS vincent boulanin and maaike verbruggen MAPPING THE DEVELOPMENT OF AUTONOMY IN WEAPON SYSTEMS vincent boulanin and maaike verbruggen November 2017 STOCKHOLM INTERNATIONAL PEACE RESEARCH INSTITUTE SIPRI is an independent international institute dedicated to research into conflict, armaments, arms control and disarmament. Established in 1966, SIPRI provides data, analysis and recommendations, based on open sources, to policymakers, researchers, media and the interested public. The Governing Board is not responsible for the views expressed in the publications of the Institute. GOVERNING BOARD Ambassador Jan Eliasson, Chair (Sweden) Dr Dewi Fortuna Anwar (Indonesia) Dr Vladimir Baranovsky (Russia) Ambassador Lakhdar Brahimi (Algeria) Espen Barth Eide (Norway) Ambassador Wolfgang Ischinger (Germany) Dr Radha Kumar (India) The Director DIRECTOR Dan Smith (United Kingdom) Signalistgatan 9 SE-169 72 Solna, Sweden Telephone: +46 8 655 97 00 Email: [email protected] Internet: www.sipri.org © SIPRI 2017 Contents Acknowledgements v About the authors v Executive summary vii Abbreviations x 1. Introduction 1 I. Background and objective 1 II. Approach and methodology 1 III. Outline 2 Figure 1.1. A comprehensive approach to mapping the development of autonomy 2 in weapon systems 2. What are the technological foundations of autonomy? 5 I. Introduction 5 II. Searching for a definition: what is autonomy? 5 III. Unravelling the machinery 7 IV. Creating autonomy 12 V. Conclusions 18 Box 2.1. Existing definitions of autonomous weapon systems 8 Box 2.2. Machine-learning methods 16 Box 2.3. Deep learning 17 Figure 2.1. Anatomy of autonomy: reactive and deliberative systems 10 Figure 2.2. Complexity factors in creating autonomy 14 3. What is the state of autonomy in weapon systems? 19 I. Introduction 19 II. Existing functions and capabilities 20 III. Autonomy for mobility 21 IV. Autonomy for targeting 24 V. Autonomy for intelligence 27 VI. Autonomy for interoperability 29 VII. Autonomy for the health management of systems 34 VIII. Mapping existing ‘semi-autonomous’ and ‘autonomous’ weapon systems 36 IX. Air defence systems 36 X. Active protection systems 41 XI. Robotic sentry weapons 44 XII. Guided munitions 47 XIII. Loitering weapons 50 XIV. Conclusions 54 Box 3.1. Typology of the human–weapon command-and-control relationship 26 according to Human Rights Watch Figure 3.1. Military systems included in the SIPRI dataset by (a) frequency of 20 weapon systems compared with unarmed systems; (b) field of use; and (c) status of development Figure 3.2. Autonomous functions in existing military systems, 21 by capability area Figure 3.3. Autonomy in ‘semi-autonomous’ and ‘autonomous’ weapon systems 36 Figure 3.4. Short-range air defence systems: Phalanx close-in weapon system 38 Figure 3.5. Long-range air defence systems: Patriot missile defence system 39 Figure 3.6. Countries with ‘automatic’ or ‘semi-automatic’ air defence systems 40 Figure 3.7. Active protection systems: T-80 Arena KAZT 41 Figure 3.8. Countries with active protection systems (APSs) 42 Figure 3.9. Robotic sentry weapons: DODAAM’s Super aEgis II 45 Figure 3.10. Countries with robotic sentry weapons 46 Figure 3.11. Guided munitions: Dual-Mode Brimstone 48 Figure 3.12. Loitering weapons 51 iv mapping the development of autonomy in weapon systems Figure 3.13. Countries with loitering weapons with and without autonomous 52 engagement Table 3.1. Command-and-control structure for collective systems, 32 including swarms 4. What are the drivers of, and obstacles to, the development of autonomy 57 in weapon systems? I. Introduction 57 II. Mapping the drivers: to what extent and why is the military interested in 57 autonomy? III. Mapping the obstacles to further incorporation of autonomy in weapon systems 65 IV. Conclusions 82 Box 4.1. Validation and verification: existing methods and their limitations 70 Box 4.2. The limits of affordability: Augustine’s Law and the escalating cost of 82 weapon systems Figure 4.1. Weapon systems accessibility based on financial and organizational 78 capital required for adoption Table 4.1. Possible missions for autonomous (weapon) systems according to US 62 strategic documents Table 4.2. Countries with the highest military expenditure, 2006–15 80 5. Where are the relevant innovations taking place? 85 I. Introduction 85 II. What is innovation and why is it difficult to track in the context of machine 85 autonomy? III. A science and technology perspective: autonomy and academia 89 IV. A geographical perspective: state-funded R&D 94 V. An industry perspective 105 VI. Conclusions 111 Box 5.1. Artificial general intelligence versus specialized artificial intelligence 92 Box 5.2. The robotics industry: an amorphous industry 107 Box 5.3. Challenges related to the commercialization of self-driving vehicles 108 Figure 5.1. Major research fields in machine autonomy 90 Figure 5.2. US Department of Defense (US DOD) funding distribution on 96 applied research and advanced technology development on autonomy in US fiscal year 2015 in millions of US dollars Figure 5.3. Robotic and autonomous systems industry 106 Table 5.1. Government research and development (R&D) spending in the 95 10 largest arms- producing countries and China 6. Conclusions 113 I. Key findings 113 II. Recommendations for future CCW discussions on LAWS 118 Glossary 123 Appendix 125 Table A. Air defence systems with autonomous engagement 125 Table B. Active protection systems with autonomous engagement 126 Table C. Robotic sentry weapons with autonomous engagement 127 Table D. A non-representative sample of guided munitions 127 Table E. Loitering weapons with autonomous engagement 128 Table F. A non-representative sample of unmanned combat systems with 129 autonomous capabilities in their critical functions Table G. Top 10 research institutions in the field of artificial intelligence 130 based on volume of academic publications in sample of relevant topics, 2011–16 Table H. Top 15 research institutions in the field of robotics based on volume 131 of academic publications in sample of relevant topics, 2000–16 Acknowledgements This report was produced through the generous financial support from the Federal Foreign Office of Germany, the Ministry of Foreign Affairs of the Netherlands, the Ministry for Foreign Affairs of Sweden, and the Federal Department for Foreign Affairs of Switzerland. SIPRI and the authors would like to express sincere gratitude to the sponsors for the support they provide for independent research and analysis. The views and opinions in this report are solely those of the authors. The authors are also indebted to all the experts who agreed to participate in back- ground interviews for their rich and invaluable input. The authors wish to thank, in particular, the peer-reviewers Martin Hagström, Ludovic Righetti and Raj Madhavan for their very comprehensive and constructive feedback. Finally, the authors would like to acknowledge the invaluable editorial support of Joey Fox, John Batho and Kate Blanchfield. Responsibility for the information set out in this report lies entirely with the authors. About the authors Dr Vincent Boulanin (France/Sweden) is a Researcher within Armament and Disarmament at SIPRI and the principal investigator of the SIPRI Mapping Study on the Development of Autonomy in Weapon Systems. He works on issues related to the production, use and control of emerging military and security technologies. Before joining SIPRI in 2014, he completed a PhD in Political Science at École des Hautes Études en Sciences Sociales (the School for Advanced Studies in the Social Sciences) in Paris. Maaike Verbruggen (Netherlands) joined SIPRI in April 2016 to work as a research assistant with the SIPRI Mapping Study on the Development of Autonomy in Weapon Systems. She left SIPRI in November 2017 to work as a PhD researcher at the Vrije Uni versiteit in Brussels. Before joining SIPRI, she completed a Master of Philosophy degree in Peace and Conflict Studies at Oslo University. Executive summary This report presents the conclusions of a one-year mapping study on the development of autonomy in weapon systems. It is intended to provide diplomats and members of civil society interested in the issue of lethal autonomous weapon systems (LAWS) with a better understanding of (a) the technological foundations of autonomy; (b) the state of autonomy in existing weapon systems; (c) the drivers of, and obstacles to, further increasing autonomy in weapon systems; and (d) the innovation ecosystems behind the advance of autonomy in weapon systems. I. Key findings What are the technological foundations of autonomy? Chapter 2 explores the technological foundations of autonomy. The main findings are as follows. 1. Autonomy has many definitions and interpretations, but is generally understood to be the ability of a machine to perform an intended task without human intervention using interaction of its sensors and computer programming with the environment. 2. Autonomy relies on a diverse range of technology but primarily software. The feasibility of autonomy depends on (a) the ability of software developers to formulate an intended task in terms of a mathematical problem and a solution; and (b) the possi- bility of mapping or modelling the operating environment in advance. 3. Autonomy can be created or improved by machine learning. The use of machine learning in weapon systems is still experimental, as it continues to pose fundamental problems regarding predictability. What is the state of autonomy in weapon systems? Chapter 3 explores the state of autonomy in deployed weapon systems and weapon systems under development. The main findings are as follows. 1. Autonomy is already used to support various capabilities in weapon systems, including mobility, targeting, intelligence, interoperability and health management. 2. Automated target recognition (ATR) systems, the technology that enables weapon systems to acquire targets autonomously, has existed since the 1970s. ATR systems still have limited perceptual and decision-making intelligence. Their performance rapidly deteriorates as operating environments become more cluttered and weather conditions deteriorate.