EXPERT MEETING AUTONOMOUS WEAPON SYSTEMS IMPLICATIONS OF INCREASING AUTONOMY IN THE CRITICAL FUNCTIONS OF WEAPONS VERSOIX, SWITZERLAND 15-16 MARCH 2016 International Committee of the Red Cross 19, avenue de la Paix 1202 Geneva, Switzerland T +41 22 734 60 01 F +41 22 733 20 57 E-mail: [email protected] www.icrc.org © ICRC, August 2016 Front cover: Photo credit EXPERT MEETING AUTONOMOUS WEAPON SYSTEMS IMPLICATIONS OF INCREASING AUTONOMY IN THE CRITICAL FUNCTIONS OF WEAPONS VERSOIX, SWITZERLAND 15-16 MARCH 2016 Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 2 Expert meeting, Versoix, Switzerland, 15-16 March 2016. CONTENTS Introduction and structure of the report 5 Part I: Summary report prepared by the International Committee of the Red Cross 7 A. Background 7 B. Summary of presentations and discussions 8 Part II: Selected presentations 23 Characteristics of autonomous weapon systems 23 – Dr Martin Hagström Focusing the debate on autonomous weapon systems: A new approach to linking 26 technology and international humanitarian law – Lt Col. Alan Schuller Missile defence systems that use computers: An overview of the Counter-Rocket, 29 Artillery, and Mortar (C-RAM) System – Dr Brian Hall Missile- and rocket-defence weapon systems 31 – Gp Capt. Ajey Lele (Ret'd) Sensor-fused munitions, missiles, and loitering munitions 33 – Dr Heather Roff Emerging technology and future autonomous weapons 36 – Dr Ludovic Righetti Legal issues concerning autonomous weapon systems 40 – Col. Zhang Xinli Autonomous weapon systems and the alleged responsibility gap 44 – Prof. Paola Gaeta Meaningful human control over individual attacks 46 – Mr Richard Moyes Human control in the targeting process 53 – Ms Merel Ekelhof Lethal Autonomous Weapon Systems (LAWS) 57 – Lt Col. John Stroud-Turp Russia's automated and autonomous weapons and their consideration from a policy standpoint 60 – Dr Vadim Kozyulin Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 3 Expert meeting, Versoix, Switzerland, 15-16 March 2016. Addressing the challenges raised by increased autonomy 65 – Ms Kerstin Vignard Part III: Background paper prepared by the International Committee of the Red Cross 69 1. Introduction 70 2. Characteristics of autonomous weapon systems 71 3. Autonomy in existing weapon systems 72 4. Emerging technology and future autonomous weapon systems 77 5. Legal and ethical implications of increasing autonomy 79 6. Human control 83 Annex 1: Expert meeting programme 86 Annex 2: List of participants 90 Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 4 Expert meeting, Versoix, Switzerland, 15-16 March 2016. INTRODUCTION AND STRUCTURE OF THE REPORT Debates on autonomous weapon systems have expanded significantly in recent years in diplomatic, military, scientific, academic and public forums. In March 2014, the ICRC convened an international expert meeting to consider the relevant technical, military, legal and humanitarian issues.1 Expert discussions at a Meeting of Experts convened by the High Contracting Parties to the UN Convention on Certain Conventional Weapons (CCW) were held in April 2014 and continued in April 2015 and April 2016.2 As a further contribution to the international discussions, the ICRC convened this second expert meeting, entitled Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons, from 15 to 16 March 2016. It brought together representatives from 20 States3 and 14 individual experts in robotics, law, policy and ethics. This report of the meeting is divided into three main sections: Part I is a summary report of the expert meeting, which was prepared by the ICRC under its sole responsibility. Part II comprises summaries of selected presentations given by individual experts at the meeting, and provided under their own responsibility. Part III is an edited version of the background paper prepared by the ICRC and circulated to participants in advance of the expert meeting in March 2016. The meeting programme and the list of participants are provided in Annexes 1 and 2. 1 ICRC (2014) Autonomous weapon systems: technical, military, legal and humanitarian aspects, https://www.icrc.org/en/download/file/1707/4221-002-autonomous-weapons-systems-full-report.pdf. 2 CCW Meetings of Experts on Lethal Autonomous Weapon Systems (LAWS), 2014, 2015 and 2016,. http://www.unog.ch/80256EDD006B9C2E/(httpNewsByYear_en)/0462FC37E62E7E73C1257E2A005A013A?OpenDocument 3 Algeria, Australia, Brazil, China, Egypt, France, Germany, India, Israel, Japan, Mexico, the Netherlands, Pakistan, the Republic of Korea, the Russian Federation, South Africa, Sweden, Switzerland, the United Kingdom and the United States. Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 5 Expert meeting, Versoix, Switzerland, 15-16 March 2016. Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 6 Expert meeting, Versoix, Switzerland, 15-16 March 2016. PART I: SUMMARY REPORT PREPARED BY THE INTERNATIONAL COMMITTEE OF THE RED CROSS Expert Meeting on Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons, 15–16 March 2016, Versoix, Switzerland. A. BACKGROUND Debates on autonomous weapon systems have expanded significantly in recent years in diplomatic, military, scientific, academic and public forums. In March 2014, the ICRC convened an international expert meeting to consider the relevant technical, military, legal and humanitarian issues.1 Expert discussions within the framework of the UN Convention on Certain Conventional Weapons (CCW) were held in April 2014 and continued in April 2015 and April 2016.2 Discussions among government experts have indicated broad agreement that “meaningful”, “appropriate” or “effective” human control over weapon systems and the use of force must be retained, but there has been less clarity on the type and degree of control necessary from a legal, ethical and policy perspective. The ICRC has called on States to set limits on autonomy in weapon systems to ensure that they can be used in accordance with international humanitarian law (IHL) and within the bounds of what is acceptable under the principles of humanity and the dictates of the public conscience.3 In view of the incremental increase of autonomy in weapon systems, specifically in the “critical functions” of selecting and attacking targets, the ICRC has stressed that experience with existing weapon systems can provide insights into where the limits on autonomy in weapon systems should be placed, and the kind and degree of human control that is necessary to ensure compliance with IHL and ethical acceptability. With this in mind, the ICRC held its second expert meeting, entitled “Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons”, from 15 to 16 March 2016. It brought together representatives from 20 States4 and 14 individual experts in robotics, law, policy and ethics, and was held under the Chatham House Rule.5 The six sessions reflected the overall objectives of the meeting, which were to: consider the defining characteristics of autonomous weapon systems; better understand autonomy in the critical functions of existing weapon systems; explore emerging technology and the implications for future autonomous weapon systems; examine the legal and ethical implications of increasing autonomy in weapon systems; consider the legal, military (operational) and ethical requirements for human control over weapon systems and the use of force; and share approaches to addressing the challenges raised by increasing autonomy. 1 ICRC (2014) Autonomous weapon systems: technical, military, legal and humanitarian aspects, https://www.icrc.org/en/download/file/1707/4221-002-autonomous-weapons-systems-full-report.pdf. 2 CCW Meetings of Experts on Lethal Autonomous Weapon Systems (LAWS), 2014, 2015 & 2016, https://www.unog.ch/ccw 3 ICRC (2016) Views of the ICRC on autonomous weapon systems, CCW Meeting of Experts on Lethal Autonomous Weapons Systems (LAWS), 11-15 April 2016, Geneva. Background paper, 11 April 2016, https://www.icrc.org/en/download/file/21606/ccw-autonomous-weapons-icrc-april-2016.pdf. 4 Algeria, Australia, Brazil, China, Egypt, France, Germany, India, Israel, Japan, Mexico, the Netherlands, Pakistan, the Republic of Korea, the Russian Federation, South Africa, Sweden, Switzerland, the United Kingdom, and the United States. 5 https://www.chathamhouse.org/about/chatham-house-rule. Autonomous Weapon Systems: Implications of Increasing Autonomy in the Critical Functions of Weapons. 7 Expert meeting, Versoix, Switzerland, 15–16 March 2016. This summary of the presentations and discussions is provided under the sole responsibility of the ICRC and reflects the key points raised by speakers and participants at the meeting.6 B. SUMMARY OF PRESENTATIONS AND DISCUSSIONS 1. Characteristics of autonomous weapon systems Speakers in this session debated the defining characteristics of autonomous weapon systems with a view to clarifying the terminology and fostering a better understanding of the types of weapons under consideration. The ICRC’s working definition was used as a basis for discussions throughout the meeting, although at times some speakers and participants expressed a different understanding of definitions. Under