OCTOBER 2019 Artificial intelligence and defense: What is at stake? Renaud Bellais RESEARCH PAPER Pensée stratégique Image : ©Daniil Peshkov 1 The Institute of Strategic and Defense Studies (IESD) is an academic research centre created in 2018 and specializing in strategic studies. The IESD is supported by the Université de Lyon (UdL) and belongs to the Law School of Université Jean Moulin – Lyon III. The institute consists of a multi-disciplinary team of researchers (Law, Political Science, Management, Economy) and unites a network of experts, researchers, PhD students and master students specialised in strategic studies. The IESD is currently a candidate for selection as one of the Defense Ministry’s (DGRIS)“National Centre of Defense Excellence” focusing on: “Interconnection of high strategic functions (air power, space, nuclear deterrence, missile defense): political and operational implications of high intensity capability interactions in homogeneous spaces and contested commons.” Director of IESD: Olivier Zajec Collection: Defense capabilities analysis Site web : https://iesd.univ-lyon3.fr/ Contact : [email protected] IESD – Faculté de droit Université Jean Moulin – Lyon III 1C avenue des Frères Lumière – CS 78242 69372 LYON CEDEX 08 2 Renaud Bellais, « Artificial Intelligence and Defense: What is at Stake? », Note de recherche IESD, coll. « Analyse technico-capacitaire », n°1, October 2019. Abstract Artificial intelligence is currently a “hot topic” in defense matters. This new technology simultaneously creates fears and sets high expectations. Despite the large amount of literature written about AI and defense, it is difficult to distinguish its true impacts and estimate when they could occur. This policy paper aims to provide a comprehensive assessment, which goes beyond prejudices and misunderstandings. It underlines how AI constitutes a general-purpose technology with huge potential that could turn defense and warfare upside down. However, we must keep in mind its advantages and limits while considering its growth potential on different time scales. In addition, the military adoption of AI-based systems should not be considered a guarantee. The integration of technology is a complex process that ultimately affects how AI would change the art of war. Résumé L’intelligence artificielle est aujourd’hui un des sujets les plus brûlants pour la défense. Cette nouvelle technologie induit à la fois de nombreuses craintes et de très hautes attentes. Malgré une littérature foisonnante sur l’IA et la défense, il n’est pas aisé de comprendre les impacts réels de cette relation, ni même de distinguer le moment où ils pourraient produire des résultats concrets. Cette note de recherche a pour objectif d’aller au- delà des préjugés et des exagérations. Elle souligne que l’IA constitue une technologie polyvalente avec un immense potentiel qui pourrait transformer la défense et les formes de la guerre contemporaine. Il est néanmoins nécessaire de garder à l’esprit ses avantages et ses limites et de considérer son évolution potentielle sur plusieurs échelles de temps. De plus, l’adoption de systèmes d’IA au sein des armées ne devrait pas être considéré comme allant de soi. L’intégration technologique suppose des mécanismes d’adaptation relativement complexes, qui affectent la manière dont l’IA transformerait effectivement l’art de la guerre. About the author Renaud Bellais is an associate researcher in economics at ENSTA Bretagne and at CESICE, Université Grenoble Alpes. He graduated from Institut d'Études Politiques de Lille (1994), PhD in economics from Université du Littoral (1998) and accredited research director from Université Grenoble-Alpes (2004). After being lecturer at Université du Littoral, he worked at the French defense procurement agency (DGA) before joining EADS, now Airbus, and MBDA since 2017. He has teaching commitments in universities and military academies. The views expressed in this article reflect solely the author's opinion. 3 NOTE DE RECHERCHE SEPTEMBER 2019 Table of Contents Artificial intelligence and defense: What is at stake? ................................................................................. 5 Artificial Intelligence as a general-purpose technology ............................................................................. 6 A multi-level “game changer” for defense? ............................................................................................... 7 Strategic dimensions ................................................................................................................................. 8 Operational features of AI systems ......................................................................................................... 10 AI and effective autonomous systems .................................................................................................... 13 Trusting AI-based systems, a technical challenge .................................................................................. 16 International regulations, a hindrance to AI deployment? ....................................................................... 20 Defense innovations: the human factor .................................................................................................. 22 Conclusion .............................................................................................................................................. 28 4 RENAUD BELLAIS Coll. Analyse techno-capacitaire Artificial Intelligence and In February 2019, for instance, the White House and Pentagon unveiled their respective AI Defense: What is at Stake? strategies. President Trump signed an executive order enacting the “American AI Initiative” that identifies AI as a priority for government research his Note aims at identifying how artificial and development. The next day, the U.S. intelligence (AI) can affect the creation of Department of Defense released its own AI T defense systems and military operations as a strategy, centred on the newly created Joint means to communicate between defense Artificial Intelligence Center (JAIC). The DoD is stakeholders. It does not look at core technical poised to spend nearly $1 billion on artificial issues, but only at conceptual and operational ones. intelligence in Fiscal Year 2020 while DARPA has It proposes a comprehensive review of related announced a multiyear $2 billion program to deliver topics, stakes and issues, and relies solely on open game-changing AI solutions. sources. Simultaneously, AI has become a hotter and There are many fears and prejudices related to hotter topic for major arms-producing countries in the use of AI in defense systems. For instance, a Europe. Recent official statements on defense coalition of NGOs launched the “Campaign to Stop innovation, e.g. French MoD’s strategic innovation Killer Robots” in 2012. However, it seems very policy paper3, have increasingly brought the 1 2 improbable that systems like Skynet or Ava could criticality of IA for defense matters to the forefront. soon be deployed. Even though the potential Therefore, it is not surprising that AI was a major implementation of advanced AI (sometimes defined topic on the agenda of the informal meeting of EU as Artificial General Intelligence) into defense defense ministers in August 2019. With the systems appears as a very distant possibility for objective to forge new ties between the defense technical, architectural and military reasons sector and the private tech industry in this field, explored here, it is indisputable that AI has achieved these ministers met with members of the Global major progress over the past decade. Therefore, Tech Panel4 consisting of tech industry leaders, exploring its military potential constitutes a investors and civil society representatives. legitimate objective. As the interest of the defense community is After decades of ups and downs, the so-called (once again) very strong for AI applications all over “AI winters”, AI has clearly made progress in recent the world, this Note aims at understanding the years because three major obstacles have been nature of AI, its effectiveness and growth potential overcome since the early 2010s: high-performing as well as its possible impacts on the way of war. computational power, massive databases, and improved algorithms associated with machine learning (Cardon et al. 2018). Recent successes of AI performances has nurtured high expectations, in particular in the realm of defense capabilities where techno-centrism remains, more than ever, a core paradigm. 1 In the series of “Terminator” movies, Skynet is an 3 Imaginer au-delà, Document d’orientation de artificial neural network-based conscious group mind l’innovation de défense [Looking beyond, Defence and artificial general intelligence. innovation policy paper]. Paris, French Ministry of Armed Forces, July 2019. 2 A female humanoid robot with artificial intelligence featuring in the 2014 British psychological science-fiction 4 https://eeas.europa.eu/topics/global-tech-panel_en movie “Ex Machina”, which is centered on the ability to pass Turing test (but who is testing who?). 5 NOTE DE RECHERCHE SEPTEMBER 2019 Artificial Intelligence as a general- spark a process of cognitization analogous to the purpose technology changes wrought by industrialization.” In the field of defense, Wunische (2018) Since artificial intelligence emerged in the proposed a similar assessment of its 1950s, innovators and researchers have filed comprehensive integration