Responsibility and AI: a Study of the Implications of Advanced Digital Technologies
Total Page:16
File Type:pdf, Size:1020Kb
Responsibility and AI Council of Europe study Prepared by the Expert Committee on human rights dimensions of automated DGI(2019)05 data processing and different forms of Rapporteur: Karen Yeung artificial intelligence (MSI-AUT) DGI(2019)05 A study of the implications of advanced digital technologies (including AI systems) for the concept of responsibility within a human rights framework Prepared by the Expert Committee on human rights dimensions of automated data processing and different forms of artificial intelligence (MSI-AUT) Rapporteur: Karen Yeung Council of Europe Study French edition: Responsabilité et IA The opinions expressed in this work are the responsibility of the authors and do not necessarily reflect the official policy of the Council of Europe. All requests concerning the reproduction or translation of all or part of this document should be addressed to the Directorate of Communication (F-67075 Strasbourg Cedex or [email protected]). All other correspondence concerning this document should be addressed to the Directorate General Human Rights and Rule of Law. Cover design: Documents and Publications Production Department (SPDP), Council of Europe Photos: Shutterstock This publication has not been copy-edited by the SPDP Editorial Unit to correct typographical and grammatical errors. © Council of Europe, September 2019 Printed at the Council of Europe 2 DGI(2019)05 TABLE OF CONTENTS Introduction .................................................................................................................................. 5 Executive Summary ...................................................................................................................... 6 Chapter 1. Introduction ............................................................................................................. 16 1.1 Scope of this study ....................................................................................................................... 16 1.2 Structure of this study ................................................................................................................. 17 1.3 Understanding the implications of AI for concepts of responsibility ......................................... 18 1.4 Implications for the concept of responsibility from a human rights perspective ...................... 25 Chapter 2. Threats, risks, harms and wrongs associated with advanced digital technologies ........................................................................................... 28 2.1 The rise of algorithmic decision-making (ADM) systems ........................................................... 28 2.1.1 How do ADM systems systematically threaten particular rights? .................................... 29 2.1.2 Societal risks associated with data-driven profiling ......................................................... 33 2.2 Collective societal threats and risks generated by other AI technologies ................................. 38 2.2.1 Malicious attacks, unethical system design or unintended system failure ..................... 38 2.2.2 Loss of authentic, real and meaningful human contact ................................................... 38 2.2.3 The chilling effect of data repurposing ............................................................................. 39 2.2.4 Digital power without responsibility ................................................................................ 39 2.2.5 The hidden privatisation of decisions about public values ............................................... 40 2.2.6 Exploitation of human labour to train algorithms ............................................................ 41 2.3 Power asymmetry and threats to the socio-technical foundations of moral and democratic community ................................................................................................ 41 2.4 Summary ...................................................................................................................................... 43 Chapter 3. Who bears responsibility for the threats, risks, harms and wrongs posed by advanced digital technologies? .............................................................. 44 3.1 What is responsibility and why does it matter? ......................................................................... 45 3.2 Dimensions of responsibility ....................................................................................................... 48 3.3 How do advanced digital technologies (including AI) implicate existing conceptions of responsibility? .................................................................................................................................... 49 3.3.1 Prospective responsibility: voluntary ethics codes and the ‘Responsible Robotics/AI’ project ................................................................................ 51 3.3.2 Machine autonomy and the alleged ‘control’ problem .................................................... 53 3.4 Models for allocating responsibility ............................................................................................ 55 3.4.1 Intention/culpability-based models .............................................................................. 57 3.4.2 Risk/Negligence-based models ..................................................................................... 58 3.4.3 Strict responsibility........................................................................................................ 60 3 Council of Europe Study 3.4.4 Mandatory Insurance .................................................................................................... 61 3.5 Responsibility challenges posed by complex and dynamic socio-technical systems ................ 62 3.5.1 The problem of ‘many hands’ ....................................................................................... 62 3.5.2 Human-Computer Interaction ....................................................................................... 64 3.5.3 Unpredictable, dynamic interactions between complex socio-technical systems ....... 66 3.6 State responsibility for ensuring effective protection of human rights ..................................... 67 3.7 Non-judicial mechanisms for enforcing responsibility for advanced digital technologies ........ 68 3.7.1 Technical protection mechanisms ................................................................................ 69 3.7.2 Regulatory governance instruments and techniques ................................................... 70 3.7.3 Standard setting, monitoring and enforcement ........................................................... 72 3.8 Reinvigorating human rights discourse in a networked digital age ........................................... 72 3.9 Summary ...................................................................................................................................... 75 Chapter 4. Conclusion ................................................................................................................ 77 Appendix A .................................................................................................................................. 80 References .................................................................................................................................. 83 4 DGI(2019)05 Introduction In the terms of reference for the Steering Committee on Media and Information Society (CDMSI) for the biennium 2018 – 2019, the Committee of Ministers of the Council of Europe asked the CDMSI to “study the development and use of new digital technologies and services, including different forms of artificial intelligence, as they may impact peoples’ enjoyment of human rights and fundamental freedoms in the digital age, with a view to giving guidance for future standard-setting in this field” and approved the committee of experts on human rights dimensions of automated data processing and different forms of artificial intelligence (MSI- AUT) as a subordinate structure to facilitate the work of the CDMSI. In its first meeting on 6-7 March 2018, the expert committee decided to focus the study on the the implications of AI decision-making for the concept of responsibility within a human rights framework. Prof. Karen Yeung was appointed as rapporteur for the preparation of the study. Composition of the Committee of Experts MSI-AUT Abraham BERNSTEIN, Professor of Informatics, University of Zürich Jorge CANCIO, International Relations Specialist, Federal Office of Communications, Switzerland Luciano FLORIDI, Professor of Philosophy and Ethics of Information, Oxford University Seda GÜRSES, Assistant Professor, Technical University Delft Gabrielle GUILLEMIN, Senior Legal Officer, ARTICLE 19 Natali HELBERGER, Professor of Information Law, University of Amsterdam Luukas ILVES (Chair), Deputy Director and Senior Fellow, Lisbon Council Tanja KERŠEVAN SMOKVINA, State Secretary, Ministry of Culture, Slovenia Joe MCNAMEE, Independent Consultant Evgenios NASTOS, Head of Information Unit, Ministry of Digital Policy, Telecoms & Media, Greece Pierluigi PERRI, Professor of Computer Law, University of Milan Wolfgang SCHULZ (Vice-Chair), Professor of Law, University of Hamburg Karen YEUNG, Interdisciplinary Professorial Fellow in Law, Ethics and Informatics, University of Birmingham 5 Council of Europe Study Executive Summary Advanced digital technologies and services, including task-specific artificial intelligence (‘AI’) bring