Consumer Law and Artificial Intelligence Challenges to the EU Consumer Law and Policy Stemming from the Business’ Use of Artificial Intelligence
Total Page:16
File Type:pdf, Size:1020Kb
LAW 2018/11 Department of Law Consumer law and artificial intelligence Challenges to the EU consumer law and policy stemming from the business’ use of artificial intelligence Final report of the ARTSY project Agnieszka Jabłonowska Maciej Kuziemski Anna Maria Nowak Hans-W. Micklitz Przemysław Pałka Giovanni Sartor European University Institute Department of Law CONSUMER LAW AND ARTIFICIAL INTELLIGENCE CHALLENGES TO THE EU CONSUMER LAW AND POLICY STEMMING FROM THE BUSINESS’ USE OF ARTIFICIAL INTELLIGENCE Final report of the ARTSY project Agnieszka Jabłonowska Maciej Kuziemski Anna Maria Nowak Hans-W. Micklitz Przemysław Pałka Giovanni Sartor EUI Working Paper LAW 2018/11 This text may be downloaded for personal research purposes only. Any additional reproduction for other purposes, whether in hard copy or electronically, requires the consent of the authors. If cited or quoted, reference should be made to the full names of the authors, the title, the working paper or other series, the year, and the publisher. ISSN 1725-6739 © Agnieszka Jabłonowska, Maciej Kuziemski, Anna Maria Nowak, Hans-W. Micklitz, Przemysław Pałka, Giovanni Sartor 2018 Printed in Italy European University Institute Badia Fiesolana I-50014 San Domenico di Fiesole (FI) Italy www.eui.eu cadmus.eui.eu Abstract Potential regulation of use of artificial intelligence by business should minimize the risks for consumers and the society without impeding the possible benefits. To do so, we argue, the legal reaction should be grounded in an empirical analysis and proceed case-by-case, bottom-up, as a series of responses to concrete research questions. The ambition of this report has been to commence and facilitate that process. We extensively document and evaluate the market practice of the corporate use of AI, map the scholarly debates about (consumer) law and artificial intelligence, and present a list of twenty five research questions which, in our opinion, require attention of regulators and academia. The report is divided into four sections. The first explains our understanding of the concepts of “artificial intelligence” (a set of socio-technological practices enabled by machine learning and big data) and “consumer law” (various legal instruments concretizing the principles of the weaker party protection, non-discrimination, regulated autonomy and consumer privacy). The second section documents the ways in which the business uses artificial intelligence in seven sectors of the economy: finance and insurance, information services, energy and “smart solutions”, retail, autonomous vehicles, healthcare and legal services. For each analyzed sector we study the gains for the businesses stemming from the deployment of AI, the potential gains, but also challenges for consumers, as well as third party effects. In the third section, we repeat the analysis through the lens of four general “uses” of AI by businesses in various sectors: knowledge generation, automated decision making, advertising and other commercial practices and personal digital assistants. Finally, in the fourth section, we present the questions which we believe should be addressed in the next stage of the research. We cluster them into: normative questions about regulatory goals, technological and governance questions about regulatory means, and theoretical questions about concepts and preconceptions. Keywords Consumer law and policy, challenges, artificial intelligence, business use Author contact details Agnieszka Jabłonowska Faculty of Law and Administration University of Lodz [email protected] Maciej Kuziemski School of Transnational Governance European University Institute Florence, Italy [email protected] Anna Maria Nowak Law Department European University Institute Florence, Italy [email protected] Hans-W. Micklitz Law Department European University Institute Florence, Italy [email protected] Przemysław Pałka Law Department European University Institute Florence, Italy Yale Law School New Haven, CT, United States [email protected] Giovanni Sartor Law Department European University Institute Florence, Italy [email protected] Table of contents INTRODUCTION: WHAT ARE WE DOING HERE AND WHY? ............................................................. 1 1. SETTING THE SCENE ............................................................................................................. 4 1.1. Artificial intelligence in context .............................................................................................. 4 1.2. The bird’s eye view: lawyers reflecting upon artificial intelligence ....................................... 7 1.3. EU consumer law: What’s that all about? ............................................................................... 8 1.3.1. Protection of the weaker party ......................................................................................... 9 1.3.2. Regulated autonomy ...................................................................................................... 12 1.3.3. Non-discrimination ........................................................................................................ 14 1.3.4. Consumer privacy .......................................................................................................... 18 2. SECTOR ANALYSIS: WHAT DOES BUSINESS USE AI FOR, WHAT ARE THE GAINS AND THE RISKS?......................................................................................................................... 19 2.1. Finance and insurance ........................................................................................................... 20 2.2. Information services .............................................................................................................. 22 2.3. Energy and “smart solutions” ................................................................................................ 25 2.4. Retail ..................................................................................................................................... 27 2.5. Autonomous vehicles ............................................................................................................ 29 2.6. Healthcare .............................................................................................................................. 30 2.7. Legal ...................................................................................................................................... 33 3. ANALYSIS BY USE: DATA GATHERING AND KNOWLEDGE GENERATION, AUTOMATIC DECISIONS, ADVERTISING, PERSONAL ASSISTANTS ................................................................... 35 3.1. Data gathering and knowledge generation ............................................................................ 35 3.2. Automated decision making .................................................................................................. 38 3.3. Advertising and other commercial practices ......................................................................... 48 3.4. Personal Assistants (“Algorithmic Consumers”, “Cyber-Butlers”) ...................................... 51 4. TIME FOR PHASE TWO: FURTHER QUESTIONS IN PLACE OF CONCLUSIONS .......................... 57 4.1. Normative questions (goals) .................................................................................................. 57 4.2. Technological (means) .......................................................................................................... 64 4.3. Governance (means) .............................................................................................................. 65 4.4. Theoretical/conceptual .......................................................................................................... 68 LIST OF LEGAL ACTS .................................................................................................................. 71 BIBLIOGRAPHY .......................................................................................................................... 73 Introduction: What are we doing here and why?1 The overall objectives of the ARTSY project have been to 1) map and document the use of artificial intelligence and other algorithmic techniques by business, 2) analyze and evaluate this practice through the prism of rules and principles of EU consumer law, including personal data protection law, 3) reflect on the challenges to consumer law and policy stemming from this practice; 4) prepare the next stage of the research, in which normative scientists, engineers and governance experts will collaborate. This will be a longer project, which will inquire into possible regulatory goals and means of achieving these goals, and put forward concrete policy recommendations. All this should be executed with a scholarly mindset, in dialogue with other researchers studying related subjects, hence 5) our ambition to also extensively map the state of the art in (consumer) law and artificial intelligence scholarship. The structure of this project rests on four methodological premises. First, that the normative (legal) research in the field of emerging technologies must be based on empirical studies. This way, we analyze the actual practices (what is really going on), instead of merely an idea of what artificial intelligence is and what it is used for. That is why we begin with extensive mapping and documenting. Second, such research must be grounded in concrete normative theory, in order to provide a clear threshold