Artificial Intelligence
Total Page:16
File Type:pdf, Size:1020Kb
ARTIFICIAL INTELLIGENCE Automated Decision-making in Catalonia © Catalan Data Protection Authority Barcelona, January 2020 LD: B 2966-2020 Design and layout: Official Journal and Publications Agency Foreword ................................................................................................................................................................................................................... 5 1. Data protection and ethics in automated decision-making ............................................................. 11 2. Ada In ACTION: research fieldwork in Catalonia ........................................................................................... 19 2.1. How artificial intelligence is changing the world ..................................................................................................20 2.2. Automated decision-making algorithm risks ......................................................................................................... 24 2.3. Automated decision-making or support for the decision? .......................................................................29 2.4. Where are ADAs used in Catalonia? Over 50 examples to help you understand… .........32 2.4.1. Healthcare ..............................................................................................................................................................................33 2.4.2. Justice system ..................................................................................................................................................................40 2.4.3. Education ...............................................................................................................................................................................44 2.4.4. Banking ....................................................................................................................................................................................49 2.4.5. Business .................................................................................................................................................................................54 2.4.6. Social .........................................................................................................................................................................................59 2.4.7. Employment .........................................................................................................................................................................64 2.4.8. Cybersecurity ..................................................................................................................................................................... 67 2.4.9. Media and communication .................................................................................................................................... 70 2.4.10. Computer vision ............................................................................................................................................................ 73 2.5. The ethics of artificial intelligence ..................................................................................................................................... 78 2.6. Unresolved dilemmas .............................................................................................................................................................. 107 2.7. Recommended reading for further insights .........................................................................................................109 2.8. Acknowledgements ....................................................................................................................................................................115 3.Data protection and AI.................................................................................................................................................................. 119 The illegitimate collection and processing of data .....................................................................................................119 The right not to be subject to automated decision-making ............................................................................. 120 The principle of transparency ..................................................................................................................................................... 120 The principle of fairness .................................................................................................................................................................... 122 The principle of purpose limitation .......................................................................................................................................... 123 The principle of minimisation ........................................................................................................................................................ 124 Analysis of ADA uses ........................................................................................................................................................................... 125 Assessment of the risk of violence.............................................................................................................................. 125 Scoring systems ........................................................................................................................................................................... 126 Credit card fraud detection ................................................................................................................................................ 126 Disease detection........................................................................................................................................................................ 127 4. Final recommendations .............................................................................................................................................................129 Recommendations for individuals ........................................................................................................................................... 129 Recommendations for organisations using AI .............................................................................................................130 Recommendations for supervisory authorities ............................................................................................................132 5. Glossary ........................................................................................................................................................................................................135 4 Mentis sae et laborep eliqui officia essitat empore nonest perit labor restis FOREWORD Towards a smart society By M. Àngels Barbarà, Director of the Catalan Data Protection Authority The last ten years have seen technological evolution so wide-ranging, so deep and so fast that we humans are unable to foresee how these technologies will be brought into our lives.1 The growth of innovation and technological development is founded on personal and non-personal data. The convergence of several technologies over time and their inte- 5 gration to use data on a mass scale and generate value greatly increases the impact on people’s lives. Moreover, the exploitation of information does not drain its value in one use but rather it can be employed several times, thus building a value chain which affords enormous power to the people who control the technology and data. I am talking about technologies such as cloud computing, which allows access from various places and huge and cheap storage capacity; the Internet of Things (IoT), which gathers information from all environments in any format and at any time; big data ana- lytics, which makes it possible to generate new knowledge in health, transport, educa- tion, the environment, etc.; and finally artificial intelligence (AI), which refers to systems that display intelligent behaviour by analysing their environment and taking actions with some degree of autonomy to achieve specific goals.2 In its broadest sense, AI can be viewed as a support tool for decision-making in any field. This support for decision-making significantly affects people since data and technology profile and draw patterns which are used to take decisions and directly impact them. 1 “We have lost the ability to censure, to govern or even to negotiate with technology, which is seen as a logical consequence of our intellectual progress” (Joaquín David Rodríguez Álvarez. La civilización ausente. 2016, p. 11). 2 Communication from the Commission to the European Parliament, the European Council, the Council, the European Economic and Social Committee and the Committee of the Regions: Artificial Intelligence for Europe. COM/2018/237 final. Invisible changes We are stepping into a digital setting which leaves us exposed to new forms of vulner- ability through the processing of our personal data. These data are used by emerging technologies to generate value and to a great extent they change the way in which people and organisations understand the world around them. This change is taking place almost invisibly. It is coming into our personal, professional, social and political lives as if it were something quite normal and convenient, enhancing our lives with no impact