DOCSLIB.ORG

A Governance Framework for Algorithmic Accountability and Transparency

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
A Governance Framework for Algorithmic Accountability and Transparency A governance framework for algorithmic accountability and transparency STUDY Panel for the Future of Science and Technology EPRS | European Parliamentary Research Service Scientific Foresight Unit (STOA) PE 624.262 – April 2019 EN A governance framework for algorithmic accountability and transparency Algorithmic systems are increasingly being used as part of decision-making processes in both the public and private sectors, with potentially significant consequences for individuals, organisations and societies as a whole. Algorithmic systems in this context refer to the combination of algorithms, data and the interface process that together determine the outcomes that affect end users. Many types of decisions can be made faster and more efficiently using algorithms. A significant factor in the adoption of algorithmic systems for decision-making is their capacity to process large amounts of varied data sets (i.e. big data), which can be paired with machine learning methods in order to infer statistical models directly from the data. The same properties of scale, complexity and autonomous model inference however are linked to increasing concerns that many of these systems are opaque to the people affected by their use and lack clear explanations for the decisions they make. This lack of transparency risks undermining meaningful scrutiny and accountability, which is a significant concern when these systems are applied as part of decision-making processes that can have a considerable impact on people's human rights (e.g. critical safety decisions in autonomous vehicles; allocation of health and social service resources, etc.). This study develops policy options for the governance of algorithmic transparency and accountability, based on an analysis of the social, technical and regulatory challenges posed by algorithmic systems. Based on a review and analysis of existing proposals for governance of algorithmic systems, a set of four policy options are proposed, each of which addresses a different aspect of algorithmic transparency and accountability: 1. awareness raising: education, watchdogs and whistleblowers; 2. accountability in public-sector use of algorithmic decision-making; 3. regulatory oversight and legal liability; and 4. global coordination for algorithmic governance. EPRS | European Parliamentary Research Service STOA | Panel for the Future of Science and Technology AUTHORS This study has been written by the following authors at the request of the Panel for the Future of Science and Technology (STOA) and managed by the Scientific Foresight Unit, within the Directorate-General for Parliamentary Research Services (EPRS) of the Secretariat of the European Parliament. Ansgar Koene, main author, University of Nottingham Chris Clifton, Purdue University Yohko Hatada, EMLS RI Helena Webb, Menisha Patel, Caio Machado, Jack LaViolette, University of Oxford Rashida Richardson, Dillon Reisman, AI Now Institute ADMINISTRATOR RESPONSIBLE Mihalis Kritikos, Scientific Foresight Unit (STOA) To contact the publisher, please e-mail [email protected] LINGUISTIC VERSION Original: EN Manuscript completed in March 2019. DISCLAIMER AND COPYRIGHT This document is prepared for, and addressed to, the Members and staff of the European Parliament as background material to assist them in their parliamentary work. The content of the document is the sole responsibility of its author(s) and any opinions expressed herein should not be taken to represent an official position of the Parliament. Reproduction and translation for non-commercial purposes are authorised, provided the source is acknowledged and the European Parliament is given prior notice and sent a copy. Brussels © European Union, 2019. PE 624.262 ISBN: 978-92-846-4656-2 doi: 10.2861/59990 QA-03-19-162-EN-N http://www.europarl.europa.eu/stoa (STOA website) http://www.eprs.ep.parl.union.eu (intranet) http://www.europarl.europa.eu/thinktank (internet) http://epthinktank.eu (blog) A governance framework for algorithmic accountability and transparency Executive summary This report presents an analysis of the social, technical and regulatory challenges associated with algorithmic transparency and accountability, including a review of existing proposals for the governance of algorithmic systems and the current state of development of related standards and consideration of the global and human rights dimensions of algorithmic governance. Motivation Algorithmic systems are increasingly being used as part of decision-making processes with potentially significant consequences for individuals, organisations and societies as a whole. When used appropriately, with due care and analysis of its impacts on people's lives, algorithmic systems, including artificial intelligence (AI) and machine learning, have great potential to improve human rights and democratic society. In order to achieve this however it is vitally necessary to establish clear governance frameworks for algorithmic transparency and accountability to make sure that the risk and benefits are equitably distributed in a way that does not unduly burden or benefit particular sectors of society. There is growing concern that unless appropriate governance frameworks are put in place, the opacity of algorithmic systems could lead to situations where individuals are negatively impacted because 'the computer says NO', with no recourse to meaningful explanation, a correction mechanism, or a way to ascertain faults that could bring about compensatory processes. As with the governance of any other aspect of society, the extent of algorithmic accountability required should be considered within the context of the good, harm, and risks these systems present. Background definitions and drivers for algorithmic transparency and accountability The study presents two 'conceptual landscapes' that explore the conceptual roles and uses of transparency and accountability in the context of algorithmic systems. The primary role of transparency is identified as a tool to enable accountability. If it is not known what an organisation is doing, it cannot be held accountable and cannot be regulated. Transparency may relate to the data, algorithms, goals, outcomes, compliance, influence and/or usage of automated decision making systems (i.e. algorithmic systems), and will often require different levels of detail for the general public, regulatory staff, third-party forensic analysts and researchers. The degree of transparency of an algorithmic systems often depends on a combination of governance processes and technical properties of the system. An important difference between transparency and accountability is that accountability is primarily a legal and ethical obligation on an individual or organisation to account for its activities, accept responsibility for them, and to disclose the results in a transparent manner. The challenges for algorithmic accountability arise from: the complex interactions between sub-systems and data sources, which might not all be under the control of the same entity; the impossibility of testing against all possible conditions when there are no formal proofs for the system's performance; difficulties in translating algorithmically derived concepts into human understandable concepts, resulting in incorrect interpretations; information asymmetries arising from algorithmic inferences; accumulation of many small (individually non-significant) algorithmic decisions; difficult to detect injections of adversarial data. When considering the governance of both transparency and accountability it is important to keep in mind the larger motivating drivers that define what is meant to be achieved. While recognising that fairness is an immensely complex concept with different, sometimes competing, definitions it is nevertheless seen as a fundamental component underpinning responsible systems and it is suggested that algorithmic processes should seek to minimise their potential to be unfair and maximise their potential to be fair. Transparency and accountability provide two important ways in which this can be achieved. Fairness is discussed through the lens of social justice, highlighting the I STOA | Panel for the Future of Science and Technology potential for algorithmic systems to systematically disadvantage, or even discriminate against, different social groups and demographics. A series of real life case studies is used to illustrate how this lack of fairness can arise, before exploring the consequences that lack of fairness can have plus the complexities inherent to trying to achieve fairness in any given societal context. The study describes ways in which lack of fairness in the outcomes of algorithmic systems might be caused by developmental decision-making and design features embedded at different points in the lifecycle of an algorithmic decision making model. A connection is made between the problem of fairness and the tools of transparency and accountability, while highlighting the value of responsible research and innovation (RRI) approaches to pursuing fairness in algorithmic systems. Technical challenges and solutions Viewing transparency as 'explaining the steps of the algorithm' is unlikely to lead to an informative outcome. On the one hand, it could result in a description that only captures the general process used to make a decision. At the other extreme would be to provide the complete set of steps taken (e.g. the complete detailed algorithm, or the machine learned model.) While this may enable the outcome to be reconstructed (provided
Load more

Recommended publications
  • Dan Calacci E15-384A MIT Media Lab 75 Amherst St
    Support for SB.1876 & H.2701: a crucial step in ensuring fair future governance in the commonwealth. 1 October, 2019 Dan Calacci E15-384a MIT Media Lab 75 Amherst St. Cambridge, MA 02139 Dear Senator Pacheco, Representative Gregoire, and members of the Committee: I’m writing in strong support of S.1876 and H.2701, An Act Establishing A Commission On Transparency And Use of Artificial Intelligence In Government Decision-Making and An Act Establishing A Commission On Automated Decision- Making, Artificial Intelligence, Transparency, Fairness, And Individual Rights. To shape the future of how we use artificial intelligence and other technologies in governance, we need to understand how they are being used in governing systems right now. This legislation will provide a desperately needed avenue for policymakers, researchers, and the public to understand how algorithms and artificial intelligence are impacting government decision-making and constituents. As a doctoral researcher studying data governance and machine learning at MIT, I have experience working with AI tools and reasoning about fairness and ethics in how they are used. My research experience includes developing cutting-edge machine learning systems, designing smartphone apps that use AI to better understand and prevent chronic disease, measuring segregation in cities using machine learning and big data, and writing about the ethics of collecting large amounts of data about citizen’s behavior, such as their location history, for use in governance. I have had the opportunity to present work on these topics at international conferences on machine learning, social science, and data ethics. My unique research experience has given me expertise on the implications of using algorithms in governance, including its promises and pitfalls, how data is collected from the public to fuel these algorithms, and how these algorithms function.
    [Show full text]
  • A Guide to Data Governance Building a Roadmap for Trusted Data a Guide to Data Governance Contents
    A Guide to Data Governance Building a roadmap for trusted data A Guide to Data Governance Contents What is Data Governance? ...............................................................................................3 Why Do We Need It? ............................................................................................................................................................................. 4 The Need to Create Trusted Data ..................................................................................................................................................... 4 The Need to Protect Data .................................................................................................................................................................... 5 Requirements For Governing Data In A Modern Enterprise ........................................7 Common Business Vocabulary ........................................................................................................................................................... 7 Governing Data Across A Distributed Data Landscape............................................................................................................ 7 Data Governance Classification ......................................................................................................................................................... 8 Data Governance Roles and Responsibilities ..............................................................................................................................
    [Show full text]
  • Can E-Government Help?
    Improving Governance and Services: Can e-Government Help? RFI Smith, Monash University, Australia Julian Teicher, Monash University, Australia Abstract: E-government can help improve governance and service delivery by refocusing consideration of the purposes and tools of government. However, E-government initiatives pose challenging questions of management, especially about coordination in government and the design of services for citizens. Progress towards implementing e-government raises critical questions about preferred styles of governance and about how governments relate to citizens. At present, interactions between citizens, the institutions of government and information and communications technology raise more agendas than governments can handle. However, trying to find ways through these agendas is to confront questions of wide interest to citizens. At the very least, e-government helps improve governance and services by asking questions. ince the late 1990s the prospect of using Information and Communication Such questions lead directly to a familiar S Technologies (ICTs) to improve effectiveness, dilemma: fairness and accountability in government has attracted widespread enthusiasm. However early hopes that e- How to capture the benefits of coordinated initiatives would bypass intractable questions of action and shared approaches while government organization and transform citizen maintaining individual agency responsibility experience of the delivery of public services have given and accountability for operations and results. way to more modest claims. (OECD 2003: 15) At the same time, thinking about how to use ICTs most effectively in government has generated Many governments have gone to great lengths widening questions about what governments should try to craft pluralist or decentralised strategies for to do and how they should do it.
    [Show full text]
  • Definitive Guide to Data Governance Contents
    Definitive Guide to Data Governance Contents Introduction: Why trusted data is the key to digital transformation 03 Chapter 1: What is data governance and why do you need it? 05 Chapter 2: Choosing the best governance model for you 11 Chapter 3: Three steps to deliver data you can trust 18 Chapter 4: Dos & don’ts: the 12 labors of the data governance hero 41 Chapter 5: New roles of data governance 46 Chapter 6: Successful trusted data delivery stories 50 Chapter 7: Managing the transition from data integration to data integrity 60 Chapter 8: Moving toward the data intelligence company 66 2 Definitive Guide to Data Governance Introduction Why trusted data is the key to digital transformation We’ve entered the era of the information economy, tools in order to get results fast While these tactics may where data has become the most critical asset of every solve for speed in the short term, they are not scalable as organization Data-driven strategies are now a competitive the company grows, and create quality and compliance imperative to succeed in every industry To support risk due to the lack of oversight On the other hand, business objectives such as revenue growth, profitability, organizations that try to solve the data trust problem often and customer satisfaction, organizations are increasingly create a culture of “no” with the establishment of strict relying on data to make decisions Data-driven controls and an authoritative approach to governance decision-making is at the heart of your digital However, it’s resource-intensive, cumbersome,
    [Show full text]
  • Understanding Corporate Governance Laws & Regulations
    I N S I D E T H E M I N D S Understanding Corporate Governance Laws & Regulations Leading Lawyers on Corporate Compliance, Advising Boards of Directors, and Sarbanes-Oxley Requirements BOOK IDEA SUBMISSIONS If you are a C-Level executive or senior lawyer interested in submitting a book idea or manuscript to the Aspatore editorial board, please email [email protected]. Aspatore is especially looking for highly specific book ideas that would have a direct financial impact on behalf of a reader. Completed books can range from 20 to 2,000 pages – the topic and “need to read” aspect of the material are most important, not the length. Include your book idea, biography, and any additional pertinent information. ARTICLE SUBMISSIONS If you are a C-Level executive or senior lawyer interested in submitting an article idea (or content from an article previously written but never formally published), please email [email protected]. Aspatore is especially looking for highly specific articles that would be part of our Executive Reports series. Completed reports can range from 2 to 20 pages and are distributed as coil-bound reports to bookstores nationwide. Include your article idea, biography, and any additional information. GIVE A VIDEO LEADERSHIP SEMINAR If you are interested in giving a Video Leadership SeminarTM, please email the ReedLogic Speaker Board at [email protected] (a partner of Aspatore Books). If selected, ReedLogic would work with you to identify the topic, create interview questions and coordinate the filming of the interview. ReedLogic studios then professionally produce the video and turn it into a Video Leadership SeminarTM on your area of expertise.
    [Show full text]
  • Data Governance Part II: Maturity Models – a Path to Progress
    NASCIO: Representing Chief Information Officers of the States Data Governance Part II: Maturity Models – A Path to Progress Introduction When government can respond effectively and expeditiously to its constituents, it In the previous report on Data Governance1 gains credibility with citizens. The an overview of data governance was opposite is also true. When government presented describing the foundational can’t respond, or responds incorrectly, or issues that drive the necessity for state too slowly, based on inaccurate informa- government to pursue a deliberate effort tion, or a lack of data consistency across for managing its key information assets. agencies, government’s credibility suffers. Data governance or governance of data, information and knowledge assets resides This research brief will present a number within the greater umbrella of enterprise of data governance maturity models2 architecture and must be an enterprise- which have been developed by widely wide program.There is a significant cost to recognized thought leaders. These models March 2009 state government when data and informa- provide a foundational reference for tion are not properly managed. In an understanding data governance and for NASCIO Staff Contact: emergency situation, conflicts in informa- understanding the journey that must be Eric Sweden anticipated and planned for achieving Enterprise Architect tion can jeopardize the lives of citizens, [email protected] first responders, law enforcement officers, effective governance of data, information fire fighters, and medical personnel. and knowledge assets. This report contin- ues to build on the concepts presented in Redundant sources for data can lead to Data Governance Part I. It presents a conflicting data which can lead to ineffec- portfolio of data governance maturity tive decision making and costly models.
    [Show full text]
  • What Is Good Governance?
    United Nations Economic and Social Commission for Asia and the Pacific What is Good Governance? Introduction institutes, religious leaders, finance institutions political parties, the military etc. The situation in urban areas is much more Recently the terms "governance" and "good complex. Figure 1 provides the governance" are being increasingly used in interconnections between actors involved in development literature. Bad governance is urban governance. At the national level, in being increasingly regarded as one of the addition to the above actors, media, root causes of all evil within our societies. lobbyists, international donors, multi-national Major donors and international financial corporations, etc. may play a role in decision- institutions are increasingly basing their aid making or in influencing the decision-making and loans on the condition that reforms that process. ensure "good governance" are undertaken. All actors other than government and the This article tries to explain, as simply as military are grouped together as part of the possible, what "governance" and "good "civil society." In some countries in addition to governance" means. the civil society, organized crime syndicates also influence decision-making, particularly in Governance urban areas and at the national level. Similarly formal government structures are The concept of "governance" is not new. It is one means by which decisions are arrived at as old as human civilization. Simply put and implemented. At the national level, "governance" means: the process of informal decision-making structures, such as decision-making and the process by "kitchen cabinets" or informal advisors may which decisions are implemented (or not exist. In urban areas, organized crime implemented).
    [Show full text]
  • The Evolving Role of the Chief Data Officer in Financial Services
    The evolving role of the chief data officer in financial services: From marshal and steward to business strategist The evolving role of the chief data officer in financial services | From marshal and steward to business strategist The evolving role of the chief data officer in financial services: From marshal and steward to business strategist Over the past few years, financial institutions core businesses, products, customers, and (FIs) have increasingly come to recognize supporting data infrastructure’s capabilities that their data assets represent highly and needs. strategic sources of insight and leverage for a wide array of business functions, More recently, the CDO’s job description–for including risk management, regulatory the most progressive organizations–has compliance, sales and marketing, product evolved from its initial focus on data asset development, and operational performance, gathering, governance, and stewardship among others. To realize this embedded to proactive business enablement, with value, however, organizations need to many institutions even marrying the CDO proactively and effectively manage their and chief analytics officer (CAO) roles into a information assets at the enterprise level. In single senior-level position. This is especially response, they have been appointing chief true for organizations that aggressively data officers (CDOs) to provide required seek to leverage data science and advanced strategic guidance and execution support, analytical modelling to generate new insights and also to assure access to and the into the markets and customers they serve, quality of critical data. In addition, CDOs the products they build and price, the risks will undoubtedly play a strategic role in they assume or pass on, and the means by helping FIs adapt and transform their data which they operate the business to benefit ecosystems in response to rapid technology stakeholders.
    [Show full text]
  • Self-Organization and Self-Governance Is Not One That Appears If We Look at the Level of Individual Components of a System
    Philosophy of the Social Sciences XX(X) 1 –25 Self-Organization and © The Author(s) 2010 Reprints and permission: http://www. Self-Governance sagepub.com/journalsPermissions.nav DOI: 10.1177/0048393110363435 http://pos.sagepub.com J. T. Ismael1,2 Abstract The intuitive difference between a system that choreographs the motion of its parts in the service of goals of its own formulation and a system composed of a collection of parts doing their own thing without coordination has been shaken by now familiar examples of self-organization. There is a broad and growing presumption in parts of philosophy and across the sciences that the appearance of centralized information-processing and control in the service of system- wide goals is mere appearance, i.e., an explanatory heuristic we have evolved to predict behavior, but one that will eventually get swept away in the advancing tide of self-organization. I argue that there is a distinction of central importance here, and that no adequate science of complex systems can dispense with it. Keywords self-organization, self-governance, selves, dynamics, intentional systems theory, Dennett The Advancing Tide of Self-organization When we see behavior that looks coordinated and directed at an end, we tend to assume that there is a guide, that is, a central intelligence collecting infor- mation, formulating goals, and choreographing the movements of its parts. So, for example, armies are led by central command, cars are guided by drivers, countries are run by governments. But the intuitive difference between a system that choreographs the motion of its parts in the service of goals of its own Received 4 March 2009 1University of Sydney, Centre for Time, Sydney, Australia 2University of Arizona, Tucson, USA Corresponding Author: J.
    [Show full text]
  • Civil Good: a Platform for Sustainable and Inclusive Online Discussion
    Civil Good: A Platform For Sustainable and Inclusive Online Discussion An Interactive Qualifying Project submitted to the faculty of Worcester Polytechnic Institute In partial fulfillment of the requirements for the degree of Bachelor of Science by: Steven Malis (Computer Science), Tushar Narayan (Computer Science), Ian Naval (Computer Science), Thomas O'Connor (Biochemistry), Michael Perrone (Physics and Mathematics), John Pham (Computer Science), David Pounds (Computer Science and Robotics Engineering), December 19, 2013 Submitted to: Professor Craig Shue, WPI Advisor Alan Mandel, Creator of the Civil Good concept Contents 1 Executive Summary1 1.1 Overview of Recommendations......................2 2 Authorship5 3 Introduction 10 3.1 Existing Work - Similar Websites.................... 11 4 Psychology 17 4.1 Online Disinhibition........................... 17 4.2 Format of Discussions.......................... 22 4.3 Reducing Bias with Self-Affirmation................... 28 4.4 Other Psychological Influences...................... 34 5 Legal Issues 38 5.1 Personally Identifiable Information................... 38 5.2 Intellectual Property........................... 42 5.3 Defamation................................ 45 5.4 Information Requests........................... 46 5.5 Use by Minors............................... 49 5.6 General Litigation Avoidance and Defense............... 51 6 Societal Impact 52 6.1 Political Polarization........................... 52 6.2 Minority Opinion Representation.................... 55 6.3 History and Political
    [Show full text]
  • Global Technology Governance a Multistakeholder Approach
    White Paper Global Technology Governance A Multistakeholder Approach In collaboration with Thunderbird School of Global Management and Arizona State University October 2019 World Economic Forum 91-93 route de la Capite CH-1223 Cologny/Geneva Switzerland Tel.: +41 (0)22 869 1212 Fax: +41 (0)22 786 2744 Email: [email protected] www.weforum.org This white paper has been published by the World Economic Forum as a contribution to a project, © 2019 World Economic Forum. All rights insight area or interaction. The findings, interpretations and conclusions expressed herein are a re- reserved. No part of this publication may be sult of a collaborative process facilitated and endorsed by the World Economic Forum, but whose reproduced or transmitted in any form or by any results do not necessarily represent the views of the World Economic Forum, nor the entirety of its means, including photocopying and recording, or Members, Partners or other stakeholders. by any information storage and retrieval system. Contents Executive summary 4 Introduction 5 Part 1: What is governance and why is it necessary? 6 1.1 Definitions 7 1.2 Technology governance frameworks 8 1.3 The dynamics of emerging technology governance 10 An overview of agile governance 11 Part 2: Sketching the technology governance landscape 12 2.1 An overview of global technology governance in 2019 12 How do research scientists experience the governance of emerging technologies? 14 2.2 Priority cross-cutting issues in technology governance 14 2.2.1 Delivering privacy and security while enabling
    [Show full text]
  • Governance Indicators Can Make Sense: Under-Five Mortality Rates Are an Example Faculty Research Working Paper Series
    Governance Indicators Can Make Sense: Under-five Mortality Rates are an Example Faculty Research Working Paper Series Matt Andrews Harvard Kennedy School Roger Hay Jerrett Myers Institute for Government April 2010 RWP10-015 The views expressed in the HKS Faculty Research Working Paper Series are those of the author(s) and do not necessarily reflect those of the John F. Kennedy School of Government or of Harvard University. Faculty Research Working Papers have not undergone formal review and approval. Such papers are included in this series to elicit feedback and to encourage debate on important public policy challenges. Copyright belongs to the author(s). Papers may be downloaded for personal use only. www.hks.harvard.edu GOVERNANCE INDICATORS CAN MAKE SENSE1 Under-five mortality rates are an example Matt Andrews,2 Roger Hay3 and Jerrett Myers4 ABSTRACT Governance indicators have come under fire in recent years, especially the World Governance Indicators (WGIs). Critics present these indicators as a-theoretical and biased. Critics of the critics counter that no better alternatives exist. We suggest otherwise, arguing that more appropriate ‘governance’ indicators will (i) have theoretical grounding, (ii) focus on specific fields of engagement, (iii) emphasize outcomes, and (iv) control for key contextual differences in comparing countries. Such measures can help indicate where countries seem to have governance problems, allowing second stage analyses of what these problems are. We present under national five mortality rates adjusted for country income groups as an example of such measure, presenting data for contextually controlled outcomes in this specific field to show where governance seems better and worse.
    [Show full text]