DOCSLIB.ORG

To Surveil and Predict: a Human Rights Analysis of Algorithmic Policing in Canada,” by Kate Robertson, Cynthia Khoo, and Yolanda Song

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text

Load more

This publication is the result of an investigation by the University of Toronto’s International Human Rights Program (IHRP) at the Faculty of Law and the Citizen Lab at the Munk School of Global Affairs & Public Policy. Authors: Kate Robertson, Cynthia Khoo, and Yolanda Song Research Principals: Yolanda Song and Cynthia Khoo Design and Illustrations: Ryookyung Kim Copyright © 2020 Citizen Lab (Munk School of Global Affairs & Public Policy, University of Toronto) and the International Human Rights Program (Faculty of Law, University of Toronto), “To Surveil and Predict: A Human Rights Analysis of Algorithmic Policing in Canada,” by Kate Robertson, Cynthia Khoo, and Yolanda Song. Document Version: 1.0 Licensed under the Creative Commons BY-SA 4.0 (Attribution-ShareAlike Licence) The Creative Commons Attribution-ShareAlike 4.0 license under which this report is licensed lets you freely copy, distribute, remix, transform, and build on it, as long as you: • give appropriate credit; • indicate whether you made changes; and • use and link to the same CC BY-SA 4.0 licence. However, any rights in excerpts reproduced in this report remain with their respective authors; and any rights in brand and product names and associated logos remain with their respective owners. Uses of these that are protected by copyright or trademark rights require the rightsholder’s prior written agreement. Electronic version first published by the International Human Rights Program and the Citizen Lab in September, 2020. This work can be accessed through https://ihrp.law.utoronto.ca/ and https://citizenlab.ca. Acknowledgements The International Human Rights Program and the Citizen Lab wish to express their sincere gratitude for the support from many people who made this work possible. This report was researched and written by Kate Robertson, criminal defence lawyer and Citizen Lab research fellow, Cynthia Khoo, Citizen Lab research fellow and technology and human rights lawyer, and Yolanda Song, lawyer at Stevenson Whelton LLP and pro bono research associate at the IHRP. Primary data collection governed by the research protocol for this report was carried out by Yolanda Song and Cynthia Khoo. This report was undertaken under the supervision of Professor Ronald J. Deibert. The report was reviewed by Ronald J. Deibert, Professor of Political Science and Director of the Citizen Lab; Christopher Parsons, Senior Research Associate at the Citizen Lab; Petra Molnar, acting Director of the IHRP; and Samer Muscati, former Director of the IHRP. The authors of this report are grateful to have received further detailed input, guidance, and support from Christopher Parsons. The authors owe a significant debt of gratitude for, and deeply appreciate, the invaluable insights and advice from the following external reviewers (presented alphabetically by last name): Vincent Chiao, Associate Professor at the University of Toronto Faculty of Law; Lex Gill, Associate at Trudel Johnston & Lespérance and Research Fellow at the Citizen Lab; Tamir Israel, Staff Lawyer at Samuelson-Glushko Canadian Internet Policy and Public Interest Clinic (CIPPIC) at the University of Ottawa, Faculty of Law; Emily Lam, Partner, Kastner Law; Brenda McPhail, Director of the Privacy, Technology & Surveillance Project at the Canadian Civil Liberties Association; Noa Mendelsohn Aviv, Director of the Equality Program at the Canadian Civil Liberties Association; Jill Presser, Principal, Presser Barristers, and Osgoode Hall Law School, Adjunct Professor; Rashida Richardson, Director of Policy Research, AI Now Institute at New York University; Teresa Scassa, Professor and Canada Research Chair in Information Law and Policy at the University of Ottawa, Faculty of Law; and Valerie Steeves, Professor at the University of Ottawa Faculty of Social Sciences, Department of Criminology. This report was copy-edited by Joyce Parsons, Principal of Stone Pillars Editing and Consulting, and fact-checked by Mashoka Maimona, a law student at the University of Toronto. The cover art and design for the report was completed by Ryookyung Kim. The authors would particularly like to thank University of Toronto law students Julie Lowenstein and Solomon McKenzie for their valuable research assistance throughout this project, and they are grateful to law student India Annamanthadoo for her contributions as well. The authors further extend gratitude to the numerous subject matter specialists and expert stakeholders who provided their insight, feedback, and time in the development of this report through informal consultations with the authors and formal research interviews. The authors are also greatly appreciative of the participants of the March 2019 symposium, “Automated Decision-Making in the Criminal Justice System,” co-hosted by the Law Commission of Ontario, the Citizen Lab, the IHRP, and the Criminal Lawyers’ Association, for their engagement and views on the issues raised in this report. The Citizen Lab would like to thank the following funders for supporting this research: the John D. and Catherine T. MacArthur Foundation, the Sigrid Rausing Trust, the Ford Foundation, and the Oak Foundation. This research was also supported in part by a grant from the Open Society Foundation. The International Human Rights Program (IHRP) would like to thank the Law Foundation of Ontario for supporting this research. About the Authors Kate Roberson is a criminal defence lawyer at Markson Law in Toronto and a Citizen Lab Research Fellow. Her criminal defence practice includes both trial and appellate work, focusing on a range of criminal law cases, including white-collar crime, sexual offences, and computer-based investigations and crime. She previously acted as a provincial Crown prosecutor in Ontario and as a Law Clerk at the Supreme Court of Canada. She holds a J.D from the University of Toronto, Faculty of Law. Cynthia Khoo is a Research Fellow at the Citizen Lab and a technology and human rights lawyer. She holds an LL.M. (Concentration in Law and Technology) from the University of Ottawa and interned as a research student and junior counsel at the Samuelson-Glushko Canadian Internet Policy and Public Interest Clinic. Her work spans across key areas of digital rights law and policy, including privacy, surveillance, intermediary liability, freedom of expression, equality, and technology-facilitated abuse. Yolanda Song is a civil litigation lawyer at Stevenson Whelton LLP in Toronto. Her practice includes government litigation and general administrative and constitutional law. Yolanda worked as the William Graham Research Fellow at the International Human Rights Program at the University of Toronto, Faculty of Law, and she continues to work as a pro bono research associate for the IHRP. She holds a J.D. from the University of Toronto, Faculty of Law. About the International Human Rights Program The International Human Rights Program (IHRP) at the University of Toronto Faculty of Law addresses the most pressing human rights issues through two avenues: The Program shines a light on egregious human rights abuses through reports, publications, and public outreach activities, and offers students unparalleled opportunities to refine their legal research and advocacy skills through legal clinic projects and global fellowships. The IHRP’s fundamental priority is impact: The Program strives to equip students and recent graduates with the skills, the knowledge, and the professional network to become effective human rights advocates. The Program also seeks to address human rights violations in Canada and abroad by engaging in comprehensive research and advocacy that aims to reform law, policy, and practice. About the Citizen Lab The Citizen Lab is an interdisciplinary laboratory based at the Munk School of Global Affairs & Public Policy, University of Toronto, focusing on research, development, and high-level strategic policy and legal engagement at the intersection of information and communication technologies, human rights, and global security. The Citizen Lab uses a “mixed methods” approach to research, combining practices from political science, law, computer science, and area studies. Our research includes: investigating digital espionage against civil society; documenting Internet filtering and other technologies and practices that impact freedom of expression online; analyzing privacy, security, and information controls of popular applications; and examining transparency and accountability mechanisms relevant to the relationship between corporations and state agencies regarding personal data and other surveillance activities. Corrections and Questions Please send all questions and corrections to the authors directly at: [email protected] [email protected] [email protected] Suggested Citation Kate Robertson, Cynthia Khoo, and Yolanda Song, “To Surveil and Predict: A Human Rights Analysis of Algorithmic Policing in Canada” (September 2020), Citizen Lab and International Human Rights Program, University of Toronto. Table of Contents 1 Executive Summary 8 1. Overview and Background 8 1.1. Introduction 11 1.2. Methodology 11 1.2.1. Data Collection 14 1.2.2. Legal Analysis of Collected Data 15 2. Social and Historical Context 15 2.1. Criminal Justice and Systemic Discrimination in Canada 18 2.2. Bias and Inaccuracies in Police Data 26 In Focus #1: Community Perspectives on Algorithmic Policing 29 3. What Is Algorithmic Policing? 29 3.1. A Technical Primer 33 3.2. What Makes Algorithmic Policing Different? 36 4. Algorithmic Policing
Recommended publications
  • ADDRESSING SYSTEMIC RACISM in MEDIA Brian Daly, Dexter Brown, Julie Sobowale and Nadia Stewart the Canadian Association of Black Journalists

    ADDRESSING SYSTEMIC RACISM in MEDIA Brian Daly, Dexter Brown, Julie Sobowale and Nadia Stewart the Canadian Association of Black Journalists

    AMPLIFYING VOICES, PROTECTING LIVES: ADDRESSING SYSTEMIC RACISM IN MEDIA Brian Daly, Dexter Brown, Julie Sobowale and Nadia Stewart The Canadian Association of Black Journalists 0 AMPLIFYING VOICES, PROTECTING LIVES: ADDRESSING SYSTEMIC RACISM IN MEDIA Acknowledgement and disclaimer: The views and positions expressed in this report are solely those of the author and do not necessarily reflect the views of the Department of Foreign Affairs, Trade and Development or the Government of Canada. The report is in its original language. Executive summary The voices of journalists of colour—within mainstream media and as citizen journalists—are needed more than ever. A government’s willingness to protect journalists and their rights reflects how much it values democracy. Effective and enforced policies protecting the lives of racialized journalists, and resources to amplify their voices and stories, are needed the world over. Public trust and citizen journalism was called in, after a Black teenager was injured during an arrest caught on cellphone video.3 The individuals who recorded George Floyd’s ill- Five months later, in the same city, a cellphone fated encounter with Minneapolis police were video of a Black woman being roughed up by police officers at a Walmart led to charges being engaging in citizen journalism. This form of 4 reporting is generally defined as when “an dropped against the victim. South of the border, ordinary person actively engages in recording, eyewitness video in the shootings of Jacob generating, and disseminating newsworthy Blake, Oscar Grant and Eric Garner played events”.1 This allows citizens to “confront issues pivotal roles in the subsequent investigations.
  • Investing in Yourself: Entrepreneurial Journalism in the Digital Age

    Investing in Yourself: Entrepreneurial Journalism in the Digital Age

    INVESTING IN YOURSELF: ENTREPRENEURIAL JOURNALISM IN THE DIGITAL AGE by Maggie Reid Master of Information, University of Toronto, Toronto, Ontario, Canada, 2012 Bachelor of Arts, University of Toronto, Toronto, Ontario, Canada, 2009 A dissertation presented to Ryerson University and York University in partial fulfillment of the requirements for the degree of Doctor of Philosophy in the Joint Program in Communication and Culture Toronto, Ontario, Canada, 2018 ©Maggie Reid, 2018 AUTHOR’S DECLARATION FOR ELECTRONIC SUBMISSION OF A DISSERTATION I hereby declare that I am the sole author of this dissertation. This is a true copy of the dissertation, including any required final revisions, as accepted by my examiners. I authorize Ryerson University to lend this dissertation to other institutions or individuals for the purpose of scholarly research. I further authorize Ryerson University to reproduce this dissertation by photocopying or by other means, in total or in part, at the request of other institutions or individuals for the purpose of scholarly research. I understand that my dissertation may be made electronically available to the public. ii Investing in Yourself: Entrepreneurial Journalism in the Digital Age Maggie Reid, PhD, Communication and Culture Ryerson University, 2018 Abstract This dissertation is grounded in a Critical Political Economy of communication theoretical framework in conjunction with extensive, qualitative interviews with eighteen emerging journalists, three journalism educators from different types of journalism schools (academic, vocational, hybrid) and four editors from different types of news organizations (legacy, public broadcaster, digital first media) in order to navigate between institutional structures and the agency of individual actors. This work examines how the current structural configurations of the news media industry are impacting how emerging journalists negotiate the expectations that they develop personal brands online, including their perceived control and autonomy over their work.
  • Amazon's Antitrust Paradox

    Amazon's Antitrust Paradox

    LINA M. KHAN Amazon’s Antitrust Paradox abstract. Amazon is the titan of twenty-first century commerce. In addition to being a re- tailer, it is now a marketing platform, a delivery and logistics network, a payment service, a credit lender, an auction house, a major book publisher, a producer of television and films, a fashion designer, a hardware manufacturer, and a leading host of cloud server space. Although Amazon has clocked staggering growth, it generates meager profits, choosing to price below-cost and ex- pand widely instead. Through this strategy, the company has positioned itself at the center of e- commerce and now serves as essential infrastructure for a host of other businesses that depend upon it. Elements of the firm’s structure and conduct pose anticompetitive concerns—yet it has escaped antitrust scrutiny. This Note argues that the current framework in antitrust—specifically its pegging competi- tion to “consumer welfare,” defined as short-term price effects—is unequipped to capture the ar- chitecture of market power in the modern economy. We cannot cognize the potential harms to competition posed by Amazon’s dominance if we measure competition primarily through price and output. Specifically, current doctrine underappreciates the risk of predatory pricing and how integration across distinct business lines may prove anticompetitive. These concerns are height- ened in the context of online platforms for two reasons. First, the economics of platform markets create incentives for a company to pursue growth over profits, a strategy that investors have re- warded. Under these conditions, predatory pricing becomes highly rational—even as existing doctrine treats it as irrational and therefore implausible.
  • The Pulitzer Prizes 2020 Winne

    The Pulitzer Prizes 2020 Winne

    WINNERS AND FINALISTS 1917 TO PRESENT TABLE OF CONTENTS Excerpts from the Plan of Award ..............................................................2 PULITZER PRIZES IN JOURNALISM Public Service ...........................................................................................6 Reporting ...............................................................................................24 Local Reporting .....................................................................................27 Local Reporting, Edition Time ..............................................................32 Local General or Spot News Reporting ..................................................33 General News Reporting ........................................................................36 Spot News Reporting ............................................................................38 Breaking News Reporting .....................................................................39 Local Reporting, No Edition Time .......................................................45 Local Investigative or Specialized Reporting .........................................47 Investigative Reporting ..........................................................................50 Explanatory Journalism .........................................................................61 Explanatory Reporting ...........................................................................64 Specialized Reporting .............................................................................70
  • 1 Statement of Justin Brookman Director, Privacy and Technology Policy Consumers Union Before the House Subcommittee on Digital

    1 Statement of Justin Brookman Director, Privacy and Technology Policy Consumers Union Before the House Subcommittee on Digital

    Statement of Justin Brookman Director, Privacy and Technology Policy Consumers Union Before the House Subcommittee on Digital Commerce and Consumer Protection Understanding the Digital Advertising Ecosystem June 14, 2018 On behalf of Consumers Union, I want to thank you for the opportunity to testify today. We appreciate the leadership of Chairman Latta and Ranking Member Schakowsky in holding today’s hearing to explore the digital advertising ecosystem and how digital advertisements affect Americans. I appear here today on behalf of Consumers Union, the advocacy division of Consumer Reports, an independent, nonprofit, organization that works side by side with consumers to create a fairer, safer, and healthier world.1 1 Consumer Reports is the world’s largest independent product-testing organization. It conducts its advocacy work in the areas of privacy, telecommunications, financial services, food and product safety, health care, among other areas. Using its dozens of labs, auto test center, and survey research department, the nonprofit organization rates thousands of products and services annually. Founded in 1936, Consumer Reports has over 7 million members and publishes its magazine, website, and other publications. 1 Executive Summary My testimony today is divided into three parts. First, I describe some of the many ways that the digital advertising ecosystem has gotten more complex in recent years, leaving consumers with little information or agency over how to safeguard their privacy. Consumers are no longer just tracked through cookies in a web browser: instead, companies are developing a range of novel techniques to monitor online behavior and to tie that to what consumers do on other devices and in the physical world.
  • Terry James Resource Library Black History

    Terry James Resource Library Black History

    TERRY JAMES RESOURCE LIBRARY BLACK HISTORY SUBJECT GUIDE BOOKINGS CAN BE SCHEDULE BY CONTACTING 519-822-4420 EXT. 553 OR [email protected] Searching for Resources? Start by selecting this link: https://ug.ent.sirsidynix.net/client/en_CA/TJR/?dt=list The following instructions may be helpful: 1. Enter your “SEARCH TERMS”, IE: “mathematics” and click on “SEARCH”. 2. Select appropriate search limiters from the left side of the search results screen NOTE: select “VIEW ALL” within categories for more options. Page 1 of 16 3. Click on the small box by each title of interest. Please note that you will need to “SELECT AN ACTION”, before moving on to next search results screen. You can “ADD TO MY LISTS” or “EMAIL TO ME”. NOTE: “SELECT AN ACTION” is located at the top & bottom of the page, in grey bar. 4. Once you have marked all the titles of interest, and you selected “ADD TO MY LISTS”, you wil now need to click on “MY LISTS” near the top of the page: 5. Resources will be listed, and can be arranged by Publication Date, Title, Author and more. Place a checkmark in the box by “SELECT ACTION” and notice that all titles in your list will have a checkmark to be included in the email message you will send to your [email protected] or [email protected] email address. Page 2 of 16 6. The list of selected resources arrives in your inbox within a few seconds. Once the list arrives in your inbox, you can close the browser.
  • Julia Angwin

    Julia Angwin

    For more information contact us on: North America 855.414.1034 International +1 646.307.5567 [email protected] Julia Angwin Topics Journalism, Science and Technology Travels From New York Bio Julia Angwin is an award-winning senior investigative reporter at ProPublica, a nonprofit newsroom in New York. From 2000 to 2013, she was a reporter at The Wall Street Journal, where she led a privacy investigative team that was a finalist for a Pulitzer Prize in Explanatory Reporting in 2011 and won a Gerald Loeb Award in 2010. Her book Dragnet Nation: A Quest for Privacy, Security and Freedom in a World of Relentless Surveillance was shortlisted for Best Business Book of the Year by the Financial Times. Julia is an accomplished and sought-after speaker on the topics of privacy, technology, and the quantified society that we live in. Among the many venues at which she has spoken are the Aspen Ideas Festival, the Chicago Humanities Festival, and keynotes at the Strata big data conference and the International Association of Privacy Professionals. In 2003, she was on a team of reporters at The Wall Street Journal that was awarded the Pulitzer Prize in Explanatory Reporting for coverage of corporate corruption. She is also the author of Stealing MySpace: The Battle to Control the Most Popular Website in America. She earned a B.A. in mathematics from the University of Chicago and an MBA from the page 1 / 3 For more information contact us on: North America 855.414.1034 International +1 646.307.5567 [email protected] Graduate School of Business at Columbia University.
  • United States

    United States

    FREEDOM ON THE NET 2016 United States 2015 2016 Population: 321.4 million Internet Freedom Status Free Free Internet Penetration 2015 (ITU): 75 percent Social Media/ICT Apps Blocked: No Obstacles to Access (0-25) 3 3 Political/Social Content Blocked: No Limits on Content (0-35) 2 2 Bloggers/ICT Users Arrested: No Violations of User Rights (0-40) 14 13 TOTAL* (0-100) 19 18 Press Freedom 2016 Status: Free * 0=most free, 100=least free Key Developments: June 2015 – May 2016 ● The USA FREEDOM Act passed in June 2015 limited bulk collection of Americans’ phone records and established other privacy protections. Nonetheless, mass surveillance targeting foreign citizens continues through programs authorized under Section 702 of the FISA Amendments Act and Executive Order 12333 (see Surveillance, Privacy, and Anonymity). ● Online media outlets and journalists face increased pressure, both financially and politically, that may impact future news coverage (see Media, Diversity, and Content Manipulation). ● Following a terrorist attack in San Bernardino in December 2015, the FBI sought to compel Apple to bypass security protections on the locked iPhone of one of the perpetrators (see Surveillance, Privacy, and Anonymity). www.freedomonthenet.org FREEDOM UNITED STATES ON THE NET 2016 Introduction Internet freedom improved slightly as the United States took a significant step toward reining in mass surveillance by the National Security Agency (NSA) with the passage of the USA FREEDOM Act in June 2015. The law ended the bulk collection of Americans’ phone records under Section 215 of the PATRIOT Act, a program detailed in documents leaked by former NSA contractor Edward Snowden in 2013 and ruled illegal by the Second Circuit Court of Appeals in May 2015.
  • Fighting Cyber-Crime After United States V. Jones Danielle K

    Fighting Cyber-Crime After United States V. Jones Danielle K

    Boston University School of Law Scholarly Commons at Boston University School of Law Faculty Scholarship Summer 2013 Fighting Cyber-Crime After United States v. Jones Danielle K. Citron Boston University School of Law David Gray University of Maryland Francis King Carey School of Law Liz Rinehart University of Maryland Francis King Carey School of Law Follow this and additional works at: https://scholarship.law.bu.edu/faculty_scholarship Part of the Privacy Law Commons Recommended Citation Danielle K. Citron, David Gray & Liz Rinehart, Fighting Cyber-Crime After United States v. Jones, 103 Journal of Criminal Law and Criminology 745 (2013). Available at: https://scholarship.law.bu.edu/faculty_scholarship/625 This Article is brought to you for free and open access by Scholarly Commons at Boston University School of Law. It has been accepted for inclusion in Faculty Scholarship by an authorized administrator of Scholarly Commons at Boston University School of Law. For more information, please contact [email protected]. Fighting Cyber-Crime After United States v. Jones David C. Gray Danielle Keats Citron Liz Clark Rinehard No. 2013 - 49 This paper can be downloaded free of charge at: The Social Science Research Network Electronic Paper Collection http://ssrn.com/abstract=2302861 Journal of Criminal Law and Criminology Volume 103 | Issue 3 Article 4 Summer 2013 Fighting Cybercrime After United States v. Jones David Gray Danielle Keats Citron Liz Clark Rinehart Follow this and additional works at: http://scholarlycommons.law.northwestern.edu/jclc Part of the Criminal Law Commons Recommended Citation David Gray, Danielle Keats Citron, and Liz Clark Rinehart, Fighting Cybercrime After United States v.
  • Literary Review of Canada a Journal of Ideas NOW AVAILABLE from HOUSE of ANANSI PRESS RIDGERUNNER

    Literary Review of Canada a Journal of Ideas NOW AVAILABLE from HOUSE of ANANSI PRESS RIDGERUNNER

    MARK NKALUBO NABETA Unrest MORGAN CAMPBELL Race and the Media DAN DUNSKY China’s Moment SHEREE FITCH Writing through Grief july | August 2020 Literary Review of Canada A journAl of ideAs NOW AVAILABLE FROM HOUSE OF ANANSI PRESS RIDGERUNNER THE HIGHLY ANTICIPATED FOLLOW-UP TO THE OUTLANDER, BY GIL ADAMSON “RIDGERUNNER IS A BRILLIANT LITERARY ACHIEVEMENT . I LOVED EVERY PAGE OF IT.” — Michael Redhill, Scotiabank Giller Prize–winning author of Bellevue Square “TRULY MAGNIFICENT.” — Robert Olmstead, award-winning author of Coal Black Horse and Savage Country “RIDGERUNNER IS A WILD ADVENTURE SPUN IN EXALTED PROSE: THE BOOK I’VE BEEN WANTING TO READ FOR YEARS.” — Marina Endicott, award-winning author of Good to a Fault and The Difference ALSO AVAILABLE: THE OUTLANDER @HOUSEOFANANSI ANANSI PUBLISHES HOUSEOFANANSI.COM VERY GOOD BOOKS july | august 2020 ◆ volume 28 ◆ number 6 a journal of ideas first word the argument playtime Summer School China’s Moment Snuffed Torch Kyle Wyatt Reckoning with an empire state of mind Can the Olympic myth survive? 3 Dan Dunsky Laura Robinson 13 26 the public square False Notions pandemic the arts Yes, certain conditions continue to exist A Northern Light North and South Mark Nkalubo Nabeta Nunavut’s hope to avoid the outbreak Cuba’s Orwellian mystery 5 Sarah Rogers Amanda Perry 15 28 Under the Guise of Research Science and subjugation compelling people literature John Baglow 6 National Personality Trying Situations The legacy of Marcel Cadieux A new collection from David Bergen An Act of Protest Bruce K. Ward David Staines Desmond Cole says his piece 16 30 Morgan Campbell 8 bygone days An Urgent Realm Harsh Treatment Mallory Tater’s dark debut this and that Cecily Ross Perspectives on internment 31 Waiting on Tables J.
  • 34:3 Berkeley Technology Law Journal

    34:3 Berkeley Technology Law Journal

    34:3 BERKELEY TECHNOLOGY LAW JOURNAL 2019 Pages 705 to 918 Berkeley Technology Law Journal Volume 34, Number 3 Production: Produced by members of the Berkeley Technology Law Journal. All editing and layout done using Microsoft Word. Printer: Joe Christensen, Inc., Lincoln, Nebraska. Printed in the U.S.A. The paper used in this publication meets the minimum requirements of American National Standard for Information Sciences— Permanence of Paper for Library Materials, ANSI Z39.48—1984. Copyright © 2019 Regents of the University of California. All Rights Reserved. Berkeley Technology Law Journal University of California School of Law 3 Law Building Berkeley, California 94720-7200 [email protected] https://www.btlj.org BERKELEY TECHNOLOGY LAW JOURNAL VOLUME 34 NUMBER 3 2019 TABLE OF CONTENTS ARTICLES THE INSTITUTIONAL LIFE OF ALGORITHMIC RISK ASSESSMENT ............................ 705 Alicia Solow-Niederman, YooJung Choi & Guy Van den Broeck STRANGE LOOPS: APPARENT VERSUS ACTUAL HUMAN INVOLVEMENT IN AUTOMATED DECISION MAKING .................................................................................. 745 Kiel Brennan-Marquez, Karen Levy & Daniel Susser PROCUREMENT AS POLICY: ADMINISTRATIVE PROCESS FOR MACHINE LEARNING ........................................................................................................................... 773 Deirdre K. Mulligan & Kenneth A. Bamberger AUTOMATED DECISION SUPPORT TECHNOLOGIES AND THE LEGAL PROFESSION .......................................................................................................................
  • Automating the Risk of Bias

    Automating the Risk of Bias

    41893-gwn_87-5 Sheet No. 99 Side B 01/29/2020 09:32:14 \\jciprod01\productn\G\GWN\87-5\GWN507.txt unknown Seq: 1 29-JAN-20 9:19 Automating the Risk of Bias Kristin N. Johnson* ABSTRACT Artificial intelligence (“AI”) is a transformative technology that has radi- cally altered decision-making processes. Evaluating the case for algorithmic or automated decision-making (“ADM”) platforms requires navigating tensions between two normative concerns. On the one hand, ADM platforms may lead to more efficient, accurate, and objective decisions. On the other hand, early and disturbing evidence suggests ADM platform results may demonstrate bi- ases, undermining claims that this special class of algorithms will democratize markets and increase inclusion. State law assigns decision-making authority to the boards of directors of corporations. State courts and lawmakers accord significant deference to the board in the execution of its duties. Among its duties, a board must employ effective oversight policies and procedures to manage known risks. Conse- quently, the board of directors and senior management of firms integrating ADM platforms must monitor operations to mitigate enterprise risks including litigation, reputation, compliance, and regulatory risks that arise as a result of the integration of algorithms. After the recent financial crisis, firms adopted structural and procedural governance reforms to mitigate various enterprise risks; these approaches may prove valuable in mitigating the risk of algorithmic bias. Evidence demon- strates that heterogeneous teams may identify and mitigate risks more success- fully than homogeneous teams. Heterogeneous teams are more likely to overcome cognitive biases such as confirmation, commitment, overconfidence, and relational biases.