A Government of Laws and Not of Machines
Total Page:16
File Type:pdf, Size:1020Kb
A GOVERNMENT OF LAWS AND NOT OF MACHINES EMILY BERMAN INTRODUCTION ............................................................................................ 1278 I. DATA MINING FOR SECURITY ............................................................. 1284 A. Defining Machine Learning...................................................... 1284 B. Use of Machine Learning in the Security Context .................... 1290 II. MACHINE LEARNING AND ITS DISCONTENTS ...................................... 1301 A. Technological Challenges ........................................................ 1302 1. Sufficiency of the Data ....................................................... 1302 2. Selecting Features ............................................................... 1305 3. Choosing a Model .............................................................. 1306 4. Verification ......................................................................... 1307 B. Machine Learning and Rule-of-Law Values ............................. 1309 1. Identifying Rule-of-Law Values ......................................... 1309 a. Ensuring Individuals Can Plan Their Affairs Effectively .................................................................... 1311 b. Constraints on Arbitrary Exercise of Government Power ........................................................................... 1312 c. Government Legitimacy ............................................... 1313 2. Machine Learning’s Tensions with Fundamental Values ................................................................................. 1315 a. Opacity ........................................................................ 1315 b. Arbitrariness of Errors ................................................ 1322 c. The Human Factor ....................................................... 1325 III. IMPLICATIONS ..................................................................................... 1331 A. The Discretion Continuum ....................................................... 1333 B. High-Discretion Decisions and Machine Learning .................. 1338 Assistant Professor of Law, University of Houston Law Center. Thanks to Seth Chandler, Ashley Deeks, Victor Flatt, Tracy Hester, Aziz Huq, David Kwok, Peter Margulies, Doug Moll, James Nelson, D. Theodore Rave, Joe Sanders, and participants in the 2018 American Association of Law Schools session on the intersection of technology and civil rights as well as a South Texas School of Law workshop for helpful comments. Thanks also go to Seth Chandler and James Winkle for their patience in explaining some of the more technical aspects of machine learning. 1277 1278 BOSTON UNIVERSITY LAW REVIEW [Vol. 98:1277 C. Low-Discretion Decisions and Machine Learning ................... 1342 1. Limited Added Value of Machine Learning in Low- Discretion Decisions ........................................................... 1343 2. Undermining the Rule of Law ............................................ 1349 CONCLUSION ................................................................................................ 1355 Each week brings another story touting the miracle of “machine learning”— a strand of artificial intelligence that uses mathematical algorithms to construct computer models that analyze enormous data sets, often for the purpose of making predictions about the future. Machine learning is all around us—it is used for spam filters, facial recognition, detecting bank fraud, calculating credit risk, and much more—and it is immensely powerful. This Article considers the government’s use of machine learning in the context of law enforcement and national security decision-making, taking a step back from the nuts and bolts questions surrounding the implementation of predictive analytics, on which most scholarly commentary has focused, to assess their use from a more conceptual perspective. The question I seek to answer is this: whether reliance on the output of machine-learning models—even if highly accurate—is consistent with the goal to maintain “a government of laws” and not of machines. I conclude that government officials operating in contexts where they enjoy broad decision- making discretion should embrace machine-learning predictions as a valuable tool. By contrast, when government discretion is highly constrained by existing constitutional, statutory, or regulatory rules, the use of machine-learning predictions represents a threat to the rule-of-law. INTRODUCTION “Machine learning” is a strand of artificial intelligence that sits at the intersection of computer science, statistics, and mathematics, and it is changing the world.1 The applications of machine learning in modern society are nearly 1 See, e.g., Steve Barrett, AI is Changing the World, but Will It End in Utopia or Dystopia?, PR WEEK (Feb. 9, 2018), https://www.prweek.com/article/1456842/ai-changing-world-will- end-utopia-dystopia (highlighting benefits and dangers of improved artificial intelligence technology); Bernard Marr, 5 Key Artificial Intelligence Predictions for 2018: How Machine Learning Will Change Everything, FORBES (Dec. 18, 2017, 12:28 AM), https://www. forbes.com/sites/bernardmarr/2017/12/18/5-key-artificial-intelligence-predictions-for-2018- how-machine-learning-will-change-everything/#7a1c79c56545 (“I expect 2018 to provide a continuous stream of small but sure steps forward, as machine learning and neural network technology takes on more routine tasks.”); Roger Parloff, Why Deep Learning Is Suddenly Changing Your Life, FORTUNE (Sept. 28, 2016, 5:00 PM), http://fortune.com/ai-artificial- 2018] A GOVERNMENT OF LAWS AND NOT OF MACHINES 1279 endless: search engines, spam filters, Amazon and Netflix recommendations, voice and facial recognition, self-driving cars, spotting bank fraud, creditworthiness determinations, medical diagnoses, apps that transform your photos into the style of your favorite painter, robotic vacuum cleaners, and automated weapons.2 Machine learning is thus an immensely powerful tool that already has transformed society’s ability to exploit data. Machine learning is, in essence, a particularly powerful version of data mining. The value of data mining stems from its capacity to make sense of so- called “big data”—enormous databases full of various bits of information. These data sets are too complex for humans to understand because of the volume of the information, because there are too many variables for humans to process, or because the meaningful relationships among the data are not self-evident. What sets machine learning apart from other forms of data mining is that the mathematical algorithms on which it relies to identify the meaningful patterns within a data set are able to learn from experience and become more accurate over time.3 These algorithms make inferences from the data to generate intelligence-deep-machine-learning/ [https://perma.cc/T7UU-CZ9G] (stating breakthroughs in voice recognition, image recognition, and machine translation are all due to artificial intelligence); Tom Simonite, The Wired Guide to Artificial Intelligence, WIRED (Feb. 1, 2018, 9:22 AM), https://www.wired.com/story/guide-artificial-intelligence/ (“The current boom in all things AI was catalyzed by breakthroughs in an area known as machine learning.”). 2 See, e.g., Danielle Keats Citron & Frank Pasquale, The Scored Society: Due Process for Automated Predictions, 89 WASH. L. REV. 1, 4 (2017) (“A credit card company uses behavioral-scoring algorithms to rate consumers’ credit risk . .”); Harry Surden, Machine Learning and Law, 89 WASH. L. REV. 87, 88 (2014) (“In the last few decades, researchers have successfully used machine learning to automate . autonomous (i.e., self-driving) cars . .”); John Brandon, Why the iRobot Roomba 980 Is a Great Lesson on the State of AI, VENTUREBEAT (Nov. 3, 2016, 4:10 PM), https://venturebeat.com/2016/11/03/why-the-irobot- roomba-980-is-a-great-lesson-on-the-state-of-ai/ [https://perma.cc/D22G-6APR] (“[T]he latest Roomba . uses true AI.”); Kyle Mizokami, Kalashnikov Will Make an A.I.-Powered Killer Robot, POPULAR MECHANICS (July 19, 2017), https://www.popularmechanics.com/ military/weapons/news/a27393/kalashnikov-to-make-ai-directed-machine-guns/ [https://per ma.cc/QX92-7F6G] (“Russian weapons maker Kalashnikov is working on an automated gun system that uses artificial intelligence to make ‘shoot/no shoot’ decisions.”); Kumba Sennaar, Machine Learning for Medical Diagnostics—4 Current Applications, TECHEMERGENCE (Jan. 11, 2018), https://www.techemergence.com/machine-learning-medical-diagnostics-4- current-applications/ [https://perma.cc/NL4C-E4NX] (“Today, AI is playing an integral role in the evolution of the field of medical diagnostics.”); PIKAZO, INC., http://www.pikazo app.com/ [https://perma.cc/T7MK-AYLF] (last visited Sept. 11, 2018). 3 See Surden, supra note 2, at 89 (explaining that machine-learning algorithms are “capable of changing their behavior to enhance their performance on some task through experience”). 1280 BOSTON UNIVERSITY LAW REVIEW [Vol. 98:1277 computer models that expose new insights about the world and, in many instances, make predictions about the future.4 Given its utility, it is not surprising that government decision-makers seek to harness machine learning’s predictive power for public-sector use. These tools already have made significant inroads in the contexts of national security and law enforcement. In these areas, predictive computer models promise to allocate government resources more