PROPHETS FOR AN ALGORITHMIC AGE

NICHOLAS MIGNANELLI

I. IT’S A COOKBOOK! As a child, I had an affinity for all things paranormal: cryptozoology, parapsychology, and especially ufology. That the first alleged alien abduction in the United States took place at a location some fifteen minutes from my childhood home may have had something to do with it.1 Yet it was only later in life that I discovered The Twilight Zone. Like many fans of this classic American television series, my favorite episode is “To Serve Man.”2 In it, a fleet of spaceships belonging to an alien species called the Kanamits—large-headed

 Research & Instructional Services Librarian, Yale Law School. My thanks to the Program Committee of the 2021 Annual Meeting of the Association of American Law Schools (“AALS”) for selecting the panel upon which this symposium is based as an open source program after it was rejected by the AALS Section on Law Libraries and Legal Information. For their encouragement and thoughtful comments on this Essay, I would also like to thank Paula Hughes Mignanelli, Nicholas F. Stump, Liz Schiller, and Sarah C. Slinger. 1 Betty and Barney Hill, a couple from Portsmouth, New Hampshire, claimed that aliens from the Zeta Reticuli star system kidnapped them while they were driving home from a vacation in Montreal on the night of September 19, 1961. Although the exact location of the alleged abduction site is unknown, it is speculated to have taken place somewhere along Tripoli Road, a remote seasonal road connecting Woodstock and Waterville Valley, New Hampshire. See STANTON T. FRIEDMAN & KATHLEEN MARDEN, CAPTURED!: THE BETTY AND BARNEY HILL UFO EXPERIENCE 165-66 (2007); see also JOHN G. FULLER, THE INTERRUPTED JOURNEY: TWO LOST HOURS “ABOARD A FLYING SAUCER” 268 (1966). The Betty and Barney Hill Papers are housed in the Special Collections of the University of New Hampshire Library. Like the author of this Essay, Betty Hill was a graduate of the University of New Hampshire. See Guide to the Betty and Barney Hill Papers, 1961-2006, U.N.H. LIBR., https://library.unh.edu/find/archives/collections/betty-barney-hill-papers-1961-2006 [https://perma.cc/BFJ2-CQE7] (last visited Apr. 5, 2021). 2 The Twilight Zone: To Serve Man (CBS television broadcast Mar. 2, 1962). In 2019, Rolling Stone ranked this episode number one in its list of the twenty-five best episodes of The Twilight Zone. See David Fear, Sean T. Collins & Angie Martoccio, 25 Best ‘Twilight Zone’ Episodes, ROLLING STONE (Apr. 1, 2019, 12:36 PM), https://www.rollingstone.com /tv/tv-features/25-best-twilight-zone-episodes-list-812043/ [https://perma.cc/CB66-4UL7]. I suspect that the late Harvard Law Professor and pioneering Critical Race Theorist Derrick Bell was also a fan of this episode. See Derrick Bell, The Space Traders, in DARK MATTER: A CENTURY OF SPECULATIVE FICTION FROM THE AFRICAN DIASPORA 326 (Sheree R. Thomas ed., 2000). I was disappointed—but not surprised—to learn that a 2021 AALS program sponsored by the Section on Biolaw entitled “Science Fiction and the Law” made no mention of Bell’s work. 41

42 BOSTON UNIVERSITY LAW REVIEW ONLINE [Vol. 101:41 humanoid creatures that stand nine feet tall and weigh 350 pounds—lands on Earth. Soon, a representative of the Kanamits appears before the United Nations, declaring that their “intentions are honorable,” that they “desire above all things to help the people of Earth,” and that they “will not force anything on [human beings].”3 In short, this interstellar ambassador explains that the Kanamits have come with advanced technologies to fix all of humanity’s problems. Exiting the meeting, the ambassador leaves behind a mysterious volume written in an extraterrestrial script. Still suspicious of the motives of the Kanamits in spite of the ambassador’s assurances, the U.S. government assigns cryptographer Michael Chambers the seemingly impossible task of decoding it. As Chambers and his team go about attempting to decipher the text, the Kanamits introduce humans to a nitrate fertilizer that ends famine, an atomic generator that serves as an unlimited source of affordable energy, and a forcefield technology that prevents international warfare. Meanwhile, Chambers’s colleague Patty decodes the title of the volume: To Serve Man. Chambers concludes that this is incontrovertible evidence of the benevolence of the Kanamits. The technological innovations introduced by the Kanamits change the very fabric of life on Earth: the military is disbanded as world peace is achieved, the Kanamits establish embassies in every major city, and a “voluntary” exchange program allows humans to travel to the Kanamits’s reportedly paradisical home planet. Despite all this, an ever-skeptical Patty persists in her attempt to fully translate the text. Chambers, on the other hand, decides to enroll in the exchange program. On the day of Chambers’s departure, an exasperated Patty runs toward the platform where he is about to board a spacecraft. She has an urgent message for Chambers, presumably regarding the text. As a Kanamit guard holds her back, Patty cries out: “Mr. Chambers, don’t get on that ship! The rest of the book, To Serve Man, it’s…it’s a cookbook!”4 Chambers attempts to disembark, but a guard blocks him from leaving, the stairs retract, and the spacecraft takes off. Now, when you spend the first several paragraphs of a scholarly essay recounting the plot of a television show that aired almost sixty years ago, you better have a good reason. So why did I write all of this? Well, because when reading law library literature on the subject of artificial intelligence (“AI”), I often feel like Patty—played by Susan Cummings—hopelessly yelling “it’s a cookbook!” as AI rushes legal research on to a metaphorical spaceship bound for horrors of cosmic proportion. The vendors of so-called “AI-powered” legal research have, after all, promised that their products will serve the legal researcher. But let me unpack this melodramatic allegory more.

3 The Twilight Zone: To Serve Man, supra note 2. 4 Id.

2021] PROPHETS FOR AN ALGORITHMIC AGE 43

Like the Kanamits, vendors have made grand promises about the potential AI technology has to transform the practice of law.5 Like Michael Chambers, a number of law librarians have displayed unwarranted trust and optimism in assessing these claims.6 And yet, the most disturbing parallel is the false sense of voluntariness. While vendors typically imply that adopting “AI-powered” legal research is entirely voluntary, certain comments made at an ABA Techshow panel last year suggest that at least some vendors believe (hope?) that this will not always be the case.7

5 See, e.g., Ed Walters, Read/Write: Artificial Intelligence Libraries, AALL SPECTRUM, Sept./Oct. 2017, at 21, 23 (“A great shift is coming, as great as the Industrial Revolution or the invention of electricity. The most successful libraries of the next 10 years will be the ones that embrace the new tools of the trade.”). I hasten to add that, for the time being, vendors have underdelivered on their promises. See Lyle Moran, Law Firms Are Slow to Adopt AI- Based Technology Tools, ABA Survey Finds, ABA J. (Oct. 22, 2020, 12:13 PM), https://www.abajournal.com/web/article/law-firms-are-slow-to-adopt-artificial-intelligence- based-technology-tools-aba-survey-finds [https://perma.cc/JFL4-VBCM] (“Just 7% of respondents to the ABA Legal Technology Resource Center’s survey reported that their firms use AI tech tools, a decrease of one percentage point from a year ago.”). 6 See, e.g., LAW LIBRARIANSHIP IN THE AGE OF AI (Ellyssa Kroski ed., 2019) (touting the benefits of AI to law librarianship); Patrick DiDomenico & James Lee, First Our Books, And Now Our Jobs?: Relax, Artificial Intelligence Won’t Replace You Anytime Soon, AALL SPECTRUM, July/Aug. 2019, at 50, 52 (“Professionals with real intelligence will embrace artificial intelligence and look for ways that it can help increase efficiency and provide greater value to their firms and clients.”); Mark Gediman, Artificial Intelligence: Not Just Sci-Fi Anymore, AALL SPECTRUM, Sept./Oct. 2016, at 34, 34 (“[I]ncorporating an AI-based system in your institution is the future of the profession.”); Jean P. O’Grady, Hand in Hand with IBM Watson, AALL SPECTRUM, Sept./Oct. 2015, at 18, 21 (“As a profession, it is important that we don’t identify with the pre-Gutenberg scribes. . . . Take a deep breath; we are living in a post-Watson world. Everything is about to change.”). But see Kim P. Nayyer, Marcelo Rodriguez & Sarah Sutherland, Artificial Intelligence & Implicit Bias: With Greater Power Comes Great Responsibility, AALL SPECTRUM, May/June 2020, at 14, 15-16 (calling on law librarians to consider the ethical implications of algorithmic bias in legal research and law practice technologies); Susan Nevelow Mart, The Algorithm as a Human Artifact: Implications for Legal [Re]search, 109 LAW LIBR. J. 387, 406-20 (2017) (demonstrating that the subjective choices made in the course of creating the algorithms that underlie a legal database determine what results that database will show researchers). 7 Thomas Hamilton, then the Vice President of Strategy and Operation at Ross Intelligence (now defunct), “argued there may come a time when courts demand lawyers use artificial intelligence to research arguments.” Stephanie Francis Ward, How Accurate Is AI in Legal Research?, ABA J. (Feb. 28, 2020, 7:30 PM), https://www.abajournal.com/news/article/how- accurate-is-ai-in-legal-research [https://perma.cc/95VG-2H7E]. These comments echo the earlier statements of vendors tucked away in the pages of law journals. See generally Andrew Arruda, An Ethical Obligation to Use Artificial Intelligence: An Examination of the Use of Artificial Intelligence in Law and the Model Rules of Professional Responsibility, 40 AM. J. TRIAL ADVOC. 443 (2017); see also Ed Walters, The Model Rules of Autonomous Conduct: Ethical Responsibilities of Lawyers and Artificial Intelligence, 35 GA. ST. U. L. REV. 1073, 1091-92 (2019) (“The Model Rules do not prohibit the use of AI or data analytics; in fact, the

44 BOSTON UNIVERSITY LAW REVIEW ONLINE [Vol. 101:41

Mindful of these circumstances, I recently published an article in Law Library Journal in which I use Professors Richard Delgado and Jean Stefancic’s theory of the “triple helix dilemma” to examine the impact that AI will have on the legal research process and, ultimately, the future of legal innovation and law reform.8 I conclude that, because it conceals the legal research process and entrenches the biases of society’s dominant interests, “AI-powered” legal research threatens to “close the legal imagination and turn the law into a monolith.”9 Adapting Nicholas F. Stump’s Critical Legal Research framework, I recommend that law librarians contend with this phenomenon by participating in algorithmic activism, practicing transgressive and archeological bibliography, and emphasizing the importance of “unplugged brainstorming” in their pedagogy.10 With my earlier article as a foundation, I will use the remaining pages of this Essay to further engage Professors Delgado and Stefancic’s commentary on the “triple helix dilemma” in the hope of suggesting where scholarship on AI and legal research might go from here.

II. OF PROPHESIES AND CHRONICLES “The gift of prophesy, when practiced by earthly oracles,” wrote the late Professor Derrick Bell, “entails a risky willingness to predict future events based on an examination of the present using the insight provided by an evaluation of the past.”11 I would add that there are also times when commentators unintentionally act as prophets: when their insights about the contours of the present are so penetrating that they cut to the very structures that have shaped it. This, I believe, is true of Professors Delgado and Stefancic and their theory of the “triple helix dilemma.” Much can be said of the far-reaching implications of their 1989 article on the subject, but what has always struck me is the prescience of certain comments they made in their 2007 article “Why Do We Ask the Same Questions? The Triple Helix Dilemma Revisited.”12 Discussing the shortcomings of the keyword

Rule 1.1 duty of competence and the Rule 1.3 duty of diligence increasingly will require lawyers to use these tools.”). 8 Nicholas Mignanelli, Critical Legal Research: Who Needs It?, 112 LAW LIBR. J. 327, 331-34 (2020) (citing Richard Delgado & Jean Stefancic, Why Do We Tell the Same Stories?: Law Reform, Critical Librarianship, and the Triple Helix Dilemma, 42 STAN. L. REV. 207 (1989) [hereinafter Delgado & Stefanic, Why Do We Tell the Same Stories?]; and then Richard Delgado & Jean Stefancic, Why Do We Ask the Same Questions? The Triple Helix Dilemma Revisited, 99 LAW LIBR. J. 307 (2007) [hereinafter Delgado & Stefanic, Why Do We Ask the Same Questions?]). 9 Id. at 335-40. 10 Id. at 340-43 (citing Nicholas F. Stump, Following New Lights: Critical Legal Research Strategies as a Spark for Law Reform in Appalachia, 23 AM. U. J. GENDER SOC. POL’Y & L. 573 (2015)). 11 Derrick Bell, Racism: A Prophecy for the Year 2000, 42 RUTGERS L. REV. 93, 93 (1989). 12 Delgado & Stefancic, Why Do We Ask the Same Questions?, supra note 8.

2021] PROPHETS FOR AN ALGORITHMIC AGE 45 searching techniques that had come to dramatically alter the traditional tools of legal research, Professors Delgado and Stefancic write: Internet searching presents a different obstacle. Most material on the Internet comes arranged in order of frequency of use. Web sites and home pages that previous users have visited many times appear high on the page, less used ones further down. This “popularity contest” approach to listing information builds in a structural bias in favor of commonplace items that have found wide use. Heretical or new ideas that are just beginning to be noticed may easily escape the attention of a busy searcher.13 The crowdsourcing that they were describing was a forerunner of what we have come to know as AI, specifically machine learning—the form of AI that employs algorithms that “discern patterns in data and infer rules on their own.”14 The rise of AI in recent years reads like a chronicle ripped from the pages of one of Professor Bell’s law review articles . . . A small group of disproportionately white, disproportionately male, and disproportionately upper-class technologists educated at America’s elite universities15 discovers a way to use complex mathematical formulas to make life more convenient—read: automated—for everyone (who matters). In doing so, they promise to save governments and corporations billions of dollars.16 The adoption of this technology does, however, require a trade- off: in order to achieve the efficiency and cost savings so valued by those in power, the hope that marginalized people have for reform and systemic change must be sacrificed.17 You see, by relying on user data riddled with racial, sex, and class biases, among others, this technology threatens to

13 Id. at 325 (footnotes omitted); see also Ronald E. Wheeler, Does WestlawNext Really Change Everything? The Implications of WestlawNext on Legal Research, 103 LAW LIBR. J. 359, 369 (2011) (theorizing that use of crowdsourcing technology in legal research databases will “limit the reach of creative thinking about the law, . . . narrow the range of alternative legal perceiptions, [and] . . . close the door to the unknown”). 14 Carolyn Elefant, Part I: The Basics – What Is AI?, MYSHINGLE.COM (July 19, 2019), https://myshingle.com/2019/07/articles/future-trends/part-i:-the-basics—-what-is-ai/ [https://perma.cc/UFY9-SDA5]. 15 See generally SARAH MYERS WEST, MEREDITH WHITTAKER & KATE CRAWFORD, AI NOW INST., DISCRIMINATING SYSTEM: GENDER, RACE, AND POWER IN AI (2019), https://ainowinstitute.org/discriminatingsystems.pdf [https://perma.cc/5MGP-DM62]. 16 See Maria Aspan, This Tech Giant Says A.I. Has Already Helped It Save $1 Billion, FORTUNE (Jan. 24, 2020, 11:00 AM), https://fortune.com/2020/01/24/ai-ibm-human- resources/ [https://perma.cc/BX8B-Y888]; PETER VIECHNICKI & WILLIAM D. EGGERS, DELIOTTE CTR. FOR GOV’T INSIGHTS, HOW MUCH TIME AND MONEY CAN AI SAVE GOVERNMENT? 14-15 (2017), https://www2.deloitte.com/content/dam/insights/us/articles /3834_How-much-time-and-money-can-AI-save-government/DUP_How-much-time-and- money-can-AI-save-government.pdf [https://perma.cc/Q4Y4-P5TD]. 17 As Bell argued, involuntary sacrifice is a mainstay of American history. See DERRICK BELL, RACE, RACISM AND AMERICAN LAW 43-45 (6th ed. 2008).

46 BOSTON UNIVERSITY LAW REVIEW ONLINE [Vol. 101:41

forever entrench the inequities that lie at the heart of our society.18 Soon this technology finds its way into law offices and courtrooms when it begins to “power” legal research. Of course, unlike the speculative fiction stories that are a hallmark of Professor Bell’s scholarship, this chronicle is not a parable or a thought experiment. We are living in it.

III. LISTEN TO THE VOICES In their 1989 article, “Why Do We Tell the Same Stories?: Law Reform, Critical Librarianship and the Triple Helix Dilemma,” Professors Delgado and Stefancic invite researchers to look to “divergent individuals,” that is to say “individuals whose life experiences have differed markedly from those of their contemporaries,” because “[t]heir ideas offer the possibility of legal transformation and growth.”19 It occurs to me that this is precisely what we must do as we contemplate how “AI-powered” legal research threatens to “entrench[] the biases of society’s dominant interests.”20 Several critical race and feminist scholars working in other fields have begun the difficult task of thinking about and criticizing AI by “looking to the bottom.”21 These scholars include Ruha Benjamin (author of Race After Technology: Abolitionist Tools for the New Jim Code),22 Safiya Umoja Noble (author of Algorithms of Oppression),23 and Virginia Eubanks (author of Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor).24 In the realm of activism, Joy Buolamwini, a computer scientist and self- described “poet of code” based at the MIT Media Lab,25 has founded the Algorithmic Justice League “to raise public awareness about the impacts of AI, equip advocates with empirical research to bolster campaigns, build the voice and choice of most impacted communities, and galvanize researchers, policymakers, and industry practitioners to mitigate AI bias and harms.”26

18 See Mignanelli, supra note 8, at 338-40. 19 Delgado & Stefancic, Why Do We Tell the Same Stories?, supra note 8, at 223. 20 Mignanelli, supra note 8, at 336. 21 By which I mean “adopting the perspective of those who have seen and felt the falsity of the liberal promise . . . [to] fathom[] the phenomenology of law and defin[e] the elements of justice.” Mari J. Matsuda, Looking to the Bottom: Critical Legal Studies and Reparations, 22 HARV. C.R.-C.L. L. REV. 323, 324 (1987). 22 See generally RUHA BENJAMIN, RACE AFTER TECHNOLOGY: ABOLITIONIST TOOLS FOR THE NEW JIM CODE (2019). 23 See generally SAFIYA UMOJA NOBLE, ALGORITHMS OF OPPRESSION (2018). 24 See generally VIRGINIA EUBANKS, AUTOMATING INEQUALITY: HOW HIGH-TECH TOOLS PROFILE, POLICE, AND PUNISH THE POOR (2018). 25 Joy Buolamwini, MIT MEDIA LAB, https://www.media.mit.edu/people/joyab/overview/ [https://perma.cc/HV8P-TS9Q] (last visited Apr. 5, 2021). 26 Our Mission, ALGORITHMIC JUST. LEAGUE, https://www.ajl.org/about [https://perma.cc/44WR-HX7W] (last visited Apr. 5, 2021).

2021] PROPHETS FOR AN ALGORITHMIC AGE 47

Buolamwini’s efforts are even the subject of a new documentary film entitled Coded Bias.27 Unfortunately, this important work has yet to gain the widespread attention of legal scholars and law librarians. But if we are to take seriously the threat bias in “AI-powered” legal research and law practice technologies poses to marginalized people and their ability to use the law to challenge the power structures that subordinate them, we must begin engaging the above scholars and thinking about how their ideas apply to legal information.

27 CODED BIAS (7th Empire Media 2020). For more information, see CODED BIAS, https://www.codedbias.com [https://perma.cc/X7CB-92VZ] (last visited Apr. 5, 2021).