(2021) 33 SAcLJ 280 JUDICIAL DECISION-MAKING AND EXPLAINABLE ARTIFICIAL INTELLIGENCE A Reckoning from First Principles In light of rapid developments in legal technology, it is timely to begin considering whether, and if so how, artificial intelligence (“AI”) can replace judges. However, given that law plays a crucial role in maintaining societal order, that judges are a crucial part of ensuring the continued well-functioning of the law, and also that there are still many unknowns in the use and deployment of AI, it would be prudent to examine and understand exactly what roles judges play in the legal system, and how they do so, before we make any bold steps towards replacing judges with AI. This article examines the current and reasonably foreseeable state of AI to consider its capabilities, as well as the process by which judges make decisions and the duties they are subject to. This article will then consider whether or how AI, given its current and foreseeable state of development, may be used in judicial decision-making, and what safeguards may be required to ensure continued confidence in a well-functioning justice system. Shaun LIM1 LLB (Hons) (National University of Singapore); Advocate and Solicitor (Singapore); Research Assistant, Centre for Technology, Robotics, Artificial Intelligence & the Law, Faculty of Law, National University of Singapore I. Introduction 1 Advancements in artificial intelligence (“AI”) techniques with demonstrable results have led to a boom in AI research, development and marketing, especially in fields dominated by specialist professionals whose knowledge was thought to be impossible for AI to replicate, such as in medicine and law.2 In law itself, AI solutions are being 1 This article expands upon a prior research paper for credit in the course of the author’s LLB degree. The author is indebted to Assoc Prof Daniel Seng for his comments and suggestions on this article, as well as to District Judge Wong Li Tein for her comments on the earlier version of this article. 2 See, eg, Xiaomin Mou, “Artificial Intelligence: Investment Trends and Selected Industry Uses” (EMCompass Note 71, September 2019) <https://www.ifc.org/ (cont’d on the next page) © 2021 Contributor(s) and Singapore Academy of Law. No part of this document may be reproduced without permission from the copyright holders. Judicial Decision-making and (2021) 33 SAcLJ Explainable Artificial Intelligence 281 commercially offered for applications such as due diligence, legal analytics, outcome predictors, document automation and practice management.3 The Singapore judiciary, long an enthusiastic supporter of technological solutions in the courts, has itself pioneered many legal technological innovations over the years, such as the Integrated Case Management System4 for criminal case management, the Automated Court Documents Assembly5 service for certain categories of cases such as bankruptcy and Magistrate’s Complaints, the Community Justice and Tribunals System6 for the Small Claims Tribunal, Community Disputes Resolution Tribunal and Employment Claims Tribunal cases, and even the Speech Transcription System,7 a voice recognition system for instant transcription of court proceedings. 2 The final frontier of the use of technology in courts then appears to be the actual rendering of judicial decisions itself, although few countries appear prepared to replace judges with AI. Even China, which has been devoting significant resources into developing technologically-advanced courts and legal assistants, appears to consider the use of AI in actual judicial decision-making to be unthinkable, saying that the expertise of judges is irreplaceable, not least by AI.8 It is of course possible to suggest several reasons for such a sentiment, such as societal confidence or lack thereof in AI, the need for the accountability that accompanies a societal presence, or the relative ease of training humans in common-sense reasoning as compared to AI. But in isolation, such a bald statement, made without substantiation, sounds merely like judicial exceptionalism, especially in light of the rapid advances that AI has made in recent years, and only exacerbates the risk of judiciaries being caught flat-footed if or when the spectre of AI judicial decision-making becomes reality. wps/wcm/connect/7898d957-69b5-4727-9226-277e8ae28711/EMCompass-Note- 71-AI-Investment-Trends.pdf?MOD=AJPERES&CVID=mR5Jvd6> (accessed 20 April 2020). 3 Daniel Faggella, “AI in Law and Legal Practice – A Comprehensive View of 35 Applications” Emerj (14 March 2020) <https://emerj.com/ai-sector-overviews/ ai-in-law-legal-practice-current-applications/> (accessed 20 April 2020). 4 See State Courts Singapore, “Integrated Case Management System” <https://www. statecourts.gov.sg/cws/CriminalCase/Pages/The-ICMS-portal.aspx> (accessed 20 April 2020). 5 See Community Justice Centre, “What is Automated Court Documents Assembly?” <https://cjc.org.sg/automated-court-documents-assembly/self-help-bankruptcy/ frequently-asked-questions/> (accessed 20 April 2020). 6 See State Courts Singapore, “Community Justice and Tribunals System” <https:// www.statecourts.gov.sg/CJTS/#!/index1> (accessed 20 April 2020). 7 Tan Tam Mei, “State Courts to Use System That Instantly Transcribes Court Proceedings” The Straits Times (14 December 2017). 8 Yan Jie, “China’s Courts Look to AI for Smarter Judgments” Sixth Tone (18 November 2016) <http://www.sixthtone.com/news/1584/china%20s-courts-look-ai-smarter- judgments> (accessed 20 April 2020). © 2021 Contributor(s) and Singapore Academy of Law. No part of this document may be reproduced without permission from the copyright holders. 282 Singapore Academy of Law Journal (2021) 33 SAcLJ 3 Whether or not AI judicial decision-making comes to pass, the pace of AI-driven change in the legal sector suggests that it may be prudent for judiciaries to begin considering now what AI judicial decision-making may look like. There is already a need for judiciaries to begin grappling with the use of AI decision-making in related fields, such as the use of algorithms in administrative decision-making.9 Even if AI judicial decision-making fails to materialise, this exercise in thinking about what truly drives judicial decision-making may provide valuable lessons and thought leadership for a legal sector that increasingly augments its services with AI. And if AI judicial decision-making does indeed become reality, judiciaries will have drawer plans that they can rely on in taking charge of shaping the use of AI in the courts. It behoves them to ensure that courts continue to deliver the justice that society needs, even if not the justice that society sometimes may expect, and if AI were one day to make judicial decisions, judges must be there every step of the way to make sure that justice is not just done but is seen to be done. 4 This article is split into three parts. The first part deals with the current state of AI as applicable to the legal sector and developments in the field of AI. As is the case with every dynamic field, it will be quite difficult to be comprehensive, and this article will not purport to be so. Nonetheless, a taster in the form of general principles will suffice for present purposes. The second part breaks down what judicial decision-making entails, as far as is possible for laypersons to do so, in the hope of distilling the essence of judicial decision-making that any AI judicial decision-making solution must be able to replicate. The third part draws upon the analyses of the former two parts to consider what implementations of AI will satisfy the requirements of judicial decision-making, and therefore what is required to put together such an implementation of AI. It also considers if any aspects of judicial decision-making could change in response to the use of AI in judicial decision-making. 5 A key caveat is that the observations in the rest of this article on the usability of AI in judicial decision-making below hold up only for current implementations of AI as weak AI.10 Paradigm changes in the field may invalidate these observations. For example, quantum-11 9 See, eg, Ashley Deeks, “The Judicial Demand for XAI” (2019) 119 Colum Law Rev 1829 at 1838–1840; and Monika Zalnieriute, Lyria Bennett Moses & George Williams, “The Rule of Law and Automation of Government Decision-Making” (2019) 82(3) Mod L Rev 425 at 427. 10 John Searle, “Minds, Brains, and Programs” (1980) 3(3) Behavioral and Brain Sciences 417 at 2 <http://cogprints.org/7150/1/10.1.1.83.5248.pdf> (accessed 23 July 2020). 11 See, eg, Nick Easen, “When Quantum Computing and AI Collide” Raconteur (28 April 2020) <https://www.raconteur.net/technology/quantum-computing-ai> (accessed 23 July 2020). © 2021 Contributor(s) and Singapore Academy of Law. No part of this document may be reproduced without permission from the copyright holders. Judicial Decision-making and (2021) 33 SAcLJ Explainable Artificial Intelligence 283 and photon-based12 intelligence technologies both have the potential to significantly increase the efficiency of AI algorithms. Without practical data as to the effects of these advances in technology, it will profit us little to speculate on possible changes; being aware that such a possibility exists suffices at this point. II. The orkingsw of AI A. Overview 6 AI generally refers to the phenomenon of computers “learning” to perform a task better, and hence simulating the human capacity to learn and process information. The foundations of AI research lay in the Dartmouth Summer