AI Sentencing in Sabah and Sarawak
Total Page:16
File Type:pdf, Size:1020Kb
Artificial Intelligence in the Courts: AI sentencing in Sabah and Sarawak VIEWS 40/20 | 18 August 2020 | Claire Lim and Rachel Gong Views are short opinion pieces by the author(s) to encourage the exchange of ideas on current issues. They may not necessarily represent the official views of KRI. All errors remain the authors’ own. This view was prepared by Claire Lim and Rachel Gong, a researcher from the Khazanah Research Institute (KRI). The authors are grateful for valuable input and comments from Aidonna Jan Ayub, Ong Kar Jin, and representatives of the courts of Sabah and Sarawak and SAINS. Corresponding author’s email address: [email protected] Introduction Attribution – Please cite the work as The Covid-19 pandemic has accelerated the need for many follows: Lim, Claire and Rachel Gong. 2020. Artificial Intelligence in the Courts: industries to undertake digital transformation. Even the AI Sentencing in Sabah and Sarawak. traditionally conservative judicial system has embraced Kuala Lumpur: Khazanah Research this ‘new normal’, for example, by holding court trials Institute. License: Creative Commons Attribution CC BY 3.0. online1. However, adapting to technological change is not foreign to the Malaysian judiciary. Earlier this year, even Translations – If you create a translation before the pandemic forced industries to embrace digital of this work, please add the following disclaimer along with the attribution: This transformation, the Sabah and Sarawak courts launched a translation was not created by Khazanah pilot artificial intelligence (AI) tool2 as a guide to help judges Research Institute and should not be considered an official Khazanah with sentencing decisions. Research Institute translation. Khazanah Research Institute shall not be liable for AI refers to machines which are able to make decisions with any content or error in this translation. human-like intelligence and tackle tasks that are arduous to Information on Khazanah Research do manually. It is important to note the distinction between Institute publications and digital products predictive statistical analysis and machine learning-based can be found at www.KRInstitute.org. AI. Predictive statistical analysis uses historical data to find Cover photo by Bill Oxford on Unsplash. patterns in order to predict future outcomes; it requires 1 Khairah N. Karim (2020) 2 Wong (2020) KRI Views | Artificial Intelligence in the Courts: AI Sentencing in Sabah and Sarawak 1 human intervention to query, make assumptions and test the data3. Machine learning based AI is able to make assumptions, learn and test autonomously4. In recent years, although there has been much hype about the rise of AI transforming industries to improve efficiency and productivity, some of these systems actually fall within data analytics or predictive analytics rather than true machine learning based AI. The Sabah and Sarawak courts’ tool at present falls more within the category of predictive statistical analysis, but aims to move towards machine learning-based AI. The impetus behind this recent push to utilise AI in the court system is to achieve greater consistency in sentencing. The AI tool is currently being trialled on two offences: drug possession under Section 12(2) of the Dangerous Drug Act and rape under Section 376(1) of the Penal Code. The algorithm analyses data from cases of these two offences which are registered in Sabah and Sarawak between 2014 and 2019, identifies patterns which it will apply to the present case and produces a sentencing recommendation that judges can choose to adopt or deviate from. According to the courts, the reason behind choosing s12(2) of the Dangerous Drug Act and s376(1) of the Penal Code for the pilot is that the dataset for those two offences is the richest dataset that they have. As with any new technology, the development of AI’s tremendous potential has to be counterbalanced against certain risks. An analysis of cases which used AI sentencing tool as at 29 May 2020 shows that judges followed the recommendation in approximately 33% of cases5. This article discusses the risks of bias and lack of transparency and accountability that surround AI and considers mitigating measures to address these risks with reference to the Sabah and Sarawak courts’ AI sentencing tool. Bias Training data Machines are generally assumed to be objective. However, a major concern with AI is its potential to replicate and exaggerate bias. Experiments with AI technology such as Microsoft’s Tay6 and Google’s autocomplete suggestions7 which rely on human engagement for input data show that machines are not immune to society’s prejudices. Tay, a Twitter chatbot, was corrupted in hours by users who flooded it with misogynistic and racist posts and began putting out its own offensive posts. Google’s offensive autocomplete predictions were based on actual searches entered into its searchbox. A popular phrase to describe this phenomenon is ‘garbage-in, garbage out’ i.e. an AI system is only as good as the data that it is trained on. Amazon’s recruiting tool8 which was trained on historical recruitment data consistently downgraded female candidates, consequently perpetuating existing gender bias. In effect, “we 3 Wade et al. (2020) 4 Reavie (2018) 5 The authors gratefully ackowledge the case data statistics as provided by the courts of Sabah and Sarawak. Exact case numbers were not available for release at the time of writing. 6 Vincent (2016) 7 Lapowsky (2018) 8 Vincent (2018) KRI Views | Artificial Intelligence in the Courts: AI Sentencing in Sabah and Sarawak 2 can’t expect an AI algorithm that has been trained on data that comes from society to be better than society – unless we’ve explicitly designed it to be”9. Global efforts are ongoing to “cure automated systems of hidden biases and prejudices”10. In 2012, Project ImageNet played a key role in providing developers with a library of images to train computers to recognise visual concepts. Scientists from Stanford University, Princeton University, and the University of North Carolina paid digital workers a small fee to label more than 14 million images, creating a large dataset which they released to the public for free. While greatly advancing AI development, researchers later found problems in the dataset, for example, an algorithm trained on the dataset may identify a “programmer” as a white man because of the pool of images labeled in that way. The ImageNet team set about analysing the data to uncover these biases and took steps such as identifying words that projected a meaning on an image (e.g 'philanthropist’) and assessing the demographic and geographic diversity in the image set. The effort showed that algorithms can be re-engineered to be fairer. A separate study was conducted by ProPublica on Correctional Offender Management Profiling for Alternative Sanctions (COMPAS), a recidivism risk assessment algorithm used by the US courts to aid in sentencing. The study criticised COMPAS as being racially biased against African- Americans and argued that COMPAS was more likely to “falsely flag black defendants as future criminals, wrongly labeling them this way at almost twice the rate as white defendants” and that “white defendants were mislabeled as low risk more often than black defendants”. Propublica’s conclusions have since been rebutted by Northpointe, the makers of COMPAS, and various academics11 who noted that the program “correctly predicted recidivism in both white and black defendants at similar rates”12. Bias has also been demonstrated in facial recognition AI tools. Despite Amazon, IBM and Microsoft’s decisions to pause the sale of their facial recognition tools to US law enforcement13, a Black man was wrongly arrested14 for a crime he didn’t commit because facial recognition had identified him as the perpetrator. Conscious of this risk of bias, the Sabah and Sarawak courts and their software developer (Sarawak Information Systems Sdn Bhd (SAINS), a Sarawak state government-owned company) held stakeholder consultations during the development process to identify prominent concerns. For example, stakeholders were concerned that the ‘race’ variable might create bias in future sentencing decisions, so the courts made the decision to remove the variable from the algorithm as it was not a significant factor in the sentencing process. Such mitigating measures are valuable, but they do not make the system perfect. A dataset of 5 years of cases seems somewhat limited in comparison with the extensive databases used in global 9 Marr (2019) 10 Knight (2019) 11 Flores, Bechtel, and Lowenkamp (2016) 12 Yong (2018) 13 Heilweil (2020) 14 Allyn (2020) KRI Views | Artificial Intelligence in the Courts: AI Sentencing in Sabah and Sarawak 3 efforts such as Project ImageNet. Furthermore, it is unclear whether the removal of the ‘race’ variable from the algorithm has any significant effect on its recommendations. Software development AI training data is not the only place where bias can occur. It is very difficult to strip human bias from algorithms themselves15, partly because it still requires humans to develop them. Bias can creep in at any stage. One of the earliest instances is at the problem structuring stage16 where, when creating a deep learning model, computer scientists need to decide what they want the model to achieve, and the parameters set by the scientists may reflect their intentions or subconscious prejudices. To mitigate this, SAINS has worked collaboratively with the Sabah and Sarawak judiciary to test the results of the AI sentencing tool. During the development process, the judiciary analysed the recommendations produced by the AI tool and debated whether they would have reached the same conclusions. This helped the software developers, who have no legal training, understand the needs of the legal system and make changes to the AI algorithm accordingly. SAINS and the Sabah and Sarawak judiciary have emphasised that this learning, consultative and collaborative process is an ongoing one as they seek to make further improvements to the AI tool.