Algorithms in the Criminal Justice System | 1
Total Page:16
File Type:pdf, Size:1020Kb
Algorithms in the Criminal Justice System | 1 Commission creator and director: Sophia Adams Bhatti Commissioners: Prof Sylvie Delacroix Prof Sofia Olhede Christina Blacklaws Guest Commissioners: Andrea Coomber, JUSTICE Sir William Blair, Queen Mary University of London Madhumita Murgia, Financial Times Lead Author: Dr Michael Veale Project Team: Olivier Roth – Project Lead Rebecca Veitch William McSweeney Catherine O’Gallachoir Bal Matharu 2019, The Law Society of England and Wales Algorithms in the Criminal Justice System | 2 Table of Contents Table of Contents ......................................................................................................... 2 Executive Summary and Recommendations ............................................................. 4 2.1 List of Recommendations of the Commission ........................................................ 5 Scope of the Commission ........................................................................................... 9 Defining Algorithms ....................................................................................................10 Introduction .................................................................................................................12 5.1 Algorithms in Criminal Justice ...............................................................................12 5.2 Drivers ..................................................................................................................13 5.2.1 Resourcing pressures .......................................................................................13 5.2.2 Desires for increased proactivity .......................................................................13 5.2.3 Data and infrastructure availability ....................................................................14 5.2.4 New crime challenges .......................................................................................14 5.2.5 Access to justice ...............................................................................................14 5.2.6 Belief in the efficacy of computational prediction ...............................................15 5.3 Benefits .................................................................................................................15 5.3.1 Efficiency from automation ................................................................................15 5.3.2 Efficacy from augmentation ...............................................................................16 5.3.3 Auditability.........................................................................................................16 5.3.4 Consistency and control ....................................................................................16 5.3.5 Monitoring performance ....................................................................................17 5.4 Dangers ................................................................................................................17 5.4.1 Instrumental concerns .......................................................................................17 5.4.2 Dignitary concerns ............................................................................................19 5.4.3 Justificatory concerns ........................................................................................21 5.4.4 Systemic concerns ............................................................................................22 Data protection and algorithms in criminal justice ...................................................25 6.1 Automated decision making for law enforcement purposes ...................................27 6.1.1 Risk of rubber-stamping ....................................................................................27 6.1.2 Omission of discrimination provisions in DPA 2018 Part 3 ................................28 6.1.3 Limited protection of groups ..............................................................................30 6.1.4 Omissions of transparency provisions in DPA 2018 Part 3 ................................30 6.2 Unclear provisions around public-private partnerships ..........................................31 6.3 Data Protection Impact Assessments ....................................................................32 6.4 Further issues .......................................................................................................32 Algorithms in Criminal Justice Today .......................................................................33 7.1 Predictive Hotspot Policing ...................................................................................33 7.1.1 Concerns ..........................................................................................................35 7.2 Facial Recognition in Policing ...............................................................................36 7.2.1 Concerns and legislation around data and privacy ............................................39 Algorithms in the Criminal Justice System | 3 7.2.2 Concerns and legislation around bias................................................................42 7.2.3 Concerns and legislation around accountability, transparency and oversight ....43 7.3 Individual Risk Assessment...................................................................................45 7.3.1 Scoring at arrest ................................................................................................45 7.3.2 Scoring suspects ...............................................................................................47 7.3.3 Scoring victims ..................................................................................................47 7.3.4 Scoring staff ......................................................................................................48 7.3.5 Scoring in the prison system .............................................................................48 7.3.6 Concerns and legislative framework ..................................................................53 7.4 Digital Forensics ...................................................................................................55 7.5 Modernisation of the Courts ..................................................................................57 7.6 Mobile Device Extraction .......................................................................................58 Cross-cutting Rights and Issues ................................................................................61 8.1 Lawfulness ............................................................................................................61 8.2 Compliance Capacity ............................................................................................62 8.3 Transparency, Explanation and Justification .........................................................64 8.4 Discrimination and Computational Power ..............................................................67 8.5 Concluding remarks ..............................................................................................70 Annexes .......................................................................................................................72 9.1 Engaged Stakeholders ..........................................................................................72 9.2 Lines of Enquiry ....................................................................................................73 References ..........................................................................................................................74 Algorithms in the Criminal Justice System | 4 Executive Summary and Recommendations The Law Society established the Technology and the Law Policy Commission to examine the use of algorithms in the justice system of England and Wales. The Commission considered both simpler ‘hand-crafted’ systems and more complex, computationally generated ones such as machine learning. It held four public evidentiary sessions, interviewed over 75 experts, and read over 82 submissions of evidence and many more supplementary studies, reports and documents on the topic. This report contains findings and recommendations concerning the use of algorithmic systems in the criminal justice system. The Commission considered a range of currently deployed systems that fell within this brief, including individual risk assessment and recidivism prediction; prospective crime mapping and hot-spotting; and mobile phone data extraction tools. At the most basic level, the Commission has found a lack of explicit standards, best practice, and openness or transparency about the use of algorithmic systems in criminal justice across England and Wales. This was concerning, as the high-stakes decisions and measures taken in the justice system demand extremely careful deployment. There are significant challenges of bias and discrimination, opacity and due process, consistency, amenability to scrutiny, effectiveness, disregard of qualitative and contextual factors, against a backdrop of the potential of these systems to more deeply change the nature of the evolution of the law. The Commission recommends that a National Register of Algorithmic Systems should be created as a crucial initial scaffold for further openness, cross-sector learning and scrutiny. While many deployments are in a pilot or experimental stage, the Commission notes that the technologies being deployed are not so technically novel that they cannot be