Conversations on Race, Risk Assessment Tools, and Pretrial
Total Page:16
File Type:pdf, Size:1020Kb
_________________________________ What Does Fairness Look Like? Conversations on Race, Risk Assessment Tools, and Pretrial Justice _________________________________ October 2018 In November 2017, our organizations hosted a two-day convening to discuss racial equity, algorithmic prediction, and pretrial justice reform. We sought to create space for a robust inter- disciplinary conversation, as well as to further explore the debate around various definitions of algorithmic “fairness,” particularly from a racial justice lens. We also hoped to further enhance our understanding of these issues by learning from participants—who brought their relative expertise in mathematical prediction, civil rights, the administration of pretrial systems, and the criminal legal system—and, in so doing, to share that discussion with partners in a candid environment. Over the course of the day, a few points of agreement—particularly around desired outcomes—were identified. In particular, participants seemed largely to agree on the need for community oversight and accountability wherever pretrial risk assessment (“PRA”) algorithms are used. Additionally, there was broad consensus around the need for any pretrial risk assessment instrument (“PRAI”) introduced to serve as a tool to promote release and decarceration. Finally, given the framing of the convening, participants agreed that racial bias and inequity are critically important considerations that must be countered by pretrial justice systems. The effort to reach broad consensus around these concerns underscored the need for further exploration of several questions that must be taken up by the collection of advocates, stakeholders, and experts engaged in the design and implementation of pretrial risk assessment instruments. Among those questions are: what “risks” should be measured, and how, in a pretrial release decision; how should the design of pretrial risk assessing instruments respond to realities of racial disparity in our criminal legal system; what role—if any—should these instruments play in determining an individual’s pretrial liberty; and what is the necessity of risk assessment tools to bail and pretrial reform? All told, the convening provided a space for dynamic conversation, information-sharing, and an opportunity to share varying perspectives on bail reform and risk assessments. We believe it marks an important beginning, and that further collaboration would be fruitful to continued progress. Vincent Southerland Andrea Woods Center on Race, Inequality, and the Law at NYU Law ACLU Table of Contents Executive Summary ................................................................................................................................................................ 3 Overview of the Discussion ................................................................................................................................................. 4 Ground Rules ........................................................................................................................................................................ 4 Realities of Pretrial Incarceration ................................................................................................................................ 4 Shared Definitions .............................................................................................................................................................. 5 Actuarial ............................................................................................................................................................................ 5 Algorithm .......................................................................................................................................................................... 5 Data ...................................................................................................................................................................................... 5 Institutional Racism ...................................................................................................................................................... 5 1 Structural Racism ........................................................................................................................................................... 5 Validation .......................................................................................................................................................................... 6 Terms We Did Not Define ........................................................................................................................................... 6 Conversation One: Defining Risk and the Current Legal Landscape .................................................................. 6 Conversation Two: The History and Future of Pretrial Risk Assessments ...................................................... 8 Conversation Three: What Does Fairness Look Like? ............................................................................................ 11 The Technical Side of Risk Assessments ............................................................................................................ 11 Fairness ............................................................................................................................................................................ 11 From the Technical to the Practical ..................................................................................................................... 12 Conversation Four: Applying Knowledge, Looking Forward .............................................................................. 14 Questions and Conclusions Drawn from Discussion .............................................................................................. 15 Are the Tools Necessary for System Change? .................................................................................................. 15 What “Risks” Matter? ................................................................................................................................................. 16 How to Address Racial Disparities in the Pretrial System .......................................................................... 17 Should We Endeavor to Redesign these Tools? .............................................................................................. 17 Takeaways: Points of Consensus ................................................................................................................................ 18 Goal of Decarceration ................................................................................................................................................. 18 Goal of Reducing Racial Disparities ...................................................................................................................... 18 Need for Community Oversight, Goal of Impacted Persons ....................................................................... 19 Need for PRAs to be Better Understood by Stakeholders ........................................................................... 20 Takeaways: Next Steps Needed .................................................................................................................................. 20 Appendix A—Overview of the Event: Format, Materials, Attendees ............................................................... 21 Agenda ...................................................................................................................................................................................21 Thursday, November 16 ........................................................................................................................................... 21 Friday, November 17 .................................................................................................................................................. 21 Materials ...............................................................................................................................................................................22 Attendees ............................................................................................................................................................................. 23 Appendix B—Materials of Interest Generated Since the Convening ........................................................... 25 Acknowledgements ...............................................................................................................................................................26 .... 2 Executive Summary The primary goals of the convening were (1) to gather leading experts on mathematical prediction, algorithmic discrimination, pretrial policy, and the criminal legal system to seek new insights into the issues surrounding algorithmic pretrial risk assessment and race, and (2) to wrestle with—and, ideally, come away with coherent frameworks about—the various ways algorithms can and cannot account for the racial disparities throughout our criminal legal system. In inviting participants to come together for this discussion, we set forth a handful of additional, “key goals.” We hoped to (1) identify areas of agreement and disagreement with respect to the permissible boundaries of preventive pretrial detention, (2) arrive at a shared and explicit vocabulary regarding the potential definitions of algorithmic