Algorithmic Equity A Framework for Social Applications Osonde A. Osoba, Benjamin Boudreaux, Jessica Saunders, J. Luke Irwin, Pam A. Mueller, Samantha Cherney C O R P O R A T I O N For more information on this publication, visit www.rand.org/t/RR2708 Library of Congress Cataloging-in-Publication Data is available for this publication. ISBN: 978-1-9774-0313-1 Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2019 RAND Corporation R® is a registered trademark. Cover: Data image/monsitj/iStock by Getty Images; Scales of Justice/DNY59/Getty Images. Limited Print and Electronic Distribution Rights This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited. Permission is given to duplicate this document for personal use only, as long as it is unaltered and complete. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial use. For information on reprint and linking permissions, please visit www.rand.org/pubs/permissions. The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. Support RAND Make a tax-deductible charitable contribution at www.rand.org/giving/contribute www.rand.org Preface This report is an examination of potential decisionmaking pathologies in social institu- tions that increasingly leverage algorithms. We frame social institutions as operating as collections of decision pipelines in which algorithms can increasingly feature either as decision aids or decisionmakers. Then we examine how different perspectives on equity or fairness inform the use of algorithms in the context of three sample institu- tions: auto insurance, recruitment, and criminal justice. We then highlight challenges and recommendations to improve equity in the use of algorithms for institutional decisionmaking. This report is funded by RAND internal funds. The goal is to inform decision- makers and regulators grappling with equity considerations in the deployment of algo- rithmic decision aids, particularly in public institutions. We hope our report can also serve to inform private sector decisionmakers and the lay public. Community Health and Environmental Policy Program RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improve the health and social and economic well-being of populations and communities throughout the world. This research was conducted in the Commu- nity Health and Environmental Policy Program within RAND Social and Economic Well-Being. The program focuses on such topics as infrastructure, science and tech- nology, community design, community health promotion, migration and population dynamics, transportation, energy, and climate and the environment, as well as other policy concerns that are influenced by the natural and built environment, technology, and community organizations and institutions that affect well-being. For more infor- mation, email [email protected]. RAND Ventures The RAND Corporation is a research organization that develops solutions to public policy challenges to help make communities throughout the world safer and more iii iv Algorithmic Equity: A Framework for Social Applications secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and commit- ted to the public interest. RAND Ventures is a vehicle for investing in policy solutions. Philanthropic con- tributions support our ability to take the long view, tackle tough and often-controversial topics, and share our findings in innovative and compelling ways. RAND’s research findings and recommendations are based on data and evidence, and therefore do not necessarily reflect the policy preferences or interests of its clients, donors, or supporters. Funding for this venture was provided by gifts from RAND supporters and income from operations. Contents Preface ................................................................................................. iii Figures and Tables ...................................................................................vii Summary .............................................................................................. ix Abbreviations ......................................................................................... xi CHAPTER ONE Introduction ........................................................................................... 1 The Rise of Algorithmic Decisionmaking ........................................................... 2 Identifying and Scoping the Research Questions .................................................. 4 Approach and Methods ............................................................................... 4 Organization of This Report .......................................................................... 5 CHAPTER TWO Concepts of Equity ................................................................................... 7 Perspectives on Equity ................................................................................. 8 Equity in Practice: Impossibilities and Trade-Offs................................................15 Competing Concepts of Equity: The Case of COMPAS ........................................17 Metaethical Aspects of Algorithmic Equity .......................................................21 CHAPTER THREE Domain Exploration: Algorithms in Auto Insurance ........................................ 23 The Auto Insurance Decision Pipeline..............................................................25 Relevant Concepts of Equity in Auto Insurance ................................................. 28 Equity Challenges .................................................................................... 30 Closing Thoughts .....................................................................................31 CHAPTER FOUR Domain Exploration: Algorithms in Employee Recruitment ...............................33 Relevant Concepts of Equity in Employment .................................................... 34 Benefits of Recruitment Algorithms ............................................................... 36 Employment Algorithm Decision Pipeline........................................................ 36 v vi Algorithmic Equity: A Framework for Social Applications Equity Challenges in Recruitment Algorithms ....................................................39 Closing Thoughts .................................................................................... 42 CHAPTER FIVE Domain Exploration: Algorithms in U.S. Criminal Justice Systems ......................45 The Criminal Justice Decision Pipeline ........................................................... 46 Relevant Concepts of Equity in Criminal Justice ................................................ 48 Equity Challenges .....................................................................................49 Closing Thoughts .....................................................................................51 CHAPTER SIX Analytic Case Study: Auditing Algorithm Use in North Carolina’s Criminal Justice System ...................................................................................53 Background ............................................................................................53 Assessment of Algorithmic Bias .................................................................... 56 Summary of Audit Analysis ..........................................................................59 CHAPTER SEVEN Insights and Recommendations ...................................................................61 Insights from Domain Explorations ................................................................61 Recommendations and Useful Frameworks .......................................................65 Closing Remarks ......................................................................................73 APPENDIX A. Supplemental Tables .............................................................................75 References ............................................................................................ 77 Figures and Tables Figures 3.1. State Auto Insurance Regulatory Regimes ........................................... 26 3.2. Notional Rendition of the Auto Insurance Decision Pipeline ..................... 27 4.1. Notional Rendition of Decision Pipeline for Employment and Recruiting ......37 5.1. Notional Rendition of Decision Pipeline for the U.S. Criminal Justice System ............................................................................45 Tables 2.1. Summary of the Simpler Statistical Concepts of Equity ............................12 6.1. Relationship Between Supervision Levels and Risk/Needs Levels .................55 6.2. Supervision Level Minimum Contact Standards ....................................55 6.3. Distribution of Probation Assessment Results by Race .............................57 6.4. One-Year Violation Rates by Race and by Supervision Level ......................57 6.5. Summary of Multivariate Logistic Regression Results—Odds Ratio .............58 7.1. Notional Basic