University of North Carolina School of Law Carolina Law Scholarship Repository AI-DR Collection AI-DR Program Spring 3-24-2021 The Auditing Imperative for Automated Hiring Ifeoma Ajunwa Follow this and additional works at: https://scholarship.law.unc.edu/aidr_collection Part of the Labor and Employment Law Commons DRAFT IN PROGRESS. Please do not circulate. THE AUDITING IMPERATIVE FOR AUTOMATED HIRING SYSTEMS IFEOMA AJUNWA* ABSTRACT The high bar of proof to demonstrate either a disparate treatment or disparate impact cause of action under Title VII of the Civil Rights Act, coupled with the “black box” nature of many automated hiring systems, renders the detection and redress of bias in such algorithmic systems difficult. This Article, with contributions at the intersection of administrative law, employment & labor law, and law & technology, makes the central claim that the automation of hiring both facilitates and obfuscates employment discrimination. That phenomenon and the deployment of intellectual property law as a shield against the scrutiny of automated systems combine to form an insurmountable obstacle for disparate impact claimants. To ensure against the identified “bias in, bias out” phenomenon associated with automated decision-making, I argue that the employer’s affirmative duty of care as posited by other legal scholars creates “an auditing imperative” for algorithmic hiring systems. This auditing imperative mandates both internal and external audits of automated hiring systems, as well as record-keeping initiatives for job applications. Such audit requirements have precedent in other areas of law, as they are not dissimilar to the Occupational Safety and Health Administration (OSHA) audits in labor law or the Sarbanes-Oxley Act audit requirements in securities law. I also propose that employers that have subjected their automated hiring platforms to external audits could receive a certification mark, “the Fair Automated Hiring Mark,” which would serve to positively distinguish them in the labor market. Labor law mechanisms such as collective bargaining could be an effective approach to combating the bias in automated hiring by establishing criteria for the data deployed in automated employment decision-making and creating standards for the protection and portability of said data. The Article concludes by noting that automated hiring, which captures a vast array of applicant data, merits greater legal oversight given the potential for “algorithmic blackballing,” a phenomenon that could continue to thwart many applicants’ future job bids. * Associate Professor of Law (with tenure), University of North Carolina School of Law; Founding Director, AI-DR program, Faculty Associate, Berkman Klein Center at Harvard Law School. Many thanks to Alvaro Bedoya, Ryan Calo, Deven Desai, Zachary Clopton, Ignacio Cofone, Maggie Gardner, Jeffrey Hirsch, Katherine Strandburg, Elana Zeide and participants of the University of Chicago Public and Legal Theory Colloquium, the Cornell Law School Colloquium, the Yale Information Society, the Yale Center for Private Law Seminar, and the Privacy Law Scholars Conference (PLSC) for helpful comments. A special thanks to my research assistants, Kayleigh Yerdon, Eric Liberatore, and Ebun Sotubo (especially for help compiling the tables). A special thanks to the National Science Foundation for generous funding for my research on automated hiring. Finally, I am grateful to the Harvard JOLT editors for their rigorous edits and fastidious cite-checking. Electronic copy available at: https://ssrn.com/abstract=3437631 2 The Auditing Imperative for Automated Hiring Systems - Active Draft TABLE OF CONTENTS INTRODUCTION 3 I. 11 A. 11 B. 14 C. 15 II. 21 A. 21 B. 22 C. 24 III. 25 A. 25 B. 28 C. 30 IV. 36 A. 37 1. Tear-off Sheets: What Information is needed for verification? 39 2. Enhancing Applicant Selection: What standards should apply? 40 B. 42 1. The Pros and Cons of a Governmental Certifying System 44 2. The Pros and Cons of a Third-Party Non- governmental Certifying System 45 C. 49 1. 50 2. 53 3. 55 4. Preventing “Algorithmic Blackballing” 54 D. 57 CONCLUSION 56 Electronic copy available at: https://ssrn.com/abstract=3437631 3 The Auditing Imperative for Automated Hiring Systems - Active Draft 1. INTRODUCTION Imagine this scenario: A woman seeking a retail job is informed that the job can only be applied for online. The position is a sales clerk for a retail company with store hours from 9:00 AM to 9:00 PM. She is interested in the morning and afternoon hours, as she has children who are in school until 3:00 PM. When completing the application, she reaches a screen where she is prompted to register her hours of availability. She enters 9:00 AM to 3:00 PM, Monday through Friday. However, when she hits the button to advance to the next screen, she receives an error message indicating that she has not completed the current section. She refreshes her screen, she re-starts her computer, and still the same error message remains. Finally, in frustration, she abandons the application. Compare the above to this second scenario: A fifty- three-year-old man is applying for a job that requires a college degree. But when he attempts to complete the application online, he finds that the drop- down menu offers only college graduation dates that go back to the year 2000. The automated hiring platform will, in effect, exclude many applicants who are older than forty years old. If the man also chooses to forgo the application like the woman in the previous scenario, the automated hiring system may not retain any record of the two failed attempts to complete the job application.1 The vignettes above reflect the real-life experiences of job applicants who must now contend with automated hiring systems in their bid for employment. 2 These stories illustrate the potential for automated hiring systems to discreetly and disproportionately cull the applications of job seekers who are from legally protected classes. 3 Given that legal scholars have identified a “bias in, bias out” problem for automated decision-making, 4 automated hiring as a socio-technical trend challenges the American bedrock ideal of equal opportunity in employment,5 as such automated practices may not only be deployed to exclude certain categories of workers but may also be 1 See generally CATHY O’NEIL, WEAPONS OF MATH DESTRUCTION: HOW BIG DATA INCREASES INEQUALITY AND THREATENS DEMOCRACY (2016). 2 Patricia G. Barnes, Behind the Scenes, Discrimination by Job Search Engines, AGE DISCRIMINATION EMP. (Mar. 29, 2017), https://www.agediscriminationinemployment.com/tag/illinois-attorney-general-lisa- madigan/ [https://perma.cc/6H7Z-WSDD]; Ifeoma Ajunwa & Daniel Greene, Platforms at Work: Data Intermediaries in the Organization of the Workplace, in WORK AND LABOR IN THE DIGITAL AGE (2019) (discussing the encountered difficulty of completing an online application when applying with constrained hours of availability). 3 Title VII of the Civil Rights Act of 1964 guarantees equal opportunity in employment irrespective of race, gender, and other protected characteristics. 42 U.S.C. §§ 2000e to 2000e- 17 (2000). 4 See Sandra G. Mayson, Bias In, Bias Out, 128 YALE L.J. 2218, 2224 (2019) (arguing that the problem of disparate impact in predictive risk algorithms lies not in the algorithmic system but in the nature of prediction itself); Sonia Katyal, Private Accountability in the Age of Artificial Intelligence, 66 UCLA L. REV. 54, 58 (2019) (noting the bias that exists within AI systems and arguing for private mechanisms to govern AI systems); Andrew Tutt, An FDA for Algorithms, 69 ADMIN. L. REV. 83, 87 (2017) (“This new family of algorithms hold enormous promise, but also poses new and unusual dangers.”). 5 Ajunwa & Greene, supra note 2; see also Pauline Kim, Data-Driven Discrimination at Work, 58 WM. & MARY L. REV. 857, 860 (2017) [hereinafter Data-Driven Discrimination at Work]. Electronic copy available at: https://ssrn.com/abstract=3437631 4 The Auditing Imperative for Automated Hiring Systems - Active Draft used to justify the inclusion of other classes as more “fit” for the job.6 This is a cause for the legal concern that algorithms may be used to manipulate the labor market in ways that negate equal employment opportunity.7 This concern is further exacerbated given that nearly all Fortune 500 companies now use algorithmic recruitment and hiring tools.8 Algorithmic hiring has also saturated the low-wage retail market, with the top twenty Fortune 500 companies, which are mostly retail and commerce companies that boast large numbers of employees, almost exclusively hiring through online platforms.9 Although it is undeniable that there could be tangible economic benefits of adopting automated decision-making, 10 the received wisdom of the objectivity of automated decision-making, coupled with an unquestioning acceptance of the results of algorithmic decision-making,11 have allowed hiring systems to proliferate without adequate legal oversight. As Professor Margot Kaminski notes, addressing algorithmic decision-making concerns requires both individual and systemic approaches. 12 Currently, the algorithmic decisions made in the private sector are largely unregulated, and Kaminski argues for a collaborative approach to governance that could satisfy both individual and collective concerns: Collaborative governance is a middle ground, a third way, that aims to harness the benefits of self-regulation without its pitfalls. The government stays significantly involved as a backdrop threat to nudge private sector involvement, as a forum for convening and empowering conflicting voices, as an arbiter or certifier in the name of the public interest, and as a hammer that can come down to enforce compliance.13 Thus, the goal of this Article is neither to argue against or for the use of automated decision-making in employment, nor is it to examine whether automated hiring systems are better than humans at making hiring decisions. For antidiscrimination law, the efficacy of any particular hiring system is a secondary concern to ensuring that any such system does not unlawfully 6 See Ifeoma Ajunwa, The Paradox of Automation as Anti-Bias Intervention, 41 CARDOZO L.