Towards a Critical Race Methodology in Algorithmic Fairness Alex Hanna∗ Emily Denton∗ Andrew Smart Jamila Smith-Loud {alexhanna,dentone,andrewsmart,jsmithloud}@google.com ABSTRACT The problemdoesnotend withthe collectionofracial We examine the way race and racial categories are adopted in al- data; it only begins. The problem accelerates when gorithmic fairness frameworks. Current methodologies fail to ade- we attempt to analyze these data statistically... The quately account for the socially constructed nature of race, instead racialization of data is an artifact of both the strug- adopting a conceptualization of race as a fixed attribute. Treating gles to preserve and to destroy racial stratification. race as an attribute, rather than a structural, institutional, and re- Before the data can be deracialized, we must deracial- lational phenomenon, can serve to minimize the structural aspects ize the social circumstances that have created racial of algorithmic unfairness. In this work, we focus on the history stratification. of racial categories and turn to critical race theory and sociolog- – Tufuku Zuberi [125, pp. 102] ical work on race and ethnicity to ground conceptualizations of race for fairness research, drawing on lessons from public health, 1 INTRODUCTION biomedical research, and social survey research. We argue that al- In recent years, there has been increasing recognition of the po- gorithmic fairness researchers need to take into account the mul- tential for algorithmic systems to reproduce or amplify existing tidimensionality of race, take seriously the processes of conceptu- social inequities. In response, the research field of algorithmic fair- alizing and operationalizing race, focus on social processes which ness has emerged. This rapidly evolving research area is focused produce racial inequality, and consider perspectives of those most on developing tools and techniques with aspirations to make devel- affected by sociotechnical systems. opment, use, and resulting impact of algorithmic systems conform to various social and legal notions of fairness. The concept of fair- CCS CONCEPTS ness, in addition to being situational, evolving, and contested from • Applied computing → Sociology; • Social and professional a number of philosophical and legal traditions, can only be under- topics → Race and ethnicity. stood in reference to the different social groups that constitute the organization of society. Consequently, the vast majority of algo- rithmic fairness frameworks are specified with reference to these KEYWORDS social groups, often requiring a formal encoding of the groups into algorithmic fairness, critical race theory, race and ethnicity the dataset and/or algorithm. However, most social groups relevant to fairness analysis re- ACM Reference Format: flect highly contextual and unstable social constructs. These social Alex Hanna, Emily Denton, Andrew Smart, and Jamila Smith-Loud. 2020. groups are often defined with recourse to legal anti-discrimination Towards a Critical Race Methodology in Algorithmic Fairness. In Confer- concepts such as "protected classes,"which, in the US, refers to race, ence on Fairness, Accountability, and Transparency (FAT* ’20), January 27–30, color, national origin, religion, sex, age, or disability. However, the 2020, Barcelona, Spain. ACM, New York, NY, USA, 12 pages. process of drawing boundaries around distinct social groups for arXiv:1912.03593v1 [cs.CY] 8 Dec 2019 https://doi.org/10.1145/3351095.3372826 fairness research is fraught; the construction of categories has a long history of political struggle and legal argumentation. Numerous recent works have highlighted the limitations of cur- rent algorithmic fairness frameworks [53, 108]. Several of these ∗Both authors contributed equally to this research. critiques point to the tendency to abstract away critical social and historical contexts and minimize the structural conditions that un- derpin problems of algorithmic unfairness. We build on this crit- ical research, focusing specifically on the use of race and racial Permission to make digital or hard copies of part or all of this work for personal or categories within this field. IN this literature, the topic of the insta- classroom use is granted without fee provided that copies are not made or distributed bility of racial categories has gone relative unexplored, with the for profit or commercial advantage and that copies bear this notice and the full citation notable exception of Benthall and Haynes [12], which we discuss on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). in detail below. FAT* ’20, January 27–30, 2020, Barcelona, Spain Race is a major axis around which algorithmic allocation of re- © 2020 Copyright held by the owner/author(s). sources and representation is bound. It may indeed be the most sig- ACM ISBN 978-1-4503-6936-7/20/02. nificant axis, given attention by investigative journalists (e.g. [7]) https://doi.org/10.1145/3351095.3372826 and critical race and technology scholars (e.g. [11, 20, 21, 24, 85]). FAT* ’20, January 27–30, 2020, Barcelona, Spain Hanna et al. Because of this, it is imperative that the race-based methodolo- the company which developed COMPAS – defended the system gies and racial categories themselves are interrogated and critically by reanalyzing the ProPublica data [28] and ProPublica later re- evaluated. sponded [6]. The debate has become a cornerstone of algorithmic In this paper we develop several lines of critique directed at the fairness research; to date, the original story has some 700 citations treatment of race within algorithmic fairness methodologies. In on Google Scholar. short, we observe that current methodologies fail to adequately ac- The Propublica analysis of COMPAS, as well as the responses count for the socially constructed nature of race, instead adopting a from Northpointe and other secondary analyses performed by third- conceptualization of race as a fixed attribute. This manifests in the party researchers, relied on data from the Broward County Sher- widespread use of racial categories as if they represent natural and iff’s Office which was obtained through a public records request. objective differences between groups. Treating race as an attribute, Race classifications in this data identified defendants as Black, White, rather than a structural, institutional, and relational phenomenon, Hispanic, Asian, Native American or Other. These categories look in turn serves to minimize the structural aspects of algorithmic familiar – they are nearly identical to the way that the US Census unfairness. defines race. However, there are the notable absences of the cate- The process of operationalizing race is fundamentally a project gories "Native Hawaiian or Other Pacific Islander", and there is a of racial classification and thus must be understood as a political redefinition of "Hispanic" as a race rather than an ethnicity. project, or, more specifically, what Omi and Winant [86] refer to as In the methodological appendix to the original article [69], Jeff a racial project. The tools for measuring individual characteristics, Larson and other ProPublica authors do not delve into how the race such as national censuses and other population-level survey instru- of the defendant is measured. Larson admits that he did not know ments, were and are still often based in the politics of racial op- how Broward County classified individuals 1. It’s not clear why pression and domination. While we acknowledge the importance Broward County used the modified Census racial schema, but, as of measuring race for the purposes of understanding patterns of detailed below, it may have to do with a 1977 directive from the fed- differential performance or differentially adverse impact of algo- eral US Office of Management and Budget. In general, approaches rithmic systems, in this work, we emphasize that data collection to measuring race for police records are inconsistent across mu- and annotation efforts must be grounded in the social and histor- nicipalities. They can rely on self-identification, state records, or ical contexts of racial classification and racial category formation. observation by criminal justice workers. In her work on racial dis- To oversimplify is to do violence, or even more, to re-inscribe vio- parities in incarceration in Wisconsin, sociologist Pamela Oliver lence on communities that already experience structural violence. notes that the race of a single individual can change across records It is useful to distinguish between two ways in which race comes even within a single jurisdiction 2. Therefore, even in the most fa- into play in algorithmic fairness research: (i) the operationalization mous debate of the field, we don’t know why the data take on a of race, i.e. the process of converting the abstract concept of race particular racial schema, nor do we have information about how into something that is concrete and measurable and (ii) the use of defendants are racially categorized. racial variables within algorithmic frameworks. These aspects are tightly interconnected since race must be operationalized before 2.1 Using race-like categories? racial variables can be utilized. While an area of major concern for critical race and technology Our contributions are as follows: we review the use of race in scholarship, the use of racial categories in algorithmic fairness re- prior fairness work and discuss an intervention in the debate on search (i.e. the research