Social Norm Bias: Residual Harms of Fairness-Aware Algorithms Myra Cheng Maria De-Arteaga
[email protected] University of Texas at Austin California Institute of Technology, Microsoft Research Austin, Texas, USA Pasadena, California, USA Lester Mackey Adam Tauman Kalai Microsoft Research Microsoft Research Cambridge, Massachusetts, USA Cambridge, Massachusetts, USA ABSTRACT In this work, we characterize Social Norm Bias (SNoB)—the Many modern learning algorithms mitigate bias by enforcing fair- associations between an algorithm’s predictions and individuals’ ness across coarsely-defined groups related to a sensitive attribute adherence to social norms—as a source of algorithmic unfairness. like gender or race. However, the same algorithms seldom account We study SNoB through the task of occupation classification on a for the within-group biases that arise due to the heterogeneity of dataset of online biographies. In this setting, masculine/feminine group members. In this work, we characterize Social Norm Bias SNoB occurs when an algorithm favors biographies written in ways (SNoB), a subtle but consequential type of discrimination that may that adhere to masculine/feminine gender norms, respectively. We be exhibited by automated decision-making systems, even when examine how existing algorithmic fairness techniques neglect this these systems achieve group fairness objectives. We study this issue issue. through the lens of gender bias in occupation classification from Discrimination based on gender norms has implications of con- biographies. We quantify SNoB by measuring how an algorithm’s crete harms, which are documented in the social science literature predictions are associated with conformity to gender norms, which (Section 3.2). For example, in the labor context, women in the work- is measured using a machine learning approach.