Reconsidering Annotator Disagreement about Racist Language: Noise or Signal? Savannah Larimore Ian Kennedy Washington University in St. Louis University of Washington Breon Haskett Alina Arseniev-Koehler University of Washington University of California - Los Angeles Abstract 2016). Indeed, social scientists have long ac- knowledged the difficulties of measuring racism, An abundance of methodological work aims even when using traditional social science meth- to detect hateful and racist language in text. However, these tools are hampered by prob- ods (e.g., interviews and surveys), due to social lems like low annotator agreement and remain desirability biases and the increasingly covert na- largely disconnected from theoretical work on ture of racism (Bonilla-Silva, 2006). race and racism in the social sciences. Using In this paper, we reconsider efforts to anno- annotations of 5188 tweets from 291 annota- tate for racism in light of sociological work on tors, we investigate how annotator perceptions race and racism. Instead of generalizing our de- of racism in tweets vary by annotator racial tection of racism, we narrow our scope to focus identity and two text features of the tweets: on anti-Black racism. In other words, we focus relevant keywords and latent topics identified through structural topic modeling. We provide on racialized language directed at or centering on 1 a descriptive summary of our data and estimate Black Americans. Using human annotations for a series of linear models to determine if anno- the racial sentiment (positive or negative) of 5188 tator racial identity and our 12 topics, alone or Tweets, we describe how ratings vary by annota- in combination, explain the way racial senti- tors’ own racial identity. Our findings suggest that ment was annotated, net of relevant annotator White raters respond differently to particular types characteristics and tweet features. Our results of racialized language on Twitter, as identified by show that White and non-White annotators ex- hibit significant differences in ratings when structural topic modeling (STM), than non-White reading tweets with high prevalence of certain raters. Failing to account for this systematic dif- racially-charged topics. We conclude by sug- ference could lead researchers to consider tweets gesting how future methodological work can which non-White annotators identify as including draw on our results and further incorporate so- negative sentiment as innocuous because more nu- cial science theory into analyses. merous White annotators rate those same tweets as positive or neutral. We conclude by suggesting 1 Introduction several ways in which future work can account for Hateful and racist language is abundant on so- the variability in annotations that comes from an- cial media platforms like Twitter and a growing notators’ own racial identities. body of work aims to develop tools to detect such 1.1 Annotating for Racism language in these spaces. Such a tool would of- fer opportunities to intervene, like providing au- Collecting annotations is a key step to develop- tomatic trigger warnings, and would provide a ing a tool to detect racial sentiment in text data. powerful barometer to measure racism. How- At the same time, the challenges of this task have ever, these efforts are hampered by low inter- been well-documented as ratings depend upon the rater agreement, low modeling performance, and ability of human annotators to consistently iden- a lack of consensus on what counts as racist lan- tify the racial sentiment of words and phrases (Zou guage (e.g., Kwok and Wang, 2013; Burnap and and Schiebinger, 2018). Empirical work often Williams, 2016; Waseem, 2016; Schmidt and Wie- finds large amounts of disagreement in these anno- gand, 2017). These efforts are also largely dis- tations, even with carefully designed annotations connected from rich understandings of race and 1We use the terms rating/annotation and rater/annotator racism in the social sciences (but, see Waseem, interchangeably throughout. schemes (e.g., Bartlett et al., 2014; Schmidt and color tended to rate these same memes as more of- Wiegand, 2017). fensive than White students. That is, while White As these efforts have shown, perceptions of students could identify that a meme was racist, racial sentiment are contextual and subjective, they rated the level of offensiveness lower than making the prevalence of racism in text sources students of color. In addition, Tynes and Markoe inherently difficult to detect. Recent social scien- (2010) find that European American college stu- tific work (Bonilla-Silva, 2006; Carter and Mur- dents were less likely than African American col- phy, 2015; Krysan and Moberg, 2016; Tynes and lege students to react negatively to racially-themed Markoe, 2010) has taken that difficulty as its sub- party images on social media. Furthermore, Euro- ject, and sought to capture and understand the pean American students reported higher levels of variation in perceptions of racism or racial senti- color-blind racial attitudes and students with lower ment, by showing how individuals’ perceptions of levels of color-blind attitudes were more likely to racism vary systematically based on the their own react as “bothered” by the images, implying that racial identity. These findings suggest that anno- both race and racial attitudes influence percep- tator disagreement is not merely noise to smooth tions of racism online. Similarly, Daniels(2009) over. Rather, annotator disagreement for racism finds that critical race literacy matters more than includes important variation that should be dis- internet literacy in identifying racially biased or aggregated and accounted for. “cloaked” websites (i.e., websites that appear to promote racial justice but are actually maintained 1.2 Varying Perceptions of Racism by White supremacist organizations). This find- Differential attitudes about and perceptions of ing suggests that students who lack a critical race racism based on an individuals’ own racial iden- consciousness may be less likely to identify racist tity are well-documented. White Americans tend materials online and that White students may be to hold more overtly racist beliefs, are less likely to particularly susceptible. believe racial discrimination is prevalent in mod- The subtlety of racism that pervades social me- ern society, and are less likely to recognize racial dia sites like Twitter may also influence percep- microaggressions than Black Americans (Bonilla- tions of racism. As Carter and Murphy(2015) Silva, 2006; Carter and Murphy, 2015; Krysan and note, Whites tend to focus only on blatant, overt Moberg, 2016; Tynes and Markoe, 2010). In ad- forms of racism (e.g., racial slurs, mentions of dition, Krysan and Moberg(2016) note that White racial violence) but are less attuned to microag- Americans increasingly disregard racial topics on gressions and other, subtler forms of racism. As questionnaires, signaling that they have “no in- such, scholars have also advocated for a method- terest” in issues of racial inequality. Likewise, ological move away from “bag of words” ap- fewer White Americans agree that racial inequal- proaches to the evaluation of racism on social ity is due to overt discrimination, arguing instead media (Watanabe et al., 2018) because these ap- that racial discrimination is a thing of the past and proaches reinforce a focus on blatant, overt forms that in contemporary society, everyone has equally of racism, and neglect more subtle, or contextually fair life chances (Bonilla-Silva, 2006). As Carter racist tweets (Chaudhry, 2015). and Murphy(2015) note, White and Black Amer- Similarly, Kwok and Wang(2013), noting the icans may differ in their views of racial inequality subtlety of racism pervading social media posts, because White Americans compare contemporary argue that to get evaluations of tweets that accu- racial inequalities to the past, referencing slavery rately assesses meaning, features of tweets other and Jim Crow and naturally concluding that condi- than the text must be included. Tweet features set tions have improved, while Black Americans com- the rules of engagement by offering markers of pare the present to an imagined future, in which an credibility, sarcasm, and persuasiveness (Sharma ideal state of racial equality has been achieved. and Brooker, 2016; Hamshaw et al., 2018). Tweet These differences also extend to how we per- features such as links, hashtags, and number of ceive racism in online platforms. Williams et al. comments have been shown to illuminate the con- (2016) find that while White students were equally text of the tweet’s message (Burnap and Williams, as likely as students of color to perceive racially- 2016). The inclusion of these features in evalua- themed internet memes as offensive, students of tion offers deeper context and more realistic eval- uation of tweets allowing for greater attention to Project (DOLLY; a repository all geotagged tweets the differential evaluations of people in racially since December 2011) (Morcos et al.). For both marginalized groups engaging with the social me- projects, tweets were restricted to those that were dia platform. sent from the contiguous United States, were writ- Here, we expand on previous research by in- ten in English, and contained at least one keyword vestigating how annotator racial identity and tweet relevant to the analysis. To limit our sample to features interact to influence perceptions of racism tweets that concerned Black Americans, we used on Twitter. Our analysis builds on previous common hate speech terms, the term “black girl research that uses racist speech as a stimulus magic”, and the same keywords to identify tweets (Leets, 2001; Cowan and Hodge, 1996; Tynes and about Black Americans as Flores(2017). Markoe, 2010), either in print or digital media, For the second project, tweets were also re- and calls for renewed attention to variations based stricted to a 10% sample of all tweets sent from on annotator racial identity and how these varia- 19 metropolitan statistical areas between January tions ultimately influence instruments to measure 2012 and December 2016. While both data collec- racial sentiment.