MojiSem: Varying linguistic purposes of emoji in (Twitter) context Noa Na’aman, Hannah Provenza, Orion Montoya Brandeis University nnaaman,hprovenza,obm @brandeis.edu { } Abstract have evolved diverse and novel linguistic uses for emoji. Early research into emoji in textual com- The expressive richness of emoji communica- munication has focused largely on high- tion would, on its own, be sufficient reason to frequency usages and ambiguity of inter- seek a nuanced understanding of its usage. But pretations. Investigation of a wide range of our initial survey of emoji on Twitter reveals many emoji usage shows these glyphs serving at cases where emoji serve direct semantic functions least two very different purposes: as con- in a tweet or where they are used as a grammat- tent and function words, or as multimodal ical function such as a preposition or punctua- affective markers. Identifying where an tion. Early work on Twitter emoticons (Schnoe- emoji is replacing textual content allows belen, 2012) pre-dated the wide spread of Uni- NLP tools the possibility of parsing them code emoji on mobile and desktop devices. Recent as any other word or phrase. Recognizing work (Miller et al., 2016) has explored the cross- the import of non-content emoji can be a platform ambiguity of emoji renderings; (Eis- a significant part of understanding a mes- ner et al., 2016) created word embeddings that sage as well. performed competitively on emoji analogy tasks; (Ljubesicˇ and Fiser,ˇ 2016) mapped global emoji We report on an annotation task on En- distributions by frequency; (Barbieri et al., 2017) glish Twitter data with the goal of classify- used LSTMs to predict them in context. ing emoji uses by these categories, and on We feel that a lexical semantics of emoji char- the effectiveness of a classifier trained on acters is implied in these studies without being di- these annotations. We find that it is pos- rectly addressed. Words are not used randomly, sible to train a classifier to tell the differ- and neither are emoji. But even when they replace ence between those emoji used as linguis- a word, emoji are used for different purposes than tic content words and those used as par- words. We believe that work on emoji would be alinguistic or affective multimodal mark- better informed if there were an explicit typology ers even with a small amount of training of the linguistic functions that emoji can serve in data, but that accurate sub-classification expressive text. The current project offered anno- of these multimodal emoji into specific tators a framework and heuristics to classify uses classes like attitude, topic, or gesture will of emoji by linguistic and discursive function. We require more data and more feature engi- then used a model based on this corpus to pre- neering. dict the grammatical function of emoji characters in novel contexts. 1 Background 2 Annotation task Emoji characters were first offered on Japanese mobile phones around the turn of the 21st cen- Although recognizing the presence of emoji char- tury. These pictographic elements reached global acters is trivial, the linguistic distinctions we language communities after being added to Uni- sought to annotate were ambiguous and seemed code 6.0 in 2010, and then being offered within prone to disagreement. Therefore in our annota- software keyboards on smartphones. In the ensu- tion guidelines we structured the process to mini- ing half-decade, digitally-mediated language users mize cognitive load and lead the annotators to in- 136 Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics- Student Research Workshop, pages 136–141 Vancouver, Canada, July 30 - August 4, 2017. c 2017 Association for Computational Linguistics https://doi.org/10.18653/v1/P17-3022 tuitive decisions. This was aided somewhat by the filtered to include: only tweets with characters observation that emoji are often used in contexts from the Emoji Unicode ranges (i.e. gener- that make them graphical replacements for exist- ally U+1FXXX, U+26XX–U+27BF); only tweets ing lexical units, and that such uses are therefore labeled as being in English. We excluded straightforward to interpret. Taking advantage of tweets with embedded images or links. Redun- such uses, our flow presented annotators with a dant/duplicate tweets were filtered by comparing few simple questions at each step, to determine tweet texts after removal of hashtags and @men- whether to assign a label or to move on to the next tions; this left only a small number of cloned du- category. plicates. After that, tweets were hand-selected to get a wide variety of emojis and context in a small 2.1 Categories and subtypes sample size — therefore, our corpus does not re- The high-level labels we defined for emoji uses flect the true distribution of emoji uses or context were: types. Function (func): stand-ins for a function • 2.3 Guidelines word in an utterance. These had a type attribute with values prep, aux, conj, Our guidelines gave annotators cursory back- dt, punc, other. An example from our ground about emoji and their uses in social media, data: “I like u”. assuming no particular familiarity with the range Content (cont): stand-ins for lexical of creative uses of emoji. In hindsight we no- • words or phrases that are part of the main ticed our assumption that annotators would have informative content of the sentence. These a fair degree of familiarity with modes of dis- have natural parts of speech, which anno- course on Twitter. The short-message social plat- tators could subtype as: noun, verb, form has many distinctive cultural and commu- adj, adv, other. “The to success is nicative codes of its own, not to mention sub- ” cultures, and continuously evolving trends com- Multimodal (mm): characters that enrich a bined with a long memory. As two of the authors • grammatically-complete text with markers of are active and engaged users of Twitter, we un- affect or stance, whether to express an atti- fortunately took it for granted that our annotators tude (“Let my work disrespect me one more would be able to decipher emoji in contexts that time... ”), to echo the topic with an iconic required nuanced knowledge of InterNet language repetition (“Mean girls ”, or to express a and Twitter norms. This left annotators occasion- gesture that might have accompanied the ut- ally bewildered: by random users begging celebri- terance in face-to-face speech (“Omg why ties to follow them, by dialogue-formatted tweets, is my mom screaming so early ”). Sub- and by other epigrammatic subgenres of the short- types: attitude, topic, gesture, text form. other. The analytical steps we prescribed were: The POS tags we chose were deliberately Identifying each emoji in the tweet coarse-grained and did not include distinctions • Deciding whether multiple contiguous emoji such as noun sub-types. We wanted to capture im- • should be considered separately or as a group portant differences while knowing that we would Choosing the best tag for the emoji (or se- have fewer instances for the function and content • labels. For all three labels, annotators were asked quence) to provide a replacement: a word or phrase Providing a translation or interpretation for • that could replace the emoji. For func and cont, each tagged span. replacements were a criterion for choosing the la- bel; for mm there was room for interpretation. Eliciting an interpretation serves two goals: first, as a coercive prompt for the user to bias them 2.2 Data Collection toward a linguistic interpretation. A replaceable Tweets were pulled from the public Twitter phrase that fits with the grammar of the sentence is streaming API using the tweepy Python pack- a different proposition than a marker that amounts age. The collected tweets were automatically to a standalone utterance such as “I am laughing” 137 or “I am sad”. Secondly, one of the eventual ap- the labeling of the emoji’s functions. Because we plications of annotated corpus may be emoji-sense were categorizing tokens, and because these cat- disambiguation (ESD), and mapping to a lexical- egories are not ordered and we presented more ized expression would be useful grounding for fu- than two labels, we used Fleiss’s κ. But Fleiss’s ture ESD tasks. The text field was very helpful κ requires that annotators have annotated the same during the adjudication process, clarifying the an- things, and in some cases annotators did not com- notators’ judgments and understanding of the task. plete the dataset or missed an individual emoji For each tweet, annotators first read without an- character in a tweet. In order to calculate the statis- notating anything, to get a sense of the general tics on actual agreement, rather than impute dis- message of the tweet and to think about the re- agreement in the case of an ‘abstention’, we re- lationship between the emoji and the text. On moved from our IAA-calculation counts any spans subsequent readings, they are asked to determine that were not marked by all annotators. There are whether the emoji is serving as punctuation or a many of these in the first dataset, and progressively function word; then if it is a content word; and if fewer in each subsequent dataset as the annotators it is neither of those, then to examine it as a multi- become more experienced. A total of 150 spans modal emoji. A key test, in our opinion, was ask- were excluded from Fleiss’ kappa calculations for ing annotators to simulate reading the message of this reason. the tweet aloud to another person. If a listener’s comprehension of the core message seemed to re- 2.5 Agreement/disagreement analysis quire a word or phrase to be spoken in place of an Content words.