A Corpus of Novel Metaphor Annotations
Total Page:16
File Type:pdf, Size:1020Kb
Weeding out Conventionalized Metaphors: A Corpus of Novel Metaphor Annotations Erik-Lanˆ Do Dinh, Hannah Wieland, and Iryna Gurevych Ubiquitous Knowledge Processing Lab (UKP-TUDA) Department of Computer Science, Technische Universitat¨ Darmstadt http://www.ukp.tu-darmstadt.de Abstract While the basic senses of tight—e.g., being phys- ically close together or firmly attached—conflict We encounter metaphors every day, but only with the more abstract budget, the metaphoric a few jump out on us and make us stum- use as meaning limited can be readily understood. ble. However, little effort has been devoted to investigating more novel metaphors in com- In contrast, the use of penumbra in (2) is more parison to general metaphor detection efforts. creative and novel. Its literal meaning is “an We attribute this gap primarily to the lack of area covered by the outer part of a shadow.”1 larger datasets that distinguish between con- Its metaphoric meaning is seldom encountered: ventionalized, i.e., very common, and novel Shadows follow objects that cast them, and espe- metaphors. The goal of this paper is to al- cially penumbras can be perceived as having fuzzy leviate this situation by introducing a crowd- outlines; attributes which are picked up by the sourced novel metaphor annotation layer for an existing metaphor corpus. Further, we an- metaphorical sense of a rather unspecified group alyze our corpus and investigate correlations of people following someone in differing vicinity. between novelty and features that are typically Common linguistic metaphor definitions used used in metaphor detection, such as concrete- in NLP (Steen et al., 2010; Tsvetkov et al., ness ratings and more semantic features like 2014) do not differentiate between convention- the Potential for Metaphoricity. Finally, we alized and novel metaphors. Some even allow present a baseline approach to assess novelty for auxiliary verbs and prepositions to be anno- in metaphors based on our annotations. tated as metaphors when they are not used in 1 Introduction their original sense (e.g., the non-spatially used on in “She wrote a study on metaphors”). While Metaphors have received considerable interest in such cases can be filtered out rather easily from NLP in recent years (see Shutova(2015)). Re- any given corpus—e.g., by using POS tag and search questions range from direct detection of lemma filters—many conventionalized metaphors metaphors in text (linguistic metaphors) to find- persist. Existing work avoids this problem par- ing mappings between conceptual source and tar- tially by only annotating certain grammatical con- get domains (conceptual metaphors). structions, such as adjective–noun or verb–object However, an important aspect of metaphors— relations (Shutova et al., 2016; Rei et al., 2017). novelty—is often overlooked, or intentionally dis- However, these too usually do not distinguish be- regarded. Consider the metaphors (bold) in the tween conventionalized and novel metaphors. following examples: Following Shutova(2015), we deem the dis- tinction between conventionalized and novel (1) We all live on tight budgets, but we still need metaphors important, because the meaning of con- to have some fun. ventionalized metaphors can usually be found in (2) They were beginning to attract a penumbra dictionaries or other resources like WordNet— of gallery-goers, as though they were offering novel metaphors on the other hand pose a more a guided tour. difficult challenge. But the lack of resources incor- porating this distinction leads to few researchers The metaphor tight budgets in (1) is an often used 1https://www.macmillandictionary.com/dictionary/ collocation and therefore highly conventionalized. american/penumbra 1412 Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1412–1424 Brussels, Belgium, October 31 - November 4, 2018. c 2018 Association for Computational Linguistics investigating novel metaphors, or the related mea- used metaphorically or not; i.e., if the token has sure of metaphoricity. They use small datasets a more basic meaning that can be understood in that have been manually annotated by experts (Del comparison with the one it expresses in its current Tredici and Bel, 2016), or focus crowdsourcing context. More basic is described as being: studies on a small number of instances (Dunn, • More concrete; what they evoke is easier to 2014). It is only recently that any work has in- imagine, see, hear, feel, smell, and taste. troduced larger-scale novel metaphor annotations (Parde and Nielsen, 2018). In contrast to our ap- • Related to bodily action. proach, they collect annotations on a relation level (see also Section 2). • More precise (as opposed to vague). Annotating metaphors is not an easy task, due • Historically older. to the inherent ambiguity and subjectivity. There- fore, we investigate approaches for annotating They note that this basic meaning does not nec- novelty in metaphors, before closing the resource essarily have to be the most frequent one. An gap for token-based annotations by creating a extended version, MIPVU, was used to annotate layer of novelty scores. parts of the British National Corpus (BNC Consor- Our contributions are the following: tium, 2007, BNC), resulting in the VU Amsterdam Metaphor Corpus (Steen et al., 2010, VUAMC). (1) We augment an existing metaphor corpus Shutova and Teufel(2010) adapt the MIP as a pre- by assessing metaphor novelty on a token requisite to annotating source and target domains level using crowdsourcing, enabling larger of metaphor—the metaphoric mapping—in parts research on novel metaphors, of the BNC. Others use rather relaxed guidelines. Tsvetkov (2) we analyze our corpus for correlation with et al.(2014) rely on intuitive definitions by their features used for general metaphor detection annotators, not specifying metaphor more closely. or metaphoricity prediction, and They ask their annotators to mark words that (3) we show that a baseline approach based on “are used non-literally in the following sentences.” features usually used for (binary) metaphor Jang et al.(2015) provide a Wikipedia definition of detection can be useful for distinguishing metaphor to users in a crowdsourcing study. Sub- novel and conventionalized metaphors. sequently, the users are tasked with annotating fo- rum posts by deciding “whether the highlighted In Section 2, we first discuss annotation guide- word is used metaphorically or literally.” lines that have been used for existing datasets, as A common trait of the listed works is that well as prior work on novel metaphor detection they do not distinguish between nuances of and crowdsourcing in NLP. This discussion is fol- metaphoricity. Instead, they impose a binary lowed by Section 3, where we detail the base cor- scheme (metaphoric/literal) on the rather diffuse pus and the crowdsourced creation of our annota- language phenomenon of metaphor. In contrast, tion layer. In Section 4, we describe our baseline Dunn(2014) introduces a scalar measurement of for detecting novel metaphors and its results. We metaphoricity on a sentence level. He created a conclude in Section 5 with a summary of our con- corpus of 60 genre-diverse sentences with varying tributions, and an outlook of what our newly intro- levels of metaphoricity, which are rated in three duced layer of annotations can enable. crowdsourcing experiments: a binary selection, a 2 Related Work ternary selection, and an ordering approach. The mean value for each sentence is then used as the Annotating metaphors is a difficult task, because metaphoricity value. Dunn(2014) uses this data to there is no single definition that can be adhered to. derive a computational measure of metaphoricity, Instead, different researchers formulate their own which he then employs in an unsupervised system versions, which can vary quite substantially. to label metaphoric sentences in the VUAMC. The The Pragglejaz Group(2007) created influen- evaluation, however, is only done on the binary la- tial metaphor annotation guidelines, the Metaphor bels provided therein. Identification Procedure (MIP). After reading a In a similar direction, Del Tredici and Bel text, an annotator assesses each token as being (2016) present their concept of Potential of 1413 genre tokens metaphors content tokens content metaphors novel metaphors acprose 75,272 9,170 42,544 5,505 102 convrsn 57,249 3,841 22,019 1,774 25 fiction 54,115 5,349 26,935 3,174 94 news 51,672 7,580 29,346 4,727 132 total 238,308 25,940 120,844 15,180 353 Table 1: VUAMC corpus statistics. Content tokens include adjectives, adverbs, verbs (without have, be, do), and nouns. For the purpose of this table, metaphors with a novelty score higher than T = 0.5 are considered novel (the possible range is [–1,1]). Metaphoricity (POM) of verbs. The POM de- good annotation results for five tasks with dif- scribes the inherent potential of a verb to take on ferent setups: two tasks ask for numerical val- a metaphoric meaning, derived from its distribu- ues (affect recognition, word similarity), two other tional behavior. They infer that low-POM verbs tasks require a binary decision (textual entailment are only able to have low degrees of metaphoricity, recognition, event ordering), and a final task pro- thus can only evoke conventionalized metaphors. vides three options for the annotators (word sense Therefore, they propose to exclude such low-POM disambiguation). Sukhareva et al.(2016) utilize verbs from novel metaphor detection systems. crowdsourcing to annotate semantic frames. They Haagsma and Bjerva(2016) use violations design their task as a decision tree, with annotators of selectional preferences (Wilks, 1978) to find moving down the tree when annotating. Moham- novel metaphors. Selectional preferences describe mad et al.(2016) use crowdsourcing for metaphor which semantic classes a verb prefers as its direct and emotion annotation in order to investigate object. Since they are often mined from large cor- their correlation. They employ an ordering ap- pora and based on frequency, they argue that this proach to annotation, and only consider verbs that feature is more suited for novel metaphor detec- already contain a metaphoric sense in WordNet.