Machine Learning for Rhetorical Figure Detection: More Chiasmus with Less Annotation Marie Dubremetz Joakim Nivre Uppsala University Uppsala University Dept. of Linguistics and Philology Dept. of Linguistics and Philology Uppsala, Sweden Uppsala, Sweden [email protected] [email protected] Abstract the label #quotation (Booten and Hearst, 2016). It is a clever reuse of the web as an annotated cor- Figurative language identification is a hard pus, but what happens if the stylistic phenomenon problem for computers. In this paper we we want to discover is not as popular on the web? handle a subproblem: chiasmus detection. When there is no available corpus and when the By chiasmus we understand a rhetorical stylistic phenomenon is rare, collecting a substan- figure that consists in repeating two el- tial amount of annotated data seems unreachable ements in reverse order: “First shall be and the computational linguist faces the limits of last, last shall be first”. Chiasmus detec- what is feasible. tion is a needle-in-the-haystack problem This study is a contribution which aims at push- with a couple of true positives for millions ing this limit. We focus on the task of automat- of false positives. Due to a lack of anno- ically identifying a playful and interesting study tated data, prior work on detecting chias- case that is rather unknown in computational lin- mus in running text has only considered guistics: the chiasmus. The chiasmus is a fig- hand-tuned systems. In this paper, we ex- ure that consists in repeating a pair of identical plore the use of machine learning on a words in reverse order. The identity criterion for partially annotated corpus. With only 31 words can be based on different linguistic prop- positive instances and partial annotation of erties, such as synonymy or morphological form. negative instances, we manage to build a Here we focus on chiasmi that have words with system that improves both precision and identical lemmas, sometimes referred to as an- recall compared to a hand-tuned system timetabole, and illustrated in Example 1. From using the same features. Comparing the now on, we will refer to this case as simply chi- feature weights learned by the machine asmus. to those give by the human, we discover common characteristics of chiasmus. (1) User services management: changing a breed or breeding a change? 1 Introduction Chiasmus is named after the greek letter χ because Recent research shows a growing interest in the the pattern of repetition is often represented as an computational analysis of style and rhetorics. ‘X’ or a cross like in Figure 1. Works like Bendersky and Smith (2012) and There are several reasons why NLP should pay Booten and Hearst (2016) demonstrate that, with attention to chiasmi. First it is a widespread lin- sufficient amounts of data, one can even train a guistic phenomenon across culture and ages. Be- system to recognize quotable sentences. Classical cause of the Greek etymology of its name, one machine learning techniques applied to text can might believe that chiasmus belongs only to the help discover much more than just linguistic struc- rhetoricians of the classical period. It is actually ture or semantic content. The techniques applied a much more ancient and universal figure. Welch so far use a lot of data already annotated by in- (1981) observes it in Talmudic, Ugaritic and even ternet users, for instance, tumblr sentences with Sumero-Akkadian literature. Contrary to what one 37 Proceedings of the 21st Nordic Conference of Computational Linguistics, pages 37–45, Gothenburg, Sweden, 23-24 May 2017. c 2017 Linkoping¨ University Electronic Press linguistic phenomena and that accuracy often de- clines drastically for the long tail of low-frequency events typical of language. Detecting chiasmus is a needle-in-the-haystack problem where all the interesting instances are in the long tail. Simply identifying word pairs repeated in reverse order is trivial, but identifying the tiny fraction of these Figure 1: Schema of a chiasmus that have a rhetorical purpose is not. may think, chiasmus is not an archaic ornament of Because of its rarity, the chiasmus is not well language and it is used far beyond advertisement suited for large-scale annotation efforts. Previous or political discourses. It is relevant for good writ- efforts aimed at chiasmus detection have there- ers of any century. Even scientists use it. For in- fore not been able to use (supervised) machine stance, currently, a small community of linguists learning for the simple reason that there has been gives a monthly ‘Chiasmus Award’ which each no training data available. These efforts have time reveals a new chiasmus produced recently by therefore mainly been based on hand-crafted rules the scientific community.1 Thus, we come to the defining categorical distinctions and typically suf- same conclusion as Nordahl (1971). If the chias- fering from either low precision or low recall. mus has for a long time seemed to be dying, this Dubremetz and Nivre (2015; 2016) proposed a is only true with respect to the interest devoted to feature-based ranking approach instead, but be- it by linguists. In reality, the chiasmus, rhetorical cause they had no annotated data to use for train- or functional, is doing well (Nordahl, 1971). Such ing, they had to resort to tuning feature weights by universality and modernity makes chiasmus detec- hand on the training set. However, an important tion a fruitful task to perform on many genres, on side effect of their work was the release of a small old text as on new texts. annotated corpus of chiasmi, containing 31 pos- Second, we can assume that the presence of itive instances, a few hundred (annotated) nega- chiasmus is a sign of writing quality because the tive instances, and several million unannotated in- author took the time to create it or to quote it. stances assumed to be negative. Nowadays the production of texts on the web is a huge industry where authors’ compensation is often based on the number of words produced, This paper presents the first attempt to use ma- which does not increase the quality. Thus, detect- chine learning to tune the weights of a model for ing such figures of speech is one clue (among oth- chiasmus detection, using the corpus released by ers) that may help distinguish masterpieces from Dubremetz and Nivre (2016). To see whether it is poorly written texts. possible to learn from this type of corpus at all, we Finally, an additional reason for studying chi- train a log-linear model with the same features as asmus, which is the focus of this paper, is its rar- Dubremetz and Nivre (2015) and Dubremetz and ity. To see just how rare it is, consider Winston Nivre (2016). The results show that the machine- Churchill’s River War, a historical narrative count- learned model, despite the small number of pos- ing more than one hundred thousand words. De- itive training instances, improves both precision spite the author’s well-known rhetorical skills, we and recall over the hand-tuned system, which is could only find a single chiasmus in the book: very encouraging. A comparison between the two types of systems reveals that they agree almost (2) Ambition stirs imagination nearly as much perfectly about which features are positive and as imagination excites ambition. negative, respectively, and that the difference in performance is therefore due simply to more well- Such rareness is a challenge for our discipline. It calibrated weights. From a more general perspec- is well known that the statistical methods domi- tive, this shows that using a hand-tuned system to nant in NLP work best for commonly occurring bootstrap a small seed corpus for machine learn- 1http://specgram.com/psammeticuspress/ ing may be a viable strategy for tackling needle- chiasmus.html in-the-haystack problems like chiasmus detection. 38 2 Related Work into precision/recall. By doing so, they come back to an essential question: When should we consider When documenting chiasmus, the computational a repetition as accidental instead of rhetoric? linguist ends up in a paradox: linguists have de- This question seems at first simpler than the veloped reflections on this rhetorical figure but question of categorising chiasmus against its alter- those reflections are not the most helpful. Indeed, native figures. But answering it leads to more uni- they never question the concept of criss-cross pat- versal and interesting answers for computational terns as an insufficient condition for producing a linguistics research. Indeed, repetition in language rhetorical effect. Typically dictionaries and stylis- is extremely banal and viewing every repetition in tic books (Fontanier, 1827; Dupriez, 2003) will a text as being rhetorical would be absurd. The explain why chiasmus should belong to the cate- very first problem in repetitive figure detection in gory of scheme and not of trope. Rabatel (2008) general, in chiasmus detection in particular, is the argues why chiasmus has different functions and disproportional number of false positives that the should therefore be divided into subcategories. On task generates. Dubremetz and Nivre (2015) point the other side, Horvei (1985) demonstrates that out that in 300 pages of historical tale the previous chiasmus should not be considered as a simple detector (Gawryjolek, 2009) extracts up to 66,000 subcategory of parallelism but rather as a figure of the criss-cross patterns (for only one true pos- on its own. All those reflections are interesting itive chiasmus to be found). At the opposite end, but they all focus on placing chiasmus into the the more strict detector of Hromada (2011) ends vast family of rhetorical figures. Following this up giving a completely empty output on the same linguistics tradition, the first computational lin- book.