Investigating the Relationship between Literary Genres and Emotional Plot Development Evgeny Kim, Sebastian Pado´ and Roman Klinger Institut fur¨ Maschinelle Sprachverarbeitung University of Stuttgart Pfaffenwaldring 5b, 70569 Stuttgart, Germany evgeny.kim,sebastian.pado,roman.klinger @ims.uni-stuttgart.de { } Abstract fall-rise), and “Oedipus” (fall-rise-fall). They also clustered fictional texts from Project Gutenberg1 Literary genres are commonly viewed as by similarity to emotion arc types, suggesting that being defined in terms of content and style. their arc types could be useful for categorizing lit- In this paper, we focus on one particular erary texts. At the same time, their analysis suf- type of content feature, namely lexical ex- fered from some limitations: it was mostly qualita- pressions of emotion, and investigate the tive and limited to the single emotion of happiness. hypothesis that emotion-related informa- Crucially, they did not investigate the relationship tion correlates with particular genres. Us- between emotions and established literary classifi- ing genre classification as a testbed, we cation schemes more concretely. compare a model that computes lexicon- The goal of our study is to investigate exactly this based emotion scores globally for complete relationship, extending the focus beyond one sin- stories with a model that tracks emotion gle emotion, and complementing qualitative with arcs through stories on a subset of Project quantitative insights. In this, we build on previous Gutenberg with five genres. work which has shown that stories from different Our main findings are: (a), the global emo- literary genres tend to have different flows of emo- tion model is competitive with a large- tions (Mohammad, 2011). The role of emotion has vocabulary bag-of-words genre classifier been investigated in different domains, including (80 % F1); (b), the emotion arc model social media (Pool and Nissim, 2016; Dodds et al., shows a lower performance (59 % F1) but 2011; Kouloumpis et al., 2011; Gill et al., 2008), shows complementary behavior to the chats (Brooks et al., 2013), and fairy tales (Alm global model, as indicated by a very good et al., 2005). performance of an oracle model (94 % F ) 1 As the basis for our quantitative analysis, we and an improved performance of an ensem- adopt the task of genre classification, which makes ble model (84 % F ); (c), genres differ in 1 it possible for us to investigate different formula- the extent to which stories follow the same tions of emotion features in a predictive setting. emotional arcs, with particularly uniform Genres represent one of the best-established clas- behavior for anger (mystery) and fear (ad- sifications for fictional texts, and are typically de- ventures, romance, humor, science fiction). fined to follow specific communicative purposes 1 Introduction and Motivation or functional traits of a text (Kessler et al., 1997), although we note that literary studies take care to Narratives are inseparable from emotional content emphasize the role of artistic and aesthetic prop- of the plots (Hogan, 2011). Recently, Reagan et al. erties in genre definition (Cuddon, 2012, p. 405), (2016) presented an analysis of fictional texts in and take a cautious stance towards genre defini- which they found that there is a relatively small tion (Allison et al., 2011; Underwood et al., 2013; number of universal plot structures that are tied Underwood, 2016). to the development of the emotion happiness over Traditionally, computational studies of genre time (“emotional arcs”). They called the arcs “Rags classification use either style-based or content- to riches” (rise), “Tragedy” (fall), “Man in a hole” (fall-rise), “Icarus” (rise-fall), “Cinderella” (rise- 1https://www.gutenberg.org 17 Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature. Proceedings, pages 17–26, Vancouver, BC, August 4, 2017. c 2017 Association for Computational Linguistics , based features. Stylistic approaches measure, for Genre Count instance, frequencies of non-content words, of adventure 569 punctuation, part-of-speech tags and character n- humor 202 grams (Karlgren and Cutting, 1994; Kessler et al., mystery 379 1997; Stamatatos et al., 2000; Feldman et al., 2009; romance 327 Sharoff et al., 2010). Content-aware characteris- science fiction 542 tics take into account lexical information in bag- of-words models or build on top of topic models 2019 (Karlgren and Cutting, 1994; Hettinger et al., 2015, P 2016). A precursor study to ours is Samothrakis Table 1: Statistics for our Gutenberg genre corpus. and Fasli(2015), who assess emotion sequence features in a classification setting. We extend their to the five literary genres found in the Brown cor- approach by carrying out a more extensive analysis. pus (Francis and Kucera, 1979): adventure (Guten- In sum, our contributions are: berg tag: “Adventure stories”), romance (“Love sto- ries” and “Romantic fiction”), mystery (“Detective 1. We perform genre classification on a corpus and mystery stories”), science fiction (“Science fic- sampled from Project Gutenberg with the gen- tion”), and humor (“Humor”). All books must addi- res science fiction, adventure, humor, roman- tionally have the tag “Fiction”. We exclude books tic fiction, detective and mystery stories. which contain one of the following tags: “Short 2. We define two emotion-based models for stories”, “Complete works”, “Volume”, “Chapter”, genre classification based on the eight fun- “Part”, “Collection”. This leads to a corpus of 2113 damental emotions defined by Plutchik(2001) stories. Out of these, 94 books (4.4 %) have more – fear, anger, joy, trust, surprise, sadness, dis- than one genre label. For simplicity, we discard gust, and anticipation. The first one is an these texts, which leads to the corpus of 2019 sto- emotion lexicon model based on the NRC dic- ries with the relatively balanced genre distribution tionary (Mohammad and Turney, 2013). The as shown in Table1. second one is an emotion arc model that mod- els the emotional development over the course 2.2 Feature Sets of a story. We avoid the assumption of Rea- We consider three different feature sets: bag-of- gan et al.(2016) that absence of happiness words features (as a strong baseline), lexical emo- indicates fear or sadness. tion features, and emotion arc features. 3. We analyze the performance of the vari- Bag-of-words features. An established strong ous models quantitatively and qualitatively. feature set for genre classification, and text classifi- Specifically, we investigate how uniform gen- cation generally, consists of bag-of-words features. res are with respect to emotion developments For genre classification, the generally adopted strat- and discuss differences in the importance of egy is to use the n most frequent words in the lexical units. corpus, whose distribution is supposed to carry more genre-specific rather than content- or domain- 2 Experimental Setup specific information. The choice of n varies across stylometric studies, from, e.g., 1,000 (Sharoff et al., To analyze the relationships between emotions ex- 2010) to 10,000 (Underwood, 2016). We set pressed in literature and genres, we formulate a n = 5, 000 here. We refer to this feature set as genre classification task based on different emotion BOW. feature sets. We start with a description of our data set in the following Section 2.1. The features are Lexical emotion features. Our second feature explained in Section 2.2 and then how they are used set, EMOLEX, is a filtered version of BOW, cap- in various classification models (in Section 2.3). turing lexically expressed emotion information. It consists of all words in the intersection between the 2.1 Corpus corpus vocabulary and the NRC dictionary (Mo- We collect books from Project Gutenberg that hammad and Turney, 2013) which contains 4,463 match certain tags, namely those which correspond words associated with 8 emotions. Thus, it incor- 18 Input Convolution Max Pool Dense Softmax 2 3 es11 es12 es13 ::: es1n 6es21 es22 es23 ::: es2n7 Adventure 6 7 6 . 7 6 . 7 Romance 6 7 6esd1 esd2 esd3 ::: esdn7 Mystery 6 7 6 1 0 0 ::: 0 7 6 7 SciFi 6 0 1 0 ::: 0 7 6 7 6 0 0 1 ::: 0 7 Humor 6 7 6 . 7 4 . 5 0 0 0 ::: 1 Figure 1: Architecture of CNN model porates the assumption that words associated with 2.3 Models for Genre Classification emotions reflect the actual emotional content (Best- In the following, we discuss the use of the feature gen, 1994). We do not take into account words sets defined in Section 2.2 with classification meth- from “positive”/“negative” categories or those that ods to yield concrete models. are not associated with any emotions. This model We use the two lexical feature sets, BOW and takes into account neither emotion labels nor posi- EMOLEX, with a random forest classifier (RF, tion of an emotion expression in the text. Breiman(2001)) and multi-layer perceptron (MLP, Hinton(1989)). RF often performs well indepen- Emotion arc features. The final feature set, dent of chosen meta parameters (Criminisi et al., EMOARC, in contrast to the lexical emotion fea- 2012), while MLP provides a tighter control for tures, takes into account both emotion labels and overfitting and copes well with non-linear prob- position of an emotion expression. It represents an lems (Collobert and Bengio, 2004). emotion arc in the spirit of Reagan et al.(2016), The emotion arc feature set (EMOARC) is used but considers all of Plutchik’s eight fundamental for classification in a random forest, multi-layer emotion classes. We split each input text into k perceptron, and a convolutional neural network equal-sized, contiguous segments S corresponding (CNN). For the first two classification methods, we to spans of tokens S = tn, . , tm . We treat k as flatten the emotion-segment matrix into an input h i a hyper-parameter to be optimized (cf.