Sentence-level Emotion Classification with Label and Context Dependence Shoushan Li†‡, Lei Huang†, Rong Wang†, Guodong Zhou†* †Natural Language Processing Lab, Soochow University, China ‡ Collaborative Innovation Center of Novel Software Technology and Industrialization {shoushan.li, lei.huang2013, wangrong2022}@gmail.com, [email protected] …… Abstract <S1>她们都睡了，我蹑手蹑脚摸黑上了 床，凑上去想亲嫣一下，她突然一个转身， Predicting emotion categories, such as anger, 小手‘啪’地搭在了我的脸颊上。</S1> <S2> joy, and anxiety, expressed by a sentence is 现在我终于如愿以偿。</S2> <S3>感受着小手 challenging due to its inherent multi-label 的温度，享受着这份她对我的依恋，生怕动 一下，会让她的小手离我而去。</S3>…… classification difficulty and data sparseness. In this paper, we address above two chal- (English: …… lenges by incorporating the label dependence <S1> The girls fall to sleep, so I make my way among the emotion labels and the context de- noiselessly onto the bed, wishing I could get a pendence among the contextual instances into chance to give a kiss to Yan, suddenly she turn a factor graph model. Specifically, we recast over to me and her little soft hand fall onto my face.</S1> <S2>Praise the Lord, that is all I sentence-level emotion classification as a fac- want.</S2> <S3>Feeling the warm of her hand tor graph inferring problem in which the label and the attachment she hold to me, I couldn’t af- and context dependence are modeled as vari- ford to move even a little, fearing I may lost her ous factor functions. Empirical evaluation hand.</S3>)……) demonstrates the great potential and effective- ------------------------------------------------------------------- ness of our proposed approach to sentence- Sentence-level Emotion Classification level emotion classification. 1 Input: S1, S2, S3 1 Introduction Output: S1 : joy, love Predicting emotion categories, such as anger, joy, S2: joy and anxiety, expressed by a piece of text encom- S3: joy, love, anxiety passes a variety of applications, such as online chatting (Galik et al., 2012), news classification Figure 1: An example of a paragraph and the (Liu et al., 2013) and stock marketing (Bollen et sentences therein with their emotion categories al., 2011). Over the past decade, there has been a from the corpus collected by Quan and Ren substantial body of research on emotion classifi- (2009) cation, where a considerable amount of work has focused on document-level emotion classification. On one hand, like document-level emotion Recently, the research community has become classification, sentence-level emotion classifica- increasingly aware of the need on sentence-level tion is naturally a multi-label classification prob- emotion classification due to its wide potential ap- lem. That is, each sentence might involve more plications, e.g. the massively growing importance than one emotion category. For example, as of analyzing short text in social media (Ki- shown in Figure 1, in one paragraph, two sen- ritchenko et al., 2014; Wen and Wan, 2014). In tences, i.e., S1 and S3, have two and three emotion general, sentence-level emotion classification ex- categories respectively. Automatically classifying hibits two challenges. instances with multiple possible categories is 1 * Corresponding author 1045 Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pages 1045–1053, Beijing, China, July 26-31, 2015. c 2015 Association for Computational Linguistics sometimes much more difficult than classifying 2 Related Work instances with a single label. On the other hand, unlike document-level emo- Over the last decade, there has been an explosion tion classification, sentence-level emotion classi- of work exploring various aspects of emotion fication is prone to the data sparseness problem analysis, such as emotion resource creation because a sentence normally contains much less (Wiebe et al., 2005; Quan and Ren, 2009; Xu et content. Given the short text of a sentence, it is al., 2010), writer’s emotion vs. reader’s emotion often difficult to predict its emotion due to the analysis (Lin et al., 2008; Liu et al., 2013), emo- limited information therein. For example, in S2, tion cause event analysis (Chen et al., 2010), doc- only one phrase “如愿以偿(that is all I want)” ex- ument-level emotion classification (Alm et al., presses the joy emotion. Once this phrase fails to 2005; Li et al., 2014) and sentence-level or short appear in the training data, it will be hard for the text-level emotion classification (Tokushisa et al., classifier to give a correct prediction according to 2008; Bhowmick et al., 2009; Xu et al., 2012). the limited content in this sentence. This work focuses on sentence-level emotion clas- In this paper, we address above two challenges sification. in sentence-level emotion classification by mod- Among the studies on sentence-level emotion eling both the label and context dependence. Here, classification, Tokushisa et al. (2008) propose a the label dependence indicates that multiple emo- data-oriented method for inferring the emotion of tion labels of an instance are highly correlated to an utterance sentence in a dialog system. They each other. For instance, the two positive emo- leverage a huge collection of emotion-provoking tions, joy and love, are more likely to appear at the event instances from the Web to deal with the data same time than the two counterpart emotions, joy sparseness problem in sentence-level emotion and hate. The context dependence indicates that classification. Bhowmick et al. (2009) and two neighboring sentences or two sentences in the Bhowmick et al. (2010) apply KNN-based classi- same paragraph (or document) might share the fication algorithms to classify news sentences into same emotion categories. For instance, in Figure multiple reader emotion categories. Although the 1, S1, S2, and S3, from the same paragraph, all multi-label classification difficulty has been no- share the emotion category joy. ticed in their study, the label dependence is not Specifically, we propose a factor graph, namely exploited. More recently, Xu et al. (2012) pro- Dependence Factor Graph (DFG), to model the la- poses a coarse-to-fine strategy for sentence-level bel and context dependence in sentence-level emotion classification. They deal with the data emotion classification. In our DFG approach, both sparseness problem by incorporating the transfer the label and context dependence are modeled as probabilities from the neighboring sentences to various factor functions and the learning task aims refine the emotion categories. To some extent, this to maximize the joint probability of all these fac- can be seen a specific kind of context information. tor functions. Empirical evaluation demonstrates However, they ignore the label dependence by di- the effectiveness of our DFG approach to captur- rectly applying Binary Relevance to overcome the ing the inherent label and context dependence. To multi-label classification difficulty. the best of our knowledge, this work is the first Unlike all above studies, this paper emphasizes attempt to incorporate both the label and context the importance of the label dependence and ex- dependence of sentence-level emotion classifica- ploits it in sentence-level emotion classification tion into a unified framework. via a factor graph model. Moreover, besides the The remainder of this paper is organized as fol- label dependence, our factor graph-based ap- lows. Section 2 overviews related work on emo- proach incorporates the context dependence in a tion analysis. Section 3 presents our observations unified framework to further improve the perfor- on label and context dependence in the corpus. mance of sentence-level emotion classification. Section 4 proposes our DFG approach to sen- tence-level emotion classification. Section 5 eval- 3 Observations uates the proposed approach. Finally, Section 6 To better illustrate our motivation of modeling the gives the conclusion and future work. label and context dependence, we systematically investigate both dependence phenomena in our evaluation corpus. 1046 0.2 0.183 0.18 0.16 0.14 0.12 0.094 0.1 0.078 0.077 0.08 0.06 0.04 0.02 0.005 0.003 0.002 0.0003 0 Figure 2: Probability distribution of most and least frequently-occurred pairs of emotion categories, with left four most frequently-occurred and right four least frequently-occurred, among all 28 pairs The corpus contains 100 documents, randomly ~25%. Table 2 shows the numbers of the sen- selected from Quan and Ren (2009). There are to- tences grouped by the emotion labels they contain. tally 2751 sentences and each of them is manually From this table, we can see that more than half annotated with one or more emotion labels. sentences have two or more emotion labels. This indicates the popularity of the multi-label issue in Table 1: The numbers of the sentences in each sentence-level emotion classification. emotion category To investigate the phenomenon of label de- pendence, we first assume that XR d denotes Emotion #Sentence Emotion #Sentence an input domain of instances and Y{ l , l ,..., l } joy 691 anxiety 567 12 m hate 532 surprise 180 be a finite domain of possible emotion labels. love 1025 anger 287 Each instance is associated with a subset of Y and sorrow 611 expect 603 this subset is described as an m-dimensional vec- tor y{ y12 , y ,..., ym } where yi =1 only if in- i Table 2: The numbers of the sentences stance x has label li . and y =0 otherwise. Then, grouped by the emotion labels they contain we can calculate the probability that an instance #Sentence takes both emotion labels li and l j , denoted as No Label 180 p(,) lij l . Figure 2 shows the probability distribu- One Label 1096 tion of most and least frequently-occurred pairs of Two Labels 1081 emotion categories, with left four most fre- Three Labels 346 quently-occurred and right four least frequently- Four or more labels 48 occurred, among all 28 pairs.