An Attention-Based Deep Learning Method for Text Sentiment Analysis

2020 International Conference on Computational Science and Computational Intelligence (CSCI) An attention-based deep learning method for text sentiment analysis Thanh Le School of Business Information Technology University of Economics HCMC, Vietnam [email protected] Abstract— Text sentiment analysis is target-oriented, aiming using unstructured text from social media to aid people on to identify the opinion or attitude from a piece of natural language information indigestion. text toward topics or entities, whether it is negative, positive or neutral using natural language processing and computational Several approaches have been proposed for text sentiment methods. With the growth of the internet, numerous business analysis. Among those that are based on lexical resources, websites have been deployed to support shopping products, methods utilize dictionary of opinionated synsets like booking services online as well as to allow online reviewing and SentiWordNet [2], or set of opinion adjective terms like Liu commenting the services in forms of either business forums or lexicon [1], are widely used these days thanks to their simplicity social networks. Use of text sentiment analysis for automatically and effectiveness. SentiWordNet is basically a lexical resource mining opinion from the feedbacks on such emerging internet for opinion mining, in which the connections between synsets platforms is not only useful for customers seeking for advice, but and opinions are defined based on WordNet [3] dictionary. also necessary for business to study customers’ attitudes toward Computationally generated from a set of seed words known to brands, products, services, or events, and has become an be popularly used, SentiWordNet consists of about 147,306 increasingly dominant trend in business strategic management. synsets, and is affordable for many real world problems in terms Current state-of-the-art approaches for text sentiment analysis of sentiment analysis. However, its large number of selected include lexicon based and machine learning based methods. In this terms, including those having no positive or negative sentiment research, we proposed a method that utilizes deep learning with polarity, makes SentiWordNet highly noisy and failed at attention word embedding. We showed that our method accounting for sentiment bearing lexical features relevant to text outperformed popular lexicon and embedding based methods. in micro-blogs [4]. On the other hand, Liu lexicon [5] [1] which Keywords— sentiment analysis, word embedding, deep learning, was initiated based on a set of seed adjective terms meaning attention mechanism, natural language processing, lexicons either good or bad and augmented through a knowledge discovery process using semantic synonym and antonym I. INTRODUCTION relations, is a set of around 6,800 English terms that are Text sentiment analysis is an ongoing field in natural categorized in positive and negative opinion groups. Thanks to language processing (NLP). It has been widely used in an array its update in the past decade with misspellings, morphological of business applications, including social media, algorithmic variants, slang, and social-media markup, Liu lexicon can trading, customer experience, and human resource management. perform well on social media text analysis without using It recently becomes an important analysis tool in financial and advanced methods for text pre-processing. It however cannot business research thanks to its ability in analyzing opinions, cover all kinds of real world problems due to its limitation in expressions, likes and dislikes of customers towards various terms of sentiment intensity. Moreover, the significant entities, namely products, services, organizations, individuals disagreement between SentiWordNet and the gold standard etc. as well as in identifying customer trends. These marketing lexicons, namely Harvard General Inquirer [6] and Linguistic matters have been always the most important issues in business Inquiry and Word Counts [7], is another reason for its worse strategic management, particularly in business decision making. performance, preventing it from being used widely in real world According to Liu et al. [1], the beliefs, or perceptions of reality, problems. and the choices one makes are somehow conditioned upon the On the other side, methods that are based on machine way the others act. This is true not only for individuals but also learning utilize supervised machine learning models such as for business. While consumers hunger for and rely on online Support Vector Machines, Naïve Bayes, Ensemble Learning, advice or recommendations of products and services, business Neural Networks… together with advanced text embedding demand for utilities that can transform customers’ expressions techniques, namely Word2vec, GloVe and FastText for word and conversations into customer insights, or those for social representation in form of numerical vector [8] [9]. Since the media monitoring, reputation management and voice of the emerge of deep learning, it has become one part of the most customer programs. Traditionally, individuals usually ask for state-of-the-art systems in various areas, especially in text opinions from friends and family members, while business rely sentiment analysis. Similar to conventional machine learning on surveys, focus groups, opinion polls and consultants. In the methods, deep learning depends heavily on the word embedding modern age of Big Data, when millions of consumer reviews and techniques. Both Word2vec and GloVe preserve word syntactic discussions flood the Internet every day, while individuals feel meanings, making possible word ranging using syntactic overwhelmed with information, it is as well impossible for similarity. They are however unable to capture the sentiment business to keep that up manually. Thus, there is a clear need of polarity of the words [10] [11] [12]. Therefore, words with computational methods for automatically analyzing sentiment opposite polarity maybe mapped into close vectors. Lack of 978-1-7281-7624-6/20/$31.00 ©2020 IEEE 282 DOI 10.1109/CSCI51800.2020.00054 sentiment information in vector representation, which plays a SentiWordNet basically consists of automatic annotations of all vital role in text sentiment analysis may significantly degrade the synsets in WordNet according to the notions of “positivity”, performance of deep learning systems for text sentiment “negativity” and “objectivity”. Each synset is associated with analysis. three numerical scores, Pos(s), Neg(s), and Obj(s) indicating how positive, negative or objective the terms in the synset are. In this research, we proposed a deep learning based method Different senses of the same term may have different opinion- with an attention word embedding technique that utilizes NLP related properties. Scores vary in the interval [0.0, 1.0], and the and popular lexicons for text sentiment analysis. We showed that total of their values is 1.0 per synset. Positive polarity score our method outperformed the popular lexicon based and deep (PosS) and negative polarity score (NegS) are independently learning methods. assigned to synsets. Each synset may consist of many terms II. SENTIMENT ANALYSIS AND RELEVANT METHODS having the same meaning in a given context, and are ranked based on their popularity in that such context. Table I shows a A. Text sentiment analysis list of synsets for the adjective term “terrible”, and describing Text sentiment analysis is a multi-discipline research field how information of synsets is stored in SentiWordNet database. aiming to analyze people’s opinions, sentiments, appraisals, attitudes, and emotions toward entities and their attributes TABLE I: SYNSETS FOR ‘TERRIBLE’ ADJECTIVE TERM expressed in written text. Entities can be products, services, Name POS PosS NegS ObjS Definition organizations, individuals, events, issues, or discussion topics. causing fear or dread In general, text sentiment analysis is a semantic analysis awful.s.02 S 0.000 0.625 0.375 problem, but highly focusing on determining “positive” or or terror exceptionally bad or “negative” without the need of fully understanding the given atrocious.s.02 S 0.000 0.875 0.125 displeasing sentence or document. Since human language is rarely precise or intensely or plainly spoken, sentiment analysis is therefore hard and extremely bad or severe.s.01 S 0.000 0.875 0.125 challenging. There are different levels of tasks in text sentiment unpleasant in degree analysis [13] [14]. While the basic task is to identify the or quality sentiment polarity from a given text to learn whether the extreme in degree or expressed opinion in that text is positive, negative, or neutral, frightful.s.02 S 0.125 0.250 0.625 extent or amount or impact tasks of higher levels are to deal with different levels of sentiment towards different features of a given entity. In this research, we aimed at improving the accuracy of identifying C. One-hot encoding opinion, which is either positive or negative, from short reviews One-hot encoding is a method to encode categorical features about something, made available on websites or social media. by a 1-of-K encoding scheme. It allows word representation by Current state-of-the-art text sentiment analysis methods can vectors where, given a word, all the elements of the vector are 0 be roughly classified into lexicon based approach, machine except one, which is corresponding to the word index. One-hot learning based approach and hybrid approach [15].

Load more