PACLIC 29 Is Wikipedia Really Neutral? A Sentiment Perspective Study of War-related Wikipedia Articles since 1945 Yiwei Zhou, Alexandra I. Cristea and Zachary Roberts Department of Computer Science University of Warwick Coventry, United Kingdom fYiwei.Zhou, A.I.Cristea, [email protected] Abstract include books, journal articles, newspapers, web- pages, sound recordings2, etc. Although a “Neutral Wikipedia is supposed to be supporting the 3 “Neutral Point of View”. Instead of accept- point of view” (NPOV) is Wikipedia’s core content ing this statement as a fact, the current paper policy, we believe sentiment expression is inevitable analyses its veracity by specifically analysing in this user-generated content. Already in (Green- a typically controversial (negative) topic, such stein and Zhu, 2012), researchers have raised doubt as war, and answering questions such as “Are about Wikipedia’s neutrality, as they pointed out there sentiment differences in how Wikipedia that “Wikipedia achieves something akin to a NPOV articles in different languages describe the across articles, but not necessarily within them”. same war?”. This paper tackles this chal- Moreover, people of different language backgrounds lenge by proposing an automatic methodology based on article level and concept level senti- share different cultures and sources of information. ment analysis on multilingual Wikipedia arti- These differences have reflected on the style of con- cles. The results obtained so far show that rea- tributions (Pfeil et al., 2006) and the type of informa- sons such as people’s feelings of involvement tion covered (Callahan and Herring, 2011). Further- and empathy can lead to sentiment expression more, Wikipedia webpages actually allow to con- differences across multilingual Wikipedia on tain opinions, as long as they come from reliable au- war-related topics; the more people contribute thors4. Due to its openness to multiple forms of con- to an article on a war-related topic, the more extreme sentiment the article will express; dif- tribution, the articles on Wikipedia can be viewed as ferent cultures also focus on different concepts a summarisation of thoughts in multiple languages about the same war and present different sen- about specific topics. Automatically detecting and timents towards them. Moreover, our research measuring the differences can be crucial in many provides a framework for performing differ- applications: public relation departments can get ent levels of sentiment analysis on multilin- some useful suggestions from Wikipedia about top- gual texts. ics close to their hearts; Wikipedia readers can get some insights about what people speaking other lan- 1 Introduction guages think about the same topic; Wikipedia ad- Wikipedia is the largest and most widely used ministrators can quickly locate the Wikipedia arti- encyclopaedia in collaborative knowledge building cles that express extreme sentiment, to better apply (Medelyan et al., 2009). Since its start in 2001, it the NPOV policy, by eliminating some edits. contains more than 33 million articles in more than 2http://en.wikipedia.org/wiki/Wikipedia: 200 languages, while only about 4 million articles Citing_sources 3 are in English1. Possible sources for the content http://en.wikipedia.org/wiki/Wikipedia: Neutral_point_of_view 1http://meta.wikimedia.org/wiki/List_of_ 4http://en.wikipedia.org/wiki/Wikipedia: Wikipedias Identifying_reliable_sources 160 29th Pacific Asia Conference on Language, Information and Computation pages 160 - 168 Shanghai, China, October 30 - November 1, 2015 Copyright 2015 by Yiwei Zhou, Alexandra Cristea and Zachary Roberts PACLIC 29 In order to further gain insight on these mat- parts only. Through extracting the subjective sen- ters, and especially, on the degree of neutrality on tences of one article, a classifier can not only achieve given topics presented in different languages, we ex- higher efficiency, because of the shorter length, but plore an approach that can perform multiple levels can also achieve higher accuracy, by leaving out the of sentiment analysis on multilingual Wikipedia ar- ‘noises’. We thus choose to extract the possible sub- ticles. We generate graded sentiment analysis re- jective parts of the articles first. sults for multilingual articles, and attribute senti- More recently, many researchers have realised ment analysis to concepts, to analyse the sentiment that multiple sentences may express sentiment about that onespecific named entity is involved in. For the the same concept, or that one sentence may contain sake of simplicity, we restrict our scenario within sentiment towards different concepts. As a result, the war-related topics, although our approach can the concept level (or aspect level) sentiment analysis be easily applied on other domains. Our results has attracted more and more attention. Researchers show that even though the overall sentiment polar- have proposed two approaches to extract concepts. ities of multilingual Wikipedia articles on the same The first approach is to manually create a list of war-related topic are consistent, the strengths of sen- interesting concepts, before the analysis (Singh et timent expression vary from language to language. al., 2013). The second approach is to extract candi- The remainder of the paper is structured as fol- date concepts from the object content, automatically lows. In Section 2, we present an overview of differ- (Mudinas et al., 2012). As in Wikipedia different ent approaches of sentiment analysis. Section 3 de- articles will mention different concepts, it is impos- scribes the approach selected in this research to per- sible to pre-create the concepts list without reading form article level and concept level sentiment analy- all the articles. We thus choose to automatically ex- sis on multilingual Wikipedia articles. In Section 4, tract the named entities in the subjective sentences experimental results are presented and analysed, and as concepts. in Section 5, we conclude the major findings and re- marks for further research. 3 Methodology 3.1 Overview 2 Related Research We employ war-related topics in Wikipedia as Researchers have been addressing the problem of counter-examples to refute the statement that sentiment analysis of user-generated content mainly ‘Wikipedia is neutral’. at three levels of granularity: sentence level senti- Based on our choice of approaches (see Sec- ment analysis, article level sentiment analysis and tion 2), we build a straightforward processing concept level sentiment analysis. pipeline (Figure 1) as briefly sketched below (with The most common level of sentiment analysis is details in the subsequent sub-sections). the sentence level, which has laid the ground for the First, we retrieve the related topic name based other two. Its basic assumption is that each sentence on some input keywords. After that, the Wikipedia has only one target concept. webpages in all available languages on this topic are Article level (or document level) sentiment analy- downloaded. Because of the diversity of the content sis is often used on product reviews, news and blogs, in Wikipedia webpages, some data pre-processing, where it is believed there is only one target concept as described in Section 3.2, is needed, in order to in the whole article. Our research performs article acquire plain descriptive text. We further translate level sentiment analysis of Wikipedia articles. We the plain descriptive text into English (see notes in believe this is applicable here, as the Wikipedia web- Section 3.2 on accuracy and the estimated errors in- pages’ structure is that with a topic as the title, the troduced), for further processing. To extract the sub- body of the webpage is the corresponding descrip- jective contents from each translated article, we to- tion of the topic. There are mainly two directions kenise the article into sentences, and then perform for article level sentiment analysis: analysis towards subjective analysis, as is described in Section 3.3, on the whole article, or analysis towards the subjective each sentence. As mentioned already, based on prior 161 PACLIC 29 Webpages Data Machine Extraction Preprocessing Translation Translated Wikipedia Webpages Articles Keywords Articles Sentiment Subjective Scores Sentences Concept Level Article Level Subjectivity Sentiment Analysis Sentiment Analysis Analysis Figure 1: Processing Pipeline research (Pang and Lee, 2004), only the subjective plemented, not the same can be said for other lan- sentences are retained, while the rest are discarded. guages. To close the gap between other languages We then leverage the English sentiment analysis re- and English sentiment analysis resources, we ap- sources to measure the sentiment score for each sub- ply machine translation on the texts in other lan- jective sentence, and utilise named entity extraction guages. To date, machine translation techniques are tools to extract the named entities as target concepts well developed; products such as Google Translate7 in this subjective sentence. We calculate the arti- and Bing Translator 8 are widely used in academic cle level sentiment scores, as is described in Sec- (Bautin et al., 2008; Wan, 2009) and business con- tion 4.1, as well as the concept level sentence scores, text. In (Balahur and Turchi, 2014), researchers as is described in Section 3.5, with both being based pointed out that the machine translation techniques on the sentence level sentiment scores. In the final have reached a reasonable level