• Abstract and Figures
• Public Full-text

A Survey on Stance Detection for Mis- and Disinformation Identification Momchil Hardalov1,2∗ , Arnav Arora1,3 , Preslav Nakov1,4 and Isabelle Augenstein1,3 1CheckStep Research 2Sofia University “St. Kliment Ohridski”, Bulgaria 3University of Copenhagen, Denmark 4Qatar Computing Research Institute, HBKU, Doha, Qatar {momchil, arnav, preslav.nakov, isabelle}@checkstep.com Abstract these notions by Claire Wardle from First Draft,2 misinforma- tion is “unintentional mistakes such as inaccurate photo cap- Detecting attitudes expressed in texts, also known tions, dates, statistics, translations, or when satire is taken as stance detection, has become an important task seriously.”, and disinformation is “fabricated or deliberately for the detection of false information online, be it manipulated audio/visual context, and also intentionally cre- misinformation (unintentionally false) or disinfor- ated conspiracy theories or rumours.”. While the intent to do mation (intentionally false, spread deliberately with harm is very important, it is also very hard to prove. Thus, the malicious intent). Stance detection has been framed vast majority of work has focused on factuality, thus treating in different ways, including: (a) as a component misinformation and disinformation as part of the same prob- of fact-checking, rumour detection, and detecting lem: spread of false information (regardless of whether this is previously fact-checked claims; or (b) as a task in done with harmful intent). This is also the approach we will its own right. While there have been prior efforts adopt in this survey. to contrast stance detection with other related so- Detecting and aggregating the expressed stances to- cial media tasks such as argumentation mining and wards a piece of information can be a powerful tool sentiment analysis, there is no survey examining for a variety of tasks like understanding idealogical de- the relationship between stance detection and mis- bates [Hasan and Ng, 2014], gathering different frames and disinformation detection from a holistic view- of a particular issue [Shurafa et al., 2020] or determining point, which is the focus of this survey. We review leanings of different media outlets [Stefanov et al., 2020]. and analyse existing work in this area, before dis- The task of stance detection has been studied from different cussing lessons learnt and future challenges. angles, e.g., in political debates [Habernal et al., 2018], for fact-checking [Thorne et al., 2018], or regarding 1 Introduction new products [Somasundaran and Wiebe, 2009]. Fur- ther, different types of text have been studied, includ- In the past decade, there has been a rapid growth in pop- ing social media posts [Zubiaga et al., 2016a] and news ularity of social media platforms such as Facebook, Twit- articles [Pomerleau and Rao, 2017]. Finally, stances ter, Reddit and Parler.1 Moreover, controversial events such expressed by different actors have been considered, as Brexit and the US presidential election, as well as the such as politicians [Johnson and Goldwasser, 2016], emergence of the COVID-19 pandemic that brought an in- journalists [Hanselowski et al., 2019], users on the fodemic [Alam et al., 2020] with it. This, in turn, has led to web [Derczynski et al., 2017]. arXiv:2103.00242v1 [cs.CL] 27 Feb 2021 a flood of dubious content, both in mainstream media and There have been a couple of recent surveys related to online, raising yet another red flag reminding us of the ever stance detection. Zubiaga et al. [2018a] present a survey growing need for effective detection of mis- and disinforma- on rumour veracity prediction, where they discussed stance tion. as a component of the rumour verification pipeline, and In this work, we examine the relationship between auto- K¨uc¸¨uk and Can [2020] give a holistic view on the stance de- matically detecting false information online – including fact- tection task in general. checking, detecting fake news, rumors, and hoaxes – and However, there is no existing overview of how different the core underlying Natural Language Processing (NLP) task formulations of the task play a role in the detection of false needed to achieve this, namely stance detection. Therein, information. This could be as a standalone task – to gather we consider both the phenomena of mis- and disinformation. stances of users or texts towards a claim (to aid in the fact- The latter two differ by the underlying intention of disinfor- checking process or studying misinformation), or as a com- mation to do harm. This is illustrated in the definitions of ponent of an automated system which uses stance as features ∗Contact Author 1https://www.digitalinformationworld.com/2021/02/social- 2http://firstdraftnews.org/wp-content/uploads/2018/07/Types-of- media-platforms-peak-points.html Information-Disorder-Venn-Diagram.png Dataset Source(s) Target Context Evidence #Instances Task English Datasets Rumour Has It [Qazvinian et al., 2011] 7 Topic Tweet ) 10K Rumours PHEME [Zubiaga et al., 2016a] 7 Claim Tweet : 7.5K Rumours Emergent [Ferreira and Vlachos, 2016] nj Headline Article∗ ) 2.6K Rumours FNC-1 [Pomerleau and Rao, 2017] nj Headline Article q 75K Fake news RumourEval ’17 [Derczynski et al., 2017] 7 Implicit‡ Tweet : 7.1K Rumours FEVER [Thorne et al., 2018] ɀ Claim Facts ) 185K Fact-checking Snopes [Hanselowski et al., 2019] Snopes Claim Snippets ) 19.5K Fact-checking RumourEval ’19 [Gorrell et al., 2019] 7 \ Implicit‡ Post : 8.5K Rumours COVIDLies [Hossain et al., 2020] 7 Claim Tweet q 6.8K Misconceptions TabFact [Chen et al., 2020] ɀ Statement WikiTable ) 118K Fact-checking Non-English Datasets Arabic [Baly et al., 2018] nj Claim Document q 3K Fact-checking DAST (Danish) [Lillie et al., 2019] \ Submission Comment : 3K Rumour Croatian [Boˇsnjak and Karan, 2019] nj Title Comment q 0.9K Claim verifiability Arabic [Khouja, 2020] nj Claim Title q 3.8K Claim verification Table 1: Key characteristics of the stance detection datasets for mis- and disinformation detection. #Instances denotes dataset size as a whole; the numbers are in thousands (K) and are rounded to the hundreds. ∗the article’s body is summarised. ‡the stance is expressed towards a topic, which is not present in the data. Sources: 7 Twitter, nj News, ɀikipedia, \ Reddit. Evidence: q Single, ) Multiple, : Thread. for determining veracity. With this survey, we aim to bridge (ii) Emotion Recognition, where the goal is to recognise emo- this gap, present some emerging trends from this space and tions such as love, anger, sadness, etc. in the text; (iii) Per- discuss the challenges ahead. spective Identification, which aims to find the point-of-view of the author (e.g., Democrat vs. Republican) and the target 2 What is Stance? is always explicit; (iv) Sarcasm Detection, where the interest is in satirical or ironic pieces of text, which are often written In order to understand the task of stance detection, we first with the intent of ridicule or mockery; (v) Sentiment Analysis, provide definitions of stance and the stance-taking process. which determines the polarity of a piece of text. Biber and Finegan (1988) define stance as the expression of a speaker’s standpoint and judgement towards a given 3 Stance and Factuality proposition. Further, Du Bois (2007) define stance as “A public act by a social actor, achieved dialogically through In this section, we discuss the different aspects of mis- overt communicative means, of simultaneously evaluating and disinformation identification, where stance detection has objects, positioning subjects (self and others), and align- been successfully applied, i.e., fake news detection, rumour ing with other subjects, with respect to any salient dimen- verification and debunking, misconception identification, and sion of the sociocultural field”, showing that the stance- fact-checking, both as a task on its own or as a component taking process is affected not only by one’s personal opin- of a pipeline. In Table 1, we provide an overview of the key ion, but also by other external factors such as cultural characteristics of the available datasets for each task. There, norms, roles in the institution of the family, etc. For the we include the source from which the data is collected, the purpose of this survey, we adopt the general definition of target towards which the stance is expressed in the provided stance detection from K¨uc¸¨uk and Can [2020]: “for an in- context. Further, we show the type of evidence: Single is a put in the form of a piece of text and a target pair, stance single document/fact, Multiple is multiple pieces of text evi- detection is a classification problem where the stance of dence, often facts or documents, Thread is a (conversational) the author of the text is sought in the form of a cate- sequence of posts or a discussion. The final column is the gory label from this set: Favor, Against, Neither. Oc- type of the target Task. casionally, the category label of Neutral is also added to the set of stance categories [Mohammad et al., 2016], 3.1 Fact-Checking as Stance Detection and the target may or may not be explicitly mentioned in As stance detection is the core task within fact-checking, the text [Augenstein et al., 2016a; Mohammad et al., 2016]. prior work has studied it in isolated, artificial task settings Note that the stance detection definitions and the label inven- – predicting the stance towards one or several documents. tories vary somewhat dependent on the target application (see Fact-Checking with One Evidence Document Section 3). Pomerleau and Rao [2017] organised the first Fake News Finally, stance detection can be distinguished from sev- Challenge3 (FNC-1) with the aim of automatically detecting eral other closely related NLP tasks: (i) Biased Language fake news. The goal was to detect the relatedness of a news Detection, where the existence of an inclination or tendency towards a particular perspective within a text is explored; 3http://www.fakenewschallenge.org/ article’s body to a headline (possibly from another news tification of mis- and disinformation, here we review its po- article), based on the stance that the former takes regarding tency to serve as a component in a larger automated pipeline.

Load more

  9

Copy Link