• Abstract and Figures
• Public Full-text

Explainable Tsetlin Machine framework for fake news detection with credibility score assessment Bimal Bhattarai Ole-Christoffer Granmo Lei Jiao University of Agder University of Agder University of Agder [email protected] [email protected] [email protected] Abstract particularly problematic as they seek to deceive people for political and financial gain (Gottfried The proliferation of fake news, i.e., news in- and Shearer, 2016). tentionally spread for misinformation, poses a threat to individuals and society. Despite In recent years, we have witnessed extensive various fact-checking websites such as Politi- growth of fake news in social media, spread across Fact, robust detection techniques are required news blogs, Twitter, and other social platforms. to deal with the increase in fake news. Sev- At present, most online misinformation is manu- eral deep learning models show promising re- ally written (Vargo et al., 2018). However, natural sults for fake news classification, however, language models like GPT-3 enable automatic gen- their black-box nature makes it difficult to ex- eration of realistic-looking fake news, which may plain their classification decisions and quality- assure the models. We here address this prob- accelerate future growth. Such growth is problem- lem by proposing a novel interpretable fake atic as most people nowadays digest news stories news detection framework based on the re- from social media and news blogs (Allcott and cently introduced Tsetlin Machine (TM). In Gentzkow, 2017). Indeed, the spread of fake news brief, we utilize the conjunctive clauses of poses a severe threat to journalism, individuals, and the TM to capture lexical and semantic prop- society. It has the potential of breaking societal be- erties of both true and fake news text. Fur- lief systems and encourages biases and false hopes. ther, we use the clause ensembles to calcu- late the credibility of fake news. For eval- For instance, fake news related to religion or gender uation, we conduct experiments on two pub- inequality can produce harmful prejudices. Fake licly available datasets, PolitiFact and Gossip- news can also trigger violence or conflict. When Cop, and demonstrate that the TM framework people are frequently exposed to fake news, they significantly outperforms previously published tend to distrust real news, affecting their ability baselines by at least 5% in terms of accuracy, to distinguish between truth and untruth. To re- with the added benefit of an interpretable logic- duce these negative impacts, it is critical to develop based representation. Further, our approach methods that can automatically expose fake news. provides higher F1-score than BERT and XL- Net, however, we obtain slightly lower accu- Fake news detection introduces various challeng- racy. We finally present a case study on our ing research problems. By design, fake news in- model’s explainability, demonstrating how it tentionally deceives the recipient. It is therefore arXiv:2105.09114v1 [cs.CL] 19 May 2021 decomposes into meaningful words and their difficult to detect fake news based on linguistic con- negations. tent, style, and diverseness. For example, fake news may narrate actual events and context to support 1 Introduction false claims (Feng et al., 2012). Thus, other than Social media platforms on the Internet have be- hand-crafted and data-specific features, we need to come an integral part of everyday life, and more employ a knowledge base of linguistic patterns for and more people obtain news from social rather effective detection. Training fake news classifiers than traditional media, such as newspapers. Key on crowdsourced data may further provide a poor drivers include cost-effectiveness, freedom to com- fit for future news events. Fake news is emerging ment, and shareability among friends. Despite continuously, quickly rendering previous textual these advantages, social media exposes people to content obsolete. Accordingly, some studies, such abundant misinformation. Fake news stories are as (Guo et al., 2018), have tried to incorporate so- cial context and hierarchical neural networks using Lukoianova, 2015). Many studies such as (Wang, attention to uncover more lasting semantic patterns. 2017)(Mitra and Gilbert, 2015)(Shu et al., 2020a) Despite significant advances in deep learning- incorporate publicly available datasets, providing a based techniques for fake news detection, few ap- basis for detailed analysis of fake news and detec- proaches can explain their classification decisions. tion methods. Currently, knowledge on the dynamics underlying Recently, deep learning-based latent representa- fake news is lacking. Thus, explaining why cer- tion of text has significantly improved fake news tain news items are considered fake may uncover classification accuracy (Karimi and Tang, 2019). new understanding. Making the reasons for deci- However, the latent representations are generally sions transparent also facilitates discovering and difficult to interpret, providing limited insight into rectifying model weaknesses. To the best of our the nature of fake news. In (Castillo et al., 2011), knowledge, previous research has not yet addressed the authors introduced features based on social con- explainability in fake news detection. text, obtained from the profiles of users and their Paper contributions: In this paper, we propose activity patterns. Other approaches depend upon so- an explainable framework for fake news detection cial platform-specific features such as likes, tweets, built using the Tsetlin Machine (TM). Our TM- and retweets for supervised learning (Volkova et al., based framework captures the frequent patterns that 2017)(Ruchansky et al., 2017). characterize real news, distilling both linguistic and While the progress on detecting fake news has semantic patterns unique for fake news stories. The been significant, limited effort has been directed to- resulting model constitutes a global interpretation wards interpretability. Existing deep learning meth- of fake news. To provide a more refined view on ods generally extract features to train classifiers individual fake news (local interpretability), we without giving any interpretable explanation. This also propose a credibility score for measuring the lack of transparency makes them black boxes when credibility of news. Finally, our framework allows it comes to understandability (Du et al., 2019). In the practitioner to see what features are critical for this paper, we propose a novel fake news detection making news fake (global interpretability). method that builds upon the TM (Granmo, 2018). Paper organization: The remainder of the pa- The TM is a recent approach to pattern classifica- per is organized as follows. In Section2, we briefly tion, regression, and novelty detection (Abeyrathna summarize related work. We then detail our ex- et al., 2021; Yadav et al., 2021; Abeyrathna et al., plainable framework in Section3. In Section4, the 2020; Bhattarai et al., 2021) that attempts to bridge datasets and experimental configurations are pre- the present gap between interpretability and accu- sented. We analyze our empirical results in Section racy in state-of-the-art machine learning. By using 5, before we conclude the work in the last section. the AND-rules of the TM to capture lexical and semantic properties of fake news, our aim is to 2 Related Work improve the performance of fake news detection. More importantly, our framework is intrinsically The problem of detecting deception is not new to interpretable, both globally, at the model level, and natural language processing (Vargo et al., 2018). locally for the individual false news predictions. Significant application domains include detecting false online advertising, fake consumer reviews, 3 Explainable Fake News Detection and spam emails (Ott et al., 2011)(Zhang and Framework Guan, 2008). The detection of fake news focuses on uncovering spread of misleading news articles In this section, we present the details of our TM- (Zhou and Zafarani, 2020)(Zhou et al., 2019). Typ- based fake news detection framework. ical detection techniques use either text-based lin- guistic features (Potthast et al., 2018) or visual 3.1 TM Architecture features (Gupta et al., 2013). Overall, fake news The TM consists of a team of two-action Tsetlin detection methods fall into two groups: knowledge- Automata (TAs) with 2N states. Each TA performs based models based on fact-checking news arti- either action “Include” (in State 1 to N) or action cles using external sources (WuYou et al., 2014), “Exclude” (in State N to 2N). Jointly, the actions and style-based models, which leverage linguis- specify a pattern recognition task. Action “Include” tic features capturing writing style (Rubin and incorporates a specific sub-pattern, while action Text Input Input Binary Encoded Input Real News Fake News Clauses Summation Threshold Argmax Output y Figure 1: Interpretable Tsetlin machine architecture. “Exclude” rejects the sub-pattern. The TAs are up- clauses. Positive polarity is assigned to one half of + dated based on iterative feedback in the form of the clauses, denoted by Cj . These are to capture rewards or penalties. The feedback signals how patterns for the target class (y = 0 for TM1 and well the pattern recognition task is solved, with the y = 1 for TM2). Negative polarity is assigned to − intent of maximizing classification accuracy. Re- the other half, denoted by Cj . Negative polarity wards reinforce the actions performed by the TAs, clauses are to capture

Load more

  11

Copy Link