MSC ARTIFICIAL INTELLIGENCE MASTER THESIS Modeling the Language of Populist Rhetoric by PERE-LLUÍS HUGUET CABOT 12345466 March 1, 2021 48 ECTS Nov 19 - Jun 20 Supervisor: Dr. Ekaterina SHUTOVA Cosupervisor: Dr. David ABADI Assessor: Dr. Giovanni COLAVIZZA INSTITUTE FOR LOGIC, LANGUAGE AND COMPUTATION iii UNIVERSITY OF AMSTERDAM Abstract Institute for Logic, Language and Computation Master of Science Modeling the Language of Populist Rhetoric by Pere-Lluís HUGUET CABOT In recent years, populism has taken the spotlight with its growth and media presence across various countries worldwide. While socio-economic factors have been con- sidered key in populist attitudes, lately the interaction between emotions and social identity is scrutinized as crucial to explain populist attitudes and its rhetoric. At the same time, Natural Language Processing (NLP) has recently provided computational models that tackle more ambitious tasks, enabling the in-depth study of political discourse and populist rhetoric. In this thesis, we will provide one of the first com- putational approaches into populist attitudes and political discourse through the use of deep learning architectures. We incorporate Multi-task Learning (MTL) with the use of auxiliary tasks that act symbiotically with political discourse. We create a new populism centered dataset (PopulismVsReddit) that enables us to model social identity in social media comments (Reddit) and the influence of biased news. In our work, we observe that metaphors and emotions play an important role when addressing political discourse. Moreover, we found evidence that emotions inter- act with the attitude different social groups receive online and provide significant improvements to identify out-group sentiments in Reddit comments. Overall, we highlight the importance of emotions on political discourse and the use of multi-task approaches that incorporate them to assess social identity and populist rhetoric. v Acknowledgements First of all, I want to acknowledge and express my gratitude to both my supervisor Ekaterina Shutova and co-supervisor David Abadi for trusting in me and offering me a project that would spark my interests, academically and personally. Thank you for going the extra mile, especially as under the current circumstances around the COVID-19 pandemic your help and commitment wasn’t affected. Katia, thank you for the patience and dedication, you helped me to keep on track while encouraging my freedom and curiosity within the research conducted. In chal- lenging times, for someone with many responsibilities you have always given a hu- man factor to your supervision and I really appreciate that. Thank you for your commitment on teaching NLP, it is because of your enthusiasm that I am invested in this field. David, I will remember our lengthy meetings fondly. You have provided me the crucial perspective from different fields (psychology and communication science), while being invested in learning about Natural Language Processing and Artificial Intelligence. Thank you for pushing the collaboration between our fields, which has made this thesis possible. I want to particularly thank Verna Dankers, who laid the ground of what this thesis is about, first by helping to spark my interest in NLP as a TA, and afterwards with her invaluable feedback, ideas and collaboration. I want to thank anyone else who has provided information, perspectives or ideas that have contributed in any way in this work, either consciously or not, through conversation, sharing a beer or an email. Thank you to the friends back home (Àdel, Gerard, Xavi, etc) for not holding a grudge for going abroad and being a remote moral support. Finally, I must thank colleagues within our MSc program. Our course has been a great example on how collaboration can be a boost for anyone who takes part in it. And I enjoyed sharing ideas and discussions in our Slack channel. Thank you Christina for making life easier and happier, those are perhaps the most important aspects for a successful thesis. Finally thank you to my mother, my father and my sister, because even if family isn’t chosen, I would choose them anyways if I had the chance and I feel immensely lucky for having their support. vii Contents 1 Introduction 1 1.1 Motivation and Research Questions .................... 1 1.2 Methodology and Contributions ...................... 2 1.3 Thesis Structure ................................ 3 2 Related Work 5 2.1 Neural Networks models in NLP ...................... 5 2.2 Populism .................................... 7 2.3 Political Bias .................................. 8 2.4 Emotions .................................... 9 2.5 Metaphors ................................... 11 3 Modeling Metaphor and Emotion in Political Discourse 13 3.1 Related work .................................. 14 3.2 Tasks and Datasets .............................. 15 3.3 Methods .................................... 15 3.4 Experiments and Results ........................... 17 3.5 Discussion ................................... 19 3.6 Conclusion ................................... 21 4 PopulismVsReddit: A Dataset on Emotions, News Bias and Social Identity 23 4.1 Dataset Creation ................................ 24 4.2 Data Analysis ................................. 30 4.3 Conclusion ................................... 38 5 Modeling the Out-Group through Multi-task Learning 39 5.1 Tasks ...................................... 40 5.2 Methods .................................... 41 5.3 Experiments .................................. 42 5.4 Results ..................................... 43 5.5 Discussion ................................... 43 5.6 Conclusion ................................... 49 6 Conclusion 55 6.1 Future Work .................................. 56 6.2 Social Impact and Responsible Use ..................... 56 6.3 Publications .................................. 57 6.4 Funding statement .............................. 57 Bibliography 63 viii A Extra Material 75 A.1 System ..................................... 75 A.2 Lists ....................................... 75 A.3 Additional Tables ............................... 76 A.4 Figures ..................................... 76 1 Chapter 1 Introduction Populism has taken the spotlight in political communication in recent years. Various countries around the globe have experienced a surge of populist rhetoric (Inglehart and Norris, 2016) in both the public and political space. Populism, when understood as a communication strategy, employs political discourse as a channel through dif- ferent types of media, such as news and social networks (Jagers and Walgrave, 2007). Through different platforms, populism uses certain rhetoric that revolves around social identity (Hogg, 2016; Abadi, 2017) and the Us vs. Them argumentation (Mudde, 2004). Social psychological and emotional perspectives have described populist communication strategies (Rico, Guinjoan, and Anduiza, 2017) and demonstrated its operationalization through experimental research (Wirz et al., 2018) as being suc- cessful in inducing emotions. Moreover, emotions have been shown to be crucial in shaping the public opinion (Demertzis, 2006; Marcus, 2002; Marcus, 2003). At the same time, metaphors also serve as a mechanism to influence public opinion within political discourse (Lakoff, 1991; Musolff, 2004). Natural Language Processing (NLP) has a wide range of applications, aiming at understanding text and perform language processing tasks computationally, which often involve comprehension of complex language, including political discourse. Previous work has proven to be successful in determining the political affiliation of politicians using parliamentary data (Iyyer et al., 2014) and the political bias in news sources (Li and Goldwasser, 2019; Kiesel et al., 2019). However, there is a lack of computational modeling approaches for populist rhetoric and populist attitudes, the closest approach being hate speech detection (Silva et al., 2016). 1.1 Motivation and Research Questions This thesis has populism as a focus through the lenses of Natural Language Process- ing (NLP). Due to the nature of populism as an umbrella term usually interpreted from different perspectives, such as ideology, political discourse or rhetoric, explor- ing populism leads to several questions regarding how to model populist rhetoric computationally. Our main research question is: 1. How can we capture populist rhetoric using Deep Learning models within Natural Language Processing? In previous research, emotions have been shown to be tied to populist rhetoric (Demertzis, 2006). Recent examples using emotions in multi-task learning models have shown how it can improve performance on other tasks such as metaphor de- tection (Dankers et al., 2019). Certain metaphors (implicit vs. explicit) are used within the populist rhetoric to evoke certain emotional reactions. Since emotions 2 Chapter 1. Introduction have a strong relation to populist attitudes, we aim to explore whether computa- tionally they offer any advantage in MTL setups with populist rhetoric, leading to the research question, 2. How do emotions interact with populist rhetoric and can they contribute to modeling it? Metaphors are used as mechanisms to engage and convince Flusberg, Matlock, and Thibodeau, 2018, and are ubiquitous within political discourse (Beigman Kle- banov, Diermeier, and Beigman, 2008). We expect them to be beneficial in the context of populist rhetoric and we also intend to explore the role of metaphor in multitask learning