Before Name-Calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation
Total Page:16
File Type:pdf, Size:1020Kb
Before Name-calling: Dynamics and Triggers of Ad Hominem Fallacies in Web Argumentation Ivan Habernal† Henning Wachsmuth‡ Iryna Gurevych† Benno Stein‡ † Ubiquitous Knowledge Processing Lab (UKP) and Research Training Group AIPHES Department of Computer Science, Technische Universitat¨ Darmstadt, Germany www.ukp.tu-darmstadt.de www.aiphes.tu-darmstadt.de ‡ Faculty of Media, Bauhaus-Universitat¨ Weimar, Germany <firstname>.<lastname>@uni-weimar.de Abstract Although the ad hominem fallacy has been known since Aristotle, surprisingly there are very Arguing without committing a fallacy is one of few empirical works investigating its properties. the main requirements of an ideal debate. But even when debating rules are strictly enforced While Sahlane(2012) analyzed ad hominem and and fallacious arguments punished, arguers of- other fallacies in several hundred newspaper edi- ten lapse into attacking the opponent by an torials, others usually only rely on few examples, ad hominem argument. As existing research as observed by de Wijze(2002). As Macagno lacks solid empirical investigation of the ty- (2013) concludes, ad hominem arguments should pology of ad hominem arguments as well as be considered as multifaceted and complex strate- their potential causes, this paper fills this gap gies, involving not a simple argument, but sev- by (1) performing several large-scale annota- eral combined tactics. However, such research, to tion studies, (2) experimenting with various neural architectures and validating our work- the best of our knowledge, does not exist. Very ing hypotheses, such as controversy or reason- little is known not only about the feasibility of ableness, and (3) providing linguistic insights ad hominem theories in practical applications (the into triggers of ad hominem using explainable NLP perspective) but also about the dynamics and neural network architectures. triggers of ad hominem (the theoretical counter- part). 1 Introduction This paper investigates the research gap at three Human reasoning is lazy and biased but it per- levels of increasing discourse complexity: ad fectly serves its purpose in the argumentative con- hominem in isolation, direct ad hominem with- text (Mercier and Sperber, 2017). When chal- out dialogical exchange, and ad hominem in large lenged by genuine back-and-forth argumentation, inter-personal discourse context. We asked the humans do better in both generating and evaluat- following research questions. First, what qualita- ing arguments (Mercier and Sperber, 2011). The tive and quantative properties do ad hominem ar- dialogical perspective on argumentation has been guments have in Web debates and how does that reflected in argumentation theory prominently by reflect the common theoretical view (RQ1)? Sec- the pragma-dialectic model of argumentation (van ond, how much of the debate context do we need Eemeren and Grootendorst, 1992). Not only for recognizing ad hominem by humans and ma- sketches this theory an ideal normative model of chine learning systems (RQ2)? And finally, what argumentation but also distinguishes the wrong ar- are the actual triggers of ad hominem arguments gumentative moves, fallacies (van Eemeren and and can we predict whether the discussion is go- Grootendorst, 1987). Among the plethora of ing to end up with one (RQ3)? prototypical fallacies, notwithstanding the contro- We tackle these questions by leveraging Web- versy of most taxonomies (Boudry et al., 2015), based argumentation data (Change my View on ad hominem argument is perhaps the most famous Reddit), performing several large-scale annotation one. Arguing against the person is considered studies, and creating a new dataset. We exper- faulty, yet is prevalent in online and offline dis- iment with various neural architectures and ex- course.1 Godwin’s_law), if a discussion goes on long enough, sooner 1According to ‘Godwin’s law’ known from the in- or later someone will compare someone or something to ternet pop-culture (https://en.wikipedia.org/wiki/ Adolf Hitler. 386 Proceedings of NAACL-HLT 2018, pages 386–396 New Orleans, Louisiana, June 1 - 6, 2018. c 2018 Association for Computational Linguistics trapolate the trained models to validate our work- 2013; Macagno, 2013; Walton, 2007; Hansen, ing hypotheses. Furthermore, we propose a list 2017; Woods, 2008). We omit examples here of potential linguistic and rhetorical triggers of as these provided in theoretical works or text- ad hominem based on interpreting parameters of books are usually artificial, as already criticized trained neural models.2 This article thus presents by (de Wijze, 2002) or (Boudry et al., 2015). the first NLP work on multi-faceted ad hominem The topic of fallacies, which might be consid- fallacies in genuine dialogical argumentation. We ered as sub-topic of argumentation quality, has re- also release the data and the source code to the re- cently been investigated also in the NLP field. Ex- search community.3 isting works are, however, limited to the mono- logical view (Wachsmuth et al., 2017; Habernal 2 Theoretical background and related and Gurevych, 2016b,a; Stab and Gurevych, 2017) work or they focus primarily on learning fallacy recog- The prevalent view on argumentation emphasizes nition by humans (Habernal et al., 2017, 2018a). its pragmatic goals, such as persuasion and group- Another related NLP sub-field includes abusive based deliberation (van Eemeren et al., 2014), al- language and personal attacks in general. Wulczyn though numerous works have dealt with argument et al.(2017) investigated whether or not Wikipedia as product, that is, treating a single argument and talk page comments are personal attacks and an- its properties in isolation (Toulmin, 1958; Haber- notated 38k instances resulting in a highly skewed nal and Gurevych, 2017). Yet the social role of distribution (only 0.9% were actual attacks). Re- argumentation and its alleged responsibility for garding the participants’ perspective, Jain et al. the very skill of human reasoning explained from (2014) examined principal roles in 80 discussions the evolutionary perspective (Mercier and Sperber, from the Wikipedia: Article for Deletion pages 2017) provide convincing reasons to treat argu- (focusing on stubbornness or ignoredness, among mentation as an inherently dialogical tool. others) and found several typical roles, including The observation that some arguments are in ‘rebels’, ‘voices’, or ‘idiots’. In contrast to our fact ‘deceptions in disguise’ was made already data under investigation (Change My View de- by Aristotle (Aristotle and Kennedy (transla- bates), Wikipedia talk pages do not adhere to strict tor), 1991), for which the term fallacy has been argumentation rules with manual moderation and adopted. Leaving the controversial typology of have a different pragmatic purpose. fallacies aside (Hamblin, 1970; van Eemeren and Reddit as a source platform has also been used Grootendorst, 1987; Boudry et al., 2015), the ad in other relevant works. Saleem et al.(2016) de- hominem argument is addressed in most theories. tected hateful speech on Reddit by exploiting par- Ad hominem argumentation relies on the strat- ticular sub-communities to automatically obtain egy of attacking the opponent and some feature training data. Wang et al.(2016) experimented of the opponent’s character instead of the counter- with an unsupervised neural model to cluster so- arguments (Tindale, 2007). With few exceptions, cial roles on sub-reddits dedicated to computer the following five sub-types of ad hominem are games. Zhang et al.(2017) proposed a set of nine prevalent in the literature: abusive ad hominem comment-level dialogue act categories and anno- (a pure attack on the character of the opponent), tated 9k threads with 100k comments and built a tu quoque ad hominem (essentially analogous to CRF classifier for dialogue act labeling. Unlike the “He did it first” defense of a three-year-old these works which were not related to argumenta- in a sandbox), circumstantial ad hominem (the tion, Tan et al.(2016) examined persuasion strate- “practice what you preach” attack and accusation gies on Change My View using word overlap fea- of hypocrisy), bias ad hominem (the attacked op- tures. In contrast to our work, they focused solely ponent has a hidden agenda), and guilt by as- on the successful strategies with delta-awarded sociation (associating the opponent with some- posts. Using the same dataset, Musi(2017) re- body with a low credibility) (Schiappa and Nordin, cently studied concession in argumentation. 2An attempt to address the plea for thinking about prob- lems, cognitive science, and the details of human language 3 Data (Manning, 2015). 3https://github.com/UKPLab/ Change My View (CMV) is an online ‘place to naacl2018-before-name-calling-habernal-et-al post an opinion you accept [...] in an effort to un- 387 derstand other perspectives on the issue’, in other words an online platform for ‘good-faith’ argu- mentation hosted on Reddit.4 A user posts a sub- mission (also called original post(er); OP) and other participants provide arguments to change the OP’s view, forming a typical tree-form Web dis- cussion. A special feature of CMV is that the OP acknowledges convincing arguments by giving a delta point (∆). Unlike the vast majority of in- ternet discussion forums, CMV enforces obeying strict rules (such as no ‘low effort’ posts, or accus- Figure 1: ‘No discussion after ad hominem.’ Dis- ing of being unwilling to change view) whose vi- tribution of the number of comments before the first olation results into deleting the comment by mod- ad hominem is committed proportional to the thread erators. These formal requirements of an ideal de- length. bate with the notion of violating rules correspond to incorrect moves in critical discussion in the nor- them start attacking ‘out of blue’, without any pre- mative pragma-dialectic theory (van Eemeren and vious interaction in the thread. On the other hand, Grootendorst, 1987). Thus, violating the rule of 11% ad hominem authors write at least one ‘nor- ‘not being rude or hostile’ is equivalent to com- mal’ argument in the thread (we found one outlier mitting ad hominem fallacy.