The Possible, the Plausible, and the Desirable: Event-Based Modality Detection for Language Processing Valentina Pyatkin∗ Shoval Sadde∗ Aynat Rubinstein Bar Ilan University Bar Ilan University Hebrew University of Jerusalem [email protected] [email protected] [email protected] Paul Portner Reut Tsarfaty Georgetown University Bar Ilan University [email protected] [email protected] Abstract (1) a. We presented a paper at ACL’19. Modality is the linguistic ability to describe b. We did not present a paper at ACL’20. events with added information such as how de- sirable, plausible, or feasible they are. Modal- The propositional content p =“present a paper at ity is important for many NLP downstream ACL’X” can be easily verified for sentences (1a)- tasks such as the detection of hedging, uncer- (1b) by looking up the proceedings of the confer- tainty, speculation, and more. Previous studies ence to (dis)prove the existence of the relevant pub- that address modality detection in NLP often p restrict modal expressions to a closed syntac- lication. The same proposition is still referred to tic class, and the modal sense labels are vastly in sentences (2a)–(2d), but now in each one, p is different across different studies, lacking an ac- described from a different perspective: cepted standard. Furthermore, these senses are often analyzed independently of the events that (2) a. We aim to present a paper at ACL’21. they modify. This work builds on the theoreti- b. We want to present a paper at ACL’21. cal foundations of the Georgetown Gradable Modal Expressions (GME) work by Rubin- c. We ought to present a paper at ACL’21. stein et al.(2013) to propose an event-based d. We are likely to present a paper at modality detection task where modal expres- ACL’21. sions can be words of any syntactic class and sense labels are drawn from a comprehensive These sentences cannot be verified or falsified taxonomy which harmonizes the modal con- simply by examining whether p actually came or cepts contributed by the different studies. We will come to pass, and in fact, such verification is present experiments on the GME corpus aim- ing to detect and classify fine-grained modal not the goal of this way of reporting. Rather, speak- concepts and associate them with their modi- ers describe such events in order to indicate PLANS fied events. We show that detecting and clas- (2a), DESIRES (2b), NORMS (2c), or the assessed sifying modal expressions is not only feasi- PLAUSIBILITY (2d) of the associated propositional ble, but also improves the detection of modal content p. Investigating how to classify these per- events in their own right. spectives on events has been the focus of exten- 1 Introduction sive research on modality in theoretical linguistics (Kratzer, 1981; Palmer, 1986; Portner, 2009). Modality refers to the linguistic ability to describe 1 In terms of NLP technology, modal concepts as arXiv:2106.08037v1 [cs.CL] 15 Jun 2021 alternative ways the world could be. Modal ex- expressed in (2) are relevant to many downstream pressions aim to identify wishes, rules, beliefs, tasks, such as the automatic detection of hedging or norms in texts (Kratzer, 1981; Portner, 2009), and speculation (Vincze et al., 2008; Malhotra et al., which is a crucial part of Natural Language Under- 2013), uncertainty (Vincze et al., 2008; Miwa et al., standing (NLU) (Morante and Sporleder, 2012). 2012; Zerva et al., 2017; Prieto et al., 2020), opin- Concretely, events in natural language are often ion (Wiebe et al., 2005; Rubin, 2010; Miwa et al., reported in a manner that emphasizes non-actual 2012), and factuality (Saurí and Pustejovsky, 2009; perspectives on them, rather than their actual propo- Rudinger et al., 2018). Although these tasks rely on sitional content. Consider examples (1a)–(1b): modality features, so far there is no accepted stan- ∗ Equal contribution dard for modal concepts and labels, which aligns 1In formal semantics, these alternatives are referred to as possible worlds or situations (Kripke, 1959; Lewis, 1973; with the semantic space of modal senses that lin- Barwise and Perry, 1981; Kratzer, 2010). guists identify. Consequently, modality features are either treated idiosyncratically or are absent from The former is uninformative, because the modal semantic frameworks (Donatelli et al., 2018, §4.6). trigger is not explicitly associated with the event In support of such downstream tasks, a different being modified. Ghia et al.(2016) take a step in the type of NLP investigations targets modality anno- right direction, offering to annotate modal sense tation and detection in its own right (Ruppenhofer constructions. and Rehbein(2012); Baker et al.(2012); Zhou The current work proposes to address all of the et al.(2015); Marasovi c´ and Frank(2016); Hen- aforementioned deficiencies as follows. We define drickx et al.(2012); Nissim et al.(2013); Ghia et al. a prediction task that we term event-based modality (2016); Mendes et al.(2016); Lavid et al.(2016), detection, where, given a sentence as input, we aim and others). However, each of these studies creates to return all of its modal triggers, their associated its own scheme, and none of these schemes has modal senses, and, for each trigger, the respective been picked up as an accepted standard by the com- event being modified. Crucially, the modal triggers munity. Moreover, different endeavors suffer from can be from any syntactic class. The modal senses one (or more) of the following types of deficiencies are drawn from a single taxonomy that we motivate with respect to their expressivity and coverage. based on linguistic research and which harmonizes First, many studies limit the modal triggers, i.e., the different modal concepts contributed in previ- the expressions that trigger the modal meaning, to ous studies (§3). Finally, we propose to view modal a closed class of auxiliary verbs (e.g., can, might, triggers as semantic modifiers of eventive heads should, must in English (Ruppenhofer and Rehbein, in event-based (a.k.a., Neo-Davidsonian; Parsons 2012; Marasovic´ et al., 2016; Quaresma et al., (1990)) semantics. This is motivated by practical 2014)). However, as acknowledged by linguists concerns – when extracting events from texts to (Kratzer, 1981) and NLP researchers (Rubin, 2010; benefit downstream tasks, one would want easy ac- Baker et al., 2012; Nissim et al., 2013), words cess to the features that indicate the perspective on of any Part-of-Speech (POS) can trigger modal- each event, above and beyond its participants. ity. Consider, for instance, the following triggers: The accompanying annotation standard we as- We should remain calm (AUX); We have a plan to sume for the task is based on the Georgetown Grad- reduce the costs (NOUN); Our agency prefers this able Modal Expressions (GME) framework (Rubin- equipment (VERB); Marx is probably patriotic stein et al., 2013), with two simplifications that are (ADV); Devaluation has been necessary (ADJ). designed to allow for more consistent annotations Second, the modal senses, i.e., the labels that and increased ease-of-use by non-experts. First, we indicate the modal perspectives, differ from one change the modal sense labels to be intuitive and study to another, with no accepted standard. Some self-explanatory. Second, instead of the event span studies focus only on a particular sense, such as (a.k.a., prejacent) in the GME, we mark the head epistemic modality (Rubin, 2010; Ghia et al., 2016). of the event being modified. Others use labels that mix modal senses with or- To assess the feasibility of the proposed task, we thogonal notions (e.g., force, distinguishing permis- use the GME corpus (Rubinstein et al., 2013) to sion from requirement as in Baker et al.(2012)), train and test the automatic detection of modal trig- thereby making their deployment into existing an- gers, their senses, and associated events. Our exper- notations and tasks less transparent. In general, iments show that while identifying a closed set of there is no single annotation standard that covers auxiliary verbs as modal triggers is straightforward, the full spectrum of modal senses attested in the expanding the set of triggers to any syntactic class data and confirmed by the latest linguistic theories, indeed makes it a harder task. Notwithstanding as portrayed by Portner(2009). this difficulty, we show that a model based on large Finally, modality detection in NLP has often pre-trained contextualized embeddings (Liu et al., been cast as a word-sense disambiguation (WSD) 2019) obtains substantial improvements over our task (Ruppenhofer and Rehbein, 2012) or as a baseline on the full task. Moreover, we show that sentence-classification task (Marasovic´ and Frank, detecting modalized events in fact improves with 2016). Both perspectives are insufficient for any the availability of information about the modal trig- practical use. The latter is too coarse-grained, as gers. All in all, we contribute a new task, a new a sentence may contain multiple events, each of standard and a set of strong baselines for the event- which potentially carries a different modal sense. based modality task we defined. 2 Linguistic Background in the non-epistemic case, distinguishing between requirements, permissions, wants, and intentions, Modal expressions allow language users to discuss but not all of these in fact track distinct modal alternative realities. For example, the sentence She senses. For example, their “require” modality con- can reach the ceiling is modal because it describes flates both rule-based obligations and goal-oriented the event of her reaching the ceiling as feasible, preferences. but potentially non-actual. Similarly, She hopefully Most importantly, the discussion of modality will reach the ceiling is modal because it describes in NLP often resorts to linguistic regimes that are such an event as desirable, and likewise potentially not understandable by non-linguists and non-expert non-actual.