So-Called Non-Subsective Adjectives Ellie Pavlick Chris Callison-Burch University of Pennsylvania University of Pennsylvania [email protected] [email protected] Abstract Privative Non-Subsective (AN N= ) anti- artificial counterfeit\ deputy; The interpretation of adjective-noun pairs erstwhile ex- fabricated fake false fictional fictitious former plays a crucial role in tasks such as rec- hypothetical imaginary mock mythical ognizing textual entailment. Formal se- onetime past phony pseudo- mantics often places adjectives into a simulated spurious virtual would-be Plain Non-Subsective (AN N and AN N = ) taxonomy which should dictate adjec- alleged apparent arguable6⇢ assumed\ 6 ; tives’ entailment behavior when placed in believed debatable disputed doubtful dubious erroneous expected faulty adjective-noun compounds. However, we future historic impossible improbable show experimentally that the behavior of likely mistaken ostensible plausible subsective adjectives (e.g. red) versus possible potential predicted presumed probable proposed putative questionable non-subsective adjectives (e.g. fake) is not seeming so-called supposed suspicious as cut and dry as often assumed. For ex- theoretical uncertain unlikely unsuccessful ample, inferences are not always symmet- ric: while ID is generally considered to be Table 1: 60 non-subsective adjectives from Nayak mutually exclusive with fake ID, fake ID is et al. (2014). Noun phrases involving non- considered to entail ID. We discuss the im- subsective adjectives are assumed not to entail the head noun. E.g. would-be hijacker hijacker. plications of these findings for automated 6) natural language understanding. (See Section 2 for definition of privative vs. plain). 1 Introduction Most adjectives are subsective, meaning that an in- In this example, recognizing that 1(a) does not en- stance of an adjective-noun phrase is an instance tail 1(b) hinges on understanding that a would-be of the noun: a red car is a car and a successful hijacker is not a hijacker. senator is a senator. In contrast, adjective-noun The observation that adjective-nouns (ANs) in- phrases involving non-subsective adjectives, such volving non-subsective adjectives do not entail the as imaginary and former (Table 1), denote a set underlying nouns (Ns) has led to the generaliza- that is disjoint from the denotation of the nouns tion that the deletion of non-subsective adjectives they modify: an imaginary car is not a car and tends to result in contradictory utterances: Mous- a former senator is not a senator. Understanding saoui is a would-be hijacker entails that it is not whether or not adjectives are subsective is critical the case that Moussaoui is a hijacker. This gen- in any task involving natural language inference. eralization has prompted normative rules for the For example, consider the below sentence pair treatment of such adjectives in various NLP tasks. from the Recognizing Textual Entailment (RTE) In information extraction, it is assumed that sys- task (Giampiccolo et al., 2007): tems cannot extract useful rules from sentences 1. (a) U.S. District Judge Leonie Brinkema containing non-subsective modifiers (Angeli et al., accepted would-be hijacker Zacarias 2015), and in RTE, it is assumed that systems Moussaoui’s guilty pleas ... should uniformly penalize insertions and deletions (b) Moussaoui participated in the Sept. 11 of non-subsective adjectives (Amoia and Gardent, attacks. 2006). Privative Plain Non-subsective Subsective (e.g. fake) (e.g. alleged) (e.g. red) N AN N AN AN Figure 1: Three main classes of adjectives. If their entailment behavior is consistent with their theoretical definitions, we would expect our annotations (Section 3) to produce the insertion (blue) and deletion (red) patterns shown by the bar graphs. Bars (left to right) represent CONTRADICTION, UNKNOWN, and ENTAILMENT While these generalizations are intuitive, there alleged is non-subsective, so there are many pos- is little experimental evidence to support them. sible worlds in which an alleged thief is not in fact In this paper, we collect human judgements of a thief. Of course, there may also be many possi- the validity of inferences following from the in- ble worlds in which the alleged thief is a thief, but sertion and deletion of various classes of adjec- the word alleged, being non-subsective, does not tives and analyze the results. Our findings suggest guarantee this to hold. that, in practice, most sentences involving non- Non-subsective adjectives can be further di- subsective ANs can be safely generalized to state- vided into two classes: privative and plain. Sets ments about the N. That is, non-subsective adjec- denoted by privative ANs are completely disjoint tives often behave like normal, subsective adjec- from the set denoted by the head N (AN N= \ tives. On further analysis, we reveal that, when ), and this mutual exclusivity is encoded in the ; adjectives do behave non-subsectively, they often meaning of the A itself. For example, fake is con- exhibit asymmetric entailment behavior in which sidered to be a quintessential privative adjective insertion leads to contradictions (ID fake ID) )¬ since, given the usual definition of fake,afake ID but deletion leads to entailments (fake ID ID). ) can not actually be an ID. For plain non-subsective We present anecdotal evidence for how the en- adjectives, there may be worlds in which the AN tailment associated with inserting/deleting a non- is and N, and worlds in which the AN is not an N: subsective adjective depends on the salient prop- neither inference is guaranteed by the meaning of erties of the noun phrase under discussion, rather the A. As mentioned above, alleged is quintessen- than on the adjective itself. tially plain non-subsective since, for example, an alleged thief may or may not be an actual thief. 2 Background and Related Work In short, we can summarize the classes of adjec- Classes of Adjectives. Adjectives are com- tives in the following way: subsective adjectives monly classified taxonomically as either subsec- entail the nouns they modify, privative adjectives tive or non-subsective (Kamp and Partee, 1995). contradict the nouns they modify, and plain non- Subsective adjectives are adjectives which pick subsective adjectives are compatible with (but do out a subset of the set denoted by the unmodified not entail) the nouns they modify. Figure 1 depicts noun; that is, AN N1. For non-subsective adjec- these distinctions. ⇢ tives, in contrast, the AN cannot be guaranteed to While the hierarchical classification of adjec- be a subset of N. For example, clever is subsective, tives described above is widely accepted and often and so a clever thief is always a thief. However, applied in NLP tasks (Amoia and Gardent, 2006; Amoia and Gardent, 2007; Boleda et al., 2012; 1We use the notation N and AN to refer both the the nat- ural language expression itself (e.g. red car) as well as its McCrae et al., 2014), it is not undisputed. Some denotation, e.g. x x is a red car . linguists take the position that in fact privative ad- { | } jectives are simply another type of subsective ad- through the Annotated Gigaword corpus (Napoles jective (Partee, 2003; McNally and Boleda, 2004; et al., 2012) for occurrences of each adjective in Abdullah and Frost, 2005; Partee, 2007). Advo- the list, restricting to cases in which the adjective cates of this theory argue that the denotation of appears as an adjective modifier of (is in an amod the noun should be expanded to include both the dependency relation with) a common noun (NN). properties captured by the privative adjectives as For each adjective, we choose 10 sentences such well as those captured by the subsective adjec- that the adjective modifies a different noun in each. tives. This expanded denotation can explain the As a control, we take a small sample 100 ANs cho- acceptability of the sentence Is that gun real or sen randomly from our corpus. We expect these to fake?, which is difficult to analyze if gun entails contain almost entirely subsective adjectives. fake gun. More recent theoretical work argues For each selected sentence s, we generate s by ¬ 0 that common nouns have a “dual semantic struc- deleting the non-subsective adjective from s.We ture” and that non-subsective adjectives modify then construct two RTE problems, one in which part of this meaning (e.g. the functional features p = s and h = s0 (the deletion direction), and one of the noun) without modifying the extension of in which p = s0 and h = s (the insertion direc- the noun (Del Pinal, 2015). Such an analysis can tion). For each RTE problem, we ask annotators explain how we can understand a fake gun as hav- to indicate on a 5-point scale how likely it is that ing many, but not all, of the properties of a gun. p entails h, where a score of -2 indicates definite Several other studies abandon the attempt to or- contradiction and a score of 2 indicates definite en- ganize adjectives taxonomically, and instead focus tailment. We use Amazon Mechanical Turk, re- on the properties of the modified noun. Nayak quiring annotators to pass a qualification test of et al. (2014) categorize non-subsective adjectives simple RTE problems before participating. We so- in terms of the proportion of properties that are licit 5 annotators per p/h pair, taking the majority shared between the N and the AN and Puste- answer as truth. Workers show moderate agree- jovsky (2013) focus on syntactic cues about ex- ment on the 5-way classification ( =0.44). actly which properties are shared. Bakhshandh Disclaimer. This design does not directly test and Allen (2015) analyze adjectives by observing the taxonomic properties of non-subsective ANs. that, e.g., red modifies color while tall modifies Rather than asking “Is this instance of AN an in- size.