Exploiting a Verb Lexicon in Automatic Semantic Role Labelling Robert S. Swier and Suzanne Stevenson Department of Computer Science University of Toronto Toronto, Ontario, Canada M5S 3G4 {swier,suzanne}@cs.toronto.edu Abstract The reliance on such a resource—one in which the arguments of each predicate are manually identified We develop an unsupervised semantic role and assigned a semantic role—limits the portability labelling system that relies on the direct of such methods to other languages or even to other application of information in a predicate genres of corpora. lexicon combined with a simple probabil- In this study, we explore the possibility of using a ity model. We demonstrate the usefulness verb lexicon, rather than a hand-labelled corpus, as of predicate lexicons for role labelling, the primary resource in the semantic role labelling as well as the feasibility of modifying an task. Perhaps because of the focus on what can existing role-labelled corpus for evaluat- be gleaned from labelled data, existing supervised ing a different set of semantic roles. We approaches have made little use of the additional achieve a substantial improvement over an knowledge available in the predicate lexicon asso- informed baseline. ciated with the labelled corpus. By contrast, we ex- ploit the explicit knowledge of the role assignment 1 Introduction possibilities for each verb within an existing lexi- Intelligent language technologies capable of full con. Moreover, we utilize a very simple probability semantic interpretation of domain-general text re- model within a highly efficient algorithm. main an elusive goal. However, statistical advances We use VerbNet (Kipper et al., 2000), a computa- have made it possible to address core pieces of tional lexicon which lists the possible semantic role the problem. Recent years have seen a wealth of assignments for each of its verbs. Our algorithm research on one important component of seman- extracts automatically parsed arguments from a cor- tic interpretation—automatic role labelling (e.g., pus, and assigns to each a list of the compatible roles Gildea and Jurafsky, 2002; Pradhan et al., 2004; Ha- according to VerbNet. Arguments which are given cioglu et al., 2004, and additional papers from Car- only a single role possibility are considered to have reras and Marquez, 2004). Such work aims to an- been assigned an unambiguous role label. This set notate each constituent in a clause with a semantic of arguments constitutes our primary-labelled data, tag indicating the role that the constituent plays with which serves as the noisy training data for a simple respect to the target predicate, as in (1): probability model which is then used to label the re- (1) [Yuka]Agent [whispered]Pred to [Dar]Recipient maining (role ambiguous) arguments. Semantic role labelling systems address a crucial This method has several advantages, the foremost first step in the automatic extraction of semantic re- of which is that it eliminates the dependence on a lations from domain-general text, taking us closer to role labelled corpus, a very expensive resource to the goal of comprehensive semantic mark-up. produce. Of course, a verb lexicon is also an expen- Most work thus far on domain-general role la- sive resource, but one that is highly reusable across a belling depends on supervised learning over statis- range of NLP tasks. Moreover, the approach points tical features extracted from a hand-labelled corpus. at some potentially useful information that current 883 Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (HLT/EMNLP), pages 883–890, Vancouver, October 2005. c 2005 Association for Computational Linguistics supervised methods have failed to exploit. Even if whisper one has access to an annotated corpus for training, Frames: our work shows that directly calling on additional Agent V information from the lexicon itself may prove useful Agent V Prep(+dest) Recipient Agent V Topic in restricting the possible labels for an argument. Verbs in same (sub)class: [bark, croon, drone, grunt, holler, ...] The method has disadvantages as well. The in- formation available in a predicate lexicon is less di- Figure 1: A portion of a VerbNet entry. rectly applicable to building a learning model. In- evitably, our results are noisier than in a super- vised approach which has access to a labelled sam- 2.1 The VerbNet Lexicon ple of what it must produce. Still, the method shows VerbNet is a manually developed hierarchical lexi- promise: on unseen test data, the system yields an con based on the verb classification of Levin (1993). F-measure of .83 on labelling of correctly extracted For each of almost 200 classes containing a total of arguments, compared to an informed baseline of .74, 3000 verbs, VerbNet specifies the syntactic frames and an F-measure of .65 (compared to .52) on the along with the semantic role assigned to each argu- overall identification and labelling task. The latter is ment position of a frame.1 Figure 1 shows an exam- well below the best supervised performance of about ple VerbNet entry. The thematic roles used in Verb- .80 on similar tasks, but it must be emphasized that Net are more general than the situation-specific roles it is achieved with a simple probability model and of FrameNet. For example, the roles Speaker, Mes- without the use of hand-labelled data. We view this sage, and Addressee of a Communication verb such as a starting point by which to demonstrate the util- as whisper in FrameNet would be termed Agent, ity of deriving more explicit knowledge from a pred- Topic, and Recipient in VerbNet. These coarser- icate lexicon, which can be later extended through grained roles are often assumed in linguistic the- the use of additional probabilistic features. ory, and have some advantages in terms of capturing We face a methodological challenge arising from commonalities of argument relations across a wide the particular choice of VerbNet for the prototyp- range of predicates. ing of our method: the lexicon has no associated 2.2 Mapping FrameNet to VerbNet Roles semantic role labelled corpus. While this under- scores the need for approaches which do not rely As noted, VerbNet lacks a corpus of example role as- on such a resource, it also means that we lack a signments against which to evaluate a role labelling labelled sample of data against which to evaluate based upon it. We create such a resource by adapting our results. To address this, we use the existing the existing FrameNet corpus. We formulate a map- labelled corpus of FrameNet (Baker et al., 1998), ping between FrameNet’s larger role set and Verb- and develop a mapping for converting the FrameNet Net’s much smaller one, and create a new corpus roles to corresponding VerbNet roles. Our mapping with our mapped roles substituted for the original method demonstrates the possibility of leveraging roles in the FrameNet corpus. existing resources to support the development of role We perform the mapping in three steps. First we labelling systems based on verb lexicons that do not use an existing mapping between the semantically- have an associated hand-labelled corpus. specific roles in FrameNet and a much smaller inter- mediate set of 39 semantic roles which subsume all FrameNet roles.2 The associations in this mapping 2 VerbNet Roles and the Role Mapping are straightforward—e.g., the Place role for Abusing verbs and the Area role for Operate-vehicle verbs are both mapped to Location. Before describing our labelling algorithm, we first 1 briefly introduce the semantic role information Throughout the paper we use the term “frame” to refer to a syntactic frame—a configuration of syntactic arguments of a available in VerbNet, and describe how we map verb—possibly labelled with roles, as in Figure 1. FrameNet roles to VerbNet roles. 2This mapping was provided by Roxana Girju, UIUC. 884 Second, from this intermediate set we create a to train a probability model, described in Section 4, simple mapping to the set of 22 VerbNet roles. Some which we employ to label the remaining arguments roles are unaffected by the mapping (e.g., Cause (those having more than one candidate role). alone in the intermediate set maps to Cause in the VerbNet set). Other roles are merged (e.g., Degree 3.1 Initialization of Candidate Roles and Measure both map to Amount). Moreover, some The frame matcher construes extracted arguments roles in FrameNet (and the intermediate set) must be from the parsed sentence as being in one of the mapped to more than one VerbNet role. For exam- four main types of syntactic positions (or slots) used ple, an Experiencer role in FrameNet is considered by VerbNet frames: subject, object, indirect object, Experiencer by some VerbNet classes, but Agent by and PP-object.3 Additionally, we specialize the lat- others. In such cases, our mappings in this step must ter by the individual preposition, such as “object of be specific to the VerbNet class. for.” For the first three slot types, alignment be- In this second step, some roles have no subsum- tween the extracted arguments and the frames is rel- ing VerbNet role, because FrameNet provides roles atively straightforward. An extracted subject would for a wider variety of relations. For example, both be aligned with the subject position in a VerbNet FrameNet and the intermediate role set contain a frame, for instance, and the subject role from the Manner role, which VerbNet does not have. We frame would be listed as a possible label for the ex- create a catch-all label, “NoRole,” to which we tracted subject. map eight such intermediate roles: Condition, Man- The alignment of PP-objects is similar to that ner, Means, Medium, Part-Whole, Property, Pur- of the other slot types, except that we add an ad- pose, and Result.