Probabilistic Frame-Semantic Parsing Dipanjan Das Nathan Schneider Desai Chen Noah A
Total Page:16
File Type:pdf, Size:1020Kb
Probabilistic Frame-Semantic Parsing Dipanjan Das Nathan Schneider Desai Chen Noah A. Smith School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA fdipanjan@cs,nschneid@cs,desaic@andrew,[email protected] Abstract TRANSITIVE_ CAUSE_TO_ ACTION MAKE_NOISE MAKE_NOISE Event Purpose Sound This paper contributes a formalization of frame-semantic parsing as a structure predic- Place Place Place tion problem and describes an implemented Time Time Time parser that transforms an English sentence Agent Agent Noisy_event into a frame-semantic representation. It finds Cause Cause Sound_source cough.v, gobble.v, words that evoke FrameNet frames, selects Patient Sound_maker ring.v, yodel.v, ... blare.v, play.v, frames for them, and locates the arguments — for each frame. The system uses two feature- ring.v, toot.v, ... based, discriminative probabilistic (log-linear) Inheritance relation Causative_of relation models, one with latent variables to permit Excludes relation disambiguation of new predicate words. The Figure 2. Partial illustration of frames, roles, and LUs parser is demonstrated to significantly outper- related to the CAUSE TO MAKE NOISE frame, from the form previously published results. FrameNet lexicon. “Core” roles are filled ovals. 8 addi- tional roles of CAUSE TO MAKE NOISE are not shown. 1 Introduction and a precision-boosting constraint that forbids ar- FrameNet (Fillmore et al., 2003) is a rich linguistic guments of the same predicate to overlap. Our parser resource containing considerable information about achieves the best published results to date on the lexical and predicate-argument semantics in En- SemEval’07 FrameNet task (Baker et al., 2007). glish. Grounded in the theory of frame semantics 2 Resources and Task (Fillmore, 1982), it suggests—but does not formally define—a semantic representation that blends word- We consider frame-semantic parsing resources. sense disambiguation and semantic role labeling. 2.1 FrameNet Lexicon In this paper, we present a computational and statistical model for frame-semantic parsing, the The FrameNet lexicon is a taxonomy of manu- 1 problem of extracting from text semantic predicate- ally identified general-purpose frames for English. argument structures such as those shown in Fig. 1. Listed in the lexicon with each frame are several We aim to predict a frame-semantic representation lemmas (with part of speech) that can denote the as a structure, not as a pipeline of classifiers. We frame or some aspect of it—these are called lexi- use a probabilistic framework that cleanly integrates cal units (LUs). In a sentence, word or phrase to- the FrameNet lexicon and (currently very limited) kens that evoke a frame are known as targets. The available training data. Although our models often set of LUs listed for a frame in FrameNet may not involve strong independence assumptions, the prob- be exhaustive; we may see a target in new data that abilistic framework we adopt is highly amenable to does not correspond to an LU for the frame it evokes. future extension through new features, relaxed in- Each frame definition also includes a set of frame el- dependence assumptions, and semisupervised learn- ements, or roles, corresponding to different aspects ing. Some novel aspects of our current approach of the concept represented by the frame, such as par- include a latent-variable model that permits disam- ticipants, props, and attributes. We use the term ar- biguation of words not in the FrameNet lexicon, a 1Like the SemEval’07 participants, we used FrameNet v. 1.3 unified model for finding and labeling arguments, (http://framenet.icsi.berkeley.edu). But there still are n't enough ringers to ring more than six of the eight bells . Frame LU N_m NOISE_MAKERS bell.n Agent Sound_maker CAUSE_TO_MAKE_NOISE ring.v UFFICIENCY Item Enabled_situation S enough.a XISTENCE Entity E there be.v Figure 1. A sentence from PropBank and the SemEval’07 training data, and a partial depiction of gold FrameNet annotations. Each frame is a row below the sentence (ordered for readability). Thick lines indicate targets that evoke frames; thin solid/dotted lines with labels indicate arguments. “N m” under bells is short for the Noise maker role of the NOISE MAKERS frame. The last row indicates that there. are is a discontinuous target. In PropBank, the verb ring is the only annotated predicate for this sentence, and it is not related to other predicates with similar meanings. FRAMENET LEXICON V. 1.3 FULL-TEXT SemEval’07 data lexical exemplars ANNOTATIONS train dev test entries counts coverage Size (words sentences documents) 8379 LUs 139K sentences, 3.1M words 70% LUs all 43.3K1.7K 22 6.3K 251 4 2.8K 120 3 795 frames 1 frame annotation / sentence 63% frames ANC (travel) 3.9K 154 2 .8K 32 1 1.3K 67 1 7124 roles 285K overt arguments 56% roles NTI (bureaucratic) 32.2K1.2K 15 5.5K 219 3 1.5K 53 2 PropBank (news) 7.3K 325 5 0 0 0 0 0 0 Table 1. Snapshot of lexicon entries and exemplar sen- Annotations (frames/word overt arguments/word) tences. Coverage indicates the fraction of types attested all 0.23 0.39 0.22 0.37 0.37 0.65 in at least one exemplar. Coverage of lexicon (% frames % roles % LUs) gument to refer to a sequence of word tokens anno- all 64.1 27.4 21.0 34.0 10.2 7.3 29.3 7.7 4.9 tated as filling a frame role. Fig. 1 shows an exam- Out-of-lexicon types (frames roles LUs) ple sentence from the training data with annotated all 14 69 71 2 4 2 39 99 189 targets, LUs, frames, and role-argument pairs. The Out-of-lexicon tokens (% frames % roles % LUs) FrameNet lexicon also provides information about all 0.7 0.9 1.1 1.0 0.4 0.2 9.8 11.2 25.3 relations between frames and between roles (e.g., Table 2. Snapshot of the SemEval’07 annotated data. INHERITANCE). Fig. 2 shows a subset of the rela- tions between three frames and their roles. lectively as the SemEval’07 data.2 For the most Accompanying most frame definitions in the part, the frames and roles used in annotating these FrameNet lexicon is a set of lexicographic exemplar documents were defined in the FrameNet lexicon, sentences (primarily from the British National Cor- but there are some exceptions for which the annota- pus) annotated for that frame. Typically chosen to il- tors defined supplementary frames and roles; these lustrate variation in argument realization patterns for are included in the possible output of our parser. the frame in question, these sentences only contain Table 2 provides a snapshot of the SemEval’07 annotations for a single frame. We found that using data. We randomly selected three documents from exemplar sentences directly to train our models hurt the original SemEval training data to create a devel- performance as evaluated on SemEval’07 data, even opment set for tuning model hyperparameters. No- though the number of exemplar sentences is an order tice that the test set contains more annotations per of magnitude larger than the number of sentences in word, both in terms of frames and arguments. More- our training set (x2.2). This is presumably because over, there are many more out-of-lexicon frame, the exemplars are neither representative as a sample role, and LU types in the test set than in the training nor similar to the test data. Instead, we make use of set. This inconsistency in the data results in poor re- these exemplars in features (x4.2). call scores for all models trained on the given data 2.2 Data split, a problem we have not sought to address here. Our training, development, and test sets consist of documents annotated with frame-semantic struc- 2http://framenet.icsi.berkeley.edu/ tures for the SemEval’07 task, which we refer to col- semeval/FSSE.html Preprocessing. We preprocess sentences in our TARGET IDENTIFICATION PR F1 dataset with a standard set of annotations: POS Our technique (x3) 89.92 70.79 79.21 tags from MXPOST (Ratnaparkhi, 1996) and depen- Baseline: J&N’07 87.87 67.11 76.10 dency parses from the MST parser (McDonald et al., Table 3. Target identification results for our system and 2005) since manual syntactic parses are not available the baseline. Scores in bold denote significant improve- for most of the FrameNet-annotated documents. We ments over the baseline (p < 0:05). used WordNet (Fellbaum, 1998) for lemmatization. used a collection of separate SVM classifiers—one We also labeled each verb in the data as having AC- for each frame—to predict a single evoked frame for TIVE or PASSIVE voice, using code from the SRL each occurrence of a word in the extended set. system described by Johansson and Nugues (2008). J&N’07 modeled the argument identification 2.3 Task and Evaluation problem by dividing it into two tasks: first, they Automatic annotations of frame-semantic structure classified candidate spans as to whether they were can be broken into three parts: (1) targets, the words arguments or not; then they assigned roles to those or phrases that evoke frames; (2) the frame type, that were identified as arguments. Both phases used defined in the lexicon, evoked by each target; and SVMs. Thus, their formulation of the problem in- (3) the arguments, or spans of words that serve to volves a multitude of classifiers—whereas ours uses fill roles defined by each evoked frame. These cor- two log-linear models, each with a single set of respond to the three subtasks in our parser, each weights, to find a full frame-semantic parse.