DOCSLIB.ORG

Evaluating Framenet-Style Semantic Parsing: the Role of Coverage Gaps in Framenet

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
Evaluating Framenet-Style Semantic Parsing: the Role of Coverage Gaps in Framenet Evaluating FrameNet-style semantic parsing: the role of coverage gaps in FrameNet Alexis Palmer and Caroline Sporleder Computational Linguistics Saarland University apalmer, csporled @coli.uni-saarland.de { } Abstract limited coverage of the available hand-coded data against which to evaluate. More realistic evalua- Supervised semantic role labeling (SRL) tions test systems on full text, but these same cov- systems are generally claimed to have ac- erage limitations mean that the assumptions made curacies in the range of 80% and higher in more restricted evaluations do not necessarily (Erk and Pado,´ 2006). These numbers, hold for full text. This paper provides an analysis though, are the result of highly-restricted of the extent and nature of the coverage gaps in evaluations, i.e., typically evaluating on FrameNet. A more precise understanding of the hand-picked lemmas for which training limitations of existing resources with respect to data is available. In this paper we con- robust semantic analysis of texts is an important sider performance of such systems when foundational component both for improving ex- we evaluate at the document level rather isting systems and for developing future systems, than on the lemma level. While it is well- and it is in this spirit that we make our analysis. known that coverage gaps exist in the re- sources available for training supervised Full-text semantic analysis SRL systems, what we have been lacking Automated frame-semantic analysis aims to ex- until now is an understanding of the pre- tract from text the key event-denoting predicates cise nature of this coverage problem and and the semantic argument structure for those its impact on the performance of SRL sys- predicates. The semantic argument structure of tems. We present a typology of five differ- a predicate describing an event encodes relation- ent types of coverage gaps in FrameNet. ships between the participants involved in the We then analyze the impact of the cov- event, e.g. who did what to whom. Knowledge of erage gaps on performance of a super- semantic argument structure is essential for lan- vised semantic role labeling system on full guage understanding and thus important for ap- texts, showing an average oracle upper plications such as information extraction (Mos- bound of 46.8%. chitti et al., 2003; Surdeanu et al., 2003), ques- tion answering (Shen and Lapata, 2007), or recog- 1 Introduction nizing textual entailment (Burchardt et al., 2009). A lot of progress has been made in semantic Evaluating an existing system for its ability to aid role labeling over the past years, but the per- such tasks is unrealistic if the evaluation is lemma- formance of state-of-the-art systems is still rel- based rather than text-based. Consequently, there atively low, especially for deep, FrameNet-style continues to be significant interest in developing semantic parsing. Furthermore, many of the re- semantic role labeling (SRL) systems able to au- ported performance figures are somewhat unre- tomatically compute the semantic argument struc- alistic because system performance is evaluated tures in an input text. on hand-selected lemmas, usually under the im- Performance on the full text task, though, is plicit assumptions that (i) all relevant word senses typically much lower than for the more restricted (frames) of each lemma are known, and (ii) there evaluations. The SemEval 2007 Task on “Frame is a suitable amount of training data for each Semantic Structure Extraction,” for example, re- sense. This approach to evaluation arises from the quired systems to identify key predicates in texts, 928 Coling 2010: Poster Volume, pages 928–936, Beijing, August 2010 assign a semantic frame to the relevant predi- it might require obtaining an analysis of the word cates, identify the semantic arguments for the tried. predicates, and finally label those arguments with (1) [Captain Thomas Preston] their semantic roles. The systems participating Defendanti was for in this task only obtained F-Scores between 55% triedTry defendanti [murder] . and 78% for frame assignment, despite the fact Chargesi,j that the task organizers adopted a lenient evalu- In the end [he] was ation scheme which gave partial credit for near- Defendantj misses (Baker et al., 2007). For the combined task [Ø] . acquittedVerdictj Chargesj of frame assigment and role labeling the perfor- mance was even lower, ranging from 35% to 54% Performance levels obtained for full text are F-Score. usually not sufficient for this kind of real-world Note that this distinction between evaluation task. FrameNet-style semantic role labeling has schemes for SRL systems corresponds to the dis- been shown to, in principle, be beneficial for ap- tinction between “lexical sample” and “all words” plications that need to generalise over individual evaluations in word sense disambiguation, where lemmas, such as recognizing textual entailment or results for the latter scheme are also typically question answering. However, studies also found lower (McCarthy, 2009). that state-of-the-art FrameNet-style SRL systems The low performances are at least partly due perform too poorly to provide any substantial ben- to coverage problems. For example, Baker et efit to real applications (Burchardt et al., 2009; al. (2007) annotated three new texts for their Shen and Lapata, 2007). SemEval 2007 task. Although these new texts Extending the value of automated semantic overlap in domain with existing FrameNet data, parsing for a variety of applications requires im- the task organizers had to create 40 new frames proving the ability of systems to process unre- in order to complete annotation. The new frames stricted text. Several methods have been pro- were for word senses found in the test set but posed to address different aspects of the cover- missing from FrameNet. The test set contained age problem, ranging from automatic data expan- only 272 frames (types), meaning that nearly 15% sion and semi-supervised semantic role labelling of the frames therein were not yet defined in (Furstenau¨ and Lapata, 2009b; Furstenau¨ and La- FrameNet. Obviously, coverage issues of this de- pata, 2009a; Deschacht and Moens, 2009; Gordon gree make full SRL a difficult task, but this is a and Swanson, 2007; Pado´ et al., 2008) to systems realistic scenario that will be encountered in real which can infer missing word senses (Pennac- applications as well. chiotti et al., 2008b; Pennacchiotti et al., 2008a; As mentioned above, for many tasks it is neces- Cao et al., 2008; Burchardt et al., 2005). How- sary to compute the semantic argument structures ever, so far there has not been a detailed analysis for the whole text, or at least for multi-sentence of the problem. In this paper we provide that de- passages. Due to non-local relations between ar- tailed analysis, by defining different types of cov- gument structures this is also true for tasks like erage problems and performing analysis of both question answering, where it might be possible coverage and performance of an automated SRL to automatically determine a subset of lemmas system on three different data sets. which are relevant for the task. For example, in (1) Section 2 of the paper provides an introduction it might be possible to determine that the second to FrameNet and introduces the basic terminol- sentence contains the answer to the question “Was ogy. Section 4 describes our approach to coverage Thomas Preston acquitted of theft?” However, evaluation, Section 3 discusses the texts analyzed, to correctly answer this question, it is necessary and the analysis itself appears in Section 5. Sec- to resolve the null instantiation of the CHARGES tion 6 then looks at one possibility for addressing role of the VERDICT frame. This null instantiation the coverage problem. The final section presents links back to the previous sentence, and resolving some discussion and conclusions. 929 (a) (b) Figure 1: Terminology: (a) Frame with core frame elements (FEs) and frame-evoking elements (FEEs) (b) Target with possible frame assignments and resultant lexical units (LUs) 2 FrameNet semantic argument annotations for nouns, adjec- tives, adverbs, prepositions and even multi-word Manual annotation of corpora with semantic ar- expressions. For example, the sentence in (2) and gument structure information has enabled the de- the NP in (3) have identical argument structures velopment of statistical and supervised machine because the verb speak and the noun comment learning techniques for semantic role labeling evoke the same frame STATEMENT. (Toutanova et al., 2008; Moschitti et al., 2008; Gildea and Jurafsky, 2002). (2) [The politician]Speaker spokeStatement The two main resources are PropBank (Palmer [about recent developments on the labour et al., 2005) and FrameNet (Ruppenhofer et al., market]T opic. 2006). PropBank aims to provide a semantic role (3) [The politician’s] com- annotation for every verb in the Penn TreeBank Speaker mentsStatement [on recent developments (Marcus et al., 1994) and assigns roles on a verb- on the labour market] by-verb basis, without making higher-level gener- T opic alizations. Whether two distinct usages of a given Since FrameNet annotations are semanti- verb are viewed as different senses or not is thus cally driven they are considerably more time- driven by both syntax (namely, differences in syn- consuming to create than PropBank annotations. tactic argument structure) and semantics (via ba- However, FrameNet also provides ‘deeper’ and sic, easily-discernable differences in meaning). more informative annotations than PropBank FrameNet1 is a lexicographic project whose analyses (Ellsworth et al., 2004). For instance, aim it is to create a lexical resource documenting the fact that (2) and (3) refer to the same state- valence structures for different word senses and of-affairs is not captured by PropBank sense dis- their possible mappings to underlying semantic tinctions. argument structure (Ruppenhofer et al., 2006). In contrast to PropBank, FrameNet is primarily se- FrameNet Terminology mantically driven; word senses (frames)2 are de- The English FrameNet data consist of an inven- fined mainly based on sometimes-subtle meaning tory of frames (i.e.
Load more

Recommended publications
  • Enriching an Explanatory Dictionary with Framenet and Propbank Corpus Examples
    Proceedings of eLex 2019 Enriching an Explanatory Dictionary with FrameNet and PropBank Corpus Examples Pēteris Paikens 1, Normunds Grūzītis 2, Laura Rituma 2, Gunta Nešpore 2, Viktors Lipskis 2, Lauma Pretkalniņa2, Andrejs Spektors 2 1 Latvian Information Agency LETA, Marijas street 2, Riga, Latvia 2 Institute of Mathematics and Computer Science, University of Latvia, Raina blvd. 29, Riga, Latvia E-mail: [email protected], [email protected] Abstract This paper describes ongoing work to extend an online dictionary of Latvian – Tezaurs.lv – with representative semantically annotated corpus examples according to the FrameNet and PropBank methodologies and word sense inventories. Tezaurs.lv is one of the largest open lexical resources for Latvian, combining information from more than 300 legacy dictionaries and other sources. The corpus examples are extracted from Latvian FrameNet and PropBank corpora, which are manually annotated parallel subsets of a balanced text corpus of contemporary Latvian. The proposed approach augments traditional lexicographic information with modern cross-lingually interpretable information and enables analysis of word senses from the perspective of frame semantics, which is substantially different from (complementary to) the traditional approach applied in Latvian lexicography. In cases where FrameNet and PropBank corpus evidence aligns well with the word sense split in legacy dictionaries, the frame-semantically annotated corpus examples supplement the word sense information with clarifying usage examples and commonly used semantic valence patterns. However, the annotated corpus examples often provide evidence of a different sense split, which is often more coarse-grained and, thus, may help dictionary users to cluster and comprehend a fine-grained sense split suggested by the legacy sources.
    [Show full text]
  • Developing a Large Scale Framenet for Italian: the Iframenet Experience
    Developing a large scale FrameNet for Italian: the IFrameNet experience Roberto Basili° Silvia Brambilla§ Danilo Croce° Fabio Tamburini§ ° § Dept. of Enterprise Engineering Dept. of Classic Philology and Italian Studies University of Rome Tor Vergata University of Bologna {basili,croce}@info.uniroma2.it [email protected], [email protected] ian Portuguese, German, Spanish, Japanese, Swe- Abstract dish and Korean. All these projects are based on the idea that English. This paper presents work in pro- most of the Frames are the same among languages gress for the development of IFrameNet, a and that, thanks to this, it is possible to adopt large-scale, computationally oriented, lexi- Berkeley’s Frames and FEs and their relations, cal resource based on Fillmore’s frame se- with few changes, once all the language-specific mantics for Italian. For the development of information has been cut away (Tonelli et al. 2009, IFrameNet linguistic analysis, corpus- Tonelli 2010). processing and machine learning techniques With regard to Italian, over the past ten years are combined in order to support the semi- several research projects have been carried out at automatic development and annotation of different universities and Research Centres. In par- the resource. ticular, the ILC-CNR in Pisa (e.g. Lenci et al. 2008; Johnson and Lenci 2011), FBK in Trento (e.g. Italiano. Questo articolo presenta un work Tonelli et al. 2009, Tonelli 2010) and the Universi- in progress per lo sviluppo di IFrameNet, ty of Rome, Tor Vergata (e.g. Pennacchiotti et al. una risorsa lessicale ad ampia copertura, 2008, Basili et al.
    [Show full text]
  • Integrating Wordnet and Framenet Using a Knowledge-Based Word Sense Disambiguation Algorithm
    Integrating WordNet and FrameNet using a knowledge-based Word Sense Disambiguation algorithm Egoitz Laparra and German Rigau IXA group. University of the Basque Country, Donostia {egoitz.laparra,german.rigau}@ehu.com Abstract coherent groupings of words belonging to the same frame. This paper presents a novel automatic approach to In that way we expect to extend the coverage of FrameNet partially integrate FrameNet and WordNet. In that (by including from WordNet closely related concepts), to way we expect to extend FrameNet coverage, to en- enrich WordNet with frame semantic information (by port- rich WordNet with frame semantic information and ing frame information to WordNet) and possibly to extend possibly to extend FrameNet to languages other than FrameNet to languages other than English (by exploiting English. The method uses a knowledge-based Word local wordnets aligned to the English WordNet). Sense Disambiguation algorithm for linking FrameNet WordNet1 [12] (hereinafter WN) is by far the most lexical units to WordNet synsets. Specifically, we ex- ploit a graph-based Word Sense Disambiguation algo- widely-used knowledge base. In fact, WN is being rithm that uses a large-scale knowledge-base derived used world-wide for anchoring different types of seman- from WordNet. We have developed and tested four tic knowledge including wordnets for languages other than additional versions of this algorithm showing a sub- English [4], domain knowledge [17] or ontologies like stantial improvement over previous results. SUMO [22] or the EuroWordNet Top Concept Ontology [3]. It contains manually coded information about En- glish nouns, verbs, adjectives and adverbs and is organized around the notion of a synset.
    [Show full text]
  • The Hebrew Framenet Project
    The Hebrew FrameNet Project Avi Hayoun, Michael Elhadad Dept. of Computer Science Ben-Gurion University Beer Sheva, Israel {hayounav,elhadad}@cs.bgu.ac.il Abstract We present the Hebrew FrameNet project, describe the development and annotation processes and enumerate the challenges we faced along the way. We have developed semi-automatic tools to help speed the annotation and data collection process. The resource currently covers 167 frames, 3,000 lexical units and about 500 fully annotated sentences. We have started training and testing automatic SRL tools on the seed data. Keywords: FrameNet, Hebrew, frame semantics, semantic resources 1. Introduction frames and their structures in natural language. Recent years have seen growing interest in the task 1.2. FrameNet in Other Languages of Semantic Role Labeling (SRL) of natural language The original FrameNet project has been adapted and text (sometimes called “shallow semantic parsing”). ported to multiple languages. The most active interna- The task is usually described as the act of identify- tional FrameNet teams include the Swedish FrameNet ing the semantic roles, which are the set of semantic (SweFN) covering close to 1,200 frames with 34K LUs properties and relationships defined over constituents (Ahlberg et al., 2014); the Japanese FrameNet (JFN) of a sentence, given a semantic context. with 565 frames, 8,500 LUs and 60K annotated ex- The creation of resources that document the realiza- ample sentences (Ohara, 2013); and FrameNet Brazil tion of semantic roles in natural language texts, such (FN-Br) covering 179 frames, 196 LUs and 12K anno- as FrameNet (Fillmore and Baker, 2010; Ruppenhofer tated sentences (Torrent and Ellsworth, 2013).
    [Show full text]
  • INF5830 Introduction to Semantic Role Labeling
    INF5830 Introduction to Semantic Role Labeling Andrey Kutuzov University of Oslo Language Technology Group With thanks to Lilja Øvrelid, Martha Palmer and Dan Jurafsky INF5830 Introduction to Semantic Role Labeling 1(36) Semantic Role Labeling INF5830 Introduction to Semantic Role Labeling 2(36) Introduction Contents Introduction Semantic roles in general PropBank: Proto-roles FrameNet: Frame Semantics Summary INF5830 Introduction to Semantic Role Labeling 2(36) Introduction Semantics I Study of meaning, expressed in language; I Morphemes, words, phrases, sentences; I Lexical semantics; I Sentence semantics; I (Pragmatics: how the context affects meaning). INF5830 Introduction to Semantic Role Labeling 3(36) Introduction Semantics I Linguistic knowledge: meaning I Meaningful or not: I Word { flick vs blick I Sentence { John swims vs John metaphorically every I Several meanings (WSD): I Words { fish I Sentence { John saw the man with the binoculars I Same meaning (semantic similarity): I Word { sofa vs couch I Sentence { John gave Hannah a gift vs John gave a gift to Hannah I Truth conditions: I All kings are male I Molybdenum conducts electricity I Entailment: I Alfred murdered the librarian I The librarian is dead I Participant roles: John is the `giver', Hannah is the `receiver' INF5830 Introduction to Semantic Role Labeling 4(36) Introduction Representing events I We want to understand the event described by these sentences: 1. IBM bought Spark 2. IBM acquired Spark 3. Spark was acquired by IBM 4. The owners of Spark sold it to IBM I Dependency parsing is insufficient. UDPipe will give us simple relations between verbs and arguments: 1. (buy, nsubj, IBM), (buy, obj, Spark) 2.
    [Show full text]
  • Using Frame Semantics in Natural Language Processing
    Using Frame Semantics in Natural Language Processing Apoorv Agarwal Daniel Bauer Owen Rambow Dept. of Computer Science Dept. of Computer Science CCLS Columbia University Columbia University Columbia University New York, NY New York, NY New York, NY [email protected] [email protected] [email protected] Abstract the notion of a social event (Agarwal et al., 2010), We summarize our experience using a particular kind of event which involves (at least) FrameNet in two rather different projects two people such that at least one of them is aware in natural language processing (NLP). of the other person. If only one person is aware We conclude that NLP can benefit from of the event, we call it Observation (OBS): for FrameNet in different ways, but we sketch example, someone is talking about someone else some problems that need to be overcome. in their absence. If both people are aware of the event, we call it Interaction (INR): for example, 1 Introduction one person is telling the other a story. Our claim We present two projects at Columbia in which we is that links in social networks are in fact made use FrameNet. In these projects, we do not de- up of social events: OBS social events give rise velop basic NLP tools for FrameNet, and we do to one-way links, and INR social events to two- not develop FramNets for new languages: we sim- way links. For more information, see (Agarwal ply use FrameNet or a FrameNet parser in an NLP and Rambow, 2010; Agarwal et al., 2013a; Agar- application.
    [Show full text]
  • Event-Based Knowledge Reconciliation Using Frame Embeddings And
    Knowledge-Based Systems 135 (2017) 192–203 Contents lists available at ScienceDirect Knowle dge-Base d Systems journal homepage: www.elsevier.com/locate/knosys Event-base d knowle dge reconciliation using frame emb e ddings and frame similarity ∗ Mehwish Alam a, Diego Reforgiato Recupero b,d, , Misael Mongiovi c, Aldo Gangemi a,d, Petar Ristoski e a Université Paris 13, 99 avenue JB Clément, Villetaneuse, Paris 93430, France b Università degli Studi di Cagliari, Department of Mathematics and Computer Science, Via Ospedale 72, Cagliari 09124, Italy c CNR, ISTC, Catania, Italy d CNR, ISTC, Via S. Martino della Battaglia 44, Rome, Italy e University of Mannheim, Mannheim, Germany a r t i c l e i n f o a b s t r a c t Article history: This paper proposes an evolution over MERGILO, a tool for reconciling knowledge graphs extracted from Received 6 April 2017 text, using graph alignment and word similarity. The reconciled knowledge graphs are typically used Revised 9 August 2017 for multi-document summarization, or to detect knowledge evolution across document series. The main Accepted 14 August 2017 point of improvement focuses on event reconciliation i.e., reconciling knowledge graphs generated by text Available online 16 August 2017 about two similar events described differently. In order to gather a complete semantic representation of Keywords: events, we use FRED semantic web machine reader, jointly with Framester, a linguistic linked data hub Knowledge reconciliation represented using a novel formal semantics for frames. Framester is used to enhance the extracted event Frame semantics knowledge with semantic frames.
    [Show full text]
  • Out-Of-Domain Framenet Semantic Role Labeling
    Out-of-domain FrameNet Semantic Role Labeling Silvana Hartmann§†, Ilia Kuznetsov†, Teresa Martin§†, Iryna Gurevych§† Research Training Group AIPHES Ubiquitous§ Knowledge Processing (UKP) Lab Department† of Computer Science, Technische Universitat¨ Darmstadt http://www.ukp.tu-darmstadt.de Abstract on a range of benchmark datasets. This is crucial as the demand for semantic textual analysis of large- Domain dependence of NLP systems is one scale web data keeps growing. of the major obstacles to their application in large-scale text analysis, also restrict- Based on FrameNet (Fillmore et al., 2003), ing the applicability of FrameNet semantic FrameNet SRL extracts frame-semantic structures role labeling (SRL) systems. Yet, current on the sentence level that describe a specific FrameNet SRL systems are still only eval- situation centered around a semantic predicate, uated on a single in-domain test set. For often a verb, and its participants, typically the first time, we study the domain depen- syntactic arguments or adjuncts of the predicate. frame dence of FrameNet SRL on a wide range of The predicate is assigned a label, essentially benchmark sets. We create a novel test set a word sense label, that defines the situation and semantic roles for FrameNet SRL based on user-generated determines the of the participants. web text and find that the major bottleneck The following sentence from FrameNet provides Grinding for out-of-domain FrameNet SRL is the an example of the frame and its roles: frame identification step. To address this [The mill]Grinding cause grindsGrinding [the problem, we develop a simple, yet efficient malt]P atient [to grist]Result.
    [Show full text]
  • Verbnet Guidelines
    VerbNet Annotation Guidelines 1. Why Verbs? 2. VerbNet: A Verb Class Lexical Resource 3. VerbNet Contents a. The Hierarchy b. Semantic Role Labels and Selectional Restrictions c. Syntactic Frames d. Semantic Predicates 4. Annotation Guidelines a. Does the Instance Fit the Class? b. Annotating Verbs Represented in Multiple Classes c. Things that look like verbs but aren’t i. Nouns ii. Adjectives d. Auxiliaries e. Light Verbs f. Figurative Uses of Verbs 1 Why Verbs? Computational verb lexicons are key to supporting NLP systems aimed at semantic interpretation. Verbs express the semantics of an event being described as well as the relational information among participants in that event, and project the syntactic structures that encode that information. Verbs are also highly variable, displaying a rich range of semantic and syntactic behavior. Verb classifications help NLP systems to deal with this complexity by organizing verbs into groups that share core semantic and syntactic properties. VerbNet (Kipper et al., 2008) is one such lexicon, which identifies semantic roles and syntactic patterns characteristic of the verbs in each class and makes explicit the connections between the syntactic patterns and the underlying semantic relations that can be inferred for all members of the class. Each syntactic frame in a class has a corresponding semantic representation that details the semantic relations between event participants across the course of the event. In the following sections, each component of VerbNet is identified and explained. VerbNet: A Verb Class Lexical Resource VerbNet is a lexicon of approximately 5800 English verbs, and groups verbs according to shared syntactic behaviors, thereby revealing generalizations of verb behavior.
    [Show full text]
  • Rs and IOS Press
    ECAI 2014 57 T. Schaub et al. (Eds.) © 2014 The Authors and IOS Press. This article is published online with Open Access by IOS Press and distributed under the terms of the Creative Commons Attribution Non-Commercial License. doi:10.3233/978-1-61499-419-0-57 Effective and Robust Natural Language Understanding for Human-R obot Interaction Emanuele Bastianelli(‡), Giuseppe Castellucci(•), Danilo Croce(†), Roberto Basili(†), Daniele Nardi() (†) DII, (‡) DICII, (•) DIE - University of Rome Tor Vergata - Rome, Italy {bastianelli,castellucci}@ing.uniroma2.it, {basili,croce}@info.uniroma2.it () DIAG - Sapienza University of Rome - Rome, Italy [email protected] Abstract. Robots are slowly becoming part of everyday life, as knowledge). Moreover, in HRI, different scenarios need to be con- they are being marketed for commercial applications (viz. telepres- sidered. In some situations, e.g. rescue robotic tasks, the precision is ence, cleaning or entertainment). Thus, the ability to interact with crucial and no command misunderstanding is allowed. non-expert users is becoming a key requirement. Even if user ut- Recently, works in the interpretation of natural language instruc- terances can be efficiently recognized and transcribed by Automatic tions for robots in controlled environments have been focused on Speech Recognition systems, several issues arise in translating them specific subsets of the human language. The interpretation process into suitable robotic actions. In this paper, we will discuss both ap- is mainly carried out through the adoption of specific grammars that proaches providing two existing Natural Language Understanding describe the human language allowed. It gives, for example, a robotic workflows for Human Robot Interaction.
    [Show full text]
  • UMCC DLSI: Textual Similarity Based on Lexical-Semantic Features
    UMCC_DLSI: Textual Similarity based on Lexical-Semantic features Alexander Chávez, Antonio Fernández Orquín, Andrés Montoyo, Rafael Muñoz Héctor Dávila, Yoan Gutiérrez, Armando DLSI, University of Alicante Carretera de Collazo, José I. Abreu San Vicente S/N Alicante, Spain. DI, University of Matanzas {montoyo,rafael}@dlsi.ua.es Autopista a Varadero km 3 ½ Matanzas, Cuba. {alexander.chavez, tony, hector.davila, yoan.gutierrez, armando.collazo, jose.abreu}@umcc.cu necessarily a student). In addition, STS differs Abstract from TE and Paraphrase in that, rather than being a binary yes/no decision, STS is a This paper describes the specifications and results of UMCC_DLSI system, which similarity-graded notion (e.g. a student is more participated in the Semantic Textual similar to a person than a dog to a person). Similarity task (STS) of SemEval-2013. Our This graded bidirectional is useful for NLP supervised system uses different types of tasks such as Machine Translation (MT), lexical and semantic features to train a Information Extraction (IE), Question Bagging classifier used to decide the correct Answering (QA), and Summarization. Several option. Related to the different features we semantic tasks could be added as modules in the can highlight the resource ISR-WN used to STS framework, “such as Word Sense extract semantic relations among words and Disambiguation and Induction, Lexical the use of different algorithms to establish Substitution, Semantic Role Labeling, Multiword semantic and lexical similarities. In order to establish which features are the most Expression detection and handling, Anaphora appropriate to improve STS results we and Co-reference resolution, Time and Date 1 participated with three runs using different resolution and Named Entity, among others” set of features.
    [Show full text]
  • Exploring the Integration of Wordnet and Framenet
    Exploring the Integration of WordNet and FrameNet Egoitz Laparra German Rigau Montse Cuadros IXA group, UPV/EHU IXA group, UPV/EHU TALP Research Center, UPC Donostia, Spain Donostia, Spain Barcelona, Spain [email protected] [email protected] [email protected] Abstract However, building large and rich enough predi- cate models for broad–coverage semantic process- This paper presents a novel automatic approach to ing takes a great deal of expensive manual effort partially integrate FrameNet and WordNet. In that involving large research groups during long peri- way we expect to extend FrameNet coverage, to ods of development. Thus, the coverage of cur- enrich WordNet with frame semantic information rently available predicate-argument resources is and possibly to extend FrameNet to languages other still unsatisfactory. For example, (Burchardt et than English. The method uses a knowledge-based al., 2005) or (Shen and Lapata, 2007) indicate the Word Sense Disambiguation algorithm for match- limited coverage of FrameNet as one of the main ing the FrameNet lexical units to WordNet synsets. problems of this resource. In fact, FrameNet1.3 Specifically, we exploit a graph-based Word Sense covers around 10,000 lexical-units while for in- Disambiguation algorithm that uses a large-scale stance, WordNet3.0 contains more than 150,000 knowledge-base derived from existing resources. words. Furthermore, the same effort should be in- We have developed and tested additional versions vested for each different language (Subirats and of this algorithm showing substantial improvements Petruck, 2003). Following the line of previous over state-of-the-art results. Finally, we show some works (Shi and Mihalcea, 2005), (Burchardt et al., examples and figures of the resulting semantic re- 2005), (Johansson and Nugues, 2007), (Pennac- source.
    [Show full text]