DOCSLIB.ORG

TINE: a Metric to Assess MT Adequacy

Total Page：16

File Type：pdf, Size：1020Kb

Download full-text PDF Read full-text
TINE: a Metric to Assess MT Adequacy TINE: A Metric to Assess MT Adequacy Miguel Rios, Wilker Aziz and Lucia Specia Research Group in Computational Linguistics University of Wolverhampton Stafford Street, Wolverhampton, WV1 1SB, UK {m.rios, w.aziz, l.specia}@wlv.ac.uk Abstract The most commonly used metrics, BLEU (Pap- ineni et al., 2002) and alike, perform simple exact We describe TINE, a new automatic evalua- tion metric for Machine Translation that aims matching of n-grams between hypothesis and refer- at assessing segment-level adequacy. Lexical ence translations. Such a simple matching proce- similarity and shallow-semantics are used as dure has well known limitations, including that the indicators of adequacy between machine and matching of non-content words counts as much as reference translations. The metric is based on the matching of content words, that variations of the combination of a lexical matching com- words with the same meaning are disregarded, and ponent and an adequacy component. Lexi- that a perfect matching can happen even if the order cal matching is performed comparing bags- of sequences of n-grams in the hypothesis and ref- of-words without any linguistic annotation. The adequacy component consists in: i) us- erence translation are very different, changing com- ing ontologies to align predicates (verbs), ii) pletely the meaning of the translation. using semantic roles to align predicate argu- A number of other metrics have been proposed ments (core arguments and modifiers), and to address these limitations, for example, by allow- iii) matching predicate arguments using dis- ing for the matching of synonyms or paraphrases tributional semantics. TINE’s performance is comparable to that of previous metrics of content words, such as in METEOR (Denkowski at segment level for several language pairs, and Lavie, 2010). Other attempts have been made with average Kendall’s tau correlation from to capture whether the reference translation and hy- 0.26 to 0.29. We show that the addition of pothesis translations share the same meaning us- the shallow-semantic component improves the ing shallow semantics, i.e., Semantic Role Labeling performance of simple lexical matching strate- (Gimenez´ and Marquez,´ 2007). However, these are gies and metrics such as BLEU. limited to the exact matching of semantic roles and their fillers. 1 Introduction We propose TINE, a new metric that comple- The automatic evaluation of Machine Translation ments lexical matching with a shallow semantic (MT) is a long-standing problem. A number of met- component to better address adequacy. The main rics have been proposed in the last two decades, contribution of such a metric is to provide a more mostly measuring some form of matching between flexible way of measuring the overlap between shal- the MT output (hypothesis) and one or more human low semantic representations that considers both the (reference) translations. However, most of these semantic structure of the sentence and the content metrics focus on fluency aspects, as opposed to ad- of the semantic elements. The metric uses SRLs equacy. Therefore, measuring whether the meaning such as in (Gimenez´ and Marquez,´ 2007). However, of the hypothesis and reference translation are the it analyses the content of predicates and arguments same or similar is still an understudied problem. seeking for either exact or “similar” matches. The 116 Proceedings of the 6th Workshop on Statistical Machine Translation, pages 116–122, Edinburgh, Scotland, UK, July 30–31, 2011. c 2011 Association for Computational Linguistics inexact matching is based on the use of ontologies matching features over dependency parses. The met- such as VerbNet (Schuler, 2006) and distributional ric then predicts the MT quality with a regression semantics similarity metrics, such as Dekang Lin’s model. The alignment is improved using ontologies. thesaurus (Lin, 1998) . He et al. (2010) measure the similarity between In the remainder of this paper we describe some hypothesis and reference translation in terms of related work (Section 2), present our metric - TINE the Lexical Functional Grammar (LFG) represen- - (Section 3) and its performance compared to pre- tation. The representation uses dependency graphs vious work (Section 4) as well as some further im- to generate unordered sets of dependency triples. provements. We then provide an analysis of these Calculating precision, recall, and F-score on the results and discuss the limitations of the metric (Sec- sets of triples corresponding to the hypothesis and tion 5) and present conclusions and future work reference segments allows measuring similarity at (Section 6). the lexical and syntactic levels. The measure also matches WordNet synonyms. 2 Related Work The closest related metric to the one proposed in this paper is that by Gimenez´ and Marquez´ (2007) A few metrics have been proposed in recent years and Gimenez´ et al. (2010), which also uses shallow to address the problem of measuring whether a hy- semantic representations. Such a metric combines a pothesis and a reference translation share the same number of components, including lexical matching meaning. The most well-know metric is probably metrics like BLEU and METEOR, as well as com- METEOR (Banerjee and Lavie, 2005; Denkowski ponents that compute the matching of constituent and Lavie, 2010). METEOR is based on a general- and dependency parses, named entities, discourse ized concept of unigram matching between the hy- representations and semantic roles. However, the se- pothesis and the reference translation. Alignments mantic role matching is based on exact matching of are based on exact, stem, synonym, and paraphrase roles and role fillers. Moreover, it is not clear what matches between words and phrases. However, the the contribution of this specific information is for the structure of the sentences is not considered. overall performance of the metric. Wong and Kit (2010) measure word choice and We propose a metric that uses a lexical similar- word order by the matching of words based on ity component and a semantic component in order surface forms, stems, senses and semantic similar- to deal with both word choice and semantic struc- ity. The informativeness of matched and unmatched ture. The semantic component is based on seman- words is also weighted. tic roles, but instead of simply matching the surface Liu et al. (2010) propose to match bags of uni- forms (i.e. arguments and predicates) it is able to grams, bigrams and trigrams considering both recall match similar words. and precision and F-measure giving more impor- tance to recall, but also using WordNet synonyms. 3 Metric Description Tratz and Hovy (2008) use transformations in or- der to match short syntactic units defined as Ba- The rationale behind TINE is that an adequacy- sic Elements (BE). The BE are minimal-length oriented metric should go beyond measuring the syntactically well defined units. For example, matching of lexical items to incorporate information nouns, verbs, adjectives and adverbs can be con- about the semantic structure of the sentence, as in sidered BE-Unigrams, while a BE-Bigram could be (Gimenez´ et al., 2010). However, the metric should formed from a syntactic relation (e.g. subject+verb, also be flexible to consider inexact matches of se- verb+object). BEs can be lexically different, but se- mantic components, similar to what is done with lex- mantically similar. ical metrics like METEOR (Denkowski and Lavie, Pado´ et al. (2009) uses Textual Entailment fea- 2010). We experiment with TINE having English tures extracted from the Standford Entailment Rec- as target language because of the availability of lin- ognizer (MacCartney et al., 2006). The Textual En- guistic processing tools for this language. The met- tailment Recognizer computes matching and mis- ric is particularly dependent on semantic role label- 117 ing systems, which have reached satisfactory perfor- get merged with modifiers due to bad semantic role mance for English (Carreras and Marquez,´ 2005). labeling (e.g. [A0 I] [T bought] [A1 something to eat TINE uses semantic role labels (SRL) and lexical se- yesterday] instead of [A0 I] [T bought] [A1 some- mantics to fulfill two requirements by: (i) compare thing to eat] [AM-TMP yesterday]). both the semantic structure and its content across matching arguments in the hypothesis and refer- P verb score(Hv,Rv) ence translations; and (ii) propose alternative ways A(H, R) = v∈V (3) of measuring inexact matches for both predicates |Vr| and role fillers. Additionally, it uses an exact lexi- In the adequacy component, V is the set of verbs cal matching component to reward hypotheses that aligned between H and R, and |Vr| is the number of present the same lexical choices as the reference verbs in R. Hereafter the indexes h and r stand for translation. The overall score s is defined using the hypothesis and reference translations, respectively. simple weighted average model in Equation (1): Verbs are aligned using VerbNet (Schuler, 2006) and VerbOcean (Chklovski and Pantel, 2004). A verb in αL(H, R) + βA(H, R) the hypothesis v is aligned to a verb in the refer- s(H, R) = max (1) h α + β R∈R ence vr if they are related according to the follow- ing heuristics: (i) the pair of verbs share at least one where H represents the hypothesis translation, R class in VerbNet; or (ii) the pair of verbs holds a re- represents a reference translation contained in the set lation in VerbOcean. of available references R; L defines the (exact) lex- For example, in VerbNet the verbs spook and ter- ical match component in Equation (2), A defines the rify share the same class amuse-31.1, and in VerbO- adequacy component in Equation (3); and α and β cean the verb dress is related to the verb wear. are tunable weights for these two components. If multiple references are provided, the score of the segment is the maximum score achieved by compar- P arg score(H ,R ) a∈Ar ∩At a a verb score(Hv,Rv) = ing the segment to each available reference.
Load more

Recommended publications
  • An Arabic Wordnet with Ontologically Clean Content
    Applied Ontology (2021) IOS Press The Arabic Ontology – An Arabic Wordnet with Ontologically Clean Content Mustafa Jarrar Birzeit University, Palestine [email protected] Abstract. We present a formal Arabic wordnet built on the basis of a carefully designed ontology hereby referred to as the Arabic Ontology. The ontology provides a formal representation of the concepts that the Arabic terms convey, and its content was built with ontological analysis in mind, and benchmarked to scientific advances and rigorous knowledge sources as much as this is possible, rather than to only speakers’ beliefs as lexicons typically are. A comprehensive evaluation was conducted thereby demonstrating that the current version of the top-levels of the ontology can top the majority of the Arabic meanings. The ontology consists currently of about 1,300 well-investigated concepts in addition to 11,000 concepts that are partially validated. The ontology is accessible and searchable through a lexicographic search engine (http://ontology.birzeit.edu) that also includes about 150 Arabic-multilingual lexicons, and which are being mapped and enriched using the ontology. The ontology is fully mapped with Princeton WordNet, Wikidata, and other resources. Keywords. Linguistic Ontology, WordNet, Arabic Wordnet, Lexicon, Lexical Semantics, Arabic Natural Language Processing Accepted by: 1. Introduction The importance of linguistic ontologies and wordnets is increasing in many application areas, such as multilingual big data (Oana et al., 2012; Ceravolo, 2018), information retrieval (Abderrahim et al., 2013), question-answering and NLP-based applications (Shinde et al., 2012), data integration (Castanier et al., 2012; Jarrar et al., 2011), multilingual web (McCrae et al., 2011; Jarrar, 2006), among others.
    [Show full text]
  • Verbnet Based Citation Sentiment Class Assignment Using Machine Learning
    (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 11, No. 9, 2020 VerbNet based Citation Sentiment Class Assignment using Machine Learning Zainab Amjad1, Imran Ihsan2 Department of Creative Technologies Air University, Islamabad, Pakistan Abstract—Citations are used to establish a link between time-consuming and complicated. To resolve this issue there articles. This intent has changed over the years, and citations are exists many researchers [7]–[9] who deal with the sentiment now being used as a criterion for evaluating the research work or analysis of citation sentences to improve bibliometric the author and has become one of the most important criteria for measures. Such applications can help scholars in the period of granting rewards or incentives. As a result, many unethical research to identify the problems with the present approaches, activities related to the use of citations have emerged. That is why unaddressed issues, and the present research gaps [10]. content-based citation sentiment analysis techniques are developed on the hypothesis that all citations are not equal. There are two existing approaches for Citation Sentiment There are several pieces of research to find the sentiment of a Analysis: Qualitative and Quantitative [7]. Quantitative citation, however, only a handful of techniques that have used approaches consider that all citations are equally important citation sentences for this purpose. In this research, we have while qualitative approaches believe that all citations are not proposed a verb-oriented citation sentiment classification for equally important [9]. The quantitative approach uses citation researchers by semantically analyzing verbs within a citation text count to rank a research paper [8] while the qualitative using VerbNet Ontology, natural language processing & four approach analyzes the nature of citation [10].
    [Show full text]
  • Towards a Cross-Linguistic Verbnet-Style Lexicon for Brazilian Portuguese
    Towards a cross-linguistic VerbNet-style lexicon for Brazilian Portuguese Carolina Scarton, Sandra Alu´ısio Center of Computational Linguistics (NILC), University of Sao˜ Paulo Av. Trabalhador Sao-Carlense,˜ 400. 13560-970 Sao˜ Carlos/SP, Brazil [email protected], [email protected] Abstract This paper presents preliminary results of the Brazilian Portuguese Verbnet (VerbNet.Br). This resource is being built by using other existing Computational Lexical Resources via a semi-automatic method. We identified, automatically, 5688 verbs as candidate members of VerbNet.Br, which are distributed in 257 classes inherited from VerbNet. These preliminary results give us some directions of future work and, since the results were automatically generated, a manual revision of the complete resource is highly desirable. 1. Introduction the verb to load. To fulfill this gap, VerbNet has mappings The task of building Computational Lexical Resources to WordNet, which has deeper semantic relations. (CLRs) and making them publicly available is one of Brazilian Portuguese language lacks CLRs. There are some the most important tasks of Natural Language Processing initiatives like WordNet.Br (Dias da Silva et al., 2008), that (NLP) area. CLRs are used in many other applications is based on and aligned to WordNet. This resource is the in NLP, such as automatic summarization, machine trans- most complete for Brazilian Portuguese language. How- lation and opinion mining. Specially, CLRs that treat the ever, only the verb database is in an advanced stage (it syntactic and semantic behaviour of verbs are very impor- is finished, but without manual validation), currently con- tant to the tasks of information retrieval (Croch and King, sisting of 5,860 verbs in 3,713 synsets.
    [Show full text]
  • Leveraging Verbnet to Build Corpus-Specific Verb Clusters
    Leveraging VerbNet to build Corpus-Specific Verb Clusters Daniel W Peterson and Jordan Boyd-Graber and Martha Palmer University of Colorado daniel.w.peterson,jordan.boyd.graber,martha.palmer @colorado.edu { } Daisuke Kawhara Kyoto University, JP [email protected] Abstract which involved dozens of linguists and a decade of work, making careful decisions about the al- In this paper, we aim to close the gap lowable syntactic frames for various verb senses, from extensive, human-built semantic re- informed by text examples. sources and corpus-driven unsupervised models. The particular resource explored VerbNet is useful for semantic role labeling and here is VerbNet, whose organizing princi- related tasks (Giuglea and Moschitti, 2006; Yi, ple is that semantics and syntax are linked. 2007; Yi et al., 2007; Merlo and van der Plas, To capture patterns of usage that can aug- 2009; Kshirsagar et al., 2014), but its widespread ment knowledge resources like VerbNet, use is limited by coverage. Not all verbs have we expand a Dirichlet process mixture a VerbNet class, and some polysemous verbs model to predict a VerbNet class for each have important senses unaccounted for. In addi- sense of each verb, allowing us to incorpo- tion, VerbNet is not easily adaptable to domain- rate annotated VerbNet data to guide the specific corpora, so these omissions may be more clustering process. The resulting clusters prominent outside of the general-purpose corpora align more closely to hand-curated syn- and linguistic intuition used in its construction. tactic/semantic groupings than any previ- Its great strength is also its downfall: adding ous models, and can be adapted to new new verbs, new senses, and new classes requires domains since they require only corpus trained linguists - at least, to preserve the integrity counts.
    [Show full text]
  • Statistical Machine Translation with a Factorized Grammar
    Statistical Machine Translation with a Factorized Grammar Libin Shen and Bing Zhang and Spyros Matsoukas and Jinxi Xu and Ralph Weischedel Raytheon BBN Technologies Cambridge, MA 02138, USA {lshen,bzhang,smatsouk,jxu,weisched}@bbn.com Abstract for which the segments for translation are always fixed. In modern machine translation practice, a sta- However, do we really need such a large rule set tistical phrasal or hierarchical translation sys- to represent information from the training data of tem usually relies on a huge set of trans- much smaller size? Linguists in the grammar con- lation rules extracted from bi-lingual train- struction field already showed us a perfect solution ing data. This approach not only results in space and efficiency issues, but also suffers to a similar problem. The answer is to use a fac- from the sparse data problem. In this paper, torized grammar. Linguists decompose lexicalized we propose to use factorized grammars, an linguistic structures into two parts, (unlexicalized) idea widely accepted in the field of linguis- templates and lexical items. Templates are further tic grammar construction, to generalize trans- organized into families. Each family is associated lation rules, so as to solve these two prob- with a set of lexical items which can be used to lex- lems. We designed a method to take advantage icalize all the templates in this family. For example, of the XTAG English Grammar to facilitate the extraction of factorized rules. We experi- the XTAG English Grammar (XTAG-Group, 2001), mented on various setups of low-resource lan- a hand-crafted grammar based on the Tree Adjoin- guage translation, and showed consistent sig- ing Grammar (TAG) (Joshi and Schabes, 1997) for- nificant improvement in BLEU over state-of- malism, is a grammar of this kind, which employs the-art string-to-dependency baseline systems factorization with LTAG e-tree templates and lexical with 200K words of bi-lingual training data.
    [Show full text]
  • Verbnet Guidelines
    VerbNet Annotation Guidelines 1. Why Verbs? 2. VerbNet: A Verb Class Lexical Resource 3. VerbNet Contents a. The Hierarchy b. Semantic Role Labels and Selectional Restrictions c. Syntactic Frames d. Semantic Predicates 4. Annotation Guidelines a. Does the Instance Fit the Class? b. Annotating Verbs Represented in Multiple Classes c. Things that look like verbs but aren’t i. Nouns ii. Adjectives d. Auxiliaries e. Light Verbs f. Figurative Uses of Verbs 1 Why Verbs? Computational verb lexicons are key to supporting NLP systems aimed at semantic interpretation. Verbs express the semantics of an event being described as well as the relational information among participants in that event, and project the syntactic structures that encode that information. Verbs are also highly variable, displaying a rich range of semantic and syntactic behavior. Verb classifications help NLP systems to deal with this complexity by organizing verbs into groups that share core semantic and syntactic properties. VerbNet (Kipper et al., 2008) is one such lexicon, which identifies semantic roles and syntactic patterns characteristic of the verbs in each class and makes explicit the connections between the syntactic patterns and the underlying semantic relations that can be inferred for all members of the class. Each syntactic frame in a class has a corresponding semantic representation that details the semantic relations between event participants across the course of the event. In the following sections, each component of VerbNet is identified and explained. VerbNet: A Verb Class Lexical Resource VerbNet is a lexicon of approximately 5800 English verbs, and groups verbs according to shared syntactic behaviors, thereby revealing generalizations of verb behavior.
    [Show full text]
  • Single Classifier Approach for Verb Sense Disambiguation Based On
    Single Classifier Approach for Verb Sense Disambiguation based on Generalized Features Daisuke Kawaharay, Martha Palmerz yGraduate School of Informatics, Kyoto University, Kyoto, Kyoto 606-8501, Japan zDepartment of Linguistics, University of Colorado Boulder, Boulder, CO 80302, USA [email protected], [email protected] Abstract We present a supervised method for verb sense disambiguation based on VerbNet. Most previous supervised approaches to verb sense disambiguation create a classifier for each verb that reaches a frequency threshold. These methods, however, have a significant practical problem that they cannot be applied to rare or unseen verbs. In order to overcome this problem, we create a single classifier to be applied to rare or unseen verbs in a new text. This single classifier also exploits generalized semantic features of a verb and its modifiers in order to better deal with rare or unseen verbs. Our experimental results show that the proposed method achieves equivalent performance to per-verb classifiers, which cannot be applied to unseen verbs. Our classifier could be utilized to improve the classifications in lexical resources of verbs, such as VerbNet, in a semi-automatic manner and to possibly extend the coverage of these resources to new verbs. Keywords: verb sense disambiguation, single classifier, word representations 1. Introduction for each verb or use class membership constraints (Abend A verb plays a primary role in conveying the meaning of et al., 2008), which limit the class candidates of a verb to its a sentence. Since capturing the sense of a verb is essential seen classes in the training data.
    [Show full text]
  • Verbnet Representations: Subevent Semantics for Transfer Verbs
    VerbNet Representations: Subevent Semantics for Transfer Verbs Susan Windisch Brown Julia Bonn James Gung University of Colorado University of Colorado University of Colorado [email protected] [email protected] [email protected] Annie Zaenen James Pustejovsky Martha Palmer Stanford University Brandeis University University of Colorado [email protected]@[email protected] Abstract verb senses and introduce a means for automati- cally translating text to these representations. We This paper announces the release of a new ver- explore the format of these representations and the sion of the English lexical resource VerbNet types of information they track by thoroughly ex- with substantially revised semantic represen- amining the representations for transfer of posses- tations designed to facilitate computer plan- sions and information. These event types are ex- ning and reasoning based on human language. We use the transfer of possession and trans- cellent examples of complex events with multi- fer of information event representations to il- ple participants and relations between them that lustrate both the general framework of the change across the time frame of the event. By representations and the types of nuances the aligning our new representations more closely new representations can capture. These repre- with the dynamic event structure encapsulated by sentations use a Generative Lexicon-inspired the Generative Lexicon, we can provide a more subevent structure to track attributes of event precise subevent structure that makes the changes participants across time, highlighting opposi- tions and temporal and causal relations among over time explicit (Pustejovsky, 1995; Pustejovsky the subevents. et al., 2016). Who has what when and who knows what when are exactly the sorts of things that we 1 Introduction want to extract from text, but this extraction is dif- ficult without explicit, computationally-tractable Many natural language processing tasks have seen representations.
    [Show full text]
  • Verbnet and Propbank
    The English VerbNet PropBank VerbNet and PropBank Word and VerbNets for Semantic Processing DAAD Summer School in Advanced Language Engineering, Kathmandu University, Nepal Day 3 Annette Hautli 1 / 34 The English VerbNet PropBank Recap Levin (1993) Detailed analysis of syntactic alternations in English Verbs with a common meaning partake in the same syntactic alternations Verb classes that share syntactic properties ! How can we use this information in nlp? 2 / 34 The English VerbNet Lexical information PropBank The VerbNet API Today 1 The English VerbNet Lexical information The VerbNet API 2 PropBank 3 / 34 The English VerbNet Lexical information PropBank The VerbNet API Overview Hierarchically arranged verb classes Inspired by Levin (1993) and further extended for nlp purposes Levin: 240 classes (47 top level, 193 second level), around 3000 lemmas VerbNet: 471 classes, around 4000 lemmas Explicit links to other resources, e.g. WordNet, FrameNet 4 / 34 The English VerbNet Lexical information PropBank The VerbNet API Outline 1 The English VerbNet Lexical information The VerbNet API 2 PropBank 5 / 34 The English VerbNet Lexical information PropBank The VerbNet API Components Thematic roles 23 thematic roles are used in VerbNet Each argument is assigned a unique thematic role When verbs have more than one sense, they usually have different thematic roles 6 / 34 The English VerbNet Lexical information PropBank The VerbNet API Components Role Example Actor Susan was chitchatting. Agent Cynthia nibbled on the carrot. Asset Carmen purchased a dress for $50. Attribute The price of oil soared. Beneficiary Sandy sang a song to me. Cause My heart is pounding from fear.
    [Show full text]
  • Framester: a Wide Coverage Linguistic Linked Data Hub
    Framester: A Wide Coverage Linguistic Linked Data Hub B Aldo Gangemi1, Mehwish Alam1( ), Luigi Asprino2, Valentina Presutti2, and Diego Reforgiato Recupero3 1 Universite Paris 13, Paris, France {gangemi,alam}@lipn.univ-paris13.fr 2 National Research Council (CNR), Rome, Italy [email protected], [email protected] 3 University of Cagliari, Cagliari, Italy [email protected] Abstract. Semantic web applications leveraging NLP can benefit from easy access to expressive lexical resources such as FrameNet. However, the usefulness of FrameNet is affected by its limited coverage and non- standard semantics. The access to existing linguistic resources is also limited because of poor connectivity among them. We present some strategies based on Linguistic Linked Data to broaden FrameNet cov- erage and formal linkage of lexical and factual resources. We created a novel resource, Framester, which acts as a hub between FrameNet, Word- Net, VerbNet, BabelNet, DBpedia, Yago, DOLCE-Zero, as well as other resources. Framester is not only a strongly connected knowledge graph, but also applies a rigorous formal treatment for Fillmore’s frame seman- tics, enabling full-fledged OWL querying and reasoning on a large frame- based knowledge graph. We also describe Word Frame Disambiguation, an application that reuses Framester data as a base in order to perform frame detection from text, with results comparable in precision to the state of the art, but with a much higher coverage. Keywords: Frame detection · Framester · FrameNet · Framenet cover- age · Knowledge graphs · Frame semantics · Linguistic linked data 1 Introduction Many resources from different domains are now published using Linked Open Data (LOD) principles to provide easy access to structured data on the web.
    [Show full text]
  • Probabilistic Frame-Semantic Parsing Dipanjan Das Nathan Schneider Desai Chen Noah A
    Probabilistic Frame-Semantic Parsing Dipanjan Das Nathan Schneider Desai Chen Noah A. Smith School of Computer Science Carnegie Mellon University Pittsburgh, PA 15213, USA fdipanjan@cs,nschneid@cs,desaic@andrew,[email protected] Abstract TRANSITIVE_ CAUSE_TO_ ACTION MAKE_NOISE MAKE_NOISE Event Purpose Sound This paper contributes a formalization of frame-semantic parsing as a structure predic- Place Place Place tion problem and describes an implemented Time Time Time parser that transforms an English sentence Agent Agent Noisy_event into a frame-semantic representation. It finds Cause Cause Sound_source cough.v, gobble.v, words that evoke FrameNet frames, selects Patient Sound_maker ring.v, yodel.v, ... blare.v, play.v, frames for them, and locates the arguments — for each frame. The system uses two feature- ring.v, toot.v, ... based, discriminative probabilistic (log-linear) Inheritance relation Causative_of relation models, one with latent variables to permit Excludes relation disambiguation of new predicate words. The Figure 2. Partial illustration of frames, roles, and LUs parser is demonstrated to significantly outper- related to the CAUSE TO MAKE NOISE frame, from the form previously published results. FrameNet lexicon. “Core” roles are filled ovals. 8 addi- tional roles of CAUSE TO MAKE NOISE are not shown. 1 Introduction and a precision-boosting constraint that forbids ar- FrameNet (Fillmore et al., 2003) is a rich linguistic guments of the same predicate to overlap. Our parser resource containing considerable information about achieves the best published results to date on the lexical and predicate-argument semantics in En- SemEval’07 FrameNet task (Baker et al., 2007). glish. Grounded in the theory of frame semantics 2 Resources and Task (Fillmore, 1982), it suggests—but does not formally define—a semantic representation that blends word- We consider frame-semantic parsing resources.
    [Show full text]
  • Redalyc.Sentence Similarity Computation Based on Wordnet
    Computación y Sistemas ISSN: 1405-5546 [email protected] Instituto Politécnico Nacional México Wali, Wafa; Gargouri, Bilel; Hamadou, Abdelmajid Ben Sentence Similarity Computation based on WordNet and VerbNet Computación y Sistemas, vol. 21, núm. 4, 2017, pp. 627-635 Instituto Politécnico Nacional Distrito Federal, México Available in: http://www.redalyc.org/articulo.oa?id=61553900006 How to cite Complete issue Scientific Information System More information about this article Network of Scientific Journals from Latin America, the Caribbean, Spain and Portugal Journal's homepage in redalyc.org Non-profit academic project, developed under the open access initiative ISSN 2007-9737 Sentence Similarity Computation based on WordNet and VerbNet Wafa Wali1, Bilel Gargouri1, Abdelmajid Ben Hamadou2 1 MIRACL Laboratory, FESGS-Sfax, Tunisia 2 MIRACL Laboratory, ISIMS-Sfax, Tunisia [email protected], [email protected], [email protected] Abstract. Sentence similarity computing is increasingly clustering, question answering, machine transla- growing in several applications, such as question tion, text summarization and others. answering, machine-translation, information retrieval A number of different metrics for computing and automatic abstracting systems. This paper firstly the semantic similarity of sentences have been sums up several methods to calculate similarity between devised. Some of these outlined syntactic sentences which consider semantic and syntactic methods, to measure the similarity between knowledge. Second, it presents a new method for sentences are based on the co-occurring words the sentence similarity measure that aggregates, in a linear function, three components: the Lexical similarity between sentences, such as [16] or on the similar Lexsim including the common words, the semantic syntactic dependencies, like [13].
    [Show full text]